tox and coverage.py

Tox makes it really easy to run multiple tests on your project: against different versions of python or different versions of a related library.

It’s still lacking proper matrix testing: you need to manually define each environment, but apparently, that is going to change:

However, that’s not what I’m writing about today.

Today is about coverage testing, using coverage.py.

It’s possible, using tox, to get coverage.py to run:

[testenv]
commands=
  coverage run setup.py test
  coverage report
deps=
  coverage

However, this will generate a coverage report for just that environment. It would be better if you generated a coverage report for the whole project (although you may want per-environment coverage testing too).

So, we can abuse the fact that the tox envlist will be created and processed in the order they appear:

[tox]
envlist = clean,py27,py34,stats

[testenv]
commands=
  coverage run -a setup.py test
deps=
  coverage

[testenv:clean]
commands=
  coverage erase

[testenv:stats]
commands=
  coverage report
  covarage html

You’ll then get a nice html report in htmlcov/, and a printed coverage report in your console.

Generating Coverage Badges

Drone.io is a pretty neat (free for open-source projects) continuous integration server. The best feature from my perspective is that it works with BitBucket repositories.

It’s pretty nice having status badges indicating if a build is passing or failing, but even better is also getting a coverage report.

I’ve been using django-coverage for this, and ages ago manually created a set of drone.io style badges, and added a patch to copy the relevant file across. But then, shortly after, drone.io changed their status badge format. I never got around to redoing my badges, as it was pretty time consuming.

Enter Pillow.

from PIL import Image, ImageDraw, ImageFont

SIZE = (95, 18)

BACKGROUND = hex_colour('#4A4A4A')
SUCCESS = hex_colour('#94B944')
WARNING = hex_colour('#E4A83C')
ERROR = hex_colour('#B10610')

# You may need a different font filename if you aren't on a Mac
FONT = ImageFont.truetype(size=10, filename="/Library/Fonts/Arial.ttf")
FONT_SHADOW = hex_colour('#525252')

PADDING_TOP = 3

def build_image(percentage, colour):
    # Create a brand-new Image object, with the background
    # as the main badge colour.
    image = Image.new('RGB', SIZE, color=BACKGROUND)
    drawing = ImageDraw.Draw(image)
    
    # Write the word 'coverage' in our specified font.
    # Fake a text-shadow by drawing the text twice.
    # TODO: Make the text-shadow better.
    drawing.text((8, PADDING_TOP+1), 'coverage', font=FONT, fill=FONT_SHADOW)
    drawing.text((7, PADDING_TOP), 'coverage', font=FONT)
    
    # Do the percentage text.
    # TODO: Make the text-shadow better.
    # TODO: Make the text centred in the coloured box.
    drawing.rectangle([(55, 0), SIZE], colour, colour)
    drawing.text((63, PADDING_TOP+1), '%s%%' % percentage, font=FONT, fill=FONT_SHADOW)
    drawing.text((62, PADDING_TOP), '%s%%' % percentage, font=FONT)

Creating the required RGB tuple from a hex colour is also fairly easy:

def hex_colour(hex):
    if hex[0] == '#':
        hex = hex[1:]
    return int(hex[:2], 16), int(hex[2:4], 16), int(hex[4:6], 16)

Finally, you can just generate an image for every percentage point, and save them:

SUCCESS_CUTOFF = 85
WARNING_CUTOFF = 45

# range(101) -> [0, 1, 2, ..., 99, 100]
for i in range(101):
    file = open('%i.png' % i, 'wb')
    
    if i < WARNING_CUTOFF:
        build_image(i, ERROR).save(file)
    elif i < SUCCESS_CUTOFF:
        build_image(i, WARNING).save(file)
    else:
        build_image(i, SUCCESS).save(file)

It’s not quite perfect: that isn’t quite the font they use, but it will do for now.