Codeship PR checker

Continuous Integration (CI) is great: basically you have a system that ensures that every time a change is committed your full test suite is run, and any failures are reported back. In our case, we have tests that take around 40 minutes to run (because we have lots of tests that need to create data and then ensure that things work based on that data), so being able to have that happen while you continue work is really nice. On top of this, we use Codecov to check that coverage does not decrease on a given commit.

Our general workflow is that we create a branch (in Mercurial, branches are different to git branches, they are long-lived, and all code within a branch retains that reference, meaning it’s easy to associate code with Jira issues), and create a Pull Request when the code is ready for review. At any one time there may be several Pull Requests open, that may or may not affect the same files.

Tests are run by Codeship (our CI service) against every commit, so it’s easy to see if a given commit is valid for merging, but there’s no way to know if a commit will (a) merge cleanly, and (b) still pass tests after merging, at least until you merge it. (a) is handled by BitBucket (it shows us if a merge will apply cleanly), but (b) is still a problem.

Until yesterday.

I was able to leverage the “Test Pipelines” feature of Codeship to make a pipeline that checks if the current branch is default, and if it is not, then it attempts an automated merge, followed by running all of the tests. It doesn’t send results to Codecov, because the commit that would be covered does not exist yet, but it does report errors if it fails to merge, or fails to run tests correctly.

if [ "$CI_BRANCH" != "default" ] ; then hg merge -r default --tool internal:merge; detox; fi

Notes: we use mercurial, so the branch name is based on that, as is the merge command. We also use tox, and detox to run tests in parallel.

This catches some big issues we had, where two migrations in one app were created in different branches, and the migrations framework is unable to deal with that. It could also pick up other issues, where a merge results in code that does not work, even though both commits contained fully passing code.

It’s not quite PR testing, because if the target revision changes (ie, a different PR is merged), then it should run the merge-checking pipeline again, but I’m not sure how to trigger this.

hg commit --prevent-stupidity

I’ve had a pre-commit hook in my mercurial ~/.hgrc for some time, that prevents me from commiting code that contains the string import pdb; pdb.set_trace().

I’ve pushed commits containing this out to testing lots of times, and I think even onto production once or twice…

So, the pre-commit hook that has been doing that this is:

[hooks]
pretxncommit.pdb_found = hg export tip | (! egrep -q '^\+[^\+].*set_trace\(\)')

This uses a regular expression check to see if the string matches. However, it does not show the filename, and the other day I was burned by leaving in a console.time(...) statement in a javascript file. So, I’ve improved the pre-commit hook, so it can do a bit more.

## <somewhere-in-your-PYTHONPATH>/hg_hooks/debug_statements.py

import sys
import _ast
import re

class colour:
    HEADER = '\033[95m'
    OKBLUE = '\033[94m'
    OKGREEN = '\033[92m'
    WARNING = '\033[93m'
    FAIL = '\033[91m'
    END = '\033[0m'

ERROR_HEADER = "*** Unable to commit. There were errors in %s files. ***"
ERROR_MESSAGE = """  File "%s/%s", line %i,
%s%s%s"""

def syntax_check(filename, data):
    try:
        tree = compile(data, filename, "exec", _ast.PyCF_ONLY_AST)
    except SyntaxError:
        value = sys.exc_info()[1]
        msg = value.args[0]

        (lineno, offset, text) = value.lineno, value.offset, value.text
        
        if text is None:
            raise
        
        return lineno, ("%s: %s" % (msg, text)).strip('\n')

ERRORS = {
    'py': [
        re.compile('(^[^#]*import pdb; pdb.set_trace\(\))'),
        syntax_check
    ],
    'js': [
        re.compile('(^[^(//)]*console\.[a-zA-Z]+\(.*\))'),
        re.compile('(^[^(//)]*debugger;)')
    ],
}

def test_data(filename, data):
    for matcher in ERRORS.get(filename.split('.')[-1], []):
        if hasattr(matcher, 'finditer'):
            search = matcher.finditer(data)
            if search:
                for match in search:
                    line_number = data[:match.end()].count('\n')
                    yield line_number + 1, data.split('\n')[line_number]
        elif callable(matcher):
            errors = matcher(filename, data)
            if errors:
                yield errors

def test_repo(ui, repo, node=None, **kwargs):
    changeset = repo[node]
    
    errors = {}
    
    for filename in changeset:
        data = changeset.filectx(filename).data()
        our_errors = list(test_data(filename, data))
        if our_errors:
            errors[filename] = our_errors        
    
    if errors:
        print colour.HEADER + ERROR_HEADER % len(errors) + colour.END
        for filename, error_list in errors.items():
            print 
            for line_number, message in error_list:
                print ERROR_MESSAGE  % (
                    repo.root, filename, line_number,
                    colour.FAIL, message, colour.END,
                )
        print
        return True

Then, add the hook to your .hgrc:

[hooks]
pretxncommit.debug_statements = python:hg_hooks.debug_statements.test_repo

Note: I’ve updated the script to correctly show the line number since the start of the file, rather than the line number within the currently processed segment. Thanks to Vinay for reminding me about that!

Patch for Mercurial.tmbundle

 1 diff -r 5e13047a2284 Support/hg_commit.rb
 2     --- a/Support/hg_commit.rb	Mon Apr 27 11:38:15 2009 +0930
 3     +++ b/Support/hg_commit.rb	Mon Apr 27 11:39:00 2009 +0930
 4     @@ -79,7 +79,7 @@
 5        commit_paths_array = matches_to_paths(commit_matches)
 6        commit_status = matches_to_status(commit_matches).join(":")
 7        commit_path_text = commit_paths_array.collect{|path| path.quote_filename_for_shell }.join(" ")
 8     -  commit_args = %x{"#{commit_tool}" --status #{commit_status} #{commit_path_text} }
 9     +  commit_args = %x{"#{commit_tool}" --diff-cmd hg,diff --status #{commit_status} #{commit_path_text} }
10      
11        status = $CHILD_STATUS
12        if status != 0
13     

A web-focused Git workflow

A nice post from Joe Maller about web-focused workflow, using Git.

A web-focused Git workflow After months of looking, struggling through Git-SVN glitches and letting things roll around in my head, I’ve finally arrived at a web-focused Git workflow that’s simple, flexible and easy to use.

From A web-focused Git workflow

I’m thinking about doing something similar with Mercurial, since that’s my DVCS of choice. Most of the concepts are directly comparable, but the commands are a bit different. When I’m done, I’ll write it up.

(This is more a note to self, than anything else!)