Highlighting Liquid Template Blocks in Marked.app

For many years, I’ve used an old version of Jekyll to write this blog. For previewing, I use Marked.app, and one of the things I like about it is how you can get it to preprocess your Markdown files before processing by the markdown processor, or use a custom markdown processor altogether.

In my case, I use Liquid Templates, although the only part of them I use often are the syntax highlighting features. I have some neat TextMate language extensions so that I see the code blocks for Python, SQL and other languages syntax highlighted in the “proper” way for that code block.

Until recently, I think I had a custom markdown processor which used to apply the syntax highlighting so I saw them in Marked.app as I would in the browser after rendering using Jekyll, but that stopped working. So tonight, I wrote a small tool in python to use Pygments to apply the syntax highlighting.

There’s not much to it: it’s more glue code: it uses re.sub to switch out the highlight block with the syntax highlighted version. Something like:

import pathlib
import sys

from pygments import highlight
from pygments.lexers import get_lexer_by_name
from pygments.formatters.html import HtmlFormatter


def highlight_block(match):
    data = match.groupdict()
    formatter = HtmlFormatter(noclasses=True, linenos=False)
    lexer = get_lexer_by_name(data['language'], stripall=True)
    return highlight(data['code'], lexer, formatter)


re.sub(
  r'{% highlight (?P<language>.*?) %}\n(?P<code>.*?)\n{% endhighlight %}',
  highlight_block,
  pathlib.Path(sys.argv[1])
)

However, it is a bit slow to syntax highlight the files. It might be nice to cache them somewhere:

import pathlib

CACHE_DIR = pathlib.Path('/tmp/pygments-cache/')
CACHE_DIR.mkdir(exist_ok=True)


def highlight_block(match):
    data = match.groupdict()
    cache = CACHE_DIR / '{language}.{hash}.html'.format(
        hash=hash(data['code']),
        language=data['language'],
    )

    if cache.exists():
        return cache.open().read()

    formatter = HtmlFormatter(
        noclasses=True,
        linenos='linenos' in data
    )
    lexer = get_lexer_by_name(data['language'], stripall=True)
    output = pygments.highlight(data['code'], lexer, formatter)
    cache.open('w').write(output)
    return output

Now it doesn’t need to rebuild syntax highlighting for blocks that have already been highlighted, and the cache automatically invalidates when there are changes to the block.

This is almost the same solution I implemented as a Jekyll plugin to make that run a bunch faster: although this version does inline styles, which means I don’t have to use the same CSS from my blog.

This is packaged up into a command line tool, and installed using:

$ pipx install --spec \
        hg+https://hg.sr.ht/~schinckel/liquid-highlight \
        liquid_highlight

(or would be if sourcehut’s public urls worked).

Highlight 'highlight' blocks in Markdown/Textmate

The other day, I mentioned that I had Marked.app nicely handling my {% highlight %} blocks, and syntax highlighting them. In passing at the end, I mentioned that TextMate was still formatting them as if they were Markdown.

Now, one way around this is to indent them, but then within the code block they are indented further, and that offends my sensibilities.

Now, within TextMate, syntax highlighting is based on scopes, so to do what I want (which is the same as how HTML may have CSS or JS embedded in it), we just need a language grammar pattern that matches, and applies the relevant scope.

TextMate 2 has even nicer features, where you can set the scope (but not, as it turns out, include rules) dynamically based on a match in the pattern.

Anyway, on to the rules.

Rather than edit the Markdown rules, I wanted to just inject the language grammars in from a bundle of my own, but had no luck with this. Instead, I decided to extend the Jekyll bundle.

This is what I wanted to put in the patterns (simplified a little):

1{
2  begin = ' "%\}\n';
3  end = ' "%\}\n';
4  name = 'source.$1.embedded.html.markdown';
5  patterns = ( { include = 'source.$1'; } );
6}

However, as I mentioned above, the expression on line 5 does not actually include source.js patterns in this case.

Instead, I needed to have a seperate pattern for each language I wanted to include patterns from. Since mostly I work in python, html and javascript, for now those ones will do.

Oh, and the last thing is that html needs to include text.html.basic.

You can see my fork at jekyll-tmbundle. The current code is:

Marked.app previews of jekyll site

Marked.app is pretty sweet. What I like most about it is that it takes about 2 minutes for my site to regenerate, so doing things like previewing a post is a bit of a pain in the arse: so I can use Marked.app to have previews every time I save.

But most of my posts are technical, and have code fragments. I’m using Liquid Templating within Jekyll (indeed, a custom highlighter that caches the files), and these were not rendered well by Marked.app. Fair enough, too, as it doesn’t know anything about them.

So, I needed a way to have the {% highlight %} tags handled by Albino, which in turn uses Pygments.

There are posts on the Marked.app site that talk about Jekyll, but they don’t actually handle the Liquid Templating syntax. For example: github-flavored-markdown.rb

But this one does:

Now all I need is for TextMate to recognise that those block delimiters mean that it is source code and to highlight it as such.