Wiki before and after

The MakerVan Wiki contains the documentation for many of my projects. Initially, I set it up as a MediaWiki installation to be able to edit it from anywhere. However, it turned out to be quite slow and thus impractical. So I decided to go ahead and make it a static website powered by Jekyll. However, there are two conditions to that:

  • All the content needs to be migrated
  • The URLs should remain the same as before

The second one is important because I link to the wiki from a number of other places, and it would be quite a hassle to adapt all those. Eventually, it might be interesting to change the URLs, and create redirects for the old URLs. But that’s for another day.

Exporting and converting contents

MediaWiki and Jekyll use different markup languages, so we cannot just export all pages, but also need to convert them to Markdown.

The ideal tool for those conversions is pandoc, and luckily there’s a few wrappers around pandoc to make this a little easier. There’s a blog post on the topic, but the referenced repository (realrubberduckdev/mediawiki-to-markdown) seems a little oldish. The ideal base seems to be outofcontrol/mediawiki-to-gfm instead, but it doesn’t provide a Dockerfile and thus requires php installed to work.

So I went ahead and got them together. Here’s the updated code while waiting for a pull request to merge:

docker build -t tiefpunkt/mediawiki2gfm .
docker run --rm -it -v $(pwd)/output/:/src/output tiefpunkt/mediawiki2gfm
# ./convert.php --filename=output/<mediawiki export>.xml --format=gfm --addmeta

Jekyll Setup and Theme

  • Setup the empty Jekyll site using jekyll new.
  • Clean the default posts and other pages you don’t need.
  • Copy the previously generated Markdown files to the Jekyll directory.

You can use any Jekyll theme. I’m using minima because it’s clean and minimalistic, and also the default theme in a new Jekyll site.

Frontmatter

The --addmeta option generates some frontmatter for all markdown files, but is still missing some details. All pages need a specified layout, for minima I’m using page. So we need to add the line layout: page to the frontmatter of all files.

Additionally, the permalinks did not match the original links, so they needed to be adapted manually.

Images

The conversion mechanism does not export images and adjust the image links in the markdown files.

Internal links are not adjusted either, and need to be updated manually.

Challenges

There are a few more challenges: Plugins & Templates.

I used a gallery plugin to show pictures of projects on the project pages. Stubs from that remained in the project page, but don’t work. I found an include snippet, which works with some adjustments. Here’s the adapted version.

The more challenging part was a template I used to summarise each project. Templates are not converted at all. I tried to convert the template to a Jekyll include, but ran into some more difficulties. Jekyll uses its own implementation of the include tag, which I tried to debug but got nowhere. The solution for now was to create a dedicated Jekyll layout with the template contents directly coded into it. Instead of parameterizing the include tag, the parameters for template were added directly to the frontmatter, e.g. for the Synthiboard, which will end up looking like here.

Open Topics

  • I want to review all links regularly. There are some plugins and other tools, like jekyll-link-checker (blog post 1 and 2) or deadlink.
  • Images are all large, I’d like to automatically create thumbnails. There are some ideas (1, 2)