NOTES:

• This is post 7 in a series; see the introduction for all of them, and the requirements for which this software was built.
• Links that start with the text “mpj:” are links to the 1.0.0 tag (1.0 release) of myPrayerJournal, unless otherwise noted.

We have spent a lot of time looking at various aspects of the application, written from the perspective of a software developer. However, there is another perspective to consider - that of a user. With a “minimalist” app, how do we let them know what the buttons do?

Setting Up GitHub Pages

In other projects, we had use GitHub Pages by creating a gh-pages branch of our repository. This has some advantages, such as a page structure that is completely separate from the source files. At the same time, though, you either have to have 2 sandboxes on different branches, or switch branches when switching from coding to documentation. Fortunately, this is not the only option; GitHub Pages can publish from a /docs folder on the master branch as well. They have a nice guide on how to set it up.

The _config.yml file that's also in the /docs folder (mpj:docs folder) was put there by GitHub when we selected the theme from the Settings page of the repo. The only file we ever put there was index.md (mpj:docs/index.md).

Writing the Documentation

As GitHub Pages uses Jekyll behind the scenes, we have Markdown and the default Liquid tags available to us. We didn't need any Liquid tags, though, as the documentation is pretty straightforward. You may notice that there isn't any front matter at the top of index.md; absent that, GitHub Pages selects the title of the page from the first h1 tag in the document, defined with a leading # sequence.

If you browse the commit history for index.md, you'll see that many of the commits reference either a version or issue number. This makes it rather simple to go back in time through the source code, and not only review the code, but see how its functionality was explained in the documentation. You can also review typos that got committed, which helps keep you humble. :)

Making It Better

There is more that could be done to improve this aspect of the project. The first would be to assign it a subdomain of prayerjournal.me instead of serving the pages from bit-badger.github.io. GitHub Pages does support custom subdomains, and even supports HTTPS for these through Let's Encrypt. The second would be to work up a lightweight theme for the page that matched the look-and-feel of the main site; this would make a more unified experience. Finally, while “minimalist” was the goal, there ended up being a few features that took lots of words to explain; a table of contents on the page would help people navigate directly to the help they need.

Our tour is drawing to a close; next time, we'll wrap it up with observations and opinions over the development process.

In the process of migrating Daniel's Weekly Devotions to Hexo, we ran into a problem that we felt sure a tag plugin could solve.

The Problem

Jekyll's Markdown parser follows the original one, where text within an HTML tag is not processed. This can be the desired behavior in many cases, as you could put what would otherwise be translated as Markdown between HTML tags, and the parser/renderer will leave it alone. One of the common features, used multiple times in most posts, are links to Scripture references and blocks of quoted text. We had an include to automate the links, but we needed a special class on the <blockquote> tag, which meant that all Scripture blockquotes could not use Markdown (or end up with “smartified” quotes and such; we had to use the HTML entities within these quotes.) We also included the verse numbers as superscripts within the quoted text; more tags.

It looked something like this… (the “ref” CSS class turns the text red)

<blockquote class="bible">
  <p>
    <sup>11</sup> …And Jesus said, <span class="ref">“Neither do I condemn you;
    go, and from now on sin no more.”</span>
  </p>
  <cite>— <a href="https://www.biblegateway.com/passage/?search=John+8:11&version=ESV"
    title="Read John 8:11 (ESV) at Bible Gateway">John 8:11</a>b <em>(ESV)</em></cite>
</blockquote>

If you've ever edited Markdown, you'll recognize how jarring all that HTML code is within the flow of the otherwise-regular text; and look at all those entities!

The Solution

We looked through at the Hexo Plugin List to find some examples, and began working towards writing a plugin to handle both the links (the part within the <cite> in the example above) as well as the entire blocks of quoted text. Some tags, like the {% codeblock %} tag, have a start tag and an end tag ({% endcodeblock %}); others, like the {% youtube %} tag, just pass arguments with the tag. (You can see all the default tags here.) Hexo passes two arguments to the tag plugin - the arguments within the (start) tag, plus the content (which is blank for tags that don't have an end tag). The returned value from the plugin call is substituted in the document.

For generating a link, that is pretty easy; it could be an inline tag, and it's just a matter of parsing the arguments and forming a link. For the quotes, we need to make sure that we include the content, and Hexo provides a way to run that content through the Markdown renderer. We are converging on a solution!

Hexo will pick up and execute any .js files in the scripts directory of the site as its generating it, so the first efforts were just local to that repo. The reference link looked something like this…

hexo.extend.tag.register('esv', (args, content) => {
  // option parsing with RegEx, similar to the way their tags do

  let reference = arg.trim()
  let urlReference = reference.split(' ').join('+')

  return `<a href="https://www.biblegateway.com/passage/?search=${urlReference}&version=${version}" `
    + `title="Read ${reference} (${version}) at Bible Gateway">${reference}</a>${extraText}${versionText}`
})

...which let the Markdown document go from…

<a href="https://www.biblegateway.com/passage/?search=John+8:11&version=ESV"
  title="Read John 8:11 (ESV) at Bible Gateway">John 8:11</a>b <em>(ESV)</em>

...to…

{% esv John 8:11 extra:b show-version %}

We refactored the link code to be version-agnostic and extracted it from the tag.register function so that we could reuse that for the blockquote citation. This made the local version of the blockquote look something like this:

hexo.extend.tag.register('bible', (args, content) => {
  let text = hexo.render.renderSync({ text: content, engine: 'markdown' })
  return `<blockquote class="bible">${text}<cite>— ${generateRef(args)}</cite></blockquote>`
})

This means that the blockquote can support all of the arguments the inline reference did. We also switched out the marked Markdown processor for the markdown-it one, which lets us do superscripts by using the ^ character. Revisiting our example under “The Problem,” our Markdown source to generate the same blockquote is now:

{% bible John 8:11 extra:b show-version %}
^11^...And Jesus said, <span class="ref">"Neither do I condemn you; go, and from
now on sin no more."</span>
{% endbible %}

The Plugin

The plugin is available on npm, is fully tested, and its source is open. If you use Hexo, and wish to cite Scripture references in your posts with links where readers can see the text for themselves - enjoy!

This blog moved today from Jekyll to Hexo (and it's now open source as well). One of the final issues I had when wrapping up the conversion was how to handle categories and tags that do not necessarily have a slug that can be naturally derived from the name. Take the “C#” category, for example. The normal slug for that category would be c, which is an entirely different programming language; interestingly, “C++” will also normally get its slug as c.

Within Hexo's default _config.yml file, there are two empty items named category_map and tag_map; their comments allude to a mapping, but I could not find what the proper syntax was for those items. We hopped onto the Hexo Gitter chat and asked the question, and someone pointed us to this issue. To define a mapping, create an item under either the category_map or tag_map top-level item. The maps for this site, as they currently are, look like this:

category_map:
  C++: c-plus-plus
  C#: c-sharp
  .NET: dot-net
tag_map:
  c#: c-sharp
  .net: dot-net

As you can see by hovering over the links in the sidebar, “Programming > .NET > C#” ends up with a URL ending with /programming/dot-net/c-sharp/, which is exactly what we were looking for.

As we mentioned in our last post, we plan to share aspects of how we moved to Jekyll. This is the first of these posts.

Background

With a database-based solution, updating is easy; when the user updates content through the user interface, the new content is served when the page or post is requested. However, with a static site, an “update” is technically any change to the underlying files from which the site is generated. Typically, though, this is marked by source control commit and push to a master repository. GitHub Pages, the product for which Jekyll was developed, uses this as a flag to regenerate the site. We weren't using GitHub*, though - we were using Mercurial (Hg) for our source code control, with the master repository on a different server than the one from which the site is served.

* There were a few reasons we did not wish to host our sites using GitHub, none of which are pertinent to how this works.

Options

With the need to regenerate the site after each site's master repository receives a push, there were a few different options we considered.

1. When a push occurs, regenerate the site on the Hg server, then use scp to delete the old files from and copy the new files to the web server.
2. Set up a sandbox on the Hg server that updates and regenerates each time a push occurs, and run rsync on the web server to check for updates every so often.
3. When a push occurs, notify the web server, and have it regenerate the site.

The first option has the potential to run afoul of SSH rate limits, plus has the potential to require much more data transfer than option 3. The second option had the advantage of running a process local to the Hg server, but would have required disk space utilization that we didn't really need; and, as Jekyll regenerates all the pages in a site, rsync would have likely ended up transferring all the data for every update anyway, losing one of its benefits. The third option required Jekyll to be installed on the web server, and uses it for processing, potentially taking cycles that could be used to serve web pages.

Eventually, we decided to go with option 3.

Script All the Things

On the Hg server, in the master repository for each site, we put the following in .hg/hgrc (the following examples are for this site):

[hooks]
incoming = /opt/jobs/notify.tech-blog.sh

...and then, notify.tech-blog.sh…

#!/bin/bash
ssh user@web.server touch /opt/jobs/jekyll/.tech-blog

That is the only logic required on the Hg server. Now, over on the web server, we need logic to regenerate the site and make it live. Since we have multiple sites, we wrote a script that has a few variables, so it could be duplicated for other sites. The following is tech-blog.sh:

##
## DJS Consulting Tech Blog Jekyll Build Script
##
## This will check out, build, and replace a Jekyll-generated site. Just update
## the parts under the "Env" heading for the specific site.
##

## Env
REPO=jekyll.tech-blog
DEST=/path/to/public/techblog
TRIGGER=.tech-blog
## /Env

cd /opt/jobs/jekyll

if [ -e $TRIGGER ]
then
  rm $TRIGGER

  ## Check out the site and build it
  hg clone ssh://user@hg.server/HgPath/$REPO $REPO
  cd $REPO
  jekyll build

  ## Copy it to the proper directory
  cd _site
  rm -r $DEST/*
  cp -r * $DEST

  ## Clean up
  cd ../..
  rm -r $REPO
fi

This script isn't perfect; it needs to check the exit code from the Jekyll build process before whacking the current site (and notifying for a failed build would be a nice addition). However, with Jekyll being the same on both development and production, and a single committer, this is fine for our purposes.

Finally, each script needs to be run to check for the presence of the semaphore (or TRIGGER, as the script calls it). The following cron definition will check every 4 minutes for a change.

*/4 *   *   *   *    /opt/jobs/jekyll/tech-blog.sh > /dev/null

Conclusion

Overall, we're pleased with the results. The inter-server communication is light, only requiring one initiated ssh connection from each server, so we won't run afoul of rate limits. With the work being done on the destination server, the amount of time where there are no files in the directory (between the rm -r $DEST/* and the time the cp -r * $DEST finishes) is very short; it would have been much longer if the directory were being repopulated across the network, or more complex if we added a staging area on the web server. Each piece can be run separately, and if we've committed a post with a future date, we can run the same touch command to make that post appear.

Next time, we'll discuss our experiences converting a non-WordPress site.

From August 2011 until today, this site has been running under WordPress. During this time, we have done many experiments with several other blog platforms, but none of them made it to the “import all the old stuff from this one - this is what we're looking for!” stage. As you may have already guessed, though, this is no longer the case. WordPress does what it does very well. However, the last post before this one was August… of 2014! That means that, every page served from this site, a script was run on the server that accessed a database and dynamically assembled the page. This content did not need to be dynamic - even when there is a new post, there is very little in the way of dynamic content here.

Jekyll is a static site generator; these types of applications generate static sites based on a collection of templates and content files, that can then be served with no backing data store. Now, we can utilize the blazing fast nginx web server to push these files as quick as people can request them, where the request does not even have to escape the process.

There will be more to come on Jekyll; there are at least two posts to be written, one on automating the build process and another on the migration from WordPress. Until then, though, there are redirects that ensure the RSS feeds for both the main blog and the xine RPMs require no changes, and the category pages have redirects as well. If something does not look right, let us know via either of the social media accounts linked above.