Changes to the blog: migration to Codeberg and ActivityPub-based comments
Ever since its start in late 2013, this blog has been hosted on Github Pages, using the Jekyll static site generator. On a technical level this always worked flawlessly, but in the current geopolitical climate I no longer want my site being hosted at a US-based tech giant. After reviewing some options, I decided to migrate the site to Codeberg Pages, which is operated by a non-profit organization that is based in Germany. I also implemented a new comments system that is based on ActivityPub. This allows readers to post comments with a Fediverse (e.g. Mastodon) account.
Overview
This post is quite technical in its nature, and is primarily aimed at readers who are also considering to move their website away from Github. I’m assuming some familiarity with building and configuring static websites and Jekyll. First I give an overview of the main differences between Github Pages and Codeberg Pages. Then I explain how I implemented the ActivityPub-based comments system, and how I created an archive of the legacy (Github-based) comments. In the final sections I briefly explain the configuration I used to make the site work on my custom web domain.
Github pages vs Codeberg pages
Unlike Github, Codeberg doesn’t have a built-in Jekyll integration. This means that for a Jekyll-based website (like mine), you explicitly need to build the static HTML locally from the source, and then upload the HTML to a dedicated repo. There are various ways to do this. In my case, I simply created 2 separate repos: one with the source data, and a “pages” repo with the generated HTML. To make this setup work, you need to add a “destination” variable to the source repo’s _config.yml file, and set the value to the path of the “pages” repo. In my case this is a sibling directory called “pages”:
destination: ../pages/
So, the main difference with the Github-based workflow is that pushing changes to the source repo doesn’t affect the live site, and that in order to update the live site one has to push changes to the “pages” repo1.
Comments section
In 2020 I added a comments feature to this site, which allowed readers to post comments using Github issues. This introduced two challenges to the current migration. First, I had to think of an alternative commenting system that is independent on Github. Since I didn’t want to lose any existing comments, it also meant I had to devise a way to archive those, and incorporate them into the site.
ActivityPub-based comments
I briefly considered a comments system based on Codeberg issues (which has an API that is similar to Github). I ultimately decided against this, mostly because of Codeberg’s much smaller user base. After some thought, I realised that ActivityPub-based comments would be a much better option, as this allows anyone with a Fediverse (e.g. Mastodon) account to post comments. This idea isn’t new, and several others have been doing this already2.
How it works
The way this works is actually pretty simple. Fediverse services such as Mastodon have a public API, which allows you to get a Toot and all its replies as structured data. As an example, consider this Toot:
https://digipres.club/@bitsgalore/114387914694269341
The following API call uses the Toot’s unique identifier, which returns the corresponding data in JSON format:
https://digipres.club/api/v1/statuses/114387914694269341/context
With some JavaScript it’s possible to parse these data files, and inject the parsed content back into a page’s HTML. To a visitor this is then visible as a comments thread. See Jan Wildeboer’s post for a more detailed explanation.
Implementation
For readers who are interested in the nitty-gritty of things, my implementation can be found here. The JavaScript is actually a straightforward adaptation of the code I previously used for the Github comments. This requires that each page (blog post) for which comments are enabled has an “ap_id” identifier. This is the aforementioned unique Toot identifier, and it is defined in the YAML frontmatter of each post’s Markdown source file. Here’s an example (full file here):
---
ap_id: 114416014116498692
comment_id: 92
description: This post describes a simple method for estimating JPEG compression quality.
It is based on a straightforward comparison of a file's quantization tables against
the quantization tables from the JPEG standard using least squares matching. It
also proposes a measure to characterize the similarity of an image's quantization
tables to these standard tables, which is useful for assessing the accuracy of the
quality estimate.
headImage: /images/2024/10/quality-sign.jpg
headImageAltText: Photograph of faded sign on building front showing the word 'Quality'.
layout: post
tags:
- JPEG
- ImageMagick
- ExifTool
title: JPEG quality estimation using simple least squares matching of quantization
tables
---
Comments bot
Although I could have used my regular Fediverse account on digipres.club for initiating comment threads, people might unknowingly end up in my blog’s comments section by replying to a Toot. To reduce the chances of this from happening, I set up a dedicated blog comments account for this instead.
From the outset I wanted to enable commenting for all posts on my blog (currently 80). This means that for each post the following actions are needed:
- Create a Toot that initiates the comments thread.
- Add the identifier of this Toot to the post’s YAML frontmatter as the “ap_id” variable.
Doing this manually for 80 posts would be very tedious, so instead I wrote a commentsBot application that completely automates this. It uses Mastodon.py to post to the blog comments Fediverse account, and Python Frontmatter to add the corresponding identifiers to the YAML frontmatter. I primarily wrote the comments bot for this blog, but I imagine the code might be useful to others as well.
Archiving the legacy Github comments
To preserve the legacy (Github-based) comments, I first downloaded the JSON files that contain the comments data using this script3. I then added all these files to a dedicated subdirectory of the source repo’s _data folder. The base name of each JSON file corresponds to the value of the “comment_id” variable in the YAML frontmatter of each post’s Markdown source file.
As an example, for this post the “comment_id” is 70. So, the corresponding data file with the comments is:
https://codeberg.org/bitsgalore/blog-src/src/branch/main/_data/comments-gh/70.json
I wrote some Liquid code that parses the JSON into styled HTML. It uses the exact same CSS classes that are also used for the ActivityPub comments, so both are rendered with identical formatting. Scroll down to the comments section of this post to see what this looks like.
Making the .domains file work
Another thing to watch out for: if (like me) you serve your site on a custom domain, you need to define this domain (and any subdomains) in a “.domains” file (here’s mine). However, by default Jekyll ignores “hidden” files that start with “.”, which means they won’t be included in the generated HTML! This in turn will break the domain configuration. The solution is to add any “hidden” files or directories to the “include” variable in _config.yml:
# By default Jekyll ignores files and dirs starting with ".", so need to bypass
# for ".well-known" dir and ".domains" file
include: [".well-known", ".domains"]
Making DNSSEC work with custom domains
My bitsgalore.org domain is secured with DNSSEC. I initially had some difficulty getting it to work on the www subdomain. I eventually made this work (thanks to a suggestion by one of the Codeberg admins) using the following configuration:
- Create A, AAAA and TXT records for both the apex domain and the www subdomain.
- Don’t create a CNAME record.
- Define both the apex domain and the www subdomain in the .domains file.
In my case this looks like this:
| Domain | Type | Data |
|---|---|---|
| A | 217.197.84.141 | |
| www | A | 217.197.84.141 |
| AAAA | 2a0a:4580:103f:c0de::2 | |
| www | AAAA | 2a0a:4580:103f:c0de::2 |
| TXT | bitsgalore.codeberg.page | |
| www | TXT | bitsgalore.codeberg.page |
And my .domains file contains the following lines:
bitsgalore.org
www.bitsgalore.org
Final thoughts
Unlike Github, Codeberg is run by a non-profit organization that is funded through donations. Many Fediverse/Mastodon instances are run by volunteers as well. If this post inspired you to move your website there as well, consider making a donation to Codeberg (or become a Codeberg member), and check if your Fediverse instance accepts donations.
Thanks are due to Codeberg and the admins at digipres.club for making this possible!
Further resources
- Blog source repo
- commentsBot code
- blogResources - various utility scripts I’ve written for managing this website
-
I supppose it would also be possible to implement this as two separate branches in one single repo, but I haven’t looked into this. ↩
-
See eg. Jan Wildeboer and Cassidy James Blaede. ↩
-
It’s important to use a Github token for this: although it’s possible to use the Github API without a token, you’re likely to hit the API’s rate limit pretty quickly, with the result that the script won’t download all JSON files. ↩
Comments