Hyperlink Your Heart - pelican

Gemini Launch!

2021-06-26T14:59:00+02:00

In the olden-times, before the Web became basically synonymous with the Internet itself in many people’s minds, there was another, competing hypertext protocol: Gopher.

I say “was”, but of course Gopher never really went away - it was kept alive by enthusiasts, and in recent years there has been a resurgence of interest in it as a sort of haven from the ubiquitous surveillance and relentless commercialisation of the Web.

I’ve long been interested in Gopher (I even made a game about it), and have intended to start a phlog for a while without ever going ahead with it. Something about it always just seemed a little bit awkward and off-putting. I was torn between using Gophermap (i.e. menu) files for everything, or using plain text for posts and sacrificing any hypertextuality. I was torn between finding the need to wrap text to be cool and retro, or a hassle that results in an inferior experience for both creating and consuming content.

Gemini is a new protocol which takes inspiration from both Gopher and the Web, and from a certain perspective, improves on both.

When I heard about Gemini I didn’t really get it at first. I thought it was just Gopher with SSL, which is nice, but I figured I’d get set up on Gopher first and then consider a Gemini mirror. A few days ago I saw a screenshot of the Lagrange browser on Mastodon and started to look into it a bit more. When I realised just how many issues of both Gopher and the web it addresses, I was hooked! I spent several days after that setting up a capsule (the Gemini equivalent of a “site”).

Static Generation

After experimenting with a Gemini server for a bit and creating a few static text/gemini files, I decided that I wanted to statically generate my gemlog the same way that I do my blog. I expected to have to write something from scratch to do this, but after some experimentation I was able to get the Pelican static site generator (which I also use for my blog) to both read and output .gmi files. It does take a bit of configuration however, and I had to monkeypatch a couple of methods in Pelican.

Unfortunately this means that it is only guaranteed to work with the current version of Pelican, 4.6.0, and could break at any time. Nonetheless, the plugin is available on GitHub if you want to try it out.

Gemini Reader

The first thing required was a custom “Reader” that can handle .gmi files instead of the usual Markdown or reStructuredText files. It’s simple enough - it just parses the file up to the first blank line as metadata, and the rest of the content is returned unmodified, since we are also going to output the same format.

class GeminiReader(BaseReader):
    enabled = True

    file_extensions = ['gmi', 'gemini']

    def read(self, filename):
        metadata = {}
        content = ""
        with open(filename, mode='r') as f:
            end_of_meta = False
            while not end_of_meta:
                current = f.readline()
                if current == '\n' or current == '':
                    end_of_meta = True
                    continue
                current = current.strip()
                split = current.split(': ')
                metadata[split[0].lower()] = split[1]
            # After the first blank line, the rest is content.
            content = f.read()

        parsed = {}
        for key, value in metadata.items():
            parsed[key] = self.process_metadata(key, value)

        return content, parsed

Handling Internal Links

Pelican has a mechanism for linking to content internal to the site where you start the URL as {static} or {filename} and it replaces those with the appropriate paths during generation. However, this didn’t work with the Gemini link syntax - the replacement is based on a regular expression that assumes the placeholder will be found in an attribute of a HTML element.

I couldn’t find any setting or hook in the plugin system to alter this regular expression. There is a setting to customise the part that specifies the braces, so you could change the placeholders to ¿¿static?? or something if you like, as long as it is still found in HTML. It seemed like my only option was to replace the method where the problem regex pattern is defined, and use something that matches Gemini links instead.

def _get_intrasite_link_regex(self):
    intrasite_link_regex = self.settings['INTRASITE_LINK_REGEX']
    regex = r"(?P<markup>=> )(?P<quote>)(?P<path>{}(?P<value>[\S]*))".format(intrasite_link_regex)
    return re.compile(regex)

You’ll notice this also has to include a “quote” group because that was present in the HTML version and was expected elsewhere - here it will always be an empty string.

Unfortunately, the problems didn’t end there. I found that the placeholders were removed, but not replaced with the absolute URL of the capsule. This turned out to be because urllib is used to join the URL components, and it doesn’t recognise the gemini protocol. To get around this I had to replace another method, and make a call to a wrapper around urllib.urljoin.

def _urljoin(base, url, *args, **kwargs):
    is_gemini = base.startswith('gemini://')
    if is_gemini:
        base = base.replace('gemini://', 'https://')
    result = urljoin(base, url, *args, **kwargs)
    if is_gemini:
        result = result.replace('https://', 'gemini://')
    return result

Gemini Output

Pelican uses Jinja2 for its templating, which is happy to work with any type of text file, so creating .gmi templates wasn’t an issue. Handily, there is a setting to look for templates with extensions other than .html.

THEME = 'themes/hypergem'
TEMPLATE_EXTENSIONS = ['.gmi', '.gemini']

To get Pelican to output files with a .gmi extension instead of .html, there are a bunch of settings for the different parts of the site. A single “extension” setting like for the templates would be nice, but whatchagonnado? I took the opportunity to customise the article location and file names as well.

# These settings are required to output files as .gmi instead of .html
ARTICLE_URL = 'articles/{date:%Y}-{date:%m}-{date:%d}-{slug}.gmi'
ARTICLE_SAVE_AS = ARTICLE_URL

DRAFT_URL = 'drafts/{slug}.gmi'
DRAFT_SAVE_AS = DRAFT_URL

PAGE_URL = 'pages/{slug}.gmi'
PAGE_SAVE_AS = PAGE_URL

DRAFT_PAGE_URL = 'drafts/pages/{slug}.gmi'
DRAFT_PAGE_SAVE_AS = DRAFT_PAGE_URL

AUTHOR_URL = 'author/{slug}.gmi'
AUTHOR_SAVE_AS = AUTHOR_URL

CATEGORY_URL = 'category/{slug}.gmi'
CATEGORY_SAVE_AS = CATEGORY_URL

TAG_URL = 'tag/{slug}.gmi'
TAG_SAVE_AS = TAG_URL

ARCHIVES_SAVE_AS = 'archives.gmi'
AUTHORS_SAVE_AS = 'authors.gmi'
CATEGORIES_SAVE_AS = 'categories.gmi'
TAGS_SAVE_AS = 'tags.gmi'

Theme

I haven’t got much to say about this. I wanted the article links to be a bit more descriptive than just the date and title, so I did something similar to what medusae.space does and included the article summary, the category, and the tags.

It’s close to general purpose but not quite - I added a custom SITELOGO setting that is used on the index page with an ASCII art version of my logo generated using ascii-generator.site, and there is also a custom template for the custom landing page. The index is renamed using a setting, and another page is renamed to index.gmi to take its place. This is so if I want to add content that isn’t generated by Pelican, I have the scope to do so.

INDEX_SAVE_AS = 'gemlog.gmi'

Title: Hyperlink Your Heart
Date: 2021-06-23 22:59
Slug: index
Authors: Kevin Houlihan
Summary: Capsule index
URL: index.gmi
save_as: index.gmi
Template: capsule_intro
Status: hidden

Hosting

I’m serving the capsule using Jetforce from a first generation Raspberry Pi which I had lying around and haven’t done anything with in a while. There was nothing really involved in setting it up beyond what is described in the documentation, except that I installed it in a virtualenv.

I also took steps to make sure it is running as a dedicated user with no permissions to anything else on the system.

Future

I’m not sure what’s next, but I’m excited! I might discuss with the Pelican crew if there are any ways around the issues I encountered that I might have overlooked, or if it could be adapted to be more suited to non-HTML output. If not, maybe a Gemini fork is in order. I have no idea if there are further issues with it beyond the functionality that I’ve used.

I have quite a few posts to port over from my blog yet, and I need to get some image optimisation happening there like I have here. Besides that, I guess all I have to do is get to know the community!

Visit My Capsule (and Beyond)

If you’re already familiar with Gemini please check out my capsule.

If you’re not, well, I still encourage you to visit, but I should probably give you some guidance on getting started.

If you just want to dip your toes you can browse the Gemini network using a HTTP proxy (here’s one, and another). For what I would consider the “full experience” you will need a dedicated browser. I’ve been using Lagrange, and highly recommend it, but there are a whole bunch of others if that doesn’t suit you. Many of them also support Gopher, which makes browsing both into a seamless experience outside of the modern Web.

When you want to move beyond my capsule, here are some others I recommend:

Medusae.space, a content directory: gemini://medusae.space
Station, a social network with user accounts: gemini://station.martinrue.com/
Geddit, an anonymous link aggregator: gemini://geddit.glv.one/
Astrobotany, a community garden game: gemini://astrobotany.mozz.us/
A gemlog post about the “smolnet”, which explains the appeal far better than I ever could: gemini://republic.circumlunar.space/users/maugre/200701-smolnet.gmi

I leave you with my anxious young poppy, Jennifer, on AstroBotany:

            O
            |
           \o
            |o
           \/
.  , _ . ., l, _ ., _ .  
^      '        `    '

name  : "Jennifer"
stage : anxious young poppy
age   : 2 days
rate  : 1st generation (x1.0)
score : 326788
water : |██████████| 100%
bonus : |          | 2%

Embedding SVGs in Pelican

2020-04-04T00:10:00+02:00

In my inaugural post I mentioned that one problem I had encountered while designing this blog was styling the SVG icons. I had grabbed a bunch of the individual icon files from Font Awesome, but because of the way SVGs, CSS and HTML interact, I wasn’t able to colour them directly using CSS color or fill properties, and instead had to use filter properties (which I calculated using this tool, so it wasn’t too much of a hardship).

I also didn’t particularly like that retrieving the icons involved numerous separate requests, nor the visible “pop-in” in Firefox that resulted from having them referenced as external files. The files are tiny, with the request overhead often as large or larger than the files themselves.

A further advantage that I was missing out on by not using Font Awesome as intended was that I couldn’t use their handy <i> tag shortcuts for specifying the icons to use.

Now, I have taken steps towards solving all of these many problems!

Just use Font Awesome normally you weirdo

Let’s back up a sec and talk about why I didn’t just use Font Awesome as intended in the first place (yes tldr; it is probably because I’m a weirdo).

Font Awesome has two ways that it can work: Web Fonts + CSS, or SVG + JavaScript. The former would involve retrieving an additional CSS file or two, as well as a couple of web fonts. The web font for the solid collection alone is 79.4KB - larger than anything else on this website. The JavaScript that would be required for the other method would likely be approaching 1MB in size - larger than this entire website so far! I want a lean, fast-loading, low-power website, and these approaches seem entirely at odds with those goals.

It also struck me as odd to be statically generating a site, yet also having the client browser swapping in SVG images. I’ve nothing against JavaScript, but clearly this is work that can be done in advance!

Doesn’t caching solve this problem?

Well… maybe? In same cases? But not necessarily.

The average size of an icon in Font Awesome’s “solid” collection is 660B. A visitor would have to encounter over 1500 such embedded icons before downloading the JavaScript and caching it would be cheaper. The Web Fonts are much better, with caching the separate files becoming worthwhile after only 214 icons. That’s about 5 views of this blog’s index page, or 15 individual posts.

As such, if somebody reads 16 posts on this blog, they will have transferred more data than they would have if I’d used the Font Awesome web fonts. However, if 15 people read one post each and never visit again, the embedded approach comes out way ahead. So it very much depends on the traffic profile of the site, and I don’t think this site is one that people will be checking in on daily.

Embedding also offers other advantages, such as reducing initial load times.

Solutions

My solution is a pelican plugin that post-processes the generated HTML files and embeds any SVGs it finds, whether specified as <img> tags or <i> tags.

It also, crucially, sets the fill attribute of any SVG paths to currentColor, which causes the fill colour to be taken from the current CSS text colour.

Taking the plugin beyond being merely a static implementation of Font Awesome, it also supports embedding of arbitrary SVG files. This can be achieved either by using <i> tags with the class pi to search a custom icon set, or through <img> tags where the SVG file is referenced by URL.

Future

The plugin probably has loads of rough edges at the moment. I haven’t at all tested if it supports Font Awesome’s more advanced behaviour, or even investigated how those features work, so there is a lot to be done there.

I may explore an approach that would combine the advantages of static generation with the advantages of a separate, cacheable SVG file. My initial thoughts on how to approach this plugin were to combine any referenced SVGs into a single file, and then reference them in the HTML using an SVG <use> tag. I need to learn a lot more about SVGs to know if that’s even feasible.

I also want to try to support other icon frameworks that support a similar <i> tag shortcut, such as Fork Awesome and Friconix.

In the meantime, it’s serving my purposes already on this site.

Remember Blogs?

2020-01-02T20:05:00+01:00

I’ve read a lot of articles recently (here’s one) lamenting the state of the web. Once distributed, egalitarian, ungovernable, and fast, now centralised, intentionally manipulative, and bloated both technically and conceptually. Even when you manage to fight your way through the popups demanding your attention or personal information, often what is underneath is not worth the effort - more likely a vehicle for advertising than for insight.

It’s also incredibly power-hungry. It’s hard to tie down an exact figure for exactly how power-hungry, but the internet as a whole could account for up to 10% of global energy use. A good chunk of that is streaming video and music, which is a topic for another day, but of the power consumed in serving the web, some of it is related to actual valuable content that people want to see, and some of it is related to the trends described above. The latter is waste. At least bloated JavaScript and CSS frameworks can be cached, but advertising has to be constantly served anew.

So, anyway, all this to say… I’ve decided to start a blog.

The Tech

My technical goals for this website are for it to be…

Lightweight & fast to load - I set up a WordPress site recently, on the best hosting I can afford. It is not lightweight or fast to load.
Content focused - Read one thing or read them all, but I’m sure you can only read one article at a time.
Nice to look at - Apparently it doesn’t take much. Also going for consistent branding between all my sites and profiles.
Responsive - Readable on phones as well as desktops!
Easy to deploy - I don’t have time to configure and maintain a teetering stack of back-end technology, and if I have to move to different hosting at some point, I want it to be a simple task.
Easy to update - If writing posts is a chore, I won’t ever do it.
Hackable - Created using technologies that I’m somewhat familiar with, so that it is feasible for me to modify or extend if I want/need to.

I decided almost immediately that a statically-generated site was going to be the best way to achieve most of those goals. I’m a big fan of Python, so although hackability could be achieved by a JavaScript or C# based generator, I checked out the Python ones first, and found plenty of viable options. I settled on Pelican because it’s…

Popular - It seems to be one of the more popular Python generators.
Blog-oriented - Some generators are geared towards documentation or are intended as replacements for content management systems, but that’s not what I’m doing.
Supports Markdown - I’m sure reST is fine, but I already have to use Markdown elsewhere so I’d rather stick with that.
Easy to update - Just create a new Markdown file and run a command to rebuild.
Extensible - It includes a plugin system to modify the output.

I also decided to hand-craft my own theme, and to avoid a CSS framework. I love the look of Bootstrap, and how quick it is to get started with, but it’s over 200kb and a lot of that is undoubtedly unnecessary for my needs. The spirit of the exercise is bare-bones and DIY!

The Theme

The first step in hand-crafting a theme was… to find an existing theme to copy! Atilla was the closest to the style I was after, so I took a copy of that and gutted it of CSS and JavaScript and other elements that didn’t meet my needs. Then I started building the CSS back up while trying to keep it as minimal as possible. It may not implement every feature supported by Pelican, but you can find it on my Github if it seems like something you could adapt for your own needs.

One departure that I made from the standard Pelican configuration was to have the social media links be taken from a collection of tuples with three elements, so that I could specify both an icon and a title to use.

# Custom social list that includes icons
SOCIAL_ICONS = (('Twitter', 'twitter.svg', 'https://twitter.com/http_your_heart'),
                ('Mastodon', 'mastodon.svg', 'https://mastodon.art/@hyperlinkyourheart'),
                ('Instagram', 'instagram.svg', 'https://www.instagram.com/hyperlinkyourheart/'),
                ('YouTube', 'youtube.svg', 'https://www.youtube.com/channel/UCc_O9Hp5UfQ-IHswi1H54Zg'),
                ('Twitch', 'twitch.svg', 'https://www.twitch.tv/hyperlinkyourheart'),
                ('Itch', 'itchio.svg', 'https://hyperlinkyourheart.itch.io/'),
                ('GitHub', 'github.svg', 'https://github.com/khoulihan'),
                ('Atom Feed', 'rss.svg', '/feeds/all.atom.xml'),)

I like that I can just throw custom configuration into the config file and then make use of it in the templates. However, it probably makes the theme less generally useful.

As it stands currently, loading this post requires less than 30kb to be transferred.

Plugins

Currently, the only plugin I’m using is the css-html-js-minify plugin that is available in the pelican-plugins repository. I haven’t found anything I need to write my own plugin to handle yet, but I’m sure I will get to it.

One problem that needs solving is that the SVG icons are a big nuisance, because it doesn’t seem to be possible to change their colour without using the CSS filter property, which is not nearly as convenient as just setting the colour directly. In order to do that, using the fill property, I would have to embed the SVGs, or reference them as symbols in a <use> tag within an <svg> tag. The individual icon files (from FontAwesome) aren’t set up like that, and I didn’t want to use their spritesheet because it is rather large.

What I might do in the future is write a plugin to compile the individual files into a single spritesheet of symbols, then find and replace any references to them with appropriate <svg> tags. Essentially this will be doing the job that the FontAwesome toolkit usually does in the browser.

The Content

Uuuh… I’ll get back to you on that. Things I like, things I do, that sort of thing.

Feedback

There’s a couple of different strategies for allowing comments on a static site - I’m not going to attempt them for now, and perhaps never will! If you have any feedback or thoughts there are many ways to reach me, such as Mastodon or Twitter, and I think that’s just fine.