Dumb things programmers do

I’ve written before about this blog’s journey away from WordPress into becoming a statically hosted blog, and how I’ve regretted it ever since. Allow me to present you with the next chapter in this increasingly silly saga.

WordPress has a lot of limitations. It had them years and years ago when I first decided to move away from it, and it still has them now. Back then, you basically had three choices: 1. host your site through wordpress.com, meaning not a lot of customization options. 2. Host everything yourself, meaning it’s on you to keep things up to date and running. And 3. go with a hosted WordPress solution that allowed you to customize some things without having to manually manage your host or software stack. I was on option 3 at the time, and the major drawback was that it just wasn’t fast, and near-impossible to make it fast while still taking advantage of WordPress’ ease of use. Oh yeah, one constraint I forgot to mention in my quest: it had to be dirt-cheap.

So at some point I decided to statically host this blog. After all, how hard could it be, right? Just generate a bunch of files, put them in an S3 bucket and you’re done. Except you need to either use a framework that does that for you, adding a thing to learn and a depedency that could cease to be updated in the future, or do it yourself, increasing the time spent on writing and maintaining your own software. I went with the latter and wrote my own blog generation software. I tried to be strict on myself in terms of maintenance burden: I only wanted the absolute barebone functionality and as little reliance on third-party libraries as possible. I would only be satisfied if, years later, I would open up the code for this blog to add a new feature without having to trudge through thousands of lines of code, or having to stare at what I wrote in disgust and confusion.

I didn’t quite make it. Even in trying to keep the codebase minimal, it’s still a maintenance burden compared to WordPress for only a fraction of the features. I spent a long time trying to figure out how I was going to do comments on a static blog without having to rely on some shitty service like disqus. I ended up with something working, but clunky, and it only added to the maintenance burden. In retrospect I am 100% sure that staying on WordPress would have been the better solution.

So… where does a lazy programmer like me go from there? Back to WordPress? Ugh. I would have to re-convert all the blog posts I wrote in my custom format back into WordPress SQL or whatever other format WordPress allows you to import. I would have to rejig the DNS, undo all the magical things I had to do to convince AWS to host things in the way that they are.. It’s all certainly possible, but it sounds like a time sink. So instead, I did what every self-disrespecting developer would do, and I made things even worse.

One major disadvantage about this static blog is that I need to run software in order to write and publish posts. Since that software is something janky that I built myself I’m not super keen on exposing that to the world wide internet, so basically the only way I can write blog posts when I’m away is if I bring a laptop that’s got the blog software on it. Old school, I know. My solution to fix that was to come up with a little script I’d run on a local raspberry pi which would ping Github for published blogposts, and in turn render them and publish them. It would solve the problem of allowing me to write blogposts remotely, but I’d still need access to a git client.

So then I thought: why not write the blogposts on WordPress instead? I could read a WordPress RSS feed and use that to convert each post into my custom static blog format, and publish whenever there’s a new post. So that’s what I did. I now have a script that runs every couple of minutes on a local raspberry pi that checks if the WordPress blog has a new post, which it then publishes. The circle of madness is now complete. I have come full circle when really what I should have been doing is just stand still.

Eventually I will probably spend some time moving everything back to a more manageable platform. The way my life has slowed down over the years I’d say that’s probably not going to be any time soon, but let’s see how long this duct-taped solution will last, and which will be the first part to break. For now at least, I am “feature complete”. Yay.

Posted in Tech | Tagged

This is a test post

More info soon. I am working on a better way to publish blog posts. This post was published on a Raspberry Pi! Woohoo!

Posted in Thoughts | Tagged ,

RSS Feed

Although in the previous post I lamented about wishing I'd stayed on Wordpress, a day after I published that this article about wordpress.com pricing model issues became the top post on Hacker News. Just changing out the pricing model for their existing users seems like a practice that would certainly have made me unhappy if I had been their customer. It made me feel slightly better about the time I spent on this blog. But only slightly.

And speaking of ancient technologies, since it seems I'll be on my self-built platform for a while I added support for RSS feed generation to my static blog platform. The front page now has an RSS feed, though I likely won't bother to add feeds for categories, tags or article comments unless there's more early-2010s nostalgia fans like me out there.

I'm still thinking about publishing this on Github but I know there's tons of static blog solutions out there that are way better than mine, so I've not been in a hurry. It's on my todo list though.

Posted in Tech | Tagged

Comments

I deeply regret moving away from Wordpress. It was a terrible choice. Self-hosting is much, much faster, more secure and I have more freedom to do what I want with this site, but even something as simple as making a comment form involves so many decisions and so much future maintenance burden that it just makes me sad.

Anyway, it's working. There's comments again now. I have to regenerate the entire site to make this work, but it's not something that should happen too often. I had planned to go with a full AWS-based solution but ended up incredibly frustrated because everything AWS is just so needlessly complex. Lambdas and SES seem like a great way to handle "serverless" comment form submissions, but hooking it all up and figuring out which set of permissions to use made me vow never to use AWS again if I can avoid it. Instead, pythonanywhere.com lets you very easily spin up any kind of python server, which made it super easy to get started, and the free tier provides more than enough capacity for a blog of this size. I gave up on SES as well and ended up going with mailjet which again provides a very usable free tier and a straightforward API.

I also had to add a captcha to avoid abuse. It's unfortunate, but kind of necessary on today's internet. An alternative would have been to ask people to tie an identity to their comment and authenticate them with Google, Facebook or whatever, but that runs counter to what I'd like this blog to be. If you do think this would have been better, feel free to leave a comment! You can do that now!

So in the end I had to use three new cloud services to make comments work on my static blog: a non-static web hosting service to handle the comment form submissions, a mail service to send out the comment notifications, and a captcha service to prevent abuse. In return, what I get is data that remains under my control: comments (and email addresses used in comments) are in my inbox and on my PC, rather than on some external commenting service. No ads, no spam, no bullshit. Mild hassle because of the captcha, but hey, you can't have everything.

What I would do different in the future if I still had to stick to a static/custom solution: just load everything in memory. All the 17 years of content on this blog, compiled into html, is less than 256 megabytes. Even if I became twice as prolific over the next 17 years it would still easily fit into memory. Single server, maybe with a CDN in front of it, though I doubt it'd need it. Optionally I could parse the original templates on-demand to save even more memory, but it doesn't really matter. That way I wouldn't need to jump through hoops to make static hosting work and could have everything be contained in a single web service. All of this is of course still worse than moving back to Wordpress.

Posted in Tech | Tagged

This blog as a hobby project

My rate of blogposting started going down around 2012. That's ten years ago. Between then and now I thought a lot about what I wanted to do with this blog, and if I even wanted to continue it. I probably changed my mind on that every year or so. The term 'blogging' itself has kind of become obsolete in the meantime, as did the code for the static blog generation framework I built to produce these pages. It feels weird to unironically work on 'comment forms' for a 'blogging framework' in 2022. All of a sudden it's gone from mainstream internet to a weird kind of nostalgia. Who even 'blogs' in this day and age? Plenty of people write, of course, but it doesn't take a self-hosted, self-published platform to do that. Just sign up for a Medium account, or if you don't like that there's plenty of alternatives. Some people write entire articles on Twitter, which is a whole other weird internet thing..

I can liken the experience of blogging and maintaining your personal blog software to that of driving and tinkering on a classic car. It's a fun weekend project with no practical purpose whatsoever, all commercially available solutions are way newer and better than what you're working on, and whatever you're doing will never actually be useful to anyone else. But it's fun somehow. It keeps me busy. And, to continue the analogy, there is tangible output: just like you can take a finished oldtimer back on the public road, this blog exists on the public internet. Even if it can't keep up with all the modern alternatives around it, it's still there for people to see. And that's worth something. In my case, pretty much all of my life is documented here - from all the way back in 2005 to now. I enjoy reading what I did back then, and I hope the people who knew me back then do too. If Medium is a modern-day Toyota, this blog is a poorly-put-together 80s kit car.

(I guess for most people reading this the car example is actually less relatable than that of maintaining a blog. Oh, how time flies..)

So that's what I spent my weekend on. Old man blogger. Here's what I did over the weekend, and how things easily spiral out of control..:

  • I wanted to make comments work so I had to first decide on a format and import comments from the wordpress days. This meant interacting with a wordpress SQL backup that only really restored well onto a MySQL installation, which I didn't have running, so I wrote a nasty little script that regexes over the SQL dump and extracts comments. Yum..
  • As I was playing around with the comments I noticed that a lot of the old articles' image links were broken due to the way they linked to S3 bucket content. So I did some search/replaces on historical content to fix that.
  • Because I fixed a bunch of old posts and changed the base templates I had to regenerate the full site from scratch. This turned out to be less user-friendly than I liked given the scripts I wrote a couple of years ago, so I ended up spending hours refactoring those scripts to be more user-friendly.
  • As part of the comment loading I ended up making some rapid-fire javascript changes to static files, which I had to upload manually myself every time I made a change. This eventually annoyed me so much that I updated the blog generation scripts to also be able to upload static files.
  • While working on the styling for comments I figured I might as well do some other drive-by fixes to padding, colors etc. so I improved the styling a bit.
  • The imported comments needed to be mass-published, so I added functionality to the generation scripts for that as well.
  • For the first time since starting this static blog I actually have non-broken 404 and 403 pages. omg
  • In the end I got comment display working but didn't manage to finish work on the comment submission form. I'll save that for another weekend..

It's my goal to eventually publish the static blog generation framework on github, even though I know for a fact that there's way better alternatives out there already, and I wouldn't recommend anyone to start a static blog themselves at this point as the hosted solutions are way easier. But if I can publish the code on github I can at least call it 'finished'. Perhaps when that happens I'll finally consider moving back to a hosted solution. But for now I will stubbornly keep tinkering..

Posted in Tech | Tagged

No more analytics

Apparently Google Analytics might become illegal soon in the Netherlands. It reminded me that I still had a Google Analytics script on this blog, which I've now removed. No more tracking!

Side note: bit rot is real. I had to update various Python libs to their latest version to make this static blog platform work with the latest Python. The maintenance burden is real.

Posted in Tech | Tagged

Comments

My self-hosted blogging woes continue..

It'd be an understatement to say that I'm not a big fan of Facebook. Facebook has been banning a lot of groups and accounts lately, seemingly algorithmically. That's kind of evil. I was seriously considering buying an Oculus headset until they started requiring that you sign in to it with your Facebook account; an utterly unnecessary step from a user point of view, only done so they can make more money and steal more of your information. Facebook sucks.

That got me thinking about how to completely distance myself from Facebook. I've still got a profile there that I regularly sign in to. Facebook is the only social network that a lot of my family and friends are on. There's just no alternative for chatting with groups of people that I know, because those people are only on Facebook, and so far this has stopped me from completely deleting my Facebook account. Today I thought I would finally remedy that, but before doing that I wanted to provide people with an alternative way to keep track of what I'm up to. My blog is where I would want to shift my activity to, but it currently does not allow people to comment, which is a big part of what makes a blog a blog, so that's really a must-have before I start pointing people to this site again.

There's two ways to add comments to a static blog: either you self-host the serverside part that stores the comments, or you use an online service like Disqus to do it for you. But don't use Disqus. The interface is utterly hideous and the free plan won't let you disable ads. Disqus also makes you jump through several unnecessary hoops to post anonymous comments, which was just clean and simple with native Wordpress. I would not recommend Disqus to anyone. Unfortunately none of the serverless alternatives offered a free plan for personal websites, so I'm out of luck there as well.

Then there's the self-hosted option. There's a few serverside packages that will deal with storing the comments for you if you don't mind spinning up a server. Some of them will even deal with span and email notifications. But for me this would mean 1. yet another piece of code that I have to maintain, and 2. cloud server resources that I have to pay for, especially if someone decides to do a spam attack on me. I have better things to do with my life than deal with that.

Sadly, this is the state of the internet right now. I want to get away from services I don't like, but the only way to do that is to invest time or money. It has never been harder than now to have a self-managed personal blog on the internet. Tim Berners-Lee's Solid project might be the best hope for a future internet, but I'm not holding my breath about it catching on any time soon. For now this blog will stay comment-less, and I will not delete my Facebook account yet..

Posted in Tech | Tagged , ,

Grown-up life and blogs don't mix

In the recent years I've begun to think very differently about the 'free' time I have available to me. Back in my mid-twenties my free time was truly free. My commute was short, my responsibilities were little, my social life was not very active, so I ended up having more than enough time to spend on all the things that interested me at the time. Now, ten years later, my social life is still not very active, but everything else has gotten a lot more busy. I lose two to three hours every day in my commute. I am fortunate enough to be able to worry about what the most efficient way is to mortgage our apartment, how to invest our savings, and so on. Once those things enter your life you have to prioritize them accordingly. Just keeping up with this took some adjusting for me. And I don't even have kids! I can only imagine how busy people are when they have kids to take care of. The free time in my life has gone from 'mostly unplanned' to 'fully planned', with almost no exception. I probably first had this thought years ago. I just didn't have time to blog about it.

What does "I don't have time" even mean? In this context it doesn't mean that I am so busy 24/7 with things that utterly vital to my life that I don't have time for anything else. I still spend lots of time playing games, watching movies, cycling, editing photos, and so on. What it means is that, after I'm done with the things that are vital to my life (social obligations, researching decisions that will affect me for the next decades, daily-life churn like laundry, dishes, commutes) I end up with very little time and 'active brain time' left, that I choose to spend it on other (less creative) activities.

Today is an exception, though. Today is the first day in months in which my to-do list has finally shrunk enough for me to feel like devoting some time to blogging. In retrospect I think it was probably a mistake to switch away from Wordpress those few years ago. Yes, my blog is now statically hosted, way faster, and fully managed by me, but the scope creep in getting the little features that I need in means it ends up taking literally years before I have something that even does the bare-minimum that Wordpress does. If I had to make this decision again, I probably would've stayed with Wordpress and moved to more expensive hosting. It's a wonderfully hands-off ecosystem. The reason I wrote my own static blog system and did not use an off-the-shelf solution is because I believe I would be locked in to someone else's solution if I used an off-the-shelf library or framework. My own system minimizes dependencies and lines of code. The actual meat of the system (template rendering, page generation, publishing) is less than 600 lines of code. Sadly, the Django app that I'm writing this in is proof that I still have dependencies and need to write hundreds more lines of code before I actually have something I can use from any random device or location. I think this solution is the right solution for someone who has the time to work on it. What I should have done is just buy more expensive Wordpress hosting, which would've gotten me a lot more happiness, time-savings and features.

One thought keeps popping back into my head over the past few months: "Efficiency/Productivity is the enemy of Creativity". That's one way to phrase it. Another way would be "Efficiency and Creativity are two sides of the same coin". The more efficient I work, the less creative I am. The most efficient people I know are people I would not rank very highly on creativity, and the opposite seems to hold true as well. I find the consequences of this interesting for someone (like me) whose professional role is "web developer". The 'developer' part is something you can improve by being more efficient/productive, but the 'web' part has a very strong element of creativity in it that is sometimes overlooked. Trying to satisfy both elements is an interesting challenge. Building my own blog is something I thought would be helpful for creativity, but in my attempts to keep the amount of work to a minimum I think it ended up being an exercise in productivity rather than creativity. Building things is not fun if you don't enjoy doing it. And (at least in the case of hobbies) if you don't have joy in what you do, why are you doing it?

Use it or lose it.

Posted in Daily Life , Thoughts | Tagged

Why I won't be moving away from WordPress

I've made some time for myself to do some personal projects, and one of the things that's been on my mind for the longest time was to move this blog away from Wordpress and host it statically on S3 instead. Serving a single static html page (plus a few resource files, but not many) from S3 would be so much faster than letting the shared hosting server parse endless lines of php, most of which I don't even want. The benefit in that aspect is clear, but in other areas it's less obvious.

Easy of deployment is one of the issues. I'm not bothered about not being able to post by email or mobile - whenever I have something to write I'll usually have my laptop with me. It doesn't bother me that it won't be WYSIWYG either (which it wouldn't be if I write my own blog software because I don't care about WYSIWYG). The problem is always with the software and the libraries. My software of choice for my would-be static blog is python, but I'll inevitably end up requiring some libraries that will need to be downloaded and/or set up on each machine that I want to publish from. Knowing myself, I will forget to do this before I go on a trip and end up having to download those libraries at ultra-slow speed at some hotel in the middle of nowhere. Or I might be on a public machine, which will be even worse.

Comments are another issue. If the blog is completely static I'd have to go for a javascript-based comment service. I also have old comments that will either have to be converted to the new commenting system or else inserted into the html somehow if I want to preserve them. Again, there are solutions, but they're hardly easier than what I have with WordPress.

Finally there's the issue of dynamic pages: the calendar, archive and search functionality. Calendar and archive pages are pretty easily generated, but I'd either have to remove the search function or rely on an external service for that. Bleh.

The system I had in mind would be a collection of small tools:

  • A converter tool to convert a wordpress database into raw blogpost/page files (the exact format of which I would have to think about).
  • A compiler tool that reads the raw posts and writes usable html
    • Inserts content into a predefined template, I was thinking Django templates. This doesn't have to be fast.
    • Regenerates related pages (front page, archives, calenders, category pages)
    • Converts any markdown text to html
  • An uploader tool that publishes the generated html
    • Would upload to S3, perhaps pluggable so other providers would work too.
    • Would be smart enough to recognize linked images/files and upload those too.
I'd leave the template design and making the javascript commenting system work all up to the user, since that'd be a one-off job (for me, anyway). The goal of all this is to make my life easier, but it's an awful lot of work for an awfully small amount of easiness.. Anyway, if someone else would also be interested in using something like this (or willing to pay for something like this!) then do let me know.

 

Posted in Tech | Tagged , ,

Stats of a trickling personal blog

Here's Wordpress/Jetpack's annual report for this blog in 2012. Not many surprises there. These days my blog gets anywhere between 50-100 hits per day, and more than half of that is from search engines. It's about what you'd expect for a non-focused personal blog. In a way I'm pleased. Even without doing any SEO, even without promoting this blog anywhere, there's still people reading it.

Posted in Uncategorized | Tagged ,