The costs of running a blog on Amazon EC2

Last month I migrated this blog from a shared hosting provider to EC2. Wordpress was already using WP supercache, and I've set up nginx to serve any cached and static content. All of the uploaded images pre-migration I moved to S3 to reduce bandwidth cost. I was planning to set this up automatically or hack into Wordpress to upload to S3 by default but didn't really find the inspiration to, or the necessity.

The month of the migration I uploaded about 800MB worth of images onto S3 and, because of not thinking things through very well, also uploaded the same set to EC2. I installed a whole bunch of software in order to get the Wordpress installation up and running. For about one whole day the EC2 instance experienced a quirk which caused a constant 100% CPU usage, driving up the cost. Since then this blog has been steadily serving anywhere between 50 and 150 views per day.

Interestingly enough response times in Pingdom went down initially, but seen settled between 800ms and 1000ms. Testing manually results sub-300ms times, so it's probably because the micro instance that comes free with AWS has to spin up for each request. This blog doesn't quite get enough traffic to warrant buying an always-on instance.

Anyway, here's the costs:

  • Initial data migration - first month cost: $5.46
  • Second month cost: $0.50
Amazon is great.

Posted in Tech

The geeky side project

Last week I suddenly got a burst of inspiration. I've been redoing the whole thing from scratch purely because I can. Still no idea what this is going to become..

(The image that was here is lost forever due to blog migration. sorry everyone!)

Posted in Tech

Nginx woohoo!

Yesterday I finally cancelled my old Bluehost account. I hadn't been using it for about a month since switching to AWS, and now it's final. Bluehost has been really great. They've always responded to my requests quickly and helpfully, and they issued me a refund immediately. If you're not looking for a self-managed server they're a great option.

Yesterday this blog was running Wordpress on an Apache server. Today this blog is served from a very speedy nginx server with some neat rules for serving cached pages from WP Super Cache directly, bypassing php entirely and speeding up stuff a whole lot. Some guides mentioned installing Varnish for even more caching, and I've also been toying with using Amazon's content delivery network for static files, but I'm rapidly reaching the overkill point. After all, this blog doesn't really get that many visitors, so this is just technological masturbation.

Well, partially. Amazon does charge you for CPU time and bandwidth, so anything saved there will reduce the monthly bill. Let's see what the introduction of Nginx will do.

For reference, here's the nginx config file I'm using. It serves Wordpress from the /blog directory, redirect the naked domain to www, redirects the root to /blog, tries the WP supercache for a blog page first if present (only if it's not a POST request, if the user is logged in and it's not an admin page), then falls back to calling index.php with the url path as the query parameter so Wordpress can do its url mapping.

server {

root /var/www;
index index.php index.html index.htm;

# Make site accessible from http://localhost/
server_name colorfulwolf.com;

rewrite ^/$ /blog/ permanent;

if ($host ~* ^colorfulwolf\.com$) {
    rewrite ^(.*)$ http://www.colorfulwolf.com$1 permanent;
}

set $cache_uri $request_uri;

if ($request_method = POST) {
    set $cache_uri 'null cache';
}

if ($query_string != "") {
    set $cache_uri 'null-cache';
}

if ($request_uri ~* "(/wp-admin/|/xmlrpc.php|/wp-(app|cron|login|register|mail).php|wp-.*.php|/feed/|index.php|wp-comments-popup.php|wp-links-opml.php|wp-locations.php|sitemap(_index)?.xml|[a-z0-9_-]+-sitemap([0-9]+)?.xml)") {
    set $cache_uri 'null-cache';
}

if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_logged_in") {
    set $cache_uri 'null cache';
}

location / {
    # First attempt to serve request as file, then
    # as directory, then fall back to index.html
    try_files /blog/wp-content/cache/supercache/$http_host/$cache_uri/index.html $uri $uri/ /index.html /blog/index.php?q=$1;
    # Uncomment to enable naxsi on this location
    # include /etc/nginx/naxsi.rules
}

location ~ /\.ht {
    deny all;
}

location /doc/ {
    alias /usr/share/doc/;
    autoindex on;
    allow 127.0.0.1;
    deny all;
}

# Only for nginx-naxsi : process denied requests
#location /RequestDenied {
    # For example, return an error code
    #return 418;
#}

error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
    root /usr/share/nginx/www;
}

# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000

location ~ \.php$ {
#location ~ brokenonpurpose {
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini

    # With php5-cgi alone:
    #fastcgi_pass 127.0.0.1:9000;
    # With php5-fpm:
    fastcgi_pass unix:/var/run/php5-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
}

# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
#   deny all;
#}

}

Posted in Tech

Region locks on phone apps

Seriously, WHY? Why does my phone need to be locked down to only load apps from the UK? What the fuck happened to this global phenomenon called the internet? But no, now I can't even install a maps app just because it's not available in the region I live in. "Android is an open platform" - yeah, right. Only if you take the risk to root your device and install dodgy region changer apps.

The internet is becoming fragmented. This is a bad thing. I am ashamed of the time I live in. We used to have shit technology and great technological freedom. Now we have great technology and no freedom.

Posted in Tech

Last month's search terms

Here's a selection of the stuff people search when they end up on my blog:

  • estate agents fucking
  • great felling
  • difference between a tourist and a traveller (x10)
  • rainy sun set photo
  • motorbike horse
  • battleship island
  • ambition vs satisfaction
  • i hate living in japan
  • good anime to watch
  • fridge disposal japan
  • puke train
  • wolves have good peripheral vision
  • loon desuka
  • "friend's glasses"
  • monkey scratching butt
  • what colors is a wulf
  • fired in japan hello work
  • getekende plaatjes van frans bauer
  • should we hate the japanese
  • bad banking experience
  • barclays sucks

Posted in Tech

MtGox

It's a strange story, that of MtGox, the most well-known Bitcoin exchange site on the internet. They were among the first to promote Bitcoin, to offer a trustworthy service that allows you to buy and sell Bitcoins. Then they got hacked and had to start all over again.

But they started over really well and at the right moment, just as Bitcoin was gaining traction. During a time where almost every week there would be a nice report about yet another Bitcoin site getting hacked and clients having their Bitcoins stolen, MtGox slowly rebuilt. They had my trust because they had already gone through all that. They learned the hard way that you have to be ultra-paranoid if you're running something as vaporous as a virtual money exchange service.

MtGox endured, but strangely enough this is not a good thing. Too large a portion of the Bitcoin economy today is dependent on what happens to MtGox. They've become the enemy of Bitcoin, in a sense, by being a centralized institution that is, simply put, a risk to the distributed network. I'm not just talking technically, but by being in the public image people equate what happens to MtGox as something that happens to 'Bitcoin' as a whole. This is very much against the original concept. The conclusion, therefore, is obvious:

MtGox must die so that Bitcoin may live.


Disclaimer / rant note: I have been waiting for two and a half months for MtGox to transfer me my balance in Euro's via bank transer. They were 6 separate transactions, of which I've only received 3. The only thing I ever heard about why the remaining 3 took so long is that they were "in a queue waiting to be processed by the bank". The utter helplessness of the MtGox staff made it clear that this was not the company to deal with any longer. So I changed the Euro's into bitcoins and rescued them from the MtGox account. A sad day.

Posted in Tech | Tagged ,

A faster blog

Even after moving to EC2 site performance was still crap, at least according to Pingdom. Every time I access the site I get <500ms response times, but Pingdom never manages to get a response time lower than 5 seconds. Time to fix that.

The first thing I did after the downtime a couple of days ago was to set up Cloudflare, and it seems to have helped greatly. After that, I moved my Wordpress theme's resource files to an S3 bucket. I also copied the uploaded images to S3, but I don't have a plugin for Wordpress yet that will do it automatically for me. So I'll have to keep doing this manually in the future, which is a bit of a pain. Besides speed, it also saves money to host static files on S3, as you only pay for data traffic and not instance usage.

The last thing I set up is BackWPup, a Wordpress plugin that backs up the database and/or Wordpress files to various clouds, including Amazon S3. I already had database snapshots on Amazon RDS but this gives me just a bit more security, especially cause I know how easy it is to get the backups out of S3 in a format that's easy for me to restore.

End result: blitz.io reports ~50-100ms response times. Pingdom response times went down to ~800-900ms, which seems a lot more reasonable. I'll run with this for a while and see how it goes. It's fun to play with professional server setups, but if this new blog setup ends up costing me more money than it did before I would definitely consider moving back.

Posted in Tech

Blog is back online, sort of

Well, that did not go well. Only days after moving to Amazon I now have an EC2 instance that consumes 100% CPU usage and doesn't allow me to log in any more. There's three options: 1. I misconfigured the server and broke it, 2. I configured the server correctly but Wordpress broke it, or 3. somebody hacked into it and broke it. My first instinct was to suspect hackers, since the instance was working perfectly fine for over 6 months, but as soon as I installed Wordpress on it and gave it a public IP, things broke. Might not be Wordpress-related, it might just be that since the public IP address was attached to it, it became a target. Ugh. For now, this blog is hosted on bluehost again. At least the database is on Amazon RDS..

Posted in Tech

Migrating Wordpress to Amazon EC2

This is not quite a guide to migrating your Wordpress blog to EC2, but having just gone through the process I thought I'd highlight some of the stuff that I had to deal with.

Why move?

25449560

For me, this was obvious. My blog was previously hosted on Bluehost's shared hosting plan. Lately response times have been getting worse and worse and I've even seen the occasional downtime. They were very much worth the money when I first signed up with them, but times have passed them by. I already have the infrastructure in place in EC2 to self-host, so it's about time I made the plunge.

Setting up the Amazon environment

This involved setting up an RDS instance with a micro database to host the Wordpress database and granting access to it from the EC2 instance. After that was done I had to install PHP + MySQL bindings, and Apache. What a pain this turned out to be. The default Apache install did some things that  I really didn't want, and finding the right configuration file to edit took forever. In the end I managed to enable mod_rewrite support (which was installed by default but not enabled for my www dir) and turn off folder indexing.

I mounted an additional volume to my instance for the uploaded files, a collection which has grown to several hundred megabytes over the years. As usual, Amazon's management console is full of buttons and screens that I don't know the meaning of, but stackoverflow and Google helped me find the right way to create, attach, mount and format the additional volume. Symlinked it into the Wordpress folder and off we go.

Uh, nope. Forgot to allow my EC2 instance to access the RDS database. Then I had tweaked the database dump a little to use InnoDB instead of MyISAM (come on, MyISAM in 2013, really?). After the dump was complete I ran some queries to change the absolute paths of images to relative ones so I could actually test to see if my new Wordpress install was working.

Configuring Wordpress

1357963301931

First off, it took me forever to get permission right. I'm not really experienced with Linux configurations, so even tiny stuff like figuring out the username of the account that runs Apache took me a while to find out. The chmod syntax and adding users to a group and a group to a folder always baffles me, so this was a frustrating uphill climb.

But eventually I got there, and managed to produce the front page of my blog in my browser. Unfortunately none of the permalinks to articles worked, which triggered another quest to get Apache to respect the .htaccess files properly. All this followed by more chmodding, of course. Wouldn't want it to be too easy, now.

At this point I started to realize that a lot of Wordpress plugins simply won't work. I used WP Minify to minimize and bundle the CSS and JS files of my blog. Bundling the CSS files worked fine but any request for the JS bundle resulted in an http 400 error. Eventually I found out where PHP logs its errors, which showed an obscure message about being unable to fork(). A little Googling suggested that PHP didn't have enough memory to spawn a new process, so after spending yet more time finding out where php.ini lives I increased PHP's memory from 128MB to 256MB. This didn't solve the problem, so I followed the second suggestion, which was to add a swap file to my EC2 instance. I would never have figured out how to do this if someone didn't already paste the exact commands in an answer on the internet somewhere. Regardless, it didn't work. According to a third answer EC2's micro instance just doesn't have enough memory, and raising the memory to 1GB should fix it. My reaction to that answer: "Fuck you, PHP, you don't need a whole fucking gigabyte of memory just to minify 100kb of javascript", so I turned off the plugin. Piece of crap.

The next problem I ran into is email. In all fairness, I haven't actually bothered to set up PHP's email system properly. Apparently email send settings are a global thing in PHP, probably hidden away in that giant config file somewhere. I installed WP-SES instead, which lets you send email via the Amazon SES email service. However, this plugin crapped out on me every time after I filled in my secret key without so much as an error message anywhere, just showing a blank page without any way to proceed. I uninstalled it and tried Amazon SES DKIM mailer instead, which worked like a charm. Well, mostly.

Another useful plugin I had installed was a backup plugin that creates a dump of the Wordpress database every week and emails it to you. Sadly, this plugin failed to send email with the SES/DKIM mailer plugin installed because... no idea. I uninstalled it too and now rely on Amazon's RDS snapshot service. I don't feel confident about that yet though. Need to spend more time understanding what it actually does.

The result

The blog feels faster to me. Pingdom stats tell me it's not. But it's not worse either. I'll monitor it for a while and post an update later. Before undertaking this endeavor I seriously considered writing my own blog platform in Django just for shits and giggles, but as I started listing out features of my current blog I realized it would take me a long time to replicate the same behavior, and I'd probably never finish it. Migrating the whole Wordpress shebabble took me at least 6 hours, but at least now it's under my own control. I get to improve it as I see fit.

Still to do

Speaking of improvements, some of the obvious improvements I want to make have to do with static files. I want to try out Amazon's content delivery network for resources and uploaded files, in combination with S3. I did find a plugin for Wordpress called WP2Cloud which lets you upload files to S3 instead of the local filesystem. But to enable that you have to install a custom cloud storage engine for MySQL. Seriously? My database is managed by Amazon so I'm not able to do that, but even if it was possible, that is probably the worst possible solution to the problem. Wordpress is already horribly linked to MySQL, don't make it worse by requiring a weird MySQL extension. Then again, this is PHP-land, where anything's possible, and if it's ridiculous enough, it's already been done by someone.

Instead of weird Wordpress plugins, a quicker and easier solution would be to slap on Nginx to serve the static files and bypass Apache. It would also be easy to modify existing content to point to S3, but I'm not sure how easy it is to insert new images into S3 dynamically (without having to resort to changing my database storage engine). Something for another time, perhaps.

Clipboard01

Posted in Tech