Startup on a budget: accepting recurring payments in the UK

It's been a busy couple of weeks. Our Flawless.QA experiment is moving along nicely and we're improving the quality of many a website. We've even started blogging about why maintaining the quality of your site is important. The next big step is to start accepting payments. For a startup, accepting payments is really the only thing that must work flawlessly. Everything else you can worry about later, but payments must be handled perfectly and professionally. Unfortunately that also means it doesn't come cheap, especially if you're not based in the US.

Handling recurring payments is a step above handling regular payments. If you were accepting one-time online payments you could get away with doing almost everything in-house, on an SSL connection, without storing credit card information. But we need to keep the user's payment details in the system so we can bill them again the next month. And that means PCI compliance, or rather, paying a company to be PCI-compliant for you so you don't have to. And that's a good business to be in, if prices of these services are any indication.

So, what options are there? The answer is: tons. But they're all not right for us. We're trying to get our site up and running with a bare minimum amount of money, so spending 69USD for a beautiful recur.ly subscription billing and an additional 25GBP for a SagePay payment gateway is just not an option. I've spent many an hour scouring the internet for useful solutions, and recur.ly+SagePay was mentioned several times as someone's preferred solution. Other solutions include Braintree, which will send you five gazillion forms to fill out and has a minimum monthly fee of 100EUR, and the un-Britishly named Authorize.net which will charge you a 50GBP setup fee and a 20GBP monthly fee (and will also send you a bunch of forms). Unlike Heroku, where the platform costs will hurt you after you've actively started using it, payment costs will bite you right from the start.

Before I tell you which provider we ended up choosing, let me mention one more option here that I only learned about after we finished our initial integration: Paymill. They seem genuinely good and I'm going to give them a try real soon. Paymill is also discussed in this thread on Hacker News about taking payments online in Europe, a must-read if you're looking for info about online payments in Europe.

The solution we ended up going for, at least for the moment, is the one big name left unspoken so far: Paypal. Much has been said about Paypal, not much of it good. You will have no problem finding horror stories about their API, their customer support, their random freezing of people's funds. But to the consumer they're a symbol of trust and familiarity. Implementing payment logic is indeed not straightforward at first, but it does make sense. Since we're Django-based, the django-paypal library seems like a good choice, but it did not work out of the box, and neither did the supposedly up-to-date fork, but it took only a small effort to get it working.

The paypal sandbox is one massive WTF. Creating dummy accounts and managing who is logged in where is a massive pain that is only made more terrible by the fact that you get logged out every other bloody minute. You'd think that they'd set the timeout to something higher on their sandbox, but no, you will suffer in inconvenience. To make matters worse the sandbox works differently from the real paypal store. Other than that I had a hard time figuring out how to make our paypal store accept credit-card payments from people who have no paypal account. I found out there's a package called "Paypal payments standard premium" which will let you do this, but it's not documented very well on the paypal site. After selecting the premium package I suddenly received an email from paypal thanking me for my purchase without mentioning any price. Trying to find out the price online gave me conflicting answers so I emailed paypal about this and received an answer two hours later: the premium package used to cost money, but now it's free. And there was much rejoicing.

The other thing we found out while testing our live paypal store is that paypal is very strict on which cards it accepts. It flat-out requires a contact phone number and will not forgive if you fail to exactly copy the billing address belonging to the card. I had to try several times with my UK card, something which will definitely be horrible for conversion rates. For some reason my Dutch card worked fine on the first try, but my friend's US card did not work either. It's my main cause of worry right now, and part of the reason I'm putting this blogpost out there: to hear other people's experiences and find out about alternatives.

If you're a UK-based startup, and you have no money to spend, there's really only one option: Paypal. I sincerely hope that Paymill can disrupt that status quo.

Posted in Tech | Tagged , , ,

Tuning gunicorn and Django performance on Heroku with blitz.io

Before starting to promote your startup to the public you really need to make sure that your landing page can handle any kind of traffic that will come your way. Blitz.io is my tool of choice. Heroku integrates with blitz and sets you up for a free account right from the start. Very nice.

Here's how Flawless.QA's landing page performance looked like, serving a page directly from memcached:

Note that we're not tuning Django here, or database performance. Our Postgres database is running on a different EC2 instance behind pgbouncer and should not have any effect on the results. This guide will show you how to set up your own Postgres instance. I suspect that using Amazon's RDS will be even easier than setting up Postgres ourselves, but that's something to look into at a later stage.

Based on Heroku's recommendations, the setup above is using gevent so we can have asynchronous workers. This seems to be a point of controversy, as Heroku's docs mention gevent but don't actually show you how to install it and set it up in your Procfile. Here's how our initial setup looks like:

web: python manage.py run_gunicorn -b 0.0.0.0:\$PORT -w 4 -k gevent --max-requests=500 --preload

The easiest thing to tune seems to be the number of workers, so let's try 9 workers instead of 4:

Well, that went to shit real fast. Performance is worse than with 4 workers, and if there's many concurrent requests they all fail, not just a small portion of the requests, which is what happened when we had 4 workers.

Now let's see what happens if we turn gevent off:

Perfect! Response times fluctuate a bit around the 200 requests per second limit, but every single requests is served quickly and correctly.

So why does gevent not give better performance on Heroku? In my test case there was nothing infrastructure-related that should have posed a problem, as the response was fetched from memcached directly, and the database was on a different instance. I'm guessing it might be because the overhead of gevent is a bit too much for a Heroku dyno to handle, especially when you're using a large number of worker processes (in gunicorn). I also wonder if this result might be more in favor of gevent if we count web page requests that include loading the static files references on the page. In our case this is a non-issue though, as our static files are hosted directly on Amazon S3, so I can't see any reason to turn gevent on when you're running a similar setup on Heroku.

But don't believe anything you read on the internet. Benchmark it yourself for your particular use case.

Posted in Tech | Tagged , , , ,

Thoughts on post-modern web development

We're getting ready to (p)(re)launch Flawless.QA with some new features that I am quite excited about. That sounds a bit like a marketing phrase, but weirdly enough I am really genuinely excited. Brian and I worked hard to get the project to the state that it's in now, and it's about to pay off. Here's some short programming-related thoughts on the stack that we're using.

Django-south: I can't believe I've lived so long without using this. Well, actually, I can, because at Potato London we're mainly on the appengine platform, which doesn't play nicely with South. That's a terrible shame because django-south is one of the two great revelations for me during this project. Database migrations have never been easier.

Django-celery: this is the other great revelation for me. Celery makes it extremely easy to set up background tasks and periodic tasks. It was wonderfully easy to set up and works brilliantly. Django-south and django-celery are the two must-have modules of any serious Django project.

Amazon Web Services: scares the shit out of me still, but I can taste the potential. It tastes like mango. The amount of free things that Amazon gives you is amazing: a free machine with an operating system of your choice, free super-fast static file storage, 2000 free email sends per day, 30GB of storage, a 20GB database with another 20GB for automatic database backups. Seemingly the only catch is that the free tier has a bandwidth cap of up to 15GB, after which they start charging you money. We haven't reached that yet so I've really no idea how much it's going to cost us.

Heroku: I feel a bit sorry about Heroku. We signed up enthusiastically for their free services and it helped us got our site in the air just a tad bit faster than if we had to configure it from scratch on AWS, but we don't even have paying customers yet and I already feel constrained on Heroku. The biggest limitation is the database. The free Postgres database Heroku offers is useless with a 10.000 row limit, and the next step up is 'not recommended for production' and artificially crippled. Even if you did go for that, it would cost you $8 per month and you get a worse product than with the free Amazon Relational Database Service. So yes, the database really is the big blocker for Heroku.

Note that I'm still a big fan of Heroku. Their developer-oriented product is beautiful to behold and calming to interact with. Finding out how to do something on Heroku is an order of magnitude more easy than it is in Amazon Web Services. AWS adds a lot of clutter and worry to my already messed up brain, but Heroku takes it away. Too bad it also takes away a lot of your money.

Cloudflare: not sure if want. We've set this up as our dns server because the nameservers of the company we bought the domain from seem terrible, but Cloudflare introduces a lot of things that we're not sure of, in particular the page caching mechanism. It seems like it could end up being a danger, and given that dns-level changes take a long time to propagate, there's no easy solution if we mess something up. I'll need to read up a bit more on what they do before I can decide what to think of Cloudflare.

Postgres vs MySQL: One of the eternal debates. We went with Postgres because Heroku gives it to you by default, and Django is pretty much database-agnostic. It's my first experience with Postgres having come from a long background in MySQL. So far I have no problem with either, but I know a lot of MySQL tricks related to configuring it to act more like a proper RDBMS (something which Postgres does by default), but also related to scaling and backing it up. It seems that we won't have to worry about this at all if we go with Amazon's RDS though, which is essentially a managed MySQL database service. Migrating the data worries me..

Appengine: I mention this here because my previous project was developed on Google Appengine. I can't help but wonder how that project would have turned out if we did not use appengine. Sure, you would lose a lot of the automatic scalability that appengine does, but in practice, having that automated scalability meant that I had to spend a lot of time learning appengine's very-specific APIs that are totally useless elsewhere. If we didn't use appengine I would have had to spend that time on a much wider variety of topics, which would surely include database tuning, configuring webservers for static file hosting, managing virtual machines etc. And that knowledge would have carried over a lot more easily into a next job or a startup. That said, appengine did teach me a lot, especially on how to scale things, and that will definitely come in handy in the future.

Posted in Tech | Tagged

Flawless.QA: automatic error finder for your website

Your website. Flawless.

Today is the day I get to reveal what we've been working on for the past two weeks. And here it is: Flawless.QA is a service that automatically checks pages on your website and notifies you via e-mail about mistakes whenever there's an update. The mistakes that we can currently detect are spelling and grammar errors, and broken links, but we're working on extending that.

At the moment we only do about one check per day, roughly. We're already making plans to improve upon this in the future, but we wanted to get our idea out to the public as soon as possible. Please give it a try, and let us know what you think. I promise I'll reply to you personally if you mention this blogpost ;)

 

Posted in Tech | Tagged