← Newer posts

Tech

Older posts →

Beautiful days

Posted on 2012-09-22 11:10 by rv

Today is another beautiful blue day. The weather in London has been contrary to its reputation lately, much to my advantage. This is a snap of a residential street near where I live. As you get closer to the subway station the neighbourhood rapidly deteriorates into a wretched hive of scum and villainy. But out where I live it's not too bad.

The reason  I went out today is the cold. The people I'm staying with are, to say the very least, peculiar. They're very nice and they're great at leaving me alone, something I much prefer. But they pretty much keep the heating off because "it's still summer". As a result I felt too cold to comfortably work in my room so I went to a local café instead. My intention was to deal with my taxes and finances, but after I arrived I realized that I forgot to bring my internet banking login things. Oh well. I guess I'll just do what I can.

I've been toying with the idea of integrating my twitter posts and facebook posts into this blog, perhaps as another column on the side, perhaps with some form of security measure for the facebook stuff that I only want my friends to see. The longer I have a presence on the internet, the more I feel the need to gather up my 'influence' in a single location. Facebook and Twitter won't last forever. Before you know it they'll become unpopular and get taken over by another company which just happens to misplace a backup and ends up losing all your data. I prefer to be in charge of my internet history myself. Now all I need is some free time..

Posted in Daily Life , Tech

Repopulating my music collection

Posted on 2012-08-13 08:28 by rv

My hard disk crashed while I was travelling, causing me to lose about 60Gb worth of music. I say lose, but I did of course have backups, namely on my computer back home, and on my iPod classic which is with me on my travels. I wanted my music back on my travel laptop while preserving the folder structure that I used to have. (I know, I'm picky like that). Unfortunately I can't just copy all the music from my home pc (which has the folder structure I want) because my home's bandwidth is nowhere near fast enough. It would take a month to get everything back, so I had to come up with another solution.

I do actually have my music with me: on my iPod. But the hugely annoying part of the iPod is that it forces you to use iTunes as an interface to your music, which completely disregards any structure that you used to have for your files. Apple does not like files. I think they're wrong. I managed to copy all my music back onto my laptop, but it's all in Apple's folder structure, eg. "F04/XCDF.mp3" is a song. Boo.

So I needed to get the folder structure of my home computer and the data of my iPod's music. I whipped up a quick python script that reads id3 tags from  all my mp3s and ran it on my laptop's obfuscated collection, outputting a json data file. Then I Teamviewered into my home PC and ran the same script on my correctly structured collection. Copying the 2mb of json metadata was quick enough, and after that it's just a matter of matching fields. Although the collections are out of sync by about 3 months the script seems to catch nearly everything correctly. The remainder, eg. the ones without ID3 tags and the files that only exist in the newer collection I can move manually without too much effort.

The resulting scripts are ugly and utilitarian, and almost not worth sharing. But you never know, someone might find a use for it, and I might need them again if I manage to crash my hard disk again, so here they are: id3er.py iterates over all mp3s and builds a json output file with id3 data. It requires Mutagen to work. Mover.py compares two json files and moves files from one location to the other.

(And in retrospect, comparing hashes would have been much easier >_<;)

Posted in Tech

On the move again

Posted on 2012-07-09 21:26 by rv

I moved my stuff into storage today. There's this service in the UK called Big Yellow which offers giant storage warehouses all around London. Getting it all set-up was very easy. All I had to do was show up with my stuff, sign a contract and move it in.

My stuff, as it turns out, is not that much. I have one reasonably large box that includes some electronics, some trinkets and other not-very-important stuff. Then some bicycle panniers that are useless now that my bike got stolen, and three trashbags of clothes, blankets, pillows etc, two of which could probably be thrown away because I never use them.

If I didn't have plans of returning to the UK I would have no problem at all with disposing of everything, but storage is cheap, so why not hold on to your things, right? The first sign of hoarding. I try to avoid hoarding physical objects as much as I can, but digitally I am the exact opposite: I must keep everything I find, to the point of downloading youtube videos just so I can have them if they ever get taken offline.

My behaviour in the physical world is very different from the digital one. In the digital world I build giant fortresses to store my information, quadruply backed up to places all over the world because I'm paranoid of losing it. In the physical world I'd rather have as few things as possible so I don't have to worry about losing anything important.

Even my primary computer, which used to be extremely important to me, is now less important than ever before. Five years ago internet was slower and not as widely spread as now, especially when travelling to peculiar places. Things were not standardized yet. There was no Facetime, no cloud sync, no Dropbox. But these days it's easy to get wifi almost anywhere, or otherwise a 3G dongle. Photos get backed up instantly, internet connections are fast enough to remote log-in to your computer in another country, or even stream video from it. Every month there is an advancement in technology that reduces the need for a carefully maintained primary physical system.

I'm travelling to New York soon, and Japan after that. There's a ton of things I need to arrange before I leave, and a lot of people still to contact to set up meetings. But most of what I've been dealing with this past weekend is my apartment. I've blogged before about what I had to deal with related to getting my apartment cancelled. This apartment has been nothing but a burden on me, and I am very reluctant to get an apartment again when I get back. It's safe to say after one year of living in London: I don't like London. I've been to worse places, but I've also been to better. It seems folly to invest money into a contract that forces me to stay in one place for at least six months.

Rental prices in central London are ridiculous. Even in the outskirts of London they're high, and you have to deal with the tube system. Instead, there is AirBnB. There's tons of cheap short-term stays available in central London. I'm planning to do this for at least several months when I get back. I tried it last year when I first moved to London and got a... memorable.. experience. It certainly wasn't bad, but at times mentally tiring. But if the choice is between mentally tiring or being mentally dead for months on end, then I'll take mentally tiring.

Clean apartment. Book hotels. E-mail friends. Sell the last of the stuff. Start packing. Have adventure.

Posted in Tech , Thoughts

Why I will not switch to Linux

Posted on 2012-06-17 15:27 by rv

Linux is used by people who work from the terminal. I prefer Windows.

Every couple of years I get frustrated with my Windows installation and feel like trying something else. The moment that sparked this adventure happened yesterday as I was trying to set up Heroku with the Play framework, and I couldn't get the damn stack to run properly on Windows. The error appeared to be some JVM issue, and online posts suggested switching to OpenJDK. OpenJDK is not available on Windows. That's not very open, is it?

I like windows. I like moving them around, arranging them with hotkeys, having many of them side by side, having a fully functional IDE that does all the fancy heavyweight stuff I want. I'm not a command line person, really. I use it for git because the git plugin for eclipse is slow, but other than that I prefer a GUI. This, apparently, is weird in the circles I hang out in. If you're a developer you're supposed to be a die-hard vim addict who never uses anything else except the terminal. I think that's ridiculous and old-fashioned. People who do that are doing it because they are the most efficient in those environments, but I will challenge any terminaler with my tricked-out windows setup and prove that I am just as efficient with windows as they are with a terminal. I'm not a typical user, and my dislike of regressions in my performance do make me a very picky eater.

So, after deciding that I needed a more compatible programming stack, I decided to give Ubuntu another try. It's been about a year since I did any serious work on it, and it's good to see if it's improved since then. The short answer is: no, it did not improve. The Ubuntu Unity shell is ridiculously minimalistic and does not offer anything useful. I quickly realized this was not going to work when a friend recommended I look into gnome shell. After installing gnome shell I was marginally more happy and felt ready to start customizing it.

And that's where I got really annoyed. I tried to install a custom user theme, which seems like a fairly standard thing to do. Not so. You have to install some custom tweak tool first, then a common extension package, then a user theme extension. After doing that I found out the user theme extension didn't work because of incompatible versions of gnome-shell. Googling the solution to this took ages. One solution offered was to disable one source repository and get it from a different one. Tried this, didn't work. Another solution said to edit the python files related to the tweak tool to pick up the extension settings from a different package. This seemed to work at first, but in reality it just made the tweak tool say that it supported user themes, when in fact it didn't really when you tried to add one. Total failure and massive waste of time.

Why do I need to install twenty thousand different packages one by one to get a goddamn piece of software to work? Why I do need to edit configuration files manually to make it work? Actually, I'm ok with both of the above, as long as the end result works. Which it didn't. In Windows you just download the binary and it works. Done. No need to get required packages, no need to compile. It. Just. Works.

The next thing I tried was to tune the performance of gnome-shell. I'm running it inside a VM so performance is less than optimal. I found many tiny performance tweaks, but even basic window dragging was sluggish after all that. It was at this point that I found out that gnome-shell uses 3d acceleration, which is not brililant when running inside a VM. At least, that's what I assume was making it sluggish, it might just have been poor design.

Then I found out about lxde, a lightweight desktop environment that replaces gnome-shell entirely. It looks perhaps 10% less pretty, but it's 1000% faster. The response is phenomal and I was in love with it right from the start. This is how GUIs are supposed to react: instantly.

As it turns out, lxde was not the answer to all my problems. The file manager decided to crash seemingly at random. It was perhaps related to moving windows around and right-clicking a lot. You know, actions that you often do in a desktop environment...

Still, it didn't happen very often, and I was fairly happy. But then the next problem appeared: somehow right-clicking in Chrome immediately caused the browser to go back to the previous page. Other programs (some, not all) had the same problem: after right-clicking the context menu pops up so quickly that when you release the mouse button, the first item is already selected and gets activated, causing Chrome to go back to the previous page, the file browser to create a new folder, etc. Some Googling discovered bug reports about this exact same issue dating back to the summer of 2011, and they've still not been solved. The solution seems known, yet the patch is in C code so if you want to fix it you have to compile your own kerner/window manager/whatever. Yes, all the useless things are configurable, all things that actually matter are broken.

The thing that finally made me decide to stick with Windows is simple: EmEditor. It's a low-resource text editor with tons and tons of options and exceptional support for multiple languages and encodings. It's great for quickly editing text files and, unfortunately, only available on Windows. I had gotten it to work on wine on OSX before, so I figured that it would be a piece of cake to get it up and running on Ubuntu as well. Of course it wasn't. I ran into some dll error that I had never seen before and couldn't solve using any of the advice found by Googling the problem.

Ubuntu is simply not meant to be a desktop OS. If you want linux, run a remote shell from Windows or even OS X (I'll save my rant about OS X's GUI for another time). Compared to Windows, Ubuntu's GUIs (lxde, unity, gnome-shell) are immature, buggy and slow (except lxde), and just not a valid replacement for a proper desktop system such as Windows 7. Microsoft perfected desktop window management in Windows 2000, why can Linux still not get it right?

If you don't believe, or if you want to find out for yourself, here's the tools I'm using on my Windows machine that I use to get the best behaviour of other OSes, while keeping the performance and reliability of Windows. (who would've thought someone would say that about Windows?)

  • windowspager for virtual desktops. This is fast, none of that sliding delay bullshit that os-x pulls on you.
  • wizmouse to scroll non-active windows as os-x does natively
  • winsplit revolution to tile your windows side by side in any number of configurations using hotkeys
This is what I'm used to, and I've been unable to replicate a similar setup in Ubuntu. I've tried it several times before and always gave up at some point. I used to blame myself, thinking my experience with Linux wasn't sufficient. I'm done doing that. I learned a lot about the way this stuff works behind the scenes, and I'm certain that if I had a month (or more), I could find all relevant config files, recompile relevant source code bits, download all the right packages and eventually get it to work. It's linux after all, it's open-source and fully customizable.

But I don't want to do all that and waste weeks of my time. I just want the damn thing to work, and offer me a stable platform, with easy-to-install extensions, a reasonable set of configuration options and a performance that is not too slow. Linux just does not have that. And that's why I'm staying with Windows.

Posted in Tech | Tagged ,

A day in the life

Posted on 2012-05-01 23:36 by rv

I wake up in the morning and talk to boss number one who is currently in Australia, catching the last hour or two of overlap between our shifts. She tells me to do urgent stuff on one project even though I'm already doing urgent stuff on the other project. I scramble together fixes, get as much feedback as I can while my boss is online and delegate the remainder of the tasks to my other team members.

Then I switch to checking how yesterday's long-running tasks turned out. It doesn't happen every day, but when it does it usually has something to do with performance/speed testing or delivering e-mail. I write scripts to analyze the results, find bugs, fix them and set up a new scenario for the next test. Then it's time for lunch.

When I'm working from home I usually watch an episode of a tv show in front of my computer while I eat lunch, at the same time monitoring work e-mails and chat channels. Sometimes I deploy some websites while eating, since it's quite easy to do in terms of keystrokes and mental power required.

After lunch I work on some tickets, some of them related to the long tasks, some of them separate. I talk to my buddies and try to help them if they're stuck, and I pester them incessantly on the chat when I am stuck. I find weird behaviour on a staging site that I cannot explain away and dive in to find the cause. I end up downloading server logs, writing devil-invoking regexps and finally realize that I was an idiot and the behaviour is actually expected. I do all this while cleaning a server to prepare for another long-running task.

I go through my list of tickets, closing invalid ones and reassigning some back to the reporter, asking for more information. I have a tendency to write lots of information in the tickets, thinking that it will be easier for another developer to take over my tickets should I not find the time for it. But the other developers in my team are just as busy as I am, so that rarely happens.

Right around this time boss number two from California comes online and starts asking me questions about the long-running stuff I'm working on. I refer him to my tickets while I try to set up a new task. There's another problem I have to debug before I can start the new task, and while I'm working on that and chatting with my boss I deploy the project I'm not working on to two websites and quality check the new deploy.

Now that my second boss is awake he reminds me of a problem that I forgot to look at during the day. Something weird is happening, and I track it down to code that is not our own, so there is no user-friendly way to work around it. I ask my boss to notify our users how to work around the problem and then continue to set up the next overnight task, which is now ready to go. I fire it off.

There's always too much of everything going on; short-term work and long-term work. Work that can be done without a context and work that absolutely requires a context. Work that needs passing back-and-forth from developer to manager many times, or from front-end dev to back-end dev. Deploying websites. Performing quality checks. Sitting in on meetings that are not really relevant to my work. Writing API documentation. Writing wiki pages. If we have time to spare perhaps we'll even write a unit-test. The last time we had time to spare was 6 months ago.

I wait for a while to see if the overnight task will finish early, but of course it doesn't. I log out and close all work-related windows, although I check my work e-mail until I go to bed just in case a live site falls out of the sky.

The end.

Repeat.

Posted in Tech

What I really want

Posted on 2012-04-28 11:23 by rv

Ten Million Pounds.

I've been toying with the idea of picking up and relaunching Moodlogger, this time for real. Moodlogger has been out of the app store for months now because my Apple developer account expired, and I haven't worked on the Android version since I got my job at Potato.

Because my job at Potato keeps me so busy I don't feel like thinking about coding in my free time at all, or only very rarely. It's hard to muster up the motivation to try my hand at a project the size of Moodlogger. I will make no excuses for that. But every once in a while I get a really good idea of something that Moodlogger could do, and I'll be thinking about how I would implement that idea and how I could make it work. I find myself still very excited to work on it, and I think that if I could focus on it full-time, Moodlogger and Moodlogger alone are what could motivate me to be a success.

And when I say success, I don't mean something silly such as changing the world. I'm talking about money. The simple fact is: I've been working for over 6 years and I've got nothing to show for it. If I had no job I would last months, perhaps a year, in the real world. I am not in a position to say "I only want to work on what I like". And I want to be.

That, for me, is the main motivating factor. I hate sounding like an old fart, but I've done a lot of coding in the past; both personal and professional. Only the ideas that you are both truly motivated for, and have the time for, become something interesting. From my perspective, Moodlogger is my only lifeline right now that could make it possible for me to solve my financial troubles.

So that's my goal. I will write a pitch and then find investors who can give me the minimum amount of money to keep me going for a year or two, probably in the range of 50,000 pounds. This may sound like a lot, but I like to bet on the safe side, and an extremely frugal lifestyle isn't going to help my programming mood either. I'll also have to spend a fair bit on creating the necessary infrastructure (servers, domains, etc.) Never spend more money than you need to, but don't use too little either.

From there on it's simple: start coding, build a user base, sell the company. I'd claim higher ethics and 'improving the world' nonsense here, and that I want to do Moodlogger forever, but I simply don't have the luxury to believe in that just yet. Perhaps if I had a more comfortable monetary base to start from, I would, but I'll save that for my next project. First, it's time to bootstrap.

I don't know if this will be a success. Perhaps there are other ways of bootstrapping myself into becoming a 'free' programmer. I've got a few options lined up in the near future, but if I am perfectly honest, the only person I trust is myself, and I think my chances of becoming successful are the best if I am in complete control of the project that I am truly motivated to do. Right now, Moodlogger is the only thing that qualifies.

What I really wanted to do was write this post.

Posted in Tech , Thoughts

Shit umbrellas

Posted on 2012-04-14 23:24 by rv

I read this article the other day about shit umbrellas. A programmer, or in more general terms: an IT worker, can be deemed a shit umbrella if he is involved in all the tiny important bits related to releasing a product, such as talking to customers about minor decisions, fixing those pesky bugs that nobody else has noticed yet, making sure that the infrastructure is in place and easy to understand, etc. etc, but not actually involved in making tons of commits and writing loads of good code.

I found a lot of things in the article that applied to me. To be fair, my manager is in fact taking most of the shit and doing a great job of shielding me from everything external, but when it comes to internal tasks I find that the description fits me quite well. I do have my periods of great uninterrupted coding, but at certain times, especially just before launching a new product or a major feature, I find myself doing all the tiny things that are required to guarantee that other people can work well (maintaining branches, changelogs, checking code quality, testing/QA, wiki-writing).

The author of the original article shows a strong desire to do actual coding instead, without having to worry about the surroundings. Focus on one task, and do it well. But no matter how well you do that task, there will still have to be a person with greater knowledge about the project to make sure that the task is done well. If there is no such person then individual tasks get completed but the project as a whole has no vision. I like being the guy who makes sure that everyone knows what we're trying to accomplish, what our priorities are and how each individual feature attributes to the whole project. Sure, you can keep your head down, only look at the information given to you in the ticket and code away, but if you don't understand why you're doing what you're doing then you'll end up producing the wrong thing. Context is important, and it's the team lead's job to provide context to the developers.

It's not always easy. There's all kinds of different programmers, and it's sometimes hard for a "big-picture" dev to see eye to eye with a "just-give-me-the-ticket" dev. I've made plenty of mistakes related to this in the past. One such mistake is giving extremely detailed descriptions of what to do in the ticket description to a big-picture dev. Big-picture devs don't need that, they just need to know what's needed and why, and they'll figure out what to do for themselves. Vice versa, you can't have a "just-give-me-the-ticket" dev work on a ticket that doesn't include the required information, or they'll mess it up.

There was such a guy on my team in my current job. He preferred to just look at the information given in the ticket and then be left alone to work on it. We did not provide him with enough context to be able to finish the tickets successfully, which led to frustrations on both sides. In the end we didn't part on good terms, but hopefully we're both a little bit wiser. I consider it my greatest professional failure to have spotted this too late, and I wish I had dealt with the situation better. That said, I do believe that it is each developer's responsibility to ask for more context if things are unclear. (In my example, I ended up clashing with this developer because I believed he needed more context whereas he thought he could figure out the big picture by himself. He was wrong.)

Consider a startup. You do NOT want a "just-give-me-the-ticket" guy as a founder, ever. From a guidance point of view these are exactly the kind of people who cannot be left alone without specific instructions. For startups you absolutely need someone who sees the big picture and who can move towards the goal without specifically being told what to do. Imagine someone with basic knowledge about coding but with a clear understanding of the big picture and the goals of the startup (and its founders). Compare that to someone who is brilliant at coding but needs to be given very specific problems to get good results. I think you get my point.

The original author proposes the point of view that a shit umbrella is someone who takes care of the fallout while 'real coders' do the work. As a professional shit umbrella I'd like to propose the opposite view: 'just-give-me-the-ticket'-developers are interchangeable cogs inside a machine whose operator is the team lead. They are the ones doing the least important tasks because those tasks are the easiest to contain in a ticket. For doing real work you need to get the big picture, and in my opinion that is the most important quality a programmer can have.

Grok the project.

Posted in Tech , Thoughts

A genetic algorithm in Python

Posted on 2012-04-03 19:10 by rv

Here's a genetic algorithm I wrote in 36 lines of Python. It can be shorter still, but the readability, which is already suffering, would suffer too much.

Posted in Tech | Tagged ,

dropcopy: backup files using dropbox

Posted on 2012-03-21 16:36 by rv

People reading this blog know that I've been looking for a good solution to get my photos from my travel laptop onto my server pc. Long story short: I don't have time to build what I want, but here's an alternative that works right now.

The dropcopy script will move all files in a subdirectory of your Dropbox folder to a different folder, deleting them from the dropbox folder. Using the script you can backup terabytes of files using even the free dropbox account. Here's how to do it:

  • Before traveling, schedule dropcopy to run on your server computer every x hours or days.
  • Copy your newly made photos to your dropbox dump folder. Dropbox syncs them to all your computers.
  • The server runs the script and moves the photos out of the dropbox folder, freeing space.
  • Your travel laptop syncs the dropbox folder so you'll know immediately when you can backup more files.
I know, it's silly and it's simple. But it works.

Posted in Tech | Tagged ,

A competent programmer

Posted on 2012-02-10 22:12 by rv

Most programmers know what's to focus on when trying to become better in their profession. This website offers a very good list. But lately I am finding increasingly that there are other things a programmer must be able to do in order to be truly successful that technology-focused sites often neglect to mention.

These are traits that are not often focused on by a lot of programmers I know. They're implicit things that some programmers just seem to be forgetting, or don't place a high enough value of. But I think if you don't possess the traits mentioned in this post then you can never be an effective programmer. Here's some of the things I found extremely useful on my current project. They apply to projects done in teams but you can just as easily apply them to lone wolf projects (but you'll be playing out all the roles mentioned below by yourself).

  • Deal with crap legacy code. Know how to deal with badly written and undocumented legacy code. Don't refactor a solution that's worked for years if it's not necessary. Don't break shit. Get to know the code before attempting to make major changes in it. If you are thinking that you will not be confronted with a situation like that then you are naive and should go back to school.
  • Know your priorities. Yes, designing that new e-mail framework from scratch is a great challenge, but there's also 5 minor bugs that break the product. Don't indulge yourself on what you like better, don't rely on other people to prioritize for you, know what's important yourself. This is one of the most important skills a programmer can have, especially when functioning in a team.
  • Communicate. Don't do something without confirming what it is you're actually doing. Don't try to get away with doing things according to the problem description if you know the problem description doesn't make sense.
  • Know better. People will ask you to do stupid shit, then change their mind and ask you to do even more stupid shit. Expect this and program your system accordingly so you don't have to change it all the time while you're in requirement limbo. (Which can last months, trust me.)
  • Don't trust people. If a person tells you that a piece of code is not in use and you can delete it, ignore him/her completely and do a full-text search. The only thing you can trust as a programmer is full-text search!
  • Don't trust new technology. If you have a crappy solution to a problem that has been working fine for years, don't change it. Almost always there will be business logic behind the crap solution that you are not aware of and will unknowingly refactor away, realizing only weeks later that you broke something important. (Unit tests avoid this problem, of course.). This point is a bit tricky and depends on the case, the crappiness of the original source code and the wackiness of the framework you're planning to replace it with.
But the most important skill is, in my opinion: achieving the right balance between writing clean code and getting things done. This is by far the most difficult thing to master for any programmer, and I have seen countless of my brilliant colleagues (and myself too) fail horribly at this. At some point in your project you will have an upcoming deadline that you simply cannot complete in time. Then what do you do? Do you try to do things properly and deliver an incomplete product? Do you fudge things up and deliver poor quality code? And where do you draw the line?

This is a bit tricky, but it's nothing compared to deciding on which technologies to use months before a deadline. That's when you really need to be able to estimate what the project's going to need and what technologies should be used. Compromises must be made and you must think about how much crap you'll tolerate given the deadline. That is the real skill of a programmer. It's not about achieving technological brilliance in every facet of the project, it's about getting the product out the door, and making compromises along the way to do it.

 

Posted in Tech | Tagged
← Newer posts

Tech

Older posts →