London Overground’s biggest problem

Nearly every working day I take the London Overground to work. I am quite lucky: my working hours are flexible so I can avoid rush hour. The stations I get on and off are near the first stop and the very last stop on the line, so I am pretty much guaranteed a seat every time. The trains are airconditioned in summer and nicely heated in winter, which is a fantastic improvement on the Bakerloo line, whose trains may be the draftiest place in the UK. There’s just one thing that bothers me *a lot* about the overground, and it’s not even the delays. But I can’t talk about my gripe with the overground without talking about the delays.

There’s always been delays on the overground, ever since I came here 3-4 years ago. Sometimes there’s this sign in the carriage telling you that x% of the overground trains ran on time in the last quarter. I’ve occasionally snapped pictures of those signs because I never once got the impression that things were getting better. TFL is pretty good in providing statistics on their website about their service, and I’ve done the math once to check if it actually was getting worse. Turns out I was wrong on that one. But things aren’t getting much better either, and I can see those numbers just as easily go down again in the near future. There’s nothing that inspired confidence in me to believe that there’s an ever-decreasing amount of delays.

This leads me to the reason I hate the overground: the utter lack of accurate real-time reporting. The most common occurrence is a delay of under ten minutes. TFL’s strategy of dealing with this is by not dealing with this. Every TFL employee just stays absolutely silent and hopes nobody notices. They certainly won’t be updating the arrival times on the signs until well after it’s too late. This is not a hard problem! Unless the staff are utterly and disastrously incompetent they would be immediately aware of the delay. They’ve got a website that everyone in London uses to check the delays which they could update immediately, but TFL deliberately chooses to take no action whatsoever whenever this happens, I guess in the hopes that the problem will magically go away?

Small delays don’t always stay small delays, though. Sometimes a train needs to be taken out of service, or is delayed even further to even out gaps in the service, or any other reason really. That’s perfectly fine. Once something’s bad happened it of course make sense to return to normal service by whatever means necessary. But you need to report that to your customers, dammit! If I’m standing there at the bloody station for a train that’s already ten minutes delayed without receiving any information at all about the state of the service, of course I am going to be even more annoyed if I suddenly hear that the next train is cancelled and I have to wait the better part of an hour for my next service. Whereas if TFL had reported immediately on the initial delay I would have stayed home just a little longer, checked the situation from their website and would have been much better off in the end. I wouldn’t even have thought worse on TFL in that case, but if they make me walk to the station and make me wait in the winter cold when they could have told me already that there were delays, that’s what really pisses me off. The problem is not the delays, it’s TFL’s lack of reporting on it which causes annoyance.

I realize that this is not a world-ending issue. No one will die from this problem, nor does it seriously affect the days of anyone involved. I can work from home, others may take taxis or busses, and in the end everything ends up just fine. But what really bothers me about this is that it’s completely preventable. There is absolutely zero need for me or any other passengers to get annoyed at TFL for the delays if they just improved their reporting. There’s staff at every station on my overground line, which is great, but they’re all doing fuck-all whenever there’s a delay when in fact they could be reporting the delay immediately so the TFL site can update. Not reporting on a delay until X minutes have passed is a terrible idea because it’s not at all uncommon for smaller delays to snowball and become something worse.

Rant over. I am working from home today.

Posted in Daily Life, Rant, UK | Tagged , , , | Leave a comment

Be direct

People are too polite. Politeness causes misunderstanding, especially across cultures or across nationalities, but even within the same culture it can be a problem. British people are sometimes ridiculously polite and indirect to the point where it serves no logical purpose and only slows down social progression.

Example: two people are getting to know each other and want to be better friends, but neither is sure about the other whether they want to improve their relationship or not. They are not sure because, at the end of their meeting, they repeat set phrases such as “That was fun, let’s do it again some time”. Then, when setting up the next meetup, quite often the tone of the next message is something like “Hey, last time was kind of fun. Do you maybe kind of perhaps in the future want to do something similar again? Only if you have time though. I wouldn’t want to impose on you or anything..” – Totally British tsundere.

Don’t fucking do this. There is absolutely no need to make communication this complicated. Just say “Last time was fun. I want to do X with you again. Do you have time Monday?” It really is that simple. There is no need to beat around the bush. Just say what you think. No one will think worse of you, or if they do, you are not a good match and there’s no point in hanging out anyway. Be direct.

I see this kind of behaviour a lot more when interacting with native English speakers, or in a group that is largely composed of people that are very adept at speaking English even if it isn’t their first language. The more adept you get, the more subtle the language becomes. This is not a good thing. At least not in this context. When it comes to social situations it is very important to be completely unambiguous. I’ve noticed this in Japan a lot while hanging out with people from various countries at the same time: eventually people realize nobody gets the cultural subtleties that they put in their speech, or they just don’t translate well to English, so after a while people tend to become more direct with each other. This is a great thing because it saves time for everyone.

Playing with language subtleties is fun when you’re having pub banter or lifelong friends or just two native speakers with an interest in language, but as soon as you’re not 100% sure that the other party will interpret your signals correctly, be direct. Use more easily understandable phrasing. Don’t leave things to be misinterpreted.

That’s for the sending end. As for the receiving end, I’m very comfortable with taking people at face value and not spending ages trying to analyze what they’re trying to say. I do find myself occasionally encountering people who throw linguistic subtleties at me. I take “That was fun, let’s meet up again” to mean “That was fun, let’s meet up again”. Even if I usually get that there is (or might be) a deeper meaning behind something, I am very comfortable pretending not to understand it. As a result people have become more direct with me and life is simpler for both me and the person I’m interacting with. It saves me a lot of mental processing power to spend instead on things that I enjoy. Miscommunication is not a thing that I enjoy.

Keep it simple. Baka.

Posted in Daily Life, Thoughts, UK | Tagged , | Leave a comment

The Peak District

IMG_0062PS

“Do you get to the Peak District very often? Oh, what am I saying, of course you don’t.

 

Posted in Photography, UK | Tagged | Leave a comment

Better or worse

A few days ago, as the London Overground was delayed yet again, I looked up at one of those ‘9x% of our trains ran on time in the last 3 months’ posters, and thought to myself: “there’s no way the overground has improved during the time that I’ve been in the UK. Or anything else in this country for that matter”. But it’s easy to complain about things being crap without actually checking them. So I did some research to find out if the things in this country that relate directly to my life have gotten better or worse during the last five years. It’ll also be a good summary post to compare with five years from now, when the effects of the brexit will have come into full force.

Public transport

To get to work I take the London Overground from Watford Junction to Euston. TFL has nice little yearly reports on the performance of the Overground [1], which they define as (iirc) ‘percentage of trains that ran within 5 minutes of their scheduled time’. According to their yearly reports, the Overground as a whole, which includes a lot more lines than the one I take, had a performance of 94.8% in 2011, and 95.2% in 2015. The only data I could find pertaining specifically to my line date back to 2013, when the Watford-Euston line’s performance was 96.64%. Last month it was 96.96%.

Things are not getting worse, apparently. The cost of taking public transport is increasing, but that’s roughly in line with inflation. What (seriously) worries me is that TFL is constantly claiming that they’re pouring all their profits right back into improving the network, yet they can barely manage even a 1% improvement over 5 years, and there’s still signal failures and train issues almost every day. Worrisome.

Lastly, TFL reporting absolutely sucks. It’s happened to me many times that a train was delayed 5 or more minutes, yet the online status report showed that there were no delays. I’m not sure if this is only happening for the Overground, but the tube lines do seem to get faster updates when something goes wrong there. It’s easy to tell because the Bakerloo line shares its tracks with the Watford-Euston overground, and the status on the website updates much quicker for the Bakerloo than for the Overground when something goes wrong. I don’t think this has improved much over the years, but don’t have enough data to prove it.

Tech salaries

Another area that obviously interests me is what kind of salary I can earn working in IT in London. I found it quite hard to get data on this as well. There’s plenty of numbers floating around, but difficult to compare them because there’s so many different job titles and qualifications out there. Speaking from anecdotal evidence, the numbers I found for average/higher-end Java developers in 2016 are higher than the Java job that I held for a while in 2010, but not significantly higher.

Lacking any other solid basis for comparison, the closest I could find to useful numbers was the average and median salary of someone working at Google. In 2011 the median salary of a Googler was £86.800, and in 2016 the average Googler salary is £160.000. I can’t really use that to decide whether things got better or worse, since median is very different from average, but it’s still interesting data points nonetheless.

House prices

It’s amazing how quickly your opinion on house/apartment prices sways once you’ve bought something yourself. In the year that we were looking for a house or apartment the prices increased insanely quickly. When we started looking we were (barely) able to afford a small house in zone 7, but by the time we found an apartment all houses and bungalows were out of our price range.

Rightmove has some good data to confirm this [2]. In 2011 the average price of a house sold in my area was between £260k and £320k. In 2016 it’s between £390k and £430k. If I was still looking I’d feel pretty crap about this, but I guess we managed to move onto the quickly departing train just before it got away. Let’s see how the brexit affects this, though..

I compared central London prices as well, just for shits and giggles, even though I’ll never be able to afford anything there [3]. In 2011 house prices were around £850k near where I work. In 2016 they’re around £1M. That shit cray.

Crime rates

Ever since we signed for the neighborhood watch mailing list we’re getting the occasional email about break-ins, theft, vandalism and so on. Since we’ve only lived here for less than a year we don’t know how things were like 5 years ago in this area, but there’s plenty of data available to compare. Additionally, we chose our area not in the least based on the low (relative) crime rate, which I’ll get into later.

In 2011 my area had on average around 200 crimes reported per month [4]. in 2016 that’s 130. That’s a pretty decent improvement. I compared my work area in Central London as well, which went down from 230 reports per month to 180 reports. Note that areas are not the same size and not the same number of inhabitants, so you can’t cross-compare. I had always assumed that Ealing, the area I previously lived in, would have much higher crime rates, because it always seemed a lot more dodgier to me than where I live now. But I looked it up and it went down from ~150 reports in 2011 to ~130 reports in 2016. It improved less than my current area, but that’s about all I can say about that.

Road incidents and accidents

Getting out of Greater London by car is a huge pain, even when you’re in zone 7. I see a *lot* of bad drivers on the road, and it pisses me off every time. Apparently, and I still can’t quite believe this, the UK is safer than the Netherlands when it comes to traffic related deaths [5]. The statistics prove me wrong, but based on what I’ve seen people in the Netherlands drive way, way safer than anyone in Greater London. But them’s the stats, so there you go. I’d rant here about the terrible quality of the road infrastructure in the UK which I think hasn’t improved either, but I’ve no hard numbers on them, so I’ll leave it at that.

In my local area there were around 110 serious or fatal incidents reported in 2011 [6]. This number went up to ~120 in 2015. A mild increase, but hardly statistically significant. If you take a larger area of north-west Greater London, there were ~1400 incidents in 2011, which went down to about 1200 in 2015. I’m not sure if that increase is on par with the increase in car ownership [7].

Deprivation

One of the best sources of information when selecting a place to live is the deprivation index, which combines a bunch of useful indexes like income, crime rate, employment etc., and gives each area a relative ranking compared to all the other areas in England. The area I live in ranked better (lower depravity) than ~91% of all other areas in 2010 [8], but that went significantly down to ~83% in 2015 [9].

Even when checking the absolute numbers, both overall and for each important subdomain (income, crime, health, employment), my area got worse in every metric over the past five years. I really did not expect that. Given that the relative ranking dropped even further, apparently my area got worse at a a faster rate than the rest of the country. People in my area are now poorer, unhealthier, less employed and more criminal than five years ago.

This appears to contradict my crime rates findings, which suggests that the number of crimes reported decreased over the years. The only way I can explain this result is that the area of the deprivation index is quite small compared to the crime reporting area. It’s quite possible that my immediate local area has gotten slightly worse, and my greater local area has still improved. In fact, looking at my county’s score, it did improve slightly over the past five years. But not significantly.

Conclusion

I was quite wrong about the public transport I use getting worse over the years, but it didn’t get much better either, which is definitely not a good sign. The deprivation index roughly confirmed what I suspected, although I didn’t think my (tiny, immediate, local) area was that much better five years ago. The thing that surprised me the most was how much the crime rate dropped over the past five years. There’s very clear progress there, and I’m very happy with that.

For future research it would be interesting to compare this with neighborhoods in other countries. I wonder if other countries I’ve lived in have experienced similar trends in crime, transport and deprivation. My suspicion is that the UK is a bit slow to change, but that’s something to find out.

Sources

[1]: https://tfl.gov.uk/corporate/publications-and-reports/
[2]: http://www.rightmove.co.uk/house-prices-in-my-area.html
[3]: http://landregistry.data.gov.uk/app/ukhpi/explore
[4]: https://www.police.uk/
[5]: https://en.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate
[6]: http://www.crashmap.co.uk/Search
[7]: http://www.bbc.co.uk/news/uk-england-35312562
[8]: https://www.gov.uk/government/statistics/english-indices-of-deprivation-2010
[9]: https://www.gov.uk/government/statistics/english-indices-of-deprivation-2015

Cool depravity map bonus: http://dclgapps.communities.gov.uk/imd/idmap.html

Posted in London, UK | Tagged , , | 1 Comment

Roads

IMG_0095PS

I still love roads. I still have energy. I still have zest. I am excited.

 

Posted in Photography, UK | Leave a comment

Dartmoor

20150523174109PSMAN

I’ve been to Dartmoor quite a few times since I came to the UK. Because of that it feels like the ‘standard’ or ‘annual’ destination for me in this country. This spring was the second time we went there to wild camp, and it was definitely a memorable experience.

This time our trip would take us to Fur Tor, apparently one of the most remote places in Dartmoor, far, far away from civilization. Or, about 15 miles, as it would turn out. We never did get that far, though. Because as beautiful as Dartmoor is, once you’re in the no-mans-land, it does get incredibly monotonous. And why bother walking 10 miles more if the scenery’s not going to change?

Well, that was part of the reason we didn’t make it to Fur Tor. The other part is that we simply ran out of time. Given that we needed to set up the tent and prepare for dinner before dark we decided to change direction about half of the way in. Progress was slow on our way to Fur Tor because the land was very boggy. Not enough to pose a danger, but enough to get your feet wet after a misstep, and certainly enough to make you want to pay attention to every single step you take. Not a relaxed walk for sure.

20150523131726PS

While the walk would have been tolerable, setting up the tent on bog land certainly wouldn’t, and neither would it be good if we could not find a stream to camp nearby (although not a disaster since we had brought plenty of water. Sadly the stream that we were supposed to cross on the way to Fur Tor turned out to be not there, and with the OS map reporting nothing but bog land on the path ahead we decided to change direction. I say ‘path’, but there really isn’t any. You can walk wherever you like, in any direction. No paths. No humans. No nothing. We went East instead in search of another stream and less bog.

Luckily we found a usable stream and some bits of land that were not boggy. Unfortunately none of the bits were entirely flat, so we ended up having to set up our tent on a slope. Setting up the tent only took a few minutes and we had a lovely evening meal and watched the sun set. During the night I left a bit of tent flap open so I could look at the stars. It was an amazing sight.

I kept waking up during the night and sliding down my sleeping mat, which was rather slippery on the angle the tent was set up on, but it could not be helped. I remember waking up a few times in the morning, seeing some light of dawn seep through the tent flap, but the sun never quite seemed to come out, so I went back to sleep again. Until finally I got up and stuck my head out, and realized that we would not be seeing the sun at all that day.

20150525180159

The mist was so intense and so humid that the tent was full of moisture droplets, and just walking outside for a few minutes would cause lots of tiny droplets to appear on your clothes on the side that the wind was coming from. We ended up having to pack everything up while it was still wet, and went on our way back to civilization, via a different route than the one we came in on. Although visibility was utterly poor we had plenty of maps, compasses and GPS devices to guide our return. The way back took us past a military practice area where a bunch of old rifle shells were littered on the ground. At least the soil was less boggy than the route of the day before, and we managed to make good progress on the way back. Our plans to have an elaborate lunch under a beautiful blue sky were put on hold though, because it did not clear up at all in Dartmoor that day.

There’s something intensely satisfying about not seeing any other people for a whole day, and also from not having to follow any paths, because there weren’t any. It’s probably the closest to true, age-old nature that I’ve ever been. It’s a really good experience, and I will do it again.

20150524111346

Posted in Travel, UK | Tagged | Leave a comment

Yup, still searching

We haven’t found a house yet, but we are learning many things as we view more properties. Here’s some thoughts I had on the topic while searching for a new home.

  • We’re getting better and better at narrowing down properties that are worth viewing.
  • As a result, filtering properties is absolutely not an issue after you’ve started looking. There’s only 1 or 2 new properties every day that match our criteria.
  • There’s no point for me to improve on Gumbug. I could make it better, but manually looking at 2 properties per day, even if they’re mismatched, is not a time sink.

We’ll get there. It’s just a matter of time.

Posted in Daily Life, UK | Leave a comment

Gumbug: A better way to browse real estate

Last summer I really wanted to find a decent rental apartment around London. Every day I scoured Gumtree, Rightmove and the likes in search of something affordable. In the end I decided to wait until I was able to buy an apartment instead, but I spent several weeks searching and getting annoyed at real estate sites nonetheless. I decided I could save myself a lot of time and effort by automating some of the steps of my search process. My search process went roughly like this:

  •  Go to Gumtree, search by location and price
  • Mentally filter out all the ads that I’d already rejected, usually because they were old or just looked crappy
  • Check the new ads, decide which ones I might be interested in based on my more subjective criteria (not ground floor, too far from public transport, high-crime area etc.)
  • Repeat the above process for a different set of locations
  • Repeat the above process for all locations on a different website (Rightmove, Zoopla etc.)

Thus Gumbug was born. Initally it was meant to search both Gumtree and Rightmove for rental apartments, but I’ve adapted it to only do Rightmove’s To Buy section, for now. I’ve found a lot of duplication between sites that are listing property to sell, whereas for rental apartments there was often a whole category of quirky private listings that would only appear on Gumtree. The need to scrape multiple sites seems a lot less when only considering things to buy.

You can find Gumbug on github: https://github.com/rheide/gumbug. I’m also running a semi-public version of it on Heroku, although it won’t be very fast if a lot of people end up using it. You can have a play with it here: http://floating-forest-4090.herokuapp.com/, or to see some example search results, have a look at this link: http://floating-forest-4090.herokuapp.com/s/gzr1vwthsd. Since it might not handle the load, I’ll describe how it works.

For each search you can add multiple sources, which are all consolidated into one page. I tried to avoid pagination of things as much as possible because I just want to see everything on one big page that I can scroll through at my leisure. If a listing appears on more than one source url it’ll only appear once in the results. If the listing is already in the system its details won’t be re-fetched every search, to save time. Adding urls as input might be a bit ‘techy’ but it saves a lot of coding time and allows me to specify a whole bunch of hard filters right at the source, since the url can already contain filters for price range, number of bedrooms etc.

Keywords

Keywords

You can add a list of keywords to ignore and a list of keywords that are required. Eg. you can ignore ‘ground floor, retirement’ and you can require ‘leasehold’. For the ignored keywords, if a listing contains at least one of the keywords, it’ll be marked as ignored and moved to the bottom. For the required keywords, if an add doesn’t contain at least one of the required keywords, it will also be marked as ignored and moved to the bottom.

Filter by distance to public transport

Filter by distance to public transport

The public transport filter lets you select the stations you wish to be near to (or far away from). The list of stations is prepopulated from the zoned stations around London, but it’ll automatically update after every search. If you add at least one station filter, all the listings will have to match at least one of your station filters, or else they will be ignored. Eg. if you add two filters: between 0.0-0.5 miles from Chesham station and between 0.2-1.0 miles from Amersham station, a listing must be either close to Chesham or close to Amersham (but not necessarily both) to match.

The distance filter is pretty stupid because distances are simply scraped from Rightmove, which (as far as I can tell) only shows straight-line distance. You might have to make a massive detour to get to the station, but Rightmove will still happily report that the listing is right next to the station.

Once the search is complete you get to see all the results on one page: all their images, important information and a map. No useless clicking through tiny thumbnails here. The key feature in the search results page is this: you can manually mark listings as either favorited or ignored, and any future searches you do from that particular search result page will preserve your favorites and ignored listings. So let’s say you haven’t searched anything for a week or so, all you have to do is press the search button to perform the exact same search again to get the new listings. Gumbug will pre-filter the new listings according to your criteria and will automatically move the ones you’ve already ignored manually down to the bottom.

So, why am I showing the ignored listings at all, if I’m clearly not interested in them? The reason for this is that humans (especially real-estate agents) make mistakes. They will mislabel things, forget to mention a keyword that every other ad that you’re interested in has, or they’ll add something stupid like “not ground floor” which throws off the keyword filters.

A second reason to display ignored listings is because you might be sharing the link to the search results with more than one person, and the other person might want to un-ignore a listing. Gumbug isn’t exactly built on security: any person that you share the search results url with can favorite and ignore listings. This is great for me because I want to share search results with my girlfriend so she can go through them as well, but when sharing in public it’s better to spawn a new search with a new url.

Lastly, there’s the map. One of the things I’ve consistently found myself doing when checking listings, is to cross-reference the area with the deprivation map, which gives a rough indication of how much crime/poverty/incidents/bad things there are in an area. You can also click the name of each public transport station to display walking directions, so you know if that 0.6 miles is actually 0.6 miles (hint: it usually isn’t).

Deprivation and Directions

Deprivation and Directions

Gumbug will continue to be a work-in-progress, but it’s reached a point where I’m quite able to use it to make my own life easier. Maybe it can help someone else too. Here’s some of its issues:

  • When you flag something as ignored and then go to the next page, the ignored listing will pop up again because it’s been moved to the back of the sort order.
  • No street view support yet
  • Some map issues when viewing on mobile
  • No floor plans yet

Feel free to give it a try on Heroku. If for some reason your search doesn’t seem to be working then that might be because the worker process is not running. Since Heroku’s not cheap I’m running the worker process on my local machine. Heroku’s database is very tiny so it might fill up very quickly. If there’s enough demand I could consider setting up a more proper version of it, so consider this an attempt to gauge the public interest. Let me know what you think

Posted in Tech, UK | Leave a comment
← Sidebar