Tech

Comments

Posted on 2021-01-23 17:58 by randy

My self-hosted blogging woes continue..

It'd be an understatement to say that I'm not a big fan of Facebook. Facebook has been banning a lot of groups and accounts lately, seemingly algorithmically. That's kind of evil. I was seriously considering buying an Oculus headset until they started requiring that you sign in to it with your Facebook account; an utterly unnecessary step from a user point of view, only done so they can make more money and steal more of your information. Facebook sucks.

That got me thinking about how to completely distance myself from Facebook. I've still got a profile there that I regularly sign in to. Facebook is the only social network that a lot of my family and friends are on. There's just no alternative for chatting with groups of people that I know, because those people are only on Facebook, and so far this has stopped me from completely deleting my Facebook account. Today I thought I would finally remedy that, but before doing that I wanted to provide people with an alternative way to keep track of what I'm up to. My blog is where I would want to shift my activity to, but it currently does not allow people to comment, which is a big part of what makes a blog a blog, so that's really a must-have before I start pointing people to this site again.

There's two ways to add comments to a static blog: either you self-host the serverside part that stores the comments, or you use an online service like Disqus to do it for you. But don't use Disqus. The interface is utterly hideous and the free plan won't let you disable ads. Disqus also makes you jump through several unnecessary hoops to post anonymous comments, which was just clean and simple with native Wordpress. I would not recommend Disqus to anyone. Unfortunately none of the serverless alternatives offered a free plan for personal websites, so I'm out of luck there as well.

Then there's the self-hosted option. There's a few serverside packages that will deal with storing the comments for you if you don't mind spinning up a server. Some of them will even deal with span and email notifications. But for me this would mean 1. yet another piece of code that I have to maintain, and 2. cloud server resources that I have to pay for, especially if someone decides to do a spam attack on me. I have better things to do with my life than deal with that.

Sadly, this is the state of the internet right now. I want to get away from services I don't like, but the only way to do that is to invest time or money. It has never been harder than now to have a self-managed personal blog on the internet. Tim Berners-Lee's Solid project might be the best hope for a future internet, but I'm not holding my breath about it catching on any time soon. For now this blog will stay comment-less, and I will not delete my Facebook account yet..

Posted in Tech | Tagged blog , rant , wordpress

Ears, motherboards and DACs

Posted on 2019-08-31 22:41 by randy

I love listening to music at home. My preferred place to do that is usually at my desk, where I play it through my PC. I've been using a Beyer Dynamic DT770 for about 10 years now. I'm on my second pair. I've gone through a lot of peripherals in-between; I used to hook up the headphones to whatever pair of crappy PC speakers I had at the time, but a few years ago I bought a small headphone amp and after that I had been using a small mixpanel. Neither of those were strictly necessary, but they provided easily accessible volume controls and were able to drive my headphones at higher volumes than the motherboard could (I have the 250 ohm version). This all worked great, up until the end of last year, when I convinced myself I was going deaf.

From January onwards, for some reason, music started sounding.. 'duller', to my ears. I really don't know how to describe it accurately. I didn't even realize that something was 'wrong' until weeks or months later. It just seemed like I didn't get my usual enjoyment from listening to music, and I couldn't figure out why. For a while I just thought it was my ears, accepted it, and thought I'd lost one of the things in life that I really enjoyed. For a while. And then I started experimenting.

I thought that maybe it was my headphones. So I tried some other (pretty decent) headphones we had lying around the house and hooked those up to my PC for a while, but all I could tell was that the audio quality was worse, not better. There was nothing wrong with the headphones.

My next suspicion was the speakers. I had been using the cheapest Logitech speakers in the past, but because they did not allow me to control the headphone output's volume with the volume control knob, I always had the headphones plugged in separately via a mixpanel. Last December I bought a new PC with new speakers: they were the second-cheapest Logitech speakers, which did allow headphone volume control using the volume know. So I ditched the mixpanel and connected my headphones up to the new speakers directly. But could that be the difference?

In order to find out I enlisted the help of my wife to do a blind test (or should it be a deaf test in this case?). I hooked up both the speakers and the mixpanel to the PC, normalized the audio, and got my wife to swap the outputs while music was playing, while I tried to guess which one was which. I had tried doing this just by myself without the blind part of the test, but I wasn't confident that it made a difference. But the blind test proved fairly conclusively that there was a difference, and that we both preferred the sound through the mixpanel, albeit only very slightly.

I ran with this for a while, but ultimately still felt like I was missing something. I really ran out of ideas for what could be causing my lack of enjoyment of music, other than my own ears.. but there was one other variable that I had changed and not tested yet: the PC. With the PC came a new motherboard: the MSI B450M Mortar, with an on-board sound card, as all motherboards have these days. I've been using on-board audio for years and never had issues with the audio experience, so it seemed very unlikely to me that the motherboard could be causing this. But it was the last thing I hadn't tested yet.

One other thing that made me wonder if it was the motherboard is that I sometimes play racing games with a separate app playing music in the background. My old motherboard used to mix these just fine, but after switching PCs the new one always seemed to cut out the volume of one app or the other at certain input levels, which sounded extremely frustrating. For a long time I suspected Windows 10 of causing this (antoher variable change: my old PC ran Windows 7), with it's "mute background apps" functionality, but I had made very sure that that was turned off.

In order to test if the motherboard audio made a difference, I bought an external DAC: the Sabaj Da2 USB DAC. It's an external USB digital-to-analog converter, which turned out to be easily powerful enough to drive my 250 ohm headphones without an amplifier in-between.

Before buying the Sabaj Da2 I had been planning to set up another blind listening test, but it took me less than an hour of using it to realize that I wouldn't have to. The sound is subtly but unmistakeably better than the motherboard audio I had been putting up with for so long. It's hard for me to articulate exactly what makes it better though. It's as if I am much better able to focus on the smaller, subtler aspects of the music, rather than just being bombarded with just the main instruments. The music seems fuller, I guess. It's a small difference, but it's impossible to not notice once you've spotted it. and then you'll never want to go back. I now wish I had tried this a lot sooner..

The Sabaj also solved the issue I was having with music not being played at equal volumes when I have more than one app playing sound at the same time. I don't know if that's just crappy drivers or crappy hardware on the motherboard, but I don't care. I'll stick to dedicated hardware for audio from now on. Side note: the Sabaj gets really hot, I'm tempted to slap a small heatsink on it.

(Disclaimer: I did not get paid to write this article. I haven't tried any other USB DACs so I don't know how the Sabaj compares with similar DACs. All I know is that it's detectably better than motherboard audio.)

Posted in Tech | Tagged dac , headphones , msi , sabaj , sabaj da2

Raspberry Pi powered Legobot: latency

Posted on 2019-01-19 17:29 by rv

I've blogged before about my neverending attempts to build the perfect robot to drive around my apartment remotely. I've gone through a lot of mechanical designs, and had a lot of failures. Eventually I settled on something that was very much simplified compared to my initial ambitions, but it works alright now. It's been serving as my remote camera every time we go on holiday, so it's doing its job. Here's a writeup on the software latencies involved.

At the core of this contraption is a Raspberry Pi that controls a whole bunch of Lego motors, and one set of lights. I wrote my own Python+Django web app to control access to the device from the internet. It uses Django Channels to send the user input to the server-side, where the web app controls the Raspberry Pi's GPIO ports directly. This means it's not very thread-safe, but then it was never really meant to be controlled by more than one user at a time anyway.

For the video stream, I tried a lot of different solutions, but all of them had terrible latency. The problem is that the Raspberry Pi just does not have enough CPU power to encode the video into a stream and send it over a websocket. However, after a long search, I did eventually find a solution someone else developed that's pretty nifty: it uses the native Raspberry Pi video tool to stream the raw (H264-encoded) camera stream directly to a websocket, hosted by a Node.js webserver. This means that the video stream has to be decoded client-side, in the javascript executed by the client's browser, using a nifty js library. (I'll eventually get around to placing links to these libraries, once I remember where I got them from). Turns out this works pretty well!

Here's a really terrible video that shows the latency of the controller software, as served from a web page on my mobile phone, over the phone's 4G connection, so not on the wifi.

This video was taken on my old Canon S110 camera, which has a super slow-mo mode which records at 240 frames per second. So, counting the frames between events, I get these results (averaged over a couple of different videos):

Time between pushing the 'lights on' button in the mobile phone browser and the UI registering the event (client-side-only lag): ~31 frames = ~129ms.
Time between the button being pushed, and the lights turning on in the video: ~12 frames = ~5ms. I'm guessing this is very short because the actual action is being sent to the Raspberry Pi well before the UI update begins, so it'll overlap with the first action. Time between finger push and real-life consequence: about 135ms.
Time until the mobile phone video stream shows the lights being turned on (since the lights actually turned on): ~55 frames = ~229ms.

All in all it's close to half a second between requesting the action, and actually seeing confirmation of the action on-screen. And that's on a fairly reliably mobile connection close to my home. It's a different story altogether on crappy hotel wifi halfway across the world, although I've also been to some amazing hotels in Japan where the speed was very similar to the speeds in this test.

Still to do: open-source the Raspberry Pi software I wrote for this. More to come!

Posted in Tech | Tagged lego , raspberry pi

Moving the goalposts

Posted on 2019-01-06 14:43 by rv

This blog is finally on https! It's been a while in the making. I first tried this over a year ago but couldn't get AWS from generating an infinite redirect loop. In the end I think it was a combination of the Cloudfront distribution doing redirects and the S3 bucket that this blog is now served from also doing redirects. I think I may have actually solved it at the time, because I did the same thing again this week and it worked, but it turns out the changes take quite a while (over a day from what I've seen) to propagate, so I never knew that my fix worked the last time I tried it.

So, 2019 now. Middle age has definitely struck. I've found myself very busy last year. Busy enough to not have enough mental energy for side projects or hobbies. My major peave of the year is definitely my commute. My commute to work is, by my own standards, way too long. It's one of the things I'd like to improve on this year, if at all possible. Currently I've got perhaps 3-4 hours of free time after work, which, after dealing with all the adulting (house chores, mortgages, that sort of thing) leaves me with no time during the week where my brain is actually functioning. While I'm unhappy about that, it's also a path I've chosen for myself, and the benefits (which I won't go into here) outweigh the costs. And there's things I can do to optimize that.

One thing I definitely should reflect on when thinking about last year is the concept of moving the goalposts. I keep perceiving my life in a way that makes it seem that I haven't accomplished what I wanted to accomplish, but I forget that I had very different goals a few years ago. While I was kind-of-sort-of aware that I had been doing this over the past year, lack of free time and preoccupation with other things left me until the December holidays to really realize how much my life has changed in the past few years. Whilst all of the world around me was going to shit in 2018, for me things have improved massively in terms of knowledge, financials, living quality, and all kinds of other things. When I'm busy living the daily life it's easy to overlook that and just focus on optimizing the next thing that's not quite perfect yet, without realizing that I've already dealt with 99.9% of the biggest worries I could be having. I think it's ridiculous to value the remaining .1% of quality of life as high as the 99.9%, but that's naturally what my brain does, and it needs conscious effort for me to stop thinking that way. My goal is to try and be more aware of that in 2019.

Here's to a good 2019. May your goalposts be moved ever further.

Posted in Daily Life , Tech , Thoughts

Hello blog!

Posted on 2018-10-05 16:43 by rv

Hello, blog. It's been a while.

Last year, after the previous blogpost, I felt a good sense of closure for the blog, so I decided to take the plunge and turn the entire site into a static site. No more wordpress, no more new comments, no more new posts. This turned out to be pretty easy to do: all I had to do was scrape all the pages on the domain and upload them to S3. I had been meaning to get an https certificate set up, but ran into some snags that I couldn't be bothered to work out. It's still on my to-do list, but at the time it was just yet another thing that prevented me from continuing the blog.

Continuing? But why? Didn't I say that I was done blogging? Well, yeess.. but..

Everything I said in the Goodbye post holds true, and I stand by what I said. The internet is not the same place it was when I first started blogging. Compared to 10 years ago I still feel quite strongly that it's way less appealing now to to put content online. The second argument, about my own life, also still holds true. I really do have way less things I want to blog about, and way less time to write blogposts. It compounds, of course: because the internet feels less inviting to me now, I feel the need to structure new blogposts better, more coherently. That costs more time. It all compounds.

But despite all that, during the past year I did find myself occasionally having the desire, the inspiration and the time to blog. Even if the amount of things I write is cut down ten-fold, there will still be the occasional blogpost. So, while the desire to blog was occasionally there, I was blocked by having made the site static: with no wordpress interface it's just a major pain to manually write something: I'd have to manually edit HTML files, update the index page, update the category pages, tag, pages, archive pages.. It's simply a non-starter. I had to make this easier.

There's a bunch of static blog generators out there. Some of them look nice. I'm willing to bet that there's a very large change that whichever one I pick won't be maintained any more ten years from now, and if there's one thing I hate it's dealing with someone else's codebase in my free time. Big nope. Since I'm a web developer anyway, I decided to build something myself. That was in January..

I got pretty stuck on doing something simple. If there's ever a second thing I hate, it's dealing with my own codebase in my free time, so I wanted to make things as simple as possible. This often meant that, as soon as I wrote something, I immediately discarded it for being too complicated. Perfect is the enemy of good, and nothing got done for months. I had decided that I wanted a flat file format consisting of a yaml section for each post's metadata, and markdown(+HTML) for the post content. But I really didn't get further than that, because I desperately tried to pretend that it was just a small project and could be contained in one simple file. I finally found myself with a lot of free time this week and decided to tackle the problems head on.

The result-for-now is that I ended up with more code than I wanted, but nothing too crazy. Since I'm the most familiar with Python, I ended up writing a template renderer using jinja2, a generator which gathers all the pages that need to be updated for each article, and a publisher that uploads to Amazon S3 using boto3. No other dependencies, although I did write a little preview site in Django just so I could get instant gratification as I was doing all the template and CSS tweaks.

I ran into major issues with transforming my original blogposts from Wordpress to the new yaml+markdown format. At first I was working on a scraper-like tool that would read my old blog's static HTML files and do some parsing. Then I found out that for some reason my old static blog was missing all posts before 2007. So I had to deal with the Wordpress SQL backup. I wasted an hour trying to import the bloody thing into SQLite, but ran into countless SQL dialect errors. Eventually I gave up and installed MySQL and ended up wasting another hour getting all the settings right and connecting to it from Python. In the end I did manage to extract all the posts from the database and into happy little human-readable flat files.

Then I hit another snag: turns out a lot of posts on this blog use special [block] tags for extra functionality, and that of course wouldn't work as-is. So I had to convert those to HTML. Since I was doing some search-replaces anyway I decided to do some anonymization as well: my goal for this blog is for it to be a one-way street: if you already know me it should be easy for you to discover this blog, but if you don't know me already, searching my name should not bring up this blog as the topmost hit. I realize I am completely failing at this at the moment by having a Github link in the header. That'll probably go away at some point.

Another snag was the post count: since each page shows the number of posts in each category, I'd have to update very single page every time I add a post for a category, just so the counts are updated. What a pain. I decided to remove the counts from the HTML entirely, and I added a separate stats.json file that adds the counts in through javascript. No javascript required for this blog, but it does do enhancements.

It was at this point that I realized that I was getting very carried away with everything and that I had to put a stop to this: I ended up spending the better part of five days rather than the two or three mornings I had estimated. But finally I am in a state where I can relatively easily write a new post and publish it. I need to have the python code and the raw blog contents on the machine that I'm writing from, and it currently only works via a command line one-liner, but it works. I intend to make a very basic front-end for it at some later point in time so I can publish from any machine using a Django app. Another thing I'd like to add back is the comments section. Who knows, perhaps I'll get around to that before 2020..

There's a lot of things I want to do, both in terms of adding features as well as things to write about. I still want to write down in detail what the Lego Raspberry Pi project ended up becoming, and I also want to publish the static blog generator on github after I'm done cleaning up the code. Who knows, I might even be tempted to write about my personal life again at some point. In any case, stay tuned for more exciting content.

More to come!

Posted in Daily Life , Tech , Thoughts

How to find out if your phone is spying on you

Posted on 2017-05-13 17:39 by rv

<tinfoil-hat>

I browse 9gag a lot during my commute. Sitting shoulder-to-shoulder with my fellow commuters I find it uncomfortable to do something on my laptop, and at least in the mornings I'm not awake enough yet to properly enjoy a book or audiobook. So instead, I turn to 9gag for my cheap entertainment. As such, I see quite a lot of ads.

Over the past few months I've managed to convince myself somehow that the ads that I'm seeing are targeting things that I've only ever mentioned verbally, but never digitally. It's just extremely suspicious to see an ad for one particular brand the day after you've spoken about it to someone. This kept happening to me, so I thought of a way to try and 'prove' that my phone is indeed listening in. My wife and I had a running joke for a while where we'd shout odd terms ("ARNE JACOBSEN CHAIR!", "NISSAN MICRA!") at my phone to see if ads for them would show up. (No for the Arne Jacobsen chair, yes for the Nissan Micra, but cars in general are already a natural search term for me and there's a digital trail showing that I like cars, so no solid evidence there).

To take it one step further I devised an experiment. For a few days my wife and I brainstormed about topics that we would never ever look up naturally; topics that are so far removed from our daily (digital) lives that it would be very unlikely for us to see targeted ads for them. We tried to focus on popular topics that people or companies would actually buy ads for. All this was done offline, on a small piece of paper, and to the best of my knowledge neither of us ever uttered one of these topics out loud or typed it in digitally until the experiment was over.

While we were devising the list of topics I took screenshots of all the ads I saw on 9gag, as well as a few on Facebook and in my browser. The idea is to compare these to the ads I got served after uttering each of these topics out loud.

With the list of topics finished, rather than manually conversating through them in the vicinity of my phone, I instead prepared an old laptop of mine with a text-to-speech script which I ran overnight. I downloaded PyTTSx, made sure that it worked, then disconnected the laptop, which had a clean Windows installation with no personal information about myself, from the network, and uninstalled the network drivers. Then I wrote a Python script to run some plausible conversations about the topics we had written down on paper thought the TTS engine, and set it to loop. I had to listen through each topic several times to make small adjustments to the spelling because the TTS didn't quite pronounce all the words right unless you spelled it slightly wrong. I added in a few voices and a few speeds and ran the script overnight with my phone charging right next to the laptop, far far away from the bedroom. Because listening to a computer having a fake conversation about topics you couldn't possibly be less interested in does get boring after a while.

At this point the topics did exist digitally, but only on a laptop that was disconnected from the internet. I continued my daily routines as usual and again took screenshots of all the ads that I saw. I kept this up for 2 days before I got bored of taking screenshots, after which I categorized all the ads I saw into one or two categories and tallied up the results.

The Topic column contains whether the category would be something I'd expect to see naturally anyway, or one of the "highly unlikely taboo topics" that I would otherwise never search for or would never even come up in my daily life. The rows highlighted in yellow are where the taboo topics overlap with the ads I saw, and the rows highlighted in red are the particularly suspicious rows of topics of which I suddenly got served a lot more ads after running the taboo topic script. There were also a whole bunch of taboo topics that I never saw ads for, presumably because no company paid for ads for those topics, or because I'm a crazy tinfoil-hat guy who's just a bit too paranoid.

I'll try and be generous and explain away both the medical category and the pets category. I was already served one ad in the medical category before running the script (which uttered some new and much more.. un-science-y medical keywords), so a slight increase is not entirely unbelievable. The same goes for pets: although the keywords in the taboo script were very heavily focused on dogs (whereas my wife and I are very much of the cat persuasion), we do have a pet and it's not unlikely that this information could have crept up in our digital life during the course of the experiment, outside of the TTS script.

Then there's the third red category that shows a sharp increase, for which I just can't find a reasonable explanation: loans. The ads in this category were for 1. credit ratings, 2. leasing cars, and 3. short term loans. I've been employed pretty much continuously over the past 12 years, have never searched for or had a loan (other than my mortgage, which is a very specific loan category that hasn't popped up either before or after the experiment). The TTS script contained several passages about wanting to buy or build (expensive) DIY things as well as the actual term "pay day loan", and some other words that could perhaps imply increased gullibility to these kind of ads. When I was categorizing the ads I saw a very clear distinction between ads that focused on investing capital (category 'Investing' in the image) and ads that focused on how to loan things or improve one's ability to loan things.

I can think of several explanations of why the ads changed after the experiment:

Coincidence. Ads change over time. Maybe the loan companies just didn't buy any (targeted) ads until in the middle of my experiment. I didn't run the experiment anywhere near long enough to rule this out.
Leaks. Maybe I changed something about my online behaviour that changed the ads I was served.
Something on my phone or in my house is listening in. Could be the phone OS, could be an app on the phone, could be a nearby TV or PS3.

It would be possible to go one step further and intercept the network traffic coming from my phone, but I don't think I would be able to conclusively prove anything one way or another. If the speech recognition happens client-side then I wouldn't see any speech data over the network. Or it could be encrypted. It might be worth a try, but I suspect I won't find anything obvious.

Is this enough evidence for me to say that my phone is definitely listening in on me? Definitely not. But this, combined with earlier incidents of ads matching conversation topics over the course of the past few months, I think is more than enough reason for me to be suspicious. It would be interesting to see other people try the same experiment. If they find the same results then we could start comparing phone OS, installed apps, other nearby electronic devices, etc.

For now, since I hardly ever speak on my phone anyway, I've put a little bit of blu tac over the microphones. I tested this with by calling my wife to confirm that no audio is heard on her end when I speak. If, over the course of the next few weeks, I find myself not noticing a single instance of an ad matching a conversation topic, I would consider that pretty strong proof that something on my phone is listening in. No matter the outcome, it's probably worth a followup experiment. Time to think of new categories.. (Don't send ideas for new categories to me, obviously! Unless it's a handwritten letter..)

</tinfoil-hat>

Posted in Daily Life , Tech | Tagged 9gag , ads , android , facebook , microphone , spying , tinfoil

Raspberry Pi controlled Lego car part 3

Posted on 2017-03-05 15:26 by rv

Things have not gone smoothly since the last post. I can't seem to stop spending money on this..

Last time my goal was to build a two two-speed gearboxes, connected via a gear switch lever, controlled via servo motor, that will allow me to switch both gearboxes simultaneously via the Raspberry Pi. I have absolutely no reason to do this. The car will work just fine with only one gear. It just seemed like an interesting thing to do. It led to me researching some rather complicated things on Wikipedia, but given that I only have one differential and (despite what these posts might make you believe) am trying to keep it simple, I decided to go with the dual gearbox option. I had already bought extra parts for this and felt too stubborn to give up on it.

That was a few weeks ago. Here's what I initially came up with.

The first image shows the interior: the M motor controls two mini actuators. You can see the top one, there's a bottom one as well controlling the axis in the other direction. The second image shows both gearboxes attached. I had to give up my plan to use the servo motor almost right away, because getting the gearing right for the servo motor to shift the axis exactly 1 block left or right proved nearly impossible, and very easy to go wrong over time. In this first design I had planned to use the regular M motor with a clutch gear to make it stop when it reached the end of its travel, but due to the way I connected the gears it didn't quite work. I also realized during building this that I couldn't have two identical gearboxes; I had to mirror the design in order for the gear-shift actuators to select the same gear every time.

This design was way more sturdy than the one from the previous post, but still not quite study enough. Other than that there are two major problems with this design. The first is that it is just massive. Both driving axes are mounted very far away from each other, meaning I needed lots of tracks to connect the two. My prototype had completely flat tracks in-between the two axes, which resulted in a massive contact surface with our thick carpet. The prototype was geared the same way as previous ones, but simply did not have enough power to move.

The second problem is the way it shifts the gear-shift axes. Each actuator is connected to gear axle with only one point, so in order to shift gear it either performs a push or a pull motion. There's nothing stopping the actuator mount point from shifting other than two little stoppers on the axle. This turned out to be rather weak, and they ended up getting pushed or pulled away from the bit that connects the actuator to the axle, making its range wider and wider until it just stopped working completely after a few shifts. Fail.

Last time I mentioned that the USB-to-9v cable that I bought was faulty, and that I was actually able to power both the Pi and the motors from a single USB battery pack. This turned out to be mostly true, yet ended up not helping me in any way. I bought a new USB-to-9v cable, and this time it did work correctly on my battery pack of choice I was using, a Tecknet 10000mah pack, which is very thin and easy to hide inside the Lego. Sadly I found out after getting the new USB cable that this battery pack doesn't provide the amperage needed to power two XL motors at the same time. It just barely manages, but as soon as you put some load on it then it shuts down and doesn't restart.

That led me into yet another unexpected quest: that of finding the perfect USB battery pack: one that has two USB connectors for input, plenty of amps on each output, and ideally allows for passthrough charging without blipping the power or requiring you to press a button after plugging/unplugging. I have spent a long time searching for a suitable battery pack on the internet and have found absolutely nothing that satifies all of these requirements. There's plenty of powerful battery packs, even powerful battery packs that allow passthrough charging, but none of them do it without blipping the power or requiring a button push to resume. There are some boards for the Raspberry Pi that allow multiple power inputs, which I think would work for autonomous recharging. That's yet more money to spend on this though, and makes things even more complicated, so I'm holding off on that for now.

Eventually I tore apart the previous prototype and started on a new gearbox assembly. I considered building the gearbox bit vertically, but eventually decided to go horizontal again for the sake of simplicity. I increased the length of the gearbox axles by one, which allows me to connect to the gear-shifting axle on both sides rather than on just one side, so there's no issues any more with the axle shifting from too much pushing or pulling.

Instead of using two actuators this version can do the shifting with just one actuator, which controls the two shifting arms connected to the gearboxes. I'm using a clutch gear directly on the M-motor used for shifting (not pictured in the picture above but it connects directly to the small gear on the top right). Some bits of the shifting arm have some tolerance, so that combined with the clutch gear seems to result in quite satisfying gear shifts.

Above is a picture of the latest gearbox core in yet another prototype shell. It ended up still being quite wide because I wanted to keep it modular, so I can take apart both sidewall tracks quite easily to access the gearbox core. In the end I just piled up all the non-Lego components on top in order to test it out. There's still plenty of space inside so my goal of having a flippable robot that willwork on both sides is quite attainable.

I found out that in the current setup the highest gear doesn't really work for turning, most likely because there's still a fair bit of contact area. I think I can gear it down by 1.5x/1.67x and it'll be perfect. I also need to buy a new USB battery pack, so the final design will depend a lot on the dimensions of that. I haven't decided yet which one. More to come much later because I'm off cycling from next week!

Still to do:

Fix the gearing.
Make it lower.
Get better USB battery that allows for more amps.
Buy UPS circuit so I can build a charging station.
Hook up the Raspberry Pi camera, camera motor and lights.
Decrease track contact surface with ground if possible.
Build internal compartments for the electronics and the battery.

Posted in Tech | Tagged lego , power bank , raspberry pi , usb battery

Raspberry Pi controlled Lego car part 2

Posted on 2017-01-31 22:06 by rv

Not many pictures this time, because unlike last time I'll rant a bit this time about getting the software on the Pi working. To start off: I did something rather stupid: I updated Raspbian. It took ages to update the OS and all the packages. The Pi isn't very fast for that. I should've just downloaded a new image. Here's how I lost a lot of time.

After updating the OS my wifi stopped working. I'm using a small TP-link USB wifi, since my Raspberry Pi B+ doesn't come with on-board wifi. I did a lot of searching on why this would suddenly stop working, and found that, only in some cases, the B+ is incapable of providing enough power to the USB ports to make my particular model of wifi adapter work. The wifi did in fact work for just a split second after booting, and I was able to ping the Pi from another PC, but it almost immediately disappeared again. This I thought was a good indication that it was indeed a power issue.

I bought a USB cable from Amazon that combines two USB plugs, one for data, one for power, into one output, for the wifi plug. I was fairly confident that this would solve my problem. First I plugged both inputs into the Pi's USB ports, but no luck. Then I tried plugging the USB power input directly into my USB battery pack, but that didn't work either. It did have a weird side effect, in that if I disconnect the Pi's regular micro-USB power connector, it miraculously stays powered through the USB wifi plug. Weird, but not useful.

So I did some more digging and eventually found out that for every Pi kernel you need to manually (re)compile your wifi plug's driver. Ugh. I got started on doing that but it took ages. I eventually managed to find a site that offered precompiled drivers and simple installation instructions. Not exactly open-source but I'm definitely not going to wait hours for the bloody thing to compile.

With the wifi fixed and the motors super-easy to control via the Python GPIO library, I got to work on getting the Pi's camera feed to work. The Pi supposedly does full-HD video capture at decent framerates so I had high hopes of getting a nice stream going. Unrealistically high hopes, it turned out. Recording decent quality video is one thing, but streaming it in real-time with little enough delay to control a moving vehicle, that's quite another. I had to turn down the resolution all the way back to 640x480 to get <500ms delay, but after that it's quite usable. The uv4l raspicam package comes with a built-in server to stream video, which is actually pretty nice when playing around with it on the local network, but since my final goal is to open it up to the internet I'll have to find something a lot more secure than that. I'll likely set up a separate Python server to forward the video feed, or stream it from the device directly. I hope it won't introduce too much extra delay..

On the hardware front I'm playing around with some of Sariel's gearboxes. I bought a set of various Lego gears on ebay since my aging Lego Technic did not have many useful parts. I built this three-speed gearbox and then figured I might as well buy the Lego Servo motor as well to play around with.

Turns out that controlling a servo motor from the Pi is a lot more difficult than controlling a regular motor. To get the servo motor to go to a pre-set position you need to use pulse width modulation with a very specific duty cycle for each position. If you get the duty cycle slightly wrong the servo will jitter around the position and move back and forth a bit. The same happens when using the Python GPIO default module, which comes PWM functionality built-in, but just can't be real-time enough to generate a stable signal. There's loads of jitter if you try it that way. I ended up going with RPIO, which uses Direct Memory Access to get the timings right. I played with it for hours but in the end still could only get about half out of the supposed 14 positions to select correctly. I think that's more of a problem on my side though, and I'm probably using the wrong duty cycles or frequencies. That said, I'm totally indulging at this point because the gearbox only needed the off position and full negative and positive duty cycles, so I didn't even need to use pulse width modulation at all. It was an interesting side track though.

Once I got the three-speed gearbox working with the servo I slapped an XL motor on to it and immediately realized that this gearbox design will never work for my tracked vehicle. As soon as I applied only a little pressure on the output wheel the servo axle would fail and start rotating. It really doesn't stand up very well to any decent amount of torque. Time for another design. I ended with a super-simple custom design for a 2-speed gearbox that I got off the internet, but all the simple designs, the designs that I actually have the parts for to build, don't seem to be meant to be controlled with the servo motor. I played around with this for quite some time last Sunday and eventually managed to rig up some kind of controlling arm to push and pull the middle axle of the gearbox so it can shift gears. I haven't tested this with load, though, so I'm not sure if it'll hold up very well in the final vehicle. But I'm a bit more confident about it than I was about the three-speed servo gearbox.

The next challenge will be to duplicate the gearbox and make a design that allows the servo to change gears in both gearboxes simultaneously. Since I'm making a vehicle with independently-controlled caterpillar tracks (but only have one servo motor) I'll need to have one gearbox for each track.

Lastly, I got one more thing wrong in the last post, about getting power to the motors. I had bought a USB-to-9V cable, which I thought didn't work in my USB battery pack because the battery pack was smart enough to disable the port if it detected that no power was being drawn. That assumption turned out to be completely wrong. Instead, what happened was that the cable had a very shitty USB connector and after I let go of the connector it would drop a bit and disconnect. I duct-taped it into place and it works perfectly, so I can get away with using just one USB battery pack (with two USB ports) for my vehicle. Yay!

Posted in Tech | Tagged gearbox , lego , raspberry pi , servo

Raspberry Pi controlled Lego car part 1

Posted on 2017-01-15 18:41 by rv

So, I have a fair bit of Lego lying around from my childhood days. Not a lot of technical parts, but enough to have some fun with. I also have a Raspberry Pi lying around after a failed experiment where I tried to use it as a security camera. Why not combine the two? I figured I could build a Lego car that I can control remotely via a little server on the Raspberry Pi, which would also stream its camera feed. It seemed easy enough, especially after discovering a page that explained exactly how to make the hardware part of it work. Never did I imagine it would take this much work to get it up and running..

This post will detail the hardware iterations and setbacks I faced. The next post will go into detail about the software side of things.

So, before starting on this project, here is the list of things I thought I'd need to make this work:

Bunch of Lego: wheels, gears, other technic parts and some generic bricks to connect it all up. (already owned)
Raspberry Pi plus camera module. (already owned)
A USB battery pack (already owned)
Two Lego power functions motors.
A 9 volt battery.
A motor controller.
Some Dupont wires to connect it all up.

Version 1

I bought the M motor starter kit and a separate M motor for my first iteration. I built a very nifty motor assembly and a very strong shell structure to house the components. Here's how it looked like:

It is thus that I found out that I should never overengineer before testing. While building the thing I discovered that I didn't have the parts required to drive all six wheels, so I chose to just drive the middle wheels instead. I figured that by gearing down a bit it should be fine. Much nope. The M motors were way too weak to power the construction, which was already quite heavy from the USB battery pack, but weighed down even more because of its overengineered sturdiness. Having just the center wheels powered was also a terrible decision. Fail.

Version 2

I built this is a test version to experiment with gear ratios and to see if in a lighter version the M motors would have enough power to drive the car. Answer: not really. I got it to go forward and backward pretty decently, but I still didn't have the parts required to power all the wheels, so the motors were not strong enough to turn the whole thing without a steering assembly. Fail again.

So I had two options at this point. Option one was to build a rack-and-pinion steering assembly, for which I had the parts available and did end up experiment with, but in the end I wasn't happy with its turning radius and how I'd either have to buy a servo motor or have an 'unclean' solution with a clutch gear to handle the stopping at the end of the wheel turning axis.

Instead, I went with option two: throw more money at it. I bought two XL motors and a crapload of caterpillar tracks. That'll give me the power and the grip I need to build something really nice.

Version 3

..or not. It does look impressive though. Here's a test build with the power functions battery box instead of the external 9 volt battery.

At this point I was still powering the motors via a separate 9 volt battery. I figured that, since my USB battery had two USB connectors, I could use it to power the motors as well, so I ordered a USB-to-9-volt cable from Amazon. That should help me save some weight by not having to put in two batteries. Sadly, this did not go well.

USB batteries are pretty smart. So smart in fact, that they only output power if they think something's plugged in. The USB-to-9-volt cable worked fine on a powered usb hub, but every time I'd try to hook it up between the battery and the motor controller it would simply not turn on. Crap.

I found another USB battery, which is actually a battery used for starting cars that happens to have a USB port on it. This one did always keep the power on if I plugged in the USB-to-9V cable. Good! Except it only had one USB port, so I couldn't also power the Raspberry Pi from it. So I ended up with two USB batteries and over double the original weight..

The big green thing is the car-charging battery. Even without the second battery the 3:1 gearing wasn't quite powerful enough to rotate this monstrosity when on certain surfaces, such as medium-thick carpet. Once again, fail. And once again, I spent way too much time overengineering the construction without testing it first with its full weight.

Version 3.5

I figured I'd try to keep the next version as small and light as possible and worry about motor powering and recharging issues later. I managed to build a pretty small prototype.

Now, I had one option left to avoid the use of two batteries. The big green car charger battery did, in addition to its one USB port, come with a 12V output. It's not the 9 volts that the Lego motors need, but I did find at least one link claiming that the motors should run fine on 12 volts as well. I had previously done this as a child when connecting an old technic 9V motor to my 12V Lego train controller, which gave it quite a nice speed boost as well. I figured this would be an acceptable trick. I butchered an old power supply, cut off it's circular connector and used it to connect up the green battery's 12V port up to the motor connector.

To avoid any more gearing issues I added another axis, bringing the final ratio down to 9:1. The build ended up much larger than I had intended, but still smaller than v3.0. With better gear ratios and a higher voltage to the motors.

You may notice the second battery sticking out at the end of this bulky thing, despite the fact that the green battery has a free USB port thanks to the makeshift 12V connector. This is because the stupid battery doesn't let you use both outputs at once. It just turns off one or the other. Not ideal!

I'm still annoyed at this but don't have an immediate solution. The reason I spent so much effort trying to get everything onto one battery was because I intended to build a little wall charging station, so I can drive the robot into the charging station, which will connect the charging port of the USB battery onto a USB wall socket. I gave this a try with the double-ported battery while the Raspberry Pi was connected, and sadly it resets the Pi as soon as the charging connector is plugged in. Crap times two. I haven't tried this trick with the green battery, but that's pointless anyway since it only lets you use one device at a time. Looks like I won't be able to charge the thing remotely while it's running.

V3.5 still has a whole bunch of issues. It works great in terms of motor and track performance though. It moves and turns fantastically. even loaded with two bulky batteries. The motor placement isn't great though; they're way too close to the ground. I need to fix that in the next version. It's also longer and wider than it needs to be. Once I get a solution to the two-batteries issue I'll decide on how large the next version will be. I'm also thinking about buying the servo motor and using it to power a small gearbox, but I'd also need to buy more technic gears and axles for that, and I've kind of already spent way more on this project than I thought I would spend..

For now though, I'll stick with this hardware design and focus on getting the software up and running. More on that next time.

Posted in Tech | Tagged lego , raspberry pi

New hobby project

Posted on 2017-01-13 01:00 by rv

Turns out it's really easy to control Lego power functions motors via a Raspberry Pi. I found myself a new hobby project :)

Posted in Tech | Tagged lego , raspberry pi

← Newer posts

Tech

Older posts →