Building Ubuntu for the Raspberry Pi

As a result of the prior musings about crowdfunding and the rather shaky VAT status of the whole sector I have been thinking quite a bit about crowdfunding and where it might be useful and how we could get involved in some way. For our normal consultancy business we have no need of capital investments and we don’t produce anything that lends itself to the crowdfunding model, however I did come up with a project I have been wanting to do for quite a long time. Allow me to introduce it by way of a little video . . .


Back when the Raspberry Pi was in development it was shown running Ubuntu 9.04, Jaunty Jackalope. This was the last Ubuntu release that supported the ARMv6 instruction set, from that point on Ubuntu was optimised for newer ARM chips and would not run on the Broadcom chip that the Pi used. I am the point of contact of the Ubuntu UK Local Community team and I was dead excited about this little computer with it’s exposed PCB and low price point. I asked some of the Ubuntu ARM folk if they could support it going forward, but that wasn’t going to be possible, they didn’t have the resources to build for two ARM platforms and the bottom line was that the Pi probably wasn’t going to provide a good user experience for the increasingly heavy Ubuntu user interface. This was sad, but it was the situation. I was a bit concerned that the Raspberry Pi foundation was proceeding on the basis that Jaunty was available – it was already old, going out of support and was a dead end, there were going to be no future updates for it. I was concerned that the UK Local community was going to be landed with a lot of new users who were having a poor user experience and there would be nothing we could do about it. Reluctantly I approached the Raspberry Pi foundation (I met the lovely Liz and Eben at an event in Oxford) and shared my concerns with them, and suggested Debian was the way forward, so the Pi would have a system based on a platform Ubuntu users would be familiar with, that would get updates.

So this was sad, I wasn’t happy about it, the foundation wasn’t happy about it, many users were not happy about it, but it was much better to have a new Debian with updates and prospects than an old dead end Ubuntu release.

Moving on to the present, the Raspberry Pi is a huge success, Rasbian is a great operating platform for it, the LXDE desktop is fine, the Wayland demo was brilliant and loads of cool projects are happening based on the Pi. We still want Ubuntu on it though. We are using it in embedded projects, it is also turning up in things like the OpenERP Point of Sale kit, situations where it doesn’t need a responsive user interface (or a user interface at all). It would be great to know that all the libraries we are using on it are the same versions we are using on other computers that are running Ubuntu. It might be nice to see what the Ubuntu Unity desktop looks like on the Pi, especially Unity 8 running in Mir, but that explicitly isn’t a goal. This project aims to build everything that will build from source without too much hassle. If that gets us a desktop then great, if it gets us a command line with python, that is great too.

Now for the armchair accountants in the audience, having seen the admin end of a campaign I can explain it a little better than before. This is a flexible funding setup rather than the all-or-nothing option and we are accepting paypal and credit card pledges. The paypal pledges happen instantly, the money goes from the end user direct to our paypal account and then there is an immediate debit of 9% of the amount which goes from us to indiegogo – so the money is not held in escrow at all, and it isn’t a big payment at the end. This is fairly clearly a purchase of a pledge to the full pledge value and a subsequent payment to indiegogo which is either a purchase of campaign hosting services, or some kind of financial services fee, not sure about that bit yet. Credit card payments are slightly different, we don’t have the money for those yet, after the campaign ends Indiegogo will do a bank transfer to us for the funds (less the 4% or 9% commission presumably). Paypal is regulated as a bank now, so I think the money should turn up in our financials when it is in the paypal account, not just when we make a transfer of it to a bricks and mortar bank. We will enter all the pledges as sales and pay VAT on them and we will reclaim the VAT on the materials purchased to build the cluster. If anyone wants a VAT invoice for a paypal pledge I can sort that out. Credit card pledges are a bit more interesting as it is questionable whether they have happened yet.

If you want to contribute to the cluster and help us build Ubuntu for the Raspberry Pi then do head on over to Indiegogo and join the 40 or so other contributors we have so far.

From the technical side of things, designing the cluster feel free to pitch in your comments and suggestions below. We have had a lot of people suggesting that we don’t use the Raspberry Pi and use some other platform instead. These suggestions include: cross compile it from Intel machines, use QEMU on fast Intel computers, use cloud computing, use a Power Mac (whut!), use the OpenSUSE Build Service, Use a Calxeda box, use Pandaboards, use Wandboard quad core arm boards. Feel free to add to the list of other platforms we should be using instead, I think I will add the yet to be delivered Parallela board to the list of things we should be using. All these suggestions are great, they would work and they might even be faster or easier. They just are not things I really want (apart from the Parallela which I don’t have) and I don’t think it works as a crowdfunding concept to raise funds to build it out of anything but the Raspberry Pi.

To provide power to lots of Pis there are a few approaches, Southampton University did this:

and other cluster projects have build custom 5v electronics for feeding the USB or direct to the GPIO pins. The custom supply option doesn’t work out particularly cheap and to run the whole cluster you are looking at parts of the circuit supporting a current heading towards 32 amps, which gets kinda complicated. At the moment I am leaning towards using a special powered hub, the Pihub which can cope with powering 4 Pi devices from a single slightly beefy supply. This keeps the plug count down (they will all need PAT testing at some point so I don’t want to go completely wild on plugs) and keeps everything neat and safe and fanless.

Networking is another area where there are options. WiFi sounds mad for a cluster, but is it really? The Pi Ethernet port kind of hangs off USB internally, so wouldn’t a 150Mbit USB wifi dongle be comparable to a 100Mbit ethernet? Lets solve this using science. Initial testing with iperf shows 74Mbit throughput on the ethernet between two Pi devices, over WiFi just 20Mbit. This is rather less than I would expect, maybe there is more performance that can be teased out of the wifi, or maybe the initial feeling is right and ethernet is the way forward. Maybe you have an opinion or advice in this area?

The funding campaign runs through to Christmas but as we have some of the money available already I am thinking we will probably start getting some bits fairly soon and start setting up the cluster controllers and do some power measurements and more detailed performance testing.

Tags: , ,

26 Comments

  • Rob says:

    Remember that wifi is shared medium. The bandwidth you get will be shared by across the devices. If you use a decent wired Ethernet switch, then you get to add up the bandwidth, instead of dividing it.

    • Alan Bell says:

      true, if they are talking to each other independently. If they are all hammering one device (like one repository server they are downloading packages from) then the bottleneck moves to that one thing (which could be a faster PC with a gigabit connection if that helps). At the moment I am getting woeful performance from wifi compared to ethernet, but I think if I can get anywhere near what 802.11N promises then wifi might have the potential to be a consideration.

  • Dave says:

    Wired – fast and simple.
    PSU, I still say a PC power supply, the mods to get it to wake up are trivial, even good ones are cheap.

  • David Talmage says:

    You definitely shouldn’t use WIFI for your cluster. As Rob pointed out, it’s a shared medium, so unless only one device talks at a time, you’ll never get anywhere near the theoretical performance of 802.11{b,g,n,…}. Plus, with all those Pis and their WIFI dongles, there’s bound to be interference and that will cut your performance down even more. You’re much, much better off with wired Ethernet and a good switch.

    • Alan Bell says:

      yeah, that is probably the case, but I would like to prove it. If for example I could get peak performance of 200Mbit out of wireless N (max 300Mbit) and I can get peak performance of 75Mbit from the pi ethernet adapter, and if the devices are mostly not doing network activity except for occasional bursts to upload a package and download a new one with build dependencies, then it could work out that a faster shared connection beats a slower one with less contention. As it stands the wifi performance varies from almost nothing up to 20Mbit, so ethernet is winning hands down at the moment, but I think there is a problem with the wifi drivers, this isn’t just wifi being poor, this is broken.

  • Andrew Back says:

    I have a feeling that you may experience WiFi issues unless you have multiple APs all on different radio channels, which even then is tricky as channels overlap and so you wouldn’t get linear bandwidth scaling. I’d really be tempted to go for wired…

    The Southampton University power supply solution just makes me cringe and I have to wonder if it was done like that for novelty value. A simple ATX power supply should give you 5V @ 40A and distribution is trivial. Have to wonder if it wouldn’t be more efficient also (think of a big multi-socket server plus hypervisor vs. lots of little boxes).

  • Dmitrijs Ledkovs says:

    Jaunty is not the last Ubuntu release compatible with RaspeberryPi. Quantal had armel set at armv5t, which should be compatible with raspberrypi. It’s not optimal however, armv6hf is best for them (same one that rasbian uses) https://launchpad.net/ubuntu/+source/gcc-4.7/4.7.0-4ubuntu2

  • Brian Gowland says:

    On a separate note…

    I know the project is about building the cluster yourselves but (just out of curiosity) had it occurred to you guys (or had you previously considered) using the “crowd” philosophy to find volunteers to use their own RPis from their homes or offices as a distributed cluster?

    For instance – I have an “up to” 120Mbps downstream and an 8Mbps upstream broadband connection and 2 RPis. As ever the “up to” is a caveat but in the early hours of the morning I can get 105-108 Mbps downstream and 6Mbps upstream fairly consistently.

    As I said, just curious, perhaps a different use for RPis and volunteers for a different project. :)

    …and one final question. What are you using to benchmark the wifi comms speed and what adapters are you using? I just got a nano USB wifi adapter for one of my RPis (not tested yet) and I’d be happy to run some benchmark testing for comparison as this obviously seems to be a concern WRT using wired or wifi.

    • Alan Bell says:

      I had thought of using crowd based building, but I had assumed that network speed would be a problem – turns out like most assumptions, it isn’t really as clear cut as you would expect with this project. One concern might be around signing the packages, normally you have trusted builders if it was massively distributed it would be possible for someone to modify a package on their pi and upload a compromised package. There are probably countermeasures to deal with this but I don’t have a full answer yet. In principal if it works then that could become very distributed and large scale indeed.

    • Alan Bell says:

      For benchmarking I am using iperf -s on one computer and iperf -c IP address of the first one
      I was testing the pihut wifi adapter http://thepihut.com/products/usb-wifi-adapter-for-the-raspberry-pi and I tested a few others I have lying about.

      • Brian Gowland says:

        Unfortunately no difference with my Edimax wifi dongle. I used my desktop as the server (it uses wired Gigabit ether direct to my router) and ran iperf – c on the Rpi 10 times and averaged ~23 Mbps. When the RPi is wired I get an average of 80 Mbps.

        On the subject of power (I can’t reply to that thread – there’s no option). How about USB distribution board using PCB USB A sockets? It would save chopping up USB leads to make micro usb tails. Something like these http://uk.rs-online.com/web/p/type-a-usb-connectors/5152049/

  • Richard says:

    Hi,
    Can i help? I have 40 raspberry pies all already powered and connected.

    I would be able to provide remote access to them.
    Regards,
    Richard

  • paulc says:

    slightly confused here… I was under the impression that CPU cycles was the problem here, not bandwidth.

    • Alan Bell says:

      well there are many problems :) CPU cycles is one, but I can’t do much about that one other than add more devices in a scalable way. Bandwidth is less of a problem, but it is worth testing the assumptions and optimising it as much as possible. To build things the builders start with a clean chroot and download and install all the build dependencies, then once done they throw it all away and start from clean again. This does use some bandwidth and the faster they can do it the better. The things we have to optimise are power distribution, disk/SD speed and networking.

  • sam says:

    Great to see your approach of measuring things, not just going off intuition.

    Another thought. What is the CPU utilisation for wifi compared to ethernet? If the wifi dongle is offloading things to the CPU that might be bad. It may also explain why the bandwidth results are so low. Also are you looking at just raw network bandwidth, or bandwidth with the additional CPU costs of secure network protocols and reading and writing to storage.

    Also does overclocking make much difference?

  • zlk says:

    Why asking money for a cluster of RPi? Cross-compile and you’re done. You don’t need a cluster.

  • fhf says:

    Any progress on this project? :)

Leave a Reply

XHTML: You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>