Rube Goldberg Streaming by Mark Stone
January 2013. The TV show M*A*S*H had a wonderful story template involving the company clerk (Radar, then later Klinger) and Hawkeye's byzantine attempts to requisition what Hawkeye needed. The goal was often something simple: a weekend pass in Seoul, or extra doses of penicillin. The episode would then take the viewer through a hilarious sequence of "I'll trade you X if you can get me Y", until the entire house of cards collapsed on itself, with Hawkeye sometimes succeeding, but often having success snatched from him at the very last turn of the plot.
Over the holidays I felt a lot of sympathy for Hawkeye, and a renewed respect for Radar and Klinger. My goal seemed simple enough, indeed it seemed like something the Android and Linux community would have made brain-dead simple years ago: stream digital music files from the file server on my home network where they are stored to an Android phone plugged into the auxiliary jack of our stereo. The journey to reach this goal has been anything but simple.
Here's what my home network looks like; I'd bet many Linux enthusiasts have something like this with a few minor variations:
- One desktop running Ubuntu 12.04.
- Two laptops; my wife's runs Ubuntu 12.04, and mine is a Chromebook. Her laptop has the DVD drive we use to rip CDs, so a copy of all of our music files is there.
- The Android gang: Three Android phones (mine, my wife's, and our "home" phone) and a Nexus 7 tablet. Thanks to the capriciousness of carriers, we're actually running four different versions of Android on these four devices (2.3, Cyanogen 7, 4.1, and 4.2).
- Two headless servers; one is for file backup, and runs the FreeNAS variant of FreeBSD. All the music files are copied to here as well. The other is my "dev" server, running Ubuntu Server 12.04, and its where I do my hobbyist programming.
- Miscellaneous devices: The Wii, and my wife's Nook eReader.
A few comments on this setup: the network is wireless, though the FreeNAS server has a wired connection to the wireless hub. The stereo is not, by default, connected to the network. Its very old school: receiver, turntable, cassette player (which never gets used), CD player, and speakers. I have a split stereo cable that enables me to connect any device with a single 3.5mm audio jack (laptop, tablet, smartphone) to the auxiliary on the stereo. The obvious way to get the stereo "onto the network" is to plug in one of the Android devices and use the stereo's auxiliary.
The challenge in meeting my goal is two-fold:
- I need appropriate server software running with access to the music files to stream those files onto the network, and
- I need an appropriate client app running on Android to receive and play the streamed file. This should be an app that is compatible with the server software, one that has a reasonable UI that allows for intuitive browsing of the music archive, and some simple organizing features like playlists.
Simple, right? Thousands of people must have exactly the same need so how hard could it be to find a solution?
Of course the obvious solution, and the one that the big cloud providers want you to use, is to store your music files remotely on their sites and then stream using their services. I have a few problems with this "obvious" solution, though:
- It grates on me to be using Internet bandwidth for a task that really doesn't need to leave my home network. All the files are already here, and all the listening will be done here; why should I have to go out to the Internet and then back again just for streaming?
- This isn't just a matter of principle. As an experiment, I've tried streaming from Amazon using their Cloud Player, and I get occasional network errors that cause the music to stop playing. And Google Play Music is just a mess, cluttered with free music files I am not interested in that do nothing more than make it difficult for me to sort through to my own music. Both Amazon's and Google's services are getting better, but neither is really ready for prime time yet.
- As much as possible, I put my music in Ogg Vorbis format. Cloud services vary widely in how well they handle Ogg files. For Google Ogg is fine, but this is nearly a complete show stopper with iTunes. Amazon has nominal Ogg support, allowing you to upload some Ogg files that they can determine match their collection, but what they stream back to you will be their matching MP3 file, not your Ogg file. Sort of defeats the purpose of using Ogg Vorbis.
So really the only satisfactory approach for me is to run my own streaming server on my local network. Wikipedia has a nice list of streaming media systems. Two obvious candidates from this list are Ampache and Icecast. I ended up using neither of these.
Ampache's appeal is that it runs on top of the near-ubiquitous Apache web server. I do run a web server on my home network, but it isn't Apache. Its my own home brew written in Node.js. And while installing Apache isn't difficult, it just seemed like an unnecessary extra step for my setup.
Icecast is an impressive piece of software, one of the oldest open source streaming servers, and arguably the market leader among open source solutions in this space. But its also massive overkill for what I'm trying to do. If I were running a public podcast station, or Internet radio station, or involved in a start-up with significant streaming audio needs, then Icecast would be great. But for my little music collection strictly on my home network, Icecast is a bit like using a cluster bomb when all you really need is a .22. I'm also not clear on what Android music app, if any, would pair well with Icecast.
Instead I approached the problem from the client end. My first attempt involved Gmote, an Android app that turns your phone into a streaming media remote control. There's also a Gmote server, written in Java, that looked pretty straightforward to install. I ran into three problems with Gmote:
- The Java server has to have a GUI, and thus will not run from my headless server. I did get it running on the laptop, but this requires that the laptop be up and running and not suspended for the server to work. Not a complete deal breaker, but a bit awkward.
- The Gmote client works more or less well with different versions of Android. With my Cyanogen 7 phone it seemed to basically work as intended, but with my Gingerbread phone it attempted to hand off actual playing of the music files to the web browser. Again, not a complete deal breaker (just don't plug the Gingerbread phone into the stereo), but awkward.
- Despite information to the contrary on the Gmote site, I could not get Ogg files to play. There's no mention of this issue on the site FAQ, but the discussion forum indicates others have had this problem, and my post requesting more information went unanswered. That's a deal breaker. Gmote is not my solution.
The option I finally settled on is Subsonic. This is another Java-based server solution with a dedicated Android client. Subsonic has several advantages over Gmote, however:
- Subsonic can be compiled headless, so it doesn't require a graphical interface to be running on the server.
- While Subsonic has an Android client, it also has a solid web interface, making it easy to browse, select, and play music from a laptop or desktop computer.
- For paying Subsonic users (more on this later), who also know how to configure IP forwarding for their home network, you can register on the Subsonic site and thus configure your home Subsonic server to be available to you over the Internet when you're away from home. I don't actually feel a need to take advantage of this option, but its definitely a plus for Subsonic that this option is available.
- Oh, and support for Ogg files just works. Like it is supposed to.
So that's the end of the story, right? I found my solution with Subsonic and lived happily ever after.
Not quite. And in what follows I freely admit the problems are of my own making. But I bet I'm not alone.
I could run Subsonic on the laptop. But -- as I mentioned in my discussion of Gmote -- that's awkward in that it requires the laptop to be up and running (and at home on the network rather than with my wife at the local Starbucks). The other place where the music files reside is on our FreeNAS server. It is certainly capable of running a Java server application, but the idea of installing Java on a server intended for backup gave me pause. What I really want is for this particular machine to be as brain-dead stupid -- and thus fault-tolerant -- as possible. The idea of installing and running even moderately complicated applications on it just goes against the grain of "dependable backup".
The ideal thing would be to run Subsonic on my dev server, which is in fact designed for experimenting with random code. That would mean, however, chewing up a lot of hard drive space copying over the music files from the laptop or backup server, and it would introduce another machine that has to be regularly synced with those files. What I really want is access to the music files on the FreeNAS backup server without having to actually copy them over.
Enter sshfs. Sshfs is an outgrowth of the FUSE (Filesystem in User Space) project. File systems, of course, are part of the backbone of the operating system and normally file system manipulation is done only when the operating system is first installed or for rare maintenance or performance tuning tasks. In other words, ordinarily file system manipulation is only within the domain of root users. FUSE, by contrast, allows a user level file system to be installed on top of the operating system level file system, and allows user manipulation of this secondary file system. While there is a modest performance hit with FUSE, you get a big increase in flexibility without compromising security.
For example, with FUSE you can use ssh to create a network file system that allows directories on a remote machine to be mounted locally. This is exactly what sshfs does. For hobbyist users like me who do not want to muck around with the complexities of Samba (or even worse, NFS) sshfs is a godsend. Easy to install, easy to use, and doesn't compromise the file system integrity of either machine. In fact, the remote machine does not know or care about sshfs; all it sees is an ssh connection and an ordinary set of file commands coming over that connection.
So for me this was the final piece of the puzzle. On the FreeNAS server I don't have to do anything. It continues to house backup copies of all music files in the Music directory as before. On my dev server, I have Subsonic installed, and I also have sshfs installed. Using sshfs, I mount the Music directory on the FreeNAS server as a local directory, and then point Subsonic at that local directory as the root directory for Music files. An Android device connected to our stereo can then play music streamed from the Subsonic server over our stereo using the Subsonic Android app, or we can listen to music directly on one of the computers using a web browser connected to the Subsonic server's web interface.
So let's review the journey of a music file to the stereo:
- Insert a CD into the Ubuntu 12.04 laptop, and rip the music files from that CD into Ogg format.
- Alternate step: purchase MP3 files and download onto the laptop.
- Those files are then copied over the network to the FreeNAS server in one of the regularly scheduled backups.
- That directory on the FreeNAS server is mounted via sshfs as a directory on the machine running the Subsonic server.
- The Subsonic server then streams the music file over the home network.
- We plug an Android device into the auxiliary on the stereo.
- And run the Subsonic client on that Android device.
- This Android device can then receive the streamed file and play it over the stereo.
One final note: I'm very happy with the solution I've settled on, but it doesn't seem like this problem should be that hard to solve. Surely enough users have enough similar needs to create demand. And an elegant, simple way to meet this demand is important enough to be worth something. I paid $20 for a license key to Subsonic not because I feel any need to stream my music beyond the home network, but because good software is worth paying for. And right now Subsonic is a good piece of software in a realm with few alternatives.