Android and Open Source
A lot of noise has been made recently about Android and it’s degree of “openness” (from both sides of the argument.) If you follow me here, or on the lists, or on twitter, etc, you probably already know I have strong feelings on the matter. Although I don’t usually write articles for this site (mostly it is just announcements) I had this one that I didn’t post to the lists. (It got away from me, and was mostly off-topic to the discussion.) I liked it enough to finish it, and so here you go. My take on Android, open source, and how we get this thing from a toy up to world domination.
Lets think about the big players in Android right now. (As best as we can see from the outside – clarity welcome, with references.) Google, of course, hosts all of the developers. Tmobile and Vodo and company pay all the bills. HTC and others make the handsets, although mostly under contract to the other members. But why? Where is the money supposed to come from? How does open source make money at all? Through the community. Community provides free support, free application coding, free advertising. In an open project, it even provides free debugging, patching, even security auditing. (In the “here’s a fix” style, not the usual “Its jailbroken now!” style..)
Google isn’t doing this out of their love of the community. (That should be obvious from their hatred treatment of the community..) Google is doing this because Tmob (and to a lesser degree Vodofone and company) are paying them to do this. If Google was doing this for love, it would be working in the open and letting dev teams (either internal or external) fork AOSP into the closed source device trees.
Instead, its the other way around, for the simple reason that that is where the fast money is. “We’ll pay you to build us a handset” is what seems to be happening. (“We’ll pay you to work on open source (and give us a couple of custom apps)” is what they claim, and its arguably where they need to be if they are to succeed.) Some very small number of their developers – I would bet fewer than 3 – currently work primarily in the open. They are noticeable because of the large quiet room they are standing in, with empty desks where the other devs should be.
Think I exaggerate? Ask where your open source SDK is for 1.5, and why its not out even though the final 1.5-compatible images already exist. (And the phone is due out in less than a month..) That will tell you where the priorities of Google lie in respect to the AOSP. (Contrast that to an open source release cycle – public bugs, alpha and beta releases, a roadmap that actually goes into the future(!!) ..)
In a successful hybrid open/closed project, the open source releases drive the closed source forks. Witness Apache httpd (open) and IBM httpd (closed). They have the same codebase, and – shockingly – its not the one in IBM’s secret dungeons that generates new releases. IBM does do their own set of releases, but they work just like everyone else – by taking changes from the open tree and integrating their closed code.
Google works with Android in exactly the opposite way – development is done mainly in secret, and occasionally someone takes the time to audit it and dump a huge, unmanageable set of changes into the open source tree. (In all but one case, this comes without including changelogs. And never do outside contributors see relevant discussions, code reviews, etc.) Periodically developers such as jbq volunteer their time to try to make the open project work vaguely correctly without the closed pieces. (And to clarify further, before you get all G1-isn’t-dev, the ADP1 – sold expressly for platform hacking – doesn’t work any better.)
So it is, technically, open. But not so fast – you cannot expect to contribute to the community version without knowledge of deadlines, requirements, UI guidelines and so forth that Google will not and can not publish. The closed source trees drive the open source development. (There are a bunch of browser changes in 1.5 – where are those patches in code review? Sure, some of them are there. Most, however, arrived – if they arrived at all – as part of a massive, opaque code dump. The same sort of bulk merging that they – correctly – refuse to do from outside contributors.)
All the handwaving and marketing in the world won’t change the way this is going. Want a happy ending? Push your developers out and make them work in public. Update the truly pitiful roadmap at least every quarter or so. Release binary versions of the closed source apps such as Gmail and Android Market. (Source is too much to ask, its not like this is open source or something.) Let Android users and community members have at least some of the capabilities that you give to rivals such as Apple’s iPhone, Nokia’s Symbian platform, and so many others. Stop talking and start acting.
The only thing Android has right now to recommend itself (to developers, handset manufacturers, carriers and community porters) is the code. The “hey neat” factor will wear off, and at the end of the day Symbian is right: Android is not an open project. Android is a closed source project with a related, incomplete pile of code. And if it is to succeed, that has to change.
(Update: minor edits to clarify, for example, that ‘hatred’ is an exaggeration. Although if that is still unclear I can add links to posts where google devs tell potential and actual platform hackers to “f*** off until its released.” And I’d love to hear from someone – with references! – about how this thing is really funded, if that summary is incorrect.)
on April 2nd, 2009 at 2:34 pm
Thank you very much.
Reply
on April 2nd, 2009 at 2:48 pm
I agree with some of your points, but i still think it’s somewhat “ok”.
But the article is good, it will tell people that android isn’t as open as it should/claims to be :)
Reply
on April 2nd, 2009 at 5:59 pm
Thanks for writing this. It’s nice that the big opaque code dumps are happening at all but it would be so much better if development, reviews, and bug tracking were done in the open.
And while we’re at it, let’s open up development of the Android Market application itself. A public twitter-like API for it would do the trick.
Reply
on April 2nd, 2009 at 7:15 pm
@Joushou: agree with you, I think it’s sort of OK to monetize a 6month period of exclusivity. They are still going to put this code out in the open very soon.
Reply
on April 3rd, 2009 at 12:24 am
Hypocricy. QT and MySQL are developed in this way. Heck, GCC was developed in this was by FSF for TEN EYARS (till egcs fork). Are you claiming all these projects are not successful? Are you claiming that all these companies (including FSF) hate community? Hard to believe that…
Reply
Disconnect Reply:
April 3rd, 2009 at 8:37 am
QT was developed in the public spaces, but they used the gpl to force everyone to purchase a commercial license. (I know, from personal experience – I had to cancel a project because the other free code wasn’t gpl-compatible and they required me to purchase a device license for every device using it.) And um.. how many qpe/qtopia phones do you see these days? And how many commercial licenses are they selling? Last I heard they finally gave up an went lgpl. So.. yes, I’d call that a fail. Nice try though.
Someone else already commented on gcc. And mysql was unabashadly commercial. It wasn’t an “open source project” it was “a commercial project with source.” Different space, different goals. And they -still- didn’t handle it as badly as google is handling android.
Reply
Disconnect Reply:
April 3rd, 2009 at 1:07 pm
oh, and I forgot. http://www.answers.com/hypocracy might help. would you like to explain where my article meets that definition? cuz you lost me..
Reply
on April 3rd, 2009 at 1:41 am
Thanks a lot for speaking up about that ! The past 4 weeks I’ve spent ‘working’ on Android have been some of the most painful I’ve ever experienced. 1/3 of the time wasted due to instability, 1/3 of the time figuring out what was useful in the opaque-code-dump-everything-in-a-commit, and the remaing 1/3 on lack of documentation. Hell… it was easier to hack on the iPhone *BEFORE* the SDK was out :)
Reply
on April 3rd, 2009 at 3:00 am
GCC was different. They did few releases, but FSF released all the sources, changelogs etc., discussion was public, and IIRC the alpha version was available (but not mirrored). FSF was born when internet was not common and expensive, the contrary of google model. Anyway EGCS opened the process and took the advantages of Interned and wide community.
Reply
on April 3rd, 2009 at 9:26 am
I worked with GCC in the pre-egcs days. Yes, the public couldn’t see the source until the release, but interested developers who showed that they knew what they were doing could get invitations to join the team, without needing to work for a particular company.
Reply
on April 4th, 2009 at 3:46 am
I never equate “open source” (small caps, don’t really care too much for the stricter Open Source which most people take fondly too) to necessitate open development.
I think Google did the minimal of what open source software entails.
I am okay with the open source -> closed vault -> put back into tree.
Any party that puts in effort should – i feel- fulfill some of the commercial self-interest (stronger influence) to be fair.
I’m more worried this becomes a vehicle for Google to be the next MacroSlob of the Web by bundling all kinds of G-stuff under the facade of OHA, or adverts. A benevolent monopolist is still better than an evil one.
Symbian is right: Android is not an open project, neither was S60. nor do I believe that by being an open project, you are any better at creating value for this world as a well-executed closed one.
only question is on everyone’s minds – in a weird Android way, what is Google’s true Intent, and how are we going to Filter it…
Reply
on April 5th, 2009 at 3:33 am
I came accross your site yesterday and can only say: You are right! Let’s change this!
Android development is not open (yet) but it’s much more open then Symbian or WM, so i think it’s a good starting point. As floks at google are ok with forks, why not set up a fork of Android, which is developed in the open community?
As a first step it would be great if one could get access to your git repository with all the features like Maps and Market app in it.
Reply
Disconnect Reply:
April 5th, 2009 at 10:16 am
What makes you think I have a git repo at all, much less one full of CLOSED SOURCE apps? You want that code, you gotta ask google for it. Or work with the bins that leak, just like the rest of us.
Reply
Brian Reply:
April 5th, 2009 at 11:40 pm
If you were in the mood to do a favor for a perfect stranger (and probably many others in the dev community that simply can’t keep up with all the git revisions happening simultaneously in master/cupcake)… would you mind posting a tarball of android source code which will build into fastbootable images enabling sound, buttons, wifi, and cellular?
(I’m going to rant now because it is late and I am frustrated. Please turn away.)
I have been trying for 2 days to use the revert.sh script you packaged with the 03-16-2009 image to produce something resembling the 03-16-2009 binary, and the closest I have come (after greatly massaging the source code) is a system with no sound (because it states at boot that it cannot find htc_acoustic.so, even though I cannot find that file on any other phone image) and the Home button disabled.
Maybe I am using your revert.sh script incorrectly. I created a new directory, ran “repo init -u git://android.git.kernel.org/platform/manifest.git”, loaded the local_manifest.xml file from android’s “Building for Dream” webpage into .repo, ran “repo sync”, then ran your revert.sh script from within that directory. I ran make, and minutes later got a Java compilation issue in
“packages/inputmethods/LatinIME/src/com/android/inputmethod/latin/LatinIME.java” because “frameworks/base/core/java/android/view/inputmethod/EditorInfo.java” lacked a bunch of attributes and methods it required. And then the fun began trying to use a completely current build to drop in missing pieces all over the system from there.
If I’m doing something wrong, please tell me. There is no documentation on building a single stable image for ADP1. I’ve pieced together my knowledge primarily by reading makefiles and shell scripts in JesusFreke’s BuildEnviroment. (Like Grandpa Simpson’s sugar packet based knowledge of history…) Not that it helped… I never managed to build a JesusFreke image with wireless or stable enough to touch anything. I would run the repoinit.sh script. I tried building that… unusable. I tried running the various patching scripts provided. Most of them introduced such egregious syntax errors, that I know he/she never used them. (The patchinit.sh script caused a git merge conflict, and after resolving the conflict: macros and functions were missing. The browser patch installed a java file that had a package name that didn’t share a word in common with where it was installed.)
Anyway, to finish up this whining… There are a lot of people who would really appreciate just one coherent source tree to use for personal projects. Fill in binary blobs where necessary for the actual hardware dependencies or have an extract-files script which is guaranteed to work. This tree does not need to be updated any more often than Google releases new sdks. The linux kernel is managed in git, but I never need to know that because I simply grab a tarball whenever I need to kernel dive. An open source operating system doesn’t imply that everyone hacking on it is trying to productively advance the state of the art.
I’ve seen you around the android-platform message boards, you follow the src changes pretty closely. And I know you can build a stable and useful system because your 03-16-2009 binary image ran great. Please, please, please… give us a well documented way to build that same image without assuming any steps.
If I am an idiot and missing something obvious, please tell me and make fun of me for it rather than ignoring me.
If you want to reach me directly, please email blackbu at gmail dot com.
Thanks
Reply
devxlar5 Reply:
April 6th, 2009 at 12:23 pm
You got me wrong, i know you (properly) don’t have the CLOSED SOURCE apps source code. I was just asking (in few words) what Brian explained with more words.
And I would just prefer git in favor of tar balls, because changes can be tracked using git.
So, do you see a way to set up a stable/running/compleate fork of Android?
Reply
Disconnect Reply:
April 6th, 2009 at 12:25 pm
repo barely allows for any synchronization between the project trees it includes. the closest i can offer is the revert.sh script, which sometimes works. (and i’m certainly not going to try to host a full one-tree git tree any more than google does. although yes, i agree, its insane that there are so many moving targets, but its due to the tools they built/chose..)
on April 6th, 2009 at 3:21 pm
The deficiencies of repo is why I suggested just making a tarball snapshot occasionally of a file tree that built successfully and runs more or less stable.
Yes, working off a tarball image will make version control of updates produced within that tarball more difficult. However, right now it is nearly impossible to get a good filetree in which to develop updates and new ideas in the first place!
If you could spend a day or two and produce a coherent file tree which builds a mostly-stable image with basic functionality (just like the images which you have been publishing here), and document any build steps outside of make and fastboot, I will host the tarball for you. As a grad student, I have plenty of access to storage with high internet bandwidth.
You can be the door through which the open source community comes to Android, because as you said, Google is making it more of a headache than it is worth.
Thanks!
Reply
on April 13th, 2009 at 3:04 pm
maybe its time to release a new image?))
Reply