On the Evil of Diverse Proprietary Things, and on Instant Messaging

A first, stream-of-consciousness draft of an essay probably additionally hampered by lack of technical insight, which I may or may not go back and refine. Feel free to offer thoughts, corrections, and comments.

While I am the first to applaud the technical efforts of Richard M. Stallman as well as his instrumental role in creating the free software movement and thus more or less directly the open source community we have today (although I identify myself as a Linux user, I am fully aware that GNU software is probably more important to me than Linux), I disagree with the Free Software Foundation's primary edict: That proprietary software is the root of all evil. (Of course I exaggerate.)

I do not personally have a problem with proprietary software (at least on an ethical level). I generally choose not to use it (with the exception of games), but I don't think there's anything wrong with selling software, I don't think it's unethical to keep the source closed (you have the freedom not to), and I don't honestly think that most our problems stem from the existance of such closed-source, proprietary software. Instead, I think that the threats to free software are

• proprietary formats,
• proprietary protocols, and
• proprietary hardware specifications.

Proprietary formats are a problem because they seek to prevent me from choosing what tool to use for a job. Let me emphasise this: I don't have a problem with, say, the Adobe Acrobat Reader being proprietary and closed. In fact I don't give a damn since I use Gnome's excellent new document viewer, evince, to view and print PDF documents. However, while I don't object to the Macromedia Flash player being proprietary, it does bother me that the Flash format is closed. Why does it bother me? Because it forces me to use not merely tools, but a toolchain of Macromedia's choosing. True, I can use Firefox (not Internet Explorer) and Linux (not Windows), so I'm lucky here, but if I happened to use a less popular OS, or a browser that doesn't support a plugin format that Macromedia chooses to offer, then I'm out of luck. Even for GNU/Linux and Firefox (a fairly significant platform as non-Windows platforms go), I can't view Shockwave animations. And this is just for viewing, not content creation!

Can I see why they would want to keep this format proprietary? Yes, I can. (Can I sympathise? Not really, but that's personal opinion…) But it is a great problem because it leaves us at the mercy of the Macromedia corporation to access content that we have every right to view. If we wish to use a platform they do not deign to support, or if Macromedia goes out of business, or if (as I expect will happen at some point in the future) Flash is deprecated and no longer actively supported, all this content will be lost. It may not be terribly critical for Flash animations, but if it weren't for open source hackers reverse engineering Microsoft's closed file formats, the same would apply to Microsoft Office files. Even as it is, it is true for a lot of file formats out there.

This tends to be a problem in monopoly environments; that is, environments where a single vendor holds a virtual monopoly on the content type. Unfortunately, this is true for a frightful number of content types; Microsoft vastly dominates office-type document creation, Macromedia is virtually the only player in the animated dynamic web content area, and so forth. In areas where several or many vendors need to co-operate, standardised formats need to be developed and agreed upon in order for new tools to work with the existing content base. But just as a diverse vendor base requires open formats, open formats open the path for a diverse environment since any new vendor can create a tool to manipulate the same files. As it is, no one has much of a chance to develop a serious competitor for Macromedia Flash because no one would use it: People go for the technology already deployed. People have Flash players.

What's the solution to this problem? Demand open formats. Support them when they're out there. Deploy files in open formats whenever remotely feasible (since free tools tend to emerge for open formats, end users can obtain proper tools to view what you deploy if they lack them—of course this inconvenience may not always be viable). Don't adopt new closed formats. Try not to fragment too much—diversity in file formats may not be an inherently bad thing (we do need innovation), but first we need to standardise on open formats that we can then proceed to evolve.

If someone then proceeds to write a proprietary application that works with these open formats, I fail to see a problem with it. Dreamweaver is a good example of just such an application; it is a well-seen and widely used application; it costs money to use, and provides returns by offering features unmatched (as far as I know; I don't use WYSIWYG HTML editors) by its competitors, and I don't begrudge its creators their chance to earn an honest buck—but at the end of the day, if I don't want to pay this money, I can get another editor to produce the same type of code: Plain old HTML, CSS, and JavaScript.

Proprietary protocols are a problem for the same reason as proprietary file formats. My favourite gripe in this area is instant messaging protocols. You can read more educated and extensive discussions on the problem elsewhere, but in a nutshell, the IM protocol situation of today is not viable. We have a profusion of closed, proprietary protocols that cannot communicate, much as in the early days, predating SMTP and proper email, you could only send messages to other users on the same network. This is not actually very useful: For email to be useful, I need to be able to email anyone, anywhere.

There is a solution underway, thanks to Microsoft and Yahoo!. It is a terrible solution, a solution that bodes ill for us all. This solution is the merger of major extant IM networks, allowing users of the MSN Messenger service and the Yahoo! Instant Messenger service to intercommunicate. (Apparently, this is even going to extend to voice and video.) This is a terrible solution because it keeps instant messaging locked to a small set of vendors, using proprietary protocols. Again, what if I want to use an OS that they don't care to support? (With Microsoft, this is every OS that isn't Windows. Yahoo! does provide a Linux client for its IM service, but it lacks features.) What if I just don't like their clients? (I don't—I think they are terrible applications full of garbage features that I would strive to disable if I used them; fortunately I use Gaim.) Instant messaging is a huge, thriving application area on the internet, like email in the nineties. Like email, I want to be able to get an account with the service provider of my choice, and I want to use the client of my choice to access it.

Multi-protocol clients like Trillian or Gaim are not a viable solution in the long run. For now, they suffice. However, they will always be playing a catch-up game in terms of features (Gaim still doesn't have voice or video chat support, although it is in the works), and worse yet (much worse!), it is possible for the proprietary network vendors to simply lock them out. Gaim is not an official client; it works because open source hackers reverse engineered the proprietary protocols. These protocols can be changed to lock out third-party clints. (This has in fact happened, several times; each time the Trillian or Gaim developers have modified their clients to cope, but there's always the chance that Microsoft and Yahoo! decide to do something serious and implement some features that make the protocol difficult to break.)

The solution here is obvious, and the same as for proprietary file formats: Use, encourage the use of, and support open protocols. In the IM world, there is a good solution: It is called Jabber. It is an IM service which, like email, offers a distributed system where anyone can set up a server and a user on one server can talk to people on any other server. (Unlike email, it was designed with features to deal with spam.)

There are a couple of problems with Jabber. First, it is not terribly widely used. This could change with GoogleTalk. Let me make my position here very clear: GoogleTalk makes me very apprehensive. It is based on the Jabber protocols, but it is not a proper Jabber service because it only allows users to communicate with other GoogleTalk users. If Google makes GoogleTalk use a proper, open set of protocols, it is a very, very wonderful things; I will love Google for it, and instant messaging may have a chance, for the very first time, to work as it should work. All messengers can communicate over the same protocol; individual messengers can experiment with additional features if they so choose, but these additional features can be included in the main protocol if they are deemed useful.

Update (Sun May 21, 2006):
A few months ago, Google did open up GoogleTalk and made it a proper Jabber service. To date, the multimedia situation—voice and video—looks a little more uncertain, but things may be off to a good start. Hooray for Google! And may more people use Jabber/GoogleTalk.

The second problem with Jabber is that it's text-only. In today's instant messaging world, that is not enough. We need voice chat. We need video; we need webcam support. GoogleTalk does offer voice chat, but it is not a proper Jabber service; even if it were, their voice protocol is not open (they promise it will be, but proof is, as always, in the proverbial pudding). In the free software world, we have standards like SIP and various open CODECs; I don't imagine this would be terribly difficult to slap onto something Jabber-like. When we have something like this, supporting voice and video, then we have a viable IM solution. I hope Google will provide it for us. If not, it has to come from somewhere else or MSN and Yahoo! may well take over the whole scene.

Proprietary hardware specifications are, of course, a problem only to users of non-mainstream operating systems. Unfortunately, this generally comprises every operating system except Windows and Mac OS (and the latter only due to hardware lockin), and it is a fairly major problem. Still more bothersome to me is the fact that as far as I can see, there is no reason for it.

If a hardware manufacturer such as (my favourite example) ATI were to release their hardware specifications (and not for hardware so out of date as to be nearly useless for anyone who wishes to maintain a competitive desktop operating system!), or better yet, open up their driver suite under, for example, the GPL or LGPL, they would reap a number of benefits. They would earn a significant amount of goodwill from open source/free software fans, of course; this is not terribly significant financially (although for what it is worth, I expect they would gain virtually the entire free software segment) but may well matter in a technical fashion. Driver maintenance could become considerably simpler (alternative OS users could write their own drivers; if the official drivers were open, the free software community could help with beta testing, submit patches, help porting efforts, and so forth).

These are not tremendous benefits, I will concede. However, they are benefits. What are the drawbacks? Why don't we see more open hardware specifications or open-source drivers? If it is fear of maintenance difficulties, I think it is unfounded; the free software community will gladly assume the responsibility of testing and patching it to make it work, if only it is free and libre (let's face it, there are lots of crazy people among us). Is it fear of giving out trade secrets?

Are you serious?

I freely admit that I am not a hardware guru (nor ever will be), and in an essay founded on personal experiences and opinions this is the weakest of a series of weak arguments—I may well be wrong, and if so, then please enlighten me. As it is, I just don't buy it. As the R300 project (among oh-so-many others) shows us, the open source community can reverse engineer hardware protocols and implement their own drivers. If they can, then competitors can do the same by simply throwing some money and programmers at the problem. Granted that it is harder, slower, and more expensive without the source code, but I don't really believe that it's a critical difference.

More importantly, however, I don't really believe that reverse engineering the hardware protocols is going to matter very much. I don't know how much of the hardware design is laid bare (ATI and NVIDIA make money from hardware, not drivers!), but even if it is a significant amount, then is it really going to matter? In the case of ATI and NVIDIA, for instance, we have a very competitive market. ATI and NVIDIA tend to be more or less neck-to-neck; the question is who is going to release the next generation of speed demon video cards first. I suspect that by the time the competitor has time to analyse the driver source and mimic the ingeneous design, it's probably too late for them to start manufacturing the copies—never mind starting up the whole hardware design and implementation process.

This essay is a work in progress. I may go back and edit it. If anyone comments, these comments may be made irrelevant or nonsensical by essay revisions (so don't blame the commenters). I may repost it.