Tuesday, November 28, 2006

How Linux and open-source development could change the way we get things done

An army of disheveled computer programmers has built an operating system called Linux based on a business model that seems to have been written with everything but business in mind. Instead of charging customers as much as the market can bear, Linux is given away for free; instead of hiding information from competitors, Linux programmers share their work with the world; instead of working for money, Linux developers are motivated primarily by adrenaline, altruism, and the respect of their peers.

Despite this unusual foundation, Linux is booming and even beginning to challenge Microsoft's control of the operating system industry. Linux may eventually pull the rug out from under the richest company in the world. It may not. But no matter what happens, it has already shown that money doesn't have to make the world, even the business world, go round. In fact, as technology improves and computers connect and create even more of our society, the principles of cooperation and collaboration that drive Linux may well spread to other fields: from computers, to medicine, to the law.

The Source

The Linux movement kick-started in 1991 when Linus Torvalds, a puckish graduate student at the University of Helsinki, got frustrated with his rickety computer. Refusing to buy another one, he wrote a new operating system--the core programs by which applications (like Microsoft Word) talk to hardware (like microprocessors). When finished, instead of running down to the patent office, he posted his code on the Internet and urged other programmers to download it and work with him to improve it. A few emailed back suggestions, some of which Torvalds took. A few more wrote the next day and a couple more the day after that. Torvalds worked constantly with these new colleagues, publicly posting each improvement and delegating responsibility to more and more programmers as the system grew. By 1994, Linux (a combination of "Linus" and "Unix," another operating system) had 100,000 users. Today, it has between 10 and 20 million users and is the fastest growing operating system in the world.

But Linux (rhymes with 'cynics') is different from almost every other operating system available. For one thing, it's downloadable for free straight off the Web. It's also open source, meaning that the source code, the program's all-important DNA, is open for anyone to look at, test, and modify. Most software is developed so that only the original authors can examine and change the code; with open-source models, however, anyone can do it if they have a computer and the right intuition.

To see the power of this model, consider what happens when you're running Microsoft Windows or Macintosh OS and your computer crashes: You stamp your feet and poke a twisted paper clip into a tiny reset button. You probably don't know what happened and it's probably going to happen again. Since you've never seen the source code, it probably doesn't even occur to you that you could fix the problem at its root. With Linux, everything's transparent and, even if you aren't an expert, you can simply post your question on a Linux-help Web page and other users can usually find solutions within hours, if not minutes. (The amorphous Linux community recently won InfoWorld's Product of the Year award for Best Technical Support.) It's also entirely possible that someone--perhaps you--will write some new code that fixes the problem permanently and that Linux developers, led by Torvalds, will incorporate into the next release. Presto, that problem's fixed and no one will need paper clips to fix it again.

To make another analogy, fixing an error caused by a normal software product is like trying to fix a car with the hood welded shut. With Linux, not only can you easily pop the hood open, there is extensive documentation telling you how everything works and how it all was developed; there's also a community of thousands of mechanics who will help you put in a new fuel pump if asked. In fact, the whole car was built by mechanics piecing it together in their spare time while emailing back and forth across the Web.

The obvious threat to this type of open development is appropriation. What if someone lifts all the clever code off the Web, copyrights it, and sells it? What if someone takes the code that you wrote to fix your crashed computer (or busted fuel pump), copyrights it, and markets it for $19.95? Well, they can't. When Torvalds created Linux, he protected it under the GNU General Public License, an intriguing form of copyright commonly known as copyleft. Under copyleft, anyone who redistributes the program, with or without changes, must pass along the freedom to further copy, change, and distribute it. Theoretically one can download Linux off the Web, add a string of useful features, and try to sell it for $50. But anyone who buys this new version can just copy it, give it away, or sell it for a dollar, thus destroying the incentive for piracy. An equivalent would be if I were to write the following at the end of this article: "Verbatim copying and redistribution of this entire article is permitted in any medium provided this notice remains at the end."

Use the Source

From the Oxford English Dictionary to hip-hop music, open-source development has always been with us to some degree. Since the mid-19th century, contributors to the OED have defined words and sent them to a centralized location to be gathered, criticized, sorted, and published. With music, as long as you stay within certain copyright laws, you can take chunks of other people's compositions and work them into your own. Most academic research is also built on open-source cooperation. Even the compact fluorescent light bulb above my head comes from data shared by researchers over centuries about electricity, properties of glass, and centralized power.

But still, most business isn't done with open source. Coca-Cola keeps its formula secret, Microsoft won't tell you how it builds its programs, and when a researcher for Ford suddenly stumbles upon the means to a more efficient fuel pump, she doesn't reflexively email her friend at Honda with a precise description of her discovery. A great deal of scientific and medical research is also done through closed source as individual laboratories race each other to determine who'll be the first to find the answer and earn the patent.

But two extraordinary changes are making open source substantially more plausible as a development and research model for 2000 than it was for 1990--and they'll make it even more so for 2010. First, the Internet. Today, I can open my Web browser and communicate instantly with Burmese refugees or writers working on projects similar to mine. Secondly, computer power has been increasing exponentially for generations and will probably continue to do so--in large part because every time you build a faster computer it allows you to build a faster one still. It's difficult to overestimate the change. The standard laptop to which I'm now dictating this article (with technology unavailable just two years ago) has more power than almost any computer owned by the government a decade ago. In four years, it could well be obsolete. As author Ray Kurzweil and others have pointed out, if cars had improved as much over the past 50 years as computers, they'd cost less than a nickel and go faster than the speed of light.

Intellectual and Physical Properties

This rate of progress is critical because the advantages of open-source development depend on the powers of technology and the balance between what can be done through thinking and what has to be done by building. Every product has certain intellectual components and certain physical components built into it. With a car, for example, the intellectual component is the thought about how to build it, how to set up the assembly lines, and how to process data you have on different kinds of tires. The physical components include the actual rubber in the tires, the machines that ran the tests, the operation and maintenance of factories.

Faster computers and increased connectivity are drastically changing the relationship between these components in two ways. First, some things that used to require physical components no longer do. You may not have to buy rubber to test the tires if you can just run a simulator online. (The 777 project at Boeing was designed so that nothing physical was produced before the plans hit the factory floor.) Second, connectivity makes the flow of information much faster and smoother and greatly facilitates the sharing of ideas and data. There is a saying known as "Linus' law" that "given enough eyes, all bugs are shallow." In other words, given enough people working on them, all problems are solvable. And the Internet has not only helped coordinate a lot more eyes: in some ways, it's given everyone glasses.

Open-source development benefits from this transition because its advantages are almost all in the realm of intellectual property. Open source improves communication and facilitates sharing ideas. But it doesn't mean that you can buy a ton of concrete for free or drive a nail into a wall without a hammer. This is why open source has come first and most prominently to computer programming: a profession where almost all of the development is intellectual, not physical, and where people have been connected over the Internet for more than 20 years. Programming also employs highly specific, common tools--computers--that a fairly large number of people have access to and that some people, such as university students, can even access for free. If a solution or improvement is found, nothing additional needs to be built; code just needs to be entered and then downloaded.

But there is still one great problem standing firmly in the way of even the most modern open-source software development project. As a 21-year-old Bill Gates asked in an angry letter to open-source programmers in 1976: "One thing you do is prevent good software from being written. Who can afford to do professional work for nothing?"

Microsoft's empire, and a great deal of the rest of our society, is built upon the assumption that there isn't an answer to this rhetorical question. To survive, organizations need to patent their information, protect their secrets, and earn as much money as they can. But the success of Linux shows that there's a way around that model. Something extraordinary has been built with a completely different set of rules. I asked David Soergel, a researcher in the department of genetics at Stanford, whether people could use the open-source model to develop medicines. "My first reaction: they'd rather get paid, and if they were any good at it, then they would be paid by somebody. My second reaction: wait a minute, that argument obviously fails for software."

Money for Nothing

Naive as it may be to think that people aren't motivated by money, it is just as naive to think that people are only motivated by money. People are motivated by a variety of factors: money, recognition, enjoyment, a belief that one is doing something good for the world, and so on. We each weigh these factors and make decisions based on our perceptions of their relative importance. At different points in our lives, we give different factors different weights. When we're poor, we tend to value simply high-paying work more than we do when we're well-off; it usually takes a high-paying job to get someone to do something really boring and it generally takes a very fulfilling job to get someone to work for less than what he should normally be able to earn.

Since people working on open-source projects generally earn nothing, or very little, there need to be other incentives. In Linux, there seem to be principally three. First, enjoyment. Computer programming can be addictive, exciting, and extraordinarily intense. Linux can be particularly enjoyable because almost every problem solved is a new one. If your Windows machine crashes, fixing the problem generally entails tediously working through scores of repair procedures which you may have used a thousand times. If A fails, try B. If B fails, try C. If a Linux computer crashes, anyone who repairs it is not only working on that one specific machine, he's finding a solution for the whole Linux community.

Eric Roberts, a computer science professor at Stanford, once explained to The New York Times that people in the profession must be "well trained to do work that is mind-numbingly boring." This is particularly true of work on most closed-source systems where programmers must continually reinvent the wheel. But with open-source projects, the things that need to be done haven't been done before. According to one only slightly hyperbolic programmer, Ali Abdin, writing to a Linux news group about how he felt after creating his first open-source project: "The feeling I got inside when I knew that I had some code out there that I can share with people is indescribable... I felt on top of the world, that I can program anything...I felt as mother would feel giving birth to a child, giving it life, for the first time."

Secondly, and similarly, Linux programmers are motivated by a feeling that they are changing the world and developing an operating system that really works. Torvalds laid out this philosophy well in a speech this summer: "I don't resent Microsoft for making lots of money. I resent them for making bad software."

Thirdly, but most significantly, Linux programmers seem motivated by prestige and, in particular, respect from their peers. Having "hacked the kernel" (contributed to the core of the operating system) gives programmers a certain stature--much as completing a four-minute mile does among runners--and, since the program is open source, everyone knows exactly who contributed what. I was once introduced to a programmer described as "the guy who wrote all the Ethernet device drivers!" as though I was meeting Jonas Salk "who came up with the cure for polio!" And, in fact, Linux programmers often discuss their work as falling in the tradition of eminent scientists. As three well-known programmers put it in the introduction to their book Open Sources: "It would be shortsighted of those in the computer industry to believe that monetary reward is the primary concern of open source's best programmers... These people are involved in a reputation game and history has shown that scientific success outlives financial success... When the history of this time is written a hundred years from now, people will perhaps remember the name of Bill Gates, but few other computer industrialists. They are much more likely to remember names like... Linus Torvalds."

Importantly, this philosophy may well be helping Linux develop creatively. There is a great deal of psychological research that shows that people actually do more creative work when they aren't motivated primarily by money. Tell a child that you'll pay her for reading a book and she'll read it with little imagination. Have one group of college poets think about getting rich and famous through their writing, according to research done by Harvard Professor Teresa Amabile, and they tend to turn out less creative work than a second group that's just asked to write poems. Is it possible that Linux programmers created such an extraordinary operating system in part because they were driven by other factors and weren't doing it for the money? I asked Professor Amabile if the implications of her research cross over to open-source programming and whether it could explain some of the remarkable innovations that have come from people working without pay. "Yes," she responded, "this [would be] entirely consistent."

Making free software affordable

Still, Linux programmers are not completely locking themselves out of the economy and there's a second response to Gates' rhetorical question: If your core open-source project is successful enough, it's possible to eventually make money off of it indirectly. No, it's not possible to make as much money as a proprietary company can--open source and copyleft will ensure this--and there's always going to be an astounding amount of work that has to be done without financial reward. But open-source programmers don't have to starve.

The trick to making money off Linux, or any open-source development, is to profit off of derivatives. No one actually makes money off the Linux code, but they can make money by offering technical support or helping customers install the program. Companies that do this follow a well-trodden path: give something away in order to sell something else. This is what cellular phone companies do when they give you the actual telephone handset for free if you agree to pay for using it a certain number of minutes a month. We do the same thing with the Monthly's Web page. We post some articles (giving them to readers for free) in the hope that visitors' interest will be piqued and that they'll subscribe.

Red Hat, the best-known Linux company, sells the operating system and other open-source programs in a box for $80, though you can download their product for free from redhat.com. Their revenue comes from the technical support they offer and from consumers' need for trust. It's much less unsettling to buy a program like Linux if you get it shrink-wrapped with a manual than if you have to download both. VA Linux, another well-known company, sells Linux hardware: You choose the memory and the motherboard; it builds you a computer.

Has the money these companies brought into the open-source movement corrupted it and made it more like the traditional model that Microsoft uses? Surely, there are a lot of people doing promotional and administrative work at Red Hat for the money and there are probably even some people working on Linux for Red Hat because they get paid a lot to do it (the company hires programmers to write code that anyone, including Red Hat's competitors, can use). But programmers mostly see the money as an added and surprising plus, and Linux is still driven by adrenaline, altruism, and peer recognition.

While it is possible that this could change, it hasn't so far. I asked Richard Stallman--the creator of copyleft, as well as many of the programs that run with the Linux through a system called GNU, and the man often considered to be the father of the open-source movement--whether he thought that money would change the attitudes of people who used to work on GNU/Linux without being paid. "In general, I don't think that wealth will make a hacker into a worse person. It would be more likely to enable the hacker to spend more time volunteering for free software instead of on work for pay."

This point is particularly germane because most open-source programmers have always had to work other jobs, and many have only been able to contribute to the project during the evenings or when their employers weren't looking. Linus Torvalds, for example, helps design microprocessors for a company called Transmeta in the daytime and does all his Linux coding after work. When I asked John Hall, vice president of VA Linux, what motivates programmers, he responded: "For some, it's altruism. For some, it's fame. For some, it's religion. For a very few, it's money."

So what's next?

To determine where open source is likely to move next, one has to imagine a scenario where these obstacles can be overcome. A project would need to be fun, or at least rewarding, to get going and it would have to primarily pose intellectual, not physical, challenges. Once it began to move, derivative financial incentives could help push it along. There also has to be a certain amount of luck, particularly with regard to organization. It's hard enough to get six people to agree to a restaurant for dinner; it's much harder to coordinate thousands of people known to each other only by email addresses. Linux has gotten around this latter problem in no small part because of Torvalds himself. He is a benevolent dictator who has earned the trust and respect of virtually everyone on the project. He's a relaxed, friendly, funny guy whose strength of character keeps his distended organization free from the internal battles that sink so many others. He also has learned how to delegate and has developed a core of equally well-respected close associates who coordinate different parts of the project.

One intriguing possibility for future open-source development comes from medicine, an area where people can become passionate and where intellectual components can far exceed physical components. Consider a smart doctor who loses a friend to rare disease X and decides to devote her life to finding a cure. Ten years ago, trying to develop anything more than a local network to collaboratively design a drug to cure the disease would have been extremely difficult. Communication would have had to be done over the phone or with photocopies slowly and expensively mailed. It made much more sense to have small groups of developers working together in laboratories, foundations, or universities.

Today the possibilities for open collaboration have improved. An ambitious doctor can network online with other researchers interested in disease X and, at the least, can quickly exchange data about the newest research techniques. In fact, there are already medical networks that pass around information about acute medical cases, using email and computers that can automatically send out patient files over a network and put X-rays into the overnight mail.

Now think another decade ahead when everyone will have high-speed Internet lines at least 500 times as fast as standard connections today (this will probably happen in three years), where it is fairly likely that we all will be able to simulate the movement of disease X online, and where it will surely be possible for medical students to run tests that approximate the human immune system on high-powered laboratory computers. Now the same doctor can farm out parts of the project to interested collaborators, many of whom have also lost friends to X and are passionate about finding a cure. If the coordinator is a good organizer and can hold people together the way that Torvalds has, the organization could grow, attracting even more people to join the collaborative effort with each success. Every breakthrough or improvement in the model could be posted online so that other participants could begin work on the next challenge. If a sample test is performed, data could be transferred to the Web simultaneously.

Eventually a prototype could be developed and adopted by an established drug company (or perhaps even a non-profit company, funded by foundations, that specializes in distributing open-source drugs and selling them at minimal costs) that licenses the product with the FDA, runs it through the necessary tests, and then manufactures, distributes and sells it--keeping prices relatively low both because no company would have exclusive copyrights and because research costs (drug companies' largest expense) would be drastically reduced.

Law

A real-life example of another possible opportunity for open source comes from Harvard where law Professors Larry Lessig and Charles Nesson have started the Open Law Project, an attempt to try cases using the open-source model. Interested people sign into the Website, read what other contributors have written, and help to develop arguments and briefs. According to the site description, "what we lose in secrecy, we expect to regain in depth of sources and breadth of argument." The program is run under the same sort of benevolent dictatorship model as Linux, with Lessig serving as chief. People brainstorm, debate, and then Lessig synthesizes and writes the briefs. Currently, the group is constructing arguments challenging, appropriately enough, the United States Copyright Extension Act.

There are great advantages to this model for law: The problems faced by lawyers are mostly intellectual, not physical; there is an abundance of people (especially law students) who are potentially willing to work on the projects for free; and there is the powerful draw of doing something you believe is a public service. If you don't agree with current copyright laws, join the group and figure out how to change them.

Of course, open-law will never be able to handle certain kinds of cases. As Nesson said to me, "open-law is not conducive to ambush." If you need to rely on secret arguments or evidence that the other side doesn't know about, you're not going to post everything on the Net. But if you just need to develop the best argument and rely on increased information flows, the model could work quite well.

Future

It is very difficult to determine exactly where open-source projects will take off next. So much depends on the personalities of the coordinators and the excitement they are able to generate; a great deal also depends on the different ways that technology develops and the way that different markets and research fields change. For some organizations, open-source development will make more sense in a couple of years than it does now. For others, it will make less. But the overall trends of technology are likely to push open source closer and closer to the mainstream.

Imagine a scale with all the advantages of a proprietary model on the left and all the advantages of an open-source model on the right. Pretend everybody who wants to solve a problem or build a project has a scale like this. If it tips to the left, the proprietary model is chosen; if it tips to the right, the open model is chosen. Now, as connectivity increases with the Internet, and computer power increases exponentially, more and more weight accumulates on the right. Every time computer power increases, another household gets wired, or a new simulator is built online, a little more weight is added to the right. Having the example of Linux to learn from adds some more weight to the right; the next successful open-source project will add even more.

Not enough is added to the right side to tip the scale for everybody and everything, but open source is presently growing and it should only continue that way. Netscape has made its Web browser open source. Sendmail, the program that routes most of our email, is open source. Most Web sites use an open-source program called Apache at their core. Even some microchip developers are starting to use open source.

Perhaps the next boom in open source will come from the law; perhaps from drug X; perhaps it will be something entirely different. Although it's difficult to tell, it is quite likely that the scale is going to tip for some projects and that there will be serious efforts at open-source development in the next decade. Moreover, it's quite likely some of these projects will work. Open source has created the fastest growing operating system in the world and it's done so by capitalizing on changes in technology that will almost certainly seem commonplace in a decade or two. Linux will continue to grow; but 10 years from now, it will probably no longer be the largest open-source project in the world.