I originally posted this on my blog, but envisioned it as a question for all of you here.
It's fairly clear that one cannot code from the ground up in bazaar style. One can test, debug and improve in bazaar style, but it would be very hard to originate a project in bazaar mode. Linus didn't try it. I didn't either.
– Eric S. Raymond
It seems likely to me that much of our future prosperity (or much of our future, at any rate) will be built on software, including free and open source (FOSS) software. Software already pervades all the other major branches of technology – it is used in agriculture, medicine, manufacturing, construction, energy, transportation, accounting, electronics and more. It constitutes the most common means of collecting, storing, retrieving and processing information at scale. It has enabled the Global Positioning System, commercial mobile phones, near-instantaneous cash transfers, cryptocurrencies, spreadsheets, 3D printing, remote work, advanced statistical modelling, forecasting platforms and the internet among much else. Few of today's great scientific discoveries could have been made without it.
There are inventions that result in new devices or processes and no more. Then there are those inventions that result in new devices or processes that are useful in coming up with new inventions. In this second category we can include human languages and writing, the wheel, the microscope, the steam engine, electricity and the ball bearing. Often these inventions are instances of what economists call general-purpose technology, that is technology that leads to productivity increases across all sectors of a region's economy.
Software is a strong candidate for a general-purpose technology. And as software is a general-purpose technology, so are there general-purpose software technologies. I mean things like programming languages, database systems, Reactive Extensions, data structure and processing libraries like NumPy and Pandas, graphics frameworks like OpenGL and distributed version control systems like git and Mercurial – in other words, the substructure and the infrastructure of software. These are very often released under FOSS licenses, for everyone to use, inspect and modify as they wish, and are consequently often used by vast numbers of people.
Austin Vernon is more bearish on productivity gains from software:
The current status quo means we don't get productivity growth until these software-driven companies become behemoths. Amazon was founded in 1994, almost thirty years ago. In 2020, it was still less than 10% of total retail sales. Is it any wonder that we haven't seen robust productivity gains? [...] In many industries, assembly lines builders are just getting started. If these companies maintain rapid growth as their market share increases, [total-factor productivity] should finally show some life. [...] Since software is a multiplier, any increase in input quality or quantity is even more beneficial. Software is more impactful in a universe with 1 trillion humans than one with 10 billion humans.
In Evolution of Programming Language Traits, I wrote about research showing that innovation rates are largely driven by a group of people's interconnectedness. I wrote:
[W]hat greatly affects a population's rate of innovation is its interconnectedness, in other words how widely ideas, information and tools are shared. In a culture that is deeply interconnected, where information is widely shared, innovations are observable and shared tools and standards mean that innovations are also more likely to be compatible. Most importantly, interconnectedness provides each individual with a large pool of ideas from which they can select the most attractive to modify, recombine, extend and spread in turn.
I then went on to quote Richard Stallman:
Software development used to be an evolutionary process, where a person would take an existing program and rewrite parts of it for one new feature, and then another person would rewrite parts to add another feature; in some cases, this continued over a period of twenty years. Meanwhile, parts of the program would be "cannibalized" to form the beginnings of other programs.
The existence of owners prevents this kind of evolution, making it necessary to start from scratch when developing a program. It also prevents new practitioners from studying existing programs to learn useful techniques or even how large programs can be structured.
In other words, the free and open source philosophies really allow us to share tools and information in a way that wasn't possible before. So supposing, as I do, that technology is on the whole good for the world, creating free and open source software, especially general-purpose software, seems good for the world.
Okay, but how much marginal impact can a single programmer expect to have? I think that is hard to estimate, but I will note, as the epigraph suggests, that many important FOSS projects have been created basically by a single person, alone. Linus Torvalds created the Linux kernel, various distributions of which has the majority of the marketshare in smartphones, supercomputers, web servers and embedded systems. Then, for good measure, he wrote git which, though it was not exactly the first of its kind, has fairly revolutionised software development and code sharing. Richard Stallman wrote the text-editor Emacs, which is being used today, nearly 40 years after it was created, and he also laid the foundations for the GNU Compiler Collection. Carsten Dominik created the widely used and hugely influential information organising tool org-mode. Satoshi Nakamoto, building on previous work by Wei Dai and Adam Back, conceptualised and implemented Bitcoin and the blockchain. Ryan Dahl created Node.js; David Heinemeier Hansson created Ruby on Rails; Salvatore Sanfilippo created Redis; Igor Sysoev created nginx. And so on.
Of course, I would say all this. As a programmer, a part of me wants FOSS to be important and impactful, and so I don't entirely trust myself when I say that it seems pretty important and impactful to me. I, too, would be sceptic if I read an accountant explaining how good pro-bono accounting is for the world. Maybe software is not as good for humanity as I think. Maybe the free and open source philosophies are not as good for software as I think. Maybe a single programmer, talented and dedicated though they may be, is just extremely unlikely to ever write a software program that is a significant improvement for the world on the margin. Software does pervade all the other major branches of technology, but so does pretty much every other major branch of technology!
How impactful is FOSS programming? 80,000 Hours, when writing about software engineering, don't seem to consider FOSS at all:
Overall, [software engineering] is good for earning to give. Software engineers can also make some direct contributions to innovation and economic productivity, especially if working for a good company. Advocacy potential is moderately good because you can be involved in the influential and innovative tech community.
I reckon that FOSS development is good for the world, likely more impactful than whatever direct proprietary contributions one can make working for a company, but I think this is really hard to quantify – certainly much harder than quantifying charitable donations – and reckon further that this is one reason why it is talked about less than those other things. But I would like to hear other opinions.
Raymond, E. (1999). The cathedral and the bazaar. Knowledge, Technology & Policy, 12(3), 23-49. ↩︎
Basu, S., & Fernald, J. (2007). Information and communications technology as a general-purpose technology: Evidence from US industry data. German Economic Review, 8(2), 146-173. ↩︎
Naughton, J. (2016). The evolution of the Internet: from military experiment to General Purpose Technology. Journal of Cyber Policy, 1(1), 5-28. ↩︎
By general purpose, I mean only that they can be used in a wide variety of different applications. ↩︎
Brien, M. & Shennan, S. (2010). Innovation in cultural systems : contributions from evolutionary anthropology. Cambridge, Mass: MIT Press. ↩︎
Of course many other people have worked to extend, maintain and test these creations since their conceptions. I do not mean to diminish that work, which is also important. ↩︎
It is not a proven fact that he is a single person, but it seems likely to me. ↩︎
Nakamoto, S. (2008). Bitcoin: A peer-to-peer electronic cash system. Decentralized Business Review, 21260. ↩︎
thanks, these are some really interesting thoughts. some comments:
i think this is true. but i think it's true for most career choices / endeavours that they only make sense for people who have the right skill set, drive, personality and so on.
i found some estimates for the eu (pdf) -- around 8% of around 3.1 million programmers are involved in open source... (read more)