Why should companies love open source
In this article I want to highlight an important relation between companies, software developers and the open source movement : the open source movement accelerates the process of production commodization (software development) in the software industry.
The process of commodization is not particular to software development, but it happens across all industrial areas. What is special in software industry is the way in which this process is accelerated (in a considerable way) by the open source movement.
Some of the ideas in this article are the result of discussions with Vali Dumitru & Hartger Visser in the last 2 years and others come from different books on economic anthropology (among them “The Great Transformation” by Karl Polanyi being major influence).
Lets go back in time to gain some perspective
At first I will briefly present the way in which the process of commodization started to work in a visible/powerfull way with the start of the industrial revolution almost two centuries ago.
As Karl Polanyi (in The Great Transformation) observes (among others) in a masterly way :
Before the industrial revolution
Owned land and social position (peasant, burger, noble, etc,) had an overriding importance over the other factors (ie. money, technology, skills, etc.). Owned land is important because was the only source of anything “productive”, it is the ultimate source of raw materials (ie. wood, stone, food, etc.) Social position is important because it offered access to labour force (ie. workers, peasants) and it was the main way to have/keep access (usually through inheritance) to land. Owned land (nature) & labour force produced/maintaned wealth (money, artifacts, social power, etc.).
As such, things get mostly reduced to the fundamentals : owned land and work force (people).
The nobles owned most of the land. But they needed peasants and craftsmen to be able to do anything with their land : plant, harvest, grow animals, etc.
Both the peasants/craftsmen and the land were irreplaceable from the whole “production cycle” (business cycle). As such, both land and labour (work force) were excluded from the process of commodization.
After the industrial revolution
The labour force can be replaced (and gets replaced) by technology (productive artifacts – things that work for you). Owned land is not that important because most of the lucrative and profitable industries don’t need land anymore (ie. you can make a weaving factory in any decent size “garden”).
As such capital begins to have an overriding importance because it :
(1) allows to buy machines (automatic labour) and hire machine attendants (manual labour – humans that work for you)
(2) allows to buy the initial stock of raw materials and boot up the production cycle
Things get mostly reduced to capital (which can be used to purchase machines and raw materials). Peoples (as workers) and land are reduced to capital.
The machine (the productive artifacts) becomes the measure of things. People become
“productive units”/”resources”, etc. and are evaluated in comparisons with the machines : we keep people in production because the machines are too expensive or non-existent (The Luddite movement is representative for the social effects of this trend)
Through productive artifacts (production related technology/machinery), peoples skills & training are replaced with equivalent machinery which gets better and better. If a machine can do it, why use people? As such people as employees become subject to the process of commodization. What started two centuries ago still goes on right now (as you read 
)
The special case of software industry
I believe that the software industry in the last 40 years repeats the same patterns of the industrial revolution (two centuries ago), but with an interesting
extra switch.
The power of industrialisation in the software industry is driven by three distinct “forces” :
(1) closed source software : database servers (Oracle, MSSql, etc.), text editors, etc.
(2) software as a service : GMail, Amazon EC, etc.
(3) open source software : Lucene, Solr, Hadoop, etc.
Closed source & software as a service follow the classical pattern
The first of them is part of the scene from the 70′s (40 years ago). Basically it consists of the software that we buy and install on our own computers.
We pay for it (sometimes quite a lot) and as such we need capital to use such a machine (productive artifact).
The second one is part of the scene in the last 10-15 years (extremely visible in the last 5) and consist of the services that we rent for a monthly fee.
We don’t own the machine, we don’t own a copy of the software, we just pay for the right to use some services and store the data on their computers.
Those two “forces” (here the term “force” is not used in the technical sense from marxist theory, but more as a figure of speech) of software industrialization (and comodisation) work in a rather classic way, in the simple sense that you need capital to use/acquire them.
Open source software introduces the new switch
Open source software (especially the one that is licensed under a friendly license like Apache) works in a peculiar way and I want to argue that here is the interesting
extra switch of it. The difference between it and the first two is that you don’t have to own capital to use it (and keep using it). Usually there is absolutely no license fee (that’s one of the reason why the “free” term stuck pretty well in the public mind) and the whole production process is driven by :
- company sponsorship
- developers drive for status in the community.
Anyone who checks the site of the Apache Foundation will see that the biggest sponsors are Google, Yahoo and Microsoft. And that they have around 2000 comitters in around 70 projects. The new switch consists in the “cooperation like behaviour” of companies so that code is written only once and then it is shared without any additional cost involved.
Open source is always economically good for companies & not always so good for developers
From the industrial/productive point of view, the open source software works :
- in the advantage of the companies by outsourcing important parts of their infrastructure to “the community” at a fraction of the cost (for production and maintenance)
- in the disadvantage of most of the developers, by accelerating the transfer of their skills/experience into the realm of commodities
In the first case it is clear how the cost is driven down. Instead of using “in house” or payed (closed software or service based) solutions, companies are choosing open source (developed/maintained) libraries. Anyone who might have the curiosity to check a JAVA job requirement will have the surprise to conclude that more than half of the required skills/libraries that the future employee needs to know are all open source (released with commercial friendly licenses) or easily replaceable by them. The companies prefers to contribute with an insignificant donation (often with nothing at all) to producing and maintaining millions of lines of code. As such software companies need less people and those people can be less skilled/experienced (although not too low on skill).
In terms of the political sciences, the companies (and of course others like : the academia, the research community, etc.) are the most prominent “free rider” of the open source software movement. This is one of the reasons why I argue that there is a special switch involved here. I don’t know any factory that got 60% of their machinery for free (and maintenance included forever).
In the second case, I want to argue that open source drives down the “entry level” of developers.
Why would you need a developer that knows the tweaks of the JDBC drivers and queries on several databases (ie. Oracle, MsSql and MySQL, etc.)? You needed that until Hibernate (and other open source ORM’s appeared). Now you just have to include 5 lines of text in a config file and forget about it.
As such, it is relevant to observe that :
(1) The requirements of many “commercial products” stayed mostly the same :
With the exception of the scale the needs for many of the software industry consumers remained mostly the same. They want to put some data in some databases and take it out quickly and make some reports out of it. 10 years ago they put 10 million rows there. Now they put 100 million. So what? Basically the target remained at the same level as 10 years ago. They just need more hardware and more software infrastructure to do that.
By this I don’t want to argue that “nothing” has changed in the last 10 years. There are several companies that were game changers : smart phones (Apple, Google, etc.), media tablets, etc. But many companies deal with customers with unchanged needs (mostly sophisticated/extensive variations on CRUD)
(2) Open source gives companies an important heads-up :
Before open source, the companies had to use their own energies/capital (and managerial skills) to jump the whole distance by buying closed solutions and skilled developers. Now they start with an important heads-up : they get for free at least half of the tools they use to make the “final product”/”commercial product”.
(3) Open source covers a good deal of the hard part of the “commercial product”
It seems consistent with the realities of most software development companies to say that open source libraries cover most of the exotic/hard parts of the “commercial product” (like persistence, transactional features,messaging,scheduled tasks, etc.). For example :
Excerpt from the Spring Framework description :
The Spring Framework is a lightweight solution and a potential one-stop-shop for building your enterprise-ready applications. [...] The Spring Framework supports declarative transaction management, remote access to your logic through RMI or web services, and various options for persisting your data.
So the skilled (and rather expensive) developers are not in such demand because the jobs got too small for them (I exclude the extremely small number of developers that maintain/extend the open source libraries or work in real R&D departments). I’m not arguing that companies don’t need peoples with skills but something weaker :
(1) fewer extensive skilled developers
Usually those developers are used to integrate existing technologies in a masterful way or extend one of the existing libraries. But most of the time this kind of in-house development is avoided as much as possible (because it ties the company to development and maintenance costs)
OR/AND
(2) developers with a narrowed down skill set
As such a considerable number of skills that involved fundamental (and hard earned) knowledge of different areas of computer science (ie. sorting algorithms, database design, distributed systems, etc.) are not needed in their full depth because the existing tools (most of them open source libraries) offer them ready-made. All what is needed is a superficial understanding of the big principles involved and of the API set of the library.
To drive the message home in more concrete terms it would be instructive to read how Terracotta or Hadoop works out of the box.
Some long term consequences
(1) the technological innovation/comodisation cycle works faster than in the “traditional” industries :
- software developers get burned down much faster (than in other industrial areas) => they become obsolete at a younger age
- software developers become more replaceable (as the jobs they do become smaller and smaller and many important tasks are outsourced to existing open source libraries)
(2) capital & connections becomes much more important that technical skill
- having “the technical edge” is becoming less and less the “important thing” in starting on your own (probably there is some combination of Open Source libraries that already does 70% of what you do)
- production is commoditiesed and as such is less the differentiating factor
(3) the importance of sales & UI design/interaction go up big time
- as production is becoming a commodity, other parts of the process of making a “commercial product” gain an excessive importance
- sales/marketing is very resistant to commoditization because it involves personal soft skills/knowledge
- UI design and “packaging design” cannot be commoditized yet
