This last Friday, I had the pleasure of attending a lecture put on by one of the professors at the NRI (Natural Resources Institute) at my university (U of Manitoba) as part of our homecoming lecture series. The talk was about environmental/resource conservation projects in so-called third world countries where they were trying to provide a way to define a successful project.
The method they came up with was by measuring the number of entities involved in any given conservation project, be it local governments, international NGOs, universities, etc. They threw up some slides showing the networks that existed for a number of the more successful projects versus the networks for the less successful projects. The conclusion was that the conservation projects that were successful were not only well networked vertically, but more surprisingly, they were very well networked horizontally.
Now if I've lost anyone yet, vertical networking was in this case defined as links between the various levels of organization, such as local->regional->national->international government support. This example would have 4 levels of vertical networks, which is pretty good to get a project started, but not enough to sustain it for the long term as it has many points of failure. If any one member of a vertical network stops supporting the project for any reason, the whole project would collapse.
On the other hand, horizontal networking refers to links between entities that are on the same level. This would refer to multiple local governments all having a shared interest in a single project, or having the support of several regional institutions (like universities) rather than just one. Having horizontal links in the network would substantially improve the resiliency of the network to any single entity dropping out.
Now here's where I had an 'aha!' moment. I started to contemplate how open source communities function, and the open source world in general vs. traditional software development.
In a business, if there were horizontal links at every level, each working on essentially the same thing, the business would be considered inefficient. Any good manager would start to work right away to remove the redundancy in order to save money. And in fact, we have a lot of people who think like this when observing the open source world, telling us that there is no good reason for multiple projects to exist that tackle the same problems.
The answer to this is that open source is infinitely more resilient due to this redundancy. Now there are a lot of examples of this within the open source world that occur in almost every single project type.
Just consider, for a moment, KDE's multimedia support. Previously we had a pretty hard link to the aRts project, but it was definitely not the only multimedia project that existed during KDE's history. There has been a wide array of suitable libraries that could have done the job, but KDE was pretty concretely linked to aRts. This would be a sort of vertical link between KDE and aRts. The problem was that aRts was no performing the job that KDE needed, so we started to form more vertical links to other projects, like Xine or GStreamer to circumvent aRts. This would not have been possible if these projects were not simultaneously working on a more-or-less similar goal. In a company, if that portion of the software project had failed, we would not have had such a readily available backup plan (management would have axed it years ago!).
This same thing happens on a number of levels all throughout the open source community. People wonder why PostgreSQL and MySQL are always locking horns. Why we need ulibc when glibc already exists. How come we keep developing zsh when bash is almost ubiquitous. Why does KDE really benefit from GNOME's existence?
Well, just like our multimedia example from earlier, imagine that one day GNOME fails for some reason (why? I dunno, I'm talking hypothetically). Well, if GNOME was the only desktop project that existed, open source would take a huge blow. Thankfully, it isn't the only project. KDE would step in to fill the whole right away with little long-term implications to the health of open source in general. And while in this scenario, it would seem that we'd be down to one desktop, it wouldn't actually be the case at all. Other desktop projects are always nipping at the heels of KDE and GNOME, projects like XFCE or ROX would rise up to meet the challenge.
The open source world will always be fractured, with multiple projects for any single task, but in this lies the strength and longevity of the world-wide community. These horizontal links provide redundancy which allows projects that live above or below you to come and go without the whole thing going to hell.
One side-effect of looking at it this way is that the project that communicates the best with it's vertical links (KDE's dependencies upstream, and distros downstream) will likely be the most popular selection amongst the choices at any given level. The CMake team communicated really well with KDE, incorporating suggestions from KDE into later versions and so forth, which made CMake an excellent choice for KDE. The same was not true of some other build systems, which may have been able to handle a large project like KDE, but were less communicative.
Turn this scenario around and we have distros that are choosing a desktop environment. They will choose the desktop environment that is more willing to listen to their needs. We (and even I am guilty of this) have been of the impression that this communication is the distro's responsibility, but this philosophy will only drive distros away from KDE. We need to behave like the CMake developers and go out of our way to ensure that the distros are happy, and KDE will thrive.
We ought to create a working group entitle Downstream Relations or similar, where people actively seek communications with the distributions. Now, I know that there are hundreds (thousands?) of distros out there, and this is a huge task, but KDE penetration can really drive forward in the long run with this ideal. I will do my part.