The Wisdom of Crowds

The Wisdom of Crowds is one of the latest book I’ve read that got me thinking. The general idea of the book is now quite well-known: combining the decision of a great number of people can reach an optimal choice more reliably than relying on an expert. But there are also several examples that are worth mentioning.

The chapter 8 highlights the fact that science is all about collaboration: first, of course, between scientists (this is more and more the case with the capacity we have to communicate nowadays) but also at the idea level. An idea first get published, then validated by the scientific community and finally accepted. This process has been pretty reliable so far: there have been incomplete ideas that were widely accepted, yes, but any plainly wrong idea that was widely accepted? I can’t think of an example (obscurantism of the Middle Ages aside). One of the condition for this collaboration to flourish is the open access to information: the example of the Philosophical Transactions journal from the Royal Society shows how this idea of open communication of idea was central to the success of Western science.

It is not mentioned in the book but this could well be extended to softwares. Either the software is designed and developed by an expert (or a small group of them), this is the traditional software development process or either the software grows into a more organic fashion that is typically the case for open source softwares. Another advantage is the access to the information on how the software is working and the possibility to adapt it: even if only a small number of the users would ever do that, the leverage is huge. The last advantage is the survival of the fittest: many open source projects start with similar goals and have to compete for the scarce resource that is programmers’ attention. The competition among these projects enable to improve most of them and finally the disappearance of the less fitted to the task.

In the open source world, decisions are usually taken by a group of people rather than imposed. The book also describes the shortcoming of small groups that could, in some conditions, easily switch to some extreme positions and fail to reach an optimal solution. This is an important problem for open source projects that usually gather like-minded people which could prevent the appearance of alternative solution: getting different people on-board is critical.

Another examples that got me thinking is the illustration of the failure of some small groups in the context of space agencies. The author compares the team of the men that managed to save the Apollo XIII mission: “most of those men had worked outside of NASA in many different industries before coming to the agency“, with those at the time of Challenger accident: “NASA employee today are far more likely to have come to the agency directly out of graduate school, which means they are also far less likely to have divergent opinions“. Does that mean that space agencies like NASA, or private companies aiming at particularly difficult tasks should emphasize the diversity of background in their employees?

Leave a Reply

Your email address will not be published. Required fields are marked *