Information overload: moving away from the wisdom of crowds

Information overload: moving away from the wisdom of crowds
By R Lemuel Lasher, chief innovation officer for CSC

Published: February 12 2009 17:45 | Last updated: February 12 2009 17:45

We have gone from a world of information scarcity to a world of information surfeit. Valuable information gets lost in the noise of an increasingly crowded digital world in which everyone has more information than needed and, therefore, an urgent need for a point of view on what the information means.

I subscribe to only a small number of all the possible blogs and media websites configured for RSS feeds, and I receive more than 3,000 updates daily. How can I possibly manage to take advantage of all of this information wealth?

While I am an avid user of Amazon.com for all of my book purchases, I never rely on the book reviews for choosing a title. Anyone can post a reader rating for a book. I don’t know their background, qualifications, education, intelligence, or motivation, so how can I trust them?

I need a filter, and not a technological one. I mean a “conceptual” or “value” filter that acts as my trusted advisor. How do I know who to trust, and what is the basis of that trust?

The answer lies in an exploration of the philosophical principles that underpin much of what occurs online. Those principles can be characterised as democratic, communitarian, libertarian and millennial:

● The internet is democratic in the sense that it is assumed that the wisdom of the crowds will level archaic hierarchy and established institutional structures, by allowing a very personalised self-selecting inclusiveness to set new patterns of social, political and economic order.

● It is communitarian in the sense that the ethos of its “community”, non-economic contribution trumps the ethos of individual economic gain.

● It is libertarian in the sense that all progress is evaluated within the context of maximising “freedom” of choice, and “liberating” the individual from organisational and social constraints.

● It is millennial in the sense that a new world order is being ushered in through the revolutionary and broad-based product, service, social and political innovations that are enabled by web-based technologies. There is an abiding faith that this new world order will be an entirely different and better one than the world today.

These philosophical principles shape how we see events, and do reflect a dimension of what is going on; but seen without the proper context, they confuse the means with the end and ignore a dialectical view that will enable us to see more clearly what is going on and what the future may hold.

The basic construct of the dialectic includes three elements: a thesis, antithesis, and synthesis. The thesis is the established order, the antithesis is the force(s) for change, and the synthesis is the result.

A thorough review of history will demonstrate that this view, properly applied, is a much more reliable way of correctly observing current events, than seeing the antithesis, and confusing it with the synthesis. This is a mistake that has been made throughout history, particularly by individuals who are passionately participating as agents of the antithesis.

There is currently a bias towards an online democratisation as a source of wisdom, and trusted sources of insight are not clearly identifiable. We are in the nascent stages of this dialectical evolution, which will eventually produce a more traditionally structured digital world and evolve towards more traditional hierarchical models, with a representative democracy of elite and proven sources.

Instead of a pure manifestation of democratic vox populi in an uncontrolled blogosphere, might not the future be more like a representative democracy, with editors and peer reviewers filtering content for us?

The editors and peer reviewers will gain credentials through societally sanctioned processes and organisational filters, but they will all have one thing in common: they will reflect a more elitist or even aristocratic representation of wisdom instead of the chattering of the crowds. The current difficulties experienced by the established media have more to do with the betrayal of their brand and role as trusted advisors, than with the “end of journalism as we know it”.

The internet has provided a platform for the emergence of a highly democratised world, and far from having reached the end game, we are seeing the beginnings of a dialectical process. This is certainly something the US Founding Fathers would recognise, who feared and eschewed the wisdom of the crowds and much preferred the wisdom of the ages.

Copyright The Financial Times Limited 2009