The balance between crowdsourced and "expert" opinion
Introduction
Some companies are turning towards crowdsourcing as a means of processing large volumes of data at relatively low cost. Crowdsourcing is “the act of taking tasks traditionally performed by an employee or contractor, and outsourcing it to an undefined, generally large group of people or community in the form of an open call.” 1
Although this can prove effective in some cases, there are risks around the quality of the data processing and the extent to which the work being done can be controlled/planned. This page explores the current (2009) state of affairs with regards to crowdsourced data processing and the counter-movement by experts.
Arguments pro and contra crowdsourcing
When deciding between sourcing expertise from an 'anonymous' group and getting an expert opinion, a trade-off needs to made between speed and accuracy of the decision.
In his book The Wisdom of Crowds5 James Surowiecki argues that there are instances when an opinion reached collectively by a crowd is better than an expert opnion. These instances are typically those in which "statistics beat an expert" or differently put: when the average of many peoples' opinions are more likely to be right than an opinion derived through in-depth analysis. Some examples are guessing the number of jellybeans in a jar or estimating the floorspace of a room.
The way Wikipedia deals with content production and moderating gives an important insight into the extent to which an anonymous group of users can 'self regulate'themselves. It took Wikipedia years to achive a situation in which the editors of the site achieved a state of self cleansing among themselves [ref] Wales had to settle many disputes himself in the early days of Wikipedia, before the self-policing social structure had matured. "You have to create a healthy civil society where there's a balance between having a lot of rules and having total freedom," he says. "And that's about very high-touch customer service."
Conclusion
To conclude the present (2009) consensus appears to be that...
Crowdsourcing makes sense for:
- processing large volumes of non-sensitive data
- processing and interprering data that is hard to process automatically, but simple to process by a human being (e.g. facial recoginition, image interpretation)
Relying on experts makes sense for:
- data processing that relies on industry/niche specific knowledge
- processing large volumes of data iteratively, where patterns/trends in data must be observed through multiple passes over the data using statistical or other methods.