Wilson's Slate Article Complicates (Not Dispels) Misconceptions About Wikipedia
2008-03-01

If Chris Wilson's recent article, "The Wisdom of the Chaperones" were published on Wikipedia instead of Slate, it would have been very quickly spotted as "POV" and removed.


The piece basically serves to call Wikipedia and Digg oligarchies, top-heavy organizations that claim to be democratic, but are really controlled by an elite few. The article is written entirely in an accusatory "get a load of this" tone, utilizing old rhetorical standbys like the "undemocratic" pejorative, and mounting up to a call for the sites to "stop pretending they're operated by the many and start thinking of ways to rein in the power of the few."


I want to address three things: two quotes from the article and then the data it used.


1) "Despite the fairy tales about the participatory culture of Web 2.0, direct democracy isn't feasible at the scale on which these sites operate"


Web 2.0 is great because it allows access for ANYBODY to participate. It's not a fairy tale. Anybody CAN participate on Wikipedia--you don't even need to make an account! Wilson even points out that "elite users aren't chosen by a corporate board of directors or by divine right. They're the people who participate the most."


While Wikipedia doesn't see itself as a democracy (it plainly states otherwise), it really seems pretty close. A democracy is a government run by the people with equal rights for all. A direct democracy is one without representatives or intermediaries (the people actually make the decisions). However, does a people lose the title "democracy" if not EVERY person votes? What if for every thousand people, there's one person who donates a ton of time to a cause they care deeply about? Does that process need to "stop pretending" to be democratic?


2) "A small number of people are writing the articles, it seems, while less-frequent users are given the tasks of error correction and typo fixing."


It's not news that there is a relatively small group of people who deeply care about Wikipedia and account for a disproportionate amount of work. This would seem to be the most popular criticism of Wikipedia as of late, edging out long-time favorites "anybody can write anything there" and "it's unreliable." I'm not sure why this comes as a surprise to so many people. The concept of a power law is pretty well known, and common sense could guide you to realize that not everybody who uses Wikipedia has the same amount of time to invest or has the same level of interest.


Wilson's talk of "territorialism...authorial domination by 1 percent of contributors" just shows he hasn't really spent much time on Wikipedia. On it there's a policy page called "Ownership of articles" which states pretty clearly "Believing that an article has an owner...is a common mistake people make on Wikipedia." As a collaborative project, nobody gets individual credit. Some editors are recognized for their contributions by other editors behind the scenes, but the meritocracy of Wikipedia is based on effort and diligence, not expertise or authority. The top 1% has simply donated more of their time--they don't actually have more power than you.


There are many reasons why people contribute to Wikipedia: passion for a particular subject, passion for information or knowledge, and/or a sense of furthering a common good. There aren't a ton of people who care SO MUCH about Wikipedia's mission of providing free access to knowledge that they are willing to sacrifice time. Thankfully, there are some. The 1% who contributes something like 50% of the content are those benevolent few who really want Wikipedia to succeed.


A study by Kittur, Suh, Pendleton, and Chi (2007) analyzed work on Wikipedia and broke it down into "direct" and "indirect" work. Direct work refers to article edits and new article creations and indirect work includes "meta" activities such as conflict resolution, discussion, community management, and undoing vandalism. They found that increasingly, people are spending a greater percentage of their time on the "indirect" activities. Between 2001 and 2006, for example, the "percentage of edits going toward policy, procedure, and other Wikipedia-specific pages has gone from roughly 2% to around 12%" The most experienced and most passionate users do the bulk of this work. Therefore, while the content itself comes from very diverse sources, there is a core group of Wikipedians who tweak, format, Wikify, organize, categorize, correct grammar, and so on.


But here he announces this finding, which contradicts what we previously held true about the 1%/50% figure:


People who've made more than 10,000 edits add nearly twice as many words to Wikipedia as they delete. By contrast, those who've made fewer than 100 edits are the only group that deletes more words than it adds. A small number of people are writing the articles, it seems, while less-frequent users are given the tasks of error correction and typo fixing.


This claim didn't quite make sense to me, although I guess maybe I could see a class of Wiki superstars researching and summarizing heaps and heaps of information while the new users are too timid to do anything more than correct grammar. There are a few factors that could contribute to this being true, but I couldn't see any of them having THAT much of an effect: new users are far more likely to "blank" a page (remove all of the words/content), experienced users are far more likely to "revert" a page (restore a previous version when someone has damaged it), etc.


3) Power of the Few vs. Wisdom of the Crowd


Let's look at the data supporting the new statistic: Palo Alto Research Center's Ed Chi's paper titled "Power of the Few vs. Wisdom of the Crowd: Wikipedia and the Rise of the Bourgeoisie


We examine "elite" vs. "common" user contributions over time, with the elite defined either by status (administrators) or by participation level (high-edit users). Two different metrics (number of edits and change in content) provide converging evidence on an answer.


While there are several odd things about this study (like why use administrators for the "elite" group instead of the users with the most edits and why factor into the "elite" group "those that might become administrators later"), the most important thing to note comes from the methodology section of the paper, most of which I quote here:


In the following analyses, we used a history dump of the English Wikipedia that was generated on 7/2/2006. The dump included over 58 million revisions, from more than 4.7 million wiki pages, of which 2.4 million are article-related entries in the encyclopedia...


To calculate the work done while editing an article, we calculated both the number of edits made and the change in content between edits. We model change as the number of words added and removed, as calculated by a traditional "diff" operation [9]. However, we used words as units instead of lines, allowing greater precision than previous studies...For both measures we aggregated edits over all 58+ million revisions, grouping by time and user participation level. User participation level was calculated based on the total number of edits made by a user.


They used all 58 million revisions in the dump! Not just the articles! They compared the contributions (words added/removed, edits, etc.) of experienced and novice Wikipedia users on TALK pages, POLICY articles and deliberations....even on those like the Request for Comment discussion page! You may have seen how long talk pages and discussions about Wikipedia decisions can get; how many edits and words go into them, with few words ever being removed? As Kittur, et al pointed out, it takes time (and a certain amount of passion for the topic at hand or for Wikipedia) to gain an understanding of the value of "indirect" work.


By including portions of Wikipedia that most newer users have never even heard of, the data becomes heavily skewed towards higher edits and word contributions on the part of the experienced Wikipedians.


I would bet that the same study, if repeated with the limitation of only examining article pages, would show exactly what we thought before--that the top 1% covers so much ground at least in part because of all the custodial work that it does.





» 1 comment «










2008-03-03
15:30:00


Good post.




  Your name (required):  


  Your URL (optional):  


  Enter your comment below: