Britannica's mad at Nature

March 23, 2006 2:26 AM

Britannica's issued a salvo against Nature's famous "Wikipedia and the EB are comparably error-strewn" analysis. Here (pdf).
posted by Tlogmer (42 comments total)

Well, shit. This is the pdf URL.

I've actually HTML-ized it and mirrored it, if anyone's interested (which was going to be my original comment post).
posted by Tlogmer at 2:28 AM on March 23, 2006

Appeal to Authority fest : nobody learns anything except that experts disagree a lot
posted by elpapacito at 3:03 AM on March 23, 2006

Though usually there can be found a mutually agreeable wording...
posted by Tlogmer at 3:07 AM on March 23, 2006

Appeal to Authority fest : nobody learns anything except that experts disagree a lot

Except the Wikipedia errors are mostly dumb factual errors (note no one has objected to any of them that I can see).
posted by cillit bang at 3:27 AM on March 23, 2006

Fundamentally, Wikipedia is a form of populism... the claim that an army of less-skilled people can do as good a job compiling facts as a few, highly-skilled ones. It could be viewed as a symptom of the anti-intellectualism in America these days.

Whether or not it's true is yet to be proven, but I've frequently found that Wikipedia articles in areas where I am knowledgeable are inaccurate or misleading. Many articles are filled with extremely expert commentary mixed with outright idiocy, and it can be hard to separate out which is which without fundamental knowledge of the subject to begin with.

The fundamental problem is that people think they are more expert than they are.

Even Britannica, as hard as they work on accuracy, and as polished as their articles tend to be, has never been considered more than a starting point for real research.

As an aside, Philip Jose Farmer defined a dullard as a person who can go to the encyclopedia, look up one item, read it, and close the book.
posted by Malor at 3:43 AM on March 23, 2006

Wikipedia: at least as reliable as peer-reviewed journal Nature
posted by PeterMcDermott at 3:45 AM on March 23, 2006

It could be viewed as a symptom of the anti-intellectualism in America these days.

Wikipedia is an intellectual endevour (unless you spend all your time editing pokemon articles). And per capita, Germany has more wikipedia editors and users than anywhere else.
posted by Tlogmer at 4:04 AM on March 23, 2006

I've frequently found that Wikipedia articles in areas where I am knowledgeable are inaccurate or misleading.

isnt this what wikipedia is about? if you find errors, correct them and leave behind your references.
posted by a. at 4:06 AM on March 23, 2006

Whether or not it's true is yet to be proven, but I've frequently found that Wikipedia articles in areas where I am knowledgeable are inaccurate or misleading.

I'm just guessing and may be wrong, but did you try to correct the article(s) in the area where you are knowledgeable and get shouted down by a bunch of 14-year olds who had read the same Readers Digest article on the topic?
posted by spazzm at 4:11 AM on March 23, 2006

"That's not a bug, that's a feature."
posted by Smart Dalek at 4:15 AM on March 23, 2006

I frequently find that people who don't mention the specific instances where they found wikipedia to be inaccurate are just talking out of their asses. I don't doubt that the wikipedia is frequenlty wrong but it is at least a self-correcting medium. Vague unsubstatiated claims of innacuracy that can't be proved or disproved are just flatulence emitted only for the pleasure of the dealer.
posted by srboisvert at 4:16 AM on March 23, 2006

No, actually, corrections I've made seem to have been well-received. And I don't mind contributing at all... but I tend to think that sins of commission (putting in outright wrong facts) are very bad, where sins of omission (just leaving out stuff you don't know) is no biggie. In an encyclopedia, the latter is expected, but the former is hard to excuse, even on an 'amateur' site.

People tend to believe the first thing they read on a subject, and it can take real argument to convince them that their 'facts' are wrong. I wish more Wikipedians realized that.
posted by Malor at 4:32 AM on March 23, 2006

Wikipedia is a tremendous, amazing resource. That said, it should not be anyone's single point of reference on a topic. But it's really the best place to start online, and once you get started editing, it's addictive.
posted by cell divide at 4:38 AM on March 23, 2006

srboisvert, for a very specific example, look at the Amiga page. I did a major edit there a few days ago in the Hardware section, correcting a number of outright errors, and expanding the text quite a bit. (look for the 'big big changes' edit by Malor.) There have been some further (good) edits of what I wrote... stuff that was anticipating further supporting entries that I haven't yet written.

That page needs a TON more work, especially the stuff about the operating system. AmigaOS physically came on disks called 'Kickstart' and 'Workbench'... but the Wikipedia article states that Workbench was 'user-mode programs on disk', and that's just flat wrong. (Workbench was the graphical presentation layer, and was embedded in the Kickstart as well.) And calling Kickstart a BIOS is a huge oversimplification; it's not a bad starting point, but it needs more data. (it was a LOT more than just a BIOS.) Much of that section of the article was written by a person or people who didn't truly understand how things were architected on the system.

i've also made numerous changes anonymously, before I bothered setting up an account. As an example, I keep the 'Malaria' entry on my watch list, because stupid stuff keeps creeping in there. (I'm not a special expert on malaria, btw.)

I also got burned by using Wikipedia as a partial source for a FPP here on MeFi a couple of weeks ago. I summarized a paragraph about relativity that, according to Stephen Den Beste, is just flat wrong. Perhaps my one-sentence summary of the original paragraph is wrong, but I don't think so.
posted by Malor at 4:49 AM on March 23, 2006

srboisvert: I wrote most of the current DVI article in Jan 2005. I revisited it a couple of months ago and had to make this correction. The idea of an aperture grille "choosing" anything is utterly idiotic. If you look at the edit where this fact was added, you'll see the first point is factually accurate but completely distracts from the point of the paragraph. The second is utter bullshit, as mentioned. The third just turns a specific sentence into a vague one. It stood for three weeks until I tidied it up.

Here's another example.
posted by cillit bang at 4:54 AM on March 23, 2006

Abstract: Does an error exist if it's not found? Here, we will demonstrate quantum superposition effects related to Wikipedian information. Using our knowledge of probability, statistical mechanics and weight theory, we found the wavefunctions for a random sample of wikipedian "facts." Non-linear fits of empirical scientific knowledge vs. the enormity of amateur participation revealed an inverse-square relationship. We conclude that Wikipedia is truthful if it is not read, and upon reading one should understand that 50% of the content is crud.
posted by gsb at 4:57 AM on March 23, 2006

Actually come to think of srboisvert, most of my edit history (and everyone else's) stands up as examples of errors found in Wikipedia.
posted by cillit bang at 5:10 AM on March 23, 2006

No matter what, the main difference between Wikipedia and a paper encyclopedia is that in the latter case there's no army of brainless trolls trying to undo in real-time what other people have done. All the energy that is wasted in pest control (repairing vandalism, user ban and administration thereof etc.) could be used at actually improving pages. A good WP page can only exists as long as its "watchers" outnumber (or at least have as much energy to spend as) the vandals. There's something sisyphean in the wiki system that doesn't exist in other knowledge-gathering systems.
posted by elgilito at 5:32 AM on March 23, 2006

It's funny. Most of my reading is academic refereed journal articles in the humanities and social sciences, which are written with a high degree of intellectual and topical sophistication. When I'm reading these articles, I'm hyper aware of potential flaws, faults, fallacies, or anthing else that might undermine the authority of the writer, his or her arguments, or conclusions. When I read Wikipedia, I just accept what's on the page for what it's worth, because if I'm looking it up there, I don't know a thing about it. The fact that it could be error-ridden doesn't bother me, because it is just a starting point, a place to learn some contextual information about a topic. Even the most rigorously edited academic writing has flaws. The difference is that no one who reads academic articles expects *Truth* out of them -- or even really facts, just premises, arguments, evidence, and conclusion. They are presumed to be situated in a particular context, body of knowledge, time, etc. As that context changes, so does the interpretation of the content. Things that are facts in one decade are quaint artifacts of naitivity in another. I think a lot of people who read encyclopedias and wikis though (not this crowd, however) expect facts and Truth from it. It is not their "starting point" of reference, it is their sole reference. Unfortunately, editing the errors in Wikipedia or the EB alike will not make people more sophisticated researchers.
posted by mrmojoflying at 5:35 AM on March 23, 2006

most of my edit history (and everyone else's) stands up as examples of errors found in Wikipedia.

Absolutely, but doesn't it also stand up as testament to the power of Wikipedia? I don't mean that as a clever paradox, because it is neither (I'm not very clever and if the errors become overwhleming, the fixers will go away), but the fact you cared enough to fix them outweighs the existence of errors for me.
posted by yerfatma at 5:51 AM on March 23, 2006

The fact that it could be error-ridden doesn't bother me, because it is just a starting point, a place to learn some contextual information about a topic. Even the most rigorously edited academic writing has flaws.

However, if it is so error-ridden it is difficult (without checking edit histories and wading through discussion pages) to be sure that at the exact moment in time you're reading the article it is even a valid starting point? The variance in the quality of a single article over time is potentially so great that it's difficult to know if there is any value to the article.
posted by patricio at 5:58 AM on March 23, 2006

Articles rarely change fundmentally over the course of just a few revisions. If the article's written in that dry, verbose, smooth-like-a-mesesoic-rock style, you can usually be sure it's seen a lot of eyes and nobody's messed with it. If you're worried, use "show changes" with a revision from a month ago.
posted by Tlogmer at 6:06 AM on March 23, 2006

It does sound like the Nature people had it in for the Britannica.

One Nature reviewer was sent only the 350-word introduction to Encyclopaedia Britannica's 6,000-word article on lipids. For Nature to have represented Britannica's extensive coverage of the subject with this short squib was absurd, and it invalidated the findings of omissions alleged by the reviewer, since those matters were covered in sections of the article he or she never saw.

Other reviewers were sent only sections taken from longer articles. For example, what the Nature editors referred to as Britannica's "articles" on "kin selection" and "punctuated equilibrium" are actually separate sections of our article on the theory of evolution, written by one of the foremost experts on evolution in the world. What they claimed to be an "article" on field-effect transistors was actually only one section of our article on integrated circuits. For Nature to have excerpted our articles in this way was irresponsible.

"Here's a few paragraphs [from a much longer article we're not telling you about]: any omissions?" Yeah, great research.
posted by languagehat at 6:13 AM on March 23, 2006

srboisvert, for a very specific example, look at the Amiga page.

And do you suppose Encyclopaedia Britannica has an article on AmigaOS as well?
posted by ruthsarian at 6:51 AM on March 23, 2006

ruthsarian, if they don't, they should. It was a major milestone in operating systems.
posted by Malor at 6:59 AM on March 23, 2006

Malor, it's hard for me to read that comment without adding 'sth' instead of 's'.
posted by Baby_Balrog at 7:05 AM on March 23, 2006

Wikipedia is great for what it is.

It's a collection of 'common knowledge' about an immense range of topics, with a relatively smaller number of thoroughly well-researched and accurate articles.

What you're getting when you read most articles is an overview of what the interested amateur knows (or think's he knows) about the topic you're reading. The best articles, IMO are the ones that are extensively sourced, where the body of the article mostly summarises the information found in the sources. That's not to say they're the most accurate, but it's enough information so that you have an idea of its reliability.

I still think the ONLY thing Wikipedia needs is an academic oversight process, where certain universities can assign students to polish, clean-up and bless nominated articles within their expertise. It would be a tough thing to negotiate with the territorial wiki-editors, but as long as it's mostly a tidying up of existing articles and mostly related to well-established areas of expertise, it would be fine. The end result would be a template that says: "This version of the article was validated by So and So, a Physical Sciences major at the University of Such and Such, as part of the Wikipedia Academic review program." Other people could continue to edit it, but any further edits would say, "There is an earlier, expert-reviewed version of this article, click here to read it"

I think that would clear up a lot of the ambiguity about Wikipedia's reliability, although, to an extent, through a false appeal to authority. The other stuff would still be there of course, but maybe people would be less likely to simply take it as gospel, which is probably a good thing.
posted by empath at 7:07 AM on March 23, 2006

*reads note at bottom of posting box*

...argh...i take that back -it was a mean thing to say.
posted by Baby_Balrog at 7:08 AM on March 23, 2006

Wikipedia could be thought of as a vast repository for human knowledge. A lot of which could be wrong...
posted by delmoi at 7:12 AM on March 23, 2006

Delmoi: I think wikipedia brings up some pretty profound questions about what 'knowledge' is that most people haven't had to deal with before.
posted by empath at 7:20 AM on March 23, 2006

a place to learn some contextual information about a topic

Exactly, what Encyclopedia's are for. They are not "the sum of all knowledge". Context is summarization.
posted by stbalbach at 7:56 AM on March 23, 2006

cillit bang: The idea of an aperture grille "choosing" anything is utterly idiotic.

I don't know about that. The original sentence is malformed, and it is hard to tell if the author fully understood the point being made. In fact, looking at the brackets, it seems possible that a couple of authors collaborated in the mangling of a good idea. Your edit completely misses the notion that the apertures in an aperture grill display does have a sampling-like effect on the intensity of the electron beam. It may not have been a good place in the article to be making that point, but it isn't the kind of nonsense thinking you make it out to be here.

Malor, your problem with frames of reference is all in the inclusion of the word inertial. I checked the wikipedia entry on inertial frame of reference thinking I would help clarify, but it kind of sucks.. :)

Anyway.. Calling wikipedia anti-intellectual is a complete misunderstanding. Wikipedia is anti-authority (the kind where individuals claim qualifications can stand in for evidence). I think that is a good thing, but there isn't any way to say that with authority (the kind where strong evidence trumps opinions like mine).
posted by Chuckles at 8:02 AM on March 23, 2006

When you can look up both "singularity" and "battle cat's real name" in the Encyclopedia Britanica, then they will have my sword.
posted by drezdn at 8:03 AM on March 23, 2006

In some cases reviewers were sent patchworks of text taken from two or more articles and pieced together in a way that made a mockery of the original entries. The "article" on "aldol reaction" that the journal sent its reviewer consisted of passages taken selectively from two different Encyclopaedia Britannica articles and joined together with text evidently written by Nature's editors. This was dishonest, and it completely misrepresented Britannica's published coverage of the subject.

Wow. If that's accurate, the folks who published the study have a lot of explaining to do. They'll respond soon, right?

I've actually HTML-ized it and mirrored it, if anyone's interested

Thank you very much for doing that. Very sweet.
posted by mediareport at 8:08 AM on March 23, 2006

empath: I think that would clear up a lot of the ambiguity about Wikipedia's reliability, although, to an extent, through a false appeal to authority. The other stuff would still be there of course, but maybe people would be less likely to simply take it as gospel, which is probably a good thing.

Sounds like the basis for a great idea empath. Wikipedia could establish a basis for what it considers 'reviewed' to mean, and then flag versions of articles as reviewed where appropriate. Then at the top, like they do now with articles under dispute, it could say "To see a reviewed version of this article click here".

By displaying the dynamic content first, that part of the nature of wikipedia is maintained. By independently establishing a review criteria the problem of a simplistic appeal to authority is reduced. On top of those, you could include style in the review standard along with content.
posted by Chuckles at 8:14 AM on March 23, 2006

That's actually a really good idea, empath. I wouldn't want to see, for instance, a high school teacher assigning students to write articles on Wikipedia as a graded project. But having talented students (at any level) spending time improving Wikipedia in an organized fashion would be a valuable undertaking. The difficulty would be in organizing such a thing, but there are a lot of ways it could help. Students could, for example, flesh out stub articles, provide links to good references, and improve cross referencing within Wikipedia. Those with advanced knowledge in a field could periodically review, correct, expand, and update related articles.

Some sort of volunteer peer review process wouldn't be a bad idea. It would in particular be nice to be able to look at a revision history page and see that a particular revision has some kind of endorsement from a [relatively] trusted authority. Different types or dimensions of endorsement suggest themselves. An article might or might not be consistent with current scientific understanding; it might be more or less complete; it might contain or lack useful references; etc.. There are all sorts of problems with this and in some part it goes against the spirit of Wikipedia, but it could have some beneficial effects.

Personally, I find Wikipedia incredibly useful. Of course it can be incomplete and inaccurate, and of course controversial topics become battlegrounds. But there are few comparable resources online in many areas of knowledge. Sure, you can search for a topic on Google and find some great stuff, some of it no doubt far superior to the Wikipedia entries. But you can also find lots of incomplete and inaccurate material, explanations based on incorrect understanding, and sites with agendas of their own. It's not necessarily any easier to tell when these flaws exist on non-Wikipedia sites, and when you do find problems elsewhere you have little or no power to help correct them.

The essential problem with Wikipedia isn't that anyone can edit it. It's that some people don't know how to use it as a reference tool.
posted by Songdog at 8:24 AM on March 23, 2006

Encyclopædia Britannica is wonderful too, by the way, and shame on Nature if they made an irresponsible comparison.
posted by Songdog at 8:27 AM on March 23, 2006

tlogmer, thanks so much for the online html version, so much handier.

It is really ironic that Nature screwed up so bad in trying to prove how the EB screwed up just as bad as WP. Wonder what they're going to say about this.

Also makes me wonder why they did that at all...

> Fundamentally, Wikipedia is a form of populism... the claim that an army of less-skilled people can do as good a job compiling facts as a few, highly-skilled ones.

Yeah, but that also has its pros. It's a double edged sword. I love Wikipedia (and I contribute to it) as a quick reference, especially on current affair items and pop culture and so on (the amount of stuff in that field is ridiculous, and that disproportion can also be a problem), the internationality of it and the constant updating makes it a great resource in that respect. So on the one hand the fact it's so open is a good thing, and more people editing means there's more chances flaws and errors and partisanship get corrected, but it also means there's always people using it as a soapbox and then it takes pages of discussion to remove or edit things that shouldn't even have been there in the first place (and I'm not even talking of vandals and trolls).

To me that's the main problem with Wikipedia, not so much factual errors or omissions as people completely disregarding the NPOV rule and sneaking in opinion and ideology in more or less overt ways, it is very time consuming and frustrating to have to fix that (much more than just adding more material or correcting mistakes, which is just part of the collaborative process). And the problem can be much worse in other languages with less users editing and less familiarity with the medium and its rules and style.

So, I dislike fanatical defenses of Wikipedia as a People's Republic of Knowledge just as I dislike complete dismissals. It's useful and it's a great idea but it has to be seen for what it is. I think even attempting to compare Wikipedia to something like the Britannica is inherently wrong, even aside from how dishonestly that comparison might have been made.
posted by funambulist at 9:12 AM on March 23, 2006

I (heart) wikipedia, but the Architecture page sucks balls, in style, innuendo and overall tone, if not actual inaccuracies.
posted by signal at 1:49 PM on March 23, 2006

My Britannica doesn't even have an article on Saturnalia. You just can't beat wikipedia on size... and one should remember that all text is to be doubted somewhat.
posted by Citizen Premier at 5:18 PM on March 23, 2006

Nature responds.
posted by Tlogmer at 5:35 AM on March 28, 2006

"We're right and they're wrong."
posted by cillit bang at 5:40 AM on March 28, 2006