a quick note for later

http://www.edge.org/discourse/whosays.html#go

“Here is my modest epistemological prediction: The more knowledge grows on Wikipedia or other similar tools on the Web, the more crucial the mastery of reputational cues about the quality of information will become. An introduction of tools for measuring”credentials” seems thus the most natural development of such a system. But of course we may disagree on what counts as “credentials” for expertise….”

edit: found the link from here, which is a really excellent essay about the role of gender in the conversation about experts & peer production.

This entry was posted in Uncategorized. Bookmark the permalink.

4 Responses to a quick note for later

  1. rimrunner says:

    Now, see, if Gorman worded his discourses on authority like that, I’d be less likely to snark at him.

    I’ve been moved recently to include this kind of discussion in the material on evaluation in my classes. And now I’m really wishing I’d included it in the presentation to the undergraduate research seminar last week…

  2. reddragdiva says:

    See discussion on past post on the expert problem, which I must find time to reshape into an arrogant pontification. The trouble with trust metrics (even under a new name such as “reputational system”) is that they are snake oil precisely because their promise is to make the hard part go away, when the hard part is the point.

  3. brassratgirl says:

    basically, evaluating a source always relies on various metrics:

    old-school (libraries)
    * author reputation
    * publisher reputation
    * peer-reviewed?
    * production values
    * does it match up with prior thinking? cite data? etc

    new school (wikipedia):
    same as above, except –
    * scratch the first two, unless you’re a wikipedian yourself and recognize usernames
    * reconsider the third
    * skip the fourth, considering our army of gnome copyeditors, which sometimes I wish didn’t exist as they give a false impression of content
    * give all the priority over to the fifth

    something is broken about both options a) and b). For one thing, “wikipedia trust” is not the same as “expert trust” in the outside world. I trust that 12 year old admin to be able to patrol wikipedia well and with the best of them. I do not trust him or her to fully understand quantum physics, or the place of literature in western society, or whatever.

    But: the people Sanger calls experts don’t trust each other, either. That’s their mission in life — coming up with new and differing opinions. There’s a world of difference between being able to accurately reflect consensus on the matter (the wikipedian skill) and being able to really understand, and come up with new ideas.

    Given this situation that no one really fully understands, it’s not coming up with new metrics that’s the problem — it’s putting all of one’s faith in them, and using them as an exclusionary measure. People are usually exceptionally bad at considering what metrics actually mean; for instance, I’d guess that a large part of our success has to do with the pretty CSS in monobook (i.e., the old “production values”, every bit as much as heavy paper and a nice leather cover).

  4. reddragdiva says:

    Yeah – Stirling Newberry has pointed this out at length: that experts aren’t the best people to expect NPOV from, as a lot of the process of becoming an expert is being the most accomplished POV pusher you possibly can, and “expert” really means “POV expert”.

Comments are closed.