Phoebe Ayers, UC Davis / psayers@ucdavis.edu
Reid Priedhorsky, IBM / reid@reidster.net

This workshop has three key goals. First, we will examine existing and proposed systems for collecting and analyzing the research literature about wikis. Second, we will discuss the challenges in building such a system and will engage participants to design a sustainable collaborative system to achieve this goal. Finally, we will provide a forum to build upon ongoing wiki community discussions about problems and opportunities in finding and sharing the wiki research literature.

Please feel free to contact us by e-mail with questions! See also our proposal PDF.

While the notion of a centralized database of wiki and Wikipedia research has been much discussed in diverse places, we believe the research community will benefit from a dedicated, in-person session at WikiSym 2011 to discuss this problem and focus toward a solution. This workshop will (1) explore efforts to collect the wiki and Wikipedia literature to date, (2) discuss further ideas for moving forward, and (3) engage participants to enumerate requirements for a sustainable system to collect, preserve, disseminate, and summarize the wiki and Wikipedia research literature.

This will be a participatory workshop. We ask all workshop participants to arrive prepared to actively engage in the subject. Ideally, we will provide concrete outcomes useful for those interested in the topic to move forward in building such a collaborative literature system. Any specifications, road map, or other artifacts we produce will be published.

The workshop organizers will solicit participation from community members who have been active in or who have expressed interest in past efforts to address this problem.

1. Introductions (led by Phoebe, 15 minutes)
   • People write names on big board when they come in
   • Call to order, quick introductions
   • Goals of the workshop
2. Presentation: History of similar efforts (led by Phoebe, 15 minutes)
   • i.e., what is the problem and what have people tried so far to fix it?
   • Discussion by participants encouraged
3. Discussion: Requirements (led by Reid, 60 minutes)
   • i.e., “what do we need?”
   • Questions to answer include:
     • What should the user experience be like?
     • What are the technical requirements (e.g. data models)?
     • What assumptions should be made explicit?
     • What are the philosophical requirements?
     • What is the endgame (e.g., should we index all of scientific knowledge?)
   • Prioritization of requirements
4. Break (led by Mr. Coffee)
5. Discussion: Practical issues (led by Phoebe, 30 minutes)
   • i.e., “how do we get there?”
   • e.g.:
     • Platform
     • Metadata
     • Existing tools & communities that meet or which might be changed to meet our specifications
   • Who will oppose the agenda?
6. Discussion: Next steps (led by Reid, 60 minutes)
   • i.e., “what do we do next?”
   • What specific steps are next?
     • e.g.: create a mailing list
     • make a wiki
     • seek out specific other interested parties
     • write a manifesto
   • Sustainable collaboration structures going forward
   • Balance between the perfect and the good enough
   • Resources needed

We hope to start a discussion on the workshop ahead of time among the research community; add your comments here.

Hi! I created and managed the first Wiki Research Bibliography until 2008. It's back online, but I plan to export it to another infrastructure. Looking back I think it was a bad decision to use WIKINDX, but it was the only software for collaborative editing bibliographies back then. Now there are many more systems, but collaborative editing is still poorly supported. However, existing tools like Zotero, Mendeley, Citavi, CiteULike etc. are suited very well for reference management I and strongly doubt that yet another system will improve the situation. Personally I would recommend to engage in Zotero, because it is Open Source and not limited to BibTeX format - two mandatory criteria of selection. You should better extend an existing system with an existing community of developers (WIKINDX was not big enough) than creating software that nobody is willing to support and develop further. An underestimated issue is the possibility to easily export the whole bibliography, so your bibliography is not stuck one a system but you can eport it to other systems as well. The only major feature that's missing from current systems is versioning. By the way, there is another collaborative bibliography system, that already supports wiki editing: The Open Library. It's only about books (similar to the marvelous LibraryThing), and less connected to reference management, but it should be considered nevertheless. My dream is a distributed system, similar like git for source code, but versioning (and identifying!) bibliographic records is more difficult than source code. Cheers and have fun at WikiSym! – Jakob Voss

Thanks Jakob!!! This is very helpful – and thanks so much for getting the wikibib back online!! – phoebe

Alain Désilets – Before we ask the question “what do we need”, I think it may be useful to define who *we* is. What are the 3-5 kinds of users who might need this, and what is their bottom line goal?

Several sites with existing lists of academic-type literature were mentioned:

• Zotero list: http://www.zotero.org/groups/6693
• http://acawiki.org
• Jakob Voss, a german librarian and wikipedian, made a list; his is revived at toolserver (I didn't find it; somebody insert it here please
• wikibibliographie encyclen, not recently updated: http://wikindx.inrp.fr/biblio_encyclen/index.php?action=powerSearchDisplay
• citeulike group about wikipedia
• mendeley group about wikipedia
• two big lit review projects Phoebe knows of:
• (1) Finn Arup Nielson – Wikipedia research and tools: review and comments
• (2) Chitu Okoli's: Protocol for a Systematic literature review of research on the wikipedia
• and A Brief Review of Studies of Wikipedia in Peer-Reviewed Journals
• on meta.wikimedia.org there's a Wiki Research Bibliography
• English Wikipedia article on academic studies about wikipedia: http://en.wikipedia.org/wiki/Wikipedia:Academic_studies_of_Wikipedia
• and in german wikipedia: wikipedistik/Bibliographie
• Academic studies about Wikipedia
• google scholar on “wiki”: http://scholar.google.com/scholar?q=wiki&hl=en&btnG=Search&as_sdt=1%2C5&as_sdtp=on

What do we need? I think: keyword searchable ; and, can't all be stored on one cite but should be cross-linked. — Peter B Meyer 2011/10/03 23:05

In one small group (table at the back of the room),

• We generated 4 use cases, and for each , we tried to answer the question: why does Google Scholar not cut it for this use case?

• Use Case1: As a practitioner I want to get a quick answer on questions like “how much of wikipedia is created by robots?”.
  • Google Scholar doesn't cut it for that need, cause I still need to read all the papers and summarize them.
  • It would be nice to have a list of 100 or so common questions that come up in the context of wiki research, and have a collaboratively created summary of our current best understanding of that question.?

• Use Case2: As a researcher with an idea for a project, I want to know if it's been done before.
  • Why doesn't Google Scholar cut it for that?
    • If the project idea is somewhat fuzzy, it can involve a lot of digging around in different papers.

• Use Case 3: As a wiki researcher, I want to find relevant literature in related fields, that is not necessarily directly associted with wiki.
  • Why does Google Scholar not cut it here?
    • Didn't have time to discuss it.

• Use Case 4: As a wiki researcher, I want to know if an article is reputable.
  • Why does Google Scholar not cut it here?
    • Didn't have time to discuss it.

Additional details on those needs:

• want to get quick answers (Alain)
• keyword searchable – use cases are suggested: “robots” (how many wikipedia edits are made by robots? how much text have they changed)? and “deletionists” (are new users more likely to be deletionist or inclusionist?) (Peter Meyer, drawing from examples from Alain D and Nisham)
• hard problem of finding relevant behavioral research that is not explicitly about wikis (Jude Yew)
• requests for some metric of credibility or other ranking / scale / voting system. (Anna S and Heather F and others)

• finding gaps
• good data model
• support reporducibility
• finding and citing datasets & tools
• ability to identify as part of a group (ie by university)
• finding hard to find research
• list of lit
• discussion of lit
• what does the subfield look like?
• massive import vs. starting from scratch
• what research has been done
  • finding relevant lit not directly about wikis
  • finding relevant reputable literature

• editability
• personalized list groups
• watchlists rss feeds
• follow new research
• competitive spirit among students/editors
• assignable categories tags
• who cites who
• good data model
• peer reviewed
• export import citation info (bibtex, plaintext)
• include API
• easy-to-use
• link to original lit (and/or pdfs)
• econding shortcomings
• encoding relationships between papers
• include works in progress
• include many kinds of literature
• free / libre licenses (and mark open access plus open licenses)
• compatibility / interoperability
• filtering, finding, searching
• discussion/commenting
• managing refs

Reid photographed the notes on the whiteboard at the conclusion of the session: