Try Reference Extract online at http://www.digref.org/

Librarians have long been seen as arbiters of quality and credibility. These librarians are now answering the questions of their users online through digital reference services. Reference Extract shall mine these questions and answers to create a new type of search engine. This search engine builds on the expert judgments of librarians in a modern, easy to use interface. Imagine searching the bookmarks of thousands of librarians and scientists.

There are few professions better suited to the world of credibility on the Internet than librarians. They have a culture of open and free expression and access to ideas. They are generalists that move agilely across different topical domains. They are skilled at searching out information, and locating potential biases in information. Their enterprises (libraries) have little invested in the production of information, and more in the consumption of information products from a wide variety of sources. Librarians already have a reputation as authoritative. It makes sense to build credibility tools upon the foundations of librarians and librarianship as well as aligned authoritative perspectives, such as those found in science and the scientific methods.

The Reference Extract architecture is expandable and easily integrated into any web site through simple HTML code. It consists of a series of “indexes,� targeted collections of searchable pages. The first index, known as the base index, searches the digital reference collection of library and expert answering services such as Ask Dr. Math, and Ask A Scientist. This index is seen as a highly credible source, but limited in terms of the number of available resources that can be found (only library and AskA sites).

The second index, the expanded index, takes all the Internet resources cited by the libraries, and makes those searchable. So if a library points a user to the NASA site, the NASA site is indexed. This expanded index is much larger than the base index and more diverse. However, because it searches against a wide range of resources beyond direct control of libraries, it is assumed to be a less credible search.

A third proposed index shall search against Internet resources selected according to criteria set forth by librarians. Once again, this index shall be larger than both the base and expanded index, but offering still less credible resources (since they have been neither created by, nor directly selected by librarians). The point is to explore a whole continuum of indexes that balance the credibility of the source versus the scope of the resource (see side bar). The research question is simply what indexes are useful (and credible) given what tasks?

In viewing the results from any of these indexes, “credibility markers� will be made available. These markers, specifically designed not to interfere with users normal searching, shall indicate how a resource was selected for indexing (library created, library selected, criteria selected), some indication of use of the resource (how many times was a resource cited by an expert, how many other Internet web sites link to this resource, etc.), and other markers determined through research and experimentation.

Associated with the tool development shall be a rigorous evaluation process. This evaluation shall look at real use of the tool and determine both the usability of the resource, and how it changes user credibility perceptions. Traditional metrics of quality in Internet search engines shall be used, such as relevance and recall & precision to determine the efficiency of the system.

Data sets, source code and evaluation data will be made available via the larger Commons site, and developers will work with other organizations to implement the successful credibility strategies developed. Furthermore tools will be provided to any organization to incorporate Reference Extract and Extract-like search into any site.