Discovering What Works: Librarians Compare Discovery Interface Experiences

Patrons are used to Google. They don’t want to use different search methods to explore different databases. They don’t even want to use different databases. Discovery services promise to enable all of a library’s material‚ print and ebooks, journal articles, streaming video, everything‚ to be uncovered through one search box. It’s different from the much-maligned federated search because instead of crawling through the catalog, then the databases, then the various ebook repositories, and so on, it compiles an enormous index of all of those things and searches it all at once. But does it really work? And how can you choose a product? Below, four librarians describe how they chose a discovery vendor, how they survived the switch to the new product, and how it has changed user satisfaction and the way they view the information future.‚ Henrietta Thornton-Verma

EBSCO’s EDS: Relying on Patron Data to Show the Way

By Amanda Clay Powers

Like many libraries, the Mississippi State University (MSU) Libraries have a long and conflicted history with federated search products. While none provided the search experience or results that could compete with stand-alone databases, our patrons wanted to use the service. Several of us were caught off guard when a web-based survey of our federated search users‚ many undergrads, grad students, and faculty‚ found that they loved it. We were confronted with the reality that convenience outstrips the need for the perfect search. It was in this environment that we became a beta tester for a discovery service, despite some suspicion, if not outright hostility, by federated-search veterans.

As we are an EBSCO house, we decided to test EBSCO Discovery Service (EDS). When we introduced EDS in August 2010, along with a complete website redesign, we had no idea how or when to use the discovery service or if indeed we even wanted it long-term. Kicking the tires, it became clear that it was a very different product from federated solutions of the past. There would no longer be a wait to search every database, since all of the library’s products were pulled into EDS. The results were displayed in one list that could be filtered and sorted. A stunning improvement, the new relevance ranking allows the best matches to reach the top of the list. As beta testers, we had no one to ask about proper use of EDS, so it featured prominently in our departmental discussions. We reported features and bugs. We discussed successes and failures. In training sessions that were ostensibly about individual databases, discussions strayed to EDS.

Becoming relevant again

Then faculty began complimenting us on the service in meetings with the dean, one member saying we were relevant again in his classes. This only fueled the fire; we couldn’t stop talking about our new toy, debating why it was so successful and how it could best be used. We knew undergrads needed something different from terminal master’s students, who again needed something different from Ph.D. students, never mind faculty.

We sorted out several levels of need and tried to identify when EDS worked. First, the perennial undergrad assignment requiring three to five relevant peer- reviewed articles was almost magically solved. For the next group, terminal master’s and upper-level undergrads, EDS wasn’t a magic bullet, however. These students don’t have much searching sophistication and need in-depth sources for articles on discipline-specific topics. Subject-specific databases still answered their needs best, grounding their searches very well without a lot of extra steps to set parameters. EDS was clumsy when it came to this type of searching, forcing students to wade through a lot of information.

The biggest surprise, however, came from doctoral students and faculty. Dissertation writers get increasingly familiar and focused with their research as they compile a literature review, and the tighter the focus, the better EDS served their needs. Serendipity returned as cross-disciplinary searches became a reality with extremely narrow topics. Breaking down faculty information needs to determine why EDS was so popular, we realized that it was great for obscure citations, interdisciplinary subjects that stretched a professor’s expertise, or finding review articles on a topic they hadn’t taught lately. It turns out that EDS excels at both ends of the information search spectrum‚ the most generic need and the narrowest‚ so that was where we began to apply it at the desk and in our classes.

Still, while speculation was one thing, we knew that our assumptions would remain flawed unless informed by our patrons’ voices. What shifts might be happening in our patron questions and in our own database use since the EDS implementation?

Virtual reference: measuring the shift

As part of the website redesign, I had reviewed the April 2010 chat transactions at the MSU Libraries, coding the content of the questions asked and the tools used by the librarian in each session. The Web Services Department used this information, along with word-frequency charts (Wordles) of the chat and email entry questions for the previous two years, to provide a qualitative perspective for the redesign. Along with that data, they used information gathered from Google Analytics and from the libraries’ content manager, Urchin; student and faculty usability testing; a review of the literature on library web design; results from analyzing the federated-search box on the MSU Libraries Databases Portal; and web-based usage surveys.

Several things became apparent in the analysis of virtual reference data. Topic questions accounted for 30 percent of questions asked, and another 27 percent concerned finding articles using only a partial citation. The third-largest category was users seeking help with the OPAC. Reflecting these findings, as well as the frequent language employed by patrons in the Wordle, the terms books, articles, and help are all featured repeatedly on the page.

EDS was given a great deal of real estate in the new website design, and we believed that this would change the distribution of virtual reference questions, decreasing the topic, citation search, and online-catalog questions, since students could now help themselves with those.

To check whether this was true, I repeated the 2010 analysis with the April 2011 chat transactions. The results were mixed. Though I knew that the total number of questions over the year had increased, the number of questions dropped by nearly 25 percent in April 2011 compared with April 2010. And though they were different kinds of questions, the changes were not as we expected. The number of topic questions dropped significantly, and citation searches declined a bit, both seeming to show that students were now able to answer these questions on their own. At the same time, questions about using the online catalog increased dramatically, which we found puzzling.

We speculated that the April 2011 decrease was because the library was repeatedly locked down for tornado warnings, or because the citation search functions were not obvious enough. From the discovery box, there was no way to limit the search to books initially, so that could be a contributing factor. Also, we had added 17 libraries to our online catalog, bringing the total to 42. While the consortium catalog expands access, it also introduces complications in requesting materials from other libraries.

The question of whether MSU librarians would adopt the discovery tool took less interpretation. The use of EBSCO’s Academic Search Premier, previously the primary database for chat transactions, dropped significantly, and EDS effectively took its place. The tool has become part of our library instruction sessions and worked its way into our departmental presentations. While the librarians have firmly stepped into the EDS camp, we still have work to do to maximize its effectiveness for our patrons. We need to get the word out about EDS as a citation search. We need to find out why the questions about the online catalog have increased. And then we’ll do it all again.

Amanda Clay Powers ( is Virtual Reference Manager at Mississippi State University Libraries. Find her online at

OCLC WorldCat: Interactivity and Mobility Create a Winning Combo

By Zinthia C. Briceño-Rosales

At Washington State University (WSU), Pullman, a localized version of the OCLC discovery service is used at the reference desk and all service points, providing quick access to known items and acting as a starting point to show students how to search for items that they don’t know about and perhaps would never have thought to request. The problem of helping patrons who have only a vague idea of a book title is also solved, along with the task of finding feature films or popular fiction, requests that are often difficult to fulfill in a traditional academic library online catalog, as those libraries may not collect such material. And if the popular item they are looking for is not in our libraries, we can point them toward a nearby OCLC customer library that owns a copy.

We began working with WSU WorldCat‚ customized with WSU branding and information on our local consortium‚ at the start of the school year in August 2009. Prior to that, WSU had separate digital locations for the OPAC, article databases, and digital collections. While these are still available, we adopted WorldCat as a fresh way of finding items with the start of the new academic year. We also customized the library’s homepage to include digitized content from our Manuscripts, Archives, and Special Collections division. Making digitized content visible is one of the aims of a discovery system, and now our digital content is on the front page for all to see.

Implementing interactivity

WorldCat was chosen by our consortium, Orbis Cascade Alliance, and we participated in task forces and implementation teams at the alliance level that worked toward the creation of a WorldCat group catalog. The process of customizing and implementing this new service at the institution level was one that included systems, reference, and technical services librarians. Several features that made WorldCat an attractive option have become heavily used here at WSU. For example, the system allows patrons to create personal, interactive accounts. They can save lists of items, searches, and favorite libraries (ones that also use OCLC cataloging, though not necessarily WorldCat). Patrons can also create profiles and watch the growing lists created by others. Yet the most striking feature is that once patrons leave‚ a student graduates, a faculty member takes a new job, a researcher finishes her work‚ they can still keep their WorldCat account. Open access, full-text materials will continue to be viewable to them, and they will have access to all of the citations they amassed while affiliated with WSU.

WorldCat enables an interactive experience with library holdings, √† la LibraryThing, since account holders can tag books with their own searchable terms. A search of WSU WorldCat’s crowdsourced tags for The Daily Show, for example, reveals books that have appeared on The Daily Show with Jon Stewart. It is also possible to add book reviews. Librarians who think this is a cataloger’s nightmare should know that adding more data to records does not mean that subject headings and firm cataloging data are altered. Instead, reader additions exist alongside professionally created information.

Another selling point was the system’s ability to integrate data gathered with Red Laser personal-use barcode scanners or smartphones. WorldCat‚ Red Laser has worked with OCLC to make ISSNs, ISBNs, and barcodes searchable. When users scan a book barcode at a local bookstore, one of the searchable sites for the item is WorldCat, so that our collection is exposed to readers who might not have thought of borrowing the book from a local library.

At the reference desk

WSU WorldCat has proven invaluable at the reference desk. While we occasionally get in-depth queries regarding thesis or grant research, most reference questions come from undergrads wanting help finding items for reports and presentations. The discovery system locates relevant books and articles in both print and electronic formats, and I see students’ eyes widen when I point out that we have thousands of items in our results. It is unbelievable to them that the library would have access to so much information about a single figure in history or a military battle.

I then point out that there are so many sources that we might want to refine the search. There’s no better way to do that than to point out the facets provided by OCLC. These search-narrowing terms include author, format, language, and year of publication. Most important, the system also suggests subtopics of the original search, and this feature allows students to focus on what they really need and perhaps discover aspects of their topic that they had not considered. A result of these facets: the product has increased student confidence in the library and in their own search capabilities.

Since WSU WorldCat is integrated into the library’s mobile site, search features are available to students on their mobile devices. Instead of waiting through the lunch rush for computers or searching before they get to the library, students can easily look for books and determine their availability right on their mobile device (they cannot see their accounts on them, however). Tablet use is rising, and soon our students, faculty, and researchers will be strolling into the library with their devices displaying the list of books they are going to pick up.

On the librarian side, OCLC-created apps are helpful in the genesis of subject guides. The subject and department liaison librarians at WSU have used these apps to make customized and new-book lists for their subjects. Defined searches integrated into the tools keep liaisons and their faculty up-to-date on new content cataloged worldwide using OCLC. The best part is that these apps can be easily plugged into any subject guide or blog. The ease of set it and forget it is something that appeals to librarians at WSU.

Since the launch of the service, the library has added more than ten centrally searchable databases. Interlibrary loan requests rose from 11,080 items in 2009 to 19,118 in 2010. We’ve made our subject pages dynamic with apps and customized our homepage app to elevate our digitized content. We’re sure this isn’t the end of our improvements, as OCLC is always adding functionality. See its WorldCat Identities, which allows users to view wiki-like author pages, and keep your eyes open for HathiTrust digitized materials.

Even though discovery tools such as OCLC’s WorldCat have only existed for about five years, they are already indispensable at the reference desk and in our digital branches. When students can carry on their mobile devices all the research they discover using this system, the future will really be here.

Zinthia C. Briceño-Rosales is a Web Design, Instruction, and Virtual Reference Librarian at Washington State University Libraries, Pullman. Find her online at and

Ex Libris’s Primo: Easy To Implement, Intuitive To Use

By Rebecca Fernandez

Midwestern State University (MSU) in Witchita Falls, TX, one of only four independent public universities in the state, has a current enrollment of about 6500 students. Its Moffett Library, which supports the university and the local community, went live with Ex Libris’s Primo discovery and delivery service in January 2011.

The library’s initial examination of discovery services involved checking what we considered critical elements: content, platform, reliability, support, ease of use, design, capacity for growth, plans for future development, reputation of the vendor, and, of course, cost. We also did some unofficial polling of other librarians.

The big picture

The candidates had varying degrees of strengths and weaknesses. One of the strongest arguments for Primo was its content neutrality‚ it doesn’t push its own material. As for pricing, one charge covered searching the full breadth of content that we had at the time and that we might add in the future, at a cost comparable to the base charge of a competitor we considered. Additionally, as a Voyager library, we were familiar with Ex Libris and had already gauged its service to be top-notch.

Most significant, we viewed any new product as only a segment of a wider library service upgrade. The timing was crucial in that we were contemplating whether to replace our server and remain dependent on the university’s IT department or move to a hosted service. The library did not have the expertise required to maintain a new server, the Voyager catalog, and any new tools we acquired, such as Primo. We also knew that our economic outlook would not allow us to hire anyone with those talents or abilities. We felt that Primo, a product that is promoted as not needing local IT backing, would solve a difficult situation and would be a four- to five-year commitment at the most.

At the same time, the library implemented Primo Central Index, a service that provides scholarly e-resources of regional and global importance, and bX Recommender Service, which generates scholarly recommendations based on article popularity. A library can choose which data these recommendations are based upon‚ they automatically come from global usage, but an institution can also add local, consortium, and/or peer institutions’ usage statistics. Both of these innovations are cloud-hosted by Ex Libris. We also began to use the company’s link resolver, SFX, a behind-the-scenes web product that connects material such as abstracts and citations to the appropriate full-text resources. Though this switch is not required for Primo to work efficiently, we had experienced an ever-increasing annual cost to our previous link resolver, and we felt SFX would give our services a more cohesive structure.

The implementation process was one of the smoothest we have ever experienced. Ex Libris assigned a team of a project manager and Primo and SFX leaders. The company also provided workbooks to help us offer guidance to the implementation team. During numerous webinars with the Ex Libris crew we saw changes as they were made. Early in the project we were given access to our Primo setup on Ex Libris’s server to give us hands-on experience. This helpful step allowed us to ask more specific questions of the team and to get a real feel for the product before we went live. The best part of the process, though, was that the Primo and the SFX lead staffers continued as our main contacts after the service was up and running.

In preparation for our move to Primo, we redesigned our website using the discovery product’s interface as our front page. At the time we decided to leave the page relatively unchanged except for branding and some minor adjustments. We kept it simple. That decision worked out very well for our students and the library staff. We are now ready to start customizing our Primo page and look forward to making the website more tailored to our university within the next few months.

Bringing the library into the classroom

The response to Primo has been overwhelmingly positive. The ease of use and intuitive nature of the single-box search and the relevance-ranked results are especially popular. Students love that these tools are available on their mobile devices and appreciate the service’s personalization features, which are available on more and more products and which users are beginning to expect. Primo allows users to save searches in an e-shelf and be notified by email or RSS of new, relevant material. They can also initiate interlibrary loan (ILL) transactions and use social reading tools such as tagging of records and rating and reviewing books and other materials.

Bibliographic instruction is much better received and easier to provide, as Primo has freed librarians to spend more time teaching the finer points of research resources and methods‚ using Primo’s facets and limiters, for example‚ instead of basic search mechanics.

Another indication of Primo’s success: faculty is much more receptive to including the library as part of the educational process by inviting us into classrooms to demonstrate it. This has presented the library with opportunities to be more visible across campus and more engaged with students and to promote our other types of training and research assistance. Professors have also been receptive to integrating Primo into the online portions of their classes, which is easily done via compatability with course management software.

As we continue to gather statistics, our analysis indicates increased usage of library resources overall and a marked increase in the use of resources previously overlooked by many. This is a clear indication to us that Primo is a huge success.

What’s next for Moffett Library? We are soon moving other services to the hosted-service model: our Voyager catalog will move to Voyager Direct, and we are creating and posting subject and research guides using LibGuides. Finally, we have signed on to be part of the Early Adopter Program for ALMA, Ex Libris’s next-generation library system. Another hosted service, ALMA, which is part of the greater Primo picture, will replace our Voyager catalog while adding a host of other capabilities to extend MSU’s range of library services. Overall, there are positive changes ahead that will extend the gains afforded by our adoption of Ex Libris’s Primo system, and we’re excited to get started.

Rebecca Fernandez is Associate University Librarian for Technical Services, Moffett Library, Midwestern State University, Wichita Falls, TX.

Serials Solutions’ Summon: Familiarity Breeds Success

By Ken Varnum

Buying a discovery system at the University of Michigan Library, Ann Arbor, began as an investigation of Google Scholar to replace our federated search tool. We formed a committee of librarians from across the organization to tackle the task, focusing on better article discovery, our greatest need. We compiled a 45-item list of desired features and functionality‚ scholarly results, technology customizability, and more‚ that an ideal tool might have, using it to assess the handful of products then on the market or announced as in development.

To bring in user needs, we adopted the concept of personas from usability testing. Library versions of these describe in broad strokes the research needs and habits of large numbers of users. These qualities of these patrons were combined with a survey of 1000 actual users’ article-searching techniques and preferences to finalize an evaluation checklist.

Keeping users in the library

Serials Solutions’ Summon discovery system emerged from this research as a likely candidate. One of Summon’s attractions is that it uses an API (Application Programmng Interface) to display its results within our library’s website. This is congruent with our philosophy to keep users in our library site until they get to the article, database, or other digital object they need. The practice reduces the number of interfaces novice researchers must tackle and provides context to all patrons. We settled on Summon after interviewing our faculty and staff who tested it at another academic institution with a similar breadth and depth of resources.

Programming and interface design took us about ten weeks, from July until late September 2010. While building the interface, we also entered our digital journal holdings into the Summon knowledge base. This latter step is critical in any discovery product implementation; if the system does not know what titles and volumes your library has licensed, it cannot provide the right search results. Once Summon knows what your library has, it answers queries with a list of citations (including author, title, publication, volume, issue, pagination, keywords, and an OpenURL or other link to the full text), facets to help the patron limit the search results, and recommendations of specific databases the patron might try, based on the subject areas that best fit the original query.

ArticlesPlus, as we branded the service, was immediately used more intensively than was its predecessor. Because of the poor user experience federated search had provided, all but the most dedicated researchers sought other alternatives; the tool was used between 200 and 500 times a day, depending on the academic calendar. In contrast, immediately after launch, ArticlesPlus searches averaged about 1050 per day; the same period this year saw about 1950 uses daily, a significant increase. We have noticed a similar jump in full-text downloads from our aggregators and a decrease in searches done directly through licensed subject database interfaces‚ indicating that users are taking to the simplicity of the new interface while accessing more full-text content in the process.

Although we’ve received comments from users noting how much they appreciate the new system, it does have drawbacks. The tool it replaced offered patrons the opportunity to build custom search sets of specific databases so that they could, for example, run a single search against PubMed and Web of Science, saving them the effort of using both interfaces and scanning two results lists. While ArticlesPlus includes both, it covers a far broader scope than either of those databases but is much harder to focus on a single collection.

Patron reaction

The results of a winter 2011 follow-up survey were positive, but highlighted the loss, for advanced users, of specific cross-database searches. Still, 74 percent of the 484 respondents who indicated they had used ArticlesPlus said they were somewhat or very satisfied with the tool. This compared with a 56 percent satisfaction rating (based on 435 users) for the product it replaced.

Building our own interface has given us several other benefits beyond the basic one of having discovery take place within the library context. Each full-text citation has a report a problem link along with it for when things go awry. This link goes to a form that asks the patron to tell us what happened (from a list of likely options) and captures the exact citation that failed. Between the reference desk and the technical services staff, we are able to identify and solve problems with full-text links that would be very difficult to resolve from the data provided by the link resolver. We can also follow up directly with patrons to make sure they get a copy of the article they want.

We are also in the process of building a favorites tool within ArticlesPlus, something that the hosted version of Summon does not offer. Because we can connect a citation (and OpenURL) with a patron account, we can save citations for the patron’s future use.

Putting article discovery in our library’s site allows us tremendous flexibility to understand both what our users are looking for (through search logs and clicks to full text) and how they are using the data and enables us to identify problems with specific journal titles or aggregators quickly. Keeping the full range of the library’s online resources‚ our chat service with reference staff, our catalog, our subject specialists, and more‚ in front of the user allows them to combine conveniently the richness of our collections with the value-added presence of library staff. And for novice or infrequent searchers, this approach looks and feels like the same interface they may have used before and is even Google-like, so it’s a win all round.

Ken Varnum is Web Systems Manager, University of Michigan Library,
Ann Arbor. Find him online at