PsycTESTS®: My How You Have Grown!
We've talked about PsycTESTS® before in these pages. In November 2010 we shared the details of Operation Test Derby and the all-hands-on-deck process that was needed to jumpstart our test collection process. In a special issue in June 2011, prior to our September launch, Introducing PsycTESTS outlined PsycTESTS' purpose, the types of tests it contained, and the new fields and limits created specifically for the database.
Now that PsycTESTS has been in the marketplace for more than a year and is available on all of our major platforms, we can catch our breath and take stock of how PsycTESTS is doing.
How is it doing? In a word: Great! So much so that in this issue, we celebrate the passage of a major milestone: We've passed 10,000 records, blowing through our goal for 2012 and setting us up for even greater growth ahead. As more researchers report to us how pleased they are with the database, we are ever more confident that we've succeeded in creating the much-needed tool we set out to build.
To mark the moment, we'd like to do something we don't normally do. We'd like to honor the individuals who have gone to extraordinary lengths to bring PsycTESTS to birth and ensure its success.
See The People of PsycTESTS for an inclusive list.
In a way, PsycTESTS is rooted in an APA Books project, the Directory of Unpublished Experimental Mental Measures (D.U.E.M.M.), which is a nine-volume collection of test instruments. The content looked so much like content for a database, and we'd received so many questions about how to find tests, that the idea of a test database was a short leap.
Electronic Books Manager Olin Nettles presented the idea in 2009, and the more APA Publisher Gary VandenBos thought about a tests and measures database, the more he liked it.
We knew it was a big undertaking, though at first, we didn't realize quite how big. This was different from anything else we'd ever attempted. Finding tests in the maze of resources available turned us into explorers, creating the interface on our existing platform when the structure and fields and limits differed so greatly turned us into mechanical engineers, and developing the production process required a whole different set of steps and skills and turned us into efficiency experts.
We'd like to acknowledge Senior Director Linda Beebe and PsycINFO Managers Olin Nettles, Jan Fleming, Susan Hillson, Brenda Evans, Alvin Walker, Kathleen Sheedy, and Lynn Willis, who have been involved at every step of PsycTESTS, as has James Whitfield, Associate Manager in Production.
From the beginning, what was to set PsycTESTS apart was that it would provide access to the text of the test itself wherever possible. Thus, for PsycTESTS to succeed, we knew we needed to find tests and fast.
We launched in September 2011 with fewer than 3,000 tests in the database and an all-out campaign to aggressively acquire suitable content. When we set a goal of 10,000 records by the end of 2012, it seemed like an impossible task. But we've learned from experience, and in addition to dedicated APA staff, freelancers, and contractors searching for tests, we also
- Emailed authors directly to see if they would like to contribute their tests. Although at first, most interactions were spent explaining what PsycTESTS would be, now so many of the authors already know PsycTESTS that we are reaching the tipping point where they are beginning to contact us to contribute their instruments. For this task, we'd like to acknowledge Content Development Manager Lynn Willis and John Firsching, Mary O'Rourke, Michelle Provenzano, and Amanda Conley.
- Collaborated with the Center for the History of Psychology at The University of Akron to digitize historical tests in their collections. That collaboration involved APA staff, Olin Nettles, and Executive Director Dave Baker and Digital Projects Manager Jodi Kearns at the Center.
- Partnered with other publishers, such as Elsevier and Sage, to be able to add their content.
- Joined with commercial publishers to provide structured information about and links to their tests.
- Hired graduate students to search (see Graduate Students: Our Not-So-Secret Weapon).
We originally envisioned a separate search page for PsycTESTS on APA PsycNET because the field structures for PsycTESTS are so different from the other APA databases.
In 2011 librarian and end user feedback at ALA–Midwinter and a university site visit to UCSD made us change course. The user's need for simple initial searching complemented with detailed faceting options at the search results stage made integration into the existing platform essential.
The technology impact of integrating PsycTESTS search functions with existing database search functions was daunting, but we felt the end result would be worth it. We thank the ALA librarians and the UCSD students for their input.
For tasks as diverse as constructing the wire frames and usability testing (Eva Winer), PsycNET technical requirements and mockups (Sahar Sheikhani), and a separate help menu (Tim McAdoo), we thank APA staff. The entire list of Information Technology Services (ITS) staff who participated is included on The People of PsycTESTS.
One of the greatest challenges with the database has always been the mechanics of how to categorize and present the data. Our first discovery of how rocky a road this would be was simply identifying the fields and limits for the record. Seven managers all looked at the same eight articles containing tests and independently filled in the fielded information they thought would comprise a basic record. The result? It was a bit like the story of five blind men describing an elephant — the results were so inconsistent they had to stop and rethink the information we really wanted to capture. That began an arduous process of defining, testing, and refining by the PsycINFO management team.
Among the many challenges they encountered were the following:
- Who knew there are so many ways tests can be developed? (for starters, from "scratch," by combining items from different tests, and by translation to a different language or context).
- Who knew there were so many kinds of test reliability and validity?
- What do we do in our records when multiple tests are named alike, when tests have no names at all, when tests appear in multiple versions, or when tests appear in multiple languages?
PsycINFO management as a whole struggled to find resolutions for all of these issues. The bibliographic production team, led by Brenda Evans, has had to adapt to a new day-by-day work flow. This database requires them to search for the test's origins into the process and to find other studies that have used the test substantively. When they find a relevant article, they must tease out the data on test development and validation from the studies themselves.
Unlike the other databases, where a record is a record, the complete record in the PsycTESTS database is actually a package composed of several components that can grow over time. This makes managing the workflow to bring these components together both interesting and never actually completed.
We thank Production Workflow Supervisor Cyndy Nixon for this challenging task and the entire production team.
All of these issues had to be thought through and required a production solution. This has been an epic case of form following function. As our needs have evolved, our production system has had to follow suit.
We'd like to acknowledge the ITS staff who designed and redesigned Artemis, our production database, even as our vision of what PsycTESTS would be kept changing. Thank you Publishing Product Services Manager Sabina Mathur and Lawrence Logan, Ashish Thomas, and Adrian Breeman.
Training, Marketing, and Sales
Of course, the list of people who have contributed goes on. The Customer Relations Team, led by Susan Hillson, has created documentation and conducted endless presentations and training webinars. The Marketing and Sales Teams, led by Peter Gaviorno, with Tim Rinda spearheading the promotional campaign and Neil Lader leading the boots-on-the- ground sales, has played a crucial role.
And those of you who have encouraged us to create PsycTESTS, contributed your time for beta tests, and have supported our efforts by subscribing have our gratitude as well.