Most of the public only has a vague awareness of what psychology is, or what psychologists do. We have a remarkable skill set, and research methods and techniques that let us engage with profoundly important issues, even if they are not easy to measure. Our decades of studies and our carefully crafted tools help no one unless they are aware of them, can find them, are able to access them, and then empowered to act on the information. With the support of a CODAPAR grant from the APA and investment by APA's Division 53 (Society for Clinical Child and Adolescent Psychology), we were able to build a partnership that has accomplished a lot of good while also having both sustainability and a foundation for continued growth.
The partners
The CODAPAR grant provided a shared goal and a timetable, as well as financial support, to move things from a fuzzy good intention to a clear action plan with deliverable products. A grant proposal is a product that academics can organize around. The discipline of having measurable goals, a finite budget, and a short period to accomplish the work all force things to focus. Our project built on long term engagement from Division 53, adding teams of experts from Division 12 Section 9 (Clinical Assessment) and Division 5 (Quantitative and Qualitative Methods) and Division 52 (International Psychology), while also partnering with a community nonprofit, Helping Give Away Psychological Scienceopens in new window, and the University of North Carolina at Chapel Hill. The project brought together stakeholders with expertise in assessment research, children and families, ethical guidelines, cultural issues in access to services, and nuances in translation.
The process
The project sounded straightforward. Gather the best free assessments, the ones with evidence of reliability and validity across a range of samples, and put them online. The more ambitious component– program them into Qualtrics, and automate the scoring; give scores and feedback, with links to further reading and resources; link the free assessment center to multiple web sites and pages; and focus group with stakeholders to provide framing and resources that are developmentally and culturally appropriate. Then work to translate the measures and resources into other languages.
Several of the divisions already had their own websites, and the measures were free. With HGAPS, we had a partner who could help build pages on Wikipedia and Wikiversity, and store the PDFs on the Open Science Framework. This creates a completely open-source infrastructure that solves many of the challenges in sustainability. The big innovation was the automated scoring, which changes the user experience from completing a paper and pencil survey to getting feedback and suggestions for next steps all linked together. The grant fed suggestions and resources to HGAPS, where a student team led by Lizzie Wilson programmed 70 measures into Qualtrics and debugged the scoring. Several Zoom calls presented early versions to experts across divisions, and the team iterated quickly.
Serendipity
Involving multiple divisions and having a series of Zooms infused expertise and perspectives that we did not know at the outset would be helpful. Some experts looked at the “straight to consumer” possibilities and raised key considerations. Others brought up social and clinical contexts, such as working with immigrant populations, that created complexity but also opportunity. Having the input early in the process created richness, depth, and balance that would have been lacking otherwise.
The products
The performance period ended at the end of December 2019. Things did not go according to script, but we accomplished a lot, and more than expected in several domains. The Assessment Centeropens in new window exists. The prototypeopens in new window that we used as the proof of concept has 9 measures that have been used 35,000 times. The version 2.0 we built has 70 measures. We have organized them as an “a la carte” menu that psychologists could shop, along with three “tasting menus”opens in new window of core batteries. Although the soft launch only happened a few months ago, these have been used 3,000 times and counting.
We also have linked these to a growing ecosystem on Wikiversity and Wikipedia. On Wikiversity, HGAPS has built more than 200 web pages around evidence-based assessment, and they have been viewed roughly 300,000 timesopens in new window. Several of the measures have pages on Wikipedia, and the pages HGAPS has edited there have accrued 150 million views. So we met or exceeded our dissemination goals in those categories. We also did a series of conference presentations, including one at the Chicago 2019 convention (which meant that we submitted the abstracts based on the pilot work used to pitch the CODAPAR proposal) as well as talksopens in new window that have been recorded for the APA convention happening virtually in 2020. The Assessment Center has been featured in workshops and other presentations, several of which are recorded and online.
Where we have a lot more work to do is in the translations. We gathered two to 25 translationsopens in new window of more than two dozen measures, but getting them programmed is challenging–if they use a different alphabet, or if the programming team doesn’t read the language, then the difficulty mounts rapidly. Still, we gathered the building blocks, and have a good process in place.
Sustainability and continued growth
After the grant ended, COVID-19 hit. Even though the project that built the team was done, the team kept collaborating. HGAPS and Divisions 53 and 5 worked together on building pages about Coping with COVID-19opens in new window and with physical distancing, as well as organizing resources and information about telepsychologyopens in new window for providers and for the public. We are now adding measures of perceived racism and stigma to the Assessment Center, adding resources to continue to strive to bring the best of psychology directly to the people who would benefit.
We are grateful for the opportunity created by the CODAPAR mechanism, which encouraged experts to work outside of their usual silos to accomplish something good. From conception to completion, the conversations and collaboration deepened our understanding of the problems and opportunities. We are proud to have built something that not only survived the end of the grant, but continues to evolve and contribute. If you are interested in learning more, please contact usopens in new window. HGAPS has ongoing community engagement meetings Thursdays at 6 to 8 pm Eastern (3pm to 5pm Pacific), and there are a growing number of ways for psychologists to partner and share more of our knowledge with the public.
Find more information on how to submit a grant proposal and on past recipients at Interdivisional Grant Program.
