“In God we trust; all others bring data.”
— W. Edwards Deming
It is in that spirit that Academic Program Reviews (APRs) became part of the academic landscape. Most faculty in psychology programs are confident that they do a competent job in delivering a well-designed psychology curriculum, but an APR provides an opportunity to evaluate their achievements objectively. APRs not only “bring the data,” but provide an analysis of programmatic strengths and weaknesses as well as direction about ways programs can improve.
Planning for the APR
Psychology programs undergo some kind of formal review process in a specified cycle (e.g., every five to 10 years) determined by the host college or university. Most departments use the year prior to the APR to collect relevant data, process the patterns revealed by the data and judge program quality. Departments tend to appoint a primary document writer who can coordinate the data collection and facilitate the best evaluation. The APR coordinator may be given extra compensation during the summer prior to the scheduled visit or release time during the prior year to encourage high caliber self-study execution.
Completing the self-study
Many consultants believe that the comprehensive self-study departments usually complete prior to a consultant’s site visit represents the most important element of a successful APR. The self-study document should include the department’s mission statement, a description of the curriculum, relevant assessment data and performance metrics (e.g., student credit hours, retention and graduation rates). Faculty vita, teaching accomplishments, research productivity and service patterns need to be included. Consultants also want to see any prior reports from past APRs to determine whether the department acted on any prior improvement recommendations. The best self-studies include a candid analysis of the department’s conclusions about what they do well and where they could improve.
Choosing a consultant
Typically, at least one seasoned psychology professor, invited from another college or university, serves as the APR consultant, who will make a site visit and provide a written report of the results. Sometimes programs invite more than one reviewer or ask local colleagues outside the department to join the team.
Programs usually recruit consultants based on specific areas of expertise. For example, consultants may specialize in assessment planning, curriculum reform, personnel management or space planning, among others.
Psychology programs specifically can benefit from using the Departmental Consulting Serviceopens in new window, which operates under the auspices of the Society for the Teaching of Psychologyopens in new window, in collaboration with the APA Education Directorate, to ensure highly qualified reviewers. (Contact Noland White to recruit an appropriate consultant or volunteer your services if you think your experience qualifies you to join the pool of reviewers).
Managing the site review
The consultant will have read the self-study and examined the website to get appropriate background prior to the visit. A typical site visit format involves meeting with the faculty members most invested in the self-study the night before the formal visit begins. The next day or day-and-a-half will be a whirlwind of interviews and questions to set the stage for a preliminary reveal of findings in the exit interview report. The consultant will follow up with a written review usually within one month of the site visit.
The reviewer will want to talk to administrators involved in the delivery of the program (e.g., chair, dean, provost). Those meetings often reveal important institutional agenda items or relevant constraints that will influence the success of APR recommendations. Subsequent meetings should involve all faculty in the program, either singly or in meaningful groups. For example, meeting with all faculty who teach methods can help concentrate attention on specific evaluation questions (e.g., Should statistics be taught by psychology or math faculty?).
Reviewers will need to meet with students, but seasoned reviewers tend to be bit skeptical about feedback that comes from only the most successful students in the program. One strategy for overcoming the selection bias problem is to allow the reviewer to occupy a class period of a regular course, which gives better exposure to a cross section of students of differing levels of academic success.
Reaping the benefits
Departments that are serious about continuous improvement will formally respond to the consultant’s comments in word and deed. They will build next year’s goals out of the APR experience, taking seriously suggestions for improvement strategies. Departments that are savvy should be able to turn favorable reviews into successful quests for more resources, faculty lines, space or recognition. Departments that don't allow the APR to have an impact waste this important institutional investment.
About the author
Jane S. Halonen has been a professor of psychology at the University of West Florida, James Madison University and Alverno College. Current publications of interest include the third edition of the Critical Thinking Companion for Introductory Psychology, co-authored with Cynthia Gray, and The Psychology Major’s Companion: How to Get You Where You Want to Go with Dana Dunn. Jane’s most recent research emphases have been in assessment, helping good departments become great ones, and trying to help legislators and others with serious misconceptions about psychology understand the true nature of discipline. Jane has been involved over the course of her career with helping the American Psychological Association develop guidelines or standards of academic performance from high school through graduate levels of education. She served as the chief reader for the Psychology Advanced Placement Reading from 2004-2009. A self-identified “teaching conference junkie,” Jane served on the NITOP Board and has presented at nearly every regional teaching conference in psychology. She co-chaired APA’s Summit on National Assessment in Psychology in Green Bay, Wisconsin, and participated in the 2017 APA Summit on High School Psychology. In 2000, she won the Distinguished Teaching Award winner by the American Psychological Foundation and the American Psychological Association named her an “Eminent Woman in Psychology” in 2003. In 2013, she won the APF Award for Applications in Education and Training. The Society of Teacher of Psychology named the “Early Career Award” in her honor as a tribute to her mentoring of generations of psychology teachers at all levels.
