HIV Prevention Program Evaluation
This page provides tools for the development and evaluation of HIV prevention programs. It was designed to serve the needs of HIV prevention providers and technical assistance providers who serve these providers. The weblinks are organized by topic and include sites that may be useful for novice evaluators, as well as those that may be more relevant for highly experienced evaluators and technical assistance providers. Each weblink has a brief annotation that describes the content and provides oration to help you to decide whether it meets your needs.
The topics here include evaluation planning, as well as links to specific types of evaluation such as outcome studies and cost-effectiveness evaluation. Other links will help you with specific research methods such as standardized surveys and qualitative approaches such as focus groups and open-ended interviews. In addition, we have included sites which will provide description of effective HIV prevention programs from throughout the world and sites which can help you use the data you collect to choose new interventions.
This site is designed to be a "how-to" guide to planning and implementing an evaluation of your prevention program. If you start by clicking on Section I of the outline below, you will be led through the step-by-step process of developing an evaluation. You can also use the outline to navigate the site and locate specific kinds of evaluation information. There are seven major sections:
I. What is Evaluation and Why do it?
II. Using a Logic Model for Evaluation Planning
III. How to Build Your Program Logic Model
IV. How to Plan Your Evaluation
V. Evaluating Your Program Using the Logic Model
VI. Analyzing, Using, and Interpreting Evaluation Information
VII. Implementing the Evaluation
CDC Framework for evaluation (downloadable PDF or HTML document); a somewhat academic presentation of an overall framework for considering evaluation and program planning.
Tools for planning an evaluation from University of Kansas' "Community Tool Box"- includes useful questions for self-assessment of evaluation needs, checklists and outlines for making timelines.
A compendium of links related to constructing logic models (not HIV specific) from CDC’s Evaluation Working Group.
CDC guidance for evaluating CDC-funded health department HIV prevention programs.
Strategies for Needs Assessment & Formative Research
The UT-Southwestern ACCESS guide provides strategies and methods for conducting rapid assessment of HIV prevention issues in the community, with an emphasis on qualitative research methods.
Parts of the "Community Toolkit" helpful guide to conducting focus groups, guide to documentation of community problems; includes checklists, basic questions, and links to a variety of topically-related resources.
Guide to conducting focus groups; general outline, checklists, examples of successful and unsuccessful focus groups; includes links to a variety of topically-related resources.
This Web site includes downloadable publications (PDF format) related to monitoring & evaluation of HIV programs, from international programs. Includes descriptions of measures used as population based indicators of knowledge, risk behavior, stigma, and disease prevalence. Also includes some methodological tools for estimating risk groups. Designed to help evaluate national HIV programs, these tools are readily applicable to evaluation monitoring at the state-level and large population areas such as regions, metropolitan areas, or large cities.
Sources of Data: When The State & CDC are Not Enough or Aren’t Handy
Kaiser Family Foundation Web site; includes new and cumulative AIDS cases, persons living with AIDS, AIDS deaths, Ryan White funding, and state AIDS policies, such as syringe access and HIV testing.
Youth Risk Behavior Survey Web site, includes state-level data and core item content.
Behavioral Risk Factor Surveillance Survey Web site, includes state-level data and core item content.
US Census Bureau site that permits searches on specific information for block, city, county, & state levels.
National Neighborhood Indicators Partnership provides local data for a variety of health, social, & economic indicators, for growing number of medium to large cities. Some datasets have capacities for simple analyses (e.g., cross-tabulations by sex, age, etc.).
Outcome Monitoring & Evaluation
Methods for evaluating community-level health interventions. Includes links to tools for evaluation of goal attainment, as well as individual and community-level measures and indicators.
National Institute on Drug Abuse (NIDA) Web site with links to downloadable PDF documents that deal with fundamentals of cost-benefit & cost-effectiveness, as well as specific examples of cost-effectiveness evaluations with populations relevant to HIV prevention.
Downloadable UNAIDS document (PDF) which reviews models of cost-benefit analysis and common approaches for designing cost-effectiveness evaluations, including a review of measures.
Geographic Information Systems
National Oceanic and Atmospheric Administration (NOAA) Web site with links to a variety of Geographic Information Systems (GIS) tutorials; the tutorials vary in difficulty, but none seem to be designed for a non-technical audience.
Plain English Presentation of Data
Developing Consultation Contracts
A PDF downloadable checklist for developing evaluation consultation contracts (not HIV-specific, but easily adapted).
A PDF downloadable checklist for writing up an evaluation (not HIV-specific, but easily adapted).
Sampling, Methods, & Measurement
Link to PDF files that describe basic steps in sampling, including sample size calculation.
Glossary of sampling terms; relatively "plain English" definitions, along with concrete examples.
Powerpoint slides from a course by Mike Raulin outline. Very good review of measurement concepts including scaling.
This presentation is by the same author & reviews threats to reliability. Good review, but more technical than the above.
This presentation reviews hypothesis testing & threats to validity. It is similar in technical level to the reliability presentation and a good review of concepts.
Bill Trochim's readable review of reliability concepts; better as a refresher for a professional audience than for non-researchers. Links to pages of similar technical level on scaling, validity, and unobtrusive measurement.
Online book that contains basics of survey design (including item development, data entry, etc.); useful as a handout or as a basis for developing TA materials. The website uses a frames format, however, the buttons on the left do not work & you need to scroll through the text (not a big deal). Provides some fundamentals for writing data.
Overview of survey research: item selection, scaling, etc. The content varies from simple to relatively academic, but much of the material would be suitable as background for developing TA materials.
Basic terminology & methods used in sampling, with an emphasis on survey applications.
Instruments developed by HRSA contractor, The Measurement Group and used in the Health Resources and Services Administration’s (HRSA) "Special Projects of National Significance". Instruments cover a variety of topics which include: demographics, barriers/facilitators of care, client satisfaction, TA evaluation, quality of life, HIV knowledge, substance abuse, distress, sexual behavior, partner disclosure, and homelessness, plus citations for other measures that were used in this project.
Evaluation Materials developed by HRSA contractor, The Measurement Group for HRSA Special Projects of National Significance (SPNS) Program Innovative Models of HIV/AIDS care. Organized by project; many could be adapted for prevention programs. Studies include key informant interviews, client interviews, instruments for evaluating training, etc.
Web site for CDC’s HIV/STD Behavioral Surveillance Working Group. Contains core survey items on HIV risk behavior (sex & drugs), and HIV testing that were complied by CDC's behavioral surveillance working group. STD and TB items are coming in the future. The site explains how these were selected and developed. Items include those for general and high-risk populations. There's also a program contact listed.
University of Illinois Extension Web page with downloadable PDF files covering key informant interviewing. Excellent and practical review of the basics in conducting key informant interviews, including a section on summarizing/writing up interviews, though weak on the analysis of interview data. Useful for background and as a handout.
Good, brief review of fundamentals for considering and conducting focus groups.
How to Read Scientific Papers
"How to read a paper": reviews statistics and their applications. Good for the person who has some scientific training and may provide useful background materials for doing technical assistance.
Different types of data need different statistical tests
"Significant" relations and their pitfalls
Papers that go beyond numbers (qualitative research)
Penn State University Web page that links to a variety of downloadable PDF "tip sheets" on a variety of topics ranging from how to make a pie chart to sampling diverse audiences to asking about age. The sheets vary in length & quality. All include helpful reference citations.
Integrating Data, Methods & Program
Reprint of an Annual Review of Public Health lit review on sampling and its implications for findings. Rather academic, but a useful background document for developing technical assistance materials.
Program Development Resources
"REP Plus": This link goes to CDC's Replicating Effective Programs (REP) project which is looking at replication/translation of interventions with demonstrated effectiveness. The individual REP projects are described in terms of contacts and core elements. The site reviews other CDC resources (with links to a compendium of effective prevention programs, Characteristics of Reputationally Strong Programs (C-RSP) Project, and a link to the APA Behavioral and Social Science Volunteer (BSSV) program.
This link goes to the UNAIDS "Best Practices" site which provides examples of "best practices" for a variety of topics (care, prevention, epidemiology, human rights, etc.). Prevention topics are better served by the population links (e.g., children) than the "care and prevention" link. The links go to project-related monographs, topical monographs and proceedings from meetings. The writing varies greatly from concise and useful to the endlessly ponderous.
CDC Division of Adolescent & School Health’s Web page for "Programs That Work." Provides contact information to receive documents; the "Programs That Work" is undergoing administrative changes.
The Center for Substance Abuse Prevention’s (CSAP) listing of "Model Programs" for drug & alcohol prevention.
Principles for drug abuse treatment programs (includes attention to HIV/AIDS risk).
Prevention and care programs for HIV-positives. Successful program principles, as well as sample programs from various HRSA-funded activities.
The Center for Substance Abuse Prevention (CSAP) Decision Support System; logic modeling is presented as a circular (recursive) seven-step process beginning at Assess Needs and progressing through Develop Capacity, Select Programs, Implement Programs, Evaluate Programs, Report Programs, and Get Technical Assistance and Training.