|
HIV Prevention Program Evaluation
|
|
This website provides tools for the development and evaluation of HIV prevention programs. It was design to serve the needs of HIV prevention providers and technical assistance providers who serve these providers. The weblinks are organized by topic and include sites that may be useful for novice evaluators, as well as those that may be more relevant for highly experienced evaluators and technical assistance providers. Each weblink has a brief annotation that describes the content and provides oration to help you to decide whether it meets your needs.
The topics here include evaluation planning, as well as links to specific types of evaluation such as outcome studies and cost-effectiveness evaluation. Other links will help you with specific research methods such as standardized surveys and qualitative approaches such as focus groups and open-ended interviews. In addition, we have included sites which will provide description of effective HIV prevention programs from throughout the world and sites which can help you use the data you collect to choose new interventions.
Evaluation Planning
This site is designed to be a "how-to" guide to planning and implementing an evaluation of your prevention program. If you start by clicking on Section I of the outline below, you will be led through the step-by-step process of developing an evaluation. You can also use the outline to navigate the site and locate specific kinds of evaluation information. There are seven major sections:
CDC Framework for evaluation (downloadable .pdf or html document); a somewhat academic presentation of an overall framework for considering evaluation and program planning.
http://www.cdc.gov/eval/framework.htm
Tools for planning an evaluation from University of Kansas’ "Community Tool Box"-includes useful questions for self-assessment of evaluation needs, checklists and outlines for making timelines.
http://ctb.lsi.ukans.edu/tools/EN/sub_section_tools_1352.htm
A compendium of links related to constructing logic models (not HIV specific) from CDC’s Evaluation Working Group.
http://www.cdc.gov/eval/resources.htm#logic%20model
CDC guidance for evaluating CDC-funded health department HIV prevention programs.
http://www.cdc.gov/hiv/aboutdhap/perb/hdg.htm
Return to top
Strategies for Needs Assessment & Formative Research
The UT-Southwestern ACCESS guide provides strategies and methods for conducting rapid assessment of HIV prevention issues in the community, with an emphasis on qualitative research methods.
http://www3.utsouthwestern.edu/preventiontoolbox/assess/assess.htm
Parts of the “Community Toolkit” helpful guide to conducting focus groups, guide to documentation of community problems; includes checklists, basic questions, and links to a variety of topically-related resources.
http://ctb.lsi.ukans.edu/tools/EN/section_1019.htm
Guide to conducting focus groups; general outline, checklists, examples of successful and unsuccessful focus groups; includes links to a variety of topically-related resources.
http://ctb.lsi.ukans.edu/tools/EN/section_1018.htm
This website includes downloadable publications (.pdf format) related to monitoring & evaluation of HIV programs, from international programs. Includes descriptions of measures used as population based indicators of knowledge, risk behavior, stigma, and disease prevalence. Also includes some methodological tools for estimating risk groups. Designed to help evaluate national HIV programs, these tools are readily applicable to evaluation monitoring at the state-level and large population areas such as regions, metropolitan areas, or large cities.
http://www.cpc.unc.edu/measure/topics/hiv_aids/hiv_aids.html
Return to top
Sources of Data: When the State & CDC are not Enough or Aren’t Handy
Kaiser Family Foundation website; includes new and cumulative AIDS cases, persons living with AIDS, AIDS deaths, Ryan White funding, and state AIDS policies, such as syringe access and HIV testing.
http://www.statehealthfacts.kff.org/cgi-bin/healthfacts.cgi? action=compare&welcome=1&category=HIV%2fAIDS
Youth Risk Behavior Survey website, includes state-level data and core item content.
http://www.cdc.gov/nccdphp/dash/yrbs/index.htm
Behavioral Risk Factor Surveillance Survey website, includes state-level data and core item content.
http://www.cdc.gov/brfss/index.htm
US Census Bureau site that permits searches on specific information for block, city, county, & state levels.
http://factfinder.census.gov/servlet/BasicFactsServlet
National Neighborhood Indicators Partnership provides local data for a variety of health, social, & economic indicators, for growing number of medium to large cities. Some datasets have capacities for simple analyses (e.g., cross-tabulations by sex, age, etc.).
http://www.urban.org/nnip/partners.html
Return to top
Outcome Monitoring & Evaluation
Methods for evaluating community-level health interventions. Includes links to tools for evaluation of goal attainment, as well as individual and community-level measures and indicators.
http://ctb.lsi.ukans.edu/tools/EN/chapter_1039.htm
Return to top
Cost-Effectiveness Evaluation
National Institute on Drug Abuse (NIDA) website with links to downloadable .pdf documents that deal with fundamentals of cost-benefit & cost-effectiveness, as well as specific examples of cost-effectiveness evaluations with populations relevant to HIV prevention.
http://www.nida.nih.gov/pdf/monographs/monograph176/download176.html
Downloadable UNAIDS document (.pdf) which reviews models of cost-benefit analysis and common approaches for designing cost-effectiveness evaluations, including a review of measures.
http://www.unaids.org/publications/documents/economics/costeffec/costtue.pdf
Return to top
Geographic Information Systems
National Oceanic and Atmospheric Administration (NOAA) webpage with links to a variety of Geographic Information Systems (GIS) tutorials; the tutorials vary in difficulty, but none seem to be designed for a non-technical audience.
http://www.ngdc.noaa.gov/seg/tools/gis/referen2.shtml
Return to top
Plain English Presentation of Data
Tips for presenting research and evaluation findings to a variety of audiences.
http://ctb.lsi.ukans.edu/tools/EN/sub_section_main_1376.htm
http://ctb.lsi.ukans.edu/tools/EN/sub_section_main_1374.htm
Return to top
Practicalities
1.) Developing Consultation Contracts
A .pdf downloadable checklist for developing evaluation consultation contracts (not HIV-specific, but easily adapted).
http://www.wmich.edu/evalctr/checklists/contracts.pdf
A pdf downloadable checklist for writing up an evaluation (not HIV-specific, but easily adapted).
http://www.wmich.edu/evalctr/checklists/kec.pdf
2.) Sampling, Methods, & Measurement
Link to .pdf files that describe basic steps in sampling, including sample size calculation.
http://www.fantaproject.org/publications/sampling.shtml
Glossary of sampling terms; relatively “plain English” definitions, along with concrete examples.
http://www.stats.gla.ac.uk/steps/glossary/sampling.html
Powerpoint slides from a course by Mike Raulin outline. Very good review of measurement concepts including scaling.
http://www.abacon.com/graziano/ch04/index.htm
This presentation is by the same author & reviews threats to reliability. Good review, but more technical than the above.
http://www.abacon.com/graziano/ch09/index.htm
This presentation reviews hypothesis testing & threats to validity. It is similar in technical level to the reliability presentation and a good review of concepts.
http://www.abacon.com/graziano/ch08/index.htm
Bill Trochim’s readable review of reliability concepts; better as a refresher for a professional audience than for non-researchers. Links to pages of similar technical level on scaling, validity, and unobtrusive measurement.
http://trochim.human.cornell.edu/kb/reliable.htm
3.) Survey Basics
On line book that contains basics of survey design (including item development, data entry, etc.); useful as a handout or as a basis for developing TA materials. The website uses a frames format, however, the buttons on the left do not work & you need to scroll through the text (not a big deal). Provides some fundamentals for writing data.
http://www.freenet.tlh.fl.us/~polland/qbook.html
Overview of survey research: item selection, scaling, etc. The content varies from simple to relatively academic, but much of the material would be suitable as background for developing TA materials.
http://www2.chass.ncsu.edu/garson/pa765/survey.htm
Basic terminology & methods used in sampling, with an emphasis on survey applications.
http://www2.chass.ncsu.edu/garson/pa765/sampling.htm
Instruments developed by HRSA contractor, The Measurement Group and used in the Health Resources and Services Administration’s (HRSA) “Special Projects of National Significance”. Instruments cover a variety of topics which include: demographics, barriers/facilitators of care, client satisfaction, TA evaluation, quality of life, HIV knowledge, substance abuse, distress, sexual behavior, partner disclosure, and homelessness, plus citations for other measures that were used in this project.
http://www.themeasurementgroup.com/modules.htm
Evaluation Materials developed by HRSA contractor, The Measurement Group for HRSA Special Projects of National Significance (SPNS) Program Innovative Models of HIV/AIDS care. Organized by project; many could be adapted for prevention programs. Studies include key informant interviews, client interviews, instruments for evaluating training, etc.
http://www.themeasurementgroup.com/resource/eval_dev.htm
Website for CDC’s HIV/STD Behavioral Surveillance Working Group. Contains core survey items on HIV risk behavior (sex & drugs), and HIV testing that were complied by CDC's behavioral surveillance working group. STD and TB items are coming in the future. The site explains how these were selected and developed. Items include those for general and high-risk populations. There's also a program contact listed.
http://www.cdc.gov/nchstp/od/core_workgroup/default.htm
4.) Qualitative Methods
University of Illinois Extension webpage with downloadable .pdf files covering key informant interviewing. Excellent and practical review of the basics in conducting key informant interviews, including a section on summarizing/writing up interviews, though weak on the analysis of interview data. Useful for background and as a handout.
http://www.aces.uiuc.edu/~PPA/KeyInform.htm
Good, brief review of fundamentals for considering and conducting focus groups.
http://www.soc.surrey.ac.uk/sru/SRU19.html
5.) How to Read Scientific Papers
“How to read a paper”: reviews statistics and their applications. Good for the person who has some scientific training and may provide useful background materials for doing technical assistance.
http://bmj.com/cgi/content/full/315/7104/364
http://bmj.com/cgi/content/full/315/7105/422
http://bmj.com/cgi/content/full/315/7110/740
Penn State University webpage that links to a variety of downloadable .pdf “tip sheets” on a variety of topics ranging from how to make a pie chart to sampling diverse audiences to asking about age. The sheets vary in length & quality. All include helpful reference citations.
http://www.extension.psu.edu/evaluation/titles.html
6.) Glossaries
Return to top
Integrating Data, Methods & Program
Reprint of an Annual Review of Public Health lit review on sampling and its implications for findings. Rather academic, but a useful background document for developing technical assistance materials.
http://www.sph.unc.edu/chsr/Dissemination/Arph_sub.htm
Return to top
Program Development Resources
"REP Plus": This link goes to CDC's Replicating Effective Programs (REP) project which is looking at replication/translation of interventions with demonstrated effectiveness. The individual REP projects are described in terms of contacts and core elements. The site reviews other CDC resources (with links to a compendium of effective prevention programs,
http://www.cdc.gov/hiv/projects/rep/compend.htm ; Characteristics of Reputationally Strong Programs (C-RSP) Project, http://www.cdc.gov/hiv/projects/rep/crspproj.htm and a link to the APA Behavioral and Social Science Volunteer (BSSV) program.
www.cdc.gov/hiv/projects/rep/default.htm
This link goes to the UNAIDS "Best Practices" site which provides examples of "best practices" for a variety of topics (care, prevention, epidemiology, human rights, etc.). Prevention topics are better served by the population links (e.g., children) than the "care and prevention" link. The links go to project-related monographs, topical monographs and proceedings from meetings. The writing varies greatly from concise and useful to the endlessly ponderous.
http://www.unaids.org/bestpractice/collection/subject/index.html
CDC compendium of effective HIV prevention programs
http://www.cdc.gov/hiv/pubs/hivcompendium/hivcompendium.htm
CDC Division of Adolescent & School Health’s webpage for “Programs That Work”. Provides contact information to receive documents; the “Programs That Work” is undergoing administrative changes.
http://www.cdc.gov/nccdphp/dash/rtc/index.htm
The Center for Substance Abuse Prevention’s (CSAP) listing of “Model Programs” for drug & alcohol prevention.
http://www.samhsa.gov/centers/csap/modelprograms/programs.cfm
Principles for drug abuse treatment programs (includes attention to HIV/AIDS risk).
http://www.samhsa.gov/centers/csat/csat.html
Prevention and care programs for HIV-positives. Successful program principles, as well as sample programs from various HRSA-funded activities.
http://www.themeasurementgroup.com/hivbttn.htm
The Center for Substance Abuse Prevention (CSAP) Decision Support System; logic modeling is presented as a circular (recursive) seven-step process beginning at Assess Needs and progressing through Develop Capacity, Select Programs, Implement Programs, Evaluate Programs, Report Programs, and Get Technical Assistance and Training.
http://www.preventiondss.org
| 
