Evaluation Criteria
Each manuscript will be scored on how comprehensively it describes an intervention and reports the appropriate study results. The scoring criteria emphasize the quality of the methodology used to assess the intervention's effectiveness; the specific topic area of the intervention is not scored. While all of the following criteria will be considered in the review of papers for the competition, it is neither anticipated nor required that any single paper will address every criterion:
I. Description of the need for the intervention, including: a) the health and safety problems addressed, and b) the injury and illness consequences for workers. Important note: severity of the problem is not a scoring criterion so as to keep an intervention to prevent fatalities from displacing a good evaluation of a comprehensive health promotion program.
II. Description of the intervention in the workplace, including: a) basic descriptive information regarding the intervention, the context in which it is implemented, and any study objectives or hypotheses; b) the organizational level(s) of workplaces included in the intervention; c) the process of development of the intervention, such as participation of workers and management and of the research/evaluation team in developing the intervention; and d) innovative approaches and noteworthy methods of addressing a problem.
III. Evaluation study design and methods (i.e., the plan for implementing the intervention and evaluating the implementation), including: a) the overall quality of the research design; b) ethical considerations, such as informed consent and other aspects of a typical IRB review; c) population, sample, recruiting participants, assignment to groups, and estimates of power; d) measurements (both quantitative and qualitative), including both intervention efficacy and effectiveness measures; and e) a plan to permit process/efficacy evaluation results to modify the intervention.
IV. Evaluation study results and analysis, including a) the quality and comprehensiveness of the results; and b) the quality of the results with respect to the overall findings and conclusions, especially the link between effectiveness measures and the overall findings. Important note: Significant results are not a prerequisite for this competition. Negative results, null results, and significant results may all be reported.
V. Implications of the evaluation results for occupational health psychology, including placing this study into the broader context of occupational health psychology, b) limitations of the intervention, c) recommendations for improvements to the intervention, d) recommendations for improved evaluation studies, and e) new research questions.
For additional information on the development of the competition criteria and scoring details, please refer to:
Scharf, T., Chapman, L., Collins, J., Limanowski, J., Heaney, C., and Goldenhar, L. (2008).
Intervention effectiveness evaluation criteria: Promoting competitions and raising the bar.
Journal of Occupational Health Psychology. v.13, no.1, pp.1-19.
As noted in Scharf, et al., (2008), the purpose for the criteria is to challenge and test them against emerging standards of the very best intervention evaluation practices. The review sheet has been adjusted from its 2008 version based on reviews from the competition at the National Occupational Injury Research Symposium (NOIRS), 2008, 2011 and the Work Stress and Health 2009 and 2011 competitions. The current score sheet is attached (PDF, 44KB). Please note: the Best Intervention Competition reviewers reserve the right to continue to adjust the relative weights on the attached score sheet until the reviews actually commence. Please consider using the attached as a content checklist while preparing your competition manuscript. Please do not use the attached score sheet as a template to write your manuscript.
Please submit your manuscript in English. Although conference presentations will take place in both English and Spanish, the Best Intervention Competition does not have the resources necessary to conduct a professional bi-lingual review with good inter-rater reliability.
Competition Procedures and Timeline
Electronic submissions are preferred, but not required. Print submissions will also be accepted.
October 1, 2012
Submit your 600-800 word abstract along with the proposal cover sheet and other required materials to the conference coordinator at APA. Check the box for the Best Intervention Competition on your cover sheet. Include one extra copy of the abstract for the competition if submitting in print form. Your abstract will remain confidential; it will be used to assign reviewers for the competition. Complete and submit the Best Intervention Competition Statement (below).
October 1, 2012 - January 7, 2013
Complete and submit to the conference coordinator a full manuscript detailing the intervention and its evaluation. Submit a draft of the manuscript describing the intervention evaluation. (Refer to the submission criteria for guidance.)
Every effort will be made to provide quick feedback on abstracts submitted to this competition. However, please do not wait to hear from APA to prepare your manuscript for this competition. Conference planning decisions regarding acceptance of abstracts will not be made until after the manuscript submission deadline.
Note: There is no required minimum or maximum length for manuscripts for this competition. Extremely short submissions (e.g., 2-5 pages) are likely to lose points simply because they do not have the capacity to address all the relevant criteria. This competition encourages students to submit theses and dissertations describing evaluations of interventions. Therefore, authors planning to submit manuscripts substantially longer than a typical journal article or book chapter are asked to contact the intervention competition coordinator.
January 7, 2013
Deadline to submit a draft of the manuscript describing the intervention evaluation. Electronic submissions are preferred. Printed copies will also be accepted if they are postmarked no later than January 5, 2013, and are sent via Air or Express Mail. If submitting in printed form, please include four copies of the manuscript.
Send your manuscript to the Best Intervention Competition Coordinator:
Ted Scharf
Best Intervention Competition Coordinator
National Institute for Occupational Safety & Health
4676 Columbia Parkway C-24
Cincinnati, Ohio 45226
January 7, 2013 – March 31, 2013
Your manuscript is peer reviewed for the Best Intervention Competition (see the competition criteria, above). Manuscripts will be held in confidence.
April 2013
The competition committee provides detailed manuscript feedback to all competition participants, and finalists are notified. Finalists will be acknowledged in a special awards session during a reception at the Work, Stress, and Health 2013 conference. All competition participants will be placed in sessions appropriate to their topic areas.
All participants are encouraged to submit their manuscripts to the Journal of Occupational Health Psychology. Submissions to JOHP are separate from this competition and will undergo the normal journal review.
Best Intervention Competition Submission Statement (PDF, 111KB)
I request that the enclosed abstract and the subsequent manuscript be considered for the Best Intervention Competition at the Work, Stress, and Health 2013 conference. Please check all that apply:
| ____ | The enclosed abstract, as well as the manuscript to be submitted by January 5, 2013, have not been previously published or presented. (Presentation at the Work, Stress, and Health 2013 conference must be the first scientific presentation.) |
| ____ | NO author of this manuscript is a member of the Work, Stress, and Health 2013 conference planning team or one of the competition reviewers. |
Signed:_____________________________________________ Date: ___________________
