Updated Standards for Educational and Psychological Testing is released for public comment
By Wayne Camara and Marianne Ernesto
The American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME) have collaborated on the joint development of Standards for Educational and Psychological Testing (hereafter referred to as the Standards) since 1966. There have been four revisions of these joint standards since they were first issued as separate technical recommendations for achievement tests and psychological tests by AERA and NCME in 1955 and by APA in 1954. The most recent edition of the Standards was published in 1999 and is currently being revised with an expected publication date of late 2012 or early 2013.
The primary purpose of the Standards is to provide criteria for evaluating tests and testing practices. The Standards apply broadly to a wide range of standardized instruments and procedures that sample an individual’s behavior, such as tests, assessments, inventories, and scales. The main exceptions in applying the Standards are unstandardized questionnaires (e.g., unstructured behavioral checklists or observational forms), teacher-made tests, and subjective decision processes (e.g., a teacher evaluating classroom participation over the semester). The Standards apply as equally to standardized multiple-choice tests as they do to performance assessments (including tests comprised only of open-ended essays) and hands-on assessments or simulations.
There is no mechanism to enforce compliance with the Standards on the part of the test developer or test user. They do not attempt to provide psychometric answers to policy or legal questions. However, the Standards have been referenced in federal law and cited in Supreme Court and other judicial decisions, lending additional authority to the document. For example, they have been cited in the Goals 2000: Educate America Act1 and Title I of the Elementary and Secondary Education Act2. They were also cited in several major court decisions involving employment testing, including a Supreme Court case in 19883.
For the past three years, AERA, APA and NCME have sponsored the revision of the 1999 Standards. As reported previously, APA in particular has supported the activities of the members of the Joint Committee (a group of well-known content experts appointed in 2008 to revise the Standards.) The Joint Committee has met four times a year for the past two years, and three meetings are currently scheduled to take place in 2011.
Primary differences between the revised draft of the Standards and the 1999 edition include the separation of the general text into “foundations,” “operations,” and “applications” sections; a condensation of several individual chapters dealing with issues of fairness into a single “foundations” chapter; the inclusion of topics such as educational accountability and technological advances in testing; and the re-organization of chapters concerning workplace testing and credentialing.
As noted, the general text of the revised draft Standards has been separated into three sections. The “foundations” section focuses on fundamental testing issues such as validity, reliability, and fairness. The “operations” section deals with operational testing issues such as test design and development, test administration, scoring and reporting, and supporting documentation for tests. The “applications” section explores issues related to workplace testing and credentialing, educational testing and assessment, and the policy implications of test use. Each chapter in the new “foundations” and “operations” sections contains an introductory/overarching standard as well as individual standards and themes. In an effort to ensure continuity and a sense of coherence, individual standards and themes are presented in the order that they appear in the introductory/overarching standard.
The decision to condense the multiple chapters that deal with fairness issues in the 1999 Standards into one “foundations” chapter in the new edition was made after members of the Joint Committee concluded that the issue of fairness was so fundamental to testing practice, it should be considered as a foundation of testing along with validity and reliability. This change means that the revised Standards will contain an expanded discussion of fairness, and the total number of chapters will decrease from the current total of 15 to 13.
The revised draft of the Standards was released for public comment January 10, 2011. An on-line template that allows individuals and organizations to review the draft document and to submit comments and recommendations is posted on the Standards website under the heading “Revisions.” All comments and recommendations must be submitted to the Joint Committee through this website by April 20, 2011.
1PL 103-227 – Goals 2000: Educate America Act – Sec. 211 Purpose states “the National Education Standards and Improvement Council shall (5) certify State assessments submitted by States or groups of States on a voluntary basis, if such assessments – (A) are aligned with and support State content standards certified by such Council; and (B) are valid, reliable, and consistent with relevant, nationally recognized, professional and technical standards for assessment when used for their intended purposes.” The Federal Register (43, pp. 38290-38315) defines assessment under the Goals 2000 Act as, “ASSESSMENT – Any method used to measure characteristics of people, programs, or objects. (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. . Standards for educational and psychological testing. Washington, DC: American Psychological Association).”
2Title I – ESEA (Guidance on Standards, Assessments, and Accountability) “Title I requires that the assessment system be used for purposes that are valid and reliable and that it meet nationally recognized professional and technical standards…The primary reference for technical quality of educational assessments is Standards for Educational and Psychological Testing (1985), developed by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education.”
Wayne Camara Is Vice President of Research and Analysis at The College Board and serves as both APA representative and chair on the Management Committee for the revision of the Standards. Marianne Ernesto is Director for Testing & Assessment in the APA Science Directorate and serves as the project director for the revision of the Standards.
PSA is a free monthly email publication from the APA Science Directorate. If you’re not already receiving PSA, request your free subscription now!