What APA policies serve as the bases for the development of clinical practice guidelines?
In 2010, the Council of Representatives approved a motion that called for APA to develop clinical practice guidelines and established an Advisory Steering Committee for Development of Clinical Practice Guidelines to design and oversee a process for guideline development, including selection of topics. The original members of the Advisory Steering Committee were appointed by the Board of Directors, following a call for nominations and review of candidates by the Board of Professional Affairs, the Board of Scientific Affairs, and the Committee for the Advancement of Professional Practice. The Feb. 19-21, 2010, Council of Representatives agenda item (PDF, 66KB)opens in new window uses the original term “treatment guidelines.”
The Advisory Steering Committee began its work by reviewing earlier APA policies. In 1995, APA approved the “Template for Developing Guidelines: Interventions for Mental Disorders and Psychosocial Aspects of Physical Disorders,” which was later revised and became the “Criteria for Evaluating Treatment Guidelines” (2000). These criteria are highly similar to the recommendations for guideline development put forward by the Institute of Medicine (2011), which the Advisory Steering Committee has used to guide APA’s approach to guideline development.
In 2005, the council approved a “Policy Statement on Evidence-Based Practice in Psychology” that defined such practice as “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.” Clinical practice guidelines are understood as a tool for synthesizing and disseminating the “best available research” on the efficacy of interventions, thus contributing to clinical decision-making and evidence-based practice. This policy serves as the general framework for the Advisory Steering Committee’s work.
How do clinical practice guidelines differ from professional practice guidelines?
APA develops two types of guidelines: clinical practice guidelines (CPGs) and professional practice guidelines (PPGs). Both types of guidelines are aspirational and consist of recommendations to practitioners to assist in the delivery of high quality care. CPGs provide specific recommendations about treatments for particular health conditions or diagnoses, while PPGs address areas of practice such as working with women and girls, multicultural practice, or telepsychology.
Both types of guidelines rely on the scientific and professional literature to inform recommendations. However, CPGs are founded on systematic reviews of evidence focused on key clinical questions, while PPGs may rely on consensus and familiarity with the body of literature but are not necessarily developed on the basis of systematic reviews.
How are APA’s Multicultural Guidelines related to clinical practice guidelines?
The Multicultural Guidelines: An Ecological Approach to Context, Identity, and Intersectionality (2017) are one of APA’s professional practice guidelines. They challenge psychologists to understand both themselves and their patients from a layered, ecological perspective that emphasizes the intersectionality of identity and roles across diversity, settings and interactions.
As noted on page 13 of the Multicultural Guidelines, they are intended to complement clinical practice guidelines and other professional practice guidelines. The Multicultural Guidelines and other guidelines each serve as a source of particular information and recommendations for the conduct of evidence-based practice (“the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences”).
How was it determined that APA would follow Institute of Medicine* standards in developing clinical practice guidelines?
The Advisory Steering Committee, established in 2010, was tasked with designing the process that APA should follow for developing clinical practice guidelines. The ASC reviewed previous APA policies, including the “Criteria for Evaluating Treatment Guidelines” (2000), and noted its similarities to the then newly released Clinical Practice Guidelines We Can Trustopens in new window (2011) from the Institute of Medicine.
The Advisory Steering Committee also investigated the guideline development processes of the UK’s National Institute for Health and Care Excellence (NICE)opens in new window as well as documents from other health care organizations that develop guidelines and the Guidelines International Networkopens in new window (the major international organization for guideline development).
After studying materials from all of these organizations and assessing directions in the field of guideline development, the Advisory Steering Committee determined that APA should aim to follow the Institute of Medicine standards. A major consideration was the goal of developing guidelines that would be broadly accepted among healthcare practitioners, researchers, and policymakers.
How does APA’s guideline development process differ from that of other organizations?
With the advent of the Institute of Medicine (IOM) standards, most organizations in the U.S. (and many outside the U.S) that develop guidelines aspire to these criteria. However, organizations that were developing guidelines prior to the release of these 2011 standards needed to examine their own development processes and decide how to modify those processes. Not all organizations have fully adopted IOM standards for their guideline development process.
For instance, the IOM standards recommend creation of guideline development panels that include professionals from multiple disciplines as well as community members who have experience with the disorder or condition. At this point, the American Psychiatric Association (ApA) generally does not include professionals, other than psychiatrists, or community members on its guideline development panels. By contrast, APA’s panels are multidisciplinary and include community members as full panel members.
IOM standards also call for guideline recommendations to be based on systematic reviews of evidence, not just consensus. All APA guideline recommendations are based on systematic reviews (see what is a systematic review below). But guidelines from the ApA and Department of Veterans Affairs may include recommendations that are based entirely on consensus, in addition to recommendations that are based on systematic reviews.
What is the ECRI Guidelines Trust™?
The ECRI Guidelines Trust™ can be found hereopens in new window. It was established by an independent, nonprofit healthcare organization to continue the legacy of the National Guideline Clearinghouse (which closed in July 2018) and meet its mission “to advance effective, evidence-based healthcare globally.” The purpose of The Guidelines Trust is “to provide physicians, nurses, other clinical specialties, and members of the healthcare community with up-to-date, clinical practices to advance safe and effective patient care.” The Guidelines Trust utilizes a rating system called a TRUST (Transparency and Rigor Using Standards of Trustworthiness) Scorecardopens in new window that measures how well the guidelines fulfill the Institute of Medicine’s standards for developing clinical practice guidelinesopens in new window.
* As of March 2016, the division of the National Academies of Sciences, Engineering, and Medicine (the National Academies) formerly known as Institute of Medicine (IOM) was renamed the Health and Medicine Division (HMD). Despite the recent name change, the guideline will use IOM when referring to the IOM standards for guideline development and systematic reviews.
Who serves on the Advisory Steering Committee?
For 2024, the members of APA’s Advisory Steering Committee for Development of Clinical Practice Guidelines are:
- Chair: Claire Collie, PhD, Veterans Affairs Central Office (2018-26)
- Vice chair: Brandon Gaudiano, PhD, Butler Hospital and Brown University (2019–26)
- Marni Axelrad, PhD, ABPP, Texas Children’s Hospital—Houston (2019–26)
- Emily Becker-Haimes, PhD, University of Pennsylvania Perelman School of Medicine (2024–27)
- Michael Constantino, PhD, University of Massachusetts, Amherst (2024–27)
- Heidi Levitt, PhD, University of Massachusetts, Boston (2023–26)
- John McQuaid, PhD, Palo Alto VA Health Care System and University of California, San Francisco (2023–26)
- J. Christopher Muran, PhD, Adelphi University (2018–26)
- Karen Tao, PhD, The University of Utah (2024–27)
- Jonathan Weinand, PhD, Community Health Centers of Southeastern Iowa (2024–27)
Members of the Advisory Steering Committee are appointed for three-year terms (renewable once) by the Board of Directors, following broadly publicized calls for nominations and vetting by the Board of Professional Affairs and Board of Scientific Affairs. (Due to changes in its governance focus, the Committee for the Advancement of Professional Practice is no longer involved in selecting members of the committee.)
How are the members of guideline development panels selected?
At APA, guideline development panels are appointed by the Board of Directors, following broadly publicized calls for nominations. Candidates are vetted by the Advisory Steering Committee, Board of Professional Affairs, and Board of Scientific Affairs, which together submit a set of nominees to the Board of Directors.
The panels of about a dozen members consist primarily of psychologists but also include researchers or practitioners in other disciplines with relevant expertise as well as knowledgeable community members who are personally affected by the condition being addressed. The aim is to have a variety of backgrounds, perspectives, and experiences represented on each panel. A comprehensive assessment of conflicts of interest (PDF, 292KB)opens in new window (financial and intellectual) for all panel members is conducted at least once per year, with appropriate steps taken to manage any conflicts.
What is a systematic review?
Systematic reviews are a primary (but not the only) source of information for the development of clinical practice guidelines. A systematic review is a methodical search for and assessment of studies of the efficacy and comparative effectiveness of particular interventions. The search for studies is guided by specific questions that are framed in terms of populations, interventions, comparators, outcomes, timing, and settings.
Studies are located from multiple databases and other sources using a comprehensive set of search terms. All abstracts are then screened and a determination is made whether to review the full article to determine whether the article meets the inclusion criteria for the review. Once studies are identified, the systematic review team evaluates each study on the following characteristics:
- Risk of bias, which covers study design characteristics such as randomization of assignment of participants to conditions, attrition of participants, validity/reliability of outcome measures, and masking of conditions; magnitudes of effects and the relation of magnitudes to amount of intervention; publication bias; and other factors.
- Consistency of results across studies.
- Directness: whether genuine health outcomes (as opposed to surrogate markers) are assessed, and whether comparisons are head-to-head (rather than from different bodies of evidence).
- Precision: how variable the results are (as captured by the confidence intervals around summary effect sizes).
The evidence is then aggregated to address the specific questions and the review team assigns an overall strength of evidence rating (typically high, moderate, low, or very low/insufficient).
Because studies are evaluated on the four criteria above, usually only randomized controlled trials (RCTs) and other studies with rigorous designs are included. Other types of studies are not considered because they usually have low strength of evidence (due, for example, to small nonrandom samples, high attrition, inadequate comparison conditions, and lack of masking of conditions).
See the 2013 systematic reviewopens in new window that was utilized in developing APA’s guideline on the treatment of PTSD. (The guideline development panel updated that systematic review to include research published through 2016.)
Who conducts systematic reviews?
For APA guidelines, most systematic reviews are conducted by federally designated evidence-based practice centersopens in new window. They are independent of APA and of the guideline development panels. This independence helps ensure that the reviews are objective and unbiased, and is in keeping with broadly accepted standards across healthcare research and policy (as described, for example, by the Institute of Medicine).
How are clinical practice guidelines developed?
A guideline development panel begins by considering what questions are key in making treatment decisions and what outcomes are critical in selecting interventions. A panel is not able to compare all identified treatments against one another on all possible outcomes as there will be no information for many comparisons. Instead, the panel focuses on the “critical” outcomes, typically symptom reduction or loss of diagnosis and adverse events. The panel also considers other “important” outcomes such as quality of life and change in other co-morbid conditions. In determining which outcomes to focus on, panel members are asked to consider the importance of each outcome to both providers and patients for decision-making regarding the treatment of the condition or disorder being examined.
For the PTSD guideline, the main critical outcome was symptom reduction, and for the draft obesity guideline, the main critical outcome is change in body mass index. Both guidelines also treat serious harms or adverse events as critical outcomes. These outcomes were selected as critical because the panel determined that they represent the outcomes necessary to decide whether or not to use a particular intervention.
Guideline development panels formulate guideline recommendations on the basis of the following considerations:
- Strength of evidence, as reported in the systematic review(s) consulted by the panel.
- Balance of benefits vs. harms/burdens of interventions.
- Patient values and preferences.
- Applicability or generalizability of the evidence to various populations and settings.
For 2, 3 and 4, evidence from all relevant sources is considered, including non-RCT studies, patient reports, and clinicians’ experience and observations.
The panels follow a formal process to address these considerations for all the interventions included in the systematic review. The end product is the formulation of guideline recommendation statements. These statements recommend for or against the use of particular interventions or types of interventions, either strongly or conditionally.
The panel presents its guideline recommendations within a guideline document, which lays out the purpose and scope of the guideline, describes the guideline development process, and comments on the guideline recommendations and the evidence on which they are based (including noting open questions and research gaps).
See the PTSD guideline document (PDF, 1MB)opens in new window the obesity guideline document (PDF, 892KB)opens in new window and the depression guideline document (PDF, 1.3MB)opens in new window for detailed accounts of those panels’ processes.
How are common/nonspecific factors, including the therapeutic relationship, addressed in guideline development?
Clinical practice guideline recommendations focus on the efficacy or comparative effectiveness of specific interventions. The guideline panels address questions related to how interventions are delivered, including the role of common and nonspecific factors, in the broader discussions that surround the recommendations within the guideline documents. For example, the PTSD guideline document includes coverage of the important role of patient factors and patient-therapist relationship factors in determining treatment success (see pp. 62-64). The obesity guideline document discusses the format and contexts in which interventions are delivered, including the need to prevent and counter weight-related bias and stigma.
Research supports the importance of the therapeutic relationship and it is presumed that psychotherapy interventions recommended in guidelines are delivered in the context of a strong therapeutic relationship.
How are public comments on guideline drafts solicited and responded to?
Clinical practice guidelines follow the review process that all APA guidelines follow. In accord with Association Rule 30-8 Standards and Guidelines, the draft version of each guideline is put out for public comment for 60 days. The call for public comments is widely disseminated to psychological communities and other scientific, practitioner, and policy communities with an interest in the guideline topic. Council members are also encouraged to submit comments during this period. Individuals who expressed an interest at any point in either the specific topic or guideline development are sent an email invitation to review the draft document. Finally, the Advisory Steering Committee and guideline development panel suggest specific reviewers or organizations that also then receive an email invitation to review the draft guideline.
For all guidelines, the comments and the guideline development panel’s responses to comments are included in the materials provided to the Council of Representatives prior to its consideration of the final guideline for approval as APA policy.
For the PTSD guideline, about 900 comments were received (more than 500 raised questions about the handling of EMDR). The panel opted to group the comments by theme and prepare responses to the issues identified around each theme. Additionally, some comments were specific to sections or pages of the document and panel members considered the specific edits suggested in these comments. A document compiling responses to comments (PDF, 1MB)opens in new window was shared with members of council.
For the obesity guideline, fewer than 100 comments were received. All received comments were grouped by theme and responded to, and included in the final public comment document (PDF, 526KB)opens in new window that accompanied the council agenda item.
For the depression guideline, around 120 comments were received. All received comments were responded to and included in the final public comment document (PDF, 1.6KB)opens in new window that accompanied the council agenda item.
Drafts of APA guidelines are also reviewed by staff in the Office of General Counsel for possible risks to the association or members.
How will APA approach development of revisions of guidelines as new research is conducted?
Both systematic reviews and clinical practice guidelines are usually considered to be out of date after five years. APA’s Advisory Steering Committee for Development of Clinical Practice Guidelines will review guidelines as they approach their expiration date and make a determination about whether the existing guideline needs to updated. If a guideline does not need updating, it will be marked with a phrase such as “reviewed and unchanged” or “no update at this time.” Several models exist for revising guidelines, varying in degree and type of update:
- Full update: Comprehensive update (including re-examination of key questions).
- Refresh: Update evidence and recommendations for some but not all key questions.
- Selective/targeted update: Update specific limited aspects of the guideline.
- “Living” guideline: Continuous updating.
The Advisory Steering Committee anticipates that as new systematic reviews are published, guidelines will be assessed to determine if the new evidence suggests possible changes to the recommendations. The committee may also recommend topics to evidence-based practice centers for new systematic reviews in areas of interest to APA.
APA will publicize the research gaps that are identified in each of the guideline documents and encourage scientists to address them. As part of this effort, APA’s government relations staff will advocate for federal science agencies to solicit and fund research proposals on these topics. The new research can then be used in revising APA’s guidelines.
What are the strengths and limitations of RCTs?
Among the central features of RCTs are inclusion of comparison conditions, randomization of participant assignment to conditions, and masking of assignments. As a result, RCTs reduce various forms of experimental bias and provide the clearest evidence for the efficacy or comparative effectiveness of interventions within a population. Also, RCTs often have large samples, which enhances the ability of the study to detect effects of interventions.
RCTs have been criticized for including only certain types of participants (e.g., those of particular demographic backgrounds or without comorbid conditions). However, over the last 20 years, many RCTs have included a wider range of participants in an effort to make the results more applicable to real world clinical practice.
Despite having more diverse samples, RCTs still may not be able to pick up differential effects across subgroups of participants. To answer questions about differential effects, either much larger RCTs or RCTs focused on particular subgroups are required.
Also, although RCTs can indicate whether an intervention works or not, they generally will not be informative about the mechanism by which the intervention works. And RCTs will provide limited information about factors that interact with the intervention to produce specific outcomes in individual patients.
Of course, not all RCTs are of high quality. As described above, RCTs are assessed in systematic reviews using a variety of criteria (risk of bias, consistency, directness, precision). Lower ratings on these criteria may be due to issues related to the design, conduct, or reporting of an RCT. In developing APA’s guidelines, only high-quality RCTs were used to determine the strength of evidence for the efficacy or comparative effectiveness of interventions.
Researchers are exploring other designs, in addition to RCTs, for assessing the efficacy or comparative effectiveness of interventions. These include historical control, case control and single case designs. However, studies using such designs often still carry a high risk of bias and lack generalizability, and are not broadly accepted as providing strong support for efficacy or comparative effectiveness.
What are the characteristics of the RCTs used in developing APA’s guidelines?
Diversity of participants
For the PTSD guideline, 92 studies were included in the systematic review. Sixty studies were conducted in the U.S. and of the remaining 32 studies, 20 were conducted in non-English speaking countries (including Israel, Brazil, Iran, Uganda, Sudan, Croatia, France and Germany). Fifty-one U.S. studies reported the percentage of non-white participants, which ranged from 6.6 percent to 100 percent across studies. Index traumas were varied: participants included individuals with mixed trauma histories, combat experience, motor vehicle accidents, interpersonal violence, sexual abuse, as well as refugees and survivors of the World Trade Center attack. The systematic review team determined that there was not sufficient evidence to identify treatment effects by subgroup.
For the obesity guideline, RCTs were conducted in the U.S., European countries, Turkey, Australia and New Zealand. Many studies did not report the race/ethnicity of their participants. The guideline document calls for future research to explicitly consider the race/ethnicity and socioeconomic status of participants.
Comorbidities
For the PTSD guideline, most RCTs did not exclude participants with comorbid conditions, although trial samples were less likely to include persons with substance use disorders, or who were violent or expressing psychosis, and were more likely to include people with higher baseline severity of PTSD, as compared with community samples of individuals with PTSD. However, the evidence was insufficient to draw conclusions about differential effects of interventions across participants with different comorbidities.
For the obesity guideline, the RCTs included participants with comorbid physical and psychological conditions. However, participants with the following diagnosed conditions were excluded: eating disorders, pregnancy, and obesity secondary to a genetic or medical condition (e.g., hypothyroidism). Again, evidence was insufficient to draw conclusions about differential effects across comorbidities.
Attrition (participant dropout)
For both the PTSD and obesity guidelines, an attrition rate of 20 percent or more (or differential attrition between conditions at 15 percent or more) resulted in the study being rated as having a high risk of bias.
Contemporary RCTs often take attrition into account by using “intention to treat” analyses. Any study that failed to use this statistical approach was also rated as having a high risk of bias.
Studies with a high risk of bias played little or no role in the formulation of guideline recommendations.
Timeframes
For the PTSD guideline, RCTs examined outcomes at a maximum of 12 months following initiation of treatment. The critical outcome considered by the guideline development panel was symptom reduction.
For the obesity guideline, RCTs examined outcomes at 12 months or more following initial assessment. The critical outcome was change in body mass index (BMI) or standardized BMI.
What disclaimers appear in APA’s clinical practice guidelines?
APA’s clinical practice guidelines begin with a disclaimer that includes the following:
“This guideline is intended to be aspirational and is not intended to create a requirement for practice. It is not intended to limit scope of practice in licensing laws for psychologists or for other independently licensed professionals, nor limit coverage for reimbursement by third party payers.
“The term guideline refers to statements that suggest or recommend specific professional behavior, endeavor, or conduct for psychologists or other independently licensed professionals. Guidelines differ from standards in that standards are mandatory and may be accompanied by an enforcement mechanism. In contrast, guidelines are aspirational in intent. They are intended to facilitate the continued systematic development of the profession and to help assure a high level of professional practice by psychologists and other professionals. Guidelines are not intended to be mandatory or exhaustive and may not be applicable to every professional and clinical situation. They are not definitive and they are not intended to take precedence over the judgment of psychologists and other professionals.”
Additionally, following the recommendations, both guideline documents make clear that the recommendations reflect the best available evidence at the time and should not be construed as a standard of care. Rather, guidelines can facilitate decision making and serve as a general guide.
How do health insurance companies use clinical practice guidelines?
In order for health insurance companies to be accredited by the National Committee on Quality Assuranceopens in new window (NCQA), their plans must adopt “evidence based practice guidelines for… at least two behavioral conditions.” (This was required for all plans in 2017 but is specified only for Medicaid plans in the 2018 requirements; see page 2 of 2017 requirements and page 11 of 2018 requirements.)
For example, Blue Cross Blue Shield of Illinoisopens in new window lists clinical practice guidelines on its website. It states that “Clinical Practice Guidelines (CPGs) are meant to serve as general guidelines and are not intended to substitute for clinical judgment in individual cases,” which is consistent with what most guideline documents also state.
Under Behavioral Healthopens in new window, the Blue Cross Blue Shield of Illinois website links to 46 guidelines, the majority of which are from the American Psychiatric Association and two are from the American Psychological Association (APA’s clinical practice guideline on PTSD and professional practice guideline on telepsychology). Psychologists who work at health insurance companies have regularly expressed concerns that few guidelines that focus on psychological interventions are available to include on such lists.
The APA Practice Directorate’s Legal and Regulatory Affairs (LRA) department has received calls from some psychologists about private insurers’ and managed Medicaid companies’ use of clinical practice guidelines. The majority of calls have been about Optum’s 2013 policy of only routinely reimbursing for the CPT code 90837 (60-minute psychotherapy) for certain treatments for three diagnoses. That policy is based primarily on the American Psychiatric Association’s guidelines. LRA’s understanding is consistent with what is observed above: When insurers need to adopt clinical practice guidelines for NCQA accreditation, they will look to other professions’ guidelines if APA has not developed them.
APA’s clinical practice guidelines can also aid advocacy with insurers. LRA received a complaint in February 2018 that a managed Medicaid company was not covering certain treatments for PTSD that are listed in APA’s guideline. LRA intends to cite APA’s guideline in arguing that the company should cover those treatments.
How are clinical practice guidelines used in malpractice suits?
This concern is raised by practitioners across all areas of healthcare. The vast majority of malpractice cases do not involve guidelines. However, in malpractice cases in which the question is whether the practitioner met the “standard of care,” clinical practice guidelines can be one of the factors used to make that determination. Nonetheless, guidelines have not been considered definitive evidence in most such cases.
Practitioners have perceived guidelines as both a “sword” and a “shield” for malpractice cases. The fear is that guidelines will be used as a sword and practitioners will be held to whatever a guideline specifies. Guidelines, however, can also serve as shields if practitioners demonstrate that their behavior was consistent with the guideline.
Several cases in which guidelines have been a factor have relied on how the guideline is defined. As APA’s PTSD guideline states, the guideline is “intended to be aspirational and is not intended to create a requirement for practice” (page vii). Furthermore, APA’s PTSD guideline notes that the application of the guideline “requires collaborative decision making that takes into consideration the unique needs and preferences of the patient including his or her contextual and cultural dimensions and the severity and comorbidity of the PTSD. A clinical practice guideline is a general guide to best practices and serves as an aid in decision making but does not define a standard of care nor supplant clinician judgment” (page 92). Judges consider these disclaimers to be important and have recognized that guidelines do not set standards of care.
The best defense, recommended by various professional associations and attorneys, is to document the treatment rationale, including whether and why particular treatments were considered but ultimately not provided.