Placing clinical practice guidelines in context

APA’s clinical practice guidelines are designed to help patients and their families, practitioners, policy makers, and administrators recommend and select treatments based on a systematic review of the scientific evidence to determine what treatments work best, on average, for a given disorder or problem.

The recommendations made in clinical practice guidelines are developed by synthesizing four types of information:

1. A systematic review of evidence about efficacy of treatments, where the efficacy data primarily derive from randomized controlled trials (RCTs; and panels also have the option to include results of carefully-controlled observational studies).
2. Data about risk of harms from treatments.
3. Data about patient preferences and values regarding treatments.
4. Data about applicability of treatments across populations (e.g., gender, ethnic, and age differences) and settings (e.g., inpatient, outpatient), etc. The systematic review of efficacy is typically the primary source of data because of the scope and depth of evidence the review provides and the methodological rigor used to evaluate the research. Also, there are usually fewer empirical studies available to provide the other types of information (e.g., limited studies of patient preferences).

Clinical practice guidelines are intended to be a central—but clearly not the only—determinant of treatment choice. To help place the guidelines in context, this supporting document discusses the strengths and limitations of basing clinical practice guidelines mainly on results of well-controlled clinical studies, and highlights some of the other factors that patients, practitioners, policy makers, and administrators will also want to consider as they recommend, select, and implement treatments. These include the importance of attending to information about patients’ preferences and histories, providers’ expertise and judgment, nonspecific factors in psychotherapy, and patients’ individual differences. Other factors are not discussed, including constraints of the practice setting or organizational context. The interested reader is encouraged to see recent versions of evidence-based practice models that outline the full range of factors that influence decision making in clinical care (e.g., Légaré et al., 2008, and Satterfield et al., 2009, among many other helpful examples).

APA’s clinical practice guidelines follow the Institute of Medicine (IOM, now National Academy of Medicine) standards for developing practice guidelines, and thus rely on data from high quality, independent systematic reviews (also conducted according to IOM standards; IOM, 2011a, 2011b; and, on occasion, include reviews that have been evaluated as sufficiently high quality following the Assessing Methodological Quality Of Systematic Reviews, or AMSTAR, process; Shea et al., 2007). These systematic reviews largely use data from RCTs to determine efficacy of interventions, though the option exists to include some high quality observational studies. (Note, while carefully constructed single-subject designs may be used to provide context for the panel’s recommendations, these designs are not typically part of systematic reviews that follow IOM standards.)

There are several advantages to this strategy. First, the RCT design allows the investigator to conclude that differences in outcomes between treatments that have been randomly assigned to participants are very likely caused by the treatments rather than by extraneous factors.

Second, the use of IOM standards is consistent with best guideline development practices across health care professions, lending credibility to the clinical practice guidelines. In fact, APA’s clinical practice guidelines are submitted to the ECRI Guidelines Trustopens in new window, a database established in 2018 by an independent, nonprofit healthcare organization to continue the legacy of the Agency for Healthcare Research and Quality’s National Guideline Clearinghouse. Through the submission process, ECRI rates for how well the guidelines follow the IOM standards utilizing the TRUST (Transparency and Rigor Using Standards of Trustworthiness) Scorecard. Health care administrators, insurance companies and other important stakeholders consult the ECRI Institute’s Guidelines Trust when making coverage, service delivery, reimbursement and other decisions.

Third, a reliance on efficacy data from RCTs to guide clinical practice guideline recommendations minimizes the impact of unintended biases when selecting a treatment (e.g., a reliance on efficacy data protects the provider from relying too heavily on whether the last patient a provider treated happened to respond well to a given treatment when making treatment recommendations for a future patient).

Fourth, a review of efficacy data allows busy providers and their patients to learn efficiently what has been shown to work for hundreds or even thousands of patients with related problems.

Nevertheless, RCTs do not answer all questions that consumers and providers want to answer. RCTs best answer the question: Which of the treatments studied in the RCT is most efficacious for the average patient who met the selection criteria for the trial? However, the RCT does not fully answer the question that the provider typically wants to answer: What treatment is likely to be most effective for this patient who is in my office at this moment (and who might not meet the selection criteria used in the RCTs)? Another limitation of RCTs is that they typically study changes in symptoms or diagnostic status, whereas patients frequently have additional idiographic symptoms and goals. Moreover, RCTs emphasize efficacy and typically do not test the mechanisms underlying the treatment approach, so guidelines leave unanswered critical questions about how treatments achieve their effects. This is a critical issue facing the field and an important consideration for future guidelines because relatively little is known about whether the mechanisms that are theorized by the specific treatments to be responsible for the effects of the treatment actually are the mechanisms that produce the effects. It is also worth noting that RCTs generally evaluate interventions as a whole. As a result, when clinical best practices involve multicomponent interventions, it is often unclear which are the necessary and sufficient sub-components (for this reason, Comparative Effectiveness Trials can be especially useful for guiding the selection of specific components).

Thus, as outlined by the IOM (2001) and the APA Presidential Task Force on Evidence-Based Practice (APA, 2006), in addition to considering efficacy data from reviews of the best available research evidence (typically RCTs), it is also important to consider the individual patient and their particular needs, background, and wishes, and the provider’s training, expertise, and judgment when selecting and implementing a treatment. We also recognize the role of nonspecific factors in therapy outcomes, and the role of patients’ individual differences in determining clinical outcomes. Each of these considerations is discussed in turn.

When discussing treatment options with patients, providers are encouraged to share the clinical practice guideline recommendations with them so they can be informed about what current research indicates are efficacious interventions. Providers and patients also will want to determine whether there are pressing reasons to consider alternative treatment options based on patient preferences, values, readiness for change, or circumstances. For instance, there may be barriers in a patient’s life that would make completing a guideline-recommended treatment unusually challenging or undesirable, or a patient may not have responded well to a given guideline-recommended treatment in the past. In the latter case, the provider will want to assess carefully to determine whether the patient received an adequate dose of the treatment and that the treatment was provided with fidelity before assuming the treatment will or will not be helpful. Attention to these factors can help the provider individualize a treatment recommendation and tailor it to address the patient’s unique needs.

Provider training, experience, expertise, and judgment are also relevant factors when recommending, selecting, and implementing treatments. Provider expertise is defined as “identifying and integrating the best research evidence with clinical data (e.g., information about the patient obtained over the course of treatment) in the context of the patient’s characteristics and preferences to deliver services that have the highest probability of achieving the goals of therapy.” (APA Presidential Task Force on Evidence-Based Practice, 2006, p. 275). Thus, the provider may be aware of characteristics about the patient, treatment setting, available resources, provider’s competence, or environmental context, among other factors, that suggest that a treatment recommended by the guideline might not work as well for a given case. Clinical practice guidelines are not a rigid prescription to be followed blindly, but are intended to make the process of treatment planning easier by providing a starting point for treatment recommendation and selection based on knowing which treatments tend to work well for a given problem area.

Because well-controlled clinical trials typically examine specific treatments, they generally have little to say about the degree to which nonspecific (also referred to as “common”) factors, such as the patient-provider relationship, patient motivation, provider’s allegiance to the treatment, provider’s empathy, patient expectations and preferences, and other factors that are not specific to a particular treatment, contribute to outcomes. But we know that nonspecific factors are quite important to treatment outcomes, and it is just as important to create a strong alliance, provide empathy, strengthen patient motivation, and address other nonspecific factors when following recommendations from clinical practice guidelines as when following any other treatment approach. Implementation of a treatment by a provider who lacks empathy is likely to be less effective, regardless of the research evidence for the treatment. Thus, nonspecific factors are the foundation for good delivery of any treatment and following clinical practice guidelines is entirely compatible with recognizing the important role of nonspecific factors. Moreover, while most available studies have not separated the effects of nonspecific from specific factors, this is certainly possible to do (e.g., Cuijpers et al., 2012), and we see this as an important direction for future research—and in turn, future guidelines.

Recommendations included in clinical practice guidelines are based on the available well-controlled clinical studies, but unfortunately, these studies often have not assessed whether or how individual differences, such as demographic and identity factors (e.g., race, ethnicity, language, nationality, socioeconomic status, religion, gender identity, sexual orientation, age, physical disability, etc.), diagnostic factors (e.g., medical and psychiatric comorbidity), environment (e.g. safety, housing, access to health care, education, nutrition, and/or transportation), and other individual patient-level factors (e.g., patient preferences about treatment, prior treatment experiences), affect clinical outcomes. That said, the current clinical practice guidelines made a formal effort to consider any available research that indicated that a treatment might work differently across various populations to help providers and patients make decisions about how to integrate information on individual differences in treatment planning. Further, the field is making advances in “personalized medicine” (e.g., by constructing algorithms that select the optimal treatment for a given patient), and so we are hopeful that future guidelines will be better able to consider individual differences in their recommendations.

To individualize treatments that have been shown to be efficacious in RCTs, the provider will want to work collaboratively with the patient and consult the research literature, experts, or guidelines relevant to the particular individual’s characteristics (e.g., APA's Guidelines for Psychological Practice with Sexual Minority Persons (PDF, 1.5MB)opens in new window, APA, 2012) to identify any modifications that might be needed to a guideline-recommended treatment to address the patient’s unique circumstances. More research is urgently needed to guide providers in determining when modifying a treatment will improve versus worsen outcomes. This makes it especially important for the provider to monitor patient outcomes during the course of treatment to assess the patient’s response and inform treatment adjustments.

It is important to underscore that guidelines reflect the state of the research at a given time point. Rather than stifle innovation in treatment development and delivery, clinical practice guidelines should continue to be updated to incorporate new evidence on older treatments and recognize new efficacious treatments. A clinical practice guideline is likely to miss new interventions that were still in the early testing phases such that the research literature for that approach was not sufficiently well-established to meet the rigorous standards for inclusion in the systematic review and subsequent guideline. Thus, it is essential to update guidelines regularly (on an every five-year basis, according to the IOM standards) to reflect the evolving literature base and encourage testing of new intervention approaches.

Relatedly, we recognize and encourage others to acknowledge that the absence of evidence for efficacy does not mean the absence of efficacy; it may simply mean that insufficient research has been conducted. “Absence of evidence is not evidence of absence,” and there is a distinction between an untested treatment and one that has been tested and found to be ineffective. These two statuses (untested vs. ineffective) raise different issues for providers and patients. For treatments that have been compared in efficacy trials, it is typically recommended that providers first select a treatment that has been shown to be helpful, if one is available. For a treatment that has not been tested, this raises pressure to test that approach so that its efficacy can be evaluated. At the same time, there is a role for treatments that are not supported by efficacy data, especially when patients have failed to respond to the evidence-based treatments. A treatment that is less effective than another, on average, might still work well for a given individual and can especially make sense to try when the treatment that works better on average has been shown not to help that given individual. Also, untested treatments can be a valid treatment option in the context of the patient’s full informed consent, a documented treatment rationale, and careful monitoring of the patient’s response. In this way, treatment innovation and experimentation are not discouraged, but new treatments are ultimately expected to be tested.

Provision of high quality, effective care that reduces symptoms and helps the patient return to higher levels of functioning is a shared goal for patients and their families, practitioners, policy makers, and administrators. But knowing which information to trust and prioritize is not easy, and synthesizing available research and determining preferable care options can be daunting. Clinical practice guidelines are an essential tool to guide clinical decision-making and make it easier for patients to get the care they need.