Client Satisfaction: The Easiest Outcome to Measure
Michael Brickey, Ph.D.
Editor's Note: This is the last article of a four part series on Outcome Measurers by Associate Editor Michael Brickey.
One of the outcomes of most interest to lay audiences (which includes clients and employers) is client satisfaction. It is the easiest and most common form of outcomes measurement.
Why do them
Satisfaction surveys are valuable to independent practitioners because:
 | They are effective marketing tools. |
| They are the least expensive way of obtaining some type of outcomes data. |
| Since the advent of consumerism and continuous quality improvement, it's the way business is conducted in America (consider how many restaurants and other businesses have satisfaction cards for consumers to fill out). |
| Satisfaction surveys can identify aspects of your services or practice that your clients do not like and that may cause you to lose clients. |
| Surveys can be a valuable source for quotes (with attention to consent and ethical principles). |
What they measure
Satisfaction surveys appear to primarily measure whether the clients like the therapist and the office atmosphere. Theoretical constructs include: 1) satisfaction is a measure of attitudes, 2) it a measure of fulfillment of needs, and 3) it represents a discrepancy between what was expected and what was received. Note that cognitive dissonance theory suggests that relatively small discrepancies may be adjusted to the expectation to prevent cognitive dissonance.
Satisfaction surveys may just focus on whether the client liked the services and practice or may venture into whether the client believes they got good results (e.g., behavior changes, problem resolution). The Consumer Reports (1995) study, for example, solicited retrospective information on symptom improvement in addition to general satisfaction. Research on the correlation between client satisfaction and behavioral outcomes have correlations of less than .40 (Lunnen & Ogles, 1997). This may not seem strange when we consider that many physician visits do not result in cures but the patient may still believe the physician was very competent and provided excellent services. Research suggests that general life satisfaction does not appear to have much of a correlation with satisfaction ratings. The provider's perceived competence, however, does correlate positively with satisfaction. There is a modest correlation between length of therapy and satisfaction.
Managed care organizations place a lot of emphasis on satisfaction surveys because the data are easy to collect, easy to understand, and liked by employer. Further, the data are usually very positive. All the managed care organizations report very positive data on consumer satisfaction. Many surveys have low return rates and clients who choose to respond to surveys may be a selective population. There also is the possibility that these reports selectively report only the most highly rated items. Accreditation organizations like the National Committee on Quality Assurance (NCQA) emphasize data collection on satisfaction measures such as promptness in getting the initial appointment, promptness in answering phone calls, 24 hour accessibility, and the office environment. These fit well with the managed care agenda and are measurable. To try to determine changes in behavior, health, and overall functioning, managed care organizations use standardized instruments like the Brief Symptoms Inventory (BSI) as well as proprietary instruments.
Validity concerns
There are many issues that may compromise validity. These include:
| Clients may want to please/not alienate their therapist for fear that unfavorable ratings may foster uncomfortable interactions or compromise therapy. |
| They may have similar concerns about pleasing/not alienating support staff. |
| Cognitive dissonance may make them reluctant to acknowledge dissatisfaction when they are continuing to spend time, money, and effort on treatment. |
| Clients may be concerned about what will be done with the data. |
| The wording of items may invite an acquiescent response set. While this could be balanced by mixing positively and negative worded items, such mixing would make the instrument more complicated. Many clients do not carefully read surveys and a complicated structure invites errors. |
| Clients who are surveyed early in treatment (before they know much about the treatment) may be making less informed responses. Surveying only clients who have had several therapy sessions excludes clients who were dissatisfied and gave up on therapy or went elsewhere. |
| Clients who are too busy or not interested in responding to the survey introduce a bias as do clients who are dissatisfied and may chose to not respond or be especially motivated to express their dissatisfaction. |
| The evaluator may selectively report results. |
| Different populations may have different biases and response styles, e.g., borderline clients, teenagers who are coming at a parent's insistence, custody evaluation cases. Because higher income clients generally are able to afford more and better care and have more choice, they may be more pleased with their choice. |
| Presumably, satisfaction is a continuous scale. Labels, such as satisfied, neutral, and unsatisfied, may not accurately represent the distribution of raw scores along that continuum.. |
Developing a satisfaction survey
Keep the survey simple and to one page.
| Make sure the survey can be filled out in five minutes or less. |
| Use a Likert-style scale with at least five (preferably seven) choices. |
| Use a coding system so clients don't have to fill out demographic information but you can access such data for analyzing the results. It is important to identify the dissatisfied clients as even a 95% approval rating means 5% of clients are dissatisfied and may seek services elsewhere. On the average, dissatisfied clients tell nine other people about their dissatisfaction. |
| Include an open-ended item or two or an "any other comments" item to elicit qualitative responses and possible quotes. |
| Have the client to fill out the survey immediately after the session (as this increases the return rate, avoids delays in responding, and avoids postage expenses). |
For most practices, once a year surveys are sufficient.
In designing a survey, a good starting point is to gather surveys from a variety or organizations including behavioral healthcare practices, and to note what you like and do not like. Then decide what your purpose is in conducting the survey, e.g., gathering impressive marketing data, finding out what your clients really think, continuous quality improvement, providing data for a quality assurance program. Have several colleagues and clients review the survey to see if it is well designed and measuring what you want it to. Decide how to conveniently administer the survey and get a high return rate, e.g., if your receptionist is the key person, how do I get him or her enthusiastic and committed to doing the survey? One option is to include an item asking the client to cite the practice's two main strengths and two areas to improve. In offices with more than one therapist, consider an item for the client to check whether it is OK to share responses with the clinician.
While satisfaction surveys should not be confused with clinical effectiveness outcomes measures, they are an important outcome measure in their own right and the easiest, least expensive form of outcomes measurement.
References
Consumer Reports. (1995). Mental health: Does therapy help? 734-739.
Lunnen, Kirk, & Ogles, Benjamin. (1997). Satisfaction ratings: Meaningful or meaningless? Behavioral Healthcare Tomorrow, 6, 49-51.