Assessing NGO Outcomes at the UN: The Role of I-O Psychology
by Harold Takooshian, Fordham University and APA Representative at the United Nations
From its origin in 1945, the United Nations has been an amalgam of three separate entities. Its Secretariat, composed of governments, which has grown nearly four-fold in number, from 51 nations in 1945 to 192 in 2008; its network of dozens of global agencies with paid staffs works on specific issues--like children (UNICEF), labor (ILO), health (WHO), or science (UNESCO); and its vast network of non-governmental organizations (NGOs) which radiate UN efforts into “civil society”—including cultural and scientific networks, community and religious groups, and trade and professional associations like APA. The registered NGOs have grown 10,000% in number, from 40 in 1945 to 4,000 in 2008, and are granted either “consultative” status (with a voice in UN deliberations) or “observer” status (without voice).
NGOs. From the start, the mission of NGOs has been two-way communication, from the UN to the NGOs, and from the NGOs to the UN. NGOs are normally limited to a team of six UN representatives who are granted a UN badge that gives them free access to much of the UN building in New York City. These mostly volunteer representatives normally spend many hours per month attending weekly briefings, monthly committee meetings, special sessions, and periodic conferences. Compared with government and other organizations, there may be no place on earth with such a high “word -to - action” ratio as the United Nations, where representatives spend countless hours listening and talking, compared with writing or doing.
Outcomes How effective are NGOs in their work? How can individual representatives best gauge the impact of their many hours of effort in the UN? Like all organizations, the UN needs job descriptions and performance criteria for its thousands of volunteers as well as staff. Yet when an I-O psychologist on the APA team failed to find any literature on NGO outcome measures, and asked the then-Director of the UN section on NGOs for any literature on this topic, the Director gave a quizzical look and admitted to being intrigued but uninformed: “What is an outcome?”
Here we see an ideal role at the UN for psychology in general (Takooshian, 2007), and I-O psychology in particular (Berry, Reichman & Schein). Compared with other fields, such as economics and social work, psychologists have arrived late to the UN table. Only a dozen of the 4,000 NGOs today are psychology groups, and most of these gained NGO status at the UN in just the past decade.
By necessity, psychology has developed expertise in assessing outcomes. In the clinic, we promote evidence-based practice, and outcomes to assess our therapies (Van den Bos, 1996). In education, we have crafted learning outcomes for the undergraduate psychology student (Murray, 2002) as well as the quality of entire psychology departments (Hanges & Lyon, 2005). In industry, we have long measured employee performance, using “critical incidents” (Flanagan, 1949) and other quantitative and qualitative methods.
When a few I-O psychologists working with the UN conferred on this issue in October of 2005, the two-fold result was not only a simple, new procedure for each NGO to develop its own outcomes, but a straightforward 4 x 4 model for psychology NGO outcomes, described in Table 1 below, based on the 4 aims of psychology, and 4 kinds of UN activity (the table lists only 3).
Aims. Based on the APA mission statement, psychology today has at least four separate aims, that can be applied to UN work: Science—conducting empirical research or theorizing related to UN goals; Practice—direct service to UN individuals, groups, units; Education—to effectively communicate psychological and other concepts to UN folks; Advocacy—to promote sound empirically-based policies infused with psychologically relevant perspectives and literature through the UN.
Levels. We can also identify four levels of activity at the UN: Process is step 1--purely instrumental activities such as listening, talking, writing interim documents, attending others’ meetings. If successful, these may lead to a more tangible step 2. Achievement—step 2—tangible actions such as publishing reports, providing direct service to others, hosting one’s own meetings, publishing policy statements. If successful, this leads to step 3. Impact—step 3—where others (the UN itself) actually act on the published reports, implement the research, or new programs/ policies/ services. Beyond this, there is a further step. Benefit—step 4 —how much the impact has made a measurable difference on society, in improved well-being. As Amatai Etzioni (1976) has long noted, the size and expenditures of hospital programs are no substitutes for indices of how much these programs have improved the health of their clients. Two things are notable about the use of this simple model in Table 1:
Table 1. A simple 4 x 3 model to chart NGO outcomes and progress
|1. PROCESS-->||2. ACHIEVEMENT-->||3. IMPACT|
|A1: unpublished reports...research/theory committee presentations||A2: published research reports...start newsletter or series||A3: use research for policy...form new offices/programs|
|B. PRACTICE :|
|B1: discuss ways to serve||B2: direct service to staff/grps||B3: implement new services|
|C1: attend/speak at meetings||C2: host symposia/conferences||C3: content in UN programs...recruit/ credential stud interns|
|D1: discuss policies||D2: publish policy statements||D3: UN implement policies|
First, where table 1 applies the first 3 levels to the 4 aims, individual psychology representatives can gauge the progress of their efforts over time. Which of the four aims are they pursuing, and are they spending less time in instrumental activities, more in tangible actions that actually impact UN operations?
Second, this psychology model can be easily adapted to non-psychology NGOs. For example, a social work or pro-environment NGO would enunciate its own aims from its mission statement, and then apply the 4 levels to this. Indeed, when I-O psychologists introduced this 4x4 model to an audience of 60 diverse NGO representatives at a symposium (Verdi, 2006), many saw how this simple process was easily adapted to their own aims at the UN.
A few examples are useful here.
Example 1. A psychology group spends many hours to draft an invited position statement on “child welfare,” but its oral and written testimony at the hearing remains unrecorded and unread. This is not a good day for psychology, as point D1 (advocacy/process) never reaches D2 (advocacy/achievement), or D3 (advocacy/impact).
Example 2. World Federation for Mental Health Director Eugene Brody credits a psychologist—Kay C. Greene—as the first to successfully introduce the phrase “mental health” among the priorities of UN Secretary General Boutros-Ghali’s—an event that was “unprecedented” (Brody, 1998, p. 182)—an example of C3, a long-in-coming educational impact on UN priorities.
Example 3 . Starting in 2007, APA and other psychology NGOs have joined to create Psychology Day at the UN, thus segueing from attending others’ conferences to hosting their own to popularize psychology at the UN—a shift from C1 (education/process) up to C2 (education/ achievement), now headed toward integrating psychology into more UN programs (C3).
Overall, this model can help all NGO representatives to monitor their own efforts, as each level leads to the next and suggesting moving beyond simply attending meetings (process), to offer tangible service/research (achievement) that actually shapes U.N. policies (impact), which will ultimately uplift society (benefit), with the 8 lofty UN Millennium Development Goals (MDGs).
Berry , M.O., Reichmann, W., & Schein, V.E. (2008, April). The United Nations Global Compact needs I-O psychology participation. The Industrial-Organizational Psychologist, 45, 33-37.
Brody, E.B. (1998). The search for mental health. NY: World Federation for Mental Health.
Etzioni, A. (1976). Social problems. Englewood NJ: Prentice-Hall.
Flanagan, J. (1949). A new approach to evaluating personnel: Critical incidents. Personnel, 26, 35-42.
Hanges, P.J., & Lyon, J.S. (2005). Rating psychology departments. American Psychologist, 60, 1035-1036.
Murray, B. (2002, July). What psychology majors need to know. Monitor on Psychology, 80-81.
Takooshian, H. (2007, April). Symposium, International I-O psychology: Growing roles at the United Nations, at the annual meetings of the Society for I-O Psychology, New York City.
Van den Bos, G. (1996). Outcomes of psychotherapy. American Psychologist, 51, 105-106.
Verdi, W.M. (2006, February 28). Symposium, Assessing NGO effectiveness? Fordham University, New York City.