|
This past February, at the Mid-Winter Conference in San Antonio, a group of Division 13 members gathered for a seminar on best practices in assessment. The pressing question on the minds of those in attendance was “What’s in your toolbox?” What instruments and processes are you currently using to assess clients and job candidates, and what can I adopt and add to my own toolbox? In addition to summarizing the catalog of instruments and techniques shared in our meeting, this article will catalog the instruments in the toolboxes of fellow consulting psychologists Ben Dattner, Tim Sheahan, and John Fennig, as well those from colleagues who recently shared some of the contents of their toolboxes in a Society of Consulting Psychology (SCP) listserv discussion. In the hope of starting a new tradition and facilitating interaction amongst members, I’m inviting members to share some of the contents of their toolboxes in the assessment forum on the SCP bulletin board. The discussion titled Assessment Toolbox has been created for this very purpose. Occupying a prominent place near the top of everyone’s toolbox at the workshop was the “Purpose Detector” tool. This, of course, is the invaluable tool every assessor employs to determine the nature and purpose of the particular assessment they are about to undertake. It is only after using this critical tool that one is able to select the appropriate instrumentation from their toolbox. This tool is used to consider critical issues such as Is the assessment being used for selection or development purposes, or both? If it is for selection, then is the position that the candidate is being evaluated for fairly unique in the organization or one that has many incumbents? and Do technical or functional skills need to be assessed in addition to non-technical dimensions such as problem solving, leadership, and communication skills? It is important to make these distinctions up front because some instruments used for developmental purposes may not be appropriate for selection purposes. For example, many of the participants in the meeting used the Myers-Briggs Type Indicator (MBTI) for pre-coaching, team building, and leadership development assessments, but were reluctant to use it for selection or promotion purposes. The information from the MBTI may be informative as to how a candidate might fit in with a particular team, but it does not necessarily provide information predictive of success in a job. Clarifying the purpose of the assessment is also critical for correctly designing and validating the assessment battery and process. Individual assessments used for selecting candidates for job positions that are relatively unique in an organization, such as a senior vice-president of marketing, need to be approached differently than assessment batteries for job positions that have many incumbents or applicants, such as customer service representatives. In the former situation, due to the small number of incumbents in that position, it is not feasible or critically necessary to conduct concurrent or predictive criterion validity studies in order to validate that you are indeed assessing what you claim to be assessing. In the latter situation, however, when you are assessing a large pool of candidates for multiple positions in the same role, it is critical to have clear criterion or content validity for any instruments used to predict job performance in order to make selection decisions with any confidence, and to provide your client with maximum legal protection in the event a candidate claims the assessment process discriminated against them in some respect. Further design issues can be addressed using the Purpose Detection tool. For example, assessments that cover technical and functional skills, in addition to the non-technical skills such as leadership potential, communication skills, and problem solving ability, may need to be conducted in conjunction with a subject matter expert capable of accurately assessing those technical dimensions. At a recent training workshop I met a psychologist from a prominent search firm, who now co-interviews all job candidates with an actual subject matter expert, in order to assess functional and technical skills in addition to interpersonal skills. Assessments that are not measuring technical and operational skill, but rather leadership potential, do not require a subject matter expert. Close by the Purpose Detector in our toolbox sits the “Position Descriptor” instrument. Before it is possible to predict who will succeed in a given job position, it is necessary to know which work activities are required in the job and which competencies are required to effectively perform those activities. This equally invaluable tool is used to determine exactly what we need to measure in the assessment. While it is valuable and necessary to use in a developmental assessment, it is absolutely critical to use in any selection assessment, for legal reasons among others. This tool goes by different names, depending on whom you talk to and what type of assessment is being performed. Some prefer to call it a success profiler; others a job model, but most tend to use the term competency model. Competencies are the underlying characteristics of an individual, which includes their knowledge, skills, and abilities, and which can be shown to predict outstanding performance in a job. Our Position Descriptor instrument provides a range of approaches to constructing competency models depending on the type of position. When conducting individual assessments for job positions that will require you to assess many job candidates for a particular position, such as a customer service representative, it is necessary to conduct a formal job analysis to determine the relevant job performance dimensions. It is only by conducting this job analysis that one can legally prove job necessity, which is critical for both accurate assessment of the relevant competencies, and for the client organization’s legal protection. Assessments of this sort are typically standardized and result in an overall score for each candidate. In my toolbox I keep a copy of SHL’s Work Profiling System, a job analysis questionnaire that enables me to query subject matter experts on the tasks that are critical to their job. By combining the job analysis questionnaire data with data collected from job shadowing and critical incident interviews, I’m able to derive valid competency models. When conducting individual assessments for positions with few incumbents in them, such as the vice president of marketing position, it is usually impractical to conduct a formal job analysis. Instead, we need to carefully utilize our Position Descriptor tool to determine the job requirements and identify the competencies related to job success. For senior managers in an organization, this often consists of measuring skills such as analyzing, strategizing, leading, influencing, communicating, motivating, and adapting. I keep a set of Lominger’s Leadership Architect cards in my toolbox to help identify the most critical competencies for individual job positions. If the assessment is being conducted for developmental purposes, it is still important to determine the competencies critical for success in the individual’s role, in order to be able to identify the individual’s relevant strengths and weaknesses. It is also important to identify the competencies required at the next level up in order to focus them on the competencies they need to be developing. Of equal importance to both selection and development assessments is the third instrument always near the top of our assessment toolbox: The “Organizational Context Detector.” This conceptual tool enables us to get a measure the organization’s culture and climate, and thereby contextualize assessment findings so that they are relevant to the organization and the individual. It also informs us about the organization’s attitude towards assessment and development, which enables us to more effectively strategize how to communicate findings to both the individual being assessed and the client organization. Once we have effectively used the previously mentioned organizing/conceptual tools to determine the purpose (selection or development) and identify the relevant competencies, we are ready to select the particular instruments and processes in our toolbox that will accurately measure these competencies. Included in everyone’s toolbox at the workshop were instruments measuring cognitive abilities, analytical skills, personality, personal preferences, and style. Included
among most used instruments for cognitive ability and analytical skills:
Included
among the most used measures of personality:
Maintaining
a prominent spot in everyone’s toolbox for both selection and developmental
use is the interview. Selection interviews are provided in either a structured
or unstructured format. Structured behavioral interviews designed to assess
specific competencies have been shown to have higher reliability and validity
than unstructured interviews. Richard Ackley recommended James Weitzul’s
excellent text, Evaluating Interpersonal
Skills in the Job Interview (Quorum Books, 1992), as a resource. This
interview approach focuses on determining a person’s specific set of
interpersonal skills and styles through analyzing their behavior traits.
Behavior traits of interest include how much of an overachiever they are,
how entrepreneurial they are, how active or passive they are, how sensitized
they are, how aggressive they are, and how compulsive they are. When
conducting developmental assessments, everyone’s toolbox typically included
interviews or survey questionnaires with relevant co-workers. These multi-rater
feedback instruments and processes, a.k.a. 360-Degree Assessments, play a
critical role in identifying How the
client is viewed from multiple perspectives, what are the client’s perceived
strengths and areas in need of development, and what are the organizational
factors the client needs to consider in becoming more successful in their
current and future role. While many large corporations have their own
competency based 360-Degree instruments already in place, Carl Robinson
recommends the Censeo online 360-Degree instrument. While I always prefer to
conduct in person interviews with the client’s co-workers, I’ve had good
experiences with both PDI’s PROFILOR 360-Degree assessments as well as those
developed by the Novations Group. Several
SCP members were generous enough to share with me the contents of their
toolboxes for the purposes of this article. While these are not exhaustive lists
of the instruments they use with their clients, they include many of their most
preferred tools. John
Fennig’s Toolbox
Ben
Dattner’s Toolbox
Tim
Sheahan’s Toolbox
Ellen
Lent’s Toolbox
Nancy
Polk’s Toolbox
|
What’s In Your Toolbox?