Scenario
I am teaching my research methods unit, and I know from past experience that students struggle with the concept of “operational definitions.” In the past, when I asked students to identify the operational definitions in a study, many of them listed the dependent variable or the entire hypothesis instead of describing how a specific variable is measured. I’m looking for a demonstration/activity that will help students encode the meaning and importance of operational definitions, and I want students to be able to describe likely operational definitions given the description of a study.
Step 1: Choosing an activity/assessment: What will students learn?
Since my learning goal is to help students understand the concept of operational definitions in ways that allow them to use that concept, I’m searching for an activity that focuses on exactly what an operational definition is in the context of a study. After a little hunting, I find this one: Smiling Operational Definitions (PDF, 825KB)opens in new window. Since I have a clear learning goal and an activity that I think matches that learning goal well, I move on to…
Step 2: How do I know if they learned?
The activity looks like it will match the learning goal well, but it doesn’t include any tasks I can look at to determine what students learned. So, it looks like I’m going to need to build a “custom” assessment for this specific activity. I take a deep breath, and dive into…
Step 3: Matching assessment to purpose
I looked through the table under step 3, and I think my purpose is close to this one in the table: “Figuring out if students understand/apply specific concepts/vocabulary,” so I decide I’m going to try to make a multiple-choice item. Thinking ahead, I know I can project the multiple-choice question on the board and have all students choose an answer quickly (my students are used to doing this in google classroom, so it will be easy and students are already used to doing this). Students’ responses are anonymous and the results are available to me immediately. I can use them to figure out if the activity worked or if I need to go back and reteach operational definitions. Now that I know what kind of assessment I need to make, I then go on to…
Step 4: Developing the assessment
I look through the multiple-choice item writing advice in the “Assessment Guide for Psychology Teachers” (PDF, 617KB)opens in new window and I dive in. Since my learning goal (from step 1) is to “help students understand the concept of operational definitions in ways that allow them to use that concept,” I decide to write a multiple-choice item that will give me information about students’ abilities to identify a likely operation definition for a variable that is not smiling. They showed me in the activity that they could figure out an operational definition for smiling, and I want my multiple-choice question to measure if they can “transfer” that knowledge of operational definitions to a new scenario.
Here’s the item I end up with:
Given the hypothesis: “Watching TV as a toddler leads to decreased ability to focus as an adult,” which is the most likely operational definition?
- ability to focus (wrong—this is the dependent variable)
- toddlers who watch TV have less ability to focus (wrong—this is the hypothesis)
- watching television (wrong—this is the independent variable)
- a control group of toddlers who don't watch television (wrong—this is a control group)
- a sample of toddlers, age 9-24 months (wrong—this is the sample)
- all children defined as toddlers (age 9-24 months) (wrong—this is the population)
- comparing the means of the two groups to see if the hypothesis is correct (wrong—this is data analysis)
- timing how long an adult can attend to a problem-solving task (one possible right answer)
- using an observational checklist measuring ability to focus (one possible right answer)
Step 5: “Field testing” and revision
We do the Smiling Operational Definitions activity and I use the multiple-choice question as a check for understanding. The data show that almost all students chose responses H or I (hooray!) so I decide to move on to the next topic in class. This decision seems to go well: Students are able to identify operational definitions later in the unit as we analyze other studies.
But this multiple-choice item is pretty long and it took a while in class for students to respond, so I cut some of the possible answers that no students chose. The next time I use this activity, I plan to use this revised version of the item:
Given the hypothesis: “Watching TV as a toddler leads to decreased ability to focus as an adult,” which is the most likely operational definition?
- ability to focus (wrong—this is the dependent variable)
- toddlers who watch TV have less ability to focus (wrong—this is the hypothesis)
- watching television (wrong—this is the independent variable)
- comparing the means of the two groups to see if the hypothesis is correct (wrong—this is data analysis)
- timing how long an adult can attend to a problem-solving task (one possible right answer)
- using an observational checklist measuring ability to focus (one possible right answer)
