Psychological researchers often collect data by asking participants to fill out a daily journal entry or describe their behaviors and feelings at several points throughout a day. But what methods ensure that participants report their behavior on time and accurately?
The debate once centered on whether to use inexpensive paper-and-pencil diaries or handheld computers that can record compliance and sound reminders. However, a paper in the March issue of Psychological Methods (Vol. 11, No. 1, pages 87-105) suggests that the method of collection may not be as important as other factors, such as study design and participant motivation.
"We want people to stop thinking 'What's better-paper or plastic?' and think more about 'What's ideal for my study design?'" explains lead author Amie S. Green, PhD, a recent graduate of New York University.
In the paper, Green's team analyzed diary entries from two previously published studies and
conducted a third, original study to track the differences between paper and electronic diary compliance. Its findings indicate that paper diaries in many instances may collect equivalent data to that of electronic diaries, though the authors consider their findings preliminary.
"If the proper methods are taken to ensure high data quality, it's premature to conclude that just because data are collected on paper it will be systematically biased," Green explains. "We found very little evidence for any bias. There's utility in both tools."
Before choosing a paper or plastic diary method, Green suggests that researchers consider:
What study design best fits the research question. In general, Green and her colleagues found that paper and plastic diaries may yield somewhat different data if a study's research questions pertain to within-person variance, such as a study of how moods may fluctuate throughout a day. However, they found that diary type probably won't greatly affect studies examining mean levels, between-person differences or correlations among variables in aggregate data. For these studies, researchers might consider factors such as whether participants would be more comfortable using a paper or electronic diary.
Green urges researchers to also consider participants' burden. If four times a day will yield quality data, then perhaps six times a day isn't necessary, she notes. In the same vein, would counting entries made within three hours of a target time still yield as solid data as entries made within one hour? The answers will vary based on the study at hand.
How to motivate participants to comply with the study. Researchers can provide a way for participants to ask questions throughout a study, give tips for remembering to complete entries and explain studies to participants in easy-to-understand language, including why participants shouldn't retrospectively fill out entries. "Don't leave them to figure out how to be a good participant," says Green. Moreover, she and her co-authors encourage researchers to create an environment in which participants are truly that: Those who have a sense of personal involvement in the study and feel as though they are collaborators are more likely to be compliant, the authors suggest.
However, they note, more studies are needed to examine how factors such as memory, compliance and motivation influence the data quality of paper and plastic diary methods.
--D. Smith Bailey