The promise and perils of using AI for research and writing

As artificial intelligence (AI) tools proliferate, the goals of ethical research and writing remain the same: to be transparent, preserve the integrity of authorship, and verify reported findings. What’s changed is that AI can provide somewhat of an assist as long as researchers and students retain rigorous oversight.

Among the ways AI tools can be useful include helping with more routine tasks, cleaning up grammar, and streamlining time-consuming steps involved with finalizing manuscripts, such as citations and the submission process, according to APA leaders whose work involves providing guidance on the use of AI. The technology can also enable non-native English speakers to improve syntax and readability, as well as to translate academic terms prior to submitting to English-language journals, said Rose Sokol, PhD, publisher of APA Journals and Books.

In addition, as AI continues to evolve, it could support the initial or brainstorming stages of research, said Emily Ayubi, senior director of APA Style. If a researcher is considering the pursuit of an avenue of study and wants to gain a better sense of gaps in the existing knowledge base, she said, generative AI “would theoretically be able to review the existing literature more expediently than a human being could. But you would still have to vet the output because there may be fabrications. It could make up studies that don’t actually exist.”

A good guideline is that although AI tools can support more routine steps of research and writing, it should not be relied upon, Ayubi and Sokol stressed.

At the heart of the APA Publishing policies as related to generative AI, Sokol said, “is that to be an author you must be a human. The threat for students and researchers is really the same—over-relying on the technology.” When that happens, you are at risk of essentially ceding control of intellectual property to the machine, she noted. “You’ve handed that over. The machine has no accountability and no responsibility.”

How and whether psychologists and students can incorporate AI tools into their research will vary depending upon the circumstances involved, said Samantha Denneny, development manager for APA Style. AI use can be unavoidable if it comprises the heart of a research project on, for instance, the role of the technology in psychology, she said.

Policies also may differ depending upon the journal or university or instructor involved, Denneny said. “You might have a professor who says, ‘Do not open ChatGPT or you’re in trouble,’ and then you might have a professor who has you use it throughout an assignment.”

Still, APA policy about generative AI use has developed consensus on several key points, as outlined in a blog post published in late 2023, including that AI cannot be listed as an author in any one of APA’s 88 scholarly publications.

“An author needs to be someone who can provide consent, who can affirm that they followed the ethical protocols of research, that they did the steps as they said they would,” said Chelsea Lee, instructional lead for APA Style. “You need a human to be able to give that consent.”

When AI is used during research, that involvement should be cited in the methods or a comparable section of the paper, said Timothy McAdoo, a manager on the APA Style team, who wrote a blog post providing guidance about how to appropriately cite ChatGPT. Although quotes and other details that are not retrievable are typically cited as personal communication, that’s not an option with ChatGPT as there’s no human involved, he noted.

The recommended citation approach is to include the precise text of the prompt, and then provide a reference that incorporates the author of the AI tool, the date of the version used, and other details. If AI produces a lengthy response, researchers should add that text to the paper’s appendix or online supplemental materials, given that the same prompt will generate a unique response each time that it’s used, McAdoo wrote.

Plagiarism software is typically not able to flag writing that AI has produced, as the tools generate sentences that haven’t existed in that combination before, members of the APA Style team pointed out. “When you plagiarize another person, there is evidence for that—you found that information from somewhere,” Denneny said. “But the output from AI is not something that is public record—it’s not trackable.”

Plus, research indicates that instructors may not be skilled in their ability to detect AI-produced text, Denneny said. She cited one recent study that found that both novice and experienced teachers struggled to identify AI versus student-written text and yet found that both groups were overconfident in their capacity to do so (Fleckenstein, J., et al., Computers and Education: Artificial Intelligence, Vol. 6, 2024opens in new window).

AI detection software exists but is not reliable, according to APA Style team members. When they fed the text of the U.S. Constitution into one program, it reported back that 99% of the historical document had been AI generated.

Another concern: the potential for bias against non-native English speakers. In one study, which evaluated seven ChatGPT detectors against 91 Test of English as a Foreign Language essays, the detectors incorrectly labeled more than half of the essays as AI-generated, with an average false-positive rate of 61.3%. But only 5.1% of 88 essays written by U.S. 8th grade students were similarly misclassified (Liang, W., et al., Patterns, Vol. 4, No. 7, 2023opens in new window).

Scholars also should be aware of the potential for bias and lack of inclusivity in the research that AI identifies and thus dig further to determine if the technology has missed key elements, Ayubi said. In addition, AI may draw from older studies that can use outdated terminology that doesn’t align with APA’s Inclusive Language Guide, Second Edition and lacks gender-inclusive language, for instance, she said.

Along with assisting with the mechanics of writing, such as checking grammar and phrasing in a paragraph, AI may provide initial insights on a subject, Denneny said. For instance, someone can ask for a quick summary, similar to checking out a Wikipedia page, she said. “And then you take that and delve deeper.”

Above all, the researcher or student must remain in the driver’s seat, checking everything that falls beneath their name, APA leaders stress. The Style Team has conducted various test runs with generative AI, and the results have not always been encouraging.

In one instance, Lee asked ChatGPT for five peer-reviewed sources on a topic with which she had familiarity.“It sounded exactly like what I was looking for,” she said, noting that the citations included authors who had studied that subject. “I went looking for [the studies], and none of them were real at all.”

Lee returned to ChatGPT and asked if it was certain that those references existed. “It said, ‘Sorry for any confusion. These sources are illustrative. I don’t have access to the information that you’re actually asking for.’”

In his blog post about citing ChatGPT, McAdoo describes how he requested five sources related to ideas about brain lateralization and how the brain operates. ChatGPT provided five, only four of which he was able to locate online. The fifth reference included a real digital object identifier (DOI), but it was one that was assigned to a different article.

With these potential pitfalls in mind, researchers and students should verify not only the legitimacy of the sources identified, but “it may be better to read those original sources to learn from that research and paraphrase or quote from those articles, as applicable, than to use the model’s interpretation of them,” McAdoo wrote.

In short, generative AI should be viewed as similar to an electric bike, with the capacity to augment but not replace one’s own skill set, Denneny said. “You don’t want to be asking AI to do something where you’re not in control,” she said.

Moreover, the goal of scholarship is to add something new to the conversation, Lee said, noting that AI only summarizes existing information. “The texts generated by AI on the whole tend to be on the surface level. Whereas in science, you want to be very precise and think about, ‘What is the thing that I’m trying to share with my audience here? What’s new? Why does it matter if anybody reads this?’”

As these tools continue to evolve, APA Style team members welcome ongoing input, Ayubi said. “What we’re presenting now is a snapshot in time,” she said. “We continue to research these technologies.”

Ideally, as AI tools progress, they hold the potential to enable researchers to focus on the more complex and cognitively demanding elements of scholarship, Sokol said.

“If you can automate the routine parts that are less essential, then you have more time for that creative process,” she said. “You have more time to think about, ‘What’s the interpretation? Was there bias introduced in my research design, and how might that affect my interpretation?’ You have more time to think through the [research] process.”