Concerns Over AI Writing Detection Tools in Education

AI detection software, designed to catch student cheating, is unintentionally altering how students write. Instead of focusing on creating engaging content for human readers, students are beginning to write for algorithms, resulting in dull prose and an unhealthy reliance on technology that these tools aim to mitigate.

Writing teachers and students have started noticing the unintended impacts of these AI detection systems. Rather than merely identifying machine-generated text, they are reshaping writing practices in ways that concern educators. These tools depend on statistical analysis, which overlooks nuanced writing, emphasizing generic, algorithm-friendly prose instead.

An illustrative case arose from a student’s essay on Kurt Vonnegut’s Harrison Bergeron. The AI detection system, installed on school Chromebooks, flagged the term “lack,” which resulted in an 18% AI score. However, swapping it for “none” dropped the score to zero, even though the essay’s core message remained intact. This highlights that current detection systems often rely on superficial word choice rather than deeper evaluations of authorship.

The way these detectors function adds to the dilemma. They gauge the likelihood that text is machine-generated by examining factors like token frequency, sentence structure, and “burstiness,” which entails variability in sentence length. Students have learned that certain stylistic elements could raise detection scores, whereas simpler, bland writing tends to go unnoticed. This leads to a understandable yet troubling outcome: students often simplify their writing or employ AI to produce safe text that blends in.

Instructor Dadland May observed his students starting to use AI tools more after realizing that specific stylistic choices could trigger detectors in his classes. For instance, one student, previously committed to writing his own essays, began using an AI system to test his drafts. After being wrongly accused of submitting AI-generated work in a prior class, another student immersed himself in various AI detection techniques to prevent future errors.

This scenario is a classic case of the cobra effect, where policies meant to deter certain behaviors instead encourage them due to misplaced incentives. Students, facing pressure regarding grades or potential disciplinary actions, are often pushed to produce more bland writing or resort to AI tools that the detectors target to craft acceptable expressions. The term “cobra effect” originates from a British initiative in colonial India, where a bounty was offered for cobra kills; this prompted people to breed cobras for profit.

This phenomenon seems especially pronounced at open-access institutions like the City University of New York, where students balance jobs and language challenges alongside inconsistent policies across classes. For instance, one student described spending hours rephrasing what were initially original passages flagged by the system, while another summed up the feeling with, “It’s just a lot of revision. It takes too much time.”

The long-term impacts on education are perhaps the most pressing issue here. Students may internalize the notion that being sophisticated in writing could be a disadvantage, leading to a writing goal focused on merely avoiding detection rather than expressing clear ideas or developing a unique voice.

Recognizing these challenges, May revamped his teaching methods. He encouraged the use of AI tools for research and planning but insisted that the actual writing should be done by the students themselves. In class, he began discussing prompt design, the barriers to effective summarization, and the risks of allowing AI to take over student thought processes.

This adjustment significantly changed classroom interactions. Instead of denying AI usage, students started coming to May with questions about how to responsibly incorporate AI into their work. They were eager to learn how to gather information without copying generated text and how to spot deviations in AI-generated summaries.

In light of these developments, new perspectives on the impact of technology in education are crucial. It seems vital to find ways to harness the benefits of AI while minimizing its unintended consequences, much like the insights offered in the upcoming book by Wynton Hall, which discusses various facets of modern society, including AI’s role in education.