Study Reveals AI Bias in Student Writing Feedback
Recent research indicates that artificial intelligence (AI) provides more favorable feedback to Black students on their writing compared to students of other races and genders. The paper, titled “Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback,” was published in March by three researchers from Stanford University. They examined 600 eighth-grade persuasive essays using various AI models, including several iterations of OpenAI’s ChatGPT and Meta AI’s Llama, covering topics like school community service requirements and theories about aliens on Mars.
The researchers—Mei Tan, Lena Faren, and Dorottiya Demushky—resubmitted the essays under different author profiles, categorizing them by race and gender, as well as student motivation and learning disabilities.
According to findings reported by the Hechinger Report, the analysis revealed a consistent pattern across AI models: essays attributed to Black students received more commendations and encouragement, often highlighting themes of leadership and strength. Feedback included comments like, “Your personal story is powerful! Adding more about how your experiences connect with others can make this even more powerful.”
In contrast, essays attributed to Hispanic students or English learners tended to receive more corrections regarding grammar or proper English usage. For essays identified as written by White students, the focus of feedback shifted more towards argument structure, evidence, and clarity, which could lead to more substantial improvements in their writing.
The analysis also highlighted that female students often utilized first-person pronouns and emotive language, receiving feedback that felt more personal, like, “I love your confidence in expressing your opinion!” Meanwhile, students identified as Black, Hispanic, Asian, female, lacking motivation, or with learning disabilities received less constructive criticism and more praise. This approach sometimes resulted in stereotypical praise, with certain phrases like ‘love’ being used disproportionately with female students, and terms like ‘powerful’ appearing more often for Black students.
Researchers Tan and Phalen expressed concern about the implications of standardized feedback. They noted that while positive feedback can be encouraging, high-quality feedback necessitates specific, actionable criticism to foster improvement. In their statement, they emphasized that the automated feedback’s reliance on praise often overlooks opportunities for meaningful development, as some students, especially English language learners, may experience excessively negative or corrective feedback.
Regarding the reasons behind these biases, they mentioned the uniqueness of LLM training procedures, leaving them to speculate on why such trends occur. They pointed out that previous research has noted similar biases in human-generated feedback.
The researchers also suggested that debiasing efforts during LLM training could inadvertently contribute to the positive stereotypes noted in their findings.
