Confident educators struggle to identify AI-generated essays, study reveals

Have you ever wondered if the text you’re reading was written by a human or an artificial intelligence? The University of California San Diego conducted an experiment using ChatGPT at a regional high school to explore this question.

To test the ability to distinguish between human-written essays and those generated by ChatGPT, the researchers presented pairs of essays to teachers and students, asking them to identify which was created by a human and which was composed by the AI language model. Teachers were correct about 70% of the time, while students averaged a 62% accuracy rate.

Although these scores may not seem terrible, the researchers argue that if differentiating between human and AI-generated essays was easy, the accuracy rates should be well above 90%.

Interestingly, confidence did not correlate with accuracy. Participants who believed they could successfully identify the work of the chatbot did not perform better than those who were less certain of their abilities.

“We were surprised to find that teachers, especially those with experience using ChatGPT or teaching high school English, struggled with the task,” said Gail Heyman, senior author and professor of psychology at the UC San Diego School of Social Sciences.

These findings raise concerns about the potential for students to submit AI-generated essays as their own and get away with it.

“One of the most interesting—and troubling—aspects of our study is that teachers performed worse when the pair of essays included a well-written student essay. In fact, many teachers guessed that the better-written essay was generated by ChatGPT. This suggests that teachers are more likely to attribute a well-written essay to AI, which has concerning implications in a real-world classroom setting,” added Heyman.

The study, published in the journal Human Behavior and Emerging Technologies, involved 69 high school teachers and 140 high school students. The essay topics were similar to those commonly assigned in schools, such as “Why is literature important?”

In addition to assessing participants’ ability to differentiate between human and AI-generated essays, the study also surveyed their opinions on ChatGPT.

Notably, students expressed greater optimism than teachers regarding the future role of ChatGPT in education. They also viewed potential academic integrity violations, like submitting AI-generated essays as their own, less negatively compared to teachers.

Riley Cox, a high school student who volunteered as a research assistant on the study, shared their perspective: “It was exciting for me to observe my classmates and teachers as they interacted with this new technology, both as a student and a psychology researcher. It was interesting to see that teachers had concerns about ChatGPT that didn’t seem to bother students.”

Reflecting on the study, one participating high school teacher commented, “I think ChatGPT could have some interesting applications in the classroom, but my concerns outweigh any positives. I am worried that we are witnessing a decline in original thought and the ability of our students to persevere through hard work.”

The researchers believe their study sheds light on the challenges and opportunities that ChatGPT presents in education.

“We are on the cusp of a major shift in educational practices as high-quality, human-like content becomes increasingly accessible to everyone,” said Tal Waltzer, co-author and postdoctoral fellow in Heyman’s lab at UC San Diego. “How we handle this transition raises important ethical considerations. For example, the fact that the paid subscription version of ChatGPT performs better on many standardized tests than the freely available version could worsen existing concerns about equity in education.”

Heyman and Waltzer plan to continue researching this area “to establish an empirical foundation for best practices regarding the ethical use of AI in secondary education,” according to Heyman. They aim to investigate activities that enhance learning and explore ways that ChatGPT can be utilized as a collaborator.

 

Reference

Denial of responsibility! TechCodex is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, and all materials to their authors. For any complaint, please reach us at – [email protected]. We will take necessary action within 24 hours.

Denial of responsibility! TechCodex is an automatic aggregator of Global media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, and all materials to their authors. For any complaint, please reach us at – [email protected]. We will take necessary action within 24 hours.