This Professor Let Half His Class Use AI. Here’s What Happened

To use artificial intelligence, or not to use AI? That is the question that all students and their educators face in today’s age. While people have been endlessly arguing the pros and cons of incorporating tools like ChatGPT in the classroom, researchers have put it to the test with actual economics students, and the results are surprising.

University of Massachusetts Amherst educators compared the impact of using AI versus not using AI in two back-to-back sections of an advanced antitrust economics course with the same lectures, assignments, and paper-and-pencil exams (without notes or technology). At the end of the semester, the experiment revealed that the structured use of generative AI increased student engagement and confidence, but not test results.

“The core result is simple: permitting and scaffolding AI use did not raise proctored exam scores in this course. Yet the intervention meaningfully changed how students learned and how they felt about their learning,” the researchers wrote in a study published earlier this month in the journal Social Science Research Network.

Positive impact

Students in one section were encouraged to use generative AI tools like ChatGPT with disclosure and guidance, while those in the other were not allowed to use AI and instead received parallel non-AI study guidance. Christian Rojas, co-author of the study and the professor who taught both sections, and his colleagues found that the students who could use AI exhibited better class participation.

Furthermore, “by semester’s end, the two sections report similar frequency of AI use in their other courses, but students in the AI section concentrate that use into longer, more substantive sessions (15–30 minutes),” the researchers added. They also associated AI access with “more positive perceptions—especially regarding efficiency, confidence, and engagement—and stronger intentions to continue using and studying AI, as well as choosing AI-intensive careers.”

Students with AI access matured more habits associated with reflective learning (learning from previous experiences or ideas), like editing AI-generated texts, identifying mistakes, and choosing their own answers over the AI’s. Moreover, they gave remarkably higher standardized course evaluations, especially in regard to instructor preparation and use of class time. All that said, AI use didn’t seem to impact exam scores or final grades.

Integration without shortcuts

“It’s not that AI helped students learn more—it helped them learn more efficiently and confidently,” Rojas explained in a university statement. “They spent less time outside the classroom on homework and exam preparation.”

According to the professor, the experiment demonstrates that educators can incorporate AI without their students taking shortcuts, though he is careful to note that the study analyzed a small sample size and involved a significant amount of self-reporting, meaning further research should be conducted on a greater scale.

Nonetheless, “taken together, structured AI access with guardrails appears, in our setting, to reshape how students learn and feel about learning, without raising exam scores,” the researchers concluded in the study.