Transforming Chemistry Exams with Artificial Intelligence

BATON ROUGE - In graduate school, cumulative exams serve as a standard method for evaluating a student's comprehensive understanding of the material covered throughout an academic program.  

Cumulative exams do more than quiz students on one chapter or one semester of material. They demand a marathon for the brain, testing students' knowledge and ability to connect the dots across an entire area.  

Imagine three LSU Chemistry graduate students already feeling the pressure of cumulative exams, learning about an inorganic chemistry cumulative exam opportunity that allows the use of ChatGPT 4, an artificial intelligence system, to solve chemistry problems.

As the students select the innovative use of artificial intelligence, or AI, on their exam, it reflects a broader trend in academia among students embracing the technological advancement.  

While attending the 2023 Neural Information Processing Systems, NeurIPS, Conference, LSU Chemistry Professor Les Butler witnessed firsthand the remarkable progress made in machine learning and large language, recognizing their benefit in the college setting.  

"Large language models, like ChatGPT, have the potential to tackle at least three educational challenges. They offer round-the-clock, non-judgmental tutoring for homework help. It also allows for solving larger, more realistic problems that capture students' interest, keeping them engaged," said Butler. "While still in the conceptual stage, these models could potentially replace conventional multiple-choice exams, although implementation specifics remain uncertain."  

Interested in utilizing a large language model, Butler designed an inorganic cumulative exam with typical questions found at the end of textbook chapters. He then provided the students with an instructional guide explaining the application of ChatGPT 4 with the Wolfram plugin two weeks before the exam. 

David Ante, a first-year chemistry graduate student at LSU, initially felt skeptical and surprised at the unconventional approach. "I never encountered this type of assessment before, so it was quite unexpected," he remarked. "In my previous experience, exams were typically rigorous, paper-based tests where you had to recollect and write down numerous memorized formulas."  

Relying on their foundation of knowledge in chemistry, students had to formulate questions guiding ChatGPT toward accurate responses. As AI can provide valuable information, students must analyze the information and articulate a solution effectively.

"We were ultimately tested on our ability to comprehend the problem, transcribe the response to produce results, and articulate a solution. The exam challenged me to apply my knowledge effectively, which I believe is a crucial skill."

David Ante, LSU Chemistry Graduate Student

The involved graduate students recognize the potential of AI in exams and research as an efficient tool but note that AI falls short when generating entirely new ideas and methods, highlighting the ongoing importance of research conducted by students and faculty.

"The one purpose of research is for us to learn and discover new things. AI will be helpful for incorporating previous literature, but not in the full aspect of using it as a tool for your personal research," Ante said.

Graduate student Afsana Mimi found the free version ChatGPT 3.5 useful for tackling chemistry problems and conducting literature searches in the past. However, she found the enhanced capabilities of the newer version particularly impressive.

"The newer version's improvements truly exceeded my expectations, opening up even more possibilities for its use in chemistry," Mimi stated.

More robust versions of AI come with costs, including ChatGPT 4. While Professor Butler covered fees for this exam, LSU Chemistry graduate student Shahriar Mahmud recognizes AI as a valuable tool for graduate students and believes universities should consider subsidizing these costs for continued use.

"I'm unsure if the utilization of ChatGPT 4 at the undergraduate level is ideal as they are still mastering the fundamentals,” said Mahmud. However, its use can be encouraged among graduate students to explore solutions for complex problems, by providing financial assistance or subsidizing the cost for ongoing access to updated versions."

With a recent NSF STEM Education Individual Postdoctoral Research Fellowship, LSU Chemistry postdoctoral researcher Jennifer García Ramos from Associate Professor Zakiya Wilson-Kennedy’s research group is investigating the motivations behind students' adoption of AI in their learning process and how instructors can create an equitable learning experience for all students.

"AI can help create equitable learning experiences by promoting educational equity, reduce achievement gaps across different groups of learners, and fill gaps within educational services by providing access to technological and general support, resources, and educational opportunities."

Dr. Jennifer García Ramos, Postdoctoral Researcher

García Ramos hopes for increased embracement and use of the technology and its potential in a new perspective on learning. Like past technologies, García Ramos believes the emergence of AI presents an opportunity for positive impact and engagement.

As faculty and researchers investigate potential AI applications, higher education may transform how students learn and introduce innovative teaching methods and assessments, paving the way for a brighter, more accessible future in education.  

 

---