Automatic feedback on physics tasks using open-source generative artificial intelligence
This study explores the feasibility of using open-source large language models (LLMs) to generate automatic feedback on physics problem-solving tasks in educational settings. A quantised version of the open-source LLM OpenChat 3.6 was employed to generate German-language feedback for high school students on standard school hardware. The study procedure involved five stages: data preparation, model selection, prompt design, response evaluation, and quality analysis of feedback. OpenChat 3.6 achieved an accuracy of 0.84 in classifying student answers. In comparison, GPT4-o achieved an accuracy of 0.85. The open-source LLM provided accurate and
Resource information
- Authors
- André Meyer, Tom Bleckmann, Gunnar Friege
- Research area
- Artificial intelligence
- Research topics
- Medicine, Neuroscience, Biological Sciences not elsewhere classified, Information Systems not elsewhere classified, Mental Health
- Publication date
- 2025
- DOI
- 10.6084/m9.figshare.29178336
- Upstream last modified
- Sep 27, 2026
- License
- Creative Commons Attribution Non Commercial No Derivatives 4.0 International
- Last record update
- Nov 5, 2026
About this resource
This study explores the feasibility of using open-source large language models (LLMs) to generate automatic feedback on physics problem-solving tasks in educational settings. A quantised version of the open-source LLM OpenChat 3.6 was employed to generate German-language feedback for high school students on standard school hardware. The study procedure involved five stages: data preparation, model selection, prompt design, response evaluation, and quality analysis of feedback. OpenChat 3.6 achieved an accuracy of 0.84 in classifying student answers. In comparison, GPT4-o achieved an accuracy of 0.85. The open-source LLM provided accurate and suitable feedback in 69% of cases, with substantial interrater agreement ( κ = 0.89) on feedback quality. However, performance varied across task types, highlighting areas for improvement in prompt specificity, especially in handling physics terminology. These findings suggest that, with optimisation, open-source LLMs can offer a locally controlled and effective solution for formative assessment in physics education, enabling real-time, targeted feedback to support student learning.