A Comparative Review of Experimental Studies on Learning Outcomes: Teacher Responses Versus Generative AI Responses
DOI:
https://doi.org/10.54097/kaqest20Keywords:
Teacher Responses, Generative AI Responses, hybrid instructional designs, student learning outcomes.Abstract
Advances in artificial intelligence (AI) have substantially broadened the scope of students’ inquiry, enabling them to engage in questioning through more varied and complex modalities. This paper presents a comparative review of recent experimental and meta-analytic studies examining the effects of teacher responses and generative AI responses on student learning outcomes. Drawing on evidence from writing instruction, general academic performance, and higher-order cognitive tasks, the review synthesizes findings across cognitive, affective, and motivational dimensions. Results indicate that generative AI responses—particularly from large language models—can produce substantial short-term gains in performance and comprehension, especially in structured tasks requiring rapid feedback. However, teacher responses remain more effective for fostering deep conceptual understanding, knowledge transfer, and emotional support. The discussion highlights theoretical explanations grounded in constructivism, trust models, and productive-struggle theory, and argues for hybrid instructional designs that integrate AI feedback with teacher scaffolding. Recommendations for future research emphasize longitudinal designs, discipline-specific analyses, and strategies for improving the accuracy and socio-emotional sensitivity of AI feedback.
Downloads
References
[1]J. Hattie and H. Timperley, The Power of Feedback. Rev. Educ. Res. 77, 81-112 (2007).
[2]V.J. Shute, Focus on Formative Feedback. Rev. Educ. Res. 78, 153-189 (2008).
[3]P. Black and D. Wiliam, Assessment and Classroom Learning. Assess. Educ. 5, 7-74 (1998).
[4]D.J. Nicol and D. Macfarlane-Dick, Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Stud. High. Educ. 31, 199-218 (2006).
[5]B.S. Bloom, The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educ. Res. 13, 4-16 (1984).
[6]K. VanLehn, The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ. Psychol. 46, 197-221 (2011).
[7]J.D. Karpicke and J.R. Blunt, Retrieval practice produces more learning than elaborative studying with concept mapping. Science 331, 772-775 (2011).
[8]R.A. Bjork, J. Dunlosky, and N. Kornell, Self-regulated learning: Beliefs, techniques, and illusions. Annu. Rev. Psychol. 64, 417-444 (2013).
[9]D.L. Roorda, H.M.Y. Koomen, J.L. Spilt, and F.J. Oort, The influence of affective teacher-student relationships on students' school engagement and achievement: A meta-analytic approach. Rev. Educ. Res. 81, 493-529 (2011).
[10]B.K. Hamre and R.C. Pianta, Can instruction and emotional support in the first-grade classroom make a difference for children at risk of school failure? Child Dev. 76, 949-967 (2005).
[11]J. Cornelius-White, Learner-centered teacher-student relationships are effective: a meta-analysis. Rev. Educ. Res. 77, 113-143 (2007).
[12]X. Gao, O. Noroozi, J. Gulikers, H.J. Biemans, and S.K. Banihashem, A systematic review of the key components of online peer feedback practices in higher education. Educ. Res. Rev. 42, 100588 (2024).
[13]K. Gomis, M. Saini, M. Arif, and C. Pathirage, Enhancing the assessment and the feedback in higher education. Qual. Assur. Educ. 32, 165-179 (2024).
[14]P. Blatchford, P. Bassett, and P. Brown, Examining the effect of class size on classroom engagement and teacher-pupil interaction. Learn. Instr. 21, 715-730 (2011).
[15]S. Burgess and E. Greaves, Test scores, subjective assessment and stereotyping of ethnic minorities. Inst. Fiscal Stud. Rep. (2009).
[16]X. Cheng, Y. Liu, and C. Wang, Understanding student engagement with teacher and peer feedback in L2 writing. System 119, 103176 (2023).
[17]Z. Ji, N. Lee, R. Frieske, et al., Survey of hallucination in natural language generation. ACM Comput. Surv. (2023).
[18]J. Wang and W. Fan, The effect of ChatGPT on students' learning performance, learning perception, and higher-order thinking: Insights from a meta-analysis. Humanit. Soc. Sci. Commun. 12, 621 (2025).
[19]X. Cheng, Y. Liu, and C. Wang, Understanding student engagement with teacher and peer feedback in L2 writing. System 119, 103176 (2023).
[20]J. Fleckenstein, L. Liebenow and J. Meyer, Automated feedback and writing: A multi-level meta-analysis of effects on students' performance. Front. Artif. Intell. 6, 1162454 (2023).
[21]J. Meyer, et al., Using LLMs to bring evidence-based feedback into the classroom: impacts on cognitive and affective outcomes. Comput. Educ. 196, 104765 (2024).
[22]A. Nie, Y. Chandak, M. Suzara, M. Ali, J. Woodrow, M. Peng, M. Sahami, E. Brunskill, and C. Piech, The GPT Surprise: Offering large language model chat in a massive coding class reduced engagement but increased adopters' exam performances. arXiv:2407.09975 (2024)
[23]A. Vanzo, S. Pal Chowdhury, and M. Sachan, GPT-4 as a Homework Tutor Can Improve Student Engagement and Learning Outcomes. arXiv:2409.15981 (2024)
[24]R. Deng, M. Jiang, X. Yu, Y. Lu, and S. Liu, Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies. Comput. Educ. 227, 105224 (2025).
[25]H. Bastani, O. Bastani, A. Sungu, H. Ge, Ö. Kabakcı, and R. Mariman, Generative AI without guardrails can harm learning: Evidence from high-school mathematics. Proc. Natl. Acad. Sci. U.S.A. 122, e2422633122 (2025).
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Education and Educational Research
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
