Explicitly Relevant Example-based Grammatical Feedback Generation for Language Learners

Abstract

Automatic feedback generation systems are widely used in language learning. However, feedback from recent systems is insufficient to help learners. To generate beneficial feedback for language learners, this study introduces a feedback generation system, GrammarMentor. It employs three components: an error type classifier, a grammatical error correction model, and an example retrieval mechanism. An error type classifier categorizes errors in learners’ sentences, a grammatical error correction model corrects these errors, and an example retrieval mechanism extracts clear examples from an example database, consisting of 129,318 examples with various tones and balanced error type distributions we built using ChatGPT. Using these components, GrammarMentor generates well-organized grammatical feedback containing error types, correction suggestions, and clear examples. To verify this system, we conducted a human evaluation comparing GrammarMentor’s feedback with that of other commercial systems. The results demonstrated that GrammarMentor’s feedback enhances learners’ writing proficiency, outperforming other commercial systems. We also conducted a quantitative evaluation, employing a GPT-4-based approach for the first time to assess feedback, and GrammarMentor’s feedback exhibited superior relevance, concreteness, and richness.