The paper explores using LLMs to automatically generate educational content with semantic annotations, focusing on creating course-specific computer science questions that target deeper understanding rather than mere factual recall.
-----
https://arxiv.org/abs/2412.04185
🤖 Original Problem:
Generating tailored educational content manually requires extensive effort and expertise. Current automated solutions lack contextual awareness and can't produce well-annotated questions targeting higher cognitive dimensions.
-----
🔍 Solution in this Paper:
→ The research implements a retrieval-augmented generation (RAG) pipeline using GPT-4-Turbo to create semantically annotated quiz questions.
→ The system takes course materials and specific parameters (concept, cognitive dimension, difficulty) as input.
→ Questions are generated with structural annotations (like question format) and relational annotations (concept references).
→ The approach focuses on the "understand" cognitive dimension and medium difficulty level.
→ Questions are automatically evaluated for quality, contextual fit, and semantic accuracy.
-----
💡 Key Insights:
→ Structural annotations (question format, feedback) are generated reliably
→ Relational annotations (concept links) remain challenging
→ Generated questions often require human verification
→ Content errors occur more frequently in complex topics
-----
📊 Results:
→ 28/30 questions matched teaching material context
→ 27/30 questions were solvable with available materials
→ 11/30 questions contained content errors
→ No fill-in-blank questions were generated, only multiple choice (18) and single choice (12)
Share this post