Reframing Assessment in the Age of Generative AI: Evaluating Engagement and Learning Outcomes through LLM Interactions

Document Type

Book Chapter

Publication Title

Progress in Education, Volume 91

Abstract

The rise of generative AI tools, particularly large language models (LLMs), has transformed educational practices, challenging traditional approaches to assessment by shifting the focus from final products to the processes underpinning student learning. This chapter outlines a novel framework for assessing learner engagement and cognitive processes in AI-driven educational environments. By analyzing interactions between students and LLMs, the framework evaluates critical thinking, problemsolving, and self-reflection, offering insights into how students develop and demonstrate learning outcomes. Using natural language processing (NLP) methodologies, the framework identifies patterns of engagement within AI-mediated dialogues, categorizing interactions into domains such as inquiry, synthesis, and iterative refinement. Its key innovation lies in visualizing these engagement levels through intuitive metrics that reveal cognitive depth and learning strategies, enabling educators to assess intellectual growth beyond polished outputs. Practical use cases highlight how this framework adapts to diverse educational contexts, from fostering critical debates in humanities to supporting iterative design in STEM disciplines. This approach aligns with competency-based education and lifelong learning goals, emphasizing process-oriented assessments that bridge the gap between generative content creation and meaningful demonstration of knowledge. By advancing transparency and equity in AI-assisted learning, this chapter advocates for reimagined assessment paradigms that reflect the complexity of modern educational ecosystems and prepare learners for success in a digitally mediated world.

Research Highlights

  • The Problem: Traditional product-oriented educational assessments fail to verify genuine student mastery because large language models can produce fluent synthetic prose that conceals individual cognitive paths, revision histories, and errors.

  • The Method: The evaluation framework applies natural language processing methodologies to analyze dialogic traces from student-large language model interactions, categorizing dialogue segments into domains of inquiry, synthesis, and iterative refinement.

  • Finding: The framework visualizes student engagement levels through metrics to map intellectual growth and adapt process-oriented assessment across humanities and STEM disciplines.

DOI

https://doi.org/10.52305/TCQS1373

Publication Date

7-2026

Link to Full Text

Share

COinS