Cohesion and Learning in a Tutorial Spoken Dialog System

Cohesion and Learning in a Tutorial Spoken Dialog System

• Art Ward
• Diane Litman

Outline

• Tutoring
• Goals
• 4 issues in measuring cohesion
• Why they’re interesting
• How we test them
• Results

Natural Language Dialog Tutoring

• Human tutors are better than classroom instruction (Bloom 84)
• Intelligent Tutoring Systems (ITSs) hope to replicate this advantage
• Is Dialog important to learning?
• Dialog acts: question answering, explanatory reasoning, deep student answers (Graesser et al. 95, Forbes-Riley et al. 05)
• Difficult to automatically tag dialog input, so:
• Automatically detectable dialog features
• Average turn length, etc. (Litman et al. 04)
• We look at Cohesion
• Lexical Co-occurrence between turns

Goals and Results

• Goals
• Want to find if cohesion is correlated with learning in our tutoring dialogs.
• If it is, may inform ITS design
• Want to find a computationally tractable measure of cohesion
• So can be used in a real-time tutor
• Results
• Do find strong correlations with learning
• For low pre-testers
• For interactive (tutor to student) measures of cohesion
• Robust to multiple measures of lexical cohesion

4 Issues

• Why/How identify cohesion in dialogs?
• Do students of different skill levels respond to cohesion in the same way?
• (Is there an aptitude/treatment interaction?)
• Is Interactivity Important?
• What other processing steps help?

Issue 1: How identify cohesion in dialogs?

• Why might cohesion be important in tutoring?
• McNamara & Kintsch (96)
• Students read high & low coherence text
• High coherence text was low coherence version altered to:
• Use consistent referring expressions
• Identify anaphora
• Supply background information
• Interaction between pre-test score & response to textual coherence
• Low pre-testers learned more from more coherent text
• High pre-testers learned LESS from more coherent text