Computer

Supported
Collaborative
Learning
Track
Carolyn Penstein Rosé
Introduction
Carnegie Mellon University
Language Technologies Institute and
Human-Computer Interaction Institute
School of Computer Science
2
Track Helpers

David Adamson Ryan Carlson

dadamson@gmail.com rcarlson@cs.cmu.edu

3
Outline

• Individual learning with technology to collaborative

learning

• Vision for dynamic collaborative learning support

• Research Issues

• Current Directions

4
Historical Perspective…
• Socratic tutoring: directed lines of reasoning
– Evidence that socratic tutoring is more beneficial than
didactic tutoring (Rosé et al., 2001a)

• Socratic style implemented in Knowledge Construction

Dialogues (KCDs)
– General attempt to model effective human tutoring
– Hierarchical structure: adaptive to student needs
– Used to elicit reflection
– First used to support individual learning in Physics
• Atlas-Andes (Rosé et al., 2001)
• WHY-Atlas (Rosé et al., 2003; Rosé & VanLehn, 2005)
Empirical Foundation for
CycleTalk
• Human tutoring not always better than non-interactive
support (VanLehn et al., 2007)
– Focus shift to capturing what it is about interaction that is
effective for instruction

• Human tutors guide students towards opportunities

for reflection (Rosé & Torrey, 2004)

• Human tutor support by effective tutors is significantly

better than hint based support (Rosé et al., 2005)
 Towards enriching the interaction…

• Student interaction with dialogue agents lacks

elaboration (Rosé et al., 2003)

• Students expect to behave differently with agents

(Rosé & Torrey, 2004)

• Students can benefit from working with another

student, even in the absence of computer
scaffolding (Gweon et al., 2006; Kumar et al., 2007)

7
 VMT-Basilica

** Students learn up to 1.25 standard deviations more when interactive

support is provided in the environment.
 Monitoring Collaboration with
Machine Learning Technology
Download tools at:
http://www.cs.cmu.edu/~emayfiel/side.html

Labeled Texts Labeled Texts

Behavior

LightSIDE

Unlabeled Texts <Triggered

A Model that can Label More Texts

Intervention>
Time
 Basilica Tutor Agent Design
One last thing on this topic, Does more
Tutor
ConcertChat Server (or less) stress in a wrench make it
easier to use?
ConcertChatActor ConcertChatListener S95 no change?
Tutor You are correct about that.. Stress
DiscourseMemory MessageFilter PresenceFilter doesn't determine ease of use.
OutputCoordinator

Tutor It's the moment achieved by the wrench

AnnotationFilter that determines the ease of use.
ProgressDetector

SocialController ActivityDetector S89 yay!

It's good to have your contributions in
Tutor

T.TakingCoordinator
the discussion Jackie :-)
PlanExecutor RequestDetector
Tutor Go team :-)
S89 Go team yay
IntroductionsManager PromptingManager TutoringManager
I am happy to be working with our
Tutor
team
IntroductionsActor PromptingActor TutoringActor S89 Me too
S95 whoa the bot knows my name
Kumar, R. & Rosé, C. P. (2011). Architecture for building Unfortunately maximum stress (12800)
Conversational Agents that support Collaborative Learning, IEEE Tutor
in our design1 is way above the
Transactions on Learning Technologies special issue on Intelligent
maximum allowed stress (i.e. 8750)
and Innovative Support Systems for Computer Supported
Collaborative Learning
Tutor This wrench cannot be safely used!
Computer Supported Learning in
Large Classroom Studies
• University Level
– Freshman Engineering
– Calculus
– Sophomore Thermodynamics
• High School
– 9th Grade Biology
– Earth Sciences
• Middle School
– Engineering Outreach
– 6th grade math
First-Year and Middle School
Projects
• First-Year Project
– Wrench Design Competition
• Part of Computer Aided Engineering Labs using Pro/ENGINEER,
Pro/MECHANICA and Pro/MANUFACTURING Software (3 sessions)
• Students Work in Teams of 3 – 4 with Unknown Team Members in Other
Parts of the Room
• Only Communication is via the Chat Interface with Agent Interactions
• Teach Relationships between Moment, Force and Stress Related to Design
Goals

• Middle-School
– Practical Considerations in Designing a Wrench
• A Simplified Version of the Freshman Project
(Using the Pro/ENGINEER CAD Package)
• Part of the CMU C-MITES Program
• Students Work in Teams of 3 – 4 with Unknown
Team Members in Other Parts of the Room
• Activity Designed to Elicit Broad Discussions on
Engineering and Product Design
Middle School Students in the
Wrench Design Project
 Second-Year Thermodynamics
Project
• Sophomore Thermodynamics Course
– Rankine Cycle Design Competition
• Part of a Thermodynamic Cycles Lab (1 session)
• Students Work in Teams of 2 with Their Partner in Another Part of the Room
• Instructed on Effects of Changing System Variables (Temperature,
Pressure) on System Output (Power, Waste Heat)
• Goal: Increase Efficiency within Practical Constraints
• Software Package is CyclePad (Developed at Northwestern University
(Forbes et. al. 1999) to Allow Students to Construct and Analyze a Variety of
Thermodynamic Cycles)
Thermo Collaborative Task

• Goal: Design a power plant based

on the Rankine Cycle paradigm
– Each pair turns in exactly one design
• Competing Student Goals:
– Power: Design a power plant that
achieves maximum power output
• Motivated by economic concerns
– Green: Design a power plant that has
the minimum impact on the environment
• Motivated by environmental concerns
Increasing Conceptual Depth

• Student: so i think the best thing to go with first is to

increase the pressure entering the connector
• Tutor: You might want to review what you saw earlier in
the sensitivity analysis investigating the effect of
changing P-max on steam quality and efficiency, from
the top half of page 13 in your book.
• Tutor: type: HELP WITH CHANGEPMAX if you want to
discuss it with me.
• Student: help with changepmax
• Tutor: Remember the sensitivity analysis you have
completed where you manipulated P-Max and observed
it's effect on the efficiency of the cycle. You can refer
back to Figure 11 on page 13 of your booklet, which …
Example Interaction
Core Research Focus

• Identify conversational
interactions that are valuable
for learning
• Automatic conversation
analysis
– Facilitates learning research
– Automates assessment of group
processes
– Enables context sensitive
triggering of support
• Interactive support
technologies

17
Operationalization of
Productive Talk for Learning
• Drawn from multiple frameworks
valuing sociocognitive engagement
– Students explicitly display
their reasoning
– Students orient their
contributions towards
previous contributions
– Students increase in their ownership of
their ideas
• Made precise through constructs from linguistics
– Integrative book chapters (Sionti, Ai, Rosé, & Resnick, in press; Howley,
Mayfield, & Rosé, in press)

18
Learning Results

• Students in accountable talk classrooms (where

transactivity is elicited) learn more than students in a
traditional classroom (O’Connor, Michaels, & Resnick, in
preparation)

• Transactivity correlates with learning (Joshi & Rosé, 2007;

Kumar et al., 2007)
– Consistent with results in connection with elaborated explanations
(e.g., Webb, Nemer, Zuniga, 2002)

• Collaboration support that increases transactivity increases

learning (Wang et al., 2007)

19
 Accountable Talk
(O’Connor, Michaels, & Resnick)
Eddie: Well, i don't think it matters what order the numbers are in. You still
get the same answer. But three times four and four times three seem
like they could be talking about different things.
Teacher: Rebecca, do you agree or disagree with what Eddie is saying?
Rebecca: Well, I agree that it doesn't matter which number is first, because
they both give you twelve. But I don't get what Eddie means about them
saying different things.
Teacher: Eddie, would you explain what you mean?
Eddie: Well, I just think that like three times four can mean three groups of
four things, like three bags of four apples. And four times three means
four bags of three apples, and those don't seem like the same thing.
Tiffany: But you still have the same number of apples, so they are the
same!
Teacher: OK, so we have two different ideas here to talk about. Eddie
says the order does matter, because the two orders can be used to
describe different situations. So Tiffany, are you saying that three times
four and four times three can't be used to describe two different
situations?
 In vivo studies
Small Group
Activity Post-Activity Whole class Post-Discussion
Pretest
(Experimental test Discussion test
Manipulation)

 3 in vivo studies in 9th grade biology

 Other similar studies in math, freshman engineering, thermodynamics, and chemistry

 Online small group activities, support from

Conversational Computer Agents
Example Intervention: Revoicing Agent
Teacher: Accountable Talk
Comparing Years 1 and 2
Auto Predicted AT

R = .45

R = .36
Student talk
Comparing Years 1 and 2
Average Student
Words per Turn

R = .59

R = .18
Teacher: Accountable Talk
Comparing Years 1 and 2

Regression Scatter Plot

Auto Predicted AT

Auto Predicted AT
R = .45

R = .36
Student talk
Comparing Years 1 and 2

Regression Scatter Plot

Average Student

Average Student
Words per Turn

Words per Turn

R = .59

R = .18
Current Directions

• Continuing to investigate social considerations

for integrating dialogue agents with groups
– Investigating how motivation orientation interacts with
treatment

• Working with groups larger than pairs (Gweon et

al., in press)
– Monitoring collaboration quality from speech
– Challenges of multi-party conversation analysis
• Multiple interwoven threads (Rosé et al., 1995; Wang et al.,
2008a,b)

26
 Thank
You
!!!

Funding: The Office of Naval Research and the National Science Foundation
27