Subscribe

Your email:

More from Allen Interactions:

UpcomingWebinar
5myths(2)

 ZEBRAZAPPS

CS&DM(2)

WhitePapers(2)

Connect with Us!

Facebookdescribe the imagedescribe the image

e-Learning Leadership Blog

Current Articles | RSS Feed RSS Feed

Improving e-Learning Interactions: This is NOT a Test!

  
  
  

Ethan Edwards

by Ethan Edwards, chief instructional strategist

Last week when flying home from Dallas after teaching an Instructional Design class there I was reading a fun little book, Theater Geek by Mickey Rapkin, and was struck by how a statement there completely duplicated a point I was trying to make in class. 

The book is about a summer youth theater camp set up in a defunct Catskills Borscht Belt resort called Stagedoor Manor—in particular it reports on the camp’s history and the influence it now holds in identifying up-and-coming actors destined for Broadway and Hollywood.  

The camp has been running since 1975, and the narrative discusses how much different the experience for the campers is now than in the early days--how the presence of YouTube has made it really difficult for kids to experiment as performers and develop skills.  Knowing that any attempt at performance nowadays will be recorded and published…and instantly critiqued and evaluated…really dampens the risks students are willing to take, exactly at a time and place where none of that should matter. 

Stagedoor Manor

Rapkin writes:  “The unique problem facing this generation:  Everything is recorded.  There is no longer a safe space to make mistakes, which is perhaps the most essential step to growth.  Brent Wagner, the head of [the University of] Michigan’s musical theater program, strongly suggests students remove their YouTube clips, because no one should be judged in the training stage. ‘Technology can put a scrutiny on your work that is intimidating,’ he says.”

This is a really important aspect of training: to learn something you need to be free to make mistakes in the process of mastering it.  Group instruction is always limited by the presence of peers and teacher(s) who are there to witness your performance; thus, every time a student has to perform some task in class, the pressure on the student is as much about not looking stupid as it is about mastering the content.  

One of the great potential advantages that e-learning provides, is that learning can be removed from this public sphere and into a place where the learner should find safety and be free—even encouraged—to make mistakes.  But what do most e-learning designers do?  They completely undermine this by making every single interaction a “Test” question with the primary purpose of any feedback is to forcefully convey judgment (“No,” “Incorrect,” “Good Job,” etc.)…where the expectation is that if you don’t get it right you will be measured and found wanting…and that your inadequacy will be immediately reported and recorded for posterity in the company’s LMS.

But all interactions don’t need to be about testing; they don’t even need to be about being right or wrong.  What needs to be done for learning to take place is to create learning experiences.  The interactions should challenge the learner to actively engage in resolving some situation, with appropriate actions available (even actions that will be unproductive) and rich feedback provided including expression of consequences, coaching, and relevant content matter.  Notice that useful feedback doesn’t necessarily have to include judgment.  If a suitable context is provided, and if students understand the challenge, they will rarely have difficulty knowing on their own whether they succeeded or not.  Instead of focusing the feedback on the obvious (“That’s incorrect”), the feedback should convey rich guidance and relevant information that will help and encourage students to try again with an improved approach—or even to try again to intentionally do even worse  — a strategy that can be highly entertaining and equally as useful as a learning event as getting a challenge correct.

The real key to making these kind of immersive, challenging, context-based simulations work is to be very clear upfront with the learner that performance is not being judged (at least at first) and that performance is not going to be recorded.  Only after the learner has had the chance to respond to and succeed at the challenges should you then begin, if necessary, to start evaluating and recording performance in more typical testing scenarios.  Students will thank you for both the opportunity to explore in a safe environment, free from judgment, and also for the fact that you are evaluating their performance only after you have given them a reasonable chance to be prepared for your evaluation.

Comments

I really like your post. Unfortunately, in today's corporations, it is very hard to develop a climate of trust. Many of us believe that we are constantly being judged, no matter what is said to the contrary. I had to "back off" doing a video of a practice presentation in a train-the-trainer class because too many people thought the presentation would be used to denigrate them or hold back promotions
Posted @ Wednesday, March 30, 2011 3:54 PM by Paul Safyan
Ethan, 
 
 
 
You were very successful in getting this point across in class last week! The examples that you shared with the class illustrated this best practice in addition to many others. Thank you for your willingness to share your expertise. 
 
 
 
Posted @ Wednesday, March 30, 2011 4:25 PM by Susan Frear
I used to give a great value to feedback until I read a research report which proved the low effect of feedback on learning. However, as in my own learning experience, feedback is meaningful for me, I tried to find out why feedback has this low impact on learning, and just by surveying some peoplé's learning experience, I've found that they just want to see a grade, a bad or wrong word and most of the times, they don't read the explanations given, or if they read it, don't take enough time to comprehend what's going on with their own answers in relation to the explanation received. 
 
What I think is that we have to be very creative in the ways we offer feedback. More than explanations, we should show (not describe) consequences for example and I've also seen that the impact is higher if we can manage feedback with a great dose of humour. 
 
It would be great to read other experiencies about the ways to offer feedback. 
 
Carlos
Posted @ Thursday, March 31, 2011 7:39 AM by Carlos
Carlos— 
 
It doesn’t surprise me that research would find most e-learning feedback ineffective. In fact, it would have been surprising if results indicated anything else. Unless motivated otherwise, most learners’ top goal is to get finished, and typical feedback provides very little value in achieving that goal. Most commonly, incorrect feedback is simply “No. Try again.” (unhelpful in any way in regard to understanding confusing content) and correct feedback (to which writers often devote a great deal of attention) is largely ignored because once a question is judged correct there’s no reason to pay any more attention; the student is rushing head-long through to completion. Again this has to do with predominant view that e-learning interactions are test questions—testing generally victimizes learners and sets up a situation for them to do no more than absolutely necessary. 
 
I think it really is question of motivation. It’s the designer’s responsibility to create interactive experiences where the feedback actually matters. It’s also important to be able to distinguish “feedback” and “judgment.” Judgment (which is a specific type of feedback) usually indicates closure. “No.” Or “Excellent. New customers are always blah blah blah blah blah……” are mainly judgment (the “blah, blah, blah…” is probably non-judgmental feedback, but at that point, who cares?) The “No” is unhelpful for learning, and irrelevant if the learner gets to move forward anyway. The “Excellent” is affirming, but once the learner determines that the answer is correct and he/she is no longer on the hook for this particular question, it’s on to the next thing rather than waste effort reading content that no longer has any consequence to the student. Feedback and judgment are hard to distinguish completely, but hopefully it’s clear that there is a whole bunch of feedback that can be delivered that is not primarily judgmental in nature. 
 
But feedback that doesn’t necessarily offer final “judgment” but rather provides useful content, illustrates actual consequences, or perhaps even offers an additional question to guide flawed thinking, can be incredibly useful. In fact, there are a number of examples I could point to where you could make a case that ALL the actual teaching and content delivery takes place in feedback. 
 
The trick is to make sure learners are committed to their own improvement and motivated to pay attention to feedback. Some really strong techniques to build motivation around feedback are: 
 
1. Build a context and challenge that the learner actually cares about. If the learner sees that the task relates directly to job performance, there’s a reason to pay attention. Learners, in general, are more interested in being able to do their job successfully than in passing an e-learning lesson, but if it seems that  
2. Hold learners accountable. Don’t let them proceed until they solve the challenge. And don’t give the answer for them to blindly copy. Continue to give useful feedback that is necessary to solve the challenge. 
3. Use intrinsic feedback to help the learner recognize the actual consequences of choices. This is really the main reason to build simulations…give the learners exposure to real-world consequences from which they can judge their own success. 
4. For questions where guessing is possible, use follow-up questions to require learners to justify the answer chosen. 
5. Delay judgment. This forces learners to evaluate whether their response is good enough (based on intrinsic feedback and access to additional content) before the lesson does. 
 
Well, sorry that answer got so long, but it is a really important question, and one that has really significant impact on whether e-learning provides any value to the learner or not. 
Posted @ Thursday, March 31, 2011 11:55 AM by Ethan Edwards
Due to constraints at work, it took me until April 14th to read this blog. As there are no accidents in the Universe, it was no mistake I read this a day after a conversation I had with 2 people who were considering joining Toastmasters. 
 
 
 
During the meeting they had seen 3 very polished presentations, 2 of which were given by Advanced Communicators. They were very concerned about making mistakes and not doing as good a job as the speakers they saw during the meeting. I explained to them everyone in the club, including our Advanced Communicators, has gone through the journey that they were setting out on: Walking into a room full of strangers and not knowing what to expect during the meeting. Then making that commitment to join and having knocking knees during the first speech.  
 
 
 
I assured them that we didn't expect them to be at the same level for performance as our Advanced Communicators. Advanced Communicators have completed over 10 speech assignments. This involves logging hours and hours preparing and delivering presentations.  
 
 
 
Toastmaster members aren't expected to be perfect. They are just people who want to refine their communication and leadership skills in a safe environment. We're all in the same boat, rowing towards the same destination. Members share the responsibilities for speaking and for evaluating. There is no value in my denigrating your performance because next time you can be evaluating me!  
 
 
 
There is great value from providing and receiving positive feedback in a supportive environment. I have found a safe harbor in my local Toastmasters club and I highly recommend it to anyone looking for a safe place to make mistakes.
Posted @ Thursday, April 14, 2011 8:43 AM by Joanne Deitsch
Comments have been closed for this article.