I recently gave a two-hour webcast for a firm. It’s been a long time since I’ve done these, so the experience was pretty nerve wracking. But apparently I got rave reviews. The training admin told me she sat in on all the firm’s webcasts and that mine was by far the best…yada, yada, yada. And that everyone agreed.
The first half of the webcast was built around two famous scenes from an iconic movie, and during the webcast I showed the scenes and then analyzed them to illustrate my main learning points. They like that—most webcasts don’t feature movie clips
The very next week I was with a group of audit seniors from the same firm to facilitate an in person (!!) session to build on the webcast and had just given. I asked the group who had seen the webcast, and they all raised their hands.
Then I asked what they remembered remembered about the movie scenes I had showed. Crickets.
I dropped hints; I cajoled; I fed them lines from the scenes. Finally someone said “ remember the movie, I just don’t remember what you said.” It had only been a week since the webcast, an event that had gotten rave reviews and undoubtedly been rated highly on the evaluations.
Which illustrates the value of most course evaluations—zero.
Firms wonder how they can send their people off to training and then they come back having not learned much. Part of the problems is how these firms evaluate the learning experience. They set a low par for success, usually centered around some version of “did you like it?
If firms want to get better results from their training, they need to be asking better questions when it’s over. The two questions every organization should ask at the end of their training sessions are:
- What did you learn/what do you remember?
- How are you going to use it?
If they can’t provide meaningful answers to those questions, then they have not been well-served by their training, irrespective of how much they liked it.
Asking these questions serves two critical purposes.
First, the answers tell you if the training is working. Partners, managers, and supervisors all have expectations for what they want their staff to learn from the training. If the participants are not learning what they were expected to learn, then the training was not successful, and it’s time for a change.
Second, by asking the participants what they learned from training, you signal very clearly to the staff that you expect them to learn something from the training. That they can use on the job. This expectation provides a basis for accountability. No matter how boring the subject matter or instructor, they are expected to learn something, and if they don’t, then they have not fulfilled their responsibilities for their professional development.
A lot of really smart people have worked to measure learning effectiveness, but I don’t have any first hand experience and can’t really common ton the usefulness of these measures.
But I do know first hand that the best way to gather anecdotal evidence about whether your training is effective is to simply ask the participants what they learned.