“Happy Sheets” have been given a violent battering over the years by otherwise peaceful L&D folk. Otherwise known as Level 1 reaction sheets on the Kirkpatrick evaluation model, they were recently nominated by learning professionals at #LearningLive to go into “Room 101”.
I understand they were saved, not by a last-minute burst of pity, but by an even greater collective loathing for words such as “millenials”, “passionate” and “blended”. Fair enough.
The Room 101 nominators were probably thinking more of Paul Merton’s lever than George Orwell’s 1984, but let’s start from the thought that handing out evaluation forms is like a living nightmare in which rats are eating our face.
Why do we dislike Level 1 so much?
I suspect there is a logical reason, and a psychological reason. This post deals with the logical reason.
This is a defence of Happy Sheets, but a very specific and partial defence.
I completely agree with the criticism that Level 1 sheets tell us little or nothing about what people have learnt. Donald Clark expresses it brilliantly in this general demolition of Kirkpatrick. “Learners can be happy and stupid.. Learners often learn.. through experiences which, although difficult at the time, prove to be useful later. Happy sheet data.. is often a skewed sample from those that have pens.”
Agreed. Move straight to Level 4. Do Not Pass Go. Even better, find another model.
But there are also two genuine, real-life benefits of Level 1:
- the collection of raw data on the customer experience
- powerful live feedback for trainers and facilitators.
I saw both of these at the National School of Government, where I went from a complete novice to a fairly experienced trainer and facilitator over six years. The National School, for all its many flaws, had a strong culture of collecting and analysing data to support an operation involving hundreds of trainers and a £30m annual turnover.
The raw data was used in various ways. It revealed variation in performance among trainers who were delivering the same course, supposedly to the same standard. You got to spot who the weak trainers were. (Or maybe they were trainers who were giving their participants a deliberately disruptive experience, enhancing their eventual learning at the expense of short-term popularity – but – no.. – just weak trainers, probably).
Some scores exposed flaws in design. Falling scores revealed complacency. Free text comments revealed some trends. Scores even revealed whether the food in the canteen was improving or deteriorating.
The overall figures meant nothing (we always scored over 90% on the key questions), and no learning should have been inferred – but the variations, the sudden dips and the overall trends did have value. That is the partial, “raw data” defence.
More importantly, though, the forms gave instant feedback to trainers and facilitators. I improved hugely through reading the evaluation forms.
It was a standing joke that the trainers would stand around nonchalantly at the end of the day, chatting and smiling as participants gathered their bags, put on their coats and left the Happy Sheets on a table – but as soon as the door closed on the last wheeled cabin-bag, we would dash for the feedback.
The next two minutes were quiet, with occasional grunts of acknowledgement, and the odd hiss of “bastard”. It was an acknowledged “end of course syndrome” that you would travel home brooding on the one negative comment and ignoring the ten good ones.
Some comments were hurtful or unfair, and many were bland and useless, but I found some were gold dust. Most importantly, I began to see how my own preferences influenced my design, and clouded my judgement.
That is the “Level 1 as Continuing Professional Development for Designers and Deliverers” defence.
So this dislike of Happy Sheets – the splash of blood on the yellow smile – I see where it comes from, on the rational side. We should all be frustrated at organisations which equate mechanistic scoring with actual learning. That means we should find better evaluation systems, but not, I think, that we should dump immediate customer feedback – it’s too valuable for that.
image credit: DC Comics