By
Rose Polchin
|
Date Published: July 01, 2014 - Last Updated September 08, 2020
|
Comments (1)
Over the course of 2014, we’ve explored a number of topics from chat to social media to one that I am exceptionally passionate about this month, learning and development. Every month, there’s been a discussion around measures, specifically what the most effective metrics are for that particular area. In keeping with that theme, it seems appropriate to put the spotlight on ways to measure the results of our investments in training.
One of the most well-known approaches to measuring training effectiveness is Kirkpatrick’s Four-Level Training evaluation model. And whether you are a trainer looking for a way to report on the impact of training or a stakeholder/budget holder trying to determine if your budget dollars are being invested wisely, this model can help objectively analyze the effectiveness and results of training programs and identify ways to improve them for the future.
So let’s start by taking a look at the four levels and then identify areas where we can apply them. Then, we’ll finish up with some key points to consider in determining if/when to apply each of the 4 levels.
We need to first give credit where credit is due, to Donald Kirkpatrick, Professor Emeritus at the University of Wisconsin. He first published his Four Level Model in 1959 in the U.S. Training and Development Journal. The model was updated in 1975 and again in 1994, when he published his best-known work, “Evaluating Training Programs.” (Source: www.kirkpatrickpartners.com)
The following is a summary of the four levels along with a few suggestions for implementing measurement at each level.
Level 1: Reaction: To what degree participants react favorably to the training.
To measure reaction, develop ways to address some of the following questions:
- Did the seminar participants believe that the training was worth the investment of their time?
- What topics were most valuable; least valuable?
- Would they recommend this course to colleagues?
Level 2: Learning: To what degree participants acquire the intended knowledge, skills, attitudes, confidence, and commitment based on their participation in a training event.
To measure learning, consider the following:
- What do you need/want to evaluate? Was the goal of the training program to change skills, knowledge, attitudes, confidence and/or commitment?
- It is often useful to test participants in this area before they attend the training and then again after they attend the session.
Level 3: Behavior: To what degree participants apply what they learned during training when they are back on the job.
To measure level 3, develop ways to address some of the following questions:
- Did the trainees put any of their learning to use?
- Are trainees able to teach their new knowledge, skills, or attitudes to other people?
- Are trainees aware that they've changed their behavior?
One of the most effective ways to assess behavior change is to conduct observations and interviews over time.
It is also important to keep in mind that the environment the trainees return to impacts whether or not they can apply what they learned. If the culture, management and reward systems do not support behavior change, then trainees may not be able to apply what they have learned.
Level 4: Results: To what degree targeted outcomes occur as a result of the training event and subsequent reinforcement.
To measure level 4 results it is important to determine which outcomes are the most directly linked to the training program. Then you’ll need to develop ways to measure these over time.
Here are a few potential benefits/outcomes to consider, that are often linked to training programs. (Please note: not every item on the list will apply to every training situation and this list is not exhaustive, there may be others that do not appear on the list.)
- Increased employee retention
- Increased employee engagement
- Increased productivity
- Increased customer satisfaction/retention
- Decreased errors and rework
- Increased sales/deepening of customer relationships
- Fewer customer complaints
(Source:
www.kirkpatrickpartners.com
)
A couple of important notes about this model:
- Evaluation should always begin with level one, and then, as time and budget allows, move sequentially through levels two, three, and four.
- Information from each prior level serves as the foundation for the next level's evaluation.
- Thus, each successive level represents a more precise measure of the effectiveness of the training program, but at the same time requires a more rigorous and time-consuming analysis.
I will be the first to admit calculating the ROI (level 4) on training is challenging for many reasons. However, that doesn’t mean we shouldn’t try! As you consider what levels you would like to pursue, it is important to consider some of the following if you are going to go to level 4 (and to a degree level 3).
- The time and resources required to use levels 3 and 4.
a. For example, it may not be practical if your organization doesn’t have a dedicated training department.
b. In addition, you need to assess the system, tools, and processes your organization has in place to collect the data necessary to report on these levels.
- The impact of other variables on all four levels. Most organizations experience change in a variety of ways and the rate and pace of those changes can influence behaviors and results as much, if not more, than the training itself. For example, the implementation of a new more user-friendly website may be the reason for an increase in customer satisfaction, rather than or in addition to the Customer Service Refresher training program all representatives just attended.
And finally, remember that measures only tell you “how you are doing”, it is up to each of us to find root causes, determine the appropriate next steps, and then to take action!
More Resources