The use of microlearning as part of a broader training strategy is an expanding practice. Determining the efficacy of microlearning alone is challenging, however, in large part due to the inherent nature of this learning method – students learn on their own time, often one very specific concept or task. Key performance indicators (KPIs) are critical to demonstrating the value of learning programs, and there are a wide variety to choose from in our industry. Which ones will provide you with the most accurate, actionable information? How can they be accurately measured? These are the questions we’ll be looking at here.
Using KPIs to measure the success of learning programs means continually evaluating job goals, employee performance, and the organization overall. For new learning, you start by focusing on KPIs specific to the learning itself, maintaining an awareness that, as the program grows (as most blended learning programs do), you’ll need to add KPIs that focus on how the learning impacts business results.
When introducing microlearning as part of a blended learning program, there are a few common KPIs to consider. At Obsidian, we distinguish two categories: inflexible KPIs and emotional KPIs. Inflexible KPIs are the tangible and trackable indicators like completion, pre and post assessments, project results post learning, etc. Emotional KPIs are related to behaviors and attitudes, and though more challenging to measure, can provide a wealth of information about the actual effectiveness of the learning offering.
The conventional means of measuring satisfaction with a learning deliverable are satisfaction surveys and learner feedback. In our opinion, the essential emotional KPIs important to learning programs in general and microlearning in particular are:
- Employees’ career progression pathway rate
- Engagement rate – pulse survey
- Likes and dislike rate
- Learning sentiment
Let’s examine how to measure these 4 KPIs, and how they can help you evaluate if your learning is hitting the mark…or if you need to go back to the drawing board.
1. Employees’ Career Progression Pathway Rate
Employees should acquire concrete benefits from any learning course. While career pathways are usually the HR department’s responsibility, training should impact an individual’s rate of progress. Hence, this specific KPI looks for an acceleration in the career progression pathway, salary increases, or movement toward cross-functional roles.
This KPI is not immediate, and it must be tracked over time, but microlearning is ideal for concrete measurement of results. If an employee’s next step in the career path is the next management level, one can clearly define the specific goals and competencies needed. Since microlearning is a very focused deliverable, the content can easily be linked to a measurable goal, hence to a result. When an employee can take their learnings and convert them into a personal value added, the learning can clearly be deemed successful.
2. Engagement Rate
Most employees participate in training, fulfil completion, and often even the assessments, but few provide feedback and interact with the content. Even fewer create the content. Innovative approaches like microlearning, gaming, and simulations create more avenues for engagement in learning programs. The key to measuring engagement rates is to provide the learner with multiple ways of interacting with the content. The use of pulse surveys instead of annual surveys or formal surveys tied to performance discussions provides more accurate data. The best way to encourage employees to participate in any survey is to act on their feedback. The response to their suggestions should be swift; otherwise, trust is broken and employee interest and involvement quickly diminish. Pulse surveys are also well suited for microlearning because, just as the learning itself, they are short and focused. Pulse surveys administered immediately following a learning experience are more likely to gather honest, spontaneous feedback.
3. Like and Dislike Rate
This is an easy KPI to track in your LMS. Just like social media, you can set up each microlearning (and each learning deliverable) with a like/dislike button and comments area. Generally, users are more inclined to click a like/dislike button than write specific comments, so this data is readily gathered and easily managed. Is this information significant? On its own, this KPI can give you a very quick sense of overall employee satisfaction with the learning. Combining this KPI with pulse surveys and sentiment analysis, however, would provide a more comprehensive view of the learning program’s success. In the example below, the higher the like/dislike rate, the more successful the learning. A rate lower than 1 represents clear dislike of the course. The response rate is high even for unsuccessful courses, indicating that even after an unsatisfactory learning experience participants remain willing to provide a quick opinion on the value of the learning experience.
| Item# | Likes | Dislikes | Completions | Like/Dislike Rate | Response Rate |
|---|---|---|---|---|---|
| CXM3245 | 23 | 75 | 110 | 0.31 | 71% |
| CXM1245 | 234 | 14 | 265 | 16.7 | 93.5% |
| CXM2301 | 76 | 7 | 100 | 10.8 | 83% |
| CXM2207 | 475 | 12 | 550 | 39.6 | 88.5% |
| CXM3567 | 32 | 140 | 200 | 0.22 | 86% |
4. Learning Sentiment
This KPI is determined by sentiment analysis. Sentiment analysis utilizes analytical models to determine the emotional tone of a person based on a series of words or a textual string. This data is analyzed to help understand behaviors, attitudes, and emotions, with the end result of providing actual satisfaction trends. Though historically used in marketing and customer service, sentiment analysis has become a valuable learning tool in large part due to the development of xAPI.
To determine the sentiment about a learning event, you need to create an associated online forum where users can comment, discuss, and express opinions. An algorithm searches for specific neutral, positive, and negative words, and determines a satisfaction ratio. Employee feedback is mostly subjective, with a high possibility of biased answers and poorly worded opinions. Sentiment analysis can cut through a large volume of very diverse feedback while determining trends and outliers. For example, a large group of students might complain about a particular course simply because they may not have the correct browser or player on their computers. Sentiment analysis can highlight these issues, opening the door to transparent conversations.
All learning should be implemented with an associated continuous improvement loop. Tracking employee satisfaction KPIs can help evaluate if your learning offerings are truly benefiting your employees and your organization. Properly taken into account, the employee satisfaction KPIs discussed here can help determine the success – or lack thereof—of your microlearning strategy. And if you are missing the mark, implementing changes based on their results will help get you back on track.
