Measuring effectiveness requires action before, during, and after taking a custom eLearning piece. Let’s explore 3 tips for each category:
BEFORE:
1. Set Aligned Learning Targets
Write great, measurable learning objectives linked to business goals (LINK HERE to recent objectives article once available). Then use these to guide your content. If your company needs to reduce technical errors, for example, create learning objectives (and the supporting content) that clearly demonstrate learners’ mastery of the problematic technical skills. Be sure you are targeting the right level of cognition. Just creating awareness is likely not enough to improve performance. Getting this right depends on excellence in design – tip No. 2.
2. Do Your Homework Up Front
One of the best advantages of custom eLearning versus an off-the-shelf course covering similar content is your ability to design something truly fit-for-purpose for your organization, audience, and unique circumstance. So take the time to make it right. Talk to the various stakeholders (target audience, their managers, star performers, IT, impacted leaders, HR) to be sure you are completely clear about what is needed in the course, how your audience might best learn it, how it ties to business targets, and what other influences might be impacting performance in this area. If a vendor is helping you develop your custom eLearning, they should have this information before beginning design. A good vendor will even perform these information-gathering sessions for you.
3. Pre-tests/Interviews/Skills Demos
Once you have a great design and know what your custom eLearning content is, you can create a pre-test for your target audience, whether a skills demonstration, written quiz, or interview/questionnaire, to get a baseline metric of their current knowledge and skill level. For practical reasons, you may choose to assess only a small focus group or the first-time participants of your custom eLearning course.
Sample
| Pre-Test Skill Demonstration | |||
|---|---|---|---|
| Section | Skill | Yes | No |
| 5.1 | Supervises back loading of bulk fluids | X | |
| 5.2 | Cleans equipment after use | X | |
| 5.3 | Completes lab reports accurately and uploads successfully | X | |
| 5.4 | Reports spillage according to regulation S-27 | X | |
| 5.5 | Disposes of all waste according to regulation S-25 | X | |
DURING:
4. Quizzes, Tests, Knowledge Checks
Another great feature of custom eLearning is that you can build measurement right into the course. Including knowledge checks, quizzes, and other interactions not only helps hold your learners’ attention and reinforce the key points, but can also help you track learner success. Most learning management systems (LMS) can generate reports so you can track scores on the various test elements in your course.
5. Case Studies, Real-world Scenarios
Since multiple choice questions are rarely sufficient to demonstrate skills improvement, be sure your custom eLearning content includes case studies and real-world scenarios. For very important skills, consider investing in 3D and/or simulation. The more realistic the course is, the more you can make it reflect the learner’s job. That can give learners the chance to practice and explore situations in a safe setting and help improve skills transfer back at work.
6. Feedback on Usability
In addition to getting input about learner skills and knowledge, you can include questions about the quality of the learning experience itself. While not a measure of learning transfer, you are getting data on usability. The best content in the world will not be as effective if the user interface is too clumsy or the learners’ bandwidth was too slow to watch the videos. Tracking usability data also shows that your team is committed to continuous improvement.
AFTER:
7. Post-tests/Interviews/Skills Demo
Once your learners have completed the training, re-test and measure the change. This can be a time-consuming process, but is generally a goldmine of good information. Talk to learners’ supervisors at 30, 90, and 180 days out to see if the learning and behavior change has persisted. Even if the results are not great, you’ll be able to see what skills need refreshing or what part of the training was ineffective from the beginning.
8. Focus Groups
Conduct focus groups from the target audience to test the usability, content, or activities to find ways to improve. Make this at least an annual process to keep tabs on the validity of your content and delivery methods.
9. Leverage All Analytics
Often included in our custom eLearning content are links to other resources or support material to continue the learning. See what learners are consistently referencing after the online experience. Be sure you’re taking advantage of all the metrics in your learning platform to track use. Generally, you’ll be able to track time spent in the course, scores on knowledge checks, who is accessing and who is completing, and if anyone is returning for reference after completion. Using more advanced analytics, you can glean even more granular information – are your learners even clicking on the video midway through your course?
Measuring the effectiveness of custom eLearning content is an ongoing challenge for organizations. Taking the time to truly assess the impact of a learning offering can seem redundant – we’ve already invested so much time in design and development! – but given the cost of custom eLearning and the beneficial data that can be gained, it is a worthwhile process. A continuous improvement loop should be built in to the process from the beginning, as both learners and your company’s bottom line will benefit.
