10 Vital Statistics of a Successful eLearning Program

Like it or not, the number of online learning programs is growing. And, for many organizations, are here to stay. Designing these programs to add value and demonstrating that in-fact they do, has never been more vital.

Here are 10 vital metrics that indicate your online programs are working.

1. Number of Subscribers

Knowing the number of subscribers for the eLearning courses you have can be quite useful. It helps you keep track of the courses that are actively being sought out by participants. It also enables you to work on and deploy relevant courses that will keep the participants engaged in their learning process.
Similarly, a lack of subscribers will be a matter of concern. You will need to go back to the drawing board and work on the courses. You can collect user feedback to determine the possible reasons for the lack of subscription.

2. Source from which participants enroll in the course

Do your participants enroll directly at a specific program or, do they shop around a bit before finding one that they like? Are they being referred to these courses by former participants? Are they being asked to take the eLearning courses by their supervisors?
Knowing the answers to these questions will help you prepare the programs accordingly. For example, if the primary sources are supervisors, you can promote a new eLearning Program through the supervisors. The supervisors can then encourage their team members to enroll and complete the new program.

3. Progress Rate

Progress rates indicate participants are consuming the content.
By measuring this, you will immediately notice the extent to which participants are engaging with the program. For example, the progress rate of programs will tell you:

• Whether or not participants are logging in
• The speed with which they are consuming the content
• The point at which they stop or drop out completely

Progress rate can also help you get an idea of where your participants are stumbling. This way, you can make rapid changes to increase the rate of progress and completion.

4. Completion Rates

Only knowing the progress rates is not enough. You must also measure the program completion rates. After all, it is not uncommon for people to start a course only to quit it midway.
For courses that lack satisfactory completion rates, there must certainly be some reason why people are quitting them midway. You can then conduct user surveys to find out what those reasons are.
User surveys will provide you with valuable data needed to improve the program so that more people complete it. After all, there is no point in having courses that people don’t complete. After all, it takes effort and money to create and make these programs available.

5. User Feedback

Gathering feedback from participants at different levels of the course provides useful information if you ask the right questions. Specific measures, such as those indicating Content Relevance and Commitment to Use, provide an indicator of value from the participant perspective. 

Net Promoter Score (NPS) can be a useful indicator and sometimes a predictor of application and impact.

The key to NPS, something that is often missed by professionals, is to ask both questions in the series when used in learner reaction feedback surveys.  While asked differently, depending on the context, the first question could be:

How likely would you recommend (name of program, coach, etc.) to others in similar positions like yours? (using a 0 – 10 point scale)

Scores of 0-6 represent detractors; 7-8 are neutral; 9-10 are promoters. Subtract %detractors from %promoters to get your NPS.
The second question is the gold mine that learning professionals often overlook. That question is:

Why did you score the way you did? (open text)

This second question provides the opportunity for improvement.
Other valuable feedback measures include those that indicate the online learning process works.  Questions around ease of access, learner platform preference, and most relevant content can help designers and developers identify opportunities to address technology issues as well as eliminate content not useful to the audience.

6. Knowledge Acquisition Scores

Learner scores are essential for any learning program. As such, this statistic is easily one of the most important for eLearning courses as well. These scores provide an understanding of how well the participants have understood the subject. A program can only be impactful if the learner has understood the topic. For example, if there are lots of employees with low scores, it is a matter of concern. In such situations, you need to identify the reasons.

7. Breakdown of Answers and Attempts

Here, you will be gathering data on the response distribution of the participants for every question in a quiz/assessment that is a part of the course.
With the breakdown, you can start analyzing the assessment’s accuracy and efficacy. You will also get to know if there are any issues with the quizzes/assessments.
If the participants are attaining a low overall score, then it could be due to common mistakes by the participants or an issue with the program content. It could also be due to some questions that are too difficult for the participants. In such cases, you may have to insert additional materials in the course to cover that section in detail.

8. Application of Learning

Let’s say you have a lot of courses with excellent progress and completion rates. Does that mean your courses are successful?
A course can only be successful if the participants apply what they have learned during the program. The application of learning will bring about a positive change in the organization in some way. This change could be in terms of the performance of the employees or the bottom-line of the organization.
As such, you need to determine the application rates of the courses. Low application rates are a matter of concern. You will have to find out why the participants are not using their newly learned knowledge or skills and take the relevant action. Further courses may be necessary. Alternatively, it is also possible that some external factors are preventing the application of those skills.

9. Impact of the Learning

The value of learning, whether online or otherwise, comes from what learners do differently with what they learn. And what the consequence is of their change in behavior or actions. This impact can get calculated by measuring the improvement in output, quality, cost, time, customer satisfaction, job satisfaction, work habits, and innovation. Connecting online learning to improved business measures is imperative if the program is a significant enough investment. Isolating the effects of learning on improvement in these measures, while challenging in some cases, is doable – especially if the program was designed, right from the outset, to deliver that improvement.

10. ROI of eLearning Programs

The ROI of online learning is calculated the same way as that of programs delivered through other modes. Converting the impact of the programs to monetary value and comparing to the fully-loaded cost of the learning programs generates an easy to understand metric. This metric clearly shows the pay off of the investment. The two measures most useful in describing the economic benefits of an online learning program are:

Benefit/Cost Ratio (BCR): Benefits/Cost And Return on Investment (ROI): Net Benefits/Cost x 100

To gain the confidence of senior leaders and to attain approvals for future projects, one should measure and present the ROI of eLearning programs to senior leaders.

You may think that it is challenging to measure effectiveness and ROI of an eLearning Program. However, that is not the case. Following the right process and principles will help you through your journey in showcasing the value of your eLearning Program.

This article was first published on Association for Talent Development's website on Tuesday, August 25, 2020