Why is measuring the effectiveness of your eLearning program so important? Well, you need to check if your methods and practices worked. You need to check for possible adjustments and corrections. Most of all, you need to know if the investment was worth it because, even if you have an efficient course production process and a good eLearning project management system, the results might miss the program’s projected goals and objectives.
As with any project, you need to measure the level of success achieved and demonstrate to your stakeholders that it worked well, and that the investment was justified. But how do you go about it? That is the million-dollar question, but just as important as the effectiveness of an eLearning program is for you so is what you need to measure. Which measurements you should focus on are as important as the methodology.
To complicate things, there is no single approach for this task since each project would have different goals and objectives, which in turn change the variables to measure; keep in mind that what is important to you may not be for others. Even if your programs are in the same area, the depth of analysis can change things completely.
The Plan
Before taking any step, you will need to design and document a plan that will set the goals and objectives of this evaluation. It is important to set up a timeline with milestones and establish who will be responsible for each one.
This plan should include the variables to be considered and measured, the methodologies used to collect the data, and the tools used to gain knowledge from the gathered data. The final report should include numbers and qualitative data that can inform those conducting the evaluation of the course.
The team in charge of this plan should include all stakeholders, including learners (they don’t need to attend meetings though), designers, instructors, and all of those who participated in the creation of the program. Participation of learners could be logged in through surveys or focus groups. The work of developing this plan can include conducting work meetings, or they could work collaboratively in a shared document.
Methodology
There are several methodologies you can use to achieve your goals of measuring the effectiveness of your eLearning program. For example, you can concentrate on the achievement level presented by your learners during different evaluations or tests taken after the program concludes. This is one of the most used methods to measure the effectiveness of an eLearning program. On the learner side, there is a certificate that demonstrate that tests were passed satisfactorily. On the program manager side, they can have access to all the data from tests, exams, learner activity in a learning management system (LMS), surveys, questionnaires, and so on. This will provide information regarding learner performance, weak areas in the curriculum, improvements to materials, suggestions for better instruction, and others.
Or maybe you would like to follow on learners’ performance on the job where you could measure the reduction in time achieved after a learner successfully passed a course that is supposed to improve on a procedure, task, routine, or any activity that can be timed. There could be goals that the learner must achieve after they completed a course: reduction in errors during the performance of a job, reduction in wasted time and materials, an amount of money saved because learners achieved goals, and many others. The important thing to keep in mind is that you must establish in the plan for the evaluation of the program what elements you will measure.
Determining the return on investment (ROI) is a methodology used by many, in this case you want to evaluate if the amount of money spent in the program produced a return, which is not necessarily money. Of course, during the early stages of development, you should have figured out if the program would bring more benefits that would outweigh the costs and hence justify the investment. For example, in a higher education setting, it is as important the money spent in creating a program as the money coming in from enrollment.
Measurements and Data
The plan must include the variables that you will measure during the evaluation process, along with the tools and methods considered for the analysis of such data. It is important to keep a data scientist or expert in data representation and reporting, for the design of the visual elements that will help understand the performance of the eLearning program. It is preferable to share the report to all stakeholders prior to any meetings so that each one can derive their own conclusions, which all will discuss during the initial meeting. It may be that the report needs some editing and improvement before it is released to upper management or those interested in the impact the program had in the goals and objectives established in the plan.
Although many of the measured variables may have been collected from a LMS, it is important to consider other variables, such as those that can help improve the usability and experience in the eLearning course (or courses, since this might be a full program). It is as important the performance of learners in eLearning courses as it is the experience and feeling they had while taking any of them. This is not necessarily related to the look or design of the course but to the difficulties that learners had navigating the course in an eLearning environment, for example.
Conclusion
It is important to develop an evaluation plan that could establish the goals and objectives of the evaluation process for an eLearning program. The plan should include all methodologies used and variables to collect for future data analysis. The plan must include the tools and methods to conduct the analysis of such data.
This process will take some meetings between stakeholders, the input from learners may be collected from surveys and focus groups. After the plan is formalized, the implementation will require a responsible party (the program director could perform this function) that will implement the plan in practice. The final report will involve data experts that can produce the data visualization part for stakeholders to evaluate.
The report review will include all stakeholders, it would be helpful to send that report to them prior to the initial review meeting, this will help with bringing ideas to the meeting that can produce discussions that will lead to potential solutions.
As always, find us in social media. You can share this content by using the buttons below. If you have questions and comments, use the form at the bottom of this page, or contact me directly, and I will get back to you as soon as I can. Thank you for reading this post.