Learning Analytics: How to Measure What Matters in Training Programmes

In this article, we look at how to measure what really matters in training programmes with effective learning analytics. We cover which metrics deserve the most attention, and how carefully designed elearning can generate insights, as well as impact.

Organisations invest significant time and budget into learning and development, but one question always matters most: is the training actually making a difference?

Learning analytics helps answer that question. It gives L&D teams the evidence they need to measure engagement, spot knowledge gaps, improve content and connect training to wider organisational goals.

What is learning analytics?

Learning analytics is the collection, analysis and interpretation of data about how people engage with training. It helps organisations move beyond assumptions and make better decisions about what is working, what needs improving, and where learning is contributing to performance.

At its best, learning analytics supports a more strategic approach to L&D by showing not just whether learners completed a course, but whether they understood the content, applied it and changed behaviour as a result.

Why completion rates are not enough

Completion data still has value. It can highlight participation trends, identify drop-off points and show whether mandatory learning is being accessed. But on its own, it doesn't say much about real effectiveness.

A course can have a strong completion rate while still failing to improve knowledge, confidence or performance. That's why organisations need a broader measurement framework that looks beyond surface-level metrics.

Strong learning analytics should consider four areas:

• Engagement - are learners actively interacting with the content?
• Knowledge - are they understanding and retaining key messages?
• Behaviour - are they applying what they learned in practice?
• Results - is the training contributing to measurable business outcomes?

The learning metrics that matter most

1. Engagement with the learning experience

Engagement is one of the earliest indicators of whether a training programme is landing well. Metrics here might include starts, completions, time on task, return visits, interaction rates and knowledge check participation.

But engagement is not just about volume. It is about whether learners are actively involved in the experience. 

For example, at First Media we created an interactive series of modules in Bite-Size Learning for The National Trust. Microlearning content, timelines, flip cards and knowledge checks have helped both staff and volunteers more easily engage with core organisational messages.

2. Knowledge retention

If the goal of training is understanding, retention needs to be measured. This can be tracked through quizzes, scenario responses, repeat assessments, confidence scores and post-learning checks carried out after the training itself.

When content is broken into manageable, well-designed pieces, it is often easier for learners to absorb and recall.

Our Microlearning for Biobest involved converting over 120 PowerPoints into digital learning and delivering over 100 micro learning modules, showing how large, information-heavy training programmes can be restructured into a more measurable and usable format.

3. Behaviour change

For many training programmes, behaviour change is the real objective. This is especially important in compliance, leadership, customer service, inclusion and workplace culture initiatives.

Behaviour-focused analytics may include observation data, manager feedback, learner self-assessment, follow-up surveys, performance reviews or error reduction over time.

First Media’s West Yorkshire Fire and Rescue Service case study is a good example of this kind of approach. The programme was made up of two modules - Data Literacy and Data Literacy Essentials - designed to help learners understand the role of data both in everyday life and in the workplace.

That kind of design supports more meaningful evaluation because it is linked to practical understanding and workplace application.

4. Efficiency and scalability

Another useful measure is how efficiently learning can be delivered at scale. This may include speed of rollout, consistency of message, ability to serve different audiences and reduction in trainer time or duplicated effort.

For organisations with large or diverse learner groups, digital learning analytics can help identify how content performs across locations, departments or learner types, making it easier to refine and improve delivery over time.

5. Alignment with business outcomes

The most valuable learning analytics connects training back to business priorities. Depending on the programme, that could mean stronger compliance, improved customer experience, reduced risk, increased confidence, faster onboarding or better operational consistency. 

For instance, our elearning for Crown Commercial Services focused on an interactive guide to help users reduce their carbon footprint and begin their journey towards carbon net zero.

In projects like this, the learning itself should be measured against the wider campaign or organisational objective, not just participation figures.

How to build a practical measurement framework for training programmes

The best time to think about learning analytics is at the start of a project, not after launch. When measurement is designed in from the outset, it becomes much easier to capture meaningful data.

A practical framework often includes:

• Clear objectives - what should learners know, feel or do differently?
• Defined success measures - what evidence will show the training has worked?
• Built-in tracking points - where will data be captured during the learner journey?
• Review cycles - how often will the data be assessed and acted on?

This approach helps organisations avoid collecting data for its own sake. Instead, every metric has a purpose and supports better decision-making.

Questions to ask when measuring training effectiveness

If you want to measure what matters in training programmes, start with these questions:

• Are learners engaging with the content in the way we intended?
• Can they demonstrate understanding during and after the programme?
• Are they applying the learning in real situations?
• Has the training helped move a business metric in the right direction?
• What does the data suggest we should improve next?

These questions shift the conversation from activity to impact, which is where learning analytics becomes most valuable.

Why better learning analytics leads to better learning design

One of the biggest advantages of learning analytics is that it does not just prove value. It also improves future learning.

When you can see where learners hesitate, where they disengage, which content performs well and which messages need reinforcing, you can continuously refine the experience. That makes training more targeted, more effective and more aligned to organisational needs.

It also allows L&D teams to demonstrate strategic value more clearly, especially when they can show a line between elearning design and measurable outcomes.

To sum up

Learning analytics is not about collecting more dashboards or reporting more numbers. It is about identifying the evidence that genuinely shows whether training is working.

Completion rates still have their place, but the most valuable insights usually come from a wider picture that includes engagement, retention, behaviour and business results.

When learning is designed with measurement in mind, organisations can build stronger programmes, make smarter improvements and show exactly how training supports performance.

If you are looking to create digital learning that is not only engaging, but measurable too, explore more of First Media’s case studies to see how different organisations are using learning design to tackle real business challenges. 

Or contact us for an informal discussion of your Learning & Development needs.