Recently, I had the chance to speak at the ATD 2023 International Conference and Expo on Tending to Your Ecosystem: Unifying Content and Data Across Systems along with Bill Conran from our sister company, Watershed. It was a fun session where we covered a lot of ground and had some great questions from the 75 folks who joined us. For those that couldn’t make it, this article originally published on the ATD blog, is a handy summary of what we talked about.
It’s springtime and things are finally blooming again! Whether the early daffodils or the rogue tulips, new growth is always a fun and gratifying reminder of the hard work you put in when planting those bulbs.
Seeing the results of your training efforts can reap a similar feeling. You spent time and care building out your courses and took great pains to deliver them to your learners. Imagine doing all that hard work and never seeing the fruits of your labor. That’s a very real possibility as learning ecosystems grow and become more scattered.
If your organization is like most, chances are your training content is shared across multiple systems—your employees might use one app for on-the-job access and an LMS for longer courses. If you have an extended enterprise use case, you’re likely bringing training to partner LMSs. This list is getting longer as organizations expand their offerings and modalities. From an experience standpoint, meeting learners where they are is ideal.
However, it presents downstream challenges for measuring the effectiveness of training content. So how can we fully reap the training seeds we’re sowing?
Plotting Your Learning Ecosystem
Having visibility into the largest datasets possible will provide insights into what’s working—and what’s not. When your training is delivered across multiple channels and systems, accessing such comprehensive datasets is harder. New systems always sprout up—either by design or default (think inherited legacy systems from an acquisition). Every time a new component enters your learning ecosystem, another level of complexity blooms.
If you’re heading down an already overgrown path or gearing up to invest in more learning tech, then it may be time to evaluate the tools and resources in your stack and how to make them work for you.
Harvesting the Results
Understanding your training programs’ effectiveness is a two-part challenge that requires different lenses. From a content perspective, you want to assess how your courses perform, both collectively and individually. From a learner perspective, you want to see how your training correlates with other business metrics to affect the organization. Thus, we both pan out to see a holistic view of training and zoom in to dig into the details at a granular level. Having a plan to provide both views is a good place to start.
Content Analytics
The Big Picture: Course Catalog Performance
Access to course data in the aggregate, across your entire learner population, allows you to answer questions like: Which titles are used most frequently? Which are used least frequently? This can help you prune out or rethink courses just sitting on the shelf. Knowing which courses to prioritize is incredibly valuable if you have limited time to refine and revise your content.
Zooming In: Course-Level Detail
Seeing how learners across your ecosystem interact with content can be revealing. Are there quiz questions that are consistently answered incorrectly? Does a question need to be redesigned or must the supporting content be revisited? Being able to drill down can help you decide if your learners are failing your courses or if the courses are failing them.
Learning Analytics
The Big Picture: Business Impacts of Your Learning Programs
Your learning analytics and how your training programs correlate with other business metrics can tell a powerful story. How does your training change or contribute to your overall business goals? If you’re looking at budgets and need to impress the value of your training and development programs upon decision makers, a panoramic view that shows the big picture, backed by data, is important.
Begin by measuring the platform or content adoption in your learning ecosystem. Knowing when and where (geographically) individuals interact with specific platforms and content can inform decision-making on redundant systems. You can use these insights to highlight how and where to promote platforms or usher them out (but remember to keep your data!).
Viewing aggregate learning data alongside performance data is often an overlooked practice. Yet, it is critical to creating and maintaining effective programs. For instance, imagine seeing how many more opportunities were created due to your training on effective meeting management for sales?
Zooming In: Learner-Level Insights
Whether you’re viewing a specific person’s actions within a course or how their training activities influenced their work performance, learner-level insights can help with individual coaching. Managers and learners gain value by drilling into the details to understand gaps and places where training has helped performance.
From a compliance perspective, ensuring teammates are current on required training—and the ability to report on it quickly—is a breeze with learning program analytics.
Understanding where learners are stumbling can also provide insight. Is there a problem with the course? Is there a topic we need to focus on? Question-level analysis helps you spot trends at a glance and maximizes the effectiveness of your instructional design team.
Planting the First Seed
Begin with a big-picture plan that considers your content management strategy and data and reporting needs. Consider centralizing all of your courses to create a “content source of truth” that you share access to across multiple systems. This helps you streamline the management of your content, allowing you to centrally update the course for every system, and allows for better visibility into how your training is used across all systems. Similarly, you want to be thoughtful about the data you’re capturing and how you can access and report on it. As you plot out your ecosystem, capturing data from both content and learning perspectives is important to help you grow and cultivate a healthy and effective learning ecosystem.
If you want to get into the weeds with us on content and learning analytics, reach out as we’d love to talk with you.
