A few months back, I mentioned in our DevLearn recap that we are seeing new capabilities and opportunities unlocked by AI.
Although one core problem remains, what did the learner actually do with the AI chatbot or what information was provided? We’ve seen a big advent of training interactions not only facilitated by AI on the backend, but AI is directly involved in the delivery of the learning experience. While the learner gets into the flow of learning resources and dynamic content, the admin may not know which resources or interactions led to improved scores and performance after the fact.
This is where xAPI has a chance to really flourish and create a layer of accountability. We’re definitely not the first to this conclusion. Chad Udell at SparkLearn wrote a great article on AI transparency, and Megan Torrance introduced MAdge, the AI that evaluates AI. I highly encourage you to give both of these a read.
The challenge and opportunity for xAPI with AI
One of the great things about xAPI can also be one of its biggest pitfalls in adoption – there’s no generally agreed-upon structure around statements. For example, nowhere in the xAPI spec is how you should record a completion. We’ve worked around this with loose conventions like Tin Can and, more recently, cmi5.
But there also exist xAPI profiles, which aim to offer a shared vocabulary for xAPI use-cases. The xAPI video profile is a great example, defining verbs like “watched” and “seeked”, etc. It lays out groupings of statement types called templates, and marks extensions as either included (ie. required) or recommended.
We here at Rustici love standards. And in an effort to explore consistency in how AI experiences are structured in xAPI, we are making available our proposed AI Profile for xAPI.
The AI xAPI Profile
This proposed profile is meant to provide a springboard for our teams and yours on what you can do with xAPI and AI. We also wanted to make a profile that provided as much flexibility as possible around the activities in the templates. With the way AI is changing every month it seems, it will be important that it can adapt to your needs as they change. This initial profile includes 3 templates:
The first, interacted-with-ai, is intentionally open-ended and can be seen as a base class from which the other templates are derived. The context information it is collecting is the recommended baseline to capture when dealing with any interaction between learner and AI.
The other two, answered-ai-question and ai-feedback-given, are scoped in a more opinionated way around the context we think is important to record in these interactions. And this context very much revolves around governance and explainability.
Explainability is the theme
If you’ve watched any of our webinars about AI, you probably noticed that explainability is a recurring theme. Anything the AI produces during learning should be explainable in terms of the sources it used, the specific model and provider, and the prompt used to generate that AI interaction or asset.
In the answered-ai-question template, we recommend a collection of both context extensions and result extensions, such as the prompt, model, and the sources used to generate the question. This serves as a valuable audit trail, enabling you to look back and ensure the sources properly addressed the question asked or that it was graded correctly, given the context. It also adds an included extension around an AI confidence score when evaluating a question answered by the learner.
Get started with AI accountability
We hope that this proposed profile helps provide learning platforms and designers with the foundational ideas of how to keep AI accountable in eLearning.
If you have questions about AI in eLearning, our webinars about AI or Rustici Generator, our API content parser, make sure to reach out and ask us anything. And if you’re coming to ATD ‘26 in L.A., come see us at booth 1802! We’d love to talk and hear from you.
