Buyer’s Guide to AI Learning Products

AI has come far. But it’s far from perfect.

AI models can generate or produce information, content, or outputs that are not based on actual data or real-world knowledge. This is known as a hallucination and can range from minor inconsistencies to factual inaccuracies to complete fabrications. Because they’re generally coherent and grammatically correct, hallucinations can be very convincing and can be mistaken for fact or believed to be accurate.

Hallucinations in AI models can occur due to various factors, including errors or biases in training data, model complexity, a lack of control and supervision during training, or the prompt itself. In natural language processing, for example, language models sometimes generate text that appears coherent but is entirely fabricated and lacks factual basis. For instance, a language model might generate a paragraph of text that describes a fictional event or scenario as if it were real.

Bias in AI refers to the presence of unfair and unjust prejudices in the decisions or predictions made by artificial intelligence systems. These biases can arise from various sources and can lead to discriminatory or unequal outcomes.

While the stakes might not be as high in learning as they are in healthcare or criminal justice, they can still undermine your programs and can manifest in the following ways:

Training data bias: One of the primary sources of bias in AI is biased training data. If the data used to train an AI model contains biases (e.g. historical, sampling, proxy, etc.) or reflects existing societal inequalities and stereotypes, the model is likely to learn and perpetuate those biases. Designing a system so that it produces gender balanced examples (regardless of the existing gender split in any examples it's trained on) would counteract this.

Algorithmic bias: Certain algorithms may inherently favor one group or produce biased results due to the way they process data. A facial recognition system, for example, might be less accurate for people with darker skin tones, or a voice recognition system might not work as well with regional dialects.

Labeling bias: The people responsible for labeling may unconsciously introduce their biases when tagging images or text. As discussed earlier, AI simply wants to meet the expectations of humans as defined by the patterns in the source data. Therefore any biases represented in the human tagging will be carried through in the statistically-based output of the AI system. It's learned to hold exactly the same amount of bias as a human, which can be problematic.

Feedback loops: AI not only learns from its dataset, but from the interaction it has with humans. If users consistently engage with or provide positive feedback on biased content, the AI system may adapt and reinforce these biases. In other words, AI systems that interact with users may learn biases from user interactions.

In the context of AI for learning, bias is a huge risk to be aware of. No two learners are the same. There's a level of effective AI implementation that brings in a degree of personalization based on what’s unique about each learner, while also leveraging pooled data within your learning platform. The combination of these two strategies can allow AI to adapt to a learner's needs on an individual level.

Mitigating bias in learning AI is critical to ensure that systems are equitable, ethical, and impartial in their decision making and that the learning experience is personalized to the individual (and not prejudiced against them).

Another way for developers to enhance the reliability of an AI model’s output is with a strategy called Retrieval Augmented Generation (RAG).

RAG is an AI framework that uses a trusted knowledge base to enhance Large Language Models (LLMs). It retrieves accurate facts to improve the generation process and ensures that the context is grounded in the latest information.

Remember: the parameters and data initially fed into an AI model are static, and to update them, the model must be retrained. RAG allows language models to skip retraining, and provide access to up-to-date, reliable information. This approach not only enhances the trustworthiness of the information provided but also reduces the likelihood of creating false or misleading content, making it a valuable technique for applications where factual accuracy and context is paramount.

The most effective, and personalized AI systems for learning will connect to and pull from your company’s internal and external knowledge base to ensure learning content is not only comprehensive, but also specific to your business. But, for this to be effective, you need to get your house in order because, as the saying goes, garbage in, garbage out.

A re-emphasis on Knowledge Management

Most L&D professionals already understand the vital importance of knowledge management—yet, it may sometimes feel like an uphill battle to build cross-functional alignment across your business around what is essentially a company-wide project.

The output of any AI model is only as good as the quality of the data. And when the data being referenced by RAG and your AI solution is your company’s internal and external knowledge base, knowledge management becomes mission critical for everyone in the organization.

The referenced knowledge base serves as the backbone for fact-checking and validation, allowing the model to generate accurate and reliable information. If this knowledge base (your organization’s knowledge) is old, incomplete, or inaccurate, the RAG process can lead to the dissemination of incorrect or outdated information.

If you’re investing in an AI product that will reference your knowledge base, knowledge management will be critical for ensuring the quality and relevance of the output.

Consider building (or rebuilding) a cross-functional team around knowledge management in your business. Your team should involve the following groups, and any others unique to your business who are critical in defining truths:

• Product
• Product Marketing
• Legal
• Enablement
• Knowledge Management / Help Desk team

Say, for example, that you adopt an AI for learning whose model has been trained on the majority of the internet and the free (and maybe some premium) courses available out there. You’ve got to ask yourself a few uncomfortable questions:

• Are these courses even good, or pedagogically sound?
• Does the training data and patterns feeding the AI represent the bar I set for my learning programs?
• Can you ever know whether a course is effective without also seeing how learners performed and retained afterwards?

When it comes to AI for learning, GenAI’s greatest strength can also represent one of its greatest vulnerabilities. When a model relies on the patterns present in its training data, it can’t make context-rich decisions that disagree with those patterns. A library of learning theories does not make one a practitioner of learning, or a master facilitator.

Without also understanding specific learner outcomes after a training program, Generative AI’s pattern-based understanding of how to create learning content relies only on whether a course is theoretically effective, and most L&D professionals know how quickly a learning program can fall off track when the learner is not central to its design.

When pedagogy meets AI

A built-in pedagogical model or framework is the linchpin of effective learning AI. It bridges the gap between AI’s technological capabilities and educational effectiveness by aligning the AI model with established educational theories and practice. It provides the foundational principles and frameworks necessary for creating effective, engaging, and impactful educational experiences. Which is important, because the more impactful the learning experience, the more impact learning will have on your business. (Because, as we all know, learning doesn’t just drive personal growth—it also drives business growth.)

Embedding pedagogical strategies can enhance the quality of the learning experience by providing tailored, goal-aligned, engaging, measurable, and personalized learning content and guide the AI processes to ensure learning objectives are met efficiently and effectively.

There are likely a number of low-stakes tasks that we should fully hand over to artificial intelligence, and never look back. We’ve always done this with technology. (Nobody manually operates the city’s telephone switchboard anymore, do they?)

However, there continue to be a number of significant tasks and functions that will always require a human in the loop (HILT). If we look outside of learning to something as fundamentally important as food, we see that AI is already present in agriculture. From weed detection to growth analysis to health monitoring, machine learning models are helping augment and improve the way that we grow food and feed our people. But that doesn’t mean that we don’t or won’t still need farmers to make these crucial decisions on critical systems. Learning is similarly critical, and regardless of how dramatically artificial intelligence improves, it will always require a human-in-the-loop system.

AI systems make predictions and decisions, but they rarely do it with 100% confidence or absolute certainty. In fact, the concept of absolute certainty is very human. That’s why HILT systems and processes are critical. We play a central role in handling the nuances and contextual factors that AI may not fully grasp, ultimately striking a balance between AI-driven insights and human wisdom.

The most powerful AI solutions won’t eliminate the need for human interaction. They’ll enable it at every stage so we can validate and oversee the output (in the broader context in which the AI model is operating), provide feedback and directions, and apply ethical and moral judgment to the decision.