I’ve spent 11 years in the trenches of Instructional Design, LMS administration, and QA management. I’ve seen projects derailed by a single misplaced comma in a compliance module and watched SMEs lose their minds over feedback that missed the mark. For the last 18 months, I’ve been integrating AI into my workflow—not as a magic button, but as a high-velocity drafting tool. And let me be clear: AI is a fantastic engine, but it is a terrible navigator.
The biggest mistake I see L&D teams make right now is treating AI-generated content like it’s a finished product. It isn't. When we use AI to churn out content at scale, our traditional “read-through” QA process fails because it doesn't account for the specific nature of AI errors: plausible-sounding lies, subtle bias, and broken logic. This is why we need to move toward risk-based validation.
What Does Validation Actually Mean in the Age of AI?
Validation isn't just checking for typos. It’s about verifying that the content is accurate, legally defensible, and pedagogically sound. When a human writes a script, they usually have an internal moral and factual compass. When an LLM writes a script, it is merely predicting the next token based on statistical probability. If that probability leads it to suggest a dangerous workplace procedure, it won’t hesitate to do so with complete confidence.
In our field, risk-based validation means adjusting the intensity of your QA process based on the potential cost of an error. If an AI generates a draft for a "Tips for Better Email Etiquette" micro-learning, the risk is low. If it drafts a "Handling Workplace Harassment" flow or a "How to Operate the Industrial Lathe" tutorial, the risk is catastrophic. You cannot—and should not—QA both of these with the same level of scrutiny.
The Spectrum: Low Stakes vs. High Stakes Training
To build an efficient pipeline, you must classify every asset before you start the drafting phase. Using a blanket QA policy leads to one of two outcomes: you either burn out your SMEs with massive, unnecessary review cycles, or you website let a critical error slip through because your process was too thin.
The Risk Assessment Matrix
I use this simple matrix to determine the "QA Weight" for every piece of content my team produces.
Risk Level Definition Example QA Intensity Low Non-critical, cultural, or reinforcement content. Sales motivation, soft skills reminders. Automated spellcheck + Peer edit. Medium Process-based or departmental workflow. How to use the new CRM, team procedures. Fact-check + Targeted SME review. High Compliance, legal, safety, or certification. Diversity training, OSHA standards, data privacy. Deep fact-check, SME sign-off, Legal review.When you start a project, ask yourself: If this content is wrong, does someone get a funny look from their boss, or does someone get sued/injured? That is your guide for how much time you spend on QA.
The 'Gotchas' Doc: Why You Need a Living Record
I keep a running "Gotchas" document—a simple spreadsheet where I record every weird hallucination, recurring grammatical quirk, or logic hole the AI has generated for my team. It sounds tedious, but it’s the most valuable asset in my design library.
Why? Because LLMs often repeat the same specific types of errors. Maybe your tool consistently misses the "Exceptions" paragraph in company policy documents, or maybe it has a habit of using exclusionary language in its persona-building. By tracking these in a "Gotchas" doc, you aren't just QAing the current project—you are building a prophylactic checklist that protects the next project.
Fact-Checking and Source Tracking: The AI "Receipts" Policy
I have zero patience for AI outputs that arrive without sources. If a junior designer on my team hands me an AI-generated draft, my first question isn't "Does this sound good?" It’s "Where is the documentation for this?"
If you are using an AI to summarize a policy, you must mandate source tracking. Here is how I enforce this:
Inline Citation: Every major claim must be followed by a bracketed reference to the source document (e.g., [Employee Handbook, pg. 12]). The "Verification Link": If the AI is used for technical information, the source document must be linked alongside the draft. If it isn’t linked, it isn't reviewed. No "Black Box" Content: If the AI draws from a broad "training set" rather than our internal knowledge base, we mark it as "Untrusted" until it is manually validated against internal documentation.Targeted SME Review: Stop Wasting Time
We’ve all been there: you send a 40-page storyboard to an SME, they send it back with a note saying "Looks good to me," and then they come back two weeks later complaining that the information on page 12 is outdated. That isn't the SME's fault—that’s poor instructional design.
When using AI to generate content, your SMEs should never have to review "for style." That is your job. When you bring in a subject matter expert, use their time for targeted review:
- The "Criticality Query": Don’t ask, "Can you review this?" Ask, "Can you verify that the steps on page 4, specifically regarding the safety interlock, are accurate to our current hardware version?" The "Truth Verification": Provide a list of specific statements generated by the AI and ask the SME to mark them as True/False/Needs Nuance. The Context Check: Ask the SME to identify what the AI *didn't* mention. AI models are often generic; they love to state the obvious but skip the "tribal knowledge" that makes a process work in your specific environment.
The "Break-It" Mentality: Testing Assessments
This is where I get pedantic. I treat every assessment question like I am an employee trying to prove that the training is stupid. AI is notoriously bad at creating challenging, unambiguous questions. It often writes multiple-choice questions where two answers are technically correct, or the distractors are so obviously wrong that the question is a waste of time.
When I review an assessment generated by an AI, I do the following:
Final Thoughts: A Culture of Accountability
As L&D practitioners, we are the stewards of organizational knowledge. AI is a tool, not a colleague. It doesn’t have a conscience, and it doesn't care if it gets the compliance procedure wrong. Your QA process isn't just about catching mistakes; it’s about maintaining the integrity of your company's knowledge base.
Stop saying "looks good to me." Stop accepting corporate fluff. If you wouldn't bet your job on the accuracy of that paragraph, keep editing. In the age of AI, the difference between a high-performing team and a liability is the person holding the red pen. Make sure that person is you.
Have thoughts instructor led training qa on how to better integrate AI into your QA process? Disagree with my risk-based approach? I’m always collecting new entries for my "Gotchas" doc. Let’s hear your horror stories.