Handling AI Hallucinations in Production

LLMs sometimes generate confident-sounding but incorrect information. In production automation, this can cause serious problems. Here's how to minimize and handle hallucinations effectively.

Understanding Hallucinations

Types of Hallucinations

Factual Errors:
- Wrong dates, numbers, or names
- Invented facts or statistics
- Incorrect technical details

Format Violations:
- JSON that doesn't parse
- Missing required fields
- Wrong data types

Logical Inconsistencies:
- Contradicting earlier statements
- Impossible combinations
- Circular reasoning

Confidence Without Basis:
- Answers when it should say "I don't know"
- Making up data to fill gaps
- Extrapolating beyond given information

Why They Happen

• Training data limitations
• Ambiguous prompts
• Pushing beyond model capabilities
• Lack of grounding data
• Temperature too high

Prevention Strategies

1. Constrain Outputs

Limit what the model can return:

// Instead of open-ended:
"What category does this belong to?"

// Use constrained:
"Classify this as one of: [support, sales, billing, spam]
Return ONLY the category name, nothing else."

2. Provide Grounding Data

Give the model source material:

// Instead of:
"What are our refund policies?"

// Use RAG:
"Based ONLY on the following policy document, answer the question.
If the answer is not in the document, say 'Not found in policy.'

Policy Document:
{{policy_text}}

Question: What are our refund policies?"

3. Lower Temperature

• Use temperature 0 for factual extraction
• Use 0.1-0.3 for classification
• Only use higher temperatures for creative tasks

4. Structured Output

Request JSON with schema:

const schema = {
  "type": "object",
  "properties": {
    "category": {"enum": ["support", "sales", "billing"]},
    "confidence": {"type": "number", "minimum": 0, "maximum": 1}
  },
  "required": ["category", "confidence"]
};

"Return a JSON object matching this schema: " + JSON.stringify(schema)

Detection Strategies

1. Schema Validation

Always validate structure:

function validateOutput(output, schema) {
  try {
    const parsed = JSON.parse(output);
    const valid = ajv.validate(schema, parsed);
    if (!valid) {
      throw new Error('Schema validation failed');
    }
    return parsed;
  } catch (e) {
    handleHallucination(output, e);
  }
}

2. Confidence Thresholds

Require confidence scores:

const result = await callLLM(prompt);

if (result.confidence < 0.7) {
  // Flag for human review
  await flagForReview(result);
  return null;
}

3. Cross-Validation

For critical tasks, use multiple approaches:

async function extractWithValidation(document) {
  // Get LLM extraction
  const llmResult = await llmExtract(document);

  // Get rule-based extraction
  const ruleResult = ruleBasedExtract(document);

  // Compare key fields
  if (llmResult.total !== ruleResult.total) {
    await flagDiscrepancy(llmResult, ruleResult);
  }

  return llmResult;
}

4. Reference Checking

Verify against source data:

// If LLM extracts a date
const extractedDate = result.date;

// Check if date appears in source
if (!sourceText.includes(extractedDate)) {
  await flagPossibleHallucination('date', extractedDate);
}

Recovery Strategies

1. Retry with Clarification

async function extractWithRetry(text, maxRetries = 2) {
  for (let i = 0; i < maxRetries; i++) {
    const result = await llmExtract(text);

    if (isValid(result)) {
      return result;
    }

    // Add clarification to prompt
    text = `Previous attempt returned invalid data.
    Please try again, ensuring:
    - All required fields are present
    - Values are from the source text only
    - Format matches the schema exactly

    Source text:
    ${text}`;
  }

  return flagForHumanReview(text);
}

2. Human-in-the-Loop

For low-confidence or critical decisions:

if (result.confidence < threshold || isCritical(task)) {
  await queueForReview({
    task,
    llmResult: result,
    source: input,
    reason: result.confidence < threshold ? 'low_confidence' : 'critical_task'
  });

  // Either wait for review or proceed with caution
  return { status: 'pending_review', provisional: result };
}

3. Fallback to Rules

Have rule-based backups:

const llmResult = await llmExtract(document);

if (!isValid(llmResult)) {
  console.log('LLM extraction failed, using rules');
  return ruleBasedExtract(document);
}

return llmResult;

Monitoring Hallucinations

Track and analyze:

• Validation failure rate by task type
• Common hallucination patterns
• Recovery success rates
• Human review volume

Use this data to improve prompts and detection.

Communicating with Clients

Be honest about limitations:

• "AI extraction has ~95% accuracy"
• "Low-confidence results are flagged for review"
• "Critical decisions have human oversight"

Don't promise perfection. Promise good processes.