:::tip 🎮 Interactive Playground Visualize this concept: Try the LLM Product Architecture demo on the EngineersOfAI Playground - no code required. :::

AI Product Design Principles

The Launch That Almost Broke Everything

It was 11:47 PM on a Thursday when the product team lead sent a Slack message: "Shipping tomorrow. Let's celebrate." Six weeks had gone into "SmartDraft" - an AI writing assistant embedded directly into their enterprise SaaS platform. The model was excellent; internal testers loved it; the board demo had gone better than anyone expected. Every internal quality metric was green. The engineering team had spent 600 person-hours fine-tuning prompts, benchmarking response quality, optimizing latency. The model could draft a professional email in under two seconds and had passed every test case the team had devised. By every measurable standard, this was the highest-quality feature the company had ever shipped.

Forty-eight hours after launch, 340 new tickets had flooded the support queue. Not because the AI was wrong - it was right roughly 85% of the time. But 85% accuracy turned out not to be good enough when the product had no mechanism for surfacing the 15%. Users were pasting AI-generated emails directly into client communications without reading them. When the AI confidently produced an incorrect contract date, the support ticket wasn't "the AI made a mistake." It was: "Your product made me look incompetent in front of my largest client." One user had an AI-drafted email sent to a prospect referencing a pricing tier deprecated eight months prior. Another user had the AI summarize a legal document and present the summary verbatim in a client meeting - the AI had missed a critical liability clause worth hundreds of thousands of dollars. A third user had the AI draft a personalized message to a client it addressed by the wrong name, because the context provided was ambiguous and the model confidently filled in a plausible-but-wrong detail.

What followed was two weeks of emergency redesign. Not of the model - the model was fine. Of everything around the model: how the UI presented uncertainty, how it scaffolded user review, how it communicated limitations, how it degraded when confidence was low. The team had spent six weeks perfecting what happens when the model works, and zero time designing what happens when it doesn't. The second version shipped with confidence badges, mandatory review prompts for high-stakes outputs, a clear "AI generated - please review before sending" notice, and highlighted numbers and names that users should verify. Same model. Same accuracy rate. Radically different product. Support tickets for the second version: 12 in the first 48 hours. This lesson is the disciplined framework those engineers learned - the set of principles that separates AI features that impress in demos from AI products that hold up in production.

---

Why This Exists: The Demo Gap

Every AI system performs better in controlled demos than in production. This is not a model quality problem - it is a product design problem. The demo gap exists because of four structural mismatches between demos and production:

1. Demos use curated inputs. Real users send edge cases, ambiguous queries, incomplete context, and things the model was never trained to handle well.
2. Demos don't show failure modes. Nobody demos the moment the model produces a confident wrong answer. Investors and stakeholders see 100% of the model's best outputs and 0% of its failures.
3. Demos don't capture trust dynamics. A user who sees one impressive demo is in a fundamentally different psychological state from a user who has been burned by an AI error once before. Burned users are hypervigilant and sometimes hostile.
4. Demos have a present guide. When a product manager demos AI, they guide the interaction, select the best follow-up, and steer away from failure paths. Real users navigate alone.

The discipline of AI product design is fundamentally about closing this gap - not by making the model better (that is ML engineering), but by designing the human-AI interaction so that model limitations don't become product failures. The model's accuracy on a benchmark tells you nothing about how safe and useful your product will be in production. Product design fills that gap.

---

The Seven Core Principles

---

Principle 1: Graceful Degradation

In traditional software, a feature either works or it doesn't. In AI products, there is a spectrum: the model might produce an excellent answer, a mediocre answer, or a wrong answer - and from the outside, all three look identical unless you design for the difference.

Graceful degradation means your product remains useful and safe even when the model underperforms. This requires three concrete design decisions:

Confidence-aware output: Don't show AI output the same way regardless of how confident the model is. Surface uncertainty. "I'm not sure about this" is a valid and valuable response. The UI should reflect the model's self-reported confidence with visual cues - a yellow border, a "review before using" badge, or a caveat text block. Confidence signals are not UX failure - they are trust-building mechanisms. Users who can see when the AI is uncertain are more equipped to catch errors than users who receive all outputs with equal authority.

Fallback content: When the model fails or produces low-quality output, have a defined fallback: a human-authored template, a simpler rule-based response, or a clear "we couldn't generate this" message with a manual path. Never leave users stranded with a raw error message or, worse, an empty content area with no explanation. Every failure path in your AI product should be explicitly designed, not implicitly ignored.

Non-blocking design: AI suggestions should enhance a user's workflow, not block it. If the AI fails to generate a summary, the user should still be able to read the original document. If the AI can't draft an email, the user should still be able to write one manually. Never make the AI output the only path through a user flow. This is the single most common mistake in AI feature design: building a flow where the AI failure state leaves the user with nowhere to go.

The edit delta - the difference between what the AI produced and what the user kept - is one of the most valuable signals in your product. Log it. High edit rates on a specific output type tell you precisely where the model is underperforming for your users, which is far more actionable than a generic benchmark score.

---

Principle 2: Progressive Disclosure of AI Capability

Don't show users everything the AI can do on day one. This sounds counterintuitive - shouldn't you want to demonstrate maximum value immediately? The problem is that users who encounter AI limitations before they understand the system's strengths tend to write off the entire feature. Trust, once lost early, is extremely difficult to rebuild. The user's mental model of your AI is built from their first five interactions; bad early experiences produce a pessimistic prior that persists long after the model has improved.

Progressive disclosure means introducing AI capability in layers, starting with the highest-confidence, lowest-stakes use cases:

Layer 1 - Autocomplete and suggestions: Low commitment, easily ignored if wrong. Users learn the system's "flavor" without risk. Think of GitHub Copilot's single-line completions - they're easy to accept or reject, and users build intuition about what the model is good at before ever trusting it with anything important.

Layer 2 - Drafts and summaries: Higher value, but presented as a starting point, not a final output. User review is built into the workflow. The AI generates a first draft; the user polishes it. The UI language matters enormously here: "Draft" and "Starting point" create appropriate expectations. "Response" and "Answer" do not. Users interpret the word "Answer" as authoritative in a way they don't interpret "Draft."

Layer 3 - Automated actions: Only after the user has built enough trust through layers 1 and 2 should you introduce actions the AI takes without explicit per-step user confirmation - auto-filing a ticket, auto-sending an email, auto-updating a record. These actions require the highest level of trust and should be opt-in, not default.

Many AI products fail because they launch at layer 3. The user hasn't built the mental model of what this AI is good at, hasn't learned to read its confidence signals, hasn't developed the habit of spot-checking. Giving a user autonomous AI actions before trust is established is a recipe for a catastrophic failure event that destroys trust for months.

:::tip Progressive Trust Building A concrete cadence that works in practice: Week 1 - suggestions only, no auto-actions. Week 2 - introduce draft generation with mandatory review step. Week 4 - introduce one low-stakes automated action with a confirmation dialog that shows what the AI is about to do. Month 3 - offer optional automation for users who've built sufficient trust signal (measured as: high accept rate, low edit rate, zero reported errors). Never force automation on users who haven't earned confidence in the AI's reliability for their specific use cases. :::

---

Principle 3: Trust Calibration

Trust calibration is the practice of helping users develop an accurate mental model of when to trust your AI and when to be skeptical. A well-calibrated user knows intuitively that the AI is reliable for task X but shaky for task Y. Miscalibration in either direction is dangerous.

Over-trust: Users accept AI output uncritically, including errors. This leads to the "confident mistake" failure pattern - the AI is wrong, the user doesn't notice, downstream consequences occur. Over-trust is the most common cause of high-severity AI product incidents. It is particularly insidious because it often doesn't show up in product metrics until the damage is done: the user accepted the output, so the product recorded a "successful interaction."

Under-trust: Users ignore AI output or spend so much time verifying it that there is no productivity gain. The feature has a 2% usage rate and gets cut in the next planning cycle. Under-trust is a feature adoption failure - often caused by a few early bad experiences that weren't properly handled. Once a user has been burned, they approach every subsequent interaction with skepticism that the product has to actively work to overcome.

To calibrate trust correctly:

• Be transparent about limitations. Not in a legal-disclaimer way - in a genuinely helpful way. "This works best for short emails. For complex legal documents, review carefully" is trust calibration. It should appear in the product, not in a terms-of-service document.
• Surface uncertainty signals. When the model is less confident, the UI should reflect that. A "review before sending" badge is not a UX failure - it is a trust-building mechanism.
• Make the AI's reasoning visible. When users can see why the AI made a suggestion, they can judge it more accurately than when they see only the output. Chain-of-thought transparency is a product feature, not just a debugging tool.
• Celebrate corrections. When users edit AI output, that should feel natural and expected - not like the product failed. Normalize the human-AI collaborative edit cycle. A product where editing is frictionless produces better-calibrated users than one where editing feels like an acknowledgment of failure.

---

Principle 4: Avoiding Over-Automation

The automation trap is one of the most common AI product design mistakes: adding automation because you can, not because the user's workflow actually benefits from it. Over-automation manifests in predictable ways:

• Removing user control from steps that benefit from human judgment
• Triggering AI actions based on implicit signals (the user scrolled past something, the user paused for three seconds) rather than explicit intent
• Chaining multiple automated steps so that a single early error cascades through the entire workflow - the AI misclassifies an email as low-priority, which triggers auto-archiving, which causes the user to miss a time-sensitive message
• Making it harder to intervene than to let the automation run - the "stop" button is buried three levels deep
• Defaulting users into automated behavior without explaining it - users discover the automation when something goes wrong, not when they configured it

The principle of "appropriate automation" asks two questions at each step in the user's workflow: (1) What is the actual cost of a mistake here? (2) Does the user have the context and capability to catch that mistake before it causes damage? If the answer to the second question is no, do not automate that step. If the error cost is high and the user's ability to catch errors is low, that step requires human-in-the-loop design - no matter how good the model is.

:::danger The Automation Ratchet Effect Once users get used to automation, removing it feels like a downgrade even if the automation was causing subtle errors. This creates a dangerous lock-in: you discover the automation is unreliable, but rolling it back produces user complaints. Design automation levels you can sustainably maintain - don't over-automate in v1 and then walk it back in v2. Every automation you ship is a commitment you are making to the reliability of that automation indefinitely. This is why starting with suggestions and graduating to actions is safer than starting with actions. :::

---

Principle 5: Designing for Failure Modes

Every AI product has predictable failure modes. The engineering discipline of AI product design requires identifying those failure modes before launch and designing explicit experiences for each one. This is not pessimism - it is professional diligence. A medical device manufacturer does not ship without a designed failure mode for battery loss. An AI product should not ship without a designed failure mode for hallucination.

Taxonomy of LLM failure modes and their designed responses:

Failure Mode	Description	Probability	Design Response
Confident hallucination	Model produces plausible but factually wrong output	Medium	Citation requirements, highlighted verifiable facts, human review gates
Context misunderstanding	Model interprets the user's request incorrectly	Medium	Show what the model "heard", offer correction path, rephrasing suggestions
Capability boundary	Request is outside what the model handles well	Low-Medium	Graceful "I can't help with that well" with alternative path or manual workflow
Safety trigger	Content policy causes refusal	Low	User-facing message that explains without shaming, alternative rephrasing
Output format failure	Model produces wrong structure (JSON, table, code)	Low	Retry with stricter format prompt, fallback to manual structure
Stale knowledge	Model's training data is outdated	Medium-High	Surface knowledge cutoff, prompt to add current context
Cascading errors	Early mistake corrupts later steps in a chain	Low-Medium	Break chains with human checkpoints between high-stakes steps
Instruction drift	Model loses track of constraints over long conversations	Medium	Remind model of key constraints in every system prompt refresh
Persona inconsistency	Model breaks character in multi-turn	Low	Hard system prompt constraints, conversation reset option

The failure mode design loop should run before every significant product release. Run a structured "red team" session where team members try to break the AI with edge cases. Document every failure you find. Design an explicit UX for each one. The ones you don't find in testing will find themselves in production - with a user on the receiving end.

---

Principle 6: AI Product vs. AI Feature

An AI feature adds AI to a product. An AI product is built around AI as its core value proposition. The distinction matters because they require fundamentally different design philosophies, different trust architectures, and different failure mode priorities.

AI feature design: The AI enhances an existing workflow. Users opt into it. It can fail without breaking the product. Design for integration with the existing mental model. Key question: does the rest of the product still work if the AI is removed?

AI product design: The AI is the product. Users come specifically for the AI capability. The product creates a new workflow, not an enhancement of an old one. Key question: is there a product here if the AI is removed? If the answer is no, you are building an AI product and must design accordingly - building the trust model, the failure modes, and the user mental model from scratch.

Most companies building "AI features" are actually building AI products and don't realize it. If your product's primary value proposition would disappear without the AI, you are building an AI product. You need to design accordingly, which means you cannot borrow the trust that users have built with the non-AI version of your product. That trust does not transfer to the AI capabilities.

Dimension	AI Feature	AI Product
Failure impact	Degraded experience, non-AI path still works	Core value destroyed, no fallback path
Trust foundation	Inherit existing product trust	Build entirely from scratch
User mental model	Extend the existing one incrementally	Requires new one from the ground up
Fallback on failure	Non-AI workflow still fully functional	Product is non-functional
Design priority	Seamless integration, minimal disruption	AI reliability, transparency, calibration
Onboarding cost	Low - users already know the product	High - new mental model required
Error tolerance	High - errors feel like degraded extras	Low - errors feel like the product broke

---

Principle 7: The Last-Mile Problem

The last mile of an AI product is the gap between what the model produces and what the user actually needs to accomplish their task. This gap is almost always larger than engineers expect - and it is almost always a product engineering problem, not a model problem.

A model might produce a technically correct SQL query. The user's actual need is to run that query against their specific database, understand the results, and make a business decision. The "last mile" is everything between the SQL string and the business decision - and if your product doesn't bridge that gap, the model's correctness is largely irrelevant to whether the user achieves their goal.

Last-mile problems include:

• Output format mismatch: The model produces prose, the user needs structured data (or vice versa). The model writes a paragraph summary; the user needs JSON for their downstream system.
• Context the model doesn't have: The user's specific situation, constraints, or preferences that weren't in the prompt. The model gives a correct general answer that doesn't account for the user's organization's specific policies.
• Action integration gap: The model recommends an action, but the product doesn't connect to the system where that action would be taken. The AI says "file this under Project X" but there's no integration with the project management system.
• Verification cost: The user has to do significant work to verify whether the output is correct before acting on it. If verification is hard, users either skip it (and get burned) or stop using the feature.
• Translation to user's domain: The model gives a correct general answer, but the user needs it adapted to their specific context - their company, their workflow, their constraints.

Solving the last-mile problem means doing the unglamorous work of connecting model output to user action: parsing, formatting, integrating with other systems, and making verification cheap and fast. Teams that focus exclusively on model quality often underinvest in this work, producing products where the AI is impressive in isolation but doesn't integrate cleanly into the user's actual workflow.

---

Implementation: Confidence-Aware Response Generation

Here is a production Python pattern for generating confidence-aware responses using the Anthropic API. The key insight is that you can prompt the model to self-assess and embed that signal in the output structure, then use that signal to drive the UI presentation.

import anthropic
import json
from dataclasses import dataclass
from enum import Enum
from typing import Optional


class ConfidenceLevel(Enum):
    HIGH = "high"
    MEDIUM = "medium"
    LOW = "low"


@dataclass
class AIResponse:
    content: str
    confidence: ConfidenceLevel
    caveat: Optional[str]
    suggested_review: bool
    reasoning: Optional[str] = None      # Chain-of-thought for transparency
    highlighted_facts: list[str] = None  # Facts user should verify


def get_confidence_aware_response(
    user_query: str,
    context: str = "",
    domain: str = "general",
    high_stakes: bool = False,
    client: Optional[anthropic.Anthropic] = None,
) -> AIResponse:
    """
    Wraps a Claude call to extract structured confidence signals.
    The model self-reports confidence and any caveats.
    Used to drive the UI's confidence indicator component.

    For high-stakes contexts (legal, financial, medical, client-facing),
    always set high_stakes=True to enforce mandatory review signaling.
    """
    if client is None:
        client = anthropic.Anthropic()

    stakes_note = (
        "\nThis is a HIGH-STAKES context (client-facing, financial, or legal)."
        " Set suggested_review to true regardless of confidence level."
        if high_stakes
        else ""
    )

    system_prompt = f"""You are a helpful assistant specializing in {domain}.
When responding, always output valid JSON with this exact structure:
{{
  "content": "your main response here",
  "confidence": "high" | "medium" | "low",
  "caveat": "any important limitation or uncertainty, or null if none",
  "suggested_review": true | false,
  "reasoning": "brief explanation of confidence level",
  "highlighted_facts": ["fact1 that user should verify", "fact2"]
}}

Confidence levels:
- high: you are very certain this is correct and complete
- medium: you believe this is correct but recommend user verification
- low: this is your best attempt but the user should definitely verify before using

Set suggested_review to true if the output will be used in a high-stakes context
(emails to clients, financial decisions, legal documents, medical advice).

The reasoning field helps users understand why you're uncertain - be specific.
If you're uncertain about a date, say "My training data may not include the most recent information."
If you're uncertain about a company policy, say "Policies change frequently - verify with the source."

highlighted_facts: list the specific claims, numbers, or names the user should double-check.
Return an empty array [] if there are no facts to highlight.
{stakes_note}
"""

    user_message = f"{context}\n\nQuery: {user_query}" if context else user_query

    message = client.messages.create(
        model="claude-opus-4-6",
        max_tokens=1024,
        system=system_prompt,
        messages=[{"role": "user", "content": user_message}],
    )

    try:
        raw = message.content[0].text
        # Handle markdown code fences if model wraps JSON in them
        if raw.startswith("```"):
            raw = raw.split("```")[1]
            if raw.startswith("json"):
                raw = raw[4:]
        parsed = json.loads(raw.strip())
        return AIResponse(
            content=parsed["content"],
            confidence=ConfidenceLevel(parsed["confidence"]),
            caveat=parsed.get("caveat"),
            suggested_review=parsed.get("suggested_review", high_stakes),
            reasoning=parsed.get("reasoning"),
            highlighted_facts=parsed.get("highlighted_facts", []),
        )
    except (json.JSONDecodeError, KeyError, ValueError):
        # Fallback: treat unparseable output as low confidence
        return AIResponse(
            content=message.content[0].text,
            confidence=ConfidenceLevel.LOW,
            caveat="Response format was unexpected - please review carefully.",
            suggested_review=True,
            reasoning="Format parsing failed; treating as low confidence for safety.",
            highlighted_facts=[],
        )


def draft_client_email(
    context: str,
    recipient: str,
    subject: str,
) -> AIResponse:
    """
    Draft a client email with confidence-aware output.
    Always sets high_stakes=True - client emails are high stakes by default.
    """
    client = anthropic.Anthropic()
    query = f"Draft a professional email to {recipient} about: {subject}"
    return get_confidence_aware_response(
        user_query=query,
        context=context,
        domain="professional communication",
        high_stakes=True,
        client=client,
    )


# Cheap classification with Haiku - use for routing/intent detection, not content
def classify_request_intent(
    user_query: str,
    client: Optional[anthropic.Anthropic] = None,
) -> dict:
    """
    Use claude-haiku-4-5-20251001 for cheap intent classification.
    Reserve claude-opus-4-6 for the actual content generation.
    Cost ratio: Haiku is ~15x cheaper per token than Opus.
    Use Haiku for: routing, classification, moderation, short answers.
    Use Opus for: drafting, analysis, complex reasoning.
    """
    if client is None:
        client = anthropic.Anthropic()

    message = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=128,
        system="""Classify the user's intent. Return JSON only:
{
  "intent": "email_draft" | "summary" | "question_answer" | "analysis" | "code" | "other",
  "high_stakes": true | false,
  "domain": "legal" | "financial" | "medical" | "technical" | "general"
}""",
        messages=[{"role": "user", "content": user_query}],
    )

    try:
        raw = message.content[0].text
        if raw.startswith("```"):
            raw = raw.split("```")[1]
            if raw.startswith("json"):
                raw = raw[4:]
        return json.loads(raw.strip())
    except (json.JSONDecodeError, ValueError):
        return {"intent": "other", "high_stakes": False, "domain": "general"}


if __name__ == "__main__":
    # Example: Classify first (cheap), then generate (expensive only if needed)
    query = "What is the penalty for late filing of Q4 corporate taxes in the US?"
    intent = classify_request_intent(query)
    print(f"Intent: {intent}")

    response = get_confidence_aware_response(
        user_query=query,
        domain="tax compliance",
        high_stakes=intent.get("high_stakes", False),
    )
    print(f"\nContent: {response.content}")
    print(f"Confidence: {response.confidence.value}")
    print(f"Caveat: {response.caveat}")
    print(f"Reasoning: {response.reasoning}")
    print(f"Review recommended: {response.suggested_review}")
    print(f"Facts to verify: {response.highlighted_facts}")

---

Frontend: Confidence-Aware UI Component

The backend confidence signal is only useful if the frontend surfaces it clearly without being alarmist. The design principle is: make uncertainty visible without making users anxious. A yellow border and a note is informative. A red warning popup is alarming. The difference between these two implementations is the difference between a trusted tool and an anxiety-inducing one.

// components/ConfidenceAwareOutput.tsx
import React, { useState } from 'react';

type ConfidenceLevel = 'high' | 'medium' | 'low';

interface AIOutputProps {
  content: string;
  confidence: ConfidenceLevel;
  caveat?: string | null;
  reasoning?: string | null;
  suggestedReview?: boolean;
  highlightedFacts?: string[];
  onEdit?: () => void;
  onAccept?: () => void;
  onRegenerate?: () => void;
}

const confidenceConfig = {
  high: {
    badge: 'High confidence',
    badgeClass: 'bg-green-50 text-green-700 border-green-200',
    containerClass: 'border-l-4 border-green-400',
    showCaveat: false,
    acceptLabel: 'Use this',
    icon: '✓',
  },
  medium: {
    badge: 'Review recommended',
    badgeClass: 'bg-yellow-50 text-yellow-700 border-yellow-200',
    containerClass: 'border-l-4 border-yellow-400',
    showCaveat: true,
    acceptLabel: 'Accept with review',
    icon: '!',
  },
  low: {
    badge: 'Verify before using',
    badgeClass: 'bg-red-50 text-red-700 border-red-200',
    containerClass: 'border-l-4 border-red-400',
    showCaveat: true,
    acceptLabel: 'Accept anyway',
    icon: '⚠',
  },
};

export function ConfidenceAwareOutput({
  content,
  confidence,
  caveat,
  reasoning,
  suggestedReview,
  highlightedFacts = [],
  onEdit,
  onAccept,
  onRegenerate,
}: AIOutputProps) {
  const config = confidenceConfig[confidence];
  const [showReasoning, setShowReasoning] = useState(false);
  const [showFacts, setShowFacts] = useState(highlightedFacts.length > 0);

  return (
    <div className={`rounded-lg bg-white p-4 shadow-sm ${config.containerClass}`}>
      {/* Header row: confidence badge + review notice */}
      <div className="flex items-center justify-between mb-3">
        <span
          className={`text-xs font-semibold px-2 py-1 rounded border ${config.badgeClass}`}
        >
          {config.icon} {config.badge}
        </span>
        {suggestedReview && confidence !== 'high' && (
          <span className="text-xs text-gray-400 italic">
            Review before using in client communications
          </span>
        )}
      </div>

      {/* Main content */}
      <div className="text-sm text-gray-800 leading-relaxed mb-3 whitespace-pre-wrap">
        {content}
      </div>

      {/* Highlighted facts - items user should verify */}
      {showFacts && highlightedFacts.length > 0 && (
        <div className="bg-yellow-50 border border-yellow-200 rounded p-3 mb-3">
          <div className="text-xs font-semibold text-yellow-800 mb-1">
            Please verify these before using:
          </div>
          <ul className="text-xs text-yellow-700 list-disc ml-4 space-y-0.5">
            {highlightedFacts.map((fact, i) => (
              <li key={i}>{fact}</li>
            ))}
          </ul>
          <button
            className="text-xs text-yellow-600 mt-1 hover:text-yellow-800"
            onClick={() => setShowFacts(false)}
          >
            Dismiss
          </button>
        </div>
      )}

      {/* Caveat - shown for medium/low confidence */}
      {config.showCaveat && caveat && (
        <div className="text-xs text-gray-500 bg-gray-50 rounded p-2 mb-3 border border-gray-100">
          <span className="font-medium">Note: </span>{caveat}
        </div>
      )}

      {/* Reasoning transparency toggle */}
      {reasoning && (
        <div className="mb-3">
          <button
            className="text-xs text-blue-500 hover:text-blue-700"
            onClick={() => setShowReasoning(!showReasoning)}
          >
            {showReasoning ? 'Hide reasoning' : 'Why this confidence level?'}
          </button>
          {showReasoning && (
            <p className="text-xs text-gray-500 mt-1 italic">{reasoning}</p>
          )}
        </div>
      )}

      {/* Action buttons - color-coded by confidence level */}
      <div className="flex gap-2 justify-end pt-2 border-t border-gray-100">
        {onRegenerate && (
          <button
            onClick={onRegenerate}
            className="text-xs px-3 py-1.5 rounded border border-gray-200 text-gray-500 hover:bg-gray-50"
          >
            Regenerate
          </button>
        )}
        {onEdit && (
          <button
            onClick={onEdit}
            className="text-xs px-3 py-1.5 rounded border border-gray-300 text-gray-600 hover:bg-gray-50"
          >
            Edit
          </button>
        )}
        {onAccept && (
          <button
            onClick={onAccept}
            className={`text-xs px-3 py-1.5 rounded text-white ${
              confidence === 'high'
                ? 'bg-green-600 hover:bg-green-700'
                : confidence === 'medium'
                ? 'bg-yellow-600 hover:bg-yellow-700'
                : 'bg-gray-600 hover:bg-gray-700'
            }`}
          >
            {config.acceptLabel}
          </button>
        )}
      </div>
    </div>
  );
}

---

Implementing a Failure Mode Registry

A failure mode registry is the operational artifact that ensures every known failure has a designed response. Build it before launch, update it after every incident, use it as a launch-readiness checklist.

# failure_modes/registry.py
from dataclasses import dataclass, field
from typing import Optional
from enum import Enum


class Severity(str, Enum):
    CRITICAL = "critical"    # Data loss, wrong financial/legal/medical output
    HIGH = "high"            # User task fails, significant disruption
    MEDIUM = "medium"        # Degraded experience, workaround exists
    LOW = "low"              # Minor annoyance, does not block task


class DesignStatus(str, Enum):
    DESIGNED = "designed"        # Explicit UX designed for this failure
    PARTIAL = "partial"          # Generic error handling only
    UNHANDLED = "unhandled"      # No handling - gap to close before launch


@dataclass
class FailureMode:
    id: str
    name: str
    description: str
    trigger_conditions: list[str]
    severity: Severity
    probability: str                       # "high", "medium", "low"
    design_status: DesignStatus
    ux_response: Optional[str]            # Description of the designed UX
    fallback: Optional[str]               # What shows when this failure occurs
    monitoring_signal: Optional[str]      # How to detect in production
    owner: Optional[str]
    test_case: Optional[str]


class FailureModeRegistry:
    """
    Central registry of all known AI failure modes for a product.
    Used in pre-launch checklists and post-incident reviews.
    Launch is blocked if any CRITICAL failures are UNHANDLED.
    """

    def __init__(self):
        self.modes: list[FailureMode] = []

    def register(self, mode: FailureMode) -> None:
        self.modes.append(mode)

    def get_critical_unhandled(self) -> list[FailureMode]:
        return [
            m for m in self.modes
            if m.severity == Severity.CRITICAL
            and m.design_status == DesignStatus.UNHANDLED
        ]

    def launch_readiness_check(self) -> dict:
        critical_unhandled = self.get_critical_unhandled()
        high_unhandled = [
            m for m in self.modes
            if m.severity == Severity.HIGH
            and m.design_status == DesignStatus.UNHANDLED
        ]
        return {
            "launch_ready": len(critical_unhandled) == 0,
            "critical_unhandled": len(critical_unhandled),
            "high_unhandled": len(high_unhandled),
            "total_modes": len(self.modes),
            "designed": len([m for m in self.modes if m.design_status == DesignStatus.DESIGNED]),
            "blocking_issues": [m.name for m in critical_unhandled],
        }

    def print_report(self) -> None:
        print("\n=== Failure Mode Registry Report ===\n")
        for mode in sorted(self.modes, key=lambda m: m.severity.value):
            status_icon = {
                DesignStatus.DESIGNED: "[OK]      ",
                DesignStatus.PARTIAL: "[PARTIAL] ",
                DesignStatus.UNHANDLED: "[BLOCKED] ",
            }[mode.design_status]
            print(f"{status_icon} [{mode.severity.value.upper():8}] {mode.name}")
            if mode.ux_response:
                print(f"             UX: {mode.ux_response}")
            if mode.design_status == DesignStatus.UNHANDLED:
                print(f"             *** LAUNCH BLOCKER - Owner: {mode.owner or 'unassigned'}")
        print()


def build_email_ai_registry() -> FailureModeRegistry:
    registry = FailureModeRegistry()

    registry.register(FailureMode(
        id="hallucination_client_details",
        name="Hallucinated client information",
        description="AI invents client names, contract dates, or pricing details not in context",
        trigger_conditions=["Long conversation", "Ambiguous context", "Numeric data in prompt"],
        severity=Severity.CRITICAL,
        probability="medium",
        design_status=DesignStatus.DESIGNED,
        ux_response="All generated names, dates, and numbers highlighted for mandatory review",
        fallback="'Review all highlighted fields before sending' notice always visible",
        monitoring_signal="User edit rate on numerical/name fields > 30%",
        owner="product-ai-team",
        test_case="Prompt with ambiguous date references, verify AI does not invent dates",
    ))

    registry.register(FailureMode(
        id="context_overflow",
        name="Context window saturation",
        description="Long conversation causes model to lose track of early constraints",
        trigger_conditions=["Conversation > 20 turns", "Large document in context"],
        severity=Severity.HIGH,
        probability="high",
        design_status=DesignStatus.DESIGNED,
        ux_response="Context usage bar shown; 'Start new conversation' prompt at 80% full",
        fallback="Auto-summary of conversation injected at 80% context",
        monitoring_signal="Context usage > 80% on more than 20% of requests",
        owner="product-ai-team",
        test_case="Send 25+ message conversation, check coherence against early constraints",
    ))

    registry.register(FailureMode(
        id="content_policy_refusal",
        name="Unexpected content policy refusal",
        description="Model refuses a legitimate business request",
        trigger_conditions=["Sensitive industry terms", "Legal/medical/financial language"],
        severity=Severity.MEDIUM,
        probability="low",
        design_status=DesignStatus.PARTIAL,
        ux_response="'I can't help with that' message with rephrasing suggestion",
        fallback="Offer to rephrase or escalate to human agent",
        monitoring_signal="Refusal rate > 5% for domain queries",
        owner="product-ai-team",
        test_case="Send legal template language, verify refusal rate and message quality",
    ))

    registry.register(FailureMode(
        id="stale_knowledge",
        name="Outdated information presented as current",
        description="Model references outdated policies, prices, or facts as if current",
        trigger_conditions=["Questions about current state", "Policy/price queries"],
        severity=Severity.HIGH,
        probability="medium",
        design_status=DesignStatus.UNHANDLED,
        ux_response=None,
        fallback=None,
        monitoring_signal="User feedback indicating outdated information",
        owner="product-ai-team",
        test_case="Ask about a policy known to have changed since training cutoff",
    ))

    return registry


if __name__ == "__main__":
    registry = build_email_ai_registry()
    readiness = registry.launch_readiness_check()
    print(f"Launch ready: {readiness['launch_ready']}")
    print(f"Blocking issues: {readiness['blocking_issues']}")
    registry.print_report()

---

Architecture: Human-in-the-Loop Review Gates

The most underused pattern in AI product engineering is the explicit human-in-the-loop review gate: a designed point in the workflow where a human reviews AI output before it produces a downstream effect. Review gates are not UX failures - they are the safety layer that makes automation sustainable.

# patterns/review_gate.py
import anthropic
import json
from dataclasses import dataclass
from typing import Callable, Any, Optional
from enum import Enum


class GateDecision(str, Enum):
    APPROVE = "approve"
    EDIT = "edit"
    REJECT = "reject"


@dataclass
class ReviewGateResult:
    decision: GateDecision
    original_content: str
    final_content: str             # May differ from original if user edited
    edit_distance: int             # 0 if approved as-is
    time_to_review_ms: Optional[int]


class ReviewGate:
    """
    A review gate wraps any AI-generated content and routes it through
    a human review step before downstream action is taken.

    In production:
    - Present content to user with clear "review required" framing
    - Provide inline editing capability
    - Log decision + edit delta for model quality tracking
    - Gate is non-blockable for CRITICAL content types

    This pattern is used when:
    - Content will be sent to external parties (emails, reports)
    - Content drives financial decisions
    - Content modifies persistent state (database writes, API calls)
    - Failure cost > cost of review time
    """

    def __init__(
        self,
        content_type: str,
        required: bool = False,       # If True, cannot be skipped
        log_fn: Optional[Callable] = None,
    ):
        self.content_type = content_type
        self.required = required
        self.log_fn = log_fn or print

    def should_gate(self, confidence: str, high_stakes: bool) -> bool:
        """Determine if a gate should be shown based on risk factors."""
        if self.required:
            return True
        if high_stakes:
            return True
        if confidence in ("medium", "low"):
            return True
        return False

    def gate(
        self,
        ai_output: str,
        confidence: str,
        high_stakes: bool,
        review_callback: Callable[[str], tuple[GateDecision, str]],
    ) -> ReviewGateResult:
        """
        Execute the review gate.

        review_callback receives the AI output and returns:
          - (GateDecision.APPROVE, original_content) - accepted as-is
          - (GateDecision.EDIT, edited_content) - accepted with edits
          - (GateDecision.REJECT, "") - rejected, no action taken
        """
        import time

        if not self.should_gate(confidence, high_stakes):
            # Auto-approve: high confidence, low stakes, gate not required
            self.log_fn(f"[ReviewGate] Auto-approved {self.content_type} "
                       f"(confidence={confidence}, stakes=low)")
            return ReviewGateResult(
                decision=GateDecision.APPROVE,
                original_content=ai_output,
                final_content=ai_output,
                edit_distance=0,
                time_to_review_ms=0,
            )

        start = time.monotonic()
        decision, final_content = review_callback(ai_output)
        elapsed_ms = int((time.monotonic() - start) * 1000)

        # Compute rough edit distance
        edit_distance = sum(
            1 for a, b in zip(ai_output, final_content) if a != b
        ) + abs(len(ai_output) - len(final_content))

        self.log_fn(
            f"[ReviewGate] {self.content_type} | decision={decision.value} "
            f"| edits={edit_distance} | time={elapsed_ms}ms"
        )

        return ReviewGateResult(
            decision=decision,
            original_content=ai_output,
            final_content=final_content,
            edit_distance=edit_distance,
            time_to_review_ms=elapsed_ms,
        )

---

Product Engineering Recommendations

Ship with an explicit "AI made this" signal. Never let AI-generated content pass as human-authored without disclosure. Users who discover undisclosed AI generation feel deceived - even if the content is excellent. A small, tasteful "AI-generated" label is both ethically required and practically valuable for trust calibration.

Log every AI output and every user edit. The delta between what the AI produced and what the user kept is your most valuable training signal and your most honest product quality metric. High edit rates on specific output types signal precisely where the model is underperforming for your users.

Build "I don't know" as a first-class response. An AI that admits uncertainty is more trustworthy than one that is always confident. Design the UI for graceful "I can't help with this well" responses - with suggested alternatives, links to documentation, and a path to human support.

Instrument your fallback paths. If you have a fallback for when the AI fails, track how often it triggers. A fallback that activates 40% of the time is a signal that your primary path has a serious reliability problem. Fallback trigger rate is one of the most underused product health metrics.

Separate AI infrastructure from product releases. Updating the model should not require a product release. Changing the product should not require touching the model infrastructure. These are different concerns and benefit from independent deployment cycles.

Test adversarially before every launch. Before every AI feature ships, spend at least one day trying to break it. Find the inputs that produce bad outputs. Design the UX for those cases. The obvious failure modes are the ones that generate the first wave of support tickets - cover those before launch.

---

Common Mistakes

:::danger Designing Only the Happy Path The most expensive mistake in AI product design is building a beautiful UI for when the model works and no UI for when it fails. Users will encounter failures. Design for them explicitly. Every failure mode that doesn't have a designed experience will produce a support ticket, and that ticket will be more expensive than the 30 minutes it would have taken to design the error state before launch. :::

:::danger Displaying Raw Model Output Without Review Especially dangerous in B2B contexts. A user who pastes AI output directly into a client email and gets it wrong will blame the product, not themselves. Always provide a review step for high-stakes outputs, and make it trivially easy to edit before using. The "copy to clipboard" pattern is safer than "send directly" for any output that affects external parties. :::

:::warning Confusing Engagement with Trust High engagement numbers in week one of an AI feature launch are misleading. Users are often exploring out of novelty. Real trust signals are week-4 and week-8 retention, and whether users are incorporating the AI into their actual daily workflow. Don't optimize for novelty clicks - optimize for habitual, confident use. :::

:::warning Launching at the Automation Ceiling Start with suggestion, not action. Give users autonomy over AI output before giving the AI autonomy over user actions. Trust is built incrementally. You can always add more automation later - removing automation that users have come to rely on feels like a downgrade even if the automation was unreliable. :::

:::info The Trust Debt Pattern Many AI products accumulate "trust debt" by over-promising in demos and under-delivering in production. Unlike technical debt, trust debt compounds: each failure makes the next failure hit harder. Pay trust debt down before it compounds - be honest about limitations in your onboarding, your documentation, and your UI copy. :::

---

Interview Q&A

Q1: What is the difference between an AI feature and an AI product, and why does it matter for engineering decisions?

An AI feature adds AI capability to an existing product where the product has value independent of the AI. An AI product's primary value proposition depends on the AI - remove it and the product doesn't work. This distinction affects almost every engineering decision. For an AI feature, you design for graceful degradation and backward compatibility - the existing UX remains fully functional if the AI fails. For an AI product, you must design an entirely new user mental model, new trust architecture, and new failure-mode UX from the ground up. Treating an AI product like an AI feature leads to a design that assumes users already know what to expect from AI, which they don't. The most common manifestation: teams build AI products but ship them with the minimal design effort appropriate for AI features, creating a trust mismatch that causes adoption failures at week 3-4 when the novelty wears off.

Q2: How do you calibrate user trust in an AI system? What goes wrong when trust is miscalibrated?

Trust calibration requires three things: transparency, consistency, and feedback loops. Transparency means telling users what the AI can and can't do - not in small print, but in the primary UX. "This works best for short emails; for complex legal documents, please review carefully" is trust calibration. Consistency means the AI behaves predictably enough that users can develop an accurate mental model of when to trust it. This is one of the biggest challenges with LLMs, which can vary significantly run-to-run. Feedback loops mean users can see the consequences of trusting or not trusting the AI, and update their behavior accordingly.

Miscalibration in either direction is dangerous. Over-trust: users accept AI output uncritically, including errors, leading to the "confident mistake" failure pattern - the AI is wrong, the user doesn't notice, downstream consequences occur. Under-trust: users ignore AI output or verify it so thoroughly that there is no productivity gain, and the feature has a 2% usage rate and gets cut. The worst pattern is an AI that is reliable 95% of the time but catastrophically wrong 5% of the time with no warning signals - users over-trust because of the 95% and are blindsided by the 5%.

Q3: What is progressive disclosure of AI capability, and when is it essential?

Progressive disclosure is the practice of introducing AI capability incrementally, starting with low-stakes, high-confidence use cases and expanding to higher-stakes, autonomous actions only after users have built sufficient trust. It matters most when users don't have prior experience with AI - which is most users, most of the time. The practical implementation is a layered rollout: week one is suggestions and autocomplete, week four is drafts and summaries, month three is automated actions. Each layer gives users time to learn the AI's capabilities and limitations before the stakes increase. It also matters for product risk management: if you discover a serious failure mode at the autocomplete layer, the blast radius is minimal compared to discovering it at the automated-action layer. Many AI products fail because they launch at layer 3 - autonomous actions - before users have built the mental model required to safely use them.

Q4: How do you design for a failure mode you haven't seen yet?

Start with a taxonomy of failure modes for the model family you're using - LLMs have well-documented failure patterns: hallucination, context misunderstanding, capability boundaries, format failures. Design a generic "something went wrong" UX that is useful and honest without being alarming. Then instrument everything: log all AI requests and responses, capture user feedback signals (edits, rejections, explicit thumbs-down), and build tooling to triage new failure patterns as they emerge. The key insight is that you don't need to anticipate every specific failure - you need to build the observability infrastructure to identify and respond to new failures quickly. A failure you can detect and fix in 24 hours is not a crisis. A failure you discover six weeks later through customer churn data is. Invest in observability before you invest in edge case handling.

Q5: What does "the last-mile problem" mean in AI products, and how do you solve it?

The last mile is the gap between what an AI model produces and what the user actually needs to complete their task. A model might generate a correct draft, but the last mile includes: formatting it for the user's specific context, connecting it to the system where they need to use it, making it easy to verify for correctness, and routing it to the right action. Teams that focus exclusively on model quality often underinvest in the last mile, which leads to products where the AI is impressive in isolation but doesn't integrate cleanly into the user's actual workflow.

Solving it requires doing the unglamorous work: building integrations with the systems users actually work in, adding formatting logic that transforms model output into the exact structure users need, providing verification affordances that make checking AI output fast rather than tedious, and connecting AI recommendations to actionable buttons and workflows. The last-mile problem is almost always a product engineering problem, not a model problem - and it is typically the hardest and most undervalued work in shipping an AI product that users actually rely on daily.

Q6: How do you decide what to automate in an AI product and what to keep human-reviewed?

The decision framework has two axes: the cost of an error and the user's ability to detect and correct that error before it causes damage. High error cost plus low detectability = manual, always. High error cost plus high detectability = automate with a mandatory review gate. Low error cost = automate more aggressively, since mistakes are cheap to recover from. The trap is that engineers consistently overestimate how easily users detect AI errors. Users are generally bad at catching AI mistakes, especially when the output looks plausible - and LLM output almost always looks plausible. Design automation levels conservatively, and expand only as you accumulate evidence through analytics, user research, and support ticket analysis that users are successfully catching and correcting errors. Every automation decision is reversible in theory but almost impossible to walk back in practice without a user trust penalty.

Q7: Why do so many AI features fail to achieve long-term adoption, even when the underlying model is strong?

The core reason is a trust-adoption mismatch: the product was designed for users in the novelty phase (first week, exploring enthusiastically) but not for users in the habitual-use phase (month two, relying on the AI for real work). In the novelty phase, users tolerate failures because they're experimenting. In the habitual-use phase, a single embarrassing failure - an AI output that got them in trouble with a client or colleague - destroys the trust they've built. Products that don't design explicitly for the post-novelty user experience see a consistent pattern: spike in adoption at launch, sharp drop at week 3-4 when real-use failures surface, plateau at a low baseline of enthusiasts who accept the failure rate. The fix is designing failure modes before launch, not after the first drop - and it requires treating "what happens when the AI is wrong" as at least as important as "what happens when the AI is right."

---

Observability: Measuring AI Product Quality

Model accuracy benchmarks tell you how well the model performs on a standardized test. They tell you almost nothing about how well your AI product performs in production for your specific users on their specific tasks. Building a separate observability layer for your AI product is not optional - it is the mechanism by which you convert production data into product improvements.

The Core Product Quality Metrics

Edit rate by output type. Track the percentage of AI outputs that users modify before using. Segment by output type (email drafts, summaries, code, structured data). A high edit rate on email drafts signals the model isn't capturing the user's voice. A high edit rate on structured data signals a format mismatch. Edit rate is a more honest quality signal than user satisfaction surveys because it measures revealed behavior, not stated preference.

Acceptance rate. Track the percentage of AI suggestions that users accept without modification. This is the inverse of edit rate, but segmented differently: you want to know which output types are accepted as-is and which are always modified. High acceptance on low-stakes outputs is expected; high acceptance on high-stakes outputs deserves scrutiny - users may be over-trusting.

Fallback trigger rate. Every time your AI fails and the fallback path activates, log it. If the fallback triggers on 25% of requests, your primary path has a serious reliability problem that no amount of product polish will fix.

Time-to-first-use after onboarding. How quickly do new users complete their first meaningful AI-assisted task? A long time suggests onboarding friction. A short time suggests either great design or users not understanding the full capability (they use the simplest feature and miss the valuable ones).

Week-4 retention by onboarding cohort. Novelty drives week-1 numbers. Week-4 retention measures real adoption. Segment by how users were onboarded - users who went through progressive disclosure typically show 30-50% higher week-4 retention than users who were given full capability on day one.

# observability/ai_product_metrics.py
import anthropic
import json
import time
from dataclasses import dataclass, field
from typing import Optional
from datetime import datetime


@dataclass
class InteractionEvent:
    """
    Structured event logged for every AI interaction.
    Used to compute product quality metrics in aggregate.
    """
    interaction_id: str
    user_id: str
    output_type: str              # "email_draft", "summary", "code", "qa"
    model: str
    request_timestamp: datetime
    ttft_ms: Optional[int]
    total_ms: Optional[int]
    input_tokens: int
    output_tokens: int
    confidence_level: str         # "high", "medium", "low"
    suggested_review: bool
    # User action fields - filled in after user interacts with output
    user_action: Optional[str] = None    # "accept", "edit", "reject", "copy"
    edit_distance: Optional[int] = None  # 0 if accepted, N chars changed if edited
    time_to_action_ms: Optional[int] = None
    fallback_triggered: bool = False


class AIProductMetrics:
    """
    Aggregates interaction events into product quality metrics.
    In production, push events to a data warehouse (BigQuery, Snowflake, Redshift)
    and compute these metrics with scheduled SQL queries or dbt models.

    This in-memory version is for development and testing.
    """

    def __init__(self):
        self.events: list[InteractionEvent] = []

    def record(self, event: InteractionEvent) -> None:
        self.events.append(event)

    def edit_rate_by_type(self) -> dict[str, float]:
        """
        Percentage of outputs that users modified before using.
        High edit rate = model is underperforming for that output type.
        """
        by_type: dict[str, list[bool]] = {}
        for e in self.events:
            if e.user_action in ("accept", "edit") and e.output_type:
                by_type.setdefault(e.output_type, [])
                by_type[e.output_type].append(e.user_action == "edit")
        return {
            t: sum(edits) / len(edits) * 100
            for t, edits in by_type.items()
            if edits
        }

    def acceptance_rate(self) -> float:
        """Percentage of outputs accepted without modification."""
        actionable = [e for e in self.events if e.user_action in ("accept", "edit", "reject")]
        if not actionable:
            return 0.0
        accepted = sum(1 for e in actionable if e.user_action == "accept")
        return accepted / len(actionable) * 100

    def fallback_rate(self) -> float:
        """Percentage of requests that triggered the fallback path."""
        if not self.events:
            return 0.0
        fallbacks = sum(1 for e in self.events if e.fallback_triggered)
        return fallbacks / len(self.events) * 100

    def avg_ttft_ms_by_model(self) -> dict[str, float]:
        """Average TTFT per model - for latency benchmarking."""
        by_model: dict[str, list[int]] = {}
        for e in self.events:
            if e.ttft_ms is not None:
                by_model.setdefault(e.model, []).append(e.ttft_ms)
        return {
            model: sum(times) / len(times)
            for model, times in by_model.items()
        }

    def confidence_calibration_report(self) -> dict:
        """
        Compare claimed confidence level to actual user behavior.
        Ideal calibration: high confidence → low edit rate, low → high edit rate.
        If high-confidence outputs have a high edit rate, the model is overconfident.
        """
        for_level: dict[str, dict] = {"high": {}, "medium": {}, "low": {}}
        for level in for_level:
            level_events = [
                e for e in self.events
                if e.confidence_level == level
                and e.user_action in ("accept", "edit")
            ]
            if not level_events:
                continue
            for_level[level] = {
                "count": len(level_events),
                "edit_rate": sum(1 for e in level_events if e.user_action == "edit") / len(level_events) * 100,
                "accept_rate": sum(1 for e in level_events if e.user_action == "accept") / len(level_events) * 100,
            }
        return for_level

    def print_dashboard(self) -> None:
        print("\n=== AI Product Quality Dashboard ===\n")
        print(f"Total interactions recorded: {len(self.events)}")
        print(f"Overall acceptance rate: {self.acceptance_rate():.1f}%")
        print(f"Fallback trigger rate: {self.fallback_rate():.1f}%")
        print("\nEdit rate by output type:")
        for t, rate in self.edit_rate_by_type().items():
            flag = " *** HIGH" if rate > 40 else ""
            print(f"  {t:20} {rate:.1f}%{flag}")
        print("\nAvg TTFT by model (ms):")
        for model, ttft in self.avg_ttft_ms_by_model().items():
            print(f"  {model:30} {ttft:.0f}ms")
        print("\nConfidence calibration:")
        for level, data in self.confidence_calibration_report().items():
            if data:
                print(f"  {level:8} accept={data['accept_rate']:.0f}%  edit={data['edit_rate']:.0f}%  n={data['count']}")

The Pre-Launch Quality Checklist

Before shipping any AI feature, run through this checklist. Every unchecked item is a potential support ticket or trust incident waiting to happen.

Trust architecture:

• Confidence signals are surfaced in the UI for all output types
• The UI language distinguishes "draft" from "answer" appropriately
• Users have an easy path to edit before committing to AI output
• High-stakes outputs have a mandatory review gate

Failure mode coverage:

• Every CRITICAL and HIGH severity failure mode has a designed UX
• The failure mode registry has zero CRITICAL + UNHANDLED combinations
• Adversarial red team session completed and documented
• Error messages are user-friendly and actionable (not raw stack traces)

Fallback paths:

• Every AI-gated flow has a non-AI fallback path
• Fallback path is tested and functional, not just theoretically designed
• Fallback trigger rate instrumentation is in place

Observability:

• Every AI interaction is logged with interaction ID, model, tokens, latency
• User action (accept/edit/reject) is captured per interaction
• Edit delta is logged for post-hoc quality analysis
• Fallback trigger events are logged separately
• Week-1 and week-4 retention reporting is set up

Infrastructure:

• AI infrastructure updates are decoupled from product releases
• Model version is logged per interaction for reproducibility
• Rate limit handling is designed and tested
• Graceful degradation is verified under API outage simulation

:::info The Edit Delta as Ground Truth The difference between what the AI produced and what the user submitted is the most honest, unbiased quality signal in your product. It requires no surveys, no user interviews, no manual annotation. It is collected automatically, at scale, in real usage conditions. Build your edit delta logging before launch and mine it weekly. High edit rate on a specific field type (dates, names, prices) tells you precisely what the model is getting wrong for your users - which is far more actionable than any benchmark score. :::

---

Summary: The Principles in Practice

AI product design is not about making AI work. It is about making AI work for humans - with all their variability, their tendency to over-trust, their low tolerance for embarrassing failures, and their need for a product that earns trust over time rather than demanding it upfront.

The seven principles in this lesson are not theoretical ideals. They are the hard-won lessons of teams who shipped AI products without them and paid the price in support queues, lost customers, and emergency redesigns. Each principle addresses a specific failure mode that emerges when these lessons are ignored:

• Skip graceful degradation, and users get stranded when the model fails.
• Skip progressive disclosure, and users over-trust before they're ready.
• Skip trust calibration, and users either over-rely or abandon.
• Over-automate, and one AI error cascades into a workflow disaster.
• Skip failure mode design, and the first real-world edge case becomes a crisis.
• Treat an AI product like an AI feature, and the trust model is wrong from day one.
• Ignore the last mile, and model accuracy becomes irrelevant to user outcomes.

The model is necessary but not sufficient. The product design is what delivers model capability as user value. That is what this discipline is for.