Fine-Tuning Ops

Fifty Customers, Fifty Models

The enterprise AI platform had a problem that started small and became unmanageable. In month one, they fine-tuned a custom LLM for their first enterprise customer - a law firm that needed a model with precise legal terminology and citation style. The fine-tuning took a week of ML engineering time. The customer was delighted.

By month six, they had 12 enterprise customers, each with a fine-tuned model. Each model was a separate set of full model weights. Serving them required 12 separate GPU instances. Each fine-tuning job was run manually, with a Jupyter notebook. Model quality was checked by a single engineer eyeballing outputs. There was no versioning - the "current model" for each customer was whichever checkpoint someone had last uploaded. Three times in those six months, a customer's production model was accidentally overwritten by a re-run fine-tuning job.

By month twelve, they had 50 customers. The infrastructure cost of 50 full-weight models was unsustainable. The manual fine-tuning process was consuming 30% of the ML team's time. Quality had degraded - nobody had bandwidth to properly evaluate 50 models after each update.

The solution was not better fine-tuning. It was industrialized fine-tuning operations: LoRA adapters instead of full weights, a data pipeline, an adapter registry, automated evaluation, and a serving infrastructure that could host 50 adapters on 2 GPUs instead of 50.

---

:::tip 🎮 Interactive Playground Visualize this concept: Try the Fine-Tuning Methods Compared demo on the EngineersOfAI Playground - no code required. :::

Why Fine-Tuning Operations Are Different

Traditional MLOps for a classification model: train one model on one dataset, evaluate on one holdout set, deploy one artifact. The pipeline is linear and the artifacts are manageable.

LLM fine-tuning ops:

• Many customers: each customer has proprietary data, unique quality requirements, and separate deployment lifecycle
• Large base models: the base model (7B–70B parameters) dwarfs the training data; most of the value is in the adapter, not the base model
• Expensive data curation: LLM training data requires careful cleaning, deduplication, format standardization, and safety filtering - more labor-intensive than tabular data
• Multi-stage pipelines: SFT → preference optimization → safety alignment → evaluation → deployment, each with its own failure modes
• Dynamic updates: customers continuously provide new data; continuous fine-tuning requires delta updates without catastrophic forgetting

---

The Fine-Tuning Data Pipeline

Before you write a single line of training code, you need clean data. LLM training data failures are silent and expensive - bad data produces a model that seems to work but generates subtly wrong outputs in production.

import json
import hashlib
from pathlib import Path
from dataclasses import dataclass
from typing import List, Optional, Iterator
import re

@dataclass
class TrainingExample:
    """One supervised fine-tuning example in chat format."""
    system: Optional[str]
    messages: List[dict]  # [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]
    source: str           # where did this example come from?
    quality_score: Optional[float] = None  # 0-1, higher is better

    def to_sharegpt_format(self) -> dict:
        """Convert to ShareGPT format for Axolotl/Unsloth training."""
        return {
            "conversations": [
                *([] if not self.system else [{"from": "system", "value": self.system}]),
                *[{"from": "human" if m["role"] == "user" else "gpt",
                   "value": m["content"]}
                  for m in self.messages]
            ]
        }

    def to_alpaca_format(self) -> dict:
        """Convert to Alpaca format for simpler instruction tuning."""
        if len(self.messages) >= 2:
            return {
                "instruction": self.messages[0]["content"],
                "input": "",
                "output": self.messages[1]["content"]
            }
        return {}

    def fingerprint(self) -> str:
        """Deterministic content hash for deduplication."""
        content = json.dumps(self.to_sharegpt_format(), sort_keys=True)
        return hashlib.md5(content.encode()).hexdigest()


class FineTuningDataPipeline:
    """
    End-to-end pipeline for preparing LLM fine-tuning data.
    Handles: cleaning, formatting, deduplication, quality filtering, versioning.
    """

    def __init__(self, output_dir: str, min_quality_score: float = 0.6):
        self.output_dir = Path(output_dir)
        self.output_dir.mkdir(parents=True, exist_ok=True)
        self.min_quality_score = min_quality_score

    def clean_text(self, text: str) -> str:
        """Remove common artifacts from scraped or extracted text."""
        # Remove HTML entities
        text = text.replace("&amp;", "&").replace("&lt;", "<").replace("&gt;", ">")
        text = text.replace("&quot;", '"').replace("&#39;", "'")

        # Normalize whitespace
        text = re.sub(r'\n{3,}', '\n\n', text)
        text = re.sub(r'[ \t]+', ' ', text)
        text = text.strip()

        return text

    def filter_quality(self, examples: List[TrainingExample]) -> List[TrainingExample]:
        """
        Apply quality filters. Returns only high-quality examples.
        Filters: minimum length, no PII patterns, no toxic content markers,
                 quality score threshold.
        """
        filtered = []
        stats = {"total": len(examples), "too_short": 0, "pii_detected": 0,
                 "low_quality": 0, "kept": 0}

        # Simple PII patterns (production: use a proper PII detector)
        pii_patterns = [
            r'\b\d{3}-\d{2}-\d{4}\b',    # SSN
            r'\b\d{16}\b',                 # credit card
            r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',  # email
        ]

        for ex in examples:
            # Length filter: skip very short outputs
            if ex.messages:
                assistant_content = next(
                    (m["content"] for m in ex.messages if m["role"] == "assistant"), ""
                )
                if len(assistant_content.split()) < 20:
                    stats["too_short"] += 1
                    continue

            # PII filter
            full_text = json.dumps(ex.messages)
            has_pii = any(re.search(p, full_text) for p in pii_patterns)
            if has_pii:
                stats["pii_detected"] += 1
                continue

            # Quality score filter
            if ex.quality_score is not None and ex.quality_score < self.min_quality_score:
                stats["low_quality"] += 1
                continue

            filtered.append(ex)
            stats["kept"] += 1

        print(f"Quality filtering: {stats}")
        return filtered

    def deduplicate(self, examples: List[TrainingExample]) -> List[TrainingExample]:
        """Remove exact and near-duplicate examples using content fingerprints."""
        seen_fingerprints = set()
        unique = []

        for ex in examples:
            fp = ex.fingerprint()
            if fp not in seen_fingerprints:
                seen_fingerprints.add(fp)
                unique.append(ex)

        removed = len(examples) - len(unique)
        print(f"Deduplication: removed {removed} duplicates ({removed/len(examples):.1%})")
        return unique

    def create_train_val_split(
        self,
        examples: List[TrainingExample],
        val_fraction: float = 0.05,
        seed: int = 42
    ) -> tuple:
        """Create deterministic train/val split."""
        import random
        random.seed(seed)
        shuffled = examples.copy()
        random.shuffle(shuffled)

        val_size = max(50, int(len(shuffled) * val_fraction))
        val = shuffled[:val_size]
        train = shuffled[val_size:]

        return train, val

    def save_versioned_dataset(
        self,
        train: List[TrainingExample],
        val: List[TrainingExample],
        customer_id: str,
        version: str
    ) -> dict:
        """
        Save dataset with versioning metadata.
        Returns dataset manifest for reproducibility.
        """
        customer_dir = self.output_dir / customer_id / version
        customer_dir.mkdir(parents=True, exist_ok=True)

        # Save in ShareGPT format for maximum compatibility
        train_path = customer_dir / "train.jsonl"
        val_path = customer_dir / "val.jsonl"

        with open(train_path, "w") as f:
            for ex in train:
                f.write(json.dumps(ex.to_sharegpt_format()) + "\n")

        with open(val_path, "w") as f:
            for ex in val:
                f.write(json.dumps(ex.to_sharegpt_format()) + "\n")

        manifest = {
            "customer_id": customer_id,
            "version": version,
            "train_examples": len(train),
            "val_examples": len(val),
            "train_path": str(train_path),
            "val_path": str(val_path),
            "train_fingerprint": hashlib.md5(
                "".join(ex.fingerprint() for ex in train).encode()
            ).hexdigest(),
            "created_at": str(Path(train_path).stat().st_mtime)
        }

        with open(customer_dir / "manifest.json", "w") as f:
            json.dump(manifest, f, indent=2)

        return manifest

---

LoRA: The Foundation of Scalable Fine-Tuning

LoRA (Low-Rank Adaptation) is the technique that makes multi-tenant fine-tuning economically viable. Instead of updating all model weights (billions of parameters), LoRA adds small trainable low-rank matrices to the attention layers. The base model is frozen; only the adapter is trained.

For a weight matrix $W \in \mathbb{R}^{d \times k}$, LoRA adds:

$W' = W + \Delta W = W + BA$

Where $B \in \mathbb{R}^{d \times r}$ and $A \in \mathbb{R}^{r \times k}$, with rank $r \ll \min(d, k)$.

A full Llama-2-7B fine-tune: ~28GB of weights. A LoRA adapter for the same model: ~200MB.

# LoRA fine-tuning configuration using PEFT
from dataclasses import dataclass, field
from typing import List

@dataclass
class LoRAConfig:
    """LoRA adapter training configuration."""
    # LoRA hyperparameters
    r: int = 16                    # LoRA rank (8-64 typical range)
    lora_alpha: int = 32           # scaling factor (alpha/r is effective lr scale)
    lora_dropout: float = 0.1
    target_modules: List[str] = field(default_factory=lambda: [
        "q_proj", "k_proj", "v_proj", "o_proj",  # attention
        "gate_proj", "up_proj", "down_proj"        # MLP
    ])
    bias: str = "none"

    # Training hyperparameters
    base_model: str = "meta-llama/Llama-2-7b-chat-hf"
    num_epochs: int = 3
    learning_rate: float = 2e-4
    per_device_train_batch_size: int = 4
    gradient_accumulation_steps: int = 4  # effective batch size = 4*4 = 16
    warmup_steps: int = 100
    max_seq_length: int = 2048
    fp16: bool = True

    # Data
    train_data_path: str = ""
    val_data_path: str = ""

    # Output
    output_dir: str = ""
    customer_id: str = ""
    adapter_version: str = ""


def create_peft_training_script(config: LoRAConfig) -> str:
    """Generate a reproducible training script from config."""
    return f"""#!/usr/bin/env python3
# Auto-generated fine-tuning script
# Customer: {config.customer_id}
# Adapter version: {config.adapter_version}

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer
from datasets import load_dataset

# Load base model (frozen)
model = AutoModelForCausalLM.from_pretrained(
    "{config.base_model}",
    torch_dtype="float16",
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("{config.base_model}")
tokenizer.pad_token = tokenizer.eos_token

# LoRA configuration
lora_config = LoraConfig(
    r={config.r},
    lora_alpha={config.lora_alpha},
    lora_dropout={config.lora_dropout},
    target_modules={config.target_modules},
    bias="{config.bias}",
    task_type=TaskType.CAUSAL_LM,
)

# Apply LoRA to frozen base model
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Expected: ~0.1-1% of total parameters are trainable

# Training arguments
training_args = TrainingArguments(
    output_dir="{config.output_dir}",
    num_train_epochs={config.num_epochs},
    per_device_train_batch_size={config.per_device_train_batch_size},
    gradient_accumulation_steps={config.gradient_accumulation_steps},
    learning_rate={config.learning_rate},
    warmup_steps={config.warmup_steps},
    fp16={config.fp16},
    evaluation_strategy="steps",
    eval_steps=100,
    save_strategy="steps",
    save_steps=100,
    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    greater_is_better=False,
    logging_steps=10,
    report_to=["wandb"],  # experiment tracking
    run_name="{config.customer_id}_{config.adapter_version}",
)

# Load data
dataset = load_dataset(
    "json",
    data_files={{
        "train": "{config.train_data_path}",
        "validation": "{config.val_data_path}"
    }}
)

# Train
trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
    tokenizer=tokenizer,
    max_seq_length={config.max_seq_length},
    dataset_text_field="conversations",  # ShareGPT format
)

trainer.train()

# Save only the adapter (NOT the full model - saves 99% of storage)
model.save_pretrained("{config.output_dir}/adapter")
tokenizer.save_pretrained("{config.output_dir}/adapter")

print(f"Adapter saved to {config.output_dir}/adapter")
print(f"Adapter size: {{sum(p.numel() for p in model.parameters() if p.requires_grad):,}} parameters")
"""

---

The Adapter Registry

When you have 50 customers each with multiple adapter versions, you need a registry - a single source of truth for which adapters exist, what they were trained on, their evaluation metrics, and their deployment status.

import sqlite3
import json
from datetime import datetime
from enum import Enum

class AdapterStatus(Enum):
    TRAINING = "training"
    EVALUATING = "evaluating"
    STAGING = "staging"
    PRODUCTION = "production"
    DEPRECATED = "deprecated"
    FAILED = "failed"

class AdapterRegistry:
    """
    Centralized registry for LoRA adapters.
    Tracks: adapter metadata, training provenance, evaluation metrics, deployment status.
    """

    def __init__(self, db_path: str = "adapter_registry.db"):
        self.conn = sqlite3.connect(db_path, check_same_thread=False)
        self._init_schema()

    def _init_schema(self):
        self.conn.execute("""
            CREATE TABLE IF NOT EXISTS adapters (
                adapter_id TEXT PRIMARY KEY,
                customer_id TEXT NOT NULL,
                base_model TEXT NOT NULL,
                version TEXT NOT NULL,
                status TEXT NOT NULL,
                storage_path TEXT,
                dataset_version TEXT,
                train_examples INTEGER,
                lora_rank INTEGER,
                training_run_id TEXT,
                eval_metrics TEXT,  -- JSON blob
                created_at TEXT,
                updated_at TEXT,
                notes TEXT,
                UNIQUE(customer_id, version)
            )
        """)
        self.conn.execute("""
            CREATE INDEX IF NOT EXISTS idx_customer_status
            ON adapters(customer_id, status)
        """)
        self.conn.commit()

    def register_adapter(
        self,
        customer_id: str,
        base_model: str,
        version: str,
        storage_path: str,
        dataset_version: str,
        train_examples: int,
        lora_rank: int,
        training_run_id: str,
    ) -> str:
        """Register a newly trained adapter. Returns adapter_id."""
        adapter_id = f"{customer_id}_{version}"
        now = datetime.now().isoformat()

        self.conn.execute("""
            INSERT INTO adapters
                (adapter_id, customer_id, base_model, version, status,
                 storage_path, dataset_version, train_examples, lora_rank,
                 training_run_id, eval_metrics, created_at, updated_at)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
        """, (
            adapter_id, customer_id, base_model, version,
            AdapterStatus.EVALUATING.value, storage_path, dataset_version,
            train_examples, lora_rank, training_run_id, json.dumps({}), now, now
        ))
        self.conn.commit()
        return adapter_id

    def update_eval_metrics(self, adapter_id: str, metrics: dict):
        """Record evaluation results for an adapter."""
        now = datetime.now().isoformat()
        self.conn.execute("""
            UPDATE adapters
            SET eval_metrics = ?, updated_at = ?
            WHERE adapter_id = ?
        """, (json.dumps(metrics), now, adapter_id))
        self.conn.commit()

    def promote_to_production(self, adapter_id: str) -> bool:
        """
        Promote adapter to production. Automatically deprecates the previous
        production adapter for the same customer.
        """
        # Get customer_id for this adapter
        row = self.conn.execute(
            "SELECT customer_id FROM adapters WHERE adapter_id = ?", (adapter_id,)
        ).fetchone()

        if not row:
            return False

        customer_id = row[0]
        now = datetime.now().isoformat()

        # Deprecate existing production adapter for this customer
        self.conn.execute("""
            UPDATE adapters
            SET status = ?, updated_at = ?
            WHERE customer_id = ? AND status = ?
        """, (AdapterStatus.DEPRECATED.value, now, customer_id, AdapterStatus.PRODUCTION.value))

        # Promote new adapter
        self.conn.execute("""
            UPDATE adapters
            SET status = ?, updated_at = ?
            WHERE adapter_id = ?
        """, (AdapterStatus.PRODUCTION.value, now, adapter_id))

        self.conn.commit()
        return True

    def get_production_adapter(self, customer_id: str) -> Optional[dict]:
        """Get the current production adapter for a customer."""
        row = self.conn.execute("""
            SELECT * FROM adapters
            WHERE customer_id = ? AND status = ?
        """, (customer_id, AdapterStatus.PRODUCTION.value)).fetchone()

        if not row:
            return None

        columns = ["adapter_id", "customer_id", "base_model", "version", "status",
                   "storage_path", "dataset_version", "train_examples", "lora_rank",
                   "training_run_id", "eval_metrics", "created_at", "updated_at", "notes"]
        result = dict(zip(columns, row))
        result["eval_metrics"] = json.loads(result["eval_metrics"])
        return result

    def list_adapters(self, customer_id: Optional[str] = None, status: Optional[str] = None) -> List[dict]:
        """List adapters with optional filters."""
        query = "SELECT * FROM adapters WHERE 1=1"
        params = []
        if customer_id:
            query += " AND customer_id = ?"
            params.append(customer_id)
        if status:
            query += " AND status = ?"
            params.append(status)
        query += " ORDER BY created_at DESC"

        rows = self.conn.execute(query, params).fetchall()
        columns = ["adapter_id", "customer_id", "base_model", "version", "status",
                   "storage_path", "dataset_version", "train_examples", "lora_rank",
                   "training_run_id", "eval_metrics", "created_at", "updated_at", "notes"]
        return [dict(zip(columns, row)) for row in rows]

---

Multi-Tenant Adapter Serving

The key innovation that makes 50 adapters economically viable: adapter hot-swapping. Load the base model once into GPU VRAM, and swap LoRA adapters on a per-request basis.

# Multi-tenant serving with dynamic adapter loading
from typing import Optional, Dict
import torch

class MultiTenantLLMServer:
    """
    Serves multiple LoRA adapters from a single base model.

    Instead of loading 50 separate models (50x GPU cost),
    load the base model once and dynamically load adapters per request.

    Libraries: vLLM (with multi-LoRA support) or TGI (with LoRA serving).
    This shows the conceptual design; production uses vLLM's LoRARequest API.
    """

    def __init__(self, base_model_id: str, max_cached_adapters: int = 10):
        """
        Args:
            base_model_id: HuggingFace model ID for the base model
            max_cached_adapters: How many adapters to keep loaded in GPU VRAM
        """
        self.base_model_id = base_model_id
        self.max_cached = max_cached_adapters
        self.adapter_cache: Dict[str, object] = {}  # customer_id -> loaded adapter
        self.cache_order: List[str] = []  # LRU order

        # In production: use vLLM with LoRARequest
        # from vllm import LLM, SamplingParams
        # from vllm.lora.request import LoRARequest
        # self.llm = LLM(model=base_model_id, enable_lora=True, max_loras=10)

    def _load_adapter(self, customer_id: str, adapter_path: str) -> object:
        """Load a LoRA adapter into GPU memory."""
        # Production: vLLM handles this transparently
        # Conceptual version:
        from peft import PeftModel
        adapter = PeftModel.from_pretrained(
            self.base_model,  # frozen base
            adapter_path,
            is_trainable=False
        )
        return adapter

    def _evict_lru(self):
        """Evict least recently used adapter when cache is full."""
        if len(self.cache_order) >= self.max_cached:
            lru_customer = self.cache_order.pop(0)
            # Unload adapter from GPU
            del self.adapter_cache[lru_customer]
            torch.cuda.empty_cache()
            print(f"Evicted adapter for customer: {lru_customer}")

    def generate(
        self,
        customer_id: str,
        adapter_path: str,
        prompt: str,
        max_tokens: int = 512,
        temperature: float = 0.1
    ) -> str:
        """
        Generate response for a customer using their specific adapter.
        """
        # Cache management
        if customer_id not in self.adapter_cache:
            self._evict_lru()
            self.adapter_cache[customer_id] = self._load_adapter(customer_id, adapter_path)

        # Update LRU order
        if customer_id in self.cache_order:
            self.cache_order.remove(customer_id)
        self.cache_order.append(customer_id)

        # Generate with customer-specific adapter
        # Production uses: self.llm.generate(prompt, LoRARequest(customer_id, 1, adapter_path))
        adapter = self.adapter_cache[customer_id]
        # ... tokenize, generate, decode ...
        return "generated response"

    def get_cache_stats(self) -> dict:
        return {
            "cached_adapters": list(self.adapter_cache.keys()),
            "cache_utilization": len(self.adapter_cache) / self.max_cached,
            "lru_order": self.cache_order
        }

Production recommendation: Use vLLM with multi-LoRA support. vLLM handles adapter loading, memory management, and batching adapters from multiple customers in a single forward pass. It can serve 100+ adapters from a single base model instance on 2× A100 GPUs - equivalent to serving 2 base models, not 100.

---

Continuous Fine-Tuning

Customers continuously provide new data. A continuous fine-tuning system incorporates new data into updated adapters on a regular cadence without catastrophic forgetting.

---

Production Engineering Notes

LoRA rank selection: r=8 is sufficient for domain adaptation (learning terminology and style). r=16 is better for behavioral alignment (learning task-specific reasoning patterns). r=64 approaches full fine-tuning expressiveness but with 8x the adapter size. Start with r=16 and reduce if adapters need to stay small.

Alpha/rank ratio: Setting lora_alpha = 2*r (e.g., alpha=32 with r=16) is a common default. The effective learning rate scales as alpha/r, so increasing alpha while keeping r fixed increases the learning rate without increasing adapter size.

Evaluation before promotion: Never promote an adapter to production without running it against your evaluation suite. At minimum: (1) held-out validation loss from training, (2) task-specific eval benchmarks for the customer's use case, (3) regression tests against a golden dataset of expected input/output pairs, (4) safety eval for toxic or off-topic outputs.

Storage strategy: Store adapter checkpoints in object storage (S3, GCS), not in your training cluster's local storage. Use versioned paths: s3://adapters/{customer_id}/{base_model}/{adapter_version}/. Only promote adapters that pass evaluation to a "production" prefix - keeping training checkpoints separate from production artifacts.

---

Common Mistakes

:::danger Fine-Tuning on Unfiltered Customer Data Customer-provided data often contains PII, proprietary information, offensive content, and factually wrong examples. Training on this directly produces models that leak PII, generate harmful content, or confidently produce misinformation. Always run PII detection, content safety filtering, and factual consistency checks before any data enters the training pipeline. :::

:::danger Full Fine-Tuning When LoRA Is Sufficient Full fine-tuning a 7B model costs ~$500 and produces a 28GB artifact per customer. LoRA fine-tuning costs ~$50 and produces a 200MB artifact. For domain adaptation and style transfer, LoRA achieves 90–95% of full fine-tuning quality at 10% of the cost and storage. Reserve full fine-tuning for cases where LoRA demonstrably underperforms - this is rare for adapter rank >= 32. :::

:::warning Catastrophic Forgetting in Continuous Fine-Tuning If you fine-tune only on new data in each update, the model forgets capabilities from earlier training. A customer who sends 50 new examples per week will have a model that by month 6 has effectively been fine-tuned only on the last 200 examples. Always mix new data with a replay buffer (random sample from the full historical dataset). A 20% replay fraction typically prevents most catastrophic forgetting. :::

:::warning Ignoring Adapter Versioning in Serving "We'll just update the adapter file in place." This is a disaster waiting to happen. In-flight requests when an adapter is being updated will see inconsistent behavior. The audit trail for why a customer's model changed disappears. A bad update cannot be rolled back. Always use immutable versioned adapter paths, and update the serving layer's configuration (via the registry) to point to a new version atomically. :::

---

Interview Q&A

Q: Why use LoRA fine-tuning instead of full fine-tuning for a multi-tenant LLM platform?

A: Three reasons: storage, cost, and serving efficiency. Full fine-tuning a 7B model produces a 28GB artifact per customer; LoRA produces ~200MB. At 50 customers, full fine-tuning requires 1.4TB of storage just for model weights versus 10GB for LoRA. Cost: full fine-tuning takes 8+ hours on 8× A100s ($500+); LoRA fine-tuning takes 1–2 hours on a single A100 ($25–50). Serving: full fine-tuning requires a dedicated GPU instance per customer; LoRA adapters can be hot-swapped onto a shared base model, serving 50 customers with 2 GPUs instead of 50. The quality difference between full fine-tuning and LoRA at rank 16–32 is negligible for domain adaptation and style transfer tasks - it only becomes relevant for deep architectural changes.

Q: Explain the LoRA technique mathematically and why it works.

A: LoRA (Low-Rank Adaptation) decomposes the weight update matrix into two smaller matrices: $\Delta W = BA$ where $B \in \mathbb{R}^{d \times r}$ and $A \in \mathbb{R}^{r \times k}$, with rank $r \ll \min(d,k)$. The base model weights $W$ are frozen - only $A$ and $B$ are trained. The key insight from the LoRA paper (Hu et al., 2021) is that pre-trained language models have a low "intrinsic dimensionality" - the effective change in weights during task-specific adaptation lies in a low-dimensional subspace. This means you can represent the necessary adaptation with far fewer parameters than the full weight matrix. The rank controls the expressiveness of the adaptation: r=4 is sufficient for style transfer, r=64 approaches full fine-tuning expressiveness. During inference, you can merge the adapter into the base model ($W' = W + BA$) for zero-latency overhead, or keep them separate for dynamic multi-tenant serving.

Q: How would you design a continuous fine-tuning pipeline for 50 enterprise customers?

A: The pipeline has five stages. First, data ingestion: each customer has an API endpoint and a data portal for submitting new training examples. All incoming data is queued, tagged with customer ID and timestamp, and staged for processing. Second, data pipeline: automated quality filtering (PII detection, content safety, length filters), deduplication against the existing customer dataset, and formatting into the training format. Third, training trigger: trigger a new fine-tuning job when the customer accumulates enough new data (e.g., 100 new examples) or on a weekly schedule. The training job uses the new data plus a replay buffer of 20% historical examples to prevent catastrophic forgetting. Fourth, evaluation gate: automated evaluation against the customer's benchmark suite and regression tests against a golden dataset. Only adapters that pass the evaluation gate are eligible for promotion. Fifth, deployment: update the adapter registry to mark the new adapter as production, and the serving layer picks up the new adapter path at next request time. A failed evaluation triggers an alert and keeps the previous adapter in production.

Q: What is catastrophic forgetting and how do you prevent it in fine-tuning?

A: Catastrophic forgetting is when a model that is fine-tuned on new data loses its previously learned capabilities. In practice for LLMs: a customer model fine-tuned on 500 new examples in month 3 might start forgetting the patterns it learned from the initial 1000 examples in months 1 and 2. The model's performance on older task types degrades even as it improves on the newest training examples. Prevention strategies: (1) Replay buffer - always mix new data with a random sample of historical data (20–30% replay fraction). This ensures the model continues to see examples from all periods during each fine-tuning update. (2) Elastic Weight Consolidation (EWC) - adds a regularization term that penalizes changes to weights that were important for previous tasks. More complex but more principled. (3) Learning rate scheduling - use a very low learning rate for fine-tuning updates to minimize the magnitude of weight changes. (4) Adapter-based methods like LoRA inherently limit catastrophic forgetting because the base model is frozen - forgetting can only occur in the adapter parameters, which are much smaller.