:::tip 🎮 Interactive Playground Visualize this concept: Try the ML System Design Framework demo on the EngineersOfAI Playground - no code required. :::

Requirements and Constraints for ML Systems

The specification is the design. If your requirements document is vague, your system will be vague - and vague systems fail in precise ways.

The Production Moment

The product manager slides a one-page brief across the conference table. "We want to add ML-powered fraud detection to our payment flow," it reads. "The model should be accurate and fast. We need it live in Q3."

You are the ML engineer in the room. You look at this document and see exactly what it doesn't say: How accurate? Accurate at what trade-off between false positives and false negatives? How fast? 10ms? 100ms? 500ms? Live in Q3 with what availability guarantee? What happens to payments if the fraud model is down? What data can you use to train it, and do you have any GDPR obligations around how you use it? Can the model make decisions automatically, or does every flag need human review for regulatory reasons?

The document says "accurate and fast." You need to turn that into a specification that a team of five engineers can build to, test against, and operate.

This is the work of requirements engineering for ML systems. It is unglamorous, often frustrating, and completely essential. More ML projects fail from under-specified requirements than from insufficient model accuracy. You can always train a better model. You cannot easily fix a system whose latency contract was never written down and turns out to be 10× more aggressive than anyone realized.

Here is how to do it right.

Why Requirements Engineering Is Different for ML Systems

Traditional software systems have well-understood requirements categories: functional (what the system does) and non-functional (how well it does it). A database either returns the correct query result or it doesn't. An API either responds within 200ms or it doesn't.

ML systems introduce a third dimension: probabilistic correctness. A fraud detection model doesn't "correctly classify a transaction" - it returns a probability score that translates into a business decision. The "correctness" of that output depends on a threshold, a business context, and an acceptable error rate that varies by use case. This makes requirements fundamentally harder to specify.

Additionally, ML systems have temporal requirements that traditional software doesn't: model freshness (how stale can the model be?), data freshness (how stale can the features be?), and prediction latency (which includes feature retrieval, model inference, and post-processing). Missing any of these dimensions produces a system that meets the letter of the requirements but fails in production.

The canonical taxonomy, codified in the ML engineering community through works like Sculley et al. (NeurIPS 2015), Zinkevich's "Rules of Machine Learning" (Google, 2016), and Huyen's "Designing Machine Learning Systems" (2022), organizes ML requirements into four categories: functional, non-functional, ML-specific, and constraints.

Category 1: Functional Requirements

Functional requirements define what the system must do. They answer: what inputs does it accept, what outputs does it produce, and what transformations happen in between?

For ML systems, functional requirements decompose into three sub-systems:

Prediction Interface

What does the system expose to consumers?

INPUT:  transaction_id (UUID), user_id (UUID), amount (float),
        merchant_category (string), timestamp (ISO 8601)

OUTPUT: fraud_score (float, 0–1), decision (ALLOW/BLOCK/REVIEW),
        confidence (float, 0–1), explanation (list of top features)

LATENCY: P99 under 80ms

This is a precise functional spec. Compare it to "the system should predict whether a transaction is fraudulent." The precise version tells an engineer exactly what to build. The vague version leaves every decision open.

Training Pipeline

What must the training system do?

- Ingest labeled transaction data from data warehouse (daily)
- Engineer features: velocity features (transactions/hour),
  device fingerprint, behavioral biometrics
- Train gradient boosting model on 6 months of data
- Evaluate on holdout set: minimum AUC-ROC 0.95, FPR at 1% at TPR >= 0.85
- Deploy to serving infrastructure if metrics pass
- Retain previous model version for 30 days (rollback capability)

Monitoring and Observability

What must the system report?

- Real-time metrics: request latency (p50/p95/p99), error rate, throughput
- ML metrics: prediction distribution (daily), feature drift (weekly),
  model accuracy on labeled samples (weekly)
- Business metrics: fraud caught rate, false positive rate,
  manual review rate, dollar losses prevented (daily)
- Alerting: page on-call if p99 latency exceeds 150ms,
  or if fraud caught rate drops > 10% from 7-day rolling average

Category 2: Non-Functional Requirements

Non-functional requirements define the quality attributes of the system. For ML systems at production scale, five are critical: latency, throughput, availability, consistency, and scalability.

Latency: The Latency Budget

Latency is not a single number - it is a distribution, and what matters is the tail of that distribution. The p99 latency (the 99th percentile - what the slowest 1% of requests experience) is what your worst users feel, and it determines whether your system is acceptable.

The industry standard notation is SLO (Service Level Objective): a target that, if missed, triggers alerting and investigation.

Latency SLO:
  p50 < 20ms
  p95 < 50ms
  p99 < 80ms
  p999 < 200ms  (1 in 1000 requests can be slower)

More importantly, you need a latency budget - an allocation of the end-to-end budget to each component:

Without an explicit latency budget, every team adds their own 20ms component and you end up at 300ms with nobody responsible for the overrun.

:::tip Real Industry Numbers Netflix's recommendation service has a p99 latency target of 100ms. Google Search requires results in under 200ms end-to-end including the neural ranking model. Stripe's fraud detection must complete in under 50ms because the payment authorization flow waits on it. These numbers are not arbitrary - they are derived from user research showing where latency starts to affect behavior. :::

Throughput

Throughput is the volume of work the system must handle: requests per second (RPS), queries per second (QPS), or tokens per second for LLM serving.

The critical number to specify is peak QPS - the maximum throughput the system must sustain without degradation. Peak is typically 3–5× average in consumer applications (morning/evening traffic spikes) and can be 10–50× average during events (Black Friday for e-commerce, breaking news for content).

def estimate_peak_qps(dau: int, avg_requests_per_user_per_day: float,
                       peak_factor: float = 3.0) -> float:
    """Estimate peak QPS from daily active users."""
    avg_qps = (dau * avg_requests_per_user_per_day) / 86_400
    peak_qps = avg_qps * peak_factor
    return peak_qps

# Payment fraud detection example
fraud_qps = estimate_peak_qps(
    dau=5_000_000,           # 5M daily active paying users
    avg_requests_per_user_per_day=3.0,  # avg 3 transactions/day
    peak_factor=5.0          # Black Friday spike
)
print(f"Peak QPS: {fraud_qps:.0f}")  # Peak QPS: 868

Availability

Availability is the percentage of time the system correctly responds to requests. It is commonly expressed as "nines":

Availability	Downtime per year	Downtime per month
99% (2 nines)	87.6 hours	7.3 hours
99.9% (3 nines)	8.76 hours	43.8 minutes
99.99% (4 nines)	52.6 minutes	4.4 minutes
99.999% (5 nines)	5.26 minutes	26.3 seconds

The key ML-specific availability question: what happens when the ML model is unavailable? The right answer depends entirely on whether the ML decision is blocking (the user cannot proceed without it) or advisory (the system falls back to a rule-based system).

# Always design a fallback for blocking ML decisions
class FraudDetectionService:
    def __init__(self, ml_model, rule_engine):
        self.ml_model = ml_model
        self.rule_engine = rule_engine  # simple rule-based fallback

    def score_transaction(self, transaction: dict) -> dict:
        try:
            # Try ML model first (fast path)
            result = self.ml_model.predict(transaction)
            result['source'] = 'ml'
            return result
        except (TimeoutError, ModelUnavailableError) as e:
            # Fall back to rules when ML is down
            # This degrades quality but maintains availability
            result = self.rule_engine.evaluate(transaction)
            result['source'] = 'rules'  # track for monitoring
            return result

This pattern - ML-primary, rules-fallback - is standard in production payment systems. The availability SLO for the overall service can be 99.99% even if the ML model itself is only 99.9% available, because the fallback absorbs the downtime.

Consistency

For ML systems, consistency requirements are more nuanced than for databases. There are three distinct consistency concerns:

1. Prediction consistency: Does the same input always return the same prediction? (Usually yes, for deterministic models. Not always, for stochastic or ensemble systems.)

2. Feature consistency: Are the features computed for a user at serving time consistent with the features that will be logged for subsequent training? (Training-serving skew if no.)

3. Model version consistency: When you deploy a new model, do all serving replicas switch simultaneously, or can different users see different model versions during rollout?

Category 3: ML-Specific Requirements

These requirements exist only in ML systems and are the most commonly under-specified.

Accuracy and Evaluation Metrics

Do not specify accuracy as a single number. Specify the business metric, the evaluation protocol, and the acceptable operating point.

WRONG:  "The model should be at least 95% accurate."

RIGHT:  "The model must achieve:
         - AUC-ROC >= 0.95 on the holdout dataset (last 30 days)
         - At a 0.5% false positive rate (FPR = 0.005),
           true positive rate (TPR) must be >= 0.80
         - Evaluated monthly on new labeled data
         - Comparison baseline: current rule-based system at AUC-ROC 0.88"

The single accuracy number is meaningless without knowing the operating point (threshold), the evaluation set (distribution matters enormously), and the baseline you're comparing to.

Model Freshness

How often does the model need to be retrained?

• Stationary systems (product category classification): monthly retraining acceptable
• Slowly drifting systems (user preference recommendation): weekly retraining
• Fast-drifting systems (fraud detection, trending content): daily or continuous retraining
• Event-driven systems (ad campaigns, breaking news): trigger-based retraining on distribution shift detection

# Automated retraining trigger based on distribution shift
class RetrainingOrchestrator:
    def __init__(self, drift_threshold: float = 0.05):
        self.drift_threshold = drift_threshold

    def should_retrain(self, current_feature_dist: dict,
                       training_feature_dist: dict) -> bool:
        """PSI (Population Stability Index) > 0.2 indicates significant shift."""
        psi = self._calculate_psi(current_feature_dist, training_feature_dist)
        return psi > self.drift_threshold

    def _calculate_psi(self, current: dict, reference: dict) -> float:
        """Population Stability Index for numerical features."""
        import numpy as np
        psi_total = 0.0
        for feature_name in current:
            c = np.array(current[feature_name]) + 1e-10  # avoid log(0)
            r = np.array(reference[feature_name]) + 1e-10
            c = c / c.sum()
            r = r / r.sum()
            psi = np.sum((c - r) * np.log(c / r))
            psi_total += psi
        return psi_total / len(current)

Feature Freshness

Distinct from model freshness: how stale can the features used for inference be?

For a fraud detection system:

• Transaction velocity features (how many transactions in the last hour): must be real-time - a feature that is 10 minutes stale misses the pattern of rapid-fire fraud
• User account features (account age, lifetime transaction volume): can be 24 hours stale
• Device fingerprint features: can be 7 days stale

Document this explicitly per feature group, because it determines your streaming infrastructure requirements. If any feature must be under 1-minute fresh, you need Kafka + Flink. If 1-hour freshness is acceptable, batch pipelines may suffice.

Fairness Requirements

Increasingly, ML systems have legal and ethical requirements around demographic fairness. These must be specified precisely because they create a tension with accuracy metrics.

Fairness Requirements for Loan Approval Model:
  - Demographic parity: approval rate for protected classes
    must not differ by more than 5pp from overall approval rate
    (legal requirement under ECOA in the US)
  - Equal opportunity: true positive rate (loan approval for
    qualified applicants) must not differ by more than 5pp
    across demographic groups
  - Disparate impact: adverse impact ratio (AIR) must be >= 0.80
    for any protected class (80% rule under EEOC guidelines)
  - Explainability: every rejection must generate a human-readable
    explanation that satisfies Adverse Action Notice requirements

Explainability Requirements

Some industries require model explanations by law. Financial services (GDPR's "right to explanation," ECOA adverse action notices), healthcare (FDA oversight of clinical decision support), and hiring (EEOC guidelines) all create explainability requirements.

# SHAP-based explanation generation for regulatory compliance
import shap

class ExplainableFraudDetector:
    def __init__(self, model, feature_names: list[str]):
        self.model = model
        self.explainer = shap.TreeExplainer(model)
        self.feature_names = feature_names

    def predict_with_explanation(self, features: dict) -> dict:
        import numpy as np
        feature_array = np.array([features[f] for f in self.feature_names])
        score = self.model.predict_proba(feature_array.reshape(1, -1))[0][1]

        # SHAP values: contribution of each feature to the prediction
        shap_values = self.explainer.shap_values(feature_array.reshape(1, -1))

        # Top 3 features driving the prediction
        feature_impacts = sorted(
            zip(self.feature_names, shap_values[1][0].tolist()),
            key=lambda x: abs(x[1]),
            reverse=True
        )[:3]

        return {
            "fraud_score": float(score),
            "decision": "BLOCK" if score > 0.8 else "ALLOW",
            "explanation": [
                {"feature": name, "impact": impact, "direction": "increases" if impact > 0 else "decreases"}
                for name, impact in feature_impacts
            ]
        }

Category 4: Constraints

Constraints are requirements imposed by context, not by choice. They are often the most important requirements because they are inviolable.

Regulatory Constraints

GDPR (EU): If you process EU user data, users have the right to explanation of automated decisions, the right to opt out of profiling, and data deletion rights. This means:

• You cannot train on deleted users' data without explicit consent
• You must be able to remove a user's contribution from a trained model (or retrain without them)
• Automated decisions with legal effects require human review

CCPA (California): Users can opt out of "sale" of their data, which can include ML feature computation shared across products.

Financial regulations: Models used in credit decisions must comply with ECOA (Equal Credit Opportunity Act) and FCRA (Fair Credit Reporting Act) in the US.

Infrastructure Constraints

Constraint: Must deploy on-premise (financial services, no cloud)
  - Impact: Cannot use managed ML services (SageMaker, Vertex AI)
  - Impact: Must manage GPU infrastructure manually
  - Impact: Limits horizontal scaling options

Constraint: Must integrate with existing Oracle data warehouse
  - Impact: Feature computation SQL must run in Oracle
  - Impact: Cannot use Spark/BigQuery for feature engineering
  - Impact: May limit feature engineering complexity

Constraint: Team has 3 ML engineers and 2 platform engineers
  - Impact: Cannot build custom feature store from scratch
  - Impact: Must use managed services where possible
  - Impact: Operational complexity is a key constraint

Budget Constraints

def estimate_monthly_serving_cost(
    peak_qps: float,
    model_latency_ms: float,
    gpu_cost_per_hour: float = 3.0,  # A10G on AWS
    safety_factor: float = 3.0
) -> float:
    """Estimate monthly GPU cost for model serving."""
    # How many requests can one GPU handle?
    requests_per_gpu_per_second = 1000 / model_latency_ms
    # How many GPUs needed at peak?
    gpus_needed = (peak_qps / requests_per_gpu_per_second) * safety_factor
    # Monthly cost (24/7 operation)
    monthly_hours = 24 * 30
    return gpus_needed * gpu_cost_per_hour * monthly_hours

# Fraud detection at 1000 QPS, 20ms model latency
cost = estimate_monthly_serving_cost(
    peak_qps=1000,
    model_latency_ms=20,
    gpu_cost_per_hour=3.0
)
print(f"Estimated monthly GPU cost: ${cost:,.0f}")  # ~$6,480/month

SLO vs SLA vs SLI: The Complete Framework

These three terms are consistently confused in engineering discussions.

SLI (Service Level Indicator): A quantitative measurement. The raw metric. Examples: request latency, error rate, availability percentage.

SLO (Service Level Objective): A target value for an SLI. Internal agreement about what "good" looks like. "P99 latency SLO: under 100ms." Violating an SLO triggers internal alerting and investigation.

SLA (Service Level Agreement): A contract with customers that includes consequences for violations. "We guarantee 99.9% availability; if we miss it, you get a service credit." SLAs are derived from SLOs with a safety buffer.

SLI: p99 request latency (measured every 60 seconds)

SLO: p99 request latency < 100ms (99.5% of measurement windows)
     → If p99 exceeds 100ms, alert the on-call engineer

SLA: 99.9% availability (contractual guarantee to paying customers)
     p99 latency < 200ms (contractual guarantee)
     → 50ms headroom between SLO and SLA
     → SLA violation triggers service credits ($X per hour of violation)

:::note The SLO Buffer Always set SLOs more aggressive than SLAs. If your SLA guarantees 200ms p99, set an internal SLO at 100ms. This gives you reaction time to fix problems before they become SLA violations. Google SRE teams typically set SLOs at 50–100% more aggressive than SLAs. :::

Real Industry Examples

Netflix Recommendation Latency SLO

Netflix's personalized homepage recommendation system must return results in under 200ms. Their architecture is designed around this: candidate retrieval from precomputed ANN indexes (sub-10ms), feature fetch from EVCache (their distributed cache, sub-5ms), ranking model inference (sub-50ms). The entire latency budget is explicitly allocated.

Source: Netflix Technology Blog, "System Architectures for Personalization and Recommendation" (2013, updated practices)

Google Search Freshness

Google's web index has a crawl freshness requirement: important news content must appear in search results within minutes of publication. This drives their infrastructure: a "freshness pipeline" separate from the main indexing pipeline, with different compute and latency characteristics. The freshness SLO is measured in minutes, not hours.

Stripe Fraud Detection

Stripe's fraud detection runs in the critical path of payment authorization. The hard requirement is that it must complete before the card network timeout (typically 30 seconds, but Stripe targets sub-100ms to allow for network round-trips and provide a good UX). This means no model that requires more than 50ms inference is acceptable, regardless of how accurate it is.

The Requirements Checklist

Use this checklist when gathering requirements for any ML system:

FUNCTIONAL
[ ] What are the inputs and their types?
[ ] What are the outputs and their types?
[ ] What does the training pipeline produce?
[ ] What does the serving pipeline expose?
[ ] What monitoring/reporting is required?

NON-FUNCTIONAL
[ ] Latency: p50/p95/p99/p999 targets?
[ ] Throughput: average and peak QPS?
[ ] Availability: target nines? Fallback behavior?
[ ] Consistency: prediction consistency? Feature consistency?

ML-SPECIFIC
[ ] Accuracy metric and target? Baseline to beat?
[ ] Evaluation protocol (dataset, time period, holdout)?
[ ] Model freshness: retraining frequency?
[ ] Feature freshness: acceptable staleness per feature group?
[ ] Fairness requirements? Demographic groups? Metrics?
[ ] Explainability: what explanations are required? For whom?
[ ] Cold-start handling: what happens for new users/items?

CONSTRAINTS
[ ] Regulatory: GDPR? CCPA? Industry-specific (financial, health)?
[ ] Data: what data is available? What requires consent?
[ ] Infrastructure: cloud/on-premise? Existing systems to integrate?
[ ] Team: size and expertise? Operational complexity budget?
[ ] Budget: monthly serving cost ceiling? Training compute budget?

:::danger The Under-Specified Accuracy Trap "The model should be accurate" is not a requirement. "The model should achieve AUC-ROC >= 0.95 on a holdout of transactions from the last 30 days, with TPR >= 0.80 at FPR = 0.01, evaluated against a labeled dataset of 100,000 transactions" is a requirement. The difference is measurability. If you can't tell whether the requirement is met by running a test, it's not a requirement - it's a wish. :::

Interview Q&A

Q1: What is the difference between an SLO, SLA, and SLI? Give a concrete example for an ML serving system.

An SLI (Service Level Indicator) is the raw measurement: "we measure p99 request latency every 60 seconds." An SLO (Service Level Objective) is the target: "p99 latency should be under 100ms in 99.5% of measurement windows." An SLA (Service Level Agreement) is the contractual commitment to customers: "we guarantee p99 latency under 200ms; violations result in service credits."

The relationship: SLI provides the data, SLO sets the internal bar, SLA is the external commitment derived from the SLO with a safety buffer. For an ML serving system, you'd also have ML-specific SLOs: "prediction distribution should not drift more than 5% from the training distribution within any 24-hour window."

Q2: How do you handle GDPR's right-to-erasure requirement for a model that was trained on a user's data?

This is the "machine unlearning" problem, and there are three practical approaches:

First, retrain from scratch excluding the deleted user's data. Simple, correct, but expensive if you retrain frequently. Acceptable for models retrained weekly or monthly.

Second, sharded training: partition training data so each shard is trained separately, and when a user's data is deleted, only retrain the affected shard. This is the approach described in "Machine Unlearning" (Cao & Yang, IEEE S&P 2015).

Third, differential privacy: train the model with formal DP guarantees so that any individual's contribution to the model is bounded. When they request deletion, you can certify that their data's influence is below the privacy bound. Used by Apple, Google, and Meta for sensitive models.

In practice, most production teams use option 1 (retrain from scratch) with a data deletion queue that marks deleted users' data with a tombstone and excludes it from the next training run.

Q3: How would you specify freshness requirements for a real-time recommendation system?

Break freshness down by feature group, because different features have different staleness tolerances:

• Session features (videos watched in the last 10 minutes): must be under 1-minute fresh - requires real-time streaming with Kafka + Flink
• Recent preference features (videos liked in the last 7 days): can be 1-hour fresh - hourly batch jobs acceptable
• Long-term preference features (category preferences over years): can be 24-hour fresh - daily batch jobs
• Item metadata (video title, category, duration): can be 24-hour fresh - daily batch jobs
• Item popularity (view counts, trending score): must be 1-hour fresh for trending content

The freshness requirement drives the infrastructure choice. If any feature needs sub-minute freshness, you need a real-time streaming architecture. If all features can tolerate 1-hour staleness, batch pipelines are simpler and cheaper.

Q4: What are the key non-functional requirements that are unique to ML systems and not found in traditional software systems?

Traditional software has latency, throughput, availability, and consistency. ML systems add:

1. Prediction freshness: How often does the model need to be retrained? A model trained 6 months ago on pre-pandemic data may perform poorly today.

2. Feature freshness: How stale can input features be? A fraud detection model with stale velocity features misses active fraud campaigns.

3. Prediction consistency: The same user query should produce deterministic results (important for explainability and debugging). Stochastic inference (beam search sampling, dropout at inference) must be explicitly specified as acceptable or not.

4. Training-serving consistency: Features computed during training must match features computed during serving. This is not a concern in traditional software - it only arises because ML systems have a training phase that is separate from the serving phase.

5. Model version safety: Can you roll back instantly if a new model degrades performance? What is the acceptable model rollout latency?

Q5: How do you handle the trade-off between model accuracy and latency when they conflict?

This is a core ML system design trade-off, and the answer depends on the use case's latency sensitivity.

First, establish the latency constraint as hard or soft. In a payment fraud system, 100ms is hard - the card network will time out. In a content recommendation system, 200ms might be soft - slightly slower is acceptable if accuracy improves significantly.

If hard: you must sacrifice accuracy for latency. Techniques: model distillation (train a smaller "student" model to approximate a large "teacher" model), quantization (INT8 instead of FP32, typically 2–4× speedup with <1% accuracy loss), feature reduction (drop slow-to-compute features), early exit (for easy cases, use a shallow model; reserve the full model for hard cases).

If soft: you can optimize inference infrastructure - batching, GPU upgrade, model parallelism - before sacrificing accuracy. Often you can meet both requirements by throwing hardware at the problem, and it is cheaper than re-engineering the model.

Always make the trade-off explicit and documented: "We are using a 10M-parameter model instead of the 100M-parameter model to meet our 50ms latency SLO, accepting a 3pp drop in AUC-ROC from 0.96 to 0.93."

Summary

Requirements engineering for ML systems is a discipline of precision. The goal is to transform ambiguous business intent ("accurate and fast") into measurable, testable specifications that a team can build to and an organization can hold itself accountable to.

The four categories - functional, non-functional, ML-specific, and constraints - cover the full requirements space. The SLO/SLA/SLI framework gives you the vocabulary to write requirements that can be measured. The latency budget technique ensures no individual component silently consumes the entire end-to-end budget.

The habit to build: never let a requirement stay vague. If you can't write a test that fails when the requirement is violated, you haven't specified the requirement yet.

:::tip Key Takeaway The most expensive line in any ML system is the one that says "TBD - to be confirmed by product." Undefined requirements become architectural debt that costs 10× more to fix after the system is built than before. Invest the time upfront to write precise, measurable requirements for every dimension of the system. :::