MLOps, Deployment & Responsible AI

Running AI reliably in the real world. Learn deployment patterns (batch, real-time API, streaming, edge), model packaging and serving (Docker, ONNX, model servers), latency and throughput SLOs, CI/CD for ML, experiment tracking and model registries, feature stores, monitoring (data drift with PSI, concept drift, performance decay), retraining strategies, A/B and shadow testing, fairness metrics (demographic parity, equal opportunity), explainability, privacy, security and AI governance — with Python computing PSI drift, disparate impact and p95 latency.

Most ML models that work in notebooks never reach production — and many that do degrade silently as the world changes. MLOps brings software-engineering discipline (versioning, testing, automation, monitoring) to machine learning.

Responsible AI ensures systems are fair, transparent, private, secure and accountable — increasingly a legal requirement, not just good practice.

An analogy

Deploying a model is like launching a satellite: building it is only half the job. You need launch procedures (CI/CD), telemetry (monitoring), course corrections (retraining) and rules of the road (governance) to keep it useful for years.

Deployment patterns

How models reach users:

MLOps practices

Version everything: code (Git), data (DVC/lakehouse snapshots), models (registry with lineage). Track experiments (MLflow, Weights & Biases). Automate pipelines (training, validation, packaging) with CI/CD; test data schemas, feature distributions and model quality gates. Feature stores keep training and serving features consistent. Release safely with shadow deployments, canaries and A/B tests; keep rollback paths.

Monitoring and retraining

Monitor system metrics (latency, errors, throughput, cost), data quality (missing values, schema changes), data drift (PSI, KS test), prediction drift and — when labels arrive — real performance (concept drift). Retrain on schedule, on drift triggers or continuously; always validate against the current champion model before promotion. For LLM apps, monitor answer quality, refusals, hallucination reports and user feedback.

Responsible AI and governance

Fairness: measure metrics per group (demographic parity, equal opportunity, calibration) — they can conflict, so choose based on context and law. Explainability: SHAP/LIME, model cards and datasheets. Privacy: data minimisation, anonymisation, differential privacy, federated learning; comply with DPDP Act/GDPR. Security: adversarial examples, data poisoning, model extraction, prompt injection. Governance: risk classification (EU AI Act), human oversight, audit trails, incident response and red-teaming.

Formulas

Worked example: PSI term

Given: Bin expected 0.10, actual 0.20

Find: Contribution.

Answer: 0.069

Worked example: Throughput

Given: Latency 50 ms, 8 concurrent workers

Find: QPS.

Answer: 160 QPS

Worked example: Disparate impact

Given: Rates 0.60 and 0.54

Find: DI.

Answer: 0.90 — passes 80% rule

Drift, fairness and latency checks

import math
# Data drift with the Population Stability Index and a fairness check (demographic parity)
expected = [0.10, 0.20, 0.40, 0.20, 0.10]      # training distribution over 5 bins
actual   = [0.05, 0.15, 0.35, 0.25, 0.20]      # production this week
psi = sum((a - e) * math.log(a / e) for a, e in zip(actual, expected))
status = "no drift" if psi < 0.1 else "moderate drift" if psi < 0.25 else "major drift -> retrain"
print(f"PSI = {psi:.3f} ({status})")
# Approval rates by group
approved = {"group A": (420, 600), "group B": (270, 500)}
rates = {g: a / n for g, (a, n) in approved.items()}
for g, r in rates.items():
    print(f"{g}: approval rate {r:.2f}")
ratio = min(rates.values()) / max(rates.values())
print(f"disparate impact ratio {ratio:.2f} ({'passes' if ratio >= 0.8 else 'fails'} the 80% rule)")
# Latency SLO check: p95 of request latencies (ms)
lat = sorted([42, 38, 55, 61, 47, 120, 44, 39, 52, 48, 58, 41, 95, 46, 50, 43, 49, 40, 57, 45])
p95 = lat[math.ceil(0.95 * len(lat)) - 1]
print(f"p95 latency {p95} ms ({'meets' if p95 <= 100 else 'violates'} 100 ms SLO)")
# PSI = 0.136 (moderate drift)
# group A: approval rate 0.70
# group B: approval rate 0.54
# disparate impact ratio 0.77 (fails the 80% rule)
# p95 latency 95 ms (meets 100 ms SLO)

The input distribution has shifted toward higher bins (PSI 0.136) — investigate before performance drops. Group B's approval rate is 77% of group A's, failing the common four-fifths rule: this needs investigation of features and labels, not an automatic fix. The p95 latency just meets its SLO, but the 120 ms outlier shows tail latency needs watching.

Common mistakes

Applications

Key takeaways

Going further