EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Kubernetes for ML Deployments
Namespace: ml-productionActive
Deployment: model-server3 replicas
image: gcr.io/myproject/model:v2
CPU req: 500m
CPU limit: 2000m
Mem req: 1Gi
Mem limit: 4Gi
Port: 8080
Deployment: feature-server2 replicas
image: gcr.io/myproject/features:v1
CPU req: 200m
CPU limit: 1000m
Mem req: 512Mi
Mem limit: 2Gi
Port: 9090
Service
model-svc
ClusterIP
10.96.12.34:8080
Service
feature-svc
ClusterIP
10.96.12.35:9090
Service
inference-lb
LoadBalancer
34.102.x.x:443
ConfigMap
model-config
model_version: ...
batch_size: ...
timeout_ms: ...
Secret
model-registry-creds
registry_user: ***
registry_pass: ***
K8s Controls
model-server replicas
Replicas3
18
feature-server replicas
Replicas2
16
Deployments manage sets of identical pods with rolling updates.

Services provide stable DNS + load balancing across pods.

ConfigMaps hold non-secret config; Secrets hold credentials - never bake secrets into images.

Kubernetes for ML Deployments - Interactive Visualization

Kubernetes is the de facto runtime for production ML systems. A model serving deployment typically has: a Deployment managing N identical model-server pods, a Service providing stable DNS and load balancing, a ConfigMap holding model version and batch size config, and a Secret for registry credentials. GPU node pools run on dedicated nodes with the nvidia.com/gpu resource label. Resource quotas prevent any single team from exhausting cluster capacity. This interactive demo lets you scale replicas, toggle GPU nodes, and inspect resource quotas in real time.

  • Deployment: declares desired replica count and pod template - Kubernetes maintains this state automatically
  • Service (ClusterIP): stable internal DNS for pod-to-pod communication, even as pods restart
  • Service (LoadBalancer): external IP assigned by cloud provider for ingress traffic to model endpoints
  • ConfigMap: non-secret configuration like model version and batch size - mounted as env vars or files
  • Secret: encoded credentials for model registry - never bake secrets into container images
  • GPU node pools: labeled with nvidia.com/gpu, pods request GPUs via resource limits

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.