KServe

• Map an InferenceService to pods, revisions and traffic splits
• Configure entrypoint and runtime (tensorflow/triton/sklearn) declaratively
• Express canary rollouts and rollbacks as config changes

• Serving becomes config in the cluster, so promotion, canary and rollback are GitOps affordances.
• It reuses Kubernetes primitives — Deployments, HPA, services — instead of a parallel world.
• Runtime images (sklearn, XGBoost, Triton, PyTorch) are selectable per model.
• Traffic splitting declvs canaries without extra deployment machinery.

Franchise headquarters (controller) provides the truck, permits and menu template; each truck (InferenceService) declares what it serves and its hours. New openings and menu changes are paperwork (config), not building new kitchens.

• You apply an InferenceService YAML naming the model and a serving runtime image.
• The controller reconciles it: creates a revision, deployment, service and autoscaling config.
• Traffic split fields route percentage between newer and current revisions.
• Knative handles scale-to-zero or minimum replicas per config.
• Rollback = resetting traffic split; the old revision is still addressable.

• A team rolls a new churn version with 10% traffic via a one-line canary field change.
• A bursty recommender uses scale-to-zero to save cluster cost off-peak.
• Promotion runs as a GitOps PR that bumps the registered model version in the InferenceService.

• Let the controller own serving; only reference registered artifacts.
• Use traffic splits for real canaries with monitoring before 100%.
• Pin runtime images and model versions together for reproducibility.
• Keep scale-to-zero off for latency-sensitive models; use minimum replicas.

• Hand-managing the deployment KServe created — you're now fighting the controller.
• Canaries without evaluation, so splits are theater.
• Missing resource limits; a bursty model starves the node beneath KServe.
• Runtime image drift between environments.

• InferenceService stuck not-ready — check the revision's PodFailed conditions and image pull logs.
• Traffic split not shifting — the revision may be from the wrong model version; verify with curl headers.
• Scale-to-zero latency — re-acquisition at idle; raise minimum replicas for interactive workloads.