# MLOps for models that have to run in production

Kitecorp builds the infrastructure that turns a model in a notebook into a service with a pipeline, a registry, a serving endpoint and a bill you can predict: training and serving on Kubernetes with GPU node pools, or on SageMaker, Vertex AI and Bedrock where managed fits, with the same discipline we bring to any production system.

Canonical: https://kitecorp.org/services/mlops

## What does an MLOps engagement deliver?

A path from experiment to production that a data scientist can walk without a platform engineer at their side: a training pipeline that runs on a schedule or on new data, a registry that records which model is where, a serving endpoint with autoscaling and a rollback, and the monitoring that tells you when the model's inputs or outputs have drifted. The runtime is Kubernetes with GPU node pools, or SageMaker, Vertex AI and Bedrock where the managed services fit, and the engagement names which and why. The same practices as any production system apply: infrastructure as code, pipelines that prove themselves, SLOs, on-call runbooks.

## How do training and serving pipelines fit on Kubernetes?

Training runs as jobs on a GPU node pool that scales to zero when idle, orchestrated by Kubeflow Pipelines or Ray where the workload is distributed. Artefacts land in object storage and the registry (MLflow or the cloud's own) records the lineage: data version, code version, parameters, metrics. Serving runs on KServe, which gives each model an endpoint with canary rollouts, autoscaling on request concurrency and scale-to-zero for the long tail of rarely called models. GPU nodes are tainted so only workloads that need them land there, and the node pool's shapes match the models' memory needs rather than the largest instance available.

## How do we serve models at scale and at a known cost?

By choosing the serving stack per model. Large language models go to vLLM, where continuous batching and paged attention make throughput per GPU the number that drives cost; smaller models batch on KServe's predictors; models that need custom pre- and post-processing run on Ray Serve. Managed endpoints on SageMaker, Vertex AI or Bedrock take the models where the convenience outweighs the margin. Every endpoint gets a cost line on the dashboard, derived from its GPU-seconds, so the cost per thousand requests is a known figure before a product depends on it.

## How do we track models, data and experiments?

With a registry as the source of truth and the pipeline as the only writer to it. MLflow, or the registry built into SageMaker or Vertex AI, holds every model version with the data snapshot, code commit and parameters that produced it, and the serving layer deploys by registry reference rather than by file path. That makes a rollback a registry operation and an audit a query. Data versioning follows the same rule: training reads from a versioned snapshot in object storage, never from a live table.

## How does an engagement run?

A discovery call to understand the models, the team that builds them and where they run today. An assessment of the current path from notebook to production, with the pipeline, registry and serving design proposed and the runtime choice argued. The implementation with your data scientists and engineers, one model end to end first. A handover with the cost dashboard, the drift alerts and the runbooks. Pricing follows the discovery call, once we know the scope.

## Frequently asked questions

### Do we need Kubernetes to run models?

No. SageMaker, Vertex AI and Bedrock run training and serving without a cluster, and for many teams that is the right start. Kubernetes with GPU node pools pays off when you run many models, need control over the serving stack, or the managed bill has outgrown a platform team.

### Which serving stack do you use?

KServe on Kubernetes for general models, vLLM for large language models where throughput per GPU matters, Ray Serve where the serving logic is itself a pipeline, and the cloud's managed endpoints where they fit. The assessment picks per model.

### How do you keep GPU costs under control?

Right-sized node pools that scale to zero, batching and quantisation on the serving side, spot capacity for training, and a cost line per model on the dashboard so nobody is surprised at month end.

### Do you work with our existing team?

Yes. Every engagement is delivered with your engineers and ends with a handover, runbooks and alerts they own.

### How do we start?

Send a message through the contact page. We schedule a discovery call, agree the scope, and pricing follows what we find.

## How to start

Send a message at https://kitecorp.org/contact or email admin@kitelang.cloud. Pricing follows a discovery call.