Your ML Platform Is a Science Fair Project

Here is a conversation I have had, almost word for word, at three different organisations in the past eighteen months:

Me: “How many models do you have in production?”

Them: “Well, we have one. But we’re planning to—”

Me: “And how does it get deployed?”

Them: “Dave runs the notebook and then copies the model file to the server.”

Me: “Right. And you want to build an MLOps platform.”

Them: “Yes. We’re looking at KubeFlow.”

This is the state of ML engineering in 2021. Every organisation with a data science team and a Gartner subscription has decided they need a Machine Learning platform. Most of them need a conversation with Dave about version control.

The hype cycle, on schedule

Gartner’s Hype Cycle for Artificial Intelligence has been tracking MLOps and related concepts for a couple of years now, and the trajectory is depressingly predictable. We’re somewhere between the Peak of Inflated Expectations and the Trough of Disillusionment, depending on who you talk to.

The narrative goes like this: data science is valuable (true). But data science experiments that never make it to production are worthless (also true). Therefore, we need platforms and processes to industrialise ML — to take models from notebooks to production reliably, reproducibly, and at scale (true in principle, wildly premature for most organisations in practice).

The gap between “we should industrialise ML” and “we have enough ML to industrialise” is enormous. And almost nobody is being honest about it.

What I’ve actually seen

I’ve worked at an AI consultancy and a global industrial firm in the past couple of years. Between them, I’ve seen the full spectrum of ML maturity — from genuinely sophisticated teams shipping models weekly, to organisations that think “AI strategy” means buying Tableau licenses.

Here’s the uncomfortable truth about most organisations’ ML capabilities:

One model in production, maybe. Often a recommendation engine, a fraud detection model, or a demand forecasting system. Frequently built by a single senior data scientist, deployed manually, monitored by… nobody. It works. It’s been working for two years. Nobody touches it because nobody fully understands it except the person who built it, and they’ve since been promoted to a management role.

A dozen experiments that never shipped. Jupyter notebooks on laptops, shared drives, or at best a shared JupyterHub instance. Experiments are tracked in spreadsheets or not at all. Data is pulled from production databases using credentials that would make a security team weep. Results are presented in PowerPoint to stakeholders who nod approvingly and then ask “so when can we use this?” The answer is always “soon,” and soon never arrives.

A data science team that’s disconnected from engineering. Data scientists report into a different part of the org from the software engineers. They use different tools, different languages, different workflows. The data scientists write Python in notebooks. The engineers write Java or Go in IDEs. The handoff between “the model works in my notebook” and “the model works in production” is a chasm that nobody has bridged because nobody owns it.

Enthusiasm massively outstripping capability. The leadership has seen the McKinsey reports. They’ve been to the conferences. They know AI is the future. They’ve committed budget. What they haven’t done is assess whether they have the foundational capabilities — clean data, reliable pipelines, production engineering skills — to actually benefit from a platform investment.

This is the landscape into which KubeFlow, SageMaker, MLflow, and a dozen other MLOps tools are being sold. It’s like selling a Formula One pit crew to someone who hasn’t passed their driving test.

KubeFlow: the Kubernetes-native nightmare

Let me talk about KubeFlow specifically, because I’ve seen it proposed (and occasionally deployed) at multiple organisations, and it deserves a proper examination.

KubeFlow’s pitch is seductive: a complete, Kubernetes-native ML platform. Pipelines for orchestrating training workflows. Experiment tracking. Model serving via KFServing (later renamed KServe, because the ML ecosystem apparently has an allergy to stable naming). Jupyter notebooks integrated into the platform. Hyperparameter tuning with Katib. All running on Kubernetes, so you get all the orchestration and scaling benefits of your existing cluster infrastructure.

In practice, here’s what deploying KubeFlow actually involves:

YAML. So much YAML. KubeFlow is deployed as a collection of Kubernetes resources — custom resource definitions, operators, controllers, services. The installation process, at the time I last dealt with it, involved either kfctl (a custom CLI tool with its own configuration format) or raw kustomize manifests that were sprawling and poorly documented. Getting it running on a fresh cluster was at least a day’s work for someone who knew Kubernetes well. Getting it running reliably, with authentication, storage, and networking configured for a real environment? Budget a week minimum.

Component soup. KubeFlow isn’t one thing — it’s a collection of loosely coupled components, each with its own release cycle, its own documentation (of wildly varying quality), and its own dependencies. Want pipelines? That’s the Pipelines component, built on Argo Workflows. Want experiment tracking? That’s separate. Want model serving? KServe, which has its own dependency on Knative or a compatible serverless runtime. Want notebooks? That’s the Notebooks component, which runs JupyterHub inside the cluster. Each piece can break independently, and diagnosing which component is misbehaving requires understanding all of them.

Breaking changes between versions. I lived through a KubeFlow upgrade that broke every pipeline in the system. The pipeline SDK had changed its API between minor versions, meaning every pipeline definition needed to be updated. The documentation for the migration path was a GitHub issue with thirty comments of people trying to figure out what had changed. For a platform that’s supposed to provide stability and reproducibility, this was — let’s say — ironic.

It assumes a maturity you don’t have. KubeFlow makes sense if you have a team of ML engineers who understand Kubernetes, a portfolio of models that need automated training and deployment, and the operational capability to maintain a complex platform. If you have a team of data scientists who are used to running notebooks on their laptops, KubeFlow is an alien spaceship. They don’t want to write pipeline definitions in Python SDK. They don’t want to think about container images, resource requests, or persistent volume claims. They want to run their notebook and get results.

The managed alternatives aren’t much better

“Fine,” I hear you say. “KubeFlow is too complex. We’ll use a managed service.” Enter Amazon SageMaker and Azure Machine Learning.

These are genuinely more polished than KubeFlow. The notebook experience is better. The training infrastructure is easier to use. Model deployment is simpler. But they come with their own problems:

Vendor lock-in. Once your pipelines, experiments, and model registry are in SageMaker, you’re in AWS. Moving to Azure or GCP is a significant engineering effort. Your pipeline definitions are SageMaker-specific. Your model artefacts are in S3 with SageMaker metadata. Your training jobs use SageMaker’s instance types and APIs. This is fine if you’ve committed to AWS forever. It’s less fine if you’re an organisation that values flexibility or has a multi-cloud strategy (and if you don’t, you should at least have the option).

Cost opacity. SageMaker’s pricing model is a masterclass in making it hard to predict your bill. Training instance hours, inference endpoint hours, data processing charges, storage, notebook instance hours — the bill arrives and it’s always more than you expected. I’ve seen organisations spend more on their SageMaker infrastructure — most of which was idle — than on the actual compute for their one production model.

It still doesn’t solve the people problem. A managed platform makes the infrastructure easier, but it doesn’t give you ML engineers who can design production-grade model serving, build monitoring for model drift, or create data pipelines that are reliable enough to train on. The platform is the easy part. The people and processes are the hard part, and no amount of managed services changes that.

The airport problem

Here’s my central metaphor, and I’m going to lean on it hard: building an ML platform for most organisations is like building an airport for one flight a week.

You’ve got one model in production. Maybe two. You’ve got a handful of experiments in various states of completion. Your total ML workload would fit comfortably on a single GPU instance. And you’re proposing to build — or buy — a platform that can orchestrate distributed training, manage hundreds of experiments, serve dozens of models, and track everything with full lineage and reproducibility.

That’s not engineering. That’s a science fair project. It’s building the thing because it’s technically interesting, not because the business needs it.

An airport for one flight a week is a vanity project. The right answer for one flight a week is a runway and a windsock.

What you actually need

If you have one or two models in production, here’s what you need:

Version control for your code and your data. Your model training code should be in Git. Your training data should be versioned — DVC does this well and it’s free. The combination of code version and data version should uniquely identify a model. This is not a platform — it’s basic hygiene.

A reproducible training process. A Dockerfile and a script that trains your model given a code version and a data version. No notebooks in the loop. The notebook is for exploration. The training script is for production. If you can’t run your training process from a clean checkout, you don’t have a process — you have a ritual that only works on Dave’s laptop.

A deployment mechanism. A container image that serves your model behind an API. Deployed the same way you deploy any other service — through your existing CI/CD pipeline. No special ML serving infrastructure needed. FastAPI and a Docker container will take you surprisingly far.

Basic monitoring. Prediction latency, error rates, input distribution drift. You don’t need a specialised ML monitoring platform. Prometheus and a few custom metrics will do. If the model’s predictions start looking weird, alert on it the same way you’d alert on any other service degradation.

A conversation about process. Who owns the model? Who decides when to retrain? What happens when the model’s performance degrades? What’s the rollback plan? These are organisational questions, not technical ones. A platform doesn’t answer them.

That’s it. That’s your MLOps strategy for a team with one or two models. It costs almost nothing. It uses tools you already have. And it’s infinitely more useful than a KubeFlow installation that nobody understands.

When you actually need a platform

Platforms become valuable when:

• You have ten or more models in production, and the operational overhead of managing them individually is becoming a bottleneck.
• You have ML engineers (not just data scientists) who can build and maintain the platform.
• Your training workloads are large enough that scheduling and resource management matter.
• You need reproducibility at scale — regulatory requirements, model governance, audit trails.
• The rate of experimentation is high enough that tracking experiments manually is no longer feasible.

If that sounds like your organisation, then yes, invest in a platform. Choose one that fits your team’s skills and your cloud strategy. Invest in the people to run it. Treat it as a product with users (your data scientists) and a roadmap.

If it doesn’t sound like your organisation — and for most of you reading this, it won’t — save your money. Invest in the fundamentals: clean data, version control, a reproducible training process, and a deployment pipeline. Get good at the boring stuff. Build the airport when you have the flights to justify it.

The real gap

The fundamental problem in ML engineering isn’t tooling. It’s the gap between data science experimentation and production software engineering.

Data scientists are trained to explore data, build models, and evaluate results. They’re good at it. What they’re generally not trained to do is build production software — write tests, handle errors, design APIs, manage dependencies, think about scale, deal with operational concerns. That’s not a criticism; it’s a different skill set.

Production ML requires both skill sets, or at least a team that collectively covers both. And most organisations don’t have that team. They have data scientists and software engineers who sit in different departments, use different tools, speak different languages, and have never worked closely together.

No platform bridges that gap. MLflow doesn’t bridge it. KubeFlow doesn’t bridge it. SageMaker doesn’t bridge it. The gap is closed by people working together — by data scientists learning enough engineering to write production-quality code, and by software engineers learning enough ML to understand model lifecycle, training data requirements, and evaluation metrics.

Until you’ve got those people — or at least a plan to develop them — buying a platform is putting the cart before the horse. Or, to return to my metaphor, building an airport before you’ve learned to fly.

Dave and his notebook might not be glamorous. But at least the flight lands.