01 · Problem
Every model was handmade
Each model or soft sensor for client plants was built by hand: connecting the data, preparing the dataset, training, saving and deploying. With a small team, that pace limited how many use cases we could take on.
02 · Solution
Treat models as a product you manufacture
An internal platform, inspired by lean manufacturing, where the whole cycle runs without code: secure database connection, dataset management, model training, registry and backup, deployment and direct integration into the SaaS product. The same platform manages the recommendations and advanced alerts, built on statistical modeling of the data.
03 · Why not another tool
We evaluated the options on the market
Before building, we evaluated existing tools. None solved the whole problem:
- MLflow handles experiment tracking and model registry well, but that was not all we needed: we wanted training and deploying a model to require no code at all.
- Amazon SageMaker covers the full machine learning lifecycle, but for our size it was oversized: it demanded a large implementation effort, its usage costs are hard to control, and it did not solve monitoring or immediate integration with our product.
So we built our own platform, with capabilities similar to an MLOps suite but tailored to our data and our product. It covers the full cycle:
- Direct, secure integration with our data sources.
- Data cleaning, both algorithmic and visual: sensor data often carries outliers that are hard for an algorithm to tell apart, and being able to review them visually speeds up the work a lot.
- Agile transformation of the variables each model needs.
- Model registry and observability.
- Direct deployment of each model, talking to the SaaS product with no intermediate steps.
04 · Evolution
From months to a day
- 1Handmade
Each model was built and deployed by hand: about three months to production.
- 2Systematized
We automated and standardized steps of the process, and the time dropped to about a month.
- 3Platform
With the platform, a model could be in production within a day. The remaining time goes into exploring the data and finding the best model.
05 · Results
What changed
Real results · no figures for confidentiality- Decrease: Far less time to take a model to production
- Increase: Models trained
06 · My contribution
I led the process change
- 01Diagnose
- Identified the bottleneck in model implementation.
- 02Design
- Designed the model production process.
- 03Decide
- Decided to build our own platform instead of adopting an existing tool.
- 04Align
- Aligned the team around the new workflow.
07 · Technologies