Levels of MLOps

Google, Microsoft, and others have defined a number of standards for measuring the maturity level of MLOps. Here is an overview of the three MLOps levels defined by Google.

Level 0: Manual process

This is a typical data science process that is performed during the initial stages of implementing ML. This level is experimental and iterative in nature.

At Level 0, each step of the pipeline, including data preparation and validation, model training, testing, and deployment, is performed entirely manually, placing MLOps at its lowest level of maturity.

MLOps level 0: Manual process

Level 0 has the following specific characteristics:

• All steps, including data extraction and data analysis, are performed manually
• Machine learning and operations teams are separated
• Cannot handle frequent tuning and releases
• Only a portion of the ML system can be deployed
• Cannot track actions in production environment

Level 1: ML pipeline automation

Level 1 is where the ML pipeline is automated. As new data becomes available, the process of retraining the model is initiated.

Level 1 adds the following components:

• Data and model validation
• Feature store
• Metadata management
• ML pipeline triggers

MLOps level 1: ML pipeline automation

Level 1 has the following specific features:

• ML experiments can be performed quickly
• Machine learning teams, development and operations teams are coordinated
• Models can be automatically trained in the production environment
• Can accommodate continuous tuning and releases

Level 2: CI/CD pipeline automation

Level 2 deploys a CI/CD system to ensure fast and reliable deployment of ML models in the production environment. The main difference from Level 1 is that it automatically builds, tests, and deploys data, ML models, and ML learning pipeline components.

MLOps level 2: CI/CD pipeline automation

Level 2 has the following specific features:

• Development and testing environments are integrated
• Build, test, package, and deploy can be automated with specific triggers
• CI/CD of the pipeline has been achieved
• The pipeline is automatically executed in the production environment based on a schedule
• Production monitoring is possible

References

https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning#devops_versus_mlops
https://ml-ops.org/content/mlops-principles