24/05/2024, 01:27 6 Levels of ML Adoption.

Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

Get unlimited access to the best of Medium for less than $1/week. Become a member

6 Levels of ML Adoption
Ali Arsanjani · Following
5 min read · Nov 6, 2022

Listen Share More

Ali Arsanjani,
Edited, from a transcription and summary of a recent talk, in collaboration with Joel
Milag, Wow AI and Team

In this insightful session, Dr. Ali Arsanjani, Director of Cloud Partner Engineering at
Google Cloud, discussed with the audience how to leverage AI/ML throughout the
lifecycle to increase organizational maturity in the use of ML and how to add
business value as a result.

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 1/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

Check out his keynote on our website and YouTube channel.

6 levels of organizational maturity of AI/ML

Ali suggested answering a few questions to measure the achievement of higher
organizational maturity levels in the adoption, usage and activation of AI/ML for
achieving business value across the ML life cycle:

Level 1: Have the data science and machine learning goals for explicitly clarified use
Open inclassification,
cases been defined in terms of clustering, app regression, forecasting,
and recommendation AI?
Search

Level 2: Is your team consistently applying the entire life cycle for machine learning
along with different phases of data cleansing, feature store, selecting an algorithm
and training, evaluating the results, explainability and going into hyperparameter
optimization, deploying to an endpoint, monitoring the endpoint, having a pipeline
and doing it all over again?

Level 3: Do you have a documented reference for activities within the machine
learning lifecycle?

Level 4: Have you been able to automate some, if not most tasks?

Level 5: Are you measuring, tracking activities, and retrospectively recognizing

bottlenecks?

Level 6: Is the team continuously use data and new learnings to improve the
machine learning process?

These are questions you need to ask yourself, your vendors, and your teams
periodically to measure and assess the process.

Addressing 4 hidden technical debts in ML systems

Dr. Arsanjani went on to reference a 2017 paper published by his colleagues called
“Hidden Technical Debt in Machine Learning Systems”. He specifically emphasized
the following tech debt areas that are often overlooked in favor of more common
tech debt.

Data Testing Debt

If data replaces code in ML and code should be tested, then some amount of input
data testing is critical to a well-functioning system.

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 2/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

Reproducibility Debt
Usually, once good results are obtained, we move on but re-running experiments
with different types of data to garner similar results is extremely important.

Process Management Debt

There is a problem with updating configurations for similar models safely and
automatically and how to manage and assign resources among the models with
these different business priorities so you can visualize and detect blockers in the
operations performed across the MLOps lifecycle using pipelines.

Cultural Debt
Creating team cultures that reward the rationalization, reduction of features,
general reduction of overall complexity, improvements in reproducibility and
scalability, and monitoring to the same degree that improvements in accuracy are
valued is one of the most important factors.

Ali believed that these less visible / less popularly considered elements of hidden
technical debt are vital to enhancing and accelerating your maturity across the
AI/ML lifecycle.

6 Components of ML adoption
He then summed up the six components that are key to the adoption of ML.

1. Data Activation requires Data Lakes ingestion and Data Prep capability via Data
Prep pipeline

2. Data for AI calls for dataprep, labeling and featurization

3. Models entail considering experiments and explainability as first class

constructs

4. Models demand training and optimization and ultimately depositing the meta
data of the experiments into a registry

5. Models are served but need to be monitored for various types of drift or skew

6. The entire ML life cycle should be supported by data, training, and inference
(deployment and monitoring) pipelines

Key components of the ML journey in more detail

Component 1: Data & The Data Lake

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 3/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

BigLake is a secure and governed data lake that gathers data from different clouds
and there are self-serve analytics as well as interoperability between AI and those
analytics. BigLake basically expands BigQuery and allows you to take into account
Spark, TensorFlow, Presto, or other sets of capabilities.

A few new BigLake capabilities also let you build a differentiated data platform,
such as Analytics Hub, BigQuery ML support, BigQuery Omni, and the upcoming
Cloud Data Loss Prevention profiling support and data masking & audit logging.

Component 2: The Feature Store

Feature sharing is used across different use cases. To leverage one feature, you have
to insert it into a feature store after it’s cleaned and prepared so everyone can use it.
The Vertex AI feature store, as recommended by Dr. Ali Arsanjani, comes in handy
in this situation. It allows continuous training, experimentation, integration with
CI/CD and serving and monitoring.

Component 3 & 4: Experiments and the Model Registry

According to Ali, companies would want to be able to manage data lineage, training
details, hyper-parameters used in each experiment by compiling this meta-data
depositing it in a model registry, such as Vertex AI Registry.

If Data Gravity is a key requirement and you do not want to move the tabular and/or
unstructured data, that may already be residing in say, BigQuery, BigQuery ML can
be used to run a diverse array of algorithms for classification, regression, model
operations, and time series right there from the BigQuery data warehouse. The
results can be integrated into Vertex AI Registry.

If you wish or are ok to move data, you can use the full power of Pre-trained,
AutoML or Custom training that Vertex AI offers. You can choose to use the Vertex
AI workbench to run data science experiments in python that can help you enrich
the data and put it in a place you can leverage, train, optimize and deploy models.

Component 5: The MLOps Pipeline

The experimentation, retraining, model deployment, and continuous monitoring
are integrated together in a governance framework to achieve an end-to-end MLOps
capability with Vertex AI pipelines. Ali also emphasized the importance of XAI :
having explainability integrated into the data, model building, inference portions of
the overall AI/ML lifecycle.

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 4/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

Screenshot taken from Ali Arsanjani’s presentation

Dr. Arsanjani ended his keynote leaving the audience with the idea that maturity,
the ability to actualize the business benefits of AI/ML, evolves as the ability of the
organization to leverage each portion of the lifecycle and to traverse the AI/ML
digital innovation spectrum grows. By picking the best-fit tool for the part of the
lifecycle your organization is currently focused on, you can nurture, develop, scale
and operationalize AI/ML capabilities, you’ll be able to integrate these capabilities
not as siloes but as a continuous digital innovation spectrum that different teams in
your organization can traverse and derive business benefit.

Mlops Machine Learning Organizational Design Artificial Intelligence

Maturity Model

Following

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 5/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

Written by Ali Arsanjani

2.1K Followers

Director Google, AI/ML & GenAI| EX: WW Tech Leader, Chief Principal AI/ML Solution Architect, AWS | IBM
Distinguished Engineer and CTO Analytics & ML

More from Ali Arsanjani

Ali Arsanjani

The GenAI Reference Architecture

26 min read · Apr 29, 2024

825 6

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 6/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

Ali Arsanjani

GenAI Roadmap : A Guide for Enterprises on How to Implement Gen AI

Applications, Part 1
7 min read · Apr 13, 2024

223 5

Ali Arsanjani

The GenAI Maturity Model

Is your AI Enterprise ready ?
https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 7/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

24 min read · 6 days ago

237 4

Ali Arsanjani

The Generative AI Life-cycle

The common AI/ML Lifecycle consists of data collection, preparation, training, evaluation,
deployment and monitoring all encompassed with…

18 min read · Mar 22, 2023

546 2

See all from Ali Arsanjani

Recommended from Medium

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 8/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

Ali Arsanjani

GenAI Strategy and ROI

4 min read · Apr 13, 2024

106 1

Architech

Building Data Architecture For AI/ML Use Cases

Originally published on Building Data Architecture For AI/ML Use Cases — Architech

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 9/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

11 min read · Feb 1, 2024

Lists

Predictive Modeling w/ Python

20 stories · 1208 saves

Natural Language Processing

1464 stories · 979 saves

AI Regulation
6 stories · 458 saves

Practical Guides to Machine Learning

10 stories · 1460 saves

Yi Zhou in Generative AI Revolution

Smart GenAI Adoption: A Strategic Guide for Executives, Board Members

& Investors — Part II
Perform a total cost of ownership (TCO) analysis of adopting generative AI, spanning 9 areas—
tools, prompt engineering, inference…

13 min read · Dec 29, 2023

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 10/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

123

Sukirti Gupta

Optimizing Generative AI Application — Part 2 (GenAI Application

Deployment Stack)
In the first part of this series, we discussed a typical machine learning Project Life Cycle. In this
section, we will talk about a…

3 min read · Feb 1, 2024

20

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 11/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

Dr Nicolas Figay

The emerging landscape of Enterprise Architecture and Ontology

Introduction

21 min read · Apr 21, 2024

13

Louis ADAM

Anatomy of a Data Platform — How to choose your data architecture

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 12/13
24/05/2024, 01:27 6 Levels of ML Adoption. Ali Arsanjani, Edited, from a… | by Ali Arsanjani | Medium

I’ve seen many data architectures during my assignments in different companies. Sometimes,
these architectures are excellently customized…

8 min read · Apr 4, 2024

120 1

See more recommendations

https://dr-arsanjani.medium.com/6-levels-of-ml-adoption-fc3a2b90a3f0 13/13