Mohamed Sabri: Operationalize machine learning with Kubeflow

•

0 likes•318 views

This document summarizes a hands-on workshop on Kubeflow Pipeline. The workshop will cover requirements, an introduction to the presenter Mohamed Sabri, and their approach of strategizing, shaping, and spreading knowledge. It then discusses operationalizing machine learning (MLOps) and provides an analysis, design, coaching, and implementation framework. Deliverables include an implemented MLOps environment, training sessions, design documents, and a recommendations roadmap. The rest of the document discusses MLOps architectures, challenges, example technologies and tools, a use case, and deployment workflows from notebooks to production.

MOHAMED
SABRI
Mentor and data science leader

Mohamed Sabri: Operationalize machine learning with Kubeflow

Spread –
Operationalize (MLOps)
Strategize
Shape
Spread

This is no traditional consulting
where we burn your cash
Our philosophy
We deliver concrete value
with full transparancy

Our approach
in MLOps
Analyze
Design
Coach
Implement

Delivrables
The implementation
of a viable environment
in MLOps
MLOps training
sessions
High-Level and Low-
Level Design
Document
A report with
recommendations
and roadmap
The following deliverables to be expected during and
after our mandate:

How to define an
MLOps efficient
architecture?
What is the level of expertise in MLOps
do we have or willing to hire?
What type of inference in ML are we looking for?
How many machine learning projects do we have
inline in short/mid/long term?
Automation vs Resource scalability?
Which type of vendors is the company working
with? Or open source vs Enterprise
Is the data science team following the state of the
art when it comes to source coding and versioning?

Microservices design environment
Machine learning
experimentation
Performance monitoring
Model serving
Retraining pipelines
Model registry
Development
environment
Dashboard for
monitoring
Data
analysis
Trigger retraining
Pipelines deployment
Experiment
tracking
Source code & versioning
Automatic
detection of
new models
Framework for
microservices
Push new model for deployment
Automated
pipelines
Challenges
• Data scientists need more
education about code submission
and production-ready code.
• The customer is looking to scale the
environment for all the
organization’s machine learning
projects.
• No clear performance metrics have
been defined by the customer to
evaluate model performance in
production.
• Helping the customer identify the
right resources internally to
maintain the environment

Microservices design environment
Challenges
• Reaching a low latency (maximum
15 ms) to allow a fast reactivity after
model inference.
• Handling a large volume of data
points per second (between 1
million to 10 million per second)
• Scaling the streams, environment
based on data volume with no
buffer.
• Automatically updating the machine
learning model if required.
Model registry
Domain events
Update
docker
image
connectors
Machine learning
experimentation
Push new model for deployment
Logs & KPI storage
Real time monitoring
Steam
processing
Data
analysis
Development
environment

Some technologies and tools
End to end platform
Continuous delivery
platform
Commercial
Microservice
deployment
Automation and
pipelines
Experimentation
tracking and
versioning
Open source

For you, what is MLOps ? Why is it
necessary ?

MLOps is like DevOps but for ML
• Continuous integration (CI)
CI is about testing and validating code and components, but also testing and validating
data, data schemas, and models.
• Continuous delivery (CD)
CD is about a system (an ML training pipeline) that should automatically deploy another
service (model prediction service).
• Continuous training (CT)
CT is concerned with automatically retraining and serving the models.

Our architecture
Data
extraction
Data pre-
processing
Building
classifier
Trigger
deployment
Model registry (persistent
volume)
Data pre-
processing
Inference
model
Automated
Training
pipeline
ML Engine
Integration with app

What's hot

20160317 - PAZUR - PowerBI & R

Łukasz Grala

This document discusses PowerBI and R. It provides an overview of Microsoft R products including Microsoft R Open, Microsoft R Server, and SQL Server R Services. It explains how SQL Server R Services integrates R with SQL Server for scalable in-database analytics. Examples of using R with PowerBI, SQL Server, and Azure are provided. The document also compares the capabilities of Microsoft R Open, Microsoft R Server, and open source R and discusses using R for advanced analytics, predictive modeling, and big data at scale.

Jeeves Grows Up: An AI Chatbot for Performance and Quality

Databricks

Sarah: CEO-Finance-Report pipeline seems to be slow today. Why Jeeves: SparkSQL query dbt_fin_model in CEO-Finance-Report is running 53% slower on 2/28/2021. Data skew issue detected. Issue has not been seen in last 90 days. Jeeves: Adding 5 more nodes to cluster recommended for CEO-Finance-Report to finish in its 99th percentile time of 5.2 hours. Who is Jeeves? An experienced Spark developer? A seasoned administrator? No, Jeeves is a chatbot created to simplify data operations management for enterprise Spark clusters. This chatbot is powered by advanced AI algorithms and an intuitive conversational interface that together provide answers to get users in and out of problems quickly. Instead of being stuck to screens displaying logs and metrics, users can now have a more refreshing experience via a two-way conversation with their own personal Spark expert. We presented Jeeves at Spark Summit 2019. In the two years since, Jeeves has grown up a lot. Jeeves can now learn continuously as telemetry information streams in from more and more applications, especially SQL queries. Jeeves now “knows” about data pipelines that have many components. Jeeves can also answer questions about data quality in addition to performance, cost, failures, and SLAs. For example: Tom: I am not seeing any data for today in my Campaign Metrics Dashboard. Jeeves: 3/5 validations failed on the cmp_kpis table on 2/28/2021. Run of pipeline cmp_incremental_daily failed on 2/28/2021. This talk will give an overview of the newer capabilities of the chatbot, and how it now fits in a modern data stack with the emergence of new data roles like analytics engineers and machine learning engineers. You will learn how to build chatbots that tackle your complex data operations challenges.

Building Data Science into Organizations: Field Experience

Databricks

We will share our experiences in building Data Science and Machine Learning (DS/ML) into organizations. As new DS/ML teams are created, many wrestle with questions such as: How can we most efficiently achieve short-term goals while planning for scale and production long-term? How should DS/ML be incorporated into a company? We will bring unique perspectives: one as a previous Databricks customer leading a DS team, one as the second ML engineer at Databricks, and both as current Solutions Architects guiding customers through their DS/ML journeys.We will cover best practices through the crawl-walk-run journey of DS/ML: how to immediately become more productive with an initial team, how to scale and move towards production when needed, and how to integrate effectively with the broader organization. This talk is meant for technical leaders who are building new DS/ML teams or helping to spread DS/ML practices across their organizations. Technology discussion will focus on Databricks, but the lessons apply to any tech platforms in this space.

Summary introduction to data engineering

Novita Sari

Data engineering involves designing, building, and maintaining data warehouses to transform raw data into queryable forms that enable analytics. A core task of data engineers is Extract, Transform, and Load (ETL) processes - extracting data from sources, transforming it through processes like filtering and aggregation, and loading it into destinations. Data engineers help divide systems into transactional (OLTP) and analytical (OLAP) databases, with OLTP providing source data to data warehouses analyzed through OLAP systems. While similar, data engineers focus more on infrastructure and ETL processes, while data scientists focus more on analysis, modeling, and insights.

Afternoons with Azure - Azure Machine Learning

CCG

Accelerating Data Science and Machine Learning Workflow with Azure Machine Le...

Aditya Bhattacharya

Afternoons with Azure - Power BI and Azure Analysis Services

CCG

CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks

Databricks

Data & ML projects bring many new complexities beyond the traditional software development lifecycle. Unlike software projects, after they were successfully delivered and deployed, they cannot be abandoned but must be continuously monitored if model performance still satisfies all requirements. We can always get new data with new statistical characteristics that can break our pipelines or influence model performance. All these qualities of data & ML projects lead us to the necessity of continuous testing and monitoring of our models and pipelines.

AI is a Team Sport

Harvinder Atwal

To analyse why operationalizing AI is so challenging, it’s important to understand the full lifecycle of an AI project, and identify the stakeholders involved. Through 2023, Gartner estimates that 50% of IT leaders will struggle to move their AI projects past proof of concept (POC) to a production level of maturity. To reduce this high failure rate, organisations need to build the right roles for AI success. In many organisations, data scientists are still wearing too many hats due to a dearth of talent across other roles. This session will highlight how, in order to successfully operationalise and scale AI POCs, organisations must to build diverse AI roles and skills with a collaborative structure that is paramount.

Learn to Use Databricks for the Full ML Lifecycle

Databricks

Machine learning development brings many new complexities beyond the traditional software development lifecycle. Unlike traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. In this talk, learn how to operationalize ML across the full lifecycle with Databricks Machine Learning.

Spark Summit East Keynote by Anjul Bhambhri

Jen Aman

Building data "Py-pelines"

Rob Winters

Empowering Real Time Patient Care Through Spark Streaming

Databricks

Takeda’s Plasma Derived Therapies (PDT) business unit has recently embarked on a project to use Spark Streaming on Databricks to empower how they deliver value to their Plasma Donation centers. As patients come in and interface without clinics, we store and track all of the patient interactions in real time and deliver outputs and results based on said interactions. The current problem with our existing architecture is that it is very expensive to maintain and has an unsustainable number of failure points. Spark Streaming is essential for allowing this use case because it allows for a more robust ETL pipeline. With Spark Streaming, we are able to replace our existing ETL processes (that are based on Lamdbas, step functions, triggered jobs, etc) into a purely stream driven architecture. Data is brought into our s3 raw layer as a large set of CSV files through AWS DMS and Informatica IICS as these services bring data from on-prem systems into our cloud layer. We have a stream currently running which takes these raw files up and merges them into Delta tables established in the bronze/stage layer. We are using AWS Glue as the metadata provider for all of these operations. From the stage layer, we have another set of streams using the stage Delta tables as their source, which transform and conduct stream to stream lookups before writing the enriched records into RDS (silver/prod layer). Once the data has been merged into RDS we have a DMS task which lifts the data back into S3 as CSV files. We have a small intermediary stream which merge these CSV files into corresponding delta tables, from which we have our gold/analytic streams. The on-prem systems are able to speak to the silver layer and allow for the near real-time latency that our patient care centers require.

What’s New with Databricks Machine Learning

Databricks

Guiding through a typical Machine Learning Pipeline

Michael Gerke

Pm.ais ummit 180917 final

Nisha Talagala

Skill up in machine learning using Azure ML

Mostafa

The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...

Databricks

Many had dubbed 2020 as the decade of data. This is indeed an era of data zeitgeist. From code-centric software development 1.0, we are entering software development 2.0, a data-centric and data-driven approach, where data plays a central theme in our everyday lives. As the volume and variety of data garnered from myriad data sources continue to grow at an astronomical scale and as cloud computing offers cheap computing and data storage resources at scale, the data platforms have to match in their abilities to process, analyze, and visualize at scale and speed and with ease — this involves data paradigm shifts in processing and storing and in providing programming frameworks to developers to access and work with these data platforms. In this talk, we will survey some emerging technologies that address the challenges of data at scale, how these tools help data scientists and machine learning developers with their data tasks, why they scale, and how they facilitate the future data scientists to start quickly. In particular, we will examine in detail two open-source tools MLflow (for machine learning life cycle development) and Delta Lake (for reliable storage for structured and unstructured data). Other emerging tools such as Koalas help data scientists to do exploratory data analysis at scale in a language and framework they are familiar with as well as emerging data + AI trends in 2021. You will understand the challenges of machine learning model development at scale, why you need reliable and scalable storage, and what other open source tools are at your disposal to do data science and machine learning at scale.

Building a Just in Time Data Warehouse by Dan Morris and Jason Pohl

Spark Summit

This document summarizes Viacom's approach to building a just-in-time data warehouse for product analytics. It discusses how Viacom uses Spark and Databricks on Amazon S3 to enable flexible exploratory analysis and A/B testing without needing to pre-define schemas or move data into a data warehouse. This just-in-time approach allows Viacom's product analysts to quickly get insights, test hypotheses, and monitor key metrics through dashboards for their mobile apps across 23 brands in over 170 countries. The summary highlights how Viacom uses this flexible approach to support their goals of growing global audiences through disciplined product development and data-driven decision making.

Spark Summit Keynote by Seshu Adunuthula

Spark Summit

1) eBay's enterprise data platform uses Apache Spark and Hadoop to process large amounts of structured and unstructured data from various sources to power applications and analytics. 2) Key aspects of the platform include an agile data warehouse, data streams platform using Apache Kafka, and data services to simplify access to data and enable collaborative analytics. 3) eBay leverages this platform to power applications such as search, personalization, fraud prevention, and business intelligence through pipelines that ingest behavioral and transactional data.

What's hot (20)

20160317 - PAZUR - PowerBI & R

Jeeves Grows Up: An AI Chatbot for Performance and Quality

Building Data Science into Organizations: Field Experience

Summary introduction to data engineering

Afternoons with Azure - Azure Machine Learning

Accelerating Data Science and Machine Learning Workflow with Azure Machine Le...

Afternoons with Azure - Power BI and Azure Analysis Services

CI/CD Templates: Continuous Delivery of ML-Enabled Data Pipelines on Databricks

AI is a Team Sport

Learn to Use Databricks for the Full ML Lifecycle

Spark Summit East Keynote by Anjul Bhambhri

Building data "Py-pelines"

Empowering Real Time Patient Care Through Spark Streaming

What’s New with Databricks Machine Learning

Guiding through a typical Machine Learning Pipeline

Pm.ais ummit 180917 final

Skill up in machine learning using Azure ML

The Future of Data Science and Machine Learning at Scale: A Look at MLflow, D...

Building a Just in Time Data Warehouse by Dan Morris and Jason Pohl

Spark Summit Keynote by Seshu Adunuthula

Similar to Mohamed Sabri: Operationalize machine learning with Kubeflow

Dmitry Spodarets: Modern MLOps toolchain 2023

Mohamed Sabri: Operationalize machine learning with Kubeflow

More Related Content

What's hot

What's hot (20)

Similar to Mohamed Sabri: Operationalize machine learning with Kubeflow

Similar to Mohamed Sabri: Operationalize machine learning with Kubeflow (20)

More from Lviv Startup Club

More from Lviv Startup Club (20)

Recently uploaded

Recently uploaded (20)

Mohamed Sabri: Operationalize machine learning with Kubeflow