Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Serverless Machine Learning

•Download as PPTX, PDF•
•
A
Asavari Tayal

From data ingestion, processing, model deployment to prediction - machine learning is hard! Join me to learn how serverless can make it all easier so you can stop worrying about the underlying infrastructure layer, and focus on getting the most value out of your data and development time.

Related slideshows

Shruti Panda Resume -Updated
Shruti Panda Resume -UpdatedShruti Panda Resume -Updated
Shruti Panda Resume -Updated
 
Getting Started with Azure AutoML
Getting Started with Azure AutoMLGetting Started with Azure AutoML
Getting Started with Azure AutoML
 
Resume for AskMe
Resume for AskMeResume for AskMe
Resume for AskMe
 
1 of 21
Serverless Machine Learning Predicting the future with the bleeding edge
About Me Product Manager on Azure Functions. Pythonista working on Serverless for over a year. Passionate about developer experiences, data science and machine learning. Asavari Tayal @tayalasavari
Plan for today 1 Machine Learning Introduction, process of ML and roles in ML. Serverless and FaaS How can they help with ML? Use Cases Real world examples of using Serverless ML. Learnings and ChallengesBenefits of Serverless ML, shortcomings and open questions. 2 3 4
What is Machine Learning? Practice of examining large amounts of data to find patterns and then generating code to find those same patterns in brand new data. ML can help you build smarter applications.
Process of Machine Learning Model Training 1 Candidate Model Chosen Model
Roles in Machine Learning Model Training Data Scientist App Developer Model Deployment ? How big is my training dataset? ? Which algorithm should I use? ? What’s the confusion matrix? ? How often should I retrain the model? ? How many servers do I need? ? How can I scale my application? ? Who monitors my app? ? How should I patch my servers? ? What packages should be on my server?
Roles in Machine Learning Model Training Data Scientist App Developer Model Deployment ? How big is my training dataset? ? Which algorithm should I use? ? What’s the confusion matrix? ? How often should I retrain the model? Full abstraction of servers Developers can just focus on their code—there are no distractions around server management, capacity planning, or availability. Simplified starting experience Easier experimentation Scale at your pace—don’t bet the farm on Day 1 Flexibility
Use Cases Serverless Machine Learning
Image Classification Images HttpTrigg er in bursts Landscape Dog Camera Prediction Inferencing Application
Solution I - PaaS Python Runtime WSGI HTTP Server Dependencies Docker Image Scoring script Application Gateway App Service Web App Input data HttpTrigger
Solution II - Serverless Scoring script Azure Function Input Data Configuration (trigger, dependencies) HttpTrigger
Learnings 1 Python is the preferred language for inferencing using Functions. 2 FaaS provides a simple and flexible approach to model deployment, empowering Data Scientists to create their own web service. 3 Serverless provides a low-cost solution for event-driven inferencing pipelines.
Topic Modeling Topic Distribution Backup Recovery Data Disk Information Security Credentials Identity Vault Permissions Network Firewall Image Virtual Private Gateway VPN Traffic Output Input Model Re-training Inferencing Solution Documents / Text Input Input IT FAQs and Trouble Shooting Guides
Topic Modeling - Serverless Topic 1 Topi c 2 Topi c 3 Input Data
Learnings Training Serverless is not preferable for compute-intensive training jobs. Short-lived, low-compute executions are still feasible. Integration Triggers and bindings simplify integrating with data sources. Data Scientists can focus on the training algorithm and scoring script, not the SDKs. Performance Downloading model from storage increases startup time, adding latency to the inferencing step.
Data processing - OCR Vehicle Number Plates Blob Trigger Result Raw image Re-shaped image Image Vehicle Number HTTP Webhook
Learnings Beyond FaaS Serverless beyond Functions is relevant to Machine Learning workloads: • Eventing infrastructure • Workflow orchestration • Pay-per-use Cognitive APIs Role of Functions Functions serve as the glue between different services for data processing.
Challenges Serverless Machine Learning
Challenges Limits Serverless Functions have strict limits on execution time and memory used.  Dedicated hosting  Durable Functions Hardware Some inferencing scenarios can require GPU or FPGA enabled hardware for execution.  Functions container on GPU enabled K8s cluster Versioning Data Science teams prefer to version models independent of their applications. This can increase startup time.  Custom container on Azure Functions on Linux Boundaries Applications dealing with sensitive data require more advanced security and networking options.  App Service Environment  Premium Plan
Takeaways Serverless is indeed a good fit for Machine Learning. FaaS programming model and event-driven pipelines can empower Data Scientists to quickly build smart applications. Let’s continue to bring the goodness of Serverless to more scenarios and developer communities.
@tayalasavari Thank you

More Related Content

Serverless Machine Learning

  • 1. Serverless Machine Learning Predicting the future with the bleeding edge
  • 2. About Me Product Manager on Azure Functions. Pythonista working on Serverless for over a year. Passionate about developer experiences, data science and machine learning. Asavari Tayal @tayalasavari
  • 3. Plan for today 1 Machine Learning Introduction, process of ML and roles in ML. Serverless and FaaS How can they help with ML? Use Cases Real world examples of using Serverless ML. Learnings and ChallengesBenefits of Serverless ML, shortcomings and open questions. 2 3 4
  • 4. What is Machine Learning? Practice of examining large amounts of data to find patterns and then generating code to find those same patterns in brand new data. ML can help you build smarter applications.
  • 5. Process of Machine Learning Model Training 1 Candidate Model Chosen Model
  • 6. Roles in Machine Learning Model Training Data Scientist App Developer Model Deployment ? How big is my training dataset? ? Which algorithm should I use? ? What’s the confusion matrix? ? How often should I retrain the model? ? How many servers do I need? ? How can I scale my application? ? Who monitors my app? ? How should I patch my servers? ? What packages should be on my server?
  • 7. Roles in Machine Learning Model Training Data Scientist App Developer Model Deployment ? How big is my training dataset? ? Which algorithm should I use? ? What’s the confusion matrix? ? How often should I retrain the model? Full abstraction of servers Developers can just focus on their code—there are no distractions around server management, capacity planning, or availability. Simplified starting experience Easier experimentation Scale at your pace—don’t bet the farm on Day 1 Flexibility
  • 10. Solution I - PaaS Python Runtime WSGI HTTP Server Dependencies Docker Image Scoring script Application Gateway App Service Web App Input data HttpTrigger
  • 11. Solution II - Serverless Scoring script Azure Function Input Data Configuration (trigger, dependencies) HttpTrigger
  • 12. Learnings 1 Python is the preferred language for inferencing using Functions. 2 FaaS provides a simple and flexible approach to model deployment, empowering Data Scientists to create their own web service. 3 Serverless provides a low-cost solution for event-driven inferencing pipelines.
  • 14. Topic Modeling - Serverless Topic 1 Topi c 2 Topi c 3 Input Data
  • 15. Learnings Training Serverless is not preferable for compute-intensive training jobs. Short-lived, low-compute executions are still feasible. Integration Triggers and bindings simplify integrating with data sources. Data Scientists can focus on the training algorithm and scoring script, not the SDKs. Performance Downloading model from storage increases startup time, adding latency to the inferencing step.
  • 16. Data processing - OCR Vehicle Number Plates Blob Trigger Result Raw image Re-shaped image Image Vehicle Number HTTP Webhook
  • 17. Learnings Beyond FaaS Serverless beyond Functions is relevant to Machine Learning workloads: • Eventing infrastructure • Workflow orchestration • Pay-per-use Cognitive APIs Role of Functions Functions serve as the glue between different services for data processing.
  • 19. Challenges Limits Serverless Functions have strict limits on execution time and memory used.  Dedicated hosting  Durable Functions Hardware Some inferencing scenarios can require GPU or FPGA enabled hardware for execution.  Functions container on GPU enabled K8s cluster Versioning Data Science teams prefer to version models independent of their applications. This can increase startup time.  Custom container on Azure Functions on Linux Boundaries Applications dealing with sensitive data require more advanced security and networking options.  App Service Environment  Premium Plan
  • 20. Takeaways Serverless is indeed a good fit for Machine Learning. FaaS programming model and event-driven pipelines can empower Data Scientists to quickly build smart applications. Let’s continue to bring the goodness of Serverless to more scenarios and developer communities.

Editor's Notes

  1. Animation