MLflow and Azure Machine Learning—The Power Couple for ML Lifecycle Management
•
1 like•1,511 views
The ML Lifecycle management process is quickly becoming the bottleneck for a lot of ML projects. With MLflow’s newest release, and its enhanced integration with Azure Machine Learning, this process is now showing the right promise and capabilities on Azure. In this talk, we intend to take a tour of the integration details and how MLOps is now becoming a strength of the platform. We’ll talk about versioning, maintaining run history, production pipeline automation, deployment to cloud and edge, and CI/CD pipelines with MLOps as the backdrop. Be prepared for an interactive conversation as we intend to seek a lot of feedback on the integration and capabilities being lit up.
Report
Share
MLflow and Azure Machine Learning—The Power Couple for ML Lifecycle Management
- 1. WIFI SSID:Spark+AISummit | Password: UnifiedDataAnalytics
- 2. Nishant Thacker, Microsoft mlFlow and Azure Machine Learning The Power Couple for ML Lifecycle Management #UnifiedDataAnalytics #SparkAISummit
- 3. Milestones Dec ‘18 Azure ML Launched Mar ‘18 Managed Spark by Databricks on Azure Apr ‘19 Managed MLflow on Azure Databricks
- 4. Azure Databricks Fast, easy, and collaborative Apache Spark™-based analytics platform Built with your needs in mind Role-based access controls Effortless autoscaling Live collaboration Enterprise-grade SLAs Best-in-class notebooks Simple job scheduling Seamlessly integrated with the Azure Portfolio Increase productivity Build on a secure, trusted cloud Scale without limits
- 5. Azure Machine Learning service Bring AI to everyone with an end-to-end, scalable, trusted platform Built with your needs in mind Support for open source frameworks Managed compute DevOps for machine learning Simple deployment Tool agnostic Python SDK Automated machine learning Seamlessly integrated with the Azure Portfolio Boost your data science productivity Increase your rate of experimentation Deploy and manage your models anywhere
- 6. Large retail customer: Use case + Persona Customer Ask: We need to build a unified platform to support a large globally diverse team of Data Engineers, Data Scientists and AI Developers for their big data, deep learning projects. These projects will help us predict and reduce churn, increase retention, and grow revenue Data Scientists want to use open source frameworks like PyTorch & TensorFlow with GPU and CPU for training. They are familiar with MLflow for managing ML lifecycle. ML Engineers want to integrate the ML models in to applications via a scalable web service. They do not want to manage the infrastructure. Recommendation (preferred by the customer): Use Azure Databricks for data prep Use Azure ML with MLflow on Azure Databricks for training OR Use Azure ML with MLflow in Notebook VM with remote Azure ML compute for training Use Azure ML for Model management and MLOps Data Engineers prefer to stay in Spark for distributed data processing on PB scale data. They do not want to manage the infra for data preparation
- 7. Experiments Local machine Virtual machine Azure ML Compute Azure Databricks Experiments and Metrics Logging Azure Machine Learning Workspace Experiments and Metrics Tracking Metric s Artifacts Logging API Tracking URI Demo summary – MLflow with Azure ML Experimentation
- 8. Models PyTorch TensorFlow Scikit-Learn ONNX … Model Deployment Azure Machine Learning Workspace Model Management Model s Artifacts Deploy API Demo summary – MLflow with Azure ML Deployment
- 9. How to get started Install • PyPi package: azureml- mlflow Set • Set Azure ML workspace • Set MLflow tracking URI to Azure ML Go • Run your MLflow experiment • Track your results in Azure ML • Deploy trained model to Azure
- 11. DON’T FORGET TO RATE AND REVIEW THE SESSIONS SEARCH SPARK + AI SUMMIT