Afternoons with Azure - Azure Machine Learning

Editor's Notes

1. Developed by Dutch Programmer, Guido van Rossum, and released in 1991 During which time he was in a big Monty Python phase, so he named it Python
2. General Purpose: can be used for almost anything Interpreted: is not compiled beforehand, but interpreted as it goes Open source: source code is available to the public for free, new packages everyday from around the world Multiple Versions: not fully backwards compatible, some syntax differences. Not a drastically new language, but Python 2 code may not run on Python 3 environment
3. Easy to read/learn: focused on simplicity, syntax tries to mimic natural language, python motto Dynamic typing: don’t need to specify variable types, defined based on the data assigned to it Everything is an object: great for object oriented programming Small/good memory management: doesn’t take up much space or use my memory Open source: tons of custom packages and tons of resources online Parallelism/concurrency: able to perform more than one task at a time Existing Python backends: if being implemented into an existing system, many environments already have Python backends Speed: it’s an interpreted language, so it can be a tad slow, faster implementations (open source) such as PyPy White-space: the fact that python doesn’t use parenthesis as much as indentation can limit customization Dynamic typing errors: variables can assume the wrong type and cause errors when pushed to production Underdeveloped DB access: there are packages for it, but still a lot of room for improvement Switching between languages: Python’s simplicity can make it more difficult to understand other languages Challenger in data science: still catching up with popularity and data science functionalities
4. - Reading data: for when CSVs are too large to be handled effectively in Excel, or in other forms such as audio or pictures - Data cleansing/preparation/munging: handling null values, transforming and assigning logic around features, to make data more usable - Creating algorithms: has many algorithms available through packages, but makes it very easy to create your own - Production solutions: great for models being introduced into existing systems, especially on-prem - Internet of things applications: tons of new “smart” devices these days, and python has become a popular tool for handling their operation - Big data solutions: some may have heard of apache spark, a big data cluster platform for processing huge volumes and streams of data, has the built in ability to run pySpark, their implementation of Python
5. Module: a collection of functions and definitions stored in a python script file Package: a group of modules and/or other packages Packages are then imported into python scripts allowing them the use functions defined in the package
6. Package manager: used for easy installation as well as managing dependencies Pip is the default package manager that is installed with Python 3 Conda is another package manager that is installed with the Anaconda distribution of python Pip installs packages from the Python Package Index, the official and largest repository Conda installs packages included with the Anaconda distribution, with packages focusing on data science Pip can only manage python dependencies Conda can also handle dependencies from other languages such as java or R Virtual environment: sometimes required to separate certain packages based on conflicting dependencies or to minimize memory usage With pip, you need an additional tool to use virtual environments Conda has this capability built in
7. NumPy: defines the numpy array object, develops arithmetic around these arrays, as well as other numerical analysis methods Pandas: most popular package for data science, built on top of numpy, defines the pandas data frame object and operations for manipulating the data within it Matplotlib: the most popular package for visualizations, huge variety of capabilities while being relatively easy to use
8. SciPy: many functions around linear algebra and optimization, which are the cornerstone of almost all machine learning, would be good to use if creating custom algorithms Scikit-Learn: almost any machine learning algorithm you could need can be found in this package, as well as many functions for customizing and evaluating your model or splitting your data in train and test data TensorFlow: a package that is getting very popular these days, developed by Google, implements a deep learning algorithm, which is a specific and complex time of machine learning, it’s popular for use with image and video analytics as well as many other things
9.  Rather than focus heavily on coding, AML lets you build models by dragging and dropping modules, we'll dive through how to build a simple regression model in our workshop
10. Since I introducted you to the select columns module, I wanted to show you what else is available. Not a comprehensive list. AML has mods for importing/exporting, mods for different models, transforms. One of the best ways to learn AML is to read through the modules and see whats available in the tool kit. A few stand out modules I want to bring to your attention
11. As you start working with AML some things may come across as a bit quirky. A huge thing is you have to work 1 step at a time. If you have a table of orders with a name, date, qty and all the fields are string. You will need a module to turn date into a datetime, and another one for qty into an integer, you can't do that in one step like you could with say a SQL select statement.  While ML is a Microsoft tool, it does not use t-sql, the engine runs sqlite. Certain functions will not work and may not throw back any errors. Example is string concatenation with "+". Will throw back a 0
12. While AML is a Microsoft tool, the SQL is not t-sql, it is sqlite. Certain functions will not work and may not throw back any errors. Example is string concatenation with "+". Uses dynamic typing of variables. Implicit type conversion is allowed Left outer join is valid. Right outer and full outer joins are not. You can rename/add column with alter table but not drop column/alter column/add constraint Views are read only
13. Another big module are the execute code scripts. 
14. When working with different script its important to kepe in mind the data types used especially when you are trying to bring in data types from other data sources
15. Now moving into connectivity. Great connectivity with other services in the Azure stack. You can import and export to/from the blob storage, table storage, and azure sql databases. Hadoop works great as well. If you work on-prem, it is currently a preview feature, meaning its not 100 fully supported, meaning its a prerelease/beta.  A couple cool things is the web URL, you can pull data from http websites and run machine learning on that that information
16. Pros Free to try, and there are multiple levels designed to prevent you from over paying/pay for what you need Easy to use and get started with Leverages the cloud, work from anywhere Web services are easy to deploy and manage Easy to retrain  Cons Generally is a little slower than traditional model training Next 2 go hand in hand Microsoft develops the premade modules, so there's a limited selection unless you can code. Closed source – you know generally what modules do, but they are black box Poor model visibility – unable to see training progress for models, 7 days max on model with not progress bar.