Programming for Data

Analytics
Python for Data Spring 2019
Week 1 ~ An
Science Introduction
Objectives for Week 1

• Course orientation, expectation

• Introduction to Python (Open Source) for Data Science
resources and related concepts/ideas
• Technologies and platforms installation and setup
• Jupyter Notebook/Python Installation
• Quick introduction to the platforms
INFS 772
• Credits: 3
• Academic term/year: Spring 2019
• Course meeting time and location: 1/10/2018 – 4/25/2018 Thursdays,
1:00PM - 3:30PM,
• Tunheim Classroom Building (TCB), Room 109
Textbooks for 772: PyData and MLPy
(softcopies can be downloaded on D2L site)
 Additional Readings for last 2 weeks:
Deep Learning with Python

• Chapter 1 and Chapter 3

• Two pdf files
Deep Learning

Summer 2019

for advanced students

(quantitative/analytical
/mathematical)
 Connections among the courses?
1. INFS 772 Programming for Data Analytics
• A foundational piece for your Data Science Pyramid

2. INFS 792 Deep Learning (Summer 2019)

• Necessary to be a data scientist
• The top piece of your Data Science Pyramid
• Relatively quantitative
• Closely related to the latest developments/literature

• INFS 776 Business Intelligence and Visualization (Summer 2019)

• Necessary skills to be a data analyst or data-powered business analyst
• Perfect for less “quantitative” students
 My courses (and major libraries covered):
The Data Science Pyramid
• Key Words:
Keras
Keras
TensorFlow TensorFlow
Deep Learning
Deep Learning
Scikit-learn
Scikit-learn
Machine Learning
Python Predictive Analytics Programming for Data
Data Science for Decision Making Analytics (Python for
(Machine Learning) Data Science)
Let’s open Will be covered in Deep
Learning class
the box of
DS/ML
tools

Comparing GitHub stars and contributors for different open source tools
 In November 2016, scikit-learn became a No. 1 open source
machine learning project for Python, according to
KDNuggets.
scikit-learn is a high level
library designed for
supervised and
unsupervised machine
learning algorithms.

It is built on other Python

libraries, and precisely for
machine learning.
INSTRUCTOR INFORMATION:
• Name: David Zeng, PhD, MBA, and MSCS
• Office: East Hall 317
• Available on Skype if you need to talk to me
• Email address: David.Zeng@dsu.edu
• Office hours: Mon. ~ Wed. 2 ~ 5 pm.
• Use Email/Outlook to schedule appointments
• Expect to use at least 5 hours/week to do the course work to
succeed!
 COURSE DESCRIPTION& GOALS
• This course introduces Python for Data Science with an emphasis on
high-level libraries and APIs.
• Hand-on exercises with Python machine learning package scikit-
learn.
• Other essential libraries: numpy (providing ndarray for mathematical
computations), pandas (providing DataFrame for data analytics),
matplotlib (for visualization), and Keras (for Deep Learning).
• Jupyter Notebook as a highly effective learning tool.
• Run Python codes
• Note-taking
• Attached images, hyperlinks, etc.
Upon completion of this course,
the students should be able to:
• Understand the core ideas of programming - flow
control, input and output, data structures (e.g., arrays,
lists, and DataFrames).
• Understand the core ideas of Classes, Objects, and high-
level APIs
• Gain solid knowledge on some of the most popular data
science modules in Python:
• Numpy
• Pandas
• Matplotlib
• Scikit-learn
• TensorFlow
• Keras
 How is the course/lecture delivered?
• How should you allocate my time/effort, roughly?
• 1/3 or less on the Basic (Python) Programming Concepts/Tools
• Maybe more if you are very very far from STEM/Computers
• 1/3 or more on Python modules for Data Science (numpy, pandas,
matplotlib, etc)
• ≈ 1/3 on scikit-learn
• Less if you have taken INFS 768 with me
• Work with Jupyter Notebook files (.ipynb) extensively
• Edit (not write) codes to enhance understanding
• Nature of the Assignments!
• Do not solve programming issues during lectures
 This is NOT… (What to expect)
• A compute science/programming course
• Not “Programming with Python”
• No debugging or details of syntax of the language
• No computational/memory efficiency issues
• No algorithm efficiency (e.g. loop vs. vectorization) issues
• A Business Analytics/Intelligence for MBA students
• We are not going to just “talk about” Analytics
• We are actually doing it, with Python modules/APIs
• Very technical and analytical (with real data sets for Machine Learning)
• For first year undergraduate students
• Not all technical details would be covered step by step
• You will need time/effort to learn/complete the assignments/project
• You will have to be resourceful
• You are expected to use google or Stackoverflow.com (with the error
message) to solve basic programming problems (e.g. handle exceptions)
Installing Anaconda Python
• We install Continuum’s Anaconda distribution by
downloading the install script from the Continuum
website. https://www.anaconda.com/download/

• The advantage of the Anaconda distribution is that lot

of the essential python packages comes in bundled.
• You do not have to struggle with all the dependencies
synchronization.
• Python version 3.7.0 (latest)
• 64-bit installer
• Mid-lecture break!
 Python!
• A foundational language for Data Scientists
• Python has libraries for data loading, visualization, statistics, natural language processing,
image processing, and more.
• This vast toolbox provides data scientists with a large array of general and special purpose
functionality.
• Will be THE language for INFS 772 and INFS 792 Deep Learning (Python + TensorFlow + Keras)
• Typing python in the command line will invoke the interpreter in immediate mode. We can
directly type in Python expressions and press enter to get the output.

• Get Python (suggested: Anaconda, Python3.6, 64bit, https://www.continuum.io/downloads )

• Or https://www.python.org/downloads/
• You can also download Python3.6 separately, https://www.python.org/downloads/ and then
install jupyter notebook with terminal/command prompt: pip3 install jupyter
• As an existing Python user, you may wish to install Jupyter using Python’s package
manager, pip, instead of Anaconda.
• First, ensure that you have the latest pip; older versions may have trouble with some
dependencies: pip3 install --upgrade pip
I recommend you use formal sources for
tutorial/examples/documentations
• Python3: https://docs.python.org/3/tutorial/
• Jupyter Notebook: http://jupyter-
notebook.readthedocs.io/en/stable/notebook.html
• Scikit-learn: http://scikit-learn.org/stable/
• Pandas: https://pandas.pydata.org/
• Pyplot: https://matplotlib.org/tutorials/introductory/pyplot.html
• Numpy: https://docs.scipy.org/doc/numpy/user/quickstart.html
 Checking Python Install
• In order to check the Python install, we issue the following commands in the
Anaconda Prompt. (cmd mode)
• python --version
• python
• Save the following into a file named hello.py with notepad
name = input("What's your name? ")
print("Nice to meet you " + name + "!")
age = input("Your age? ")
print("So, you are already " + age + " years old, " +
name + "!")

• Use python hello.py in cmd mode or import hello from python prompt to
run the Python script
• Use help() whenever needed
• Use quit() to quit
Python script (.py), IPython, !type, %run
• Use your notepad (or IDLE) to save this
import sys
print('version is', sys.version)
into a file sys-version.py
• This is a Python script
• Editable with a text editor such as Notepad
• Open your IPython mode (>ipython from cmd)
• Use this !type sys-version.py to stream the file’s content
• Run the script with %run sys-version.py
• quit() to quit
• Again, outside of python prompt, you just use python to
run Python scripts
 Testing Jupyter Notebook
• The Jupyter Notebook is an interactive environment for
running code in the browser.
• It is a great tool for exploratory data analysis and is
widely used by data scientists.
• A foundational tool for learning, research, computing,
and data-powered communications.
• The primary tool for INFS 772 ~ Python Programming for
Data Analytics.
• Download the introduction_to_notebook file from D2L
course site under Python Files.
• We are going to test the Jupyter Notebook file.
• Do not open Jupyter Notebook files with Notepad! Make
sure your jupyter notebook kernel is running and then
open your notebook files from the browser.
Work with a Notebook (browser-based)

• Start a Notebook
• Open terminal/command prompt jupyter notebook
• Or you can open it via the menu of Anaconda programs
• Notebook will open at http://127.0.0.1:8888

• Exit by closing the browser, then typing Ctrl+C in the terminal window

• Introduction to Notebook (.ipynb) file

 Week 2 Outline
Intro to • Second half of Get Started with Python
Python and • A Light Introduction to Python II
Jupyter • List, Function, Class/Object, indentation, for loop
Notebook
• Introduction to Jupyter notebook II
• Course schedule
• Your job as a data scientist with the example of VGG-16
• Instructions for Assignment 1
Key skills (my definition) for Data Scientists

• 1. Quantitative,
• 2. Utilizing open-source/high level libraries, APIs, and
pre-trained models,
• 3. Communicating verbally and visually,
• 4. Learning the latest advances.
 VGG-16: a 16-hidden-layer deep neural network
Your job as a data scientist:
1. Understand the problem and your
data (business processes)
2. Understand the nature of the model
you use (how it works)
3. Utilize pre-trained model and APIs
4. Fine-tune the learning process with
a state-of-the-art optimizer
5. Evaluate the model
6. Test the model with your new data
7. Predict! Or deploy the model
8. Communicate the results!
 Instructions on Assignment 1
1. Download the data files from the UCI ML data depository site before you
can use them in your python program.
Make sure they are in your working directory (for me, it
is C:\Users\dzeng\772 Programming for Data Analytics)
And you may want to use
import os
os.getcwd()
to display it for you if you are not sure. This is where your jupyter notebook
file is.
They are “text” files but make sure they have the correct extensions (.data
and .attributes) when you save them. More specifically, change the .names
file into .attributes when you save it in your machine:
https://archive.ics.uci.edu/ml/machine-learning-databases/mushroom/
2. Download and install a separate module call “simple_ml”.
It is on D2L under Course Files, you need to download it into either (for
me)
C:\Users\dzeng\AppData\Local\Programs\Python\Python36\Lib
If you installed Python separately, or
C:\Users\dzeng\AppData\Local\Continuum\Anaconda3\Lib\site-
packages\
If you use Anaconda3
3. The codes in the cells are highly connected. You (almost) have to run
them one by one from the beginning in order to run the cells related to
assignment 1 later in the file.
4. Remove the two exception-handling examples in the file
Outline for Week 3
• Review: basic data structures: list, tuple, dict
• for loop
• Functions (lambda function/operator)
• Classes
• Assignment 1
• Defining and calling functions
• Starting point:
single_instance_list : a list of attribute values of a single instance
attribute_names : a list of attribute names
File I/O
Week Date Topics Reading/Working Document Assignments
1 1/10 Introduction, Python/Jupyter Notebook Setup PyData: Chapter 2
System Testing Introduction to Python and Jupyter Notebook
2 1/17 Data Structures, Functions, and Files PyData: Chapter 3 Assignment 1 out
3 1/24 NumPy Basics PyData: Chapter 4
4 1/31 Getting Started with pandas PyData: Chapter 5 Assignment 2 out
10 minutes to pandas
5 2/7 Input/Output Tools PyData: Chapter 6 Assignment 1 due
6 2/14 Data Cleaning and Preparation PyData: Chapter 7
7 2/21 Data Wrangling: Join, Combine, and Reshape PyData: Chapter 8
8 2/28 Plotting and Visualization PyData: Chapter 9 Assignment 2 due
Assignment 3 out
9 3/7 Spring Break
10 3/14 Introduction to scikit-learn I: A First Application: MLPy: Chapter 1
Classifying Iris Species
11 3/21 Introduction to scikit-learn II: k-Nearest Neighbors MLPy: Chapter 2: p37 - 42
12 3/28 Introduction to scikit-learn III: Linear Models for MLPy: Chapter 2: p58 - 69 Assignment 3 due
Classification Assignment 4 out
Project out
13 4/4 Introduction to scikit-learn IV: Neural Networks MLPy: Chapter 2: p106 - 119
14 4/11 Introduction to keras I Introducing Keras: deep learning with Python
DLPy: Chapter 1
15 4/18 Introduction to keras II Classifying movie reviews: a binary classification Assignment 4 due
example
DLPy: Chapter 3
16 4/25 Review/Future Trends of Machine/Deep Project due
Learning/AI
 Review
List
Lists are very similar to strings, except that each element can be of any type.
The syntax for creating lists in Python is [...]

•The indexing operator consists of square brackets [] rather than parentheses

• indexing starts at 0
• Adding, inserting, modifying, and removing elements from lists (mutate the list)

Dictionaries
Dictionaries are also like lists, except that each element is a key-value pair. The syntax for
dictionaries is {key1 : value1, ...}
Classes
• Classes provide a means of bundling data and functionality together.
• Creating a new class creates a new type of object, allowing new
instances of that type to be made.
• Each class instance can have attributes attached to it for maintaining
its state.
• Class instances can also have methods (defined by its class) for
modifying its state.

• Class objects support two kinds of operations: attribute references

and instantiation.
 attribute references and instantiation
• Attribute references use the standard syntax used for all attribute
references in Python: obj.name.

• then MyClass.i and MyClass.f are valid attribute references, returning

an integer and a function object, respectively.

• Class instantiation uses function notation.

x = MyClass()
• Creates a new instance of the class and assigns this object to the local
variable x.