Python

Abstract

Python is increasingly being used as a scientific language. Matrix and vector

manipulations are extremely important for scientific computations. Both NumPy and
Pandas have emerged to be essential libraries for any scientific computation, including
machine learning, in python due to their intuitive syntax and high-performance matrix
computation capabilities.

In this post, we will provide an overview of the common functionalities of NumPy and
Pandas. We will realize the similarity of these libraries with existing toolboxes in R and
MATLAB. This similarity and added flexibility have resulted in wide acceptance of
python in the scientific community lately. Topic covered in the blog are:

1. Overview of NumPy
2. Overview of Pandas
3. Using Matplotlib

This post is an excerpt from a live hands-on training conducted by CloudxLab on 25th
Nov 2017. It was attended by more than 100 learners around the globe. The
participants were from countries namely; United States, Canada, Australia, Indonesia,
India, Thailand, Philippines, Malaysia, Macao, Japan, Hong Kong, Singapore, United
Kingdom, Saudi Arabia, Nepal, & New Zealand.

 NumPy

NumPy stands for ‘Numerical Python’ or ‘Numeric Python’. It is an open source

module of Python which provides fast mathematical computation on arrays and
matrices.

NumPy provides the essential multi-dimensional array-oriented computing functionalities

designed for high-level mathematical functions and scientific computation. Numpy can be
imported into the notebook using

1 >>> import numpy as np

NumPy’s main object is the homogeneous multidimensional array. It is a table with same
type elements, i.e, integers or string or characters (homogeneous), usually integers. In
NumPy, dimensions are called axes. The number of axes is called the rank.

There are several ways to create an array in NumPy like np.array, np.zeros, no.ones, etc.
Each of them provides some flexibility.
Command to create
Example
an array

1 >>> a = np.array([1, 2, 3])

2 >>> type(a)

3 <type 'numpy.ndarray'>

np.array 5 >>> b = np.array((3, 4, 5))

6 >>> type(b)

7 <type 'numpy.ndarray'>

1 >>> np.ones( (3,4), dtype=np.int16 )

2 array([[ 1, 1, 1, 1],

3 [ 1, 1, 1, 1],
np.ones
4 [ 1, 1, 1, 1]])

1 >>> np.full( (3,4), 0.11 )

2 array([[ 0.11, 0.11, 0.11, 0.11],

3 [ 0.11, 0.11, 0.11, 0.11],

np.full
4 [ 0.11, 0.11, 0.11, 0.11]])

np.arange 1 >>> np.arange( 10, 30, 5 )

2 array([10, 15, 20, 25])
 3

4 >>> np.arange( 0, 2, 0.3 )

5 # it accepts float arguments

6 array([ 0. , 0.3, 0.6, 0.9, 1.2, 1.5, 1.8])

>>> np.linspace(0, 5/3, 6)

1
2 array([0. , 0.33333333 , 0.66666667 , 1. ,
np.linspace 1.33333333 1.66666667])

>>> np.random.rand(2,3)
1 array([[
2 0.55365951, 0.60150511, 0.36113117],

np.random.rand(2,3) 3 [ 0.5388662
, 0.06929014, 0.07908068]])

>>> np.empty((2,3))
1 array([[
2 0.21288689, 0.20662218, 0.78018623],

np.empty((2,3)) 3 [
0.35294004, 0.07347101, 0.54552084]])

Some of the important attributes of a NumPy object are:

4. Ndim: displays the dimension of the array

5. Shape: returns a tuple of integers indicating the size of the array
6. Size: returns the total number of elements in the NumPy array
7. Dtype: returns the type of elements in the array, i.e., int64, character
8. Itemsize: returns the size in bytes of each item
9. Reshape: Reshapes the NumPy array

examples:

 A[2:5] will print items 2 to 4. Index in NumPy arrays starts from 0

 A[2::2] will print items 2 to end skipping 2 items
 A[::-1] will print the array in the reverse order
 A[1:] will print from row 1 to end

 Pandas

Similar to NumPy, Pandas is one of the most widely used python libraries in data science. It
provides high-performance, easy to use structures and data analysis tools. Unlike NumPy
library which provides objects for multi-dimensional arrays, Pandas provides in-memory 2d
table object called Dataframe. It is like a spreadsheet with column names and row labels.

Hence, with 2d tables, pandas is capable of providing many additional functionalities like
creating pivot tables, computing columns based on other columns and plotting graphs.
Pandas can be imported into Python using:

1 >>> import pandas as pd

Some commonly used data structures in pandas are:

10. Series objects: 1D array, similar to a column in a spreadsheet

11. DataFrame objects: 2D table, similar to a spreadsheet
12. Panel objects: Dictionary of DataFrames, similar to sheet in MS Excel

Pandas Series object is created using pd.Series function. Each row is provided with an index
and by defaults is assigned numerical values starting from 0. Like NumPy, Pandas also
provide the basic mathematical functionalities like addition, subtraction and conditional
operations and broadcasting.

Pandas dataframe object represents a spreadsheet with cell values, column names, and row
index labels. Dataframe can be visualized as dictionaries of Series. Dataframe rows and
columns are simple and intuitive to access. Pandas also provide SQL-like functionality to
filter, sort rows based on conditions. For example,
 >>> people_dict = { "weight": pd.Series([68, 83, 112],index=["alice", "bob", "charles"]), "birthyear":
1
pd.Series([1984, 1985, 1992], index=["bob", "alice", "charles"], name="year"),
2
"children": pd.Series([0, 3], index=["charles", "bob"]),
3
"hobby": pd.Series(["Biking", "Dancing"], index=["alice", "bob"]),}

1 >>> people = pd.DataFrame(people_dict)

2 >>> people

1 >>> people[people["birthyear"] < 1990]

New columns and rows can be easily added to the dataframe. In addition to the basic
functionalities, pandas dataframe can be sorted by a particular column.

Dataframes can also be easily exported and imported from CSV, Excel, JSON, HTML and
SQL database. Some other essential methods that are present in dataframes are:

 head(): returns the top 5 rows in the dataframe object

 tail(): returns the bottom 5 rows in the dataframe
 info(): prints the summary of the dataframe
 describe(): gives a nice overview of the main aggregated values over each column
Matplotlib

Matplotlib is a 2d plotting library which produces publication quality figures in a

variety of hardcopy formats and interactive environments. Matplotlib can be used in
Python scripts, Python and IPython shell, Jupyter Notebook, web application servers
and GUI toolkits.

examples:

Example 1: Plotting a line graph Example 2: Plotting a histogram

>>> import matplotlib.pyplot as plt

1 >>> x =
1 >>> import matplotlib.pyplot as plt 2 [21,22,23,4,5,6,77,8,9,10,31,32,33,34,35,36,37,18,
2 >>> plt.plot([1,2,3,4])
3 49,50,100]
3 >>> plt.ylabel('some numbers')
4 >>> num_bins = 5
4 >>> plt.show()
5 >>> plt.hist(x, num_bins, facecolor='blue')
>>> plt.show()

Submitted by ,.

Shambhavi b
 Usha M

Lakshmi s nandi