SNBP INTERNATIONAL SCHOOL & KIDZONE

SENIOR SECONDARY SCHOOL

MORWADI , PIMPRI, PUNE
CBSE AFFILIATION NO. 1130522
Grade 12

Informatics Practices

Unit 1: Data Handling using Pandas and Data Visualization

DataFrame
Sometimes we need to work on multiple columns at a time, i.e., we
need to process the tabular data. For example, the result of a class,
items in a restaurant’s menu, reservation chart of a train, etc.
Pandas store such tabular data using a DataFrame. A DataFrame is a
two-dimensional labelled data structure like a table of MySQL. It
contains rows and columns, and therefore has both a row and
column index. Each column can have a different type of value such
as numeric, string, boolean, etc., as in tables of a database.
Basic Features of DataFrame
1. It has two index – a row index and a column index
2. The row index is known as index and column index is known as
column name.
3. Index can be of number or letters or strings.
4. Columns may be of different types
5. Size can be changed(Mutable)
6. We can change column/row values.(value- mutable)

Creating a DataFrame

<Dataframe object> =pandasobject.DataFrame(<2 dimen

datastructure>,\
[columns=<col sequence>] , [index=<index sequence>])

Creation of an empty DataFrame

An empty DataFrame can be created as follows:
DF1= pd.DataFrame() …. Empty DataFrame

Empty DataFrame
Columns: []
Index: []

We can create DataFrame by various ways:

1] 2D dictionaries
2] 2D ndarray
3] Series object
4] another dataframe

1. Creation of DataFrame object from 2D dictionary:

A] Creating DataFrame having value as lists/ndarray:

# EG:1Creating DataFrame having value as array

import pandas as pd
D1={"Student":["a","b","c"],"Marks":[12,14,15]}
print("Dictionary is:\n",D1)
Df1=pd.DataFrame(D1)
print("DataFrame is:")
print(Df1)

OUTPUT:
Dictionary is:
{'Student': ['a', 'b', 'c'], 'Marks': [12, 14, 15]}
DataFrame is:
Student Marks
0 a 12
1 b 14
2 c 15

# EG:2 Creating DataFrame having value as lists

import pandas as pd
D1={"Student":[1,"b","abc"],"Marks":[12,"AB",15]}
print("Dictionary is:\n",D1)
Df1=pd.DataFrame(D1)
print("DataFrame is:")
print(Df1)

OUTPUT:
Dictionary is:
{'Student': [1, 'b', 'abc'], 'Marks': [12, 'AB', 15]}
DataFrame is:
Student Marks
0 1 12
1 b AB
2 abc 15
# EG:3 Creating DataFrame having value as lists[putting
index]
import pandas as pd
D1={"Student":[1,"b","abc"],"Marks":[12,"AB",15]}
print("Dictionary is:\n",D1)
Df1=pd.DataFrame(D1,index=["A","B","C"])
print("DataFrame is:")
print(Df1)

Note: Here index will be 0,1,2 … and columns will be created

from keys of dictionary .

OUTPUT:
Dictionary is:
{'Student': [1, 'b', 'abc'], 'Marks': [12, 'AB', 15]}
DataFrame is:
Student Marks
A 1 12
B b AB
C abc 15

B] Creating DataFrame having value as dictionary:

import pandas as pd
D1={"Sales":{"Name":"a","Age":10},"Marketing":{"Name":"b
","Age":20}}
print("Dictionary is:\n",D1)
Df1=pd.DataFrame(D1)
print("DataFrame is:")
print(Df1)

OUTPUT:
Dictionary is:
 {'Sales': {'Name': 'a', 'Age': 10}, 'Marketing': {'Name': 'b',
'Age': 20}}
DataFrame is:
Sales Marketing
Name a b
Age 10 20

2. Creation of DataFrame from NumPy ndarrays

Consider the following three NumPy ndarrays. Let us create a simple
DataFrame without any column labels, using a single ndarray:
>>> import numpy as np
>>> array1 = np.array([10,20,30])
>>> array2 = np.array([100,200,300])
>>> array3 = np.array([-10,-20,-30, -40])
>>> dFrame4 = pd.DataFrame(array1)
>>> dFrame4
0
0 10
1 20
2 30
We can create a DataFrame using more than one ndarrays, as shown
in the following example:
>>> dFrame5 = pd.DataFrame([array1, array3, array2], columns=[
'A', 'B', 'C', 'D'])
>>> dFrame5
A B C D
0 10 20 30 NaN
1 -10 -20 -30 -40.0
2 100 200 300 NaN

(C) Creation of DataFrame from List of Dictionaries

We can create DataFrame from a list of Dictionaries, for example:
# Create list of dictionaries
>>> listDict = [{'a':10, 'b':20}, {'a':5, 'b':10, 'c':20}]
>>> dFrameListDict = pd.DataFrame(listDict)
>>> dFrameListDict
A b c
0 10 20 NaN
1 5 10 20.0
Here, the dictionary keys are taken as column labels, and the values
corresponding to each key are taken as rows. There will be as many
rows as the number of dictionaries present in the list.

In the above example there are two dictionaries in the list. So, the
DataFrame consists of two rows. Number of columns in a DataFrame
is equal to the maximum number of keys in any dictionary of the list.
Hence, there are three columns as the second dictionary has three
elements. Also, note that NaN (Not a Number) is inserted if a
corresponding value for a column is missing.

In the output, VDF is now displayed as the middle column instead of

last.

(D) Creation of DataFrame from Series

Consider the following three Series:
seriesA = pd.Series([1,2,3,4,5], index = ['a', 'b', 'c', 'd', 'e'])
seriesB = pd.Series ([1000,2000,-1000,-5000,1000], index = ['a', 'b',
'c', 'd', 'e'])
seriesC = pd.Series([10,20,-10,-50,100],index = ['z', 'y', 'a', 'c', 'e'])
We can create a DataFrame using a single series as shown below:
>>> dFrame6 = pd.DataFrame(seriesA)
>>> dFrame6
0
a1
b2
c3
d4
e5
Here, the DataFrame dFrame6 has as many numbers of rows as the
numbers of elements in the series, but has only one column. To
create a DataFrame using more than one series, we need to pass
multiple series in
the list as shown below:
>>> dFrame7 = pd.DataFrame([seriesA, seriesB])
>>> dFrame7
a b c d e
0 1 2 3 4 5
1 1000 2000 -1000 -5000 1000
Observe that the labels in the series object become the column
names in the DataFrame object and each series becomes a row in
the DataFrame. Now look at the following example:
>>> dFrame8 = pd.DataFrame([seriesA, seriesC])
>>> dFrame8
a b c d e z y
0 1.0 2.0 3.0 4.0 5.0 NaN NaN
1 -10.0 NaN -50.0 NaN 100.0 10.0 20.0
Here, different series do not have the same set of labels. But, the
number of columns in a DataFrame equals to distinct labels in all the
series. So, if a particular series does not have a corresponding value
for a label, NaN is inserted in the DataFrame column

(E) Creation of DataFrame from Dictionary of Series

A dictionary of series can also be used to create a DataFrame. For
example, ResultSheet is a dictionary of series containing marks of 5
students in three subjects. The names of the students are the keys
to the dictionary, and the index values of the series are the subject
names
as shown below:
>>> ResultSheet={ 'Arnab': pd.Series([90, 91, 97],
index=['Maths','Science','Hindi']),
'Ramit': pd.Series([92, 81, 96], index=['Maths','Science','Hindi']),
'Samridhi': pd.Series([89, 91, 88], index=['Maths','Science','Hindi']),
'Riya': pd.Series([81, 71, 67], index=['Maths','Science','Hindi']),
'Mallika': pd.Series([94, 95, 99], index=['Maths','Science','Hindi'])}
>>> ResultDF = pd.DataFrame(ResultSheet)
>>> ResultDF
Arnab Ramit Samridhi Riya Mallika
Maths 90 92 89 81 94
Science 91 81 91 71 95
Hindi 97 96 88 67 99
The following output shows that every column in the
DataFrame is a Series:
>>> type(ResultDF.Arnab)
<class 'pandas.core.series.Series'>

When a DataFrame is created from a Dictionary of Series, the

resulting index or row labels are a union of all series indexes used to
create the DataFrame.

For example:
dictForUnion = { 'Series1' : pd.Series([1,2,3,4,5], index = ['a', 'b', 'c',
'd', 'e']) , 'Series2' : pd.Series([10,20,-10,-50,100], index = ['z', 'y', 'a',
'c', 'e']),
'Series3' : pd.Series([10,20,-10,-50,100], index = ['z', 'y', 'a', 'c', 'e']) }
>>> dFrameUnion = pd.DataFrame(dictForUnion)
>>> dFrameUnion
Series1 Series2 Series3
a 1.0 -10.0 -10.0
b 2.0 NaN NaN
c 3.0 -50.0 -50.0
d 4.0 NaN NaN
e 5.0 100.0 100.0
y NaN 20.0 20.0
z NaN 10.0 10.0

Accessing/Selecting Data from DataFrames Element

Arnab Ramit Samridhi Riya Mallika
Maths 90 92 89 81 94
Science 91 81 91 71 95
Hindi 97 96 88 67 99

1. Selecting /Accessing a Column

<Dataframe obj >[<column name>] ….Using square bracket
Eg: 1. print(ResultDF[['Arnab','Ramit']])
Output:
Arnab Ramit
Maths 90 92
Science 91 81
Hindi 97 96

<DataFrame obj>.<column name>…..using dot notation

Eg: print(ResultDF.Riya)
Output:
Maths 81
Science 71
Hindi 67
Name: Riya, dtype: int64

2. Selecting/Accessing Multiple Column

<Dataframe obj >[[<column name>,<column name>] ]
Eg: print(ResultDF[['Arnab','Ramit']])
Output:
Arnab Ramit
Maths 90 92
Science 91 81
Hindi 97 96
 3. Selecting /Accessing a Subset from a dataframe using
Row/Column names

<dataframe obj>.loc[<startrow>:<endrow>, <startcol>:<endcol>]

1. To access a row:
Dft.loc[<rowlabel>,:]
Eg: print(ResultDF.loc['Maths',:])

output:
Arnab 90
Ramit 92
Samridhi 89
Riya 81
Mallika 94
Name: Maths, dtype: int64

2. To access Multiple rows:

Dft.loc[<startrow>:<endrow>],:]
Eg: print(ResultDF.loc['Maths':'Science':])
Arnab Ramit Samridhi Riya Mallika
Maths 90 92 89 81 94
Science 91 81 91 71 95

3. To access Selective Columns:

Dft.loc[:,<startcol>:<endcol>]
Eg: print(ResultDF.loc[:,'Arnab':'Samridhi'])
Arnab Ramit Samridhi
Maths 90 92 89
Science 91 81 91
Hindi 97 96 88

4. To access range of column from a range of rows:

<dataframe obj>.loc[<startrow>:<endrow>, <startcol>:<endcol>]
 Eg: print(ResultDF.loc['Maths':'Science','Arnab':'Samridhi',])

Output:
Arnab Ramit Samridhi
Maths 90 92 89
Science 91 81 91

Index:

<dataframe obj>.iloc[<startrow>:<endrow>, <startcol>:<endcol>]

The end index is excluded in result.

Selecting/Accessing Individual Value:

1) <dataframe obj>.<column>[row name or row index]
Eg: print(ResultDF.Arnab[1])
o/p : 91

print(ResultDF.Arnab['Maths'])
o/p : 90

2) We can you at or iat attributes with DF.

<datframe obj>.at[<row label>,<col label>]
Eg: print(ResultDF.at[‘Maths’,’Arnab’])
O/p : 90

<datframe obj>.iat[<row index>,<col index>]

Eg: print(ResultDF.iat[2,3])
o/p : 67

Operations on rows and columns in DataFrames

We can perform some basic operations on rows and columns of a
DataFrame like selection, deletion, addition, and renaming.

Adding a New Column to a DataFrame

We can easily add a new column to a DataFrame. Let us consider the
DataFrame ResultDF defined earlier. In order to add a new column
for another student ‘Preeti’, we can write the following statement:

>>> ResultDF['Preeti']=[89,78,76]
>>> ResultDF
Arnab Ramit Samridhi Riya Mallika Preeti
Maths 90 92 89 81 94 89
Science 91 81 91 71 95 78
Hindi 97 96 88 67 99 76
Assigning values to a new column label that does not exist will
create a new column at the end. If the column already exists in the
DataFrame then the assignment statement will update the values of
the already existing column, for example:
>>> ResultDF['Ramit']=[99, 98, 78]
>>> ResultDF

Arnab Ramit Samridhi Riya Mallika Preeti

Maths 90 99 89 81 94 89
Science 91 98 91 71 95 78
Hindi 97 78 88 67 99 76
We can also change data of an entire column to a particular value in
a DataFrame. For example, the following statement sets marks=90
for all subjects for the column name 'Arnab':
>>> ResultDF['Arnab']=90
>>> ResultDF
Arnab Ramit Samridhi Riya Mallika Preeti
Maths 90 99 89 81 94 89
Science 90 98 91 71 95 78
Hindi 90 78 88 67 99 76
Adding a New Row to a DataFrame

We can add a new row to a DataFrame using the DataFrame.loc[ ]

method. Consider the DataFrame ResultDF that has three rows for
the three subjects – Maths, Science and Hindi. Suppose, we need to
add the marks for English subject in ResultDF, we can use the
following statement:
>>> ResultDF
Arnab Ramit Samridhi Riya Mallika Preeti
Maths 90 92 89 81 94 89
Science 91 81 91 71 95 78
Hindi 97 96 88 67 99 76

ResultDF.loc['English'] = [85, 86, 83, 80, 90, 89]

>>> ResultDF

Eflect Arnab Ramit Samridhi Riya Mallika Preeti

Maths 90 92 89 81 94 89
Science 91 81 91 71 95 78
Hindi 97 96 88 67 99 76
English 85 86 83 80 90 89

We cannot use this method to add a row of data with already

existing (duplicate) index value (label). In such case, a row with this
index label will be updated, for example:
>>> ResultDF.loc['English'] = [95, 86, 95, 80, 95,99]
>>> ResultDF

DataFRame.loc[] method can also be used to change the data values

of a row to a particular value. For example, the following statement
sets marks in 'Maths' for all columns to 0:

ResultDF.loc['Maths']=0
>>> ResultDF
If we try to add a row with lesser values than the number of
columns in the DataFrame, it results in a ValueError, with the error
message: ValueError: Cannot set a row with mismatched columns.
Similarly, if we try to add a column with lesser values than the
number of rows in the DataFrame, it results in a ValueError, with the
error message: ValueError: Length of values does not match length
of index.

Further, we can set all values of a DataFrame to a particular value,

for example:
>>> ResultDF[: ] = 0 # Set all values in ResultDF to 0

(C) Deleting Rows or Columns from a DataFrame

We can use the DataFrame.drop() method to delete rows and

columns from a DataFrame. We need to specify the names of the
labels to be dropped and the axis from which they need to be
dropped. To delete a row, the parameter axis is assigned the value 0
and for deleting a column,the parameter axis is assigned the value 1.
Consider the following DataFrame:

ResultDF = ResultDF.drop('Science', axis=0)

>>> ResultDF

ResultDF = ResultDF.drop(['Samridhi','Rami t','Riya'], axis=1)

If the DataFrame has more than one row with the same label, the
DataFrame.drop() method will delete all the matching rows from it.
For example, consider the following DataFrame:

(D) Renaming Row Labels of a DataFrame

We can change the labels of rows and columns in a DataFrame using
the DataFrame.rename() method. Consider the following
DataFrame. To rename the row indices Maths to sub1, Science to
sub2, Hindi to sub3 and English to sub4 we can write the following
statement:

ResultDF=ResultDF.rename({'Maths':'Sub1',
‘Science':'Sub2','English':'Sub3',
'Hindi':'Sub4'}, axis='index')
>>> print(ResultDF)

The parameter axis='index' is used to specify that the row label is to

be changed. If no new label is passed corresponding to an existing
label, the existing row label is left as it is.

(E) Renaming Column Labels of a DataFrame

To alter the column names of ResultDF we can again use the

rename() method, as shown below. The parameter axis='columns'
implies we want to change the column labels:
Think and Reflect
>>>
ResultDF=ResultDF.rename({'Arnab':'Student1','Ramit':'Student2','
Samridhi':'Student3','Mallika':'Student4'},axis='columns')
>>> print(ResultDF)

Selecting Dataframe Rows/columns based on Boolean

Conditions

1] ResultDF['Arnab']>90
# applied to only one column of DF and return the result of
each column
o/p Maths False
Science True
 Hindi True
Name: Arnab, dtype: bool

2] To extract the subset of dF, just write condition inside the

square bracket next to dF obj

<dataframe obj>[condition]
Eg: print(ResultDF[ResultDF['Arnab']>90])
o/p :
Arnab Ramit Samridhi Riya Mallika
Science 91 81 91 71 95
Hindi 97 96 88 67 99
Or

<dataframe obj>.loc[condition]

Eg: print(ResultDF.loc[ResultDF['Arnab']>90])

o/p
Arnab Ramit Samridhi Riya Mallika
Science 91 81 91 71 95
Hindi 97 96 88 67 99

Creating Data Frames with Boolean Index

Import pandas as pd
Days=[“Mon”, “Tue”, “Wed”, “Thur”, “Fri”]
Classes = [6,0,3,0,8]
Dc= {“Days”:Days, ‘No. of Classes’:Classes}
Clasdf= pd.DataFrame(Dc,index=[True,False,True,False,True])

You can also provide Boolean indexing to dataframes as 1s and

0s.
Accessing Rows from dataframe with Boolean Indexes

<df>.loc[True]
<df>.loc[False]
<df>.loc[1]
<df>.loc[0]