What Is Data Analysis

First things first: what IS data analysis?

In short, data analysis involves sorting through massive amounts of

unstructured information and deriving key insights from it. These insights are
enormously valuable for decision-making at companies of all sizes.

A quick note here: data analysis and data science are not the same. Although
they belong to the same family, data science is typically more advanced (a lot
more programming, creating new algorithms, building predictive models, etc.).

Introduction to the data analytics process:

1. Define the question or goal behind the analysis: what are you trying to
discover?

2. Collect the right data to help answer this question.

3. Perform data cleaning/data wrangling to improve data quality and prepare it

for analysis and interpretation–getting data into the right format, getting rid
of unnecessary data, correcting spelling mistakes, etc.

4. Manipulate data using Excel or Google Sheets. This may include plotting the
data out, creating pivot tables, and so on.

5. Analyze and interpret the data using statistical tools (i.e. finding correlations,
trends, outliers, etc.).

6. Present this data in meaningful ways: graphs, visualizations, charts, tables,

etc. Data analysts may report their findings to project managers, department
heads, and senior-level business executives to help them make decisions and
spot patterns and trends.

Is data analytics hard? Well, the great thing about data analysis is that it’s more
of an entry-level role, meaning you can jump right in with basic knowledge after
you take some data analysis courses for beginners and sharpen a few key skills.

1
(Of course, it certainly won’t hurt if you already have experience with coding,
math, or statistics!)

Becoming a data analyst can also open the door to lucrative careers like data
science and data engineering (just to name a few) as you gain more experience
on the job.

☝️ Types of Data Analysis

What is the key objective of data analysis? That depends on what type of data
analysis skills you’re using. Here are five kinds of data analytics.

Descriptive analysis: Descriptive analytics is designed to answer the question

“What happened?” The goal of descriptive analytics is to summarize data in a
meaningful and descriptive manner, not to make any predictions. Examples
include monthly revenue reports and KPI dashboards.

Exploratory analysis: Exploratory analysis dives a bit deeper than descriptive

analytics, skimming for detectable patterns and trends in data. Another way to
think of this is the initial investigation phase.

Diagnostic analysis: Takes the insights found from both descriptive and
exploratory analytics and investigates further to find the causes.

Predictive analysis: This type is often used more by data scientists, rather than
data analysts. It uses data, statistics, and machine learning algorithms and
techniques to figure out the likelihood of future outcomes based on data.
Examples include sales forecasting and risk assessment.

Prescriptive analysis: Takes insights found from all types of data analysis
(descriptive, exploratory, diagnostic, predictive) to determine the best course of
action.

☝️Data Analysis Methods

Next, what are the methods data analysts use to accomplish these various
objectives? Here’s a quick introduction to data analytics methods.

Cluster analysis: Organizes data into groups, or clusters, that share common
characteristics. More on this here:

2
Regression analysis: A set of statistical processes that allows you to examine
the relationship between two or more variables. Learn more about this method
here:

Factor analysis: Condenses several variables into just a few to make data
analysis easier. Learn more:

Data mining: The process of finding trends, patterns, and correlations in large
data sets. Learn more

Text analysis: Extract machine-readable information from unstructured text

(e.g., PDFs, word processing documents, emails).

Data Analysis is a process of collecting, transforming, cleaning, and modeling data

with the goal of discovering the required information. The results so obtained are
communicated, suggesting conclusions, and supporting decision-making. Data
visualization is at times used to portray the data for the ease of discovering the useful
patterns in the data. The terms Data Modeling and Data Analysis mean the same.
Data Analysis Process consists of the following phases that are iterative in
nature −

• Data Requirements Specification

• Data Collection
• Data Processing
• Data Cleaning
• Data Analysis
• Communication

3
Data Requirements Specification
The data required for analysis is based on a question or an experiment. Based on the
requirements of those directing the analysis, the data necessary as inputs to the
analysis is identified (e.g., Population of people). Specific variables regarding a
population (e.g., Age and Income) may be specified and obtained. Data may be
numerical or categorical.

Data Collection
Data Collection is the process of gathering information on targeted variables identified
as data requirements. The emphasis is on ensuring accurate and honest collection
of data. Data Collection ensures that data gathered is accurate such that the related
decisions are valid. Data Collection provides both a baseline to measure and a target
to improve.
Data is collected from various sources ranging from organizational databases to the
information in web pages. The data thus obtained, may not be structured and may
contain irrelevant information. Hence, the collected data is required to be subjected
to Data Processing and Data Cleaning.

Data Processing
The data that is collected must be processed or organized for analysis. This includes
structuring the data as required for the relevant Analysis Tools. For example, the data
might have to be placed into rows and columns in a table within a Spreadsheet or
Statistical Application. A Data Model might have to be created.

Data Cleaning
The processed and organized data may be incomplete, contain duplicates, or contain
errors. Data Cleaning is the process of preventing and correcting these errors. There
are several types of Data Cleaning that depend on the type of data. For example,
while cleaning the financial data, certain totals might be compared against reliable
published numbers or defined thresholds. Likewise, quantitative data methods can
be used for outlier detection that would be subsequently excluded in analysis.

Data Analysis
Data that is processed, organized and cleaned would be ready for the analysis.
Various data analysis techniques are available to understand, interpret, and derive
conclusions based on the requirements. Data Visualization may also be used to
examine the data in graphical format, to obtain additional insight regarding the
messages within the data.
Statistical Data Models such as Correlation, Regression Analysis can be used to
identify the relations among the data variables. These models that are descriptive of
the data are helpful in simplifying analysis and communicate results.

4
The process might require additional Data Cleaning or additional Data Collection, and
hence these activities are iterative in nature.

Communication
The results of the data analysis are to be reported in a format as required by the users
to support their decisions and further action. The feedback from the users might result
in additional analysis.
The data analysts can choose data visualization techniques, such as tables and
charts, which help in communicating the message clearly and efficiently to the users.
The analysis tools provide facility to highlight the required information with color codes
and formatting in tables and charts.

PYTHON Introduction:

Python is a widely used general-purpose, high level programming language.

It was created by Guido van Rossum in 1991 and further developed by the
Python Software Foundation. It was designed with an emphasis on code
readability, and its syntax allows programmers to express their concepts in
fewer lines of code.

It is used for:

• web development (server-side),

• software development,
• mathematics,
• system scripting.

Python Features

Python is a dynamic, high level, free open source and interpreted programming
language. It supports object-oriented programming as well as procedural
oriented programming. In Python, we don’t need to declare the type of variable
because it is a dynamically typed language. For example, x = 10 Here, x can be
anything such as String, int, etc.
Features in Python
There are many features in Python, some of which are discussed below –

5
1. Easy to code: Python is a high-level programming language. Python is very
easy to learn the language as compared to other languages like C, C#, Javascript,
Java, etc. It is very easy to code in python language and anybody can learn python
basics in a few hours or days. It is also a developer-friendly language.
2. Free and Open Source: Python language is freely available at the official
website and you can download it from there Since it is open-source, this means
that source code is also available to the public. So you can download it as, use it
as well as share it.
3. Object-Oriented Language: One of the key features of python is Object-
Oriented programming. Python supports object-oriented language and concepts
of classes, objects encapsulation, etc.
4. GUI Programming Support: Graphical User interfaces can be made using a
module such as PyQt5, PyQt4, wxPython, or Tk in python. PyQt5 is the most
popular option for creating graphical apps with Python.
5. High-Level Language: Python is a high-level language. When we write
programs in python, we do not need to remember the system architecture, nor
do we need to manage the memory.
6. Extensible feature: Python is a Extensible language. We can write us some
Python code into C or C++ language and also we can compile that code in C/C++
language.
7. Python is Portable language: Python language is also a portable language.
For example, if we have python code for windows and if we want to run this code
on other platforms such as Linux, 10 Unix, and Mac then we do not need to change
it, we can run this code on any platform.
8. Python is Integrated language: Python is also an Integrated language
because we can easily integrated python with other languages like c, c++, etc.
9. Interpreted Language: Python is an Interpreted Language because Python
code is executed line by line at a time. like other languages C, C++, Java, etc. there
is no need to compile python code this makes it easier to debug our code. The
source code of python is converted into an immediate form called bytecode.
10. Large Standard L Library Python has a large standard library which
provides a rich set of module and functions so you do not have to write your own
code for every single thing. There are many libraries present in python for such
as regular expressions, unit-testing, web browsers, etc.

11. Dynamically Typed Language: Python is a dynamically-typed language.

That means the type (for example- int, double, long, etc.) for a variable is decided

6
at run time not in advance because of this feature we don’t need to specify the
type of variable.

What is Python Interpreter?

Python executes the code line by line i.e., Python is an interpreter language.
As we know the computer can’t understand our language, it can only
understand the machine language i.e., binary language. So, the Python
interpreter converts the code written in python language by the user to the
language which computer hardware or system can understand. It does all the
time whenever you run your python script.

We write the python code in any text editor and save the same file using the
“.py” extension in our system. Now, how will this code run? There must be
some application or program like “python” or “python3” that must be
installed in your system, and this is their duty to run this python code. This
type of program is called Interpreter.

Python does provide a python shell, which is used to execute a single

command in python and display the result accordingly.

The Python interpreter first reads the command, then evaluates the
command, prints the results, and then again loops it back to read the
command and because of this only Python is known as REPL i.e., (Read,
Evaluate, Print, Loop).

MODES OF PYTHON INTERPRETER:

Python Interpreter is a program that reads and executes Python code. It

uses 2 modes of Execution.

1. Interactive mode
2. Script mode
1. Interactive mode:
❖ Interactive Mode, as the name suggests, allows us to interact with OS.
❖ When we type Python statement, interpreter displays the
result(s) immediately.

7
Advantages:
❖ Python, in interactive mode, is good enough to learn, experiment or
explore.
❖ Working in interactive mode is convenient for beginners and for testing
small pieces of code.

Drawback:
❖ We cannot save the statements and have to retype all the statements once
again to re-run them.

In interactive mode, you type Python programs and the interpreter displays
the result:
>>> 1 + 1
2
The chevron, >>>, is the prompt the interpreter uses to indicate that it is
ready for you to enter code. If you type 1 + 1, the interpreter replies 2.
>>> print ('Hello, World!')
Hello, World!
This is an example of a print statement. It displays a result on the screen. In
this case, the result is the words.

8
2. Script mode:

❖ In script mode, we type python program in a file and then use interpreter
to execute the content of the file.

❖ Scripts can be saved to disk for future use. Python scripts have
the extension .py, meaning that the filename ends with .py

❖ Save the code with filename.py and run the interpreter in script mode to
execute the script.

Integrated Development Learning Environment (IDLE):

❖ Is a graphical user interface which is completely written in Python.
❖ It is bundled with the default implementation of the python language and
also comes with optional part of the Python packaging.
Features of IDLE:
Multi-window text editor with syntax highlighting.
❖ Auto completion with smart indentation.

9
Python shell to display output with syntax highlighting.

VALUES AND DATA TYPES

Values: Data can be stored in a program in a variety of ways. The most basic unit
of data is a value.
A value is a specific piece of data, such as a word or a number. Some examples
are 5, 5.2, and "Hello, World!".
Each value belongs to a category called a data type.

Python Data Types

Data types are the classification or categorization of data items.

Python

supports the following built-in data types.

• You can get the data type of any object by using

the type() function:

• Every value in Python has a datatype. Since everything is an object

in Python programming, data types are actually classes and
variables are instance (object) of these classes.

Example

Print the data type of the variable x:

x = 5
print(type(x))

Scalar Types

• int: Positive or negative whole numbers (without a fractional part)

e.g. -10, 10, 456, 4654654.
• float: Any real number with a floating-point representation in
which a fractional component is denoted by a decimal symbol or
scientific notation e.g. 1.23, 3.4556789e2.
• complex: A number with a real and imaginary component
represented as x + 2y.
• bool: Data with one of two built-in values True or False. Notice
that 'T' and 'F' are capital. true and false are not valid booleans
and Python will throw an error for them.
10
 • None: The None represents the null object in Python. A None is
returned by functions that don't explicitly return a value.

Sequence Type

A sequence is an ordered collection of similar or different data

types. Python has the following built-in sequence data types:

• String: A string value is a collection of one or more characters

put in single, double or triple quotes.
• x = "Hello World"
• List: A list object is an ordered collection of one or more data
items, not necessarily of the same type, put in square brackets.
• Lists are used to store multiple items in a single variable.

list Items

List items are ordered, changeable, and allow duplicate values.

List items are indexed, the first item has index [0], the second item has
index [1] etc.

Example

Create a List:

thislist = ["apple", "banana", "cherry"]

print(thislist)

Ordered

When we say that lists are ordered, it means that the items have a
defined order, and that order will not change.

If you add new items to a list, the new items will be placed at the end
of the list.

Changeable

The list is changeable, meaning that we can change, add, and remove
items in a list after it has been created.

Allow Duplicates

11
 Since lists are indexed, lists can have items with the same value:

Example

Lists allow duplicate values:

thislist = ["apple", "banana", "cherry", "apple", "cherry"]

print(thislist)
output:

• List items can be of any data type:

Example

String, int and boolean data types:

list1 = ["apple", "banana", "cherry"]

list2 = [1, 5, 7, 9, 3]
list3 = [True, False, False]

• A list can contain different data types:

Example

A list with strings, integers and boolean values:

list1 = ["abc", 34, True, 40, "male"]

Tuple: A Tuple object is an ordered collection of one or more data

items, not necessarily of the same type, put in parentheses.

• A tuple is a collection which is ordered and unchangeable.

• Tuples are written with round brackets.

• Create a Tuple:
• thistuple = ("apple", "banana", "cherry")
print(thistuple)

• Tuple items are ordered, unchangeable, and allow duplicate values.

• Tuple items are indexed, the first item has index [0], the second item
has index [1] etc.

Mapping Type
12
Dictionary: A dictionary Dict() object is an unordered collection
of data(python version3.6 and earlier) in a key:value pair form.
A collection of such pairs is enclosed in curly brackets.

For example: {1:"Steve", 2:"Bill", 3:"Ram", 4: "Farha"}

• Dictionaries are used to store data values in key:value pairs.

• A dictionary is a collection which is ordered*, changeable and does

not allow duplicates.

• As of Python version 3.7, dictionaries are ordered. In Python 3.6

and earlier, dictionaries are unordered.

• Dictionaries are written with curly brackets, and have keys and
values:

• Create and print a dictionary:

• thisdict = {
"brand": "Ford",
"model": "Mustang",
"year": 1964
}
print(thisdict)

Set Types

• set: Set is mutable, unordered collection of distinct hashable objects.

The set is a
• Python implementation of the set in Mathematics. A set object has
suitable methods
• to perform mathematical set operations like union, intersection,
difference, etc.

• A set is a collection which is both unordered and unindexed.

• Sets are written with curly brackets.

• Create a Set:
• thisset = {"apple", "banana", "cherry"}
print(thisset)
• Set items are unordered, unchangeable, and do not allow duplicate
values.
• frozenset: Frozenset is immutable version of set whose elements are
added from other iterables.

13
 Mutable and Immutable Types
Data objects of the above types are stored in a computer's memory
for processing. Some of these values can be modified during
processing,
but contents of others can't be altered once they are created in the
memory.
Numbers, strings, and Tuples are immutable, which means
their contents can't be altered after creation.

On the other hand, items in a List or Dictionary object can

be modified. It is possible to add, delete, insert, and rearrange

items in
a list or dictionary. Hence, they are mutable objects.

Python Variables
Python is not “statically typed”. We do not need to declare variables before
using them or declare their type. A variable is created the moment we first
assign a value to it. A variable is a name given to a memory location. It is the
basic unit of storage in a program.
• The value stored in a variable can be changed during program
execution.
• A variable is only a name given to a memory location, all the
operations done on the variable effects that memory location.
Rules for creating variables in Python:
• A variable name must start with a letter or the underscore
character.
• A variable name cannot start with a number.
• A variable name can only contain alpha-numeric characters and
underscores (A-z, 0-9, and _ ).
• Variable names are case-sensitive (name, Name and NAME are
three different variables).
• The reserved words(keywords) cannot be used naming the
variable.
Example:

# An integer assignment
age = 45

# A floating point
salary = 1456.8

# A string
name = "John"

print(age)
14
 print(salary)
print(name)

Assigning a single value to multiple variables:

Also, Python allows assigning a single value to several variables
simultaneously with “=” operators.
For example: a = b = c = 10
print(a)
print(b)
print(c)
Assigning different values to multiple variables:
Python allows adding different values in a single line with “,”operators.
a, b, c = 1, 20.2, "Hello World"

print(a)
print(b)
print(c)

Python Identifiers
A Python identifier is a name used to identify a variable, function, class, module or
other object. An identifier starts with a letter A to Z or a to z or an underscore (_)
followed by zero or more letters, underscores and digits (0 to 9).
Python does not allow punctuation characters such as @, $, and % within identifiers.
Python is a case sensitive programming language.
Thus, Manpower and manpower are two different identifiers in Python.
Here are naming conventions for Python identifiers −

• Class names start with an uppercase letter. All other identifiers start with a
lowercase letter.
• Starting an identifier with a single leading underscore indicates that the identifier
is private.
• Starting an identifier with two leading underscores indicates a strongly private
identifier.
• If the identifier also ends with two trailing underscores, the identifier is a
language-defined special name.

15
Python Statement

Instructions that a Python interpreter can execute are called

statements. For example, a = 1 is an assignment
statement. if statement, for statement, while statement, etc. are other

kinds of statements which will be discussed later.

Multi-line statement

In Python, the end of a statement is marked by a newline character. But we

can make a statement extend over multiple lines with the line continuation
character (\). For example:

a = 1 + 2 + 3 + \
4 + 5 + 6 + \
7 + 8 + 9

This is an explicit line continuation. In Python, line continuation is implied

inside parentheses ( ), brackets [ ], and braces { }. For instance, we can
implement the above multi-line statement as:

a = (1 + 2 + 3 +
4 + 5 + 6 +
7 + 8 + 9)

Here, the surrounding parentheses ( ) do the line continuation implicitly.

Same is the case with [ ] and { }. For example:

colors = ['red',
'blue',
'green']

We can also put multiple statements in a single line using semicolons, as

follows:

a = 1; b = 2; c = 3

16