Introduction to Advanced Data

Analytics
Stewart Muchuchuti
MSc Comp, BSc Eng, ML Eng, AI Eng.
stewartm@bac.ac.bw
Here is wisdom
• "In God we trust, all others must bring data." - W. Edwards Deming,
statistician and quality control expert

• "Without big data analytics, companies are blind and deaf, wandering out
onto the web like deer on a freeway." - Geoffrey Moore, author and
consultant

• "Data is a precious thing and will last longer than the systems themselves." -
Tim Berners-Lee, inventor of the World Wide Web
What is data Analytics
• Process of analyzing and interpreting large sets of data to identify patterns,
draw conclusions, and inform decision-making
• Involves using a variety of techniques and tools, such as statistical analysis,
data mining, machine learning, and data visualization, to extract insights
from complex data sets.
• Businesses:
• identify trends in consumer behavior
• optimize marketing campaigns
• improve operational efficiency, and reduce costs.
• Healthcare providers: improve patient outcomes, reduce medical errors, and
identify opportunities for preventive care
• Governments: improve public services, monitor social trends, and inform
policy decisions.
Key definitions
What is a Data Analyst?

A professional who uses data analysis techniques to extract insights and

knowledge from large and complex data sets

They must have a basic understanding of statistics, a perfect sense of databases,

the ability to create new views, and the perception to visualize the data.
Data Analyst
1. Collecting and organizing data from various sources.
2. Cleaning and preparing data for analysis.
3. Performing statistical analysis and data mining to identify patterns and
trends.
4. Creating visualizations and reports to communicate insights and findings to
stakeholders.
5. Making data-driven recommendations to improve business processes,
customer experience, or other key performance indicators.
6. Collaborating with other stakeholders to ensure data accuracy and quality.
Key Skills

Mathematics Statistics

Excel Python/R

Databases
Example – Data Analytics at play
• Say the Operations Director of a multinational tyre
company wants to do a detailed analysis of defects
during the tyre production at its various
manufacturing plants across the globe. Every time
there are defects on the tires during the
manufacturing process, the defect is stored with a
predefined defect code.
9
Data sources
• Sources of data for a business can range from customer feedback to sales
figures to product or service demands.
• Few sources of data a business may utilize:
• Social media: LinkedIn, Twitter, and Facebook can provide insight into
the kind of customer traffic your web page receives. These platforms
also provide cost-effective ways to conduct surveys about customer
satisfaction with products or services and customer preferences.
• Online Engagement Reporting: Using tools such as Google Analytics can
provide you with data about how customers interact with your website.
• Transactional Data: This kind of data will include information collected
from sales reports, ledgers, and web payment transactions. With a
customer relationship management system, you will also be able to
collect data about how customers spend their money on your products.
• So now, data has become so big!
How data can improve your business

Improving Marketing Strategies

• If a company knows how customers are reacting to current marketing techniques, it will allow
Improving them to make changes that will fall in line with trends and expectations of their customers.

Identifying pain points

• predetermined processes - data can help identify points of deviation. Small deviations from the
Identifying norm can be the reason behind increased customer complaints, decreased sales, or a decrease
in productivity. By collecting and analyzing data regularly, you will be able to catch a mishap
early enough to prevent irreversible damages.
How data can improve your
business contd..
• Fraud detection
• With access to sales numbers in hand, it will be easy to detect when
and where fraud may be occurring.
• For instance, if you have a purchase invoice for 100 units, but your sales
reports only show that 90 units have been sold, you know that ten units
are missing from inventory and you will know where to look.
• Many companies are silent victims of fraud because they fail to utilize
the data to realize that fraud is even occurring.
• Identifying data breaches
• Tools for data analytics can help a company to develop and implement
data tests that will detect early signs of fraudulent activity.
How data can improve your
business contd..
Customer feedback
Improving Customer Alter business practices, services or
products to better satisfy the
Experience customer
Customized emails – care

Market trends data, customer bases,

prices offered by competitors.
Making decisions If data does not influence decision
making process, it could cost the
company immensely

Hiring process Data helps get the right talent for

driving business success
Big data – a new advantage
• In the healthcare industry, data pioneers are looking at the outcomes of
some pharmaceuticals that are widely prescribed.
• There are other industries that are using the sensors in their products to
gain data that they can use. This can be seen in children’s toys, large-scale
industrial goods, and so many others. The data that they gather show how
the products are used in real life.
• “With this data, companies can make improvements on the products based
on how people are really using them. This will make these products so
much better for the future users. ”
• New growth opportunities – there will be companies who will sit right in the
middle of flowing data of organisations in order to analyse it for them
Big data – a new advantage

Now casting – capability of companies to make real time

decisions that could help them to improve their business
e.g real time measurement of consumer confidence

Big data can unlock serious value – makes information

transparent – open areas of innovation

Big data helps companies put customers into small groups

allowing them to tailor their services and product offerings.
What’s Big Data?
No single definition; here is from Wikipedia:
• Big data is the term for a collection of data sets so large and complex that it
becomes difficult to process using on-hand database management tools or
traditional data processing applications.
• The challenges include capture, curation, storage, search, sharing, transfer,
analysis, and visualization.
• Big data is high-volume, high- velocity and high-variety information assets
that demand cost-effective, innovative forms of information processing for
enhanced insight and decision making.” -- Gartner

33
 What’s Big Data?
Processor or Virtual Storage Disk Storage

1 Bit = Binary Digit 1 Bit = Binary Digit

8 Bits = 1 Byte 8 Bits = 1 Byte
1000 Bytes = 1 Kilobyte
1024 Bytes = 1 Kilobyte
1000 Kilobytes = 1 Megabyte
1024 Kilobytes = 1 Megabyte
1000 Megabytes = 1 Gigabyte
1024 Megabytes = 1 Gigabyte 1000 Gigabytes = 1 Terabyte
1024 Gigabytes = 1 Terabyte 1000 Terabytes = 1 Petabyte
1024 Terabytes = 1 Petabyte 1000 Petabytes = 1 Exabyte
1024 Petabytes = 1 Exabyte 1000 Exabytes = 1 Zettabyte
1024 Exabytes = 1 Zettabyte 1000 Zettabytes = 1 Yottabyte
1024 Zettabytes = 1 Yottabyte 1000 Yottabytes = 1 Brontobyte
1000 Brontobytes = 1 Geopbyte
1024 Yottabytes = 1 Brontobyte
1024 Brontobytes = 1 Geopbyte 34
Big Data: 3V’s

35
Volume (Scale)
• Data Volume
• 44x increase from 2009 to 2020
• From 0.8 zettabytes to 35zb
• Data volume is increasing exponentially

Exponential increase in
collected/generated data

36
 4.6
30 billion RFID billion
tags today
camera
(1.3B in 2005)
phones
world wide

100s of
millions
of GPS
data every day
? TBs of

enabled
devices sold
annually

25+ TBs of
log data 2+
every day billion
people on
the Web
76 million smart meters by end
in 2009… 2011
200M by 2014
 Variety (Complexity)
• Relational Data (Tables/Transaction/Legacy Data)
• Text Data (Web)
• Semi-structured Data (XML)
• Graph Data
• Social Network, Semantic Web (RDF), …

• Streaming Data
• You can only scan the data once

• A single application can be generating/collecting

many types of data

• Big Public Data (online, weather, finance, etc)

To extract knowledge all these types of

data need to linked together 38
 Velocity (Speed)

• Data is being generated fast and need to be processed

fast
• Online Data Analytics
• Late decisions  missed opportunities
• Examples
• E-Promotions: Based on your current location, your purchase history, what
you like  send promotions right now for store next to you

• Healthcare monitoring: sensors monitoring your activities and body 

any abnormal measurements require immediate reaction

39
 Real-time/Fast Data

Mobile devices
(tracking all objects all the time)

Social media and networks Scientific instruments

(all of us are generating data) (collecting all sorts of data)

Sensor technology and networks

(measuring all kinds of data)

• The progress and innovation is no longer hindered by the ability to collect data
• But, by the ability to manage, analyze, summarize, visualize, and discover
knowledge from the collected data in a timely manner and in a scalable fashion

40
Real-Time Analytics/Decision Requirement

Product
Recommendations Learning why Customers
Influence
that are Relevant Behavior Switch to competitors
& Compelling and their offers; in
time to Counter

Friend Invitations
Improving the Customer to join a
Marketing Game or Activity
Effectiveness of a that expands
Promotion while it business
is still in Play
Preventing Fraud
as it is Occurring
& preventing more
proactively
Some Make it 4V’s

42
 Harnessing Big Data

• OLTP: Online Transaction Processing (DBMSs)

• OLAP: Online Analytical Processing (Data Warehousing)
• RTAP: Real-Time Analytics Processing (Big Data Architecture & technology)

43
Big Data Technology

44