Python Data Visualization Essentials Guide: Become a Data Visualization expert by building strong proficiency in Pandas, Matplotlib, Seaborn, Plotly, Numpy, and Bokeh

CHAPTER 1

Introduction to Data Visualization

Human visual perception is grounded on a single set of biological and psychological principles, regardless of culture. Cultural differences should be taken into account but be aware that there are common foundations for what we do.

– Alberto Cairo

Data Visualization and data storytelling have taken the world by storm. Visualization skills are one of the hot skills in the market. Like the introduction, the focus of this book is to give an introduction to data visualization using Python as a primary tool of choice. Specifically, coming to the objective of this chapter, the idea is to introduce data visualization and the difference between various aspects of analysis and the importance of data visualization to the world of business analysis, product management, and its correlation. The importance of data visualization is so big that it determines the winners and losers. Some of the leading technology companies are successful due to their ability to derive insights into their customers' data, products, and services. Some of the large companies with a goldmine are data are not as successful as they should be due to the lack of ability to generate actionable insights.

Structure

In this chapter, we will cover the following topics:

What is data visualization

Key elements of data visualization

Importance of data visualization

Objective

This chapter aims at giving a good amount of introduction to what data visualization is about, why data visualization is important, how it evolved over time, and its key elements. This chapter will set the context for approaching data visualization from atechnology solution point of view using Python later in the chapters.

Data visualization is a visual art of storytelling with an intent to share insights with a meaningful purpose. Data visualization leverages graphical elements such as graphs, charts, maps, and other elements to produce a meaningful graphical representation of data and information. It is a powerful way to share insights on trends, patterns, and outliers in a set of data. The users can analyze the patterns and gain an insight into the data shared. It can be said that visualization is an art due to the creative aspects involved. Data visualization is a very powerful concept in today's world and an important skill to imbibe to succeed.

What is data visualization?

Human brains are trained to spot patterns through our experiential learning throughout life. There is a popular adage - A picture is worth a thousand words. Our eyes are attracted to patterns and colors. In combination with the brain's cognitive abilities, we are attracted by visuals that make an impact. It could be an image, scenery, or an image in a movie or a TV advertisement.

Visualization is a process that transforms the representation of real data from something into something meaningful in a visual representation. The key is meaningful rendering in a simple visualization, even for complex data. Similar to the quote, a picture is worth a thousand words, a good visualization tells a story simply and efficiently, like a good painting. It is visual art that can be used as a powerful storytelling tool.

In literature, I can't say that story A is better than poem B; I have to compare stories with stories and poems with poems, despite being all literature. The same applies to data visualization.

- Jorge Camões

Data visualization, in a metaphorical way, is one way to leverage the visual art of storytelling. Visualization is an intent to share insights with a meaningful purpose. Data visualization leverages graphical elements such as graphs, charts, maps, and other elements to produce a meaningful graphical representation of data and information. It is a powerful way to share insights on trends, patterns, and outliers in a set of data. The users can analyze the patterns and gain an insight into the data shared. It can be said that visualization is an art due to the creative aspects involved.

Data visualization is both an art (visuals) and science (the method of rendering the data) combined. Data visualization can be leveraged to display insights of both quantitative and qualitative data under analysis¹.

The scientific part of data visualization is done using software or libraries available for rendering graphical visualization. This book is primarily dedicated to this aspect, focusing on a particular language we've chosen – Python.

The French cave paintings, at Chauvet-Pont-d'Arc Cave in France, albeit an art form, showcase some inspiring elements of visualization. Some of the themes used in data visualization were, in a way, let's say, inspired by some of the visual art forms that followed for years. It progressed further with abbreviations used for tax notifications by governments to simple graphs of a line graph or a bar chart before graduating to mind-boggling real-time interactive visualization.

Brilliant use of data visualization in history

Some of the greatest visualization examples include the following:

Visualization of the cholera deaths by John Snow, known as the father of epidemiology, is a study of disease and patterns to identify measures to solve the issues. He visualized the cholera deaths for a London borough. While visualizing the outbreak in the city, he noted that the number of deaths at a particular street (Broad Street) near a water pump was high. This led to an insight that cholera was caused by germ-contaminated water than particles in the air. This changed the course of medicine and treatments for outbreaks.

The brilliant use of wonderful data visualization by Florence Nightingale to record the causes of mortality during the Crimean war. Her fact-oriented visualizations proved that more soldiers died due to infections than that of actual fatality in the war. Her visualizations proved the power of inference of data. Data visualizations thrive on the power of insights and inference, and Nightingale's visualization brought the idea of a single picture being more powerful than thousand words to the fore. Florence Nightingale also produced other data visualization charts to prove a point to the government, healthcare professionals, and the public that sanitation is key for healthy lives. Florence Nightingale, also known as The Lady with the Lamp, was a pioneering icon in statistics and data visualization.

Data visualization has transformed many organizations to become wildly successful and has helped governments make decisions to improve the lives of the citizens. Some of the data used are sales and profit numbers, market coverage, employee productivity, etc. It could be budget and revenue figures, health indicators for citizens, employment data, and education data to make policy decisions for governments. For humans, one major use case of impactful contribution by data visualization is the efficient usage to expand the average lifespan.

By helping the healthcare professionals to do the right type of diagnosis and analyzing to understand the patterns and outliers

By focusing on statistically important aspects to build procedures, discover and develop medicines, and choose treatment

By giving an insight into the trends, progress, and to make an informed decision for the betterment of business

Data visualization is a powerful way to tell a visual story that can help determine outliers, patterns, trends, and correlations of data available and make meaningful decisions.

Key elements of data visualization

In the 21st century, data visualization has picked a lot of momentum with the advent of increased use of artificial intelligence and data science. The use of data visualization for research and development, education, and commercial usage have expanded exponentially using interactive dashboards, infographics, and other data rendering tools. It is used in every aspect of our daily life. It is a lot easier to generate powerful visual stories, and I see at least 5 to 10 data visualization elements daily.

One of the keys to the success of the popularity is the evolution of data visualization as an art and science discipline. If you take the examples of visualizations by Florence Nightingale and John Snow, both took a considerable amount of manual analysis and attention to details and great application of key data visualization elements to make them ground-breaking. There are many definitions of key elements that vary from designer-to-designer and author-to-author. From a simplicity standpoint, let us see some of the key elements of data visualization.

Elements of data visualization

There are plenty of guides available covering the key elements and themes to be considered for effective data visualization. We shall cover some of the essential elements to focus on and consider while designing data visualization. Let us see the key elements for data visualization in a diagram. We can call this DUSSSS – Data, User, Strategy, Structure, Style, and Story.

Figure 1.1: Elements of data visualization

Figure 1.1 shows the key elements in a simple visualization in the form of a pictorial. We shall cover each of the elements in detail. We shall explore each element in a question-and-answer format. At a high level, the six key elements to focus on are:

Strategy: What is your data visualization strategy?

Structure: How are you planning to structure your story?

Data: What type of data are you planning to use? How many datasets are you planning to use?

Style: A key element on your visualization style, choice of visualization elements such as graphs and charts, choice of colors and other visual elements, use of qualitative and quantitative information to convey a message

User: The key to the success of the data visualization exercise. Who are your users? Why should they be using your product? What is the key takeaway for them?

Story: Most important aspect of the exercise. What are you trying to convey, what would the key insights, messages, actions, inspirations they can take away to implement actions?

Let us delve into a bit more detail on these elements.

Strategy

Having a good strategy for your visualization exercise is very important. This is because the visualization outcome is purely based on the data being used. Like bad content or a theme could derail a movie or an advertisement, bad data can result in poor outcomes in story, elegance, insights, etc. Having a good strategy is important for a visualization exercise. This includes data strategy, design strategy in terms of user persona and visual elements, etc.

Having good data capturing, data extracting, data cleansing, data integration strategy is very important. This strategy is especially important for planning interactive, real-time, update-oriented dashboards and data visualization. There should be a data strategy for the visualization exercise.

Another element to consider is the user experience and design thinking strategy to address the needs and wants of the users. Using a persona-based design of visual elements can help in designing better visualization elements.

As one size fits all does not exist, a designer bias can be avoided by taking the user requirements, and user needs into consideration through the empathy-based user-centric design of elements.

Design elements, style elements, visual themes, templates, messaging, colors, form factors, devices, and gesture-based themes and actions can all be thought in advance.

Having a clear structure, simplicity, better visibility, and consistency in design could be thought through before the design is done.

Story

Data visualization can be used for two purposes when it comes to stories and messages for the users. The best stories (or visualizations) are not used only to share information or create a user reaction. They can influence and inspire people.

Some of the advertisements on TV have very strong visuals that tell, inspire, motivate and connect with the viewers at a deep level. Similarly, a good data visualization output should connect with the users. It should cater to the needs of the user. It should address a specific need or wants they have while viewing the data visualization product.

A simple run-of-the-mill output may not appeal to everyone. This is one reason why the current set of successful data visualization tools gives the end-users many choices to play around with and customize the data visualization outputs to suit their needs.

We need to be very clear in what we want to communicate and how the users and viewers will interpret it.

A key takeaway message – be it an executive summary, call-for-action, insights should be included in the story getting told

Style

Style is very important for a data visualization element. This is similar to branding exercises carried out by various firms. This gives a unique association of the product with the style and a very consistent expectation, and a potential wow factor for the users. The use of essential styling themes will help convey the message in a brilliant and highly influential manner to the users/viewers. There are numerous techniques available for styling. Some of the tips are:

Have an aesthetic element that hooks the users. The style should be impressive and beautiful but simple to connect with the users quickly

Have a simple, easy to understand and decipher structure rather than a complex one. You can choose to have an intricate style depending on the audience.

Have a better, simple, effective, and efficient design element that is used in a modular fashion across the board

Have a style guide for the following:

The basic structure of the visualization

Text styles – for header, section, axes, messaging, legend, etc.

The text used with context will explain the data visualization story in a better manner

Colors

Background colors and images

Colors to be used for data display

Colors for special elements such as maps, density, outliers, highlighting patterns and trends

It is advisable to minimize the variations in different colors, such as dark and lighter shades

It is advisable to use color palettes available in the software or charting libraries

It is better to use standard colors (recommended by most tools and libraries) for ease of use and readability

Structure

Data visualization, as mentioned earlier, is an art and a science combined. Every aesthetic art has a structure to it, and so does a scientific experiment, and it's the outcome. Hence having a structure for data visualization is an important aspect in deciding how the data can be presented by determining the structure of data to be used, structure of the data and insights presentation, the structure of data formatting, structure of how data will be presented. The presentation of the final data step of the data storytelling design journey including data collection, curation, format, visualization, and presentation. It would also include the frequency of data collection.

Having a good data visualization structure also corresponds to keeping the user experience, needs, and wants in mind in delivering a design that matters the most.

Structuring the content to be displayed in the visualization is important – such as what, when, how the users see.

Structuring the timing of data collection and visualization, including the update frequency (if it is one-way visualization; and the data refresh in case of an interactive