Spatial Data Science

Esri Press, 380 New York Street, Redlands, California 92373-8100

Copyright © 2024 Esri

All rights reserved.

Version 2. Updated 10/3/24.

e-ISBN: 9781589486119

The Library of Congress has cataloged the print edition as follows: 2024941723

The information contained in this document is the exclusive property of Esri or its licensors. This work is protected under United States copyright law and other international copyright treaties and conventions. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, except as expressly permitted in writing by Esri. All requests should be sent to Attention: Director, Contracts and Legal Department, Esri, 380 New York Street, Redlands, California 92373-8100, USA.

The information contained in this document is subject to change without notice.

US Government Restricted/Limited Rights: Any software, documentation, and/or data delivered hereunder is subject to the terms of the License Agreement. The commercial license rights in the License Agreement strictly govern Licensee’s use, reproduction, or disclosure of the software, data, and documentation. In no event shall the US Government acquire greater than RESTRICTED/LIMITED RIGHTS. At a minimum, use, duplication, or disclosure by the US Government is subject to restrictions as set forth in FAR §52.227-14 Alternates I, II, and III (DEC 2007); FAR §52.227-19(b) (DEC 2007) and/or FAR §12.211/12.212 (Commercial Technical Data/Computer Software); and DFARS §252.227-7015 (DEC 2011) (Technical Data–Commercial Items) and/or DFARS §227.7202 (Commercial Computer Software and Commercial Computer Software Documentation), as applicable. Contractor/Manufacturer is Esri, 380 New York Street, Redlands, California 92373-8100, USA.

Esri products or services referenced in this publication are trademarks, service marks, or registered marks of Esri in the United States, the European Community, or certain other jurisdictions. To learn more about Esri marks, go to: links.esri.com/EsriProductNamingGuide. Other companies and products or services mentioned herein may be trademarks, service marks, or registered marks of their respective mark owners.

For purchasing and distribution options (both domestic and international), please visit esripress.esri.com.

173515

Contents

Illustrations vii

Tables viii

Preface ix

Chapter 1: Introduction 1

Chapter 2: The emergence of the spatial sciences as a new and integrative field 7

2.1 The formation and elaboration of the spatial sciences 7

2.2 The fusion of theory, practice, and technology 15

2.3 The rapid growth and elaboration of the spatial sciences community 29

2.4 Syncing science and the technology platforms 33

Chapter 3: Cloud computing, big data, data science, and open science in the digital era 39

3.1 Cloud computing 40

3.2 Big data 42

3.3 Data science 44

3.4 Open science 46

3.5 Implications for the spatial sciences 50

Chapter 4: Geospatial big data 53

4.1 Location-based devices and services 54

4.2 Volunteered and ambient geographic information 56

4.3 Remote sensing 61

4.3.1 In situ sensing 61

4.3.2 Traditional satellite and high-altitude airborne remote sensing systems 62

4.3.3 Nanosatellites 69

4.3.4 Street-level imagery 70

4.3.5 Unoccupied aerial systems 71

4.4 Sensor networks and the Internet of Things 73

4.5 3D modeling, video, and augmented and virtual reality systems 76

4.5.1 Digital twins and building information models 78

4.5.2 3D city models 80

4.5.3 Spatial video 82

4.5.4 Virtual and augmented reality 85

4.6 Syncing science and geospatial big data 88

Chapter 5: The Esri geospatial cloud 91

5.1 Geospatial knowledge discovery 91

5.2 ArcGIS as an open platform 95

5.3 ArcGIS and interoperability 97

5.4 Scenarios and patterns 99

5.4.1 Enabling location intelligence across an organization 100

5.4.2 Operationalizing spatial data science 101

5.4.2.1 Working with real-time data 101

5.4.2.2 Working with big vector and tabular data 103

5.4.2.3 Working with big image and raster data 106

5.4.2.4 Working with artificial intelligence, machine learning, and deep learning methods 113

5.4.3 Supporting national data standards using industry and open standards 127

5.5 Enabling 3D across ArcGIS 129

5.6 Moving ArcGIS indoors 131

5.7 Using knowledge graphs in ArcGIS 132

5.8 Deploying ArcGIS on Kubernetes 134

5.9 Syncing science and the enabling technologies 134

Chapter 6: Conclusions and future prospects 139

6.1 Adopting and using the latest advances in computing 141

6.2 Adopting and using new data streams 141

6.3 Adopting and using the latest spatial methods 143

6.4 The role of geodesign and spatial methods in analysis, modeling, prediction, and decision-making 150

Abbreviations 153

References 161

Index 199

Illustrations

Figure 2.1 The exposome concept 12

Figure 2.2 The focus on place, space, time and spatiotemporal information and the complementary roles of theory, practice, and technology in the spatial sciences 16

Figure 2.3 ArcGIS as a system of insight (analytics), engagement (apps and maps), and record (transactions) 17

Figure 2.4 ArcGIS foundation products 17

Figure 2.5 Five-star guide to encourage more researchers and GIS practitioners to share their data and code 38

Figure 3.1 Promoting openness at different stages of the research process 47

Figure 4.1 Global mean sea level observed with satellite altimetry, GRACE, and Argo floats between 2005 and 2016 64

Figure 4.2 Operational drought monitoring supported by GRACE 65

Figure 5.1 A conceptual three-pillar (advanced computing, artificial intelligence, and geospatial big data) view of GeoAI 92

Figure 5.2 The four levels that make up the ArcGIS stack 98

Figure 5.3 One possible deployment showing how GeoEvent and GeoAnalytics Servers and a spatiotemporal big data store can be deployed using ArcGIS Enterprise and connected to numerous apps 102

Figure 5.4 The ideal deployment of raster analysis requires the integration of three server sites to perform the primary roles of the hosting server, raster analysis server, and image hosting server 112

Figure 5.5 The relationship between artificial intelligence, machine learning, and deep learning 117

Figure 5.6 A geoprocessing model used by a conservation organization to identify potential habitats for a native bird species based on vegetation type, distance from major roads, climate, slope, and elevation 127

Figure 6.1 The reproducible, semantic data processing framework used by Li et al. (2022) to evaluate the performance of semantic data repositories 145

Figure 6.2 Approaches for choosing appropriate block sizes to minimize spatial extrapolation problems 147

Figure 6.3 Overview of the study area and results of the evaluation of validation strategies 149

Tables

For this ebook edition, some tables have been converted to lists. The table numbering remains unchanged.

Table 2.1 Special-interest groups represented at the 2023 Esri User Conference 8

Table 2.2 The 19 research initiatives sponsored by the NCGIA, from 1988 to 1997 9

Table 2.3 Geospatial domains and applications featured in ArcUser, from 2021 to 2023 13

Table 2.4 ArcGIS Enterprise extensions, including specialized servers, data and workflow extensions, and industry-specific data management solutions 19

Table 2.5 ArcGIS Pro extensions, data and workflows, and industry-specific data management solutions 21

Table 2.6 ArcGIS Online extensions and premium feature data stores 23

Table 2.7 ArcGIS apps 24

Table 2.8 ArcGIS-focused products 25

Table 2.9 ArcGIS location data 26

Table 2.10 ArcGIS developer tools 28

Table 2.11 Twenty leading spatial professional organizations by domain 30

Table 2.12 Forty leading spatial academic journals by domain and subdomain 32

Table 4.1 Application scenarios and guiding concepts reviewed by Rieke et al. 2018 77

Table 4.2 3D city model use cases and example applications (adapted from list in Biljecki

et al. 2015) 81

Table 5.1 Common patterns enabling interoperability across the ArcGIS stack 99

Table 5.2 The most popular data types and formats used (January 2024) 102

Table 5.3 GeoAnalytics desktop and GeoAnalytics Server toolsets (January 2024) 104

Table 5.4 The tools available in each of Esri’s GeoAnalytics products (January 2024) 105

Table 5.5 Categories of capabilities, functions, and tools included in ArcGIS Pro Image Analyst extension (January 2024) 107

Table 5.6 Image Analyst geoprocessing toolsets (January 2024) 110

Table 5.7 Descriptions of Esri’s pretrained publicly available DL models (January 2024) 122

Preface

This book traces the growth of the spatial sciences thus far and how spatial professionals will need to evolve in future years to advance spatiotemporal understanding, analysis, modeling, prediction, and decision-making. The recipe is straightforward: spatial professionals will need to add some data science workflows and methods to their toolboxes. Consequently, I set out to write a book that engages with both spatial scientists and data scientists because their methods and ways of thinking about the world complement one another.

Spatial Data Science consists of six chapters. The first chapter introduces the book and describes its two complementary goals. The first goal is to show spatial scientists and practitioners how they can enhance their current use of geographic information systems (GIS), Global Positioning System (GPS), and remote sensing by adding and using geospatial big data and data science methods to extend their existing toolboxes. The second goal is to show data scientists how they can better realize the potential of big data and data science by using spatial perspectives and geospatial technologies to discover new knowledge.

The second chapter traces the emergence of the spatial sciences as a new and integrative field over the past 50 years. This chapter describes the two core threads: (1) the representation, measurement, and manipulation of geospatial information, and (2) the significance and meaning of place for the functioning of the earth and human well-being. The ArcGIS® sections show how GIS has moved from stand-alone software platforms to open and distributed systems during the past several years. The rapid growth and elaboration of the spatial science community is also described through the lens of professional organizations and academic journals that cover theory, practice, and technology. The chapter closes by noting what is missing and what needs to be added to advance the work of spatial scientists and practitioners in the future.

The third chapter looks beyond the spatial sciences that I grew up with to the digital era and the rise of cloud computing, big data, data science, and open science during the past few decades. These fields represent important innovations and help set the stage for how the geospatial community will need to evolve in the next few years to sustain its success.

The fourth chapter describes the rise of geospatial big data and the various types of digital data that spatial scientists and practitioners can use to inform and guide their work. These types of data include location-based devices and services; volunteered and ambient geographic information; remote sensing; sensor networks and the Internet of Things (IoT); and 3D modeling, video, and virtual and augmented reality systems. This chapter describes how we all live and work in an era that is awash with geospatial data.

The fifth chapter describes the Esri® geospatial cloud. The chapter starts with comments about geospatial knowledge discovery and moves quickly to describe how this platform can support a variety of new work scenarios and patterns. The latter include enabling location intelligence across an organization, operationalizing spatial data science, and supporting national standards using industry and open standards. The various types of geospatial big data are woven throughout the descriptions of the four main spatial data science patterns—real-time data, big vector and tabular data, big image and raster data, and artificial intelligence (AI) and machine learning (ML) methods. The chapter closes with a brief discussion of what we need to do to sync our science with the rapidly evolving enabling technologies.

The sixth and final chapter makes the case for the adoption and use of the latest advances in computing, new data sources and methods, and, most important, the deployment of these new spatial methods and data to improve understanding, analysis, modeling, prediction, and decision-making.

Writing this book was a labor of love. It afforded me the opportunity to think of the arc of my academic career, from the first faculty position I took in 1984 to where I find myself today. This career has witnessed numerous milestones—the launch of the Geographic Information and Analysis Center at Montana State University in 1986, the launch of the journal Transactions in GIS with Wiley-Blackwell in 1996, the launch of the GIS Research Laboratory at the University of Southern California (USC) in 1998, and the launch of the Spatial Sciences Institute at USC in June 2010. These milestones tell the story of my evolution as a spatial scientist and how I see the world today, in which the best spatial sciences graduates possess an advanced knowledge of GIS, GPS, remote sensing, experience in the interpretation and processing of satellite images and other digital data streams, a broad understanding of computer applications and database management, and, most important, the spatial principles and methods used to characterize the role of location in the functioning of the earth and everything that people do on it. My USC colleagues and I have built a series of multidisciplinary academic programs at the bachelor’s, master’s, and doctoral levels that epitomize this vision and draw hundreds of students to our programs and classes each year.

Given this state of play, I would be remiss if I were not to acknowledge and thank all the scholars, staff, and students who have shared their knowledge and showed me the path forward during the past four decades. There are too many to call out individually, but my hope is that I listened carefully and that I have used their best ideas to build a series of academic units that can support a myriad of spatially inspired academic programs and cutting-edge research projects that produce knowledge and provide experiential learning opportunities for students. Much of what I have learned in my career is encapsulated in this book.

I hope that all of you will find something of value as you read this book and that you will remember that any shortcomings, blunders, and errors you find were completely of my own making. I will judge this book a success if it motivates individuals in the spatial and data sciences to search for common ground and opportunities to collaborate with one another in the future.

Chapter 1

Introduction

This book traces the rise of the spatial sciences as an integrative discipline and provides a road map for spatial scientists and practitioners to use the opportunities afforded by big data and elevate the importance of geographic knowledge in today’s world. Numerous authors and commentators have argued that geospatial information is the key to understanding our changing planet—the oceans, the atmosphere, and the land; the biosphere, including plants, animals, and ecology; and climate change, sustainable economic development, and the interactions connecting food, energy, and water (Shekhar et al. 2016; Wright and Harder 2019, 2021). The beautifully illustrated GIS for science monograph edited by Dawn J. Wright and Christian Harder (2019), for example, uses a series of case studies to illustrate how the earth works, how the earth looks, and how we look at the earth. The case studies cover global ecosystem mapping, landslides and other natural hazards, the anatomy of supervolcanoes, the modeling of global seagrass habitats, extreme heat events, homelessness, restoring coastal marine habitats, modeling bird responses to climate change, mapping ancient landscapes, and identifying the best places for ecological restoration. Wright and Harder (2019) conclude this monograph with brief discussions of the training of future generations of scientists and the various ways in which modern science works hand in hand with technology.

The case studies used by Wright and Harder (2019), coupled with the concepts and approaches they employed, are inspiring and help explain why geographic information science (GIScience) academic programs and courses have grown rapidly in colleges and universities across the globe during the past 30 years (Wikle and Finchum 2003; Wikle and Fagin 2014; Wikle 2018; Wikle and Sinton 2020). These programs and courses use multiple pedagogies (Mathews and Wikle 2019) and, more often than not, serve two distinctly different groups of students—geography or geographic information science majors on one hand and students from other disciplines seeking spatial literacy and training in GIS skills on the other.

The geographic information science story is a useful precursor because this book argues that spatial data science should also seek to engage two communities simultaneously—those using or studying the spatial sciences and the business professionals, computer scientists, engineers, planners, policymakers, statisticians, and information and library scientists leading the development and deployment of data science writ large.

This book therefore has two complementary goals. The first is to show spatial scientists and practitioners how they can enhance their current work using geographic information systems (GIS), Global Positioning System (GPS), and remote sensing by adding and using geospatial big data and the new analytic methods from data science to extend their current methods and toolboxes. The second is to show data scientists how they can better realize the potential of big data and data science by using spatial perspectives and the existing geospatial technologies to transform data into knowledge and actionable information.

The two aforementioned goals are equally important, and the second in particular will be a big lift because few of the university courses and GIS books published thus far focus on the built environment and the role of the business and engineering communities in supporting everyday life in the 21st century. GIS plays a prominent role in regional and urban planning—for example, by helping with site selection, land use analysis, watershed management, and urban development, and the contributions of GIS to the design, construction, upgrading, and repair of many forms of infrastructure have grown in the past decade. We also anticipate substantial growth in business applications in the coming years, considering the ways in which GIS can help with asset management across many business sectors, including the energy and transportation sectors, which have made large investments in transmission networks and vehicle fleets. GIS can also help manage business information so that firms can keep track of the locations of customers and use this information to site business nodes, build marketing campaigns, manage and optimize sales territories and supply chains, and maximize return on investment. The need to deploy geospatial information has grown with globalization and the sourcing of goods and services from faraway locations. The insurance industry also makes extensive use of GIS to predict and help manage risk, as evidenced by the threats accompanying climate change and the COVID-19 pandemic. The paucity of university programs and courses and, for that matter, books and case studies depicting these types of applications speaks to a series of missed opportunities that the rise of spatial data science may help us address during the next few decades.

The value proposition for carving out a niche for spatial data science within the rapidly evolving field of data science stems from the treatment of location, distance, and spatial relationships as core aspects of the methods and software used to acquire, store, analyze, and visualize such data. Luc Anselin, founding director of the Center for Spatial Data Science at the University of Chicago, views the emergence of spatial data science as a natural evolution of geographic information scientists’ work, considering that spatial data science relates to data science as spatial statistics does to statistics, spatial databases do to databases, and geocomputation does to computation (Anselin 2019a). Michael Goodchild (2024) takes a similar stance in his foreword to the Handbook of Geospatial Artificial Intelligence, edited by Song Gao, Yingjie Hu, and Wenwen Li (Gao et al. 2024a). This view also shows how spatial data science and geographic information science complement one another, because the former endeavors to extract meaningful information from large and varied sources of data whereas the latter transforms spatiotemporal data into actionable information.

This book therefore explores how to use the Esri® GIS software ecosystem to support these new spatial data science methods to help extract additional insights from the large and varied sources and streams of geospatial data characterizing the digital era. The benefits are twofold: the first is to introduce key concepts of spatial data science and the second is to provide some guidance on how to implement spatial data science approaches using Esri’s geospatial cloud.

The remainder of this book consists of the five chapters that are briefly introduced below.

The second chapter traces the emergence of the spatial sciences as a new and integrative field. This chapter starts with some preliminaries and then paints a brief history of the formation and elaboration of the spatial sciences over the past five decades. The latter theme focuses on two threads: (1) the representation, measurement, and manipulation of geospatial information and (2) the significance and meaning of place for the functioning of the earth and human well-being. The chapter then moves on to discuss some of the ways in which the spatial sciences combine theory, practice, and technology before briefly describing the ArcGIS® ecosystem today and the use patterns from a variety of perspectives. The focus then shifts again to offer brief descriptions of some of the other technology options, and this, coupled with the description of ArcGIS, shows how GIS in all its guises has moved from stand-alone software to open and distributed systems during the past decade. The next part of the chapter describes the rapid growth and elaboration of the spatial sciences community through the lens of professional organizations and academic journals that cover theory, practice, and technology from a variety of starting points, including geography, cartography, geographic information science, remote sensing, and computer science. The chapter concludes by noting what is missing, what needs to be improved, and what needs to be added to the current constellation of technology platforms to support knowledge discovery and improve the replicability and reproducibility when working with GIS software and algorithms.

The third chapter looks beyond the spatial sciences to the digital era and the rise of cloud computing, big data, data science, and open science during the past 20 years. This chapter also discusses the kinds of education programs, advanced computing platforms, and digital infrastructures that commentators indicate will be required to realize the full potential of data science in the years ahead. This discussion helps set the stage for how the geospatial community will need to evolve in the next decade to position itself for continued success moving forward.

The fourth chapter traces the rise of geospatial big data over the past decade and describes the kinds of digital data that we can now use to inform and guide our spatial data science work. This data spans five major classes: (1) location-based devices and services; (2) volunteered and ambient geographic information; (3) remote sensing; (4) the Internet of Things and sensor networks; and (5) 3D modeling, video, and virtual and augmented reality systems. Their character and value are described using a series of exemplary use cases and applications. The magnitude and rate of change are staggering—for example, the description of remote sensing offered in this chapter tackles in situ sensors, traditional satellite and high-altitude airborne remote sensing systems, nanosatellites, street-level imagery, and unmanned aerial systems, even though Landsat 1 was launched just 50 years ago on July 23, 1972. The choice of and assignment of individual use cases and applications to the five groups is in some ways arbitrary because the spatial sciences is an integrative discipline that finds meaning and value by gathering and integrating data from many sources and domains. The enduring message is that we now live in a digital era that is awash with geospatial data, and this calls for a new focus on fitness of use, replicability, and reproducibility on one hand and new analysis, modeling, and visualization methods on the other.

The fifth chapter describes Esri’s geospatial cloud and the opportunities it provides to extend the work of spatial professionals everywhere. This chapter begins with a brief discussion of geospatial knowledge discovery and how it has evolved with the rise of cloud computing, big data, data science, and open science. The attention then shifts to the functionality of the various elements that make up Esri’s geospatial cloud. The second and third sections, for example, describe the emergence of ArcGIS as an open platform and ArcGIS and interoperability, respectively. The fourth section describes three broad categories of use patterns—the opportunities to enable location intelligence across an organization, the operationalization of spatial workflows in the era of geospatial big data, and ways to support national data standards using industry and open standards. The largest part of this discussion focuses on the new and rapidly evolving spatial data science patterns along with the computing and artificial intelligence approaches used today and how interoperability can be used to work with new data types and geospatial big data at scale. This chapter then switches to brief descriptions of how 3D has been enabled across ArcGIS, how ArcGIS has been extended to support work indoors, connecting ArcGIS with knowledge graphs, and implementing ArcGIS on Kubernetes. The chapter concludes by reminding us of the continued need to sync our science with Esri’s geospatial cloud and other enabling technologies.

The sixth and final chapter starts by briefly summarizing the contributions of the earlier chapters and then uses this summary to paint a picture of the spatial sciences as a fast-moving field than can be combined and used to add value to many academic disciplines and use cases in the public, private, and non-for-profit sectors. This chapter then pivots to explain the two things spatial professionals will need to do to ensure their continued success. The first entails taking up geodesign to describe what a future world would or should look like (as opposed to simply describing the current state of the world), and the second entails the adoption and use of the latest advances in computing, new geospatial data streams, and the latest spatial methods in everything we do. The chapter closes with a call for spatial professionals everywhere to think and work collaboratively with the goal of using spatial methods and data to improve spatiotemporal understanding, analysis, modeling, prediction, and decision-making so they can contribute to and help show the way to a more sustainable, more inclusive, more equitable, and more resilient world.

Chapter 2

The emergence of the spatial sciences as a new and integrative field

This chapter traces the intellectual origins and current focus of work in the spatial sciences today; the varying contributions made by theory, practice, and technology; and the flourishing academic, government, business, and not-for-profit communities that have sprung up around the spatial sciences during the past 20 years.

This is a challenging task considering that more than 50 years have passed since