Professional Documents
Culture Documents
Geospatial Data Analytics on AWS 1st Edition Scott Bateman full chapter instant download
Geospatial Data Analytics on AWS 1st Edition Scott Bateman full chapter instant download
https://ebookmass.com/product/data-wrangling-on-aws-clean-and-
organize-complex-data-for-analysis-shukla/
https://ebookmass.com/product/aws-cookbook-recipes-for-success-
on-aws-1st-edition-john-culkin/
https://ebookmass.com/product/data-analytics-anil-maheshwari/
https://ebookmass.com/product/data-analytics-for-social-
microblogging-platforms-1st-edition-soumi-dutta/
ISE Data Analytics for Accounting 2nd Edition Vernon
Richardson Professor
https://ebookmass.com/product/ise-data-analytics-for-
accounting-2nd-edition-vernon-richardson-professor/
https://ebookmass.com/product/data-analytics-for-intelligent-
transportation-systems-1st-edition-edition-mashrur-chowdhury/
https://ebookmass.com/product/ebook-pdf-human-resource-
management-people-data-and-analytics-1st-edition/
https://ebookmass.com/product/data-engineering-with-aws-acquire-
the-skills-to-design-and-build-aws-based-data-transformation-
pipelines-like-a-pro-2nd-edition-eagar/
https://ebookmass.com/product/google-cloud-platform-for-data-
science-a-crash-course-on-big-data-machine-learning-and-data-
analytics-services-dr-shitalkumar-r-sukhdeve/
Geospatial Data Analytics on AWS
Scott Bateman
Janahan Gnanachandran
Jeff DeMuth
BIRMINGHAM—MUMBAI
Geospatial Data Analytics on AWS
Copyright © 2023 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, without the prior written permission of the publisher, except in the case
of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information
presented. However, the information contained in this book is sold without warranty, either express
or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable
for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and
products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot
guarantee the accuracy of this information.
ISBN 978-1-80461-382-5
www.packtpub.com
This book is dedicated to my father, Orren, for instilling my passion for both computers and maps.
To the memory of my mother, Jolene, for teaching me the persistence and patience needed to get my
thoughts on the page. To my wife, Angel, and wonderful children Jackson and Emily for the support
and encouragement to bring this book to life.
Scott Bateman
This book is dedicated to my parents, Gnanachandran and Rajaletchumy, for guiding me through
life’s challenges and celebrating my successes. To my amazing wife, Dilakshana and lovely children,
Jashwin, and Dhruvish, for your unwavering support and belief in me. Finally, to the team at Amazon
Web Services (AWS), whose relentless pursuit of excellence has transformed the way we harness
technology. Thank you for inspiring this work and for your commitment to empowering businesses
worldwide.
Janahan (Jana) Gnanachandran
Contributors
Janahan (Jana) Gnanachandran is a Principal Solutions Architect at AWS. He partners with customers
across industries to increase speed, agility, and drive innovation using the cloud for digital transformation.
Jana holds a BE in Electronics and Communication Engineering from Anna University, India, and
an MS in Computer Engineering from the University of Louisiana at Lafayette. Alongside designing
scalable, secure, and cost-effective cloud solutions, Jana conducts workshops, training sessions, and
public speaking engagements on cloud best practices, architectural best practices, and data and AI/
ML strategies. When not immersed in cloud computing, he enjoys playing tennis or golf, photography,
and traveling with his wife and their 2 kids.
Jeff DeMuth is a solutions architect who joined Amazon Web Services (AWS) in 2016. He focuses
on the geospatial community and is passionate about geographic information systems (GIS) and
technology. Outside of work, Jeff enjoys travelling, building Internet of Things (IoT) applications,
and tinkering with the latest gadgets.
About the reviewers
Faizan Tayyab is a GIS Professional with over 16 years of experience in the oil and gas industry. He
holds a master’s degree and has several certifications covering various technologies including web &
cloud technologies. He is also a software trainer, teaching courses to worldwide audience on technologies
through Udemy, a popular online training platform. He is well-versed with various technologies and
actively contributes to open source geospatial community through development and distribution of
software components available free for use by other geographers and developers.
Angela Orthmeyer is currently the Lead Geospatial Data Analyst at RapidSOS, the world’s leading
intelligent safety platform. Angela is a creative problem solver with experience in GIS, data science and
project management. Prior to her position at RapidSOS, she was a Data Scientist at CKM Analytix, a
Natural Resources Social Scientist at the National Oceanic and Atmospheric Administration, and a
Peace Corps Volunteer in Panama. Her education spans the social and natural sciences. Angela received
a Master of Environmental Management from Yale University, a B.S. in Biology from the University
of Richmond, and a certificate in Data Analytics from Principal Analytics Prep.
Rohit Mendadhala is a Geospatial Data Scientist and an FAA Certified Drone Pilot with over 7 years
of professional experience in developing, managing, and implementing geospatial solutions at scale for
organizations across a wide range of industries such as government, environmental, transportation,
software, telecom, real-estate research. His core areas of expertise include Geospatial Data Analytics,
Data Visualization, Spatial Analysis and Mapping, Spatial Data Science, GIS Development, Web and
Enterprise GIS, Market Research and Analysis using ArcGIS and open-source geospatial platforms.
He enjoys discovering underlying patterns in large datasets with a spatial context and curating answers
to critical thought-provoking questions.
Shital Dhakal is a seasoned GIS professional with over seven years’ experience in the field of GIS and
remote sensing. He has acquired industry and research experience in North America, Europe, and
Asia. Currently, he works at a San Francisco Bay Area-based start-up and helps local government to
implement enterprise GIS strategies. He is a certified GIS Professional (GISP) and has an MSc from
Boise State University, Idaho. When he is not playing with spatial data, writing blogs, or making maps,
he can be found hiking in the Sierra Nevada and, occasionally, in the Himalayas.
Table of Contents
Prefacexiii
2
Quality and Temporal Geospatial Data Concepts 15
Quality impact on geospatial data 15 Considering temporal dimensions 21
Transmission methods 17 Summary22
Streaming data 18 References22
Understanding file formats 18
Normalizing data 20
viii Table of Contents
4
Using Geospatial Data with Amazon Redshift 43
What is Redshift? 44 Redshift geospatial support 47
Understanding Redshift partitioning 44 Launching a Redshift cluster and
Redshift Spectrum 45 running a geospatial query 48
Redshift geohashing support 46 Summary55
Redshift AQUA 46 References56
5
Using Geospatial Data with Amazon Aurora PostgreSQL 57
Lab prerequisites 57 Architectural considerations 78
Setting up the database 58 Summary78
Connecting to the database 65 References79
Installing the PostGIS extension 66
6
Serverless Options for Geospatial 81
What is serverless? 81 Serverless hosting security and
Serverless services 81 performance considerations 83
Object storage and serverless websites with S3 82
Python with Lambda and API Gateway83
Geospatial applications and S3 web Deploying your first serverless
hosting82 geospatial application 84
Summary91
References91
7
Querying Geospatial Data with Amazon Athena 93
Setting up and configuring Athena 93 Spatial functions 100
Geospatial data formats 97 AWS service integration 102
WKT97 Architectural considerations 103
JSON-encoded geospatial data 97 Summary104
Spatial query structure 99 References104
9
Using Geospatial Data with Amazon EMR 115
Introducing Hadoop 115 Geospatial with EMR 117
Introduction to EMR 116 Launching EMR 118
10
Geospatial Data Analysis Using R on AWS 127
Introduction to the R geospatial data Analyzing and visualizing geospatial
analysis ecosystem 127 data using RStudio 153
Setting up R and RStudio on EC2 129 Summary156
RStudio on Amazon SageMaker 138 References156
11
Geospatial Machine Learning with SageMaker 159
AWS ML background 159 First-time use steps 164
AWS service integration 160 Geospatial data processing 166
Geospatial data visualization 172
Common libraries and algorithms 161
Introducing Geospatial ML with Architectural considerations 174
SageMaker162 Summary174
Deploying a SageMaker Geospatial References174
example163
12
Using Amazon QuickSight to Visualize Geospatial Data 177
Geospatial visualization background 177 Configuring Athena 181
Amazon QuickSight overview 180 Configuring QuickSight 183
14
Leveraging OpenStreetMap on AWS 211
What is OpenStreetMap? 211 Architectural considerations 222
OSM’s data structure 211 Summary223
OSM benefits 214 References223
Accessing OSM from AWS 214
Application – ski lift scout 218
The OSM community 222
xii Table of Contents
15
Feature Servers and Map Servers on AWS 225
Types of servers and deployment ECR and EC2 228
options226 Summary235
Capabilities and cloud integrations 226 Further reading 235
Deploying a container on AWS with
16
Satellite and Aerial Imagery on AWS 237
Imagery options 237 Demonstrating satellite imagery
Sentinel237 using AWS 239
Landsat238 Summary243
NAIP238 References243
Architectural considerations 239
Index245
No matter who you are, you will find most of the chapters in this book helpful. The readers who will
get the most out of this book are technologists that have 2-5 years of cloud and/or GIS experience.
Knowledge of either cloud or GIS, but not both, will amplify the value you’ll get reading this book.
We don’t dive too deep into either cloud or geospatial, and if you have a firm footing in either one,
you’re going to learn a lot. If you know how to use geospatial data to make beautiful maps but are
frustrated by some of the technical limitations you face, this book is for you. Even if you didn’t know
that geospatial was a word until reading this book, I think the following 16 chapters will still be enjoyable
and help you understand the power of analyzing geospatial data in the AWS cloud.
Chapter 11, Geospatial Machine Learning with SageMaker, SageMaker is the cornerstone AWS service
for statistical and machine learning computing. This chapter provides readers with step by step guidance
to import, analyze, and visualize geospatial data on AWS using SageMaker
Chapter 12, Using Amazon QuickSight to Visualize Geospatial Data, delves into how geospatial data
on AWS can be converted into visualizations that can be shared with others and combined with web
maps and other geospatial visualizations.
Chapter 13, Open Data on AWS, Open Data on AWS offers public data made available through AWS
services from across the globe. Whether directly interacting with the source or downloading for
analysis and transformation, datasets on demographics, public health, industry, and environment
are ready to use.
Chapter 14, Leveraging OpenStreetMap, OpenStreetMap has more crowdsourced updates than any
other geospatial dataset on the planet. From roads and buildings to businesses and parks, millions
of places can be found on OpenStreetMap. This chapter will show what data can be leveraged using
Amazon Athena queries directly against the latest updates.
Chapter 15, Map and Feature Services on AWS, looks at tools and services available on AWS to create
a durable, scalable platform optimized for the cloud.
Chapter 16, Satellite Imagery on AWS, talks about how to find and use this data and use Amazon
SageMaker for incorporating near real-time machine learning into your applications.
If you are using the digital version of this book, we advise you to type the code yourself or access the
code from the book’s GitHub repository (a link is available in the next section). Doing so will help
you avoid any potential errors related to the copying and pasting of code.
xvi Preface
Conventions used
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions,
pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “ This lets you run
a command like spark.read().text(s3://myfile); and Hadoop can natively read that
file as if it was local.“
A block of code is set as follows:
#!/bin/bash
sudo python3 -m pip install jupyter
mkdir /home/hadoop/.jupyter
touch /home/hadoop/.jupyter/jupyter_notebook_config.py
When we wish to draw your attention to a particular part of a code block, the relevant lines or items
are set in bold:
Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words
in menus or dialog boxes appear in bold. Here is an example: “ Click Create Cluster in the Top
Right Corner.”
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at customercare@
packtpub.com and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen.
If you have found a mistake in this book, we would be grateful if you would report this to us. Please
visit www.packtpub.com/support/errata and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would
be grateful if you would provide us with the location address or website name. Please contact us at
copyright@packt.com with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you
are interested in either writing or contributing to a book, please visit authors.packtpub.com.
https://packt.link/free-ebook/9781804613825
In this part we will learn how to work with Geospatial Data in the cloud, and the economics of storing
and analyzing the data in the cloud.
This part has the following chapters:
• Chapter 1, Introduction to Geospatial Data in the Cloud, shows us how to work with Geospatial
Data in the cloud, and the economics of storing and analyzing the data in the cloud.
• Chapter 2, Quality and Temporal Geospatial Concepts, explores the different quality characteristics
of geospatial data. Additionally, concepts will be presented that show how the time-specific
(temporal) aspects of data can be captured and designated in the data structure.
Another random document with
no related content on Scribd:
si lues; statim Christianus ad leonem acclamatur». Il popolo romano
avea, a tale riguardo, privilegi speciali. Nel Digesto leggiamo: «Ad
bestias damnatos favore populi praeses dimittere non debet: sed si
eius roboris vel artificii sint, ut digne populo romano exhiberi possint,
principem consulere debet» [916].
Ai tempi dell’Impero i cittadini romani erano esenti per legge dalla
damnatio ad bestias; i Cristiani però furono ben presto condannati a
quella pena, qualunque si fosse la loro condizione. Così, ad
esempio, in Lione nell’anno 177, reclamante populo, fu condannato
ad bestias un Cristiano il quale era cittadino Romano. Gli stessi
Imperatori si dilettavano di siffatte condanne; e di Caligola si legge
che un giorno, non essendovi in pronto rei da darsi alle fiere, fè
prendere alcuni spettatori, e, sospintili nell’arena, diè compimento
allo spettacolo. E quanto più prendevan voga i giuochi anfiteatrali,
tanto più gli Imperatori cercarono di trovar materia onde più
frequentemente celebrarli; e fra i condannati ad bestias vennero
annoverati i parricidi, i fratricidi e i rei di lesa maestà: «Etsi antea
insuti culleo in mari praecipitabantur, hodie tamen vivi exuruntur vel
ad bestias dantur»; e dei rei di lesa maestà leggiamo: «Hi antea in
perpetuum aqua et igni interdicebantur; nunc vero humiliores bestiis
obiiciuntur, vel vivi exuruntur» [917].
Dagli scritti di Tertulliano e di altri scrittori apprendiamo che, alla fine
del secondo secolo, le pene da subirsi dai Cristiani erano
determinate dall’arbitrio dei magistrati; ma il fatto ci ha dimostrato
che fra quelle non era ultima la «damnatio ad bestias». Onde il
condannato venisse meglio dilaniato dalle belve, legavasi ad un
palo; e perchè gli spettatori meglio lo vedessero, il palo collocavasi
in un punto alquanto elevato. Così leggiamo di S. Policarpo, che
ricusò di essere legato (nel rogo) al palo [918]; di Saturo: ad ursum
substrictum..... in ponte [919]; di s. Blandina: Blandina vero ad palum
suspensa bestiis obiecta est [920]. Questo pulpito o ponte vedesi
rappresentato in varî cimelî; come, ad esempio, in una lampada di
Cartagine, illustrata dal P. Bruzza [921], nella quale vedesi il
disgraziato paziente legato ad un palo che sorge su di un ponte,
mentre un feroce leone lo assalisce per dilaniarlo.
Non sempre le belve uccidevano la vittima, perchè per lo più, anzi
che di ultimo supplicio, quella condanna serviva per torturare e far
soffrire il paziente, usandosi in tali casi di belve ammaestrate [922]. E
questa fu forse la ragione per cui s. Ignazio scriveva ai romani che
nutriva la speranza di trovare nelle belve tale disposizione, che non
gli perdonassero la vita.
È notissimo che i seguaci di Cristo si propagarono in un modo
straordinario; e non possiamo negare che il Cristianesimo,
specialmente in Roma, abbia avuto uno sviluppo rapido e trionfale.
In una lettera, che s. Paolo scrisse ai Romani nell’anno 58 di C.,
leggiamo il nome di un gran numero di fedeli, ai quali in gran parte
erano connesse intiere famiglie. In quella lettera si ricordano infatti i
coniugi Aquila e Prisca et domesticam Ecclesiam eorum; Epitteto,
Maria, Andronico, Giunia, Ampliato, Urbano, Stachyn, Apelle; quei
della casa di Aristobolo; Erodione; quei della casa di Narcisso,
Trifena e Trifosa, Perside, Rufo, Asincrito, Flegonte, Erma, Patroba,
Ermine, et qui cum eis sunt fratres; Filologo, Giulia, Nereo e la sua
sorella Olimpiade, et omnes qui cura eis sunt sanctos [923]; e nella
lettera ai Filippesi S. Paolo fa menzione di coloro, qui de Caesaris
domo sunt. Nell’anno 64 dell’êra nostra, al triste momento
dell’incendio neroniano, fu tradotta innanzi ai tribunali, secondo la
frase di Tacito, una multitudo ingens di Cristiani: frase, dice
l’Armellini [924] che ha fatto impazzire un povero scrittore straniero,
Hochart P. [925], il quale non sapendo, per odio al Cristianesimo,
accettare questa testimonianza, ha finito col sentenziare, Tacito
essere non un autore genuino, ma uno pseudonimo d’uno scrittore
del medio evo!!
Nè per la persecuzione la nuova fede perdè terreno; giacchè,
secondo la espressione di Tertulliano, semen est sanguis
Christianorum; e talmente s’ingrossarono le sue file, che poco
mancò che nell’anno 80, coi nipoti di Domiziano, il Cristianesimo non
salisse al trono dei Cesari. Esso combattè gloriosamente per due
secoli ancora, e si propagò in tal guisa, che secondo lo stesso
Tertulliano, se i Cristiani, ritenuti dai pagani per loro nemici, avessero
emigrato in remote parti dell’orbe, i gentili avrebbero avuti più nemici
da combattere che cittadini cui comandare [926].
Una prova materiale poi del gran progresso del Cristianesimo in
Roma, l’abbiamo finalmente nelle aree primitive e nei cimiteri
sotterranei. Nel solo raggio di cinque chilometri dal recinto di Servio
Tullio, e senza considerare le aree ed i cimiteri minori, noi troviamo
circa 30 cimiteri detti maggiori, i quali tutti furono iniziati non oltre il III
secolo inclusive.
Da quanto fin qui si è detto, possiamo dedurre:
1.º Che Nerone proscrisse la religione cristiana, e che colle sue leggi
si diè principio all’êra delle persecuzioni;
2.º Che i delitti connessi col nome cristiano erano puniti da quelle
leggi con varie pene, e fra queste non era ultima la damnatio ad
bestias;
3.º Che i Cristiani furono tradotti dinanzi ai tribunali, sia perchè
ricercati d’ufficio dai magistrati, sia perchè accusati dai delatores, o
pagani o ebrei o rinnegati fratelli;
4.º Che i seguaci di Cristo furono realmente condannati ad bestias;
ed abbiamo addotto, per incedens, qualche esempio [927];
5.º Che essendo i Cristiani a quei tempi in gran numero, numerose
pur dovettero essere le vittime, nel mondo pagano in genere, e nella
sua capitale in ispecie.
*
**
S. Ignazio,
S. Eustachio e famiglia,
S. Taziana,
Ss. Abdon e Sennen,
S. Martina,
Ss. CCLXII soldati,
Ss. Vito e Modesto,
Ss. Sempronio e compagni.
Fig. 14.ª