Infrastructures

Getting access to data is not as easy as it is to access water or electricity. There are thousands
of formats of data, and millions of places to look in order to find the data. Geographic information
systems (GIS) are data-driven (data dependenent) however the task of locating (geo)data,
reformating it, and then exploiting it often is very difficult and labourious. On the other hand,
multiple public and private sector agencies have traditionally collected geodata describing the
same areas of the Earth, in a redundant and very inefficient manner...in a manner that does not
contemplate reuse or sharing of data.
There must be SOME way of facilitating access to (geo)data, of better connecting geodata
providers to geodata users. If the electrical power infrastructure (standards, agreements,
generators, substations, electrical cables, delivery methods, etc.) was constructed to facilitate
standardized access of businesses and houses to electricity, then why cannot a similar
infrastructure be created to provide standardized access to data, in this case geodata?
Fortunately many nations and many subnational regions are now designing such an
infrastructure: SDI.

The notion of a coordinated national SDI began in the USA in the early 1990s. President William
J Clinton signed an Executive Order in 1994, which ordered the US federal government agencies
to start cooperating in terms of geodata collection and use. 

What was the government's (Clinton's) main argument for the need to create a national
SDI? Read Clinton's Executive Order number 12906 (PDF) for yourself. 

The Order is worded in typical (very direct!) American English. The first 8 words essentially tell
the whole story: "Geographic information is critical..." critical for what?

The next paragraph states that the US government needs to stop wasting money.

How does the Order claim that the money can be saved? Please, continue reading the
PDF...spend at least 15 minutes studying its details.
Further readings (in the sections below) will support this same view of the need for SDI: the need
on the part of the public sector data providers. But for now let us look in more detail at the
(different) needs of the geodata user.

Feel the pain

Why SDI? Feel the pain

Before we ask you to read two rather long documents explaining the basic ideas of what is
contained in an SDI, we find it necessary to revisit the problem. New methodologies or
technologies are created as a response or solution to some problem. Therefore SDI must, or
should, be a solution to a problem. Actually there are two related problems: we have seen one
of them in the Clinton Executive Order: SDIs should help public agencies to minimize wasted
resources (a huge problem), by promoting data sharing amongst themselves.
The other related problem is a user problem. Some readers following this course will already
understand this SDI problem, however some other readers may not. If you are already an active
user of GIS, then you probably DO understand: it is difficult for you to locate geodata that wuill be
useful for your GIS analysis. (Some of you may have had difficulty finding data for your class
projects already this semester.) Thus, many times you need to start each analysis from zero! Or
if there are data available, then they are often very expensive to acquire, difficult to access, in
some strange format, are undocumented and therefore not trustworthy, etc.
THINK: Why is geodata not like electricity? Both are goods (products) that may be distributed to
users through network infrastructures. But why is electricity more or less the same in Madrid as
in Lima or Tokyo, yet geodata describing these places can be very, very different in each
case? What is the key factor for this difference?
If you are NOT yet an active user of GIS in real-world situations (this is the case for many
university students), then you probably do not "feel the pain". Unless you feel the pain (truely
understand the problem) then it is difficult to truely comprehend the benefit of the proposed
remedy. If you are not a GIS user but rather you work for a geodata provider, then this lesson
might also be a valuable one, because you should want to better understand your users!

Feel the Pain 2

Ok, you´re back.  How many hours/days have passed since you left step 2?

Even if you were lazy and did not redo the exercise, I think you get the point. 

Were you able to find similar data, or did you stop after quite some frustration?

Are you now feeling the "pain" experienced by many GIS users?

Even if you have local geodata in your possession, simply getting it into the correct format is
often quite difficult. And if you already have geodata, then perhaps you remember how difficult it
was to find it and to reformat it or clean it! The current practice (status quo) in most countries of
the world, is to find relevant geodata not by simple web searches, but rather by phoning a friend
of a friend, who may know someone with information regarding where the data are, how to
access them, licensing details, etc. Or if you are fortunate, you live and work in an area that has
a pro-active mapping agency that more easily facilitates information on what geodata are
available and how to access and exploit these data. The SDI is a key vehicle in making that easy
access possible.

Getting access to geodata should not be a painful process, especially given the fact that much of
that data was collected and maintained with OUR OWN tax money. Are they not ALREADY "our"
data?

Definition of SDI
Definition of SDI
We still have not defined what is a Spatial Data Infrastructure. There are multiple definitions, or
perspectives, that you should read and be familiar with. After reading these you can decide,
depending upon your context (you may be a geodata provider, user, teacher, technical person,
politician,...) what an SDI is and what it is not.
You have already had a look at the U.S. NSDI definition of an SDI, from the Clinton Executive
Order. The following definitions will differ slightly, but hopefully you will see that the central
message is the same.
Please take some time to read these documents carefully, and take notes of the key points.
First reading: Chapter 1 of the INSPIRE Architecture and Standards working group position
paper (2002).
INSPIRE is a European Commission-led initiative, designed to create legislation (a so-called
Framework Directive) which will guide the European member states in the construction of their
national SDIs. We will see more details on INSPIRE, later in this course.
Second reading: Chapter 1 of the GSDI Cookbook.
We will come back to this cookbook at several times during the course. In general the cookbook
is a quite useful guide to SDIs, although from an international perspective we might say that it
has perhaps too much a US (NSDI) flavour to it.
Third reading: SDI case studies (Esri). Have a quick look at what government agencies are
doing to connect the worlds of GIS and SDI.

Summary
After reading the above materials:
You should now have a good idea about what a Spatial Data Infrastructure is.
An electricity infrastructure is more than simply wires and transformers, and a water
infrastructure is more than simply wells, pumps and pipes. These infrastructures exist and run
more or less smoothly thanks to a wide range of agreements among multiple companies and
water authorities. These agreements, and indeed the electricity and water markets, are also
controlled by higher-level political and legal agreements. Without these agreements and without
industry standards, these services would be a complete chaos. When electricity was first used at
factories and other businesses, each building installed its own power generator (locally-
generated power): this was extremely expensive because it did not allow for volume-based
economy of scale. Today factories receive electricity from standardized networks that carry
electricity from centralized generation stations: this facilitates maintenance, build-out
(expansion), and in the end it converts electricity from a local competitive advantage (factories
with it could be more efficient than factories without it) to a global commodity.
 The SDI is similar, is it not? Rather than each GIS user storing and processing his/her own copy
of GIS data locally (like a local generator), the SDI allows (or should allow) for universal access
to common versions of key (basic or framework) geodata. This does not happen by accident:
industry standards and agreements are developed by the market itself, under the guidance of
lawyers and politicians.
How many organisations are required to build an electrical infrastructure? Is it built by a single
Ministry, or a single company? Or is it necessarily built as a collaboration among various
entities? An SDI also is, by definition, a collaborative project or process, involving geodata
producers, users, integrators, educators and trainers, etc.
We understand that most students following this course want to know more about these software
components, so that is the topic of the next section.

Visit an SDI
Visit an SDI
It is time for you to visit, briefly, a Spatial Data Infrastructure. Or at least you should see the web
portal that is the public access point to the SDI. This is a key difference (the portal is not the
entire SDI), one that we will explore in more detail in lesson 2.
The first SDI portal (geoportal) to visit is the INSPIRE European geoportal.
The European Geoportal was created as a pilot to demonstrate some of the SDI components
being implemented as a collaborative effort by institutions in several European member states.
Its funding has been limited, and therefore also the modest quantity of geodata that is stored and
can be accessed with the discovery service.
Visit the INSPIRE geoportal here.
ADVICE: It will be easier for you to navigate through the course, if you open each new
web site in a new browser window. That will allow you to more easily navigate back and
forth, from this course window to the other web sites.

Let's say that you would like to find government geodata, in this case satellite imagery,
for the area around Lisbon, Portugal.
- Select "Discovery/Viewer”
- Then, to view data in Portugal, select the "draw bounding box" tool (second icon) from the right
side menu, to create a zoom box over an area INSIDE Portugal near Lisbon  (even by doing this,
the system still manages to include datasets from Spain!)
- Then enter the text "imagery" in the search box on the right panel. Scroll through the various
datasets identified.
- Are these datasets what you were looking for? If not, then try searching on other imagery-
related terms.
- Select any of the datasets, and study the details that are provided.  What are these details
telling you?
- These details are the summary metadata or data documentation describing a data product or
mapping service. We will see more about metadata in the 2nd and 3rd Lessons. You can try to
go back and test other features of the European geoportal, but it is quite simple.

Visit Data.gov
Visit another SDI
The second portal to visit is the U.S. Data.gov portal. The "one-stop shop” concept is quite
familiar in the world of portals, as it indicates that through this single door (the web portal) the
user can get access to all his or her data or services. This is similar to shopping at a supermarket
instead of visiting several smaller shops in the city centre. This portal has similar basic
functionality to the INSPIRE geoportal: discovery, visualization, download.

Connect to the Data.gov catalogue to browse geospatial data.

You will see that you can filter on data types, and one is geospatial data (see left menu, dataset
type).
Try searching for Environmental Sensitivity Index maps for Florida. Here are the steps to follow:
1. In the main search box enter "ESI Florida"
2. You should receive a list of about 53 data sources.
3. Click on the title (link) of the Environmental Senstivity Index (ESI) Atlas, Gulf of Mexico,
Upper Coast of Texas  (the 4th dataset in the list)

Note: when I was a   Masters student, many years ago, I helped NOAA to create these ESI maps! They are

used in case of marine oil or chemical spills, to know which parts of the coastline to protect first. -MG
4. The system takes you a webpage showing the possible Downloads and resources. You can
try the HTTP download option, which takes you a NOAA webpage where you can Download
individual files (click that and you will see a folder 1.1 with a bunch of data files for the state
of Florida).
So that was a bit complicated and, in the end, you are not really sure which files you have.  What
would happen if you just use Google to search for "ESI Atlas Florida"? Try it.
5. After trying Google, RETURN to the Data.gov data page, and repeat your search, but this time
with keywords "ESI Atlas Florida". 
6. You should get about 740 resources or datasets. Choose the resource called "EnviroAtlas".
Click on Esri REST API Endpoint. Then click on the URL.
(REST is simply the language/protocol used to call the map services; you will see more about
REST in future lessons.)
7. In this case, you are taken not to a data download page, but rather to a data  service
description page. You will see a list of possible datasets to connect to, not data to be
downloaded to your local disk. You will see a long list of service details: what the service does,
what is the geographical extent of the data, which data layers are available, etc.
8. In the "View in" options, click on "ArcGIS.com Map" to open an Esri webmap showing the
data layers. These layers are being served by the ArcGIS Server at Environmental Protection
Agency (EPA) offices.
9. The map viewer zooms into the data's geographical extent (the USA 48 states) and allows you
to choose to view the legend and to choose layers to view. Try opening the Contents and
turning on Climate, Precipitation and Annual Precipitation. Better to view and inspect the data
online in this manner, than go through the trouble of downloading a large data file only to find
that it is not what you were looking for. You can open these data in ArcMap also.

You have now seen two of many examples of SDI geoportals, allowing access to geodata. For
now let us leave the geoportals, a fun and public side of SDIs but really not the central part of the
SDI.
You should now be ready to answer some simple questions regarding the definition of an
SDI. If you do not score 80% or higher then the system will not allow you to access
Lesson 2, and you will need to go back and read the materials in Lesson 1 more closely.
Good luck!