*56
"];
t
7PM

/P

t
.

BSDI

t 

Web scraping and mapping urban

data to support urban design
decisions

Elif ENSARİ1, Bilge KOBAŞ2

1
FMJGFOTBSJ!HNBJMDPN
t
"SDIJUFDUVSBM
%FTJHO
$PNQVUJOH
1SPHSBN
*TUBOCVM

Technical University, Istanbul, Turkey
2
bilge@super-eight.com t
#JUTO
#SJDLT
"OBMZUJDBM
3FTFBSDI
$FOUFS
*TUBOCVM

Turkey

3FDFJWFE
0DUPCFS

t Final Acceptance: November 2017

Abstract
Cities generate data in increasing speed, volume and variety which is more eas-
ily accessed and processed by the advance of technology every day. Consequently,
the potential for this data to feedback into the city to improve living conditions
and efficiency of utilizing resources grows. Departing from this potential, this pa-
per presents a study that proposes methods to collect and visualize urban data
with the aim of supporting urban design decisions. We employed web scraping
techniques to collect a variety of publicly available data within the Kadıköy mu-
nicipal boundaries of Istanbul and utilized a visual programming software to map
and visualize this information. Through this method and superposition of our re-
doi: 10.5505/itujfa.2018.40360

sulting maps, we visually communicate urban conditions including demographic

and economic trends based on online real estate listings as well as spatial distri-
bution and accessibility of public and commercial resources. We propose that
this method and resulting visualizations present valuable potential in supporting
urban design decision-making processes.

Keywords
Geospatial data, Urban data, Urban design decision support, Web scraping.
6
1. Introduction
Data turns out to be more relevant
for urban design as it becomes more
ubiquitously available and accessible
through the advance of technology and
as it is more widely incorporated in the
decision-making processes regarding
UIF
VSCBO
FOWJSPONFOU
3FDFOU
ZFBST

have seen the proliferation of big data as
well as research concerned with its inte-
gration into the analysis, management
BOE
EFTJHO
PG
DJUJFT
/VNFSPVT
BDB-
demic, civic and commercial research
centers have been founded dedicated
to studying the urban environment
through information generated by the
DJUJFT
#BUUZ



8IJMF
UIF
TDBMF

done using the visual programing en-
variety and production speed of data
7T

JODSFBTF
FWFSZ
EBZ

JUT
BWBJM
BCJMJUZ

Our research has revealed various
accessibility and communication to
designers, decision makers and to gen-
eral public remain a critical issue. This
is due to the range of skills required to
access and process the data in order to
make it relevant and meaningful.
The aim of our study is to present an
urban data collection and visualization
methodology which can help urban de-
signers produce maps to communicate
various urban patterns to stake holders
and decision makers in order to sup-
port urban design decisions, as well as
to communities to better inform par-
ticipatory processes. We also present
a set of maps produced following this
workflow, along with some basic geo-
spatial analyses we have performed to
create them. The questions we take on
to answer through the study presented
in this paper are as follows.
t
How can sources of publicly avail-
able data with geolocation informa-
tion be relevant and useful to urban
designers?
t
How can this data be gathered and
visualized?
t
What kind of knowledge can be
generated through the visualization
of this data that may be presentable
to decision makers even before any
geostatistical analysis is carried out?


5PXOTFOE




The study is unique in that we pres-
ent a methodology to compile a rich
dataset which we propose to be rele-
vant for urban design decision making
processes, consisting of information
drawn from multiple sources rather
than studying a single source of data
*56
"];
t
7PM

/P

t
.BSDI

t
&
&OTBSŔ

#
,PCBʰ
and its relevance for a specific urban
issue. We also present visualization
of each data set with some basic spa-
tial analysis. Through this, we aim to
aid quick and easy visual analysis and
better communication of this data to
designers and decision makers. Fur-
thermore, Kadıköy region of Istanbul
has never been subject to such a com-
prehensive study in terms of the variety
of urban data and data sources utilized
as well as the techniques adopted for
mapping and visualization. The data
DPMMFDUJPO
XBT
EPOF
VUJMJ[JOH
1ZUIPO

scripting and all mapping, visualiza-
tion and visual analysis processes were
WJSPONFOU
(SBTTIPQQFS
GPS
3IJOPE
urban patterns that became apparent
through the maps we have created uti-
lizing the data collected from multiple
geolocated urban data sources. Two
most significant cases we have ob-
served to have potential in contributing
in the urban design decision dialogues
are the exposition of public and com-
mercial resource distribution, and de-
mographic and economic trends read
through the various maps we present
in the following sections.
We aim to develop this study by in-
tegrating more comprehensive analy-
ses to our workflow in future research.
2. Literature review
While cities around the world are
growing in size and population every
day, digital devices and systems are be-
coming more and more integral to their
functioning in parallel with technolog-
ical advancements. This results in the
generation of massive amounts of data
in the urban environment which have
already been subject to a large body of
research motivated with the belief that
cities can learn from this data and be-
DPNF
ATNBSUFS
JO
PSEFS
UP
QSPWJEF
CFU-
ter living conditions for their residents
while utilizing resources more efficient-
MZ
#BUUZ
FU
BM
.PCJMJUZ
QBUUFSOT
PG
VSCBO
EXFMMFST

drawn from digital public transporta-
tion ticketing systems, cellular phones
or cameras detecting traffic patterns,
GIS data compiled and made avail-
able to public by governments, various
use and activity data shared through
 7

personal mobile devices and publicly to draw data from public web sites, web
available social media content are some scraping which we utilize in our study
of many examples of such data. involves automated means to query and
"
QBSU
PG
UIFTF
TUVEJFT
JO
VSCBO
download large sets of geospatial data.
data focuses on gathering, analyzing #PFJOH
BOE
8BEEFMM


VUJMJ[F
UIJT

and visualizing data which is inher- technique to analyze real-estate listings
ently linked to geolocation informa- in the US and are able to compare elev-
tion, namely geospatial big data (Lee en million listings all over the country

,BOH


NBLJOH
JU
QPTTJCMF
UP
with government declared fair market
map and observe geospatial patterns of SFOUT
JO

NFUSPQPMJUBO
BSFBT
"1*T

urban phenomenon. While such stud- BQQMJDBUJPO
QSPHSBNNJOH
JOUFSGBDFT


ies often bring together geographers provided by many online platforms
and computer scientists based on in- such as Google, Instagram, Facebook,
UFSFTU
BOE
UFDIOJDBM
TLJMMT
SFRVJSFE
VS- Flickr, Foursquare or Twitter make it
ban planners, designers and architects possible to query and download user
also see value in this data which can generated content, which becomes in-
provide precious insight on the func- teresting for urban researchers when
tioning of cities and support urban the data is accessed together with geo-
decisions in order to improve the qual- MPDBUJPO
JOGPSNBUJPO
"NPOH
TFWFSBM

ity of the urban environment (Glaeser, recent studies that utilize social net-
%VLF,PNJOFST
-VDB

/BML


work data to better understand urban
One of the most popular subjects patterns, Cranshaw and colleagues
of study of dynamic urban data is real 

EFUFDU
BSFBT
PG
DPODFOUSBU-
estate rental and sales prices, mapping FE
VSCBO
BDUJWJUZ
i-JWFIPPETw

VT-
and geographical analysis of which al- ing Foursquare check-ins and Jiang,
low for understanding location driven "MWFT
3PESJHVFT
'FSSFJSB

1FSFJSB

contributors to property values (Co- 

FTUJNBUF
MBOE
VTF
CBTFE
PO
:B-
IFO

$PVHIMJO

(FPHIFHBO
IPPT
QPJOU
PG
JOUFSFTU
10*

EBUB
XJUI

8BJOHFS

#PDLTUBFM

8BEEFMM
a higher accuracy than with traditional
#FSSZ

)PDI


.PSF
SFDFOUMZ
NFUIPET
$IFO


QSPWJEFT
B
HPPE

studies utilize housing price informa- overview of data collecting methods
tion from public real-estate websites from social media platforms including
#FODZ
3BMMBQBMMJ
(BOUJ

4SJWBUTB
XFC
TDSBQJOH
BOE
UIF
VTF
PG
"1*T
UIBU


#PFJOH

8BEEFMM


GB- were also utilized in our study.
cilitating the use of very large and up "MM
HFPTQBUJBM
EBUB
JT
BMSFBEZ
CJH

to date datasets, which allows for the EBUB
-FF

,BOH


BOE
WFMPDJUZ

development of more accurate hous- WBSJFUZ
BOE
WPMVNF
BSF
UIF
7T
DPO-
ing price prediction models. One such sidered to define big data. While col-
study finds correlations of housing pric- lecting, managing and analyzing such
es with geo-tagged social media con- rapidly generated large scales of data
UFOU
-J
:F
-FF
(POH

2JO


already pose a significant challenge,
which is another type of data that has visualizing and communicating it is ut-
been subject to urban analysis research most critical in deriving meaning and
as an indicator of urban activity (Jen- value from geospatial data in order for
kins, Croitoru, Crooks, & Stefanidis, cities to learn from it. There are several

4DISFDL

,FJN


8BMLJOH
tools from geographical information
running and other forms of exercise TZTUFNT
(*4

CVJMEJOH
JOGPSNBUJPO

data tracked and shared by urban res- models to virtual simulation environ-
idents have also been studied to better ments that make available various visu-
understand qualities of preferred loca- BMJ[BUJPO
UFDIOJRVFT
1FUUJU
FU
BM



tions and routes for recreational activ- for geospatial data.
JUZ
JO
UIF
VSCBO
FOWJSPONFOU
#BMBCBO
"
QSPKFDU
UIBU
EFNPOTUSBUFT
UIF


5VODFS

$MBSLF

4UFFMF


power of visualization of geospatial
There are various methods and tech- EBUB
i.JMMJPO
%PMMBS
#MPDLTw
,VS-
niques which allow for management of HBO


NBQT
UIF
IPNF
BEESFTTFT

big data, yet means of its collection and of criminals as they are admitted into
visualization is the main subject of our prison as well as the length of their stay
JOUFSFTU
JO
UIJT
SFTFBSDI
"T
B
NFUIPE
JO
ĕWF
PG
64T
MBSHFTU
DJUJFT
BOE
SFWFBMT

Web scraping and mapping urban data to support urban design decisions


Figure 1. The workflow.

a remarkable concentration of these these visualizations created through
addresses around a few city blocks. basic spatial analysis and mapping
0OF
PG
UIF
QSPKFDUT
NPTU
TUSJLJOH
WJ- techniques allow for human visual as-
suals maps the money spent by the sessment of a rich set of urban data,
state on each criminal to their home which is deemed as valuable as auto-
BEESFTT
EJTQMBZJOH

NJMMJPO
EPM- mated analytical processes (Schreck &
lars spent on criminals coming from ,FJN


XJUIJO
KVTU
GPVS
CMPDLT
JO
#SPXOTWJMMF

#SPPLMZO
"
TUVEZ
DPOEVDUFE
JO
B
OFX- 3. Methodology
MZ
BDUJWBUJOH
BSFB
PG
"NTUFSEBN
ćF
Our process involves the following
,OPXMFEHF
.JMF
/JFEFSFS
$PMPNCP
steps: drawing of data from online
.BVSJ

"[[J


QSFTFOUT
HFP- sources, organizing the data, perform-
graphical maps of companies and their ing mathematical operations to calcu-
online hyperlinks, most shared images late some additional values on the da-
and most shared locations using data tabase, mapping the data based on geo
gathered from online platforms Goo- location information, performing basic
HMF
4FBSDI
1BOPSBNJP
*OTUBHSBN
BOE
geometrical analysis and visualization.
'PVSTRVBSF
.BLJOH
BQQBSFOU
UIF
PO- Our workflow is presented in Figure 1.
line presence and character of the ur- We utilized multiple sources of pub-
ban axis in the focus of the research, licly available data from within the
these maps were later shared with local Kadıköy municipal boundaries of Is-
stakeholders in participatory design tanbul, to draw already geo-tagged data
sessions and reportedly initiated con- or data with address information that
versation and further field work. we converted to latitude and longitude
Having various similarities with the WBMVFT
UISPVHI
(PPHMFT
(FPDPEF
"1*

research presented above, our study While mapping and visualizing the raw
presents multiple layers of geographi- data we collected, we applied different
cally linked urban data drawn from dy- MFWFMT
PG
JOUFSWFOUJPO
GSPN
TJNQMZ
QP-
namic sources of public web content, sitioning the circles at geographical
mapped to reveal patterns of urban ac- locations and scaling them based on
tivity and distribution of commercial their values within the domain of the
and public facilities. We propose that data set or connecting the geolocation

*56
"];
t
7PM

/P

t
.BSDI

t
&
&OTBSŔ

#
,PCBʰ

Figure 2. Unit rental prices (rent price per sqm).
points to create routes and superposing
them, to performing surface to surface
intersections to calculate areas of influ-
ence or accessibility of amenities auto-
mated through custom codes.
Our data comprises real estate rental
and sales prices, property square me-
ters, property bedroom numbers, ages
of buildings, uses of buildings, social
media activity, public park boundaries,
commercial facilities, public transpor-
tation and other public facilities.
For the data collection phase, we use
1ZUIPO
QSPHSBNNJOH
MBOHVBHF
UP
TFOE

SFRVFTUT
DPNQPTFE
CZ
UIF
1PTUNBO
UP


TJ[F
JO
TRNT

OVNCFS

UIF
TPVSDF
XFC
TJUFT
TFSWFST
BOE
HFOFS-
ally download the data in Json format.
We then convert the data to csv (com-
NB
TFQBSBUFE
WBMVF

GPSNBU
BOE
JNQPSU
page of each listing to draw further text
it to Grasshopper visual programming
FOWJSPONFOU

BO
BEEPO
GPS
3IJOPE

of heating, additional amenities and
VTJOH
.FFSLBU
QMVHJO
)FSF
XF
QSPDFTT

the values within the database or run
simple spatial analysis after converting
UIFN
UP
HFPNFUSJDBM
FOUJUJFT
"MM
UIF

steps carried out in the Grasshopper
environment besides the final graphic
touch-ups are automated so that they
are easily repeatable for different loca-
tions in future studies.


#FMPX
XF
QSFTFOU
UIF
EFUBJMT
PG
PVS

methods grouped based on the sources
Web scraping and mapping urban data to support urban design decisions

of data and the themes of the maps that
were created.
3.1. Housing and real estate
We utilized one of the most wide-
ly used real estate listings websites in
5VSLFZ
XJUI

NJMMJPO
BDUJWF
NFN-
CFST
WJTJUJOH
UIF
QPSUBM

NJMMJPO

times per month that allows for post-
ing and managing of apartment rental
and sales listings by both the owner
of the property as well as real estate
agents. The website has a map interface
through which we were able to draw
HFPMPDBUJPO
of rooms and comments of the listing
owner of each apartment in the Json
format. Then we sent a request to the
based information including the type
whether the listing was posted by the
owner or the real estate agent. The text
EBUB
XBT
QBSTFE
VTJOH
UIF
#FBVUJGVM

4PVQ
-JCSBSZ
GPS
1ZUIPO
BOE
DPOWFSU-
ed into csv format with Csv Library for
1ZUIPO
ćF
DTW
ĕMFT
XFSF
UIFO
JNQPSU-
ed to Grasshopper and processed to
calculate the rental and sales prices per
TRN
BOE
DSFBUF
NBQT
'JHVSFT



In these maps, the radius of each cir-
cle positioned at the geolocation of the


Figure 3. Sales prices per sqm.

Figure 4. Number of bedrooms for each house listing (rental and for sale).
listing is scaled proportionately with from both rental and sales listings and
its price divided by sqms, after the out- NBQQFE
XJUI
DPMPST
'JHVSFT





liers were excluded from the dataset. 1BZCBDL
WBMVFT
XFSF
DBMDVMBUFE
QFS

The number of rooms per apartment N
CZ
N
BSFB
CBTFE
PO
UIF
BW-
and building age data were combined erage apartment rental and sales prices

*56
"];
t
7PM

/P

t
.BSDI

t
&
&OTBSŔ
#
,PCBʰ
 11

Figure 5. Ages of the buildings of the listings (rental and for sale).

Figure 6. Payback time.

per sqm available in each grid square ing use information, with the build-
'JHVSF

 ing footprints colored using a gradient
"
(*4
ĕMF
DSFBUFE
CZ
*TUBOCVM
.FUSP- based on the proportion of the number
QPMJUBO
.VOJDJQBMJUZ
XBT
VTFE
UP
NBQ
of non-residential units to the residen-
the residential/non-residential build- UJBM
POFT
GPS
FBDI
CVJMEJOH
'JHVSF



Web scraping and mapping urban data to support urban design decisions
12

Figure 7. Residential and non-residential uses in buildings.

Figure 8. Circle grid system for Instagram queries.

3.2. Social media LFZ
XJUI

NJMMJPO
VTFST
CZ


Instagram was the source of social 4UBUJTUBDPN


1ZUIPO
DPEF
BOE

media data we utilized for this study as *OTUBHSBNT
PXO
"1*
XBT
VTFE
UP
RVFSZ

it is known to be favored over Twitter 
QPTUT
BU
B
UJNF
GSPN
XJUIJO

N

and other forms of social media in Tur- SBEJJ
PG

QPJOUT
BSSBOHFE
UP
DPWFS
BMM

*56
"];
t
7PM

/P

t
.BSDI

t
&
&OTBSŔ
#
,PCBʰ
 

Figure 9. Cafes, Instagram posts and their like numbers.

the area within the municipal bound-
BSJFT
PG
,BE‘LÚZ
'JHVSF


#FTJEFT
UIF
QPTUT

XF
RVFSJFE

QPJOUT
XJUI
N

geolocation data, number of likes and
the text accompanying each image was
drawn. The duplicate posts download-
ed due to inevitable overlaps of circles
XFSF
DMFBOFE
VQ
1PTUT
UIBU
XFSF
MJLFE

PWFS

UJNFT
XFSF
DPVOUFE
BT



XJUI
PWFS

NJMMJPO
VTFST
CZ

due to their low frequency and posts

UIBU
EJSFDUT
PSEFST
GSPN

XJUIJO
B
EJTUBODF
PG
NT
UP
FBDI
PUI-
er were combined by adding up their
number of likes for the sake of a more
MFHJCMF
SFQSFTFOUBUJPO
"
NBQ
XBT
DSF-
ated with the scale of each circle repre-
senting the like numbers per post point
'JHVSF

 VTJOH
(PPHMFT
(FPDPEF
"1*
ćFO

The text data drawn from the Insta-
gram posts were analyzed using a cus-
tom definition separating each word us-
JOH
1ZUIPO
TQMJU
GVODUJPO
EFĕOJOH
TQBDF


XFSF
RVFSJFE

sign “ “ as a delimiter and hashtags were
appended to the data of each post in
the csv file. Then the frequency of each
IBTIUBH
XBT
DBMDVMBUFE
QFS
N
CZ

N
HSJE
BOE
NBQQFE
'JHVSF

 scraper script was developed which
3.3. Public and commercial
amenities
6TJOH
UIF
(PPHMF
1MBDFT
"1*

XF
Soup Library was used to convert the
drew the geolocations and ratings of
$BGÏT
BOE
3FTUBVSBOUT
XIJDI
XFSF

NBSLFE
PO
(PPHMF
.BQT
4JNJMBSMZ

with the method we drew Instagram
SBEJJ
BOE
SFDPSEFE

QMBDFT
BU
B
UJNF

Then we cleaned up the duplicates and
created a simple point-location map.
One of our resources was a widely
used food delivery portal, Yemeksepe-
UJDPN


users to member restaurants in Tur-
LFZ
&BDI
JUFN
PO
B
SFTUBVSBOUT
NFOV

is displayed with its price, however, the
SFTUBVSBOUT
BEESFTT
JT
OPU
QSPWJEFE
PO

the website. First, we queried the geo-
location information of each restaurant
the prices of the most commonly sold
JUFNT
XIJDI
XFSF
B
NM
DBO
PG
$PLF

BOE
NM
DVQ
PG
"ZSBO
B
ZPHVSU

CBTFE
5VSLJTI
CFWFSBHF
We also drew the hours of operation for
each restaurant. The website does not
provide the menu items in a database
GPSNBU
UIFSFGPSF
B
DVTUPN
1ZUIPO
XFC

searches for all restaurants delivering to
the area and then follows each of their
MJOLT
UP
BDDFTT
UIFJS
NFOVT
#FBVUJGVM

)5.-
GPSNBUUFE
TJUF
JOUP
B
GPSNBUUFE

spreadsheet, the prices were converted
to the scales of circle radii distributed

Web scraping and mapping urban data to support urban design decisions


Figure 10. Hashtag map.

on the map based on the geolocation calculated the number of stops within
of each restaurant and the number of these circles first, and then multiplied
operating hours were converted to col- the seated passenger capacity of each
or codes to be visualized on the map vehicle with its frequency throughout
'JHVSFT

 the day. Without drawing the route
lines, we did a similar calculation for
3.4. Transportation and access ferries and subway lines, and added
We drew the geolocation data of stops up all the seated passenger capacities
for buses and metrobuses (a rapid bus within the mentioned radii distance
transit line using dedicated bus lanes from each building centroid. We illus-
GPS
NVDI
PG
UIF
SPVUFT

GSPN
*&55T
trated these numbers through a gradi-
(the general transportation department ent of colors for each building, repre-
PG
UIF
$JUZ
PG
*TUBOCVM

XFCTJUF
BT
XFMM
senting a scale of accessibility to public
as the number of times a bus or metro- USBOTQPSUBUJPO
'JHVSF


bus passes through each stop to create "OPUIFS
NBQ
XF
DSFBUFE
BJNT
UP
WJ-

SPVUFT
PO
B
NBQ
'JHVSF




8F
VTFE
sualize the accessibility of recreational
their frequency data to define the thick- green spaces from buildings. For this,
ness of the route lines on our map. we utilized the geographically located
Then for each building centroid, outlines of public parks drawn from
we computed the number of public the GIS files and made surface inter-
USBOTQPSU
TFBUT
BDDFTTJCMF
XJUIJO
N
sections of them with the circles of
N
BOE
N
SBEJJ
'PS
UIJT
XF
N
N
BOE
N
SBEJJ
GSPN

*56
"];
t
7PM

/P

t
.BSDI

t
&
&OTBSŔ
#
,PCBʰ
 

Figure 11. Soft drink prices.

Figure 12. Operating hours of the restaurants.

each building centroid. The areas of We drew the exercise routes of Kadıköy
intersecting regions gave us a domain residents from an activity tracking social
of accessible green space sqms that we network that allows for sharing of walk-
mapped as a gradient of colors for each ing, running, biking and swimming activ-
CVJMEJOH
QPMZHPO
'JHVSF

 ity data. Superposing these routes revealed

Web scraping and mapping urban data to support urban design decisions
16

Figure 13. Bus and metrobus lines.

Figure 14. Accessibility to public transportation seats.

the most popular streets and in one case a 4. Observations and urban design
swimming route, as well as the approxi- decision support
mately estimated geographic locations of We propose that the set of maps pro-
the buildings, residents of which use these duced by our automated workflow are
SPVUFT
NPTU
GSFRVFOUMZ
'JHVSF

 powerful in demonstrating how urban

*56
"];
t
7PM

/P

t
.BSDI

t
&
&OTBSŔ
#
,PCBʰ
 17

Figure 15. Accessibility to green spaces.

Figure 16. Exercise routes.

data can be basically organized and visu-
alized to communicate meaningful obser-
vations and aid urban design decisions.
"MUIPVHI
UIF
RVBOUJUBUJWF
SFTVMUT

we aim to generate in future studies
through geospatial analysis of this data
will advance this research significant-
ly, we believe that the current stage of
the study we present here is as useful
in revealing the relevance of geospatial

Web scraping and mapping urban data to support urban design decisions

information for urban design. Some
of our observations at this stage of our
research are presented below. We also
present ways in which the visual in-

ćF
HFPHSBQIJDBM

formation revealed in these maps can
support urban planning and design
decisions following each of our obser-
vations.
There is a concentration of apart-
ments for sale and higher sales prices
along the coast and a concentration
the liveliness and feeling of safety on
a street, contributing to the “Eyes on
the street,” a concept first proposed by
+BOF
+BDPCT

distribution of commercial amenities
and social media activity can inform
urban decisions of zoning and invest-
ment in new facilities or services by re-
vealing if the existing facilities are suf-
ficient or attractive to urban dwellers
and how commercially active a street is
PG
SFOUBMT
PG

BOE

VOJUT
PO
UIF
UISPVHIPVU
UIF
EBZ
"SFBT
UIBU
BSF
MBDL-
XFTU
PG
4ÚʓàUMà
±FʰNF
'JHVSFT




ing in such amenities can be prioritized


ćJT
JT
JOGPSNBUJWF
BCPVU
UIF
EF- for making changes in planning to en-
mographics of residents around these courage a more vibrant street life and
neighborhoods and can guide the mu- commercial activity on their streets.
nicipality in deciding on the kind of Our public transportation and ac-
investments they should make, may DFTT
NBQT
'JHVSFT




WJTVBM-
they be introducing co-working spac- ly present one of the most common
es, libraries, professional education indicators of walkability for urban
facilities, kindergartens or retirement OFJHICPSIPPET
"MUIPVHI
UIF
EJTUBODF

homes. Our Instagram hashtags and from each building was not measured
TPę
ESJOL
QSJDFT
NBQT
'JHVSFT


through the street network but simply


TJNJMBSMZ
DPOUSJCVUF
JO
SFWFBMJOH
calculated based on a radius, the map
the demographics of the local residents showing access to public transpor-
and visitors, as well as providing a tation seats provides an idea of how
glimpse of their consumer trends. democratically distributed the trans-
Urban transformation which is portation facilities are within the mu-
encouraged by the policy to renew OJDJQBM
CPVOEBSJFT
PG
,BE‘LÚZ
"DDFTT

non-earthquake resistant housing to greenspace that we created a map
stock of Istanbul is made apparent in GPS
'JHVSF


JT
B
TJNJMBSMZ
TJHOJĕDBOU

our maps displaying the ages of build- contributor to walkability and quality
JOHT
'JHVSF



TIPXJOH
UIF
HFPHSBQI- of life. Our exercise routes map (Figure
ic distribution of recent construction. 

HJWFT
BO
JEFB
PG
XIJDI
SFTJEFOUJBM

The rapid transformation within the locations have better access to recre-
Caddebostan area in the last couple of BUJPOBM
BDUJWJUJFT
XJUIJO
UIF
DJUZ
#F-
years is revealed which may be obvious sides distance, we argue that this map
to the neighborhood residents but no may reveal additional contributors to
known maps exist with updated infor- the exercise behavior in the city such as
mation available to public. The local the pleasantness, comfort and safety of
authorities can utilize this information these routes, as well as their accessibili-
not only for understanding areas where ty from different neighborhoods.
transformation is slow, but also where .VOJDJQBMJUJFT
BOE
DFOUSBM
BVUIPS-
it is rapid, and they can work with ities can manage the allocation of
urban designers to take precautions their resources based on information
against disadvantageous consequences regarding the distribution of public
of such a high density of construction services and facilities easily observ-
sites and the rapid change of the built able through our maps demonstrating
environment for the local residents. access to public transport, green ar-
Together with the cafes & Insta- eas and routes used for sports activity.
HSBN
MJLFT
NBQ
'JHVSF



PVS
SFTJ- Since residential neighborhoods less
dential-nonresidential buildings map fortunate in terms of access to public
SFWFBM
'JHVSF


UIF
NBJO
DPNNFS- transportation can quickly be identi-
cial axes and the mixed-use pattern in fied in these maps, they can encourage
Kadıköy which is known to be an in- local residents to demand an increase
dicator influencing the walkability in of routes and more frequent services
a neighborhood. The operating hours GSPN
USBOTQPSUBUJPO
BVUIPSJUJFT
"D-
PG
SFTUBVSBOUT
'JHVSF


JOĘVFODF
cess to green areas and the shoreline

*56
"];
t
7PM

/P

t
.BSDI

t
&
&OTBSŔ

#
,PCBʰ

which is obviously attractive for ex-
ercisers can be enhanced by new bike
lanes and improved street conditions
for better walkability especially in areas
that link the northern parts of Kadıköy
to these areas.
We present these observations
and suggestions for urban designers
through the case study of Kadıköy,
however, we aim this study and these
maps to constitute a model for reveal-
ing and communicating information
that can guide urban decisions in im-
proving the street life, commercial
activity, local development and life
quality of urban residents through
better allocation and more democratic
geographic distribution of resources.

OPUF
B
DBTF
XIFSF
B
GFEFSBM
64

Without doubt, each urban area from
each country is unique in many aspects
that also contribute to the urban qual-
ities that we propose can be improved
through our observations, thus all data
should be mapped and interpreted with
various such differences in mind to be
relevant in supporting urban decisions.
5. Discussion
We do not present any geostatistical
analysis of urban data in this paper, but
rather propose this research as a pre-
liminary step to be followed by rigor-
ous spatial analyses in studies to follow.

B
TJHOJĕDBOU
QPSUJPO
PG

/FWFSUIFMFTT

XF
QSPQPTF
UIF
NFUIPE- public and commercial activity takes
ology we present for urban data collec-
tion and visualization as a guide for de-
signers to better understand and utilize
means to gather and process data rele-
vant for urban design decision support.
We acknowledge that there may be
possible errors in information provid-
ed by the listing owners regarding the
SFBMFTUBUF
EBUB
XF
VUJMJ[F
7PMVOUBSJ-
ly generated data may also cause in-
accuracies due to the free data-entry
format, as in the case for social media

XIFO

IPX
IPX
UIFZ

entries. The social media users tend to
create their own use of language, ab-
breviations, and personal methods of
VTJOH
IBTIUBHT
4DISFDL

,FJN



what was designed and how they are
or simply make misspellings. Further-
more, Turkish is an agglutinative lan-
guage and multiple suffixes make it un-
favorable for text based analytics.
"EEJUJPOBMMZ

UIF
TPVSDFT
XF
IBWF
"MUIPVHI
VSCBO
EFTJHOFST
BOE
BV-
used for this research are limited:
There are several websites and appli-
cations dedicated for similar purpos-
Web scraping and mapping urban data to support urban design decisions

es, and at this stage of the study, the
datasets were created using the most
widely used website and/or application
in their respective category. While this
approach gave us a strong initial start,
additional sources may be added to the
datasets in further research. In such a
case, a mutual database format should
be created as the type of data retrieved
from different sources will also begin
to differ.
Final discussion marks include
DPQZSJHIU
JTTVFT
BT
BMNPTU
BMM
PG
UIF

websites used in these studies limit re-
production and re-distribution of the
data for commercial use in their legal
VTFS
BHSFFNFOUT
#PFJOH
BOE
8BEEFMM


court decided that web scraping pub-
licly available data did not constitute
a copyright violation. Even though
non-commercial research does not fall
under legal restrictions, only a handful
of the websites and applications make
the data collecting process openly
available to developers and researches.
This means that the sources without
developer access may easily limit or re-
strict our access in the future by mak-
ing changes to their web structures.
6. Conclusion
/PXBEBZT
place on online social networks. We
swim in an enormous pool of data
streaming all around us. The tweets,
selfies, check-ins, likes, ignores, pokes,
snaps, listings, stars, user comments
constantly keep updating and adding
UP
UIJT
QPPM
.PTUMZ
DPOTJTUJOH
PG
VO-
organized, unfiltered and uncatego-
SJ[FE
SBX
EBUB
UIJT
QPPM
JT
UIF
VMUJNBUF

collective subconscious of our society.
It has the information on what people
MJLF
EPJOH
XIFSF
USBWFM
IPX
UIFZ
TQFOE
UIFJS
UJNF
IPX

UIFZ
TQFBL
UP
FBDI
PUIFS
XIBU
UIFJS

TIBSF
PG
FDPOPNZ
JT
IPX
UIFZ
UBLF

transformed by that. This is a valuable
source of information for anyone inter-
ested in exploring the geospatial past,
current time and future in great detail.
thorities already make use of the data
available to them, traditional data col-
lecting methods tend to be more static


and data is not so easy to update, where- #BUUZ
.
"YIBVTFO
,
(JBOOPUUJ

as spatial information is exponentially '
1P[EOPVLIPW
"
#B[[BOJ
"
8B-
more valuable when the time layer is DIPXJD[
.
0V[PVOJT
(
1PSUVHBMJ

included in the equation. Deriving :


4NBSU
DJUJFT
PG
UIF
GVUVSF
Eu-
the data from constantly self-updating ropean Physical Journal: Special Topics,
sources, and establishing a workflow 
o
that collects, organizes, and processes #FODZ
"
3BMMBQBMMJ
4
(BOUJ
3

this data ensures access to up to date 4SJWBUTB
.

.BOKVOBUI
#



TDFOBSJPT
BU
BMM
UJNFT
"EEJUJPOBMMZ
#FZPOE
4QBUJBM
"VUP3FHSFTTJWF
.PE-
the accumulation of this data creates a FMT
1SFEJDUJOH
)PVTJOH
1SJDFT
XJUI

spatio-temporal archive in time, which 4BUFMMJUF
*NBHFSZ
*O

IEEE Winter
would illustrate the development and Conference on Applications of Comput-
evolution of urban use patterns. er Vision
QQ
o

4BOUB
3PTB

Through this research we have in- $"
64"
*&&&
tegrated different computational ap- #PFJOH
(

8BEEFMM
1



proaches together to create an auto- /FX
*OTJHIUT
JOUP
3FOUBM
)PVTJOH

mated workflow that draws data from .BSLFUT
BDSPTT
UIF
6OJUFE
4UBUFT
8FC

online sources and visualizes them on 4DSBQJOH
BOE
"OBMZ[JOH
$SBJHTMJTU

NBQT
"
NBKPS
QBSU
PG
UIJT
XPSLĘPX
3FOUBM
-JTUJOHT
Journal of Planning Ed-
can be utilized to draw data from var- ucation and Research,
o

ious other sources with minor tweaks $IFO
/
$


Urban Data
in the query code. We present multi- Mining: Social Media Data Analysis as
ple layers of urban data that we believe a Complementary Tool for Urban De-
to be relevant for aiding urban design sign
.*5
decisions by revealing various urban $MBSLF
"

4UFFMF
3


)PX

use patterns and distribution of public personal fitness data can be re-used by
and commercial amenities and services smart cities. In 7th International Con-
within a neighborhood. We visual- ference on Intelligent Sensors, Sensor
ize our data layers on maps which we Networks and Information Processing
propose to constitute a basis for fur- QQ
o

"EFMBJEF
4"
"VTUSBMJB

ther study where geospatial statistical IEEE.
analyses will be performed to achieve $PIFO
+
1

$PVHIMJO
$
$

quantitative results. 

4QBUJBM
IFEPOJD
NPEFMT
PG
BJS-
We believe that with the further port noise, proximity, and housing
development of web technologies and prices. Journal of Regional Science,
integration of mobile devices into our  

o

lives, the definition of a city and what it $SBOTIBX
+
4DIXBSU[
3
)POH
+

encompasses is already being redefined *

4BEFI
/


ćF
-JWFIPPET

fundamentally. Therefore, approaches 1SPKFDU
6UJMJ[JOH
4PDJBM
.FEJB
UP
6O-
to urban analysis and design must be derstand the Dynamics of a City. In
re-evaluated and become as dynamic Proceedings of the Sixth International
our urban environments already have. AAAI Conference on Weblogs and So-
cial Media
QQ
o

1BMP
"MUP
$"

Acknowledgements 64"
ćF
"""*
1SFTT

The authors would like to thank Geoghegan, J., Wainger, L. a., &
Can Kadir Sucuoğlu and Cemal Koray #PDLTUBFM
/
&


"OBMZTJT
4QB-
#JOHÚM
GPS
UIFJS
PVUTUBOEJOH
DPOUSJCV- tial landscape indices in a hedonic
tions in this study. framework:an ecological economics
analysis using GIS. Ecological Econom-
References ics

o

#BMBCBO
0

5VODFS
#


7J- Glaeser, E. L., Duke-Kominers, S.,
TVBMJ[JOH
6SCBO
4QPSUT
.PWFNFOU
*O
-VDB
.

/BML
/


Big Data
Complexity & Simplicity - Proceedings and Big Cities: The Promises and Lim-
of the 34th eCAADe Conference
7PM

itations of Improved Measures of Urban
QQ
o

0VMVMV
'JOMBOE
F$""%F
Life. HKS Faculty Research Working Pa-
and University of Oulu. per Series.
$BNCSJEHF
."
#BUUZ
.


Urban Informatics +BDPCT
+


The Death and Life
and Big Data. London. of Great American Cities.
/FX
:PSL

*56
"];
t
7PM

/P

t
.BSDI

t
&
&OTBSŔ
#
,PCBʰ
 21

/:
64"
3BOEPN
)PVTF
1FUUJU
$
8JEKBKB
*
3VTTP
1
4JO-

+FOLJOT
"
$SPJUPSV
"
$SPPLT
OPUU
3
4UJNTPO
3

5PNLP
.

"
5

4UFGBOJEJT
"


$SPXE- 

7JTVBMJTBUJPO
TVQQPSU
GPS
FY-
sourcing a collective sense of place. ploring urban space and place. ISPRS
PLoS ONE, 11 

o
Annals of Photogrammetry, Remote
+JBOH
4
"MWFT
"
3PESJHVFT
'
Sensing and Spatial Information Scienc-
'FSSFJSB
+

1FSFJSB
'
$


es, I-2
o
.JOJOH
QPJOUPGJOUFSFTU
EBUB
GSPN
TP- 4DISFDL
5

,FJN
%


7J-
cial networks for urban land use clas- TVBM
"OBMZTJT
PG
4PDJBM
.FEJB
%BUB

sification and disaggregation. Comput- Computer, 46 

o
ers, Environment and Urban Systems, 1FOH
%
#JBHJ
-

*UP
6



53
o
Leading countries based on number of
,VSHBO
-


.JMMJPO%PMMBS
monthly active Instagram users as of
#MPDLT
*O
Close up at a distance: map- TU
RVBSUFS

3FUSJFWFE
0DUPCFS


ping, technology, and politics
;POF

GSPN
IUUQXXXTUBUJTUBDPN
#PPLT 5PXOTFOE
"
.


Smart Cit-
-FF
+(

,BOH
.


(FP- ies: Big Data, Civic Hackers, and the
TQBUJBM
#JH
%BUB
$IBMMFOHFT
BOE
0Q- Quest for a New Utopia
/FX
:PSL
8

portunities. Big Data Research
 

8
/PSUPO

$PNQBOZ
*OD
o 8BEEFMM
1
#FSSZ
#
+
-

)PDI

Li, S., Ye, X., Lee, J., Gong, J., & Qin, *


3FTJEFOUJBM
QSPQFSUZ
WBMVFT

$


4QBUJPUFNQPSBM
"OBMZTJT
PG
JO
B
NVMUJOPEBM
VSCBO
BSFB
/FX
FWJ-
)PVTJOH
1SJDFT
JO
$IJOB
"
#JH
%BUB
dence on the implicit price of location.
1FSTQFDUJWF
Applied Spatial Analysis The Journal of Real Estate Finance and
and Policy
 

o Economics, 7 

o
/JFEFSFS
4
$PMPNCP
(
.BVSJ
.
:FNFLTFQFUJOEFO

MF[[FU
BM-

"[[J
.


4USFFUMFWFM
$JUZ
"OB- NBOBʓ‘


3FUSJFWFE
0DUPCFS


MZUJDT
.BQQJOH
UIF
"NTUFSEBN
,OPXM- 
GSPN
IUUQXXXZFNFLTFQFUJ
FEHF
.JMF
*O
Hybrid City 2015: Data to com/
the People
QQ
o

63*"$

Web scraping and mapping urban data to support urban design decisions