Progressing Higher Goals in Data Analysis

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 9

r

d
Conference of Academies for Applied Studies in Serbia (CAASS)
University Business Academy in Novi Sad, Faculty of Contemporary
Arts, Belgrade

BOOK OF PROCEEDINGS
International Multidisciplinary Conference
"Challenges of Contemporary Higher Education" - CCHE 2024
Kopaonik January 29th - February 3rd 2024
Vol_2

edited by:

Dr Branko Savić, professor

Belgrade, 2024
International Multidisciplinary Conference
"Challenges of Contemporary Higher Education" - CCHE 2024

Impressum
Book of Proceedings, International Multidisciplinary Conference
"Challenges of Contemporary Higher Education" - CCHE 2024
Kopaonik January 29th - February 2nd 2024 https://cche.rs/

Editor
Dr Branko Savić, professor

Publisher
Conference of Academies for Applied Studies in Serbia (CAASS)
Bulevar Mihaila Pupina 2,, Novi Beograd, Serbia https://www.caass.rs/
University Business Academy in Novi Sad
Faculty of Contemporary Arts, 12 Svetozara Miletića Street, Belgrade, Serbia www.fsu.edu.rs

Cover design
Information Technology School ITS -Belgrade

Composition
Conference of Academies for Applied Studies in Serbia (CAASS)
Information Technology School ITS -Belgrade

Print
The Higher Education Technical School of Professional Studies in Novi Sad, Serbia

Circulation
100 copies

Publication year
2024

First edition

ISBN-978-86-82744-03-0 (tom)
ISBN-978-86-82744-00-9 (niz)
International Multidisciplinary Conference "Challenges of
Contemporary Higher Education" - CCHE 2024

PROGRESSING HIGHER GOALS IN DATA ANALYSIS


dr Ivan Šećerov 1, dr Jelena Dunjić1, dr Daniela Arsenović 2, dr Stevan Savić1, dr Dragan Milošević 3, dr Srđan Popov 4

Abstract: Understanding better climate conditions which shape urban environment, its influence on local
citizen, has become an increasing need. Large time series acquisition, detecting and measuring urban climate
peculiarities, analyzing large amount of data is challenge of its own. During last decade, different systems were
developed under one combined name - NSUNET. Its achievements, approach and pitfalls were used to expand
knowledge through 'Big data analysis' course. Bringing students closer to the development of programming tools
used for large time series analysis which could find its place in back-end development, turned out to be worthy
goal.

Key words: urban climate, urban meteorological networks, wireless sensor networks, data analysis, NSUNET,
Novi Sad

1. INTRODUCTION
Due to the intensive growth of urban population over past century [1], understanding threads
interconnecting all pieces which combine urban climate, becomes more essential than ever. Heat
waves occurrences become, unfortunately, more frequent, blending in and becoming in many cities
around globe, the fact in everyday life [2]. Wide range of predictions about how hot summers become
and how hot it would be, are published within scientific literature [3], with increasing intensity.
Unfortunately, heat weaves do not present only discomfort in life of urban citizen, they become causes
of environmentally related deaths globally [4]. Scholars around the globe are trying to present different
sustainable cooling strategies [5] mainly how to mitigate this effect within current urban
establishments, while urbanization remains only possibility.
To understand climate effects which surround us and to analyze their peculiarities, science
requires wide range of data. One, of the common methods to observe Earth, remote sensing, now days
mostly utilize satellite sensor systems which provide data from emitted, reflected, and transmitted
energy from all parts of the electromagnetic spectrum [6]. Even though global trends in climate change
are well known, to understand changes in more detail, studies focus also on micro climate. Remote
sensing provides promising results even in this area [7].
But there are still limitations in maximum data resolution provided from this technology [8]. To
allow monitoring urban climate in more details with much higher time resolution, different kind of
surveillance systems are required. Last decades, wide deployment of urban meteorological networks
(UMNs) through the cities around the Globe has been made. Their goal - to resolve understanding of
heterogeneous nature of urban environments on the local climate [9].
Different attempts, to build and deploy, UMNs were made in many countries in the world, with
goal to record influence in urban climate made by vast difference between urban and rural areas.
Oklahoma City Micronet (OKCNET) was only one of examples [10]. As such, attempt to develop and
build combined system through decade of European Union funds (such as Instrument for Pre-
accession Assistance - IPA) were Novi Sad Urban Network (NSUNET).

1
Chair of Geoecology, Faculty of Sciences, University of Novi Sad, Trg Dositeja Obradovića 3, 21000 Novi
Sad, Serbia; e-mail: ivan.secerov@dgt.uns.ac.rs, jdunjic@dgt.uns.ac.rs, stevan.savic@dgt.uns.ac.rs
2
Department of Geography, Tourism and Hotel Management, Faculty of Sciences, University of Novi Sad, Trg
Dositeja Obradovića 3, 21000 Novi Sad, Serbia: e-mail: daniela.arsenovic@dgt.uns.ac.rs
3
Meteorology and Air Quality Group, Hydrology and Environmental Hydraulics Group, Wageningen University
& Research, 6708 PB Wageningen, The Netherlands; e-mail: dragan.milosevic@wur.nl
4
Faculty of Technical Sciences, Chair of Applied Computer Science, University of Novi Sad, Trg Dositeja
Obradovića 6, 21000 Novi Sad, Serbia; e-mail: srdjanpopov@uns.ac.rs

377
International Multidisciplinary Conference "Challenges of
Contemporary Higher Education" - CCHE 2024

2. RELATED WORK BACKGROUND


1.1. Enviroinment monitoring and challenges
Monitoring networks, represents systems built from nodes of different sensor sets [11]. Types and
selection of sensors, is decided based on environment variable which would be measured, level of
accuracy of measured data and available budget. In climate monitoring, some variables happen in
short time interval and are very hard to detect and measure. For example, urban rainfalls which could
lead to urban floods [12], sometimes occur within less than 30 minutes. Short gaps of high wind
velocity are example as well. On the other side, air temperature in urban environment is different. Its
variations take more time. Selecting most suitable monitoring technology from which data analysis, as
final goal of this kind of monitoring, would benefit the most, is formidable task [13]. Some areas, for
which environment variables are being measured, are covered with available power source and data
transfer technologies such as wired based (nowadays mostly used network transmission protocol -
Ethernet). But most of the time, only power source is solar panel combined with battery backup and
only transmission method is some of the widely available radio technologies such as GSM network.
Such scenarios, pose additional challenge to provide reliable power supply and data transfer
mechanism, which would make environment monitoring reliable.
1.2. Wireles sensor networks
Detecting desired environment variables and its occurrences in most scenarios is provided using
sensor network combined of distributed nodes connected with radio technologies. Such topology is
defined as wireless sensor network (WSN) [14]. Monitoring sensor sets are placed in protective
casing, usually with battery backup included (eq. stations). Required power supply, could be available
continuously. Example of this scenario are stations which are connected to public buildings and
urbanized areas. But to monitor rural, and areas which does not fit in this scope, power supply can be
available only for some short period of time (solar panel power supply) [15]. With goal to achieve
better understanding of micro related relations between variables in urban climate, founded through
different IPA Interreg projects, in Novi Sad wider area, three wireless sensor network systems are
deployed in approximately ten years period. Since monitoring networks are built on same principles
and their improvement were based on previous developments, this urban micro meteorological sensor
networks are named NSUNET. First one, was developed through the URBAN-PATH project, with
goal to detect urban heat island effect and its influence on urban and rural area, which covered Novi
Sad wider area. Second one, the URBAN-PREX had goal to successfully detect short and high amount
of precipitation in urban area, while providing constant information about critical water levels on
streets which could form urban floods. Third, the GReENERGY system was developed to provide
valuable data about human thermal comfort in urban area, more specifically on micro level, especially
in period of high air temperatures.

3. CHALLENGES, PITFALLS AND SUCCESS


Since different behavior of monitoring variables, which the NSUNET system is developed to
detect and measure, different challenges had to be solved. Urban Heat Island (UHI), as climatological
effect, experience changes, slowly during 24 hours period of time. Its recording is less demanding then
for example, urban flood detection. But, since URBAN-PATH stations were placed on public lamp-
posts, which provide power supply only during night (and such time varies through seasonal period),
providing continuous power supply to sensor station could be a challenge. Obvious choice is battery
backup-ed power supply. But such technologies, experience decay over time and at one moment,
measuring urban heat island (in case of URBAN-PATH system) would fail. Reliable information
about states on which station algorithm is based, were mandatory. This was achieved using debug
messages accompanied by measured data via data transmission and were analyzed by back-end
software placed on core segment run on servers.
URBAN-PREX has included same idea, but in order to detect variable, which rarely occurs and
can happen only in short period of time, had new problem to be solved. On one side, data transmission
should be infrequent to lower data transmission costs. This was acceptable in case of the URBAN-

378
International Multidisciplinary Conference "Challenges of
Contemporary Higher Education" - CCHE 2024

PATH system. In case of high amount of precipitation,


even if data transmission occurred frequently (in 10-
minutes which was default interval for all mentioned systems), detection of urban floods could be
missed. URBAN-PREX had such challenge to solve. Since its precipitation sensors had hardware
based measured period in approximately 2-minutes, data transmission would occur in default period of
time in periods when precipitation is absent. If precipitation would be detected, data transmission
period would become lower possible interval for which station hardware could measure variables and
prepare them to be transmitted in defined data structure. This working mode would take effect until
there is no precipitation detected.
Development of the GReENERGY system allowed more relaxed approach. Monitoring and
calculating human thermal comfort on micro-scale, for which monitoring stations were placed on
narrow area, was an easier task. Even though, all GReENERGY stations had included most of the
findings from previously mentioned projects.
Common for all mentioned networks, were unfortunately poor selection of hardware components,
especially of GSM modems, which literally lead to entire NSUNET system to become degraded in
such amount, that it stopped providing data during 2023.

4. DATA STRUCTURE AND ANALYSIS


Detected variables from different monitoring conditions, after being measured by sensors, in most
cases are stored on local storage mechanism (for backup purposes and to be prepared for data
transmission). Transmission process can take different forms of communication between segments
(remote - which is represented by measuring stations and core segment, which is represented by server
and storage area). Protocols included in this process could be suitable for binary transfer but also can
be adequate for character-based transfer also. The NSUNET in its implementations had utilized file
transfer protocol (URBAN-PATH) and hypertext transfer protocol (HTTP). Selection was based on
fact that, when data reach core segment, become available to analysis as soon as possible. URBAN-
PATH and URBAN-PREX systems used same data structure. Each measurement was represented as
variable name, followed by ':' character, measured value followed by end of line character '0xa'.
Storing measured data using this mechanism in plain text files allowed easier parsing and further data
analysis, while maintaining measured data human readable. The GReENERGY implemented modified
version of previous example.
Entire core segment performed some predefined operations to check for data errors (whether
made by the sensors or the transmission process). Analysis for missing data was performed in
predefined periods of day. Main goal which had been achieved by the NSUNET system was and still
is, to detect and better understand urban climate peculiarities and combined processes which has direct
influence on urban citizen and to alert on critical urban climate conditions when they occur.
The NSUNET system was founded through three major and one additional IPA Interreg project
which was developed within University of Novi Sad, Faculty of Sciences. To make its discoveries
available for students, NSUNET achievements and problems were integrated in 'Big data analysis'
course. Actual data measured from nearby urban surrounding are used for educational purposes. Since
'Big data analysis' was designed be advanced programming course, students are given opportunity to
learn how to think in order to develop programming code suitable for work with large amount of data
in short period of time. Not all students attending DGTH Geoinformatics classes has same affection
toward IT and its technologies. For that reason, ‘Big data analysis’ course has made as optional one.
To prepare better future students for ‘Big data analysis’, Geoinformatics mandatory course
'Introduction to the programming', has goal to bring closer knowledge about what is computer process
in every operating system and what is actually involving in its work. Using comparison between 'ideas'
within Microsoft Windows and Linux, students could better understand new knowledge.
Student are given opportunity to use programming language for data analysis which suits them
most. Until now, most students have chosen Python (mostly because it is at the moment popular
selection) even though its design place lesser focus on actual processes involved in its code execution.

379
International Multidisciplinary Conference "Challenges of
Contemporary Higher Education" - CCHE 2024

As additional development within NSUNET, offline tool


was developed using Perl, mainly for easier and less error
prone data analysis. Since Perl syntax is closer to C programming, it is easier to rewrite code for time
critical online back-end application when there is need for it. Using actual example of how to 'think
while developing' time critical code involving large amount of data, helps students better understand
how to write their own implementation.
Additional attention is brought, always to ‘think about how reliable data exported for analysis
are’, focus on error detection and defining actual critical states in program algorithm.
2021-05-10 00:00:00,1620604800,25.479167,20.549792,48.00,31.11,8.00,7.94
2021-06-05 00:00:00,1622851200,29.064516,23.645269,47.00,31.50,12.00,12.00
2021-07-06 02:15:00,1625537700,31.117647,27.574314,48.00,33.44,22.00,20.72
2021-08-01 00:00:00,1627776000,30.285714,25.468429,54.00,36.94,21.00,19.94
2021-09-12 00:00:00,1631404800,25.752688,20.486774,48.00,30.50,12.00,11.33
2022-05-12 05:00:00,1652331600,31.434211,25.593289,48.00,32.22,18.00,15.83
2022-06-03 00:30:00,1654216200,28.283019,23.900377,47.00,33.06,20.00,18.50
2022-07-01 00:00:00,1656633600,33.944444,28.643750,54.00,37.50,23.00,22.22
2022-08-01 00:00:00,1659312000,29.797872,25.181809,48.00,32.28,17.00,16.72
2022-09-06 08:15:00,1662452100,43.055556,28.990556,47.00,31.61,34.00,24.11
---------------------------------------------------------------------------------------------------------------------
-----------------
*** CLIMATOLOGY STATS *** hottest day count: [10]
---------------------------------------------------------------------------------------------------------------------
-----------------

** ERROR DETECTED IN CURRENT DATA *** -> 2022-08-01


06:00:00,1659333600,36.00,-17.78

---------------------------------------------------------------------------------------------------------------------
-----------------
*** STATS: *** -> detected measurements: [96]/[66.67%], expected measurements: [144]
---------------------------------------------------------------------------------------------------------------------
-----------------
Table 1. Selected parts from the NSUNET offline tool data output
While 'raw' data analysis is always good approach, special care is given on visual representation
of the analyzed data. The NSUNET, offline tool produce data visualisation in real time.

Figure 1. NSUNET offline tool output data visualization

380
International Multidisciplinary Conference "Challenges of
Contemporary Higher Education" - CCHE 2024

5. CONCLUSION
Even though, at first, purpose for developing systems
through presented the EU projects, was scientific analysis, together with producing micro-climate
measurements, better understanding of urban climate conditions in Novi Sad wider area and critical
conditions alert to local citizens. At first, it was unexpected to combine teaching courses scope with
actual development of meteorological systems. Together with the fact that more students embrace IT
technologies every day, helping them to get involved, even in small parts, with achievements from the
EU projects has turned out to be right path. Bringing students closer to the development of
programming tools used for large time series analysis which could find its place in back-end
development, turned out to be worthy goal. Examples presented in this paper are just main points of
interest in knowledge sharing approach used in 'Big data analysis' course. Continuous work with
students and exploring their creativity can help us to better shape future courses at the Faculty of
Sciences. As finding proper gap filling methodology to fit in recovering the NSUNET missing data
has yet to be implemented in its back-end, introducing students to such area is something to be
implemented in future Faculty`s courses also.

6. REFERENCES
[1] Sharifi, A.; Khavarian-Garmsi, A.; Allam, Z.; Asadzadeh, A.; Progress and prospects in
planning: A bibliometric review of literature in Urban Studies and Regional and Urban
Planning, 1956–2022, in Progress in Planning, Vol. 173, July 2023,
https://doi.org/10.1016/j.progress.2023.100740
[2] Muñoz-Pizza, D, M.; Sanchez-Rodriguez, Roberto A.; Gonzalez-Manzano, Eduardo.: Linking
climate change to urban planning through vulnerability assessment: The case of two cities at
the Mexico-US border, in Urban Climate, Vol. 51, 2023,
https://doi.org/10.1016/j.uclim.2023.101674
[3] Brimicombe, C.; Porter, James J.; Di Napoli, C.; Pappenberger, F.; Cornforth, R.; Petty, C.; L.
Cloke, H.: Heatwaves: An invisible risk in UK policy and research, in Environmental Science
& Policy, Vol. 116, 2021, pp. 1-7, https://doi.org/10.1016/j.envsci.2020.10.021
[4] Jedlovec, G.; Crane, D.; Quattrochi, D.: Urban heat wave hazard and risk assessment, in
Results in Physics, Vol. 7, 2017, pp. 4294-4295, https://doi.org/10.1016/j.rinp.2017.10.056
[5] Jay, O.; Capon, A.; Berry, P.; Broderick, C.; de Dear, R.; Havenith, G.; Honda, Y.; Kovats, S.;
Ma, W.; Malik, A.; B. Morris, N.; Nybo, L.; I. Seneviratne, S.; Vanos, J.; L. Ebi, K.:
Reducing the health effects of hot weather and heat extremes: from personal cooling strategies
to green cities, in Heat and Health, Vol. 398, 2021, pp. 709-724,
https://doi.org/10.1016/S0140-6736(21)01209-5
[6] Estes, J.; Kline, K.; Collins, E.: Remote Sensing, in International Encyclopedia of the Social
& Behavioral Sciences, 2001, pp. 13144-13150, https://doi.org/10.1016/B0-08-043076-
7/02526-2
[7] Zellweger, F.; De Frenne, P.; Lenoir, J.; Rocchini, D.; Coomes, D.: Advances in Microclimate
Ecology Arising from Remote Sensing, in Trends in Ecology & Evolution, Vol. 34, 2019, pp.
327-341, https://doi.org/10.1016/j.tree.2018.12.012
[8] Dubovik, O.; L. Schuster, G.; Xu, F.; Hu, Y.; Bösch, H.; Landgraf, J.; Li, Z.: Grand
Challenges in Satellite Remote Sensing, in Frontiers in Remote Sensing, Vol. 2, 2021,
https://doi.org/10.3389/frsen.2021.619818
[9] L. Muller, K.; Chapman, L.; Grimmond, C. S. B.; T. Young, D.; Cai, X.: Sensors and the city:
a review of urban meteorological networks, in International Journal of Climatology, Vol, 33,
2013, pp. 1585-1600, https://doi.org/10.1002/joc.3678
[10] J. Schroeder, A.; B. Basara, J.; G. Illston, B.: Challenges Associated with Classifying Urban
Meteorological Stations: The Oklahoma City Micronet Example, in The Open Atmospheric
Science Journal, Vol, 4, 2010, pp. 88-100, https://doi.org/10.2174/1874282301004010088
[11] K. Hart, J.; Martinez, K.: Environmental Sensor Networks: A revolution in the earth system
science?, in Earth-Science Reviews, Vol. 78, 2006, pp. 177-191,
https://doi.org/10.1016/j.earscirev.2006.05.001

381
International Multidisciplinary Conference "Challenges of
Contemporary Higher Education" - CCHE 2024

[12] Hu, C.; Liu, C.; Yao, Y.; Wu, Q.; Ma, B.; Jian,
S.: Evaluation of the Impact of Rainfall Inputs on
Urban Rainfall Models: A Systematic Review, in Water, Vol. 12, 2020,
https://doi.org/10.3390/w12092484
[13] Middel, A.; Nazarian, N.; Demuzere, M.; Bechtel, B.: Urban Climate Informatics: An
Emerging Research Field, in Frontiers in Environmental Science, Vol. 10, 2022,
https://doi.org/10.3389/fenvs.2022.867434
[14] Kingsley, E. U.; Ituabhor, O.; Silas S., T.; Rout, G. K.; Akinola S. O.; Ayodotun, O. B.:
Wireless Sensor Networks: Applications and Challenges, in Wireless Sensor Networks -
Design, Deployment and Applications, 2020, IntechOpen,
https://doi.org/10.5772/intechopen.93660
[15] Guo, W.; M. Healy, W.: Power Supply Issues in Battery Reliant Wireless Sensor Networks:
A Review, in International Journal of Intelligent Control and Systems, Vol. 19, 2014, pp. 15-
23

382

You might also like