Professional Documents
Culture Documents
PID2706771
PID2706771
PID2706771
net/publication/261093626
CITATIONS READS
15 378
6 authors, including:
All content following this page was uploaded by Bartosz Balis on 08 January 2020.
Abstract—Early Warning Systems (EWS) may become a pow- warning systems [2]. We focus on deployment and runtime
erful tool for mitigating the negative impact of natural disasters, capabilities of CIS not described in previous publications and
especially when combined with advanced IT solutions – such as highlight its mission-critical EWS execution features.
on-demand scenario simulations, semi-automatic impact assess-
ment, or real-time analysis of measurements from in-situ sensors. This paper is organized as follows. Section II overviews the
However, such complex systems require a proper computing state of the art. Section III presents the early warning system
environment supporting their development and operation. We software model and runtime components of the Common
propose the Common Information Space (CIS), a software frame- Information Space. In section IV details of CIS solutions re-
work facilitating design, deployment and execution of early warn- garding mission-critical aspects of EWS are presented. Section
ing systems based on real-time monitoring of natural phenomena
and computationally intensive, time-critical computations. CIS V describes the CIS-powered Flood Early Warning System.
provides a service-oriented technology stack which helps design Section VI concludes the paper.
‘blueprints’ for early warning application scenarios and deploy
these blueprints as services – system factories enabling users II. S TATE OF THE A RT
to rapidly deploy new EWSs in new settings. CIS also provides Early warning systems leveraging real-time sensor data and
advanced runtime services and resource orchestration capabilities
in order to address the specific requirements of EWSs: continuous time-critical computing are becoming increasingly popular.
operation, highly variable resource demands and mission-critical One notable example is the Indian Tsunami Early Warning [3]
computations. The CIS concept is validated through the Flood system which combines sensors for earthquake detection,
Early Warning System whose goal is to monitor embankments in ocean-based sensors to detect tsunamis, satellites to correlate
urban areas and assist in rapid decision making whenever a dike weather information and area maps to correctly predict where
failure results in a flooding threat.
the tsunami will strike. The US Geological Survey [4] offers
access to a global network of seismic sensors and performs
I. I NTRODUCTION
data processing in order to generate warnings for concerned
Early warning systems can become a crucial factor in stakeholders. The Fire Warning System [5] developed at the
mitigating the impact of natural disasters on urban envi- University of Freiburg has a similar goal – to deliver a map
ronments. The maturation of in-situ monitoring technologies which plots all forest fires in the shortest time possible.
and ever-increasing supply of cheap computing power brings However, while the aforementioned systems aggregate existing
this vision ever closer to reality. Early warning scenarios resources in order to deliver a high quality early warning
involve real-time analysis of large volumes of data concerning system for a specific domain, they do not attempt to propose
natural phenomena, and predictions based on CPU-intensive a computing environment facilitating the very development
simulations. Fidelity and timeliness are two crucial quality and operation of early warning systems in general.
indicators in effective early warning systems. Consequently, Mission-critical applications, including early warning sys-
such systems require proper IT support at every stage of tems, operate continuously and monitor selected phenomena
their lifetime: (i) tools for planning early warning scenarios in order to invoke resource-demanding and computationally-
at design time; (ii) frameworks facilitating implementation intensive simulations whenever necessary. These computations
of new early warning system blueprints; (iii) technologies must meet strict deadlines – otherwise their results will present
for rapid deployment of EWS blueprints at new locations; no practical value. Therefore they must be hosted in special
(iv) runtime services supporting mission-critical execution of environments that feature urgent computing facilities [6]. Over
EWSes. the last decade a number of attempts have been made to con-
In this paper, we present a concept of a software framework struct such systems on the basis of grid infrastructures. While
facilitating design, deployment and execution of early warning grids can easily provide extensive resources they also have
systems based on real-time monitoring of natural phenomena important drawbacks. Since they rely on queuing systems, it
and computationally-intensive, time-critical computations — is impossible to predict when execution will commence and
The Common Information Space. In our previous paper [1] when all the required resources will become available. The
we described the concept of CIS at its early development latter problem is partially addressed by advanced reservation
stage. This paper presents CIS as a mature concept and a fully mechanisms, enabling users to specify all required resources
implemented system, validated in the context of flood early that have to be available simultaneously for an application to
execute. One example of such a solution is HARC [7]: an infrastructures are garnering interest not only from commercial
open-source system enabling users to reserve different types users but also from scientific communities [13].
of resources (e.g. CPU, GPU, network connections) in a single Although cloud infrastructures are becoming ever more
step. Another example is QosCosGrid [8] middleware, which mature, operating early warning systems on clouds remains
uses, among others, various advance reservation techniques to a great challenge. Problems that need to be addressed include
integrate multiple computing resources into a single powerful specifying the system as a set of cooperating components run-
virtual supercomputer. ning on virtual machines, automatic deployment, monitoring
However in the above mentioned solutions resources may and scaling of resources.
only be reserved following a significant delay, which calls for
a reservation prioritization mechanism. Moreover, an urgent III. T HE C OMMON I NFORMATION S PACE
computation must not wait for a long time in a queue – The Common Information Space is a software framework
rather, it should be invoked immediately. This makes the listed facilitating development, deployment and execution of com-
systems poorly suited to urgent computing tasks. In [9] the plex mission-critical systems which rely on scientific com-
authors present several ways to address these issues (imple- puting, in particular early warning systems protecting against
mented in the SPRUCE Urgent Computing Environment). In natural disasters. This section explains the system model
order to reduce the resource procurement time there is a pool adopted by CIS and describes core CIS runtime services.
of sites with ready-to-use preinstalled applications from which
the least used are chosen for any emerging task. Afterwards, A. Early warning system model
the urgent job is submitted to the corresponding queue. The In complex systems, such as EWS, there is a need to
queuing system may apply various policies with regard to compose and orchestrate diverse and distributed resources in
working jobs: wait for completion, move to another site, force order to enact complex application scenarios. For example,
a checkpoint or even kill the given job to ensure that the urgent a relatively simple scenario – prediction of inundation of
request is served immediately. Still, the presented approach an area threatened by a dike breach – involves at least the
does not take into account external storage resources which following elements: (i) scientific applications implementing
may be extensively used by scientific computations involved computational models of inundation, (ii) computing infrastruc-
in simulations. This deficiency is addressed by the Urgent Data ture (HPC clusters, clouds) capable of running CPU-intensive
Management Framework [10] which focuses on providing jobs; (iii) current dike sensor readings (e.g. water level);
various services, including storage, for urgent computing. (iv) terrain elevation data for the area.
Yet another disadvantage of grid systems is the fact that
every application runs on resources with one specific software
configuration (operating system and a set of libraries and
programs). Therefore it is difficult or even impossible to
satisfy specific requirements of custom applications. Even
if the administrator were to create a pool of customized
workers, the number of such nodes in traditional grids cannot
be managed dynamically. These problems can be solved by
combining grid infrastructures with virtualization technolo-
gies. Each application can run in a virtual machine which may
be customized in terms of operating systems, installed libraries
and tools. The number of such virtual worker nodes can be
increased or decreased to best meet current needs. An example
of such a system is the Virtual Spaces Job Manager [11].
Job Description Language extensions allow users to specify
the required virtual machines and level of urgency. If a task Fig. 1. Complexity of early warning application scenarios, flood simulation
requires resources that are not available at a given moment, example. Composition and orchestration of diverse distributed resources in
a request to start a virtual machine is submitted to a queue. the Common Information Space based on the service orientation approach.
In spite of the described enhancements of grid infrastruc-
tures, they still lack some desired characteristics that are pro- In CIS, we adopt a service orientation approach in which
vided by cloud infrastructures. The former require users to go resources are exposed as services which are, in turn, composed
through a certification process before access is granted and do and orchestrated in order to enact application scenarios (Fig.
not give administrative access to resources. On the other hand 1).
clouds can be used by anyone on a pay-per-use basis. They Fig. 2 provides a more detailed view of the CIS architecture
generate an illusion of infinite resources, making acquisition of and its system model. The architecture proposes a SOA solu-
fully configurable resources easier and faster. The availability tion stack for mission-critical compute-intensive systems, such
of cluster instances in Amazon’s EC2 makes high performance as EWSs. In creating this design, we wanted to take advantage
computing on the cloud possible [12]. As a consequence, cloud of SOA architectural patterns for enterprise applications while
Fig. 2. Common Information Space architecture and early warning system model. Domain resources (lower layer) are exposed as basic services. These can
be composed into system parts – blueprints for early warning application scenarios and building blocks for early warning systems. Early warning system is
a collection of loosely coupled parts configured for a particular setting (e.g. a dike section).
sensor data and periodically publishes messages containing the and implementation of CIS-based EWSs, let us examine thor-
likelihood of an anomaly. All remaining application scenarios oughly the implementation of the Flood Simulation scenario
are implemented as services (system parts). The Dike Stability blueprint. Fig. 9 gives a detailed overview of resources in-
Monitoring Part is event-driven. It continuously consumes volved in the Flood Simulation application scenario. The entry
anomaly probabilities produced by the AI appliance and when point for the scenario is the Flood Simulation Part (a WPS-
the probability exceeds a designated threshold (whose value compliant service). Actual computations are handled by two
is part of the Part’s configuration), it publishes an ‘Alert level appliances encapsulating computational models provided by
change’ message for the dike section where the anomaly has HR Wallingford: Hydrograph (computation of water volumes
been detected (which results in changing the color of the flowing through a breach) and Dynamic RFSM (rapid flood
section in the system control center GUI). Subsequently, the spreading algorithm) [1]. The Flood Simulation service is
Part consumes sensor readings from the affected section of designed so that some input data can be provided as links to
the dike and invokes the Dike Reliability Part in order to external data services, implemented in compliance with OGC
compute the probability of dike failure. When this probability standards: Web Coverage Service (WCS) and Web Feature
again exceeds a given threshold, the alert level is raised again. Service (WFS). This applies to terrain elevation data and data
Reliability analysis is disabled when the AI appliance no describing dike parameters, geometry, etc. In addition, the
longer reports high likelihood of an anomaly for the given output inundation data is archived and exposed through a WCS
dike section. The remaining three simulation parts are request- service.
driven: they are invoked directly from the system control Such compliance with standards increases the interoperabil-
center. ity and reusability of the Flood Simulation service blueprint.
While currently we provide our own in-house implementation
of both WCS services (the WFS service is not currently
implemented), in the future many spatial data sets owned by
public organizations in Europe should be exposed as services,
as dictated by the INSPIRE directive [19]. Our blueprint
provides the opportunity to instantly leverage such services in
order to perform flooding simulations at a selected location.
Another important service is the Data archiver whose task
is to publish the simulation output to the inundation data
repository. To this end, the inundation data sets are published
as a series of messages to the message bus. The data archiver
subscribes to these messages, receives them and updates the
inundation repository. A system control center retrieves the
inundation data from the WCS service and visualizes it on the
map of the affected area. It is interesting to note that loose cou-
pling through the message bus eliminates direct dependencies
Fig. 8. High-level architecture of the Flood Early Warning System powered between services. We might change the configuration of the
by the Common Information Space. system control center so that it would consume the inundation
data directly from the message bus and disable the Data
In order to highlight the most important aspects of design archiver without breaking the system. Similarly, a new service
run software components of the Common Information Space
and the Flood Early Warning System, as well as to per-
form related experiments. The software described in this
paper was developed within the EU FP7 UrbanFlood project
(http://dice.cyfronet.pl/products/cis). This work is partially
supported by the European Union Regional Development
Fund, POIG.02.03.00-00-096/10 as part of the PLGrid Plus
Project. AGH Grant 11.11.230.015 is also acknowledged.
R EFERENCES
[1] B. Balis, M. Kasztelnik, M. Bubak, T. Bartynski, T. Gubała,
P. Nowakowski, and J. Broekhuijsen, “The UrbanFlood Common Infor-
mation Space for Early Warning Systems,” Procedia Computer Science,
vol. 4, pp. 96–105, 2011, proceedings of the International Conference
on Computational Science, ICCS 2011.
[2] V. Krzhizhanovskaya, G. Shirshov, N. Melnikova, R. Belleman, F. Ru-
sadi, B. Broekhuijsen, B. Gouldby, J. Lhomme, B. Balis, M. Bubak,
Fig. 9. Implementation of the Flood Simulation application scenario. A. Pyayt, I. Mokhov, A. Ozhigin, B. Lang, and R. Meijer, “Flood early
warning system: design, implementation and computational modules,”
Procedia Computer Science, vol. 4, pp. 106–115, 2011, proceedings of
the International Conference on Computational Science, ICCS 2011.
could be connected to the message bus at any time, consume [3] T. Kumar, Srinivasa, “Implementation of the indian national tsunami
messages, and perform additional tasks without affecting other early warning system,” in Fostering e-Governance: Selected Com-
pendium of Indian Initiatives, Pune, India, 2009, pp. 380–391.
services. [4] ANSS Technical Integration Committee, Technical Guidelines for the
Implementation of The Advanced National Seismic System. U.S.
VI. C ONCLUSION Department of the Interior, U.S. Geological Survey, 2002.
[5] W. de Groot, J. Goldammer, T. Keenan, M. Brady, T. Lynham, C. Justice,
We presented the Common Information Space, a service- I. Csiszar, and K. O’Loughlin, “Developing a global early warning
oriented software framework for early warning systems guard- system for wildland fire,” Forest Ecology and Management, vol. 234,
ing against natural disasters. CIS supports both EWS devel- no. 1 S 10, p. 15, 2006.
[6] N. Trebon, “Enabling urgent computing within the existing distributed
opment and execution. It provides a framework for creating computing infrastructure,” Ph.D. dissertation, The Univeristy of Chicago,
blueprints for complete early warning systems and individual 2011.
EWS application scenarios. Furthermore, these blueprints can [7] J. Maclaren and M. M. Keown, “Harc: A highly-available robust co-
scheduler, submitted to the 5th uk e-science all hands meeting,” 2006.
be deployed as services and used as factories for new instances [8] B. S. Werner Dubitzky, Krzysztof Kurowski, Large-Scale Computing
of early warning systems or individual services performing Techniques for Complex System Simulations, 1st ed., 2011.
specific tasks. CIS also provides a set of runtime services for [9] S. Marru, D. Gannon, S. Nadella, P. Beckman, D. B. Weber, K. A.
Brewster, and K. K. Droegemeier, “Lead cyberinfrastructure to track
execution of EWSes as mission-critical applications. Runtime real-time storms using spruce urgent computing,” CTWatch Quarterly,
support includes self-monitoring of EWSes, detection of and vol. 4, no. 1, 2008.
recovery from failures, and optimization of resource allocation, [10] J. Cope and H. Tufo, “Adapting grid services for urgent computing
environments,” in Proceedings of the 2008 International Conference on
leveraging cloud infrastructures. Software and Data Technologies (ICSOFT 2008), 2008, pp. 135–142.
Early warning systems feature characteristics of enterprise [11] A. Cencerrado, M. A. Senar, and A. Cortés, “Support for urgent
applications (e.g. continuous monitoring, mission-critical func- computing based on resource virtualization,” in Proceedings of the 9th
International Conference on Computational Science: Part I, ser. ICCS
tion, ad-hoc scenarios) and scientific applications (resource- ’09. Berlin, Heidelberg: Springer-Verlag, 2009, pp. 227–236.
intensive computations). Consequently, in the CIS computing [12] High performance computing on aws. [Online]. Available: http:
environment we combined architectural patterns, industry stan- //aws.amazon.com/hpc-applications/
[13] C. Vecchiola, S. Pandey, and R. Buyya, “High-performance cloud
dards and technologies commonly found in high-quality enter- computing: A view of scientific applications,” in Pervasive Systems,
prise applications with computing infrastructure and resource Algorithms, and Networks (ISPAN), 2009 10th International Symposium
orchestration algorithms typical for e-Science. on. IEEE, 2009, pp. 4–16.
[14] P. Schut, “Open Geospatial Consortium Inc. OpenGIS R Web Processing
The capabilities of the CIS framework have been validated Service,” Open Geospatial Consortium, pp. 1–87, 2007.
for the Flood Early Warning System. However, the scope of [15] G. Hohpe, Enterprise integration patterns : designing, building, and
CIS applicability extends beyond early warning systems for deploying messaging solutions. Boston: Addison-Wesley, 2004.
[16] M. Logan, E. Merritt, and R. Carlsson, Erlang and OTP in Action.
natural disasters. In fact, CIS is suitable for any complex Manning Publications Co., 2010.
mission-critical system levaraging resource-intensive compu- [17] J. Dabrowski, S. Feduniak, B. Balis, T. Bartynski, and W. Funika, “Au-
tations. tomatic Proxy Generation and Load-Balancing-based Dynamic choice
of Services,” Computer Science, vol. 13, no. 3, pp. 45–59, 2012.
[18] A. Pyayt, I. Mokhov, B. Lang, V. Krzhizhanovskaya, and R. Meijer,
ACKNOWLEDGMENT “Machine Learning Methods for Environmental Monitoring and Flood
The results described in this paper have been made Protection,” World Academy of Science, Engineering and Technology
journal, vol. 78, pp. 118–124, 2011.
possible thanks to the access to the the computing re- [19] “Inspire Directive,” http://inspire.jrc.ec.europa.eu/.
sources of the Polish National Grid infrastructure (PL-Grid,
http://www.plgrid.pl) which were used by the authors to