An approach to monitoring, data analytics, and decision support for

An approach to monitoring, data analytics, and decision support for

An approach to monitoring, data analytics, and decision support
for levee supervision
Marian Bubak1,2, Bartosz Baliś1,2, Daniel Harężlak2, Marek Kasztelnik2, Piotr Nowakowski2
Tomasz Bartyński2, Tomasz Gubala2, Maciej Malawski1,2, Maciej Pawlik2, Bartosz Wilk2
1
AGH University of Science and Technology, Department of Computer Science
2
ACC Cyfronet AGH
Krakow, Poland
emails: {balis,bubak}@agh.edu.pl
Keywords: levee monitoring, natural disasters, flood protection, data analytics
In this paper, we present a concept of a comprehensive software platform facilitating levee
supervision [1] through on-line processing of levee sensor monitoring data, offline data
analytics, and decision support. The platform is developed as part of the ISMOP project [2]
whose objectives span construction of an artificial levee, design of wireless sensors for levee
instrumentation, development of a sensor communication infrastructure, and a software
platform for execution management, data management and decision support.
Execution Platform
Provisioning platform
Composite App
Optimizer & Scheduler
App model
Scaling rules
Autoscaler
Data access (DAP)
Events
Enact
Reconfigure app
App state
HyperFlow Enactment Engine
Initial deployment
Provisioner
Start/Stop/Reconfigure VM
measurements
Cloud
Execute
Input data
Executor
Trigger app execution
VM
VM
VM
VM
VM
Monitoring
Fig. 1. Architecture of the execution and provisioning (EXP) platform.
Fig. 1 presents the execution and provisioning platform (EXP), the first part our
solution supporting execution and management of complex distributed applications. While
the EXP platform is generic, in the ISMOP project it will be responsible for hosting an
application responsible for online levee monitoring. This application continuously fetches
current monitoring data collected from in-situ sensors, and coordinates the execution of
various analytical modules, depending on the emergency level. The platform will be
designed for monitoring very large areas spanning many kilometers of levees. Consequently,
it will be capable of hosting many instances of the levee monitoring application
(responsible for different sections of levees), dynamic provisioning of cloud resources, and
autoscaling based on current resource demands. The need for such an architecture stems
from the fact that while most of the time (for low emergency levels) the monitoring
applications will work in a standby mode demanding only little amount of computing
resources, in high-emergency situations their resource demands may rapidly grow, as new
computing and data-intensive analyses will be enabled. The solution is inspired by our work
on the Common Information Space for early warning systems against natural disasters [3,4].
From sensor network communication infrastructure
Sensor data collection
Apache Flume
propagation
propagation
DB interfaces
Time Series DB
GIS database
HDFS
Namenode
Storage node
2nd NN
Storage node
Offline data analytics
Storage node
synchronization
Online data access
API
Hive
Pig
Data access
To application services
Fig. 2. Architecture of the data access and analytics platform (DAP).
The second part of our solution is a data access and analytics platform (DAP, Fig. 2),
responsible for storage and access to various data sets including geospatial data, sensor data,
and metadata. The DAP platform also includes a big data infrastructure which is necessary
for data-intensive online analytics scenarios, one of which includes searching through a very
large collection of data sets representing various simulated levee monitoring scenarios as
a basis of current risk prediction algorithm.
Finally, the third part of the solution is a visualization and decision support system
responsible for visualization of levee state, results of risk assessments and predictions, as
well as dissemination of warnings.
Acknowledgment: This work was supported by the ISMOP project [2]. The authors are
very grateful to the members of this project for many inspiring discussions and suggestions.
References
1.
2.
3.
4.
International Levee Handbook, http://www.leveehandbook.net/
Project ISMOP: Informatyczny System Monitorowania Obwałowań Przeciwpowodziowych
http://www.ismop.edu.pl/
Bartosz Balis, Tomasz Bartynski, Marian Bubak, Grzegorz Dyk, Tomasz Gubala, and Marek
Kasztelnik. A Development and Execution Environment for Early Warning Systems for Natural
Disasters. In Cluster, Cloud and Grid Computing (CCGrid), 2013 pp. 575-582. IEEE, 2013.
B. Balis, M. Kasztelnik, M. Bubak, T. Bartynski, T. Gubala, P. Nowakowski, and J. Broekhuijsen.
The UrbanFlood Common Information Space for Early Warning Systems. Procedia Computer
Science, 4:96-105, 2011. Proc. International Conference on Computational Science, ICCS 2011.