MDE

Towards the model-driven

engineering of open data
applications as data services
ngel Mora Segura
Jess Snchez Cuadrado
Juan de Lara
ODaaS
Modelling and Software Engineering Group
Department of Computer Science
Universidad Autnoma de Madrid (Spain)
OpenData
The 6th Workshop on
Service oriented Enterprise Architecture for Enterprise Engineering
in conjunction with EDOC2014
MDE
ODaaS
MDE is a development paradigm exploiting the use of domain models to raise
the level of abstraction and automation at which software is developed.
MDE is based on two key elements.
1. Meta-modelling denes the structure and semantics of domain
models.



Model-Driven Engineering
2
Object Diagram
User dened model
Metamodel
Meta-metamodel
<<instance of>>
<<instance of>>
<<instance of>>
MDE
ODaaS
MDE is a development paradigm exploiting the use of domain models to raise
the level of abstraction and automation at which software is developed.
MDE is based on two key elements.
2. Transformations permit the automatic model manipulation for
different purposes



Model-Driven Engineering
3

Refactoring

Model querying

Code generation

Language mappings

Conversions between tech. spaces

Model A Model B
Metamodel A Metamodel B
<<instance of>> <<instance of>>
model
transformation
OpenData
ODaaS
The Data-as-a-Service (DaaS)
approach is a variant of the
Servi ce-Ori ent ed Comput i ng
paradigm that allows suppliers to
expose their internal data using a
richer data model than the one in
which the data is originally stored.
What does DaaS mean?
4
OpenData
ODaaS
The Data-as-a-Service (DaaS)
approach is a variant of the
Servi ce-Ori ent ed Comput i ng
paradigm that allows suppliers to
expose their internal data using a
richer data model than the one in
which the data is originally stored.
What does DaaS mean?
Open Data
[if is freely used
and distributed
then]
5
OpenData
ODaaS
The Data-as-a-Service (DaaS)
approach is a variant of the
Servi ce-Ori ent ed Comput i ng
paradigm that allows suppliers to
expose their internal data using a
richer data model than the one in
which the data is originally stored.
What does DaaS mean?
Open Data
[if is freely used
and distributed
then]
[if data are related to government behavior
or the performance of public services
then]
Open
Government
6
OpenData
ODaaS
The Data-as-a-Service (DaaS)
approach is a variant of the
Servi ce-Ori ent ed Comput i ng
paradigm that allows suppliers to
expose their internal data using a
richer data model than the one in
which the data is originally stored.
What does DaaS mean?
Challenges
!
Lack of an underlying data model
!
Heterogenity of data and sources
!
Dynamic nature in some cases
!
Unavailability
7
OpenData
ODaaS
The Data-as-a-Service (DaaS)
approach is a variant of the
Servi ce-Ori ent ed Comput i ng
paradigm that allows suppliers to
expose their internal data using a
richer data model than the one in
which the data is originally stored.
What does DaaS mean?
Open Data Applications
need to be handcrafted
by skilled technical
people
8
Challenges
!
Lack of an underlying data model
!
Heterogenity of data and sources
!
Dynamic nature in some cases
!
Unavailability
Problem
OpenData
ODaaS
Requirements of our framework
9
Open Data Applications
need to be handcrafted
by skilled technical
people
To avoid
this problem
OpenData
ODaaS
Requirements of our framework
!
Dealing with heterogeneous
data formats.
We have to get a mechanisms to
facilitate data processing and
giving data a homogenous
structure are needed for data
consumers to make the most of
open data possibilities.

10
Open Data Applications
need to be handcrafted
by skilled technical
people
To avoid
this problem
OpenData
ODaaS
Requirements of our framework
!
Semantic data domains.
Semantically rich data domains
must be dened in formalisms
that facilitate mapping existing
data to them, as well as creating
new data domains, possibly
derived from the existing ones.
11
Open Data Applications
need to be handcrafted
by skilled technical
people
To avoid
this problem
OpenData
ODaaS
Requirements of our framework
!
User interaction.
Applications with rich data
analysis and visualization means
(e.g., geolocalization of elements
in interactive maps) need to be
created for people to be able to
make use of open data.
12
Open Data Applications
need to be handcrafted
by skilled technical
people
To avoid
this problem
OpenData
ODaaS
What are we proposing?
!
Dealing with heterogeneous data
formats. Mechanisms to facilitate data
processing and giving data a homogenous
structure are needed for data consumers to
make the most of open data possibilities.
!
Semantic data domains. Semantically rich
data domains must be dened in formalisms
that facilitate mapping existing data to them,
as well as creating new data domains,
possibly derived from the existing ones.
!
User interaction. Applications with rich
data analysis and visualization means (e.g.,
geolocalization of elements in interactive
maps) need to be created for people to be
able to make use of open data.
13
OpenData
ODaaS
What are we proposing?
!
Dealing with heterogeneous data
formats. Mechanisms to facilitate data
processing and giving data a homogenous
structure are needed for data consumers to
make the most of open data possibilities.
!
Semantic data domains. Semantically rich
data domains must be dened in formalisms
that facilitate mapping existing data to them,
as well as creating new data domains,
possibly derived from the existing ones.
!
User interaction. Applications with rich
data analysis and visualization means (e.g.,
geolocalization of elements in interactive
maps) need to be created for people to be
able to make use of open data.
Make data sources uniform
through models
Excel, XML, CSV, PDF
APIs and Queries
Relational data bases
Streaming data, like sensors
14
OpenData
ODaaS
What are we proposing?
!
Dealing with heterogeneous data
formats. Mechanisms to facilitate data
processing and giving data a homogenous
structure are needed for data consumers to
make the most of open data possibilities.
!
Semantic data domains. Semantically rich
data domains must be dened in formalisms
that facilitate mapping existing data to them,
as well as creating new data domains,
possibly derived from the existing ones.
!
User interaction. Applications with rich
data analysis and visualization means (e.g.,
geolocalization of elements in interactive
maps) need to be created for people to be
able to make use of open data.
Make data sources uniform
through models
Personal information
Temporal information
Geopositions
Give semantics to the data
through
domain metamodels
15
Transportations
<<instance of>>
<<instance of>>
OpenData
ODaaS
What are we proposing?
!
Dealing with heterogeneous data
formats. Mechanisms to facilitate data
processing and giving data a homogenous
structure are needed for data consumers to
make the most of open data possibilities.
!
Semantic data domains. Semantically rich
data domains must be dened in formalisms
that facilitate mapping existing data to them,
as well as creating new data domains,
possibly derived from the existing ones.
!
User interaction. Applications with rich
data analysis and visualization means (e.g.,
geolocalization of elements in interactive
maps) need to be created for people to be
able to make use of open data.
Make data sources uniform
through models
REST service with a console Make the system accessible
for other users
Give semantics to the data
through
domain metamodels
16
Domain
injectors
M d id
Madrid
Concerts
Functional
Sources
File-based
sources
(XML, Excel,
CSV,)
Relational
sources Gene
Servi
Dom
Spec
Serv
On-deman

Heterogeneous data sources

Madrid
Bus
Dynamic
Sources
(web services) Doma
Injectio
London
Metro
Model
exploration
console
Domain
Metamodels
Generic
Metamodels
S
T

A
P
I
eric
ices
main
cific
ices
nd
for
for
instance of
instance of

Models
Meta-Modelling
Infrastructure
R
E
S
Domain
Specific
Applications
ain
on
OpenData
ODaaS
What are we proposing?
Architecture of our approach
17
Domain
injectors
M d id
Madrid
Concerts
Functional
Sources
File-based
sources
(XML, Excel,
CSV,)
Relational
sources Gene
Servi
Dom
Spec
Serv
On-deman

Heterogeneous data sources

Madrid
Bus
Dynamic
Sources
(web services) Doma
Injectio
London
Metro
Model
exploration
console
Domain
Metamodels
Generic
Metamodels
S
T

A
P
I
eric
ices
main
cific
ices
nd
for
for
instance of
instance of

Models
Meta-Modelling
Infrastructure
R
E
S
Domain
Specific
Applications
ain
on
OpenData
ODaaS
What are we proposing?
Architecture of our approach
To import data into the MDE technical space,
we use domain injectors.
18
Platform: File system, web service
Format: XML, JSON,
Model (e.g., transportation). +
Giving semantics
Domain
injectors
M d id
Madrid
Concerts
Functional
Sources
File-based
sources
(XML, Excel,
CSV,)
Relational
sources Gene
Servi
Dom
Spec
Serv
On-deman

Heterogeneous data sources

Madrid
Bus
Dynamic
Sources
(web services) Doma
Injectio
London
Metro
Model
exploration
console
Domain
Metamodels
Generic
Metamodels
S
T

A
P
I
eric
ices
main
cific
ices
nd
for
for
instance of
instance of

Models
Meta-Modelling
Infrastructure
R
E
S
Domain
Specific
Applications
ain
on
OpenData
ODaaS
What are we proposing?
Architecture of our approach
Data can be dynamically fetched from data sources on demand.
19
Retrieving data
A data retrieval component activates the corresponding domain
injector to retrieve the data from the external source when data
is needed.
Domain
injectors
M d id
Madrid
Concerts
Functional
Sources
File-based
sources
(XML, Excel,
CSV,)
Relational
sources Gene
Servi
Dom
Spec
Serv
On-deman

Heterogeneous data sources

Madrid
Bus
Dynamic
Sources
(web services) Doma
Injectio
London
Metro
Model
exploration
console
Domain
Metamodels
Generic
Metamodels
S
T

A
P
I
eric
ices
main
cific
ices
nd
for
for
instance of
instance of

Models
Meta-Modelling
Infrastructure
R
E
S
Domain
Specific
Applications
ain
on
OpenData
ODaaS
What are we proposing?
Architecture of our approach
! The system is accessible as a REST service.
! There is a console to discover and use the different data
services, and domain-specic applications.
20
REST Api + Console
Domain
injectors
M d id
Madrid
Concerts
Functional
Sources
File-based
sources
(XML, Excel,
CSV,)
Relational
sources Gene
Servi
Dom
Spec
Serv
On-deman

Heterogeneous data sources

Madrid
Bus
Dynamic
Sources
(web services) Doma
Injectio
London
Metro
Model
exploration
console
Domain
Metamodels
Generic
Metamodels
S
T

A
P
I
eric
ices
main
cific
ices
nd
for
for
instance of
instance of

Models
Meta-Modelling
Infrastructure
R
E
S
Domain
Specific
Applications
ain
on
MDE
ODaaS
What are we proposing?
Architecture of our approach
21
How we use MDE elements
!
Characterizing the data.
!
Validation.
!
Checking data conformance to integrity constraints.
!
Transformation: data mappings.
Domain
injectors
M d id
Madrid
Concerts
Functional
Sources
File-based
sources
(XML, Excel,
CSV,)
Relational
sources Gene
Servi
Dom
Spec
Serv
On-deman

Heterogeneous data sources

Madrid
Bus
Dynamic
Sources
(web services) Doma
Injectio
London
Metro
Model
exploration
console
Domain
Metamodels
Generic
Metamodels
S
T

A
P
I
eric
ices
main
cific
ices
nd
for
for
instance of
instance of

Models
Meta-Modelling
Infrastructure
R
E
S
Domain
Specific
Applications
ain
on
MDE
ODaaS
What are we proposing?
Architecture of our approach
Data sets from different sources
22
Making data compatible and homogeneous
Services are dened generically, based on the concepts of the domain.
Same meta-model
map to
visualization,
interaction,
MDE
ODaaS
Domain modelling
Geoposition Temporal
Transport
Madrid Bus
Night Lines
London
Metro
Generic
Meta-models
Domain
Meta-models
Domain
Models

instance of instance of
instance of instance of
Night Lines Metro
23
Data sets from different sources
Services are dened generically, based on the concepts of the domain.
Same meta-model
map to
visualization,
interaction,
Architecture of our approach
Making data compatible and homogeneous
MDE
ODaaS
Domain modelling
Place
Transp
S i
LineGroup:
Region
Line:
Region
Stop:Place
Coordinates
StopLocation: Coordinates
Geographical
Element
Region
contains
*
*
descBy
stops:
descBy
1..*
0..1
at
groups
lines:
contains
1..*
1..*
lat: double
long: double
name: String
Geoposition, potency *
Transport, potency *
instance of
lineId: String
Service
MadridBus:
TranspService
NightLines:
LineGroup
N13:Line
Cibeles: Stop
:stops
lat= 40.419167
long= -3.693056
StopLocation: Coordinates
CibelesLoc: StopLocation
coords: at
name= Cibeles
Madrid bus, potency 0
instance of
lineId=N13
24
Architecture of our approach
Making data compatible and homogeneous
Geoposition Temporal
Transport
Madrid Bus
Night Lines
London
Metro
Generic
Meta-models
Domain
Meta-models
Domain
Models

instance of instance of
instance of instance of
Night Lines Metro
MDE
ODaaS
Domain modelling
Place
Transp
S i
LineGroup:
Region
Line:
Region
Stop:Place
Coordinates
StopLocation: Coordinates
Geographical
Element
Region
contains
*
*
descBy
stops:
descBy
1..*
0..1
at
groups
lines:
contains
1..*
1..*
lat: double
long: double
name: String
Geoposition, potency *
Transport, potency *
instance of
lineId: String
Service
MadridBus:
TranspService
NightLines:
LineGroup
N13:Line
Cibeles: Stop
:stops
lat= 40.419167
long= -3.693056
StopLocation: Coordinates
CibelesLoc: StopLocation
coords: at
name= Cibeles
Madrid bus, potency 0
instance of
lineId=N13
Multi-level modelling*
allow a stratied
classication of
concepts and their
successive
renement.

25
Architecture of our approach
Making data compatible and homogeneous
* The Essence of Multilevel Metamodeling. Atkinson, Khne, 2001
MDE
ODaaS
Domain modelling
Place
Transp
S i
LineGroup:
Region
Line:
Region
Stop:Place
Coordinates
StopLocation: Coordinates
Geographical
Element
Region
contains
*
*
descBy
stops:
descBy
1..*
0..1
at
groups
lines:
contains
1..*
1..*
lat: double
long: double
name: String
Geoposition, potency *
Transport, potency *
instance of
lineId: String
Service
MadridBus:
TranspService
NightLines:
LineGroup
N13:Line
Cibeles: Stop
:stops
lat= 40.419167
long= -3.693056
StopLocation: Coordinates
CibelesLoc: StopLocation
coords: at
name= Cibeles
Madrid bus, potency 0
instance of
lineId=N13
Operations dened over
the top-most meta-
models become
applicable any number
of meta-levels below.
26
Architecture of our approach
Making data compatible and homogeneous
MDE
OpenData
ODaaS
Prototype implementation
27
MDE
OpenData
ODaaS
Prototype implementation
!
Model-to-model
transformations
!
In-place
transformations
!
Queries
!
Code generators
28
METADEPTH
MDE
OpenData
ODaaS
Prototype implementation
Data services are
exposed using a
REST API.
29
Jersey
MDE
OpenData
ODaaS
Prototype implementation
! Explore the
different data
services
! Invoking its
queries
! Browsing the
resulting models
30
Generic Web Application
MDE
OpenData
ODaaS
Related work
It is a data access protocol which uses an object-oriented entity-
relationship model, and denes a mapping between CRUD
operations and a REST service to access the data model.
31
Open Data Procotol (OData)
MDE
OpenData
ODaaS
Related work
Ontologies could be used as an alternative to multi-level
modelling. However, by remaining in the MDE technical space,
we can use languages and tools for model manipulation, like
transformations, code generation, and in-place transformations.
32
Ontologies
MDE
OpenData
ODaaS
Related work
Some efforts for the systematic use of open data are starting to
emerge. Some tools, like Any23, inject data from different
sources (CSV and HTML micro-formats) into RDF triples.
However, these tools are not meant to discover the underlying
structure of the data, validate it, or synthesize a full interactive
application.
33
Tools
MDE
OpenData
ODaaS
Conclusions

The approach is based on multi-level modelling to

represent domain knowledge.

It supports on demand data loading through domain

injectors, which are described through semantic-rich
query descriptions.

We believe an MDE-based approach for ODaaS

applications is pertinent, as it makes it possible to assign
semantics to data by promoting data as semantically rich
models.

MDE enables the generation of domain-specic

applications for different platforms, like the web or mobile
devices.
34
MDE
OpenData
ODaaS
Future work
!
Complete the framework for generating domain-
specic applications on top of the Data Services
using MDE.
!
A deeper study of the integration of dynamic and
static data sources.
!
Interoperability of our frameworks with OWL
ontologies, and RDF triplestores.
35
MDE
OpenData
ODaaS
Some References
J. de Lara and E. Guerra. Deep meta-modelling with METADEPTH. In
TOOLS10, volume 6141 of LNCS, pages 120. Springer, 2010.
A. Gregory. Open data and metadata standards: Should we be satised
with good enough?, 2011. Open Data Foundation.
D. Lathrop and L. Ruma. Open Government: Collaboration,
Transparency, and Participation in Practice. OReilly, 2010.
The Open Data Protocol (OData). http://www.odata.org/.
D. Varr. Model transformation by example. In MoDELS, volume 4199 of
LNCS, pages 410424. Springer, 2006.
M. Vlter and T. Stahl. Model-driven software development. Wiley, 2006.
36

Any question?
MDE
OpenData
ODaaS
37
MDE

Towards the model-driven
engineering of open data
applications as data services
ngel Mora Segura
Jess Snchez Cuadrado
Juan de Lara
ODaaS
OpenData
38
Modelling and Software Engineering Group
Department of Computer Science
Universidad Autnoma de Madrid (Spain)
http://angelmorasegura.wordpress.com/