DV Reference Architectures

REFERENCE ARCHITECTURE FOR ENTERPRISE DATA LAYER FOR BIG DATA AND ANALYTICS

REFERENCE ARCHITECTURE FOR ENTERPRISE DATA LAYER FOR OPERATIONAL AND

TRANSACTIONAL
 DV Reference Architectures
DATA VIRTUALIZATION USAGE PATTERNS
Data Virtualization technology offers a number of solutions for the major challenges that today’s
businesses are facing. These challenges range from having a Single Customer View across
many different data sources to creating, managing and maintaining Logical Data Warehouses.
Denodo offers solutions providing Agile BI for Big Data environments. These Data Virtualization
solutions aim to deliver real-time information to end users, so they can make more informed
decisions and provide better services.

Horizontal Data Virtualization Solutions Include:

● Agile BI. The Denodo Platform provides Agile BI for enterprise. This is vital as demand
escalates for reports and dashboards with better business insights, more information
sources, real-time data for Operational BI, and more self-service capabilities. Traditional
BI, based mainly on intensive physical data replication, results in complex multi-tier
infrastructures that are not designed to be agile or to deliver real-time business insights
to the organization.
● Big Data solutions. The Denodo Platform offers a common layer to get access to Big
Data solutions like Hadoop, NoSQL and traditional enterprise environments. This means
they can coexist seamlessly, avoiding "yet another silo" when deploying Big Data
solutions. Traditional reporting tools can execute reports through the data virtualization
layer with data coming from Hadoop, NoSQL, etc., using SQL queries – making these
new data repositories accessible to people who haven’t learned the intricacies of writing
Map/Reduce jobs in Hadoop or the API semantics of MongoDB, and so on.
● Data Governance. Data virtualization can be considered a strategic tool in today’s data
integration toolkit. And having a centralized point of access to all kinds of information in
the enterprise brings a multitude of benefits.
● Data Services. Through Denodo Platform it is possible to create a virtual data layer that
exposes data services at enterprise scale to any consuming applications in the
enterprise.
● Logical Data Warehouse. The logical data warehouse augments the capability of an
existing enterprise data warehouse allowing the integration of additional sources (data
warehouse appliances, Hadoop, NoSQL, etc.) and offering greater flexibility to publish
the required data sets in consumable formats to multiple applications.
● Single Customer View. For enterprises, having a single customer view that offers a
360º view of key business entities such as customer, product, patient, etc. is a critical
requirement in many business processes today.
● Web, SaaS and Cloud Integration Services. Denodo advanced data virtualization
technology allows Web, SaaS and Cloud integration for multiple purposes like Web data
extraction, Cloud / SaaS Application Integration, B2B integration and social media
integration.

Data Virtualization for Agile BI

IT cannot keep up with the demands for more timely information from the business. Three
 DV Reference Architectures
months to create an ETL process to load data into a data warehouse is no longer acceptable –
even three weeks is not acceptable! People want to access the data immediately and IT’s
inability to satisfy this need has resulted in ‘shadow IT’ projects which have circumnavigated
normal IT processes to get to information.
The Denodo Data Virtualization Platform enables IT to be more agile in responding to the
businesses almost insatiable demand for actionable information – usually in the form of reports
or dashboards. With the Denodo Platform, the data remains in the source data stores –
replication, with its accompanying staging, transformation, and batch copying tools and
processes, is not required. Access to the data is through virtual views created in the Denodo
Platform – virtual views are lightweight and quickly created (and discarded, if necessary).
Changes are similarly quick, making iterative report and dashboard creation, with almost
immediate involvement and feedback from the business stakeholders, a reality. No more waiting
for the ETL load to be configured and processed before deciding if you’ve got the data that the
business actually wants – the Denodo Platform makes reviews and changes very simple and
quick.

The Denodo Platform also provides a common and consistent view onto the underlying data.
One of the usual complaints from the business users is that reports from different teams –
sometimes using different reporting or visualization tools - often provide conflicting information
when they are ostensibly reports based on the same data. This inability to get a consistent view
of the business through reports and dashboards is a common driver for using data virtualization
in Agile BI projects. Having a single and consistent view of a customer, or product, or an asset
 DV Reference Architectures
is critical if decisions are going to be based on information derived from this data.

Using the Denodo Platform for Agile BI Projects Can:

● Significantly minimizes replication: from the data virtualization layer the user can point to
the original data sources, partially cache any data and also build “virtual data marts” that
are defined in the data virtualization layer thus avoiding the creation of new repositories
(and yet more copies of the data).
● Make it easier to change views and be more reactive to business requests: changes are
carried out in the data virtualization layer, avoiding modifying ETL scripts across the
whole replication chain.
● Access to real-time data for Operational BI by enabling direct access to operational
systems.
● Provide a shorter time to market for delivering reports to users.
● Integrate of any type of data from semi-structured, to structured and unstructured.
● Enable self-service BI: the users can run any report against “self-service ready” data
services created and managed by IT.

Data Virtualization for Big Data

The Denodo Platform supports many patterns, or use cases, with Big Data – whether with
Hadoop distributions (Cloudera, Hortonworks, Amazon’s Elastic Map reduce on EC2, etc.) or
NoSQL data stores such as MongoDB, Cassandra, Neo4j, Aerospike, and so on. These newer
data stores do not typically provide a standard JDBC/ODBC-based SQL interface which make
them difficult to use for BI and reporting tools and for data analysts who are familiar with SQL
queries. The Denodo Platform provides an abstraction and federation layer that hides the
complexities of the Big Data stores and makes it easy to integrate data from these stores with
other data within the enterprise.
The Denodo Platform supports a number of Big Data patterns, including:

Hybrid Data Warehouse

Also called ‘Data Warehouse Offloading’ or ‘Horizontal Partitioning’, this offloads older, less
frequently accessed data from the data warehouse into cheaper commodity storage, such as
Hadoop HDFS. The Denodo Platform sits on top of both data stores and federates queries
across both so that the hybrid data warehouse appears as a single data store to users. The
Denodo Platform optimizes queries so that unnecessary query branches are ignored to provide
better performance to allow you to meet your SLAs.
 DV Reference Architectures

Data Lakes and Enterprise Data Hubs

Both of these patterns involve storing data in a central Hadoop data store. This makes it
cheaper to store massive amounts of data in a central repository and make it available
throughout the organization.
However, the users still need to access the data with their existing tools – whether using
JDBC/ODBC for SQL queries with BI and reporting tools, SOAP/XML Web Services for
ESB/BPM systems, RESTful Web Services for mobile or web applications, and so on.
The Denodo Platform provides a standardized access layer for data stored in a data lake or
enterprise data hub. It abstracts the complexities of the data lake or enterprise data hub
interfaces to present the data as standards-based protocols such as SQL (JDBC, ODBC, and
ADO.NET), Web Services (SOAP/XML and REST), and Web Parts for Microsoft SharePoint
integration. The Denodo Platform also makes it easy to integrate data in the data lake with data
from other data sources, such as external Web services, Web data, operational databases,
enterprise applications, and so on.
 DV Reference Architectures

Analytical Data Integration

Hadoop is great for cost and time-effectively performing analysis over huge data sets –
extracting critical insights from sensor data, clickstream data, mobile data, etc. However,
performing the analysis to get these insights is only the first step in taking advantage of the
benefits of Hadoop and Big Data generally.
Once you have performed the analysis and got the results that you were looking for, you need
to turn this into actionable information. Typically, this means integrating the analytics results with
other information with your enterprise – information from, say, your CRM system or your Order
Management System.
This is where the Denodo Platform comes into play. The Denodo Platform makes it quick and
easy to integrate the data from Hadoop with data from more traditional data sources – such as
data warehouses, operational databases, enterprise applications (on-premise or Cloud-based),
and so on. The Denodo Platform exposes this integrated data as services to consuming
applications – BI reporting tools, dashboards, Excel, web applications, mobile applications, etc.
– making it easy to turn the analytics results into actionable information that can be used by the
business stakeholders.
 DV Reference Architectures

Big Data as an Analytics Sandbox

Your data warehouse is a critical part of your data architecture. It’s designed to provide timely
response to queries for analysis and reporting. The last thing that you want is a business
analyst unleashing complex, resource consuming queries as he ‘plays’ with the data – this sort
of thing can disrupt the smooth running of the data warehouse and threaten your SLAs.
You need to move the business analyst to a sandbox where they can play with all the data
without affecting the performance and stability of your data warehouse operations. This is where
Hadoop and the Denodo Platform come in…the required data is offloaded from the data
warehouse into low cost storage in Hadoop, saving the data warehouse from additional
unplanned work load. The Denodo Platform provides a standards-based interface to the
offloaded data in Hadoop, allowing the business analyst to use his usual tools to perform the
analysis of the data. The Denodo Platform supports JDBC and ODBC APIs allowing the analyst
to use the usual reporting and visualization tools together with programming languages such as
‘R’ to develop new analytical applications.
 DV Reference Architectures

Data Virtualization for Data Governance

Data virtualization can be considered a strategic tool in today’s data integration toolkit. And
having a centralized point of access to all kinds of information in the enterprise brings a
multitude of benefits.
● Security Management: Data virtualization offers a common security management
capability – including definition of users/roles (or importing them from LDAP/AD), fine
grained access restrictions (down to the data cell level), data masking, custom security
policy enforcement, etc.
● Data Governance: Central definition of the business glossary with advanced data
lineage, change impact analysis, catalog search, etc.
● Performance Monitoring: End-to-end performance monitoring from sources to
consumers to detect bottlenecks that can hinder performance.
● Auditing: Tracking all data access activity, so the access to the information can be
properly audited (“who”, “what” and “when” access to any piece of information).
 DV Reference Architectures

Data Virtualization for Data Services

With the Denodo Platform, it's possible to create a virtual data layer that exposes the underlying
data sources as Data Services to any consuming applications in the enterprise – for both
operational and informational purposes. This service-oriented approach to data has numerous
benefits, including:
● A consistent view onto the underlying data sources, reducing the confusion caused by
having reports showing different results because the report developers interpreted the
data differently.
● Support for multiple access protocols so that the consuming application or system can
access the data service in the manner that is best for them. For example, a reporting tool
can access the service using an ODBC connection and an ESB can access the same
service (and same data) using a SOAP/XML Web service.
● A single control point for security and governance. Being able to see who accessed the
data (via the data service), when they accessed the data, and what queries did they
perform (and result sets returned) is important in all industries and is critical in regulated
industries dealing with private or confidential data. The Denodo Platform provides the
control point for enforcing and auditing data access through the exposed data services.
● In addition, the Denodo Platform has a unique Linked Data Services capability which
allows users to browse, search, and query the virtual data model to discover the data
that they need.
 DV Reference Architectures

Data Virtualization for Logical Data Warehouse

Mark Beyer of Gartner introduced the term Logical Data Warehouse and defined it as
"a new data management architecture for analytics which combines the strengths of
traditional repository warehouses with alternative data management and access
strategies."
Gartner predicts that the Logical Data Warehouse will form a new best practice by the end of
2015.

Data Virtualization is a critical part of the Logical Data Warehouse architecture enabling queries
to be federated across multiple data sources – both traditional structured data sources, such as
databases, data warehouses, etc., and less traditional data sources, such as Hadoop, NoSQL,
Web Services, SaaS applications, and so on – while still appearing as a single ‘logical’ data
source to the user.
 DV Reference Architectures

The Denodo Platform provides query federation capabilities to support Logical Data Warehouse
architecture while also isolating the user from the complexities – different technologies, formats,
protocols, locations, etc. – of underlying data sources and provides a common and consistent
view into this data.

Many Logical Data Warehouse implementations augment and extend existing enterprise data
warehouse systems with data from additional sources, such as Hadoop or NoSQL databases,
and offer tremendous flexibility to publish the augmented information in many different formats
to suit the consuming applications.

Data Virtualization for Single View of Customer

For most organizations having a single view of customer data is a critical process
requirement. Whether this is patient information in a health care organization or customer data
in a consumer focused organization, having a 360º view of this data is critical.
The Denodo Platform makes it easy to connect to all of the data source containing customer
data – this could be the CRM system, billing system, logistics system, customer services
incident management system, and so on – and create a single view of this data on demand. The
data can be used in operational scenarios (e.g. providing a 360º view of the customer to a call
center agent) or in analytical scenarios (looking at all of the customer data for trend analysis).
 DV Reference Architectures

Data Virtualization for Web Automation

Denodo advanced Data Virtualization technology supports Web, SaaS and Cloud integration
for multiple purposes:
● Web Data Extraction: For competitive intelligence, compliance, market research, etc.
For example, extracting competitive pricing or special offer information from a
competitor’s web site and making this available to a call center agent who is handling a
customer retention issue. This information might allow the agent to provide a matching
offer to retain the customer.
● Cloud/SaaS Application Integration: When moving to a Cloud-based application, this
can help to integrate traditional on-premises data with Cloud/SaaS data.
● B2B Integration Through Web Automation: Automating Web-based B2B processes
with Denodo advanced Web automation capabilities.
● Social Media integration: Integrating information from LinkedIn, Facebook, Twitter, etc.
for enhancing customer information. For example, adding Facebook posts to customer
information to be displayed to a call center agent – seeing the posts might help the agent
to understand how customer’s disposition towards the organization or its products.
DV Reference Architectures