335 BROUGHT TO YOU IN PARTNERSHIP WITH

CONTENTS

•  What Is Distributed SQL?

Distributed SQL •  Fundamentals of Distributed SQL

Architecture
−  Consider Your Workload

Essentials
−  Mixed Models
−  Additional Infrastructure
−  Consider Mixed Workload Support
−  Making and Meeting Performance
Requirements
−  Understanding Consistency
−  Cloud, Home, or Hybrid

ANDREW OLIVER •  Conclusion and Further Reading

SR. DIRECTOR OF PRODUCT MARKETING, MARIADB

Distributed SQL databases combine the resilience and scalability of
a NoSQL database with the full functionality of a relational database.
Since the 1980s, using the relational model and SQL query language has
been the dominant approach that businesses adopt to develop critical
applications.
While the 2010s saw the emergence of databases ranging from NoSQL
to various data stores in the Hadoop ecosystem, most mission-critical
transactional applications have remained on traditional relational
databases like Oracle and SQL Server.
WHAT IS DISTRIBUTED SQL?
Distributed SQL databases are becoming popular with organizations
that are interested in moving data infrastructure to the cloud and/or
cloud-native environments in order to reduce TCO and move away from
the horizontal scaling limitations of monolithic RDBMSs like Oracle,
PostgreSQL, MySQL, and SQL Server.
Traditional relational databases were architected for transactional
integrity, low disk storage, and high performance, all on a single-server
architecture. This requires a larger server or instance whenever there
is an increase in the amount of data, query volume, or data complexity.
Monolithic architectures do not support high availability and disaster
recovery.
Clustering technologies like Oracle RAC and Galera have addressed
some of the issues of high availability, and replication technologies
like Oracle Streams (Golden Gate) have allowed for forms of disaster
recovery and cross-data center replication, but they come at a heavy
cost in both dollars and performance.
Traditional databases added forms of data partitioning to better use
storage hardware, but these partitioning methods increase application

Basic characteristics of distributed SQL:

•  A SQL API for querying and modeling data with support for
traditional RDBMS features like foreign keys, partial indexes,
stored procedures, and triggers

•  An automatic distributed query execution so that no single node

becomes a bottleneck

•  Automatic and transparent distributed data storage, including

indexes sharded across multiple nodes of the cluster, so that no
single node becomes a bottleneck — data distribution ensures
high performance and high availability

•  Replication with strong consistency and distributed ACID

transactions

REFCARD | JANUARY 2022 1

 XPAND YOUR
EXPECTATIONS
Distributed SQL now available in SkySQL

Get started with a $500 credit:

mariadb.com/skyview

SkySQL is the only DBaaS capable of deploying MariaDB as a distributed SQL database for
scalable, high-performance transaction processing or as a multi-node columnar database for
data warehousing and ad hoc analytics. SkySQL makes it easy to start small and scale when
needed, as much as needed – whether it’s the result of continued business growth or an
exponential surge (e.g., successful Black Friday/Cyber Monday promotions).

complexity and do not deliver performance at "internet scale" without
in-memory caching technologies. When combined, caching, clustering,
replication, and manual sharding strategies lead to an extremely brittle
architecture.
NoSQL has not been a panacea, either. Transactional integrity, flexible
query, and join functionality are the primary reasons organizations
have not adopted NoSQL technologies outside of low-risk applications
and niche areas where there was no choice but to sacrifice database
functionality in order to deal with the required high data volume and
multi-data center deployments.
Traditional databases excel at joins and transactional data integrity
— but not when combined with the requirement to handle large data
volumes and global data replication.
The promise of distributed SQL databases is that they allow you to
achieve internet scale and high availability without sacrificing data
Figure 1
Google Spanner and most distributed SQL databases follow the
CAP theorem as "CP" databases, meaning they are consistent and
partitionable. According to the CAP theorem, this means these
databases sacrifice "100% availability"; however, this is a "theoretical"
analysis of their storage and replication model. If the database achieves
a high enough level of reliability through other means, then theoretical
fault tolerance is less of a concern.
Data in a distributed SQL database is exposed as tables but stored in
key-value pairs. Tables in distributed SQL databases are divided into
smaller units, essentially sliced into "partitions" (which the Spanner
paper calls tablets). Each partition is replicated to a number of server
nodes. Peers live on separate instances and generally separate
availability zones and data centers. For instance, if the replication
factor is 3, then there are three replicas of each table partition. This
replication has strong consistency.
REFCARD | JANUARY 2022
REFCARD | DISTRIBUTED SQL ESSENTIALS
integrity, full RDBMS features (such as joins), and a standard query
language (SQL). These new distributed architectures embrace and
take advantage of modern technologies like cloud computing and
Kubernetes.
HOW DISTRIBUTED SQL DATABASES WORK
NoSQL databases dropped much of the functionality of SQL databases
because they are difficult to implement at scale. Features like
transactional consistency were not as strictly important for the "big
data" use cases these databases were created for. As Google faced
increasingly critical workloads, it developed a technology called
"Spanner," which was outlined in a paper published in 2012.
The basic structure outlined in the paper is common across all
distributed SQL databases. Distributed SQL databases include a SQL
execution engine that distributes queries to multiple servers. They
include a distributed storage engine that shards and replicates data
(see Figure 1).
One node is elected as the leader using a consensus protocol (either
PAXOS or Raft for most other distributed SQL databases). When an
application writes to the database, a set of keys is locked, and the state
change is replicated among the nodes in strict order. A log ensures
ordered data replication regardless of failures or cluster changes.
Distributed SQL databases allow for multi-zone, where data for each
partition is replicated using the consensus protocol. Leaders coordinate
writes and are usually distributed among multiple zones evenly.
In the event a zone or region becomes unavailable, a new leader is
elected in one of the remaining zones or regions. Data is copied from
surviving replicas to existing nodes to maintain fault tolerance and
data distribution (see Figures 2 and 3 on the next page). If new nodes
are added, the data is rebalanced among the new nodes, increasing
distribution and performance.
3 BROUGHT TO YOU IN PARTNERSHIP WITH

Figure 2
Figure 3
NEWSQL VS. DISTRIBUTED SQL
Before the Spanner paper, there were a number of "NewSQL" databases
developed. These databases do not offer the same level of functionality
as distributed SQL databases. In some cases, they are merely a sharding
strategy built on existing monolithic databases like MySQL and
PostgreSQL. Some do not provide global consistency or failover.
Distributed SQL databases are a subset of NewSQL databases that
scale, shard, fail over, and provide global consistency. Because there
is no governing standard for distributed SQL databases, some NewSQL
vendors have begun calling their systems "distributed SQL," despite
lacking distributed SQL features.
FUNDAMENTALS OF DISTRIBUTED SQL
ARCHITECTURE
Using or migrating to a distributed SQL database requires considering
your workload, application architecture, performance, consistency
requirements, geographic topography, and execution environment.
Match application requirements to the capabilities and configuration
of a distributed SQL database to ensure your business and customer
needs are met.
CONSIDER YOUR WORKLOAD
Distributed SQL databases are good for moving mission-critical
and system-of-record workloads to cloud computing environments
REFCARD | JANUARY 2022
REFCARD | DISTRIBUTED SQL ESSENTIALS
while providing resilience and consistency across redundant zones.
However, this distribution inherently causes more latency. Some legacy
workloads may not tolerate distribution. In those cases, a client-server
RDBMS with read replicas or multiple writers may be more appropriate
even though they do not scale to the same level as a distributed SQL
database.
To migrate an existing application, consider whether the distributed
SQL database is sufficiently compatible with your previous database’s
APIs and SQL dialect. Most distributed SQL databases are either
compatible with MariaDB and MySQL or PostgreSQL. Some distributed
SQL databases also provide compatibility features for Oracle (even
including PL/SQL) and SQL Server.
If an application is written using an Object-Relational Mapping (ORM)
tool, consider a distributed SQL database that is well supported by that
ORM.
MIXED MODELS
While NoSQL databases, particularly document databases, are famous
for storing JSON, SQL databases now also support JSON. In fact,
JSON support is part of the SQL 2016 standard. Different distributed
SQL and SQL databases have extended support for JSON. It may be
possible to consolidate both SQL and JSON workloads into a single
distributed SQL database.
4 BROUGHT TO YOU IN PARTNERSHIP WITH

ADDITIONAL INFRASTRUCTURE
A full distributed SQL system is more than a database. It requires
administration tools, load balancers, and a database proxy in order
to fully serve its purpose. A load balancer may not be sufficient for a
modern everyday application. In the case of a node failure, a database
proxy can potentially retry inflight transactions and fail over sessions
to the new instance. This capability reduces the burden on application
clients and removes the need for developers to write complicated
exception case code to handle transaction failures.
CONSIDER MIXED WORKLOAD SUPPORT
Distributed SQL databases are transactional by nature, meaning they
support a large number of short reads, writes, and simple queries that
pull back a small number of rows. Traditional relational databases
supported parallel queries and other types of relatively long-running,
semi-analytical queries. Some distributed SQL databases support
columnar indexes and the ability to run more ad hoc and analytical
queries. Look at your entire workload and usage patterns to ensure the
distributed SQL database can handle most of your application needs. If
consuming the database as a service, look at add-on capabilities such
as columnar storage.
MAKING AND MEETING PERFORMANCE
REQUIREMENTS
Using your business requirements as a foundation, establish technical
requirements for scalability and latency. Scalability requirements are
expressed in terms of queries per second, writes per second, reads per
second, data volume, and relative growth. This enables the cluster to
be sized appropriately. Latency requirements should be expressed as
nominal and maximum delay for writes and reads on the system. As the
application matures, develop more specific requirements for different
components or services. For instance, a sale might take 10ms, but other
requests might be allowed to take 3ms. Consider the database as a part
of your total "latency budget."
UNDERSTANDING CONSISTENCY
Business-critical applications rely upon ACID guarantees. When data is
written in one row in one table on one server, it is easy for databases
to be "ACID." When multiple rows or multiple tables are written, it is
important to understand the database’s consistency guarantees and
their failure modes. Ensuring consistency requires some degree of
locking or copying and waiting on acknowledgment, so when seeking
higher throughput or lower latency, it may make sense to compromise
read consistency guarantees for select datasets or clients.
Write consistency should never be compromised for system-of-record
applications. For anything from ever-growing data to informational
status, consistency may not be as important, or at least not equally as
important to all applications or application clients. The database may
offer different modes of isolation — for instance, REPEATABLE_READ and
SERIALIZABLE. These offer different trade-offs in terms of consistency
REFCARD | JANUARY 2022
REFCARD | DISTRIBUTED SQL ESSENTIALS
and performance under concurrent access. Making these compromises
in integrity depends on your database’s capabilities, the nature of the
data, and the needs of your application.
CLOUD, HOME, OR HYBRID
Many applications and services have moved to public clouds. Some
distributed SQL databases are offered as a vendor-hosted database as
a service. Although it may be more costly, the distributed SQL database
provider has the most expertise with running that type of database and
can apply the best practices learned across multiple customer installs.
The cost may also balance out when labor, hardware, and risk are
accounted for.
The downside of a managed offering is the loss of control. A vendor
may not support a really old version of their database because of costs.
An organization may not want to assume the risk of an upgrade until
their application or service is revised. Some applications, datasets, or
industries cannot consider the public cloud due to cost, data sensitivity,
or specific security requirements. Running locally also may achieve
lower latency with data at the edge of the network.
Many organizations have a combination of sensitive and less-sensitive
data and may want a hybrid cloud consisting of local instances as well
as public cloud instances. Data may even be replicated between private
and public instances in various configurations. While there are many
potential advantages of a hybrid cloud approach to database hosting,
it is more complex to operate and requires a platform or tool to manage
the database cost effectively.
COST CONSIDERATIONS
Distributed SQL databases replicate at the database layer as opposed
to the storage layer. Competitive technologies like Amazon Aurora use
the storage layer for replication and redundancy. Generally, distributed
SQL databases are cost-competitive compared to storage layer
replication. However, when measuring cost in cloud-hosted distributed
SQL databases, instance pricing is only one small component of the
overall system. Generally speaking, IOPS are a larger component.
Additionally, while one database may support cheaper instances, it
may require more of them to achieve the same performance. When
measuring and comparing costs, price for performance is the most
important factor.
CONCLUSION
Distributed SQL databases are critical infrastructure for taking systems
of record to the cloud or operating them at internet scale. These
systems match the scale and resilience of a NoSQL database with
the full-featured performance of a relational database. Whether the
consideration is for an existing workload or a new system of record, any
application that requires scale, low latency, transactional integrity, or
general resilience is a good candidate for a distributed SQL database.
Anyone architecting a modern application that takes advantage of a
5 BROUGHT TO YOU IN PARTNERSHIP WITH

distributed database should investigate tools like Hasura or Prisma
as well as Object-Relational Mapping tools like Hibernate. These tools
may allow for overall distributed application optimization including
optimizing database operations. Deploying and managing a distributed
database is more complex than a monolithic database and requires
either extensive DevOps expertise and tooling or a distributed SQL
management platform that takes care of the details. This is also true
when monitoring key performance and cost metrics. The rewards of
distributed SQL technology, including globally consistent transactions,
high availability, and scalability, make these challenges worthwhile
— especially when deployed with the right DevOps tooling and
architecture.
FURTHER READING
•  "Spanner: Google's Globally-Distributed Database" – https://
research.google/pubs/pub39966/
•  "What You Need to Know About Distributed SQL" – https://dzone.
com/articles/what-you-need-to-know-about-distributed-sql
•  "Getting Started With Distributed SQL" Refcard – https://dzone.
com/refcardz/getting-started-with-distributed-sql
•  "Beyond NoSQL: The Case for Distributed SQL" – https://www.
infoworld.com/article/3564543/beyond-nosql-the-case-for-
distributed-sql.html
REFCARD | JANUARY 2022
REFCARD | DISTRIBUTED SQL ESSENTIALS
WRITTEN BY ANDREW OLIVER,
SR. DIRECTOR OF PRODUCT MARKETING,
MARIADB
Andrew C. Oliver is the Senior Director of Product
Marketing for MariaDB. He is a prolific writer about
technology — particularly open-source and distributed database
technologies. In the past, he served on the board of the Open
Source Initiative, founded Apache POI and was an early part of
JBoss, Inc. before its acquisition by Red Hat. Find him over on
Twitter @acoliver.
600 Park Offices Drive, Suite 300
Research Triangle Park, NC 27709
888.678.0399 | 919.678.0300
At DZone, we foster a collaborative environment that empowers developers and
tech professionals to share knowledge, build skills, and solve problems through
content, code, and community. We thoughtfully — and with intention — challenge
the status quo and value diverse perspectives so that, as one, we can inspire
positive change through technology.
Copyright © 2022 DZone, Inc. All rights reserved. No part of this publication
may be reproduced, stored in a retrieval system, or transmitted, in any form or
by means of electronic, mechanical, photocopying, or otherwise, without prior
written permission of the publisher.
6 BROUGHT TO YOU IN PARTNERSHIP WITH