ASE Cluster Edition

Technical Overview

Sybase Confidential
 Agenda
• ASE Cluster Edition Background
• Connection Management
• Cluster Resources
• Administration
• Roadmap

Sybase Confidential
 Agenda
• ASE Cluster Edition Background
• Connection Management
• Cluster Resources
• Administration
• Roadmap

Sybase Confidential
 Use Sybase ASE Cluster Edition
Protect Service Levels
• Protect against downtime from node failure because other
nodes in the cluster continue unaffected
• Maintain SLAs during peak loads by balancing workload
across multiple nodes

Maximize Resource Utilization

• Consolidate multiple applications onto the cluster, maximizing
use of resources and reducing underutilized hardware
• Leverage redundant hardware by distributing workloads across
all nodes in the cluster

Reduce Infrastructure Costs

• Deploy clusters on standard, commodity hardware,
reducing cost of purchase, and support
• Provide a computing grid that can be easily expanded and
maintained by bringing nodes online and offline as needed

Sybase Confidential
 What is ASE Cluster Edition?
ASE with new Shared-disk Clustering technology

The best & latest technology for high-availability

ASE Cluster Edition

Load balancing

Manageability

Virtualization

Architected to support up to

SAN
32 ASE servers

Sybase Confidential
 Sybase’s Clustering Strategy
Cost Savings
MANAGEABILITY

Scalability

Virtualization

SDC

AVAILABILITY

Sybase Confidential
 Comparison of Various HA
Solutions
Shared disk Failover Database Shared Nothing
Clustering Clustering replication Clustering

Downtime 0 < 5mins < 30mins < 15mins

Data Loss 0 0 Depends on Depends on

architecture architecture

Transparent Yes Yes No No

App failover

Distance Yes No No No
Limitations

Data No Yes Yes No

Duplication

Sybase Confidential
 Cluster Edition and
Other Sybase BC/DR Solutions
Business Continuity solutions for multi site disaster recovery

Block Replication
101100010 1011000101101 1011
1

DI Suite – Replication Mirror Activator

Replication of DB transactions Combines disk/block replication
with DB replication

Business Continuity solutions for single site high availability

A C

standby
B
standby

D +

ASE High Availability (HA) option ASE Cluster Edition

High availability for a single primary nodeContinuous availability for all nodes,
and so much more!

Sybase Confidential
 Business Continuity
There are tradeoffs between system latency due to distance, the area
impacted by the cause of downtime, and frequency of the cause

High- availability Campus cluster – spans across a data center

Disaster Recovery Metro cluster – spans across a city or metropolitan area

Disaster Recovery Geo cluster – spans between cities and across countries

Sybase Confidential
 Cluster Edition Installation
& Components

<ASE_SDC>.cfg
Public Network
Node
Instance Instance
errorlogs

!!
!!
Private
Interconnects

$SYBASE

CFS Quorum Device Database Devices SAN Storage

(or NFS) (raw partition) (raw partitions)

Sybase Confidential
 Target Configurations (1)
Continuous Availability for Mission Critical Apps
Availability Configuration
MC Application
− Topology: n+1 or 1+1 in the simplest case
− Usage Profile: Mission critical application, often
user facing, that needs to be continuously
available. Upgrade from ASE/HA option for
higher availability
− Key benefit of SDC:
− Continuous Operation: Node removal (either
because of failure or for planned maintenance) does
not affect other nodes.

Minimal application changes required

Sybase Confidential
 Target Configurations (2)
Application Consolidation
Consolidation Configuration
Different Applications
− Topology: n+m or n+1 in the simplest case
− Usage Profile: Multiple ASE applications App A App B App C Spare

with desired availability protection on

underutilized hardware
− Key benefits of SDC:
− Lower cost: Applications can be consolidate to fewer systems.
System Redundancy is achieved by fewer “substitute nodes”
(e.g. the +1, +2 or a max of *2 support nodes.) No additional
cluster software to buy. Lower power and footprint
consumption.
− Continuous Operation: Node removal (either because of
failure or for planned maintenance) does not affect other
nodes.

Minimal application changes required

Sybase Confidential
 Target Configurations (3)
Well Partitioned Applications
Well Partitioned Applications
− Topology: n+m or n+1 in simplest case Well Partitioned Application
Slice A Slice B Slice C Spare

− Usage Profile: A single, partition-able

ASE application with desired availability
protection on underutilized hardware.
− Benefits of SDC:
− Some scale: Application can be distributed to “n” nodes as long as:
Applications partitions do not need to talk to each other.
Application is “aware” of the node partitioning and ensures that it
only talks to one node at a time.
Fully linear scale is not desired.
− Continuous Operation: Node removal (either because of failure or for
planned maintenance) does not affect other nodes.

Minimal application changes required

Sybase Confidential
 Sybase ASE Cluster
Edition: Higher Availability Configuration
Cluster Edition Advantages over Existing Sybase ASE HA Option

Sybase Confidential
 Competition
We are 2nd in the market. Key Differentiators:
– VRM™ Technology
– Workload Management
– Open Architecture

Vendor Comparison Shared Disk Shared Nothing Automatic Load Balancing Dependency on third
Failover party Cluster-ware
Architecture Architecture

Sybase ASE Cluster X Yes Advanced None

Edition

Oracle 11g X Yes Minimal ASM & CRS

MySQL Cluster X No 4 No CRS

Microsoft SQL Server X No 4 No MS Cluster Server

IBM DB2 X No 4 No IBM ICE

IBM DB2 on ZOS X Yes Minimal

Sybase Confidential
 Virtualized Resource
Management™
Applications
& Workloads

Logical Workload
Clusters Manager

Physical
Cluster

NODES ASE SERVER INSTANCES

Sybase Confidential
 Oracle 10g RAC vs. ASE Cluster
Edition: Workload Management
Features ASE Cluster Edition (ACE) Oracle 10g RAC

Virtual Workload Manager Database Services

Architecture:

Sybase architecture is based on logical clusters

which makes it easier to use over Oracle’s database
sessions
Advanced Load Balancing features:

ASE CE provides more routing policies and out of box

connection distribution models. RAC requires manual
configurations.

Cluster Maintenance Support:

Automated planned downtime management

Ease of Use:

Installation & configuration is much easier than

Oracle RAC
Cluster Optimization Metrics:

Collects and uses more matrices for better utilization

Sybase Confidential
 Platform and Version Support
– Supported platforms
SuSE Linux 9, 10 (X86_64)
RedHat Enterprise Linux 4, 5 (X86_64)
Note: No 32-bit Linux or RHEL 3
64-bit Solaris Sparc 9, 10
– Supported Upgrades from ASE versions
12.5 to 12.5.3 ESD #6
15.0 and 15.0.1 ESD#3

Sybase Confidential
 Cluster Edition
Hardware Requirements
– Homogonous hardware nodes
– Same O/S and architecture required
– Boxes can be multi-way SMP
– At least three networks
– A “public” network for normal host / ASE connections
– Two private cluster networks used exclusively for ASE intra-cluster
messaging
– Raw database devices on SAN storage
– Devices must have the same path on all nodes
– $SYBASE on shared file system (NFS / CFS)
– 3rd party cluster file systems are ideal for higher availability

Sybase Confidential
 Storage Area Network
– Certified for raw devices on SAN
– Devices must have the same path from all hosts
– One sysdevices table
– SAN fabric should be multipathed
– Dual HBAs is required in production, with multipathing
preventing a single link failure from knocking out the cluster
– Disk subsystem should support SCSI-3 persistent reservation

Sybase Confidential
 Supported Interconnects
– Cluster Edition currently supports UDP
– Vanilla Ethernet NIC
– InfiniBand/IPoIB
– Typical Configuration
– “Public” interface not used for CIPC
– This is what clients use to talk to ASE, i.e. what is in
directory service
– Interconnect has a private IP address, like 192.168.0.1, and is
cabled to a switch that only talks to other cluster members
– The switch is not hooked to a router
– There are two of these interconnects using a physically
separate fabric, i.e. each interconnect goes to a separate
switch

Sybase Confidential
 ASE Shared Disk
Cluster Architecture
An Instance Data Service

Cluster Logging Cluster Space / Cluster Meta-Data /

Local/Global Temp DB
Recovery Threshold DDL / Statistics

Buffer Cache Cluster Lock

Object Coherency Peer Coordination
Coherency Management

Workload Connection/Context Cluster RPC, Cluster SPID, DBCC,

Management Management Replication Agent Monitor, Config, etc

Kernel

Quorum Management, Cluster Membership Cluster Event Reliable

IO Fencing Service Service Cluster Interconnect

Basis I/O and Platform Abstraction Interconnect I/O Abstraction

Operating System / Device Drivers UDP TCP SDP* VERBS*

Sybase Confidential
 Workload Manager Overview
The workload manager …
can customize workload management and failover for each of your
business applications so that each performs most efficiently.
The logical cluster is the container …
that allows the workload manager to provide individualized working
environments.
Routing rules direct incoming connections …
to specific logical clusters based on an application, user login, or server
alias supplied by the client.
By creating logical clusters on top of the physical cluster…
you can partition applications with different workloads on the same
system.

Sybase Confidential
 Workload Manager Functions
– Through Sybase central administrator can
– Configures and manages a logical cluster
– Selects or configures load profiles that the system uses
– Monitors instances in the cluster and the workload on each
instance

Sybase Confidential
 Logical Clusters
– A logical cluster is assigned
resources from the physical Client 1 Client 2 Client 3 Client 4

cluster
– Instances are a logical cluster’s Private network
All instances provide
base instances service to a single
database store
– Failover resources are an ordered
list of instances on which a
logical cluster is to run should
one or more of the base “Sales LC” “HRLC” “Catchall LC”

instances fail. “my cluster” shared-

disk cluster
– You can create logical clusters using
sp_cluster logical, "create".

Clustered databases Quorum disk

Shared-disk storage

Sybase Confidential
 Adding Cluster Instance

Sybase Confidential
 Configuring Logical Clusters

Sybase Confidential
 Agenda
• ASE Cluster Edition Background
• Connection Management
• Cluster Resources
• Administration
• Roadmap

Sybase Confidential
 New Client Technologies
– New client technologies allow clients to connect logically to a
cluster while remaining connected physically to individual
instances.
– New client technologies include:
– Login redirection – when the client reconnects to another instance
in the cluster. OCS version 15.0
– Connection migration – when an established connection moves to
another instance in the cluster. OCS version 15.0 ESD #3
– Extended high-availability failover – allows the client to fail over
multiple times until it finds the first available or least-loaded
instance. OCS version 15.0, ESD #8

Sybase Confidential
 Application Changes
to Enable Failover
Any existing application can connect to the Cluster Edition.
However, to use the Cluster Edition extended high-availability
(HA) capabilities, application code may need to be changed.
For existing HA applications based on existing Adaptive Server HA
functionality, no application code changes are required.
Existing non-HA applications may benefit from some aspects of the HA
capabilities of the Cluster Edition with no code changes, or minor ones.
− However, in these cases, failover is not transparent.
− The application receives an error message when failover is first detected.
− The user must then resubmit the batch or transaction to initiate failover.
To make failover transparent to users…
the application must actively check for failover error status, and
automatically resubmit the batch or transaction.

Sybase Confidential
 Login Redirection
Login redirection occurs at login time
an instance tells a client to log in to another instance because of load
considerations.
You do not need to perform any additional configuration for login
redirection.
Adaptive Server workload manager
Uses login redirection to send incoming connections to specific instances
based on the logical cluster configuration and the cluster’s current
workload.

Sybase Confidential
 Connection Migration
– Connection migration occurs when an existing client is
transferred from one instance of a cluster to another.
– Connection migration allows the workload manager to
gracefully move existing connections between instances
during administrative failover, failback, or logical clusters
going offline.
– Connection migration is enabled automatically when an
instance uses the Open Client 15.0. client libraries.

Sybase Confidential
 Migration Vs. Failover
Migration vs. Failover
Migration is a planned, controlled event that Adaptive Server requests.
Failover is an unplanned event that occurs after an Adaptive Server crash
or network disconnect.
Applications are not aware of—and you do not need to write code
for—migration.
However, you must specifically code applications for failover support.
Migration restores the full session context on the new instance.
Failover does not
it is up to the application to restore its own context.

Sybase Confidential
 When Can Migration Occur?
Migration is possible only when a request can successfully be sent
to a client. Specifically,

To connections that have completed their login:

− After an instance receives a new batch from the client, but before the
batch is parsed and executed
− After an instance completes the processing of a client batch but
before it sends a final completion to the client.
− When an instance is not executing any batch on behalf of the client.

Sybase Confidential
 Criteria for Migration
– Migration is an asynchronous event: a request is issued for the
task to migrate, and the task migrates only when it reaches a
quiescent state. For the Cluster Edition, a quiescent state is
one that:
– Is not executing a query batch
– Has no open transactions
– Has no session-level temporary tables
– Has no declared cursors
– Has not changed its password since its initial connection
– Has not run set user or set proxy
– Is not connected to the cluster using SSL
– Is not bound to an engine using the logical process manager
– Is not using real-time data services (RTDS)
– Is not a logical connection associated with an inbound site handler

Sybase Confidential
 Extended High-Availability Failover
Adaptive Server provides a list of failover addresses to “HA-
aware” clients when they connect.

This allows high-availability-aware clients or applications to failover

multiple times whenever the instance to which it is connected becomes
unavailable.
If the instance has not sent a failover list to the client, the client
uses the HAFAILOVER entry information in the interfaces file.

The cluster edition can handle multiple failures if they do not

happen concurrently, and can recover fully from the initial failure
before a subsequent failure occurs.

Sybase Confidential
 Agenda
• ASE Cluster Edition Background
• Connection Management
• Cluster Resources
• Administration
• Roadmap

Sybase Confidential
 Named Data Caches
Cluster cache configuration defines multiple named caches to
have local or global caches.

Objects can be bound to local or global caches.

Multiple buffer pool support provides better access performance
The maximum data cache size is limited only by the amount of
available memory on your system.

The memory required to create the new cache is taken from the Adaptive
Server global memory.

Sybase Confidential
 Global Caches
Global caches are defined for every instance in a cluster

You can increase the size of local and global caches dynamically, but you
cannot reduce them dynamically.
It is possible to change certain attributes of a global cache to be
instance-specific using local definitions.

Local definitions override the global definitions and settings.

Sybase Confidential
 Local Caches
Your application can define local caches for each instance in the
cluster.

Local caches are instance-specific.

Recommendation is to partition applications across instances
within a cluster.

For application specific caches, a local cache can then be used

An object can be bound to only one cache, either local or global,
at any particular instance.

Sybase Confidential
 Temporary Databases in Cluster
– Two types of temporary
databases can be created
– Local temporary databases
−Accessed only by the
owning instance Instance S1 Instance S2
−Used to store session- system local system local
temp.db temp.db
specific private temporary temp.db temp.db

objects private private

−#tables, worktables and disk disk
fake tables (good-old) global
tempdb temp.db
– Global temporary databases
−Accessed by all instances in shared
the cluster disk

−Used to store temporary

objects that can exist
beyond current session

Sybase Confidential
 Agenda
• ASE Cluster Edition Background
• Connection Management
• Cluster Resources
• Administration
• Roadmap

Sybase Confidential
 Cluster Edition and
Directory Services
–Servers
– Each server instance uses directory services to determine the
address at which it should listen for clients.
– When an instance is started, it looks for the server name
provided in –s option. If not, it checks DLISTEN environment
variable. If not, it assumes sever name is SYBASE.
–Clients
– Determines the name of the server either programmatically or by
referring to the DSQUERY environment variable. If not, defaults
to SYBASE.

Sybase Confidential
 I/O Fencing
– I/O fencing is used to stop the non-cooperating instance
from writing to the shared disk even though the instance is
no longer part of the cluster.
– The Cluster Edition supports the SCSI-3 Persistent Group
Reservation (PGR) feature of SCSI-3 devices to provide I/O
fencing.
– The I/O fencing provided by the SCSI-3 PGR feature
operates only on devices, not on partitions.

Sybase Confidential
 Monitoring Tables
– Monitoring tables in cluster environment report on a per-
instance basis instead of cluster-wide results.
– This allows you to monitor the activities of processes and
queries across the cluster to get a better understanding of
the statistics for objects that may be opened on more than
one instance and resource usage on each instance in the
cluster.

Sybase Confidential
 Lock Management
– The cluster lock manager (CLM) provides distributed
locking services for sharing globally accessible and
cacheable objects.
– Locks managed by local lock manager can have task or
transaction ownership.
– Locks managed by the Cluster Lock Manager have instance
ownership.

Sybase Confidential
 Cluster Edition and Replication
– The Cluster Edition supports replication using Replication
Server and the RepAgent thread.
– A clustered database can be a source or a destination.
– You can perform all of the tasks, such as configuring
RepAgent or marking tables for replication, from any
instance in the cluster.
– During configuration of primary database in a clustered
system, the server name should be the cluster name.
– By default, RepAgent starts on the coordinator unless
configured otherwise.

Sybase Confidential
 Deployment Guidelines
OLTP Applications:

The best method for scaling OLTP workloads on a shared-disk cluster

is to partition the application and data into mutually exclusive sets
(that is, to separate the data to different databases) to avoid
processing coordination across server instances and access the data
for an application from the same instance.
Logical clusters allow you to allocate distinct instances to different
applications or workloads, logically enabling these groups of data to
operate under a single cluster.
DSS reporting applications

Consider creating a multiple-node cluster in which all Adaptive Server

instances service these same applications.

Sybase Confidential
 Agenda
• ASE Cluster Edition Background
• Connection Management
• Cluster Resources
• Administration
• Roadmap

Sybase Confidential
 Roadmap
The development, release, and timing on any features and functionality described for our products
remains at our sole discretion. The information on roadmaps is intended to outline our general
product direction and should not be relied on in making a purchasing decision.
2007 2008 H1
Reduce TCO • Platforms: Linux on Intel/AMD, • ASEP/ICU Tool – to manage backup server,
Solaris monitor server and historical server
• Enhanced Cluster Admin Tool
Better Resource • Virtual Workload Manager •Optimized garbage collection of locks
Utilization • Logical Clusters
Increased • 2-node: Active-Passive • 2-node: Active-Active
Availability • Built-in Clusterware • Improved interconnect redundancy
• IO Fencing
•Cluster file system certification
GFS, VCFS
•Context migration support for clients
executing Dynamic SQL/LWP
Scalability • Private devices • Support for non-partitioned apps.
• Infiniband --IP over IB
• Efficient allocation space management for
utilities (ex: create index, bcp)
• Improved hash mgmt for cluster lock
manager
• Better concurrency for cluster buffer
manager
Sybase Confidential
2008 H2
• 4- node cluster (well-partitioned)
• Enhanced systems management
• AIX, HP-UX platform support
• 3+1 cluster for application
consolidation
• 4-node
• Cluster file system certification
OCFS
• Volume manager certification
CLVM
• Support graceful multi-node departure
upon a failure
• Infiniband– SDP
2009 H1 (Pyxis)
• ASE SMP parity with 15.0.3
• Faster utilities
• Improved security (decrypt defaults for
app transparency; enhanced controls for
login/password mgmt)
• SQL UDFs, Instead-of-triggers
• Monitoring enhancements
• ADA
• Reduced resource utilization
• Faster optimized recovery
• Optimized buffer management
• Improved concurrency (DDL RLC for
clusters )
• Improved DQP (advanced aggregation,
relocated joins)
• Enhanced BCP performance
• tempdb enhancements
• Other performance enhancements
 Sybase ASE Cluster Edition…
Ensures continuous availability Minimizes performance impact
with shared-disk clusters of maintenance activities

Handle multiple Use standby nodes

failures gracefully to take others offline
with faster failover for routine
and failback maintenance

Balances system load Simplifies implementation with

automatically and transparently virtual workloads

Leverage all nodes in Deploy and configure

the cluster by routing workloads easily for
requests automatically maximum
based on business performance and
rules reliability

Sybase Confidential
 For more information on ASE Cluster Edition
www.sybase.com/clusters

Sybase
SybaseConfidential
Confidential