2

High Availability

In the first chapter we learned about Backing Up and restoring our BizTalk Server 2010
Environments. In this chapter we will untangle the complexity of implementing High
Availability.
The prime goal of high availability is to minimize downtime. Downtime can be caused by
several sources. Some of these sources are:
 Data Corruption
 Hardware Component Failure
 Application Failure
 Server Failure
 Human Error
 Maintenance
 Site Outages
In order to achieve high availability for our BizTalk Environment we will learn about the
following topics.
 High Availability and the Microsoft Operations Framework
 Host separation and multiple host instances
 High availability setup of BizTalk both virtual and physical setup
 Setup of handlers in a high available environment
 Third-party clustering and virtualization platforms
 High availability and the Microsoft Operation Framework

Microsoft Operations Framework

Microsoft provides us with Solution Accelerators. One of these accelerators is the
Microsoft Operations Framework (MOF). The MOF provides relevant, practical, and
actionable guidance that we can utilize to blend business and IT goals. There are many
benefitals factors to implement a framework, the main keys is to enhance the
productivity, effectively and and the cost-efficiently. Implmenting the guidline
throughout the entire administrator role is important to gain the maxium efficiency.

The current version is 4.0.

More information about the MOF 4.0 framework can be found on TechNet.
http://technet.microsoft.com/en-
us/solutionaccelerators/dd320379.aspx

High availability and the Microsoft Operations

Framework
Applying the Microsoft Operations Framework for the planning and implementation of a
highly available BizTalk Server 2010 Environment will insure that we have taken the
appropriate steps at the different stages of the release life cycle. This allows us to look
ahead at all the life cycle stages where high availability surfaces. It will make the
installation, maintenance, and troubleshooting issues in our environment easier.
Central to the MOF process model is its division into four quadrants of operational
processes and procedures, named Service Management Functions (SMFs). The SMFs are
the foundation-level best practices and prescriptive guidance for operating and
maintaining an IT environment. The following figure shows the MOF processes where
we have to consider high availability:

2
Changing Quadrant
The canging quadrant includes the service management functions (SMF), this is required
in order to review, approve, incorporate and identify changes for a managed BizTalk
environment. You will also includes artefacts like software, hardware, roles,
reposnibilities, documentation and different processes and procedural changes.

Change Management
The Change manamgnent is responsible for any changes made in technology,
applications, integrations, tools, hardware and any changes in responsibility and roles.

Configuration Management
The configuration management is to identify, control and event track versions of
software, hardware, processes, procedures, documentation and all other components of
the IT evnrionment, all under the control of a good change management.

3
Operating Quadrant
While the operating quadrant also includes the SMF, its directed to the SMFs requirement
to monitor, control, manage and administer service solution, daily to achieve and
maintant any service levels within dererminded parameters and bounderies.

Job Scheduling
Scheduling jobs involces the continuous the organization of jobs and processes in an
efficient sequence, this will help yo maximize the system throughput and to meet service
level agreement (SLA) requirements. This involves that we schedule planned downtime,
this may be during system updates, upgrade of application or migration of the
environment. This should be done at times when the message load is low or according to
predefined agreements in the SLA or OLA requirements. This is to minimize the
potential effect on our business.

Supporting Quadrant
While the supporting quadrant includes the SMFs requirments to idendtify, diagnose,
track, assign and resolve incidents, requests and problems within the approved
requirements that are contained in the service level agreements.

Optimizing Quadrant
The optimizing quadrant contributes to maintain BizTalk and business alignment by
focusing on low IT costs while maintaining and improving service levels. This will
include a review of outages and incidents, examination of cost structures, staff
assessments, availability and performance analysis and capacity forecasting.

Service Level Management

The focus of the service level management is to maintain and improve the quality of
services, through a constant cycle of negotiating and monitoring service level
requirements.

Availability Management
The availability management is to make sure that our customer can use a particular IT
service whenever they want. By estrablishing mechanisms for notifying personnel if
hardware failure occurs so that it may be fixed or replacement of hardware can be done as
quickly as possible. This may also involve if hardware load is larger than a particular
threshold

4
Service Continuity Management
The service continuity management function is to ensure that specified IT services
provides a value to clients and customers if regular-availabilty solutions fail.
During the service continuity function you must examine what high-availability
configuration to implement to make sure that you continue providing your customers
with the services they expect even when a planned or unplanned downtime occurs.
Examples of unplanned downtime are hardware failures.

For more detailed information regarding Microsoft Operation Configuration take

a look at Microsoft TechNet http://technet.microsoft.com/en-
us/library/cc506049.aspx

Planning for High Availability

The first step in our High availability planning is to properly configure our Hosts and
Host Instances.

Separate Hosts and Multiple Host Instances

A BizTalk host is a logical container of zero or more runtime processes. The Host owns
its own tables in the BizTalk databases and the dedicated workflow for each host is added
to the desired table in the BizTalk MessageBox (BizTalkMsgBoxDb). For processing
data all hosts need a physical representation, these are called host instances they perform
the processing of data, example receiving, sending or orchestrating and the processing of
a message on the BizTalk servers. To maintain a high availability environment there
should be at least two Host instances running for each Host.
It is also important to configure separate Hosts for receiving data, sending data and
processing orchestrations and tracking. BizTalk provides us with built-in load balancing
that automatically distributes workloads across multiple Host Instances. Separating the
host will also make changes for thresholds and throttling easier, we will talk more about
this in chapter 6.

The SQL Agent job

“MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb” will check to
see if a server running a Host Instance is up and running, if it cannot get a hold
of the servers this job will release the workflow for all host instances connected
to the server that is no longer responding so that BizTalk can dedicate a new
Host Instance to continue with the processing. We will talk more about BizTalk
SQL Agent jobs in a later chapter.

5
Host clustering
Some receive handlers cannot run on multiple servers at the same time. These handlers
are FTP, MSMQ, POP3, SQL, FTPs and SAP or any adapters that supports Ordered
Delivery. These hosts need to be clustered to maintain a high availability environment..

Creating multiple Host Instances where ordered delivery is not supported causes
unexpected issues such as duplicate files, corrupt files, etc.

You can cluster hosts by referring to the hosts in the BizTalk Administration Console by
right clicking on the host and choosing Cluster.

All clustered hosts will have its own symbol under host instances. Be aware that one
should always be disabled. But do not disable the host by changing the configuration

6
To cluster a host you need to follow these steps.
 Open the “Host” window under the “Platform settings” in the “BizTalk Administration
Console”
 Right click the Host you want to cluster and choose the option “Cluster”.
 Add the host to the cluster and click ok.
 You will now have the text behind your host with information like the image above. And
under Host Instances you will see which server the host instance is currently running at.

Dedicated Tracking Host

When global tracking is turned on, BizTalk by default tracks all in and out events,
including information and count of all instances, other tracking options can be turned on
by modifying the tracking options found under properties for the dedicated artifact. All
tracking data is stored in the BizTalkMsgBoxDb database under the tables
dbo_TrackingData_x_x (where as X is a number) all the tables starting with 0_x is
tracking information for BAM (BAMPrimaryImport). If you don’t use BAM these tables
will be empty at all time. Tables starting with 1_x tracks data information for the
Tracking database (BizTalkDTADb).
By enabling the option on the host “Enable host tracking” will tell BizTalk that this host
is designed to move information from the MessageBox to the BAM and Tracking
database.

Enabling host tracking grants this host access to write to the tracking database;
the other Hosts only have read access, but they will still write tracking info to
the MessageBox database). This host should never run any artefacts in BizTalk,
all available resources should be dedicated to moving the tracking data for this
host.

We will also need to create a dedicated BizTalk Host Instance for Tracking.
The movement of tracking data out of the MessageBox(s) is critical. Without this, the
BizTalk MessageBox(s) can become bloated and will greatly affect performance over
time and if the databases goes full it can aggregate a full stop in BizTalk..

7
The Tracking Host should be run on a minimum of two BizTalk Servers. To insure
optimal performance and insure Fault Tolerance, It is recommended to have at least one
Tracking Host per BizTalk MessageBox pluss one extra for redundancy.

Separate Hosts and Multiple Host Instances

BizTalk Host Instances are created and assigned to Hosts. The BizTalk Host Instance is
a physical representation of a BizTalk Host on multiple servers. To provide High
Availability we must run multiple Host Instances.

We will need to configure separate Hosts for Receiving Messages, Sending Messages,
transfering tracking data and Processing Orchestrations. . This is defined as best
practices, by assigning the hosts for different processes is will enhance the security and
overall structure of your environment. It’s also recommended to maintain separate users
for the different roles, a receive host should never have access to destination locations
and vica versa.
BizTalk provides us with built-in load balancing that automatically distributes workloads
across multiple BizTalk Host Instances. In most situations, this eliminates the need to
cluster our BizTalk Hosts.
Splitting the host in to designated tasks will enhance the functionality and flexibility
when configuring the workload in our BizTalk environment. This will also allow you to
8
stop one host without affecting other hosts and other processes in our BizTalk
environment. One exemple is during a problem like a great load of messages coming into
your environment and not leaving, you may want to stop all receiving of messages while
sending still should occur to decrease the load on your server.
Planning for our virtual BizTalk 2010 environmentFinding the best combination for our
environment is vital, what should be virtual and physical platforms. Remember that the
split of virtual and physical servers may differ from companies.
You should also be aware that the virtual platform you have available is supported by
Microsoft and is either a Microsoft product or SVVP-Certified.

All the SVVP-Certified servers are listed here

http://www.windowsservercatalog.com/svvp.aspx?svvppage=svvp.htm

Scenario: Configuring a highly available VM using

Hyper-V
For this scenario we have decided to virtualize both our BizTalk 2010 and SQL 2008 R2
clusters.

Planning our Hyper-V Host server

In order for you plan your Hyper-V Host server you should fill out the following points
prior to setting up your virtual platform. Physical and virtual machine requirement
 Software requirements
 Basic configuration for Hyper-V
 Basic virtual server configurations
 NIC configuration

Windows Server 2008R2 IP configurationWindows Server 2008 R2 Server

Failover Cluster Is a group of two or more interconnected Windows 2008 R2
Servers correlating and interacting with each other over the network.

9
Planning our Virtual BizTalk 2010 Enterprise Cluster
When planning the setup of our virtual BizTalk 2010 cluster we need to ensure that we
follow all steps, if you are not using Microsoft Hyper-V you need to be sure that the
virtualized environment is SVVP-Certified.

The following steps must be followed when establishing our Virtual BizTalk 2010
Enterprise Cluster Environment:
1. Verify the Required Server Roles and Features that you need to have on
your machine, this may be elements like IIS, MSMQ or similar software
that is needed for your BizTalk environment.
2. Verify our Firewall Configuration and that all the needed exceptions is
correct and allowed.
3. Then Configure IIS for the clusterwhere you need to have it.
4. Configure Microsoft Distributed Transaction Coordinator (MSDTC) to
allow communication.
5. Add the dedicated Storage for our BizTalk Cluster
6. Create the BizTalk Cluster Group
7. Verify our Cluster by trying a failover
8. Configuring the Cluster Quorum Settings
9. Add Storage as a Disk Resource for the cluster
10. Create the BizTalk Cluster Resource
11. Cluster IIS and add it to the BizTalk Cluster Resource
12. Add the IIS HTTP Response Header to the cluster
13. Create and add the Generic Script Resource for IIS Clustering

10
 14. Cluster MSMQ and add it to our BizTalk Cluster Resource (optional)

The following are the steps we will follow when planning our Virtual SQL 2008 R2
Cluster.
1. Configure the Firewall
2. Configure Microsoft Distributed Transaction Coordinator (MSDTC)
3. Create the SQL Server Cluster
4. Assign Storage to the SQL Server Cluster
5. Add storage for the SQL servers
6. Add the assigned storage as a Disk Resource in the SQL cluster
7. Install SQL Server on the cluster
8. Verify the SQL Server configuration
9. Complete the Cluster configuration
10. Verify the IP Settings
11. Add additional storage to the Cluster instance
12. Add a Clustered Distributed Transactions Server (DTC)
13. Verify the Cluster by performing a manual failover test

BizTalk 2010 Failover Cluster Configuration

You need to configure the failover configuration to your environment to make sure you
will have a valid and good setup, by following the steps in the image below we will
maintain and have a good failover cluster.

11
The steps to create our BizTalk Failover Cluster are:
1. Install MS SQL Server DTS 2008
2. Install BizTalk 2010 on the servers
3. Install the BizTalk adapters
4. Configure BizTalk on the main node
5. Configure BizTalk on the other nodes
6. Modify the BizTalk Application Configuration
7. Add our BizTalk Hosts
8. Add our BizTalk Host Instances
9. Add the correct handlers to the adapters
10. Cluster our BizTalk Server Hosts
11. Verify our BizTalk Cluster Resources and Dependencies

Install MS Office Excel 2010 for BAM (optional if you are using BAM)If you
want to read more about BizTalk clustering we recommend Rene Brauwers'
article about clusters, it provides an excellent tutorial at:
http://blog.brauwers.nl/2011/05/14/part-6-biztalk-high-
availability-server-environmentbiztalk-2010-failover-
cluster-creation/

12
Third-party high availability products
Applying and using third-party products to cluster or as virtual platforms to create a high
available BizTalk environment is supported. Remember that all third-party virtual
platforms must be SVVP-Certified. Some of the current platforms supported at the
moment are:
 Third-Party Virtual Platforms:
 Red Hat, Inc: http://no.redhat.com/
 Sun Microsystems:
http://www.oracle.com/us/products/servers-
storage/servers/index.html
 VMware, Inc.: http://www.vmware.com/
 There are more: You can read more about the SVVP certified
virtual platforms
here:http://windowsservercatalog.com/svvp.aspx?svvp
page=svvp.htm
 Third-Party clustering services:
 Symantec Veritas:
http://www.symantec.com/business/cluster-server

High Availability for BizTalk 2010 Adapters

Now that we covered the foundations about setting up your BizTalk environment to
support High Availability, we will focus our attention into configuring our adapters in a
high available fashion as well, as our BizTalk environment won´t be of much use if there
are no messages that can be received and so processed by our system.
Even though we have that rich variety, we are lucky that we can easily group them into
two discrete groups in terms of how to configure them for high availability.
 Adapters whose protocol or transactionality requires that one instance receiving
from the same location at the same time. We will call these “MUTEX” adapters
 Adapters whose protocol or transactionality allow having multiple instances
receiving/sending from the same location at the same time. We will call “NON-
MUTEX” adapters

13
MUTEX adapters
There are certain protocols or Message Exchange Patterns (MEP) that requires that the
vendors we are receiving information from is only accessed by one host at a time. The
scenarios will vary:
 Having multiple instance accessing the same resource could lead into
messages duplicity
 Messages reception needs strictly to keep in an order.
 The resource being accessed gets locked by the party that is receiving
from it.
Some of these are the MSMQ, WCF-NetMSMQ, FTP, POP3 or in certain scenarios the
WCF-Custom/WCF-NetTCP, MLLP adapters.
For all of these, the solution is to configure the adapter to run in a clustered host. This
way we´ll make sure that only one instance of the adapter interacts with the
corresponding resource, at the same time we are safe against any situation where the
active host for that adapter becomes unavailable and by keeping our business running on
another host instance.

NON-MUTEX adapters
This is the vast majority of send adapters (except those that require strict ordered
delivery), all of the HTTP-based adapters and those scenarios with WCF-Custom/WCF-
NetTCP adapters that match the non-mutex requirement. We have two different options
depending on whether the adapter handler is a receive or a send handler.

14
Send handlers
Send handlers are the easiest scenario due to the pub-sub pattern, so deeply engraved in
BizTalk’s architecture, is the one helping us here.
We will just configure the adapter send handler on multiple Hosts Instances in our
environment, and so assigning that logical host to the corresponding send port(s) we want
to make highly available.
All those ports will be subscribed to the corresponding message type or schema in the
messagebox, so even if one of the nodes fails, messages will be still delivered as long as
there is at least one instance of that host up and running.

Receive handlers
Network Load Balancing (NLB) can be used in many scenarios to improve stability. It
will take care of having our messages delivered to whichever reception node that remains
active even though other nodes might be down.

The following are the steps we will follow when adding Network Load Balancing to our
Virtual Environment:
14. Prepare Server for Network Load Balancing
15. Add Network Load Balancing to our Environment
16. Add a DNS Entry for our NLB
17. Install and Configure BizTalk on our new NLB Servers
18. Configure NLB
19. Configure IIS
20. Add finally, we add the BAM Portal
We will talk more about Network Load Balancing in chapter 6 Performance and
Optimizing of BizTalk.

15
 Note that configuring both NLB and the Cluster service on the same machine is
a scenario not supported by Microsoft, so we should keep those adapter handlers
that need to be clustered apart from those that require NLB to provide high
availability. http://support.microsoft.com/kb/235305

The exception for NLB

As it happens in many aspects in life, there´s always an exception that proves the rule (or
at least goes against those rules we try to back up with our best arguments).
In our case, it comes with the MQSeries adapter that is really a mix of the approaches
taken for the scenarios we proposed earlier.

See the following link for more information regarding the MQSeries adapters:
http://technet.microsoft.com/en-us/library/aa547973.aspx

We will configure multiple instances of the host running the MQSeries adapter which
won´t be clustered (as it happened to our NON-MUTEX scenarios), but at the same time,
we will cluster (like in the MUTEX scenarios) the MQSeries queue managers as well as
the MQSeries Server itself.

You can read more about cluster the MQSeries here

http://msdn.microsoft.com/en-US/library/dd897482(v=BTS.10).aspx

Summary
We have learned about implementing the Microsoft Operation Freamework into a high
availability environment. Including the setup of dedicatd hosts for different processes in
BizTalk. And dedicated tracking host. We have learned that by doing it correctly will
give us a better overview and flexibility in our environment.
We have seen how you can cluster BizTalk in a virtual platform and specially the usage
of Microsoft Hyper-V, we have also seen that other virtualization platforms are useable
as long as they are SVVP-Certified. We have also taken a look at third-party products for
high availability.
We have discussed the principle planning og setup and configuration of clustering and
the single instance adapters that can only run on one host instance at a time. We have
described the issues with MUTEX adapter and how to cluster them correctly.

16
The next chapter will cover the most important aspects of monitoring your BizTalk
solution.

17