VMware Validated Design

for Software-Defined Data Center 4.3

Logical Component Architecture

In a dual-region Software-Defined Data Center, the two Platform Service Controllers and two vCenter Server instances are deployed in each region. In a dual-region Software-Defined Data Center, a vRealize Log Insight cluster is deployed in each region.
In a dual-region Software-Defined Data Center, two primary NSX Manager instances are deployed in Region A. The design integrates solutions for compute, storage, network, cloud operations, and cloud management.
Core vSphere Management This includes a vCenter Server for the management domain and a vCenter Server for the shared edge and compute domains.
Each vCenter Server instance is connected to a load-balanced pair of Platform Services Controllers using an NSX Edge Services Gateway. NSX One for the management domain and one for the shared edge and compute domains, along with associated NSX Universal Controller Clusters.
In Region B, secondary NSX Manager instances automatically import the configurations of the NSX Universal Controller Clusters from Region A.
vRealize Operations A single vRealize Operations analytics cluster monitors and performs diagnostics across the Software-Defined
Data Center by using a series of remote collectors and solution management packs.
vRealize Log Insight Each cluster consists of three nodes, enabling continued availability and increased log ingestion rates.
vRealize Log Insight collects and analyzes log data across the domain using the syslog protocol and the ingestion API.
To enable enhanced linked mode, the design joins the Platform Services Controller instances into a unified Single Sign-On domain. vRealize Log Insight also integrates with vRealize Operations Manager to facilitate root cause analysis.

Region A Region B Region A Region B Region A Region B

Management / Management /
Common vCenter Single Sign-On Domain Management /
vRealize
Management /
Compute
vRealize
Compute
vRealize
Automation
(Ring Topology) Management Domain Management Domain Compute
vCenter Servers
Automation
Compute
vCenter Servers
vCenter Servers
Automation
vCenter Servers Proxy Agents
vCenter Server vCenter Server
Appliance Appliance
NSX vRealize Log Insight Cluster vRealize Log Insight Cluster NSX
NSX Remote Collectors Remote Collectors NSX
Region A Region B Event
Forwarding
Management Domain Management Domain Clctr Clctr Clctr Clctr
Master Worker Worker via Ingestion Master Worker Worker
NSX Manager Pairing Shared Node Node Node API Node Node Node
NSX Manager NSX Manager Storage
Node Node Node Node Shared
Storage vSAN vSAN
(Primary) (Secondary) Systems Systems

vRealize

Platform Services Platform Services SDPlatform Services Platform Services NSX Edge Management Domain Import of Management Domain NSX Edge Analytics Cluster
vRealize
Operations
Any Supported NFS Any Supported NFS Operations
Remote
Controller Controller Controller Controller Services Gateways NSX Universal NSX Controller Configuration Services Gateways
Collectors

Appliance Appliance Appliance Appliance (N/S Routing) Controller Cluster

from Primary NSX Manager
(N/S Routing)
Master
Node
Replica
Node
Data
Node

Primary Storage Log Archives Primary Storage Log Archives

NSX Edge Services Compute Domain Compute Domain NSX Edge Services Region A Management Domain Region B Management Domain Region A Management Domain Region B Management Domain
NSX Edge Services NSX Edge Services Gateway w/ HA vCenter Server vCenter Server Gateway w/ HA
Gateway w/ HA Gateway w/ HA (One-Arm Load Balancer) Appliance Appliance (One-Arm Load Balancer)
(Load Balancer) (Load Balancer)

Compute Domain NSX Manager Pairing

Compute Domain The design establishes a Cloud Management Platform with vRealize Automation to provide a The design implements a single vRealize Automation tenant. Business groups can be created to fit your needs.
NSX Manager NSX Manager vRealize Automation, vRealize Orchestrator service catalog and self-service portal to deploy, update, and manage the workloads. Its embedded instance of vRealize Automation Within each business group the tenant administrators are able to manage users and groups, apply tenant-specific
(Primary) (Secondary) and vRealize Business for Cloud vRealize Orchestrator provides a repository of extensible workflows and integrations. vRealize Business for Cloud Business Groups & Reservations branding, enable notifications, configure business policies, and manage the service catalog.
Management Domain Compute Domain Management Domain Compute Domain provides visibility into the financial aspects of the cloud infrastructure, allowing cost to be tracked and optimized. The IT Automating IT Use Case documenation provides implementation steps for a set of scenarios.
vCenter Server vCenter Server vCenter Server vCenter Server
Appliance Appliance Appliance Appliance
Business Business
Region A Management Domain Region B Management Domain Tenant
Admin Group Group
Tenant
Admin
vRealize vRealize vRealize vRealize
Sign In

Manager Manager
VRA
Automation Business Automation Business
https://my.sddc.local/vcac/org/company

vSphere Update vSphere Update NSX Edge Compute Domain NSX Import of Compute Domain NSX Edge IWS IMS DEM IAS SQL BUS IAS BUC
VRA
Manager Download Manager Download Services Gateways Universal Controller NSX Controller Configuration Services Gateways Edge Business Group Business Group Business Group Business Group Edge
Service Service (N/S Routing) Cluster
from Primary NSX Manager
(N/S Routing) Reservation Reservation Reservation Reservation Reservation Reservation
VRA IWS IMS DEM IAS BUC IAS Fabric Fabric
Region A Fabric Group Region B Fabric Group
Admin Admin
Region A Data Center Infrastructure Fabric Region B Data Center Infrastructure Fabric IaaS
Region A Management Domain Region B Management Domain Region A Shared Edge and Compute Domain Region B Shared Edge and Compute Domain IaaS
Admin Admin
(Edge Resource Pool) Shared Edge/Compute Domain Additional Compute Domain(s) Shared Edge/Compute Domain Additional Compute Domain(s)
(Edge Resource Pool)
Region A Management Domain Region B Management Domain

Core and Domain Architecture Workload Domains

Universal Logical Switch Universal Logical Switch
The design uses standardized building blocks called workload

Application Virtual Networks

Network Transport Domains Host Connectivity

Workload Virtual Networks

domains. Below is the standard design based on a two domain L2
UDLR External
L2
UDLR & DLR
model with a dedicated management domain and shared

for SDDC Solutions

L3 Networks L3
edge/compute domain. APP APP APP APP APP APP APP APP APP
OS OS OS OS OS OS OS OS OS

Management Domain ESXi Host Shared Edge and Compute Domain ESXi Host
Universal Logical Switch Universal Logical Switch
Universal Compute Transport Zone

North/South
Spine Spine Spine L2 L2
UDLR UDLR & DLR

Routing
10 GigE nic0 nic1 10 GigE 10 GigE nic0 nic1 10 GigE
L3 L3 L3 L3

Spine Spine Universal Management Transport Zone Local Compute Transport Zone APP APP APP APP APP APP
L3
APP APP APP APP
L3

vDS MTU 9000 vDS MTU 9000 OS OS OS OS OS OS OS OS OS OS

Routed Uplinks (ECMP) L3 L3
Management Distributed Switch Compute Distributed Switch
Layer 3 ToR Switch Management Distributed Switch Compute Distributed Switch VDP vCenter vCenter L2 L2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Core Platform
(Management) (Compute) (Management)

Services
L3 SRM PSC PSC
VMK MTU VMK MTU Edge
ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi Management 9000 Management 9000 (Management) (Compute) (Management) NSX Controllers N/S NSX EDGE N/S NSX EDGE NSX Controllers
40 GigE 40 GigE L2 (Management)
Resource
(Management) (Compute) (Compute)
Leaf Leaf Leaf Leaf Leaf Leaf Pool
Leaf Leaf VMK MTU VMK MTU VR NSX Manager NSX Manager
vMotion 9000 vMotion 9000 (Management) (Compute) (Management)
L3 L3 L3 UDLR UDLR & DLR

Any Supported Storage Any Supported Storage

VMK MTU VMK MTU
IGMP IGMP IGMP IGMP IGMP IGMP
VLAN 1611 VLAN 1612 VLAN 1613 VLAN 1614 vSAN vSAN
9000 9000
L2 L2 IGMP IGMP
L2 vSAN Recommended vSAN Recommended
Management Management

Span of VLANs
VMK MTU VMK MTU
VTEP (VXLAN) VTEP (VXLAN)

Distributed
9000 9000

Switches
10 GigE 10 GigE vMotion vMotion
Management Domain plus NFS plus NFS

Span of VLANs
(4+ Hosts) VMK MTU VMK MTU
vSAN vSAN
NFS 9000 NFS 9000 VTEP (VXLAN) VTEP (VXLAN)
Minimum 4 Nodes | vSAN ReadyNodes Recommended Minimum 4 Nodes | vSAN ReadyNodes Recommended VLAN Trunk (802.1Q) NFS NFS
Server vSphere HA and DRS Enabled vSphere HA and DRS Enabled | Business Workload Requirements vSphere Replication
VMK MTU
Uplink 01
9000 vSphere Replication North/South Uplink(s)
ESXi Host
North/South Uplink(s) External Connectivity
The management domain hosts the infrastructure components used to instantiate, Uplink 01 Uplink 02 External Connectivity
manage and monitor the SDDC. This includes the core infrastructure Workloads running in the SDDC do not have direct access to external networks.
components, such as the Platform Services Controllers, vCenter Server instances, To access external networks, tra c is routed through distributed routing to
NSX Managers, NSX Controllers for the management domain, vSphere Replication, Management vMotion VXLAN vSAN Uplink 02 External Connectivity
the NSX Edge Services Gateways in the shared edge and compute domain.
Shared Edge and Compute Domain Site Recovery Manager, as well as the SDDC monitoring and automation 172.16.11.0/24 172.16.12.0/24 172.16.13.0/24 172.16.14.0/24
(4+ Hosts) solutions like vRealize Operations, vRealize Log Insight and vRealize Automation. Expansions beyond the initial shared domain are simply compute Domains. DGW: DGW:
172.16.11.253 172.16.12.253 External Connectivity

Edge/Compute Cluster
ESXi-MGMT-01 ESXi-MGMT-02 ESXi-MGMT-03

Transport Zones
VTEPs VTEPs VTEPs VTEPs ESX-COMP-01

Management Custer
UDLR UDLR UDLR UDLR & DLR

Universal
Additional Compute Domains Management Domain Shared Edge and Compute Domain
Managed by Management Domain vCenter Server Managed by Compute Domain vCenter Server
When using the recommended L3 network transport, the top-of-rack leaf switches of each rack act as the The two 10GbE NICs on each host are connected across the top-of-rack leaf switches and teamed on the vSphere Distributed Switch via an active-active configuration. Any Supported Any Supported
The design supports L3 or L2 network transport services. For a scalable and vendor-neutral data center network, use an L3 transport. corresponding L3 interface for the associated subnets. The management domain and the shared edge and compute All port groups, except for the ones that carry VXLAN tra c, are configured for the 'Route based on physical NIC load' teaming algorithm. Storage Storage
A consolidated management and compute design is also available. domain are provided with externally accessible VLANs to access to the Internet and corporate networks. VTEP kernel ports and VXLAN tra c use the ’Route based on SRC-ID' algorithm.
All design documentation is provided for an L3 transport. Adjust the deployment and operations guidance under the context of an L2 transport. Refer to the VVD documentation. The vSphere Distributed Switch has a MTU of 9000 configured for Jumbo Frames along with with necessary VMkernel ports.
. Core and Domain Architecture Core and Domain Architecture

Distributed Logical Routing and Application Virtual Networks for Management, Operations and Automation Solutions Storage

Distributed Logical Routing All design documentation for is provided for an L3 transport with BGP based peering. vRealize Operations vRealize Automation Primary Storage Secondary Storage
and Application Virtual Networks A TechNote is provided for the alternative mixed-use or end-to-end use of OSPF. and vRealize Log Insight and vRealize Business for Cloud

Region A Region B
Caching
SSD PCIe NVMe Tier
ECMP
NSX Edge
Region A Region B ECMP
NSX Edge Read and Write Cache NFS Storage Array NFS Storage Array
Services Gateways Services Gateways

Region A Region B Region A Region B

ECMP ECMP
NSX Edge NSX Edge BGP Universal Transit Network Universal Transit Network BGP
Internet or Internet or Services Gateways Services Gateways Peering Universal Logical Switch / VXLAN Segment 192.168.10.0/24 192.168.10.0/24 Universal Logical Switch / VXLAN Segment Peering
Enterprise Enterprise BGP Universal Transit Network Universal Transit Network BGP
WAN/MPLS WAN/MPLS Peering Universal Logical Switch / VXLAN Segment 192.168.10.0/24 192.168.10.0/24 Universal Logical Switch / VXLAN Segment Peering

Management Universal Distributed Logical Router

Management Universal Distributed Logical Router

Internet or Enterprise WAN/MPLS

Internet or Enterprise WAN/MPLS

Region Independent Application Virtual Network Region Independent Application Virtual Network
Spine To Shared Edge and Compute Domain To Shared Edge and Compute Domain Spine Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment
Switches Switches Region Independent Application Virtual Network Region Independent Application Virtual Network Volume 1 Volume 2 Volume 1 Volume 2
Workload Domains Workload Domains Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment Capacity
NSX Edge Services Gateway NSX Edge Services Gateway
One-Arm Load Balancer One-Arm Load Balancer
NSX Edge Services Gateway 192.168.11.0/24 192.168.11.0/24 Capacity Export for Export for Export for Export for Export for Export for
NSX Edge Services Gateway
One-Arm Load Balancer One-Arm Load Balancer
or
Tier Content Library Log Archives Backups Content Library Log Archives Backups
L3 L3 192.168.11.0/24 192.168.11.0/24
Top-of-Rack Top-of-Rack and Templates and Templates
172.16.11.0/24 172.17.11.0/24 APP APP APP APP APP APP
Leaf Switches
L2 L2
Leaf Switches
VRA VRA VRA VRA
OS OS OS OS OS OS
APP APP APP APP APP APP
BGP Peering BGP Peering
OS OS OS OS OS OS All design documentation and validation is provided using vSAN as the primary storage system.
NSXM VDP PSC VC VC PSC VDP NSXM IWS APP APP
IWS IWS APP APP
IWS vSAN enables both all-flash and hybrid architectures. Adjust deployment and operations for supported storage systems.
Master Replica Data Master Replica Data
OS OS
The design uses NFS as a secondary storage tier.
ECMP OS OS OS OS OS OS OS OS ECMP
Node Node Node Node Node Node
OS OS
NSX Edge NSX Edge NFS is used for the content library and templates consumed by vRealize Automation blueprints and for vRealize Log Insight log archives.
Services Gateways Services Gateways
Use of vSAN ReadyNodes is recommended to ensure seamless compatibility and support. NFS is also used by any vSphere APIs for Data Protection compatible solution to store backups.
BGP Universal Transit Network Universal Transit Network vRealize Operations Replicated for Disaster Recovery
Peering Universal Logical Switch / VXLAN Segment 192.168.10.0/24 192.168.10.0/24 Universal Logical Switch / VXLAN Segment
BGP
Peering IMS APP APP
IMS IMS APP APP
IMS The configuration and assembly of the components are standardized to eliminate system variability.
OS OS OS OS

Region Dependent Application Virtual Network Region Dependent Application Virtual Network APP APP
Management Universal Distributed Logical Router DEM APP
DEM DEM APP
DEM
Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment OS OS
OS OS

192.168.31.0/24
192.168.32.0/24

Reference
Region Independent Application Virtual Network Region Independent Application Virtual Network APP APP APP APP

Region Protection and Disaster Recovery

Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment SQL BUS SQL BUS
S
OS OS OS OS

NSX Edge Services Gateway NSX Edge Services Gateway

One-Arm Load Balancer One-Arm Load Balancer APP APP APP APP
192.168.11.0/24 vRealize Automation / vRealize Orchestrator Replicated for Disaster Recovery
192.168.11.0/24 OS OS OS OS vRealize Business for Cloud
Collector Collector Collector Collector
APP APP APP APP APP
Reserved for Disaster Recovery
Node Node Node Node Region A Region A Replicated Region B Replicated Region B Networks Notable Acronyms
OS OS OS OS OS
vRealize Operations vRealize Operations
Non-Replicated Non-Replicated
BUC vRealize Business Data Collector
Region Dependent Application Virtual Network Region Dependent Application Virtual Network Protection Groups Protection Groups External Transit Network(s) BUS vRealize Business Appliance
Region Dependent Application Virtual Network Region Dependent Application Virtual Network
Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment 192.168.31.0/24 192.168.32.0/24 Universal Logical Switch / VXLAN Segment SRM DEM vRealize Automation Distributed Execution Manager
• vRealize Automation (failover/failback)
• vRealize Automation
192.168.31.0/24 192.168.32.0/24 vRealize Log Insight vRealize Log Insight IAS vRealize Automation IaaS vSphere Proxy Agent
• vRealize Business for Cloud • vRealize Business for Cloud
APP APP APP APP APP APP IMS vRealize Automation IaaS Manager Service
APP APP APP APP APP APP • vRealize Operations • vRealize Operations
OS OS OS OS OS OS Replication IWS vRealize Automation IaaS Web Server
APP APP APP APP APP APP APP APP APP APP OS OS OS OS OS OS Universal Transit Network VXLAN
Cluster Master Worker Worker Cluster Master Worker Worker vSphere Replication when using vSAN
NSXM NSX Manager
OS OS OS OS OS OS OS OS OS OS VIP Node Node Node IAS IAS BUC IAS IAS BUC
VIP Node Node Node PSC Platform Services Controller
vRealize Log Insight
SQL Microsoft SQL Server Database
vRealize Log Insight
Application Virtual Networks for SDDC Management Solutions in Region A Application Virtual Networks for SDDC Management Solutions in Region B Region A Infrastructure Management Region B Infrastructure Management SRM Site Recovery Manager
Management Distributed Port Group UDLR Universal Distributed Logical Router
vSphere Update Manager Download Service, vSphere Update Manager Download Service, 192.168.11.50 > Active Node
vRealize Operations Analytics Cluster and Remote Collectors, Regional vRealize Log Insight Cluster, vRealize Operations Remote Collectors, Regional vRealize Log Insight Cluster, 192.168.11.57 > Active Node VDP vSphere Data Protection
Distributed vRealize Automation and Proxy Agents, and vRealize Business for Cloud Server and Collector. vRealize Automation Proxy Agents and vRealize Business for Cloud Collector.
VRA VIP: 192.168.11.53 192.168.11.51 > Active Node IMS VIP: 192.168.11.59 vSphere vSphere
192.168.11.58 > Passive Node VR vSphere Replication
192.168.11.52 > Active Node NSX NSX VRA vRealize Automation Appliance
Disaster Recovery vRealize Operations Analytics Cluster,
Distributed vRealize Automation, and vRealize Business for Cloud Server. Site Recovery Manager Site Recovery Manager VTEP VXLAN Tunnel Endpoint
192.168.11.54 > Active Node Management Application Virtual Network VXLAN
IWS VIP: 192.168.11.56
192.168.11.55 > Active Node
One region is designated as the primary region and the other as the secondary region. SDDC management, automation and operations solutions are
deployed in the primary region and configured to migrate to the secondary region in the event of a disaster. All regions actively run business workloads.

Copyright © 2018 VMware, Inc. All rights reserved. Refer to the design release notes for products and versions included in the design. @vmwcf | vmware.com/go/vvd-docs