Enterprise-Scale Delivery Guide & Resources

Cloud Democratisation

Policy Driven Governance

Single Control and Management Plane

Application Centric and Archetype-

Neutral

Azure Native Design and Platform

Roadmap Alignment
Enterprise Enrolment Management Group
Identity and Access
and Azure AD and Subscription
Management
Tenants Organisation

Business Continuity
Network Topology Management and
and Disaster
and Connectivity Monitoring
Recovery

Security, Governance Platform Automation Layered Control

and Compliance and DevOps Framework *
 Enterprise-scale: Audience
Microsoft Customer Leads *
Arch Lead/Chief Architect
Customer Success Network (NetOps)
Identity and Access Management (IAM)
Customer Engineers Systems (SysOps: OS, DB)
Cloud Solution Architects Operations and Management
Customer Field Roles Security (SecOps)
DevOps and Automation
Partner/MCS/Fast Track Application Architecture

Technical Lead

(*) Customer may bring additional SMEs where and when required.
Enterprise-Scale: Workshop Objective

Design

Target DC architecture is designed that is

aligned with Enterprise-scale and in
compliance with customer requirements

All technical requirements to build a DC on Target DC architecture design agreed and

Azure are understood and captures. documented.
Enterprise-scale Delivery

• Be loyal to the design principles across the 8

critical design areas
• Capture decisions jointly agreed on with
customer
• Capture trade-offs; make customer aware of
the implications

• The next slides provides an example of how

to review and make design decisions jointly
with the customer, through the lenses of
the Enterprise-scale critical design areas
Reminder

The impact of decisions made within these critical areas will reverberate
across the Enterprise-scale architecture and influence other decisions.

Readers are strongly advised to familiarize themselves with these eight

areas, to better understand the consequences of encompassed decisions,
which may later produce trade-offs within related areas
 Enterprise-scale: Workshop
Agenda
Day 1
Day 2 Day 3 Day 4 Day 5 - [Optional]
[Enterprise Scale Architecture ->
[Networking & Connectivity] [Management and Security] [DevOps] [Show & Tell]
Landing Zone]
09:00 Introductions - Goals and Objective [as-is] Business Continuity and Disaster Critical Design Path - Show and Validate
[as-is] Networking and [as-is] Management and Monitoring
09:30 Recovery
Connectivity Review
10:00 1. File->New->Landing Zone and
Platform Management and Monitoring HA/BCDR Subscription democratisation
10:30 [as-is] Subscription Organisation and 2. IAM and AAD PIM
11:00 Application Portfolio 3. Complaint VM
Global Azure Network Topology Application Management and
11:30 [as-is] DevOps - Capability Review 4. Management and Security at-scale
Monitoring
12:00
12:30
Lunch
13:00

13:30 Enterprise Scale Design Principles and

[as-is] Security and Compliance What-must-be-true for the “Enterprise-
14:00 Critical Design Areas Connectivity to Azure - ER, NGF/WAF,
scale”
Network, DNS Ingress/Egress,
14:30 Landing Zone Design File -> New -> Region and Critical Design Path - Show and Validate
Segmentation Encryption and Key Management File -> New -> “Landing Zone”
15:00
EA, Identity and Access Management
15:30 and Management Group and 6. Platform DevOps with Enterprise-
Connectivity to Azure PaaS Services, Policy Driven Governance
16:00 Subscription Organisation "Monday Morning" Implementation - Scale in-a-box
Inbound and Outbound Internet
16:30 Accelerators/Blockers and Next Steps
Platform - Operating Model Connectivity Service Whitelisting Framework
17:00
A. Enterprise Enrollment and Azure AD Tenants
1. Planning for Enterprise Enrolment Design Decision

Only use the authentication type "Work and School Account" for all account types. Avoid using the MSA account type.
EA Administrator
Distribution Group email address
Setup the notification contact email address to ensure notifications are sent to an appropriate group mailbox. Notification Account

Assign a budget for each account and establish an alert associated with the budget. Number of Departments Name of Departments

Organisation can have a variety of structures such as functional, divisional, geographic, matrix or team structure. Leverage organizational
structure to map organization structure to enterprise enrolment. Number of EA Account Number of AAD Tenants + (Total Number of Subscription/5000)

Create a new department for IT if business domains have independent IT capabilities. Audit Interval for EA Portal
90 Days
for Microsoft account
Restrict and minimize the number of Account Owners within the Enrolment to avoid the proliferation of admin access to Subscriptions and
associated Azure resources. Environment isolation Dev/Test/Prod on EA account level

If multiple Azure AD tenants are used, ensure the Account Owner is associated with the same tenant as where subscriptions for the account Notes
are provisioned.

Setup enterprise Dev/Test/Prod environments at an EA account level to support holistic isolation.

Do not ignore notification emails sent to the notification account email address. Microsoft sends important EA wide communications to this
account.

Do not move or rename an EA Account in Azure AD. Periodically audit EA portal to review who has access.
2. Azure AD Tenants Design Decision

Leverage Azure AD SSO based on the selected planning topology. AAD Tenants Azure AD Connect for each Azure AD Tenant if identity is to be synchronized from on -
premises.
In case organisation does not have existing identity infrastructure, then it is recommended to start by implementing Azure AD only identity
deployment. Such deployment with Azure AD domain services and Enterprise mobility suite provide end to end protection for Saa S & enterprise
AAD PIM Yes/No
application as well as devices.

MFA provides a second barrier of authentication adding another layer of security. It is recommended to enforce MFA and conditional access
policies for all privileged accounts to make it more secure. MFA does provide another barrier of authentication but does not stop phishing or Notes
social engineering such as hacker taking physical possession of your phone or Sim Swapping or cloning. It is recommended that MFA should
be implemented with device management policy(such as strong pin locking and encryption and erasing device remotely when its l ost). Out of
band multifactor authentication (such as biometric) is also consider secure form of MFA.

Plan and implement for emergency access or break-glass accounts to prevent tenant-wide account lockout.

Use Azure AD privileged identity management for Identity and access management.

If Dev/Test/Prod are going to be completely isolated environments from an identity perspective, separate them at a tenant level (i.e. use
multiple tenants).

Avoid creating a new Azure AD tenant unless there is a strong IAM justification and processes are already in-place.
B. Identity and Access Management
1. Planning for identity and access management Design Decision

Use Azure RBAC to manage data plane access to resources where possible (e.g. Key Vault, Storage account, Azure SQL Database).
Azure AD Conditional Access
Document MFA Policy
policies
Deploy Azure AD Conditional Access policies for any user with rights to the Azure environment(s). Doing so will provide another mechanism to
help protect a controlled Azure environment from unauthorized access.
"Azure AD only" groups for
Naming convention for AAD groups
Azure control plane
Enforce MFA for any user with rights to the Azure environment(s). This is a requirement of many compliance frameworks and greatly lowers the
risk of credential theft and unauthorized access.
Custom Role Definitions +
Use Azure AD Privileged Identity Management (PIM) to establish zero standing access and least privilege. We recommend that customers map AAD PIM for Azure roles
the organization roles to the minimum level of access needed. Azure AD PIM can either be an extension of existing tools and processes, utilize
Azure native as outlined above, or both as needed. AAD Diagnostic Logs Log Analytics Workspace name

Use "Azure AD only" groups for Azure control plane resources in Azure AD PIM when granting access to resources.
Custom RBAC Roles and
Azure Platform Owner, NetOps, SecOps, DevOps
Add on-premises groups to the "Azure AD only" group if there is an existing group management system already in place. Usage
Use Azure AD PIM access reviews to periodically validate resource entitlements. Access reviews are part of many compliance fr ameworks so
many organizations will already have a process in place to address this requirement.
Notes

Integrate Azure AD logs with the platform-central Azure Monitor. Azure Monitor allows for a single source of truth around log and monitoring
data in Azure, giving organizations a cloud native options to meet requirements around log collection and retention.

If any data sovereignty requirements exist, custom user policies can be deployed to enforce them.

Use custom RBAC role definitions within the Azure AD tenant, considering the following key roles:
2. Authentication Inside the Landing Zone Design Decision

Leverage Azure AD SSO based on the selected planning topology. AD Placement of domain controller in each region.

In case organisation does not have existing identity infrastructure, then it is recommended to start by implementing Azure AD only identity
deployment. Such deployment with Azure AD domain services and Enterprise mobility suite provide end to end protection for SaaS & enterprise AAD DS (If required)
application as well as devices.

MFA provides a second barrier of authentication adding another layer of security. It is recommended to enforce MFA and conditional access Notes
policies for all privileged accounts to make it more secure. MFA does provide another barrier of authentication but does not stop phishing or
social engineering such as hacker taking physical possession of your phone or Sim Swapping or cloning. It is recommended that MFA should
be implemented with device management policy(such as strong pin locking and encryption and erasing device remotely when its l ost). Out of
band multifactor authentication (such as biometric) is also consider secure form of MFA.

Plan and implement for emergency access or break-glass accounts to prevent tenant-wide account lockout.

Use Azure AD privileged identity management for Identity and access management.

If Dev/Test/Prod are going to be completely isolated environments from an identity perspective, separate them at a tenant level (i.e. use
multiple tenants).

Avoid creating a new Azure AD tenant unless there is a strong IAM justification and processes are already in-place.
C. Management Group and Subscription Organization
1. Management Group Hierarchy Design Decision

• Keep the Management Group hierarchy reasonably flat with ideally no more than 3-4 levels.
• Avoid duplicating your organizational structure into a deeply nested management group hierarchy.
Management groups should be used for policy assignment purposes, NOT for billing purposes. Tenant Root Group

• Create management groups under your root level management group that represent the types of
workloads (archetype) that you will host, based on their security, compliance, connectivity, and feature
needs.

• Use resource tags, which can be enforced or appended through Azure Policy, to query and horizontally
navigate across the Management Group hierarchy.

• Create a top-level Sandboxes management group to allow users to immediately experiment with Azure. Contoso

• Use a dedicated service principal (SPN) to execute management group management operations,
subscription management operations, and role assignment.

• Assign the User Access Administrator RBAC role at the tenant root scope (/) to grant the SPN mentioned
above access at the root level. Platform Landing Zones Sandbox Decommissioned

• Create a Platform Management Group under the top-level (root) Management Group to support
common platform policy and RBAC assignment.

• Limit the number of Azure Policy assignments made at the root Management Group scope.

• Do not create any subscriptions under the "root" Management Group. Corp Online SAP
Management Connectivity Identity
2. Subscription Organization and Governance
Design Decision

Criteria's for new Subscription, Process definition and agreement on

File -> New Subscription Criteria
• Subscriptions serve as a boundary for the assignment of Azure policies. For example, secure workloads, automation process
such as PCI workloads, typically require additional policies to achieve compliance. Instead of using a Create subscriptions for platform
management group to group workloads that require PCI compliance, you can achieve the same isolation resources, such as management,
by using a subscription. connectivity, and identity (if
needed)

• Subscriptions serve as a scale unit so that component workloads can scale within the platform Notes
subscription limits.

• Subscriptions provide a management boundary for governance and isolation, allowing for a clear
separation of concerns.
C. Management Group and Subscription Organization
3. Configure Subscription Quota and Capacity Design Decision

• Leverage subscriptions as scale units, scaling out resources and subscriptions as required. App teams can
create/request new
subscriptions
• Use reserved instances to prioritize reserved capacity in required regions.
App owners can purchase
• This ensures that your workload will have the required services, even if there is a high demand for that reserved instances
resource in a given region at any time. Capacity and Limit
Monitoring Dashboard
• Establish a dashboard with custom views to monitor utilized capacity levels. Setup alerts if capacity Services and Quota needed
utilization is reaching critical levels (e.g. 90% CPU utilization). in each Region

Policies to govern
• Configure quota increase as a part of subscription provisioning (e.g. total available VM cores within a resources and quotas per
subscription). landing zones

Allowed resources within

• Govern subscription quotas using Azure Policy Subject to policy and regional availability
landing zones

• Ensure required services and features are available within the chosen deployment regions. Notes

4. Establish Cost Management

Design Decision

• Use Azure Cost Management for the first level of aggregation and make it available to Tags enforced for billing
application owners.
Additional tags being
• Use Azure Resource Tags for cost categorization and resource grouping. enforced or optional

Shared Service e.g. ASE

Notes
D. Network Topology and Connectivity
1. Planning for IP Addressing Design Decision

Plan for non-overlapping IP address spaces across Azure regions and on-premises locations well in advance.
Examples:
Use IP addresses from the address allocation for private internets (RFC 1918). VNet CIDR range for each North Europe: 10.1.0.0/16
Azure region West Europe: 10.2.0.0/16
For environments with limited private IP addresses (RFC 1918) availability, consider using IPv6. North Central US: 10.3.0.0/16
Examples:
Do not create unnecessarily large Virtual Networks (for example: /16) to ensure there is no unnecessary wastage of IP address space. XS: /28 (available IPs: 6)
S: /27 (available IPs: 22)
Azure VNet sizing
Do not create Virtual Networks without planning the required address space in advance, since adding address space will cause an outage once M: /26 (available IPs: 54)
a Virtual Network is connected via Virtual Network Peering. L: /25 (available IPs: 118)
XL: /24 (available IPs: 246)
Do not use public IP addresses for Virtual Networks, especially if the public IP addresses do not belong to the customer.
Notes

2. DNS and name resolution for on-premises and Azure resources Design Decision

For environments where name resolution in Azure is all that is required, use Azure Private DNS for resolution by creating a d elegated zone for Azure Private DNS Zones Examples:
name resolution (example azure.contoso.com). Name privatelink.blob.core.windows.net
privatelink.database.windows.net
For environments where name resolution across Azure and on-premises is required, use existing DNS infrastructure (for example, Active
Directory integrated DNS) deployed onto at least two Azure VMs and configure DNS settings in Virtual Networks to use those DNS servers. Hybrid DNS Resolver

Azure Private DNS can still be used where the Azure Private DNS Zone is linked to the VNets and DNS servers are used as hybrid resolvers with
conditional forwarding to on-premises DNS names (such as onprem.contoso.com) leveraging on-premises DNS servers.
Notes
On-premises servers can be configured with conditional forwarders to resolver VMs in Azure for the Azure Private DNS zone (for e xample,
azure.contoso.com).

Special workloads that require and/or deploy their own DNS (i.e. OpenShift) should use their preferred DNS solution.

Auto-registration should be enabled for Azure DNS to automatically manage the life cycle of the DNS records for the Virtual Machines deployed
within a Virtual Network.

Use Virtual Machines as a resolver for cross-premises DNS resolution with Private DNS.

Create the Azure Private DNS zone within a global "Connectivity" subscription.
D. Network Topology and Connectivity
3. Define an Azure Networking Topology Design Decision
4. Connectivity to Azure
A. Azure Virtual WAN Based Networking Topology (Example values) North Europe:

B. Traditional Azure Networking Topology

Resource Resource Group Parameters

Connectivity (Hub)
Loca tion: North Europe
VNet na me: contoso-hub-neu
Hub VNet contos o-global-
VNet a ddress space: 10.1.0.0/24
network-rg network
Expres sRoute Gateway: Yes (ErGw2AZ)
VPN Ga teway: No
Azure Security
Monitor Center
VNet NSG’s Route
Table
Fi rewall Policy name: contoso-fw-policy-global
Azure Fi rewall contos o-fw-
Outbound rules: (as required, for example, a llow outbound tra ffic to
Pol i cy pol icies
*.mi cros oft.com)

ER-Rg VPN-Rg NVA/AFW/

DNS contos o-global-
Azure Fi rewall Fi rewall name: contoso-azfw-neu
network
Azure Pri va te contos o-global-
VPN Gateway Pri va te DNS Zone name: azure.Contoso.com
DNS dns
Azure DDoS contos o-global- Na me: contoso-ddos-std-plan
Standard ddos Regi on: North Europe

Expres sRoute resource ID:

/s ubscriptions/<subscription-id>/resourceGroups/<resourcegroup-
Expres sRoute contos o-er-
na me>/providers/Microsoft.Network/expressRouteCircuits/<er-circuit-
ci rcui t ci rcui ts
na me>
Expres sRoute authorization key:XXXXXXXXXXXX

Network NetworkWatcher
Loca ti on: North Europe
Wa tcher RG
D. Network Topology and Connectivity
5. Connectivity to Azure PaaS Services Design Decision

Use VNet injection for supported Azure services to make them available from within a customer Virtual Network.
VNet-injected services (examples) Databricks, SQL MI, Azure Data Factory, App Gateway v2
Azure PaaS services that have been injected into a Virtual Network still perform management plane operations using public IP addresses.
Ensure that this communication is locked down within the Virtual Network using UDRs and NSGs.
Private Link services
(examples) Azure SQL, Storage (blob), Cosmos DB and Key Vault
Use Azure Private Link, where available, for shared Azure PaaS services.

Private Link is GA for several services and is in public preview for numerous ones. Private Link availability is detailed her e. Service Endpoint services
(examples) none
Access Azure PaaS services from on-premises via ExpressRoute Private Peering, using either VNet injection for dedicated Azure services or
Azure Private Link for available shared Azure services.
NVA Filtering
(examples) none
To access Azure PaaS services from on-premises when VNet injection or Private Link are not available, use ExpressRoute with Microsoft
Peering when there are no data exfiltration concerns. This would avoid transiting over the public internet.
Notes
Use VNet Service Endpoints to secure access to Azure PaaS services from within a customer VNet, but only when Private Link is not available
and when there are no data exfiltration concerns. To address data exfiltration concerns with Service Endpoints:

• Use NVA filtering.

(Examples)
• Use VNet Service Endpoint Policies for Azure Storage. • Access to Azure PaaS services from on-premises will be done via ExpressRoute
Private Peering
Do not enable VNet Service Endpoints by default on all subnets.

Do not use VNet Service Endpoints when there are data exfiltration concerns, unless NVA filtering is used.

It is strongly recommended to not implement forced tunneling to enable communication from Azure to Azure resources.
6. Planning for Inbound and Outbound Internet Connectivity Design Decision

Use Azure Firewall to govern: NVA for East-West (example) Azure Firewall
• Azure outbound traffic to the internet
• Non-HTTP/S inbound connections
• East-West traffic filtering (if required by customer) NVA for South-North (example) Azure Firewall

Use Firewall Manager (in preview) to deploy and manage Azure Firewalls across VWAN or in hub Vnets
NVA for North-South (example) Azure App Gateway with WAF and Azure Front Door
Create a global Azure Firewall policy to govern security posture across the global network environment and assign it to all A zure Firewalls. Allow
for granular policies to meet requirements of specific regions by delegating incremental Firewall Policies to local security teams via RBAC.

Use Application Gateway WAF within a "Landing Zone" Virtual Network for protecting inbound HTTP/S traffic from the internet.

Use Azure Front Door WAF policies to provide global protection across Azure regions for inbound HTTP/S connections to a "Landing Zone".

If 3rd party NVAs are required for east-west and/or south-north traffic protection/filtering:
• For VWAN network topologies, deploy the NVAs to a separate VNet (i.e. NVA VNet)
• For non-Virtual WAN network topologies, deploy the 3rd party NVAs in the central Hub VNet.

If 3rd party NVAs are required for inbound HTTP/S connections, they should be deployed within a "Landing Zone" Virtual Networ k, together with
the applications that they are protecting and exposing to the internet.
D. Network Topology and Connectivity
7. Planning for Application Delivery Design Decision

Application delivery for both internal and external facing applications should be performed within a "Landing Zones“
Application delivery from
Yes/No
For secure delivery of HTTP/S applications, use Application Gateway v2 and ensure WAF protection/policies are enabled within Landing Zones

Use a 3rd party NVAs if Application Gateway v2 cannot be used for the security of HTTP/S applications NVA
(example) Azure App Gateway and Azure Front Door
Application Gateway v2 or 3rd party NVAs used for inbound HTTP/S connections, should be deployed within the "Landing Zone" Vi rtual
Network, together with the applications that they are securing DDoS Standard
Yes/No
All public IP addresses in a "Landing Zone" should be protected with a DDoS Standard protection plan

Global HTTP/S applications that span Azure regions should be delivered and protected using Azure Front Door with WAF policies WAF Policies
(example) WAF Policies in Azure Front Door will be used
When using Azure Front Door and Application Gateway to protect HTTP/S applications, use WAF policies in Front Door and lock d own
Application Gateway to receive traffic only from Azure Front Door Notes
Global applications that span protocols other than HTTP/S should be delivered using Azure Traffic Manager
Design Decision

Subnet delegation to app- Yes/No

8. Planning for "Landing Zone" Network Segmentation owner

Delegate subnet creation to the "Landing Zone" owner. This will enable them to define how to segment workloads across subnets (i.e. single NSG default configuration (example)
large subnet, multi-tier app, VNet injected app) • Deny inbound RDP/SSH connections
• The Platform team can use Azure Policy to ensure an NSG with specific rules (such as deny inbound SSH or RDP from Internet, or • Allow communication with domain controllers
allow/block traffic across landing zones) is always associated to subnets with Deny only policies
Enable NSG Flow logs and Yes/No
NSGs must be used to protect traffic across subnets, as well as east-west traffic across the platform (inter "Landing Zone" traffic) send them to Traffic
Analytics
Application team should use Application Security Groups at the subnet level NSGs to protect multi -tier VMs within their "Landing Zone“
Notes
Use NSGs and ASGs to microsegment traffic within the "Landing Zone" and avoid using a central NVA to filter these traffic flows

Is recommended to enable NSG Flow Logs and feed them into Traffic Analytics to gain insights into internal and external traffic flows

Use NSGs to selectively allow inter "Landing Zone" connectivity

For VWAN-based network topologies, route traffic across "Landing Zones" via Azure Firewall only if the customer requires filtering and logging
capabilities for traffic flowing across "Landing Zones"
D. Network Topology and Connectivity
9. Define Network Encryption Requirements Design Decision

Yes/No

A
Yes/No

vHub Subscription
(vNet)
Partner Edge
Microsoft
Edge ER VPN Yes/No
Subscription
(vNet)

(Native) MACSec (only for ExpressRoute Direct)

On Premises Azure
B •
• (Native) Azure VWAN + ExpressRoute Private Peering + VPN Gateway
Encryption Requirement • (Native) ExpressRoute Microsoft Peering + VPN Gateway
C • (Third-party) ExpressRoute Private Peering + NVAs

Notes

10. Planning for Traffic Inspection

Azure Virtual Network TAP (VTAP) is in preview, but your must reach to azurevnettap@microsoft.com for availability details.

Network Watcher packet captures in Network Watcher is GA, but captures are limited to a maximum period of 5 hours.

As alternative to Virtual Network TAP, evaluate the following options:

Design Decision
• Use Network Watcher packet capture despite the limited capture window.

• Evaluate if NSG Flow Logs v2 provide the level of detail required. Packet capture required Yes/no

• Use 3rd party solutions for scenarios where sustained deep packet inspection is required.
Packet capture technology
• Do not develop a custom solution to mirror traffic. While this might be acceptable for small scale scenarios, this approach i s not
encouraged at scale due to complexity and supportability issues which may arise.
Notes
E - Management and Monitoring
1. Planning for Platform Management and Monitoring Design Decision

• Use a single Log Analytics workspace for centralized platform management except where Use dedicated
RBAC, retention of data types and data sovereignty requirements mandate the consideration subscription for
Subscription Name
management, separated
of separate workspaces. from app landing zones
• Export logs to Azure Storage if log retention requirements exceed 2 years. Use Azure native/first-
Leverage immutable storage with WORM policy (Write Once, Read Many) to make data party monitoring & Yes
management services
non-erasable and non-modifiable for a user-specified interval.
• Use Azure Policy for access control and compliance reporting. Data retention 1 year
This provides the ability to enforce the settings across an organization to ensure
consistent policy adherence and fast violation detection, as described in this article. Log Workspace name Name of Log Analytics Workspace
• Monitor in-guest VM configuration drift using Azure Policy.
Azure region for the Log
• Enabling guest configuration audit capabilities through policy provides application teams Analytics workspace
Region
with the ability to immediately consume the feature capabilities for their workloads with very
little effort. Automation account name Name of automation account
• Use Update Management in Azure Automation as a long-term patching mechanism, for both
Azure region for the
Windows and Linux VMs. automation account
Region

• Enforcing configuration of update management through policy ensures all VMs are included
Solutions enabled for the
in the patch management regimen, provides application teams with the ability to manage List of Solutions
workspace, such as Sentinel
patch deployment for their VMs, and provides central IT with visibility and enforcement
RBAC model for Workspace:
capabilities across all VMs. Resource centric
Centric vs resource centric
• Use Azure Network Watcher to proactively monitor traffic flows via NSG Flow Logs v2. Azure Policies to ensure
• Use deny policies to supplement Azure RBAC assignments. auditing, security,
ASC monitoring, diangostics, activity logs, VM extension etc.
diagnostics and metrics from
• Include Service and Resource Health events as part of the overall platform monitoring platform perspective
solution. Delegate update
• Do not send raw logs entries back to on premise monitoring systems, but instead adopt the management to application Yes
owners
principal of "data born in Azure, stays in Azure".
Notes
If on-premises SIEM integration is required send critical alerts instead of logs
E - Management and Monitoring
1. Planning for Application Management and Monitoring Design Decision

• Application monitoring can use dedicated Log Analytics workspaces.

• For applications that are deployed to virtual machines, logs should be stored centrally to the Allow app owners to
create/use their own
dedicated Log Analytics workspace from a platform perspective, and application teams can Yes, using Custom Roles
monitoring tools within
access the logs subject to the RBAC they have on their applications/virtual machines. the Landing Zone (RBAC)

• Use Azure Monitor Metrics for time sensitive analysis. Azure Policies to provide
built-in monitoring,
• Use shared storage accounts within the "Landing Zone" for Azure Diagnostic Extension log Yes, providing baseline for resources within scope
security and auditing for
storage when required. applications

• Leverage Azure Monitor Alerts for the generation of operational alerts. Notes
F - Business Continuity and Disaster Recovery
1. Planning for BCDR Design Decision

• Employ Azure Site Recovery for Azure to Azure Virtual Machine DR scenarios to replicate Provide baseline
backup/restore Yes, using Azure Policy
workloads across regions. capabilities
ER Failover
• Utilize native PaaS service DR capabilities.
Paired Regions
• Leverage Azure native backup capabilities. (Workload)
Notes
• Use multiple regions and peering locations for ExpressRoute connectivity.

• Refer to Azure Region Pairs documentation when selecting locations for your organizations
DR layouts.

• Use Azure paired regions when planning for BCDR.

• Planned Azure system updates are rolled out to paired regions sequentially (not at the same
time) to minimize downtime, the effect of bugs, and logical failures in the rare event of a bad
update.

• In the event of a broad outage, recovery of one region is prioritized out of every pair.
Applications that are deployed across paired regions are guaranteed to have one of the
regions recovered with priority. If an application is deployed across regions that are not
paired, recovery might delayed in the worst case the chosen region may be the last two to be
recovered.

• Avoid using overlapping IP address ranges for Production and DR sites.

G - Security, Governance and Compliance
1. Define Encryption and Key Management Design Decision
• Encryption to ensuring data privacy, compliance, and data residency in Azure.
Encryption/Key mgmt
requirement
• Federated Key Vault model to avoid transaction scale limits. Do not use centralized Key Vault
instances for application keys or secrets Key vault security
boundaries

• Required Key and Secrets compliance and encryption level Key Vault SKU / data
classification

• Default to Microsoft managed keys (MMK) for principal encryption functionality and when Key alerting, monitoring Use the platform-central Log Analytics workspace to audit key, certificate, and secret usage
and rotation
required to use customer managed keys (CMK). Including disk encryption for Virtual Machines within each Key Vault

Notes

2. Planning for Governance Design Decision

• Mechanisms and processes to maintain control over your applications and resources in Azure.
Enforce vital management and security conventions across Azure platform services, Role Enabled services All, except XYZ
Based Access Controls that control what actions authorized users can perform.
Azure policies at scope
• Based on existing requirements, regulatory and compliance controls (internal/external) -
Determine what Azure Policies and Azure RBAC role are needed. RBAC at scope

• If appropriate, addressing data exfiltration concerns. Defining requirements for private Private Endpoint
endpoints (Service Endpoints, Private Link)
Regulatory standards
e.g. SOC, HIPAA, PCI DSS, SOC TSP
• Regulatory/compliance industry standards and policies such as HIPAA, PCI DSS, SOC TSP for needed for Landing Zones

applications (landing zones). Not all application in an organization have to implement the Notes
same standards
G - Security, Governance and Compliance
3. Define Security Monitoring and Audit policy Design Decision
• Target: Visibility into what is happening within technical cloud estate. Security monitoring and
audit logging of Azure platform services. Using Azure Security Center and Azure Sentinel as E2E Security Monitoring Policy assigned at top level management group
core components to.
Update management for
Enforced using Azure Policy
Virtual machines
• Requirements for “real-time” monitoring and alerting. SIEM integration with Azure Security
Center and Azure Sentinel Azure Policies for security ASC

• Determine patch level assessment, management and update process and responsibilities for Data retention policy N days for N services
VM. Including emergency patch process.
Holisitc visibility of
Yes, using Log Analytics, Sentinel and ASC together with Policy
• Vulnerability scanning, assessment and management definition security posture

Notes
• Baseline security requirements for Azure services and desired state for Virtual machine

• Identify required data retention policy and log term storage services

4. Planning for Platform Security

Design Decision
• Target: Maintain a healthy security posture for the Azure adoption. Visibility and initial settings
and changes as the Azure Services evolve. Therefore, planning for platform security is
extremely important. Platform Security Baseline Security on by default using Azure Policy

• Capture security requirements and map it to Azure Services. Define implementation plans Azure roadmap alignment
(short/mid/long term) and align Azure roadmap items.
Incident process
• Determine incident response plan for Azure services.
Shared responsibility
• Define responsibilities, Azure (MSFT), Platform (Customer) and Application teams (Customer)
Notes
H - Platform Automation and DevOps
1. Planning for a DevOps Approach Design Decision

Establish a cross functional DevOps Platform Team to build, manage and maintain your Enterprise Scale architecture. This Infrastructure as code Git repo will have 100s if not 1000s of configuration artifacts tracked and version controlled. Platfo rm developers will
team should include members from your central IT, security, compliance, and business units teams to ensure a wide spectrum be modifying a very small subset of these artifacts on an on-going basis via Pull Requests. Since Git represents the source of truth and change
of your enterprise is represented. history, Contoso will leverage Git to determine differential changes in each pull request and trigger subsequent Azure deployment actions for
only those artifacts which have been changed instead of triggering a full deployment of all artifacts.
The list below presents a recommended set of DevOps roles for the central Platform Team.
Discover existing configurations in Azure that can serve as a platform baseline. The consequence of not discovering the exist ing
PlatformOps (Platform Operations) to environment will be no reference point to rollback or roll-forward after a deployment. Initialization is also important since it can pr
• Subscription provisioning and delegation of required network, IAM, and policies. ovide a crucial on-ramp path to infrastructure as code and allow transitioning without starting all-over. For the purpose of
initialization, the following resources are considered within the scope of the overall Azure platform. This will initialize a n empty Git
• Platform management and monitoring (holistic).
• Cost Management (holistic). repo with the current configuration establish a baseline configuration encompassing the following:
• "Platform as Code" (management of templates, scripts and other assets).
Resource Type Deployment Scope
Description
• Responsible for overall operations on Azure within the Azure AD tenant, such as managing service principles, Graph API
registration, and role definitions. Management groups, which can
Microsoft.Management/managementGroups Tenant root contain child management groups
SecOps (Security Operations) and subscriptions
• Role based access control (holistic). Subscriptions, which will be the de-
• Key management (for central services, for example SMTP, Domain Controller). Microsoft.Subscription/subscriptions Tenant root facto resource containers for
• Policy management and enforcement (holistic). workloads in Azure.
• Security monitoring and audit (holistic).
Microsoft.Management/managementGroups/subscript Placement of a subscription into a
Tenant root
NetOps (Network Operations) ions management group
Policy definitions can be created at
• Network Management (holistic).
• Allow application owners to create and manage application resources through a DevOps model. management groups and
• The list below presents a recommended DevOps role for application teams. Management group, subscriptions and can contain audit,
Microsoft.Authorization/policyDefinitions
subscription deny, append, auditIfNotExists,
AppDevOps
deployIfNotExists, and modify
• Application migration and/or transformation. policy effects
• Application management and monitoring.
• Role based access control (app resources).
PolicySetDefinitions can represent
Management group,
• Security monitoring and audit (app resources). Microsoft.Authorization/policySetDefinitions multiple policyDefinitions to
subscription
• Cost Management (app resources). simplify policyAssignment lifecycle
• Network Management (app resources).
PolicyAssignments will manifests
Management group,
In some instances, customers may wish to break AppDevOps into more granular roles such as AppDataOps for database Microsoft.Authorization/policyAssignments the runtime representation of a
management like traditional DBA roles, or AppSecOps where more security sensitive applications are concerned; this is to be subscription
policyDefinition at the given scope
expected.
Role-based access control
Management group, definition, containing actions,
Microsoft.Authorization/roleDefinitions
subscription notActions, dataActions,
dataNotActions
RoleAssignments will manifests the
Management group,
Microsoft.Authorization/roleAssignments runtime representation of a
subscription
roleDefinition at the given scope
H - Platform Automation and DevOps
2. Define Central and Federated Responsibilities Design Decision

Central teams strive to maintain full control whilst application owners seek to maximise agility. The balance between these goals can greatly The list below presents a recommended distribution of responsibilities between central IT and application teams,
influence the success of the migration striving to empower migration/transformation activities with minimal central dependencies, while still supporting
the centralized governance of security and operability across the entire estate.

Application Functions

• Application migration and/or transformation.

• Application management and monitoring (app resources).

• Key management (app keys).

• Role based access control (app resources).

• Security monitoring and audit (app resources).

• Cost Management (app resources).

• Network Management (app resources).

Central Functions

• Architecture governance.

• Subscription management.

• Platform as Code (management of templates, scripts and other assets).

• Policy management and enforcement (holistic).

• Platform management and monitoring (holistic).

• Role based access control (holistic).

• Key Management (central services).

• Network management (networks, NVAs, etc.).

• Security monitoring and audit (holistic).

Enterprise-Scale – Resources

Start with Cloud Adoption Framework Enterprise-Scale Landing Zone

Overview Enterprise-scale Executive Briefing
Enterprise-Scale Architecture and Design Guideline (GitHub)
Enterprise-Scale Reference Implementation
Enterprise-Scale – Resources

Step 1: Join the Enterprise-scale Readiness Team with this code lvi2dtb
Step 2: Check out the recorded Virtual Training Sessions to learn about Enterprise-scale
Step 3: Review the CSA Delivery Guide here
Step 4: Try out the reference implementation on your own here
Enterprise-scale pilot CSA Delivery Recording - Plan your workshop, review the CSA delivery video and CSA Delivery
Guide here
CSU Customer Pilot and how to nominate a customer - More information on Pilot requirements and process for
nomination.
Azure Learning Circles Overview of Enterprise-scale – Overview presentation of Enterprise-scale and demonstration of the
reference implementation.
Office hours Recordings- Pilot Office hours recordings for June.
Enterprise-scale Workshop

CSA Guidance for Partner

Scenarios
Enterprise-scale | Guidance for Partner Scenarios
Scenario Guidance

Customer pursing CSU CSA can look-up the list of eligible partners participating and determine which partners may be a good fit. CSU CSA to reach out to
engagement with the OCP CSA(s) for alignment and further discussion on which partner is the best fit for this customer scenario.
a partner (no Once a partner is confirmed, CSU CSA must (1) Submit customer nomination with partner name, and (2) Create MSX success engage ment
preference)

Customer pursuing The expectation is for Partner CSA to reach out to the customer (CSU) CSA and rest of the customer account team to ensure alignment.
engagement with From there, if the decision has been made to move forward, the CSU CSA is responsible for creating a success engagement in MSX.
preferred (incumbent)
partner Once a partner is confirmed, CSU CSA must (1) Submit customer nomination with partner name, and (2) Create MSX success engage ment

Partner pursuing OCP CSA can look up the list of eligible/nominated customers . However, it is “critical” that in this scenario, the OCP CSA reaches out to the
engagement a CSU CSA to align and coordinate PRIOR to confirming the possibility of an engagement between the partner and customer.
customer (no
preference) Once a partner is confirmed, CSU CSA must (1) Submit customer nomination with partner name, and (2) Create MSX success engage ment

Partner pursuing Note that only a select number of partners are eligible and registered for participating.
engagement with The very first step here is for the CSU CSA to look up the Partner name on the dashboard to confirm that the Partner is Enterprise-scale
preferred customer(s) ready. From there, the OCP CSA and CSU CSA need to have a joint discussion and ensure alignment prior to reaching out to the customer.
Ultimately, this is a joint decision between customer, OCP CSA, customer CSA on the best partner for the circumstance, considering which
partner is able to act fastest and in a way that is compliant to selection criteria from Corp. Ideally, Microsoft (CSU + OCP) should have a
solid recommendation for the customer on which partner to engage. While we want to honor the strong customer preference, we a lso
want to make sure the partner is capable and bought into the mission of the project.
Once a partner is confirmed, CSU CSA must (1) Submit customer nomination with partner name, and (2) Create MSX success engagement
Enterprise-scale | Guidance for Partner Scenarios
Scenario Guidance

Please note: The criteria below has been outlined to highlight specifics and nuances of the Enterprise-scale Remember that at a
Customer and Partner high level, the partner-customer engagement here is no different than any other engagement. As such, please be sure to follow
alignment on success standard processes, rules, etc.
criteria
Partner Engagement success criteria
1. Partner has awareness of 1st party Enterprise-scale architecture reference implementation.
2. Partner has their own toolchain for production build-out of an Enterprise-scale architecture
3. Partner is committed to publish GTM offering (at later date) to deliver Enterprise-scale engagements using the Enterprise-
scale architecture design guidance

Customer Engagement success criteria

1. Customer has defined “what is a landing zone”?
2. Customer has agreed on critical design decisions to define their target state by following the Enterprise-scale architecture
design methodology
3. Customer has at least one workload running in production by the end of 8 weeks post-workshop using a landing zone

Understanding Enterprise Scale Architecture vs. Reference Implementation

1. The Enterprise-scale architecture represents the strategic design path and target technical state for your Azure environment.
2. It will continue to evolve in lockstep with the Azure platform and is ultimately defined by design decisions aligned with
defined principles to safeguard your Azure journey

Enterprise-scale comes in two buckets:

• Architecture – This is the Microsoft opinion on what should be done based on real world work with our strategic customers.
* Note; If partner deviates from the design principles the value prop is differentiated
• Reference implementation – This is a first partner example on how this could be implemented. You do not have to use
Enterprise-scale reference implementation if you are following the design guidelines and principles.
Enterprise-scale | Guidance for Partner Scenarios
Scenario Guidance

OCP and CSU CSAs Before Design Workshop

aligned. • OCP CSA: Ensure partner is well educated on the Enterprise-scale landing zone and guidance (solution briefing)
• CSU CSA: Ensure customer is well educated on the Enterprise-scale landing zone and guidance (solution briefing)
Partner is contributing • Joint OCP-CSU:
to the customer • Ensure Alignment between CSU-OCP and Customer-Partner, as well as confirmation of expectations, goals and review of agenda
Design workshop. should happen prior to the workshop.
• OCP CSA expected to join workshop with Partner
• CSU CSA expected to join workshop Customer

During Design Workshop

• Review customer environment:
• Deep understanding of customer environment, workloads, and technical constraints
• Review Success Criteria: All technical requirements to build a DC on Azure are understood and captured.

• Design & Finalize Customer Architecture Approach:

• Target DC architecture is designed that is aligned with Enterprise-scale and in compliance with customer requirements
• Design Success Criteria: Target DC architecture design agreed and documented.

After Design Workshop

Follow-up on architectural decisions taken in the workshop. Agreement for partners to translate the decision to the customer’s
implementation. Next step is to set project timelines/expectations, similar to any standard partner-customer engagement.

• Be loyal to the design principles across the 8 critical design areas

• Capture decisions jointly agreed on with customer
• Capture trade-offs; make customer aware of the implications
• Be sure to review Enterprise-Scale CSA Delivery Guide.pptx for a deep dive into these 8 areas:

• Important Expected Outcome:

• Customer production workload is expected to land in customer tenants within 7 weeks
from workshop delivery
Enterprise-scale | Guidance for Partner Scenarios
Scenario Guidance

OCP and CSU CSAs Before Design Workshop

aligned. • OCP CSA: Ensure partner is well educated on the Enterprise-scale landing zone and guidance (solution briefing)
• CSU CSA: Ensure customer is well educated on the Enterprise-scale landing zone and guidance (solution briefing)
Partner is contributing • Joint OCP-CSU:
to the customer • Ensure Alignment between CSU-OCP and Customer-Partner, as well as confirmation of expectations, goals and review of agenda
Design workshop. should happen prior to the workshop.
• OCP CSA expected to join workshop with Partner
• CSU CSA expected to join workshop Customer

During Design Workshop

• Review customer environment:
• Deep understanding of customer environment, workloads, and technical constraints
• Review Success Criteria: All technical requirements to build a DC on Azure are understood and captured.

• Design & Finalize Customer Architecture Approach:

• Target DC architecture is designed that is aligned with Enterprise-scale and in compliance with customer requirements
• Design Success Criteria: Target DC architecture design agreed and documented.

After Design Workshop

Follow-up on architectural decisions taken in the workshop. Agreement for partners to translate the decision to the customer’s
implementation. Next step is to set project timelines/expectations, similar to any standard partner-customer engagement.

• Be loyal to the design principles across the 8 critical design areas

• Capture decisions jointly agreed on with customer
• Capture trade-offs; make customer aware of the implications
• Be sure to review Enterprise-Scale CSA Delivery Guide.pptx for a deep dive into these 8 areas:

• Important Expected Outcome:

• Customer production workload is expected to land in customer tenants within 7 weeks
from workshop delivery
Design Decision (Mock Side)