Professional Documents
Culture Documents
74a - Hyer-V+configuration+guide
74a - Hyer-V+configuration+guide
Virtualization
Guarded fabric and shielded VMs
Overview
Plan
Deploy
Manage
Troubleshoot
Hyper-V
Technology Overview
What's new in Hyper-V
System requirements
Supported Windows guest operating systems
Supported Linux and FreeBSD VMs
Feature compatibility by generation and guest
Get started
Plan
Deploy
Manage
Hyper-V Performance Tuning
Hyper-V Virtual Switch
Remote Direct Memory Access (RDMA) and Switch Embedded Teaming (SET)
Manage Hyper-V Virtual Switch
Virtualization
11/3/2017 • 3 min to read • Edit Online
Virtualization in Windows Server 2016 is one of the foundational technologies required to create
your software defined infrastructure. Along with networking and storage, virtualization features
deliver the flexibility you need to power workloads for your customers.
Windows Server 2016 Virtualization technologies include updates to Hyper-V, Hyper-V Virtual Switch, and Guarded
Fabric and Shielded Virtual Machines (VMs), that improve security, scalability, and reliability. Updates to failover
clustering, networking, and storage make it even easier to deploy and manage these technologies when used with
Hyper-V.
Windows Containers is a new technology that offers you another way to deploy flexible, software-based computing
power.
NOTE
To download Windows Server 2016, see Windows Server Evaluations.
The following sections contain brief technology overviews and links to Virtualization documentation.
Hyper-V
The Hyper-V technology provides computing resources through hardware virtualization. Hyper-V creates a
software version of computer, called a virtual machine, which you use to run an operating system and applications.
You can run multiple virtual machines at the same time, and can create and delete them as needed.
Hyper-V requires specific hardware to create the virtualization environment. For details, see System requirements
for Hyper-V on Windows Server 2016.
Hyper-V on Windows Server 2016
Hyper-V is a server role in both Windows Server 2016 Datacenter and Standard editions.
Learn more about Hyper-V, the hardware you need, the operating systems you can run in your virtual machines,
and more. If you're new to Hyper-V, start with the Hyper-V Technology Overview.
For more information, see Hyper-V on Windows Server 2016
Hyper-V on Windows 10
Hyper-V is available in some versions of Windows 10, Windows 8.1, and Windows 8.
Hyper-V on Windows is geared toward development and test activities and gives you a quick and easy way to run
different operating systems without deploying more hardware.
For more information, see Hyper-V on Windows 10.
Microsoft Hyper-V Server 2016
The Hyper-V technology is also available separately from Windows and Windows Server, as a free, standalone
product. Hyper-V Server is commonly used as the host in a virtualized desktop infrastructure (VDI) environment.
For more information, see Microsoft Hyper-V Server 2016.
One of the most important goals of providing a hosted environment is to guarantee the security of the virtual
machines running in the environment. As a cloud service provider or enterprise private cloud administrator, you
can use a guarded fabric to provide a more secure environment for VMs. A guarded fabric consists of one Host
Guardian Service (HGS) - typically, a cluster of three nodes - plus one or more guarded hosts, and a set of shielded
virtual machines (VMs).
IMPORTANT
Ensure that you have installed the latest cumulative update before you deploy shielded virtual machines in production.
Videos, blog, and overview topic about guarded fabrics and shielded
VMs
Video: Introduction to Shielded Virtual Machines in Windows Server 2016
Video: Dive into Shielded VMs with Windows Server 2016 Hyper-V
Video: Deploying Shielded VMs and a Guarded Fabric with Windows Server 2016
Blog: Datacenter and Private Cloud Security Blog
Overview: Guarded fabric and shielded VMs overview
Planning topics
Planning guide for hosters
Planning guide for tenants
Deployment topics
Deployment Guide
Quick start
Deploy HGS
Deploy guarded hosts
Configuring the fabric DNS for hosts that will become guarded hosts
Deploy a guarded host using AD mode
Deploy a guarded host using TPM mode
Confirm guarded hosts can attest
Shielded VMs - Hosting service provider deploys guarded hosts in VMM
Deploy shielded VMs
Create a shielded VM template
Prepare a VM Shielding helper VHD
Set up Windows Azure Pack
Create a shielding data file
Deploy a shielded VM by using Windows Azure Pack
Deploy a shielded VM by using Virtual Machine Manager
TPM-trusted attestation: Offers the strongest possible Guarded hosts that can run shielded VMs are approved based
protections but also requires more configuration steps. Host on their TPM identity, measured boot sequence and code
hardware and firmware must include TPM 2.0 and UEFI 2.3.1 integrity policies so that you can ensure that these hosts are
with secure boot enabled. only running approved code.
Admin-trusted attestation: Intended to support existing Guarded hosts that can run shielded VMs are approved by
host hardware where TPM 2.0 is not available. Requires fewer the Host Guardian Service based on membership in a
configuration steps and is compatible with commonplace designated Active Directory Domain Services (AD DS) security
server hardware. group.
BitLocker encrypted disks (OS disks and data disks) Shielded VMs use BitLocker to protect their disks. The
BitLocker keys needed to boot the VM and decrypt the disks
are protected by the shielded VM's virtual TPM using
industry-proven technologies such as secure measured boot.
While shielded VMs only automatically encrypt and protect
the operating system disk, you can encrypt data drives
attached to the shielded VM as well.
Deployment of new shielded VMs from "trusted" When deploying new shielded VMs, tenants are able to
template disks/images specify which template disks they trust. Shielded template
disks have signatures that are computed at a point in time
when their content is deemed trustworthy. The disk
signatures are then stored in a signature catalog, which
tenants securely provide to the fabric when creating shielded
VMs. During provisioning of shielded VMs, the signature of
the disk is computed again and compared to the trusted
signatures in the catalog. If the signatures match, the shielded
VM is deployed. If the signatures do not match, the shielded
template disk is deemed untrustworthy and deployment fails.
Protection of passwords and other secrets when a When creating VMs, it is necessary to ensure that VM secrets,
shielded VM is created such as the trusted disk signatures, RDP certificates, and the
password of the VM's local Administrator account, are not
divulged to the fabric. These secrets are stored in an
encrypted file called a shielding data file (a .PDK file), which is
protected by tenant keys and uploaded to the fabric by the
tenant. When a shielded VM is created, the tenant selects the
shielding data to use which securely provides these secrets
only to the trusted components within the guarded fabric.
Tenant control of where the VM can be started Shielding data also contains a list of the guarded fabrics on
which a particular shielded VM is permitted to run. This is
useful, for example, in cases where a shielded VM typically
resides in an on-premises private cloud but may need to be
migrated to another (public or private) cloud for disaster
recovery purposes. The target cloud or fabric must support
shielded VMs and the shielded VM must permit that fabric to
run it.
What are the types of virtual machines that a guarded fabric can run?
Guarded fabrics are capable of running VMs in one of three possible ways:
1. A normal VM offering no protections above and beyond previous versions of Hyper-V
2. An encryption-supported VM whose protections can be configured by a fabric admin
3. A shielded VM whose protections are all switched on and cannot be disabled by a fabric admin
Encryption-supported VMs are intended for use where the fabric administrators are fully trusted. For example, an
enterprise might deploy a guarded fabric in order to ensure VM disks are encrypted at-rest for compliance
purposes. Fabric administrators can continue to use convenient management features, such VM console
connections, PowerShell Direct, and other day-to-day management and troubleshooting tools.
Shielded VMs are intended for use in fabrics where the data and state of the VM must be protected from both
fabric administrators and untrusted software that might be running on the Hyper-V hosts. For example, shielded
VMs will never permit a VM console connection whereas a fabric administrator can turn this protection on or off
for encryption supported VMs.
The following table summarizes the differences between encryption-supported and shielded VMs.
CAPABILITY GENERATION 2 ENCRYPTION SUPPORTED GENERATION 2 SHIELDED
Secure Boot Yes, required but configurable Yes, required and enforced
Encrypt VM state and live migration Yes, required but configurable Yes, required and enforced
traffic
Virtual Machine Connection (Console), On, cannot be disabled Disabled (cannot be enabled)
HID devices (e.g. keyboard, mouse)
1 Traditional debuggers that attach directly to a process, such as WinDbg.exe, are blocked for
shielded VMs
because the VM's worker process (VMWP.exe) is a protected process light (PPL). Alternative debugging techniques,
such as those used by LiveKd.exe, are not blocked. Unlike shielded VMs, the worker process for encryption
supported VMs does not run as a PPL so traditional debuggers like WinDbg.exe will continue to function normally.
Both shielded VMs and encryption-supported VMs continue to support commonplace fabric management
capabilities, such as Live Migration, Hyper-V replica, VM checkpoints, and so on.
Host Guardian Service (HGS) A Windows Server role that is installed on a secured cluster of
bare-metal servers that is able to measure the health of a
Hyper-V host and release keys to healthy Hyper-V hosts
when powering-on or live migrating shielded VMs. These two
capabilities are fundamental to a shielded VM solution and are
referred to as the Attestation service and Key Protection
Service respectively.
TERM DEFINITION
guarded host A Hyper-V host on which shielded VMs can run. A host can
only be considered guarded when it has been deemed healthy
by HGS' Attestation service. Shielded VMs cannot be
powered-on or live migrated to a Hyper-V host that has not
yet attested or that failed attestation.
guarded fabric This is the collective term used to describe a fabric of Hyper-V
hosts and their Host Guardian Service that has the ability to
manage and run shielded VMs.
shielded virtual machine (VM) A virtual machine that can only run on guarded hosts and is
protected from inspection, tampering and theft from malicious
fabric admins and host malware.
HGS administrator A trusted administrator in the public or private cloud that has
the authority to manage the policies and cryptographic
material for guarded hosts, that is, hosts on which a shielded
VM can run.
provisioning data file or shielding data file (PDK file) An encrypted file that a tenant or user creates to hold
important VM configuration information and to protect that
information from access by others. For example, a shielding
data file can contain the password that will be assigned to the
local Administrator account when the VM is created.
See also
Guarded fabric and shielded VMs
Blog: Datacenter and Private Cloud Security Blog
Video: Introduction to Shielded Virtual Machines in Windows Server 2016
Video: Dive into Shielded VMs with Windows Server 2016 Hyper-V
Planning a Guarded Fabric
6/19/2017 • 1 min to read • Edit Online
The following topics cover planning for the deployment of a guarded fabric and shielded virtual machines (VMs):
Guarded Fabric and Shielded VM Planning Guide for Hosters
Compatible hardware with Windows Server 2016 Virtualization-based protection of Code Integrity
Guarded Fabric and Shielded VM Planning Guide for Tenants
Guarded Fabric and Shielded VM Planning Guide for
Hosters
10/17/2017 • 6 min to read • Edit Online
This topic covers planning decisions that will need to be made to enable shielded virtual machines to run on your
fabric. Whether you upgrade an existing Hyper-V fabric or create a new fabric, running shielded VMs consists of
two main components:
The Host Guardian Service (HGS) provides attestation and key protection so that you can make sure that
shielded VMs will run only on approved and healthy Hyper-V hosts.
Approved and healthy Windows Server 2016 Hyper-V hosts on which shielded VMs (and regular VMs) can run
— these are known as guarded hosts.
AREA DETAILS
Sizing Each mid-size (8 core/4 GB) HGS server node can handle
1,000 Hyper-V hosts
Management Designate specific people who will manage HGS. They should
be separate from fabric administrators. For comparison, HGS
clusters can be thought of in the same manner as a Certificate
Authority (CA) in terms of administrative isolation, physical
deployment and overall level of security sensitivity.
Host Guardian Service Active Directory By default, HGS installs its own internal Active Directory for
management. This is a self-contained, self-managed forest and
is the recommended configuration to help isolate HGS from
your fabric.
Host Guardian Service keys A Host Guardian Service uses two asymmetric key pairs — an
encryption key and a signing key — each represented by an
SSL certificate. There are two options to generate these keys:
1. Internal certificate authority – you can generate these
keys using your internal PKI infrastructure. This is
suitable for a datacenter environment.
2. Publicly trusted certificate authorities – use a set of
keys obtained from a publicly trusted certificate
authority. This is the option that hosters should use.
Note that while it is possible to use self-signed certificates, it is
not recommended for deployment scenarios other than
proof-of-concept labs.
Host Guardian Service key storage For the strongest possible security, we recommend that HGS
keys are created and stored exclusively in a Hardware Security
Module (HSM). If you are not using HSMs, applying BitLocker
on the HGS servers is strongly recommended.
Branch office considerations If your Hyper-V host is running in a branch office, it needs to
have connectivity to the Host Guardian Service to power-on
or to live migrate shielded VMs.
Compatible hardware with Windows Server 2016
Virtualization-based protection of Code Integrity
6/19/2017 • 1 min to read • Edit Online
Windows Server 2016 introduces a new Virtualization-based code protection to help protect physical and virtual
machines from attacks that modify system code. To achieve this high protection level, Microsoft works in tandem
with the computer hardware manufactures (Original Equipment Manufacturers, or OEMs) to prevent malicious
writes into system execution code. This protection can be applied to any system and is being used as one of the
building blocks for implementing the Hyper-V host health for shielded virtual machines (VMs).
As with any hardware based protection, some systems might not be compliant due to issues such as incorrect
marking of memory pages as executables or by actually trying to modify code at run time, which may result in
unexpected failures including data loss or a blue screen error (also called a stop error).
To be compatible and fully support the new security feature in Windows Server 2016, OEMs need to implement
the Memory Address Table defined in UEFI 2.6, which was published in Jan. 2016. The adoption of the new UEFI
standard takes time; meanwhile, to prevent customers encountering issues, we want to provide information about
systems and configurations that we have tested this feature set with as well as systems that we know to be not
compatible.
Non-compatible systems
The following configurations are known to be non-compatible with Virtualization-based protection of code
integrity and cannot be used as a host for Shielded VMs:
Dell PowerEdge Servers running PERC H330 RAID Controllers For more information, see the following article
from Dell Support H330 – Enabling “Host Guardian Hyper-V Support” or “Device Guard” on Win 2016 OS
causes OS boot failure.
Compatible systems
These are the systems we and our partners have been testing in our environment. Please make sure that you verify
the system works as expected in your environment:
Virtual Machines – You can enable Virtualization-based protection of code integrity on virtual machines that
run on a Windows Server 2016 Hyper-V host.
Guarded Fabric and Shielded VM Planning Guide for
Tenants
10/17/2017 • 6 min to read • Edit Online
This topic focuses on VM owners who would like to protect their virtual machines (VMs) for compliance and
security purposes. Regardless of whether the VMs run on a hosting provider’s guarded fabric or a private guarded
fabric, VM owners need to control the security level of their shielded VMs, which includes maintaining the ability to
decrypt them if needed.
There are three areas to consider when using shielded VMs:
The security level for the VMs
The cryptographic keys used to protect them
Shielding data—sensitive information used to create shielded VMs
Shielding data
Shielding data contains the secrets necessary to deploy shielded or encryption-supported VMs. It is also used when
converting regular VMs to shielded VMs.
Shielding data is created using the Shielding Data File Wizard and is stored in PDK files which VM owners upload
to the guarded fabric.
Shielded VMs help protect against attacks from a compromised virtualization fabric, so we need a safe mechanism
to pass sensitive initialization data, such as the administrator’s password, domain join credentials, or RDP
certificates, without revealing these to the virtualization fabric itself or to its administrators. In addition, shielding
data contains the following:
1. Security level – Shielded or encryption-supported
2. Owner and list of trusted Host Guardians where the VM can run
3. Virtual machine initialization data (unattend.xml, RDP certificate)
4. List of trusted signed template disks for creating the VM in the virtualization environment
When creating a shielded or encryption-supported VM or converting an existing VM, you will be asked to select the
shielding data instead of being prompted for the sensitive information.
How many shielding data files do I need? A single shielding data file can be used to create every shielded VM.
If, however, a given shielded VM requires that any of the four items be different, then an additional shielding data
file is necessary. For example, you might have one shielding data file for your IT department and a different
shielding data file for the HR department because their initial administrator password and RDP certificates differed.
While using separate shielding data files for each shielded VM is possible, it is not necessarily the optimal choice
and should be done for the right reasons. For example, if every shielded VM needs to have a different administrator
password, consider instead using a password management service or tool such as Microsoft’s Local Administrator
Password Solution (LAPS).
« PL A N A GUA RDED Q U IC K S T A R T
F A B R IC »
One of the most important goals of providing a hosted environment is to guarantee the security of the virtual
machines running in the environment. As a cloud service provider or enterprise private cloud administrator, you
can use a guarded fabric to provide a more secure environment for VMs. A guarded fabric consists of one Host
Guardian Service (HGS) - typically, a cluster of three nodes - plus one or more guarded hosts, and a set of shielded
virtual machines (VMs).
See also
Guarded fabric and shielded VMs
Quick start for guarded fabric deployment
12/4/2017 • 7 min to read • Edit Online
This topic explains what a guarded fabric is, its requirements, and a summary of the deployment process. For
detailed deployment steps, see Deploying the Host Guardian Service for guarded hosts and shielded VMs.
Prefer video? See the Microsoft Virtual Academy course Deploying Shielded VMs and a Guarded Fabric with
Windows Server 2016.
If your existing Hyper-V servers don’t meet the prerequisites for TPM mode (for example, they do not have TPM
2.0), you can initialize HGS using Admin-based attestation (AD mode), which requires an Active Directory trust with
the fabric domain.
In our example, let’s say Contoso initially deploys in AD mode in order to immediately meet compliance
requirements, and plans to convert to the more secure TPM-based attestation after suitable server hardware can be
purchased.
Step 3: Extract identities, hardware baselines, and code integrity
policies
The process to extract identities from Hyper-V hosts depends on the attestation mode being used.
For AD mode, the ID of the host is its domain-joined computer account, which must be a member of a designated
security group in the fabric domain. Membership in the designated group is the only determination of whether the
host is healthy or not.
In this mode, the fabric admin is solely responsible for ensuring the health of the Hyper-V hosts. Since HGS plays
no part in deciding what is or is not allowed to run, malware and debuggers will function as designed.
However, debuggers that attempt to attach directly to a process (such as WinDbg.exe) are blocked for shielded
VMs because the VM’s worker process (VMWP.exe) is a protected process light (PPL). Alternative debugging
techniques, such as those used by LiveKd.exe, are not blocked. Unlike shielded VMs, the worker process for
encryption supported VMs does not run as a PPL so traditional debuggers like WinDbg.exe will continue to
function normally.
Stated another way, the rigorous validation steps used for TPM mode are not used for AD mode in any way.
For TPM mode, three things are required:
1. A public endorsement key (or EKpub) from the TPM 2.0 on each and every Hyper-V host. To capture the EKpub,
use Get-PlatformIdentifier .
2. A hardware baseline. If each of your Hyper-V hosts are identical, then a single baseline is all you need. If they
are not, then you’ll need one for each class of hardware. The baseline is in the form of a Trustworthy Computing
Group logfile, or TCGlog. The TCGlog contains everything that the host did, from the UEFI firmware, through the
kernel, right up to where the host is entirely booted. To capture the hardware baseline, install the Hyper-V role
and the Host Guardian Hyper-V Support feature and use Get-HgsAttestationBaselinePolicy .
3. A Code Integrity policy. If each of your Hyper-V hosts are identical, then a single CI policy is all you need. If they
are not, then you’ll need one for each class of hardware. Windows Server 2016 and Windows 10 both have a
new form of enforcement for CI policies, called Hypervisor-enforced Code Integrity (HVCI). HVCI provides
strong enforcement and ensures that a host is only allowed to run binaries that a trusted admin has allowed it
to run. Those instructions are wrapped in a CI policy that is added to HGS. HGS measures each host’s CI policy
before they’re permitted to run shielded VMs. To capture a CI policy, use New-CIPolicy . The policy must then be
converted to its binary form using ConvertFrom-CIPolicy .
That’s all—the guarded fabric is built, in terms of the infrastructure to run it.
Now you can create a shielded VM template disk and a shielding data file so shielded VMs can be provisioned
simply and securely.
A trustworthy administrator, such as the fabric administrator or the VM owner, will need a certificate (often
provided by a Hosting Service Provider) to sign the VHDX template disk.
The disk signature is computed over the OS partition of the virtual disk. If anything changes on the OS partition,
the signature will also change. This allows users to strongly identify which disks they trust by specifying the
appropriate signature.
Review the template disk requirements before you get started.
The shielding data file also includes the security policy setting for the shielded VM. You must choose one of two
security policies when you create a shielding data file:
Shielded
The most secure option, which eliminates many administrative attack vectors.
Encryption supported
A lesser level of protection that still provides the compliance benefits of being able to encrypt a VM, but
allows Hyper-V admins to do things like use VM console connection and PowerShell Direct.
You can add optional management pieces like VMM or Windows Azure Pack. If you’d like to create a VM without
installing those pieces, see Step by step – Creating Shielded VMs without VMM.
« Q U IC K P REP A RE FOR HG S
STA RT »
See also
Deploying the Host Guardian Service for guarded hosts and shielded VMs
Configuration steps for Hyper-V hosts that will become guarded hosts
Review prerequisites for the Host Guardian Service
10/24/2017 • 1 min to read • Edit Online
« DEPLOY O B T A IN C E R T IF IC A T E S F O R H G S
HG S »
This topic covers HGS prerequisites and initial steps to prepare for the HGS deployment.
Prerequisites
Hardware: HGS can be run on physical or virtual machines, but physical machines are recommended.
If you want to run HGS as a three-node physical cluster (for availability), you must have three physical
servers. (As a best practice for clustering, the three servers should have very similar hardware.)
Operating system: Windows Server 2016, Standard or Datacenter edition.
Server Roles: Host Guardian Service and supporting server roles.
Configuration permissions/privileges for the fabric (host) domain: You will need to configure DNS
forwarding between the fabric (host) domain and the HGS domain. If you are using Admin-trusted
attestation (AD mode), you will need to configure an Active Directory trust between the fabric domain and
the HGS domain.
See also
Deploying the Host Guardian Service for guarded hosts and shielded VMs
Configuration steps for Hyper-V hosts that will become guarded hosts
Obtain certificates for HGS
10/17/2017 • 3 min to read • Edit Online
« IN S T A L L H G S
P R E R E Q U IS IT E S »
When you deploy HGS, you will be asked to provide signing and encryption certificates that are used to protect the
sensitive information needed to start up a shielded VM. These certificates never leave HGS, and are only used to
decrypt shielded VM keys when the host on which they're running has proven it is healthy. Tenants (VM owners)
use the public half of the certificates to authorize your datacenter to run their shielded VMs. This section covers the
steps required to obtain compatible signing and encryption certificates for HGS.
Crypto provider Any Key Storage Provider (KSP). Legacy Cryptographic Service
Providers (CSPs) are not supported.
Key renewal policy Renew with the same key. Renewing HGS certificates with
different keys will prevent shielded VMs from starting up.
These requirements apply whether you are using certificates backed by hardware or software. For security reasons,
it is recommended that you create your HGS keys in a Hardware Security Module (HSM) to prevent the private keys
from being copied off the system. Follow the guidance from your HSM vendor to request certificates with the
above attributes and be sure to install and authorize the HSM KSP on every HGS node.
Every HGS node will require access to the same signing and encryption certificates. If you are using software-
backed certificates, you can export your certificates to a PFX file with a password and allow HGS to manage the
certificates for you. You can also choose to install the certs into the local machine's certificate store on each HGS
node and provide the thumbprint to HGS. Both options are explained in the Initialize the HGS Cluster topic.
$certificatePassword = Read-Host -AsSecureString -Prompt "Enter a password for the PFX file"
Subject name Name of your HGS cluster (distributed network name). This will
be the concatenation of your HGS service name provided to
Initialize-HgsServer and your HGS domain name.
Subject alternative name If you will be using a different DNS name to reach your HGS
cluster (e.g. if it is behind a load balancer), be sure to include
those DNS names in the SAN field of your certificate request.
The options for specifying this certificate when initializing the HGS server are covered in Configure the first HGS
node. You can also add or change the SSL certificate at a later time using the Set-HgsServer cmdlet.
Choose whether to install HGS in its own dedicated
forest or in an existing bastion forest
10/17/2017 • 1 min to read • Edit Online
« O B T A IN C E R T IF IC A T E S F O R IN IT IA L IZ E H G S
HG S »
The Active Directory forest for HGS is sensitive because its administrators have access to the keys that control
shielded VMs. The default installation will set up a new forest dedicated for HGS and configure other
dependencies. This option is recommended because the environment is self-contained and known to be secure
when it is created.
The only technical requirement for installing HGS in an existing forest is that it be added to the root domain; non-
root domains are not supported. But there are also operational requirements and security-related best practices
for using an existing forest. Suitable forests are purposely built to serve one sensitive function, such as the forest
used by Privileged Access Management for AD DS or an Enhanced Security Administrative Environment (ESAE)
forest. Such forests usually exhibit the following characteristics:
They have few admins (separate from fabric admins)
They have a low number of logons
They are not general-purpose in nature
General purpose forests such as production forests are not suitable for use by HGS. Fabric forests are also
unsuitable because HGS needs to be isolated from fabric administrators.
Choose the installation option that best suits your environment:
Install HGS in its own dedicated forest
Install HGS in an existing bastion forest
Install HGS in a new forest
10/17/2017 • 1 min to read • Edit Online
« PREPA RE FOR IN IT IA L IZ E H G S
HG S »
Install HGS
The Host Guardian Service should be installed in a separate Active Directory forest than the Hyper-V hosts and
fabric managers. If a secure bastion forest is not already available in your environment, follow these steps to have
HGS set up one for you.
Ensure that the HGS machine is not joined to a domain before you start.
1. In an elevated Windows PowerShell console, run the following commands to install the Host Guardian
Service and configure its domain. The password you specify here will only apply to the Directory Services
Restore Mode password for Active Directory; it will not change your admin account's login password. You
may provide any domain name of your choosing to the -HgsDomainName parameter.
Next steps
For the next steps to set up TPM-based attestation, see Initialize the HGS cluster using TPM mode in a new
dedicated forest (default).
For the next steps to set up Admin-based attestation, see Initialize the HGS cluster using AD mode in a new
dedicated forest (default).
Install HGS in an existing bastion forest
11/27/2017 • 4 min to read • Edit Online
« PREPA RE FOR IN IT IA L IZ E H G S
HG S »
If your datacenter has a secure bastion forest where you want to join HGS nodes, follow these steps. You can also
use these steps to configure 2 or more independent HGS clusters that are joined to the same domain.
NOTE
Group managed service accounts are available beginning with the Windows Server 2012 Active Directory schema. For more
information, see group managed service account requirements.
Cluster objects
If the account you are using to set up HGS does not have permission to create new computer objects in the
domain, you will need to pre-stage the cluster objects. These steps are explained in Prestage Cluster Computer
Objects in Active Directory Domain Services.
To set up your first HGS node, you will need to create one Cluster Name Object (CNO) and one Virtual Computer
Object (VCO). The CNO represents the name of the cluster, and is primarily used internally by Failover Clustering.
The VCO represents the HGS service that resides on top of the cluster and will be the name registered with the
DNS server.
IMPORTANT
The user who will run Initialize-HgsServer requires Full Control over the CNO and VCO objects in Active Directory.
To quickly prestage your CNO and VCO, have an Active Directory admin run the following PowerShell commands:
# Create the CNO
$cno = New-ADComputer -Name 'HgsCluster' -Description 'HGS CNO' -Enabled $false -Passthru
# Allow time for your new CNO and VCO to replicate to your other Domain Controllers before continuing
Next steps
For the next steps to set up TPM-based attestation, see Initialize the HGS cluster using TPM mode in an existing
bastion forest.
For the next steps to set up Admin-based attestation, see Initialize the HGS cluster using AD mode in an existing
bastion forest.
Initialize the Host Guardian Service (HGS)
10/17/2017 • 1 min to read • Edit Online
« IN S T A L L C O N F IG U R E H T T P S
HG S »
When you initialize HGS, you specify the mode that HGS will use to measure the health of guarded hosts. There
are two mutually exclusive options. For background information about which mode to choose, see Guarded Fabric
and Shielded VM Planning Guide for Hosters.
The following topics cover deployment steps for each mode:
TPM-trusted attestation (TPM mode)
Admin-trusted attestation (AD mode)
You should perform these steps on a physical server with Windows Server 2016 installed.
Initialize HGS using TPM-trusted attestation
10/17/2017 • 1 min to read • Edit Online
These steps vary depending on whether you are initializing HGS in a new forest or an existing bastion forest:
1. Initialize the HGS cluster in a new forest (default)
-Or-
Initialize the HGS cluster in an existing bastion forest
2. Install trusted TPM root certificates
3. Configure the fabric DNS
Initialize the HGS cluster using TPM mode in a new
dedicated forest (default)
10/17/2017 • 3 min to read • Edit Online
« IN S T A L L H G S IN A N E W IN S T A L L T R U S T E D T P M R O O T C E R T IF IC A T E S
FOREST »
1. Determine a suitable distributed network name (DNN) for the HGS cluster. This name will be registered in
the HGS DNS service to make it easy for Hyper-V hosts to contact any node in the HGS cluster. As an
example, if you have 3 HGS nodes with hostnames HGS01, HGS02, and HGS03, you might decide to choose
"HGS" or "HgsCluster" for the DNN. Do not provide a fully qualified domain name to the
Initialize-HgsServer cmdlet (use "hgs" not "hgs.relecloud.com").
2. Locate your HGS guardian certificates. You will need one signing certificate and one encryption certificate to
intitialize the HGS cluster. The easiest way to provide certificates to HGS is to create a password-protected
PFX file for each certificate which contains both the public and private keys. If you are using HSM-backed
keys or other non-exportable certificates, make sure the certificate is installed into the local machine's
certificate store before continuing. For more information about which certificates to use, see Obtain
certificates for HGS.
3. Run Initialize-HgsServer in an elevated PowerShell window on the first HGS node. The syntax of this cmdlet
supports many different inputs, but the 2 most common invocations are below:
If you are using PFX files for your signing and encryption certificates, run the following commands:
If you are using non-exportable certificates that are installed in the local certificate store, run the
following command. If you do not know the thumbprints of your certificates, you can list available
certificates by running Get-ChildItem Cert:\LocalMachine\My .
4. If you provided any certificates to HGS using thumbprints, you will be instructed to grant HGS read access to
the private key of those certificates. On a server with Desktop Experience installed, complete the following
steps:
a. Open the local computer certificate manager (certlm.msc)
b. Find the certificate(s) > right-click > all tasks > manage private keys
c. Click Add
d. In the object picker window, click Object types and enable service accounts
e. Enter the name of the service account mentioned in the warning text from Initialize-HgsServer
f. Ensure the gMSA has "Read" access to the private key.
On server core, you will need to download a PowerShell module to assist in setting the private key
permissions.
a. Run Install-Module GuardedFabricTools on the HGS server if it has Internet connectivity, or run
Save-Module GuardedFabricTools on another computer and copy the module over to the HGS server.
b. Run Import-Module GuardedFabricTools . This will add additional properties to certificate objects found in
PowerShell.
c. Find your certificate thumbprint in PowerShell with Get-ChildItem Cert:\LocalMachine\My
d. Update the ACL, replacing the thumbprint with your own and the gMSA account in the code below
with the account listed in the warning text of Initialize-HgsServer .
If you are using HSM-backed certificates, or certificates stored in a third party key storage provider, these
steps may not apply to you. Consult your key storage provider's documentation to learn how to manage
permissions on your private key. In some cases, there is no authorization, or authorization is provided to the
entire computer when the certificate is installed.
5. That's it! In a production environment, you should continue to add additional HGS nodes to your cluster. In a
test environment, you can skip to validating your HGS configuration.
Initialize the HGS cluster using TPM mode in an
existing bastion forest
11/27/2017 • 1 min to read • Edit Online
« IN S T A L L H G S IN A N E X IS T IN G B A S T IO N IN S T A L L T P M R O O T C E R T S
FOREST »
Active Directory Domain Services will be installed on the machine, but should remain unconfigured.
Locate your HGS guardian certificates. You will need one signing certificate and one encryption certificate to
intitialize the HGS cluster. The easiest way to provide certificates to HGS is to create a password-protected PFX file
for each certificate which contains both the public and private keys. If you are using HSM-backed keys or other
non-exportable certificates, make sure the certificate is installed into the local machine's certificate store before
continuing. For more information about which certificates to use, see Obtain certificates for HGS.
Before you continue, ensure that you have prestaged your cluster objects for the Host Guardian Service and
granted the logged in user Full Control over the VCO and CNO objects in Active Directory. The virtual computer
object name needs to be passed to the -HgsServiceName parameter, and the cluster name to the -ClusterName
parameter.
TIP
Double check your AD Domain Controllers to ensure your cluster objects have replicated to all DCs before continuing.
If you are using PFX-based certificates, run the following commands on the HGS server:
If you are using certificates installed on the local machine (such as HSM-backed certificates and non-exportable
certificates), use the -SigningCertificateThumbprint and -EncryptionCertificateThumbprint parameters instead.
Install trusted TPM root certificates
11/21/2017 • 1 min to read • Edit Online
« IN IT IA L IZ E C O N F IG U R E F A B R IC D N S
HG S »
If you chose TPM mode, or expect to migrate to TPM mode in the future, you need to install root certificates to
issue the endorsement key in each host's TPM module. These root certificates are different from those installed by
default in Windows and represent the specific root and intermediate certificates used by TPM vendors. The
following steps help you install certificates for TPMs produced by Microsoft's partners. Some TPMs do not use
endorsement key certificates or use certificates not included in this package. Consult your TPM vendor or server
OEM for further assistance in these cases.
1. Download the latest package from http://tpmsec.microsoft.com/OnPremisesDHA/TrustedTPM.cab.
2. Validate that the package is digitally signed by Microsoft. Do not expand the CAB file if it does not have a
valid signature.
mkdir .\TrustedTPM
expand.exe -F:* <Path-To-TrustedTpm.cab> .\TrustedTPM
4. By default, the configuration script will install certificates for every TPM vendor. If you only want to import
certificates for your specific TPM vendor, delete the folders for TPM vendors not trusted by your
organization.
5. Install the trusted certificate package by running the setup script in the expanded folder.
cd .\TrustedTPM
.\setup.cmd
The TrustedTpm.cab file is updated regularly with new vendor certificates as they become available. To add new
certificates or ones intentionally skipped during an earlier installation, simply repeat the above steps on every
node in your HGS cluster. Existing certificates will remain trusted but new certificates found in the expanded cab
file will be added to the trusted TPM stores.
8/7/2017 • 1 min to read • Edit Online
« IN S T A L L T P M R O O T C O N F IG U R E H T T P S
C E RTS »
A fabric administrator needs to configure the fabric DNS takes to allow guarded hosts to resolve the HGS cluster.
The HGS cluster must already be set up by the HGS administrator.
There are many ways to configure name resolution for the fabric domain. One simple way is to set up a conditional
forwarder zone in DNS for the fabric. To set up this zone, run the following commands in an elevated Windows
PowerShell console on a fabric DNS server. Substitute the names and addresses in the Windows PowerShell syntax
below as needed for your environment. Add master servers for the additional HGS nodes.
See also
Configuration steps for Hyper-V hosts that will become guarded hosts
Deployment tasks for guarded fabrics and shielded VMs
Initialize HGS using Admin-trusted attestation
10/17/2017 • 1 min to read • Edit Online
These steps vary depending on whether you are initializing HGS in a new forest or an existing bastion forest:
1. Initialize the HGS cluster in a new forest (default)
-Or-
Initialize the HGS cluster in an existing bastion forest
2. Configure DNS forwarding in the fabric domain
3. Configure DNS forwarding and a one-way trust in the HGS domain
Initialize the HGS cluster using AD mode in a new
dedicated forest (default)
10/17/2017 • 3 min to read • Edit Online
« IN S T A L L H G S IN A N E W C O N F IG U R E F A B R IC D N S
FOREST »
1. Determine a suitable distributed network name (DNN) for the HGS cluster. This name will be registered in the
HGS DNS service to make it easy for Hyper-V hosts to contact any node in the HGS cluster. As an example, if
you have 3 HGS nodes with hostnames HGS01, HGS02, and HGS03, you might decide to choose "HGS" or
"HgsCluster" for the DNN. Do not provide a fully qualified domain name to the Initialize-HgsServer cmdlet
(use "hgs" not "hgs.relecloud.com").
2. Locate your HGS guardian certificates. You will need one signing certificate and one encryption certificate to
intitialize the HGS cluster. The easiest way to provide certificates to HGS is to create a password-protected
PFX file for each certificate which contains both the public and private keys. If you are using HSM-backed
keys or other non-exportable certificates, make sure the certificate is installed into the local machine's
certificate store before continuing. For more information about which certificates to use, see Obtain
certificates for HGS.
3. Run Initialize-HgsServer in an elevated PowerShell window on the first HGS node. The syntax of this cmdlet
supports many different inputs, but the 2 most common invocations are below:
If you are using PFX files for your signing and encryption certificates, run the following commands:
If you are using non-exportable certificates that are installed in the local certificate store, run the
following command. If you do not know the thumbprints of your certificates, you can list available
certificates by running Get-ChildItem Cert:\LocalMachine\My .
4. If you provided any certificates to HGS using thumbprints, you will be instructed to grant HGS read access to
the private key of those certificates. On a server with Desktop Experience installed, complete the following
steps:
a. Open the local computer certificate manager (certlm.msc)
b. Find the certificate(s) > right-click > all tasks > manage private keys
c. Click Add
d. In the object picker window, click Object types and enable service accounts
e. Enter the name of the service account mentioned in the warning text from Initialize-HgsServer
f. Ensure the gMSA has "Read" access to the private key.
On server core, you will need to download a PowerShell module to assist in setting the private key
permissions.
a. Run Install-Module GuardedFabricTools on the HGS server if it has Internet connectivity, or run
Save-Module GuardedFabricTools on another computer and copy the module over to the HGS server.
b. Run Import-Module GuardedFabricTools . This will add additional properties to certificate objects found in
PowerShell.
c. Find your certificate thumbprint in PowerShell with Get-ChildItem Cert:\LocalMachine\My
d. Update the ACL, replacing the thumbprint with your own and the gMSA account in the code below
with the account listed in the warning text of Initialize-HgsServer .
If you are using HSM-backed certificates, or certificates stored in a third party key storage provider, these
steps may not apply to you. Consult your key storage provider's documentation to learn how to manage
permissions on your private key. In some cases, there is no authorization, or authorization is provided to the
entire computer when the certificate is installed.
5. That's it! In a production environment, you should continue to add additional HGS nodes to your cluster. In a
test environment, you can skip to validating your HGS configuration.
Initialize the HGS cluster using AD mode in an
existing bastion forest
11/27/2017 • 1 min to read • Edit Online
« IN S T A L L H G S IN A N E W C O N F IG U R E F A B R IC D N S
FOREST »
Active Directory Domain Services will be installed on the machine, but should remain unconfigured.
Locate your HGS guardian certificates. You will need one signing certificate and one encryption certificate to
intitialize the HGS cluster. The easiest way to provide certificates to HGS is to create a password-protected PFX file
for each certificate which contains both the public and private keys. If you are using HSM-backed keys or other
non-exportable certificates, make sure the certificate is installed into the local machine's certificate store before
continuing. For more information about which certificates to use, see Obtain certificates for HGS.
Before you continue, ensure that you have prestaged your cluster objects for the Host Guardian Service and
granted the logged in user Full Control over the VCO and CNO objects in Active Directory. The virtual computer
object name needs to be passed to the -HgsServiceName parameter, and the cluster name to the -ClusterName
parameter.
TIP
Double check your AD Domain Controllers to ensure your cluster objects have replicated to all DCs before continuing.
If you are using PFX-based certificates, run the following commands on the HGS server:
If you are using certificates installed on the local machine (such as HSM-backed certificates and non-exportable
certificates), use the -SigningCertificateThumbprint and -EncryptionCertificateThumbprint parameters instead.
8/7/2017 • 1 min to read • Edit Online
« IN IT IA L IZ E C O N F IG U R E H G S D N S A N D A O N E -W A Y T R U S T
HG S »
A fabric administrator needs to configure the fabric DNS takes to allow guarded hosts to resolve the HGS cluster.
The HGS cluster must already be set up by the HGS administrator.
There are many ways to configure name resolution for the fabric domain. One simple way is to set up a conditional
forwarder zone in DNS for the fabric. To set up this zone, run the following commands in an elevated Windows
PowerShell console on a fabric DNS server. Substitute the names and addresses in the Windows PowerShell syntax
below as needed for your environment. Add master servers for the additional HGS nodes.
See also
Configuration steps for Hyper-V hosts that will become guarded hosts
Deployment tasks for guarded fabrics and shielded VMs
Configure DNS forwarding in the HGS domain and a
one-way trust with the fabric domain
10/17/2017 • 1 min to read • Edit Online
« C O N F IG U R E F A B R C C O N F IG U R E H T T P S
DNS »
Use the following steps to set up DNS forwarding and establish a one-way trust with the fabric domain. These
steps allow the HGS to locate the fabric domain controllers and validate group membership of the Hyper-V hosts.
1. Run the following command in an elevated PowerShell session to configure DNS forwarding. Replace
fabrikam.com with the name of the fabric domain and type the IP addresses of DNS servers in the fabric
domain. For higher availability, point to more than one DNS server.
2. To create a one-way forest trust, run the following command in an elevated Command Prompt:
Replace relecloud.com with the name of the HGS domain and fabrikam.com with the name of the fabric
domain. Provide the password for an admin of the fabric domain.
« IN IT IA L IZ E A DD HG S N ODES
HG S »
By default, when you initialize the HGS server it will configure the IIS web sites for HTTP-only communications. All
sensitive material being transmitted to and from HGS (including the encryption keys for the VM) are always
encrypted using message-level encryption, however if you desire a higher level of security you can also enable
HTTPS by configuring HGS with an SSL certificate.
First, obtain an SSL certificate for HGS from your certificate authority. Each Hyper-V host will need to trust the SSL
certificate, so it is recommended that you issue the SSL certificate from your company's public key infrastructure
or a third party CA. Any SSL certificate supported by IIS is supported by HGS, however the subject name on the
certificate must match the fully qualified HGS service name (cluster distributed network name). For instance,
if the HGS domain is "secure.contoso.com" and your HGS service name is "hgs", your SSL certificate should be
issued for "hgs.secure.contoso.com". You can add additional DNS names to the certificate's subject alternative
name field if necessary.
Once you have the SSL certificate, you can either provide the certificate to the Initialize-HgsServer cmdlet if you
haven't already run it, or use Set-HgsServer if you've already initialized HGS.
If you haven't already initialized HGS
Append the following SSL-related parameters to the Initialize-HgsServer command from the Initialize the HGS
cluster or Initialize HGS in the bastion forest sections.
If your certificate is installed in the local certificate store and cannot be exported to a PFX file with the private key
intact, you can provide the SSL certificate by its thumbprint instead:
Or, if you have already installed the certificate into the local certificate store and want to reference it by
thumbprint:
Set-HgsServer -Http -Https -HttpsCertificateThumbprint 'A1B2C3D4E5F6...'
IMPORTANT
Configuring HGS with an SSL certificate does not disable the HTTP endpoint. If you wish to only allow use of the HTTPS
endpoint, configure Windows Firewall to block inbound connections to port 80. Do not modify the IIS bindings for HGS
websites to remove the HTTP endpoint; it is unsupported to do so.
Configure additional HGS nodes
10/17/2017 • 6 min to read • Edit Online
« C O N F IG U R E V E R IF Y T H E C O N F IG U R A T IO N
HT TP S »
In production environments, HGS should be set up in a high availability cluster to ensure that shielded VMs can be
powered on even if an HGS node goes down. For test environments, secondary HGS nodes are not required.
Use one of these methods to add HGS nodes, as best suited for your environment.
Prerequisites
Make sure that each additional node:
Has the same hardware and software configuration as the primary node
Is connected to the same network as the other HGS servers
Can resolve the other HGS servers by their DNS names
It will take up to 10 minutes for the encryption and signing certificates from the first HGS server to replicate to this
node.
It will take up to 10 minutes for the encryption and signing certificates from the first HGS server to replicate to this
node.
Install the private keys for the certificates
If you did not provide a PFX file for either the encryption or signing certificates on the first HGS server, only the
public key will be replicated to this server. You will need to install the private key by importing a PFX file
containing the private key into the local certificate store or, in the case of HSM-backed keys, configuring the Key
Storage Provider and associating it with your certificates per your HSM manufacturer's instructions.
It will take up to 10 minutes for the encryption and signing certificates from the first HGS server to replicate to this
node.
It will take up to 10 minutes for the encryption and signing certificates from the first HGS server to replicate to this
node.
Install the private keys for the certificates
If you did not provide a PFX file for either the encryption or signing certificates on the first HGS server, only the
public key will be replicated to this server. You will need to install the private key by importing a PFX file
containing the private key into the local certificate store or, in the case of HSM-backed keys, configuring the Key
Storage Provider and associating it with your certificates per your HSM manufacturer's instructions.
If you already installed the certificate into the local certificate store and want to reference it by thumbprint, run the
following command instead:
HGS will always expose both the HTTP and HTTPS ports for communication. It is unsupported to remove the HTTP
binding in IIS, however you can use the Windows Firewall or other network firewall technologies to block
communications over port 80.
Next steps
Validate the HGS configuration
Verify the HGS configuration
10/17/2017 • 1 min to read • Edit Online
« A DD HG S D E P L OY G U A RD E D HOSTS
N ODES »
Next, we need to validate that things are working as expected. To do so, run the following command in an elevated
Windows PowerShell console:
Get-HgsTrace -RunDiagnostics
Because the HGS configuration does not yet contain information about the hosts that will be in the guarded fabric,
the diagnostics will indicate that no hosts will be able to attest successfully yet. Ignore this result, and review the
other information provided by the diagnostics.
NOTE
When running the Guarded Fabric diagnostics tool (Get-HgsTrace -RunDiagnostics), incorrect status may be returned
claiming that the HTTPS configuration is broken when it is, in fact, not broken or not being used. This error can be returned
regardless of HGS’ attestation mode. The possible root-causes are as follows:
HTTPS is indeed improperly configured/broken
You’re using admin-trusted attestation and the trust relationship is broken
- This is irrespective of whether HTTPS is configured properly, improperly, or not in use at all.
Note that the diagnostics will only return this incorrect status when targeting a Hyper-V host. If the diagnostics are
targeting the Host Guardian Service, the status returned will be correct.
« DEPLOY D E P L O Y S H IE L D E D V IR T U A L M A C H IN E S
HG S »
The topics in this section describe the steps that a fabric administrator takes to configure Hyper-V hosts to work
with the Host Guardian Service (HGS). Before you can start these steps, at least one node in the HGS cluster must
be set up.
For Admin-trusted attestation:
1. Configure the fabric DNS: Tells how to set up a DNS forwarder from the fabric domain to the HGS domain.
2. Create a security group: Tells how to set up an Active Directory security group in the fabric domain, add
guarded hosts as members of that group, and provide that group identifier to the HGS administrator.
3. Confirm guarded hosts can attest
For TPM-trusted attestation:
1. Configure the fabric DNS: Tells how to set up a DNS forwarder from the fabric domain to the HGS domain.
2. Capture information required by HGS: Tells how to capture TPM identifiers (also called platform identifiers),
create a Code Integrity policy, and create a TPM baseline. Then you will provide this information to the HGS
administrator to configure attestation.
3. Confirm guarded hosts can attest
See also
Deployment tasks for guarded fabrics and shielded VMs
Prerequisites for guarded hosts
10/17/2017 • 1 min to read • Edit Online
Review the host prerequisites for the mode of attestation you've chosen, then click on the next step to add guarded
hosts.
TPM-trusted attestation
Guarded hosts using TPM mode must meet the following prerequisites:
Hardware: One host is required for initial deployment. To test Hyper-V live migration for shielded VMs, you
must have at least two hosts.
Hosts must have:
IOMMU and Second Level Address Translation (SLAT)
TPM 2.0
UEFI 2.3.1 or later
Configured to boot using UEFI (not BIOS or "legacy" mode)
Secure boot enabled
Operating system: Windows Server 2016 Datacenter edition
IMPORTANT
Make sure you install the latest cumulative update.
Role and features: Hyper-V role and the Host Guardian Hyper-V Support feature. The Host Guardian
Hyper-V Support feature is only available on Datacenter editions of Windows Server 2016.
WARNING
The Host Guardian Hyper-V Support feature enables Virtualization-based protection of code integrity that may be
incompatible with some devices. We strongly recommend testing this configuration in your lab before enabling this feature.
Failure to do so may result in unexpected failures up to and including data loss or a blue screen error (also called a stop
error). For more information, see Compatible hardware with Windows Server 2016 Virtualization-based protection of Code
Integrity.
Admin-trusted attestation
« DEPLOY GUA RDED P L A C E H O S T S IN A S E C U R IT Y G R O U P
HOSTS »
IMPORTANT
Install the latest cumulative update.
Role and features: Hyper-V role and the Host Guardian Hyper-V Support feature, which is only available in
Windows Server 2016 Datacenter edition.
WARNING
The Host Guardian Hyper-V Support feature enables Virtualization-based protection of code integrity that may be
incompatible with some devices. We strongly recommend testing this configuration in your lab before enabling this feature.
Failure to do so may result in unexpected failures up to and including data loss or a blue screen error (also called a stop
error). For more information, see Compatible hardware with Windows Server 2016 Virtualization-based protection of Code
Integrity.
Authorize guarded hosts using TPM-based
attestation
10/17/2017 • 7 min to read • Edit Online
« R E V IE W C O N F IR M A T T E S T A T IO N
P R E R E Q U IS IT E S »
TPM mode uses a TPM identifier (also called a platform identifier or endorsement key [EKpub]) to begin
determining whether a particular host is authorized as "guarded." This mode of attestation uses secure boot and
code integrity measurements to ensure that a given Hyper-V host is in a healthy state and is running only trusted
code. In order for attestation to understand what is and is not healthy, you must capture the following information:
1. TPM identifier (EKpub)
This information is unique to each Hyper-V host
2. TPM baseline (boot measurements)
This is applicable to all Hyper-V hosts that run on the same class of hardware
3. Code Integrity policies (a white list of allowed binaries)
This is applicable to all Hyper-V hosts that share common hardware and software
We recommend that you capture the baseline and CI policies from a "reference host" that is representative of each
unique class of Hyper-V hardware configuration within your datacenter.
For a video that illustrates the deployment process for TPM mode, see Guarded fabric deployment using TPM
mode.
Capture the TPM identifier (platform identifier or EKpub) for each host
1. In the fabric domain, make sure the TPM on each host is ready for use - that is, the TPM is initialized and
ownership obtained. You can check the status of the TPM by opening the TPM Management Console
(tpm.msc) or by running Get-Tpm in an elevated Windows PowerShell window. If your TPM is not in the
Ready state, you will need to initialize it and set its ownership. This can be done in the TPM Management
Console or by running Initialize-Tpm.
2. On each guarded host, run the following command in an elevated Windows PowerShell console to obtain
its EKpub. For <HostName> , substitute the unique host name with something suitable to identify this host -
this can be its hostname or the name used by a fabric inventory service (if available). For convenience, name
the output file using the host's name.
3. Repeat the preceding steps for each host that will become a guarded host, being sure to give each XML file a
unique name.
4. Provide the resulting XML files to the HGS administrator.
5. In the HGS domain, open an elevated Windows PowerShell console on an HGS server and run the following
command. Repeat the command for each of the XML files.
NOTE
If you encounter an error when adding a TPM identifier regarding an untrusted Endorsement Key Certificate (EKCert),
ensure that the trusted TPM root certificates have been added to the HGS node. Additionally, some TPM vendors do
not use EKCerts. You can check if an EKCert is missing by opening the XML file in an editor such as Notepad and
checking for an error message indicating no EKCert was found. If this is the case, and you trust that the TPM in your
machine is authentic, you can use the -Force flag to override this safety check and add the host identifier to HGS.
Note The above command creates a CI policy in audit mode only. It will not block unauthorized
binaries from running on the host. You should only use enforced policies in production.
2. Keep your Code Integrity policy file (XML file) where you can easily find it. You will need to edit this file later
to enforce the CI policy or merge in changes from future updates made to the system.
3. Apply the CI policy to your reference host:
a. Copy the binary CI policy file (HW1CodeIntegrity.p7b) to the following location on your reference
host (the filename must exactly match):
C:\Windows\System32\CodeIntegrity\SIPolicy.p7b
b. Restart the host to apply the policy.
4. Test the code integrity policy by running a typical workload. This may include running VMs, any fabric
management agents, backup agents, or troubleshooting tools on the machine. Check if there are any code
integrity violations and update your CI policy if necessary.
5. Change your CI policy to enforced mode by running the following commands against your updated CI
policy XML file.
6. Apply the CI policy to all of your hosts (with identical hardware and software configuration) using the
following commands:
Restart-Computer
Note Be careful when applying CI policies to hosts and when updating any software on these
machines. Any kernel mode drivers that are non-compliant with the CI Policy may prevent the machine
from starting up. For best practices regarding CI policies, see Planning and getting started on the Device
Guard deployment process and Deploy Device Guard: deploy code integrity policies.
7. Provide the binary file (in this example, HW1CodeIntegrity_enforced.p7b) to the HGS administrator.
8. In the HGS domain, copy the code integrity policy to an HGS server and run the following command.
For <PolicyName> , specify a name for the CI policy that describes the type of host it applies to. A best
practice is to name it after the make/model of your machine and any special software configuration running
on it.
For <Path> , specify the path and filename of the code integrity policy.
Warning The Host Guardian Hyper-V Support feature enables Virtualization-based protection of code
integrity that may be incompatible with some devices. We strongly recommend testing this
configuration in your lab before enabling this feature. Failure to do so may result in unexpected failures
up to and including data loss or a blue screen error (also called a stop error).
Install-WindowsFeature Hyper-V, HostGuardian -IncludeManagementTools -Restart
2. To capture the baseline policy, run the following command in an elevated Windows PowerShell console.
Note You will need to use the -SkipValidation flag if the reference host does not have Secure Boot
enabled, an IOMMU present, Virtualization Based Security enabled and running, or a code integrity
policy applied. These validations are designed to make you aware of the minimum requirements of
running a shielded VM on the host. Using the -SkipValidation flag does not change the output of the
cmdlet; it merely silences the errors.
« R E V IE W C O N F IR M A T T E S T A T IO N
P R E R E Q U IS IT E S »
This topic describes the intermediate steps to prepare Hyper-V hosts to become guarded hosts using Admin-
trusted attestation (AD mode). Before taking these steps, complete the steps in Configuring the fabric DNS for
hosts that will become guarded hosts.
For a video that illustrates the deployment process, see Guarded fabric deployment using AD mode.
Next step
Confirm hosts can attest successfully
See also
Deploying the Host Guardian Service for guarded hosts and shielded VMs
Confirm guarded hosts can attest
10/17/2017 • 1 min to read • Edit Online
« R E V IE W D E P L O Y S H IE L D E D V M S
P R E R E Q U IS IT E S »
A fabric administrator needs to confirm that Hyper-V hosts can run as guarded hosts. Complete the following
steps on at least one guarded host:
1. If you have not already installed the Hyper-V role and Host Guardian Hyper-V Support feature, install them
with the following command:
Through VMM: If you are using System Center 2016 - Virtual Machine Manager (VMM), you can
configure Attestation and Key Protection URLs in VMM. For details, see Configure global HGS
settings in Provision guarded hosts in VMM.
Notes
If the HGS administrator enabled HTTPS on the HGS server, begin the URLs with https:// .
If the HGS administrator enabled HTTPS on the HGS server and used a self-signed certificate, you
will need to import the certificate into the Trusted Root Certificate Authorities store on every host.
To do this, run the following command on each host:
Import-Certificate -FilePath "C:\temp\HttpsCertificate.cer" -CertStoreLocation
Cert:\LocalMachine\Root
3. To initiate an attestation attempt on the host and view the attestation status, run the following command:
Get-HgsClientConfiguration
The output of the command indicates whether the host passed attestation and is now guarded. If
IsHostGuarded does not return True, you can run the HGS diagnostics tool, Get-HgsTrace, to investigate.
To run diagnostics, enter the following command in an elevated Windows PowerShell prompt on the host:
Get-HgsTrace -RunDiagnostics -Detailed
See also
Deploy the Host Guardian Service (HGS)
Deploy shielded VMs
Deploy shielded VMs
11/14/2017 • 1 min to read • Edit Online
« C O N F IR M C R E A T E A S H IE L D E D V M T E M P L A T E
A T T E S T A T IO N »
The following topics describe how a tenant can work with shielded VMs.
1. (Optional) Create a Windows template disk or create a Linux template disk. The template disk can be
created by either the tenant or the hosting service provider.
2. (Optional) Convert an existing Windows VM to a shielded VM.
3. Create shielding data to define a shielded VM.
For a description and diagram of a shielding data file, see What is shielding data and why is it necessary?
For information about creating an answer file to include in a shielded data file, see Shielded VMs -
Generate an answer file by using the New-ShieldingDataAnswerFile function.
4. Create a shielded VM:
Using Windows Azure Pack: Deploy a shielded VM by using Windows Azure Pack
Using Virtual Machine Manager: Deploy a shielded VM by using Virtual Machine Manager
See also
Guarded fabric and shielded VMs
Create a Windows shielded VM template disk
11/14/2017 • 8 min to read • Edit Online
« D E P L O Y S H IE L D E D C R E A T E A S H IE L D IN G D A T A F IL E
VM S »
As with regular VMs, you can create a VM template (for example, a VM template in Virtual Machine Manager
(VMM)) to make it easy for tenants and administrators to deploy new VMs on the fabric using a template disk.
Because shielded VMs are security-sensitive assets, there are additional steps to create a VM template that
supports shielding. This topic covers the steps to create a shielded template disk and a VM template in VMM.
To understand how this topic fits in the overall process of deploying shielded VMs, see Hosting service provider
configuration steps for guarded hosts and shielded VMs.
Must be a GUID Partition Table (GPT) disk Needed for generation 2 virtual machines to support UEFI
Disk type must be Basic as opposed to Dynamic. BitLocker does NOT support dynamic disks.
Note: This refers to the logical disk type, not the "dynamically
expanding" VHDX feature supported by Hyper-V.
The disk has at least two partitions. One partition must Needed for BitLocker
include the drive on which Windows is installed. This is the
drive that BitLocker will encrypt. The other partition is the
active partition, which contains the bootloader and remains
unencrypted so that the computer can be started.
The operating system installed on the VHDX is one of the Needed to support generation 2 virtual machines and the
following: Microsoft Secure Boot template
- Windows Server 2016, Windows Server 2012 R2, or
Windows Server 2012
- Windows 10, Windows 8.1, Windows 8
Operating system must be generalized (run sysprep.exe) Template provisioning involves specializing VMs for a specific
tenant's workload
NOTE
If you use VMM, do not copy the template disk into the VMM library at this stage.
Prepare and protect the VHDX with the template disk wizard
To use a template disk with shielded VMs, the disk must be prepared and encrypted with BitLocker by using the
Shielded Template Disk Creation Wizard. This wizard will generate a hash for the disk and add it to a volume
signature catalog (VSC). The VSC is signed using a certificate you specify and is used during the provisioning
process to ensure the disk being deployed for a tenant has not been altered or replaced with a disk the tenant
does not trust. Finally, BitLocker is installed on the disk's operating system (if it is not already there) to prepare the
disk for encryption during VM provisioning.
NOTE
The template disk wizard will modify the template disk you specify in-place. You may want to make a copy of the
unprotected VHDX before running the wizard to make updates to the disk at a later time. You will not be able to modify a
disk that has been protected with the template disk wizard.
Perform the following steps on a computer running Windows Server 2016 (does not need to be a guarded host or
a VMM server):
1. Copy the generalized VHDX created in Prepare an operating system VHDX to the server, if it is not already
there.
2. To administer the server locally, install the Shielded VM Tools feature from Remote Server
Administration Tools on the server.
You can also administer the server from a client computer on which you have installed the Windows 10
Remote Server Administration Tools.
3. Obtain or create a certificate to sign the VSC for the VHDX that will become the template disk for new
shielded VMs. Details about this certificate will be shown to tenants when they create their shielding data
files and are authorizing disks they trust. Therefore, it is important to obtain this certificate from a certificate
authority mutually trusted by you and your tenants. In enterprise scenarios where you are both the hoster
and tenant, you might consider issuing this certificate from your PKI.
If you are setting up a test environment and just want to use a self-signed certificate to prepare your
template disk, run a command similar to the following:
4. Start the Template Disk Wizard from the Administrative Tools folder on the Start menu or by typing
TemplateDiskWizard.exe into a command prompt.
5. On the Certificate page, click Browse to display a list of certificates. Select the certificate with which to
prepare the disk template. Click OK and then click Next.
6. On the Virtual Disk page, click Browse to select the VHDX that you have prepared, then click Next.
7. On the Signature Catalog page, provide a friendly disk name and version. These fields are present to help
you identify the disk once it has been prepared.
For example, for disk name you could type WS2016 and for Version, 1.0.0.0
8. Review your selections on the Review Settings page of the wizard. When you click Generate, the wizard
will enable BitLocker on the template disk, compute the hash of the disk, and create the Volume Signature
Catalog, which is stored in the VHDX metadata.
Wait until the prep process has finished before attempting to mount or move the template disk. This
process may take a while to complete, depending on the size of your disk.
9. On the Summary page, information about the disk template, the certificate used to sign the VSC, and the
certificate issuer is shown. Click Close to exit the wizard.
If you use VMM, follow the steps in the remaining sections in this topic to incorporate a template disk into a
shielded VM template in VMM.
After the template is created, tenants can use it to create new virtual machines. You will need to verify that the VM
template is one of the resources available to the Tenant Administrator user role (in VMM, user roles are in the
Settings workspace).
See also
Hosting service provider configuration steps for guarded hosts and shielded VMs
Guarded fabric and shielded VMs
Create a Linux shielded VM template disk
11/14/2017 • 8 min to read • Edit Online
This topic explains how to prepare a template disk for Linux shielded VMs that can be used to instantiate one or
more tenant VMs.
Prerequisites
To prepare and test a Linux shielded VM, you will need the following resources available:
A server with virtualization capababilities running Windows Server, version 1709 or later
A second computer (Windows 10 or Windows Server 2016) capable of running Hyper-V Manager to connect to
the running VM's console
An ISO image for one of the supported Linux shielded VM OSes:
Ubuntu 16.04 LTS with the 4.4 kernel
Red Hat Enterprise Linux 7.3
SUSE Linux Enterprise Server 12 Service Pack 2
Internet access to download the lsvmtools package and OS updates
IMPORTANT
Newer versions of the OSes listed above may include a known TPM driver bug which will prevent them from successfully
provisioning as shielded VMs. It is not recommended that you update your templates or shielded VMs to a newer release
until a fix is available. The list of supported OSes above will be updated when the updates are made public.
Prepare a Linux VM
Shielded VMs are created from secure template disks. Template disks contain the operating system for the VM and
metadata including a digital signature of the /boot and /root partitions to ensure core OS components are not
modified before deployment.
To create a template disk, you must first create a regular (unshielded) VM that you will prepare as the base image
for future shielded VMs. The software you install and configuration changes you make to this VM will apply to all
shielded VMs created from this template disk. These steps will walk you through the bare minimum requirements
to get a Linux VM ready for templatization.
NOTE
Linux disk encryption is configured at the time the disk is partitioned. This means that you must create a new VM that is pre-
encrypted using dm-crypt to create a Linux shielded VM template disk.
1. On the virtualization server, ensure that Hyper-V and the Shielded VM Tools features are installed by
running the following commands in an elevated PowerShell console:
Install-WindowsFeature RSAT-Hyper-V-Tools
4. Open Hyper-V Manager on your management computer and connect to your virtualization server. You can
do this by clicking "Connect to Server..." in the Actions pane or by right clicking on Hyper-V Manager and
choosing "Connect to Server..." Provide the DNS name for your Hyper-V server and, if necessary, the
credentials needed to connect to it.
5. Using Hyper-V Manager, configure an external switch on your virtualization server so the Linux VM can
access the Internet to obtain updates.
6. Next, create a new virtual machine to install the Linux OS onto. In the Actions pane, click New > Virtual
Machine to bring up the wizard. Provide a friendly name for your VM, such as "Pre-templatized Linux" and
click Next.
7. On the second page of the Wizard, select Generation 2 to ensure the VM is provisioned with a UEFI-based
firmware profile.
8. Complete the rest of the wizard according to your preferences. Do not use a differencing disk for this VM;
shielded VM template disks cannot use differencing disks. Lastly, connect the ISO image you downloaded
earlier to the virtual DVD drive for this VM so that you can install the OS.
9. In Hyper-V Manager, select your newly-created VM and click Connect... in the Actions pane to attach to a
virtual console of the VM. In the window that appears, click Start to turn on the virtual machine.
10. Proceed through the setup process for your selected Linux distribution. While each Linux distribution uses a
different setup wizard, the following requirements must be met for VMs that will become Linux shielded VM
template disks:
The disk must be partitioned using the GUID Paritioning Table (GPT) layout
The root partition must be encrypted with dm-crypt. The passphrase should be set to passphrase (all
lowercase). This passphrase will be randomized and the partition re-encrypted when a shielded VM is
provisioned.
The boot partition must use the ext2 file system
11. Once your Linux OS has fully booted and you have signed in, it is recommended that you install the linux-
virtual kernel and associated Hyper-V integration services packages. Additionally, you will want to install an
SSH server or other remote management tool to access the VM once it is shielded.
On Ubuntu, run the following command to install these components:
12. Configure your Linux OS as desired. Any software you install, user accounts you add, and systemwide
configuration changes you make will apply to all future VMs created from this template disk. You should
avoid saving any secrets or unnecessary packages to the disk.
13. If you are planning to use System Center Virtual Machine Manager to deploy your VMs, install the VMM
guest agent to enable VMM to specialize your OS during VM provisioning. Specialization allows each VM to
be set up securely with different users and SSH keys, networking configurations, and custom setup steps.
Learn how to obtain and install the VMM guest agent in the VMM documentation.
14. Next, add the Microsoft Linux Software Repository to your package manager.
15. Using your package manager, install the lsvmtools package which contains the Linux shielded VM
bootloader shim, provisioning components, and disk preparation tool.
# Ubuntu 16.04
sudo apt-get install lsvmtools
# SLES 12 SP2
sudo zypper install lsvmtools
# RHEL 7.3
sudo yum install lsvmtools
16. When you're done customizing the Linux OS, locate the lsvmprep installation program on your system and
run it.
# The path below may change based on the version of lsvmprep installed
# Run "find /opt -name lsvmprep" to locate the lsvmprep executable
sudo /opt/lsvmtools-1.0.0-x86-64/lsvmprep
Details about this certificate will be shown to tenants when they create their shielding data files and are authorizing
disks they trust. Therefore, it is important to obtain this certificate from a certificate authority mutually trusted by
you and your tenants. In enterprise scenarios where you are both the hoster and tenant, you might consider issuing
this certificate from your enterprise certificate authority. Protect this certificate carefully, as anyone in posession of
this certificate can create new template disks that are trusted the same as your authentic disk.
In a test lab environment, you can create a self-signed certificate with the following PowerShell command:
IMPORTANT
The Remote Server Administrator Tools available on Windows Server 2016 or Windows 10 cannot be used to prepare a Linux
shielded VM template disk. Only use the Protect-TemplateDisk cmdlet available on Windows Server, version 1709 to prepare
a Linux shielded VM template disk.
# Replace "THUMBPRINT" with the thumbprint of your template disk signing certificate in the line below
$certificate = Get-Item Cert:\LocalMachine\My\THUMBPRINT
Your template disk is now ready to be used to provision Linux shielded VMs. If you are using System Center Virtual
Machine Manager to deploy your VM, you can now copy the VHDX to your VMM library.
You may also want to extract the volume signature catalog from the VHDX. This file is used to provide information
about the signing certificate, disk name, and version to VM owners who want to use your template. They need to
import this file into the Shielding Data File Wizard to authorize you, the template author in posession of the signing
certificate, to create this and future template disks for them.
To extract the volume signature catalog, run the following command in PowerShell:
This topic describes how a hosting service provider can configure Windows Azure Pack so that tenants can use it to
deploy shielded VMs. Windows Azure Pack is a web portal that extends the functionality of System Center Virtual
Machine Manager to allow tenants to deploy and manage their own VMs through a simple web interface. Windows
Azure Pack fully supports shielded VMs and makes it even easier for your tenants to create and manage their
shielding data files.
To understand how this topic fits in the overall process of deploying shielded VMs, see Hosting service provider
configuration steps for guarded hosts and shielded VMs.
4. Once completed, you should be able to see the VM clouds set up in your VMM environment. Ensure you
have at least one VM cloud that supports shielded VMs available to WAP before continuing.
Create a plan in Windows Azure Pack
In order to allow tenants to create VMs in WAP, you must first create a hosting plan to which tenants can subscribe.
Plans define the allowed VM clouds, templates, networks, and billing entities for your tenants.
1. On the lower pane of the portal, click +NEW > PLAN > CREATE PLAN.
2. In the first step of the wizard, choose a name for your Plan. This is the name your tenants will see when
subscribing.
3. In the second step, select VIRTUAL MACHINE CLOUDS as one of the services to offer in the plan.
4. Skip the step about selecting any add-ons for the plan.
5. Click OK (check mark) to create the plan. Although this creates the plan, it is not yet in a configured state.
10. Scroll down to the section titled templates, and then select one or more templates to offer to your tenants.
You can offer both shielded and unshielded templates to tenants, but a shielded template must be offered to
give tenants end-to-end assurances about the integrity of the VM and their secrets.
11. In the networks section, add one or more networks for your tenants.
12. After setting any other settings or quotas for the Plan, click Save at the bottom.
13. At the top left of the screen, click on the arrow to take you back to the Plan page.
14. At the bottom of the screen, change the Plan from being Private to Public so that tenants can subscribe to
the Plan.
At this point, Windows Azure Pack is configured and tenants will be able to subscribe to the plan you just
created and deploy shielded VMs. For additional steps that tenants need to complete, see Shielded VMs for
tenants - Deploying a shielded VM by using Windows Azure Pack.
See also
Hosting service provider configuration steps for guarded hosts and shielded VMs
Guarded fabric and shielded VMs
10/17/2017 • 12 min to read • Edit Online
« C R E A T E A S H IE L D E D V M D E P L O Y A S H IE L D E D U S IN G P O W E R S H E L L
TE M P L A TE »
A shielding data file (also called a provisioning data file or PDK file) is an encrypted file that a tenant or VM owner
creates to protect important VM configuration information, such as the administrator password, RDP and other
identity-related certificates, domain-join credentials, and so on. This topic provides information about how to
create a shielding data file. Before you can create the file, you must either obtain a template disk from your hosting
service provider, or create a template disk as described in Shielded VMs for tenants - Creating a template disk
(optional).
For a list and a diagram of the contents of a shielding data file, see What is shielding data and why is it necessary?.
IMPORTANT
The steps in this section should be completed on a tenant machine running Windows Server 2016. That machine must not
be part of a guarded fabric (that is, should not be configured to use an HGS cluster).
If you are evaluating shielded VMs and are not yet ready to request a certificate from your certificate authority, you
can create a self-signed certificate on the tenant machine by running the following Windows PowerShell
command (where contoso.com is the tenant's domain):
ComputerName @ComputerName@
TimeZone @TimeZone@
ProductKey @ProductKey@
IPAddr4-1 @IP4Addr-1@
IPAddr6-1 @IP6Addr-1@
MACAddr-1 @MACAddr-1@
Prefix-1-1 @Prefix-1-1@
NextHop-1-1 @NextHop-1-1@
Prefix-1-2 @Prefix-1-2@
NextHop-1-2 @NextHop-1-2@
When using substitution strings, it is important to ensure that the strings will be populated during the VM
provisioning process. If a string such as @ProductKey@ is not supplied at deployment time, leaving the
<ProductKey> node in the unattend file blank, the specialization process will fail and you will be unable to connect
to your VM.
Also, note that the networking-related substitution strings towards the end of the table are only used if you are
leveraging VMM Static IP Address Pools. Your hosting service provider should be able to tell you if these
substitution strings are required. For more information about static IP addresses in VMM templates, see the
following in the VMM documentation:
Guidelines for IP address pools
Set up static IP address pools in the VMM fabric
Finally, it is important to note that the shielded VM deployment process will only encrypt the OS drive. If you
deploy a shielded VM with one or more data drives, it is strongly recommended that you add an unattend
command or Group Policy setting in the tenant domain to automatically encrypt the data drives.
$vsc.WriteToFile(".\templateDisk.vsc")
The tenant has access to the template disk file. This may be the case if the tenant creates a template disk to
uploaded to a hosting service provider or if the tenant can download the hoster's template disk. In this case,
without VMM in the picture, the tenant would run the following cmdlet (installed with the Shielded VM
Tools feature, part of Remote Server Administration Tools):
Invoke-WebRequest 'http://hgs.relecloud.com/KeyProtection/service/metadata/2014-07/metadata.xml' -
OutFile .\RelecloudGuardian.xml
Obtain the guardian metadata from VMM using the VMM PowerShell cmdlets:
$relecloudmetadata = Get-SCGuardianConfiguration
Obtain the guardian metadata files for each guarded fabric you wish to authorize your shielded VMs to run on
before continuing.
Create a shielding data file and add guardians
Run the Shielding Data File wizard to create a shielding data (PDK) file. Here, you'll add the RDP certificate,
unattend file, volume signature catalogs, owner guardian and the downloaded guardian metadata obtained in the
preceding step.
1. Install Remote Server Administration Tools > Feature Administration Tools > Shielded VM Tools on
your machine using Server Manager or the following Windows PowerShell command:
Install-WindowsFeature RSAT-Shielded-VM-Tools
2. Open the Shielding Data File Wizard from the Administrator Tools section on your Start menu or by
running the following executable C:\Windows\System32\ShieldingDataFileWizard.exe.
3. On the first page, use the second file selection box to choose a location and file name for your shielding
data file. Normally, you would name a shielding data file after the entity who owns any VMs created with
that shielding data (for example, HR, IT, Finance) and the workload role it is running (for example, file server,
web server, or anything else configured by the unattend file). Leave the radio button set to Shielding data
for Shielded templates.
NOTE
In the Shielding Data File Wizard you will notice the two options below:
Shielding data for Shielded templates
Shielding data for existing VMs and non-Shielded templates
The first option is used when creating new shielded VMs from shielded templates. The second option allows you
to create shielding data that can only be used when converting existing VMs or creating shielded VMs from non-
shielded templates.
Additionally, you must choose whether VMs created using this shielding data file will be truly shielded or
configured in "encryption supported" mode. For more information about these two options, see What are
the types of virtual machines that a guarded fabric can run?.
IMPORTANT
Pay careful attention to the next step as it defines the owner of your shielded VMs and which fabrics your shielded
VMs will be authorized to run on.
Possession of owner guardian is required in order to later change an existing shielded VM from Shielded to
Encryption Supported or vice-versa.
5. On the Volume ID Qualifiers page, click Add to authorize a signed template disk in your shielding data file.
When you select a VSC in the dialog box, it will show you information about that disk's name, version, and
the certificate that was used to sign it. Repeat this process for each template disk you wish to authorize.
6. On the Specialization Values page, click Browse to select your unattend.xml file that will be used to
specialize your VMs.
Use the Add button at the bottom to add any additional files to the PDK that are needed during the
specialization process. For example, if your unattend file is installing an RDP certificate onto the VM (as
described in Generate an answer file by using the New-ShieldingDataAnswerFile function), you should add
the RDPCert.pfx file referenced in the unattend file here. Note that any files you specify here will
automatically be copied to C:\temp\ on the VM that is created. Your unattend file should expect the files to
be in that folder when referencing them by path.
7. Review your selections on the next page, and then click Generate.
8. Close the wizard after it has completed.
See also
Deploy shielded VMs
Guarded fabric and shielded VMs
Create a shielded VM using PowerShell
11/14/2017 • 4 min to read • Edit Online
« C R E A T E A S H IE L D IN G D A T A D E P L O Y A S H IE L D E D U S IN G V M M
F IL E »
In production, you would typically use a fabric manager (e.g. VMM) to deploy shielded VMs. However, the steps
illustrated below allow you to deploy and validate the entire scenario without a fabric manager.
In a nutshell, you will create a template disk, a shielding data file, an unattended installation answer file and other
security artifacts on any machine, then copy these files to a guarded host and provision the shielded VM.
# Import the HGS guardian for each fabric you want to run your shielded VM
$Guardian = Import-HgsGuardian -Path C:\HGSGuardian.xml -Name 'TestFabric'
```powershell
<#
.SYNOPSIS
This script will provisioin a new shielded VM from existing disk template and a PDK file.
.DESCRIPTION
You will need a PDK and associated disk template file prior for shielded VM provisioning
.PARAMETER VMName
The name of the VM to be created
.PARAMETER PdkFile
The path for the pdk file
.PARAMETER TemplateDiskPath
The path for the disk template
.PARAMETER VMPath
The path for VM location.
.PARAMETER Linux
Specifies whether the target OS is Linux
#>
Param
(
[Parameter (Mandatory=$true)][string] $VMName,
[Parameter (Mandatory=$true)][string] $PdkFile,
[Parameter (Mandatory=$true)][string] $TemplateDiskPath,
[Parameter (Mandatory=$true)][string] $VMPath,
[string] $switch = 'External',
[Int64] $VMMemSize = 1GB,
[switch] $Linux = $False
)
#create VM
$vm = New-VM -Name $VMName -Generation 2 -VHDPath $VmVhdPath -MemoryStartupBytes $VMMemSize -Path $VMPath -
SwitchName $switch -erroraction Stop
if ($Linux) {
Set-VMFirmware -VM $vm -SecureBootTemplate OpenSourceShieldedVM
}
While the shielded VM is being provisioned, you can use the following cmdlet to check the progress:
Once it's completed, make sure the shielded VM has the correct network adapter configured so it can be accessed
through RDP.
« D E P L O Y A S H IE L D E D V M U S IN G D E P L O Y A S H IE L D E D U S IN G W IN D O W S A Z U R E P A C K
P OW ERSHEL L »
If you have access to System Center 2016 - Virtual Machine Manager (VMM), you can deploy a shielded VM for
which a shielded VM template has already been created.
To deploy a shielded VM in VMM, use the instructions in Provision a new shielded VM.
See also
Deploy shielded VMs
Guarded fabric and shielded VMs
10/17/2017 • 1 min to read • Edit Online
« D E P L O Y A S H IE L D E D U S IN G S H IE L D A N E X IS T IN G V M
VM M »
If your hosting service provider supports it, you can use Windows Azure Pack to deploy a shielded VM.
Complete the following steps:
1. Subscribe to one or more plans offered in Windows Azure Pack.
2. Create a shielded VM by using Windows Azure Pack.
Use shielded virtual machines, which is described in the following topics:
Create shielding data (and upload the shielding data file, as described in the second procedure in the
topic).
NOTE
As part of creating shielding data, you will download your guardian key file, which will be an XML file in UTF-
8 format. Do not change the file to UTF-16.
Create a shielded virtual machine - with Quick Create, through a shielded template, or through a
regular template.
WARNING
If you Create a shielded virtual machine by using a regular template, it is important to note that the VM is
provisioned unshielded. This means that the template disk is not verified against the list of trusted disks in
your shielding data file, nor are the secrets in your shielding data file used to provision the VM. If a shielded
template is available, it is preferable to deploy a shielded VM with a shielded template to provide end-to-
end protection of your secrets.
NOTE
If you convert a virtual machine to a shielded virtual machine, existing checkpoints and backups are not
encrypted. You should delete old checkpoints when possible to prevent access to your old, decrypted data.
See also
Hosting service provider configuration steps for guarded hosts and shielded VMs
Guarded fabric and shielded VMs
Shielded VMs - Preparing a VM Shielding Helper
VHD
11/14/2017 • 2 min to read • Edit Online
« D E P L O Y A S H IE L D E D U S IN G W IN D O W S A Z U R E S H IE L D A N E X IS T IN G V M
PACK »
IMPORTANT
Before beginning these procedures, ensure that you have installed the latest cumulative update for Windows Server 2016 or
are using the latest Windows 10 Remote Server Administration Tools. Otherwise, the procedures will not work.
This section outlines steps performed by a hosting service provider to enable support for converting existing VMs
to shielded VMs.
To understand how this topic fits in the overall process of deploying shielded VMs, see Hosting service provider
configuration steps for guarded hosts and shielded VMs.
IMPORTANT
The VM Shielding Helper VHD must not be related to the template disks you created in Hosting service provider
creates a shielded VM template. If you re-use a template disk, there will be a disk signature collision during the
shielding process because both disks will have the same GPT disk identifier. You can avoid this by creating a new
(blank) VHD and installing Windows Server 2016 onto it using your ISO installation media.
2. Start the VM, complete any setup steps, and log into the desktop. Once you have verified the VM is in a
working state, shut down the VM.
3. In an elevated Windows PowerShell window, run the following command to prepare the VHDX created
earlier to become a VM shielding helper disk. Update the path with the correct path for your environment.
4. Once the command has completed successfully, copy the VHDX to your VMM library share. Do not start up
the VM from step 1 again. Doing so will corrupt the helper disk.
5. You can now delete the VM from step 1 in Hyper-V.
See also
Hosting service provider configuration steps for guarded hosts and shielded VMs
Guarded fabric and shielded VMs
Managing a Guarded Fabric
10/17/2017 • 1 min to read • Edit Online
See also
Deploying a guarded fabric
Managing the Host Guardian Service
10/17/2017 • 40 min to read • Edit Online
The Host Guardian Service (HGS) is the centerpiece of the guarded fabric solution. It is responsible for ensuring
that Hyper-V hosts in the fabric are known to the hoster or enterprise and running trusted software and for
managing the keys used to start up shielded VMs. When a tenant decides to trust you to host their shielded VMs,
they are placing their trust in your configuration and management of the Host Guardian Service. Therefore, it is
very important to follow best practices when managing the Host Guardian Service to ensure the security,
availability and reliability of your guarded fabric. The guidance in the following sections addresses the most
common operational issues facing administrators of HGS.
Create standard users for the HGS administrator and reviewer roles
New-ADUser -Name 'hgsadmin01' -AccountPassword (Read-Host -AsSecureString -Prompt 'HGS Admin Password') -
ChangePasswordAtLogon $false -Enabled $true
Add-ADGroupMember -Identity $adminGroup -Members 'hgsadmin01'
New-ADUser -Name 'hgsreviewer01' -AccountPassword (Read-Host -AsSecureString -Prompt 'HGS Reviewer Password')
-ChangePasswordAtLogon $false -Enabled $true
Add-ADGroupMember -Identity $reviewerGroup -Members 'hgsreviewer01'
You can then check which commands are allowed in the session with Get-Command and run any allowed commands
to audit the configuration. In the below example, we are checking which policies are enabled on HGS.
Get-Command
Get-HgsAttestationPolicy
Type the command Exit-PSSession or its alias, exit , when you are done working with the JEA session.
Add a new policy to HGS using the administrator role
To actually change a policy, you need to connect to the JEA endpoint with an identity that belongs to the
'hgsAdministrators' group. In the below example, we show how you can copy a new code integrity policy to HGS
and register it using JEA. The syntax may be different from what you are used to. This is to accommodate some of
the restrictions in JEA like not having access to the full file system.
$cipolicy = Get-Item "C:\temp\cipolicy.p7b"
$session = New-PSSession -ComputerName <hgsnode> -Credential '<hgsdomain>\hgsadmin01' -ConfigurationName
'microsoft.windows.hgs'
Copy-Item -Path $cipolicy -Destination 'User:' -ToSession $session
# Now that the file is copied, we enter the interactive session to register it with HGS
Enter-PSSession -Session $session
Add-HgsAttestationCiPolicy -Name 'New CI Policy via JEA' -Path 'User:\cipolicy.p7b'
Monitoring HGS
Event sources and forwarding
Events from HGS will show up in the Windows event log under 2 sources:
HostGuardianService-Attestation
HostGuardianService-KeyProtection
You can view these events by opening Event Viewer and navigating to Microsoft-Windows-HostGuardianService-
Attestation and Microsoft-Windows-HostGuardianService-KeyProtection.
In a large environment, it is often preferable to forward events to a central Windows Event Collector to make
analyzation of the events easier. For more information, check out the Windows Event Forwarding documentation.
Using System Center Operations Manager
You can also use System Center 2016 - Operations Manager to monitor HGS and your guarded hosts. The guarded
fabric management pack has event monitors to check for common misconfigurations that can lead to datacenter
downtime, including hosts not passing attestation and HGS servers reporting errors.
To get started, install and configure SCOM 2016 and download the guarded fabric management pack. The included
management pack guide explains how to configure the management pack and understand the scope of its
monitors.
NOTE
If you are using admin-trusted attestation, you must separately back up membership in the security groups used by HGS to
authorize guarded hosts. HGS will only back up the SID of the security groups, not the membership within them. In the event
these groups are lost during a disaster, you will need to recreate the group(s) and add each guarded host to them again.
Backing up certificates
The Export-HgsServerState command will back up any PFX-based certificates added to HGS at the time the
command is run. If you added certificates to HGS using a thumbprint (typical for non-exportable and hardware-
backed certificates), you will need to manually back up the private keys for your certificates. To identify which
certificates are registered with HGS and need to be backed up manually, run the following PowerShell command
on any working HGS server node.
For each of the certificates listed, you will need to manually back up the private key. If you are using software-
based certificate that is non-exportable, you should contact your certificate authority to ensure they have a backup
of your certificate and/or can reissue it on demand. For certificates created and stored in hardware security
modules, you should consult the documentation for your device for guidance on disaster recovery planning.
You should store the certificate backups alongside your attestation policy backups in a secure location so that both
pieces can be restored together.
Additional configuration to back up
The backed up HGS server state will not include the name of your HGS cluster, any information from Active
Directory, or any SSL certificates used to secure communications with the HGS APIs. These settings are important
for consistency but not critical to get your HGS cluster back online after a disaster.
To capture the name of the HGS service, run Get-HgsServer and note the flat name in the Attestation and Key
Protection URLs. For example, if the Attestation URL is "http://hgs.contoso.com/Attestation", "hgs" is the HGS
service name.
The Active Directory domain used by HGS should be managed like any other Active Directory domain. When
restoring HGS after a disaster, you will not necessarily need to recreate the exact objects that are present in the
current domain. However, it will make recovery easier if you back up Active Directory and keep a list of the JEA
users authorized to manage the system as well as the membership of any security groups used by admin-trusted
attestation to authorize guarded hosts.
To identify the thumbprint of the SSL certificates configured for HGS, run the following command in PowerShell.
You can then back up those SSL certificates according to your certificate provider's instructions.
TIP
The new HGS cluster does not need to use the same certificates, service name, or domain as the HGS instance from which
your backup file was exported.
If you only want to import admin-trusted attestation policies or TPM-trusted attestation policies, you can do so by
specifying the -ImportActiveDirectoryModeState or -ImportTpmModeState flags to Import-HgsServerState.
Ensure the latest cumulative update for Windows Server 2016 is installed before running Import-HgsServerState .
Failure to do so may result in an import error.
NOTE
If you restore policies on an existing HGS node that already has one or more of those policies installed, the import command
will show an error for each duplicate policy. This is an expected behavior and can be safely ignored in most cases.
Get-HgsTrace -RunDiagnostics
If the "Overall Result" is not "Pass", additional steps are required to finish configuring the system. Check the
messages reported in the subtest(s) that failed for more information.
Patching HGS
It is important to keep your Host Guardian Service nodes up to date by installing the latest cumulative update when
it comes out. If you are setting up a brand new HGS node, it is highly recommended that you install any available
updates before installing the HGS role or configuring it. This will ensure any new or changed functionality will take
effect immediately.
When patching your guarded fabric, it is strongly recommended that you first upgrade all Hyper-V hosts before
upgrading HGS. This is to ensure that any changes to the attestation policies on HGS are made after the Hyper-V
hosts have been updated to provide the information needed for them. If an update is going to change the behavior
of policies, they will not automatically be enabled to avoid disrupting your fabric. Such updates require that you
follow the guidance in the following section to activate the new or changed attestation policies. We encourage you
to read the release notes for Windows Server and any cumulative updates you install to check if the policy updates
are required.
Updates requiring policy activation
If an update for HGS introduces or significantly changes the behavior of an attestation policy, an additional step is
required to activate the changed policy. Policy changes are only enacted after exporting and importing the HGS
state. You should only activate the new or changed policies after you have applied the cumulative update to all
hosts and all HGS nodes in your environment. Once every machine has been updated, run the following
commands on any HGS node to trigger the upgrade process:
If a new policy was introduced, it will be disabled by default. To enable the new policy, first find it in the list of
Microsoft policies (prefixed with 'HGS_') and then enable it using the following commands:
Get-HgsAttestationPolicy
Hgs_CiPolicy Negative policy to ensure hosts are using one of the admin-
defined CI policies.
Hgs_FullBoot Ensures the host did not resume from sleep or hibernation.
Hosts must be properly restarted or shut down to pass this
policy.
Admin-trusted attestation
To register a new host in HGS when using admin-trusted attestation, you must first add the host to a security
group in the domain to which it's joined. Typically, each domain will have one security group for guarded hosts. If
you have already registered that group with HGS, the only action you need to take is to restart the host to refresh
its group membership.
You can check which security groups are trusted by HGS by running the following command:
Get-HgsAttestationHostGroup
To register a new security group with HGS, first capture the security identifier (SID) of the group in the host's
domain and register the SID with HGS.
Add-HgsAttestationHostGroup -Name "Contoso Guarded Hosts" -Identifier "S-1-5-21-3623811015-3361044348-
30300820-1013"
Instructions on how to set up the trust between the host domain and HGS are available in the deployment guide.
TPM-trusted attestation
When HGS is configured in TPM mode, hosts must pass all locked policies and "enabled" policies prefixed with
"Hgs_", as well as at least one TPM baseline, TPM identifier, and code integrity policy. Each time you add a new host,
you will need to register the new TPM identifier with HGS. As long as the host is running the same software (and
has the same code integrity policy applied) and TPM baseline as another host in your environment, you will not
need to add new CI policies or baselines.
Adding the TPM identifier for a new host On the new host, run the following command to capture the TPM
identifier. Be sure to specify a unique name for the host that will help you look it up on HGS. You will need this
information if you decommission the host or want to prevent it from running shielded VMs in HGS.
Copy this file to your HGS server, then run the following command to register the host with HGS.
Adding a new TPM baseline If the new host is running a new hardware or firmware configuration for your
environment, you may need to take a new TPM baseline. To do this, run the following command on the host.
NOTE
If you receive an error saying your host failed validation and will not successfully attest, do not worry. This is a prerequisite
check to make sure your host can run shielded VMs, and likely means that you have not yet applied a code integrity policy or
other required setting. Read the error message, make any changes suggested by it, then try again. Alternatively, you can skip
the validation at this time by adding the -SkipValidation flag to the command.
Copy the TPM baseline to your HGS server, then register it with the following command. We encourage you to use
a naming convention that helps you understand the hardware and firmware configuration of this class of Hyper-V
host.
Adding a new code integrity policy If you have changed the code integrity policy running on your Hyper-V
hosts, you will need to register the new policy with HGS before those hosts can successfully attest. On a reference
host, which serves as a master image for the trusted Hyper-V machines in your environment, capture a new CI
policy using the New-CIPolicy command. We encourage you to use the FilePublisher level and Hash fallback for
Hyper-V host CI policies. You should first create a CI policy in audit mode to ensure that everything is working as
expected. After validating a sample workload on the system, you can enforce the policy and copy the enforced
version to HGS. For a complete list of code integrity policy configuration options, consult the Device Guard
documentation.
# Capture a new CI policy with the FilePublisher primary level and Hash fallback and enable user mode code
integrity protections
New-CIPolicy -FilePath 'C:\temp\ws2016-hardware01-ci.xml' -Level FilePublisher -Fallback Hash -UserPEs
# Check the event log for any untrusted binaries and update the policy if necessary
# Consult the Device Guard documentation for more details
Once you have your policy created, tested and enforced, copy the binary file (.p7b) to your HGS server and register
the policy.
Alternatively, you can directly provide the string representation of the hash to the cmdlet.
Be sure to add each unique dump encryption key to HGS if you choose to use different keys across your guarded
fabric. Hosts that are encrypting memory dumps with a key not known to HGS will not pass attestation.
Consult the Hyper-V documentation for more information about configuring dump encryption on hosts.
Check if the system passed attestation
After registering the necessary information with HGS, you should check if the host passes attestation. On the
newly-added Hyper-V host, run Set-HgsClientConfiguration and supply the correct URLs for your HGS cluster.
These URLs can be obtained by running Get-HgsServer on any HGS node.
Set-HgsClientConfiguration -KeyProtectionServerUrl 'http://hgs.relecloud.com/KeyProtection' -
AttestationServerUrl 'http://hgs.relecloud.com/Attestation'
If the resulting status does not indicate "IsHostGuarded : True" you will need to troubleshoot the configuration. On
the host that failed attestation, run the following command to get a detailed report about issues that may help you
resolve the failed attestation.
If you find a policy enabled that no longer meets your security requirement (e.g. an old code integrity policy which
is now deemed unsafe), you can disable it by replacing the name of the policy in the following command:
After the diagnostics complete, review the outputted information to determine if any hosts would have failed
attestation in TPM mode. Re-run the diagnostics until you get a "pass" from each host, then proceed to change HGS
to TPM mode.
Changing to TPM mode takes just a second to complete. Run the following command on any HGS node to
update the attestation mode.
Set-HgsServer -TrustTpm
If you run into problems and need to switch back to Active Directory mode, you can do so by running
Set-HgsServer -TrustActiveDirectory .
Once you have confirmed everything is working as expected, you should remove all trusted Active Directory host
groups from HGS and remove the trust between the HGS and fabric domains. If you leave the Active Directory trust
in place, you risk someone re-enabling the trust and switching HGS to Active Directory mode, which could allow
untrusted code to run unchecked on your guarded hosts.
Key management
The guarded fabric solution uses several public/private key pairs to validate the integrity of various components in
the solution and encrypt tenant secrets. The Host Guardian Service is configured with at least two certificates (with
public and private keys), which are used for signing and encrypting the keys used to start up shielded VMs. Those
keys must be carefully managed. If the private key is acquired by an adversary, they will be able to unshield any
VMs running on your fabric or set up an imposter HGS cluster that uses weaker attestation policies to bypass the
protections you put in place. Should you lose the private keys during a disaster and not find them in a backup, you
will need to set up a new pair of keys and have each VM re-keyed to authorize your new certificates.
This section covers general key management topics to help you configure your keys so they are functional and
secure.
Adding new keys
While HGS must be initialized with one set of keys, you can add more than one encryption and signing key to HGS.
The two most common reasons why you would add new keys to HGS are:
1. To support "bring your own key" scenarios where tenants copy their private keys to your hardware security
module and only authorize their keys to start up their shielded VMs.
2. To replace the existing keys for HGS by first adding the new keys and keeping both sets of keys until each VM
configuration has been updated to use the new keys.
The process to add your new keys differs based on the type of certificate you are using.
Option 1: Adding a certificate stored in an HSM
Our recommended approach for securing HGS keys is to use certificates created in a hardware security module
(HSM). HSMs ensure use of your keys is tied to physical access to a security-sensitive device in your datacenter.
Each HSM is different and has a unique process to create certificates and register them with HGS. The steps below
are intended to provide rough guidance for using HSM backed certificates. Consult your HSM vendor's
documentation for exact steps and capabilities.
1. Install the HSM software on each HGS node in your cluster. Depending on whether you have a network or local
HSM device, you may need to configure the HSM to grant your machine access to its key store.
2. Create 2 certificates in the HSM with 2048 bit RSA keys for encryption and signing
a. Create an encryption certificate with the Data Encipherment key usage property in your HSM
b. Create a signing certificate with the Digital Signature key usage property in your HSM
3. Install the certificates in each HGS node's local certificate store per your HSM vendor's guidance.
4. If your HSM uses granular permissions to grant specific applications or users permission to use the private key,
you will need to grant your HGS group managed service account access to the certificate. You can find the name
of the HGS gMSA account by running (Get-IISAppPool -Name KeyProtection).ProcessModel.UserName
5. Add the signing and encryption certificates to HGS by replacing the thumbprints with those of your
certificates' in the following commands:
IMPORTANT
You will need to manually install the private key and grant read access to the gMSA account on each HGS node. HGS cannot
automatically replicate private keys for any certificate registered by its thumbprint.
Identifying and changing the primary certificates While HGS can support multiple signing and encryption
certificates, it uses one pair as its "primary" certificates. These are the certificates that will be used if someone
downloads the guardian metadata for that HGS cluster. To check which certificates are currently marked as your
primary certificates, run the following command:
To set a new primary encryption or signing certificate, find the thumbprint of the desired certificate and mark it as
primary using the following commands:
Get-HgsKeyProtectionCertificate
Set-HgsKeyProtectionCertificate -CertificateType Encryption -Thumbprint "AABBCCDDEEFF00112233445566778899" -
IsPrimary
Set-HgsKeyProtectionCertificate -CertificateType Signing -Thumbprint "99887766554433221100FFEEDDCCBBAA" -
IsPrimary
3. If you used thumbprints, you'll need to go to each node in the cluster to install the private key and grant the
HGS gMSA access to the key.
4. Make the new certificates the default certificates in HGS
At this point, shielding data created with metadata obtained from the HGS node will use the new certificates, but
existing VMs will continue to work because the older certificates are still there. In order to ensure all existing VMs
will work with the new keys, you will need to update the key protector on each VM. This is an action that requires
the VM owner (person or entity in possession of the "owner" guardian) to be involved. For each shielded VM,
perform the following steps:
1. Shut down the VM. The VM cannot be turned back on until the remaining steps are complete or else you will
need to start the process over again.
2. Save the current key protector to a file: Get-VMKeyProtector -VMName 'VM001' | Out-File '.\VM001.kp'
3. Transfer the KP to the VM owner
4. Have the owner download the updated guardian info from HGS and import it on their local system
5. Read the current KP into memory, grant the new guardian access to the KP, and save it to a new file by
running the following commands:
NOTE
If the VM owner sets an incorrect key protector on the VM and does not authorize your fabric to run the VM, you will be
unable to start up the shielded VM. To return to the last known good key protector, run
Set-VMKeyProtector -RestoreLastKnownGoodKeyProtector
Once all VMs have been updated to authorize the new guardian keys, you can disable and remove the old keys.
1. Get the thumbprints of the old certificates from Get-HgsKeyProtectionCertificate -IsPrimary $false
2. Disable each certificate by replacing the certificate type and thumbprint in the following command:
Set-HgsKeyProtectionCertificate -CertificateType Encryption -Thumbprint <Thumbprint> -IsEnabled $false
3. After ensuring VMs are still able to start with the certificates disabled, remove the certificates from HGS with
Remove-HgsKeyProtectionCertificate -CertificateType Encryption -Thumbprint <Thumbprint>
IMPORTANT
VM backups will contain old key protector information that allow the old certificates to be used to start up the VM. If you are
aware that your private key has been compromised, you should assume that the VM backups can be compromised, too, and
take appropriate action. Destroying the VM configuration from the backups (.vmcx) will remove the key protectors, at the
cost of needing to use the BitLocker recovery password to boot the VM the next time.
Unconfiguring HGS
If you need to decommission or significantly reconfigure an HGS server, you can do so using the Clear-HgsServer
or Uninstall-HgsServer cmdlets.
Clearing the HGS configuration
To remove a node from the HGS cluster, use the Clear-HgsServer cmdlet. This cmdlet will make the following
changes on the server where it is run:
Unregisters the attestation and key protection services
Removes the "microsoft.windows.hgs" JEA management endpoint
Removes the local computer from the HGS failover cluster
If the server is the last HGS node in the cluster, the cluster and its corresponding Distributed Network Name
resource will also be destroyed.
After the clear operation completes, the HGS server can be re-initialized with Initialize-HgsServer. If you used
Install-HgsServer to set up an Active Directory Domain Services domain, that domain will remain configured and
operational after the clear operation.
Uninstalling HGS
If you wish to remove a node from the HGS cluster and demote the Active Directory Domain Controller running on
it, use the Uninstall-HgsServer cmdlet. This cmdlet will make the following changes on the server where it is run:
Unregisters the attestation and key protection services
Removes the "microsoft.windows.hgs" JEA management endpoint
Removes the local computer from the HGS failover cluster
Demotes the Active Directory Domain Controller, if configured
If the server is the last HGS node in the cluster, the domain, failover cluster, and the cluster's Distributed Network
Name resource will also be destroyed.
# Removes the local computer from the HGS cluster and demotes the ADDC (restart required)
$newLocalAdminPassword = Read-Host -AsSecureString -Prompt "Enter a new password for the local administrator
account"
Uninstall-HgsServer -LocalAdministratorPassword $newLocalAdminPassword -Restart
After the uninstall operation is complete and the computer has been restarted, you can reinstall ADDC and HGS
using Install-HgsServer or join the computer to a domain and initialize the HGS server in that domain with
Initialize-HgsServer.
If you no longer intend to use the computer as a HGS node, you can remove the role from Windows.
Uninstall-WindowsFeature HostGuardianServiceRole
Troubleshooting a Guarded Fabric
6/19/2017 • 1 min to read • Edit Online
See also
Deploying the Host Guardian Service for guarded hosts and shielded VMs
Managing a guarded fabric
Troubleshooting Using the Guarded Fabric Diagnostic
Tool
10/17/2017 • 13 min to read • Edit Online
This topic describes the use of the Guarded Fabric Diagnostic Tool to identify and remediate common failures in
the deployment, configuration, and on-going operation of guarded fabric infrastructure. This includes the Host
Guardian Service (HGS), all guarded hosts, and supporting services such as DNS and Active Directory. The
diagnostic tool can be used to perform a first-pass at triaging a failing guarded fabric, providing administrators
with a starting point for resolving outages and identifying misconfigured assets. The tool is not a replacement for a
sound grasp of operating a guarded fabric and only serves to rapidly verify the most common issues encountered
during day-to-day operations.
Documentation of the cmdlets used in this topic can be found on TechNet.
NOTE
When running the Guarded Fabric diagnostics tool (Get-HgsTrace -RunDiagnostics), incorrect status may be returned
claiming that the HTTPS configuration is broken when it is, in fact, not broken or not being used. This error can be returned
regardless of HGS’ attestation mode. The possible root-causes are as follows:
HTTPS is indeed improperly configured/broken
You’re using admin-trusted attestation and the trust relationship is broken
- This is irrespective of whether HTTPS is configured properly, improperly, or not in use at all.
Note that the diagnostics will only return this incorrect status when targeting a Hyper-V host. If the diagnostics are targeting
the Host Guardian Service, the status returned will be correct.
Quick Start
You can diagnose either a guarded host or an HGS node by calling the following from a Windows PowerShell
session with local administrator privileges:
This will automatically detect the role of the current host and diagnose any relevant issues that can be
automatically detected. All of the results generated during this process are displayed due to the presence of the
-Detailed switch.
The remainder of this topic will provide a detailed walkthrough on the advanced usage of Get-HgsTrace for doing
things like diagnosing multiple hosts at once and detecting complex cross-node misconfiguration.
Diagnostics Overview
Guarded fabric diagnostics are available on any host with shielded virtual machine related tools and features
installed, including hosts running Server Core. Presently, diagnostics are included with the following
features/packages:
1. Host Guardian Service Role
2. Host Guardian Hyper-V Support
3. VM Shielding Tools for Fabric Management
4. Remote Server Administration Tools (RSAT)
This means that diagnostic tools will be available on all guarded hosts, HGS nodes, certain fabric management
servers, and any Windows 10 workstations with RSAT installed. Diagnostics can be invoked from any of the above
machines with the intent of diagnosing any guarded host or HGS node in a guarded fabric; using remote trace
targets, diagnostics can locate and connect to hosts other than the machine running diagnostics.
Every host targeted by diagnostics is referred to as a "trace target." Trace targets are identified by their hostnames
and roles. Roles describe the function a given trace target performs in a guarded fabric. Presently, trace targets
support HostGuardianService and GuardedHost roles. Note it is possible for a host to occupy multiple roles at once
and this is also supported by diagnostics, however this should not be done in production environments. The HGS
and Hyper-V hosts should be kept separate and distinct at all times.
Administrators can begin any diagnostic tasks by running Get-HgsTrace . This command performs two distinct
functions based on the switches provided at runtime: trace collection and diagnosis. These two combined make up
the entirety of the Guarded Fabric Diagnostic Tool. Though not explicitly required, most useful diagnostics require
traces that can only be collected with administrator credentials on the trace target. If insufficient privileges are held
by the user executing trace collection, traces requiring elevation will fail while all others will pass. This allows partial
diagnosis in the event an under-privileged operator is performing triage.
Trace collection
By default, Get-HgsTrace will only collect traces and save them to a temporary folder. Traces take the form of a
folder, named after the targeted host, filled with specially formatted files that describe how the host is configured.
The traces also contain metadata that describe how the diagnostics were invoked to collect the traces. This data is
used by diagnostics to rehydrate information about the host when performing manual diagnosis.
If necessary, traces can be manually reviewed. All formats are either human-readable (XML) or may be readily
inspected using standard tools (e.g. X509 certificates and the Windows Crypto Shell Extensions). Note however that
traces are not designed for manual diagnosis and it is always more effective to process the traces with the
diagnosis facilities of Get-HgsTrace .
The results of running trace collection do not make any indication as to the health of a given host. They simply
indicate that traces were collected successfully. It is necessary to use the diagnosis facilities of Get-HgsTrace to
determine if the traces indicate a failing environment.
Using the -Diagnostic parameter, you can restrict trace collection to only those traces required to operate the
specified diagnostics. This reduces the amount of data collected as well as the permissions required to invoke
diagnostics.
Diagnosis
Collected traces can be diagnosed by provided Get-HgsTrace the location of the traces via the -Path parameter
and specifying the -RunDiagnostics switch. Additionally, Get-HgsTrace can perform collection and diagnosis in a
single pass by providing the -RunDiagnostics switch and a list of trace targets. If no trace targets are provided, the
current machine is used as an implicit target, with its role inferred by inspecting the installed Windows PowerShell
modules.
Diagnosis will provide results in a hierarchical format showing which trace targets, diagnostic sets, and individual
diagnostics are responsible for a particular failure. Failures include remediation and resolution recommendations if
a determination can be made as to what action should be taken next. By default, passing and irrelevant results are
hidden. To see everything tested by diagnostics, specify the -Detailed switch. This will cause all results to appear
regardless of their status.
It is possible to restrict the set of diagnostics that are run using the -Diagnostic parameter. This allows you to
specify which classes of diagnostic should be run against the trace targets, and suppressing all others. Examples of
available diagnostic classes include networking, best practices, and client hardware. Consult the cmdlet
documentation to find an up-to-date list of available diagnostics.
WARNING
Diagnostics are not a replacement for a strong monitoring and incident response pipeline. There is a System Center
Operations Manager package available for monitoring guarded fabrics, as well as various event log channels that can be
monitored to detect issues early. Diagnostics can then be used to quickly triage these failures and establish a course of
action.
Targeting Diagnostics
Get-HgsTrace operates against trace targets. A trace target is an object that corresponds to an HGS node or a
guarded host inside a guarded fabric. It can be thought of as an extension to a PSSession which includes
information required only by diagnostics such as the role of the host in the fabric. Targets can be generated
implicitly (e.g. local or manual diagnosis) or explicitly with the New-HgsTraceTarget command.
Local Diagnosis
By default, Get-HgsTrace will target the localhost (i.e. where the cmdlet is being invoked). This is referred as the
implicit local target. The implicit local target is only used when no targets are provided in the -Target parameter
and no pre-existing traces are found in the -Path .
The implicit local target uses role inference to determine what role the current host plays in the guarded fabric. This
is based on the installed Windows PowerShell modules which roughly correspond to what features have been
installed on the system. The presence of the HgsServer module will cause the trace target to take the role
HostGuardianService and the presence of the HgsClient module will cause the trace target to take the role
GuardedHost . It is possible for a given host to have both modules present in which case it will be treated as both a
HostGuardianService and a GuardedHost .
Get-HgsTrace
TIP
Get-HgsTrace can accept targets via the pipeline or directly via the -Target parameter. There is no difference between
the two operationally.
NOTE
In most cases, it is only necessary that the localhost be a part of the same Active Directory forest and that a valid DNS
hostname is used. If your environment utilizes a more complicated federation model or you wish to use direct IP addresses
for connectivity, you may need to perform additional configuration such as setting the WinRM trusted hosts.
You can verify that a trace target is properly instantiated and configured for accepting connections by using the
Test-HgsTraceTarget cmdlet:
This command will return $True if and only if Get-HgsTrace would be able to establish a remote diagnostic
session with the trace target. Upon failure, this cmdlet will return relevant status information for further
troubleshooting of the Windows PowerShell remoting connection.
Implicit Credentials
When performing remote diagnosis from a user with sufficient privileges to connect remotely to the trace target, it
is not necessary to supply credentials to New-HgsTraceTarget . The Get-HgsTrace cmdlet will automatically reuse the
credentials of the user that invoked the cmdlet when opening a connection.
WARNING
Some restrictions apply to reusing credentials, particularly when performing what is known as a "second hop." This occurs
when attempting to reuse credentials from inside a remote session to another machine. It is necessary to setup CredSSP to
support this scenario, but this is outside of the scope of guarded fabric management and troubleshooting.
NOTE
You do not need to diagnose your entire guarded fabric when diagnosing multiple nodes. In many cases it is sufficient to
include all nodes that may be involved in a given failure condition. This is usually a subset of the guarded hosts, and some
number of nodes from the HGS cluster.
NOTE
Traces are not anonymized and reveal network configuration, PKI settings, and other configuration that is sometimes
considered private information. Therefore, traces should only be transmitted to trusted entities within an organization and
never posted publicly.
2. Request that each host administrator package the resulting traces folder and send it to you. This process can
be driven over e-mail, via file shares, or any other mechanism based on the operating policies and
procedures established by your organization.
3. Merge all received traces into a single folder, with no other contents or folders.
For example, assume you had your administrators send you traces collected from four machines
named HGS-01, HGS-02, RR1N2608-12, and RR1N2608-13. Each administrator would have sent you
a folder by the same name. You would assemble a directory structure that appears as follows:
FabricTraces
|- HGS-01
| |- TargetMetadata.xml
| |- Metadata.xml
| |- [any other trace files for this host]
|- HGS-02
| |- [...]
|- RR1N2608-12
| |- [...]
|- RR1N2608-13
|- [..]
4. Execute diagnostics, providing the path to the assembled trace folder on the -Path parameter and
specifying the -RunDiagnostics switch as well as those diagnostics for which you asked your administrators
to collect traces. Diagnostics will assume it cannot access the hosts found inside the path and will therefore
attempt to use only the pre-collected traces. If any traces are missing or damaged, diagnostics will fail only
the affected tests and proceed normally. For example:
The diagnostic cmdlet will identify all pre-collected hosts, and the one additional host that still needs to be traced
and will perform the necessary tracing. The sum of all pre-collected and freshly gathered traces will then be
diagnosed. The resulting trace folder will contain both the old and new traces.
Troubleshooting the Host Guardian Service
10/17/2017 • 6 min to read • Edit Online
This topic describes resolutions to common problems encountered when deploying or operating a Host Guardian
Service (HGS) server in a guarded fabric. If you are unsure of the nature of your problem, first try running the
guarded fabric diagnostics on your HGS servers and Hyper-V hosts to narrow down the potential causes.
Certificates
HGS requires several certificates in order to operate, including the admin-configured encryption and signing
certificate as well as an attestation certificate managed by HGS itself. If these certificates are incorrectly configured,
HGS will be unable to serve requests from Hyper-V hosts wishing to attest or unlock key protectors for shielded
VMs. The following sections cover common problems related to certificates configured on HGS.
Certificate Permissions
HGS must be able to access both the public and private keys of the encryption and signing certificates added to
HGS by the certificate thumbprint. Specifically, the group managed service accoung (gMSA) that runs the HGS
service needs access to the keys. To find the gMSA used by HGS, run the following command in an elevated
PowerShell prompt on your HGS server:
How you grant the gMSA account access to use the private key depends on where the key is stored: on the machine
as a local certificate file, on a hardware security module (HSM), or using a custom third-party key storage provider.
Grant access to software-backed private keys
If you are using a self-signed certificate or a certificate issued by a certificate authority that is not stored in a
hardware security module or custom key storage provider, you can change the private key permissions by
performing the following steps:
1. Open local certificate manager (certlm.msc)
2. Expand Personal > Certificates and find the signing or encryption certificate that you want to update.
3. Right click the certificate and select All Tasks > Manage Private Keys.
4. Click Add to grant a new user access to the certiciate's private key.
5. In the object picker, enter the gMSA account name for HGS found earlier, then click OK.
6. Ensure the gMSA has Read access to the certificate.
7. Click OK to close the permission window.
If you are running HGS on Server Core or are managing the server remotely, you will not be able to manage private
keys using the local certificate manager. Instead, you will need to download the Guarded Fabric Tools PowerShell
module which will allow you to manage the permissions in PowerShell.
1. Open an elevated PowerShell console on the Server Core machine or use PowerShell Remoting with an account
that has local administrator permissions on HGS.
2. Run the following commands to install the Guarded Fabric Tools PowerShell module and grant the gMSA
account access to the private key.
$certificateThumbprint = '<ENTER CERTIFICATE THUMBPRINT HERE>'
Gemalto SafeNet Ensure the Key Usage Property in the certificate request file is
set to 0xa0, allowing the certificate to be used for signing and
encryption. Additionally, you must grant the gMSA account
read access to the private key using the local certificate
manager tool (see steps above).
Thales nShield Ensure each HGS node has access to the security world
containing the signing and encryption keys. You do not need
to configure gMSA-specific permissions.
Utimaco CryptoServers Ensure the Key Usage Property in the certificate request file is
set to 0x13, allowing the certificate to be used for encryption,
decryption, and signing.
Certificate requests
If you are using a certificate authority to issue your certificates in a public key infrastructure (PKI) environment, you
will need to ensure your certificate request includes the minimum requirements for HGS' usage of those keys.
Signing Certificates
CSR PROPERTY REQUIRED VALUE
Algorithm RSA
Encryption Certificates
Algorithm RSA
Time Drift
If your server's time has drifted significantly from that of other HGS nodes or Hyper-V hosts in your guarded fabric,
you may encounter issues with the attestation signer certificate validity. The attestation signer certificate is created
and renewed behind the scenes on HGS and is used to sign health certificates issued to guarded hosts by the
Attestation Service.
To refresh the attestation signer certificate, run the following command in an elevated PowerShell prompt.
Alternatively, you can manually run the scheduled task by opening Task Scheduler (taskschd.msc), navigating to
Task Scheduler Library > Microsoft > Windows > HGSServer and running the task named
AttestationSignerCertRenewalTask.
This topic describes resolutions to common problems encountered when deploying or operating a guarded Hyper-
V host in your guarded fabric. If you are unsure of the nature of your problem, first try running the guarded fabric
diagnostics on your Hyper-V hosts to narrow down the potential causes.
Get-WindowsFeature HostGuardian
If the feature is not installed, install it with the following PowerShell command:
Attestation failures
If a host does not pass attestation with the Host Guardian Service, it will be unable to run shielded VMs. The output
of Get-HgsClientConfiguration on that host will show you information about why that host failed attestation.
The table below explains the values that may appear in the AttestationStatus field and potential next steps, if
appropriate.
ATTESTATIONSTATUS EXPLANATION
InsecureHostConfiguration The host did not pass attestation because it did not comply
with the attestation policies configured on HGS. Consult the
AttestationSubStatus table for more information.
NotConfigured The host is not configured to use a HGS for attestation and
key protection. It is configured for local mode, instead. If this
host is in a guarded fabric, use Set-HgsClientConfiguration to
provide it with the URLs for your HGS server.
TpmError The host could not complete its last attestation attempt due
to an error with your TPM. Consult your TPM logs for more
information.
UnauthorizedHost The host did not pass attestation becuase it was not
authorized to run shielded VMs. Ensure the host belongs to a
security group trusted by HGS to run shielded VMs.
Unknown The host has not attempted to attest with HGS yet.
DumpEncryptionKey The host is configured to allow and encrypt dumps, but is not
using a certificate known to HGS to encrypt them. To resolve
this, update the dump encryption key on the host or register
the key with HGS.
PagefileEncryption Page file encryption is not enabled on the host. To resolve this,
run fsutil behavior set encryptpagingfile 1 to enable
page file encryption. For more information, see fsutil behavior.
SecureBoot Secure Boot is either not enabled on this host or not using the
Microsoft Secure Boot template. Enable Secure Boot with the
Microsoft Secure Boot template to resolve this issue.
SecureBootSettings The TPM baseline on this host does not match any of those
trusted by HGS. This can occur when your UEFI launch
authorities, DBX variable, debug flag, or custom Secure Boot
policies are changed by installing new hardware or software. If
you trust the current hardware, firmware, and software
configuration of this machine, you can capture a new TPM
baseline and register it with HGS.
The Hyper-V role in Windows Server lets you create a virtualized computing environment where you can create
and manage virtual machines. You can run multiple operating systems on one physical computer and isolate the
operating systems from each other. With this technology, you can improve the efficiency of your computing
resources and free up your hardware resources.
See the topics in the following table to learn more about Hyper-V on Windows Server 2016.
Evaluate Hyper-V
- Choose a generation
- Hyper-V scalability
- Hyper-V networking
- Hyper-V security
Blogs
Stay connected
Follow us on Twitter
Related technologies
The following table lists technologies that you might want to use in your virtualization computing environment.
TECHNOLOGY DESCRIPTION
Failover Clustering Windows Server feature that provides high availability for
Hyper-V hosts and virtual machines.
Nano Server New installation option for Windows Server 2016 that is a
remotely administered server operating system optimized for
hosting in private clouds and datacenters.
TECHNOLOGY DESCRIPTION
Hyper-V is Microsoft's hardware virtualization product. It lets you create and run a software version of a computer,
called a virtual machine. Each virtual machine acts like a complete computer, running an operating system and
programs. When you need computing resources, virtual machines give you more flexibility, help save time and
money, and are a more efficient way to use hardware than just running one operating system on physical
hardware.
Hyper-V runs each virtual machine in its own isolated space, which means you can run more than one virtual
machine on the same hardware at the same time. You might want to do this to avoid problems such as a crash
affecting the other workloads, or to give different people, groups or services access to different systems.
Related technologies
These are some technologies from Microsoft that are often used with Hyper-V:
Failover Clustering
Remote Desktop Services
Virtual Machine Manager
Various storage technologies: cluster shared volumes, SMB 3.0, storage spaces direct
Windows containers offer another approach to virtualization. See the Windows Containers library on MSDN.
What's new in Hyper-V on Windows Server 2016
10/17/2017 • 13 min to read • Edit Online
This article explains the new and changed functionality of Hyper-V on Windows Server 2016 and Microsoft Hyper-
V Server 2016. To use new features on virtual machines created with Windows Server 2012 R2 and moved or
imported to a server that runs Hyper-V on Windows Server 2016, you'll need to manually upgrade the virtual
machine configuration version. For instructions, see Upgrade virtual machine version.
Here's what's included in this article and whether the functionality is new or updated.
IMPORTANT
The vmguest.iso image file is no longer needed, so it isn't included with Hyper-V on Windows Server 2016.
For more information about Linux virtual machines on Hyper-V, see Linux and FreeBSD Virtual Machines on
Hyper-V. For more information about the cmdlet, see Set-VMFirmware.
IMPORTANT
After you update the cluster functional level, you can't return it to Windows Server 2012 R2.
For a Hyper-V cluster with a functional level of Windows Server 2012 R2 with nodes running Windows Server
2012 R2 and Windows Server 2016, note the following:
Manage the cluster, Hyper-V, and virtual machines from a node running Windows Server 2016 or Windows
10.
You can move virtual machines between all of the nodes in the Hyper-V cluster.
To use new Hyper-V features, all nodes must run Windows Server 2016 and the cluster functional level
must be updated.
The virtual machine configuration version for existing virtual machines isn't upgraded. You can upgrade the
configuration version only after you upgrade the cluster functional level.
Virtual machines that you create are compatible with Windows Server 2012 R2, virtual machine
configuration level 5.
After you update the cluster functional level:
You can enable new Hyper-V features.
To make new virtual machine features available, use the Update-VmConfigurationVersion cmdlet to
manually update the virtual machine configuration level. For instructions, see Upgrade virtual machine
version.
You can't add a node to the Hyper-V Cluster that runs Windows Server 2012 R2.
NOTE
Hyper-V on Windows 10 doesn't support failover clustering.
For details and instructions, see the Cluster Operating System Rolling Upgrade.
NOTE
As of Technical Preview 5, shielded virtual machines are compatible with Hyper-V Replica. To replicate a shielded virtual
machine, the host you want to replicate to must be authorized to run that shielded virtual machine.
IMPORTANT
The .vmcx file name extension indicates a binary file. Editing .vmcx or .vmrs files isn't supported.
IMPORTANT
After you update the version, you can't move the virtual machine to a server that runs Windows Server 2012 R2.
You can't downgrade the configuration to a previous version.
The Update-VMVersion cmdlet is blocked on a Hyper-V Cluster when the cluster functional level is Windows Server 2012
R2.
See also
What's new in Hyper-V on Windows 10
Not finding content you need? Windows 10 users, tell us what you want on Feedback Hub.
System requirements for Hyper-V on Windows
Server 2016
6/19/2017 • 3 min to read • Edit Online
Hyper-V has specific hardware requirements, and some Hyper-V features have additional requirements. Use the
details in this article to decide what requirements your system must meet so you can use Hyper-V the way you
plan to. Then, review the Windows Server catalog. Keep in mind that requirements for Hyper-V exceed the general
minimum requirements for Windows Server 2016 because a virtualization environment requires more
computing resources.
If you're already using Hyper-V, it's likely that you can use your existing hardware. The general hardware
requirements haven't changed significantly from Windows Server 2012 R2 . But, you will need newer hardware to
use shielded virtual machines or discrete device assignment. Those features rely on specific hardware support, as
described below. Other than that, the main difference in hardware is that second-level address translation (SLAT)
is now required instead of recommended.
For details about maximum supported configurations for Hyper-V, such as the number of running virtual
machines, see Plan for Hyper-V scalability in Windows Server 2016. The list of operating systems you can run in
your virtual machines is covered in Supported Windows guest operating systems for Hyper-V on Windows
Server.
General requirements
Regardless of the Hyper-V features you want to use, you'll need:
A 64-bit processor with second-level address translation (SLAT). To install the Hyper-V virtualization
components such as Windows hypervisor, the processor must have SLAT. However, it's not required to
install Hyper-V management tools like Virtual Machine Connection (VMConnect), Hyper-V Manager, and
the Hyper-V cmdlets for Windows PowerShell. See "How to check for Hyper-V requirements," below, to
find out if your processor has SLAT.
VM Monitor Mode extensions
Enough memory - plan for at least 4 GB of RAM. More memory is better. You'll need enough memory for
the host and all virtual machines that you want to run at the same time.
Virtualization support turned on in the BIOS or UEFI:
Hardware-assisted virtualization. This is available in processors that include a virtualization option -
specifically processors with Intel Virtualization Technology (Intel VT) or AMD Virtualization (AMD-V)
technology.
Hardware-enforced Data Execution Prevention (DEP) must be available and enabled. For Intel
systems, this is the XD bit (execute disable bit). For AMD systems, this is the NX bit (no execute bit).
Hyper-V supports several versions of Windows Server, Windows, and Linux distributions to run in virtual
machines, as guest operating systems. This article covers supported Windows Server and Windows guest
operating systems. For Linux and FreeBSD distributions, see Supported Linux and FreeBSD virtual machines for
Hyper-V on Windows.
Some operating systems have the integration services built-in. Others require that you install or upgrade
integration services as a separate step after you set up the operating system in the virtual machine. For more
information, see the sections below and Integration Services.
Windows Server 2008 with 8 Install all critical Windows Datacenter, Enterprise,
Service Pack 2 (SP2) updates after you set up the Standard and Web editions
guest operating system. (32-bit and 64-bit).
Windows 10 32 Built-in
Windows Server 2012 R2 and Windows 8.1 - Supported Windows Guest Operating Systems for Hyper-V
in Windows Server 2012 R2 and Windows 8.1
- Linux and FreeBSD Virtual Machines on Hyper-V
Windows Server 2012 and Windows 8 Supported Windows Guest Operating Systems for Hyper-V in
Windows Server 2012 and Windows 8
Windows Server 2008 and Windows Server 2008 R2 About Virtual Machines and Guest Operating Systems
See also
Linux and FreeBSD Virtual Machines on Hyper-V
Supported Guest Operating Systems for Client Hyper-V in Windows 10
Supported Linux and FreeBSD virtual machines for
Hyper-V on Windows
10/17/2017 • 2 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1,
Windows 8, Windows 7.1, Windows 7
Hyper-V supports both emulated and Hyper-V-specific devices for Linux and FreeBSD virtual machines. When
running with emulated devices, no additional software is required to be installed. However emulated devices do
not provide high performance and cannot leverage the rich virtual machine management infrastructure that the
Hyper-V technology offers. In order to make full use of all benefits that Hyper-V provides, it is best to use Hyper-
V-specific devices for Linux and FreeBSD. The collection of drivers that are required to run Hyper-V-specific
devices are known as Linux Integration Services (LIS) or FreeBSD Integration Services (BIS).
LIS has been added to the Linux kernel and is updated for new releases. But Linux distributions based on older
kernels may not have the latest enhancements or fixes. Microsoft provides a download containing installable LIS
drivers for some Linux installations based on these older kernels. Because distribution vendors include versions
of Linux Integration Services, it is best to install the latest downloadable version of LIS, if applicable, for your
installation.
For other Linux distributions LIS changes are regularly integrated into the operating system kernel and
applications so no separate download or installation is required.
For older FreeBSD releases (before 10.0), Microsoft provides ports that contain the installable BIS drivers and
corresponding daemons for FreeBSD virtual machines. For newer FreeBSD releases, BIS is built in to the FreeBSD
operating system, and no separate download or installation is required except for a KVP ports download that is
needed for FreeBSD 10.0.
TIP
Download Windows Server 2016 from the Evaluation Center.
DownloadMicrosoft Hyper-V Server 2016 from the Evaluation Center.
The goal of this content is to provide information that helps facilitate your Linux or FreeBSD deployment on
Hyper-V. Specific details include:
Linux distributions or FreeBSD releases that require the download and installation of LIS or BIS drivers.
Linux distributions or FreeBSD releases that contain built-in LIS or BIS drivers.
Feature distribution maps that indicate the features in major Linux distributions or FreeBSD releases.
Known issues and workarounds for each distribution or release.
Feature description for each LIS or BIS feature.
Want to make a suggestion about features and functionality? Is there something we could do better?You
can use the Windows Server User Voice site to suggest new features and capabilities for Linux and FreeBSD
Virtual Machines on Hyper-V, and to see what other peopleare saying.
In this section
Supported CentOS and Red Hat Enterprise Linux virtual machines on Hyper-V
Supported Debian virtual machines on Hyper-V
Supported Oracle Linux virtual machines on Hyper-V
Supported SUSE virtual machines on Hyper-V
Supported Ubuntu virtual machines on Hyper-V
Supported FreeBSD virtual machines on Hyper-V
Feature Descriptions for Linux and FreeBSD virtual machines on Hyper-V
Best Practices for running Linux on Hyper-V
Best practices for running FreeBSD on Hyper-V
Not finding content you need? Windows 10 users, tell us what you want on Feedback Hub.
Supported CentOS and Red Hat Enterprise Linux
virtual machines on Hyper-V
12/21/2017 • 11 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
The following feature distribution maps indicate the features that are present in built-in and downloadable
versions of Linux Integration Services. The known issues and workarounds for each distribution are listed after the
tables.
The built-in Red Hat Enterprise Linux Integration Services drivers for Hyper-V (available since Red Hat Enterprise
Linux 6.4) are sufficient for Red Hat Enterprise Linux guests to run using the high performance synthetic devices
on Hyper-V hosts.These built-in drivers are certified by Red Hat for this use. Certified configurations can be viewed
on this Red Hat web page: Red Hat Certification Catalog. It isn't necessary to download and install Linux
Integration Services packages from the Microsoft Download Center, and doing so may limit your Red Hat support
as described in Red Hat Knowledgebase article 1067: Red Hat Knowledgebase 1067.
Because of potential conflicts between the built-in LIS support and the downloadable LIS support when you
upgrade the kernel, disable automatic updates, uninstall the LIS downloadable packages, update the kernel, reboot,
and then install the latest LIS release, and reboot again.
NOTE
Official Red Hat Enterprise Linux certification information is available through the Red Hat Customer Portal.
In this section:
RHEL/CentOS 7.x Series
RHEL/CentOS 6.x Series
RHEL/CentOS 5.x Series
Notes
Table legend
Built in - LIS are included as part of this Linux distribution. The kernel module version numbers for the built
in LIS (as shown by lsmod, for example) are different from the version number on the Microsoft-provided
LIS download package. A mismatch does not indicate that the built in LIS is out of date.
✔ - Feature available
Availabil LIS 4.2 LIS 4.1 Built in Built in Built in Built in Built in
ity
Core 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
2012 R2,
2012,
2008 R2
Windows 2016 ✔
Server
2016
Accurate
Time
Networki
ng
Jumbo 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
frames 2012 R2,
2012,
2008 R2
VLAN 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
tagging 2012 R2,
and 2012,
trunking 2008 R2
Live 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
Migration 2012 R2,
2012,
2008 R2
vRSS 2016, ✔ ✔ ✔ ✔ ✔ ✔
2012 R2
TCP 2016, ✔ ✔ ✔ ✔ ✔ ✔
Segmenta 2012 R2,
tion and 2012,
Checksu 2008 R2
m
Offloads
SR-IOV 2016 ✔ ✔ ✔
Storage
VHDX 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
resize 2012 R2
WINDOWS
SERVER
FEATURE VERSION 7.0-7.4 7.0-7.2 7.4 7.3 7.2 7.1 7.0
TRIM 2016, ✔ ✔ ✔ ✔ ✔
support 2012 R2
SCSI 2016, ✔ ✔
WWN 2012 R2
Memory
Configura 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
tion of 2012 R2
MMIO
gap
Runtime 2016 ✔ ✔
Memory
Resize
Video
Hyper-V- 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
specific 2012 R2,
video 2012,
device 2008 R2
Miscellan
eous
Key-Value 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
Pair 2012 R2,
2012,
2008 R2
WINDOWS
SERVER
FEATURE VERSION 7.0-7.4 7.0-7.2 7.4 7.3 7.2 7.1 7.0
Non- 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
Maskable 2012 R2
Interrupt
lsvmbus 2016, ✔ ✔
command 2012 R2,
2012,
2008 R2
Hyper-V 2016 ✔ ✔
Sockets
PCI 2016 ✔ ✔ ✔
Passthrou
gh/DDA
Generati
on 2
virtual
machines
Secure 2016 ✔ ✔ ✔ ✔ ✔ ✔ ✔
boot
WINDOWS
SERVER
FEATURE VERSION 6.0-6.8 6.0-6.8 6.9, 6.8 6.6, 6.7 6.5 6.4
Windows 2016
Server 2016
Accurate
Time
WINDOWS
SERVER
FEATURE VERSION 6.0-6.8 6.0-6.8 6.9, 6.8 6.6, 6.7 6.5 6.4
Networkin
g
SR-IOV 2016
Storage
Live virtual 2016, 2012 ✔ Note 5 ✔ Note 5 ✔ Note 4, ✔ Note 4, ✔ Note 4, ✔ Note 4,
machine R2 5 5 5, 6 5, 6
backup
Memory
Runtime 2016
Memory
Resize
Video
Miscellane
ous
Hyper-V 2016 ✔ ✔
Sockets
PCI 2016
Passthroug
h/DDA
Generation
2 virtual
machines
WINDOWS
SERVER
FEATURE VERSION 6.0-6.8 6.0-6.8 6.9, 6.8 6.6, 6.7 6.5 6.4
WINDOWS SERVER
FEATURE VERSION 5.2 -5.11 5.2-5.11 5.9 - 5.11
Networking
VLAN tagging and 2016, 2012 R2, 2012, ✔ Note 1 ✔ Note 1 ✔ Note 1
trunking 2008 R2
SR-IOV 2016
Storage
Memory
Dynamic Memory - 2016, 2012 R2, 2012 ✔ Note 7, 9, 10, 11 ✔ Note 7, 9, 10, 11
Ballooning
Video
Miscellaneous
PCI 2016
Passthrough/DDA
Generation 2 virtual
machines
Notes
1. For this RHEL/CentOS release, VLAN tagging works but VLAN trunking does not.
2. Static IP injection may not work if Network Manager has been configured for a given synthetic network
adapter on the virtual machine. For smooth functioning of static IP injection please make sure that either
Network Manager is either turned off completely or has been turned off for a specific network adapter
through its ifcfg-ethX file.
3. On Windows Server 2012 R2 while using virtual fibre channel devices, make sure that logical unit number 0
(LUN 0) has been populated. If LUN 0 has not been populated, a Linux virtual machine might not be able to
mount fibre channel devices natively.
4. For built-in LIS, the "hyperv-daemons" package must be installed for this functionality.
5. If there are open file handles during a live virtual machine backup operation, then in some corner cases, the
backed-up VHDs might have to undergo a file system consistency check (fsck) on restore. Live backup
operations can fail silently if the virtual machine has an attached iSCSI device or direct-attached storage
(also known as a pass-through disk).
6. While the Linux Integration Services download is preferred, live backup support for RHEL/CentOS 5.9 -
5.11/6.4/6.5 is also available through Hyper-V Backup Essentials for Linux.
7. Dynamic memory support is only available on 64-bit virtual machines.
8. Hot-Add support is not enabled by default in this distribution. To enable Hot-Add support you need to add a
udev rule under /etc/udev/rules.d/ as follows:
a. Create a file /etc/udev/rules.d/100-balloon.rules. You may use any other desired name for the
file.
b. Add the following content to the file: SUBSYSTEM=="memory", ACTION=="add", ATTR{state}="online"
The Linux Integration Services download can be applied to existing Generation 2 VMs but does not impart
Generation 2 capability.
15. In Red Hat Enterprise Linux or CentOS 5.2, 5.3, and 5.4 the filesystem freeze functionality is not available, so
Live Virtual Machine Backup is also not available.
See Also
Set-VMFirmware
Supported Debian virtual machines on Hyper-V
Supported Oracle Linux virtual machines on Hyper-V
Supported SUSE virtual machines on Hyper-V
Supported Ubuntu virtual machines on Hyper-V
Supported FreeBSD virtual machines on Hyper-V
Feature Descriptions for Linux and FreeBSD virtual machines on Hyper-V
Best Practices for running Linux on Hyper-V
Red Hat Hardware Certification
Supported Debian virtual machines on Hyper-V
6/19/2017 • 3 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
The following feature distribution map indicates the features that are present in each version. The known issues
and workarounds for each distribution are listed after the table.
Table legend
Built in - LIS are included as part of this Linux distribution. The Microsoft-provided LIS download package
doesn't work for this distribution so do not install it. The kernel module version numbers for the built in LIS
(as shown by lsmod, for example) are different from the version number on the Microsoft-provided LIS
download package. A mismatch does not indicate that the built in LIS is out of date.
✔ - Feature available
WINDOWS SERVER
FEATURE OPERATING SYSTEM VERSION 8.0-8.8 (JESSIE) 7.0-7.11 (WHEEZY)
Networking
SR-IOV 2016
WINDOWS SERVER
FEATURE OPERATING SYSTEM VERSION 8.0-8.8 (JESSIE) 7.0-7.11 (WHEEZY)
Storage
Memory
Video
Miscellaneous
Generation 2 virtual
machines
WINDOWS SERVER
FEATURE OPERATING SYSTEM VERSION 8.0-8.8 (JESSIE) 7.0-7.11 (WHEEZY)
Notes
1. Creating file systems on VHDs larger than 2TB is not supported.
2. On Windows Server 2008 R2 SCSI disks create 8 different entries in /dev/sd*.
3. Windows Server 2012 R2 a VM with 8 cores or more will have all interrupts routed to a single vCPU.
4. Starting with Debian 8.3 the manually-installed Debian package "hyperv-daemons" contains the key-value
pair, fcopy, and VSS daemons. On Debian 7.x and 8.0-8.2 the hyperv-daemons package must come from
Debian backports.
5. Live virtual machine backup will not work with ext2 file systems. The default layout created by the Debian
installer includes ext2 filesystems, you you must customize the layout to not create this filesystem type.
6. While Debian 7.x is out of support and uses an older kernel, the kernel included in Debian backports for
Debian 7.x has improved Hyper-V capabilities.
7. OnWindows Server 2012 R2 Generation 2 virtual machines have secure boot enabled by default and some
Linux virtual machines will not boot unless the secure boot option is disabled. You can disable secure boot
in the Firmware section of the settings for the virtual machine in Hyper-V Manager or you can disable it
using Powershell:
See Also
Supported CentOS and Red Hat Enterprise Linux virtual machines on Hyper-V
Supported Oracle Linux virtual machines on Hyper-V
Supported SUSE virtual machines on Hyper-V
Supported Ubuntu virtual machines on Hyper-V
Supported FreeBSD virtual machines on Hyper-V
Feature Descriptions for Linux and FreeBSD virtual machines on Hyper-V
Best Practices for running Linux on Hyper-V
Supported Oracle Linux virtual machines on Hyper-V
10/24/2017 • 8 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
The following feature distribution map indicates the features that are present in each version. The known issues
and workarounds for each distribution are listed after the table.
In this section:
Red Hat Compatible Kernel Series
Unbreakable Enterprise Kernel Series
Notes
Table legend
Built in - LIS are included as part of this Linux distribution. The kernel module version numbers for the
built in LIS (as shown by lsmod, for example) are different from the version number on the Microsoft-
provided LIS download package. A mismatch doesn't indicate that the built in LIS is out of date.
✔ - Feature available
Availabil LIS 4.2 LIS 4.1 Built in Built in Built in Built in Built in
ity
Core 2016, ✔ ✔ ✔ ✔ ✔ ✔
2012 R2,
2012,
2008 R2
Windows 2016
Server
2016
Accurate
Time
WINDOWS 6.4-6.8 6.4-6.8
SERVER AND 7.0- AND 7.0- RHCK 7.0- RHCK 6.6,
FEATURE VERSION 7.3 7.2 7.2 RHCK 6.8 6.7 RHCK 6.5 RHCK6.4
Networki
ng
Jumbo 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
frames 2012 R2,
2012,
2008 R2
Live 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
Migration 2012 R2,
2012,
2008 R2
Static IP 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
Injection 2012 R2,
2012
vRSS 2016, ✔ ✔ ✔ ✔ ✔
2012 R2
TCP 2016, ✔ ✔ ✔ ✔ ✔
Segmenta 2012 R2,
tion and 2012,
Checksu 2008 R2
m
Offloads
SR-IOV 2016
Storage
VHDX 2016, ✔ ✔ ✔ ✔ ✔ ✔
resize 2012 R2
TRIM 2016, ✔ ✔ ✔ ✔
support 2012 R2
SCSI 2016, ✔ ✔
WWN 2012 R2
WINDOWS 6.4-6.8 6.4-6.8
SERVER AND 7.0- AND 7.0- RHCK 7.0- RHCK 6.6,
FEATURE VERSION 7.3 7.2 7.2 RHCK 6.8 6.7 RHCK 6.5 RHCK6.4
Memory
Configura 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
tion of 2012 R2
MMIO
gap
Runtime 2016
Memory
Resize
Video
Hyper-V- 2016,201 ✔ ✔ ✔ ✔ ✔ ✔
specific 2 R2,
video 2012,
device 2008 R2
Miscellan
eous
Non- 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
Maskable 2012 R2
Interrupt
lsvmbus 2016, ✔ ✔
command 2012 R2,
2012,
2008 R2
Hyper-V 2016 ✔ ✔
Sockets
PCI 2016
Passthrou
gh/DDA
Generati
on 2
virtual
machines
Secure 2016
boot
WINDOWS SERVER
FEATURE VERSION UEK R4 UEK R3 QU3 UEK R3 QU2 UEK R3 QU1
Networking
SR-IOV 2016
Storage
Memory
Video
Miscellaneous
WINDOWS SERVER
FEATURE VERSION UEK R4 UEK R3 QU3 UEK R3 QU2 UEK R3 QU1
Key-Value Pair 2016, 2012 R2, ✔ Note 11, 12 ✔ Note 11, 12 ✔ Note 11, 12 ✔ Note 11, 12
2012, 2008 R2
PCI 2016
Passthrough/DD
A
Generation 2
virtual
machines
Notes
1. For this Oracle Linux release, VLAN tagging works but VLAN trunking does not.
2. While using virtual fibre channel devices, ensure that logical unit number 0 (LUN 0) has been populated. If
LUN 0 has not been populated, a Linux virtual machine might not be able to mount fibre channel devices
natively.
3. If there are open file handles during a live virtual machine backup operation, then in some corner cases, the
backed-up VHDs might have to undergo a file system consistency check (fsck) on restore.
4. Live backup operations can fail silently if the virtual machine has an attached iSCSI device or direct-attached
storage (also known as a pass-through disk).
5. Live backup support for Oracle Linux 6.4/6.5/UEKR3 QU2 and QU3 is available through Hyper-V Backup
Essentials for Linux.
6. Dynamic memory support is only available on 64-bit virtual machines.
7. Hot-Add support is not enabled by default in this distribution. To enable Hot-Add support you need to add
a udev rule under /etc/udev/rules.d/ as follows:
a. Create a file /etc/udev/rules.d/100-balloon.rules. You may use any other desired name for the
file.
b. Add the following content to the file: SUBSYSTEM=="memory", ACTION=="add", ATTR{state}="online"
The Linux Integration Services download can be applied to existing Generation 2 VMs but does not impart
Generation 2 capability.
See Also
Set-VMFirmware
Supported CentOS and Red Hat Enterprise Linux virtual machines on Hyper-V
Supported Debian virtual machines on Hyper-V
Supported SUSE virtual machines on Hyper-V
Supported Ubuntu virtual machines on Hyper-V
Supported FreeBSD virtual machines on Hyper-V
Feature Descriptions for Linux and FreeBSD virtual machines on Hyper-V
Best Practices for running Linux on Hyper-V
Supported SUSE virtual machines on Hyper-V
10/24/2017 • 5 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
The following is a feature distribution map that indicates the features in each version. The known issues and
workarounds for each distribution are listed after the table.
The built-in SUSE Linux Enterprise Service drivers for Hyper-V are certified by SUSE. An example configuration can
be viewed in this bulletin: SUSE YES Certification Bulletin.
Table legend
Built in - LIS are included as part of this Linux distribution.The Microsoft-provided LIS download package
does not work for this distribution, so do not install it.The kernel module version numbers for the built in
LIS (as shown by lsmod, for example) are different from the version number on the Microsoft-provided LIS
download package. A mismatch doesn't indicate that the built in LIS is out of date.
✔ - Feature available
WINDOWS
SERVER
OPERATIN
G SYSTEM SLES 12 SLES 12 SLES 11 SLES 11 SLES 11 OPEN SUSE
FEATURE VERSION SP2 SP1 SLES 12 SP4 SP3 SP2 12.3
Core 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
2012 R2,
2012,
2008 R2
Windows 2016 ✔
Server
2016
Accurate
Time
Networki
ng
Jumbo 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
frames 2012 R2,
2012,
2008 R2
WINDOWS
SERVER
OPERATIN
G SYSTEM SLES 12 SLES 12 SLES 11 SLES 11 SLES 11 OPEN SUSE
FEATURE VERSION SP2 SP1 SLES 12 SP4 SP3 SP2 12.3
VLAN 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
tagging 2012 R2,
and 2012,
trunking 2008 R2
Live 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
migration 2012 R2,
2012,
2008 R2
vRSS 2016, ✔ ✔ ✔
2012 R2
TCP 2016, ✔ ✔ ✔ ✔
Segmenta 2012 R2,
tion and 2012,
Checksu 2008 R2
m
Offloads
SR-IOV 2016 ✔
Storage
VHDX 2016, ✔ ✔ ✔ ✔ ✔
resize 2012 R2
Virtual 2016, ✔ ✔ ✔ ✔ ✔
Fibre 2012 R2
Channel
TRIM 2016, ✔ ✔ ✔ ✔
support 2012 R2
SCSI 2016, ✔
WWN 2012 R2
Memory
Configura 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
tion of 2012 R2
MMIO
gap
Video
Hyper-V- 2016, ✔ ✔ ✔ ✔ ✔
specific 2012 R2,
video 2012,
device 2008 R2
Miscellan
eous
Non- 2016, ✔ ✔ ✔ ✔ ✔ ✔ ✔
Maskable 2012 R2
Interrupt
lsvmbus 2016, ✔
command 2012 R2,
2012,
2008 R2
Hyper-V 2016
Sockets
PCI 2016 ✔ ✔
Passthrou
gh/DDA
WINDOWS
SERVER
OPERATIN
G SYSTEM SLES 12 SLES 12 SLES 11 SLES 11 SLES 11 OPEN SUSE
FEATURE VERSION SP2 SP1 SLES 12 SP4 SP3 SP2 12.3
Generati
on 2
virtual
machines
Secure 2016 ✔ ✔ ✔
boot
Notes
1. Static IP injection may not work if Network Manager has been configured for a given Hyper-V-specific
network adapter on the virtual machine. To ensure smooth functioning of static IP injection please ensure
that Network Manager is turned off completely or has been turned off for a specific network adapter
through its ifcfg-ethX file.
2. If there are open file handles during a live virtual machine backup operation, then in some corner cases, the
backed-up VHDs might have to undergo a file system consistency check (fsck) on restore.
3. Live backup operations can fail silently if the virtual machine has an attached iSCSI device or direct-
attached storage (also known as a pass-through disk).
4. Dynamic memory operations can fail if the guest operating system is running too low on memory. The
following are some best practices:
Startup memory and minimal memory should be equal to or greater than the amount of memory
that the distribution vendor recommends.
Applications that tend to consume the entire available memory on a system are limited to
consuming up to 80 percent of available RAM.
5. Dynamic memory support is only available on 64-bit virtual machines.
6. If you are using Dynamic Memory on Windows Server 2016 or Windows Server 2012 operating systems,
specify Startup memory, Minimum memory, and Maximum memory parameters in multiples of 128
megabytes (MB). Failure to do so can lead to Hot-Add failures, and you may not see any memory increase
in a guest operating system.
7. In Windows Server 2016 or Windows Server 2012 R2, the key/value pair infrastructure might not function
correctly without a Linux software update. Contact your distribution vendor to obtain the software update
in case you see problems with this feature.
8. VSS backup will fail if a single partition is mounted multiple times.
9. On Windows Server 2012 R2, Generation 2 virtual machines have secure boot enabled by default and
Generation 2 Linux virtual machines will not boot unless the secure boot option is disabled. You can disable
secure boot in the Firmware section of the settings for the virtual machine in Hyper-V Manager or you can
disable it using Powershell:
Set-VMFirmware -VMName "VMname" -EnableSecureBoot Off
See Also
Set-VMFirmware
Supported CentOS and Red Hat Enterprise Linux virtual machines on Hyper-V
Supported Debian virtual machines on Hyper-V
Supported Oracle Linux virtual machines on Hyper-V
Supported Ubuntu virtual machines on Hyper-V
Supported FreeBSD virtual machines on Hyper-V
Feature Descriptions for Linux and FreeBSD virtual machines on Hyper-V
Best Practices for running Linux on Hyper-V
Supported Ubuntu virtual machines on Hyper-V
10/24/2017 • 7 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
Beginning with Ubuntu 12.04, loading the "linux-virtual" package installs a kernel suitable for use as a guest
virtual machine. This package always depends on the latest minimal generic kernel image and headers used for
virtual machines. While its use is optional, the linux-virtual kernel will load fewer drivers and may boot faster and
have less memory overhead than a generic image.
To get full use of Hyper-V, install the appropriate linux-tools and linux-cloud-tools packages to install tools and
daemons for use with virtual machines. When using the linux-virtual kernel, load linux-tools-virtual and linux-
cloud-tools-virtual.
The following feature distribution map indicates the features in each version. The known issues and workarounds
for each distribution are listed after the table.
Table legend
Built in - LIS are included as part of this Linux distribution. The Microsoft-provided LIS download package
doesn't work for this distribution, so don't install it. The kernel module version numbers for the built in LIS
(as shown by lsmod, for example) are different from the version number on the Microsoft-provided LIS
download package. A mismatch doesn't indicate that the built in LIS is out of date.
✔ - Feature available
WINDOWS
SERVER
OPERATING
SYSTEM
FEATURE VERSION 17.04 16.10 16.04 LTS 14.04 LTS 12.04 LTS
Windows 2016 ✔ ✔ ✔
Server 2016
Accurate Time
Networking
SR-IOV 2016 ✔ ✔ ✔
Storage
Memory
Runtime 2016 ✔ ✔ ✔ ✔
Memory
Resize
Video
Miscellaneou
s
Hyper-V 2016
Sockets
PCI 2016 ✔ ✔ ✔ ✔
Passthrough/
DDA
Generation 2
virtual
machines
Boot using 2016, 2012 ✔ Note 11, ✔ Note 11, ✔ Note 11, ✔ Note 11,
UEFI R2 12 12 12 12
# apt-get update
# apt-get install linux-virtual-lts-xenial
To install the virtual HWE kernel on 14.04, run the following commands as root (or sudo):
# apt-get update
# apt-get install linux-virtual-lts-xenial
12.04 does not have a separate virtual kernel. To install the generic HWE kernel on 12.04, run the following
commands as root (or sudo):
# apt-get update
# apt-get install linux-generic-lts-trusty
On Ubuntu 12.04, 14.04, and 16.04 the following Hyper-V daemons are in a separately installed package:
VSS Snapshot daemon - This daemon is required to create live Linux virtual machine backups.
KVP daemon - This daemon allows setting and querying intrinsic and extrinsic key value pairs.
fcopy daemon - This daemon implements a file copying service between the host and guest.
To install these Hyper-V daemons on 16.04, run the following commands as root (or sudo):
To install these Hyper-V daemons on 14.04, run the following commands as root (or sudo).
To install the KVP daemon on 12.04, run the following commands as root (or sudo).
# apt-get install hv-kvp-daemon-init linux-tools-lts-trusty linux-cloud-tools-generic-lts-trusty
Whenever the kernel is updated, the virtual machine must be rebooted to use it.
6. On Ubuntu 17.04 and 16.10, use the latest virtual kernel to have up-to-date Hyper-V capabilities.
To install the virtual kernel on 17.04 and 16.10, run the following commands as root (or sudo):
# apt-get update
# apt-get install linux-image-virtual
On Ubuntu 17.04 and 16.10 the following Hyper-V daemons are in a separately installed package:
VSS Snapshot daemon - This daemon is required to create live Linux virtual machine backups.
KVP daemon - This daemon allows setting and querying intrinsic and extrinsic key value pairs.
fcopy daemon - This daemon implements a file copying service between the host and guest.
To install these Hyper-V daemons on 17.04 and 16.10, run the following commands as root (or sudo):
Whenever the kernel is updated, the virtual machine must be rebooted to use it.
7. Dynamic memory support is only available on 64-bit virtual machines.
8. Dynamic Memory operations can fail if the guest operating system is running too low on memory. The
following are some best practices:
Startup memory and minimal memory should be equal to or greater than the amount of memory
that the distribution vendor recommends.
Applications that tend to consume the entire available memory on a system are limited to
consuming up to 80 percent of available RAM.
9. If you are using Dynamic Memory on Windows Server 2016 or Windows Server 2012 operating systems,
specify Startup memory, Minimum memory, and Maximum memory parameters in multiples of 128
megabytes (MB). Failure to do so can lead to Hot-Add failures, and you might not see any memory increase
on a guest operating system.
10. In Windows Server 2016 or Windows Server 2012 R2, the key/value pair infrastructure might not function
correctly without a Linux software update. Contact your distribution vendor to obtain the software update
in case you see problems with this feature.
11. On Windows Server 2012 R2, Generation 2 virtual machines have secure boot enabled by default and
some Linux virtual machines will not boot unless the secure boot option is disabled. You can disable secure
boot in the Firmware section of the settings for the virtual machine in Hyper-V Manager or you can
disable it using Powershell:
12. Before attempting to copy the VHD of an existing Generation 2 VHD virtual machine to create new
Generation 2 virtual machines, follow these steps:
a. Log in to the existing Generation 2 virtual machine.
b. Change directory to the boot EFI directory:
# cd /boot/efi/EFI
# cd boot
See Also
Supported CentOS and Red Hat Enterprise Linux virtual machines on Hyper-V
Supported Debian virtual machines on Hyper-V
Supported Oracle Linux virtual machines on Hyper-V
Supported SUSE virtual machines on Hyper-V
Feature Descriptions for Linux and FreeBSD virtual machines on Hyper-V
Best Practices for running Linux on Hyper-V
Set-VMFirmware
Ubuntu 14.04 in a Generation 2 VM - Ben Armstrong's Virtualization Blog
Not finding content you need? Windows 10 users, tell us what you want on Feedback Hub.
Supported FreeBSD virtual machines on Hyper-V
9/7/2017 • 3 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
The following feature distribution map indicates the features in each version. The known issues and workarounds
for each distribution are listed after the table.
Table legend
Built in - BIS (FreeBSD Integration Service) are included as part of this FreeBSD release.
✔ - Feature available
WINDOWS
SERVER
OPERATING
SYSTEM
FEATURE VERSION 11.1 11.0 10.3 10.2 10.0 - 10.1 9.1 - 9.3, 8.4
Windows 2016 ✔
Server
2016
Accurate
Time
Networkin
g
SR-IOV 2016
Memory
Runtime 2016
Memory
Resize
Video
Miscellane
ous
Hyper-V 2016
Sockets
PCI 2016 ✔
Passthroug
h/DDA
Generatio
n 2 virtual
machines
Notes
1. Suggest to Label Disk Devices to avoid ROOT MOUNT ERROR during startup.
2. The virtual DVD drive may not be recognized when BIS drivers are loaded on FreeBSD 8.x and 9.x unless
you enable the legacy ATA driver through the following command.
Additional Notes: The feature matrix of 10 stable and 11 stable is same with FreeBSD 11.1 release. In
addition, FreeBSD 10.2 and previous versions (10.1, 10.0, 9.x, 8.x) are end of life. Please refer here for an
up-to-date list of supported releases and the latest security advisories.
See Also
Feature Descriptions for Linux and FreeBSD virtual machines on Hyper-V
Best practices for running FreeBSD on Hyper-V
Feature Descriptions for Linux and FreeBSD virtual
machines on Hyper-V
6/19/2017 • 8 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
This article describes features available in components such as core, networking, storage, and memory when
using Linux and FreeBSD on a virtual machine.
Core
FEATURE DESCRIPTION
Integrated shutdown With this feature, an administrator can shut down virtual
machines from the Hyper-V Manager. For more information,
see Operating system shutdown.
Time synchronization This feature ensures that the maintained time inside a virtual
machine is kept synchronized with the maintained time on
the host. For more information, see Time synchronization.
Windows Server 2016 Accurate Time This feature allows the guest to use the Windows Server 2016
Accurate Time capability, which improves time
synchronization with the host to 1ms accuracy. For more
information, see Windows Server 2016 Accurate Time
Multiprocessing support With this feature, a virtual machine can use multiple
processors on the host by configuring multiple virtual CPUs.
Heartbeat With this feature, the host to can track the state of the virtual
machine. For more information, see Heartbeat.
Integrated mouse support With this feature, you can use a mouse on a virtual machine's
desktop and also use the mouse seamlessly between the
Windows Server desktop and the Hyper-V console for the
virtual machine.
Hyper-V specific Storage device This feature grants high-performance access to storage
devices that are attached to a virtual machine.
Hyper-V specific Network device This feature grants high-performance access to network
adapters that are attached to a virtual machine.
Networking
FEATURE DESCRIPTION
Jumbo frames With this feature, an administrator can increase the size of
network frames beyond 1500 bytes, which leads to a
significant increase in network performance.
VLAN tagging and trunking This feature allows you to configure virtual LAN (VLAN) traffic
for virtual machines.
Live Migration With this feature, you can migrate a virtual machine from one
host to another host. For more information, see Virtual
Machine Live Migration Overview.
Static IP Injection With this feature, you can replicate the static IP address of a
virtual machine after it has been failed over to its replica on a
different host. Such IP replication ensures that network
workloads continue to work seamlessly after a failover event.
vRSS (Virtual Receive Side Scaling) Spreads the load from a virtual network adapter across
multiple virtual processors in a virtual machine.For more
information, see Virtual Receive-side Scaling in Windows
Server 2012 R2.
TCP Segmentation and Checksum Offloads Transfers segmentation and checksum work from the guest
CPU to the host virtual switch or network adapter during
network data transfers.
SR-IOV Single Root I/O devices use DDA to allow guests access to
portions of specific NIC cards allowing for reduced latency
and increased throughput. SR-IOV requires up to date
physical function (PF) drivers on the host and virtual function
(VF) drivers on the guest.
Storage
FEATURE DESCRIPTION
Virtual Fibre Channel With this feature, virtual machines can recognize a fiber
channel device and mount it natively. For more information,
see Hyper-V Virtual Fibre Channel Overview.
FEATURE DESCRIPTION
Live virtual machine backup This feature facilitates zero down time backup of live virtual
machines.
TRIM support TRIM hints notify the drive that certain sectors that were
previously allocated are no longer required by the app and
can be purged. This process is typically used when an app
makes large space allocations via a file and then self-manages
the allocations to the file, for example, to virtual hard disk
files.
SCSI WWN The storvsc driver extracts World Wide Name (WWN)
information from the port and node of devices attached to
the virtual machine and creates the appropriate sysfs files.
Memory
FEATURE DESCRIPTION
PAE Kernel Support Physical Address Extension (PAE) technology allows a 32-bit
kernel to access a physical address space that is larger than
4GB. Older Linux distributions such as RHEL 5.x used to ship
a separate kernel that was PAE enabled. Newer distributions
such as RHEL 6.x have pre-built PAE support.
Configuration of MMIO gap With this feature, appliance manufacturers can configure the
location of the Memory Mapped I/O (MMIO) gap. The MMIO
gap is typically used to divide the available physical memory
between an appliance's Just Enough Operating Systems
(JeOS) and the actual software infrastructure that powers the
appliance.
FEATURE DESCRIPTION
Dynamic Memory - Hot-Add The host can dynamically increase or decrease the amount of
memory available to a virtual machine while it is in operation.
Before provisioning, the Administrator enables Dynamic
Memory in the Virtual Machine Settings panel and specify the
Startup Memory, Minimum Memory, and Maximum Memory
for the virtual machine. When the virtual machine is in
operation Dynamic Memory cannot be disabled and only the
Minimum and Maximum settings can be changed. (It is a best
practice to specify these memory sizes as multiples of
128MB.)
Dynamic Memory - Ballooning The host can dynamically increase or decrease the amount of
memory available to a virtual machine while it is in operation.
Before provisioning, the Administrator enables Dynamic
Memory in the Virtual Machine Settings panel and specify the
Startup Memory, Minimum Memory, and Maximum Memory
for the virtual machine. When the virtual machine is in
operation Dynamic Memory cannot be disabled and only the
Minimum and Maximum settings can be changed. (It is a best
practice to specify these memory sizes as multiples of
128MB.)
Runtime Memory Resize An administrator can set the amount of memory available to
a virtual machine while it is in operation, either increasing
memory ("Hot Add") or decreasing it ("Hot Remove").
Memory is returned to Hyper-V via the balloon driver (see
"Dynamic Memory - Ballooning"). The balloon driver
maintains a minimum amount of free memory after
ballooning, called the "floor", so assigned memory cannot be
reduced below the current demand plus this floor amount.
The Memory tab of Hyper-V manager will display the amount
of memory assigned to the virtual machine, but memory
statistics within the virtual machine will show the highest
amount of memory allocated. (It is a best practice to specify
memory values as multiples of 128MB.)
Video
FEATURE DESCRIPTION
Hyper-V-specific video device This feature provides high-performance graphics and superior
resolution for virtual machines. This device does not provide
Enhanced Session Mode or RemoteFX capabilities.
Miscellaneous
FEATURE DESCRIPTION
KVP (Key-Value pair) exchange This feature provides a key/value pair (KVP) exchange service
for virtual machines. Typically administrators use the KVP
mechanism to perform read and write custom data
operations on a virtual machine. For more information, see
Data Exchange: Using key-value pairs to share information
between the host and guest on Hyper-V.
File copy from host to guest This feature allows files to be copied from the host physical
computer to the guest virtual machines without using the
network adaptor. For more information, see Guest services.
lsvmbus command This command gets information about devices on the Hyper-
V virtual machine bus (VMBus) similiar to information
commands like lspci.
PCI Passthrough/DDA With Windows Server 2016 administrators can pass through
PCI Express devices via the Discrete Device Assignment
mechanism. Common devices are network cards, graphics
cards, and special storage devices. The virtual machine will
require the appropriate driver to use the exposed hardware.
The hardware must be assigned to the virtual machine for it
to be used.
Boot using UEFI This feature allows virtual machines to boot using Unified
Extensible Firmware Interface (UEFI).
Secure boot This feature allows virtual machines to use UEFI based secure
boot mode. When a virtual machine is booted in secure
mode, various operating system components are verified
using signatures present in the UEFI data store.
See Also
Supported CentOS and Red Hat Enterprise Linux virtual machines on Hyper-V
Supported Debian virtual machines on Hyper-V
Supported Oracle Linux virtual machines on Hyper-V
Supported SUSE virtual machines on Hyper-V
Supported Ubuntu virtual machines on Hyper-V
Supported FreeBSD virtual machines on Hyper-V
Best Practices for running Linux on Hyper-V
Best practices for running FreeBSD on Hyper-V
Best Practices for running Linux on Hyper-V
6/19/2017 • 4 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
This topic contains a list of recommendations for running Linux virtual machine on Hyper-V.
The ext4 format is preferred to ext3 because ext4 is more space efficient than ext3 when used with dynamic
VHDX files.
When creating the filesystem specify the number of groups to be 4096, for example:
Specifying a kickstart file to the pre-install kernel would also avoid the need for keyboard and mouse input during
installation.
Add "numa=off" if the Linux virtual machine has more than 7 virtual
processors or more than 30 GB RAM
Linux virtual machines configured to use more than 7 virtual processors should add numa=off to the GRUB
boot.cfg to work around a known issue in the 2.6.x Linux kernels. Linux virtual machines configured to use more
than 30 GB RAM should also add numa=off to the GRUB boot.cfg.
See also
Supported Linux and FreeBSD virtual machines for Hyper-V on Windows
Best practices for running FreeBSD on Hyper-V
Deploy a Hyper-V Cluster
Best practices for running FreeBSD on Hyper-V
6/19/2017 • 2 min to read • Edit Online
Applies To: Windows Server 2016, Hyper-V Server 2016, Windows Server 2012 R2, Hyper-V Server 2012 R2,
Windows Server 2012, Hyper-V Server 2012, Windows Server 2008 R2, Windows 10, Windows 8.1, Windows
8, Windows 7.1, Windows 7
This topic contains a list of recommendations for running FreeBSD as a guest operating system on a Hyper-V
virtual machine.
1. Reboot the system into single user mode. This can be accomplished by selecting boot menu option 2 for
FreeBSD 10.3+ (option 4 for FreeBSD 8.x), or performing a 'boot -s' from the boot prompt.
2. In Single user mode, create GEOM labels for each of the IDE disk partitions listed in your fstab (both root
and swap). Below is an example of FreeBSD 10.3.
# cat /etc/fstab
# Device Mountpoint FStype Options Dump Pass#
/dev/da0p2 / ufs rw 1 1
/dev/da0p3 none swap sw 0 0
Additional information on GEOM labels can be found at: Labeling Disk Devices.
3. The system will continue with multi-user boot. After the boot completes, edit /etc/fstab and replace the
conventional device names, with their respective labels. The final /etc/fstab will look like this:
4. The system can now be rebooted. If everything went well, it will come up normally and mount will show:
# mount
/dev/label/rootfs on / (ufs, local, journaled soft-updates)
devfs on /dev (devfs, local, mutilabel)
# sysctl net.link.ether.inet.max_age=60
See also
Supported FreeBSD virtual machines on Hyper-V
Hyper-V feature compatibility by generation and
guest
10/17/2017 • 2 min to read • Edit Online
The tables in this article show you the generations and operating systems that are compatible with some of the
Hyper-V features, grouped by categories. In general, you'll get the best availability of features with a generation 2
virtual machine that runs the newest operating system.
Keep in mind that some features rely on hardware or other infrastructure. For hardware details, see System
requirements for Hyper-V on Windows Server 2016. In some cases, a feature can be used with any supported
guest operating system. For details on which operating systems are supported, see:
Supported Linux and FreeBSD virtual machines
Supported Windows guest operating systems
Compute
FEATURE GENERATION GUEST OPERATING SYSTEM
Mobility
FEATURE GENERATION GUEST OPERATING SYSTEM
Networking
FEATURE GENERATION GUEST OPERATING SYSTEM
Single root input/output virtualization 1 and 2 64-bit Windows guests, starting with
(SR-IOV) Windows Server 2012 and Windows 8.
Discrete device assignment (DDA) 1 and 2 Windows Server 2016, Windows Server
2012 R2 only with Update 3133690
installed, Windows 10
Note: For details on Update 3133690,
see this support article.
Storage
FEATURE GENERATION GUEST OPERATING SYSTEM
Shared virtual hard disks (VHDX only) 1 and 2 Windows Server 2016, Windows Server
2012 R2, Windows Server 2012
Use the following resources to set up and try out Hyper-V on the Nano Server, Server Core or GUI installation
option of Windows Server 2016. But before you install anything, check the System Requirements for Windows
Server and the System Requirements for Hyper-V.
Download and install Windows Server
To use Nano Server as a virtual machine host:
Deploy Nano Server
Use Hyper-V on Nano Server
To use the Server Core or GUI installation option of Windows Server 2016 as a virtual machine host:
Install the Hyper-V role on Windows Server 2016
Create a virtual switch for Hyper-V virtual machines
Create a virtual machine in Hyper-V
Install the Hyper-V role on Windows Server 2016
10/24/2017 • 3 min to read • Edit Online
To create and run virtual machines, install the Hyper-V role on Windows Server 2016 by using Server Manager or
the Install-WindowsFeature cmdlet in Windows PowerShell. To install the Hyper-V role on a Nano Server, see
Getting Started with Nano Server. For Windows 10, see Install Hyper-V on Windows 10.
To learn more about Hyper-V, see the Hyper-V Technology Overview. To try out Windows Server 2016, you can
download and install an evaluation copy. See the Evaluation Center.
Before you install Windows Server 2016 or add the Hyper-V role, make sure that:
Your computer hardware is compatible. For details, see System Requirements for Windows Server and System
requirements for Hyper-V on Windows Server 2016.
You don't plan to use third-party virtualization apps that rely on the same processor features that Hyper-V
requires. Examples include VMWare Workstation and VirtualBox. You can install Hyper-V without uninstalling
these other apps. But, if you try to use them to manage virtual machines when the Hyper-V hypervisor is
running, the virtual machines might not start or might run unreliably. For details and instructions for turning
off the Hyper-V hypervisor if you need to use one of these apps, see Virtualization applications do not work
together with Hyper-V, Device Guard, and Credential Guard.
If you want to install only the management tools, such as Hyper-V Manager, see Remotely manage Hyper-V hosts
with Hyper-V Manager.
If you're connected locally to the server, run the command without -ComputerName <computer_name> .
4. After the server restarts, you can see that the Hyper-V role is installed and see what other roles and features
are installed by running the following command:
If you're connected locally to the server, run the command without -ComputerName <computer_name> .
NOTE
If you install this role on a server that runs the Server Core installation option of Windows Server 2016 and use the
parameter -IncludeManagementTools , only the Hyper-V Module for Windows PowerShell will be installed. You can use the
GUI management tool, Hyper-V Manager, on another computer to remotely manage a Hyper-V host that runs on a Server
Core installation. For instructions on connecting remotely, see Remotely manage Hyper-V hosts with Hyper-V Manager.
See also
Install-WindowsFeature
Create a virtual switch for Hyper-V virtual machines
10/24/2017 • 3 min to read • Edit Online
A virtual switch allows virtual machines created on Hyper-V hosts to communicate with other computers. You can
create a virtual switch when you first install the Hyper-V role on Windows Server 2016. To create additional virtual
switches, use Hyper-V Manager or Windows PowerShell. To learn more about virtual switches, see Hyper-V Virtual
Switch.
Virtual machine networking can be a complex subject. And there are several new virtual switch features that you
may want to use like Switch Embedded Teaming (SET). But basic networking is fairly easy to do. This topic covers
just enough so that you can create networked virtual machines in Hyper-V. To learn more about how you can set
up your networking infrastructure, review the Networking documentation.
Allow management operating system to share this Select this option if you want to allow the Hyper-V host
network adapter to share the use of the virtual switch and NIC or NIC team
with the virtual machine. With this enabled, the host can
use any of the settings that you configure for the virtual
switch like Quality of Service (QoS) settings, security
settings, or other features of the Hyper-V virtual switch.
Enable single-root I/O virtualization (SR-IOV) Select this option only if you want to allow virtual machine
traffic to bypass the virtual machine switch and go directly
to the physical NIC. For more information, see Single-Root
I/O Virtualization in the Poster Companion Reference:
Hyper-V Networking.
7. If you want to isolates network traffic from the management Hyper-V host operating system or other
virtual machines that share the same virtual switch, select Enable virtual LAN identification for
management operating system. You can change the VLAN ID to any number or leave the default. This is
the virtual LAN identification number that the management operating system will use for all network
communication through this virtual switch.
8. Click OK.
9. Click Yes.
Get-NetAdapter
4. Create a virtual switch by using the New-VMSwitch cmdlet. For example, to create an external virtual switch
named ExternalSwitch, using the ethernet network adapter, and with Allow management operating
system to share this network adapter turned on, run the following command.
For more advanced Windows PowerShell scripts that cover improved or new virtual switch features in Windows
Server 2016, see Remote Direct Memory Access and Switch Embedded Teaming.
Next step
Create a virtual machine in Hyper-V
Create a virtual machine in Hyper-V
10/24/2017 • 4 min to read • Edit Online
Learn how to create a virtual machine by using Hyper-V Manager and Windows PowerShell and what options you
have when you create a virtual machine in Hyper-V Manager.
4. Use the New-VM cmdlet to create the virtual machine. See the following examples.
NOTE
If you may move this virtual machine to a Hyper-V host that runs Windows Server 2012 R2, use the -Version
parameter with New-VM to set the virtual machine configuration version to 5. The default virtual machine
configuration version for Windows Server 2016 isn't supported by Windows Server 2012 R2 or earlier versions. You
can't change the virtual machine configuration version after the virtual machine is created. For more information, see
Supported virtual machine configuration versions.
Existing virtual hard disk - To create a virtual machine with an existing virtual hard disk, you can
use the following command where,
-Name is the name that you provide for the virtual machine that you're creating.
-MemoryStartupBytes is the amount of memory that is available to the virtual machine at start
up.
-BootDevice is the device that the virtual machine boots to when it starts like the network
adapter (NetworkAdapter) or virtual hard disk (VHD).
-VHDPath is the path to the virtual machine disk that you want to use.
-Path is the path to store the virtual machine configuration files.
-Generation is the virtual machine generation. Use generation 1 for VHD and generation 2 for
VHDX. See Should I create a generation 1 or 2 virtual machine in Hyper-V?.
-Switch is the name of the virtual switch that you want the virtual machine to use to connect
to other virtual machines or the network. See Create a virtual switch for Hyper-V virtual
machines.
For example:
This creates a generation 2 virtual machine named Win10VM with 4GB of memory. It boots
from the folder VMs\Win10.vhdx in the current directory and uses the virtual switch named
ExternalSwitch. The virtual machine configuration files are stored in the folder VMData.
New virtual hard disk - To create a virtual machine with a new virtual hard disk, replace the -
VHDPath parameter from the example above with -NewVHDPath and add the -
NewVHDSizeBytes parameter. For example,
New-VM -Name Win10VM -MemoryStartupBytes 4GB -BootDevice VHD -NewVHDPath .\VMs\Win10.vhdx -Path
.\VMData -NewVHDSizeBytes 20GB -Generation 2 -Switch ExternalSwitch
New virtual hard disk that boots to operating system image - To create a virtual machine with a
new virtual disk that boots to an operating system image, see the PowerShell example in Create
virtual machine walkthrough for Hyper-V on Windows 10.
5. Start the virtual machine by using the Start-VM cmdlet. Run the following cmdlet where Name is the name
of the virtual machine you created.
For example:
VMConnect.exe
Specify Name and Location Name: New Virtual Machine. You can also enter your own name and
choose another location for the virtual
Location: machine.
C:\ProgramData\Microsoft\Window
s\Hyper-V\. This is where the virtual machine
configuration files will be stored.
Assign Memory Startup memory: 1024 MB You can set the startup memory from
32MB to 5902MB.
Dynamic memory: not selected
You can also choose to use Dynamic
Memory. For more information, see
Hyper-V Dynamic Memory Overview.
Configure Networking Not connected You can select a network connection for
the virtual machine to use from a list of
existing virtual switches. See Create a
virtual switch for Hyper-V virtual
machines.
Connect Virtual Hard Disk Create a virtual hard disk You can also choose to use an existing
virtual hard disk or wait and attach a
Name: <vmname>.vhdx virtual hard disk later.
Location:
C:\Users\Public\Documents\Hyper-
V\Virtual Hard Disks\
Size: 127GB
Installation Options Install an operating system later These options change the boot order of
the virtual machine so that you can
install from an .iso file, bootable floppy
disk or a network installation service,
like Windows Deployment Services
(WDS).
Summary Displays the options that you have Tip: You can copy the summary from
chosen, so that you can verify they are the page and paste it into e-mail or
correct. somewhere else to help you keep track
of your virtual machines.
- Name
- Generation
- Memory
- Network
- Hard Disk
- Operating System
See also
New-VM
Supported virtual machine configuration versions
Should I create a generation 1 or 2 virtual machine in Hyper-V?
Create a virtual switch for Hyper-V virtual machines
Plan for Hyper-V on Windows Server 2016
8/24/2017 • 1 min to read • Edit Online
Applies To: Microsoft Hyper-V Server 2016, Windows 10, Windows Server 2016
Your choice to create a generation 1 or generation 2 virtual machine depends on which guest operating system
you want to install and the boot method you want to use to deploy the virtual machine. We recommend that you
create a generation 2 virtual machine to take advantage of features like Secure Boot unless one of the following
statements is true:
The VHD you want to boot from is not UEFI-compatible.
You plan to move your virtual machine to Azure.
Generation 2 doesn't support the operating system you want to run on the virtual machine.
Generation 2 doesn't support the boot method you want to use.
For more information about what features are available with generation 2 virtual machines, see Hyper-V feature
compatibility by generation and guest.
You can't change a virtual machine's generation after you've created it. So, we recommend that you review the
considerations here, as well as choose the operating system, boot method, and features you want to use before
you choose a generation.
Windows 10 ✔ ✔
Windows 8.1 ✔ ✔
Windows 8 ✔ ✔
Windows 7 ✔ ✖
The following table shows which 32-bit versions of Windows you can use as a guest operating system for
generation 1 and generation 2 virtual machines.
Windows 10 ✔ ✖
Windows 8.1 ✔ ✖
Windows 8 ✔ ✖
Windows 7 ✔ ✖
CentOS and Red Hat Enterprise Linux guest operating system support
The following table shows which versions of Red Hat Enterprise Linux (RHEL) and CentOS you can use as a guest
operating system for generation 1 and generation 2 virtual machines.
For more information, see CentOS and Red Hat Enterprise Linux virtual machines on Hyper-V.
Debian guest operating system support
The following table shows which versions of Debian you can use as a guest operating system for generation 1 and
generation 2 virtual machines.
OPERATING SYSTEM VERSIONS GENERATION 1 GENERATION 2
FreeBSD 8.4 ✔ ✖
The following table shows which versions of Unbreakable Enterprise Kernel you can use as a guest operating
system for generation 1 and generation 2 virtual machines.
Ubuntu 12.04 ✔ ✖
IDE controller Virtual SCSI controller Boot from .vhdx (64 TB maximum size,
and online resize capability)
IDE CD-ROM Virtual SCSI CD-ROM Support for up to 64 SCSI DVD devices
per SCSI controller.
Legacy network adapter Synthetic network adapter Network boot with IPv4 and IPv6
Universal asynchronous Optional UART for debugging Faster and more reliable
receiver/transmitter (UART) for COM
ports
i8042 keyboard controller Software-based input Uses fewer resources because there is
no emulation. Also reduces the attack
surface from the guest operating
system.
2. Add a COM port. Use the Set-VMComPort cmdlet to do this. For example, the following command
configures the first COM port on virtual machine, TestVM, to connect to the named pipe, TestPipe, on the
local computer:
NOTE
Configured COM ports aren't listed in the settings of a virtual machine in Hyper-V Manager.
See Also
Linux and FreeBSD Virtual Machines on Hyper-V
Use local resources on Hyper-V virtual machine with VMConnect
Plan for Hyper-V scalability in Windows Server 2016
Plan for Hyper-V networking in Windows Server 2016
8/24/2017 • 4 min to read • Edit Online
A basic understanding of networking in Hyper-V helps you plan networking for virtual machines. This article also
covers some networking considerations when using live migration and when using Hyper-V with other server
features and roles.
This article gives you details about the maximum configuration for components you can add and remove on a
Hyper-V host or its virtual machines, such as virtual processors or checkpoints. As you plan your deployment,
consider the maximums that apply to each virtual machine, as well as those that apply to the Hyper-V host.
Maximums for memory and logical processors are the biggest increases from Windows Server 2012, in response
to requests to support newer scenarios such as machine learning and data analytics. The Windows Server blog
recently published the performance results of a virtual machine with 5.5 terabytes of memory and 128 virtual
processors running 4 TB in-memory database. Performance was greater than 95% of the performance of a
physical server. For details, see Windows Server 2016 Hyper-V large-scale VM performance for in-memory
transaction processing. Other numbers are similar to those that apply to Windows Server 2012. (Maximums for
Windows Server 2012 R2 were the same as Windows Server 2012.)
NOTE
For information about System Center Virtual Machine Manager (VMM), see Virtual Machine Manager. VMM is a Microsoft
product for managing a virtualized data center that is sold separately.
Size of physical disks attached directly Varies Maximum size is determined by the
to a virtual machine guest operating system.
Virtual hard disk capacity 64 TB for VHDX format; Each virtual hard disk is stored on
2040 GB for VHD format physical media as either a .vhdx or a
.vhd file, depending on the format used
by the virtual hard disk.
- Hardware-assisted virtualization
- Hardware-enforced Data Execution
Prevention (DEP)
Memory 24 TB None.
Network adapter teams (NIC Teaming) No limits imposed by Hyper-V. For details, see NIC Teaming.
Virtual network switch ports per server Varies; no limits imposed by Hyper-V. The practical limit depends on the
available computing resources.
Virtual switches Varies; no limits imposed by Hyper-V. The practical limit depends on the
available computing resources.
Running virtual machines per cluster 8,000 per cluster Several factors can affect the real
and per node number of virtual machines you can run
at the same time on one node, such as:
- Amount of physical memory being
used by each virtual machine.
- Networking and storage bandwidth.
- Number of disk spindles, which affects
disk I/O performance.
Plan for Hyper-V security in Windows Server 2016
8/24/2017 • 3 min to read • Edit Online
Secure the Hyper-V host operating system, the virtual machines, configuration files, and virtual machine data. Use
the following list of recommended practices as a checklist to help you secure your Hyper-V environment.
Discrete Device Assignment allows physical PCIe hardware to be directly accessible from within a virtual machine.
This guide will discuss the type of devices that can use Discrete Device Assignment, host system requirements,
limitations imposed on the virtual machines as well as security implications of Discrete Device Assignment.
For Discrete Device Assignment's initial release, we have focused on two device classes to be formally supported
by Microsoft. These are Graphics Adapters and NVMe Storage devices. Other devices are likely to work and
hardware vendors are able to offer statements of support for those devices. For these other devices, please reach
out to those hardware vendors for support.
If you are ready to try out Discrete Device Assignment, you can jump over to Deploying Graphics Devices Using
Discrete Device Assignment or Deploying Storage Devices using Discrete Device Assignment to get started!
System Requirements
In addition to the System Requirements for Windows Server and the System Requirements for Hyper-V, Discrete
Device Assignment requires server class hardware that is capable of granting the operating system control over
configuring the PCIe fabric (Native PCI Express Control). In addition, the PCIe Root Complex has to support "Access
Control Services" or ACS which enables Hyper-V to force all PCIe traffic through the I/O MMU.
These capabilities usually aren't exposed directly in the BIOS of the server and are often hidden behind other
settings. For example, the same capabilities are required for SR-IOV support and in the BIOS you may need to set
"Enable SR-IOV." Please reach out to your system vendor if you are unable to identify the correct setting in your
BIOS.
To help ensure hardware the hardware is capable of Discrete Device Assignment, our engineers have put together
a Machine Profile Script that you can run on an Hyper-V enabled host to test if your server is correctly setup and
what devices are capable of Discrete Device Assignment.
Device Requirements
Not every PCIe device can be used with Discrete Device Assignment. For example, older devices that leverage
legacy (INTx) PCI Interrupts are not supported. Jake Oshin's blog posts go into more detail however, for the
consumer, running the Machine Profile Script will tell you which devices are capable of being used for Discrete
Device Assignment.
Device manufactures can reach out to their Microsoft representative for more details.
Device Driver
As Discrete Device Assignment passes the entire PCIe device into the Guest VM, a host driver is not required to be
installed prior to the device being mounted within the VM. The only requirement on the host is that the device's
PCIe Location Path can be determined. The device's driver can optionally be installed if this helps in identifying the
device. An example of this is, a GPU that doesn't has it's device driver installed shows up as a Microsoft Basic
Render Device. If the device driver is installed, it's manufacture and model will likely be displayed.
Once the device is mounted inside the guest, the Manufacturer's device driver can now be installed like normal
inside the guest virtual machine.
Security
Discrete Device Assignment passes the entire device into the VM. This means all capabilities of that device are
accessible from the guest operating system. Some capabilities, like firmware updating, may adversely impact the
stability of the system. As such, numerous warnings are presented to the admin when dismounting the device
from the host. We highly recommend that Discrete Device Assignment is only used where the tenants of the VMs
are trusted.
If the admin desires to use a device with an untrusted tenant, we have provided device manufactures with the
ability to create a Device Mitigation driver that can be installed on the host. Please contact the device manufacturer
for details on whether they provide a Device Mitigation Driver.
If you would like to bypass the security checks for a device that does not have a Device Mitigation Driver you will
have to pass in --force to the Dismount-VMHostAssignableDevice call. Understand that by doing so, you have
changed the security profile of that system and this is only recommended during prototyping or trusted
environments.
MMIO Space
Some devices, especially GPUs, require additional MMIO space to be allocated to the VM for the memory of that
device to be accessible. By default, each VM starts off with 128MB of low MMIO space and 512MB of high MMIO
space allocated to it, but a device might require more, or you may pass through multiple devices that the
combined requirements exceed these values. Changing MMIO Space is straight forward, you do it in PowerShell
using the following commands.
The easiest way to determine how much MMIO space to allocate is to use the Machine Profile Script. Alternatively,
you can calculate it based using device manager and the blog post, Discrete Device Assignment - GPUs, has more
details.
Can Discrete Device Assignment be used to pass a USB device into a VM?
Although not officially supported, our customers have used Discrete Device Assignment to do this by passing the
entire USB3 controller into a VM. As the whole controller is being passed in, each USB device plugged into that
controller will also be accessible in the VM. Note, only some USB3 controllers may work, and USB2 controllers
definitely will not work.
Deploy Hyper-V on Windows Server 2016
6/19/2017 • 1 min to read • Edit Online
Use these resources to help you deploy Hyper-V on Windows Server 2016.
Configure virtual local area networks for Hyper-V
Set up hosts for live migration without Failover Clustering
Upgrade virtual machine version in Hyper-V on Windows 10 or Windows Server 2016
Deploy graphics devices using Discrete Device Assignment
Deploy storage devices using Discrete Device Assignment
10/2/2017 • 3 min to read • Edit Online
Restore
To import the virtual machine specifying your own path for the virtual machine files, run a command like this,
replacing the examples with your values:
Import as a copy
To complete a copy import and move the virtual machine files to the default Hyper-V location, run a command like
this, replacing the examples with your values:
This article shows you how to set up hosts that aren't clustered so you can do live migrations between them. Use
these instructions if you didn't set up live migration when you installed Hyper-V, or if you want to change the
settings. To set up clustered hosts, use tools for Failover Clustering.
PS C:\> Enable-VMMigration
Set-VMHost also lets you choose a performance option (and many other host settings). For example, to choose SMB
but leave the authentication protocol set to the default of CredSSP, type:
Next steps
After you set up the hosts, you're ready to do a live migration. For instructions, see Use live migration without
Failover Clustering to move a virtual machine.
Upgrade virtual machine version in Hyper-V on
Windows 10 or Windows Server 2016
6/19/2017 • 4 min to read • Edit Online
Make the latest Hyper-V features available on your virtual machines by upgrading the configuration version.
Don't do this until:
You upgrade your Hyper-V hosts to the latest version of Windows or Windows Server.
You upgrade the cluster functional level.
You're sure that you won't need to move the virtual machine back to a Hyper-V host that runs a previous
version of Windows or Windows Server.
For more information, see Cluster Operating System Rolling Upgrade and Upgrading Windows Server 2012 R2
clusters to Windows Server 2016 in VMM.
Update-VMVersion <vmname>
Run the PowerShell cmdlet Get-VMHostSupportedVersion to see what virtual machine configuration versions
your Hyper-V Host supports. When you create a virtual machine, it's created with the default configuration
version. To see what the default is, run the following command.
Get-VMHostSupportedVersion -Default
If you need to create a virtual machine that you can move to a Hyper-V Host that runs an older version of
Windows, use the New-VM cmdlet with the -version parameter. For example, to create a virtual machine that you
can move to a Hyper-V host that runs Windows Server 2012 R2 , run the following command. This command
will create a virtual machine named "WindowsCV5" with a configuration version 5.0.
Virtual hard disk Stores virtual hard disks for the virtual machine.
File name extension: .vhd or .vhdx
Default location: C:\ProgramData\Microsoft\Windows\Hyper-
V\Virtual Hard Disks
Automatic virtual hard disk Differencing disk files used for virtual machine checkpoints.
File name extension: .avhdx
Default location: C:\ProgramData\Microsoft\Windows\Hyper-
V\Virtual Hard Disks
For more information about these features, see What's new in Hyper-V on Windows Server 2016.
Deploy graphics devices using Discrete Device
Assignment
6/19/2017 • 4 min to read • Edit Online
If no Partitioning driver is provided, during dismount you must use the -force option to bypass the security
warning. Please read more about the security implications of doing this on Plan for Deploying Devices using
Discrete Device Assignment.
What’s Next
After a device is successfully mounted in a VM, you’re now able to start that VM and interact with the device as you
normally would if you were running on a bare metal system. This means that you’re now able to install the
Hardware Vendor’s drivers in the VM and applications will be able to see that hardware present. You can verify this
by opening device manager in the Guest VM and seeing that the hardware now shows up.
You can then re-enable the device in device manager and the host operating system will be able to interact with the
device again.
Examples
Mounting a GPU to a VM
In this example we use PowerShell to configure a VM named “ddatest1” to take the first GPU available by the
manufacturer NVIDIA and assign it into the VM.
#Configure the VM for a Discrete Device Assignment
$vm = "ddatest1"
#Set automatic stop action to TurnOff
Set-VM -Name $vm -AutomaticStopAction TurnOff
#Enable Write-Combining on the CPU
Set-VM -GuestControlledCacheTypes $true -VMName $vm
#Configure 32 bit MMIO space
Set-VM -LowMemoryMappedIoSpace 3Gb -VMName $vm
#Configure Greater than 32 bit MMIO space
Set-VM -HighMemoryMappedIoSpace 33280Mb -VMName $vm
Starting with Windows Server 2016, you can use Discrete Device Assignment, or DDA, to pass an entire PCIe
Device into a VM. This will allow high performance access to devices like NVMe storage or Graphics Cards from
within a VM while being able to leverage the devices native drivers. Please visit the Plan for Deploying Devices
using Discrete Device Assignment for more details on which devices work, what are the possible security
implications, etc. There are three steps to using a device with DDA:
Configure the VM for DDA
Dismount the Device from the Host Partition
Assigning the Device to the Guest VM
All command can be executed on the Host on a Windows PowerShell console as an Administrator.
You can then re-enable the device in device manager and the host operating system will be able to interact with
the device again.
Manage Hyper-V on Windows Server 2016
6/19/2017 • 1 min to read • Edit Online
Use the resources in this section to help you manage Hyper-V on Windows Server 2016.
Choose between standard or production checkpoints
Enable or disable checkpoints
Manage Windows virtual machines with PowerShell Direct
Set up Hyper-V Replica
Use live migration without Failover Clustering to move a virtual machine
Choose between standard or production checkpoints
in Hyper-V
10/17/2017 • 1 min to read • Edit Online
Starting with Windows Server 2016 and Windows 10, you can choose between standard and production
checkpoints for each virtual machine. Production checkpoints are the default for new virtual machines.
Production checkpoints are "point in time" images of a virtual machine, which can be restored later on in a
way that is completely supported for all production workloads. This is achieved by using backup technology
inside the guest to create the checkpoint, instead of using saved state technology.
Standard checkpoints capture the state, data, and hardware configuration of a running virtual machine and
are intended for use in development and test scenarios. Standard checkpoints can be useful if you need to
recreate a specific state or condition of a running virtual machine so that you can troubleshoot a problem.
NOTE
Only Production Checkpoints are supported on guests that run Active Directory Domain Services role (Domain Controller)
or Active Directory Lightweight Directory Services role.
See also
Production checkpoints
Enable or disable checkpoints
Create Hyper-V VHD Set files
6/19/2017 • 1 min to read • Edit Online
VHD Set files are a new shared Virtual Disk model for guest clusters in Windows Server 2016. VHD Set files support
online resizing of shared virtual disks, support Hype-V Replica, and can be included in application-consistent
checkpoints.
VHD Set files use a new VHD file type, .VHDS. VHD Set files store checkpoint information about the group virtual
disk used in guest clusters, in the form of metadata.
Hyper-V handles all aspects of managing the checkpoint chains and merging the shared VHD set. Management
software can run disk operations like online resizing on VHD Set files in the same way it does for .VHDX files. This
means that management software doesn't need to know about the VHD Set file format.
PS c:\>Remove-VMHardDiskDrive existing.vhdx
PS c:\>Add-VMHardDiskDrive new.vhds
Enable or disable checkpoints in Hyper-V
6/19/2017 • 1 min to read • Edit Online
You can choose to enable or disable checkpoints for each virtual machine.
1. In Hyper-V Manager, right-click the virtual machine and click Settings.
2. Under the Management section, select Checkpoints.
3. To allow checkpoints to be taken of this virtual machine, make sure Enable checkpoints is selected. To
disable checkpoints, clear the Enable checkpoints check box.
4. Click Apply to apply your changes. If you are done, click OK to close the dialog box.
See also
Choose between standard or production checkpoints in Hyper-V
Remotely manage Hyper-V hosts with Hyper-V
Manager
10/17/2017 • 6 min to read • Edit Online
Applies To: Windows Server 2016, Windows Server 2012 R2, Windows 10, Windows 8.1
This article lists the supported combinations of Hyper-V hosts and Hyper-V Manager versions and describes how
to connect to remote and local Hyper-V hosts so you can manage them.
Hyper-V Manager lets you manage a small number of Hyper-V hosts, both remote and local. It's installed when you
install the Hyper-V Management Tools, which you can do either through a full Hyper-V installation or a tools-only
installation. Doing a tools-only installation means you can use the tools on computers that don't meet the
hardware requirements to host Hyper-V. For details about hardware for Hyper-V hosts, see System requirements.
If Hyper-V Manager isn't installed, see the instructions below.
Windows 2016, Windows 10 - Windows Server 2016—all editions and installation options,
including Nano Server, and corresponding version of Hyper-V
Server
- Windows Server 2012 R2—all editions and installation
options, and corresponding version of Hyper-V Server
- Windows Server 2012—all editions and installation options,
and corresponding version of Hyper-V Server
- Windows 10
- Windows 8.1
Windows Server 2012 R2, Windows 8.1 - Windows Server 2012 R2—all editions and installation
options, and corresponding version of Hyper-V Server
- Windows Server 2012—all editions and installation options,
and corresponding version of Hyper-V Server
- Windows 8.1
Windows Server 2012 - Windows Server 2012—all editions and installation options,
and corresponding version of Hyper-V Server
HYPER-V MANAGER VERSION HYPER-V HOST VERSION
Windows Server 2008 R2 Service Pack 1, Windows 7 Service - Windows Server 2008 R2—all editions and installation
Pack 1 options, and corresponding version of Hyper-V Server
Windows Server 2008, Windows Vista Service Pack 2 - Windows Server 2008—all editions and installation options,
and corresponding version of Hyper-V Server
NOTE
Service pack support ended for Windows 8 on January 12, 2016. For more information, see the Windows 8.1 FAQ.
Enable-PSRemoting
NOTE
This will only work for Windows Server 2016 or Windows 10 remote hosts.
NOTE
This will only work for Windows Server 2016 or Windows 10 remote hosts.
Connect to a Windows 2016 or Windows 10 remote host outside your domain, or with no domain
To do this:
1. On the Hyper-V host to be managed, open a Windows PowerShell session as Administrator.
2. Create the necessary firewall rules for private network zones:
Enable-PSRemoting
3. To allow remote access on public zones, enable firewall rules for CredSSP and WinRM:
NOTE
This will only work for Windows Server 2016 or Windows 10 remote hosts.
add-windowsfeature rsat-hyper-v-tools
See also
Install Hyper-V
Not finding content you need? Windows 10 users, tell us what you want on Feedback Hub.
8/28/2017 • 8 min to read • Edit Online
IMPORTANT
Each service you want to use must be enabled in both the host and guest so they can communicate. All integration services
are on by default on Windows guest operating systems, but you can turned them off individually. The next sections show you
how.
Each integration service is listed as a service in Windows. To turn an integration service on or off from inside the
virtual machine, you'll start or stop the service.
Use Windows Services
1. Open Services manager
2. Find the services with Hyper-V in the name.
3. Right-click the service you want start or stop.
Use Windows PowerShell
1. To get a list of integration services, run:
3. Run either Start-Service or Stop-Service. For example, to turn off Windows PowerShell Direct, run:
3. To find out if the required daemons are running, use this command.
ps -ef | grep hv
compgen -c hv_
hv_vss_daemon
hv_get_dhcp_info
hv_get_dns_info
hv_set_ifconfig
hv_kvp_daemon
hv_fcopy_daemon
Integration service daemons that might be listed include the following. If they're not, they might not be
supported on your system or they might not be installed. Find details, see Supported Linux and FreeBSD
virtual machines for Hyper-V on Windows.
hv_vss_daemon: This daemon is required to create live Linux virtual machine backups.
hv_kvp_daemon: This daemon allows setting and querying intrinsic and extrinsic key value pairs.
hv_fcopy_daemon: This daemon implements a file copying service between the host and guest.
Examples
These examples stop and start the KVP daemon, named hv_kvp_daemon .
1. Use the process ID (PID) to stop the daemon's process. To find the PID, look at the second column of the
output, or use pidof . Hyper-V daemons run as root, so you'll need root permissions.
ps -ef | hv
sudo hv_kvp_daemon
4. To verify that the hv_kvp_daemon process is listed with a new process ID, run:
ps -ef | hv
NOTE
The image file vmguest.iso isn't included with Hyper-V on Windows 10 because it's no longer needed.
Windows Vista (SP 2) Windows Update Requires the Data Exchange integration
service.*
Windows Server 2012 Windows Update Requires the Data Exchange integration
service.*
Windows Server 2008 R2 (SP 1) Windows Update Requires the Data Exchange integration
service.*
Windows Server 2008 (SP 2) Windows Update Extended support only in Windows
Server 2016 (read more).
Windows Home Server 2011 Windows Update Will not be supported in Windows
Server 2016 (read more).
Windows Small Business Server 2011 Windows Update Not under mainstream support (read
more).
Linux guests package manager Integration services for Linux are built
into the distro but there may be
optional updates available. ********
* If the Data Exchange integration service can't be enabled, the integration services for these guests are available
from the Download Center as a cabinet (cab) file. Instructions for applying a cab are available in this blog post.
For virtual machines running on Windows 8.1 hosts:
GUEST UPDATE MECHANISM NOTES
Windows Server 2008 (SP 2) Integration Services disk See instructions, below.
Windows Home Server 2011 Integration Services disk See instructions, below.
Windows Small Business Server 2011 Integration Services disk See instructions, below.
Windows Server 2003 R2 (SP 2) Integration Services disk See instructions, below.
Windows Server 2003 (SP 2) Integration Services disk See instructions, below.
Linux guests package manager Integration services for Linux are built
into the distro but there may be
optional updates available. **
-
GUEST UPDATE MECHANISM NOTES
Windows Server 2008 (SP 2) Integration Services disk See instructions, below.
Windows Home Server 2011 Integration Services disk See instructions, below.
Windows Small Business Server 2011 Integration Services disk See instructions, below.
Windows Server 2003 R2 (SP 2) Integration Services disk See instructions, below.
Windows Server 2003 (SP 2) Integration Services disk See instructions, below.
Linux guests package manager Integration services for Linux are built
into the distro but there may be
optional updates available. **
For more details about Linux guests, see Supported Linux and FreeBSD virtual machines for Hyper-V on Windows.
You can use PowerShell Direct to remotely manage a Windows 10 or Windows Server 2016 virtual machine from
a Windows 10 or Windows Server 2016 Hyper-V host. PowerShell Direct allows Windows PowerShell
management inside a virtual machine regardless of the network configuration or remote management settings on
either the Hyper-V host or the virtual machine. This makes it easier for Hyper-V Administrators to automate and
script virtual machine management and configuration.
There are two ways to run PowerShell Direct:
Create and exit a PowerShell Direct session using PSSession cmdlets
Run script or command with the Invoke-Command cmdlet
If you're managing older virtual machines, use Virtual Machine Connection (VMConnect) or configure a virtual
network for the virtual machine.
Exit-PSSession
See Also
Enter-PSSession
Exit-PSSession
Invoke-Command
Set up Hyper-V Replica
6/19/2017 • 11 min to read • Edit Online
Hyper-V Replica is an integral part of the Hyper-V role. It contributes to your disaster recovery strategy by
replicating virtual machines from one Hyper-V host server to another to keep your workloads available. Hyper-V
Replica creates a copy of a live virtual machine to a replica offline virtual machine. Note the following:
Hyper-V hosts: Primary and secondary host servers can be physically co-located or in separate
geographical locations with replication over a WAN link. Hyper-V hosts can be standalone, clustered, or a
mixture of both. There's no Active Directory dependency between the servers and they don't need to be
domain members.
Replication and change tracking: When you enable Hyper-V Replica for a specific virtual machine, initial
replication creates an identical replica virtual machine on a secondary host server. After that happens,
Hyper-V Replica change tracking creates and maintains a log file that captures changes on a virtual machine
VHD. The log file is played in reverse order to the replica VHD based on replication frequency settings. This
means that the latest changes are stored and replicated asynchronously. Replication can be over HTTP or
HTTPS.
Extended (chained) replication: This lets you replicate a virtual machine from a primary host to a
secondary host, and then replicate the secondary host to a third host. Note that you can't replicate from the
primary host directly to the second and the third.
This feature makes Hyper-V Replica more robust for disaster recovery because if an outage occurs you can
recover from both the primary and extended replica. You can fail over to the extended replica if your
primary and secondary locations go down. Note that the extended replica doesn't support application-
consistent replication and must use the same VHDs that the secondary replica is using.
Failover: If an outage occurs in your primary (or secondary in case of extended) location you can manually
initiate a test, planned, or unplanned failover.
When should I run? Verify that a virtual During planned downtime During unexpected events
machine can fail over and and outages
start in the secondary site
Where is it initiated? On the replica virtual Initiated on primary and On the replica virtual
machine completed on secondary machine
How often should I run? We recommend once a Once every six months or Only in case of disaster
month for testing in accordance with when the primary virtual
compliance requirements machine is unavailable
TEST PLANNED UNPLANNED
Is there any data loss? None None. After failover Hyper- Depends on the event and
V Replica replicates the last recovery points
set of tracked changes
back to the primary to
ensure zero data loss.
Is there any downtime? None. It doesn't impact The duration of the The duration of the
your production planned outage unplanned outage
environment. It creates a
duplicate test virtual
machine during failover.
After failover finishes you
select Failover on the
replica virtual machine and
it's automatically cleaned
up and deleted.
Recovery points: When you configure replication settings for a virtual machine, you specify the recovery
points you want to store from it. Recovery points represent a snapshot in time from which you can recover a
virtual machine. Obviously less data is lost if you recover from a very recent recovery point. You can access
recovery points up to 24 hours ago.
Deployment prerequisites
Here's what you should verify before you begin:
Figure out which VHDs need to be replicated. In particular, VHDs that contain data that is rapidly
changing and not used by the Replica server after failover, such as page file disks, should be excluded from
replication to conserve network bandwidth. Make a note of which VHDs can be excluded.
Decide how often you need to synchronize data: The data on the Replica server is synchronized
updated according to the replication frequency you configure (30 seconds, 5 minutes, or 15 minutes). The
frequency you choose should consider the following: Are the virtual machines running critical data with a
low RPO? What are you bandwidth considerations? Your highly-critical virtual machines will obviously need
more frequent replication.
Decide how to recover data: By default Hyper-V Replica only stores a single recovery point that will be the
latest replication sent from the primary to the secondary. However if you want the option to recover data to
an earlier point in time you can specify that additional recovery points should be stored (to a maximum of
24 hourly points). If you do need additional recovery points you should note that this requires more
overhead on processing and storage resources.
Figure out which workloads you'll replicate: Standard Hyper-V Replica replication maintains
consistency in the state of the virtual machine operating system after a failover, but not the state of
applications that running on the virtual machine. If you want to be able to recovery your workload state you
can create app-consistent recovery points. Note that app-consistent recovery isn't available on the extended
replica site if you're using extended (chained) replication.
Decide how to do the initial replication of virtual machine data: Replication starts by transferring the
needs to transfer the current state of the virtual machines. This initial state can be transmitted directly over
the existing network, either immediately or at a later time that you configure. You can also use a pre-existing
restored virtual machine (for example, if you have restored an earlier backup of the virtual machine on the
Replica server) as the initial copy. Or, you can save network bandwidth by copying the initial copy to external
media and then physically delivering the media to the Replica site. If you want to use a preexisting virtual
machine delete all previous snapshots associated with it.
Deployment steps
Step 1: Set up the Hyper-V hosts
You'll need at least two Hyper-V hosts with one or more virtual machines on each server. Get started with Hyper-V.
The host server that you'll replicate virtual machines to will need to be set up as the replica server.
1. In the Hyper-V settings for the server you'll replicate virtual machines to, in Replication Configuration,
select Enable this computer as a Replica server.
2. You can replicate over HTTP or encrypted HTTPS. Select Use Kerberos (HTTP) or Use certificate-based
Authentication (HTTPS). By default HTTP 80 and HTTP 443 are enabled as firewall exceptions on the
replica Hyper-V server. If you change the default port settings you'll need to also change the firewall
exception. If you're replicating over HTTPS, you'll need to select a certificate and you should have certificate
authentication set up.
3. For authorization, select Allow replication from any authenticated server to allow the replica server to
accept virtual machine replication traffic from any primary server that authenticates successfully. Select
Allow replication from the specified servers to accept traffic only from the primary servers you
specifically select.
For both options you can specify where the replicated VHDs should be stored on the replica Hyper-V server.
4. Click OK or Apply.
Step 2: Set up the firewall
To allow replication between the primary and secondary servers, traffic must get through the Windows firewall (or
any other third-party firewalls). When you installed the Hyper-V role on the servers by default exceptions for HTTP
(80) and HTTPS (443) are created. If you're using these standard ports, you'll just need to enable the rules:
To enable the rules on a standalone host server:
1. Open Windows Firewall with Advance Security and click Inbound Rules.
2. To enable HTTP (Kerberos) authentication, right-click Hyper-V Replica HTTP Listener (TCP-In)
>Enable Rule. To enable HTTPS certificate-based authentication, right-click Hyper-V Replica
HTTPS Listener (TCP-In) > Enable Rule.
To enable rules on a Hyper-V cluster, open a Windows PowerShell session using Run as Administrator,
then run one of these commands:
For HTTP:
get-clusternode | ForEach-Object {Invoke-command -computername $_.name -scriptblock {Enable-
Netfirewallrule -displayname "Hyper-V Replica HTTP Listener (TCP-In)"}}
For HTTPS:
get-clusternode | ForEach-Object {Invoke-command -computername $_.name -scriptblock {Enable-
Netfirewallrule -displayname "Hyper-V Replica HTTPS Listener (TCP-In)"}}
Run a failover
After completing these deployment steps your replicated environment is up and running. Now you can run
failovers as needed.
Test failover: If you want to run a test failover right-click the primary virtual machine and select Replication >
Test Failover. Pick the latest or other recovery point if configured. A new test virtual machine will be created and
started on the secondary site. After you've finished testing, select Stop Test Failover on the replica virtual machine
to clean up it up. Note that for a virtual machine you can only run on test failover at a time. Read more.
Planned failover: To run a planned failover right-click the primary virtual machine and select Replication >
Planned Failover. Planned failover performs prerequisites checks to ensure zero data loss. It checks that the
primary virtual machine is shut down before beginning the failover. After the virtual machine is failed over, it starts
replicating the changes back to the primary site when it's available. Note that for this to work the primary server
should be configured to recive replication from the secondary server or from the Hyper-V Replica Broker in the
case of a primary cluster. Planned failover sends the last set of tracked changes. Read more.
Unplanned failover: To run an unplanned failover, right-click on the replica virtual machine and select
Replication > Unplanned Failover from Hyper-V Manager or Failover Clustering Manager. You can recover
from the latest recovery point or from previous recovery points if this option is enabled. After failover, check that
everything is working as expected on the failed over virtual machine, then click Complete on the replica virtual
machine. Read more.
Live Migration Overview
6/28/2017 • 1 min to read • Edit Online
Live migration is a Hyper-V feature in Windows Server 2016. It allows you to transparently move running Virtual
Machines from one Hyper-V host to another without perceived downtime. The primary benefit of live migration is
flexibility; running Virtual Machines are not tied to a single host machine. This allows actions like draining a specific
host of Virtual Machines before decommissioning or upgrading it. When paired with Windows Failover Clustering,
live migration allows the creation of highly available and fault tolerant systems.
Live migration is a Hyper-V feature in Windows Server 2016. It allows you to transparently move running Virtual
Machines from one Hyper-V host to another without perceived downtime. The primary benefit of live migration is
flexibility; running Virtual Machines are not tied to a single host machine. This allows actions like draining a specific
host of Virtual Machines before decommissioning or upgrading it. When paired with Windows Failover Clustering,
live migration allows the creation of highly available and fault tolerant systems.
This article shows you how to set up hosts that aren't clustered so you can do live migrations between them. Use
these instructions if you didn't set up live migration when you installed Hyper-V, or if you want to change the
settings. To set up clustered hosts, use tools for Failover Clustering.
Step 2: Set up the source and destination computers for live migration
This step includes choosing options for authentication and networking. As a security best practice, we recommend
that you select specific networks to use for live migration traffic, as discussed above. This step also shows you
how to choose the performance option.
Use Hyper-V Manager to set up the source and destination computers for live migration
1. Open Hyper-V Manager. (From Server Manager, click Tools >>Hyper-V Manager.)
2. In the navigation pane, select one of the servers. (If it isn't listed, right-click Hyper-V Manager, click
Connect to Server, type the server name, and click OK. Repeat to add more servers.)
3. In the Action pane, click Hyper-V Settings >>Live Migrations.
4. In the Live Migrations pane, check Enable incoming and outgoing live migrations.
5. Under Simultaneous live migrations, specify a different number if you don't want to use the default of 2.
6. Under Incoming live migrations, if you want to use specific network connections to accept live migration
traffic, click Add to type the IP address information. Otherwise, click Use any available network for live
migration. Click OK.
7. To choose Kerberos and performance options, expand Live Migrations and then select Advanced
Features.
If you have configured constrained delegation, under Authentication protocol, select Kerberos.
Under Performance options, review the details and choose a different option if it's appropriate for
your environment.
8. Click OK.
9. Select the other server in Hyper-V Manager and repeat the steps.
Use Windows PowerShell to set up the source and destination computers for live migration
Three cmdlets are available for configuring live migration on non-clustered hosts: Enable-VMMigration, Set-
VMMigrationNetwork, and Set-VMHost. This example uses all three and does the following:
Configures live migration on the local host
Allows incoming migration traffic only on a specific network
Chooses Kerberos as the authentication protocol
Each line represents a separate command.
PS C:\> Enable-VMMigration
Set-VMHost also lets you choose a performance option (and many other host settings). For example, to choose
SMB but leave the authentication protocol set to the default of CredSSP, type:
OPTION DESCRIPTION
Next steps
After you set up the hosts, you're ready to do a live migration. For instructions, see Use live migration without
Failover Clustering to move a virtual machine.
Use live migration without Failover Clustering to
move a virtual machine
6/19/2017 • 2 min to read • Edit Online
This article shows you how to move a virtual machine by doing a live migration without using Failover Clustering.
A live migration moves running virtual machines between Hyper-V hosts without any noticeable downtime.
To be able to do this, you'll need:
A user account that's a member of the local Hyper-V Administrators group or the Administrators group on
both the source and destination computers.
The Hyper-V role in Windows Server 2016 or Windows Server 2012 R2 installed on the source and
destination servers and set up for live migrations. You can do a live migration between hosts running
Windows Server 2016 and Windows Server 2012 R2 if the virtual machine is at least version 5.
For version upgrade instructions, see Upgrade virtual machine version in Hyper-V on Windows 10 or
Windows Server 2016. For installation instructions, see Set up hosts for live migration.
The Hyper-V management tools installed on a computer running Windows Server 2016 or Windows 10,
unless the tools are installed on the source or destination server and you'll run them from there.
Troubleshooting
Failed to establish a connection
If you haven't set up constrained delegation, you must sign in to source server before you can move a virtual
machine. If you don't do this, the authentication attempt fails, an error occurs, and this message is displayed:
"Virtual machine migration operation failed at migration Source.
Failed to establish a connection with host computer name: No credentials are available in the security package
0x8009030E."
To fix this problem, sign in to the source server and try the move again. To avoid having to sign in to a source
server before doing a live migration, set up constrained delegation. You'll need domain administrator credentials
to set up constrained delegation. For instructions, see Set up hosts for live migration.
Failed because the host hardware isn't compatible
If a virtual machine doesn't have processor compatibility turned on and has one or more snapshots, the move fails
if the hosts have different processor versions. An error occurs and this message is displayed:
The virtual machine cannot be moved to the destination computer. The hardware on the destination
computer is not compatible with the hardware requirements of this virtual machine.
To fix this problem, shut down the virtual machine and turn on the processor compatibility setting.
1. From Hyper-V Manager, in the Virtual Machines pane, right-click the virtual machine and click Settings.
2. In the navigation pane, expand Processors and click Compatibility.
3. Check Migrate to a computer with a different processor version.
4. Click OK.
To use Windows PowerShell, use the Set-VMProcessor cmdlet:
Hyper-V is the virtualization server role in Windows Server 2016. Virtualization servers can host multiple virtual
machines that are isolated from each other but share the underlying hardware resources by virtualizing the
processors, memory, and I/O devices. By consolidating servers onto a single machine, virtualization can improve
resource usage and energy efficiency and reduce the operational and maintenance costs of servers. In addition,
virtual machines and the management APIs offer more flexibility for managing resources, balancing load, and
provisioning systems.
See also
Hyper-V terminology
Hyper-V architecture
Hyper-V server configuration
Hyper-V processor performance
Hyper-V memory performance
Hyper-V storage I/O performance
Hyper-V network I/O performance
Detecting bottlenecks in a virtualized environment
Linux Virtual Machines
Hyper-V Virtual Switch
10/24/2017 • 4 min to read • Edit Online
This topic provides an overview of Hyper-V Virtual Switch, which provides you with the ability to connect virtual
machines (VMs) to networks that are external to the Hyper-V host, including your organization's intranet and the
Internet.
You can also connect to virtual networks on the server that is running Hyper-V when you deploy Software Defined
Networking (SDN).
NOTE
In addition to this topic, the following Hyper-V Virtual Switch documentation is available.
Manage Hyper-V Virtual Switch
Remote Direct Memory Access (RDMA) and Switch Embedded Teaming (SET)
Network Switch Team Cmdlets in Windows PowerShell
What's New in VMM 2016
Set up the VMM networking fabric
Create networks with VMM 2012
Hyper-V: Configure VLANs and VLAN Tagging
Hyper-V: The WFP virtual switch extension should be enabled if it is required by third party extensions
For more information about other networking technologies, see Networking in Windows Server 2016.
Hyper-V Virtual Switch is a software-based layer-2 Ethernet network switch that is available in Hyper-V Manager
when you install the Hyper-V server role.
Hyper-V Virtual Switch includes programmatically managed and extensible capabilities to connect VMs to both
virtual networks and the physical network. In addition, Hyper-V Virtual Switch provides policy enforcement for
security, isolation, and service levels.
NOTE
Hyper-V Virtual Switch only supports Ethernet, and does not support any other wired local area network (LAN) technologies,
such as Infiniband and Fibre Channel.
Hyper-V Virtual Switch includes tenant isolation capabilities, traffic shaping, protection against malicious virtual
machines, and simplified troubleshooting.
With built-in support for Network Device Interface Specification (NDIS) filter drivers and Windows Filtering
Platform (WFP) callout drivers, the Hyper-V Virtual Switch enables independent software vendors (ISVs) to create
extensible plug-ins, called Virtual Switch Extensions, that can provide enhanced networking and security
capabilities. Virtual Switch Extensions that you add to the Hyper-V Virtual Switch are listed in the Virtual Switch
Manager feature of Hyper-V Manager.
In the following illustration, a VM has a virtual NIC that is connected to the Hyper-V Virtual Switch through a
switch port.
Hyper-V Virtual Switch capabilities provide you with more options for enforcing tenant isolation, shaping and
controlling network traffic, and employing protective measures against malicious VMs.
NOTE
In Windows Server 2016, a VM with a virtual NIC accurately displays the maximum throughput for the virtual NIC. To view
the virtual NIC speed in Network Connections, right-click the desired virtual NIC icon and then click Status. The virtual NIC
Status dialog box opens. In Connection, the value of Speed matches the speed of the physical NIC installed in the server.
This topic provides information on configuring Remote Direct Memory Access (RDMA) interfaces with Hyper-V in
Windows Server 2016, in addition to information about Switch Embedded Teaming (SET).
NOTE
In addition to this topic, the following Switch Embedded Teaming content is available.
TechNet Gallery Download: Windows Server 2016 NIC and Switch Embedded Teaming User Guide
TIP
In editions of Windows Server previous to Windows Server 2016, it is not possible to configure RDMA on network adapters
that are bound to a NIC Team or to a Hyper-V Virtual Switch. In Windows Server 2016, you can enable RDMA on network
adapters that are bound to a Hyper-V Virtual Switch with or without SET.
In Windows Server 2016, you can use fewer network adapters while using RDMA with or without SET.
The image below illustrates the software architecture changes between Windows Server 2012 R2 and Windows
Server 2016.
The following sections provide instructions on how to use Windows PowerShell commands to enable Data Center
Bridging (DCB), create a Hyper-V Virtual Switch with an RDMA virtual NIC (vNIC), and create a Hyper-V Virtual
Switch with SET and RDMA vNICs.
Enable Data Center Bridging (DCB )
Before using any RDMA over Converged Ethernet (RoCE) version of RDMA, you must enable DCB. While not
required for Internet Wide Area RDMA Protocol (iWARP) networks, testing has determined that all Ethernet-based
RDMA technologies work better with DCB. Because of this, you should consider using DCB even for iWARP RDMA
deployments.
The following Windows PowerShell example commands demonstrate how to enable and configure DCB for SMB
Direct.
Turn on DCB
Install-WindowsFeature Data-Center-Bridging
Enable-NetQosFlowControl -Priority 3
If you have a kernel debugger installed in the system, you must configure the debugger to allow QoS to be set by
running the following command.
Override the Debugger - by default the debugger blocks NetQos:
NOTE
Using SET teams with RDMA-capable physical NICs provides more RDMA resources for the vNICs to consume.
New-VMSwitch -Name RDMAswitch -NetAdapterName "SLOT 2"
Get-NetAdapterRdma
Many switches won't pass traffic class information on untagged VLAN traffic, so make sure that the host adapters
for RDMA are on VLANs. This example assigns the two SMB_* host virtual adapters to VLAN 42.
Get-NetAdapterRdma | fl *
SET Overview
SET is an alternative NIC Teaming solution that you can use in environments that include Hyper-V and the
Software Defined Networking (SDN) stack in Windows Server 2016. SET integrates some NIC Teaming
functionality into the Hyper-V Virtual Switch.
SET allows you to group between one and eight physical Ethernet network adapters into one or more software-
based virtual network adapters. These virtual network adapters provide fast performance and fault tolerance in the
event of a network adapter failure.
SET member network adapters must all be installed in the same physical Hyper-V host to be placed in a team.
NOTE
The use of SET is only supported in Hyper-V Virtual Switch in Windows Server 2016. You cannot deploy SET in Windows
Server 2012 R2 .
You can connect your teamed NICs to the same physical switch or to different physical switches. If you connect
NICs to different switches, both switches must be on the same subnet.
The following illustration depicts SET architecture.
Because SET is integrated into the Hyper-V Virtual Switch, you cannot use SET inside of a virtual machine (VM).
You can, however use NIC Teaming within VMs.
For more information, see NIC Teaming in Virtual Machines (VMs).
In addition, SET architecture does not expose team interfaces. Instead, you must configure Hyper-V Virtual Switch
ports.
SET Availability
SET is available in all versions of Windows Server 2016 that include Hyper-V and the SDN stack. In addition, you
can use Windows PowerShell commands and Remote Desktop connections to manage SET from remote
computers that are running a client operating system upon which the tools are supported.
NOTE
When you use SET in conjunction with Packet Direct, the teaming mode Switch Independent and the load balancing mode
Hyper-V Port are required.
Because the adjacent switch always sees a particular MAC address on a given port, the switch distributes the
ingress load (the traffic from the switch to the host) to the port where the MAC address is located. This is
particularly useful when Virtual Machine Queues (VMQs) are used, because a queue can be placed on the specific
NIC where the traffic is expected to arrive.
However, if the host has only a few VMs, this mode might not be granular enough to achieve a well-balanced
distribution. This mode will also always limit a single VM (i.e., the traffic from a single switch port) to the
bandwidth that is available on a single interface.
Dynamic
This load balancing mode provides the following advantages.
Outbound loads are distributed based on a hash of the TCP Ports and IP addresses. Dynamic mode also re-
balances loads in real time so that a given outbound flow can move back and forth between SET team
members.
Inbound loads are distributed in the same manner as the Hyper-V port mode.
The outbound loads in this mode are dynamically balanced based on the concept of flowlets. Just as human
speech has natural breaks at the ends of words and sentences, TCP flows (TCP communication streams) also have
naturally occurring breaks. The portion of a TCP flow between two such breaks is referred to as a flowlet.
When the dynamic mode algorithm detects that a flowlet boundary has been encountered - for example when a
break of sufficient length has occurred in the TCP flow - the algorithm automatically rebalances the flow to
another team member if appropriate. In some uncommon circumstances, the algorithm might also periodically
rebalance flows that do not contain any flowlets. Because of this, the affinity between TCP flow and team member
can change at any time as the dynamic balancing algorithm works to balance the workload of the team members.
NOTE
SET always presents the total number of queues that are available across all SET team members. In NIC Teaming, this is
called Sum-of-Queues mode.
Most network adapters have queues that can be used for either Receive Side Scaling (RSS) or VMQ, but not both
at the same time.
Some VMQ settings appear to be settings for RSS queues but are really settings on the generic queues that both
RSS and VMQ use depending on which feature is presently in use. Each NIC has, in its advanced properties, values
for *RssBaseProcNumber and *MaxRssProcessors .
Following are a few VMQ settings that provide better system performance.
Ideally each NIC should have the *RssBaseProcNumber set to an even number greater than or equal to two (2).
This is because the first physical processor, Core 0 (logical processors 0 and 1), typically does most of the
system processing so the network processing should be steered away from this physical processor.
NOTE
Some machine architectures don't have two logical processors per physical processor, so for such machines the base
processor should be greater than or equal to 1. If in doubt, assume your host is using a 2 logical processor per physical
processor architecture.
The team members' processors should be, to the extent that it's practical, non-overlapping. For example, in a 4-
core host (8 logical processors) with a team of 2 10Gbps NICs, you could set the first one to use base processor
of 2 and to use 4 cores; the second would be set to use base processor 6 and use 2 cores.
If you want to create a SET-capable switch with a single team member so that you can add a team member at a
later time, then you must use the EnableEmbeddedTeaming parameter.
Remove-VMSwitch "SETvSwitch"
This topic is examined in more depth in section 4.2.5 of the Windows Server 2016 NIC and Switch Embedded
Teaming User Guide.
Manage Hyper-V Virtual Switch
10/17/2017 • 1 min to read • Edit Online
You can use this topic to access Hyper-V Virtual Switch management content.
This section contains the following topics.
Configure VLANs on Hyper-V Virtual Switch Ports
Create Security Policies with Extended Port Access Control Lists
Configure and View VLAN Settings on Hyper-V
Virtual Switch Ports
10/17/2017 • 2 min to read • Edit Online
You can use this topic to learn best practices for configuring and viewing virtual Local Area Network (VLAN)
settings on a Hyper-V Virtual Switch port.
When you want to configure VLAN settings on Hyper-V Virtual Switch ports, you can use either Windows® Server
2016 Hyper-V Manager or System Center Virtual Machine Manager (VMM).
If you are using VMM, VMM uses the following Windows PowerShell command to configure the switch port.
If you are not using VMM and are configuring the switch port in Windows Server, you can use the Hyper-V
Manager console or the following Windows PowerShell command.
Because of these two methods for configuring VLAN settings on Hyper-V Virtual Switch ports, it is possible that
when you attempt to view the switch port settings, it appears to you that the VLAN settings are not configured -
even when they are configured.
Use the same method to configure and view switch port VLAN settings
To ensure that you do not encounter these issues, you must use the same method to view your switch port VLAN
settings that you used to configure the switch port VLAN settings.
To configure and view VLAN switch port settings, you must do the following:
If you are using VMM or Network Controller to set up and manage your network, and you have deployed
Software Defined Networking (SDN), you must use the VMNetworkAdapterIsolation cmdlets.
If you are using Windows Server 2016 Hyper-V Manager or Windows PowerShell cmdlets, and you have not
deployed Software Defined Networking (SDN), you must use the VMNetworkAdapterVlan cmdlets.
Possible issues
If you do not follow these guidelines you might encounter the following issues.
In circumstances where you have deployed SDN and use VMM, Network Controller, or the
VMNetworkAdapterIsolation cmdlets to configure VLAN settings on a Hyper-V Virtual Switch port: If you use
Hyper-V Manager or Get VMNetworkAdapterVlan to view the configuration settings, the command output
will not display your VLAN settings. Instead you must use the Get-VMNetworkIsolation cmdlet to view the
VLAN settings.
In circumstances where you have not deployed SDN, and instead use Hyper-V Manager or the
VMNetworkAdapterVlan cmdlets to configure VLAN settings on a Hyper-V Virtual Switch port: If you use the
Get-VMNetworkIsolation cmdlet to view the configuration settings, the command output will not display your
VLAN settings. Instead you must use the Get VMNetworkAdapterVlan cmdlet to view the VLAN settings.
It is also important not to attempt to configure the same switch port VLAN settings by using both of these
configuration methods. If you do this, the switch port is incorrectly configured, and the result might be a failure in
network communication.
Resources
For more information on the Windows PowerShell commands that are mentioned in this topic, see the following:
Set-VmNetworkAdapterIsolation
Get-VmNetworkAdapterIsolation
Set-VMNetworkAdapterVlan
Get-VMNetworkAdapterVlan
Create Security Policies with Extended Port Access
Control Lists
10/17/2017 • 9 min to read • Edit Online
This topic provides information about extended port Access Control Lists (ACLs) in Windows Server 2016. You can
configure extended ACLs on the Hyper-V Virtual Switch to allow and block network traffic to and from the virtual
machines (VMs) that are connected to the switch via virtual network adapters.
This topic contains the following sections.
Detailed ACL rules
Stateful ACL rules
NOTE
If you want to add an extended ACL to one network adapter rather than all, you can specify the network adapter with
the parameter -VMNetworkAdapterName.
Add-VMNetworkAdapterExtendedAcl [-VMName] <string[]> [-Action] <VMNetworkAdapterExtendedAclAction>
{Allow | Deny}
[-Direction] <VMNetworkAdapterExtendedAclDirection> {Inbound | Outbound} [[-LocalIPAddress]
<string>]
[[-RemoteIPAddress] <string>] [[-LocalPort] <string>] [[-RemotePort] <string>] [[-Protocol]
<string>] [-Weight]
<int> [-Stateful <bool>] [-IdleSessionTimeout <int>] [-IsolationID <int>] [-Passthru] [-
VMNetworkAdapterName
<string>] [-ComputerName <string[]>] [-WhatIf] [-Confirm] [<CommonParameters>]
2. Add an extended ACL to a specific virtual network adapter on a specific VM. Syntax:
3. Add an extended ACL to all virtual network adapters that are reserved for use by the Hyper-V host
management operating system.
NOTE
If you want to add an extended ACL to one network adapter rather than all, you can specify the network adapter with
the parameter -VMNetworkAdapterName.
4. Add an extended ACL to a VM object that you have created in Windows PowerShell, such as $vm = get-vm
"my_vm". In the next line of code you can run this command to create an extended ACL with the following
syntax:
NOTE
The values for the rule parameter Direction in the tables below are based on traffic flow to or from the VM for which you are
creating the rule. If the VM is receiving traffic, the traffic is inbound; if the VM is sending traffic, the traffic is outbound. For
example, if you apply a rule to a VM that blocks inbound traffic, the direction of inbound traffic is from external resources to
the VM. If you apply a rule that blocks outbound traffic, the direction of outbound traffic is from the local VM to external
resources.
DESTINATION DESTINATION
SOURCE IP IP PROTOCOL SOURCE PORT PORT DIRECTION ACTION
Following are two examples of how you can create rules with Windows PowerShell commands. The first example
rule blocks all traffic to the VM named "ApplicationServer." The second example rule, which is applied to the
network adapter of the VM named "ApplicationServer," allows only inbound RDP traffic to the VM.
NOTE
When you create rules, you can use the -Weight parameter to determine the order in which the Hyper-V Virtual Switch
processes the rules. Values for -Weight are expressed as integers; rules with a higher integer are processed before rules with
lower integers. For example, if you have applied two rules to a VM network adapter, one with a weight of 1 and one with a
weight of 10, the rule with the weight of 10 is applied first.
DESTINATION DESTINATION
SOURCE IP IP PROTOCOL SOURCE PORT PORT DIRECTION ACTION
DESTINATION DESTINATION
SOURCE IP IP PROTOCOL SOURCE PORT PORT DIRECTION ACTION
Following are examples of how you can create these rules with Windows PowerShell commands.
NOTE
IGMP has a designated IP protocol number of 0x02.
DESTINATION DESTINATION
SOURCE IP IP PROTOCOL SOURCE PORT PORT DIRECTION ACTION
* * 0x02 * * In Allow
Following is an example of how you can create these rules with Windows PowerShell commands.
NOTE
Due to formatting restrictions and the amount of information in the table below, the information is displayed differently than
in previous tables in this document.
Source IP * * *
Destination IP * * *
Protocol * * TCP
Source Port * * *
Destination Port * * 80
Stateful No No Yes
The stateful rule allows the VM application server to connect to a remote Web server. When the first packet is sent
out, Hyper-V Virtual switch dynamically creates two flow states to allow all packets sent to and all returning packets
from the remote Web server. When the flow of packets between the servers stops, the flow states time out in the
designated timeout value of 3600 seconds, or one hour.
Following is an example of how you can create these rules with Windows PowerShell commands.