Hardening Virtual Machine Hosts Managed by VMM
Applies To: Virtual Machine Manager 2008, Virtual Machine Manager 2008 R2, Virtual Machine Manager 2008 R2 SP1
This topic explains security requirements and security best practices in System Center Virtual Machine Manager (VMM) 2008 and VMM 2008 R2 for virtual machine hosts that are running Microsoft Hyper-V, Microsoft Virtual Server, and VMware ESX Server. The configuration of ports, protocols, and authentication methods is in large part determined by the host type, which is based on the network location and the type of operating system that runs on the host. The first part of this topic describes those requirements. The second part of this topic describes specific security requirements in VMM for Hyper-V, Virtual Server, and ESX Server.
Security Requirements Based on Host Type
Security requirements for virtual machine hosts in VMM vary based on the location of the host and the operating system that runs on it.
Host Types in VMM
Security requirements generally vary based on the following host types:
Hosts in an Active Directory domain that has a two-way trust relationship with the domain of the VMM server
VMM supports managing hosts and host clusters in a disjointed namespace in a domain that has a two-way trust relationship with the domain of the VMM server. For more information about disjointed namespaces, see Naming conventions in Active Directory for computers, domains, sites, and OUs (http://go.microsoft.com/fwlink/?LinkId=123886).
Windows Server-based hosts on a perimeter network
A perimeter network is separate from an organization's internal network and the Internet and has the following characteristics:
Allows external users to access specific computers located on the perimeter network.
Prevents access to computers on the organization's internal network.
Can be set up to allow limited access from users on the internal network to computers located on the perimeter network.
For example, a perimeter network can include the company's Web server so that it can deliver Web content to the Internet. However, the perimeter network does not allow external users to access any private company data on computers on the internal network. Even if an external user penetrates the perimeter network security, only the perimeter network servers might be compromised.
You can deploy virtual machines on a host on a perimeter network from within the internal network. However, after you deploy virtual machines on a host on a perimeter network, you cannot migrate those virtual machines back to a host on the internal network or to another host on the perimeter network.
Because a perimeter network is separate from the internal network, security for a host on a perimeter network must be provided by a local service account.
Hosts in a non-trusted Active Directory domain
For a Windows Server–based host in an Active Directory domain that does not have a two-way trust relationship with the domain of the VMM server, VMM uses the same authentication and encryption methods that it uses for a Windows Server–based host on a perimeter network. For that reason, security requirements for those two topologies will be discussed together.
Hosts that don’t run on a Windows Server operating system
Non-Windows Server–based hosts in a managed VMware Infrastructure 3 (VI3) environment have different security requirements than do Windows Server–based hosts, and are discussed separately.
The following sections describe the security requirements for Windows Server–based and non-Windows Server-based hosts. For a quick comparison of the security features of each type of host described above, see the summary table in Comparison of VMM Security Requirements by Host Type, at the end of this section.
Ports and Protocols for Windows Server–Based Hosts
For Windows Server–based hosts, the host location does not affect file transfers or client communications. However, the transfer of control data between the VMM server and a host is managed and secured differently.
For Windows-based hosts running either Hyper-V or Windows Server, VMM uses the WS-Management protocol to transfer control data. WS-Management is an HTTP protocol that connects via port 80 by default. Windows Remote Management (WinRM), the Microsoft implementation of the WS-Management protocol, handles authentication and encryption internally.
The authentication method that is used depends on the host location:
Trusted Active Directory domain—For Windows-based hosts in an Active Directory domain that has a two-way trust relationship with the domain of the VMM server, Kerberos is used for authentication.
Non-trusted Active Directory domain or perimeter network—For Windows Server–based hosts in a non-trusted Active Directory domain or on a perimeter network, the VMM agent uses NTLM for authentication and a CA-signed certificate that is installed on the host during agent installation to encrypt communications between VMM and the host. The credentials are created at random and support mutual authentication.
A host on a perimeter network requires local installation of the VMM agent. The host then must be added to VMM by using the Add Hosts Wizard to provide credentials and to retrieve the certificate and public key that were generated during agent installation. Any updates to the VMM agent on a host on a perimeter network also require manual agent installation followed by updating the host credentials in VMM.
In a non-trusted Active Directory domain, local installation of the VMM agent or any future updates to the agent is not required. VMM installs the agent when the host is added to VMM.
For all file transfer operations between Windows Server–based hosts, VMM uses the BITS 2.5 protocol, which is encrypted using Secure Sockets Layer (SSL) on default port 443. The port number must not exceed 32768.
If you have implemented another form of encryption, such as IPsec, or have otherwise secured your virtual environment, you might want to take advantage of the new option of VMM 2008 R2 to allow unencrypted file transfers for individual host groups and library servers. Allowing unencrypted file transfers can improve performance during virtual machine creation and migration. For files to be transferred unencrypted, unencrypted file transfers must be allowed on both the source and the destination computer. This option is available by updating the properties of a host group. For a procedure, see How to Modify the Properties of a Host Group (http://go.microsoft.com/fwlink/?LinkId=162967).
To ensure the integrity of data within your trusted Active Directory domains, do not migrate virtual machines from a host that is on a perimeter network or is in an Active Directory domain that does not have a two-way trust relationship with the domain that contains the VMM server to a host in a trusted Active Directory domain, and do not transfer data or files from those hosts to a host or library server in a trusted Active Directory domain.
When a host is on a perimeter network or in a non-trusted Active Directory domain, VMM uses the NTLM protocol for communications between VMM and the host. On the NTLM protocol, the VMM server is authenticated to the host by using the local user credentials on the host that were provided during VMM agent installation and transferred securely to the VMM server. However, the host is not authenticated to VMM on this channel.
During the BITS transfers over the HTTPS protocol that occur during virtual machine migrations, if the source host is on a perimeter network or in a non-trusted Active Directory domain, the source host uses its own certificate to authenticate itself during the BITS transfer; however, the target host cannot be authenticated. Because the target host is not authenticated on either the NTLM or BITS protocol, both the VMM server and the host are susceptible to a spoofing attack.
For connections to clients, VMM uses Windows Communication Foundation (WCF) over default port 8100. WCF uses TCP internally, with encryption enabled. Kerberos is used for authentication.
During authorization, VMM checks the type of client, all Active Directory group memberships, and all user role memberships in VMM. User role settings determine the operations that VMM will perform on the user’s behalf on objects within the scope of the user roles. For more information, see Role-Based Security in VMM.
Ports and Protocols for Non-Windows Server–Based Hosts
This section summarizes VMM security requirements for hosts that are not running Windows Server operating systems. This section is intended to provide a quick reference for comparing security requirements for all host types managed by VMM. For instructions on configuring security for non-Windows Server–based hosts, see Configuring Security for a Managed VMware Environment in VMM.
For non-Windows Server–based hosts that are managed by a VMware VirtualCenter server, VMM performs supported management tasks by communicating with the VirtualCenter server. For these communications, VMM uses the WebServices protocol over default port 443. The WebServices protocol is based on HTTPS and uses Secure Sockets Layer (SSL) for encryption. Authentication uses credentials provided when the VirtualCenter server is added to VMM.
By default, VMM manages VMware environments in secure mode, which requires authentication of ESX Server hosts for communications on all protocols. If your environment does not require this level of authentication, you can turn secure mode off when you add the VirtualCenter server to VMM.
For file transfer operations, VMM connects directly to ESX Server hosts. To transfer files between hosts running non-embedded versions of ESX Server and Windows Server–based computers, VMM must have virtual machine delegate credentials in ESX Server. If you use the default root credentials as the delegate, you must enable SSH root logon on the host. As an added security measure, you can configure a lower-privilege account as the virtual machine delegate in ESX Server. The account must have administrative rights and permissions in ESX Server.
If you are managing your VMware environment in secure mode, certificate authentication also is required for ESX Server hosts. Additionally, for non-embedded versions of ESX Server, you must provide an RSH public key.
VMM uses a different file transfer protocol for embedded versions of ESX Server than for non-embedded versions:
For embedded versions of ESX Server (including VMware® ESX Server 3i), VMM uses HTTPS over default port 443. In secure mode, encryption using Secure Sockets Layer (SSL) requires certificate authentication.
For non-embedded versions of ESX Server (VMware® ESX Server 3.5, VMware® ESX Server 3.0.2), VMM uses SFTP over default port 22. In secure mode, encryption using Secure Shell (SSH) requires RSA public key authentication.
When you add a VirtualCenter server to VMM, VMM adds the ESX Server hosts with OK (Limited) status until you provide the required security information for each host. While the host is in OK (Limited) status, the VMM administrator can perform only actions that do not require file transfers between the host and Windows Server–based computers. For a list of actions allowed on hosts in OK (Limited) status and under full management, see Supported Virtual Machine Actions for ESX Server Hosts (http://go.microsoft.com/fwlink/?LinkID=134489).
Comparison of VMM Security Requirements by Host Type
The following table summarizes security requirements for each host type in VMM.
|Security Settings||Windows Server–Based Host in Trusted Active Directory Domain||Windows Server–Based Host in Non-Trusted Domain or Perimeter Network||Non-Windows Server–Based Host|
Connection type: Control data
Note To implement client authentication on a perimeter network, set up IPsec between the intranet and the perimeter network.
Connection type: File transfers
Embedded ESX Server (including VMware® ESX Server 3i):
Non-Embedded ESX Server (VMware® ESX Server 3.5, VMware® ESX Server 3.0.2):
VMM account requirements
Domain account with administrative rights on the host.
Local account with administrative rights on the host.
VirtualCenter server—VMM must log on under an account that is a member of the Administrator role in VirtualCenter. A domain account is not required. Use a dedicated account that is not used by any other users.
ESX Server hosts—To perform file transfer operations between the host and Windows Server–based computers, VMM must have virtual machine delegate credentials in ESX Server.
Important You must use the same virtual machine delegate (by default, vimuser) on all ESX Server 3i hosts that use the NFS datastore.
Use the Add Hosts Wizard to install an agent and add the host to VMM.
Alternatively, install the agent locally by using the Install Agent Locally Setup option, and then use Add Hosts Wizard to add the host and provide credentials.
Perimeter network: Requires local installation of the VMM agent on the host computer. By default, the installer generates a self-signed certificate for use in encrypting communications between VMM and the host. After installing the agent, use the Add Hosts Wizard to discover the host. At that point, you can substitute a CA-signed certificate.
Non-trusted domain: Add Hosts Wizard installs the agent remotely. The installation generates a self-signed certificate for use when encrypting communications between VMM and the host.
No VMM agent is installed on an ESX Server host. VMM proxies all operations except file transfers through the VirtualCenter server.
Virtual machines that have been deployed on a host on a perimeter network cannot be migrated to hosts in the internal network or to other hosts on a perimeter network.
These restrictions do not apply to hosts in a non-trusted domain.
Security During Agent Deployment
When you add a virtual machine host or library server to VMM, VMM remotely installs a VMM agent on the managed computer. The VMM agent deployment process uses both the Server Message Block (SMB) ports and the Remote Procedure Call (RPC) port (TCP 135) and the DCOM port range. You can use either SMB packet signing or IPsec to help secure the agent deployment process.
You can install the VMM agent locally on a host, discover the computer by adding the host to VMM, and then control the host using only the WS-Management port (default port 80) and BITS port (default port 443).
A host on a perimeter network requires local installation of the VMM agent. The installation generates a self-signed certificate for use in encrypting communications between VMM and the host. You can substitute a CA-signed certificate from a trusted source when you add the host by using the Add Hosts Wizard. All agent updates must be performed locally on the host before updating the agent in VMM.
Adding a host cluster can require additional configuration:
While a host cluster or a highly available library server is being added to VMM 2008, port 135 (DCOM port) must be open on the target computer (that is, the cluster node that was specified in the Add Hosts or Add Library Server wizard) to enable VMM to send remote WMI calls to that computer. The port needs to be open only while the cluster is being added to VMM. For information, see Connecting to WMI Remotely Starting with Windows Vista (http://go.microsoft.com/fwlink/?LinkId=153786).
Before you can add a host cluster in a disjointed namespace to a VMM server that is not in a disjointed namespace, you must add the DNS suffix for the host cluster to the TCP/IP connection settings on the VMM server.
Account Requirements for Adding and Removing Hosts
When you add or remove Hyper-V or Virtual Server hosts, you cannot use the same domain account that is used as the VMM service account.
When adding a host, VMM adds the VMM service account to the local Administrators group on the host computer, and VMM uses that account for communications with the host. While adding a host, VMM needs to be able to successfully roll back the process and remove the VMM agent from the host computer if agent installation fails. If the administrator were using the same account that VMM runs under, that operation would fail.
To remove a host, VMM must remove the VMM service account from the local Administrators group on the host computer, which would result in loss of communication with the host and a failed host removal if the administrator were using the VMM service account for the connection.
For those reasons, VMM blocks use of the same domain account that is used as the VMM service account while adding and removing hosts.
Security Considerations for Hyper-V, Virtual Server, and ESX Server Hosts
This section explains security requirements in VMM that are specific to the type of virtualization software—Hyper-V, Virtual Server, or ESX Server—that is running on the host. Your first line of defense for your hosts is to follow security best practices for Hyper-V, Virtual Server, and ESX Server.
VMM Security Considerations for Hyper-V Hosts
Perform the following security configuration tasks for Hyper-V hosts that VMM is managing.
Use VMM User Roles Instead of Hyper-V Role Definitions
If you use VMM to manage Hyper-V hosts, you will need to use delegated administration in VMM to configure the permissions that you had assigned through your Hyper-V role definitions.
VMM uses its own role-based security to authorize operations performed on virtual machines. In addition to the Administrator user role, this includes Delegated Administrator user roles, which enable almost all management tasks within the host groups and library servers defined by the role’s scope. You also can create self-service user roles, which define a set of specific operations that users can perform on their own virtual machines within a restricted environment. For more information about VMM user roles, see Role-Based Security in VMM.
Hyper-V security is based on Authorization Manager API (known as AZMan). Similarly to VMM’s delegated administration model, an administrator can configure a set of role objects and assign Active Directory user and group accounts to those roles. Each role can be granted a set of permissions for virtual machine access and management, and securable objects can be assigned to scopes, which determine the objects against which access checks are performed. When a Hyper-V host is added to VMM, VMM applies its own authorization layer, defined by the VMM user roles, to determine the actions that VMM administrators and self-service users can perform on the Hyper-V virtual machines while working in VMM. To do this, VMM creates its own AZMan authorization store on the host computer. In VMM 2008 R2, the method for implementing user roles in AZMan was changed to preserve role definitions and role memberships in the root scope of the Hyper-V authorization store while VMM is managing a Hyper-V host. In VMM 2008, the Hyper-V roles are not used while a host is managed by VMM.
Authorization for Hyper-V Virtual Machines in VMM 2008 R2
VMM 2008 R2 preserves changes to role definitions and role memberships in the root scope of the Hyper-V authorization store. The VMM agent overwrites all changes to other scopes. As a result, while a Hyper-V host is managed by VMM 2008 R2, access is determined by the union of all roles in the root scope plus the VMM role assigned to each virtual machine’s scope.
VMM 2008 R2 makes the following changes to authorization stores during agent installation on a Hyper-V host:
VMM creates its own AZman authorization store—HyperVAuthStore.xml, stored in the folder %ProgramData%\Microsoft Virtual Machine Manager—which will store the roles and memberships that are used to authorize virtual machine access and management through VMM.
VMM updates the registry entry HKLM\Software\Microsoft\Windows NT\CurrentVersion\Virtualization on the Hyper-V host to point Hyper-V to the VMM authorization store.
VMM imports any user roles and role memberships in the root scope of Hyper-V’s initialstore.xml authorization store into the VMM authorization store.
After VMM agent installation, the VMM user role refresher overwrites any changes to scopes other than the root scope every half hour. However, VMM preserves any role definitions and memberships that subsequently are added to the root scope.
When a Hyper-V host is removed from VMM 2008 R2, instead of pointing Hyper-V back to initialstore.xml, VMM saves a copy of the VMM authorization store on the Hyper-V computer, removes all user roles and scopes that were defined in VMM from the copy, and then points Hyper-V to that store.
Authorization for Hyper-V Virtual Machines in VMM 2008
When a Hyper-V host is added to VMM 2008, VMM creates its own authorization store without importing any role and membership settings from initialstore.xml on the Hyper-V computer, and then updates the registry so that Hyper-V points to the VMM authorization store.
VMM 2008 does not modify the existing Hyper-V role definitions; it simply ignores them. When a Hyper-V host is removed from VMM 2008, the registry is updated to point Hyper-V back to the original Hyper-V authorization store.
In a Hyper-V environment, the pre-release version of VMM 2008 R2 does not support the use of shared ISO image files when creating Hyper-V virtual machines. By default, when an ISO image file is added to a drive on a virtual machine, VMM attaches a copy of an ISO image file to the virtual machine instead of sharing the image file. In the pre-release version of VMM 2008 R2, if a user instead chooses the Share image file instead of copying it option on a Hyper-V virtual machine, the operation will fail.
Enabling Shared ISO Image Files for Hyper-V Virtual Machines
To enable sharing of ISO image files on Hyper-V virtual machines, you must perform the following configurations:
You must specify an Active Directory domain account for the VMM Server service account.
You must configure share permissions and NTFS permissions on the library shares. The VMM Server service account and the machine account for each server that is running Hyper-V must have Read permission on the shares.
To allow a Hyper-V virtual machine to access an ISO file on a shared folder in a domain environment, you must configure the server running Hyper-V for constrained delegation. Configure the computer account of the server running Hyper-V to present delegated credentials for the Common Internet File System (CIFS) service type.
In VMM 2008, use of a shared ISO image file is not supported for self-service users. Instead, VMM attaches a copy of the ISO image file to the new virtual machine. VMM 2008 R2 supports shared ISO images for self-service.
Virtual Machine Connections
When you add a Hyper-V host to VMM, VMM enables remote connections to virtual machines using the default remote connection port for Hyper-V hosts (by default, port 2179), which is a configurable general setting in VMM. You can change the remote connection port for a host while adding the host to VMM or by modifying the setting on the Remote tab of the host properties. For instructions about configuring the default remote connection ports in VMM, see How to Configure Remote Access to Virtual Machines (http://go.microsoft.com/fwlink/?LinkId=162936).
Hyper-V Manager uses VMConnect for connections to virtual machines from remote consoles. VMConnect uses the Remote Desktop Protocol (RDP) listener port to provide console connections to a virtual machine via the parent partition in Hyper-V.
VMM uses the same Single Port Listener technology to provide the administrator with live thumbnails in the VMM Administrator Console. The VMM Self-Service Portal and VirtualMachineViewer.exe also use the Single Port Listener technology of RDP.
However, the Single Port Listener for RDP is available only for console connections from a computer that is running one of the following operating systems: Windows Vista with Service Pack 1 (SP1), Windows Server 2008, or Windows Server 2008 R2. If the client computer is running any other operating system, VMM connects directly to the guest operating system that is running inside the virtual machine by using standard RDP. In that case, VMM can make the connection only if the virtual machine is in a running state and virtual guest services are installed on the virtual machine.
Additional Resources for Hyper-V Hosts
VMM Security Considerations for Virtual Server Hosts
When you take a Virtual Server host under management by VMM, you only need to configure remote connections. The security model for VMM fully supports the rights and permissions assigned in Virtual Server.
Access to Virtual Machines
VMM fully supports the Virtual Server security model for virtual machines. For each virtual machine that is created on Virtual Server hosts, VMM maintains a virtual machine configuration (.vmc) file that reflects the rights and permissions that are assigned through VMM role-based security. VMM does not use the virtual machine configuration file. The file is maintained to enable seamless management of the virtual machine in Virtual Server by using the Virtual Server Administration Website.
Virtual Machine Connections
When you add a Virtual Server host to VMM, VMM enables virtual machine connections by using Virtual Machine Remote Control (VMRC) in Virtual Server. By default, VMM uses the default remote connections port for VMRC connections (port 5900), which is a configurable general setting in VMM. You can change the remote connection port for a host while adding the host to VMM or by modifying settings on the Remote tab of the host properties. For instructions about configuring default remote connection ports for VMM, see How to Configure Remote Access to Virtual Machines (http://go.microsoft.com/fwlink/?LinkId=162936).
Encryption is not enabled for VMRC connections when you add a Virtual Server host to VMM. After you add the host, it is recommended that you implement Secure Sockets Layer (SSL) encryption, particularly if you use Basic authentication, which transmits passwords in plain text. By modifying settings on the Remote tab of the host properties, you can enable SSL by using an unsigned certificate from Virtual Server. For instructions, see How to Change Remote Connections to Virtual Machines on a Host (http://go.microsoft.com/fwlink/?LinkID=123607).
By default, VMM does not allow multiple users to connect to a virtual machine by using VMRC. To support training and test lab scenarios where one user must demonstrate a task to other users connected to the same remote session, VMRC can allow multiple users to connect to a virtual machine, and each user can access the guest operating system without knowledge of the other users. VMM turns off this feature by default. If you want to turn it on, update another setting on the Remote tab of the host properties.
Additional Resources for Virtual Server Hosts
VMM Security Considerations for ESX Server Hosts
ESX Server hosts require the following security configuration after adding the VirtualCenter server to VMM.
Access to Virtual Machines
As noted earlier, when you add a VirtualCenter server to VMM, VMM adds each ESX Server host with OK (Limited) status until you configure the security information that VMM needs to perform file transfers between the host and Windows Server–based computers. The unsupported virtual machine operations include creating a virtual machine from a virtual hard disk that is stored on a library server or storing a virtual machine in the VMM library.
When a host has OK (Limited) status, VMM administrators are limited to a subset of virtual machine operations that do not require that type of file transfer. For a list of supported virtual machine operations in VMM under limited or full management, see Supported Virtual Machine Actions for ESX Server Hosts (http://go.microsoft.com/fwlink/?LinkID=134489).
To change a host’s status to OK and enable all management capabilities that VMM supports, VMM must have virtual machine delegate credentials on the host. Additionally, if you are managing your VMware environment in secure mode, VMM must be able to identify the host by its certificate. For non-embedded versions of ESX Server, the host’s RSH public key also is required. For detailed instructions about configuring security for ESX Server hosts, see Configuring Security for a Managed VMware Environment in VMM.
Access to VMware Virtual Machines Through the VMM Self-Service Portal
To manage VMware virtual machines, users of the VMM Self-Service Portal must download and install a VMware ActiveX control. This control must be downloaded through a secure SSL channel. VMM connects to the VMware host by using SSL. However, to ensure that users can download and install the ActiveX control, you must enable SSL on the VMware host computers. Alternatively, you can install the Virtual Infrastructure client on the client machine, which will also install the ActiveX control, thereby eliminating the need to download the ActiveX control from the host.
Virtual Machine Connections
For virtual machine connections, ESX Server hosts use VMware Control. ESX Server hosts the control, which runs on the user client.