Azure for secure worldwide public sector cloud adoption
Microsoft Azure is a multi-tenant cloud services platform that government agencies can use to deploy various solutions. A multi-tenant cloud platform implies that multiple customer applications and data are stored on the same physical hardware. Azure uses logical isolation to segregate each customer's applications and data. This approach provides the scale and economic benefits of multi-tenant cloud services while rigorously helping prevent customers from accessing one another's data or applications. Azure is available globally in more than 60 regions and can be used by government entities worldwide to meet rigorous data protection requirements across a broad spectrum of data classifications, including unclassified and classified data.
This article addresses common data residency, security, and isolation concerns pertinent to worldwide public sector customers. It also explores technologies available in Azure to safeguard both unclassified and classified workloads in the public multi-tenant cloud in combination with Azure Stack Hub and Azure Stack Edge deployed on-premises and at the edge.
Microsoft Azure provides strong customer commitments regarding data residency and transfer policies. Most Azure services enable the customer to specify the deployment region. For those services, Microsoft will not store customer data outside the customer specified geography. Customers can use extensive and robust data encryption options to help safeguard their data in Azure and control who can access it.
Listed below are some of the options available to customers to safeguard their data in Azure:
- Customers can choose to store their most sensitive customer content in services that store customer data at rest in Geo.
- Customers can obtain further protection by encrypting data with their own key using Azure Key Vault.
- While customers cannot control the precise network path for data in transit, data encryption in transit helps protect data from interception.
- Azure is a 24x7 globally operated service; however, support and troubleshooting rarely require access to customer data.
- Customers who want added control for support and troubleshooting can use Customer Lockbox for Azure to approve or deny access to their data.
- Microsoft will notify customers of any breach of customer or personal data within 72 hours of incident declaration.
- Customers can monitor potential threats and respond to incidents on their own using Azure Security Center.
Using Azure data protection technologies and intelligent edge capabilities from the Azure Stack portfolio of products, customers can process confidential and secret data in secure isolated infrastructure within the public multi-tenant cloud or top secret data on premises and at the edge under the customer’s full operational control.
Governments around the world are in the process of a digital transformation, actively investigating solutions and selecting architectures that will help them transition many of their workloads to the cloud. There are many drivers behind the digital transformation, including the need to engage citizens, empower employees, transform government services, and optimize government operations. Governments across the world are also looking to improve their cybersecurity posture to secure their assets and counter the evolving threat landscape.
For governments and the public sector industry worldwide, Microsoft provides Azure – a public multi-tenant cloud services platform – that government agencies can use to deploy various solutions. A multi-tenant cloud services platform implies that multiple customer applications and data are stored on the same physical hardware. Azure uses logical isolation to segregate each customer's applications and data. This approach provides the scale and economic benefits of multi-tenant cloud services while rigorously helping prevent customers from accessing one another's data or applications.
A hyperscale public cloud provides resiliency in time of natural disaster and warfare. The cloud provides capacity for failover redundancy and empowers sovereign nations with flexibility regarding global resiliency planning. Hyperscale public cloud also offers a feature-rich environment incorporating the latest cloud innovations such as artificial intelligence, machine learning, IoT services, intelligent edge, and many more to help government customers increase efficiency and unlock insights into their operations and performance.
Using Azure’s public cloud capabilities, customers benefit from rapid feature growth, resiliency, and the cost-effective operation of the hyperscale cloud while still obtaining the levels of isolation, security, and confidence required to handle workloads across a broad spectrum of data classifications, including unclassified and classified data. Using Azure data protection technologies and intelligent edge capabilities from the Azure Stack portfolio of products, customers can process confidential and secret data in secure isolated infrastructure within the public multi-tenant cloud or top secret data on-premises and at the edge under the customer’s full operational control.
This article addresses common data residency, security, and isolation concerns pertinent to worldwide public sector customers. It also explores technologies available in Azure to help safeguard unclassified, confidential, and secret workloads in the public multi-tenant cloud in combination with Azure Stack products deployed on-premises and at the edge for fully disconnected scenarios involving top secret data. Given that unclassified workloads comprise most scenarios involved in worldwide public sector digital transformation, Microsoft recommends that customers start their cloud journey with unclassified workloads and then progress to classified workloads of increasing data sensitivity.
Established privacy regulations are silent on data residency and data location, and permit data transfers in accordance with approved mechanisms such as the EU Standard Contractual Clauses (also known as EU Model Clauses). Microsoft commits contractually in the Online Services Terms Data Protection Addendum (DPA) that all potential transfers of customer data out of the EU, European Economic Area (EEA), and Switzerland shall be governed by the EU Model Clauses. Microsoft will abide by the requirements of the EEA and Swiss data protection laws regarding the collection, use, transfer, retention, and other processing of personal data from the EEA and Switzerland. All transfers of personal data are subject to appropriate safeguards and documentation requirements. However, many customers considering cloud adoption are seeking assurances about customer and personal data being kept within the geographic boundaries corresponding to customer operations or location of customer’s end users.
Data sovereignty implies data residency; however, it also introduces rules and requirements that define who has control over and access to customer data stored in the cloud. In many cases, data sovereignty mandates that customer data be subject to the laws and legal jurisdiction of the country or region in which data resides. These laws can have direct implications on data access even for platform maintenance or customer-initiated support requests. Customers can use Azure public multi-tenant cloud in combination with Azure Stack products for on-premises and edge solutions to meet their data sovereignty requirements, as described later in this article. These other products can be deployed to put customers solely in control of their data, including storage, processing, transmission, and remote access.
Among several data categories and definitions that Microsoft established for cloud services, the following four categories are discussed in this article:
- Customer data is all data that customers provide to Microsoft to manage on customer’s behalf through customer’s use of Microsoft online services.
- Customer content is a subset of customer data and includes, for example, the content stored in a customer’s Azure Storage account.
- Personal data means any information associated with a specific natural person, for example, names and contact information of customer’s end users. However, personal data could also include data that is not customer data, such as user ID that Azure can generate and assign to each customer administrator – such personal data is considered pseudonymous because it cannot identify an individual on its own.
- Support and consulting data mean all data provided by customer to Microsoft to obtain Support or Professional Services.
The following sections address key cloud implications for data residency and the fundamental principles guiding Microsoft’s safeguarding of customer data at rest, in transit, and as part of customer-initiated support requests.
Data at rest
Microsoft provides transparent insight into data location for all online services available to customers from “Where your data is located” page – expand Cloud service data residency and transfer policies section to reveal links for individual online services. Customers who want to ensure their customer data is stored only in Geo should select from the many regional services that make this commitment.
Regional vs. non-regional services
Microsoft Azure provides strong customer commitments regarding data residency and transfer policies:
- Data storage for regional services: Most Azure services are deployed regionally and enable the customer to specify the region into which the service will be deployed. Microsoft will not store customer data outside the customer-specified Geo except for a few regional services and Preview services as described on the Azure data location page. This commitment helps ensure that customer data stored in a given region will remain in the corresponding Geo and will not be moved to another Geo for most regional services. For service availability, see Products available by region.
- Data storage for non-regional services: Certain Azure services do not enable the customer to specify the region where the services will be deployed as described on the data location page. For a complete list of non-regional services, see Products available by region.
Customer data in an Azure Storage account is always replicated to help ensure durability and high availability. Azure Storage copies customer data to protect it from transient hardware failures, network or power outages, and even massive natural disasters. Customers can typically choose to replicate their data within the same data center, across availability zones within the same region, or across geographically separated regions. Specifically, when creating a storage account, customers can select one of the following redundancy options:
- Locally redundant storage (LRS)
- Zone-redundant storage (ZRS)
- Geo-redundant storage (GRS)
- Geo-zone-redundant storage (GZRS)
Data in an Azure Storage account is always replicated three times in the primary region. Azure Storage provides LRS and ZRS redundancy options for replicating data in the primary region. For applications requiring high availability, customers can choose geo-replication to a secondary region that is hundreds of kilometers away from the primary region. Azure Storage offers GRS and GZRS options for copying data to a secondary region. More options are available to customers for configuring read access (RA) to the secondary region (RA-GRS and RA-GZRS), as explained in Read access to data in the secondary region.
Azure Storage redundancy options can have implications on data residency as Azure relies on paired regions to deliver geo-redundant storage (GRS). For example, customers concerned about geo-replication across regions that span country boundaries, may want to choose LRS or ZRS to keep Azure Storage data at rest within the geographic boundaries of the country in which the primary region is located. Similarly, geo replication for Azure SQL Database can be obtained by configuring asynchronous replication of transactions to any region in the world, although it is recommended that paired regions be used for this purpose as well. If customers need to keep relational data inside the geographic boundaries of their country/region, they should not configure Azure SQL Database asynchronous replication to a region outside that country.
As described on the data location page, most Azure regional services honor the data at rest commitment to ensure that customer data remains within the geographic boundary where the corresponding service is deployed. A handful of exceptions to this rule are noted on the data location page. Customers should review these exceptions to determine if the type of data stored outside their chosen deployment Geo meets their needs.
Non-regional Azure services do not enable customers to specify the region where the services will be deployed. Some non-regional services do not store customer data at all but merely provide global routing functions such as Azure Traffic Manager or Azure DNS. Other non-regional services are intended for data caching at edge locations around the globe, such as the Content Delivery Network – such services are optional and customers should not use them for sensitive customer content they wish to keep in Geo. One non-regional service that warrants extra discussion is Azure Active Directory, which is discussed in the next section.
Customer data in Azure Active Directory
Azure Active Directory (Azure AD) is a non-regional service that may store identity data globally, except for Azure AD deployments in:
- The United States, where identity data is stored solely in the United States.
- Europe, where Azure AD keeps most of the identity data within European datacenters except as noted in Identity data storage for European customers in Azure Active Directory.
- Australia and New Zealand, where identity data is stored in Australia except as noted in Customer data storage for Australian and New Zealand customers in Azure Active Directory.
Azure AD provides a dashboard with transparent insight into data location for every Azure AD component service. Among other features, Azure AD is an identity management service that stores directory data for customer’s Azure administrators, including user personal data categorized as End User Identifiable Information (EUII), for example, names, email addresses, and so on. In Azure AD, customers can create User, Group, Device, Application, and other entities using various attribute types such as Integer, DateTime, Binary, String (limited to 256 characters), and so on. Azure AD is not intended to store customer content and it is not possible to store blobs, files, database records, and similar structures in Azure AD. Moreover, Azure AD is not intended to be an identity management service for customer’s external end users – Azure AD B2C should be used for that purpose.
Azure AD implements extensive data protection features, including tenant isolation and access control, data encryption in transit, secrets encryption and management, disk level encryption, advanced cryptographic algorithms used by various Azure AD components, data operational considerations for insider access, and more. Detailed information is available from a whitepaper Active Directory Data Security Considerations.
Generating pseudonymous data for internal systems
Personal data is defined broadly. It includes not just customer data but also unique personal identifiers such as Probably Unique Identifier (PUID) and Globally Unique Identifier (GUID), the latter being often labeled as Universally Unique Identifier (UUID). These unique personal identifiers are pseudonymous identifiers. This type of information is generated automatically to track users who interact directly with Azure services, such as customer’s administrators. For example, PUID is a random string generated programmatically via a combination of characters and digits to provide a high probability of uniqueness. Pseudonymous identifiers are stored in centralized internal Azure systems.
Whereas EUII represents data that could be used on its own to identify a user (for example, user name, display name, user principal name, or even user-specific IP address), pseudonymous identifiers are considered pseudonymous because they cannot identify an individual on their own. Pseudonymous identifiers do not contain any information uploaded or created by the customer.
Data in transit
While customers cannot control the precise network path for data in transit, data encryption in transit helps protect data from interception.
Data in transit applies to the following scenarios involving data traveling between:
- Customer’s end users and Azure service
- Customer’s on-premises datacenter and Azure region
- Microsoft datacenters as part of expected Azure service operation
While data in transit between two points within the Geo will typically remain in Geo, it is not possible to guarantee this 100% of the time because of the way that networks automatically reroute traffic to avoid congestion or bypass other interruptions. That said, data in transit can be protected through encryption as detailed below and in Data encryption in transit section.
Customer’s end users connection to Azure service
Most customers will connect to Azure over the Internet, and the precise routing of network traffic will depend on the many network providers that contribute to Internet infrastructure. As stated in the Microsoft Online Services Terms Data Protection Addendum (DPA), Microsoft does not control or limit the regions from which customer or customer’s end users may access or move customer data. Customers can increase security by enabling encryption in transit. For example, customers can use Azure Application Gateway to configure end-to-end encryption of traffic. As described in Data encryption in transit section, Azure uses the Transport Layer Security (TLS) protocol to help protect data when it is traveling between customers and Azure services. However, Microsoft cannot control network traffic paths corresponding to customer’s end-user interaction with Azure.
Customer’s datacenter connection to Azure region
Virtual Network (VNet) provides a means for Azure virtual machines (VMs) to act as part of a customer’s internal (on-premises) network. Customers have options to securely connect to a VNet from their on-premises infrastructure – choose an IPSec protected VPN (for example, point-to-site VPN or site-to-site VPN) or a private connection by using Azure ExpressRoute with several data encryption options.
- IPSec protected VPN uses an encrypted tunnel established across the public Internet, which means that customers need to rely on the local Internet service providers for any network-related assurances.
- ExpressRoute allows customers to create private connections between Microsoft datacenters and their on-premises infrastructure or colocation facility. ExpressRoute connections do not go over the public Internet and offer lower latency and higher reliability than IPSec protected VPN connections. ExpressRoute locations are the entry points to Microsoft’s global network backbone and they may or may not match the location of Azure regions. For example, customers can connect to Microsoft in Amsterdam through ExpressRoute and have access to all Azure cloud services hosted in Northern and Western Europe. However, it’s also possible to have access to the same Azure regions from ExpressRoute connections located elsewhere in the world. Once the network traffic enters the Microsoft backbone, it is guaranteed to traverse that private networking infrastructure instead of the public Internet.
Traffic across Microsoft global network backbone
As described in Data at rest section, Azure services such as Storage and SQL Database can be configured for geo-replication to help ensure durability and high availability especially for disaster recovery scenarios. Azure relies on paired regions to deliver geo-redundant storage (GRS), and paired regions are also recommended when configuring active geo-replication for Azure SQL Database. Paired regions are located within the same Geo.
Inter-region traffic is encrypted using Media Access Control Security (MACsec), which protects network traffic at the data link layer (Network Layer 2) and relies on AES-128 block cipher for encryption. This traffic stays entirely within the Microsoft global network backbone and never enters the public Internet. The backbone is one of the largest in the world with more than 160,000 km of lit fiber optic and undersea cable systems. However, network traffic is not guaranteed to always follow the same path from one Azure region to another. To provide the reliability needed for the Azure cloud, Microsoft has many physical networking paths with automatic routing around failures for optimal reliability. Therefore, Microsoft cannot guarantee that network traffic traversing between Azure regions will always be confined to the corresponding Geo. In networking infrastructure disruptions, Microsoft can reroute the encrypted network traffic across its private backbone to ensure service availability and best possible performance.
Data for customer support and troubleshooting
Azure is a 24x7 globally operated service; however, support and troubleshooting rarely requires access to customer data. Customers who want added control for support and troubleshooting can use Customer Lockbox for Azure to approve or deny access to their data.
Microsoft Azure support is available in markets where Azure is offered. It is staffed globally to accommodate 24x7 access to support engineers via email and phone for technical support. Customers can create and manage support requests in the Azure portal. As needed, frontline support engineers can escalate customer requests to Azure DevOps personnel responsible for Azure service development and operations. These Azure DevOps engineers are also staffed globally. The same production access controls and processes are imposed on all Microsoft engineers, which include support staff comprised of both Microsoft full-time employees and subprocessors/vendors.
As explained in Data encryption at rest section, customer data is encrypted at rest by default when stored in Azure and customers can control their own encryption keys in Azure Key Vault. Moreover, access to customer data is not needed to resolve most customer support requests. Microsoft engineers rely heavily on logs to provide customer support. As described in Insider data access section, Azure has controls in place to restrict access to customer data for support and troubleshooting scenarios should that access be necessary. For example, Just-in-Time (JIT) access provisions restrict access to production systems to Microsoft engineers who are authorized to be in that role and were granted temporary access credentials. As part of the support workflow, Customer Lockbox puts customers in charge of approving or denying access to customer data by Microsoft engineers. When combined, these Azure technologies and processes (data encryption, JIT, and Customer Lockbox) provide appropriate risk mitigation to safeguard confidentiality and integrity of customer data.
Government customers worldwide expect to be fully in control of protecting their data in the cloud. As described in the next section, Azure provides extensive options for data encryption through its entire lifecycle (at rest, in transit, and in use), including customer control of encryption keys.
Azure has extensive support to safeguard customer data using data encryption. Customers who require extra security for their most sensitive customer data stored in Azure services can encrypt it using their own encryption keys they control in Azure Key Vault. While customers cannot control the precise network path for data in transit, data encryption in transit helps protect data from interception. Azure supports the following data encryption models:
- Server-side encryption that uses service-managed keys, customer-managed keys (CMK) in Azure, or CMK in customer-controlled hardware.
- Client-side encryption that enables customers to manage and store keys on-premises or in another secure location.
Data encryption provides isolation assurances that are tied directly to encryption key access. Since Azure uses strong ciphers for data encryption, only entities with access to encryption keys can have access to data. Deleting or revoking encryption keys renders the corresponding data inaccessible.
Encryption key management
Proper protection and management of encryption keys is essential for data security. Azure Key Vault is a cloud service for securely storing and managing secrets. The Key Vault service supports two resource types:
- Vault supports software-protected and hardware security module (HSM)-protected secrets, keys, and certificates.
- Managed HSM supports only HSM-protected cryptographic keys.
Key Vault enables customers to store their encryption keys in hardware security modules (HSMs) that are FIPS 140-2 validated. With Azure Key Vault, customers can import or generate encryption keys in HSMs, ensuring that keys never leave the HSM protection boundary to support bring your own key (BYOK) scenarios. Keys generated inside the Azure Key Vault HSMs are not exportable – there can be no clear-text version of the key outside the HSMs. This binding is enforced by the underlying HSM.
Azure Key Vault is designed, deployed, and operated such that Microsoft and its agents do not see or extract customer cryptographic keys.
For more information, see Azure Key Vault.
Data encryption in transit
Azure provides many options for encrypting data in transit. Data encryption in transit isolates customer network traffic from other traffic and helps protect data from interception. For more information, see Data encryption in transit.
Data encryption at rest
Azure provides extensive options for encrypting data at rest to help customers safeguard their data and meet their compliance needs using both Microsoft-managed encryption keys and customer-managed encryption keys. This process relies on multiple encryption keys and services such as Azure Key Vault and Azure Active Directory to ensure secure key access and centralized key management. For more information about Azure Storage encryption and Azure Disk encryption, see Data encryption at rest.
Azure SQL Database provides transparent data encryption (TDE) at rest by default. TDE performs real-time encryption and decryption operations on the data and log files. Database Encryption Key (DEK) is a symmetric key stored in the database boot record for availability during recovery. It is secured via a certificate stored in the master database of the server or an asymmetric key called TDE Protector stored under customer control in Azure Key Vault, which is Azure’s cloud-based external key management system. Key Vault supports bring your own key (BYOK), which enables customers to store TDE Protector in Key Vault and control key management tasks including key permissions, rotation, deletion, enabling auditing/reporting on all TDE Protectors, and so on. The key can be generated by the Key Vault, imported, or transferred to the Key Vault from an on-premises HSM device. Customers can also use the Always Encrypted feature of Azure SQL Database, which is designed specifically to help protect sensitive data by allowing clients to encrypt data inside client applications and never reveal the encryption keys to the database engine. In this manner, Always Encrypted provides separation between those users who own the data (and can view it) and those users who manage the data (but should have no access).
Data encryption in use
Microsoft enables customers to protect their data throughout its entire lifecycle: at rest, in transit, and in use. Azure confidential computing and Homomorphic encryption are two techniques that safeguard customer data while it is processed in the cloud.
Azure confidential computing
Azure confidential computing is a set of data security capabilities that offers encryption of data while in use. This approach means that data can be processed in the cloud with the assurance that it is always under customer control. Confidential computing ensures that when data is in the clear, which is needed for efficient data processing in memory, the data is protected inside a hardware-based trusted execution environment (TEE, also known as an enclave), as depicted in Figure 1. TEE helps ensure that there is no way to view data or the operations from outside the enclave and that only the application designer has access to TEE data; access is denied to everyone else including Azure administrators. Moreover, TEE helps ensure that only authorized code is permitted to access data. If the code is altered or tampered with, the operations are denied, and the environment is disabled.
Figure 1. Trusted execution environment protection
Azure DCsv2-series virtual machines have the latest generation of Intel Xeon processors with Intel Software Guard Extensions (SGX) technology, which provides a hardware-based TEE. Intel SGX isolates a portion of physical memory to create an enclave where select code and data are protected from viewing or modification. The protection offered by Intel SGX, when used appropriately by application developers, can prevent compromise due to attacks from privileged software and many hardware-based attacks. An application using Intel SGX needs to be refactored into trusted and untrusted components. The untrusted part of the application sets up the enclave, which then allows the trusted part to run inside the enclave. No other code, irrespective of the privilege level, has access to the code executing within the enclave or the data associated with enclave code. Design best practices call for the trusted partition to contain just the minimum amount of content required to protect customer’s secrets. For more information, see Application development on Intel SGX.
Based on customer feedback, Microsoft has started to invest in higher-level scenarios for Azure confidential computing. Customers can review the scenario recommendations as a starting point for developing their own applications using confidential computing services and frameworks.
Homomorphic encryption refers to a special type of encryption technology that allows for computations to be performed on encrypted data, without requiring access to a key needed to decrypt the data. The results of the computation are encrypted and can be revealed only by the owner of the encryption key. In this manner, only the encrypted data are processed in the cloud and only the customer can reveal the results of the computation.
To help customers adopt homomorphic encryption, Microsoft SEAL provides a set of encryption libraries that allow computations to be performed directly on encrypted data. This approach enables customers to build end-to-end encrypted data storage and compute services where the customer never needs to share their encryption keys with the cloud service. Microsoft SEAL aims to make homomorphic encryption easy to use and available to everyone. It provides a simple and convenient API and comes with several detailed examples demonstrating how the library can be used correctly and securely.
Data encryption in the cloud is an important risk mitigation requirement expected by government customers worldwide. As described in this section, Azure helps customers protect their data through its entire lifecycle whether at rest, in transit, or even in use. Moreover, Azure offers comprehensive encryption key management to help customers control their keys in the cloud, including key permissions, rotation, deletion, and so on. End-to-end data encryption using advanced ciphers is fundamental to ensuring confidentiality and integrity of customer data in the cloud. However, customers also expect assurances regarding any potential customer data access by Microsoft engineers for service maintenance, customer support, or other scenarios. These controls are described in the next section.
Insider data access
Insider threat is characterized as potential for providing back-door connections and cloud service provider (CSP) privileged administrator access to customer’s systems and data. Microsoft provides strong customer commitments regarding who can access customer data and on what terms. Access to customer data by Microsoft operations and support personnel is denied by default. Access to customer data is not needed to operate Azure. Moreover, for most support scenarios involving customer troubleshooting tickets, access to customer data is not needed.
No default access rights and Just-in-Time (JIT) access provisions reduce greatly the risks associated with traditional on-premises administrator elevated access rights that typically persist throughout the duration of employment. Microsoft makes it considerably more difficult for malicious insiders to tamper with customer applications and data. The same access control restrictions and processes are imposed on all Microsoft engineers, including both full-time employees and subprocessors/vendors.
For more information on how Microsoft restricts insider access to customer data, see Restrictions on insider access.
Government requests for customer data
Government requests for customer data follow a strict procedure according to Microsoft practices for responding to government requests. Microsoft takes strong measures to help protect customer data from inappropriate access or use by unauthorized persons. These measures include restricting access by Microsoft personnel and subcontractors and carefully defining requirements for responding to government requests for customer data. Microsoft ensures that there are no back-door channels and no direct or unfettered government access to customer data. Microsoft imposes special requirements for government and law enforcement requests for customer data.
As stated in the Microsoft Online Services Terms Data Protection Addendum (DPA), Microsoft will not disclose customer data to law enforcement unless required by law. If law enforcement contacts Microsoft with a demand for customer data, Microsoft will attempt to redirect the law enforcement agency to request that data directly from the customer. If compelled to disclose customer data to law enforcement, Microsoft will promptly notify the customer and provide a copy of the demand unless legally prohibited from doing so.
Government requests for customer data must comply with applicable laws.
- A subpoena or its local equivalent is required to request non-content data.
- A warrant, court order, or its local equivalent is required for content data.
Every year, Microsoft rejects many law enforcement requests for customer data. Challenges to government requests can take many forms. In many of these cases, Microsoft simply informs the requesting government that it is unable to disclose the requested information and explains the reason for rejecting the request. Where appropriate, Microsoft challenges requests in court.
CLOUD Act provisions
The CLOUD Act is a United States law that was enacted in March 2018. For more information, see Microsoft’s blog post and the follow-up blog post that describes Microsoft’s call for principle-based international agreements governing law enforcement access to data. Key points of interest to government customers procuring Azure services are captured below.
- The CLOUD Act enables governments to negotiate new government-to-government agreements that will result in greater transparency and certainty for how information is disclosed to law enforcement agencies across international borders.
- The CLOUD Act is not a mechanism for greater government surveillance; it is a mechanism toward ensuring that customer data is ultimately protected by the laws of each customer’s home country/region while continuing to facilitate lawful access to evidence for legitimate criminal investigations. Law enforcement in the US still needs to obtain a warrant demonstrating probable cause of a crime from an independent court before seeking the contents of communications. The CLOUD Act requires similar protections for other countries seeking bilateral agreements.
- While the CLOUD Act creates new rights under new international agreements, it also preserves the common law right of cloud service providers to go to court to challenge search warrants when there is a conflict of laws – even without these new treaties in place.
- Microsoft retains the legal right to object to a law enforcement order in the United States where the order clearly conflicts with the laws of the country/region where customer data is hosted. Microsoft will continue to carefully evaluate every law enforcement request and exercise its rights to protect customers where appropriate.
- For legitimate enterprise customers, US law enforcement will, in most instances, now go directly to the customer rather than Microsoft for information requests.
Microsoft does not disclose extra data as a result of the CLOUD Act. This law does not practically change any of the legal and privacy protections that previously applied to law enforcement requests for data – and those protections continue to apply. Microsoft adheres to the same principles and customer commitments related to government demands for user data.
Most government customers have requirements in place for handling security incidents, including data breach notifications. Microsoft has a mature security and privacy incident management process in place that is described in the next section.
Microsoft will notify customers of any breach of customer or personal data within 72 hours of incident declaration. Customers can monitor potential threats and respond to incidents on their own using Azure Security Center.
Microsoft is responsible for monitoring and remediating security and availability incidents affecting the Azure platform and notifying customers of any security breaches involving customer or personal data. Microsoft Azure has a mature security and privacy incident management process that is used for this purpose. Customers are responsible for monitoring their own resources provisioned in Azure, as described in the next section.
The NIST SP 800-145 standard defines the following cloud service models: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). The shared responsibility model for cloud computing is depicted in Figure 2. With on-premises deployment in their own datacenter, customers assume the responsibility for all layers in the stack. As workloads get migrated to the cloud, Microsoft assumes progressively more responsibility depending on the cloud service model. For example, with the IaaS model, Microsoft’s responsibility ends at the Hypervisor layer, and customers are responsible for all layers above the virtualization layer, including maintaining the base operating system in guest Virtual Machines.
Figure 2. Shared responsibility model in cloud computing
In line with the shared responsibility model, Microsoft does not inspect, approve, or monitor individual customer applications deployed on Azure. For example, Microsoft does not know what firewall ports need to be open for customer’s application to function correctly, what the back-end database schema looks like, what constitutes normal network traffic for the application, and so on. Microsoft has extensive monitoring infrastructure in place for the cloud platform; however, customers are responsible for provisioning and monitoring their own resources in Azure. Customers can deploy a range of Azure services to monitor and safeguard their applications and data, as described in the next section.
Essential Azure services for extra protection
Azure provides essential services that customers can use to gain in-depth insight into their provisioned Azure resources and get alerted about suspicious activity, including outside attacks aimed at their applications and data. The Azure Security Benchmark provides security recommendations and implementation details to help customers improve the security posture with respect to Azure resources.
For more information about essential Azure services for extra protection, see Customer monitoring of Azure resources.
Breach notification process
Security incident response, including breach notification, is a subset of Microsoft’s overall incident management plan for Azure. All Microsoft employees are trained to identify and escalate potential security incidents. A dedicated team of security engineers within the Microsoft Security Response Center (MSRC) is responsible for always managing the security incident response for Azure. Microsoft follows a five-step incident response process when managing both security and availability incidents for Azure services. The process includes the following stages:
- Stabilize and recover
The goal of this process is to restore normal service operations and security as quickly as possible after an issue is detected, and an investigation started. Moreover, Microsoft enables customers to investigate, manage, and respond to security incidents in their Azure subscriptions. For more information, see Incident management implementation guidance: Azure and Office 365.
If during the investigation of a security or privacy event, Microsoft becomes aware that customer or personal data has been exposed or accessed by an unauthorized party, the security incident manager is required to trigger the incident notification subprocess in consultation with Microsoft legal affairs division. This subprocess is designed to fulfill incident notification requirements stipulated in Azure customer contracts (see Security Incident Notification in the Microsoft Online Services Terms Data Protection Addendum). Customer notification and external reporting obligations (if any) are triggered by a security incident being declared. The customer notification subprocess begins in parallel with security incident investigation and mitigation phases to help minimize any impact resulting from the security incident.
Microsoft will notify customers, Data Protection Authorities, and data subjects (each as applicable) of any breach of customer or personal data within 72 hours of incident declaration. The notification process upon a declared security or privacy incident will occur as expeditiously as possible while still considering the security risks of proceeding quickly. In practice, this approach means that most notifications will take place well before the 72-hr deadline to which Microsoft commits contractually. Notification of a security or privacy incident will be delivered to one or more of customer’s administrators by any means Microsoft selects, including via email. Customers should provide security contact details for their Azure subscription – this information will be used by Microsoft to contact the customer if the MSRC discovers that customer data has been exposed or accessed by an unlawful or unauthorized party. To ensure that notification can be delivered successfully, it is the customer’s responsibility to maintain correct administrative contact information for each applicable subscription.
Most Azure security and privacy investigations do not result in declared security incidents. Most external threats do not lead to breaches of customer or personal data because of extensive platform security measures that Microsoft has in place. Microsoft has deployed extensive monitoring and diagnostics infrastructure throughout Azure that relies on big-data analytics and machine learning to get insight into the platform health, including real-time threat intelligence. While Microsoft takes all platform attacks seriously, it would be impractical to notify customers of potential attacks at the platform level.
Aside from controls implemented by Microsoft to safeguard customer data, government customers deployed on Azure derive considerable benefits from security research that Microsoft conducts to protect the cloud platform. Microsoft global threat intelligence is one of the largest in the industry, and it is derived from one of the most diverse sets of threat telemetry sources. It is both the volume and diversity of threat telemetry that makes Microsoft machine learning algorithms applied to that telemetry so powerful. All Azure customers benefit directly from these investments as described in the next section.
Threat detection and prevention
The Microsoft Graph Security API uses advanced analytics to synthesize massive amounts of threat intelligence and security signals obtained across Microsoft products, services, and partners to combat cyberthreats. Millions of unique threat indicators across the most diverse set of sources are generated every day by Microsoft and its partners and shared across Microsoft products and services (Figure 3). Across its portfolio of global services, each month Microsoft scans more than 400 billion email messages for phishing and malware, processes 450 billion authentications, executes more than 18 billion page scans, and scans more than 1.2 billion devices for threats. Importantly, this data always goes through strict privacy and compliance boundaries before being used for security analysis.
Figure 3. Microsoft global threat intelligence is one of the largest in the industry
The Microsoft Graph Security API provides an unparalleled view into the evolving threat landscape and enables rapid innovation to detect and respond to threats. Machine learning models and artificial intelligence reason over vast security signals to identify vulnerabilities and threats. The Microsoft Graph Security API provides a common gateway to share and act on security insights across the Microsoft platform and partner solutions. Azure customers benefit directly from the Microsoft Graph Security API as Microsoft makes the vast threat telemetry and advanced analytics available in Microsoft online services, including Azure Security Center. These services can help customers address their own security requirements in the cloud.
Microsoft has implemented extensive protections for the Azure cloud platform and made available a wide range of Azure services to help customers monitor and protect their provisioned cloud resources from attacks. Nonetheless, for certain types of workloads and data classifications, government customers expect to have full operational control over their environment and even operate in a fully disconnected mode. The Azure Stack portfolio of products enables customers to provision private and hybrid cloud deployment models that can accommodate highly sensitive data, as described in the next section.
Private and hybrid cloud with Azure Stack
Azure Stack portfolio is an extension of Azure that enables customers to build and run hybrid applications across on-premises, edge locations, and cloud. As shown in Figure 4, Azure Stack includes Azure Stack Hyperconverged Infrastructure (HCI), Azure Stack Hub (previously Azure Stack), and Azure Stack Edge (previously Azure Data Box Edge). The last two components (Azure Stack Hub and Azure Stack Edge) are discussed in this section. For more information, see Differences between global Azure, Azure Stack Hub, and Azure Stack HCI.
Figure 4. Azure Stack portfolio
Azure Stack Hub and Azure Stack Edge represent key enabling technologies that allow customers to process highly sensitive data using a private or hybrid cloud and pursue digital transformation using Microsoft intelligent cloud and intelligent edge approach. For many government customers, enforcing data sovereignty, addressing custom compliance requirements, and applying maximum available protection to highly sensitive data are the primary driving factors behind these efforts.
Azure Stack Hub
Azure Stack Hub (formerly Azure Stack) is an integrated system of software and validated hardware that customers can purchase from Microsoft hardware partners, deploy in their own data center, and then operate entirely on their own or with the help from a managed service provider. With Azure Stack Hub, the customer is always fully in control of access to their data. Azure Stack Hub can accommodate up to 16 physical servers per Azure Stack Hub scale unit. It represents an extension of Azure, enabling customers to provision various IaaS and PaaS services and effectively bring multi-tenant cloud technology to on-premises and edge environments. Customers can run many types of VM instances, App Services, Containers (including Cognitive Services containers), Functions, Azure Monitor, Key Vault, Event Hubs, and other services while using the same development tools, APIs, and management processes they use in Azure. Azure Stack Hub is not dependent on connectivity to Azure to run deployed applications and enable operations via local connectivity.
In addition to Azure Stack Hub, which is intended for on-premises deployment (for example, in a data center), a ruggedized and field-deployable version called Tactical Azure Stack Hub is also available to address tactical edge deployments for limited or no connectivity, fully mobile requirements, and harsh conditions requiring military specification solutions. Azure Stack Hub can be operated disconnected from Azure or the Internet. Customers can run the next generation of AI-enabled hybrid applications where their data lives. For example, government agencies can rely on Azure Stack Hub to bring a trained AI model to the edge and integrate it with their applications for low-latency intelligence, with no tool or process changes for local applications.
Azure and Azure Stack Hub can help government customers unlock new hybrid use cases for customer-facing and internal line-of-business application, including edge and disconnected scenarios, cloud applications intended to meet data sovereignty and compliance requirements, and cloud applications deployed on-premises in customer data center. These use cases may include mobile scenarios or fixed deployments within highly secure data center facilities. Figure 5 shows Azure Stack Hub capabilities and key usage scenarios.
Figure 5. Azure Stack Hub capabilities
Azure Stack Hub brings the following value proposition for key scenarios shown in Figure 5:
- Edge and disconnected solutions: Address latency and connectivity requirements by processing data locally in Azure Stack Hub and then aggregating in Azure for further analytics, with common application logic across both, connected or disconnected. Aircraft, ship, or truck-delivered, Azure Stack Hub meets the tough demands of exploration, construction, agriculture, oil and gas, manufacturing, disaster response, government, and military efforts in the most extreme conditions and remote locations. Government customers can use Azure Stack Hub architecture for edge and disconnected solutions, for example, bring the next generation of AI-enabled hybrid applications to the edge where the data lives and integrate it with existing applications for low-latency intelligence.
- Cloud applications to meet data sovereignty: Deploy a single application differently depending on the country or region. Customers can develop and deploy applications in Azure, with full flexibility to deploy on-premises with Azure Stack Hub based on the need to meet data sovereignty or custom compliance requirements. Customers can use Azure Stack Hub architecture for data sovereignty, for example, transmit data from Azure VNet to Azure Stack Hub VNet over private connection and ultimately store data in SQL Server database running in a VM on Azure Stack Hub. Government customers can use Azure Stack Hub to accommodate even more restrictive requirements such as the need to deploy solutions in a disconnected environment managed by security-cleared, in-country personnel. These disconnected environments may not be permitted to connect to the Internet for any purpose because of the security classification they operate at.
- Cloud application model on-premises: Use Azure Stack Hub to update and extend legacy applications and make them cloud ready. With App Service on Azure Stack Hub, customers can create a web front end to consume modern APIs with modern clients while taking advantage of consistent programming models and skills. Customers can use Azure Stack Hub architecture for legacy system modernization, for example, apply a consistent DevOps process, Azure Web Apps, containers, serverless computing, and microservices architectures to modernize legacy applications while integrating and preserving legacy data in mainframe and core line-of-business systems.
Azure Stack Hub requires Azure Active Directory (Azure AD) or Active Directory Federation Services, backed by Active Directory as an identity provider. Customers can use role-based access control (RBAC) to grant system access to authorized users, groups, and services by assigning them roles at a subscription, resource group, or individual resource level. Each role defines the access level a user, group, or service has over Azure Stack Hub resources.
Azure Stack Hub protects customer data at the storage subsystem level using encryption at rest. By default, Azure Stack Hub's storage subsystem is encrypted using BitLocker with 128-bit AES encryption. BitLocker keys are persisted in an internal secret store. At deployment time, it is also possible to configure BitLocker to use 256-bit AES encryption. Customers can store and manage their secrets including cryptographic keys using Key Vault in Azure Stack Hub.
Azure Stack Edge
Azure Stack Edge (formerly Azure Data Box Edge) is an AI-enabled edge computing device with network data transfer capabilities. It enables customers to pre-process data at the edge and move data to Azure efficiently. Azure Stack Edge uses advanced Field-Programmable Gate Array (FPGA) hardware natively integrated into the appliance to run machine learning algorithms at the edge efficiently. The size and portability allow customers to run Azure Stack Edge as close to users, apps, and data as needed. Figure 6 shows Azure Stack Edge capabilities and key use cases.
Figure 6. Azure Stack Edge capabilities
Azure Stack Edge brings the following value proposition for key use cases shown in Figure 6:
- Preprocess data: Analyze data from on-premises or IoT devices to quickly obtain results while staying close to where data is generated. Azure Stack Edge transfers the full data set (or just the necessary subset of data when bandwidth is an issue) to the cloud to perform more advanced processing or deeper analytics. Preprocessing can be used to aggregate data, modify data (for example, remove personally identifiable information or other sensitive data), transfer data needed for deeper analytics in the cloud, and analyze and react to IoT events.
- Inference Azure Machine Learning: Inference is a part of deep learning that takes place after model training, such as the prediction stage resulting from applying learned capability to new data. For example, it’s the part that recognizes a vehicle in a target image after the model has been trained by processing many tagged vehicle images, often augmented by computer synthesized images (also known as synthetics). With Azure Stack Edge, customers can run Machine Learning (ML) models to get results quickly and act on them before the data is sent to the cloud. The necessary subset of data (in case of bandwidth constraints) or the full data set is transferred to the cloud to continue to retrain and improve customer’s ML models.
- Transfer data over network to Azure: Use Azure Stack Edge to transfer data to Azure to enable further compute and analytics or for archival purposes.
Being able to gather, discern, and distribute mission data is essential for making critical decisions. Tools that help process and transfer data directly at the edge make this capability possible. For example, Azure Stack Edge, with its light footprint and built-in hardware acceleration for ML inferencing, is useful to further the intelligence of forward-operating units or similar mission needs with AI solutions designed for the tactical edge. Data transfer from the field, which is traditionally complex and slow, is made seamless with the Azure Data Box family of products.
These products unite the best of edge and cloud computing to unlock never-before-possible capabilities like synthetic mapping and ML model inferencing. From submarines to aircraft to remote bases, Azure Stack Hub and Azure Stack Edge allow customers to harness the power of cloud at the edge.
Using Azure in combination with Azure Stack Hub and Azure Stack Edge, government customers can process confidential and sensitive data in a secure isolated infrastructure within the Azure public multi-tenant cloud or highly sensitive data at the edge under the customer’s full operational control. The next section describes a conceptual architecture for classified workloads.
Figure 7 shows a conceptual architecture using products and services that support various data classifications. Azure public multi-tenant cloud is the underlying cloud platform that makes this solution possible. Customers can augment Azure with on-premises and edge products such as Azure Stack Hub and Azure Stack Edge to accommodate critical workloads over which customers seek increased or exclusive operational control. For example, Azure Stack Hub is intended for on-premises deployment in a customer-owned data center where the customer has full control over service connectivity. Moreover, Azure Stack Hub can be deployed to address tactical edge deployments for limited or no connectivity, including fully mobile scenarios.
Figure 7. Conceptual architecture for classified workloads
For classified workloads, customers can provision key enabling Azure services to secure target workloads while mitigating identified risks. Azure, in combination with Azure Stack Hub and Azure Stack Edge, can accommodate private and hybrid cloud deployment models, making them suitable for many government workloads involving both unclassified and classified data. The following data classification taxonomy is used in this article:
- Top secret
Similar data classification schemes exist in many countries.
For top secret data, customers can deploy Azure Stack Hub, which can operate fully disconnected from Azure and the Internet. Tactical Azure Stack Hub is also available to address tactical edge deployments for limited or no connectivity, fully mobile requirements, and harsh conditions requiring military specification solutions. Figure 8 depicts key enabling services that customers can provision to accommodate various workloads on Azure.
Figure 8. Azure support for various data classifications
Listed below are key enabling technologies and services that customers may find helpful when deploying confidential data and workloads on Azure:
- All recommended technologies used for Unclassified data, especially services such as Virtual Network (VNet), Azure Security Center, and Azure Monitor.
- Public IP addresses are disabled allowing only traffic through private connections, including ExpressRoute and Virtual Private Network (VPN) gateway.
- Data encryption is recommended with customer-managed keys (CMK) in Azure Key Vault backed by multi-tenant hardware security modules (HSMs) that have FIPS 140-2 Level 2 validation.
- Only services that support VNet integration options are enabled. Azure VNet enables customers to place Azure resources in a non-internet routable network, which can then be connected to customer’s on-premises network using VPN technologies. VNet integration gives web apps access to resources in the virtual network.
- Customers can use Azure Private Link to access Azure PaaS services over a private endpoint in their VNet, ensuring that traffic between their VNet and the service travels across the Microsoft global backbone network, which eliminates the need to expose the service to the public Internet.
- Customer Lockbox for Azure enables customers to approve/deny elevated access requests for customer data in support scenarios. It’s an extension of the Just-in-Time (JIT) workflow that comes with full audit logging enabled.
Using Azure public multi-tenant cloud capabilities, customers can achieve the level of isolation and security required to store confidential data. Customers should use Azure Security Center and Azure Monitor to gain visibility into their Azure environments including the security posture of their subscriptions.
Listed below are key enabling technologies and services that customers may find helpful when deploying secret data and workloads on Azure:
- All recommended technologies used for confidential data.
- Use Azure Key Vault Managed HSM, which provides a fully managed, highly available, single-tenant HSM as a service that uses FIPS 140-2 Level 3 validated HSMs. Each Managed HSM instance is bound to a separate security domain controlled by the customer and isolated cryptographically from instances belonging to other customers.
- Azure Dedicated Host provides physical servers that can host one or more Azure VMs and are dedicated to one Azure subscription. Customers can provision dedicated hosts within a region, availability zone, and fault domain. They can then place VMs directly into provisioned hosts using whatever configuration best meets their needs. Dedicated Host provides hardware isolation at the physical server level, enabling customers to place their Azure VMs on an isolated and dedicated physical server that runs only their organization’s workloads to meet corporate compliance requirements.
- Accelerated FPGA networking based on Azure SmartNICs enables customers to offload host networking to dedicated hardware, enabling tunneling for VNets, security, and load balancing. Offloading network traffic to a dedicated chip guards against side-channel attacks on the main CPU.
- Azure confidential computing offers encryption of data while in use, ensuring that data is always under customer control. Data is protected inside a hardware-based trusted execution environment (TEE, also known as enclave) and there is no way to view data or operations from outside the enclave.
- Just-in-time (JIT) virtual machine (VM) access can be used to lock down inbound traffic to Azure VMs by creating network security group (NSG) rules. Customers select ports on the VM to which inbound traffic will be locked down and when a user requests access to a VM, Azure Security Center checks that the user has proper role-based access control (RBAC) permissions.
To accommodate secret data in the Azure public multi-tenant cloud, customers can deploy extra technologies and services on top of those technologies used for confidential data and limit provisioned services to those services that provide sufficient isolation. These services offer various isolation options at run time. They also support data encryption at rest using customer-managed keys in single-tenant HSMs controlled by the customer and isolated cryptographically from HSM instances belonging to other customers.
Top secret data
Listed below are key enabling products that customers may find helpful when deploying top secret data and workloads on Azure:
- All recommended technologies used for secret data.
- Azure Stack Hub (formerly Azure Stack) enables customers to run workloads using the same architecture and APIs as in Azure while having a physically isolated network for their highest classification data.
- Azure Stack Edge (formerly Azure Data Box Edge) allows the storage and processing of highest classification data but also enables customers to upload resulting information or models directly to Azure. This approach creates a path for information sharing between domains that makes it easier and more secure.
- In addition to Azure Stack Hub, which is intended for on-premises deployment (for example, in a data center), a ruggedized and field-deployable version called Tactical Azure Stack Hub is also available to address tactical edge deployments for limited or no connectivity, fully mobile requirements, and harsh conditions requiring military specification solutions.
- User-provided hardware security modules (HSMs) allow customers to store their encryption keys and other secrets in HSMs deployed on-premises and controlled solely by customers.
Accommodating top secret data will likely require a disconnected environment, which is what Azure Stack Hub provides. Azure Stack Hub can be operated disconnected from Azure or the Internet. Even though “air-gapped” networks do not necessarily increase security, many governments may be reluctant to store data with this classification in an Internet connected environment.
Azure offers an unmatched variety of public, private, and hybrid cloud deployment models to address each customer’s concerns regarding the control of their data. The following section covers select use cases that might be of interest to worldwide government customers.
Select workloads and use cases
This section provides an overview of select use cases that showcase Azure capabilities for workloads that might be of interest to worldwide governments. In terms of capabilities, Azure is presented via a combination of public multi-tenant cloud and on-premises + edge capabilities provided by Azure Stack Hub and Azure Stack Edge.
Processing highly sensitive or regulated data on Azure Stack Hub
Microsoft provides Azure Stack Hub as an on-premises, cloud-consistent experience for customers who do not have the ability to directly connect to the Internet, or where certain workload types are required to be hosted in-country due to law, compliance, or sentiment. Azure Stack Hub offers IaaS and PaaS services and shares the same APIs as the public Azure cloud. Azure Stack Hub is available in scale units of 4, 8, and 16 servers in a single-server rack, and 4 servers in a military-specification, ruggedized set of transit cases, or multiple racks in a modular data center configuration.
Azure Stack Hub is a solution for customers who operate in scenarios where:
- Microsoft does not have an in-country cloud presence and therefore cannot meet data sovereignty requirements.
- For compliance reasons, the customer cannot connect their network to the public Internet.
- For geo-political or security reasons, Microsoft cannot offer connectivity to other Microsoft clouds.
- For geo-political or security reasons, the host organization may require cloud management by non-Microsoft entities, or in-country by security-cleared personnel.
- Cloud management would pose significant risk to the physical well-being of Microsoft resources operating the environment.
For most of these customers, Microsoft and its partners offer a customer-managed, Azure Stack Hub-based private cloud appliance on field-deployable hardware from major vendors such as Avanade, Cisco, Dell EMC, Hewlett Packard Enterprise, and Lenovo. Azure Stack Hub is manufactured, configured, and deployed by the hardware vendor, and can be ruggedized and security-hardened to meet a broad range of environmental and compliance standards, including the ability to withstand transport by aircraft, ship, or truck, and deployment into colocation, mobile, or modular data centers. Azure Stack Hub can be used in exploration, construction, agriculture, oil and gas, manufacturing, disaster response, government, and military efforts in hospitable or the most extreme conditions and remote locations. Azure Stack Hub allows customers the full autonomy to monitor, manage, and provision their own private cloud resources while meeting their connectivity, compliance, and ruggedization requirements.
Machine learning model training
Artificial intelligence (AI) holds tremendous potential for governments. Machine learning (ML) is a data science technique that allows computers to learn to use existing data, without being explicitly programmed, to forecast future behaviors, outcomes, and trends. Moreover, ML technologies can discover patterns, anomalies, and predictions that can help governments in their missions. As technical barriers continue to fall, decision-makers face the opportunity to develop and explore transformative AI applications. There are five main vectors that can make it easier, faster, and cheaper to adopt ML:
- Unsupervised learning
- Reducing need for training data
- Accelerated learning
- Transparency of outcome
- Deploying closer to where data lives
In the following sections, we expand on areas that can help government agencies with some of the above vectors.
In recent years, we have been witnessing massive proliferation of Internet of Things (IoT) devices and sensors. In almost all cases, these sensors gather signals and data from the environments and conditions they’re designed for. The spectrum of capabilities for IoT sensors expands from measuring the level of moisture in soil all the way to gathering intelligence at 5,000-meters altitude. The high number of use cases imposes the necessity of applying data-analysis tools and procedures to realize value from the huge volumes of gathered data by IoT devices.
Governments are increasingly employing IoT devices for their missions, which could include maintenance predictions, borders monitoring, weather stations, smart meters, and field operations. In many cases, the data is often analyzed and inferred from where it’s gathered. The main challenges of IoT analytics are: (1) large amount of data from independent sources, (2) analytics at the edge and often in disconnected scenarios, and (3) data and analysis aggregation.
Precision Agriculture with Farm Beats
Agriculture plays a vital role in most economies worldwide. In the US, over 70% of the rural households depend on agriculture as it contributes about 17% to the total GDP and provides employment to over 60% of the population. In project Farm Beats, we gather numerous data from farms that we couldn’t get before, and then by applying AI and ML algorithms we are able to turn this data into actionable insights for farmers. We call this technique data-driven farming. What we mean by data-driven farming is the ability to map every farm and overlay it with data. For example, what is the soil moisture level 6 inches below soil, what is the soil temperature 6 inches below soil, etc. These maps can then enable techniques, such as Precision Agriculture, which has been shown to improve yield, reduce costs, and benefit the environment. Despite the fact the Precision Agriculture as a technique was proposed more than 30 years ago, it hasn’t taken off. The biggest reason is the inability to capture numerous data from farms to accurately represent the conditions in the farm. Our goal as part of the Farm Beats project is to be able to accurately construct precision maps at a fraction of the cost.
Unleashing the power of analytics with synthetic data
Synthetic data is data that is artificially created rather than being generated by actual events. It is often created with the help of computer algorithms and it is used for a wide range of activities, including usage as test data for new products and tools, as well as for ML models validation and improvements. Synthetic data can meet specific needs or conditions that are not available in existing real data. For governments, the nature of synthetic data removes many barriers and helps data scientists with privacy concerns, accelerated learning, and data volume reduction needed for the same outcome. The main benefits of synthetic data are:
- Overcoming restrictions: Real data may have usage constraints due to privacy rules or other regulations. Synthetic data can replicate all important statistical properties of real data without exposing real data.
- Scarcity: Providing data where real data does not exist for a given event.
- Precision: Synthetic data is perfectly labeled.
- Quality: The quality of synthetic data can be precisely measured to fit the mission conditions.
Synthetic data can exist in several forms, including text, audio, video, and hybrid.
The exponential growth of unstructured data gathering in recent years has created many analytical problems for government agencies. This problem intensifies when data sets come from diverse sources such as text, audio, video, imaging, etc. Knowledge mining is the process of discovering useful knowledge from a collection of diverse data sources. This widely used data mining technique is a process that includes data preparation and selection, data cleansing, incorporation of prior knowledge on data sets, and interpretation of accurate solutions from the observed results. This process has proven to be useful for large volumes of data in different government agencies.
For instance, captured data from the field often includes documents, pamphlets, letters, spreadsheets, propaganda, videos, and audio files across many disparate structured and unstructured formats. Buried within the data are actionable insights that can enhance effective and timely response to crisis and drive decisions. The objective of knowledge mining is to enable decisions that are better, faster, and more humane by implementing proven commercial algorithm-based technologies.
Scenarios for confidential computing
Security is a key driver accelerating the adoption of cloud computing, but it’s also a major concern when customers are moving sensitive IP and data to the cloud.
Microsoft Azure provides broad capabilities to secure data at rest and in transit, but sometimes the requirement is also to protect data from threats as it’s being processed. Microsoft Azure confidential computing is designed to address this scenario by performing computations in a hardware-based trusted execution environment (TEE, also known as enclave) based on Intel Software Guard Extensions (SGX) technology. The hardware provides a protected container by securing a portion of the processor and memory. Only authorized code is permitted to run and to access data, so code and data are protected against viewing and modification from outside of TEE.
TEEs can directly address scenarios involving data protection while in use. For example, consider the scenario where data coming from a public or unclassified source needs to be matched with data from a highly sensitive source. Azure confidential computing can enable that matching to occur in the public cloud while protecting the highly sensitive data from disclosure. This circumstance is common in highly sensitive national security and law enforcement scenarios.
A second scenario involves data coming from multiple sources that needs to be analyzed together, even though none of the sources have the authority to see the data. Each individual provider encrypts the data they provide and only within the TEE is that data decrypted. As such, no external party and even none of the providers can see the combined data set. This capability is valuable capability for secondary use of healthcare data.
Customers deploying the types of workloads discussed in this section typically seek assurances from Microsoft that the underlying cloud platform security controls for which Microsoft is responsible are operating effectively. To address the needs of customers across regulated markets worldwide, Azure maintains a comprehensive compliance portfolio based on formal third-party certifications and other types of assurances to help customers meet their own compliance obligations.
Compliance and certifications
Azure has the broadest compliance coverage in the industry, including key independent certifications and attestations such as ISO 27001, ISO 27017, ISO 27018, ISO 22301, ISO 9001, ISO 20000-1, SOC 1/2/3, PCI DSS Level 1, PCI 3DS, HITRUST, CSA STAR Certification, CSA STAR Attestation, US FedRAMP High, Australia IRAP, Germany C5, Japan CS Gold Mark, Singapore MTCS Level 3, Spain ENS High, UK G-Cloud and Cyber Essentials Plus, and many more. Azure compliance portfolio includes more than 90 compliance offerings spanning globally applicable certifications, US Government-specific programs, industry assurances, and regional / country-specific offerings. Government customers can use these offerings when addressing their own compliance obligations across regulated industries and markets worldwide.
When deploying applications that are subject to regulatory compliance obligations on Azure, customers seek assurances that all cloud services comprising the solution are included in the cloud service provider’s audit scope. Azure offers industry-leading depth of compliance coverage judged by the number of cloud services in audit scope for each Azure certification. Customers can build and deploy realistic applications and benefit from extensive compliance coverage provided by Azure independent third-party audits.
Azure Stack Hub also provides compliance documentation to help customers integrate Azure Stack Hub into solutions that address regulated workloads. Customers can download the following Azure Stack Hub compliance documents:
- PCI DSS assessment report produced by a third-party Qualified Security Assessor (QSA).
- Cloud Security Alliance (CSA) Cloud Controls Matrix (CCM) assessment report, including Azure Stack Hub control mapping to CCM domains and controls.
- FedRAMP High System Security Plan (SSP) precompiled template to demonstrate how Azure Stack Hub addresses applicable controls, Customer Responsibility Matrix for the FedRAMP High baseline, and FedRAMP assessment report produced by an independent Third-Party Assessor Organization (3PAO).
Azure Blueprints is a service that helps automate compliance and cybersecurity risk management in cloud environments. For more information on Azure Blueprints, including production-ready blueprint solutions for ISO 27001, NIST SP 800-53, PCI DSS, HITRUST, and other standards, see the Azure Blueprint guidance.
Azure compliance and certification resources are intended to help customers address their own compliance obligations with various regulations. Some governments across the world have already established cloud adoption mandates and the corresponding regulation to facilitate cloud onboarding. However, there are many government customers that still operate traditional on-premises datacenters and are in the process of formulating their cloud adoption strategy. Azure’s extensive compliance portfolio can be of assistance to customers irrespective of their cloud adoption maturity level.
Frequently asked questions
This section addresses common customer questions related to Azure public, private, and hybrid cloud deployment models.
Data residency and data sovereignty
- Data location: How does Microsoft keep data within a specific country’s boundaries? In what cases does data leave? What data attributes leave? Answer: Microsoft provides strong customer commitments regarding cloud services data residency and transfer policies:
- Data storage for regional services: Most Azure services are deployed regionally and enable the customer to specify the region into which the service will be deployed, for example, Europe. Microsoft will not store customer data outside the customer-specified Geo except for a few regional services and Preview services as described on the Azure data location page. This commitment helps ensure that customer data stored in a given region will remain in the corresponding Geo and will not be moved to another Geo for most regional services, including Storage, SQL Database, Virtual Machines, and many more.
- Data storage for non-regional services: Certain Azure services do not enable the customer to specify the region where the services will be deployed as described on the data location page. For a complete list of non-regional services, see Products available by region.
- Sovereign cloud deployment: Why doesn’t Microsoft deploy a sovereign, physically isolated cloud instance in every country that requests it? Answer: Microsoft is actively pursuing sovereign cloud deployments where a business case can be made with governments across the world. However, physical isolation or “air gapping”, as a strategy, is diametrically opposed to the strategy of hyperscale cloud. The value proposition of the cloud, rapid feature growth, resiliency, and cost-effective operation, break down when the cloud is fragmented and physically isolated. These strategic challenges compound with each extra sovereign cloud or fragmentation within a sovereign cloud. Whereas a sovereign cloud might prove to be the right solution for certain customers, it is not the only option available to worldwide public sector customers.
- Sovereign cloud customer options: How can Microsoft support governments who need to operate cloud services completely in-country by local security-cleared personnel? What options does Microsoft have for cloud services operated entirely on-premises within customer owned datacenter where government employees exercise sole operational and data access control? Answer: Government customers can use Azure Stack Hub to deploy a private cloud on-premises managed by the customer’s own security-cleared, in-country personnel. Customers can run many types of VM instances, App Services, Containers (including Cognitive Services containers), Functions, Azure Monitor, Key Vault, Event Hubs, and other services while using the same development tools, APIs, and management processes they use in Azure. With Azure Stack Hub, customers have sole control of their data, including storage, processing, transmission, and remote access.
- Local jurisdiction: Is Microsoft subject to local country jurisdiction based on the availability of Azure public cloud service? Answer: Yes, Microsoft must comply with all applicable local laws; however, government requests for customer data must also comply with applicable laws. A subpoena or its local equivalent is required to request non-content data. A warrant, court order, or its local equivalent is required for content data. Government requests for customer data follow a strict procedure according to Microsoft practices for responding to government requests. Every year, Microsoft rejects many law enforcement requests for customer data. Challenges to government requests can take many forms. In many of these cases, Microsoft simply informs the requesting government that it is unable to disclose the requested information and explains the reason for rejecting the request. Where appropriate, Microsoft challenges requests in court. Our Law Enforcement Request Report and US National Security Order Report are updated every six months and show that most of our customers are never impacted by government requests for data. For example, in the second half of 2019, Microsoft received 39 requests from law enforcement for accounts associated with enterprise cloud customers. Of those requests, only one warrant resulted in disclosure of customer content related to a non-US enterprise customer whose data was stored outside the United States.
- Autarky: Can Microsoft cloud operations be separated from the Internet or the rest of Microsoft cloud and connected solely to local government network? Are operations possible without external connections to a third party? Answer: Yes, depending on the cloud deployment model.
- Public Cloud: Azure regional datacenters can be connected to local government network through dedicated private connections such as ExpressRoute. Independent operation without any connectivity to a third party such as Microsoft is not possible in public cloud.
- Private Cloud: With Azure Stack Hub, customers have full control over network connectivity and can operate Azure Stack Hub in fully disconnected mode.
- Data flow restrictions: What provisions exist for approval and documentation of all data exchange between customer and Microsoft for local, in-country deployed cloud services? Answer: Options vary based on the cloud deployment model.
- Private cloud: For private cloud deployment using Azure Stack Hub, customers can control which data is exchanged with third parties. Azure Stack Hub telemetry can be turned off based on customer preference and Azure Stack Hub can be operated fully disconnected. Moreover, Azure Stack Hub offers the capacity-based billing model in which no billing or consumption data leaves the customer’s premises.
- Public cloud: In Azure public cloud, customers can use Network Watcher to monitor network traffic associated with their workloads. For public cloud workloads, all billing data is generated through telemetry used exclusively for billing purposes and sent to Microsoft billing systems. Customers can download and view their billing and usage data; however, they cannot prevent this information from being sent to Microsoft. Microsoft engineers do not have default access to customer data. For customer-initiated support requests, Customer Lockbox for Azure can be used to enable customers to approve/deny elevated requests for customer data access. Moreover, customers have control over data encryption at rest using customer-managed encryption keys.
- Patching and maintenance for private cloud: How can Microsoft support patching and other maintenance for Azure Stack Hub private cloud deployment? Answer: Microsoft has a regular cadence in place for releasing update packages for Azure Stack Hub. Government customers are sole operators of Azure Stack Hub and they can download and install these update packages. An update alert for Microsoft software updates and hotfixes will appear in the Update blade for Azure Stack Hub instances that are connected to the Internet. If your instance isn’t connected and you would like to be notified about each update release, subscribe to the RSS or ATOM feed, as explained in our online documentation.
Safeguarding of customer data
- Microsoft network security: What network controls and security does Microsoft use? Can customer requirements be considered? Answer: For insight into Azure infrastructure protection, customers should review Azure network architecture, Azure production network, and Azure infrastructure monitoring. Customers deploying Azure applications should review Azure network security overview and network security best practices. To provide feedback or requirements, contact your Microsoft account representative.
- Customer separation: How does Microsoft logically or physically separate customers within its cloud environment? Is there an option for select customers to ensure complete physical separation? Answer: Azure uses logical isolation to segregate each customer's applications and data. This approach provides the scale and economic benefits of multi-tenant cloud services while rigorously enforcing controls designed to keep customers from accessing one another's data or applications. There is also an option to enforce physical compute isolation via Azure Dedicated Host, which provides physical servers that can host one or more Azure VMs and are dedicated to one Azure subscription. Customers can provision dedicated hosts within a region, availability zone, and fault domain. They can then place VMs directly into provisioned hosts using whatever configuration best meets their needs. Dedicated Host provides hardware isolation at the physical server level, enabling customers to place their Azure VMs on an isolated and dedicated physical server that runs only their organization’s workloads to meet corporate compliance requirements.
- Data encryption at rest and in transit: Does Microsoft enforce data encryption by default? Does Microsoft support customer-managed encryption keys? Answer: Yes, many Azure services, including Azure Storage and Azure SQL Database, encrypt data by default and support customer-managed keys. Azure Storage encryption for data at rest ensures that data is automatically encrypted before persisting it to Azure Storage and decrypted before retrieval. Customers can use their own encryption keys for Azure Storage encryption at rest and manage their keys in Azure Key Vault. Storage encryption is enabled by default for all new and existing storage accounts and it cannot be disabled. When provisioning storage accounts, customers can enforce “secure transfer required” option, which allows access only from secure connections. This option is enabled by default when creating a storage account in the Azure portal. Azure SQL Database enforces data encryption in transit by default and provides transparent data encryption (TDE) at rest by default allowing customers to use Azure Key Vault and bring your own key (BYOK) functionality to control key management tasks including key permissions, rotation, deletion, and so on.
- Data encryption during processing: Can Microsoft protect customer data while it is being processed in memory? Answer: Yes, Microsoft Azure confidential computing is designed to address this scenario by performing computations in a hardware-based trusted execution environment (TEE, also known as enclave) based on Intel Software Guard Extensions (SGX) technology. The hardware provides a protected container by securing a portion of the processor and memory. Only authorized code is permitted to run and to access data, so code and data are protected against viewing and modification from outside of TEE.
- FIPS 140-2 validation: Does Microsoft offer FIPS 140-2 Level 3 validated hardware security modules (HSMs) in Azure? Answer: Yes, Azure Key Vault Managed HSM provides a fully managed, highly available, single-tenant HSM as a service that uses FIPS 140-2 Level 3 validated HSMs (certificate #3718). Each Managed HSM instance is bound to a separate security domain controlled by the customer and isolated cryptographically from instances belonging to other customers.
- Customer provided crypto: Can customers bring their own cryptography or encryption hardware? Answer: Yes, customers can use their own HSMs deployed on-premises with their own crypto algorithms. However, if customers expect to use customer-managed keys for services integrated with Azure Key Vault (for example, Azure Storage, SQL Database, Disk encryption, and others), then they need to use hardware security modules (HSMs) and cryptography supported by Azure Key Vault.
- Access to customer data by Microsoft personnel: How does Microsoft restrict access to customer data by Microsoft engineers? Answer: Microsoft engineers do not have default access to customer data in the cloud. Instead, they are granted access, under management oversight, only when necessary using the restricted access workflow. For customer-initiated support requests, Customer Lockbox for Azure provides customers with the capability to control how a Microsoft engineer accesses their data. As part of the support workflow, a Microsoft engineer may require elevated access to customer data. Customer Lockbox for Azure puts the customer in charge of that decision by enabling the customer to approve/deny such elevated requests.
- Code review: What can Microsoft do to help ensure that no malicious code has been inserted into the services that customers use? Can customers review Microsoft code deployments? Answer: Microsoft has full control over all source code that comprises Azure services. The procedure for patching guest VMs differs greatly from traditional on-premises patching where patch verification is necessary following installation. In Azure, patches are not applied to guest VMs; instead, the VM is simply restarted and when the VM boots, it is guaranteed to boot from a known good image that Microsoft controls. There is no way to insert malicious code into the image or interfere with the boot process. PaaS VMs offer more advanced protection against persistent malware infections than traditional physical server solutions, which if compromised by an attacker can be difficult to clean, even after the vulnerability is corrected. With PaaS VMs, reimaging is a routine part of operations, and it can help clean out intrusions that have not even been detected. This approach makes it more difficult for a compromise to persist. Customers cannot review Azure source code; however, online access to view source code is available for key products through the Microsoft Government Security Program (GSP).
- DevOps personnel (cleared nationals): What controls or clearance levels does Microsoft have for the personnel that have DevOps access to cloud environments or physical access to data centers? Answer: Microsoft conducts background screening on operations personnel with access to production systems and physical data center infrastructure. Microsoft cloud background check includes verification of education and employment history upon hire, and extra checks conducted every two years thereafter (where permissible by law), including criminal history check, OFAC list, BIS denied persons list, and DDTC debarred parties list.
- Data center site options: Is Microsoft willing to deploy a data center to a specific physical location to meet more advanced security requirements? Answer: Customers should inquire with their Microsoft account team regarding options for data center locations.
- Service availability guarantee: How do we ensure that Microsoft (or particular government or other entity) can’t turn off our cloud services? Answer: Customers should review the Microsoft Online Services Terms (OST) and the OST Data Protection Addendum (DPA) for contractual commitments Microsoft makes regarding service availability and use of online services.
- Non-traditional cloud service needs: What is the recommended approach for managing scenarios where Azure services are required in periodically internet free/disconnected environments? Answer: In addition to Azure Stack Hub which is intended for on-premises deployment and disconnected scenarios, a ruggedized and field-deployable version called Tactical Azure Stack Hub is also available to address tactical edge deployments for limited or no connectivity, fully mobile requirements, and harsh conditions requiring military specification solutions.
Transparency and audit
- Audit documentation: Does Microsoft make all audit documentation readily available to customers to download and examine? Answer: Yes, Microsoft makes all independent third-party audit reports and other related documentation available to customers under a non-disclosure agreement from the Azure portal. You will need an existing Azure subscription or free trial subscription to access Azure Security Center audit reports blade.
- Process auditability: Does Microsoft make its processes, data flow, and documentation available to customers or regulators for audit? Answer: Yes, Microsoft offers a Regulator Right to Examine, which is a program Microsoft implemented to provide regulators with direct right to examine Azure, including the ability to conduct an on-site examination, to meet with Microsoft personnel and Microsoft external auditors, and to access any related information, records, reports, and documents.
- Service documentation: Can Microsoft provide in-depth documentation covering service architecture, software and hardware components, and data protocols? Answer: Yes, Microsoft provides extensive and in-depth Azure online documentation covering all these topics. For example, customers can review documentation on Azure products, global infrastructure, and API reference.
Learn more about: