您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

使用应用服务环境进行企业部署

Azure Active Directory
应用程序网关
应用服务
防火墙
密钥保管库
管道
虚拟网络

Azure App Service 是一种 PaaS 服务,用于在 Azure 上托管各种应用: web 应用、API 应用、函数和移动应用。Azure App Service is a PaaS service used to host a variety of apps on Azure: web apps, API apps, functions, and mobile apps. 应用服务环境或 ASE 允许企业在其自己的 Azure 虚拟网络中的子网中部署其应用服务应用,为其云工作负荷提供一个独立、高度可缩放的专用环境。App Service Environment or ASE allows enterprises to deploy their App Service apps in a subnet in their own Azure Virtual Network, providing an isolated, highly scalable, and dedicated environment for their cloud workloads.

此参考体系结构演示了使用 ASE 的常见企业工作负荷,以及加强此工作负荷的安全性的最佳做法。This reference architecture demonstrates a common enterprise workload using ASE, and best practices to tighten security of this workload.

GitHub 徽标 github上提供了此体系结构的参考实现。GitHub logo A reference implementation for this architecture is available on GitHub.

标准 ASE 部署的参考体系结构

体系结构Architecture

此体系结构的主要组件是 应用服务环境The main component of this architecture is the App Service environment. ASE 可以作为带有公共 IP 地址的 外部 ase 进行部署,也可以作为 内部 ase 部署,其内部 IP 地址属于 内部负载均衡器 (ILB)An ASE can be deployed either as external ASE with a public IP address, or as internal ASE, with an internal IP address belonging to the Internal Load Balancer (ILB). 此参考体系结构在内部 ASE (也称为 ILB ASE)中部署企业 web 应用程序。This reference architecture deploys an enterprise web application in an internal ASE, also called an ILB ASE. 如果你的方案要求你执行以下操作,请使用 ILB ASE:Use an ILB ASE when your scenario requires you to:

  • 在云中安全托管可通过站点到站点连接或 ExpressRoute 访问的 Intranet 应用程序。Host intranet applications securely in the cloud, which you access through a site-to-site or ExpressRoute.
  • 使用 WAF 设备保护应用。Protect apps with a WAF device.
  • 在云端托管未在公用 DNS 服务器中列出的应用。Host apps in the cloud that aren't listed in public DNS servers.
  • 创建与 Internet 隔离且前端应用可以与之安全集成的后端应用。Create internet-isolated back-end apps, which your front-end apps can securely integrate with.

ASE 必须始终部署在企业 "虚拟网络" 中其自己的子网中,允许严格控制传入和传出流量。An ASE must be always deployed in its own subnet within the enterprise' virtual network, allowing a tight control of the incoming and outgoing traffic. 在此子网中,应用服务应用程序部署在一个或多个 应用服务计划中,这是运行应用所需的物理资源的集合。Within this subnet, App Service applications are deployed in one or more App Service plans, which is a collection of physical resources required to run the app. 根据复杂性和资源要求,可以在多个应用之间共享应用服务计划。Depending on the complexity and resource requirement, an App Service plan can be shared between multiple apps. 此引用实现部署了一个名为 投票应用 程序的 web 应用,该应用与专用 web API 和函数交互。This reference implementation deploys a web app named Voting App, that interacts with a private web API and a function. 它还部署一个名为 " 测试应用程序 " 的虚拟 web 应用来演示多应用部署。It also deploys a dummy web app named Test App to demonstrate multi-app deployments. 每个应用服务应用都托管在其自身的应用服务计划中,允许根据需要单独缩放。Each App Service app is hosted in its own App Service plan, allowing each to be scaled independently if necessary. 这些托管应用所需的所有资源(如存储和计算)以及缩放需求都由应用服务环境基础结构进行完全管理。All resources required by these hosted apps, such as storage and compute, as well as scaling needs are fully managed by the App Service Environment infrastructure.

此实现中的简单投票应用允许用户查看现有条目或创建新条目,以及对现有条目进行投票。The simple voting app in this implementation, allows users to view existing or create new entries, as well as vote on the existing entries. Web API 用于创建和检索条目和投票。The web API is used for creation and retrieval of entries and votes. 数据本身存储在 SQL Server 数据库中。The data itself is stored in a SQL Server database. 为了演示异步数据更新,web 应用会在服务总线实例中对新添加的投票进行排队。To demonstrate asynchronous data updates, the web app queues newly added votes in a Service Bus instance. 该函数选取已排队的投票并更新 SQL 数据库。The function picks up queued votes and updates the SQL database. CosmosDB 用于存储模拟广告,web 应用将在浏览器中进行检索。CosmosDB is used to store a mock-up ad, that the web app retrieves to display on the browser. 应用程序使用用于 Redis 的 Azure 缓存来管理缓存。The application uses Azure Cache for Redis for cache management. 使用用于 Redis 的高级层 Azure 缓存,可在其自己的子网中配置与 ASE 相同的虚拟网络。A Premium tier Azure Cache for Redis is used, which allows configuration to the same virtual network as the ASE, in its own subnet. 这为缓存提供了增强的安全性和隔离性。This provides enhanced security and isolation to the cache.

Web apps 是通过应用程序网关可访问的唯一组件。The web apps are the only components accessible to the internet via the App Gateway. API 和函数无法从 internet 客户端访问。The API and the function are inaccessible from an internet client. 入站流量受在应用程序网关上配置的 Web 应用程序防火墙的保护。The inbound traffic is protected by a Web Application Firewall configured on the App Gateway.

以下服务是锁定此体系结构中的 ASE 的关键:The following services are key to locking down the ASE in this architecture:

Azure 虚拟网络 或 VNet 是企业拥有的专用 azure 云网络。Azure Virtual Network or VNet is a private Azure cloud network owned by the enterprise. 它提供基于网络的安全性和隔离。It provides network-based security and isolation. ASE 是企业拥有的 VNet 的子网中的应用服务部署。ASE is an App Service deployment into a subnet of the enterprise-owned VNet. 它通过使用 网络安全组服务终结点,使企业能够严格控制其访问的网络空间和资源。It allows the enterprise to tightly control that network space and the resources it accesses, using Network Security groups and Service Endpoints.

应用程序网关 是应用程序级的 web 流量负载均衡器,使用 TLS/SSL 卸载,web 应用程序防火墙 (WAF) 保护。Application Gateway is an application-level web traffic load balancer, with TLS/SSL offloading, and web application firewall (WAF) protection. 它侦听公共 IP 地址,并将流量路由到 ILB ASE 中的应用程序终结点。It listens on a public IP address and routes traffic to the application endpoint in the ILB ASE. 由于这是应用程序级别的路由,因此它可以将流量路由到同一 ILB ASE 内的多个应用程序。Since this is application-level routing, it can route traffic to multiple apps within the same ILB ASE. 请参阅 应用程序网关多站点托管 ,了解应用程序网关如何支持多个站点。See Application Gateway multiple site hosting to learn how App Gateway supports multiple sites.

Azure 防火墙 用于限制来自 web 应用程序的出站流量。Azure Firewall is used to restrict the outbound traffic from the web application. 不通过服务终结点通道和 ASE 所需的路由表的传出流量将定向到防火墙子网。Outgoing traffic that does not go through the service endpoint channels and a route table required by ASE, is directed to the firewall subnet. 尽管建议 在防火墙子网上配置服务终结点 以实现可跟踪性,但这可能并不总是可行。Although it is recommended for service endpoints to be configured on the firewall subnet for traceability, it may not be always feasible. 例如,ASE 基础结构需要某些服务终结点,才能将其置于 ASE 子网中。For example, some service endpoints are required by the ASE infrastructure to be on the ASE subnet. 为简单起见,此体系结构配置 ASE 子网上的所有服务终结点。For simplicity, this architecture configures all service endpoints on the ASE subnet.

Azure Active Directory 或 Azure AD 提供对 Azure 资源和服务的访问权限管理。Azure Active Directory or Azure AD provides access rights and permissions management to Azure resources and services. 托管标识 将标识分配给服务和应用,由 Azure AD 自动管理。Managed Identities assigns identities to services and apps, automatically managed by Azure AD. 这些标识可用于向支持 Azure AD 身份验证的任何服务进行身份验证。These identities can be used to authenticate to any service that supports Azure AD authentication. 这样就无需为这些应用显式配置凭据。This removes the need to explicitly configure credentials for these apps. 此体系结构将托管标识分配给 web 应用。This architecture assigns a managed identity to the web app.

Key Vault 存储应用所需的任何机密和凭据。Key Vault stores any secrets and credentials required by the apps. 使用此选项直接在应用程序中存储机密。Use this option over storing secrets directly in the application.

Azure Pipelines 在此体系结构中提供 持续集成和持续部署 功能。Azure Pipelines provides Continuous Integration and Continuous Deployment capabilities in this architecture. 由于 ASE 是虚拟网络的内部,因此 虚拟机 用作 VNet 中的 jumpbox ,以便在应用服务计划中部署应用。Since the ASE is internal to the virtual network, a virtual machine is used as a jumpbox inside the VNet to deploy apps in the App Service plans. 管道在 VNet 之外构建应用。The pipeline builds the apps outside the VNet. 为增强安全性和无缝 RDP/SSH 连接,请考虑使用最近发布的 Azure 堡垒 作为 jumpbox。For enhanced security and seamless RDP/SSH connectivity, consider using the recently released Azure Bastion as the jumpbox.

多站点配置Multi-site configuration

多站点部署

内部 ASE 可以托管多个包含 HTTP 终结点的 web 应用和 Api。Internal ASE can host several web apps and APIs with HTTP endpoints. 从公共 internet 锁定这些应用程序,因为 ILB IP 只能从虚拟网络内部访问。These applications are locked down from the public internet as the ILB IP is only accessible from within the Virtual Network. 应用程序网关用于有选择地将这些应用程序公开到 internet。The Application Gateway is used to selectively expose these applications to the internet. ASE 将默认 URL 作为分配给每个应用服务应用程序 <default-appName>.<ASE-domain>.appserviceenvironment.netASE assigns a default URL to each App Service application as <default-appName>.<ASE-domain>.appserviceenvironment.net. 将创建一个 专用 DNS 区域 ,该区域将 ASE 域名映射到 ASE ILB IP 地址。A private DNS zone is created that maps the ASE domain name to the ASE ILB IP address. 这可以避免使用自定义 DNS 访问 VNet 中的应用。This avoids using a custom DNS to access the apps within the VNet.

已配置应用程序网关,以便 侦听器 侦听 HTTPS 端口以请求发送到网关的 IP 地址。The App Gateway is configured such that a listener listens on the HTTPS port for requests to the Gateway's IP address. 为简单起见,此实现不使用公共 DNS 名称注册,并要求您在计算机上修改 localhost 文件,使任意选定的 URL 指向应用程序网关的 IP。For simplicity, this implementation does not use a public DNS name registration, and requires you to modify the localhost file on your computer to point an arbitrarily chosen URL to the App Gateway's IP. 为简单起见,侦听器使用自签名证书来处理这些请求。For simplicity, the listener uses a self-signed certificate to process these requests. 应用程序网关具有每个应用服务应用程序的默认 URL 的 后端池The App Gateway has backend pools for each App Service application's default URL. 路由规则配置为将侦听器连接到后端池。A routing rule is configured to connect the listener to the backend pool. 创建HTTP 设置,以确定是否将加密网关和 ASE 之间的连接。HTTP settings are created that determine whether the connection between the gateway and the ASE will be encrypted. 这些设置还用于替代传入的 HTTP 主机标头,其中包含从后端池中选取的主机名。These settings are also used to override the incoming HTTP host header with a host name picked from the backend pool. 此实现使用为默认 ASE 应用 Url 创建的默认证书,这些 Url 是网关信任的。This implementation uses default certificates created for the default ASE app URLs, which are trusted by the gateway. 请求将重定向到相应应用的默认 URL。The request is redirected to the default URL of the corresponding app. 链接到 VNet的专用 DNS 将此请求转发到 ILB IP。The private DNS linked to the VNet forwards this request to the ILB IP. 然后,ASE 将此转发到请求的应用服务。The ASE then forward this to the requested App service. ASE 应用内的任何 HTTP 通信都经过专用 DNS。Any HTTP communication within the ASE apps, goes through the private DNS. 请注意,需要在应用程序网关上为每个 ASE 应用配置侦听器、后端池、路由规则和 HTTP 设置。Note that the listener, backend pool, routing rule, and the HTTP settings need to be configured on the App Gateway for each ASE app.

浏览 appgw.js上 的,并 dns.js 了解如何进行这些配置以允许多个站点。Explore appgw.json and dns.json to understand how these configurations are made to allow multiple sites. 名为的 web 应用 testapp 是为演示此配置而创建的空应用程序。The web app named testapp is an empty app created to demonstrate this configuration. 可以从部署脚本 deploy_std sh访问这些 JSON 文件。它们还可通过 deploy_ha sh访问,用于 高可用性多站点 ASE 部署These JSON files are accessed from the deployment script deploy_std.sh. These are also accessed by deploy_ha.sh, which is used for a high availability multi-site ASE deployment.

安全注意事项Security considerations

应用服务环境App Service Environment

内部 ASE 部署在企业虚拟网络中,并在公共 internet 上隐藏。An internal ASE is deployed in the enterprise virtual network, hidden from the public internet. 它允许企业在网络级别锁定其后端服务,如 web Api 和功能。It allows the enterprise to lock down their backend services, such as web APIs and functions, at the network level. 使用 HTTP 终结点的任何 ASE 应用都可以从虚拟网络中通过 ILB 访问。Any ASE app with an HTTP endpoint, can be accessed through the ILB, from within the virtual network. 应用程序网关配置为通过 ILB 将请求转发到 web 应用。The App Gateway is configured to forward requests to the web app through the ILB. Web 应用本身通过 ILB 访问 API。The web app itself goes through the ILB to access the API. 关键后端组件(即 API 和函数)实际上无法从公共 internet 中访问。The critical backend components, that is, the API and the function, are effectively inaccessible from the public internet.

为 ASE 分配的默认域名创建每个应用服务的默认证书。Default certificates are created for each App service for the default domain name assigned by ASE. 此证书可以增强网关与应用之间的流量的安全性,在某些情况下可能需要此证书。This certificate can tighten the security of the traffic between the gateway and the app, and may be required in certain scenarios. 客户端浏览器将不会显示这些证书,只能响应在应用程序网关上配置的证书。These certificates will not be visible by the client browser, it can only respond to the certificates configured at the App Gateway.

应用程序网关App Gateway

引用实现以编程方式在的 appgw.js中配置应用程序网关。The reference implementation configures the App Gateway programmatically in the appgw.json. 部署网关时, deploy_std. sh 使用此配置:The file deploy_std.sh uses this configuration when deploying the gateway:

az deployment group create --resource-group $RGNAME --template-file templates/appgw.json --parameters vnetName=$VNET_NAME appgwSubnetAddressPrefix=$APPGW_PREFIX appgwApplications=@appgwApps.parameters.json
APPGW_PUBLIC_IP=$(az deployment group show -g $RGNAME -n appgw --query properties.outputs.appGwPublicIpAddress.value -o tsv)

加密Encryption

tls 终止和端到端 tls (使用应用程序网关)概述中所述,应用程序网关可以使用 传输层安全性 (TLS) /Secure 套接字层 (SSL) 来加密和保护来自 web 浏览器的所有流量。As described in the Overview of TLS termination and end to end TLS with Application Gateway, App Gateway can use Transport Layer Security (TLS)/Secure Sockets Layer (SSL) to encrypt and protect all traffic from web browsers. 可以通过以下方式配置加密:Encryption can be configured in the following ways:

  1. 加密在网关上终止Encryption terminated at the gateway. 在此示例中,将为 HTTP 配置后端池。The backend pools in this case are configured for HTTP. 加密停止于网关,网关和应用服务之间的流量未加密。The encryption stops at the gateway, and traffic between the gateway and the app service is unencrypted. 由于加密是占用大量 CPU 的,因此后端的未加密流量将提高性能,并允许更简单的证书管理。Since encryption is CPU-intensive, unencrypted traffic at the backend improves performance and allows simpler certificate management. 这提供了合理的安全级别,因为在这种情况下,会通过网络配置保护后端。This provides a reasonable level of security since the backend is secured by virtue of the network configuration.

  2. 端对端加密End to end encryption. 在某些情况下,例如特定的安全性或符合性要求,可能需要在网关和应用之间对流量进行加密。In some scenarios, such as specific security or compliance requirements, the traffic might be required to be encrypted between the gateway and the app. 这是通过使用 HTTPS 连接并在后端池配置证书来实现的。This is achieved by using HTTPS connection, and configuring certificates at the backend pool.

此引用实现使用适用于应用程序网关的自签名证书。This reference implementation uses self-signed certificates for App Gateway. 对于生产代码,应使用 证书颁发机构 颁发的证书。For production code, a certificate issued by a Certificate Authority should be used. 有关支持的证书类型的列表,请阅读 TLS 终止支持的证书For a list of supported certificate types, read Certificates supported for TLS termination. 阅读 使用 Azure 门户配置具有 TLS 终止的应用程序网关 ,了解如何创建网关终止的加密。Read Configure an application gateway with TLS termination using the Azure portal to learn how to create Gateway-terminated encryption. appgw.js上的以下代码行以编程方式配置此项:The following lines of code in the appgw.json configure this programmatically:

          {
            "name": "httpListeners",
            "count": "[length(parameters('appgwApplications'))]",
            "input": {
              "name": "[concat(variables('appgwListenerName'), parameters('appgwApplications')[copyIndex('httpListeners')].name)]",
              "properties": {
                "frontendIPConfiguration": {
                  "id": "[concat(variables('appgwId'), '/frontendIPConfigurations/', variables('appgwFrontendName'))]"
                },
                "frontendPort": {
                  "id": "[concat(variables('appgwId'), '/frontendPorts/port_443')]"
                },
                "protocol": "Https",
                "sslCertificate": {
                  "id": "[concat(variables('appgwId'), '/sslCertificates/', variables('appgwSslCertificateName'), parameters('appgwApplications')[copyIndex('httpListeners')].name)]"
                },
                "hostName": "[parameters('appgwApplications')[copyIndex('httpListeners')].hostName]",
                "requireServerNameIndication": true
              }
            }
          },

对于 ASE 上的 web 应用和网关之间的流量,将使用默认 SSL 证书。For traffic between the web apps on the ASE and the gateway, the default SSL certificates are used. 此实现中的后端池配置为使用由与 web 应用关联的默认域名替代的主机名重定向 HTTPS 流量。The backend pools in this implementation are configured to redirect HTTPS traffic with host name overridden by the default domain names associated to the web apps. 应用程序网关信任默认的 SSL 证书,因为这些证书由 Microsoft 颁发。The App Gateway trusts the default SSL certificates since they are issued by Microsoft. 请参阅 配置应用程序服务和应用程序网关 ,了解如何进行这些配置。Read Configure App Service with Application Gateway to know how these configurations are made. appgw.js中的以下行显示了如何在引用实现中配置此内容:The following lines in the appgw.json show how this is configured in the reference implementation:

         {
            "name": "backendHttpSettingsCollection",
            "count": "[length(parameters('appgwApplications'))]",
            "input": {
              "name": "[concat(variables('appgwHttpSettingsName'), parameters('appgwApplications')[copyIndex('backendHttpSettingsCollection')].name)]",
              "properties": {
                "Port": 443,
                "Protocol": "Https",
                "cookieBasedAffinity": "Disabled",
                "pickHostNameFromBackendAddress": true,
                "requestTimeout": 20,
                "probe": {
                  "id": "[concat(variables('appgwId'), '/probes/', variables('appgwHealthProbeName'), parameters('appgwApplications')[copyIndex('backendHttpSettingsCollection')].name)]"
                }
              }
            }
          },

Web 应用程序防火墙Web Application Firewall

应用程序网关上 (WAF) 的 Web 应用程序防火墙 可防止 Web 应用受到恶意攻击,例如 SQL 注入。Web Application Firewall (WAF) on the Application Gateway protects the web apps from malicious attacks, such as SQL injection. 它还与 Azure Monitor 集成,使用实时日志监视攻击。It is also integrated with Azure Monitor, to monitor attacks using a real-time log. 需要在网关上启用 WAF,如 使用 Azure 门户创建具有 Web 应用程序防火墙的应用程序网关中所述。WAF needs to be enabled on the gateway, as described in Create an application gateway with a Web Application Firewall using the Azure portal. 引用实现通过以下代码片段以编程方式在文件的 appgw.js中启用 WAF:The reference implementation enables WAF programmatically in the appgw.json file with the following snippet:

        "webApplicationFirewallConfiguration": {
          "enabled": true,
          "firewallMode": "Detection",
          "ruleSetType": "OWASP",
          "ruleSetVersion": "3.0"
        },

虚拟网络Virtual Network

网络安全组 可与 VNet 中的一个或多个子网相关联。Network security groups can be associated with one or more subnets in the VNet. 这些是允许或拒绝流量在各个 Azure 资源之间流动的安全规则。These are security rules that allow or deny traffic to flow between various Azure resources. 此体系结构为每个子网关联单独的 NSG。This architecture associates a separate NSG for each subnet. 这允许根据子网中包含的服务 (s) 对这些规则进行微调。This allows fine-tuning of these rules per subnet, as per the service(s) contained in that subnet. 例如, "type": "Microsoft.Network/networkSecurityGroups" 在用于 ASE 子网 NSG 的文件的文件 ase.js 中,以及 NSG For App 网关子网的文件 appgw.js 中浏览配置。For example, explore the configuration for "type": "Microsoft.Network/networkSecurityGroups" in the file ase.json for the NSG for the ASE subnet, and in the file appgw.json for the NSG for App Gateway subnet.

服务终结点 将 VNet 的专用地址空间和标识扩展到支持 Azure 服务,并将对这些服务的访问仅限制到 VNet。Service endpoints extend your VNet's private address space and identity to supporting Azure services, restricting the access to these services to only your VNet. 为了提高安全性,应在 ASE 子网中为支持它的任何 Azure 服务启用服务终结点。For improved security, service endpoints should be enabled on the ASE subnet for any Azure service that supports it. 然后,可以通过对服务设置虚拟网络规则来保护服务的企业 VNet,有效地阻止公共 internet 的访问。The service can then be secured to the enterprise VNet by setting virtual network rules on the service, effectively blocking access from public internet. ASE 基础结构设置事件中心、SQL Server 和存储的服务终结点,以便进行自己的管理,即使体系结构本身不能使用这些终结点(如 本文的系统体系结构部分所述)。ASE infrastructure sets up service endpoints for Event Hub, SQL Server, and Storage, for its own management, even if the architecture itself may not use them, as mentioned in the System Architecture section of this article. 此体系结构配置所使用服务的服务终结点,该服务终结点支持: Service Bus、SQL Server、Key Vault 和 CosmosDB。This architecture configures service endpoints for the services used, that support this: Service Bus, SQL Server, Key Vault, and CosmosDB. 上的ase.js中浏览此配置,如下所示 "type": "Microsoft.Network/virtualNetworks/subnets" --> "properties" --> "serviceEndpoints"Explore this configuration in the ase.json, as "type": "Microsoft.Network/virtualNetworks/subnets" --> "properties" --> "serviceEndpoints".

在子网上启用服务终结点 的过程分为两个步骤。Enabling service endpoints on a subnet is a two-step process. 需要配置这些资源,以便从此虚拟网络接收服务终结点上的流量。The resources themselves need to be configured to receive traffic on the service endpoints from this virtual network. 有关详细信息,请参阅 限制对资源的网络访问 For more information, read Restrict network access to a resource .

防火墙Firewall

Azure 防火墙和服务终结点彼此互为补充。Azure Firewall and service endpoints complement each other. 尽管服务终结点通过限制来自虚拟网络的传入流量来保护资源,但 Azure 防火墙允许你限制应用程序的出站流量。While service endpoints protect the resources by restricting incoming traffic to originate from your virtual network, Azure Firewall allows you to restrict the outbound traffic from your applications. 建议将所有出站流量传递给防火墙子网,包括服务终结点流量。It is recommended to let all outbound traffic to pass through the firewall subnet, including service endpoint traffic. 这样就可以更好地监视出站流量。This allows better monitoring of the outbound traffic. 不过,ASE 基础结构需要在 ASE 子网上配置 SQL Server、存储和事件中心的服务终结点。However, ASE infrastructure requires service endpoints for SQL Server, Storage, and Event Hubs to be configured on the ASE subnet. 有关防火墙集成的讨论,请参阅 锁定应用服务环境See the Locking down an App Service Environment for discussion on firewall integration. 此外,此实现 在直接连接模式下使用 CosmosDB,这需要打开一定范围的端口。Additionally, this implementation uses CosmosDB in direct connection mode, which requires a range of ports to be opened up. 这可能会使防火墙配置变得复杂。This may complicate the firewall configuration. 为简单起见,此参考体系结构配置 ASE 子网上的所有服务终结点,而不是防火墙子网。For simplicity, this reference architecture configures all service endpoints on the ASE subnet instead of the firewall subnet. 这包括服务总线、CosmosDB 和 Key Vault 的终结点,以及 ASE 基础结构所需的终结点。This includes endpoints for Service Bus, CosmosDB, and Key Vault, in addition to those required by the ASE infrastructure.

请参阅为 Ase 配置 Azure 防火墙 ,了解如何将防火墙与 ase 集成。See Configuring Azure Firewall with your ASE to learn how firewall is integrated with the ASE. 不会通过服务终结点和虚拟网络路由表的任何流量将受到防火墙的监视和控制。Any traffic not going over the service endpoints and virtual network route table, will be monitored and gated by the firewall. 下一节中描述了对路由表的需求。The need for the route table is described in the following section.

ASE 管理 IpASE Management IPs

ASE 管理流量通过企业虚拟网络流动。The ASE management traffic flows through the enterprise virtual network. 应将此流量直接路由到虚拟网络外部的专用 IP 地址,以避免防火墙。This traffic should be routed directly to the dedicated IP addresses outside the virtual network, avoiding the firewall. 这是通过创建 用户定义的虚拟网络路由表来实现的。This is achieved by creating a user-defined virtual network route table. 应用服务环境管理地址的文章列出了这些 IP 地址。The article App Service Environment management addresses lists these IP addresses. 手动在防火墙中配置此列表可能太长。This list can get too long to be configured in the firewall manually. 此实现演示如何以编程方式填充此。This implementation shows how this can be populated programmatically. Deploy_std中的以下行通过命令行 JSON 分析器使用 Azure CLI 和jq获取此列表:The following line in the deploy_std.sh obtains this list using Azure CLI and jq, a command line JSON parser:

ENDPOINTS_LIST=$(az rest --method get --uri $ASE_ID/inboundnetworkdependenciesendpoints?api-version=2016-09-01 | jq '.value[0].endpoints | join(", ")' -j)

这等效于 从 API 获取管理地址的记录方法。This is equivalent to the documented method of getting the management addresses from API.

Deploy_std中的以下命令将创建虚拟网络以及路由表,如network.js上的所配置:The following commands in the deploy_std.sh create the virtual network along with the route table, as configured in the network.json:

VNET_NAME=$(az network vnet list -g $RGNAME --query "[?contains(addressSpace.addressPrefixes, '${NET_PREFIX}')]" --query [0].name -o tsv)
az deployment group create --resource-group $RGNAME --template-file templates/network.json --parameters existentVnetName=$VNET_NAME vnetAddressPrefix=$NET_PREFIX
VNET_NAME=$(az deployment group show -g $RGNAME -n network --query properties.outputs.vnetName.value -o tsv)
VNET_ROUTE_NAME=$(az deployment group show -g $RGNAME -n network --query properties.outputs.vnetRouteName.value -o tsv)

创建 ASE 子网后,路由表与之关联:When the ASE subnet is created, the route table is associated with it:

az deployment group create --resource-group $RGNAME --template-file templates/ase.json -n ase --parameters vnetName=$VNET_NAME vnetRouteName=$VNET_ROUTE_NAME aseSubnetAddressPrefix=$ASE_PREFIX

创建防火墙后,配置文件 上的firewall.js 将使用 ASE 管理 ip 更新此路由表,后跟防火墙 IP。When the firewall is created, the firewall.json configuration file updates this route table with the ASE management IPs, followed by the firewall IP. 这会通过防火墙 IP 驱动剩余流量:This drives the remaining traffic through the firewall IP:

"variables": {
    "firewallSubnetName": "AzureFirewallSubnet",
    "firewallPublicIpName": "[concat('firewallIp', '-', uniqueString(resourceGroup().id))]",
    "firewallName": "[concat('firewall', '-', uniqueString(resourceGroup().id))]",
    "aseManagementEndpoints": "[split(replace(parameters('aseManagementEndpointsList') ,' ', ''), ',')]",
    "copy": [
      {
        "name": "aseManagementIpRoutes",
        "count": "[length(variables('aseManagementEndpoints'))]",
        "input": {
          "name": "[replace(variables('aseManagementEndpoints')[copyIndex('aseManagementIpRoutes')], '/', '-')]",
          "properties": {
            "addressPrefix": "[variables('aseManagementEndpoints')[copyIndex('aseManagementIpRoutes')]]",
            "nextHopType": "Internet"
          }
        }
      }
    ]
  },
  "resources": [
    {
      "type": "Microsoft.Network/routeTables",
      "apiVersion": "2019-11-01",
      "name": "[parameters('vnetRouteName')]",
      "location": "[parameters('location')]",
      "tags": {
        "displayName": "UDR - Subnet"
      },
      "dependsOn": [
        "[resourceId('Microsoft.Network/azureFirewalls', variables('firewallName'))]"
      ],
      "properties": {
        "routes": "[concat(variables('aseManagementIpRoutes'), array(json(concat('{ \"name\": \"Firewall\", \"properties\": { \"addressPrefix\": \"0.0.0.0/0\", \"nextHopType\": \"VirtualAppliance\", \"nextHopIpAddress\": \"', reference(concat('Microsoft.Network/azureFirewalls/', variables('firewallName')),'2019-09-01','Full').properties.ipConfigurations[0].properties.privateIPAddress, '\" } }'))))]"
      }
    },

部署路由表后,管理 IP 列表可能会更改,在这种情况下,将需要重新部署此路由表。The management IP list may change after the route table is deployed, in which case this route table will need to be redeployed.

Azure Active DirectoryAzure Active Directory

Azure Active Directory 提供了用于对应用程序进行身份验证和授权访问资源的安全功能。Azure Active Directory provides security features to authenticate applications and authorize access to resources. 此参考体系结构使用 Azure Active Directory 的两个主要功能:托管标识和 Azure 基于角色的访问控制。This reference architecture uses two key features of Azure Active Directory: managed identities, and Azure role-based access control.

构建云应用程序时,必须确保对云服务进行身份验证所需的凭据。When building cloud applications, the credentials required to authenticate to cloud services, must be secured. 理想情况下,凭据绝不会出现在开发人员工作站上或已签入源代码管理中。Ideally, the credentials should never appear on developer workstations or checked into source control. Azure Key Vault 提供了一种安全地存储凭据和机密的方式,但应用程序仍需要进行身份验证才能 Key Vault 检索它们。Azure Key Vault provides a way to securely store credentials and secrets, but the app still has to authenticate to Key Vault to retrieve them. Azure 资源的托管标识为 azure 服务提供了 Azure AD 中的自动托管标识。Managed Identities for Azure resources provides Azure services with an automatically managed identity in Azure AD. 此标识可用于向支持 Azure AD 身份验证的任何服务进行身份验证,包括 Key Vault,而无需在应用程序中使用任何凭据。This identity can be used to authenticate to any service that supports Azure AD authentication, including Key Vault, without any credentials in the application.

Azure RBAC) (azure 基于角色的访问控制管理对 azure 资源的访问权限。Azure role-based access control (Azure RBAC) manages access to Azure resources. 这包括:This includes:

  • 哪个实体具有访问权限: user、managed identity、security principal。which entity has the access: user, managed identity, security principal.
  • 它具有的访问权限类型:所有者、参与者、读者和管理员。what type of access it has: owner, contributor, reader, admin.
  • 访问作用域:资源、资源组、订阅或管理组。scope of the access: resource, resource group, subscription, or management group.

你可以通过严格控制每个应用所需的角色和访问类型,来锁定对 ASE 应用程序的访问。You can lock down access to ASE applications by tightly controlling the role required and the type of access for each app. 这样一来,就可以将多个应用程序部署到不同开发团队中的同一 ASE。This way, multiple apps can be deployed on the same ASE from different development teams. 例如,前端可能由一个团队处理,而后端则由另一个团队处理。For example, the frontend might be handled by one team, and the backend by another. 可以使用 Azure RBAC 来限制每个团队访问 () 其正在处理的应用。Azure RBAC can be used to limit each team's access to the app(s) it is working on. 了解 Azure 自定义角色 ,以创建适用于你的组织的角色。Explore Azure custom roles to create roles suitable to your organization.

Key VaultKey Vault

某些服务支持托管标识,但是它们使用 Azure RBAC 为应用设置权限。Some services support managed identities, however they use Azure RBAC to set up permissions for the app. 例如,请参阅服务总线 内置角色,以及 Azure Cosmos DB 中的 Azure RBACFor example, see Service Bus' built-in roles, as well as Azure RBAC in Azure Cosmos DB. 需要对订阅的 所有者 访问权限才能授予这些权限,即使具有 参与者 访问权限的人员可以部署这些服务也是如此。Owner access to the subscription is required for granting these permissions, even though personnel with Contributor access can deploy these services. 为了使更广泛的开发人员能够运行部署脚本,下一种最佳选择是使用该服务的本机访问控制策略:To allow a wider team of developers to be able to run the deployment scripts, the next best option is to use native access control policies of the service:

然后,将这些访问控制策略的连接字符串存储在 Key Vault 中。The connection strings for these access control policies are then stored in the Key Vault. 保管库本身可通过托管标识访问,这不需要 Azure RBAC。The vault itself is accessed through managed identities, which does not require Azure RBAC. 适当地设置这些连接字符串的访问策略。Set the access policy for these connection strings appropriately. 例如,对于后端,为只读,对于前端为只读,而不是使用默认根访问策略。For example, read-only for the backend, write-only for the frontend, and so on, instead of using default root access policy.

services.js上的以下代码片段显示了这些服务的 KeyVault 配置:The following snippet in services.json shows the KeyVault configuration for these services:

    {
      "type": "Microsoft.KeyVault/vaults/secrets",
      "name": "[concat(variables('keyVaultName'), '/CosmosKey')]",
      "apiVersion": "2015-06-01",
      "dependsOn": [
        "[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]",
        "[resourceId('Microsoft.DocumentDB/databaseAccounts', variables('cosmosName'))]"
      ],
      "properties": {
        "value": "[listKeys(resourceId('Microsoft.DocumentDB/databaseAccounts',variables('cosmosName')),'2015-04-08').primaryMasterKey]"
      }
    },
    {
      "type": "Microsoft.KeyVault/vaults/secrets",
      "name": "[concat(variables('keyVaultName'), '/ServiceBusListenerConnectionString')]",
      "apiVersion": "2015-06-01",
      "dependsOn": [
        "[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]",
        "[resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules', variables('serviceBusName'), 'ListenerSharedAccessKey')]"
      ],
      "properties": {
        "value": "[listKeys(resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules',variables('serviceBusName'),'ListenerSharedAccessKey'),'2015-08-01').primaryConnectionString]"
      }
    },
    {
      "type": "Microsoft.KeyVault/vaults/secrets",
      "name": "[concat(variables('keyVaultName'), '/ServiceBusSenderConnectionString')]",
      "apiVersion": "2015-06-01",
      "dependsOn": [
        "[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]",
        "[resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules', variables('serviceBusName'), 'SenderSharedAccessKey')]"
      ],
      "properties": {
        "value": "[listKeys(resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules',variables('serviceBusName'),'SenderSharedAccessKey'),'2015-08-01').primaryConnectionString]"
      }
    },

这些连接字符串值由应用通过引用 Key Vault 键/值对来访问。These connection string values are accessed by the apps, by referencing the Key Vault key/value pair. 此操作在 sites.js 的文件中完成,如以下代码片段所示:This is done in the sites.json file as the following snippet shows for the Voting App:

   "properties": {
        "enabled": true,
        "name": "[variables('votingWebName')]",
        "hostingEnvironment": "[variables('aseId')]",
        "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('votingWebPlanName'))]",
        "siteConfig": {
          "appSettings": [
            {
...
...
...
            {
              "name": "ConnectionStrings:sbConnectionString",
              "value": "[concat('@Microsoft.KeyVault(SecretUri=', reference(resourceId('Microsoft.KeyVault/vaults/secrets', parameters('keyVaultName'), variables('serviceBusSenderConnectionStringSecretName')), '2016-10-01').secretUriWithVersion, ')')]"
            },
...
...
            {
              "name": "ConnectionStrings:RedisConnectionString",
              "value": "[concat('@Microsoft.KeyVault(SecretUri=', reference(resourceId('Microsoft.KeyVault/vaults/secrets', parameters('keyVaultName'), variables('redisSecretName')), '2016-10-01').secretUriWithVersion, ')')]"
            },
...
...
            {
              "name": "ConnectionStrings:CosmosKey",
              "value": "[concat('@Microsoft.KeyVault(SecretUri=', reference(resourceId('Microsoft.KeyVault/vaults/secrets', parameters('keyVaultName'), variables('cosmosKeySecretName')), '2016-10-01').secretUriWithVersion, ')')]"
            }

函数还以类似方式访问服务总线侦听器连接字符串。The function also accesses the Service Bus listener connection string in a similar manner.

可伸缩性注意事项Scalability considerations

设计可缩放的应用Design scalable apps

此参考体系结构中的应用程序是结构化的,以便可以根据使用情况缩放单个组件。The application in this reference architecture is structured so that individual components can be scaled based on usage. 每个 web 应用、API 和函数都部署在其自己的应用服务计划中。Each web app, API, and function is deployed in its own App Service plan. 可以监视每个应用程序的性能瓶颈,并在需要时进行 扩展You can monitor each app for any performance bottlenecks, and then scale it up if required. 阅读 提高 Azure web 应用程序的可伸缩性 ,了解如何使用 Azure App Service 设计可缩放的 web 应用程序。Read Improve scalability in an Azure web application to learn how to design scalable web applications using Azure App Service.

自动缩放应用程序网关Autoscaling App Gateway

可以在 Azure 应用程序网关 V2 上启用自动缩放。Autoscaling can be enabled on Azure Application Gateway V2. 这允许应用程序网关根据流量负载模式进行扩展或缩减。This allows App Gateway to scale up or down based on the traffic load patterns. 此参考体系结构 autoscaleConfigurationappgw.js 的文件中进行配置,以在零到10个附加实例之间进行缩放。This reference architecture configures autoscaleConfiguration in the file appgw.json to scale between zero and 10 additional instances. 有关更多详细信息,请参阅 缩放应用程序网关和 WAF v2See Scaling Application Gateway and WAF v2 for more details.

部署注意事项Deployment considerations

此参考体系结构中的部署脚本用于部署 ASE、其他服务和 ASE 内的应用程序。The deployment scripts in this reference architecture are used to deploy ASE, other services, and the applications inside the ASE. 部署这些应用程序后,企业可能需要制定持续集成和部署的计划,以便进行应用程序维护和升级。Once these applications are deployed, enterprises might want to have a plan for continuous integration and deployment for app maintenance and upgrades. 本部分介绍开发人员用于 ASE 应用程序的 CI/CD 的一些常用方法。This section shows some of the common ways developers use for CI/CD of ASE applications.

只能从虚拟网络中将应用部署到内部 ASE。Apps can be deployed to an internal ASE only from within the virtual network. 以下三种方法广泛用于部署 ASE 应用:The following three methods are widely used to deploy ASE apps:

  1. 虚拟网络内部手动-在 ASE VNet 内创建虚拟机,并在其中创建部署所需的工具。Manually inside the Virtual Network - Create a virtual machine inside the ASE VNet with the required tools for the deployment. 使用 NSG 配置打开到 VM 的 RDP 连接。Open up the RDP connection to the VM using an NSG configuration. 将代码项目复制到 VM,生成并部署到 ASE 子网。Copy your code artifacts to the VM, build, and deploy to the ASE subnet. 这是设置初始生成和测试开发环境的一种简单方法。This is a simple way to set up an initial build and test development environment. 但对于生产环境,不建议这样做,因为它无法缩放所需的部署吞吐量。It is however not recommended for production environment since it cannot scale the required deployment throughput.

  2. 指向本地工作站中的站点连接 -这允许你将 ASE VNet 扩展到你的开发计算机,并从该位置进行部署。Point to site connection from local workstation - This allows you to extend your ASE VNet to your development machine, and deploy from there. 这是设置初始开发环境的另一种方法,不建议用于生产环境。This is another way to set up an initial dev environment, and not recommended for production.

  3. 通过 Azure Pipelines -实现完整的 CI/CD 管道,以位于 VNet 内的代理结尾。Through Azure Pipelines - This implements a complete CI/CD pipeline, ending in an agent located inside the VNet. 这非常适合需要高吞吐量部署的生产环境。This is ideal for production environments requiring high throughput of deployment. 生成管道完全保留在 VNet 的外部。The build pipeline remains entirely outside the VNet. 部署管道将生成的对象复制到 VNet 内的生成代理,然后部署到 ASE 子网。The deploy pipeline copies the built objects to the build agent inside the VNet, and then deploys to the ASE subnet. 有关详细信息,请阅读有关 管道和 ASE VNet 之间的自承载生成代理的此讨论。For more information, read this discussion on the self-hosted build agent between Pipelines and the ASE VNet.

建议为生产环境使用 Azure Pipelines。Using Azure Pipelines is recommended for production environments. 利用 AZURE PIPELINES YAML 架构 的帮助编写 CI/CD 脚本,有助于自动执行生成和部署过程。Scripting CI/CD with the help of Azure Pipelines YAML schema helps to automate the build and deployment processes. Azure-pipelines在此参考实现中为 web 应用实现此类 CI/CD 管道。The azure-pipelines.yml implements such a CI/CD pipeline for the web app in this reference implementation. WEB API函数都有类似的 CI/CD 脚本。There are similar CI/CD scripts for the web API as well as the function. 阅读 使用 Azure Pipelines 了解如何使用这些方法来自动执行每个应用程序的 CI/CD。Read Use Azure Pipelines to learn how these are used to automate CI/CD for each application.

某些企业可能不希望在 VNet 内维护永久性的生成代理。Some enterprises may not want to maintain a permanent build agent inside the VNet. 在这种情况下,可以选择在 DevOps 管道中创建生成代理,并在部署完成后将其关闭。In that case, you can choose to create a build agent within the DevOps pipeline, and tear it down once the deployment is completed. 这会在 DevOps 中添加另一个步骤,但这会降低 VM 维护的复杂性。This adds another step in the DevOps, however it lowers the complexity of maintaining the VM. 甚至可以考虑使用容器作为生成代理,而不是虚拟机。You may even consider using containers as build agents, instead of VMs. 通过从 位于 VNet 外部的压缩文件(通常在存储帐户中)进行部署,也可以完全避免生成代理。Build agents can also be completely avoiding by deploying from a zipped file placed outside the VNet, typically in a storage account. 需要从 ASE 访问存储帐户。The storage account will need to be accessible from the ASE. 管道应能够访问存储。The pipeline should be able to access the storage. 在发布管道结束时,可以将压缩文件放入 blob 存储。At the end of the release pipeline, a zipped file can be dropped into the blob storage. 然后,ASE 可以选取并部署。The ASE can then pick it up and deploy. 请注意此方法的以下限制:Be aware of the following limitations of this approach:

  • DevOps 与实际部署过程之间存在断开连接,从而导致监视和跟踪任何部署问题的困难。There is a disconnect between the DevOps and the actual deployment process, leading to difficulties in monitoring and tracing any deployment issues.
  • 在具有受保护流量的锁定环境中,可能需要更改规则以访问存储中的压缩文件。In a locked down environment with secured traffic, you may need to change the rules to access the zipped file on the storage.
  • 需要在 ASE 上安装特定的扩展和工具才能从 zip 部署。You will need to install specific extensions and tools on the ASE to be able to deploy from the zip.

若要了解如何将应用部署到应用服务计划的更多方式,请阅读 重点介绍部署策略的应用服务文章To know some more ways the apps can be deployed to the App Service plans, read the App Service articles focusing on deployment strategies.

成本注意事项Cost considerations

使用 Azure 定价计算器估算成本。Use the Azure pricing calculator to estimate costs. 其他注意事项,请参阅 Microsoft Azure Well-Architected 框架的 "成本" 部分。Other considerations are described in the Cost section in Microsoft Azure Well-Architected Framework. Azure 预留项可帮助你节省资金,因为它可以承诺许多 Azure 资源的一年期或三年期计划。Azure Reservations help you save money by committing to one-year or three-years plans for many Azure resources. 有关详细信息,请参阅 购买预订Read more in the article Buy a reservation.

对于此体系结构中使用的某些关键服务,请注意以下几点。Here are some points to consider for some of the key services used in this architecture.

应用服务环境App Service Environment

应用服务可以使用多种定价选项There are various pricing options available for App Service. 使用 独立服务计划部署应用服务环境。An App Service environment is deployed using the Isolated Service Plan. 在此计划中,有三个用于 CPU 大小-I1、I2 和 I3 的选项。Within this plan, there are three options for CPU sizes - I1, I2, and I3. 此参考实现正在每个实例使用三个 I1's。This reference implementation is using three I1's per instance.

应用程序网关Application Gateway

应用程序网关定价 提供适用于应用程序网关的各种定价选项。Application Gateway pricing provides various pricing options for App Gateway. 我们使用的是 应用程序网关 Standard_v2 和 WAF_V2 SKU,它支持自动缩放和区域冗余。We are using the Application Gateway Standard_v2 and WAF_v2 SKU, that supports auto scaling and zone redundancy. 阅读 本文 ,了解有关用于此 SKU 的定价模型的详细信息。Read this article to know more about the pricing model used for this SKU.

用于 Redis 的 Azure 缓存Azure Cache for Redis

Azure Cache For Redis 定价 提供此服务的各种定价选项。Azure Cache for Redis pricing provides the various pricing options for this service. 此体系结构使用 高级 SKU 作为虚拟网络支持。This architecture uses the Premium SKU, for the virtual network support.

下面是用于锁定 ASE 的其他服务的定价页:The following are pricing pages for other services used to lock down the ASE:

部署解决方案Deploy the solution

若要部署此体系结构的引用实现,请参阅 GitHub 自述文件,并遵循用于 标准部署 的脚本。To deploy the reference implementation for this architecture, see the GitHub readme, and follow the script for standard deployment.

后续步骤Next steps

若要了解如何扩展此体系结构以支持高可用性,请参阅 使用应用服务环境进行高可用性应用部署To learn how to extend this architecture to support high availability, read High availability app deployment using App Services Environment.