使用 App Service 環境進行企業部署

Azure Active Directory
應用程式閘道
App Service
防火牆
Key Vault
Pipelines
虛擬網路

Azure App Service 是一項 PaaS 服務,可用來在 Azure 上裝載各種應用程式: web 應用程式、API 應用程式、功能和行動應用程式。Azure App Service is a PaaS service used to host a variety of apps on Azure: web apps, API apps, functions, and mobile apps. App Service 環境或 ASE 可讓企業在自己的 Azure 虛擬網路中,將其 App Service 應用程式部署在子網中,為其雲端工作負載提供隔離、可高度擴充且專用的環境。App Service Environment or ASE allows enterprises to deploy their App Service apps in a subnet in their own Azure Virtual Network, providing an isolated, highly scalable, and dedicated environment for their cloud workloads.

此參考架構示範使用 ASE 的一般企業工作負載,以及加強此工作負載安全性的最佳作法。This reference architecture demonstrates a common enterprise workload using ASE, and best practices to tighten security of this workload.

GitHub 標誌您 可在 github上取得此架構的參考實行。GitHub logo A reference implementation for this architecture is available on GitHub.

標準 ASE 部署的參考架構

架構Architecture

此架構的主要元件是 App Service 環境The main component of this architecture is the App Service environment. ASE 可部署為具有公用 IP 位址的 外部 ase ,或作為 內部 ase,且內部 IP 位址屬於 內部 Load Balancer (ILB)An ASE can be deployed either as external ASE with a public IP address, or as internal ASE, with an internal IP address belonging to the Internal Load Balancer (ILB). 此參考架構會在內部 ASE (也稱為 ILB ASE)中部署企業 web 應用程式。This reference architecture deploys an enterprise web application in an internal ASE, also called an ILB ASE. 當您的案例要求您執行下列動作時,請使用 ILB ASE:Use an ILB ASE when your scenario requires you to:

  • 在您可透過端對端或 ExpressRoute 存取的雲端,安全地裝載內部網路應用程式。Host intranet applications securely in the cloud, which you access through a site-to-site or ExpressRoute.
  • 使用 WAF 裝置保護應用程式。Protect apps with a WAF device.
  • 在雲端裝載未在公用 DNS 伺服器中列出的應用程式。Host apps in the cloud that aren't listed in public DNS servers.
  • 建立與網際網路隔離,且您的前端 app 可以安全地與之整合的後端應用程式。Create internet-isolated back-end apps, which your front-end apps can securely integrate with.

ASE 必須一律部署在企業虛擬網路內自己的子網中,以充分掌控連入和連出流量。An ASE must be always deployed in its own subnet within the enterprise' virtual network, allowing a tight control of the incoming and outgoing traffic. 在這個子網內,App Service 的應用程式會部署在一或多個 App Service 方案中,這是執行應用程式所需的實體資源集合。Within this subnet, App Service applications are deployed in one or more App Service plans, which is a collection of physical resources required to run the app. 根據複雜度和資源需求,可以在多個應用程式之間共用 App Service 的方案。Depending on the complexity and resource requirement, an App Service plan can be shared between multiple apps. 此參考實行會部署名為「 投票」應用 程式的 web 應用程式,該應用程式會與私用 web API 和函式互動。This reference implementation deploys a web app named Voting App, that interacts with a private web API and a function. 它也會部署名為 Test app 的虛擬 web 應用程式,以示範多應用程式部署。It also deploys a dummy web app named Test App to demonstrate multi-app deployments. 每個 App Service 應用程式都裝載于自己的 App Service 方案中,讓每個應用程式都能在必要時獨立調整。Each App Service app is hosted in its own App Service plan, allowing each to be scaled independently if necessary. 這些託管應用程式所需的所有資源(例如儲存體和計算),以及調整需求,都受到 App Service 環境基礎結構的管理。All resources required by these hosted apps, such as storage and compute, as well as scaling needs are fully managed by the App Service Environment infrastructure.

這項實行中的簡單投票應用程式可讓使用者查看現有的專案或建立新的專案,以及對現有專案進行投票。The simple voting app in this implementation, allows users to view existing or create new entries, as well as vote on the existing entries. Web API 會用來建立及抓取專案和投票。The web API is used for creation and retrieval of entries and votes. 資料本身會儲存在 SQL Server 資料庫中。The data itself is stored in a SQL Server database. 為了示範非同步資料更新,web 應用程式會在服務匯流排實例中將新加入的投票排入佇列。To demonstrate asynchronous data updates, the web app queues newly added votes in a Service Bus instance. 此函數會挑選佇列的投票,並更新 SQL 資料庫。The function picks up queued votes and updates the SQL database. CosmosDB 用來儲存模擬 ad,web 應用程式會在瀏覽器上進行抓取以顯示。CosmosDB is used to store a mock-up ad, that the web app retrieves to display on the browser. 應用程式會使用 Azure Cache for Redis 來進行快取管理。The application uses Azure Cache for Redis for cache management. 使用進階層 Azure Cache for Redis,可讓您在其自己的子網中,將設定至與 ASE 相同的虛擬網路。A Premium tier Azure Cache for Redis is used, which allows configuration to the same virtual network as the ASE, in its own subnet. 這可為快取提供增強的安全性和隔離。This provides enhanced security and isolation to the cache.

Web 應用程式是可透過應用程式閘道存取網際網路的唯一元件。The web apps are the only components accessible to the internet via the App Gateway. API 和函數無法從網際網路用戶端存取。The API and the function are inaccessible from an internet client. 輸入流量會受到應用程式閘道上設定的 Web 應用程式防火牆所保護。The inbound traffic is protected by a Web Application Firewall configured on the App Gateway.

下列服務是在此架構中鎖定 ASE 的關鍵:The following services are key to locking down the ASE in this architecture:

Azure 虛擬網路 或 VNet 是企業所擁有的私人 Azure 雲端網路。Azure Virtual Network or VNet is a private Azure cloud network owned by the enterprise. 它提供以網路為基礎的安全性和隔離。It provides network-based security and isolation. ASE 是在企業擁有的 VNet 子網中部署 App Service。ASE is an App Service deployment into a subnet of the enterprise-owned VNet. 它可讓企業使用 網路安全性群組服務端點,嚴密地控制網路空間和其所存取的資源。It allows the enterprise to tightly control that network space and the resources it accesses, using Network Security groups and Service Endpoints.

應用程式閘道 是應用層級的 web 流量負載平衡器、TLS/SSL 卸載,以及 web 應用程式防火牆 (WAF) 保護。Application Gateway is an application-level web traffic load balancer, with TLS/SSL offloading, and web application firewall (WAF) protection. 它會接聽公用 IP 位址,並將流量路由傳送至 ILB ASE 中的應用程式端點。It listens on a public IP address and routes traffic to the application endpoint in the ILB ASE. 由於這是應用層級的路由,因此可以將流量路由傳送至相同 ILB ASE 內的多個應用程式。Since this is application-level routing, it can route traffic to multiple apps within the same ILB ASE. 請參閱 應用程式閘道多網站裝載 ,以瞭解應用程式閘道如何支援多個網站。See Application Gateway multiple site hosting to learn how App Gateway supports multiple sites.

Azure 防火牆 可用來限制來自 web 應用程式的輸出流量。Azure Firewall is used to restrict the outbound traffic from the web application. 未通過服務端點通道的連出流量,以及 ASE 所需的路由表,會導向至防火牆子網。Outgoing traffic that does not go through the service endpoint channels and a route table required by ASE, is directed to the firewall subnet. 雖然建議 在防火牆子網上設定服務端點 以供追蹤,但這可能不一定是可行的。Although it is recommended for service endpoints to be configured on the firewall subnet for traceability, it may not be always feasible. 例如,ASE 基礎結構需要某些服務端點,才能在 ASE 子網上。For example, some service endpoints are required by the ASE infrastructure to be on the ASE subnet. 為了簡單起見,此架構會在 ASE 子網上設定所有的服務端點。For simplicity, this architecture configures all service endpoints on the ASE subnet.

Azure Active Directory 或 Azure AD 提供對 Azure 資源和服務的存取權和版權管理。Azure Active Directory or Azure AD provides access rights and permissions management to Azure resources and services. 受控 識別會將身分識別指派給服務和應用程式,並由 Azure AD 自動管理。Managed Identities assigns identities to services and apps, automatically managed by Azure AD. 這些身分識別可以用來向支援 Azure AD authentication 的任何服務進行驗證。These identities can be used to authenticate to any service that supports Azure AD authentication. 這樣就不需要為這些應用程式明確設定認證。This removes the need to explicitly configure credentials for these apps. 此架構會將受控識別指派給 web 應用程式。This architecture assigns a managed identity to the web app.

Key Vault 會儲存應用程式所需的任何秘密和認證。Key Vault stores any secrets and credentials required by the apps. 使用這個選項,將秘密直接儲存在應用程式中。Use this option over storing secrets directly in the application.

Azure Pipelines 在此架構中提供 持續整合和持續部署 功能。Azure Pipelines provides Continuous Integration and Continuous Deployment capabilities in this architecture. 因為 ASE 是虛擬網路的內部,所以會使用 虛擬機器 作為 VNet 內的 jumpbox ,以在 App Service 方案中部署應用程式。Since the ASE is internal to the virtual network, a virtual machine is used as a jumpbox inside the VNet to deploy apps in the App Service plans. 管線會在 VNet 外部建立應用程式。The pipeline builds the apps outside the VNet. 為了加強安全性和順暢的 RDP/SSH 連線能力,請考慮使用最近發行的 Azure 防禦作為 jumpbox。For enhanced security and seamless RDP/SSH connectivity, consider using the recently released Azure Bastion as the jumpbox.

多網站設定Multi-site configuration

多網站部署

內部 ASE 可以裝載數個具有 HTTP 端點的 web 應用程式和 Api。Internal ASE can host several web apps and APIs with HTTP endpoints. 這些應用程式會從公用網際網路鎖定,因為 ILB 的 IP 只能從虛擬網路中存取。These applications are locked down from the public internet as the ILB IP is only accessible from within the Virtual Network. 應用程式閘道可用來選擇性地將這些應用程式公開至網際網路。The Application Gateway is used to selectively expose these applications to the internet. ASE 會將每個 App Service 應用程式的預設 URL 指派為 <default-appName>.<ASE-domain>.appserviceenvironment.netASE assigns a default URL to each App Service application as <default-appName>.<ASE-domain>.appserviceenvironment.net. 建立 私人 DNS 區域 ,將 ase 功能變數名稱對應至 ASE ILB IP 位址。A private DNS zone is created that maps the ASE domain name to the ASE ILB IP address. 這可避免使用自訂 DNS 來存取 VNet 內的應用程式。This avoids using a custom DNS to access the apps within the VNet.

已設定應用程式閘道,讓 接聽程式 接聽 HTTPS 埠,以取得閘道 IP 位址的要求。The App Gateway is configured such that a listener listens on the HTTPS port for requests to the Gateway's IP address. 為了簡單起見,此執行不會使用公用 DNS 名稱註冊,而且需要您修改電腦上的 localhost 檔案,以將任意選擇的 URL 指向應用程式閘道的 IP。For simplicity, this implementation does not use a public DNS name registration, and requires you to modify the localhost file on your computer to point an arbitrarily chosen URL to the App Gateway's IP. 為了簡單起見,接聽程式會使用自我簽署的憑證來處理這些要求。For simplicity, the listener uses a self-signed certificate to process these requests. 應用程式閘道會針對每個 App Service 應用程式的預設 URL 提供 後端 集區。The App Gateway has backend pools for each App Service application's default URL. 路由規則設定為將接聽程式連接到後端集區。A routing rule is configured to connect the listener to the backend pool. 系統會建立HTTP 設定,以判斷閘道和 ASE 之間的連接是否會加密。HTTP settings are created that determine whether the connection between the gateway and the ASE will be encrypted. 這些設定也用來覆寫傳入 HTTP 主機標頭,並從後端集區挑選主機名稱。These settings are also used to override the incoming HTTP host header with a host name picked from the backend pool. 這項實行會使用針對預設 ASE 應用程式 Url 所建立的預設憑證,此憑證是由閘道所信任。This implementation uses default certificates created for the default ASE app URLs, which are trusted by the gateway. 要求會重新導向至對應應用程式的預設 URL。The request is redirected to the default URL of the corresponding app. 連結至 VNet的私人 DNS 會將此要求轉送至 ILB IP。The private DNS linked to the VNet forwards this request to the ILB IP. ASE 接著會將此轉送到要求的 App service。The ASE then forward this to the requested App service. ASE 應用程式內的任何 HTTP 通訊都會通過私人 DNS。Any HTTP communication within the ASE apps, goes through the private DNS. 請注意,必須在應用程式閘道上為每個 ASE 應用程式設定接聽程式、後端集區、路由規則和 HTTP 設定。Note that the listener, backend pool, routing rule, and the HTTP settings need to be configured on the App Gateway for each ASE app.

探索 appgw.js ,並 dns.js 瞭解如何進行這些設定以允許多個網站。Explore appgw.json and dns.json to understand how these configurations are made to allow multiple sites. 名為的 web 應用程式 testapp 是建立的空白應用程式,用來示範這項設定。The web app named testapp is an empty app created to demonstrate this configuration. 這些 JSON 檔案可從部署腳本 deploy_std sh進行存取。這些也可透過 deploy_ha,用來進行 高可用性多網站 ASE 部署These JSON files are accessed from the deployment script deploy_std.sh. These are also accessed by deploy_ha.sh, which is used for a high availability multi-site ASE deployment.

安全性考量Security considerations

App Service 環境App Service Environment

內部 ASE 會部署在企業虛擬網路中,並在公用網際網路中隱藏。An internal ASE is deployed in the enterprise virtual network, hidden from the public internet. 它可讓企業在網路層級鎖定其後端服務(例如 web Api 和函式)。It allows the enterprise to lock down their backend services, such as web APIs and functions, at the network level. 任何具有 HTTP 端點的 ASE 應用程式,都可以從虛擬網路內的 ILB 存取。Any ASE app with an HTTP endpoint, can be accessed through the ILB, from within the virtual network. 應用程式閘道已設定為透過 ILB 將要求轉送至 web 應用程式。The App Gateway is configured to forward requests to the web app through the ILB. Web 應用程式本身會經歷 ILB 來存取 API。The web app itself goes through the ILB to access the API. 重要的後端元件(即 API 和函式)實際上無法從公用網際網路存取。The critical backend components, that is, the API and the function, are effectively inaccessible from the public internet.

針對 ASE 所指派的預設功能變數名稱,會為每個 App service 建立預設憑證。Default certificates are created for each App service for the default domain name assigned by ASE. 此憑證可以加強閘道和應用程式之間流量的安全性,而且在某些情況下可能會需要此憑證。This certificate can tighten the security of the traffic between the gateway and the app, and may be required in certain scenarios. 用戶端瀏覽器將看不到這些憑證,只會回應在應用程式閘道上設定的憑證。These certificates will not be visible by the client browser, it can only respond to the certificates configured at the App Gateway.

應用程式閘道App Gateway

參考實行會以程式設計的方式,在 appgw.js中設定應用程式閘道。The reference implementation configures the App Gateway programmatically in the appgw.json. 在部署閘道時, deploy_std 的檔案會使用此設定:The file deploy_std.sh uses this configuration when deploying the gateway:

az deployment group create --resource-group $RGNAME --template-file templates/appgw.json --parameters vnetName=$VNET_NAME appgwSubnetAddressPrefix=$APPGW_PREFIX appgwApplications=@appgwApps.parameters.json
APPGW_PUBLIC_IP=$(az deployment group show -g $RGNAME -n appgw --query properties.outputs.appGwPublicIpAddress.value -o tsv)

加密Encryption

如同 使用應用程式閘道的 tls 終止和端對端 tls 的總覽所述,應用程式閘道可以使用 傳輸層安全性 (TLS) /Secure 通訊端層 (SSL) ,來加密及保護來自網頁瀏覽器的所有流量。As described in the Overview of TLS termination and end to end TLS with Application Gateway, App Gateway can use Transport Layer Security (TLS)/Secure Sockets Layer (SSL) to encrypt and protect all traffic from web browsers. 您可以透過下列方式設定加密:Encryption can be configured in the following ways:

  1. 加密在閘道上終止Encryption terminated at the gateway. 此案例中的後端集區是針對 HTTP 進行設定。The backend pools in this case are configured for HTTP. 加密會在閘道上停止,且閘道與 app service 之間的流量不會加密。The encryption stops at the gateway, and traffic between the gateway and the app service is unencrypted. 由於加密需要耗用大量 CPU,因此後端未加密的流量會改善效能,並允許更簡單的憑證管理。Since encryption is CPU-intensive, unencrypted traffic at the backend improves performance and allows simpler certificate management. 這會提供合理的安全性層級,因為後端會受到網路設定的保護。This provides a reasonable level of security since the backend is secured by virtue of the network configuration.

  2. 端對端加密End to end encryption. 在某些情況下(例如特定安全性或合規性需求),可能需要在閘道和應用程式之間加密流量。In some scenarios, such as specific security or compliance requirements, the traffic might be required to be encrypted between the gateway and the app. 這可透過使用 HTTPS 連線,並在後端集區設定憑證來達成。This is achieved by using HTTPS connection, and configuring certificates at the backend pool.

此參考程式會使用適用于應用程式閘道的自我簽署憑證。This reference implementation uses self-signed certificates for App Gateway. 若為實際執行程式碼,則應使用由 憑證授權單位 單位所發行的憑證。For production code, a certificate issued by a Certificate Authority should be used. 如需支援的憑證類型清單,請參閱 TLS 終止支援的憑證For a list of supported certificate types, read Certificates supported for TLS termination. 若要瞭解如何建立閘道終止的加密,請參閱 使用 Azure 入口網站設定 TLS 終止的應用程式閘道Read Configure an application gateway with TLS termination using the Azure portal to learn how to create Gateway-terminated encryption. appgw.js中的下列幾行程式碼會以程式設計方式進行設定:The following lines of code in the appgw.json configure this programmatically:

          {
            "name": "httpListeners",
            "count": "[length(parameters('appgwApplications'))]",
            "input": {
              "name": "[concat(variables('appgwListenerName'), parameters('appgwApplications')[copyIndex('httpListeners')].name)]",
              "properties": {
                "frontendIPConfiguration": {
                  "id": "[concat(variables('appgwId'), '/frontendIPConfigurations/', variables('appgwFrontendName'))]"
                },
                "frontendPort": {
                  "id": "[concat(variables('appgwId'), '/frontendPorts/port_443')]"
                },
                "protocol": "Https",
                "sslCertificate": {
                  "id": "[concat(variables('appgwId'), '/sslCertificates/', variables('appgwSslCertificateName'), parameters('appgwApplications')[copyIndex('httpListeners')].name)]"
                },
                "hostName": "[parameters('appgwApplications')[copyIndex('httpListeners')].hostName]",
                "requireServerNameIndication": true
              }
            }
          },

針對 ASE 和閘道上的 web 應用程式之間的流量,會使用預設 SSL 憑證。For traffic between the web apps on the ASE and the gateway, the default SSL certificates are used. 此實行中的後端集區會設定為使用與 web 應用程式相關聯的預設功能變數名稱覆寫的主機名稱來重新導向 HTTPS 流量。The backend pools in this implementation are configured to redirect HTTPS traffic with host name overridden by the default domain names associated to the web apps. 應用程式閘道會信任預設 SSL 憑證,因為它們是由 Microsoft 所發行。The App Gateway trusts the default SSL certificates since they are issued by Microsoft. 請參閱 使用應用程式閘道設定 App Service ,以瞭解如何進行這些設定。Read Configure App Service with Application Gateway to know how these configurations are made. appgw.js中的下列幾行會示範如何在參考實行中設定:The following lines in the appgw.json show how this is configured in the reference implementation:

         {
            "name": "backendHttpSettingsCollection",
            "count": "[length(parameters('appgwApplications'))]",
            "input": {
              "name": "[concat(variables('appgwHttpSettingsName'), parameters('appgwApplications')[copyIndex('backendHttpSettingsCollection')].name)]",
              "properties": {
                "Port": 443,
                "Protocol": "Https",
                "cookieBasedAffinity": "Disabled",
                "pickHostNameFromBackendAddress": true,
                "requestTimeout": 20,
                "probe": {
                  "id": "[concat(variables('appgwId'), '/probes/', variables('appgwHealthProbeName'), parameters('appgwApplications')[copyIndex('backendHttpSettingsCollection')].name)]"
                }
              }
            }
          },

Web 應用程式防火牆Web Application Firewall

應用程式閘道上的 Web 應用程式防火牆 (WAF) 可保護 web 應用程式免于遭受惡意攻擊,例如 SQL 插入式攻擊。Web Application Firewall (WAF) on the Application Gateway protects the web apps from malicious attacks, such as SQL injection. 它也與 Azure 監視器整合,以使用即時記錄來監視攻擊。It is also integrated with Azure Monitor, to monitor attacks using a real-time log. 您必須在閘道上啟用 WAF,如 使用 Azure 入口網站建立具有 Web 應用程式防火牆的應用程式閘道所述。WAF needs to be enabled on the gateway, as described in Create an application gateway with a Web Application Firewall using the Azure portal. 參考實 WAF 會使用下列程式碼片段,以程式設計的方式在檔案的 appgw.js中啟用:The reference implementation enables WAF programmatically in the appgw.json file with the following snippet:

        "webApplicationFirewallConfiguration": {
          "enabled": true,
          "firewallMode": "Detection",
          "ruleSetType": "OWASP",
          "ruleSetVersion": "3.0"
        },

虛擬網路Virtual Network

網路安全性群組 可以與 VNet 中的一或多個子網相關聯。Network security groups can be associated with one or more subnets in the VNet. 這些是安全性規則,可允許或拒絕在不同 Azure 資源之間流動的流量。These are security rules that allow or deny traffic to flow between various Azure resources. 此架構會為每個子網建立個別的 NSG 關聯。This architecture associates a separate NSG for each subnet. 這可讓您根據子網所包含的每個服務 () ,微調這些規則。This allows fine-tuning of these rules per subnet, as per the service(s) contained in that subnet. 例如,在的檔案ase.js的設定 "type": "Microsoft.Network/networkSecurityGroups" 中, 探索 ASE 子網的 NSG, appgw.js以及 NSG for App Gateway 子網的檔案中的設定。For example, explore the configuration for "type": "Microsoft.Network/networkSecurityGroups" in the file ase.json for the NSG for the ASE subnet, and in the file appgw.json for the NSG for App Gateway subnet.

服務端點 會擴充您 vnet 的私人位址空間和身分識別,以支援 Azure 服務,將這些服務的存取限制為只有您的 vnet。Service endpoints extend your VNet's private address space and identity to supporting Azure services, restricting the access to these services to only your VNet. 為了提高安全性,您應該針對支援的任何 Azure 服務,在 ASE 子網上啟用服務端點。For improved security, service endpoints should be enabled on the ASE subnet for any Azure service that supports it. 然後,您可以在服務上設定虛擬網路規則,將服務安全地保護企業 VNet,以有效地封鎖公用網際網路的存取。The service can then be secured to the enterprise VNet by setting virtual network rules on the service, effectively blocking access from public internet. ASE 基礎結構會設定事件中樞、SQL Server 和儲存體的服務端點,以進行自己的管理,即使架構本身可能不會使用它們,如本文的 系統架構一節中所述。ASE infrastructure sets up service endpoints for Event Hub, SQL Server, and Storage, for its own management, even if the architecture itself may not use them, as mentioned in the System Architecture section of this article. 此架構會設定所使用服務的服務端點,其支援:服務匯流排、SQL Server、Key Vault 和 CosmosDB。This architecture configures service endpoints for the services used, that support this: Service Bus, SQL Server, Key Vault, and CosmosDB. ase.js中探索此設定,如下所示 "type": "Microsoft.Network/virtualNetworks/subnets" --> "properties" --> "serviceEndpoints"Explore this configuration in the ase.json, as "type": "Microsoft.Network/virtualNetworks/subnets" --> "properties" --> "serviceEndpoints".

在子網上啟用服務端點 是一個兩步驟的處理常式。Enabling service endpoints on a subnet is a two-step process. 資源本身必須設定為從這個虛擬網路接收服務端點的流量。The resources themselves need to be configured to receive traffic on the service endpoints from this virtual network. 如需詳細資訊,請參閱 限制對資源的網路存取 For more information, read Restrict network access to a resource .

防火牆Firewall

Azure 防火牆和服務端點彼此互補。Azure Firewall and service endpoints complement each other. 當服務端點藉由限制來自您虛擬網路的連入流量來保護資源時,Azure 防火牆可讓您限制來自您應用程式的輸出流量。While service endpoints protect the resources by restricting incoming traffic to originate from your virtual network, Azure Firewall allows you to restrict the outbound traffic from your applications. 建議讓所有輸出流量通過防火牆子網,包括服務端點流量。It is recommended to let all outbound traffic to pass through the firewall subnet, including service endpoint traffic. 這可讓您更妥善地監視輸出流量。This allows better monitoring of the outbound traffic. 不過,ASE 基礎結構需要在 ASE 子網上設定 SQL Server、儲存體和事件中樞的服務端點。However, ASE infrastructure requires service endpoints for SQL Server, Storage, and Event Hubs to be configured on the ASE subnet. 如需有關防火牆整合的討論,請參閱 鎖定 App Service 環境See the Locking down an App Service Environment for discussion on firewall integration. 此外,這項實行會使用 直接連線模式中的 CosmosDB,需要開啟某個範圍的埠。Additionally, this implementation uses CosmosDB in direct connection mode, which requires a range of ports to be opened up. 這可能會使防火牆設定變得複雜。This may complicate the firewall configuration. 為了簡單起見,此參考架構會在 ASE 子網上設定所有的服務端點,而不是設定防火牆子網。For simplicity, this reference architecture configures all service endpoints on the ASE subnet instead of the firewall subnet. 這包括服務匯流排、CosmosDB 和 Key Vault 的端點,以及 ASE 基礎結構所需的端點。This includes endpoints for Service Bus, CosmosDB, and Key Vault, in addition to those required by the ASE infrastructure.

請參閱 使用 ase 設定 Azure 防火牆 ,以瞭解如何將防火牆與 ase 整合。See Configuring Azure Firewall with your ASE to learn how firewall is integrated with the ASE. 任何未通過服務端點和虛擬網路路由表的流量,都會受到防火牆的監視和閘道。Any traffic not going over the service endpoints and virtual network route table, will be monitored and gated by the firewall. 下一節將說明路由表的需求。The need for the route table is described in the following section.

ASE 管理 IpASE Management IPs

ASE 管理流量會流經企業虛擬網路。The ASE management traffic flows through the enterprise virtual network. 此流量應直接路由傳送至虛擬網路外的私人 IP 位址,以避免防火牆。This traffic should be routed directly to the dedicated IP addresses outside the virtual network, avoiding the firewall. 這是藉由建立 使用者定義的虛擬網路路由表來達成。This is achieved by creating a user-defined virtual network route table. App Service 環境管理位址的文章會列出這些 IP 位址。The article App Service Environment management addresses lists these IP addresses. 這份清單可能太久,無法手動在防火牆中設定。This list can get too long to be configured in the firewall manually. 此實作為示範如何以程式設計的方式填入。This implementation shows how this can be populated programmatically. Deploy_std中的下面這一行會使用 Azure CLI 和jq(命令列 JSON 剖析器)取得這份清單:The following line in the deploy_std.sh obtains this list using Azure CLI and jq, a command line JSON parser:

ENDPOINTS_LIST=$(az rest --method get --uri $ASE_ID/inboundnetworkdependenciesendpoints?api-version=2016-09-01 | jq '.value[0].endpoints | join(", ")' -j)

這相當於 從 API 取得管理位址的記錄方法。This is equivalent to the documented method of getting the management addresses from API.

Deploy_std中的下列命令會建立虛擬網路以及路由表,如network.js中所設定:The following commands in the deploy_std.sh create the virtual network along with the route table, as configured in the network.json:

VNET_NAME=$(az network vnet list -g $RGNAME --query "[?contains(addressSpace.addressPrefixes, '${NET_PREFIX}')]" --query [0].name -o tsv)
az deployment group create --resource-group $RGNAME --template-file templates/network.json --parameters existentVnetName=$VNET_NAME vnetAddressPrefix=$NET_PREFIX
VNET_NAME=$(az deployment group show -g $RGNAME -n network --query properties.outputs.vnetName.value -o tsv)
VNET_ROUTE_NAME=$(az deployment group show -g $RGNAME -n network --query properties.outputs.vnetRouteName.value -o tsv)

建立 ASE 子網時,路由表會與其相關聯:When the ASE subnet is created, the route table is associated with it:

az deployment group create --resource-group $RGNAME --template-file templates/ase.json -n ase --parameters vnetName=$VNET_NAME vnetRouteName=$VNET_ROUTE_NAME aseSubnetAddressPrefix=$ASE_PREFIX

建立防火牆時,設定檔 上的firewall.js 會使用 ASE 管理 ip 來更新此路由表,後面接著防火牆 IP。When the firewall is created, the firewall.json configuration file updates this route table with the ASE management IPs, followed by the firewall IP. 這會透過防火牆 IP 驅動剩餘的流量:This drives the remaining traffic through the firewall IP:

"variables": {
    "firewallSubnetName": "AzureFirewallSubnet",
    "firewallPublicIpName": "[concat('firewallIp', '-', uniqueString(resourceGroup().id))]",
    "firewallName": "[concat('firewall', '-', uniqueString(resourceGroup().id))]",
    "aseManagementEndpoints": "[split(replace(parameters('aseManagementEndpointsList') ,' ', ''), ',')]",
    "copy": [
      {
        "name": "aseManagementIpRoutes",
        "count": "[length(variables('aseManagementEndpoints'))]",
        "input": {
          "name": "[replace(variables('aseManagementEndpoints')[copyIndex('aseManagementIpRoutes')], '/', '-')]",
          "properties": {
            "addressPrefix": "[variables('aseManagementEndpoints')[copyIndex('aseManagementIpRoutes')]]",
            "nextHopType": "Internet"
          }
        }
      }
    ]
  },
  "resources": [
    {
      "type": "Microsoft.Network/routeTables",
      "apiVersion": "2019-11-01",
      "name": "[parameters('vnetRouteName')]",
      "location": "[parameters('location')]",
      "tags": {
        "displayName": "UDR - Subnet"
      },
      "dependsOn": [
        "[resourceId('Microsoft.Network/azureFirewalls', variables('firewallName'))]"
      ],
      "properties": {
        "routes": "[concat(variables('aseManagementIpRoutes'), array(json(concat('{ \"name\": \"Firewall\", \"properties\": { \"addressPrefix\": \"0.0.0.0/0\", \"nextHopType\": \"VirtualAppliance\", \"nextHopIpAddress\": \"', reference(concat('Microsoft.Network/azureFirewalls/', variables('firewallName')),'2019-09-01','Full').properties.ipConfigurations[0].properties.privateIPAddress, '\" } }'))))]"
      }
    },

部署路由表之後,管理 IP 清單可能會變更,在此情況下,必須重新部署此路由表。The management IP list may change after the route table is deployed, in which case this route table will need to be redeployed.

Azure Active DirectoryAzure Active Directory

Azure Active Directory 可提供安全性功能來驗證應用程式,並授權資源的存取權。Azure Active Directory provides security features to authenticate applications and authorize access to resources. 此參考架構使用 Azure Active Directory 的兩個主要功能:受控識別,以及 Azure 角色型存取控制。This reference architecture uses two key features of Azure Active Directory: managed identities, and Azure role-based access control.

當您建立雲端應用程式時,必須保護向雲端服務驗證所需的認證。When building cloud applications, the credentials required to authenticate to cloud services, must be secured. 在理想情況下,認證絕對不應該出現在開發人員工作站上,也不會簽入原始檔控制。Ideally, the credentials should never appear on developer workstations or checked into source control. Azure Key Vault 提供安全地儲存認證和秘密的方式,但應用程式仍必須向 Key Vault 驗證才能取得它們。Azure Key Vault provides a way to securely store credentials and secrets, but the app still has to authenticate to Key Vault to retrieve them. 適用于 azure 資源的受控 識別會在 Azure AD 中,為 azure 服務提供自動管理的身分識別。Managed Identities for Azure resources provides Azure services with an automatically managed identity in Azure AD. 此身分識別可用來向任何支援 Azure AD 驗證的服務進行驗證,包括 Key Vault,而不需要在應用程式中使用任何認證。This identity can be used to authenticate to any service that supports Azure AD authentication, including Key Vault, without any credentials in the application.

Azure角色型存取控制 (AZURE RBAC) 管理對 azure 資源的存取。Azure role-based access control (Azure RBAC) manages access to Azure resources. 這包括:This includes:

  • 哪些實體具有存取權:使用者、受控識別、安全性主體。which entity has the access: user, managed identity, security principal.
  • 其擁有的存取類型:擁有者、參與者、讀者、管理員。what type of access it has: owner, contributor, reader, admin.
  • 存取範圍:資源、資源群組、訂用帳戶或管理群組。scope of the access: resource, resource group, subscription, or management group.

您可以藉由嚴格控制所需的角色和每個應用程式的存取類型,來鎖定 ASE 應用程式的存取權。You can lock down access to ASE applications by tightly controlling the role required and the type of access for each app. 如此一來,多個應用程式可以部署在不同開發小組的相同 ASE 上。This way, multiple apps can be deployed on the same ASE from different development teams. 例如,前端可能由一個小組處理,而後端則是另一個。For example, the frontend might be handled by one team, and the backend by another. 您可以使用 Azure RBAC 來限制每個小組在處理 () 應用程式的存取權。Azure RBAC can be used to limit each team's access to the app(s) it is working on. 探索 Azure 自訂角色 ,以建立適合您組織的角色。Explore Azure custom roles to create roles suitable to your organization.

Key VaultKey Vault

某些服務支援受控識別,不過它們會使用 Azure RBAC 來設定應用程式的許可權。Some services support managed identities, however they use Azure RBAC to set up permissions for the app. 例如,請參閱服務匯流排的內 建角色,以及 Azure Cosmos DB 中的 Azure RBACFor example, see Service Bus' built-in roles, as well as Azure RBAC in Azure Cosmos DB. 您必須擁有訂用帳戶的 擁有 者存取權,才能授與這些許可權,即使具有 參與者 存取權的人員可以部署這些服務也是一樣。Owner access to the subscription is required for granting these permissions, even though personnel with Contributor access can deploy these services. 為了讓更廣大的開發人員能夠執行部署腳本,下一個最佳選項是使用服務的原生存取控制原則:To allow a wider team of developers to be able to run the deployment scripts, the next best option is to use native access control policies of the service:

這些存取控制原則的連接字串接著會儲存在 Key Vault 中。The connection strings for these access control policies are then stored in the Key Vault. 保存庫本身會透過受控識別來存取,而不需要 Azure RBAC。The vault itself is accessed through managed identities, which does not require Azure RBAC. 適當地設定這些連接字串的存取原則。Set the access policy for these connection strings appropriately. 例如,後端的唯讀、前端的唯讀等,而不是使用預設的根存取原則。For example, read-only for the backend, write-only for the frontend, and so on, instead of using default root access policy.

services.js中的下列程式碼片段會顯示這些服務的 KeyVault 設定:The following snippet in services.json shows the KeyVault configuration for these services:

    {
      "type": "Microsoft.KeyVault/vaults/secrets",
      "name": "[concat(variables('keyVaultName'), '/CosmosKey')]",
      "apiVersion": "2015-06-01",
      "dependsOn": [
        "[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]",
        "[resourceId('Microsoft.DocumentDB/databaseAccounts', variables('cosmosName'))]"
      ],
      "properties": {
        "value": "[listKeys(resourceId('Microsoft.DocumentDB/databaseAccounts',variables('cosmosName')),'2015-04-08').primaryMasterKey]"
      }
    },
    {
      "type": "Microsoft.KeyVault/vaults/secrets",
      "name": "[concat(variables('keyVaultName'), '/ServiceBusListenerConnectionString')]",
      "apiVersion": "2015-06-01",
      "dependsOn": [
        "[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]",
        "[resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules', variables('serviceBusName'), 'ListenerSharedAccessKey')]"
      ],
      "properties": {
        "value": "[listKeys(resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules',variables('serviceBusName'),'ListenerSharedAccessKey'),'2015-08-01').primaryConnectionString]"
      }
    },
    {
      "type": "Microsoft.KeyVault/vaults/secrets",
      "name": "[concat(variables('keyVaultName'), '/ServiceBusSenderConnectionString')]",
      "apiVersion": "2015-06-01",
      "dependsOn": [
        "[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]",
        "[resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules', variables('serviceBusName'), 'SenderSharedAccessKey')]"
      ],
      "properties": {
        "value": "[listKeys(resourceId('Microsoft.ServiceBus/namespaces/AuthorizationRules',variables('serviceBusName'),'SenderSharedAccessKey'),'2015-08-01').primaryConnectionString]"
      }
    },

這些連接字串值是由應用程式存取,方法是參考 Key Vault 機碼/值組。These connection string values are accessed by the apps, by referencing the Key Vault key/value pair. 這是在檔案的 sites.js 中完成,如下列程式碼片段所示:投票應用程式:This is done in the sites.json file as the following snippet shows for the Voting App:

   "properties": {
        "enabled": true,
        "name": "[variables('votingWebName')]",
        "hostingEnvironment": "[variables('aseId')]",
        "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('votingWebPlanName'))]",
        "siteConfig": {
          "appSettings": [
            {
...
...
...
            {
              "name": "ConnectionStrings:sbConnectionString",
              "value": "[concat('@Microsoft.KeyVault(SecretUri=', reference(resourceId('Microsoft.KeyVault/vaults/secrets', parameters('keyVaultName'), variables('serviceBusSenderConnectionStringSecretName')), '2016-10-01').secretUriWithVersion, ')')]"
            },
...
...
            {
              "name": "ConnectionStrings:RedisConnectionString",
              "value": "[concat('@Microsoft.KeyVault(SecretUri=', reference(resourceId('Microsoft.KeyVault/vaults/secrets', parameters('keyVaultName'), variables('redisSecretName')), '2016-10-01').secretUriWithVersion, ')')]"
            },
...
...
            {
              "name": "ConnectionStrings:CosmosKey",
              "value": "[concat('@Microsoft.KeyVault(SecretUri=', reference(resourceId('Microsoft.KeyVault/vaults/secrets', parameters('keyVaultName'), variables('cosmosKeySecretName')), '2016-10-01').secretUriWithVersion, ')')]"
            }

函數也會以類似的方式存取服務匯流排接聽程式連接字串。The function also accesses the Service Bus listener connection string in a similar manner.

延展性考量Scalability considerations

設計可調整的應用程式Design scalable apps

此參考架構中的應用程式是結構化的,因此可以根據使用方式調整個別的元件。The application in this reference architecture is structured so that individual components can be scaled based on usage. 每個 web 應用程式、API 和函式都會部署在自己的 App Service 方案中。Each web app, API, and function is deployed in its own App Service plan. 您可以監視每個應用程式是否有任何效能瓶頸,然後視需要相應 增加You can monitor each app for any performance bottlenecks, and then scale it up if required. 閱讀 改善 Azure web 應用程式中 的擴充性,以瞭解如何使用 Azure App Service 來設計可調整的 web 應用程式。Read Improve scalability in an Azure web application to learn how to design scalable web applications using Azure App Service.

自動調整應用程式閘道Autoscaling App Gateway

可以在 Azure 應用程式閘道 V2 上啟用自動調整。Autoscaling can be enabled on Azure Application Gateway V2. 這可讓應用程式閘道根據流量負載模式來擴大或縮小。This allows App Gateway to scale up or down based on the traffic load patterns. 此參考架構會 autoscaleConfiguration 在檔案 appgw.js 中設定,以在零和10個額外的實例之間進行調整。This reference architecture configures autoscaleConfiguration in the file appgw.json to scale between zero and 10 additional instances. 如需詳細資訊,請參閱 調整應用程式閘道和 WAF v2See Scaling Application Gateway and WAF v2 for more details.

部署考量因素Deployment considerations

此參考架構中的部署腳本會用來部署 ase、其他服務,以及 ASE 內的應用程式。The deployment scripts in this reference architecture are used to deploy ASE, other services, and the applications inside the ASE. 部署這些應用程式之後,企業可能會想要針對應用程式維護和升級規劃持續整合和部署。Once these applications are deployed, enterprises might want to have a plan for continuous integration and deployment for app maintenance and upgrades. 本節說明開發人員用於 ASE 應用程式 CI/CD 的一些常用方式。This section shows some of the common ways developers use for CI/CD of ASE applications.

應用程式只能從虛擬網路內部部署到內部 ASE。Apps can be deployed to an internal ASE only from within the virtual network. 下列三種方法廣泛用來部署 ASE 應用程式:The following three methods are widely used to deploy ASE apps:

  1. 在虛擬網路內部手動 建立虛擬機器,並使用部署所需的工具來建立 ASE VNet 內的虛擬機器。Manually inside the Virtual Network - Create a virtual machine inside the ASE VNet with the required tools for the deployment. 使用 NSG 設定開啟對 VM 的 RDP 連線。Open up the RDP connection to the VM using an NSG configuration. 將您的程式碼成品複製到 VM、建立並部署至 ASE 子網。Copy your code artifacts to the VM, build, and deploy to the ASE subnet. 這是設定初始組建和測試開發環境的簡單方式。This is a simple way to set up an initial build and test development environment. 但不建議用於生產環境,因為它無法調整所需的部署輸送量。It is however not recommended for production environment since it cannot scale the required deployment throughput.

  2. 從本機工作站的點對站 連線-這可讓您將 ASE VNet 延伸至您的開發電腦,並從該處部署。Point to site connection from local workstation - This allows you to extend your ASE VNet to your development machine, and deploy from there. 這是設定初始開發環境的另一種方式,不建議用於生產環境。This is another way to set up an initial dev environment, and not recommended for production.

  3. 透過 Azure Pipelines -這會執行完整的 CI/CD 管線,並以位於 VNet 內的代理程式結尾。Through Azure Pipelines - This implements a complete CI/CD pipeline, ending in an agent located inside the VNet. 這適用于需要高輸送量部署的生產環境。This is ideal for production environments requiring high throughput of deployment. 組建管線完全維持在 VNet 之外。The build pipeline remains entirely outside the VNet. 部署管線會將建立的物件複製到 VNet 內的組建代理程式,然後部署至 ASE 子網。The deploy pipeline copies the built objects to the build agent inside the VNet, and then deploys to the ASE subnet. 如需詳細資訊,請閱讀這 段關於管線與 ASE VNet 之間自我裝載組建代理程式的討論。For more information, read this discussion on the self-hosted build agent between Pipelines and the ASE VNet.

建議在生產環境中使用 Azure Pipelines。Using Azure Pipelines is recommended for production environments. 使用 AZURE PIPELINES YAML 架構 的協助編寫 CI/CD 的腳本有助於將組建和部署程式自動化。Scripting CI/CD with the help of Azure Pipelines YAML schema helps to automate the build and deployment processes. >azure-pipelines.yml會在此參考實行中,為 web 應用程式執行這類的 CI/CD 管線。The azure-pipelines.yml implements such a CI/CD pipeline for the web app in this reference implementation. WEB API函數都有類似的 CI/CD 腳本。There are similar CI/CD scripts for the web API as well as the function. 請參閱使用 Azure Pipelines ,瞭解如何使用這些應用程式將每個應用程式的 CI/CD 自動化。Read Use Azure Pipelines to learn how these are used to automate CI/CD for each application.

某些企業可能不會想要在 VNet 內維護永久的組建代理程式。Some enterprises may not want to maintain a permanent build agent inside the VNet. 在此情況下,您可以選擇在 DevOps 管線內建立組建代理程式,並在部署完成後將其卸載。In that case, you can choose to create a build agent within the DevOps pipeline, and tear it down once the deployment is completed. 這會在 DevOps 中新增另一個步驟,但它會降低維護 VM 的複雜度。This adds another step in the DevOps, however it lowers the complexity of maintaining the VM. 您甚至可以考慮使用容器做為組建代理程式,而不是 Vm。You may even consider using containers as build agents, instead of VMs. 組建代理程式也可以完全避免,方法是從 放置於 VNet 外部的 zip 壓縮檔 進行部署,通常是在儲存體帳戶中。Build agents can also be completely avoiding by deploying from a zipped file placed outside the VNet, typically in a storage account. 儲存體帳戶將需要可從 ASE 存取。The storage account will need to be accessible from the ASE. 管線應該能夠存取儲存體。The pipeline should be able to access the storage. 在發行管線結束時,可將壓縮檔案放入 blob 儲存體。At the end of the release pipeline, a zipped file can be dropped into the blob storage. ASE 接著可以挑選並部署。The ASE can then pick it up and deploy. 請注意這種方法的下列限制:Be aware of the following limitations of this approach:

  • DevOps 和實際的部署程式之間有中斷連線,導致監視和追蹤任何部署問題的困難。There is a disconnect between the DevOps and the actual deployment process, leading to difficulties in monitoring and tracing any deployment issues.
  • 在具有安全流量的鎖定環境中,您可能需要變更規則以存取儲存體上的 zip 壓縮檔案。In a locked down environment with secured traffic, you may need to change the rules to access the zipped file on the storage.
  • 您將需要在 ASE 上安裝特定的擴充功能和工具,才能從 zip 部署。You will need to install specific extensions and tools on the ASE to be able to deploy from the zip.

若要瞭解一些可將應用程式部署至 App Service 方案的其他方法,請閱讀 著重于部署策略的 App Service 文章To know some more ways the apps can be deployed to the App Service plans, read the App Service articles focusing on deployment strategies.

成本考量Cost considerations

使用 Azure 定價計算機來估計成本。Use the Azure pricing calculator to estimate costs. Microsoft Azure Well-Architected Framework的「成本」一節中會說明其他考慮。Other considerations are described in the Cost section in Microsoft Azure Well-Architected Framework. Azure 保留可協助您藉由承諾多項 Azure 資源一年或三年期的方案來節省成本。Azure Reservations help you save money by committing to one-year or three-years plans for many Azure resources. 如需詳細資訊,請參閱 購買保留專案。Read more in the article Buy a reservation.

以下是此架構中所使用的一些重要服務的一些考慮事項。Here are some points to consider for some of the key services used in this architecture.

App Service 環境App Service Environment

App Service 有各種 定價選項可用There are various pricing options available for App Service. 使用 隔離服務方案部署 App Service 環境。An App Service environment is deployed using the Isolated Service Plan. 在此方案中,有三個適用于 CPU 大小的選項: I1、I2 和 I3。Within this plan, there are three options for CPU sizes - I1, I2, and I3. 此參考實行會針對每個實例使用三個 I1's。This reference implementation is using three I1's per instance.

應用程式閘道Application Gateway

應用程式閘道定價 提供適用于應用程式閘道的各種定價選項。Application Gateway pricing provides various pricing options for App Gateway. 我們使用的 應用程式閘道 Standard_v2 和 WAF_V2 SKU,可支援自動調整和區域冗余。We are using the Application Gateway Standard_v2 and WAF_v2 SKU, that supports auto scaling and zone redundancy. 閱讀本文 ,以深入瞭解此 SKU 所使用的定價模型。Read this article to know more about the pricing model used for this SKU.

Azure Cache for RedisAzure Cache for Redis

Azure Cache for Redis 定價 提供適用于此服務的各種定價選項。Azure Cache for Redis pricing provides the various pricing options for this service. 此架構會使用 PREMIUM SKU 來支援虛擬網路。This architecture uses the Premium SKU, for the virtual network support.

以下是用來鎖定 ASE 的其他服務的定價頁面:The following are pricing pages for other services used to lock down the ASE:

部署解決方案Deploy the solution

若要部署此架構的參考執行,請參閱 GitHub 讀我檔案,並遵循 標準部署 的腳本。To deploy the reference implementation for this architecture, see the GitHub readme, and follow the script for standard deployment.

後續步驟Next steps

若要瞭解如何擴充此架構以支援高可用性,請 使用應用程式服務環境來讀取高可用性應用程式部署To learn how to extend this architecture to support high availability, read High availability app deployment using App Services Environment.