Understand on-premises data gateways for Microsoft Flow
Use the on-premises data gateway with Microsoft Flow to establish secure connections to your on-premises data sources such as Microsoft SQL Server.
Installation and configuration
- .NET Framework 4.6
- 64-bit version of Windows 7 or Windows Server 2008 R2 (or later)
- 8 Core CPU
- 8 GB Memory
- 64-bit version of Windows Server 2012 R2 (or later)
- You can't install a gateway on a domain controller.
- You shouldn't install a gateway on a computer, such a laptop, that may be turned off, asleep, or not connected to the Internet.
- Gateway performance might suffer over a wireless network.
Install a gateway
Microsoft SharePoint data gateways now support both HTTP and HTTPS traffic.
Download the installer, and then run it.
On the first screen of the installation wizard, select Next to acknowledge the reminder about installing a gateway on a laptop.
Select the installation location.
In the User Account Control dialog boxes, select Yes to continue.
On the On-premises data gateway screen, enter the email address for the account you will use to sign into the gateway, select Sign in, and then complete the sign in process.
Register new gateway or take over existing gateway
Select either Register a new gateway on this computer or Migrate, restore, or takeover an existing gateway, and then select Next.
To configure a new gateway, enter a name in the New on-premises data gateway name box, enter a recovery key in the Recovery key box, enter the same recovery key into the Confirm recovery key box. Select Configure, and then select Close.
Specify a recovery key that contains at least eight characters, and keep it in a safe place. You'll need this key if you want to migrate, restore, or take over its gateway.
To migrate, restore, or take over an existing gateway, provide the name of the gateway and its recovery key, select Configure, and then follow any additional prompts.
Restart the gateway
The gateway runs as a Windows service and, as with any other Windows service, you can start and stop it in multiple ways. For example, you can open a command prompt with elevated permissions on the machine where the gateway is running, and then run either of these commands:
- To stop the service, run this command:
net stop PBIEgwService
- To start the service, run this command:
net start PBIEgwService
Configure a firewall or proxy
For information about how to provide proxy information for your gateway, see Configure proxy settings.
You can verify whether your firewall, or proxy, may be blocking connections by running the following command from a PowerShell prompt. This command tests connectivity to the Azure Service Bus. This command only tests network connectivity and doesn't impact the cloud server service or the gateway. It helps to determine whether your machine has connectivity to the Internet.
Test-NetConnection -ComputerName watchdog.servicebus.windows.net -Port 9350
The results should look like the output below. If TcpTestSucceeded is not true, you may be blocked by a firewall.
ComputerName : watchdog.servicebus.windows.net RemoteAddress : 18.104.22.168 RemotePort : 5672 InterfaceAlias : vEthernet (Broadcom NetXtreme Gigabit Ethernet - Virtual Switch) SourceAddress : 10.120.60.105 PingSucceeded : False PingReplyDetails (RTT) : 0 ms TcpTestSucceeded : True
If you want to be exhaustive, substitute the ComputerName and Port values with those listed under Configure ports later in this topic.
The firewall may also be blocking the connections that the Azure Service Bus makes to the Azure data centers. If that's the case, you'll want to whitelist (unblock) all of the IP addresses for your region for those data centers.
The gateway creates an outbound connection to Azure Service Bus. It communicates on outbound ports: TCP 443 (default), 5671, 5672, 9350 thru 9354. The gateway doesn't require inbound ports.
Learn more about hybrid solutions.
|Domain names||Outbound ports||Description|
|*.servicebus.windows.net||5671-5672||Advanced Message Queuing Protocol (AMQP)|
|*.servicebus.windows.net||443, 9350-9354||Listeners on Service Bus Relay over TCP (requires 443 for Access Control token acquisition)|
|*.msftncsi.com||443||Used to test internet connectivity if the gateway is unreachable.|
If you need to white list IP addresses instead of the domains, you can download and use the Microsoft Azure Datacenter IP ranges list. In some cases, the Azure Service Bus connections will be made with IP address instead of the fully qualified domain names.
Users will sign in with either a work or school account. This is your organization account. If you signed up for an Office 365 offering and didn’t supply your work email, it may look like email@example.com. Your account, within a cloud service, is stored within a tenant in Azure Active Directory (AAD). In most cases, your AAD account’s UPN will match the email address.
Windows Service account
The on-premises data gateway is configured to use NT SERVICE\PBIEgwService for the Windows service logon credentials. By default, it has the right of Log on as a service. This is in the context of the machine on which you're installing the gateway.
This isn't the account used to connect to on-premises data sources or the work or school account with which you sign into cloud services.
Tenant level administration
There is currently no single place where tenant administrators can manage all the gateways that other users have installed and configured. If you’re a tenant administrator, we recommend that you ask the users in your organization to add you as an administrator to every gateway they install. This allows you to manage all the gateways in your organization through the Gateway Settings page or through PowerShell commands.
Frequently asked questions
Question: What data sources does the gateway support? Answer:
- SQL Server
Question: Do I need a gateway for data sources in the cloud, such as SQL Azure? Answer: No. A gateway connects to on-premises data sources only.
Question: Do I have to install my gateway in the default environment? Answer: Yes, Microsoft Flow only supports gateways that are installed in the default environment.
Question: What is the actual Windows service called? Answer: In Services, the gateway is called Power BI Enterprise Gateway Service.
Question: Are there any inbound connections to the gateway from the cloud? Answer: No. The gateway uses outbound connections to Azure Service Bus.
Question: What if I block outbound connections? What do I need to open? Answer: See the ports and hosts that the gateway uses.
Question: Does the gateway have to be installed on the same machine as the data source? Answer: No. The gateway will connect to the data source using the connection information that was provided. Think of the gateway as a client application in this sense. It will just need to be able to connect to the server name that was provided.
Question: What is the latency for running queries to a data source from the gateway? What is the best architecture? Answer: To reduce network latency, install the gateway as close to the data source as possible. If you can install the gateway on the actual data source, it will minimize the latency introduced. Consider the data centers as well. For example, if your service is using the West US data center and you have SQL Server hosted in an Azure VM, you'll want to have the Azure VM in West US as well. This will minimize latency and avoid egress charges on the Azure VM.
Question: Are there any requirements for network bandwidth? Answer: It is recommended to have good throughput for your network connection. Every environment is different, and the amount of data being sent will affect the results. Using ExpressRoute could help guarantee a level of throughput between on-premises and the Azure data centers.
You can use the third-party tool Azure Speed Test app to determine your throughput.
Question: Can the gateway Windows service run with an Azure Active Directory account? Answer: No. The Windows service must have a valid Windows account. By default, it will run with the Service SID, NT SERVICE\PBIEgwService.
Question: How are results sent to the cloud? Answer: Results are sent using Azure Service Bus. For more information, see how it works.
Question: Where are my credentials stored? Answer: The credentials that you enter for a data source are encrypted and stored in the gateway cloud service. The credentials are decrypted at the gateway on-premises.
High availability/disaster recovery
Question: Are there any plans for enabling high availability scenarios with the gateway? Answer: Yes, high availability is now available.
Question: What options are available for disaster recovery? Answer: You can use the recovery key to restore or move a gateway.
Question: What is the benefit of the recovery key? Answer: It provides a way to migrate or recover your gateway settings.
Question: Where are the gateway logs? Answer: See Tools later in this topic.
Question: How can I see what queries are being sent to the on-premises data source? Answer: You can enable query tracing, which will include the queries being sent. Remember to change it back to the original value when done troubleshooting. Leaving query tracing enabled will cause the logs to be larger.
You can also look at tools that your data source has for tracing queries. For example, you can use Extended Events or SQL Profiler for SQL Server and Analysis Services.
How the gateway works
When a user interacts with an element that's connected to an on-premises data source:
- The cloud service creates a query, along with the encrypted credentials for the data source, and sends the query to the queue for the gateway to process.
- The gateway cloud service analyzes the query and pushes the request to the Azure Service Bus.
- The on-premises data gateway polls the Azure Service Bus for pending requests.
- The gateway gets the query, decrypts the credentials, and connects to the data source(s) with those credentials.
- The gateway sends the query to the data source for execution.
- The results are sent from the data source back to the gateway and then onto the cloud service. The service then uses the results.
Update to the latest version
Many issues can surface when the gateway version is out of date. Ensure you're on the latest version. If you haven't updated the gateway recently, consider installing the latest version and see if you can reproduce the issue.
Error: Failed to add user to group. (-2147463168 PBIEgwService Performance Log Users )
You may receive this error if you're trying to install the gateway on a domain controller, which isn't supported. You'll need to install the gateway on a machine that isn't a domain controller.
Collecting logs from the gateway configurator
You can collect several logs for the gateway. Always start with the logs!
%localappdata%\Microsoft\on-premises data gateway\GatewayConfigurator*.log
Enterprise gateway service logs
C:\Users\PBIEgwService\AppData\Local\Microsoft\on-premises data gateway\Gateway*.log
The On-premises data gateway service event logs are present under Applications and Services Logs.
Fiddler is a free tool from Telerik that monitors HTTP traffic. You can see the back and forth with the Power BI service from the client machine. This may show errors and other related information.