question

AkashVerma-9570 avatar image
1 Vote"
AkashVerma-9570 asked jikuja edited

Purpose of Allow public network access on Azure Data Factory

I created one Data Factory and i have one 'Networking' tab under settings.

I want to know the purpose of 'Allow public network access' setting under networking.

What is the impact of enabling and disabling it. What is the use of this control.22511-sscreenshot.jpg


azure-data-factory
sscreenshot.jpg (40.2 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

NandanHegde-7720 avatar image
0 Votes"
NandanHegde-7720 answered AkashVerma-9570 edited

Hi Akash,

There are multiple REST APIs for an Azure data factory which we can use to either create a pipeline, trigger a job, delete a pipeline etc.
Now in case if you disable the Network access, you need to whitelist specific Ips from which you can trigger the REST APIs.
In case if it is public, you can trigger them from any system.

This is a high level explanation for the networking part :)

Hope this clarifies your query.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks @NandanHegde-7720 for your response. Could you please let me know from where i can whitelist specific IP address, i am not able to find the option to whitelist IP in data factory.

And how can i connect with Data factory using Rest APIs (means what parameters are required - like azure portal username or password ) ?

To add, if i select disable it gives message "This Data Factory is only accessible via private endpoint"
so how then how can i access it then. If you can give more detailed info as i am unable to find it on internet

0 Votes 0 ·
KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered jikuja edited

Firstly, thank you so much @NandanHegde-7720 for sharing your inputs and helping the community.

Hi @AkashVerma-9570, Thank you for using Microsoft Q&A forum and reaching out regarding this query.

Regarding "This Data Factory is only accessible via private endpoint":

If you want to block public access to your Azure Data Factory and only allow access through Private Link, you can disable network access of Azure Data Factory in Azure portal.

Please note that disabling public network access is only applicable to Self-hosted Integration Runtime, not to Azure Integration Runtime and Azure SSIS Integration Runtime.

If you would like to disable Public Network access to ADF, then you will have to create a Private Endpoint. A private endpoint is a private IP address within a specific virtual network and subnet. To protect your Azure resources from attacks in public network or let them securely communicate with each other, you can set up an Azure Virtual Network as a logical representation of your network in the cloud. You can also connect an on-premises network to your virtual network by setting up IPSec VPN (site-to-site) or ExpressRoute (private peering). The Self-hosted Integration Runtime can be installed on an on- premise machine or virtual machine in Virtual Network to run copy activities between a cloud data store and a data store in a private network or dispatch transform activities against compute resources in an on-premises network or an Azure virtual network.

With the support of Azure Private Link for Azure Data Factory, you can create a Private Endpoint (PE) in your virtual network and enable the private connection to specific Azure Data Factory.

23928-image.png

As shown in the above image, the benefits of using private endpoint is that you can do authoring and monitoring of Azure Data Factory in your virtual network, even you block all outbound communications.

The command communications between Self-hosted Integration Runtime and Azure Data Factory service can be performed securely in a private network environment. The traffic between Self-hosted Integration Runtime and Azure Data Factory service goes through Private Link.

  1. Coming back to your question - " Could you please let me know from where i can whitelist specific IP address, i am not able to find the option to whitelist IP in data factory." - You do not need to apply whitelist on Data factory side, because Data factory initiates communication. This applies when you are using Self Hosted Integration Runtime to perform data movement using Azure Data Factory from an on Premise VM or an Azure VM or other data sources which support firewall (eg: Azure Data Lake store, Azure SQL, etc). For additional clarification please refer to : Firewall requirements for on-premises/private network

  2. 'And how can i connect with Data factory using Rest APIs (means what parameters are required - like azure portal username or password ) ?' - Here is a tutorial that explain step-by-step process on how to create an Azure data factory and pipeline by using the REST API: Quickstart: Create an Azure data factory and pipeline by using the REST API

Helpful resources for reference:

Security considerations for data movement in Azure Data Factory
Azure Private Link for Azure Data Factory
What is Azure Private Link?

Hope this info helps.

Thank you



Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.



image.png (131.5 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @AkashVerma-9570,

Just checking to see if the above information was helpful? If you have further query do let us know.

Thank you

0 Votes 0 ·

Hi @AkashVerma-9570,

We still have not heard back from you. Just wanted to check if the above response was helpful? Otherwise, let us know if you have further queries. Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

Thank you

0 Votes 0 ·

You do not need to apply whitelist on Data factory side, because Data factory initiates communication.

Not true because incoming ADF endpoints are still open for public internet. adf.azure.com is still available for authoring. The official document states following:

You can still access the Azure Data Factory portal through a public network after you create private endpoint for portal.

Also connections between SHIR and data factory are initiated from SHIR side: connections to: adf.azure.com, .{region}.datafactory.azure.net and .servicebus.windows.net. Usage of reverse connections would be impractical because SHIR would need to open ports on public internet.

The current name of the setting is more descriptive but still not perfect and Microsoft should use a lot of more time describing that the setting will harden only connection between SHIR and ADF. Documentation also fails to mention if it is possible to connect SHIR through public internet to ADF.

0 Votes 0 ·