question

GurvinderKandhola-1509 avatar image
0 Votes"
GurvinderKandhola-1509 asked PRADEEPCHEEKATLA-MSFT commented

ADF Unable to write to BLOB Storage using Private end Point

Can any one please help on this, we are stuck on this and this is in production.

We are getting this error message in ADF while writing to the BLOB (ADLS Gen2) storage.(
"Failure happened on 'Sink' side. Error Code=AdlsGen2OperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 operation failed for: Operation returned an invalid status code 'Forbidden'.Account ")
While using the **selected network*s option under Networking in Storage Account. We have given the Storage blob data contributor Role* to ADF in the Storage Account. This is working fine with All networks, but not with the Private End Point. We had created a private end point and have approved the permissions to the request from ADF , which is generated while creating the end point in data factory.

107192-image.png



azure-data-factoryazure-blob-storage
· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @GurvinderKandhola-1509 ,

Thank you for posting query in Microsoft Q&A platform. Could you please help on below clarifications to help you better in resolving issue.

  • Is linked service test connection is successful? If No, Kindly share error details.

Thank you

0 Votes 0 ·

Hi @ShaikMaheer-MSFT

Thanks for your reply.

In our case the Linked Service is working fine.
We have also given the approval to the ADF on the BLOB as shown in the below picture. Storage Blob Data Contributor Role has been provided to the Data Factory using managed identity. If we used the All Network option under Networking, then its working fine with the same Linked Service .Its failing (Ref : Previous post) when we are using Selected networks.
107694-pendptapprvl.jpg


0 Votes 0 ·
pendptapprvl.jpg (46.0 KiB)

Hi @PRADEEPCHEEKATLA-MSFT
When i click on the Accept Answer , it takes me to this error page
"404 - Page not found
Hmm, we could not find this link. If you feel you are seeing this page in error, please enter a Site feedback and tell us which page you expected to find."

0 Votes 0 ·

@GurvinderKandhola-1509 - Are you referring the 404 error when clicking on "Accept the answer" under the highlighted answer section ?


110745-image.png


0 Votes 0 ·
image.png (58.3 KiB)

Yes @Nandha22

1 Vote 1 ·

@GurvinderKandhola-1509 - Thanks for the confirmation. The Article moved to new location https://docs.microsoft.com/en-us/answers/support/accepted-answers and the old path is no longer exist. This could have created confusion here. Going forward we will share the new path.

1 Vote 1 ·
Show more comments

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
2 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT edited

Hi @GurvinderKandhola-1509,

As per my repro, I did experience the same error message as described above:

108896-image.png

Here are the steps which I followed to resolve the issue:

Step1: I had set Firewalls and virtual networks to selected networks:

108874-image.png

Step2: Approved the Data Factory requested private endpoint connection:

108780-image.png

Step3: Grant Data Factory service managed identity access to your Azure Data Lake Storage Gen2.

108944-image.png

Note: After completing the step3, please do wait at least for 5mins for the permission to reflect.

Step4: Created linked service for the Azure Storage account with Authentication method Managed Identity:

108935-image.png

Successfully able to run the copy activity pipeline without any issue:

108895-image.png

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.


image.png (93.1 KiB)
image.png (68.0 KiB)
image.png (83.8 KiB)
image.png (178.2 KiB)
image.png (67.9 KiB)
image.png (50.6 KiB)
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @GurvinderKandhola-1509,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·

Hi @PRADEEPCHEEKATLA-MSFT ,Yes this helped and I was able to resolve the issue. I was missing the Managed Identity part and were using the Account Key for Authentication. After changing the Authentication Method to Managed Identity, the Pipeline was working fine. Thanks for your Help.

0 Votes 0 ·

Hi @GurvinderKandhola-1509,

Glad to know that your issue has resolved. Now, you can "Accept the answer". This can be beneficial to other community members. Thank you.

0 Votes 0 ·

Thanks for the Reply, This helped to resolve the issue.

0 Votes 0 ·

Hi @GurvinderKandhola-1509,

Did you get a chance to click on Accept Answer?

0 Votes 0 ·