question

NagarajanAnand-3891 avatar image
0 Votes"
NagarajanAnand-3891 asked HarithaMaddi-MSFT commented

How to disable server side validation when using web activity in DataFactory

We are planning to use data factory to fetch data from an API . The API provider has implemented a Client Credentials apporach where we will have to first call an API with a client secret and Client password to obtain a token . The obtained token will be used subsequently used with other Api calls to get data.

Our challenge here is that when we try to call the API to get the tokens , the web activity returns a 2108 error and it turns out the issue is due to the fact that server side validation certification fails and we are unable to figure out how to the disable server side validation in the web api.

Please note that the same call works in Postman when the server side validation is disabled.

Could you guide us how we can overcome this problem , is using webapi justified or is there any other activity that is more apt for this and means to use the same
58312-config.png
Note that we have set the header to the correct Content type and have included the granttype , client_id and client_secret values without quotes in the body


azure-data-factoryazure-webapps-apis
config.png (23.6 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @NagarajanAnand-3891,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

Please share the pipeline run id that is throwing the error as that would help us to access logs for further investigation.

Looking forward for your response!

0 Votes 0 ·

Hei @HarithaMaddi-MSFT
Here is the pipeline id : 569959e7-8442-489b-8267-8cc13009c710
I can add that the error code is : 2108
and that when i test the same url using postman with ssl disabled i see a warning " unable to get local issuer certificate" , although i get the results
and when i test the same using postman with ssl enabled , i get an error "unable to get local issuer certificate"

0 Votes 0 ·

Hi @NagarajanAnand-3891,

Thanks for sharing the details. I have shared the information with product team and waiting to hear back from them. I will get back to you as soon as I get more insights.

Stay tuned!

0 Votes 0 ·

1 Answer

HarithaMaddi-MSFT avatar image
0 Votes"
HarithaMaddi-MSFT answered HarithaMaddi-MSFT commented

Hi @NagarajanAnand-3891,

Thanks for your patience. Product team mentioned it as a bug and they are working on fixing it by having an approach to add the following feature flag to the end of the data factory URL which will allow to select an option in the web activity to disable cert validation as below. This will be available in future releases of data factory and please follow the updates page to check this once available.

 &feature.disableWebCertValidation=true

A new option will appear once the url is appended in web activity as below, please check the box, save and publish your pipeline, then, try executing it again.

60129-image.png

Please let us know for further queries and we will be glad to assist.


  • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification.




image.png (21.9 KiB)
· 15
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @HarithaMaddi-MSFT
Unfortunately , this does not function even after adding the feature.disablewebcertvalidation = true on the url .
I have also tried adding the same to the headers and as well to the body but still the same error.

Here is the pipeline ID : 314693d9-e3bb-4280-adda-ea376eb9c498

1 Vote 1 ·

Thanks @NagarajanAnand-3891 for confirming the workaround suggested. I shared the same to product team and I will get back to you as soon as I hear from them.

Appreciate your support!

0 Votes 0 ·

Thanks @NagarajanAnand-3891 for sharing the additional details. Product team mentioned that feature flag has to be appended to data factory URL and then these options appear where BASIC auth can be selected. From the shared pipeline run id, it is observed that the feature flag is appended to web activity URL but not MADRID UI.

Please check and let us know if this is working fine else share the pipeline run id with us again to look at the details.

0 Votes 0 ·
Show more comments

Hi @HarithaMaddi-MSFT
Thank you very much for your quick response. However the issue is still not solved.
As the authentication scheme that we use is basic authentication
Here is the pipeline ID :
25fdc616-3622-4ba3-bf54-a0e24dba7754

0 Votes 0 ·

Hi @NagarajanAnand-3891,

Thanks for sharing the details. Is there any possibility to switch to this authentication mode for your requirement. I also reached out to product team to understand if there is any other workaround that can be used with Basic authentication and will update you once I hear from them.

0 Votes 0 ·

hi @HarithaMaddi-MSFT Thank you very much for your quick response. Unfortunately we are not able to switch to anyother authentication other than basic authentication as the solution is delivered by a third party.
The third party solutioroviders have implemented oauth with a bearer token approach and the flow consists of authenticating first with a user name and password to get the bearer token and using the bearer token in subsequent calls.

0 Votes 0 ·

Hi @NagarajanAnand-3891,

Thanks for sharing more details. Product team confirmed that we can use BASIC authentication by adding "&feature.disableWebCertValidation=true" to the URL and it should work well without server side validation. Please let us know how it goes and if issue persists, please share pipeline run id with us to share with internal team for additional investigation.

0 Votes 0 ·

Hi @NagarajanAnand-3891,

As discussed, feature flag needs to be added to the URL of the data factory, sample one is as shown below and then it will work without server validation in web activity.

Please let us know for further queries and we will be glad to assist.

 https://ms-adf.azure.com/en-us/authoring/pipeline/pipeline1?factory=%2Fsubscriptions%xxxx%2FresourceGroups%2Fxxx%2Fproviders%2FMicrosoft.DataFactory%2Ffactories%2Fharithaadf73&l=en-us&feature.disableWebCertValidation=true

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members


0 Votes 0 ·

Hi @HarithaMaddi-MSFT
Thank you very much for your help , but it still does not work
ON appending the feature.disableWebCertValidation= true to the datafactory Url , I get a non trusted error and then iam redirected back to the select page for data factory

Here is a snapshot of the error that i get

62387-image.png


0 Votes 0 ·
image.png (11.3 KiB)

Thanks @NagarajanAnand-3891 for sharing. I reached out to product team with this error to know their thoughts, I will get back to you once I hear from them.

0 Votes 0 ·

Thanks @NagarajanAnand-3891 for sharing the details. Product team confirmed that it is a bug and they are working on it. The changes should be live in future releases. Apologize the inconvenience.

Please let us know for further queries and we will be glad to assist. Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members.

0 Votes 0 ·
Show more comments

Hi @HarithaMaddi-MSFT
As it stands it is a bug at your end.
Once the bug is fixed , I can test the same and after it works can i accept the answer, isnt that correct.
Could you also let me know if there are any other work arounds, say use of another data factory task which perhaps iam not aware of to address the issue ?

0 Votes 0 ·

Hi @NagarajanAnand-3891,

Thanks for understanding. The bug deployment may take a month or more depending on the priority of the release items and it is recommended to accept answer as other customers knows that it is confirmed as a bug and awaiting deployment who are looking at community forum.

As per my understanding, there is no possible workaround for this scenario using data factory activities other than custom coding. Please suggest if this helps in clarifying the question.

0 Votes 0 ·