question

NickBaker-1941 avatar image
0 Votes"
NickBaker-1941 asked MartinJaffer-MSFT commented

ADF Web activity can't connect to API When API Key has special character

I am trying to use a Web activity in an ADF pipeline to issue a GET to an API. The API authentication is through an Authorization header with an API key provided to be by the API developer. The API Key is of the form username:password (note the colon)

102542-image.png

When I try and authenticate against the API with my correct API key containing that colon character, I get the following error (sensitive data redacted):

"Error calling the endpoint 'https://api.XXXXX.com/XXX'. Response status code: ''. More details:Exception message: 'The format of value 'USERNAME:PASSWORD' is invalid.'. No response from the endpoint. Possible causes: network connectivity, DNS failure, server certificate validation or timeout."

When I try with an incorrect API key with no colon, I get a valid response from the API:

"{"payload":{"error_code":401,"message":"Not authorized."},"status":401,"requestTime":"2021-06-04T12:14:33","responseTime":"2021-06-04T12:14:33","statusText":"Unauthorized","url":"XXXX","method":"GET"}"

There is nothing wrong with my original API call - I have tried this exact setup in Postman and it works perfectly. I can only think that the way ADF is sending that header string differs in some way to how Postman sends it when there is a special character.

Any help would be appreciated!



azure-data-factory
image.png (51.2 KiB)
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @NickBaker-1941 and welcome to Microsoft Q&A. Thank you for bringing this to our attention.

I agree with your assessment. There are a number of possibilities.

  • escape the : as %3A (`@encodeUriComponent('USERNAME:PASSWORD')`)

  • escape the : as \: in the JSON code

  • there could be a deeper problem in Data Factory (needs testing)

The error sounds to me like somewhere in the Data Factory side, it is having trouble parsing the : and reading as "Header":"Value":"Value". The results of trying with each escape method should help give greater insight into the cause. I expect the resulting error(s) will be different.

Please let me know what you find.

Thank you.



0 Votes 0 ·

Thanks @MartinJaffer-MSFT - I've tried both options you've suggested:

Using @encodeUriComponent() results in the same 'Unauthorised' response from the API. I assume the replacement of the : with %3A results in an incorrect string being submitted.

Adding the escape character directly into the JSON code seems to result in an error in ADF of 'invalid JSON syntax' - screenshot below in case I've misunderstood your suggestion.

103051-screenshot-2021-06-07-120511.png


0 Votes 0 ·

Also just an aside - I don't have this issue when I use this exact same API as a linked service\dataset in a copy data activity. In fact I can return the data I want in this web activity and store it - that is my current workaround, but seems clunky when I only want to do a simple call.

0 Votes 0 ·

If you found your own solution, please share here with the community.

0 Votes 0 ·

0 Answers