question

DunkleyScott-7708 avatar image
0 Votes"
DunkleyScott-7708 asked HanyanWang-3936 commented

ADF Schema import get 403, copy also gets 403, Linked service test is fine

i am trying to setup ADF to connect to HTTP API source and get a CSV file.

Works fine in Postman so I know basic auth is working ok.

I setup linked service and test connection which is fine, auth set to Basic and same credentials as used in Postman

I then setup delimited dataset to use linked service and it wont let me import schema, getting 403 Forbiden

So I tried to create the copy data object anyway but get the same 403 error on debug.

i have tried self hosted Ir and autoresolve, both work when doing Linked Service "test connection".

i also tried changing the base URL to be the full URL in the linked service so no relatuive URL required. Again Linked service test is fine but dataset schema import gets 403

What am I missing here, its like my auth details are being applied from dataset.

azure-data-factory
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @DunkleyScott-7708,


Welcome to Microsoft Q&A platform and thanks for your query.


Could you please share complete error message that you are receiving? Have you tried using Web activity before Copy activity to get the access token? If not could you please try having a web activity before the copy to get the access token from an AAD app that has access to the API endpoint your are calling and then use the token as a header in the source settings (additional headers) as described in below document.


Related MS doc: https://docs.microsoft.com/azure/data-factory/control-flow-web-activity


In case if you still encounter the issue, please do trigger run the pipeline and share below info for further assistance.


What is the linked service (Connector) used? If it is REST connector, what is the IR used?



Failed activity runID



Failed pipeline runID



Region of your data factory?




Do let us know how it goes.





Thank you




0 Votes 0 ·

Thanks for reply, will try getting token first.

Not really getting as far as pipeline run as I cant even get an import of the schema from dataset.

Error is Failed to read data from http server. Check the error from http server:The remote server returned an error: (403) Forbidden. The remote server returned an error: (403) Forbidden. . Activity ID: eef980a6-f84c-4568-8487-733e8bd0b3ec

ADF is un UK West region

Will report back with findings

0 Votes 0 ·

Thanks for your response @DunkleyScott-7708. Please let us know your findings after passing access token from web activity to your API call.

0 Votes 0 ·

Hi @DunkleyScott-7708, Just wanted to followup to see if you have any progress using Access token in your HTTP API call ? Please let us know how it goes.

Thanks

0 Votes 0 ·
Show more comments

1 Answer

DunkleyScott-7708 avatar image
2 Votes"
DunkleyScott-7708 answered HanyanWang-3936 commented

its fixed thanks, was down to a 64 bit encoding issue

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @DunkleyScott-7708, Glad to know your issue was resolved and appreciate much for sharing your findings here with community, as it would be helpful for others. I have converted your comment to an answer. Please feel free to 'Accept Answer' as it would help others to find the solution easily.



Thank you

0 Votes 0 ·

Hi,

I met the same problem, when I tried to preview the source data of a copy activity, it prompted 403. If I use POSTMAN to test, it works fine.
Can you elaborate on how you solved that? Thanks.

0 Votes 0 ·