question

RobRichardson-9222 avatar image
0 Votes"
RobRichardson-9222 asked SaurabhSharma-msft commented

Azure Synapse Data Factory REST Connector issues within Data Flows

We're trying to use the REST connector as a Source within a Data Flow which is supposedly supported based on this Announcement. However, when we attempt to Import the Schema for the endpoint, the process never completes; just seeing a loading indicator for an indefinite amount of time until the Debug Session eventually times out. The same endpoint works just fine when using it from an HTTP Connector. Is this a known issue?


azure-data-factoryazure-synapse-analytics
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @robrichardson-9222,

Thanks for using Microsoft Q&A !!
I have tried using REST source for a public REST API from here and I could see projection coming fine in DataFlow.
131202-image.png
Even Data Preview shows data correctly.
131224-image.png

Is REST API internal to you ? Also, what is the format (JSON or XML) your API is returning? Have you tried changing the format under Source Options
131232-image.png

Also, could you provide a sample data ?

Thanks
Saurabh


0 Votes 0 ·
image.png (24.0 KiB)
image.png (25.1 KiB)
image.png (32.2 KiB)
SaurabhSharma-msft avatar image SaurabhSharma-msft SaurabhSharma-msft ·

Hi @RobRichardson-9222,

Following up to know if you are still having issues.

Thanks
Saurabh

0 Votes 0 ·
RobRichardson-9222 avatar image RobRichardson-9222 SaurabhSharma-msft ·

Saurabh,
Thanks for your reply. This is a public API we maintain. It is protected with Basic Authentication & the format is JSON. This same API works just fine when using the HTTP Connector. This API also works fine when using the REST connector from a Copy Data Activity - I can succesfully import the schema and copy all data to a JSON file in Storage easily. The problem only arises when we attempt to use the REST connector as a Data Flow Source.

Also, we tried the public REST API that you mentioned above from within a Data Flow and it worked fine so I'm not sure what's going on - perhaps the issue is related to Basic Authentication? Regardless, the problem appears to be isolated to when the REST connector is used as a Source within a Data Flow.

0 Votes 0 ·
SaurabhSharma-msft avatar image SaurabhSharma-msft RobRichardson-9222 ·

@robrichardson-9222 It may be an authentication issue. Can you please try running your dataflow and provide me the runId so that I can try checking in backend for issues.

Thanks
Saurabh

0 Votes 0 ·

0 Answers