Parse Data Flow -> External Call response using UTF-8 encoding?

Question

question

AshTT-7826 asked Apr 13, '22 KranthiPakala-MSFT edited Apr 27, '22

Parse Data Flow -> External Call response using UTF-8 encoding?

Hey all,

Having a little trouble getting ADF to correctly parse a UTF-8 XML response from an external call activity within a data flow. The API XML response doesn't have an encoding tag (limitation of the service we're using). Is there any way to force the encoding of the response in the activity.. have I missed a trick here?

Example data:

Directorâ s

Thanks, Ash.

azure-data-factory

· 2

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KranthiPakala-MSFT · Apr 15 at 03:33 PM

Hello @AshTT-7826,

Thanks for the question and using MS Q&A platform.

As I understand the ask here that your External endpoint returns a response of XML type. Please correct me if I'm not correct.

As you are using External call transformation, you will be using REST connector. Please note that in ADF REST Connector, only response payload in JSON is supported.

You may have to try with HTTP connector in regular ADF pipeline and not Dataflows.

Please let us know if you have questions.

0 ·

image.png (37.3 KiB)

AshTT-7826 KranthiPakala-MSFT · Apr 18 at 09:58 PM

Hey Kranthi,

Thanks for getting back to me! Ah okay, you are correct I am using External Call connector to consume an XML endpoint.

I didn't realise the connector only supported JSON, the UI itself within the External Call connector does seem to only allow REST inline connector but also does give the option to choose an XML format for the source data.

Not sure if this is a mistake in the UI or maybe I'm misunderstanding the use case? It's certainly working, parsing the XML into an object correctly just incorrect encoding when doing so..

Thanks, Ash.

0 ·

image.png (530.2 KiB)

Answer 1 · 2022-04-19T21:49:03.703Z

AshTT-7826 answered Apr 19, '22 KranthiPakala-MSFT edited Apr 27, '22

Managed to find a workable solution, adjusting the Content-Type of my APIs response to contain the correct charset (e.g. Content-Type: text/xml; charset=utf-8) appears to work as expected and is parsed correctly by Data Factory. Thanks, Ash.

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KranthiPakala-MSFT · Apr 27 at 11:55 PM

Hello @AshTT-7826,

Appreciate much for sharing your findings and glad to know that you were able to figure it out. Regarding the XML support, looks like product team may have added it recently but I have passed on the feedback to respective product owners to update the documentation with updated information. Good call out and thanks for helping us improve the Azure Product and our documentation.

Have a great day.

0 ·

question