question

HiteshSrivastava-4993 avatar image
0 Votes"
HiteshSrivastava-4993 asked HiteshSrivastava-4993 commented

Mapping of XML in ADF

Hey,

I'm trying to do mapping of XML file from HTTP connector in ADF, now the issue is that in a single column some rows are there as an Object and others as a node.
199605-microsoftteams-image-6.png199673-microsoftteams-image-5.png
Here, in 1st screen shot the customer-work-region is an object and in the another one it is a node.
If we do the mapping according to one case, then we get an error and if we select "Skip Incompatible Rows"  we are loosing some data.
199701-microsoftteams-image-4.png
So is there any other way by which we can get data without any data loss?


azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

AnnuKumari-MSFT avatar image
0 Votes"
AnnuKumari-MSFT answered HiteshSrivastava-4993 commented

Hi @HiteshSrivastava-4993 ,
Welcome to Microsoft Q&A platform and thanks for posting your query.
As I understand your ask here, it seems you want to fetch the xml data , however, the data which you shared in the screenshot looks like JSON data. Please confirm on that. Here the issue is that there is an inconsistency in the type of data, few rows are of String data type and few are JSON data. Please let me know if my understanding about your query is incorrect.

In order to process the data in ADF, we need to make sure the data is consistent from the source . The error which you are getting is an expected error in this case.

To process the file without any data loss, you can consider using fault tolerance to skip incompatible data and process the log file to fetch the skipped data.

Also, you can consider writing some custom code using Azure function to convert the source data into similar datatype and then use copy activity once the datatype among all the rows match.

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
    Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators


· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @AnnuKumari-MSFT,
First, we are fetching data from API's using HTTP connector which gives XML format in ADF.
Second, how can we create and process the log files? and can you please tell what type of azure functions can help in resolving this issue?

0 Votes 0 ·