question

Raviyaramati-1837 avatar image
0 Votes"
Raviyaramati-1837 asked KranthiPakala-MSFT commented

how to capture output of append variable in adf and write to json or csv file in adls


i have a set up like below , i need to capture data from for each activity to json or csv



113964-image.png


azure-data-factory
image.png (9.0 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT commented

Hi @raviyaramati-1837,

Welcome to Microsoft Q&A forum and thanks for posting your query.

One way to achieve this requirement is by utilizing "Add additional columns during copy" feature as explained in this stackoverflow thread - Transfer the output of 'Set Variable' activity into a json file [Azure Data Factory

After your append variable activity, have a subsequent Copy activity and define a column and pass the value of append variable as described in stackoverflow thread.

Hope this info helps. Do let us know if you have further query.



Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.


· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @raviyaramati-1837,

Just checking in to see if the above information was helpful. If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you

0 Votes 0 ·

Hi @raviyaramati-1837,

We still have not heard back from you. Just wanted to check if the above suggestion was helpful? If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

0 Votes 0 ·

Hello @KranthiPakala-MSFT

i have gone through stack overflow thread earlier as well, but no luck , as it say point dataset to json , but to create dataset we need linked service point connection ,

1) where should we place or point dataset link service to point to

2) if we tried placing source adls link service and created json dataset , but this seems like we to place output of variable to json (sink) , but source , json seems invalid

can you suggest if there is any other way , other than this


Regards

Ravi

0 Votes 0 ·

Hi @raviyaramati-1837,

Thanks for your response and apologies for the delayed reply.

1) where should we place or point dataset link service to point to

You will have to create a linked service pointing to the sink data store where you would want to write the sink data (csv/json based on the requirement).

2) if we tried placing source adls link service and created json dataset , but this seems like we to place output of variable to json (sink) , but source , json seems invalid - Sorry I didn't get this ask here, could you please elaborate a bit so that we can better assist on this?

Thank you



0 Votes 0 ·

Hi @raviyaramati-1837,

We still haven't heard back from you. Could you please confirm the above details for further assistance on this.

Thank you

0 Votes 0 ·