question

KlausNrregaard-1904 avatar image
0 Votes"
KlausNrregaard-1904 asked MartinJaffer-MSFT answered

DataFactory sink from variable array?

Hi

I am looking for the best strategy or method to sink from a defined pipeline array. I have with success created a foreach loop, that iterates over a list of objects. Then each object have been stored within a Pipeline variable of type Array. Now I would like to sink the array to csv. How to do that?



Thanks
Klaus

azure-data-factory
· 6
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

You'll need to provide some sampling... too many arrays to understand what you have and what you want.

0 Votes 0 ·

It is just one array. In the Foreach loop how do I sink the data to a data store? The variable is located on the Pipeline and is called RecipientList. It is ready to be stored, but I dont know how to

113744-capture.png



0 Votes 0 ·
capture.png (8.2 KiB)

If the array is the actual data you want saved, use the web request as your source dataset within a copy activity

If it's detail of the source that you need to subsequently fetch, use the output of the web request in a foreach loop

0 Votes 0 ·
Show more comments

If you found your own solution, please share here with the community.

0 Votes 0 ·

1 Answer

MartinJaffer-MSFT avatar image
0 Votes"
MartinJaffer-MSFT answered

Hello @KlausNrregaard-1904 , @RyanAbbey-0701 . Welcome to Microsoft Q&A.

I don't want to interrupt the great collaboration you have going on, but I thought of suggesting another way.

If you just need to get the data from a web-based source, use the HTTP dataset or the REST dataset. Web activity is more for getting meta-data.

Web activity can be used to write via leveraging a storage api.

It is also possible to write from an array by using the "additional column" feature in a delimited text dataset. The trick in this, is to use an almost-empty file as source, put the array into additional column, and map to sink.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.