question

arkiboys avatar image
0 Votes"
arkiboys asked arkiboys commented

read each row in csv

Hello,
using a lookup activity, I can loop through each row in on-prem sql table and get each of the field values separately...
usingthe query this is what I am doing:
select * from eol.configTable where Source = '@{pipeline().parameters.pSource}' and Check = '@{pipeline().parameters.pCheck}'

This time I would like to loop through a csv file instead of a sql table?
I have a dataset pointing to the csv in the blobstorage.
Question:
How do I do the same select query as there is no query option when dataset is csv?
Thank you

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

MarkKromer-MSFT avatar image
0 Votes"
MarkKromer-MSFT answered arkiboys commented

Use a data flow activity and use the Filter transformation

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi,
I achieved this without a dataflow.
Lookup to read csv, filter activity to filter the rows
Thank you

0 Votes 0 ·

Maximum 5,000 rows / 4 MB in pipeline lookup. No such max in data flow.

0 Votes 0 ·

Thank you

0 Votes 0 ·