question

yosboss-7571 avatar image
0 Votes"
yosboss-7571 asked yosboss-7571 commented

how to use cached lookup in sink partition

Hello,

In Azure Data Factory, Data Flow, I want to use dynamic partitioning in sink. so I used a cached sink to store row count, and then in the actual sink that will output the files I used sink1#outputs()[1].rowCount in "Number of partition" field.
but I get this error : "Job failed due to reason: at : Unsupported syntax in expression"

79741-screenshot-2021-03-19-154935.png


azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

Kiran-MSFT avatar image
1 Vote"
Kiran-MSFT answered yosboss-7571 commented

This is not supported but in our backlog of items to address. Until then you will need to use 3 activities. 1 dataflow to aggregate and write a single count to a file. Then use a lookup activity to read that file. Pass the read value as a parameter to the 2nd dataflow to use in partition configuration.
@MarkKromer-8019

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

yes, that's what I'm doing right now. thanks!
how can I stay tuned for this upgrade ?

0 Votes 0 ·