question

Pranay-9917 avatar image
0 Votes"
Pranay-9917 asked Pranay-9917 commented

Dataflows : remove partitonkey and equals sign while writing

HI,

Atm after partitioning the data using DataFlows in DataFactory, it creates the folder structure including the key and equals sign. Is there a possibility to remove these from folder structure.

Thanks

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

Kiran-MSFT avatar image
0 Votes"
Kiran-MSFT answered Pranay-9917 commented

Don't use key based partitioning. Look up documentation on "As data in column/folder" on the sink.
https://twitter.com/KromerBigData/status/1366842335481393153

· 8
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

HI @Kiran-MSFT : Thanks for the quick reply.

Although, I would like to have multiple partitions, which won't be possible using this solution.

0 Votes 0 ·

You can create as many partitions you want using key based partitioning or any partition methods at a transformation before the sink.

0 Votes 0 ·

HI @Kiran-MSFT true, atm im doing the same with key based partitoning. Although the partition name is present while creating the folder. I am not able to get rid of that and "As data in column/folder" doesnt work when key based partitioning is present.

0 Votes 0 ·
Show more comments