question

Pranay-9917 avatar image
0 Votes"
Pranay-9917 asked Pranay-9917 commented

Dataflows : remove partitonkey and equals sign while writing

HI,

Atm after partitioning the data using DataFlows in DataFactory, it creates the folder structure including the key and equals sign. Is there a possibility to remove these from folder structure.

Thanks

azure-data-factory
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

Kiran-MSFT avatar image
0 Votes"
Kiran-MSFT answered Pranay-9917 commented

Don't use key based partitioning. Look up documentation on "As data in column/folder" on the sink.
https://twitter.com/KromerBigData/status/1366842335481393153

· 8
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Pranay-9917 avatar image Pranay-9917 ·

HI @Kiran-MSFT : Thanks for the quick reply.

Although, I would like to have multiple partitions, which won't be possible using this solution.

0 Votes 0 ·
Kiran-MSFT avatar image Kiran-MSFT Pranay-9917 ·

You can create as many partitions you want using key based partitioning or any partition methods at a transformation before the sink.

0 Votes 0 ·
Pranay-9917 avatar image Pranay-9917 Kiran-MSFT ·

HI @Kiran-MSFT true, atm im doing the same with key based partitoning. Although the partition name is present while creating the folder. I am not able to get rid of that and "As data in column/folder" doesnt work when key based partitioning is present.

0 Votes 0 ·
Show more comments