question

Anshal-7634 avatar image
0 Votes"
Anshal-7634 asked KranthiPakala-MSFT answered

Parquet file compression

Hello friends when file compression for parquet happens before writing the data or after data write help me please

azure-data-factoryazure-databricks
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered

Hello @Anshal-7634,

Thanks for the question and using MS Q&A platform.

My understanding from below article is that the parquet data is compressed in memory before the file is being written.

For more detailed explanation, please refer to this article by community volunteer - How Parquet Files are Written – Row Groups, Pages, Required Memory and Flush Operations

Hope this info helps.



· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @Anshal-7634,

Just checking in to see if the above response was helpful. And, if you have any further query do let us know.

0 Votes 0 ·