question

PraveenNaregal-5988 avatar image
0 Votes"
PraveenNaregal-5988 asked PRADEEPCHEEKATLA-MSFT commented

Handle 3 Billion records

We have a view built in Synapse Data warehouse which pulls out data around 3 Billion.
Need to understand how can we handle this data and also if there is other solution on storing this 3 Billion data in Azure.

azure-synapse-analytics
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @PraveenNaregal-5988,

Following up to see if the below suggestion was helpful. And, if you have any further query do let us know.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.

0 Votes 0 ·

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT commented

Hello @PraveenNaregal-5988,

Welcome to the Microsoft Q&A platform.

‎Azure Data Lake Storage Gen2 is a set of capabilities dedicated to big data analytics, built on Azure Blob Storage.

Azure Data Lake Storage Gen2 supports individual file sizes as high as 190.7 TiB and most of the hard limits for performance have been removed.

Azure Synapse removes the traditional technology barriers between using SQL and Spark together. You can seamlessly mix and match based on your needs and expertise.

  • Tables defined on files in the data lake are seamlessly consumed by either Spark or Hive.

  • SQL and Spark can directly explore and analyze Parquet, CSV, TSV, Delta Lake, and JSON files stored in the data lake.

  • Fast, scalable data loading between SQL and Spark databases

133567-image.png

For more details, refer to Introduction to Azure Data Lake Storage Gen2.

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators


image.png (85.5 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @PraveenNaregal-5988,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·