Delta log stats are not captured in non Databricks environment. Can data skipping algorithm works without Databricks ?
Delta log stats are not captured in non Databricks environment. Can data skipping algorithm works without Databricks ?
Hello @MMathan-3430,
Following up to see if the below suggestion was helpful. And, if you have any further query do let us know.
Please don't forget to click on
or upvote
button whenever the information provided helps you.
Hello @MMathan-3430,
Thanks for the question an using MS Q&A platform.
Z-Ordering is a technique to co-locate related information in the same set of files. This co-locality is automatically used by Delta Lake on Azure Databricks data-skipping algorithms to dramatically reduce the amount of data that needs to be read.
For more details, refer to Z-Ordering (multi-dimensional clustering).
Hope this will help. Please let us know if any further queries.
Please don't forget to click on
or upvote
button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
Hello @MMathan-3430,
Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
19 people are following this question.