Summary

Apache Spark offers high performance when you need to work with Big Data workloads, because of its distributed computing architecture. When working with high volume, high velocity, or a variety of data. Apache Spark can help you process all combinations of these types of Big Data scenarios. Azure Databricks lets you use Apache Spark in a fully managed and finely tuned environment.

Understanding the Spark Cluster and Jobs architecture is a great starting point for learning how to perform data engineering and data science tasks in Azure Databricks.