What is Azure Data Explorer?

  • 5 minutes

Let's start by defining the service and taking a tour of the core features of Azure Data Explorer. This overview should help you decide if it's the right service for you to manage and analyze your data.

What is Azure Data Explorer?

Azure Data Explorer is a big data analytics platform that makes it easy to analyze high volumes of data in near real time. The Azure Data Explorer toolbox gives you an end-to-end solution for data ingestion, query, visualization, and management.

By analyzing structured, semi-structured, and unstructured data across time series, and by using Machine Learning, Azure Data Explorer makes it simple to extract key insights, spot patterns and trends, and create forecasting models. Azure Data Explorer is fully managed, scalable, secure, robust, and enterprise-ready, and is useful for log analytics, time series analytics, IoT, and general-purpose exploratory analytics.

How to understand your big data

If you remember our example clothing company, they have a lot of types of data coming in from varied domains. They need to use very different kinds of analytics on these data types, and then share their results with a range of stakeholders. They're going to use Azure Data Explorer to get insights from all of their data across the company.

Production analyzes their product logs to manage their inventory and make manufacturing decisions. These decisions are also informed by geospatial analytics, which are used to identify geographical areas of high-performing ads and anticipate inventory.

The company's warehouses are outfitted with IoT devices, some of which are used by security to manage warehouse entry/exit logs, while others are used by operations to monitor the environment inside the warehouse. Individual stores use time series analytics to identify sales anomalies and predict future inventory events.

The global marketing team uses clickstream data (also a form of log analytics) to optimize and scan online both ad campaigns and the customer funnel, while the customer success department uses text search to analyze user feedback on social media.

Every minute of the day, a company decision is being made based on data flowing into Azure Data Explorer.

What are some of Azure Data Explorer's key features?

Now that you have an idea of what Azure Data Explorer can be used for, let's look at some of its key features.

Data velocity, variety, and volume

Azure Data Explorer can ingest terabytes of data in minutes in batch or streaming mode, and petabytes of data can be queried with results returned within milliseconds to seconds. This allows for high velocity (millions of events per second), low latency (seconds), and linear scale ingestion of raw data. Data can be ingested in different formats and structures. It can flow in from various pipelines and sources.

User-friendly query language

Azure Data explorer uses the Kusto Query Language (KQL), an open-source language initially invented by the team. The language is simple to understand and learn, and highly productive. You can use simple operators as well as advanced analytics.

Advanced analytics

Use Azure Data Explorer for time series analysis with a large set of functions; adding, subtracting time series, filtering, regression, seasonality detection, geospatial analysis, anomaly detection, scanning, and forecasting. Time series functions are optimized for processing thousands of time series in seconds. Pattern detection is made easy with cluster plugins that can diagnose anomalies and do root cause analysis. You can also extend Azure Data Explorer capabilities by embedding python code in KQL queries.

Easy-to-use wizard

The ingestion wizard makes the data ingestion process easy, fast, and intuitive. The web UI provides an intuitive and guided experience that helps customers ramp-up quickly to start ingesting data, creating database tables, and mapping structures. It enables one time or a continuous ingestion from various sources and in a variety of data formats. Table mappings and schema are auto suggested and easy to modify.

Versatile data visualization

Data visualization helps you gain important insights. Azure Data Explorer offers built-in visualization and dashboarding out of the box, with support for various charts and visualizations. It has native integration with Power BI, native connectors for Grafana, Kibana and Databricks, ODBC support for Tableau, Sisense, Qlik and more.

Automatic ingest, process, and export

Azure Data Explorer supports server-side stored functions, continuous ingest, and continuous export to Azure Data Lake store. It also supports ingestion time mapping transformations on the server side, update policies, and precomputed scheduled aggregates with materialized views.

Integration with other services

Integrate easily and seamlessly with other tools in all aspects of your workflow, such as Ingestion, Visualization, Orchestration, and Monitoring.