Use one-click ingestion to ingest CSV data from a container to a new table in Azure Data Explorer
One-click ingestion enables you to quickly ingest data in JSON, CSV, and other formats into a table and easily create mapping structures. The data can be ingested either from storage, from a local file, or from a container, as a one-time or continuous ingestion process.
This document describes using the intuitive one-click wizard in a specific use case to ingest CSV data from a container into a new table. You can use the same process with slight adaptations to cover a variety of different use cases.
For an overview of one-click ingestion and a list of prerequisites, see One-click ingestion. For information about ingesting data into an existing table in Azure Data Explorer, see One-click ingestion to an existing table
Ingest new data
In the left menu of the Web UI, right-click a database and select Ingest new data (Preview).
In the Ingest new data (Preview) window, the Source tab is selected.
Select Create new table and enter a name for the new table. You can use alphanumeric, hyphens, and underscores. Special characters aren't supported.
Table names must be between 1 and 1024 characters.
Select an ingestion type
Under Ingestion type, do the following steps:
Select from container
In the Link to storage field, add the SAS URL of the container, and optionally enter the sample size.
For ingestion from file, see Use one-click ingestion to ingest JSON data from a local file to an existing table in Azure Data Explorer
A sample of the data appears. If you want to, filter the data to ingest only files that begin end with specific characters. When you adjust the filters, the preview automatically updates.
For example, filter for all files that begin with the word .csv extension.
Edit the schema
Select Edit schema to view and edit your table column configuration. The system will select one of the blobs at random and the schema will be generated based on that blob. By looking at the name of the source, the service automatically identifies if it is compressed or not.
In the Schema tab:
Select Data format:
In this case, the data format is CSV
If you want to use JSON files, see Use one-click ingestion to ingest JSON data from a local file to an existing table in Azure Data Explorer.
You can select the check box Include column names to ignore the heading row of the file.
In the Mapping name field, enter a mapping name. You can use alphanumeric characters and underscores. Spaces, special characters, and hyphens aren't supported.
Edit the table
When ingesting to a new table, alter various aspects of the table when creating the table.
In the table:
Double-click the new column name to edit.
Select new column headers and do any of the following actions:
Action Description Change data type Change the data type from the one automatically selected by the service to one of the other supported data types Rename column Change the column name New column Add a new column. For tabular formats, each column can be ingested into one column on ADX. A new column can be created from a different JSON level. Delete column Delete the selected column Sort ascending Sort the table by the selected column in ascending order (existing columns only) Sort descending Sort the table by the selected column in descending order (existing columns only)
For tabular formats, each column can be ingested into one column in Azure Data Explorer.
Above the Editor pane, select the v button to open the editor. In the editor, you can view and copy the automatic commands generated from your inputs.
You can't update the name and datatype for existing tables.
Select Start ingestion to create a table and mapping and to begin data ingestion.
Complete data ingestion
In the Data ingestion completed window, all three steps will be marked with green check marks when data ingestion finishes successfully.
Explore quick queries and tools
In the tiles below the ingestion progress, explore Quick queries or Tools:
Quick queries includes links to the Web UI with example queries.
Tools includes links to Undo or Delete new data on the Web UI, which enable you to troubleshoot issues by running the relevant
You might lose data when you use
.dropcommands. Use them carefully. Drop commands will only revert the changes that were made by this ingestion flow (new extents and columns). Nothing else will be dropped.
Create continuous ingestion for container
Continuous ingestion enables you to create an event grid that listens for new files in the source container. Any new file that meets the criteria of the pre-defined parameters (prefix, suffix, and so on) will be automatically ingested into the destination table.
Select Event Grid in the Continuous ingestion tile to open the Azure portal. The data connection page opens with the event grid data connector opened and with source and target parameters already entered (source container, tables, and mappings).
Select Create to create a data connection which will listen for any changes, updates, or new data in that container.