Quickstart: Create an Azure Cognitive Search knowledge store in the Azure portal
Knowledge store is a feature of Azure Cognitive Search that persists output from a cognitive skills pipeline for subsequent analyses or downstream processing.
A pipeline accepts images and unstructured text as raw content, applies AI through Cognitive Services (such as image and natural language processing), and creates enriched content (new structures and information) as output. One of the physical artifacts created by a pipeline is a knowledge store, which you can access through tools to analyze and explore content.
In this quickstart, you'll combine services and data in the Azure cloud to create a knowledge store. Once everything is in place, you'll run the Import data wizard in the portal to pull it all together. The end result is original plus AI-generated content that you can view in the portal (Storage explorer).
If you don't have an Azure subscription, create a free account before you begin.
Create services and load data
This quickstart uses Azure Cognitive Search, Azure Blob storage, and Azure Cognitive Services for the AI.
Because the workload is so small, Cognitive Services is tapped behind the scenes to provide free processing for up to 20 transactions daily when invoked from Azure Cognitive Search. As long as you use the sample data we provide, you can skip creating or attaching a Cognitive Services resource.
Download HotelReviews_Free.csv. This data is hotel review data saved in a CSV file (originates from Kaggle.com) and contains 19 pieces of customer feedback about a single hotel.
Create an Azure storage account or find an existing account under your current subscription. You'll use Azure storage for both the raw content to be imported, and the knowledge store that is the end result.
There are two requirements for this account:
Choose the same region as Azure Cognitive Search.
Choose the StorageV2 (general purpose V2) account type.
Open the Blob services pages and create a container.
Select the HotelReviews-Free.csv file you downloaded in the first step.
- Create an Azure Cognitive Search service or find an existing service under the same subscription. You can use a free service for this quickstart.
You are now ready to move on the Import data wizard.
Run the Import data wizard
In the search service Overview page, click Import data on the command bar to create a knowledge store in four steps.
Step 1: Create a data source
In Connect to your data, choose Azure Blob storage, select the account and container you created.
For the Name, enter
For Parsing mode, select Delimited text, and then select the First Line Contains Header checkbox. Make sure the Delimiter character is a comma (,).
Enter your storage service Connection String that you saved in a previous step.
For Container name, enter
Click Next: Add AI enrichment (Optional).
Continue to the next page.
Step 2: Add cognitive skills
In this wizard step, you will create a skillset with cognitive skill enrichments. The skills we use in this sample will extract key phrases and detect the language and sentiment. In a later step, these enrichments will be "projected" into a knowledge store as Azure tables.
Expand Attach Cognitive Services. Free (Limited enrichments) is selected by default. You can use this resource because number of records in HotelReviews-Free.csv is 19 and this free resource allows up to 20 transactions a day.
Expand Add cognitive skills.
For Skillset name, enter
For Source data field, select reviews_text.
For Enrichment granularity level, select Pages (5000 characters chunks)
Select these cognitive skills:
Extract key phrases
Expand Save enrichments to knowledge store.
Enter the Storage account Connection String that you saved in a previous step.
Select these Azure table projections:
- Key phrases
Continue to the next page.
Step 3: Configure the index
In this wizard step, you will configure an index for optional full-text search queries. The wizard will sample your data source to infer fields and data types. You only need to select the attributes for your desired behavior. For example, the Retrievable attribute will allow the search service to return a field value while the Searchable will enable full text search on the field.
For Index name, enter
For attributes, make these selections:
- Select Retrievable for all fields.
- Select Filterable and Facetable for these fields: Sentiment, Language, Keyphrases
- Select Searchable for these fields: city, name, reviews_text, language, Keyphrases
Your index should look similar to the following image. Because the list is long, not all fields are visible in the image.
Continue to the next page.
Step 4: Configure the indexer
In this wizard step, you will configure an indexer that will pull together the data source, skillset, and the index you defined in the previous wizard steps.
- For Name, enter
- For Schedule, keep the default Once.
- Click Submit to run the indexer. Data extraction, indexing, application of cognitive skills all happen in this step.
Cognitive skill indexing takes longer to complete than typical text-based indexing. The wizard should open the Indexer list in the overview page so that you can track progress. For self-navigation, go to the Overview page and click Indexers.
In the Azure portal, you can also monitor the Notifications activity log for a clickable Azure Cognitive Search notification status link. Execution may take several minutes to complete.
Now that you have enriched your data using Cognitive Services and projected the results into a knowledge store, you can use Storage Explorer or Power BI to explore your enriched data set.
You can view content in Storage Explorer, or take it a step further with Power BI to gain insights through visualization.
If you want to repeat this exercise or try a different AI enrichment walkthrough, delete the hotel-reviews-idxr indexer. Deleting the indexer resets the free daily transaction counter back to zero for Cognitive Services processing.