Get started: Form Recognizer Studio | Preview

Note

Form Recognizer Studio is currently in public preview. Some features may not be supported or have limited capabilities.

Form Recognizer Studio preview is an online tool for visually exploring, understanding, and integrating features from the Form Recognizer service in your applications. Get started with exploring the pre-trained models with sample documents or your own. Create projects to build custom form models and reference the models in your applications using the Python SDK preview and other quickstarts.

Form Recognizer Studio demo

Migrating from the sample labeling tool

If you are a previous user of the sample labeling tool, skip the prerequisites to sign into the Studio preview to use your existing Azure account and Form Recognizer or Cognitive Services resources with the Studio.

To migrate your existing custom projects to the Studio, jump ahead to the Custom model getting started section to create a new project and point it to the same Azure Blob storage location assuming you have access to it in Azure. Once you configure a new project, the Studio will load all documents and interim files for labeling and training.

Prerequisites for new users

Sign into the Form Recognizer Studio preview

Note

Virtual networks (VNETs)

If you are using the Studio with service endpoints and blob storage configured within a virtual network (VNET), ensure that your computer is in the same VNET as the endpoint and the storage container.

After you have completed the prerequisites, navigate to the Form Recognizer Studio preview.

  1. Select a Form Recognizer service feature from the Studio home page.

  2. Select your Azure subscription, resource group, and resource. (You can change the resources anytime in "Settings" in the top menu.)

  3. Review and confirm your selections.

Form Recognizer Studio Getting Started example

Layout

In the Layout view:

  1. Select the Analyze command to run Layout analysis on the sample document or try your document by using the Add command.

  2. Observe the highlighted extracted text, the table icons showing the extracted table locations, and highlighted selection marks.

  3. Use the controls at the bottom of the screen to zoom in and out and rotate the document view.

  4. Show and hide the text, tables, and selection marks layers to focus on each one of them at a time.

  5. In the output section's Result tab, browse the JSON output to understand the service response format. Copy and download to jumpstart integration.

Form Recognizer Layout example

Prebuilt models

There are several prebuilt models to choose from, each of which has its own set of supported fields. The model to use for the analyze operation depends on the type of document to be analyzed. Here are prebuilt models currently supported by the Form Recognizer service:

  • 🆕 General document—Analyze and extract text, tables, structure, key-value pairs and named entities.
  • Invoice: extracts text, selection marks, tables, key-value pairs, and key information from invoices.
  • Receipt: extracts text and key information from receipts.
  • ID document: extracts text and key information from driver licenses and international passports.
  • Business card: extracts text and key information from business cards.

In the Prebuilt view:

  1. From the Studio home, select one of the prebuilt models. In this example, we are using the Invoice model.

  2. Select the Analyze command to run analysis on the sample document or try your invoice by using the Add command.

  3. In the visualization section, observe the highlighted fields and values and invoice line items. All extracted text and tables are also shown.

  4. In the output section's Fields tab, note the listed fields and values, and select the line items to view in a table-like format.

  5. In the output section's Result tab, browse the JSON output to understand the service response format. Copy and download to jumpstart integration.

Form Recognizer Prebuilt example

Additional prerequisites for custom projects

In addition to the Azure account and a Form Recognizer or Cognitive Services resource, you'll need:

Azure Blob Storage container

A standard performance Azure Blob Storage account. You'll create containers to store and organize your blob data within your storage account. If you don't know how to create an Azure storage account with a container, following these quickstarts:

  • Create a storage account. When creating your storage account, make sure to select Standard performance in the Instance details → Performance field.
  • Create a container. When creating your container, set the Public access level field to Container (anonymous read access for containers and blobs) in the New Container window.

Configure CORS

CORS (Cross Origin Resource Sharing) needs to be configured on your Azure storage account for it to be accessible from the Form Recognizer Studio. To configure CORS in the Azure portal, you will need access to the CORS blade of your storage account.

Screenshot that shows CORS configuration for a storage account.

  1. Select the CORS blade for the storage account.
  2. Start by creating a new CORS entry in the Blob service.
  3. Set the Allowed origins to https://formrecognizer.appliedai.azure.com.
  4. Select all the available 8 options for Allowed methods.
  5. Approve all Allowed headers and Exposed headers by entering an * in each field.
  6. Set the Max Age to 120 seconds or any acceptable value.
  7. Click the save button at the top of the page to save the changes.

CORS should now be configured to use the storage account from Form Recognizer Studio.

Sample documents set

  1. Go to the Azure portal and navigate as follows: Your storage accountData storageContainers

    Screenshot: Data storage menu in the Azure portal.

  2. Select a container from the list.

  3. Select Upload from the menu at the top of the page.

    Screenshot: container upload button in the Azure portal.

  4. The Upload blob window will appear.

  5. Select your file(s) to upload.

    Screenshot: upload blob window in the Azure portal.

Note

By default, the Studio will use form documents that are located at the root of your container. However, you can use data organized in folders if specified in the Custom form project creation steps. See Organize your data in subfolders

Custom projects

Getting started

To create custom models, you start with configuring your project:

  1. From the Studio home, select the Custom form project to open the Custom form home page.

  2. Use the "Create a project" command to start the new project configuration wizard.

  3. Enter project details, select the Azure subscription and resource, and the Azure Blob storage container that contains your data.

  4. Review and submit your settings to create the project.

Form Recognizer Custom project Getting Started example

Basic flow

After the project creation step, in the custom model phase:

  1. From the labeling view, define the labels and their types that you are interested in extracting.

  2. Select the text in the document and select the label from the drop-down list or the labels pane.

  3. Label four more documents to get at least five documents labeled.

  4. Select the Train command and enter model name and description to start training your custom model.

  5. Once the model is ready, use the Test command to validate it with your test documents and observe the results.

Form Recognizer Custom project basic workflow example

Other features

In addition, view all your models using the Models tab on the left. From the list view, select model(s) to perform the following actions:

  1. Test the model from the list view.

  2. Use the Delete command to delete models that are not required.

  3. Download model details for offline viewing.

  4. Select multiple models and compose them into a new model to be used in your applications.

Labeling as tables

While creating your custom models, you may need to extract data collections from your documents. These may appear in a couple of formats. Using tables as the visual pattern:

  • Dynamic or variable count of values (rows) for a given set of fields (columns)

  • Specific collection of values for a given set of fields (columns and/or rows)

Label as dynamic table

Use dynamic tables to extract variable count of values (rows) for a given set of fields (columns):

  1. Add a new "Table" type label, select "Dynamic table" type, and name your label.

  2. Add the number of columns (fields) and rows (for data) that you need.

  3. Select the text in your page and then choose the cell to assign to the text. Repeat for all rows and columns in all pages in all documents.

Form Recognizer labeling as dynamic table example

Label as fixed table

Use fixed tables to extract specific collection of values for a given set of fields (columns and/or rows):

  1. Create a new "Table" type label, select "Fixed table" type, and name it.

  2. Add the number of columns and rows that you need corresponding to the two sets of fields.

  3. Select the text in your page and then choose the cell to assign it to the text. Repeat for other documents.

Form Recognizer Labeling as fixed table example

Labeling for signature detection

To label for signature detection:

  1. In the labeling view, create a new "Signature" type label and name it.

  2. Use the Region command to create a rectangular region at the expected location of the signature.

  3. Select the drawn region and choose the Signature type label to assign it to your drawn region. Repeat for other documents.

Form Recognizer labeling for signature detection example

Next steps

Get started with the Form Recognizer Studio preview.