Form Recognizer custom and composed models

Form Recognizer uses advanced machine learning technology to detect and extract information from document images and return the extracted data in a structured JSON output. With Form Recognizer, you can train standalone custom models or combine custom models to create composed models.

• Custom models. Form Recognizer custom models enable you to analyze and extract data from forms and documents specific to your business. Custom models are trained for your distinct data and use cases.

• Composed models. A composed model is created by taking a collection of custom models and assigning them to a single model that encompasses your form types. When a document is submitted to a composed model, the service performs a classification step to decide which custom model accurately represents the form presented for analysis.

What is a custom model?

A custom model is a machine learning program trained to recognize form fields within your distinct content and extract key-value pairs and table data. You only need five examples of the same form type to get started and your custom model can be trained with or without labeled datasets.

What is a composed model?

With composed models, you can assign multiple custom models to a composed model called with a single model ID. It is useful when you have trained several models and want to group them to analyze similar form types. For example, your composed model may include custom models trained to analyze your supply, equipment, and furniture purchase orders. Instead of manually trying to select the appropriate model, you can use a composed model to determine the appropriate custom model for each analysis and extraction.

Development options

The following resources are supported by Form Recognizer v2.1:

The following resources are supported by Form Recognizer v3.0:

Try Form Recognizer

See how data is extracted from your specific or unique documents using custom models. You'll need the following:

• An Azure subscription—you can create one for free

• A Form Recognizer instance in the Azure portal. You can use the free pricing tier (F0) to try the service. After your resource deploys, select Go to resource to get your API key and endpoint.

Form Recognizer Studio (preview)

1. On the Form Recognizer Studio home page, select Custom form.

2. Under My Projects, select + Create a project.

3. Complete the project details fields.

4. Configure the service resource.

5. Add your Storage account and Blob container to Connect your training data source.

6. Review and create your project.

7. A set of sample documents has been provided for you to build and test your custom model.

   Try Form Recognizer Studio

Sample Labeling tool

You'll need a set of at least six forms of the same type. You'll use this data to train the model and test a form. You can use our sample data set. Download and extract the sample_data.zip, then upload the contents to your Azure Blob Storage container.

In the Form Recognizer UI:

1. On the Sample Labeling tool home page, select Use Custom to train a model with labels and get key value pairs.

   

2. In the next window, select New project:

   

   For more detailed instructions, see our Sample Labeling tool quickstart.

   Try Sample Labeling tool

Input requirements

• For best results, provide one clear photo or high-quality scan per document.

• Supported file formats: JPEG, PNG, BMP, TIFF, and PDF (text-embedded or scanned). Text-embedded PDFs are best to eliminate the possibility of error in character extraction and location.

• For PDF and TIFF, up to 2000 pages can be processed (with a free tier subscription, only the first two pages are processed).

• The file size must be less than 50 MB.

• Image dimensions must be between 50 x 50 pixels and 10000 x 10000 pixels.

• PDF dimensions are up to 17 x 17 inches, corresponding to Legal or A3 paper size, or smaller.

• The total size of the training data is 500 pages or less.

• If your PDFs are password-locked, you must remove the lock before submission.

• For unsupervised learning (without labeled data):

  • Data must contain keys and values.
  • Keys must appear above or to the left of the values; they can't appear below or to the right.

  Tip

  Training data

  • If possible, use text-based PDF documents instead of image-based documents. Scanned PDFs are handled as images.
  • For filled-in forms, use examples that have all of their fields filled in.
  • Use forms with different values in each field.
  • If your form images are of lower quality, use a larger data set (10-15 images, for example).

Supported languages and locales

Form Recognizer preview version introduces additional language support for custom models. See our Language Support for a complete list of supported handwritten and printed text.

Form Recognizer preview v3.0

Form Recognizer v3.0 (preview) introduces several new features and capabilities:

• Custom model API (v3.0) supports signature detection for custom forms. When training custom models, you can specify certain fields as signatures. When a document is analyzed with your custom model, it will indicate whether a signature has been detected or not.

• Follow our Form Recognizer v3.0 migration guide to learn how to use the preview version in your applications and workflows.

• Explore our REST API (preview) to learn more about the preview version and new capabilities.

Try signature detection

1. Build your training data set.

2. Navigate to the Form Recognizer Studio and select Custom form under Custom models:

   

3. Follow the workflow to create a new project:

4. Follow the Custom model input requirements.

5. Label your documents. For signature fields, using region labeling is recommended for better accuracy.

6. Label your documents. For signature fields, using region labeling is recommended for better accuracy.

   

Once your training set has been labeled, you can train your custom model and use it to analyze documents. The signature fields will specify whether a signature was detected or not.

Next steps

• Complete a Form Recognizer quickstart:

  Form Recognizer quickstart

• Explore our REST API:

  Form Recognizer API v2.1