What is Form Recognizer?

Azure Form Recognizer is a cognitive service that uses machine learning technology to identify and extract key/value pairs and table data from form documents. It then outputs structured data that includes the relationships in the original file. You can call your custom Form Recognizer model by using a simple REST API to reduce complexity and easily integrate it into your workflow or application. To get started, you just need five filled-in form documents or two filled-in forms plus an empty form of the same type as your input material. You quickly get accurate results that are tailored to your specific content without heavy manual intervention or extensive data science expertise.

Custom models

The Form Recognizer custom model trains to your own data, and you only need five sample input forms to start. When you submit your input data, the algorithm clusters the forms by type, discovers what keys and tables are present, and associates values to keys and entries to tables. It then outputs structured data that includes the relationships in the original file. After you train the model, you can test and retrain it and eventually use it to reliably extract data from more forms according to your needs.

Unsupervised learning allows the model to understand the layout and relationships between fields and entries without manual data labeling or intensive coding and maintenance. By contrast, pre-trained machine learning models require standardized data. They're less accurate with input material that deviates from traditional formats, like industry-specific forms.

Prebuilt receipt model

Form Recognizer also includes a model for reading sales receipts. This model extracts key information such as the time and date of the transaction, merchant information, amounts of taxes and totals and more. In addition, the prebuilt receipt model is trained to recognize and return all of the text on a receipt.

What it includes

Form Recognizer is available as a REST API. You can create, train, and score a custom model or access the prebuilt model by invoking these APIs. If you want, you can train and run custom models in a local Docker container.

Input requirements

Custom model

Form Recognizer works on input documents that meet these requirements:

  • Format must be JPG, PNG, or PDF (text or scanned). Text-embedded PDFs are best because there's no possibility of error in character extraction and location.
  • If your PDFs are password-locked, you must remove the lock before submitting them.
  • File size must be less than 4 MB.
  • For images, dimensions must be between 600 x 100 pixels and 4200 x 4200 pixels.
  • If scanned from paper documents, forms should be high-quality scans.
  • Text must use the Latin alphabet (English characters).
  • Data must contain keys and values.
  • Keys can appear above or to the left of the values, but not below or to the right.

Form Recognizer doesn't currently support these types of input data:

  • Complex tables (nested tables, merged headers or cells, and so on).
  • Checkboxes or radio buttons.
  • PDF documents longer than 50 pages.

Prebuilt receipt model

The input requirements for the receipt model are slightly different.

  • Format must be JPEG, PNG, BMP, PDF (text or scanned) or TIFF.
  • File size must be less than 20 MB.
  • Image dimensions must be between 50 x 50 pixels and 10000 x 10000 pixels.
  • PDF dimensions must be at most 17 x 17 inches, corresponding to Legal or A3 paper sizes and smaller.
  • For PDF and TIFF, only the first 200 pages are processed (with a free tier subscription, only the first two pages are processed).

Request access

Form Recognizer is available in a limited-access preview. To get access to the preview, fill out and submit the Form Recognizer access request form. The form requests information about you, your company, and the user scenario for which you'll use Form Recognizer. If your request is approved by the Azure Cognitive Services team, you'll receive an email with instructions for accessing the service.

Where do I start?

Step 1: Create a Form Recognizer resource in the Azure portal.

Step 2: Follow a quickstart to use the REST API:

We recommend that you use the free service when you're learning the technology. Keep in mind that the number of free pages is limited to 500 per month.

Step 3: Review the REST APIs

You use the following APIs to train and extract structured data from forms.

Train Model Train a new model to analyze your forms by using five forms of the same type. Or, train with an empty form and two filled-in forms.
Analyze Form Analyze a single document passed in as a stream to extract key/value pairs and tables from the form with your custom model.
Analyze Receipt Analyze a single receipt document to extract key information and other receipt text.

Explore the REST API reference documentation to learn more.

Data privacy and security

This service is offered as a preview of an Azure service under the Online Service Terms. As with all the cognitive services, developers using the Form Recognizer service should be aware of Microsoft policies on customer data. See the Cognitive Services page on the Microsoft Trust Center to learn more.

Next steps

Complete a quickstart to get started with the Form Recognizer APIs.