What's new in Azure Form Recognizer
Form Recognizer service is updated on an ongoing basis. Bookmark this page to stay up to date with release notes, feature enhancements, and documentation updates.
June 2022
Form Recognizer Studio June Update
The June release is the latest update to the Form Recognizer Studio. There are considerable UX and accessbility improvements addressed in this update:
- 🆕 Code sample for Javascript and C#. Studio code tab now includes sample codes written in Javascript and C# in addition to the already existing Python code.
- 🆕 New document upload UI. Studio now supports uploading a document with drag & drop into the new upload UI.
- 🆕 New feature for custom projects. Custom projects now support creating storage account and file directories when configuring the project. In addition, custom project now supports uploading training files directly within the Studio and copying the existing custom model.
Form Recognizer v3.0 preview release
The 2022-06-30-preview release is the latest update to the Form Recognizer service for v3.0 capabilities and presents extensive updates across the feature APIs:
- 🆕 Layout extends structure extraction. Layout now includes added structure elements including sections, section headers, and paragraphs. This update enables finer grain document segmentation scenarios. For a complete list of structure elements identified, see enhanced structure.
- 🆕 Custom neural model tabular fields support. Custom document models now support tabular fields. Tabular fields by default are also multi page. To learn more about tabular fields in custom neural models, see tabular fields.
- 🆕 Custom template model tabular fields support for cross page tables. Custom form models now support tabular fields across pages. To learn more about tabular fields in custom template models, see tabular fields.
- 🆕 Invoice model output now includes general document key-value pairs. Where invoices contain required fields beyond the fields included in the prebuilt model, the general document model supplements the output with key-value pairs. See key value pairs.
- 🆕 Invoice language expansion. The invoice model includes expanded language support. See supported languages.
- 🆕 Prebuilt business card now includes Japanese language support. See supported languages.
- 🆕 Prebuilt ID document model. The ID document model now extracts DateOfIssue, Height, Weight, EyeColor, HairColor, and DocumentDiscriminator from US driver's licenses. See field extraction.
- 🆕 Read model now supports common Microsoft Office document types. Document types like Word (docx) and PowerPoint (ppt) are now supported with the Read API. See page extraction.
Form Recognizer SDK beta preview release
The latest beta release version of the Azure Form Recognizer SDKs incorporates new features, minor feature updates and bug fixes.
Note
The 4.0.0-beta.4 (C# and JavaScript), 4.0.0-beta.5 (Java), and 3.2.0b5 (Python) previews contain the same updates and bug fixes but the versioning is no longer in sync across all programming languages.
This new release includes the following updates:
Version 4.0.0-beta.4 (2022-06-08)
Changelog/Release History
Package (NuGet)
SDK reference documentation
February 2022
Form Recognizer v3.0 preview release
Form Recognizer v3.0 preview release introduces several new features and capabilities and enhances existing one:
- 🆕 Custom neural model or custom document model is a new custom model to extract text and selection marks from structured forms, semi-strutured and unstructured documents.
- 🆕 W-2 prebuilt model is a new prebuilt model to extract fields from W-2 forms for tax reporting and income verification scenarios.
- 🆕 Read API extracts printed text lines, words, text locations, detected languages, and handwritten text, if detected.
- General document pre-trained model is now updated to support selection marks in addition to API text, tables, structure, key-value pairs, and named entities from forms and documents.
- Invoice API Invoice prebuilt model expands support to Spanish invoices.
- Form Recognizer Studio adds new demos for Read, W2, Hotel receipt samples, and support for training the new custom neural models.
- Language Expansion Form Recognizer Read, Layout, and Custom Form add support for 42 new languages including Arabic, Hindi, and other languages using Arabic and Devanagari scripts to expand the coverage to 164 languages. Handwritten language support expands to Japanese and Korean.
Get started with the new REST API, Python, or .NET SDK for the v3.0 preview API.
Form Recognizer model data extraction
Model | Text extraction | Key-Value pairs | Selection Marks | Tables | Entities | Signatures |
---|---|---|---|---|---|---|
🆕Read | ✓ | |||||
🆕General document | ✓ | ✓ | ✓ | ✓ | ✓ | |
Layout | ✓ | ✓ | ✓ | |||
Invoice | ✓ | ✓ | ✓ | ✓ | ||
Receipt | ✓ | ✓ | ||||
ID document | ✓ | ✓ | ||||
Business card | ✓ | ✓ | ||||
Custom template | ✓ | ✓ | ✓ | ✓ | ✓ | |
Custom neural | ✓ | ✓ | ✓ | ✓ |
Form Recognizer SDK beta preview release
The latest beta release version of the Azure Form Recognizer SDKs incorporates new features, minor feature updates and bug fixes.
Note
The 4.0.0-beta.3 (C# and JavaScript), 4.0.0-beta.4 (Java), and 3.2.0b4 (Python) previews contain the same updates and bug fixes but the versioning is no longer in sync across all programming languages.
This new release includes the following updates:
🆕 Custom Document models and modes:
- Custom template (formerly custom form)
- Custom neural.
- Custom model—build mode.
🆕 W-2 prebuilt model (prebuilt-tax.us.w2).
🆕 Read prebuilt model (prebuilt-read).
🆕 Invoice prebuilt model (Spanish) (prebuilt-invoice).
Version 4.0.0-beta.3 (2022-02-10)
Changelog/Release History
Package (NuGet)
SDK reference documentation
November 2021
Form Recognizer v3.0 preview SDK release update (beta.2)
The beta.2 version of the Azure Form Recognizer SDKs has been released. This new beta release incorporates bug fixes and minor feature updates.
Version 4.0.0-beta.2 (2021-11-09)
| Package (NuGet) | Changelog | API reference documentation
Bugs Fixed
The BuildModelOperation
and CopyModelOperation
now correctly populate the PercentCompleted
property, and no longer return a constant value of 0.
October 2021
Form Recognizer v3.0 preview release (beta.1)
Version 4.0.0-beta.1 (2021-10-07)
Form Recognizer v3.0 preview release introduces several new features and capabilities:
General document model is a new API that uses a pre-trained model to extract text, tables, structure, key-value pairs, and named entities from forms and documents.
Hotel receipt model added to prebuilt receipt processing.
Expanded fields for ID document the ID model supports endorsements, restrictions, and vehicle classification extraction from US driver's licenses.
Signature field is a new field type in custom forms to detect the presence of a signature in a form field.
Language Expansion Support for 122 languages (print) and 7 languages (handwritten). Form Recognizer Layout and Custom Form expand supported languages to 122 with its latest preview. The preview includes text extraction for print text in 49 new languages including Russian, Bulgarian, and other Cyrillic and more Latin languages. In addition extraction of handwritten text now supports seven languages that include English, and new previews of Chinese Simplified, French, German, Italian, Portuguese, and Spanish.
Tables and text extraction enhancements Layout now supports extracting single row tables also called key-value tables. Text extraction enhancements include better processing of digital PDFs and Machine Readable Zone (MRZ) text in identity documents, along with general performance.
Form Recognizer Studio To simplify use of the service, you can now access the Form Recognizer Studio to test the different prebuilt models or label and train a custom model
Get stared with the new REST API, Python, or .NET SDK for the v3.0 preview API.
Form Recognizer model data extraction
Model | Text extraction | Key-Value pairs | Selection Marks | Tables | Entities |
---|---|---|---|---|---|
🆕General document | ✓ | ✓ | ✓ | ✓ | ✓ |
Layout | ✓ | ✓ | ✓ | ||
Invoice | ✓ | ✓ | ✓ | ✓ | |
Receipt | ✓ | ✓ | |||
ID document | ✓ | ✓ | |||
Business card | ✓ | ✓ | |||
Custom | ✓ | ✓ | ✓ | ✓ | ✓ |
September 2021
- Azure metrics explorer advanced features are available on your Form Recognizer resource overview page in the Azure portal.
Monitoring menu
Charts
ID document model update: given names including a suffix, with or without a period (full stop), process successfully:
Input Text Result with update William Isaac Kirby Jr. FirstName: William IsaacLastName: Kirby Jr. Henry Caleb Ross Sr FirstName: Henry Caleb LastName: Ross Sr
July 2021
System-assigned managed identity support
You can now enable a system-assigned managed identity to grant Form Recognizer limited access to private storage accounts including accounts protected by a Virtual Network (VNet) or firewall or have enabled bring-your-own-storage (BYOS). See Create and use managed identity for your Form Recognizer resource to learn more.
June 2021
Form Recognizer containers v2.1 released in gated preview
Form Recognizer features are now supported by six feature containers—Layout, Business Card,ID Document, Receipt, Invoice, and Custom. To use them, you must submit an online request, and receive approval.
See Install and run Docker containers for Form Recognizer and Configure Form Recognizer containers
Form Recognizer connector released in preview
The Form Recognizer connector integrates with Azure Logic Apps, Microsoft Power Automate, and Microsoft Power Apps. The connector supports workflow actions and triggers to extract and analyze document data and structure from custom and prebuilt forms, invoices, receipts, business cards and ID documents.
Form Recognizer SDK v3.1.0 patched to v3.1.1 for C#, Java, and Python
The patch addresses invoices that don't have subline item fields detected such as a FormField
with Text
but no BoundingBox
or Page
information.
May 2021
Form Recognizer 2.1 API Generally Available (GA) release
Form Recognizer 2.1 is generally available. The General Availability (GA) release marks the stability of the changes introduced in prior 2.1 preview package versions. This release enables you to detect and extract information and data from the following document types:
Get started
Go to the Form Recognizer Sample Tool and follow the quickstart
Layout adds table headers
The updated Layout API table feature adds header recognition with column headers that can span multiple rows. Each table cell has an attribute that indicates whether it's part of a header or not. This update can be used to identify which rows make up the table header.
SDK updates
| Reference documentation | NuGet package version 3.0.1 |
Non-breaking changes
- FormRecognizerModelFactory class now supports updates to TextAppearance and ReadingOrder and removal of TextStyle models. See Breaking changes
Breaking changes (May)
Client defaults to the latest supported service version, currently v2.1. You can specify version 2.0 in the FormRecognizerClientOptions object's Version property.
StartRecognizeIdentityDocuments. Renamed methods and method parameters using Identity to replace Id keyword for all related identity documents recognition API functionalities.
FormReadingOrder. ReadingOrder renamed to FormReadingOrder.
AsCountryRegion. AsCountryCode renamed to AsCountryRegion.
TextAppearance now includes StyleName and StyleConfidence properties (formerly part of the TextStyle object).
FieldValueType. Value Gender removed from the model.
TextStyle model removed.
FieldValueGender type removed.
April 2021
SDK preview updates for API version 2.1-preview.3
NuGet package version 3.1.0-beta.4
New methods to analyze data from identity documents:
StartRecognizeIdDocumentsFromUriAsync
StartRecognizeIdDocumentsAsync
For a list of field values, see Fields extracted in our Form Recognizer documentation.
Expanded the set of document languages that can be provided to the StartRecognizeContent method.
New property
Pages
supported by the following classes:RecognizeBusinessCardsOptions RecognizeCustomFormsOptions RecognizeInvoicesOptions RecognizeReceiptsOptions
The
Pages
property allows you to select individual or a range of pages for multi-page PDF and TIFF documents. For individual pages, enter the page number, for example,3
. For a range of pages (like page 2 and pages 5-7) enter the p age numbers and ranges separated by commas:2, 5-7
.New property
ReadingOrder
supported for the following class:The
ReadingOrder
property is an optional parameter that allows you to specify which reading order algorithm—basic
ornatural
—should be applied to order the extraction of text elements. If not specified, the default value isbasic
.
Breaking changes (April)
The client defaults to the latest supported service version, which is currently 2.1-preview.3.
StartRecognizeCustomForms method now throws a
RequestFailedException()
when an invalid file is passed.
March 2021
Form Recognizer v2.1 public preview 3 is now available. v2.1-preview.3 has been released, including the following features:
New prebuilt ID model The new prebuilt ID model enables customers to take IDs and return structured data to automate processing. It combines our powerful Optical Character Recognition (OCR) capabilities with ID understanding models to extract key information from passports and U.S. driver licenses.
Line-item extraction for invoice model - Prebuilt Invoice model now supports line item extraction; it now extracts full items and their parts - description, amount, quantity, product ID, date and more. With a simple API/SDK call, you can extract useful data from your invoices - text, table, key-value pairs, and line items.
Supervised table labeling and training, empty-value labeling - In addition to Form Recognizer's state-of-the-art deep learning automatic table extraction capabilities, it now enables customers to label and train on tables. This new release includes the ability to label and train on line items/tables (dynamic and fixed) and train a custom model to extract key-value pairs and line items. Once a model is trained, the model will extract line items as part of the JSON output in the documentResults section.
In addition to labeling tables, you can now label empty values and regions. If some documents in your training set don't have values for certain fields, you can label them so that your model will know to extract values properly from analyzed documents.
Support for 66 new languages - The Layout API and Custom Models for Form Recognizer now support 73 languages.
Natural reading order, handwriting classification, and page selection - With this update, you can choose to get the text line outputs in the natural reading order instead of the default left-to-right and top-to-bottom ordering. Use the new readingOrder query parameter and set it to "natural" value for a more human-friendly reading order output. In addition, for Latin languages, Form Recognizer will classify text lines as handwritten style or not and give a confidence score.
Prebuilt receipt model quality improvements This update includes many quality improvements for the prebuilt Receipt model, especially around line item extraction.
November 2020
New features
Form Recognizer v2.1 public preview 2 is now available. v2.1-preview.2 has been released, including the following features:
New prebuilt invoice model - The new prebuilt Invoice model enables customers to take invoices in various formats and return structured data to automate the invoice processing. It combines our powerful Optical Character Recognition (OCR) capabilities with invoice understanding deep learning models to extract key information from invoices in English. It extracts key text, tables, and information such as customer, vendor, invoice ID, invoice due date, total, amount due, tax amount, ship to, and bill to.
Enhanced table extraction - Form Recognizer now provides enhanced table extraction, which combines our powerful Optical Character Recognition (OCR) capabilities with a deep learning table extraction model. Form Recognizer can extract data from tables, including complex tables with merged columns, rows, no borders and more.
Client library update - The latest versions of the client libraries for .NET, Python, Java, and JavaScript support the Form Recognizer 2.1 API.
New language supported: Japanese - The following new languages are now supported: for
AnalyzeLayout
andAnalyzeCustomForm
: Japanese (ja
). Language supportText line style indication (handwritten/other) (Latin languages only) - Form Recognizer now outputs an
appearance
object classifying whether each text line is handwritten style or not, along with a confidence score. This feature is supported only for Latin languages.Quality improvements - Extraction improvements including single digit extraction improvements.
New try-it-out feature in the Form Recognizer Sample and Labeling Tool - Ability to try out prebuilt Invoice, Receipt, and Business Card models and the Layout API using the Form Recognizer Sample Labeling tool. See how your data will be extracted without writing any code.
Try the Form Recognizer Sample Labeling tool
Feedback Loop - When Analyzing files via the Sample Labeling tool you can now also add it to the training set and adjust the labels if necessary and train to improve the model.
Auto Label Documents - Automatically labels added documents based on previous labeled documents in the project.
August 2020
New features
Form Recognizer v2.1 public preview is now available. V2.1-preview.1 has been released, including the following features:
- REST API reference is available - View the v2.1-preview.1 reference
- New languages supported In addition to English, the following languages are now supported: for
Layout
andTrain Custom Model
: English (en
), Chinese (Simplified) (zh-Hans
), Dutch (nl
), French (fr
), German (de
), Italian (it
), Portuguese (pt
) and Spanish (es
). - Checkbox / Selection Mark detection – Form Recognizer supports detection and extraction of selection marks such as check boxes and radio buttons. Selection Marks are extracted in
Layout
and you can now also label and train inTrain Custom Model
- Train with Labels to extract key-value pairs for selection marks. - Model Compose - allows multiple models to be composed and called with a single model ID. When you submit a document to be analyzed with a composed model ID, a classification step is first performed to route it to the correct custom model. Model Compose is available for
Train Custom Model
- Train with labels. - Model name - add a friendly name to your custom models for easier management and tracking.
- New pre-built model for Business Cards for extracting common fields in English, language business cards.
- New locales for pre-built Receipts in addition to EN-US, support is now available for EN-AU, EN-CA, EN-GB, EN-IN
- Quality improvements for
Layout
,Train Custom Model
- Train without Labels and Train with Labels.
v2.0 includes the following update:
- The client libraries for NET, Python, Java, and JavaScript have entered General Availability.
New samples are available on GitHub.
- The Knowledge Extraction Recipes - Forms Playbook collects best practices from real Form Recognizer customer engagements and provides usable code samples, checklists, and sample pipelines used in developing these projects.
- The Sample Labeling tool has been updated to support the new v2.1 functionality. See this quickstart for getting started with the tool.
- The Intelligent Kiosk Form Recognizer sample shows how to integrate
Analyze Receipt
andTrain Custom Model
- Train without Labels.
July 2020
New features
v2.0 reference available - View the v2.0 API Reference and the updated SDKs for .NET, Python, Java, and JavaScript.
Table enhancements and Extraction enhancements - includes accuracy improvements and table extractions enhancements, specifically, the capability to learn tables headers and structures in custom train without labels.
Currency support - Detection and extraction of global currency symbols.
Azure Gov - Form Recognizer is now also available in Azure Gov.
Enhanced security features:
- Bring your own key - Form Recognizer automatically encrypts your data when persisted to the cloud to protect it and to help you to meet your organizational security and compliance commitments. By default, your subscription uses Microsoft-managed encryption keys. You can now also manage your subscription with your own encryption keys. Customer-managed keys, also known as bring your own key (BYOK), offer greater flexibility to create, rotate, disable, and revoke access controls. You can also audit the encryption keys used to protect your data.
- Private endpoints – Enables you on a virtual network to securely access data over a Private Link.
June 2020
New features
- CopyModel API added to client SDKs - You can now use the client SDKs to copy models from one subscription to another. See Back up and recover models for general information on this feature.
- Azure Active Directory integration - You can now use your Azure AD credentials to authenticate your Form Recognizer client objects in the SDKs.
- SDK-specific changes - This change includes both minor feature additions and breaking changes. For more information, see the SDK changelogs.
April 2020
New features
SDK support for Form Recognizer API v2.0 Public Preview - This month we expanded our service support to include a preview SDK for Form Recognizer v2.0 (preview) release. Use the links below to get started with your language of choice:
The new SDK supports all the features of the v2.0 REST API for Form Recognizer. You can share your feedback on the SDKs through the SDK Feedback form.
Copy Custom Model You can now copy models between regions and subscriptions using the new Copy Custom Model feature. Before invoking the Copy Custom Model API, you must first obtain authorization to copy into the target resource. This authorization is secured by calling the Copy Authorization operation against the target resource endpoint.
- Generate a copy authorization REST API
- Copy a custom model REST API
Security improvements
- Customer-Managed Keys are now available for FormRecognizer. For more information, see Data encryption at rest for Form Recognizer.
- Use Managed Identities for access to Azure resources with Azure Active Directory. For more information, see Authorize access to managed identities.
March 2020
New features
Value types for labeling You can now specify the types of values you're labeling with the Form Recognizer Sample Labeling tool. The following value types and variations are currently supported:
string
- default,
no-whitespaces
,alphanumeric
- default,
number
- default,
currency
- default,
date
- default,
dmy
,mdy
,ymd
- default,
time
integer
See the Sample Labeling tool guide to learn how to use this feature.
Table visualization The Sample Labeling tool now displays tables that were recognized in the document. This feature lets you view recognized and extracted tables from the document prior to labeling and analyzing. This feature can be toggled on/off using the layers option.
The following image is an example of how tables are recognized and extracted:
The extracted tables are available in the JSON output under
"pageResults"
.Important
Labeling tables isn't supported. If tables are not recognized and extrated automatically, you can only label them as key/value pairs. When labeling tables as key/value pairs, label each cell as a unique value.
Extraction enhancements
This release includes extraction enhancements and accuracy improvements, specifically, the capability to label and extract multiple key/value pairs in the same line of text.
Sample Labeling tool is now open-source
The Form Recognizer Sample Labeling tool is now available as an open-source project. You can integrate it within your solutions and make customer-specific changes to meet your needs.
For more information about the Form Recognizer Sample Labeling tool, review the documentation available on GitHub.
TLS 1.2 enforcement
TLS 1.2 is now enforced for all HTTP requests to this service. For more information, see Azure Cognitive Services security.
January 2020
This release introduces the Form Recognizer 2.0 (preview). In the sections below, you'll find more information about new features, enhancements, and changes.
New features
Custom model
- Train with labels You can now train a custom model with manually labeled data. This method results in better-performing models and can produce models that work with complex forms or forms containing values without keys.
- Asynchronous API You can use async API calls to train with and analyze large data sets and files.
- TIFF file support You can now train with and extract data from TIFF documents.
- Extraction accuracy improvements
Prebuilt receipt model
- Tip amounts You can now extract tip amounts and other handwritten values.
- Line item extraction You can extract line item values from receipts.
- Confidence values You can view the model's confidence for each extracted value.
- Extraction accuracy improvements
Layout extraction You can now use the Layout API to extract text data and table data from your forms.
Custom model API changes
All of the APIs for training and using custom models have been renamed, and some synchronous methods are now asynchronous. The following are major changes:
- The process of training a model is now asynchronous. You initiate training through the /custom/models API call. This call returns an operation ID, which you can pass into custom/models/{modelID} to return the training results.
- Key/value extraction is now initiated by the /custom/models/{modelID}/analyze API call. This call returns an operation ID, which you can pass into custom/models/{modelID}/analyzeResults/{resultID} to return the extraction results.
- Operation IDs for the Train operation are now found in the Location header of HTTP responses, not the Operation-Location header.
Receipt API changes
The APIs for reading sales receipts have been renamed.
- Receipt data extraction is now initiated by the /prebuilt/receipt/analyze API call. This call returns an operation ID, which you can pass into /prebuilt/receipt/analyzeResults/{resultID} to return the extraction results.
Output format changes
The JSON responses for all API calls have new formats. Some keys and values have been added, removed, or renamed. See the quickstarts for examples of the current JSON formats.
Next steps
Complete a quickstart to get started writing a forms processing app with Form Recognizer in the development language of your choice.
See also
Feedback
Submit and view feedback for