Transparency note and use cases for Form Recognizer

What is a transparency note?

An AI system includes not only the technology, but also the people who will use it, the people who will be affected by it, and the environment in which it is deployed. Creating a system that is fit for its intended purpose requires an understanding of how the technology works, its capabilities and limitations, and how to achieve the best performance.

Microsoft provides transparency notes to help you understand how our AI technology works. This includes the choices system owners can make that influence system performance and behavior, and the importance of thinking about the whole system, including the technology, the people, and the environment. You can use transparency notes when developing or deploying your own system, or share them with the people who will use or be affected by your system.

Transparency notes are part of a broader effort at Microsoft to put our AI principles into practice. To find out more, see Microsoft's AI principles.

Introduction to Form Recognizer

Form Recognizer is a service in Azure Cognitive Services, accessed via a set of APIs, that allows developers to easily extract structured data from their documents. This structured data can include extracted text, key and value pairs, selection marks, and tables. Form Recognizer is composed of Layout, prebuilt models for documents (such as invoices, receipts, and business cards), and custom forms models.

Form Recognizer's print text extraction supports 73 languages for Layout and Custom Forms features. Handwritten text extraction is exclusively supported for English.

Form Recognizer terms

Term Definition
Layout This feature extracts text, selection marks, and table structure (the row and column numbers associated with the text). See Form Recognizer Layout.
Prebuilt models Prebuilt models are document-specific models for unique form types. These models don't require custom training before use. For example, the prebuilt invoice model extracts key fields from invoices. For more information, see Form Recognizer prebuilt invoice model.
Custom models Form Recognizer allows you to train a custom model that is tailored to your forms. This model extracts text, key/value pairs, selection marks, and table data from forms. Custom models can be improved with human feedback by applying human review, updating the labels, and retraining the model using the API.
Confidence value All Get Analysis Results operations return confidence values in the range between 0 and 1 for all extracted words and key-value mappings. This value represents the service's estimate of how many times it correctly extracts the word out of 100 or correctly maps the key-value pairs. For example, a word that's estimated to be extracted correctly 82% of the time will result in a confidence value of 0.82.

Example use cases for Form Recognizer

Form Recognizer includes features that enable customers from a variety of industries to extract data from their documents. The following scenarios are examples of appropriate use cases:

  • Accounts payable: - A company increases the efficiency of its accounts payable clerks by using the prebuilt invoice model and custom forms to speed up invoice data entry with a human-in-the loop. The prebuilt invoice model can extract key fields, such as Invoice Total and Shipping Address.

  • Insurance form processing: - A customer trains a model by using custom forms to extract a key/value pair in insurance forms, and then feed the data to their business flow to improve the accuracy and efficiency of their process. For their unique forms, customers can build their own model that extracts key values by using custom forms. These extracted values then become actionable data for various workflows within their business.

  • Bank form processing: - A bank uses the prebuilt ID model and custom forms to speed up the data entry for "know your customer" documentation, or to speed up data entry for a mortgage packet. If a bank requires their customers to submit personal identification as part of a process, the prebuilt ID model can extract key values, such as Name and Document Number, speeding up the overall time for data entry.

  • Robotic process automation (RPA): - Content extraction for a wide variety of form types and fields is supported via custom forms. Many industries have technical form types that have distinct structures and key/value pairs. To extract this data and make it actionable, customers can use custom forms to train custom models and enable RPA scenarios.

  • Consumer behavior and market analysis: - Using the prebuilt receipt model, customers can quickly extract key values, such as the merchant name and transaction total, from retail receipts. Customers can then use the extracted data to do consumer behavior analysis. For more information, see Form Recognizer prebuilt receipt model.

Considerations when choosing other use cases

Consider the following factors when you choose a use case.

  • Carefully consider when using for awarding or denying of benefits - Medical insurance: These include healthcare records and medical prescriptions that are a basis for decisions on insurance reward or denial. Loan approvals: These include applications for new loans or refinancing of existing ones.

  • Carefully consider applying human review when sensitive data is involved - It is important to include a human-in-the-loop for a manual review when you're dealing with high-stakes or sensitive data. Machine learning models are not perfect. Consider carefully when to include a manual review step for certain workflows. For example, identity verification at a port of entry such as airports should include human oversight.

  • Carefully consider the supported document types and locales - Prebuilt models have a pre-defined list of supported fields and are built for specific locales. Be sure to carefully check the officially supported locales and document types to ensure best results. For example, see Form Recognizer prebuilt Receipt locales.