Create a form-processing model

After you review the requirements, you can get started creating your form-processing model.

Sign in to AI Builder

  1. Go to Power Apps or Power Automate, and sign in with your organizational account.
  2. In the left pane, select AI Builder > Build.
  3. Select Form processing.
  4. Type a name for your model.
  5. If you want to create your model by using your own documents, make sure that you have at least five examples that use the same layout. Otherwise, you can use sample data to create the model.
  6. Select Create.

Upload and analyze documents

You need some sample documents to train your model for the type of forms you'll be working with.

  1. Select Add documents.

    Add documents screen

  2. Select at least five sample documents that represent the type of forms you you want to train the model for. Only JPG, PNG, and PDF files are accepted.

  3. Verify the selection, and then select Upload documents.

  4. After the upload is completed, select Close.

  5. Select Analyze.

    Select Analyze

Analyze

During the analysis step, AI Builder examines the documents that you uploaded, and detects the fields and tables in your document. The time it takes to complete this operation depends on the number of documents provided, but in most cases, it should only take a few minutes.

When the analysis has finished, select the thumbnail to open the field selection experience.

Select form fields

Fields and tables that are automatically detected

To help you get started, some fields and tables have been automatically detected for you. They're shown by dotted rectangles. If you want your model to return those fields, you just have to click to select them and validate the selection.

Select detected fields

Fields that aren't detected automatically

Some fields in your document might not have been automatically detected by AI Builder. Draw a rectangle around the field you're interested in, give it a name, and validate the selection.

Draw fields

When you hover over different words in your documents, light blue boxes appear. These indicate that you can draw a rectangle around those words to select a field.

Select fields

When you draw a rectangle to indicate a field, if your initial selection didn't pick up all the words you wanted—or it picked up additional ones—you can adjust your selection.

Adjust drawn fields

Note

Adding tables that haven't been detected isn't currently supported.

Resizing fields that are automatically detected

If a field that you want your model to return has been automatically detected, but doesn't contain all the values for that field, you can adjust the selection by selecting the field, resizing the selection, and then validating the selection.

Resize fields

Hiding fields

If you're finding it difficult to select fields because of automated detected field suggestions that are incorrect, or fields that are too close to each other, you can click a selected or suggested field and hide it. To show all the hidden fields, select the Show hidden fields option that appears at the top as you hide fields.

Hide fields

Rename fields

Both for automatically detected fields and fields that you manually draw, you have the option to rename them so they're easier to reference.

Rename fields

Note

Only fields can be renamed. Renaming tables isn't currently supported.

Confirming fields

If you've drawn new fields or resized automatically detected fields, you'll be presented with all the documents that you've uploaded. In this step you'll need to draw the fields that are marked as pending in the right panel. By doing this, you're teaching your AI Builder model to recognize these fields in documents of this type.

To draw a field on a document, just start drawing a selection on the document and you'll be asked which field it corresponds to. You can also select Draw in document on the fields list to the right to start the selection process for that field.

If a field isn't present in one of the documents, select Field not in document.

Confirm fields

Next step

Train and publish your form-processing model

See also

Use a form-processing model in Power Automate
Use the form-processor component in Power Apps