Use form processing model in Power Automate

[This topic is pre-release documentation and is subject to change.]

Important

To use AI Builder models in Power Automate, you have to create the flow inside a solution. The steps below won't work if you don't follow these instructions first: Create a flow in a solution.

Create your flow

  1. Sign in to Power Automate.

  2. Select My flows in the left-side navigation pane, and then select Create from blank.

  3. Select +New > +Instant-from blank.

  4. Name your flow, select Manually trigger a flow under Choose how to trigger this flow, and then select Create.

  5. Expand Manually trigger a flow, select +Add an input, and then select File as the input type.

  6. Select + New step. Don't worry about the file information fields at this point - that comes later. Expand 'manually trigger a flow'.

  7. Search for Predict in the Search for filters and actions box, and then select Predict Common Data Service (current Environment) in the list of actions. If this action doesn't appear, make sure you're working with a solution as described in Create a flow in a solution.

  8. Select the form processing model you want to use, and specify this Request Payload:

    • For a .jpeg image of the form:

      {
          "base64Encoded": "EXPRESSION",
          "mimeType": "image/jpeg"
      }
      
    • For a .pdf document of the form:

      {
         "base64Encoded": "EXPRESSION",
        "mimeType": "application/pdf"
      }
      
    • In the formula bar on the right, replace Expression with the following expression:

      string(triggerBody()?['file']?['contentBytes'])

      Replace expression screens

Note

Depending on which connector the file comes from, the expression will need to be enclosed by base64() instead of string().

Test and edit your flow

  1. Select Test on the upper right, select I’ll perform the trigger action, and then select Save & Test.

  2. Import a document that can be processed by your trained form processing model, and then select Run flow.

  3. Copy the results to an editor like Visual Studio Code and remove all the " \" characters.

  4. In the Flow editor, select + New step, search for Parse JSON, and then select Parse JSON – Data Operations from the list of actions.

    Parse JSON screens

  5. In the Parse JSON screen, next to Content, select Response Payload.

  6. Select Use sample payload to generate schema link, paste the output from your test, and then select Done.

  7. Copy the generated schema and then paste it into an editor like Visual Studio Code.

  8. Replace all instances of integer with number, and then copy the modified schema back into the Parse JSON screen in Power Automate.

    Visual Studio  screen

    Paste schema

Use form processing model output in Power Automate

Now you can use the output of the form processing model in later Power Automate actions.

For example, to retrieve the value of a field named Total, use the following expression:

     body('Parse_JSON')?['predictionOutput']?['labels']?['Total']?['value']

To iterate over tables, put the entries value on an Apply to each' loop. To access the value of a column named Amount—for instance, inside the table—use the expression:

        items('Apply_to_each')?['Amount']?['value']

Congratulations! You've created a flow that uses an AI Builder form processing model. Select Save on the top right, and then select Test to try out your flow.

Form processing model overview