Preview policy performance of Bonsai brains


  • Total time to complete: 10 minutes
  • Active time: 5 minutes

You can use the View policy performance feature in Bonsai to confirm the brain is communicating correctly with your simulation before starting a formal assessment. Use the streaming test episodes to review the learned behavior and test responses in simulation.

Before you start

  • You must have a simulator. You cannot train or assess a brain without a simulator. Your simulator can be local (unmanaged) or packaged in Bonsai (managed).
  • You must have a trained (or partially trained brain) that is not actively training. You cannot preview the policy for a brain version that is untrained or in the process of training. You can stop training manually when you are satisfied with the performance metrics or wait for training to stop automatically when the brain reaches your no progress iteration limit.

Step 1: Confirm the brain is stopped

To stop a brain that is currently training:

  • In the CLI: Run bonsai brain version stop-training -n {BRAIN_NAME} --version {BRAIN_VERSION}
  • In the UI: Click the red Stop training button.

    Stop Training button in Bonsai UI

    Screenshot of the Stop Training button that needs to be clicked before Assessment can start.

Tip

Bonsai automatically stops training and marks training complete if learning does not progress within the NoProgressIterationLimit window set for the AI curriculum. :::image-end:::

Step 2: Start the preview

  1. Open the Bonsai UI
  2. Select the brain version you want to review.
  3. Select the Train tab.
  4. Select the View policy performance tab at the bottom of the data panel.

If you are previewing a brain trained with a managed simulator, the preview starts automatically with the package configured in your Inkling file.

Step 3: Review the data stream

Use the interactive, streaming performance charts in the preview panel to review policy responses from the Bonsai brain.

The streaming charts show how data points in your simulation change over training sessions. You can use the data to verify your your AI responds to environmental changes appropriately and is making progress toward training goals.

If your simulator tracks multiple data points, you can click the + Add Chart button to replicate the data chart. Toggle the available fields on or off in each chart to make analysis easier.

Animation of using brain policy preview

Animation of the Bonsai policy preview feature in UI with a brain that has not finished training. The animated mouse cursor clicks "View policy performance", then the screen scrolls down to show the performance chart of Moab demo.

Tip

Previewing a partially trained brain is a good way to perform early verification of your control policy and confirm the brain is developing solutions for the problem you actually want to solve with AI.

Preview performance with an unmanaged (local) simulator

To preview the brain performance with a local simulator, you must connect the simulator to Bonsai with the Bonsai CLI before selecting the View policy performance tab:

  1. Comment out the package statement in your Inkling file (if it exists).
  2. Start the simulator locally.
  3. Connect the unmanaged simulator for assessment:
    bonsai simulator unmanaged connect \
      --simulator-name=SIMULATOR_NAME  \
      --brain-name=BRAIN_NAME          \
      --brain-version=VERSION          \
      --concept_name CONCEPT_NAME      \
      --action Assess
    
    Simulators Found: 1
    Simulators Connected: 1
    Simulators Not Connected: 0
    

When you select the View policy performance tab, you can select your unmanaged simulator from the available list.

Next steps

Once the brain seems to meet your performance expectations, you can formally evaluate the behavior with a formal assessment.