Edit

View orchestration workflow model details

  • Article

After model training is completed, you can view your model details and see how well it performs against the test set. Observing how well your model performed is called evaluation. The test set consists of data that wasn't introduced to the model during the training process.

Note

Using the Automatically split the testing set from training data option may result in different model evaluation result every time you train a new model, as the test set is selected randomly from your utterances. To make sure that the evaulation is calcualted on the same test set every time you train a model, make sure to use the Use a manual split of training and testing data option when starting a training job and define your Testing set when add your utterances.

Prerequisites

Before viewing a model's evaluation, you need:

See the project development lifecycle for more information.

Model details

In the view model details page, you'll be able to see all your models, with their current training status, and the date they were last trained.

  1. Go to your project page in Language Studio.

  2. Select Model performance from the menu on the left side of the screen.

  3. In this page you can only view the successfully trained models, F1 score for each model and model expiration date. You can select the model name for more details about its performance.

  4. You can find the model-level evaluation metrics under Overview, and the intent-level evaluation metrics. See Evaluation metrics for more information.

    A screenshot of the model performance metrics in Language Studio

  5. The confusion matrix for the model is located under Test set confusion matrix. You can see the confusion matrix for intents.

    Note

    If you don't see any of the intents you have in your model displayed here, it is because they weren't in any of the utterances that were used for the test set.

Load or export model data

To load your model data:

  1. Select any model in the model evaluation page.

  2. Select the Load model data button.

  3. Confirm that you do not have any unsaved changes you need to capture in window that appears, and select Load data.

  4. Wait until your model data finishes loading back into your project. On completion, you'll be redirected back to the Schema design page.

To export your model data:

  1. Select any model in the model evaluation page.

  2. Select the Export model data button. Wait for the JSON snapshot of your model to be downloaded locally.

Delete model

To delete your model from within the Language Studio:

  1. Select Model performance from the left side menu.

  2. Select the model name you want to delete and select Delete from the top menu.

  3. In the window that appears, select OK to delete the model.

Next steps