TruncationSelectionPolicy Class

  • Reference

Defines an early termination policy that cancels a given percentage of runs at each evaluation interval.

Initialize a TruncationSelectionPolicy.

Inheritance
azureml.train.hyperdrive.policy.EarlyTerminationPolicy
TruncationSelectionPolicy

Constructor

TruncationSelectionPolicy(truncation_percentage, evaluation_interval=1, delay_evaluation=0)

Parameters

truncation_percentage
int
Required

The percentage of runs to cancel at each evaluation interval.

evaluation_interval
int
default value: 1

The frequency for applying the policy.

delay_evaluation
int
default value: 0

The number of intervals for which to delay the first policy evaluation. If specified, the policy applies every multiple of evaluation_interval that is greater than or equal to delay_evaluation.

truncation_percentage
int
Required

The percentage of runs to cancel at each evaluation interval.

evaluation_interval
int
Required

The frequency for applying the policy.

delay_evaluation
int
Required

The number of intervals for which to delay the first policy evaluation. If specified, the policy applies every multiple of evaluation_interval that is greater than or equal to delay_evaluation.

Remarks

This policy periodically cancels the given percentage of runs that rank the lowest for their performance on the primary metric. The policy strives for fairness in ranking the runs by accounting for improving model performance with training time. When ranking a relatively young run, the policy uses the corresponding (and earlier) performance of older runs for comparison. Therefore, runs aren't terminated for having a lower performance because they have run for less time than other runs.

The Truncation Selection policy takes the following configuration parameters:

  • truncation_percentage: The percentage of lowest performing runs to terminate at each evaluation interval.

  • evaluation_interval: The frequency for applying the policy. Each time the training script logs the primary metric counts as one interval.

  • delay_evaluation: The number of intervals to delay policy evaluation. Use this parameter to avoid premature termination of training runs. If specified, the policy applies every multiple of evaluation_interval that is greater than or equal to delay_evaluation.

For example, when evaluating a run at a interval N, its performance is only compared with the performance of other runs up to interval N even if they reported metrics for intervals greater than N.

Attributes

delay_evaluation

Return the value for number of sequences the first evaluation is delayed.

Returns

The delay evaluation.

Return type

int

evaluation_interval

Return evaluation interval value.

Returns

The evaluation interval.

Return type

int

truncation_percentage

Return truncation percentage value.

Returns

The truncation percentage.

Return type

int

POLICY_NAME 

POLICY_NAME = 'TruncationSelection'