NlpSearchSpace Class

  • Reference

Search space for AutoML NLP tasks.

Inheritance
azure.ai.ml.entities._mixins.RestTranslatableMixin
NlpSearchSpace

Constructor

NlpSearchSpace(*, gradient_accumulation_steps: int | SweepDistribution | None = None, learning_rate: float | SweepDistribution | None = None, learning_rate_scheduler: str | SweepDistribution | None = None, model_name: str | SweepDistribution | None = None, number_of_epochs: int | SweepDistribution | None = None, training_batch_size: int | SweepDistribution | None = None, validation_batch_size: int | SweepDistribution | None = None, warmup_ratio: float | SweepDistribution | None = None, weight_decay: float | SweepDistribution | None = None)

Parameters

Name Description
gradient_accumulation_steps
Required
Optional[Union[int, <xref:SweepDistribution>]]

number of steps over which to accumulate gradients before a backward pass. This must be a positive integer., defaults to None
learning_rate
Required
Optional[Union[float, <xref:SweepDistribution>]]

initial learning rate. Must be a float in (0, 1), defaults to None
learning_rate_scheduler
Required
Optional[Union[str, <xref:SweepDistribution>]]

the type of learning rate scheduler. Must choose from 'linear', 'cosine', 'cosine_with_restarts', 'polynomial', 'constant', and 'constant_with_warmup', defaults to None
model_name
Required
Optional[Union[str, <xref:SweepDistribution>]]

the model name to use during training. Must choose from 'bert-base-cased', 'bert-base-uncased', 'bert-base-multilingual-cased', 'bert-base-german-cased', 'bert-large-cased', 'bert-large-uncased', 'distilbert-base-cased', 'distilbert-base-uncased', 'roberta-base', 'roberta-large', 'distilroberta-base', 'xlm-roberta-base', 'xlm-roberta-large', xlnet-base-cased', and 'xlnet-large-cased', defaults to None
number_of_epochs
Required
Optional[Union[int, <xref:SweepDistribution>]]

the number of epochs to train with. Must be a positive integer, defaults to None
training_batch_size
Required
Optional[Union[int, <xref:SweepDistribution>]]

the batch size during training. Must be a positive integer, defaults to None
validation_batch_size
Required
Optional[Union[int, <xref:SweepDistribution>]]

the batch size during validation. Must be a positive integer, defaults to None
warmup_ratio
Required
Optional[Union[float, <xref:SweepDistribution>]]

ratio of total training steps used for a linear warmup from 0 to learning_rate. Must be a float in [0, 1], defaults to None
weight_decay
Required
Optional[Union[float, <xref:SweepDistribution>]]

value of weight decay when optimizer is sgd, adam, or adamw. This must be a float in the range [0, 1], defaults to None

Keyword-Only Parameters

Name Description
gradient_accumulation_steps
Required
learning_rate
Required
learning_rate_scheduler
Required
model_name
Required
number_of_epochs
Required
training_batch_size
Required
validation_batch_size
Required
warmup_ratio
Required
weight_decay
Required

Examples

creating an nlp search space


   from azure.ai.ml import automl
   from azure.ai.ml.constants import NlpLearningRateScheduler
   from azure.ai.ml.sweep import Uniform

   nlp_search_space = automl.NlpSearchSpace(
       learning_rate_scheduler=NlpLearningRateScheduler.LINEAR,
       warmup_ratio=0.1,
       model_name="roberta-base",
       weight_decay=Uniform(0.01, 0.1),
   )