We are deploying an ML model through the Azure ML designer. Over time the underlying data changes and so the model needs to be regularly retrained. The actual designer pipeline and the dataset definition (a query on a SQL database) are not changed, only the underlying data in the Azure SQL database.
Right now, the pipeline API can be triggered, but it does not execute (as expected). This is equivalent to the default allow_reuse = True in the Azure ML SDK. Is there a way to disable this setting (or set in to False) in the designer so that when the API is triggered we can force it to re-execute the pipeline every time we want to do a retraining (eg once a week) as new data comes in, so that a new model version is generated every time.
To be clear, the training takes around 20 minutes, and the compute cluster it runs on has a 120 second scale-down time, so cost considerations etc (ie the reason for this feature being enabled by default) are not a concern.
Thanks in advance for any help.
