Hi @ramr-msft , this doesn't answer my question - I am already outputting the model data from the designer pipeline into a know location as a FileDataset. I specifically want to know how I can define this dataset to have the ModelDirectory type, rather than the AnyDirectory type, when executing from a "Execute Python Code" block within the designer, so that I can use this user defined model dataset in a designer inference pipeline as the model input to the scoring block.

As things stand right now, the designer pipeline won't validate because the FileDataset has this incorrect sub-type. Since this directory is a direct clone of the trained_model_outputs standard ouptut folder from a "Tune Model Hyperparameters" block, there is no reason it won't work besides have the wrong metadata associated with it on registration.

The appropriate AML package to use here look like its this, but I cannot find online documentation for it:

azureml.studio.core.io.model_directory import ModelDirectory

@ramr-msft that would be great - looking forward to connecting on that

@ramr-msft I will file a support ticket. To give some more context for the sake of this question that others might see - we are trying to bring the designer in line with enterprise change management principles, including (as much as is possible) code freezes, pipeline version control, automated CI/CD and enforced naming conventions for artifacts (models, transforms etc).

We have a training pipeline that is compliant, however the designer requires a high degree of manual manipulation to update the AKS in the deployment phase. If we can define fixed dataset objects which are updated in the training phase, then we can code freeze the inference pipeline, and updating the AKS webservice becomes a case of a couple of button pushes (since we can define the datasets to use the most recent dataset versions only), which could be automated through an RPA process.

@DustinReagan-3231 I managed to take this all the way to the Microsoft product development team, and the answer (paraphrased) was as follows. This ModelDirectory class is 100% abstracted from the users, and thus cannot be called from with a user defined block.

The group told me that the designer is not meant to be a production grade tool in it's current implemention - more of a prototyping tool - and it is not compatible with CI/CD processes like python scripts are. Automated retraining in the designer is not a supported scenario as of today either, since we can't create an API for the training pipeline. I can't speak to the roadmap though, but I did advocate that some method of enabling automation and MLOps in the designer would be highly desirable.

So I don't think you're missing anything. In the end we had to work with the team to convert the prototyped pipelines into python equivalents.

Totally agree - if this one thing were solved for then CI would be totally possible with the designer

Thanks for the update. As the designer matures into an enterprise grade tool, more people are going to need this functionality (I have multiple clients that are asking about it), as they look to integrate this with their enterprise-wide CI/CD processes. If you could look into this and let me know when this feature is being looked at that would be very helpful.

Thanks for the response - I added this in, but the pipeline is still short-circuiting, it didn't seem to do anything. Is there a setting in one of the modules that I need to change as well to add this in so that it has the equivalent functionality to the SDK?

I've seen a way of doing this with the Import Data module's "regenerate output" setting, but I would much rather use datasets so we can keep the SQL queries versioned if possible.