I need to create Single Node Databricks Job Cluster from Azure Data Factory. Currently in Azure Data Factory, there is no option to choose the Cluster mode like Standard or Single Node. We cant mention Worker as 0 since the Standard cluster needs atleast one worker node to execute the Spark commands whereas it is not the case with Single Node. There is an alternate way of using Single Node cluster via Interactive Cluster option, but we want to have it as job cluster so that it gets deleted automatically after the process completes.
Will this feature get added to ADF in near future?