@TAN Thiam Huat Currently, there is no offering from Azure Machine Learning with a managed RStudio as AWS Sagemaker.
You can use execute R script module as part of Azure ML designer but this offers very limited functionality with only some pre-installed packages in the R script. This module also needs to be part of the experiment combined with other data processing and inference modules.
The only offering that has R studio preinstalled is the Azure DSVM in windows and linux configurations. This is similar to what you want to achieve with a standard VM by installing the same. These DSVM also come with pre-configured sizes so you can choose the required RAM based on compute requirements. I do not think you can configure auto scaling but you can choose to create highly available configuration using the advanced options or choosing more than one availability zones. I hope this helps!!
If an answer is helpful, please click on or upvote which might help other community members reading this thread.