question

suba1987-5883 avatar image
0 Votes"
suba1987-5883 asked PRADEEPCHEEKATLA-MSFT commented

Automatically delete HDInsight cluster after period of inactivity

Hi,
I have created an Azure template for creating HDInsight cluster.
I would like to be able to set an option to delete the cluster after a period of inactivity. Ideally, I could set up this parameter during the creation and after (let's say) an hour of inactivity the cluster would be automatically deleted to save on costs.
Other acceptable solutions would be to have an email sent to the user who created the account that the cluster has been inactive (or running for a predefined period of time) reminding them to delete it.

I know that Azure Data Factory can create a cluster and set the time for it to be active, but I'd like to utilize the template I already created and I'd like to retain the full control of the cluster once created (access to Jupyter, Ambari, etc.)

Thanks

azure-automationazure-hdinsight
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT commented

Hi @suba1987-5883,

Welcome to the Microsoft Q&A platform.

Unfortunately, you cannot delete Azure HDInsight cluster after a period of inactivity.

I would suggest you to vote up an idea submitted by another Azure customer.

https://feedback.azure.com/forums/217335-hdinsight/suggestions/5663773-start-stop-cluster-hdinsight

All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.

Currently, if you want to delete the cluster after a period of inactivity, you need to use create on-demand Apache Hadoop clusters in HDInsight using Azure Data Factory.

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi, thanks for the answer.
I've seen that idea, but since it's from 2016 I don't have any real hopes it will be implemented any time soon. So I'm looking for a workaround.

If I create a cluster using Data Factory, can I use it as a regular cluster? Login to the Jupyter and run notebooks? Also, is it possible to utilize already existing template for this, I'd hate to be required to input all of the parameters every time I'm creating a cluster.

Thanks

0 Votes 0 ·

Hi @suba1987-5883,

Yes, this is the most awaiting feature in Azure HDInsight. I don't think this will be implemented soon.

Currently, you need to manually delete the HDInsight cluster once you finish your work on HDInsight cluster.

No, it is not possible to utilize Jupyter on on-demand HDInsight clusters.

Hope this helps. Do let us know if you any further queries.

0 Votes 0 ·

Hi @suba1987-5883,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·
Show more comments