Spot node pools in AKS and alerting on evictions

Question

I want to start using spot instances in AKS for some of my nodepools. Is there a way to get notified when nodes are evicted? Or see a history of when it happens at least?

Thanks,

Answer 1

@Todd Bowman Thanks for posting your query on Microsoft Q&A.

Issue summary:
Is there a way to get notified when nodes are evicted? Or see a history of when it happens at least?

Recommended solution:
You can opt-in to receive in-VM notifications through Azure Scheduled Events. This will notify you if your VMs are being evicted and you will have 30 seconds to finish any jobs and perform shutdown tasks prior to the eviction.

You can find more details in this document about Azure Spot VMs and Eviction policy.

You can also see the eviction history on the Azure Portal as described here:

Update: Ability to Query Spot eviction rates via API is also in the roadmap in the near future.
Let me know if you have further questions and I would be happy to help.

Answer 2

So, it sounds like there is no built-in way with Azure Monitor a similar to get alerts when a VM is evicted? Nothing in a dashboard I can easily see?

And also, no good way to view the history of evictions in my particular tenant/subscription/resource groups?

Basically, if I have a spot VM and it goes away before I set up the in-VM notification, it's just gone with no visible way for me to find out why, short of opening a support ticket?