question

bluestreaktx avatar image
0 Votes"
bluestreaktx asked ·

Spot Restart

I'm pretty sure I know the answer, but wanted to ask anyway...

I have a spot VM that gets deallocated every morning, I assume this is due to the need for capacity:

"At any point in time when Azure needs the capacity back, the Azure infrastructure will evict Spot VMs with 30 seconds notice."

All good, but can it be set to restart the VM automatically once the capacity is not needed. Like if my CPU gets to 90%, I can kill some long running process, until the CPU drops back below 10% usage and then restart the process.

Thanks

azure-spot
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

pituach avatar image
1 Vote"
pituach answered ·

Good day,
I am not sure if I fully understand your question.

can it be set to restart the VM automatically once the capacity is not needed.

Do you mean that when Azure does not needs the capacity you want to restart (stop and start the machine or do you mean to start the machine after eviction ? like you do in cases the CPU high to stop processes?

If you mean to stop and start then it should be clear that the spot is based on what the Azure needs and not what your VM needs. When Azure needs the capacity it will take it. Why do you use spot VM if you do not want to use the capacity when it is available?!? It make no sense in first glance to stop when Azure does not need it. The entire idea of spot VM is that you use the time that Azure does not use it.

In case the answer is that you can simply try to start the machine when it is evicted, then If there is capacity, then the VM will deployed, and if there is't then you get error and can try again after few minutes. Note that after eviction if you start the machine it will be charged at the current price. You can do it manually using PowerShell.

As I said I do not fully understand your description.

please clarify and elaborate a bit more on your architecture.

Maybe you meant the opposite?
"restart the VM automatically once the capacity is needed by the Azure (during the 30 seconds notice)"

Like if my CPU gets to 90%, I can kill some long running process

Do you want to restart the machine or kill process?

  • Off-topic: you should probably not design a system that based on killing process. It is sound very poor solution for most cases. I HIGHLY recommend you to try to find a different design like not starting process if you are going to kill it (basically find the reason for the long running and avoid it if possible or try to predict these cases in order to avoid starting the process)

· 2 · Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

All good, but can it be set to restart the VM automatically once the capacity is not needed. Like if my CPU gets to 90%, I can kill some long running process, until the CPU drops back below 10% usage and then restart the process.


0 Votes 0 · ·

The point is Azure should (optionally) be doing the restarting, not the user. Like a bid in a stock market where you buy one unit whenever available at that price or less. Right now we're missing both an automatic "buy when it's available at that price" (if it isn't already) and "buy it again if it was evicted automatically, but is now available".

0 Votes 0 · ·
GreenReaper avatar image
0 Votes"
GreenReaper answered ·

I'm pretty sure what was meant was "can't Azure just restart it for me once there is capacity again at my selected price?" (which may be any spot price)

Currently the answer appears to be no, though it does not explicitly say this for capacity, just for price:

After eviction the price for the VM goes back to being < the max price.

The VM will not be automatically re-started. You can restart the VM yourself, and it will be charged at the current price.

Right now, it seems you have to manually check the price, and manually restart the server. Ideally, you could set a target price (or the max price) and it might run whenever there was capacity at that price. In a true market-oriented world, there'd be no guaranteed instances or prices, but it'd always be possible to guarantee service - if you're willing to pay more than other users were for the last unit of capacity.












· Share
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.