maximum runbooks

Question

question

vrakking asked Sep 25, '20 vrakking commented Nov 16, '20

maximum runbooks

Hi :)

At a customer we have a lot of events telling us "the number of runbooks on the runbook server is approaching the maximum number allowed for this runbook server" which is, in our case, 200.
when I take a look with the orchestrator health checker:

As you can see we are not nearly at that amount of running runbooks.

I have no idea where to look or even how to solve this ?? maybe you guys can give me some hints or tips where to start ???

thanks in advanced,
Bart

msc-orchestrator

image.png (36.2 KiB)

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 1 · 2020-09-25T09:21:10.467Z

LeonLaude answered Sep 25, '20 LeonLaude commented Sep 28, '20

Hi @vrakking,

By default, each Runbook Server is configured to simultaneously run a maximum of 50 runbooks, it appears that you have modified this value, which is OK as long as the server performance allows it.
You might be receiving this event because one of your runook server's runbook usage percentage is "yellow", the 86.50% one.

This is mainly just a warning, nothing to be worried about.

(If the reply was helpful please don't forget to upvote or accept as answer, thank you)

Best regards,
Leon

· 2

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

vrakking · Sep 25, 2020 at 12:47 PM

I'm not worried about the event, although it's very annoying getting every hour 15 events or so, but i don't want them, because they are false / positive.

0 ·

LeonLaude vrakking · Sep 28, 2020 at 06:38 AM

Have you tried balancing the runbooks more on all your runbook servers?

0 ·

Answer 2 · 2020-09-25T10:10:05.107Z

AndreasBaumgarten answered Sep 25, '20 vrakking commented Oct 22, '20

In addition to LeonLaude:
The 200 runbooks aren't the total of all Runbooks ;-)
It's the total of the "running Runbook instances/jobs".

If the maximum of Runbook jobs is reached (by default 50 in your case 200) the requests to start a runbook will be queued.

More information you could find here:
How Orchestrator processes a runbook
Job concurrency

Regards
Andreas Baumgarten

(Please don't forget to Accept as answer if the reply is helpful)

· 3

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

vrakking · Sep 25, 2020 at 12:49 PM

I know, but there are 310 date/time monitoring runbook, that's why the high ammount. in total we have 1200 runbooks.....

0 ·

AndreasBaumgarten vrakking · Sep 25, 2020 at 09:14 PM

And how many Runbook Servers are you running?

Kind regards
Andreas

0 ·

vrakking AndreasBaumgarten · Oct 22, 2020 at 03:31 PM

we have 5 Runbook servers running

0 ·

Answer 3 · 2020-09-25T14:10:16.5Z

StefanHorz answered Sep 25, '20

Hi,
in addition to Leon and Andreas:
Rubook Servers work in failover mode (not load balancing) . A Runbook will be executed on the primary Runbook Server until the maximum number of Runbooks (default: 50, in your case: 200) is reached or the Runbook Server has problems. Then on the secondary, then on secondary(2), ...
The order of the Runbook Servers you can see in Runbook Designer in folder "Runbook Servers".
You can change the Runbook Servers from this order with "Invoke Runbook" Activity or in the properties of each Runbook.

Regards,
Stefan

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 4 · 2020-11-03T08:48:49.11Z

MichaelSeidlau2matorcom-9815 answered Nov 3, '20 vrakking commented Nov 16, '20

Hi,

additionally, you said you have 300+ Monitoring Runbooks, that's, as Andreas already said, is the "Problem" that SCO has a limit of 50 actively Running Runbooks.

This limit can be changed, but what i would recommend, to combine your Monitoring Runbooks in one Runbook.

What we do, when we have many Runbokks which should be triggered each hour or each 15 Minutes
We configure one Master Runbook with the Monitor-DatEtime, which will then trigger the Child Runbooks.

image.png (35.1 KiB)

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

vrakking · Nov 16, 2020 at 11:49 AM

Hi Michael,

Thanks for your thoughts, but 99% of the constant running runbooks are monitoring runbooks with 80% of different times to start runbook.

I don't think this will be solvable, because max 50 runbooks was something determined already more than 10 years ago by Opalis and with current servers, this resource issue is not really an issue anymore.

regards,
Bart

0 ·

question