如何建立基準測試 runbookHow to create a benchmark runbook

重要

此版本的 Orchestrator 已達終止支援,建議您 升級至 Orchestrator 2019This version of Orchestrator has reached the end of support, we recommend you to upgrade to Orchestrator 2019.

您將需要建立效能基準測試,以便將 Orchestrator runbook 的效能優化。You will need to create a performance benchmark in order to optimize the performance for your Orchestrator runbook. 在建立基準測試的過程中,您應該分析 runbook 中的活動。As part of creating the benchmark you should analyze the activities in your runbook.

您可以將 Orchestrator Runbook 活動視為具有兩種不同類型的程式碼:平台程式碼和網域程式碼。Orchestrator runbook activities can be thought of as having two distinct types of code: platform code and domain code. 網域程式 代碼 一詞是用來識別 runbook 活動內的程式碼,該活動通常與 Orchestrator 平臺本身無關 ( ,但有明顯的例外狀況,例如叫用 runbook接合和其他 ) 。The term domain code is used to identify code within a runbook activity that is typically not associated with the Orchestrator platform itself (with notable exceptions, such as Invoke Runbook, Junction, and others). 例如,[叫用 Web 服務 ] 標準活動包含 Orchestrator 平臺程式碼,活動的「配管」以及 ( ) 叫用 SOAP 型 Web 服務的唯一網域程式碼 - 。For example, the Invoke Web Service standard activity would contain Orchestrator platform code (the "plumbing" of the activity) as well as domain code unique to invoking a SOAP-based web service. 由於平台程式碼是從通用架構建置而來,因此大部分活動的平台程式碼均非常類似。The platform code will be very similar for most activities, since it is built on a common framework. 不過,不同活動的網域程式碼可能會有很大的差異。However, there will potentially be great variation in domain code for different activities.

資料記錄Data Logging

資料記錄對 runbook 效能有重大影響。Data logging has a major impact on runbook performance. 為了瞭解效能,請考量下列兩種記錄設定:預設記錄和一般已發佈資料記錄。For the purpose of understanding performance consider two logging configurations: Default logging and Common Published Data logging. 每當活動執行時,預設記錄會將大約 524 位元組的資料寫入 Orchestrator 資料庫。Default logging results in approximately 524 bytes of data being written to the Orchestrator database each time an activity is run. 一般已發佈資料的記錄寫入大約6082個位元組的資料( ( 預設記錄層級的12倍) ) 。Logging of common published data writes approximately 6,082 bytes of data (12 times the default logging level). 這兩種記錄層級之間有顯著的效能差異。There is a notable difference in performance between these logging levels.

請試想以下案例:執行相同的 Runbook 活動兩次,一次採用預設層級的資料記錄,另一次則啟用一般已發佈資料記錄。Consider the scenario where the same runbook activity is run twice, once with data logging at the default level and once with logging of common published data enabled. 網域程式碼的完成時間應該相同,The domain code should take the same amount of time to complete. 但在啟用一般已發佈資料記錄的情況下,平台程式碼的執行時間將會延長。However, the platform code will take longer to run with common published data logging enabled. 基本上,啟用一般已發佈資料時,平台程式碼必須支援比預設記錄層級還多出 12 倍的資料。Essentially, the platform code has to support logging 12 times more data with common published data enabled than it did when running at the default logging level.

「比較值」 **** 標準活動可用來建立 Orchestrator 環境的效能評定基準。The Standard Activity Compare Values can been used to create benchmarks of an Orchestrator environment.

建立可用來評定 Orchestrator 環境效能的 RunbookTo create a runbook that can be used to benchmark your Orchestrator environment

  1. 建立新的 Runbook。Create a new runbook.
  2. 從 [標準活動] 調色盤新增 Compare Values 活動。Add a Compare Values activity from the Standard Activity palette. 按兩下 - 活動以進行設定。Double-click the activity to configure it.
  3. 按一下 [ 一般 ] 索引標籤,並將此活動設定為比較字串 ( 的預設值 ) 。Click the General tab and configure this activity to compare strings (the default value).
  4. 按一下 [詳細資料] **** 索引標籤,在 [測試] **** 方塊中輸入值 STRING ,並且選取 [是空的] ****。Click the Details tab, type the value STRING in the Test box and select is empty.
  5. 按一下 [完成] **** ,儲存活動的更新。Click Finish to save the updates to the activity.
  6. 在活動上按一下滑鼠右鍵並選取 [迴圈] ****。Right-click the activity and select Looping.
  7. 選取 [啟用] 核取方塊,並0在 ( ) 嘗試之間的延遲輸入數位0零。Select the Enable checkbox and enter the number 0 (zero) for Delay between attempts.
  8. 按一下 [結束] **** 索引標籤。Click the Exit tab.
  9. 變更預設的結束條件。Change the default exit condition. 按一下 [ 比較值]、勾選 [ 顯示一般已發佈資料 ] 核取方塊,然後選取 [ 迴圈:嘗試次數]。Click Compare Values, check the Show Common Published Data checkbox, and select Loop: Number of attempts. 按一下 [確定] **** 儲存此變更。Click OK to save this change.
  10. 從更新結束條件選取 [ ],然後輸入 10000 ( 萬分之一的數位 - ) 。Select value from the updated exit condition and type the number 10000 (ten-thousand). 按一下 [確定] **** 儲存此變更。Click OK to save this change.
  11. 按一下 [完成] **** 儲存上述更新。Click Finish to save these updates.
  12. 按一下 [存回] **** 將變更儲存到 Orchestrator 資料庫。Click Check In to save the changes to the Orchestrator database.

這個簡單 - 的單一活動 runbook 將會執行「 比較值 」活動10000次。This simple one-activity runbook will run a Compare Values activity 10,000 times. 「比較值」**** 是網域程式碼極少的簡單活動。Compare Values is a very simple activity whose domain code is quite minimal. 您可以在各種情況下叫用此 Runbook,以便歸納指定之 Orchestrator 執行階段環境的整體效能。This runbook can be invoked under a variety of circumstances to characterize the overall performance of a given Orchestrator runtime environment.

此 Runbook 可用來實驗不同的 Orchestrator 設定。This runbook can be used to experiment with different configurations of Orchestrator. 例如,假設您要判斷四部部署在不同資料中心內的 Runbook 伺服器效能。For example, supposed you wanted to determine the performance of four Runbook Servers deployed to different data centers.

資料中心Data Center 記錄設定Logging Configuration 平臺程式碼執行時間 ( 秒數)Platform Code Run Time (seconds) ms / 活動ms/Activity 擴充係數Scale Factor
位置 1Location 1 預設記錄Default logging 819819 8282 1.0 ( 參考)1.0 (reference)
位置 1Location 1 記錄一般已發佈資料Logging common published data 20122012 201201 2.52.5
位置 2Location 2 預設記錄Default logging 12291229 123123 1.51.5
位置 2Location 2 記錄一般已發佈資料Logging common published data 36863686 369369 4.54.5
位置 3Location 3 預設記錄Default logging 24572457 426426 3.03.0
位置 3Location 3 記錄一般已發佈資料Logging common published data 44224422 442442 5.45.4
位置 4Location 4 預設記錄Default logging 14741474 147147 1.81.8
位置 4Location 4 記錄一般已發佈資料Logging common published data 26542654 265265 3.23.2

請注意由記錄一般已發佈資料所造成平台效能大幅降低的現象。Notice the significant decrease in platform performance caused by logging of common published data. 最差的情況發生在位置 2 的一般已發佈資料記錄作業。The worst scenario appears to be logging of common published data at Location 2. 在表面上,這個現象似乎是明確而相關的結論。On the surface, this appears to be a clear and relevant conclusion.

不過請注意,這些圖表反映的是平台程式碼的負荷,不是網域程式碼。However, it should be noted that these figures reflect the overhead of the platform code, not the domain code. 網域程式碼的執行階段可能會大幅延長。Domain code runtimes can be significantly longer. 例如,Virtual Machine Manager 整合套件中的「從範本建立 VM」 **** 活動可能會在建立 VM 時執行數分鐘之久。For example, the Create VM from Template activity in the Virtual Machine Manager Integration Pack may run for several minutes as the VM is created. 展開先前的範例,請考慮 runbook 活動的平臺程式碼成本,無論位置為何,都需要1分鐘的時間來執行 ( 1 分鐘的 = 60000 毫秒 ) 。Expanding on the previous example, consider the platform code costs on a runbook activity that takes 1 minute to run (1 minute = 60,000 milliseconds) regardless of location.

資料中心Data Center 記錄設定Logging Configuration 平臺程式碼執行時間 ( 秒數)Platform Code Run Time (seconds) 網域程式碼 %% Domain Code 平台程式碼 %% Platform Code
位置 1Location 1 預設記錄Default logging 819819 98.6%98.6% 1.4%1.4%
位置 1Location 1 記錄一般已發佈資料Logging common published data 20122012 96.7%96.7% 3.3%3.3%
位置 2Location 2 預設記錄Default logging 12291229 98.0%98.0% 2.0%2.0%
位置 2Location 2 記錄一般已發佈資料Logging common published data 36863686 93.9%93.9% 6.1%6.1%
位置 3Location 3 預設記錄Default logging 24572457 95.9%95.9% 4.1%4.1%
位置 3Location 3 記錄一般已發佈資料Logging common published data 44224422 92.6%92.6% 7.4%7.4%
位置 4Location 4 預設記錄Default logging 14741474 97.5%97.5% 2.5%2.5%
位置 4Location 4 記錄一般已發佈資料Logging common published data 26542654 95.6%95.6% 4.4%4.4%

這些資料所代表的意義逐漸成形。A clearer picture begins to emerge from the data. 在位置 2 啟用一般已發佈資料記錄的案例蟬聯效能表現最差的案例。The scenario where logging of common published data is enabled at Location 2 continues to be the worst performer. 不過,平台程式碼和記錄作業僅佔總執行時間的 6%。However, the platform code and logging only accounts for 6% of the total runtime. 雖然這是很重要的圖表,但最佳 - 情況是1.4%。While this is a significant figure, the best-case scenario is 1.4%. 基本上,此範例花在網域程式碼的時間比花在執行平台程式碼的時間多出許多。Essentially, the time spent in the domain code in the example far outweighs the time spent running platform code. 正確來說,假設您可以完全排除平台程式碼成本,產生的 Runbook 效能提升也僅限於 1.4 到 7.4% 的範圍之間。To put this in perspective, if you were able to completely eliminate the platform code costs, you would only see runbook performance improvements in the range of 1.4 to 7.4%.

當然,大部分的真實 - 世界案例都是不同的。Of course most real-world scenarios will be different. 活動行為可能會因網域程式碼接獲的作業指示不同而改變。Activity behavior may change depending on what the domain code is told to do. 例如,[從範本複製 VM] **** 活動可能會花一分鐘從伺服器範本 A 複製 VM,但花 5 分鐘從伺服器範本 B 複製 VM。此外,Runbook 伺服器可能位於不同的網路,而且網路的效能特性也不相同,從而影響網域程式碼效能及 Orchestrator 資料記錄效能。For example, a Clone VM from Template activity may take one minute to clone a VM from Server Template A, but take 5 minutes to clone a VM from Server Template B. Also, Runbook Servers may reside on different networks with different performance characteristics which can potentially impact both domain code performance as well as Orchestrator data logging performance.

總括來說:To summarize:

  • 請謹慎拿捏已發佈資料的記錄時機。Make careful decisions about when to log published data.
  • 請仔細評估記錄一般已發佈資料所造成的影響。Carefully consider the impact of logging common published data. 請切記記錄的資料數量取決於活動執行的次數。Remember that the number of times activities run determines the volume of logged data. 活動數目較少但執行次數較多的 Runbook,其資料記錄數量可能比執行次數較少但較大型的 Runbook 還多。A runbook with a small number of activities run many times can result in more data logging than a larger runbook run a small number of times.
  • 請勿在生產環境中啟用活動特定已發佈資料的記錄功能。Do not enable logging of activity specific published data in production environments.
  • 請設法瞭解 Runbook 在執行網域程式碼和平台程式碼時所需時間的差異。Develop an understanding of how much time your runbooks spend running domain code compared to running platform code.
  • 請使用本文件概述的技術來預估平台程式碼成本。Estimate platform code costs using the techniques outlined in this document. 請參考這些技術,考量您應從何處著手改善 Runbook 的效能。Use as a reference in considering where to make improvements in runbook performance.
  • 請使用本文件概述的技術來深入瞭解不同執行階段環境的相對效能。Use the techniques outlined in this document to gain a deeper understanding of the relative performance of your different runtime environments. 請採用標準的測量比較方式,藉此找出改進的機會。Identify opportunities for improvement by making normalized comparisons of your measurements.

後續步驟Next steps

取得在 建立和測試範例 runbook時建立 runbook 的逐步指示。Get step by step instructions for creating runbooks at Creating and testing a sample runbook.