MAT – More is better. Multi-server conversions

  • Article

train3

When I introduced the MAT last post, I didn’t spend much time discussing how to convert using multiple servers. So I better get to it. This post details how to configure the MAT to use multiple machines to convert at the same time (let’s call these “Helpers”). Using the MAT on many servers is pretty simple but there are some things you want to take into consideration.

Prerequisites:

  • Install MVMC on your Helper(s)
  • Verify your credentials in the <SchedCreds> section in Variables.xml
  • Add one <ConvertServer>HELPERx</ConvertServer> to the <RemoteHost> section of the Variable.xml file for each Helper you wish to use.
  • Copy the MAT files to your Helper(s) in C:\MAT (assuming you haven’t edited that variable in main.ps1)
  • Verify your SQL server is configured to accept remote connections
  • Log into each Helper using the account assigned in <SchedCreds>
  • You do NOT need to install PowerCLI on Helpers, only on the control server that runs the collection.

How it works

When you start a set of conversions from the .\main.ps1 menu (and then select Convert) or if you launch the script with the Convert-Global switch the process will read the Variable.xml file and load all the <ConvertServer> values you have added.

It will then iterate through a function Start-Remotes that will do the following for each Helper machine in the list:

  1. Connect-TaskService - This function will connect to the Helper server’s Task Scheduler service. It will also invoke the Remove-Task function to delete any previous versions of the task.
  2. New-Task – The New-Task function will create a task (the default is MAT based on $RemoteJobName = "MAT" in the main.ps1 variables section) on the remote server that will call main.ps1 with the Convert parameter. If you recall from previous posts, this option will run a local conversion (bypassing the logic to start Helpers)
  3. Start-Task – this function triggers the remote task to start immediately (as opposed to scheduling it)
  4. The process will then wait 15 seconds to give the current Helper enough time to start up and grab it’s initial VM from the queue.

Why do I need to be logged in?

The MVMC.exe gets kinda angry if you try to launch it using Task Scheduler and it will fail. This is because it expects to write its temp files (during the VM file copy) and its log output to the user’s temp folder. If the user isn’t logged in, MVMC will fail without any real explanation. The process can’t write a log, so you can’t see what it might have reported and its running via Task Scheduler so you can’t see the DOS box either. Not good.

You do not need to have an active connection to the Helper, a disconnected session you started over RDP will work just fine. So RDP in and disconnect before you start your conversion. It’s not a perfect approach but it works.

Shared Logs

Why have five MAT.log files on five Helpers, you ask. Good news! You can change this if you don’t’ like my approach. (Isn’t PowerShell grand?)

In the General section of variable.xml, edit the value for <Variable Name="Logpath"> and point it to a shared location and you’ll end up with one big MAT.log file.

WARNING: I will caution you, if you have a large number of conversions running, the log can be a challenge to read (it’s usually enough fun on one system) and there is a chance you will have file contention issues when the Helpers write to the log file. So for these reasons I prefer leaving the MAT.log on each Helper.

Sharing XML

Some folks will want to use the same central XML file rather than copying the XML file to every machine and then securing them all (it does have some sensitive data in it after all). I get that.

If you want to use this approach, create a share in your C:\MAT folder (or wherever you use it) and edit main.ps1 find $XMLPath = "C:\MAT\Variable.xml" (line 21 by default) and update it.

 

Spreading the love, spreading the load

So what’s the downside of using a common XML file? Simply put everything ends up on the same Hyper-V host and you might overload it.  So you may want to customize how each Helper operates. For example, “Helper1” points to “Hyper-VHost1”, “Helper2” points to “Hyper-VHost2” and so on. Doing this limits the total number of VMs being written to each Hyper-V host and will improve your overall performance. Your Hyper-V hosts should be able to easily accommodate the maximum of three VMs per Helper. Following this approach you should be able to scale out Helpers limited only by your Hyper-V hosts that will accept the VMs.

Once the VMs are on Hyper-V you can move them very easily to a better location using Hyper-V Manager, or better yet using VMM.

Next time I’ll post about sizing and figuring out how many Helpers do I need…

Happy Converting!