Making MSMQ's persistent storage ... persistent

I had a customer the other day find that their performance with MSMQ was pretty poor until they defragged the disk. Not only were the storage files fragmented so reading and writing recoverable messages was slow but also creating new storage files on demand took time as free space was not contiguous. This got me thinking about ways to avoid the problem.

Obviously you could perform defragmentation on a regular basis but only during quiet periods as otherewise you would impact disk I/O - which is what you are trying to avoid doing in the first place.

So how about setting up MSMQ with pre-generated storage files on a freshly defragmented drive and not deleting them when they are empty?

There's two parts here:

  • Preventing cleanup of empty storage files
  • Generating the storage files

The first part is easy but with a "gotcha". You can set MessageCleanupInterval to 0xFFFFFFFF, or 4294967295 milliseconds which I think is 7 weeks. (Must check if this can be set to zero to disable it completely...). So, instead of every 6 hours (the default), the MSMQ service won't delete empty storage files until 7 weeks is up. Unless at some point you restart the MSMQ service when it will just go ahead and clean up regardless as it loads the storage files into memory.

So once you have set the registry value (and restarted MSMQ so the change is picked up), use something like MQBench to fire off enough large recoverable messages to a local test queue to create the volume of storage files you want. Don't go crazy with this - Windows 2000 isn't going to appreciate more than about 1.6GB of files - so just create what you would need for normal load. Remember these are memory-mapped files so you are pre-allocating disk and virtual memory space. Once you have enough messages, purge the queue. This should leave all the files in place and, as you defregmented the drive beforehand, disk I/O to read or write messages should be optimised.

Make sure you test this first - permanently pre-allocating memory space for storage may have a major impact on something else on your system.
And remember that you will need to repeat the process any time you restart MSMQ.

Reference articles

199312 You must restart the Message Queuing service to clean up message files
174307 Interpreting file names in the Storage directory in Microsoft Message Queue Server and in Microsoft Message Queuing
186194 MQBench.exe Measures Time to Deliver MSMQ Messages