SPSiteManager 2.0 Perftest and Update

  • Article

A couple more changes to SPSiteManager 2.0 before I spread the icing on top (Uh, I mean finish the documentation :))

 

When I distributed the last package of the SharePoint Utility Suite It was based on an argument processor I wrote called XAP (That wasn't it's first name :), but I was limited with options (See my post on Bluedog about this). Anyways, it was pretty cool, and allowed me to handle argument processing in a consistent way across my tools. But, I found one fallacy in this. They way I first implemented it, required you to ensure you had the .config file in the same location where you're putting the binary. I hated that. Oh well, I promised myself that I would get back to that, and eventually include the argument syntax as a resource into the binary.

Well, I finally got a round to taking care of that :) So, in the next release of the utility suite, you'll see that the additional config file is gone! 

Some recent changes SPSiteManager 2.0

  • .config file dependency removed.
  • An additional level of analysis that does a token search on aspx pages (This was added due to a recent issue I worked on in which we needed to actually examine the contents of pages to look for specific tags (In this case, register directives). I wasn’t going to leave it in the official version, but after a bit of thinking, I think this could be useful in other scenarios. 
  • Some tweaks to improve performance and reduce memory stress.
  • Before a site is repartitioned I check to see if it is a portal site or MySite.
    • If it's a portal site, it doesn't move it (This is important, because portal sites cannot be backed up/deleted/restored, etc using STS)
    • If it's a MySite, it checks to ensure that the target URL is the same as the source. If it isn't it doesn't move it. See https://support.microsoft.com/?kbid=889236

I'm now completing the documentation (Up to 60 pages now!) and testing, testing, testing :)

Some interesting performance metrics

I recently had the opportunity to run it on an installation as described below

The following operation was run on the farm:

SPSiteManager -o analyze -analysislevel 6 -tokenfile tokens.txt -log -allvs –verbose

The farm had: 

  • 4 Virtual Servers (a couple were portal instances)
  • A total of nearly 1500 site collections
  • Nearly 2000 webs
  • Approximately 12 Gig of data

The SPSiteManager logs showed the following results:

Beginning SPSiteManager

Operation=analyze

Time:3/21/2006 5:15:52 PM

Completed SPSiteManager

Operation=analyze

Time:3/21/2006 5:32:16 PM

It only took approximately 16 minutes to run!! Pretty darn good, but once I decide to implement the multithreading I want to do, it could only decrease the time.

During the course of the analysis, once it kicked into high gear.

  • Private Bytes was 288,694,272 and maxed to about 500 MB
  • At the end when it generated the SDD, it jumped up to about 650 MB.

Since SPSiteManager currently does not deal with Portal Areas and utilize the Portal Area Hierarchy objects it obviously complained and bawked a good deal when it hit the portal bucket webs. I think some additional exception handling needs to be added into this, to ensure that some of the SPSite and SPWeb objects are properly disposed in this situation, which I think will account for some of the increase, but obviously your going to see an gradual increase in private bytes as we’re caching all of this data in memory

Of course you’re not really going to want to run a deep analysis across all your virtual servers in a very large farm anyways, and the documentation will be noted so, but I was rather impressed with the current builds ability to handle crawling that much data and never really getting over 650 MB of private bytes.

In a really large farm scenario, you would run a level 1 analysis across the entire farm, then run deeper analysis on specific virtual servers, or down to the granularity of a single site on a virtual server.

Overall, a really interesting perf test!

I’ll be generating tons of sites and webs using SPSiteBuilder and letting it run over the weekend. And let you know the results once those are in.

There will definitely be prescriptive guidance for determining which scope of analysis you should run, as this puppy will definitely chew up some memory when run with a deep analysis across all virtual servers J

- Keith