Debugging Content Deployment issues
(Last updated June 17th with the Object Reference Not Set to an Instance)
SharePoint's Content Deployment feature is a very complex functionality that, unfortunately, does not work all the time. I've had scenarios where it ran very smoothly and I've had other scenarios (albeit similar in terms of architecture and design) that simply seems to have a different issue everyday.
What I'll attempt here is to include a few ways of debugging Content Deployment as well as links to fixes (or at least, known issues).
Is the error reproducible?
First of all, is the error reproducible. If it's not, then it'll make it harder to determine the issue but I've had cases where we could still work it out. If it is reproducible, I'd suggest 2 things:
- Use STSADM -o export [...] and STSADM -o import [...] These commands are essentially a Full Content Deployment but it's also a little bit more verbose and you may be able to get the exact error and item.
- Open the exported CAB file and look at the item that's failing. For example, if you successfully 6872 objects, look at the 6873e object in the CAB file. (note that this also works if the error is not reproducible)
Look at the ULS logs
The Unified Logging System (ULS) logs may contain more information regarding your issue. Look at the logs at the same "time" and see if you have more information depending on the type of error and how verbose you are logging.
Look at the Windows Event Log
For some reason, since we have SharePoint logs, we tend to go less often in the Windows Event Logs. It's actually surprising how much more relevant the information can be there and it's also easier to search on an event ID than a string, especially if you have a localized error message.
Unfortunately, a default installation of SharePoint may also bring a lot of errors in the event log (that could be .NET errors, DCOM errors, etc.) so I would suggest taking a look at it before going to production and apply any other fixes that would help.
Run SPSReport, especially if you plan on opening a support call
You can download it here : http://www.codeplex.com/spsreport.
Try between Incremental & Full Content Deployments
Okay, that's not a funny one but I've had cases where I had a reproducible error during incremental and we couldn't figure out why. We ran a Full Content Deployment and the error disappeared for a couple weeks. I had the opposite cases too though :)
While this may seem trivial, I often forget to look at the CAB file myself so I guess it's possible that not everyone knows about it; same with the Event Log. Hopefully, they will give you enough information to find out which item is breaking your deployment or if a fix is available.
Read the exported Manifest.xml file
The manifest file contains all deployed objects and may help determining which of them broke. However, you need it to break during Import because the manifest file is only completed at the end of the Export phase. Plus, the Incremental jobs almost always delete the file when the content deployment fails (and succeeds). You can read about it here.
List of issues I've worked with (with or without fixes, I will update with comments & new issues/fixes)
The given key was not present in the dictionary. at System.ThrowHelper.ThrowKeyNotFoundException() at System.Collections.Generic.Dictionary`2.get_Item
Save Conflict Your changes conflict with those made concurrently by another user
Solution: None yet, often case, a following Content Deployment may work. I have to check at the CAB file for this one but I didn't get the chance
The specified file is already used
Solution: Delete the file and create it under another name; or delete it, deploy, and recreate it. Also, the new post-SP1 WSS roll-up may fix this as it seems to fix it for lists (hopefully list items as well) : KB941422.
Transaction logs are growing fast, especially when using Full Content Deployment
Solution: While this seems to be by design, you can limit the transaction log file by using a SQL "Simple" recovery mode. However, it also means that you can only recover your database with a backup (not with a backup + log backup). Also, Database Mirroring requires a Full recovery mode.
Object reference not set to an instance during export
Solution: The .NET Framework 2.0 SP1 fixes an instance of that issue (related to memory issues). However, there's another case when you have a Quick Deploy job running while you flag a page to be quick deployed. You can read it here.
I noticed "Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt." messages in the ULS logs and Event Logs. See KB923028.
Full Content Deployment may be slower after installing SP1
Solution: None yet, not a "known issue"
I documented this one here : http://blogs.msdn.com/maximeb/archive/2008/02/13/make-sure-you-test-your-content-deployment-after-installing-sp1.aspx. I basically ran some performance tests (after noticing a performance degradation) after installing SP1 and got some concerns. Update: Note that in some cases, Anti-Virus software may be the cause to this as well (pre and post SP1).
HTTP (400) Bad request, Invalid Hostname or "the file type is blocked" ... error message
Solution: Configure your SharePoint Alternate Access Mappings (AAM) for your site so that the default zone for the Central Administration isn't load balanced; or configure IIS so that it responds when the server name and port number are used instead of the load balanced URL.
The URL "Style Library/en-us" is invalid
Solution: Delete your site collection at the destination, create a Publishing Site at the destination, delete it, and now create a Blank Site.
I documented this one here : http://blogs.msdn.com/maximeb/archive/2007/10/10/content-deployment-the-url-style-library-en-us-is-invalid.aspx. Update: In fact, doing this also creates the correct Timer Jobs for other operations such as Variations to work so it's a requirement.
The TCP connection was forcibly closed
Solution: Run the "Netsh int ip set chimney ENABLED" command; KB912222
Links are updated at the destination and are still pointing on your authoring farm
Solution: None officially nor planned so far. I ended up creating a simple "StringReplacement" HttpModule for a customer.
You cannot perform this action on a checked out document
Solution: Someone probably checked out a document at the destination. Unfortunately, the logs do not tell you which document is checked out. I ended up creating an STSADM extension that would list all checked out documents by anyone. If anyone knows a CAML query to do this, it would also be an easy way to do it in the Content Structure Reports.
Master pages are not honoured
Solution: Install SP1 or create an STSADM extension to update it. However, note that incremental content deployment will not "erase" your update so you can update it manually at the destination after the initial content deployment.
Violation of Primary Key
Solution: Install SP1.
Special characters are not deployed correctly
Solution: Install SP1.
Update : Error 4958: The stream does not support concurrent IO read or write operations
Solution: Set the FileMaxSize property to 2MB in the Content Deployment Configuration. Thanks to Jian Gao for this one.
Update : Violation of Primary Key on #ExportObjects
Solution: This one occurs sometimes when you modified some files in the Style Library and an exception is thrown early in the Export stage. If you have this, go in the Style Library of your source web site, click on List settings and then Versioning settings and update the versioning so that it only accepts Major versions (NOT major & minor). Execute the Content Deployment and it execute fine after.