An adventure in the sccmcontentlib - single instance store
Sometimes I have nightmares which are work related. One such nightmare was about Configuration Manager 2012 and it prompted me to write this blog post. In my dream, the telephone rang and it was my Configuration Manager 2012 Distribution point. It kindly told me that one of my packages disappeared for no reason. Geez I need a vacation.
With SCCM 2007, content is stored on a distribution point in package ID folders. If you knew the package ID, you could browse to the file share and install/run manually.
This option is more complicated now in Configuration Manager 2012. The files are not just stored on a drive in a manner you could just copy and execute.
The following blog post is about the new Content Libraryand my attempts to reconstitute a package from the Single Instance Store.
By default, the content is single storage and browsing to a DP does not store all the files in a package id folder where you could find an install and execute it. You can enable the older method which will create a separate folder share. This will require double the disk space! Obviously, this is a pain point for those who use the distribution points this way as a mitigation step in a crisis.
To understand how the Content Library works in Configuration Manager 2012, I decided to find a way to solve this pain point.
This functional diagram is my review of the single content library:
In the Content Layer, individual files are sent and only stored once in a single storage location. For example, there might be three packages that need to be distributed in a Configuration Manager 2012 infrastructure. All three packages contain the same file Psexec.exe with the same md5\sha1\sha2 hash value.
Why send the file and store it in three places?
Great efficiencies are gained with the new shared single storage Content Library in Configuration Manager 2012. The file is hashed, given an ID and sent only once. Each package is stored as pointers to the necessary files. This means that packages share files if they are the same. This saves on network usage and file storage requirements. Sounds awesome; doesn’t it!
Configuration Manager Layer
Here is an example:
I created two packages:
- 1. Filezilla – package id HH00005
- 2. Notepad ++ - Package id HH00006
Each has psexec.exe, one of my favorite sysinternal tools. Note you can get it here at http://live.sysinternals.com
Next I “Distribute Content” in the CM2012 console to my distribution point.
Distribution Point Layer
When content is sent to the distribution point, it is stored by default on the largest available drive in a folder, sccmcontentlib. During the install you can set this location, however, you cannot configure the drive settings in the distribution point properties after installation completes.
The following folders get created:
- · Datalib = Data Library
- · Filelib = File Library
- · Pkglib = Package Library
For each package that you distribute a corresponding package+id.version number folder and file will be created. For patches the folder will appear as the patch guid and application folders will appear as Content_guid
Now let’s look at the package HH00005.1 (Filezilla)
Each DataLib folder contains .ini file which includes important information for the actual file.
- · Attributes
- · Size
- · TimeModified
- · Hash
Viewing the psexec.ini file in the HH00005.1 package folder:
A Hash value of F8DB… is found. The first four entries of this hash are important. This is the locater folder name for the actual binary file in the pkgLib folder.
The default hashing algorithm for content is SHA-256
To find the file:
The hash value .ini contains a list of the type and references of which packages use this file. Notice that there are two packages that are listed. This is single instance storage. Any package/program/content/update that has this exact hash file requirement will use this file and location.
The actual file will be the hash with no extension. Copying and renaming the file to psexec.exe, I verified that is the psexec file:
SHA-256 hash value matches.
Now back to the HH00005 package. It contains the setup for FileZilla and psexec.
Note that the psexec.exe.ini file points to the F8DB filelib folder as I demonstrated above.
To summarize the Content Library, Single Content Storage and this review:
Both HH00005 and HH00006 CM2012 packages contain the file psexec.exe. The file is only sent once to the Content Library and stored as single instance. A SHA-256 hash value is used to track the unique file and the first four digits are used as a locator. When a client requests the download the files are assembled from the Content Library via single instance storage.
Now suppose you want to reconstitute an entire package from the sccmcontentlib folder. I created a basic vb.net application to assist me in this exercise.
Following each PkgLib file to the DataLib, I was able to regenerate all the files from the FileLib for any given package.
I hope my blog posting was helpful in providing an understanding the Content Library and the Single Instance Storage in Configuration Manager 2012.