Staging folders and Conflict and Deleted folders
Applies To: Windows Server 2003 R2
Staging folders
DFS Replication uses staging folders to act as caches for new and changed files to be replicated from sending members to receiving members. The sending member begins staging a file when it receives a request from the receiving member. The process involves reading the file from the replicated folder and building a compressed representation of the file in the staging folder. This is the staged file. After being constructed, the staged file is sent to the receiving member; if remote differential compression [RDC] is used, only a fraction of the staging file might be replicated. The receiving member downloads the data and builds the file in its staging folder. After the file has completed downloading on the receiving member, DFS Replication decompresses the file and installs it into the replicated folder.
Each replicated folder has its own staging folder, which by default is located under the local path of the replicated folder in the DfsrPrivate\Staging folder. The default size of each staging folder is 4,096 MB. This is not a hard limit, however. It is only a quota that is used to govern cleanup and excessive usage based on high and low watermarks (90 percent and 60 percent of staging folder size, respectively). For example, when the staging folder reaches 90 percent of the configured quota the oldest staged files are purged until the staging folder reaches 60 percent of the configured quota.
It is important to note that the staging folder quota does not determine the largest file that can be replicated. In other words, it is possible to replicate a file that is larger than the configured quota of a staging folder. The large file is placed in the staging folder, and the staging folder cleanup process is triggered when the file is finished staging and space usage is at or above the high watermark. If the cleanup fails because the large file is still in the process of being replicated to receiving members, the cleanup process will be retried later and eventually the large file will be purged from the staging folder.
Although you can adjust the size of each staging folder, you must take the following factors into account while doing so:
For good operational performance, increasing the quota size of a staging folder is recommended when you have multiple large files that change frequently. We also recommend that you increase the staging folder quota on hub members that have many replication partners.
The size of each staging folder on a member is cumulative per volume. For example, if you have three replicated folders on a member on the same volume, it is possible for DFS Replication to use 12 GB or more for staging purposes. However, staging space is not preallocated; disk space is only used when staged files are present.
Note
In this example, staging cleanup might not be triggered when the staging files occupy 10.8 GB (0.94 GB3), because staging space is distributed across the three replicated folders. If free disk space is a concern, you might need to configure the staging quota to be lower than the default quota when several replicated folders share staging space on the same volume. This ensures that staging cleanup is triggered.
If the size of a staging folder is below 90 percent of configured capacity (the high watermark), then staged files are kept in the folder and can be used in case new members are added.
Staged files are purged (based on a "least recently used" algorithm) when a staging folder reaches 90 percent of the configured quota (the high watermark). Files are purged until the staging size falls below the low watermark (default is 60 percent) of the configured quota.
If a staging folder quota is configured to be too small, DFS Replication might consume additional CPU and disk resources to regenerate the staged files. Replication might also slow down because the lack of staging space can effectively limit the number of concurrent transfers with partners.
For the initial replication of existing data on the primary member, it is important that you size the staging folder quota large enough so that if multiple large files are blocked in staging due to partners not being able to download the files, the remaining files can continue replicating. To properly size the staging folder for initial replication, you must take into account the size of the files to be replicated. At a minimum, the staging folder quota for servers running Windows Server 2003 R2 should be at least the size of the nine largest files in the replicated folder. To improve performance, set the size of the staging folder quota as close as possible to the size of the replicated folder.
Tip
To determine the size of the largest files in a replicated folder, you can either use Windows Explorer (sort by size and add the nine largest file sizes to get the minimum staging folder size), or, from a Windows PowerShell™ command prompt, use this Windows PowerShell command where <replicatedfolderpath> is the path to the replicated folder:
Get-ChildItem<replicatedfolderpath>-recurse | Sort-Object length -descending | select-object -first 9 | ft name,length -wrap –autoTo install Windows PowerShell on a server running Windows Server 2003, see Windows PowerShell 1.0 English-Language Installation Package for Windows Server 2003 (KB926139) in the Microsoft Download Center (https://go.microsoft.com/fwlink/?LinkId=184089).During normal operation, if the event that indicates the staging quota is over its configured size (event ID 4208 in the DFS Replication event log) is logged multiple times in a day, increase the staging quota by 20 percent.
Conflict and Deleted folders
DFS Replication uses a "last-writer wins" method for determining which version of a file to keep when a file is modified on two or more members. The losing file is stored in the Conflict and Deleted folder on the member that resolves the conflict. This member might not be the member where the changes originated.
Each replicated folder has its own Conflict and Deleted folder, which is located under the local path of the replicated folder in the DfsrPrivate\ConflictandDeleted folder. The quota size of the Conflict and Deleted folder is 660 MB by default. Like the staging folder, the Conflict and Deleted folder has high and low watermarks (90 percent and 60 percent of Conflict and Deleted folder quota, respectively) that govern cleanup and excessive usage of the folder. And like the staging folder, the size of each Conflict and Deleted folder on a member is cumulative per volume.
The files in the Conflict and Deleted folder are renamed and are accessible only by members of the local Administrators group. The access control lists (ACLs) on the conflict files are preserved to help ensure security. You can view a log of conflict files, as well as their original file names, by viewing the ConflictandDeletedManifest.xml file in the DfsrPrivate folder.
The Conflict and Deleted folder can also be used to store files that are deleted from replicated folders. You can configure this option by viewing the properties of a membership. Deleted files are treated like conflict files in that they are purged when the Conflict and Deleted folder reaches 90 percent of the configured quota. They are also renamed and listed in the ConflictandDeletedManifest.xml file.