How do WIndows Shadow Copies store much more information than the size of the Shadow Copy itself?

Question

Using WizTree, I have been indexing the contents of my external hard drives and noticed that a few of these drive have what look like GUID-tagged Shadow Copies stored in the System Volume Information folder around 30-50 GB.

When using Shadow Explorer to open one of these Shadow Copies on a 5 TB HDD, however, I was able to both see and extract the entire contents of the shadow copy, which was essentially a snapshot of the external drive from 2 years ago that was around 3 TB in size. That's approximately 100 times the size of the Shadow Copy file and also over half the size of the drive. I had previously deleted files and re-written to the external drive multiple times, but only about 10 files failed to export.

I would like to ask how this is possible?

1. If it only requires around 30-50 GB to "store" around 3 TB of data, where is the bulk of this backup actually located? It wouldn't make sense if the Shadow Copy is just referencing the original location of the file on the physical drive because if you permanently delete and overwrite that file, you wouldn't be able to recover it anyway.
2. Given that a Shadow Copy "stores" an entire snapshot of the drive, if it more than half full then it is theoretically not possible to shadow copy the entire drive unless there is some serious lossless compressions happening.
3. It just doesn't make sense that a 3 TB Shadow Copy of data was able to be made and kept at all on a 5 TB drive, especially when the drive was later filled to over 4 TB over the past 2 years and yesterday I was still able to extract the entire 3 TB of Shadow Copy data off.

Thank you if anyone is able to shed some light on how Shadow Copies actually work. It seems a little like magic and data is being stored in the ether.

Answer 1

Hello Ruohan,

Thank you for your question and for reaching out with your question today.

Shadow Copies (also known as Volume Shadow Copies or Previous Versions) are a feature in Windows that allows users to restore previous versions of files or folders. They work by creating snapshots of the volume at different points in time. These snapshots are stored in the "System Volume Information" folder on the volume.

Here's how Shadow Copies work and why they seem to take up less space than the actual data:

1. Differential Storage: Shadow Copies use a technology called "differential storage." Instead of storing complete copies of each file in every snapshot, they store only the differences (or changes) between the current version and the previous version. This way, the amount of space required for each snapshot is significantly reduced.
2. Sparse Copies: Shadow Copies use sparse copies for files. A sparse copy is a type of file that contains large sections of zeros. When a file is deleted or overwritten, the sparse copy only needs to store the changed data, not the entire file, resulting in significant space savings.
3. Copy-on-Write: When a file is modified or deleted after a snapshot is taken, the original data is not immediately removed. Instead, a mechanism called "copy-on-write" is used. The original data remains in the snapshot until it's needed to store a new version of the file. This helps maintain the integrity of the snapshots and allows for more efficient use of space.
4. Snapshot Linking: When multiple snapshots are taken, they can share the same storage for unchanged data. This is achieved through snapshot linking. As a result, common data between different snapshots is not duplicated, leading to further space savings.
5. Retention Policy: Shadow Copies have a retention policy that determines how many snapshots are kept and for how long. Older snapshots may be automatically deleted as new ones are created, depending on the settings.

Overall, Shadow Copies are designed to efficiently store previous versions of files while minimizing the storage space required. The technology used, such as differential storage, sparse copies, and copy-on-write, enables effective space management and allows for the creation of multiple snapshots without requiring a full copy of the data each time. This is why a Shadow Copy can seemingly take up less space than the actual data being referenced.

I used AI provided by ChatGPT to formulate part of this response. I have verified that the information is accurate before sharing it with you.

If the reply was helpful, please don’t forget to upvote or accept as answer.