How hard links work

I was asked in a prior comment to give greater detail on how hard linking works in the file system and how it applies to the component store and the way files are serviced. I’ve enlisted the assistance of my co-worker and uber disk expert Robert Mitchell to help describe this. I hope you all enjoy the material.

Hardlinks

 

First I would like to thank Joseph Conway for asking me to contribute to his blog.

The following information has been compiled to help you to better understand how hard links work in the context of using them with the component store. Hard links have been around for a while but it wasn’t until we started using them in the component store that they got much attention. They are used to allow us to have a file that exists in multiple directories without taking up extra space on the hard drive.

Before you start learning about hard links there are some basic file system concepts you must understand first.

· Master File Table

· File Record Segments

· File attributes

The Master File Table or MFT is a collection of 1k entries called File Record Segments or simply file records. A file record contains the metadata for a single file. Each file record has a number assigned to it that is used by NTFS to refer to the file.

For example, the first file created is called $MFT and is always file 0h. The root directory is file 5h. Special NTFS files, called metafiles, always use the same file record number. For a list of most of these metafiles and what they are used for please refer to http://blogs.technet.com/b/askcore/archive/2009/12/30/ntfs-metafiles.aspx.

File attributes are the building blocks of a file. Do not get these confused with attributes like READ-ONLY. Here is a simplified diagram of how a file record segment looks.

clip_image001

It starts out with a header. This has very basic information about the file, including something called a link count, which we will discuss later.

Each attribute has its own header that defines things about the attribute, like at what byte it starts at and its size.

The attribute itself provides metadata about the file. Common attribute types are $Standard_Information, $File_Name, and $Data. For a more complete list of file attributes, please refer to http://blogs.technet.com/b/askcore/archive/2010/08/25/ntfs-file-attributes.aspx.

When a file is created a relationship is forged between the new file and its parent directory. This relationship has three parts:

1. The parent directory has an index entry that refers to the file.

2. The file has a reference to its parent folder.

3. The file has a link count.

So let’s take this file and directory as an example. Here we see the directory TestFolder with a file called TestFile within.

clip_image003

Next we will use a tool to look at the directory’s file attributes using its file record number (4757h in this case). Such tools exist for download but not from Microsoft. We will NOT be discussing how they are used.

NOTE: Do not get caught up on the bulk of the metadata. We will be reviewing only what is important to the understanding hard links. Also in the spirit of simplification, I have removed any attributes that aren’t involved in any way with the demonstrating of how hard links function.

File Record 0x4757

 

_FILE_RECORD_SEGMENT_HEADER {

    _MULTI_SECTOR_HEADER MultiSectorHeader {

            ULONG Signature : 0x454c4946 "FILE"

            USHORT SequenceArrayOffset : 0x0030

            USHORT SequenceArraySize : 0x0003

    }

  LONGLONG Lsn : 0x00000005a660a744

    USHORT SequenceNumber : 0x0025

    USHORT ReferenceCount : 0x0001

    USHORT FirstAttributeOffset : 0x0038

    USHORT Flags : 0x0003

    ULONG FirstFreeByte : 0x000001f8

    ULONG BytesAvailable : 0x00000400

    _MFT_SEGMENT_REFERENCE BaseFileRecordSegment {

        ULONGLONG SegmentNumber : 0x0000000000000000

        USHORT SequenceNumber : 0x0000

    }

  USHORT NextAttributeInstance : 0x0005

    ULONG SegmentNumberLowPart : 0x00004757

    USHORT SegmentNumberHighPart : 0x0000

}

    _ATTRIBUTE_RECORD_HEADER { $STANDARD_INFORMATION:""

        ULONG TypeCode : 0x00000010

        ULONG RecordLength : 0x00000060

        UCHAR FormCode : 0x00

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0000

        USHORT Flags : 0x0000

        USHORT Instance : 0x0000

        Resident {

            ULONG ValueLength : 0x00000048

            USHORT ValueOffset : 0x0018

            UCHAR ResidentFlags : 0x00

            UCHAR Reserved : 0x00

        }

    }

        _STANDARD_INFORMATION {

            LONGLONG CreationTime : 0x01cbad237cfab9d1 01/05/2011 LCL 16:57:04.245

            LONGLONG LastModificationTime : 0x01cbad238c3239f8 01/05/2011 LCL 16:57:29.775

            LONGLONG LastChangeTime : 0x01cbad238c808309 01/05/2011 LCL 16:57:30.288

            LONGLONG LastAccessTime : 0x01cbad238c3239f8 01/05/2011 LCL 16:57:29.775

        ULONG FileAttributes : 0x00000000

            ULONG MaximumVersions : 0x00000000

            ULONG VersionNumber : 0x00000000

            ULONG ClassId : 0x00000000

            ULONG OwnerId : 0x00000000

            ULONG SecurityId : 0x00000539

            ULONGLONG QuotaCharged : 0x0000000000000000

            ULONGLONG Usn : 0x00000000f65da758

        }

    _ATTRIBUTE_RECORD_HEADER { $FILE_NAME:""

        ULONG TypeCode : 0x00000030

        ULONG RecordLength : 0x00000070

        UCHAR FormCode : 0x00

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0000

        USHORT Flags : 0x0000

        USHORT Instance : 0x0003

        Resident {

            ULONG ValueLength : 0x00000056

            USHORT ValueOffset : 0x0018

            UCHAR ResidentFlags : 0x01

            UCHAR Reserved : 0x00

        }

    }

        _FILE_NAME {

            _MFT_SEGMENT_REFERENCE ParentDirectory {

                ULONGLONG SegmentNumber : 0x0000000000000005

                USHORT SequenceNumber : 0x0005

            }

            _DUPLICATED_INFORMATION Info {

                LONGLONG CreationTime : 0x01cbad237cfab9d1 01/05/2011 LCL 16:57:04.245

                LONGLONG LastModificationTime : 0x01cbad237cfab9d1 01/05/2011 LCL 16:57:04.245

                LONGLONG LastChangeTime : 0x01cbad237cfab9d1 01/05/2011 LCL 16:57:04.245

                LONGLONG LastAccessTime : 0x01cbad237cfab9d1 01/05/2011 LCL 16:57:04.245

                LONGLONG AllocatedLength : 0x0000000000000000

                LONGLONG FileSize : 0x0000000000000000

                ULONG FileAttributes : 0x10000000

                PackedEaSize : 0x0000

                ULONG ReparsePointTag : 0x00000000

            }

            UCHAR FileNameLength : 0x0a

            UCHAR Flags : 0x00

            ..... FileName : "TestFolder"

        }

<Attribute removed from display for simplification>

    _ATTRIBUTE_RECORD_HEADER { $INDEX_ROOT:"$I30"

        ULONG TypeCode : 0x00000090

        ULONG RecordLength : 0x000000c0

        UCHAR FormCode : 0x00

        UCHAR NameLength : 0x04

        USHORT NameOffset : 0x0018

        USHORT Flags : 0x0000

        USHORT Instance : 0x0001

        Resident {

            ULONG ValueLength : 0x000000a0

            USHORT ValueOffset : 0x0020

            UCHAR ResidentFlags : 0x00

            UCHAR Reserved : 0x00

        }

    }

        _INDEX_ROOT {

            ULONG IndexedAttributeType : 0x00000030

            ULONG CollationRule : 0x00000001

            ULONG BytesPerIndexBuffer : 0x00001000

            _INDEX_HEADER {

                ULONG FirstIndexEntry : 0x00000010

                ULONG FirstFreeByte : 0x00000090

                ULONG BytesAvailable : 0x00000090

                UCHAR Flags : 0x00

                ..... Reserved :

            }

        }

        _INDEX_ENTRY {

            _MFT_SEGMENT_REFERENCE FileReference {

                ULONGLONG SegmentNumber : 0x00000000000054f7

                USHORT SequenceNumber : 0x0567

            }

            USHORT Length : 0x0070

            USHORT AttributeLength : 0x005a

            USHORT Flags : 0x0000

            ..... Reserved :

            _FILE_NAME {

                _MFT_SEGMENT_REFERENCE ParentDirectory {

                  ULONGLONG SegmentNumber : 0x0000000000004757

                    USHORT SequenceNumber : 0x0025

                }

                _DUPLICATED_INFORMATION Info {

                    LONGLONG CreationTime : 0x01cbad2389b8ec62 01/05/2011 LCL 16:57:25.624

                    LONGLONG LastModificationTime : 0x01cbad2392dec574 01/05/2011 LCL 16:57:40.972

                    LONGLONG LastChangeTime : 0x01cbad2392dec574 01/05/2011 LCL 16:57:40.972

                    LONGLONG LastAccessTime : 0x01cbad2389b8ec62 01/05/2011 LCL 16:57:25.624

                    LONGLONG AllocatedLength : 0x0000000000000008

                    LONGLONG FileSize : 0x0000000000000008

                    ULONG FileAttributes : 0x00000020

                    PackedEaSize : 0x0000

                    ULONG ReparsePointTag : 0x00000000

                }

                UCHAR FileNameLength : 0x0c

                UCHAR Flags : 0x00

                ..... FileName : "TestFile.txt"

            }

        }

 

The section marked in RED is the index entry for the one file that is housed by this directory, TestFile.txt. If the directory had multiple files, then there would be additional index entries.

Inside the index entry you can see a segment number…

                ULONGLONG SegmentNumber : 0x00000000000054f7

 

This segment number is the file record number for example file. Also listed in the index entry is the name of the file.

                ..... FileName : "TestFile.txt"

What this tells us is that this directory has a file named ‘TestFile.txt’ and it can be found at file record number 54f7h in the MFT.

Looking at the file itself…

File Record 0x54f7

 

_FILE_RECORD_SEGMENT_HEADER {

    _MULTI_SECTOR_HEADER MultiSectorHeader {

            ULONG Signature : 0x454c4946 "FILE"

            USHORT SequenceArrayOffset : 0x0030

            USHORT SequenceArraySize : 0x0003

    }

    LONGLONG Lsn : 0x00000005a660c4e6

    USHORT SequenceNumber : 0x0567

    USHORT ReferenceCount : 0x0001

    USHORT FirstAttributeOffset : 0x0038

    USHORT Flags : 0x0001

    ULONG FirstFreeByte : 0x00000160

    ULONG BytesAvailable : 0x00000400

    _MFT_SEGMENT_REFERENCE BaseFileRecordSegment {

        ULONGLONG SegmentNumber : 0x0000000000000000

        USHORT SequenceNumber : 0x0000

    }

    USHORT NextAttributeInstance : 0x0005

    ULONG SegmentNumberLowPart : 0x000054f7

    USHORT SegmentNumberHighPart : 0x0000

}

    _ATTRIBUTE_RECORD_HEADER { $STANDARD_INFORMATION:""

        ULONG TypeCode : 0x00000010

        ULONG RecordLength : 0x00000060

        UCHAR FormCode : 0x00

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0000

        USHORT Flags : 0x0000

        USHORT Instance : 0x0000

        Resident {

            ULONG ValueLength : 0x00000048

            USHORT ValueOffset : 0x0018

            UCHAR ResidentFlags : 0x00

            UCHAR Reserved : 0x00

        }

    }

        _STANDARD_INFORMATION {

            LONGLONG CreationTime : 0x01cbad2389b8ec62 01/05/2011 LCL 16:57:25.624

            LONGLONG LastModificationTime : 0x01cbad2392dec574 01/05/2011 LCL 16:57:40.972

            LONGLONG LastChangeTime : 0x01cbad2392dec574 01/05/2011 LCL 16:57:40.972

            LONGLONG LastAccessTime : 0x01cbad2389b8ec62 01/05/2011 LCL 16:57:25.624

            ULONG FileAttributes : 0x00000020

            ULONG MaximumVersions : 0x00000000

            ULONG VersionNumber : 0x00000000

            ULONG ClassId : 0x00000000

            ULONG OwnerId : 0x00000000

            ULONG SecurityId : 0x00000538

            ULONGLONG QuotaCharged : 0x0000000000000000

            ULONGLONG Usn : 0x00000000f65db1a8

        }

    _ATTRIBUTE_RECORD_HEADER { $FILE_NAME:""

        ULONG TypeCode : 0x00000030

        ULONG RecordLength : 0x00000078

        UCHAR FormCode : 0x00

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0000

        USHORT Flags : 0x0000

        USHORT Instance : 0x0003

        Resident {

            ULONG ValueLength : 0x0000005a

            USHORT ValueOffset : 0x0018

            UCHAR ResidentFlags : 0x01

            UCHAR Reserved : 0x00

        }

    }

        _FILE_NAME {

            _MFT_SEGMENT_REFERENCE ParentDirectory {

                ULONGLONG SegmentNumber : 0x0000000000004757

                USHORT SequenceNumber : 0x0025

            }

            _DUPLICATED_INFORMATION Info {

                LONGLONG CreationTime : 0x01cbad2389b8ec62 01/05/2011 LCL 16:57:25.624

                LONGLONG LastModificationTime : 0x01cbad2389b8ec62 01/05/2011 LCL 16:57:25.624

                LONGLONG LastChangeTime : 0x01cbad2389b8ec62 01/05/2011 LCL 16:57:25.624

               LONGLONG LastAccessTime : 0x01cbad2389b8ec62 01/05/2011 LCL 16:57:25.624

                LONGLONG AllocatedLength : 0x0000000000000000

                LONGLONG FileSize : 0x0000000000000000

                ULONG FileAttributes : 0x00000020

                PackedEaSize : 0x0000

                ULONG ReparsePointTag : 0x00000000

            }

            UCHAR FileNameLength : 0x0c

            UCHAR Flags : 0x00

            ..... FileName : "TestFile.txt"

        }

    <Attribute removed from display for simplification>

        _FILE_OBJECTID_BUFFER {

            GUID BirthVolumeId {00000000-0000-0000-0000000000000000}

            GUID BirthObjectId {00000000-0000-0000-0000000000000000}

            GUID DomainId {00000000-0000-0000-0000000000000000}

            GUID ObjectId {0df3b845-0e3c-11e0-b14e001f2901477c}

            CHAR ExtendedInfo[] {

                ...

            }

        }

    _ATTRIBUTE_RECORD_HEADER { $DATA:""

        ULONG TypeCode : 0x00000080

        ULONG RecordLength : 0x00000020

        UCHAR FormCode : 0x00

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0018

        USHORT Flags : 0x0000

        USHORT Instance : 0x0001

        Resident {

            ULONG ValueLength : 0x00000008

     USHORT ValueOffset : 0x0018

            UCHAR ResidentFlags : 0x00

            UCHAR Reserved : 0x00

        }

    }

    <Attribute removed from display for simplification>

 

The parts marked in RED are the reference count aka link count and the file name attribute. The link count tells us how many times a directory links back to this file. Since our link count is 1, we know that this file can only be found in a single directory.

The file name attribute is where we keep track of who the parent directory is. Looking closer at the attribute, we can see the following line…

                ULONGLONG SegmentNumber : 0x0000000000004757

This tells us that the file has a parent directory that can be found at file record number 4757h.

So the three parts to the file/directory relationship are now visible. Here is a simplified diagram of this relationship…

clip_image004

1. The directory has an index entry that tells us the file record number for the child file.

2. The file has a file name attribute that tells us what the file record number of the parent directory.

3. The file has a link count that tells us that it only has one parent directory.

 

Once you understand this, adding in a hard link is a fairly simple matter.

clip_image005

We still use basically the same diagram but now we have two directories, each with its own relationship with the file.

1. Each directory has an index entry that tells us the file record number for the child file.

2. The file has two file name attributes. One for each parent directory.

3. The link count is incremented to 2h.

This is how the component store functions. All operating system files are placed into organized subdirectories under the WinSXS directory. When a role is added, we simply create a new links to the appropriate files, usually linking them to the System32 or SysWOW64 directories.

Here is a such a file.

File Record 0x94df

 

_FILE_RECORD_SEGMENT_HEADER {

    _MULTI_SECTOR_HEADER MultiSectorHeader {

            ULONG Signature : 0x454c4946 "FILE"

            USHORT SequenceArrayOffset : 0x0030

            USHORT SequenceArraySize : 0x0003

    }

    LONGLONG Lsn : 0x000000057b08b20d

    USHORT SequenceNumber : 0x0001

    USHORT ReferenceCount : 0x0002

    USHORT FirstAttributeOffset : 0x0038

    USHORT Flags : 0x0001

    ULONG FirstFreeByte : 0x000001d8

    ULONG BytesAvailable : 0x00000400

    _MFT_SEGMENT_REFERENCE BaseFileRecordSegment {

        ULONGLONG SegmentNumber : 0x0000000000000000

        USHORT SequenceNumber : 0x0000

    }

    USHORT NextAttributeInstance : 0x0005

    ULONG SegmentNumberLowPart : 0x000094df

    USHORT SegmentNumberHighPart : 0x0000

}

    _ATTRIBUTE_RECORD_HEADER { $STANDARD_INFORMATION:""

        ULONG TypeCode : 0x00000010

        ULONG RecordLength : 0x00000060

        UCHAR FormCode : 0x00

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0000

        USHORT Flags : 0x0000

        USHORT Instance : 0x0000

        Resident {

            ULONG ValueLength : 0x00000048

    USHORT ValueOffset : 0x0018

            UCHAR ResidentFlags : 0x00

            UCHAR Reserved : 0x00

        }

    }

        _STANDARD_INFORMATION {

            LONGLONG CreationTime : 0x01ca04156a9a588b 07/13/2009 LCL 19:55:36.280

            LONGLONG LastModificationTime : 0x01ca042443976710 07/13/2009 LCL 21:41:53.281

            LONGLONG LastChangeTime : 0x01caee3b49efdbbe 05/07/2010 LCL 19:16:14.441

            LONGLONG LastAccessTime : 0x01ca04156a9a588b 07/13/2009 LCL 19:55:36.280

            ULONG FileAttributes : 0x00000020

            ULONG MaximumVersions : 0x00000000

            ULONG VersionNumber : 0x00000000

           ULONG ClassId : 0x00000000

            ULONG OwnerId : 0x00000000

            ULONG SecurityId : 0x000001f7

            ULONGLONG QuotaCharged : 0x0000000000000000

            ULONGLONG Usn : 0x0000000001843698

        }

    _ATTRIBUTE_RECORD_HEADER { $FILE_NAME:""

        ULONG TypeCode : 0x00000030

        ULONG RecordLength : 0x00000078

        UCHAR FormCode : 0x00

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0000

        USHORT Flags : 0x0000

        USHORT Instance : 0x0002

        Resident {

            ULONG ValueLength : 0x0000005a

            USHORT ValueOffset : 0x0018

            UCHAR ResidentFlags : 0x01

            UCHAR Reserved : 0x00

        }

    }

        _FILE_NAME {

            _MFT_SEGMENT_REFERENCE ParentDirectory {

                ULONGLONG SegmentNumber : 0x0000000000001709

                USHORT SequenceNumber : 0x0001

            }

            _DUPLICATED_INFORMATION Info {

                LONGLONG CreationTime : 0x01caee3b3b1feba9 05/07/2010 LCL 19:15:49.590

                LONGLONG LastModificationTime : 0x01caee3b3b1feba9 05/07/2010 LCL 19:15:49.590

                LONGLONG LastChangeTime : 0x01caee3b3b1feba9 05/07/2010 LCL 19:15:49.590

  LONGLONG LastAccessTime : 0x01caee3b3b1feba9 05/07/2010 LCL 19:15:49.590

                LONGLONG AllocatedLength : 0x0000000000000000

                LONGLONG FileSize : 0x0000000000000000

                ULONG FileAttributes : 0x00000020

                PackedEaSize : 0x0000

                ULONG ReparsePointTag : 0x00000000

            }

            UCHAR FileNameLength : 0x0c

            UCHAR Flags : 0x00

            ..... FileName : "OobeFldr.dll"

        }

    _ATTRIBUTE_RECORD_HEADER { $FILE_NAME:""

        ULONG TypeCode : 0x00000030

        ULONG RecordLength : 0x00000078

   UCHAR FormCode : 0x00

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0000

        USHORT Flags : 0x0000

        USHORT Instance : 0x0004

        Resident {

            ULONG ValueLength : 0x0000005a

            USHORT ValueOffset : 0x0018

            UCHAR ResidentFlags : 0x01

            UCHAR Reserved : 0x00

        }

    }

        _FILE_NAME {

            _MFT_SEGMENT_REFERENCE ParentDirectory {

                ULONGLONG SegmentNumber : 0x00000000000010f4

                USHORT SequenceNumber : 0x0001

            }

            _DUPLICATED_INFORMATION Info {

              LONGLONG CreationTime : 0x01caee3b3b1feba9 05/07/2010 LCL 19:15:49.590

                LONGLONG LastModificationTime : 0x01caee3b3b1feba9 05/07/2010 LCL 19:15:49.590

                LONGLONG LastChangeTime : 0x01caee3b3b1feba9 05/07/2010 LCL 19:15:49.590

                LONGLONG LastAccessTime : 0x01caee3b3b1feba9 05/07/2010 LCL 19:15:49.590

                LONGLONG AllocatedLength : 0x00000000000dc000

                LONGLONG FileSize : 0x00000000000db600

                ULONG FileAttributes : 0x00000020

                PackedEaSize : 0x0000

                ULONG ReparsePointTag : 0x00000000

            }

            UCHAR FileNameLength : 0x0c

            UCHAR Flags : 0x00

            ..... FileName : "OobeFldr.dll"

        }

    _ATTRIBUTE_RECORD_HEADER { $DATA:""

        ULONG TypeCode : 0x00000080

        ULONG RecordLength : 0x00000048

        UCHAR FormCode : 0x01

        UCHAR NameLength : 0x00

        USHORT NameOffset : 0x0000

        USHORT Flags : 0x0000

        USHORT Instance : 0x0003

        Nonresident {

            LONGLONG LowestVcn : 0x0000000000000000

            LONGLONG HighestVcn : 0x00000000000000db

            USHORT MappingPairsOffset : 0x0040

            UCHAR CompressionUnit : 0x00

            ..... Reserved :

            LONGLONG AllocatedLength : 0x00000000000dc000

            LONGLONG FileSize : 0x00000000000db600

            LONGLONG ValidDataLength : 0x00000000000db600

        }

<Attribute removed from display for simplification>

 

The link count has been set to 2h.

    USHORT ReferenceCount : 0x0002

This tells us that there are two directory relationships to this file.

Also notice that this file has two file name attributes. If you compare the file record number for each attribute, you can see that they differ.

                ULONGLONG SegmentNumber : 0x0000000000001709

                ULONGLONG SegmentNumber : 0x00000000000010f4

 

Each file record number points to one of the parent directories of this file. This is how a single file can exist in multiple directories.

That is the nature of how hard links work.

As a final note, this is completely different than how a reparse point functions. But that is a subject for another time.

Thank you for your time.

Robert Mitchell

Microsoft Enterprise Support - Windows Server CoreTeam

Want to know more about Microsoft storage? Check out my other blogs...

http://blogs.technet.com/askcore/archive/2010/02/18/understanding-the-2-tb-limit-in-windows-storage.aspx

http://blogs.technet.com/askcore/archive/2009/10/16/the-four-stages-of-ntfs-file-growth.aspx

http://blogs.technet.com/askcore/archive/2009/12/30/ntfs-metafiles.aspx

http://blogs.technet.com/b/askcore/archive/2010/08/25/ntfs-file-attributes.aspx

http://blogs.technet.com/b/askcore/archive/2010/10/08/gpt-in-windows.aspx

NEW NEW NEW!!

http://blogs.technet.com/b/askperf/archive/2010/12/03/performance-counter-for-iscsi.aspx