More on new .NET path handling

In my prior post I talked briefly about the new path handling in .NET 4.6.2. In the latest drop (14367) PowerShell opts into both of the new behaviors so you can easily play around with the changes.

There are two basic changes that can be controlled via two compatibility switches. The changes both originated in CoreFX so you can get a more detailed look if you so choose. I'm going to talk about these this week on On .NET, so bring your questions.

Normalization Changes

The first change is that we now try to defer to the OS to normalize paths. Trying to emulate what the OS does led to some discrepancies- one notable one being that you couldn't do anything with directories that end in a space. You can create directories with a space at the end from the CMD prompt, but they were impossible to do anything with in .NET. Refer back to my post on normalization to see a bit more about what is/isn't possible in Windows.

The second change is that we allow better access to DOS device paths. This includes both the normalized (\\.\) and non-normalized (\\?\) syntaxes. These were poorly documented and subsequently poorly understood. Handling was spotty and inconsistent. Getting at paths that were possible using non-normalized syntax (such as \\?\C:\foo.) used to be impossible in .NET.

Both of the DOS device path syntaxes skip most character validity checks. Why? Those checks were wrong and essentially impossible to get correct. Named pipes, for example, can use many characters you can't use in a "normal" NTFS path (and do, Nvidia creates a pipe like this on my machine). This sort of validation is better left to the OS and the various drivers involved, so we do.

The extended DOS device syntax will only work for core APIs that live in mscorlib if you're in full trust. FileIOPermissions historically is the class responsible for kicking \\?\ paths and still is. Comparing extended paths is too costly to do in FileIOPermissions (partially because paths don't have to exist, which makes the logic crazy). For this release the internal usages in mscorlib have a special out when we're running full trust. This should cover the vast majority of usages. I'm looking into expanding this to all usages of FileIOPermissions in full trust in a future .NET. (Note that there are no such limitations for CoreFX as there is no limited trust model.)

Normalization changes are controlled via the UseLegacyPathHandling switch, which is off by default if you target 4.6.2 or higher (the semantics here are part of the AppContext design- you opt into backwardly compatible behavior on newer Framework versions).

Long Path Changes

The key driver in re-evaluating the normalization was the need to support long paths (paths over MAX_PATH, or 260 characters). The old logic didn't scale and prevented usage of the current "long path" solution on Windows, extended DOS device paths (\\?\).

In 4.6.2 we will no longer throw PathTooLongException if we see a path that is >= MAX_PATH. If the OS doesn't like it we'll surface whatever the OS returns as an error (which may be PathTooLong), but we won't second guess what the OS will do. This enables you to use extended long paths and, if the OS would let you, regular long paths. *cough* :)

(Note that in many cases the OS will return DirectoryNotFound for paths that are too long. The primary case is with creating files. We don't have a way to programmatically check whether paths are actually too long so we can't translate DirectoryNotFound into PathTooLong. We're looking at updating the exception text in the future to say it "may also be caused by a path that is too long" where such a thing is possible.)

In CoreFX, .NET will put the extended syntax on for you when calling Windows APIs. This allows you to use long paths without any extra thought or new OS support (if it should ever arrive *cough* *cough*). This implicit handling isn't done in 4.6.2 as it was to costly to fit in the schedule (it is a much more difficult task to get this implicit support in the desktop code as opposed to CoreFX).

The long path changes are controlled via the BlockLongPaths switch, which again is off by default if you target 4.6.2 or higher.

Config File Setting

If you target 4.6.2 this isn't necessary, but if you want to enable the behavior for existing code, here is config file snippet you need to use.

 <?xml version="1.0" encoding="utf-8" ?>
    <AppContextSwitchOverrides value="Switch.System.IO.UseLegacyPathHandling=false;Switch.System.IO.BlockLongPaths=false" />