SQL Server on Linux: How is Delete-On-Close Handled
Windows provides an option for CreateFile to delete a file when the file is closed (FILE_FLAG_DELETE_ON_CLOSE.)
Host Extension Handling
Linux does not expose such an option as part of the open syscall. Instead the host extension remembers that the open request was made with the FILE_FLAG_ON_DELETE option and after closing a file issues the remove syscall on your behalf.
Linux does provide a delete-on-close like capability using the unlink or remove syscalls. Intentionally, the Host Extension does not open the file and then invoke unlink or remove. Linux allows a file open syscall, followed by a call to unlink or remove, and as long as a file descriptor is valid the file can be used by the application for reads and writes. The unlink or remove syscall makes changes to the file system so searching for the file by name returns no entries and when the file descriptor is closed the file is deleted.
The downside of using unlink or remove, and keeping the file descriptor open, is that the file becomes hidden. This can add difficulty tracking space usage or other aspects a hidden file descriptor (still in use) can present.
Update Oct 2018 – SQL Server 2019 CTP 2.0
SQL Server 2019 changes the DELTE On CLOSE behavior used on Linux installations. I continued to work with our Linux partners and we are able to use unlink, right after an open to achieve DELETE ON CLOSE behavior.
I changed the code to call unlink after open completes preventing new access to the file. Only currently open file descriptors may continue to perform reads, writes and fstat activities. Attempts to access the file with cat, open, ‘by name’ fail as the file is no longer available for name resolution.
However, ls conveniently continues to list the file as well as display size changes allowing you to monitor disk space.
When the last file descriptor is closed the file honors the ‘delete on close’ behavior and is fully removed from the system. I even tested the following scenario.
- Linux VM
- mounted location on a Windows UNC path
- open and unlink
The file can be seen from a Windows directory listing.
- Power down the VM (simulate power outage)
While the VM is stopped I can still see the file in the Windows directory listing.
- Restart the VM
File is removed on startup of the VM. The file system journaling intelligently removes the file during startup.
Bob Dorr - Principal Software Engineer SQL Server