Perils of Disk I/O

Being aware of the myriad ways in which disk I/O can fail goes a long way toward creating a robust system and diagnosing problems. The following contains Windows specifics, but each category will have equivalents in any OS.

The exception documentation for File.Open is a good place to start, but wait there’s more:

• File or directory not found
  • An obvious situation, but with edge case subtlety. If you use File.Exists before acting on it, there is no guarantee that the file will still exist by the time your next statement runs. The closest you can come to an atomic operation is to use File.Open with a FileShare restriction. This will either give you a lock or throw an exception in one statement. But beware this will block other users from modifying the file. User-friendly and reliable file sync apps are a challenge.
• File is locked by another process
  • Fails with IOException: The process cannot access the file '{X}' because it is being used by another process.
  • Another process has already opened the file with a FileShare restriction. Note that for testing purposes just opening a file in Notepad does not create a lock (although Notepad cannot save if another process has a lock).
  • This SuperUser post has many options for finding which process has a lock on a file (the accepted answer may not be the best, scroll down).
• Access denied
  • This can happen as a result of OS user/file permissions, modifying a Read Only file, getting blocked by User Account Control, or attempting to operate on a directory as if it were a file.
  • It’s unfortunate these all share the same exception and message in .NET: UnauthorizedAccessException: Access to the path '{X}' is denied.
• Illegal path
  • The rules for legal file names and paths are much longer than you might think.
  • My favorite hidden gems are the reserved names such as CON, PRN, COM1 (see previous link). For historical reasons, you cannot use these names alone or with any extension. Go ahead, try to create COM1.txt in Explorer. The .NET exception for these is: System.ArgumentException: FileStream will not open Win32 devices such as disk partitions and tape drives. Avoid use of "\\.\" in the path.
• Out of disk space
  • This will produce an IOException, but to know it’s for disk space you will need to grab the HResult and compare against system error codes ERROR_DISK_FULL and ERROR_HANDLE_DISK_FULL. This is now a bit easier in .NET 4.5.
  • This may seem like and edge case, but if you have log files better make sure there is an expiration policy (speaking from experience). Removable media is also more likely to fill up.
• Bad timing
  • I’ve observed timing issues on rare occasion, such as when attempting to delete and immediately re-create a directory. The Directory.Delete will return before the file system is truly done getting rid of it, and the Directory.Create operation will fail. The only solution appears to be waiting and retrying. Don’t be 100% sure any file system operation is synchronous.
• Buffers
  • The framework, drivers, OS, and hardware layers can all contain I/O buffers for efficiency. The Win32 FlushFileBuffers is as close as you can get to a sure thing if you are paranoid.
• You opened a file for reading (FileAccess.Read), but then tried to write to it.
  • System.NotSupportedException: Stream does not support writing
  • Should not generally handle this specific exception as it indicates a software logic error.
• Failed parse of previously corrupted file
  • If you deal with a large number of devices, you will eventually encounter file corruption from bad hardware, abrupt system halts, or bad software. This is more likely to occur for files which are modified regularly. Consider implementing checksums and fallbacks for critical files.
• Too many open file handles
  • Windows can handle millions of open file handles in a process, but a long running process on a server that fails to free resources might just get you there. Beware that other libraries may have lower limits than the OS.
• Maximum file size
  • The limit depends on file system, such as NTFS and FAT32.
• If your file I/O is over a network, you get too include all of the ways a network can fail too!

Featured Image Credit

Cliff Shelf Trail at Badlands National Park, South Dakota. Photo by Nick Bushby.