Building Windows 8 strikes again, this time with a 1/16/2012 blog from Surendra Verma, development manager for the Windows 8 Storage and File system team. It’s entitled “Building the next generation file system for Windows: ReFS.” The ReFS acronym is expanded in the FAQ that follows the blog post to mean “Resilient File System;” that same FAQ also documents some awe-inspiring maximum file system attributes.
What’s New About ReFS?
Verma takes several cuts at answering this question without necessary addressing it completely explicitly, so I’m doing my best to read between the lines here. It appears that ReFS comes with a new engine for file access and management, and also includes enhanced verification and auto-correction facilities. In Verma’s words, ReFS is optimized for “extreme scale” to “use scalable structures for everything.” Interestingly, ReFS volume will never be taken offline: even when corruption occurs, unaffected volume elements will remain available and accessible. And a new resiliency architecture (when used in concert with Storage Spaces) will help to protect and preserve volume contents. Verma’s list of key features also highlights plenty of new capability:
- Metadata integrity with checksums
- Integrity streams providing optional user data integrity
- Allocate on write transactional model for robust disk updates (also known as copy on write)
- Large volume, file and directory sizes
- Storage pooling and virtualization makes file system creation and management easy
- Data striping for performance (bandwidth can be managed) and redundancy for fault tolerance
- Disk scrubbing for protection against latent disk errors
- Resiliency to corruptions with “salvage” for maximum volume availability in all cases
- Shared storage pools across machines for additional failure tolerance and load balancing
What Remains in Common between ReFS and NTFS?
Not surprisingly, Microsoft has also elected to “maintain a high degree of compatibility with a subset of NTFS features that are widely adopted” although they do intend to “deprecate other … [features] … that provide limited value at the cost of system complexity and footprint.” This means that BitLocker encryption, access-control lists (ACLs), the USN journal, change notifications, symbolic links, junction, mount and reparse points, volume snapshots, file IDs, and oplocks will remain the same as those used in NTFS.
However, there will be no conversion utility to transform NTFS formatted volumes to ReFS. Instead Microsoft advocates copying data from NTFS into ReFS to make the change, and also to allow the resulting ReFS volumes, file collections, and data sets to enjoy the benefits of the new ReFS structures, maximum sizes, and resiliency. For the time being, ReFS will not be used to boot volumes in Windows 8 (though I’ve seen some discussions of making ReFS volumes bootable for the upcoming version of Windows Server due out late 2012 or early 2013).
What’s Gone from ReFS?
Verma does address this as “What semantics or features of NTFS are no longer supported on ReFS?”
The NTFS features we have chosen to not support in ReFS are: named streams, object IDs, short names, compression, file level encryption (EFS), user data transactions, sparse, hard-links, extended attributes, and quotas.
This is an interesting grab-bag of features, among which there are some obvious items: EFS was never that popular, and had sufficient problems to make lots of enterprise users steer clear, short names have been passe for 10 years or more, and named streams (aka “alternate streams”) have posed some clear and present security dangers. Extended attribute elements (also including object IDs) support the distributed link tracking service, which can be convenient but also had some problems, as did sparse files and hard links. It’s a little surprising to see file compression and quotas going away, and user data transaction handling was never implemented much anyway.
Once again, it’s “in with the old, and out with the new.” This should give Mark Russinovich and his colleagues something interesting to update in the next edition of Windows Internals, which should give me an opportunity to understand the impact of these changes in more detail.