ReFS: What you need to know about the Resilient File System (Part 1)

If you would like to be notified when Deb Shinder releases the next part of this article series please sign up to the WindowsNetworking.com Real time article update newsletter.

Introduction

The Resilient File System (ReFS) was introduced in August 2012 as part of the Windows Server 2012 operating system, as the long-awaited replacement for Microsoft’s venerable NT File System (NTFS). In this article, we’ll take a look at how ReFS came to be and where it is two years after its release, then we’ll discuss how to work with ReFS in Microsoft’s current operating systems and speculate a little about where the Windows file system might be headed in the future.

A Brief History of Windows File Systems

The file system used by a computer operating system defines how data is structured, stored and handled. An operating system can include support for more than one file system. For purposes of this article, we’re talking about disk file systems, although there are many other types such as network file systems (SMB, NFS), optical disc file systems such as UDF and flat file systems such as that used by Amazon’s S3 cloud storage service. In particular, we’re focusing on the disk file systems supported by Windows.

Those of us who have been working with Windows for a long time have been through several evolutions of the file system upon which the OS depends. MS-DOS and Windows 3.x (which ran on top of MS-DOS) used the FAT (File Allocation Table) file system. For the limited uses of which operating systems and applications were capable at the time, it worked. It was simple and provided good performance. But it had some serious limitations that came to light as subsequent versions of the Windows operating system became more sophisticated and applications became more demanding.

The biggest problem with FAT was that hard drives kept outgrowing it. The earliest version of FAT was an 8 bit file system that most of us never used. The next version of FAT was known as FAT12 and it was used by MS-DOS and IBM’s PC-DOS that ran on many of the early personal computers in the 1980s when PCs started to really catch on. FAT 12 had a disk size limit of 32 MiB, or 256 MiB with a 64KiB cluster size).

Note:
MiB stands for Mebibyte, which is slightly different from MB or megabyte. For a full explanation of the difference, see this article.

This limitation wasn’t a problem with my first “gigantic” hard drive that could storage a whole 10 MB, but available drive sizes quickly expanded in response to ever-growing file sizes, and soon we needed more space. Unfortunately, the next version of FAT, the original FAT16, was still hampered by the same 32 MB limitation for 512 KB clusters. A few years later, we got some relief in the improved version of FAT16 (also known as FAT16B), which was able to handle partitions up to 4 GB in size. It still wasn’t enough, so Microsoft brought out FAT32 in 1996 when they introduced Windows 95 OSR2. Windows 98 let you convert old FAT16 formatted disks to FAT32 without reformatting and losing your data. FAT32 could support a volume that was 32 GB in size.

Note:
Also note that the partition/volume size supported is also dependent on the operating system as well as the file system format. The theoretical limitations of a file system are also usually greater than the practical limits imposed by the hardware and OS.

Partition size limitations wasn’t the only problem with FAT. The NTFS file system was first introduced by Microsoft in 1993 in Windows NT 3.1, the “business” fork of the Windows OS. In addition to better support for larger partitions and file sizes, NTFS has better reliability and error recovery, faster performance, and better security through file-level permissions/access control lists. NTFS further supports file-level encryption and compression, which FAT does not do.

NTFS has been the file system of choice for twenty years, but it’s getting a bit long in the tooth. In the 1990s, Microsoft was already experimenting with a new file system called Object File System (OFS) but it never saw the light of day. Then in the early 2000s, Microsoft again raised expectations about a new file system that was code named WinFS, which included a relational database and a shared schema. It was expected to be released as part of Windows Vista (at that time known by code name Longhorn). That didn’t happen, either. NTFS continued to endure.

Introducing ReFS

Finally, a new file system did emerge in 2012, when Microsoft released Windows Server 2012 with support for ReFS. Today’s computing needs are very different from those in the days of FAT. We routinely create huge presentation and video files that can be several gigabytes in size. A single HD movie file can be 20 GB or more. Hard drive capacity if measured in terabytes. ReFS supports a theoretical maximum volume size of one yottabyte, which is equal to one trillion terabytes. A single file on an ReFS volume can be 16 exabytes (one million terabytes) in size. That should be sufficient to handle expanding file sizes for a while.

ReFS is designed as a “next gen” file system for Windows, but current versions of the OS – Windows Server 2012/2012 R2, Windows 7 and 8/8.1 – still support NTFS and even FAT32. To format a drive larger than 32 GB in FAT32, though, you’ll need a utility such as fat32formatter.

Most hard drives on the new operating systems are still formatted in NTFS, and you won’t see the option to format a partition in ReFS in the Windows 8.1 disk management console. ReFS is currently supported only for use by file servers, and you have to jump through some hoops to enable full read/write support for ReFS (which isn’t something you would normally want or need to do other than for experimental purposes). That involves creating a new registry key, after which you can format new partitions in ReFS using the diskpart utility. For instructions on how to do that, check out this article.

While ReFS has some great new features that NTFS doesn’t, it’s also lacking some of NTFS’s capabilities. You can’t (at least at this time) boot Windows from an ReFS volume and the first versions of ReFS don’t include file-level compression and encryption, disk quotas or hard links, all of which are advantages of NTFS over the FAT file systems. Note that ReFS does support sparse files, reparse points, case-sensitive file names and Unicode in file names and perhaps most important, it preserves and enforces access control lists (ACLs).

It’s obvious that ReFS in its current iteration is not a replacement for NTFS, but rather is intended for use in specific circumstances, particularly for storage of very large data sets. The different underlying structure and lack of support for some of the features that we’ve come to take for granted in NTFS present a “gotcha” for anyone who would aspire to use ReFS as the main file system, because some applications that rely on specific NTFS features might not work with ReFS. Many NTFS disk tools don’t work with ReFS, either, because it handles the file metadata differently.

Now for the good news: Storage of most conventional data doesn’t require the specific NTFS features that aren’t supported by ReFS and so ReFS can handle that duty nicely. Its primary use case is on file servers that store extremely large amounts of data. It has data integrity and recovery mechanisms built into the file system, as well. That means those tools that are designed to detect and repair file corruption in other file systems aren’t necessary, so their incompatibility with ReFS isn’t really an issue. Additionally, although ReFS doesn’t support file level (Encrypting File System) encryption, BitLocker can be used to protect ReFS volumes so that’s not so much of an issue, either – and with today’s gigantic hard drives that cost only a few pennies per gigabyte, does anyone really use disk compression anymore anyway?

Summary

ReFS has some distinct advantages over current reigning Windows file system NTFS, but it also has some drawbacks. It boasts self-healing powers, ability to repair files without down time, less risk that data will be lost when there’s a power failure (due to the way it writes metadata), and of course the ability to create huge volumes and files and even give those files names that are longer than 255 characters if you wish. But it’s not quite ready for prime time yet. Disk quotas are commonly relied upon in business environments to prevent “data hoarders” from overloading the file servers, for example.

The new file system is expected to undergo some changes in the next generation of Windows (Windows 10 and its server counterpart). Will it someday replace NTFS? That was the obvious plan of Steve Sinofsky, who wrote an enthusiastic blog post about the next-gen file system back in early 2012.

Of course, it took quite some time for NTFS to overtake FAT32 as the default file system for Windows client systems, in particular. Such transitions don’t (and probably shouldn’t) generally come quickly. In the meantime, ReFS is being used primarily with Microsoft’s Storage Spaces feature that some are predicting will be a hardware RAID killer. In Part 2 of this series, we’ll look at how ReFS and Storage Spaces work together.

If you would like to be notified when Deb Shinder releases the next part of this article series please sign up to the WindowsNetworking.com Real time article update newsletter.

Leave a Comment

Your email address will not be published.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top