NTFS vs. ReFS – How to Decide Which to Use

Ever since Windows Server 2012 you’ve had two choices for a file system to use, Resilient File System (ReFS) or New Technology File System (NTFS), which is now way over 20 years old, so perhaps not so “new”). ReFS exceeds NTFS in scalability and reliability and while it originally didn’t work well with Hyper-V VM storage, it’s now the preferred option. In this article, we’ll look at ReFS and if it’s the right choice for your situation.

Creating a Storage Spaces Direct volume for a Hyper-V cluster

What is ReFS?

As mentioned, “ReFS” means “resilient file system” and it has built-in features to guard against data corruption. Microsoft’s docs provide a detailed exploration of Microsoft ReFS and its features. A brief recap:

● Integrity streams: ReFS uses checksums to check for file corruption.

● Automatic repair: When ReFS detects problems in a file, it will automatically enact corrective action.

● Performance improvements: In some situations, ReFS provides performance benefits over NTFS.

● Very large volume and file support: ReFS’s upper limits exceed NTFS’s without incurring the same performance hits.

● Mirror-accelerated parity: Mirror-accelerated parity uses a lot of raw storage space, but it’s very fast and very resilient.

● Integration with Storage Spaces: Many of ReFS’s features only work to their fullest in conjunction with Storage Spaces.

Before you get excited about some of the earlier points, we need to emphasize one thing: ReFS requires Storage Spaces in order to do its best work.

ReFS Benefits for Hyper-V

ReFS has features that accelerate some virtual machine activities.

● Block cloning is the ability of the file system itself to copy a large file as a metadata operation, rather than reading all the blocks of one file and then writing them to a different area of storage. Anyone who’s copied a 200 GB VHDX file knows the time this takes, on top of ReFS the blocks are essentially cloned in place so it’s lightning-fast. This also greatly speeds checkpoint merges.

● Sparse VDL (valid data length): All file systems record the amount of space allocated to a file. Microsoft ReFS uses VDL to indicate how much of that file has data. So, when you instruct Hyper-V to create a new fixed VHDX on ReFS, it can create the entire file in about the same amount of time as creating a dynamically expanding VHDX. It will similarly benefit expansion operations on dynamically expanding VHDXs.

Take a little bit of time to think about these features and how they might benefit your situation.

Formatting a Storage Spaces Volume with ReFS

ReFS vs. NTFS for Hyper-V: Technical Comparison

With the general explanation out of the way, now you can make a better assessment of ReFS vs NTFS question. First, check the comparison tables on Microsoft’s ReFS overview page. For typical Hyper-V deployments, most of the differences mean very little. For instance, you probably don’t need quotas for your Hyper-V storage locations. Let’s make a table of our own, scoped more appropriately for Hyper-V:

● ReFS wins: Really large storage locations and really large VHDXs

● ReFS wins: Environments with large numbers of created, checkpointed or merged VHDXs

● ReFS wins: Storage Space and Storage Spaces Direct deployments

● NTFS wins: Single-volume deployments

● NTFS wins (potentially): Mixed-purpose deployments

I think most of these things speak for themselves. The last two probably need a bit more explanation.

Single-Volume Deployments Require NTFS

In this context, I intend “single-volume deployment” to mean installations where you have Hyper-V (including its management operating system) and all VMs on the same volume. You cannot format a boot volume with ReFS, nor can you place a page file on ReFS. This type of installation also does not allow for Storage Spaces or Storage Spaces Direct, so it would miss out on most of ReFS’s capabilities anyway.

Mixed-Purpose Deployments Might Require NTFS

Some of us have the good fortune to deploy nothing but virtual machines in dedicated storage locations but not everyone has that. If your Hyper-V storage volume also hosts files for other purposes, you might need to continue with NTFS. Go over the last table near the bottom of the overview page. It shows the properties that you can only find in NTFS. For standard file sharing scenarios, you lose quotas. You may have legacy applications that require NTFS’s extended properties or short names. In these situations, only NTFS will do.

Note: If you have any other option, do not use the same host to run non-Hyper-V roles alongside Hyper-V, Microsoft does not support this. Similarly, separate Hyper-V VMs onto volumes separately to volumes that hold other file types.

Another question that often arises is if ReFS is faster than NTFS and realistically (apart from the block cloning scenarios for VHDX merge and backup archive storage where it definitely is) there isn’t a noticeable difference in speed for most file system operations, particularly on modern SSD/flash-based storage.

Unexpected ReFS Behavior

The official content goes to some lengths to describe the benefits of ReFS’s integrity streams. It uses checksums to detect file corruption. If it finds problems, it engages in corrective action. On a Storage Spaces volume that uses protective schemes, it has an opportunity to fix the problem. It does that with the volume online, providing a seamless experience. What happens when ReFS can’t correct the problem? That’s where you need to pay real attention.

On the overview page, the documentation uses exceptionally vague wording: “ReFS removes the corrupt data from the namespace”. The integrity streams page does worse: “If the attempt is unsuccessful, ReFS will return an error.” While researching this article, I was told of a more troubling activity: ReFS deletes files that it deems unfixable. I did find an entry from a Microsoft program manager in a forum that states:

ReFS deletes files in two scenarios:

1. 1. ReFS detects Metadata corruption AND there is no way to fix it. Meaning ReFS is not on a Storage Spaces redundant volume where there are other copies of the data on other hosts so it can fix the corrupted copy.
   2. ReFS detects data corruption AND Integrity Streams is enabled AND there is no way to fix it. Meaning if Integrity Stream is not enabled, the file will be accessible whether data is corrupted or not. If ReFS is running on a mirrored volume using Storage Spaces, the corrupted copy will be automatically fixed.

The upshot: If ReFS decides that a VHDX has sustained unrecoverable damage, it will delete it. It will not ask, nor will it give you any opportunity to try to salvage what you can. If ReFS isn’t backed by Storage Spaces’s redundancy, then it has no way to perform a repair. So, from one perspective, that makes Microsoft ReFS on non-Storage Spaces look like a very high-risk approach. But…

Mind Your Backups!

You should not overlook the severity of the previous section. However, you should not let it scare you away, either. I certainly understand that you might prefer a partially readable VHDX to a deleted one. To that end, you could simply disable integrity streams on your VMs’ files. I also have another suggestion.

Do not neglect your backups! If ReFS deletes a file, retrieve it from backup. If a VHDX goes corrupt on NTFS, retrieve it from backup. With ReFS, at least you know that you have a problem. With NTFS, problems can lurk much longer. No matter your configuration, the only thing you can depend on to protect your data is a solid backup solution.

It’s also worth noting that System Center Data Protection Manager (DPM) from 2016 onwards also supports ReFS for backup storage and actually prefers it and this saves storage space and increases performance. Also, Azure Stack Hub, Microsoft’s all-in-one “appliance” rack of servers that gives you the Azure platform in your datacenter uses Storage Spaces Direct with the ReFS filesystem.

Settings for an ReFS formatted volume

Converting NTFS to ReFS and vice versa

There are no options to convert an NTFS volume to ReFS with the data intact. If you do want to change the file system you have to back up the data, reformat the volume and restore the data. This also works for the reverse situation, if you want to go from ReFS to NTFS you also have to copy the data someplace else, format it as NTFS and then restore the data.

When to Choose NTFS for Hyper-V

You now have enough information to make an informed decision. These conditions indicate a good condition for NTFS:

● Configurations that do not use Storage Spaces, such as single-disk or manufacturer RAID. This alone does not make an airtight point; please read the “Mind Your Backups!” section above.

● Single-volume systems (your host only has a C: volume)

● Mixed-purpose systems (please reconfigure to separate roles)

● Storage on hosts older than 2016 — ReFS was not as mature on previous versions.

● Your backup application vendor does not support ReFS

● If you’re uncertain about ReFS

As time goes on, NTFS will lose favorability over ReFS in Hyper-V deployments, however, that does not mean that NTFS has reached the end of its usefulness. ReFS has staggeringly higher limits, but very few systems use more than a fraction of what NTFS can offer. ReFS does have impressive resilience features, but NTFS also has self-healing powers and you have access to RAID technologies to defend against data corruption.

Microsoft has continued to develop ReFS and it’s the recommended file system for virtualized workloads and network-attached storage.

When to Choose ReFS for Hyper-V

Some situations make ReFS the clear choice for storing Hyper-V data:

● Storage Spaces (and Storage Spaces Direct) environments

● Extremely large volumes

● Extremely large VHDXs

You might make an additional performance-based argument for ReFS in an environment with a very high churn of VHDX files. However, do not overestimate the impact of those performance enhancements. The most striking difference appears when you create fixed VHDXs.

However, I do not want to gloss over the benefit of ReFS for very large volumes. If you have a storage volume of a few terabytes and VHDXs of even a few hundred gigabytes, then ReFS will rarely beat NTFS significantly. When you start thinking in terms of hundreds of terabytes, NTFS will likely show bottlenecks. If you need to push higher, then ReFS becomes your only choice.

ReFS really shines when you combine it with Storage Spaces Direct. Its ability to automatically perform a non-disruptive online repair is truly impressive. On the one hand, the odds of disruptive data corruption on modern systems constitute a statistical anomaly. On the other, no one that has suffered through such an event really cares how unlikely it was.

ReFS vs NTFS on Hyper-V Guest File Systems

All of the above deals only with Hyper-V’s storage of virtual machines. What about ReFS in guest operating systems?

To answer that question, we need to go back to ReFS’s strengths. So far, we’ve only thought about it in terms of Hyper-V. Guests have their own conditions and needs. Let’s start by reviewing Microsoft’s ReFS overview. Specifically the following:

“Microsoft has developed NTFS specifically for general-purpose use with a wide range of configurations and workloads, however for customers especially requiring the availability, resiliency, and/or scale that ReFS provides, Microsoft supports ReFS for use under the following configurations and scenarios…”

I added emphasis on the part that I want you to consider. The sentence itself makes you think that they’ll go on to list some usages, but they only list one: “backup target”. The other items on their list only talk about the storage configuration. So, we need to dig back into the sentence and pull out those three descriptors to help us decide: “availability”, “resiliency”, and “scale”. You can toss out the first two right away — you should not focus on storage availability and resiliency inside a VM. That leaves us with “scale”. So, really big volumes and really big files. Remember, that means hundreds of terabytes and up.

For a more accurate decision, read through the feature comparisons. If any application that you want to use inside a guest needs features only found on NTFS, use NTFS. Personally, I still use NTFS inside guests almost exclusively. ReFS needs Storage Spaces to do its best work, and Storage Spaces does its best work at the physical layer.

Combining ReFS with NTFS across Hyper-V Host and Guests

Keep in mind that the file system inside a guest has no bearing on the host’s file system and vice versa. As far as Hyper-V knows, VHDXs attached to virtual machines are nothing other than a bundle of data blocks. You can use any combination that works.

To properly protect your Hyper-V virtual machines, use Altaro VM Backup to securely backup and replicate your virtual machines. We work hard perpetually to give our customers confidence in their Hyper-V backup strategy.

To keep up to date with the latest Hyper-V best practices, become a member of the Hyper-V DOJO now (it’s free).

Conclusion

ReFS is a more modern file system than NTFS with some amazing data resiliency benefits when used with Storage Spaces Direct, as well as performance benefits in specific circumstances, which is why it’s the preferred file system for Hyper-V cluster deployments. However, as covered in this article there are situations where it’s not the best option so make sure you do your research before formatting, as there’s no way to convert from one to the other, without backing up all the data beforehand.