ReFS – the file system of the future? ReFS file system

The first developments of the ReFS file system appeared in 2012 directly in Windows Server 2012. Now the technology is seen in the Windows 8 and 10 operating systems as a replacement for NTFS. You need to figure out why ReFS is better than other file systems and whether it can be used on home computers.

Concept of ReFS

ReFS (Resilient file system)– is a fault-tolerant technology that replaced NTFS. Designed to eliminate the shortcomings of its predecessor and reduce the amount of information that can be lost during various operations. Supports working with large files.

So, one of the advantages of the technology is high data security from destruction. The media contains checksums and metadata designed to determine the integrity of the data on the partitions. The scan occurs during read/write operations and immediately detects damaged files.

Benefits of ReFS

The ReFS file system (FS) has the following features:

Great productivity;
Improving the ability to check media for errors;
Low degree of data loss when file system errors and bad blocks occur;
Implementation of EFS encryption;
Disk quota functionality;
Increased maximum file limit to 18.3 EB;
Increased number of files stored in a folder to 18 trillion;
Maximum disk capacity up to 402 EB;
The number of characters in the file name has been increased to 32767.

There are, of course, many opportunities, but that’s not all. However, it is worth considering one point: how useful will all these advantages be to the average user?

For a user working on a computer at home, the only thing that will be useful is the fast speed of checking partitions for errors and reducing the loss of files in the event of these errors. Of course, in this case, security is carried out only at the file system level, that is, it only solves its own problems, and the problem of losing important files still remains a pressing issue. For example, this could happen due to a hard drive failure. The technology has the greatest effect in.

The advantage of RAID is high fault tolerance and data safety, as well as high speed of operation; the most used RAID levels are 1 and 2. The disadvantages of the system are the high cost of purchasing equipment, as well as the time spent on implementation. I think that this is of no use to the average user unless he creates a home server that works 24/7.

Performing tests based on ReFS and NTFS

Using software tools, it was possible to find out that using the ReFS file system compared to NTFS does not provide a noticeable increase in performance. Tests based on similar read and write cycles occurring on the same disk and file sizes, the Crystal Disk Mark utility showed identical results. ReFS had a slight advantage when copying small files.

There were tests using large files, and a slow hard drive partition was used as a guinea pig. The results were disappointing as ReFS showed lower performance compared to NTFS.

There is no doubt that the technology is still raw, the indicators were carried out at the end of 2017, but in Windows 10 the technology can be widely used. The best option for using FS would be based on SSD – solid state drives. These drives are better than HDDs in almost every way.

Benefits of ReFS for other users

The system has such a function as a hypervisor - Hyper-V. This technology is a virtual machine. When using a partition formatted in ReFS, there was an advantage in operating speed. Since the file system uses checksums and metadata, it only needs to refer to them when copying files; if there is a match, it does not have to physically copy the data.

Creating virtual disks in ReFS takes seconds. In NTFS this process takes minutes. Fixed virtual disks in NTFS are created by delays and heavily load the hard drive; with SSDs this is an even bigger problem, since a large number of rewrite cycles is “deadly” for the media. Because of this, working in the background with other applications will be problematic.

It is also planned that ReFS will be highly compatible with virtual machines such as VMware.

Disadvantages of the ReFS file system

Above we looked at the advantages of ReFS technology and touched a little on the disadvantages. Let's talk about the disadvantages in more detail. We must understand that until Microsoft implements the technology in Windows, there will be no development. Now we have the following features:

Existing Windows partitions are not eligible for use by ReFS, that is, only partitions not used by the system must be used, for example, those intended for storing files.
External drives are not supported.
It is not possible to convert an NTFS disk to a ReFS disk without losing data, only formatting and backing up important files.
Not all software is able to recognize this file system.

That's it. Now look at the image below. This Windows 7 and here the FS is not recognized, and an error appears when opening the partition.

In Windows 8, the partition will need to be formatted, since the FS is also not recognized. Before using a new file system on your home PC, it is better to think about the consequences several times. In Windows 8.1, the problem is solved by activating the FS using the registry editor, but this does not always work, especially since using ReFS implies formatting the disk and destroying data.

Some problems arise in Windows 10. If the new partition with ReFS works stably, then the existing one, which was formatted into it, is not recognized by Windows.

How to format a disk or partition in ReFS

Let’s say the user doesn’t care about the shortcomings and shortcomings of the new product. God bless you, friends, let's start analyzing the instructions for formatting a partition in ReFS. I’ll tell you one thing: if something suddenly happens and the partition fails, you can use the R-Studio tool to restore it.

This is interesting:

To format, just follow the following procedure:

Open “This PC” and right-click on the desired section;
In the context menu, click the item "Format";
In the window that opens, in the “File system” field we find REFS;
Press the button "Begin" and wait.

The same can be done using the command line, where you need to enter the following commands one by one:

diskpart– utility for working with disks;
lis vol– display all partitions of the computer;
sel vol 3– where 3 is the number of the required volume;
format fs=refs– formatting into the desired file system.

How to enable ReFS using the registry

If you don't have anything that points to FS, it may need to be enabled. For this we need a registry editor. The procedure works correctly on Windows 8.1 and 10:

Launch the registry editor (Win+R and enter regedit);
Let's go to this thread - HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem;
On the right side of the window, create a 32-bit DWORD parameter with the name RefsDisableLastAccessUpdate;
Enter the number 1 as the value.
Finding a branch HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control;
Create a section with the name MiniNT
As you can see, the ability to use ReFS exists, but it is not recommended to use it yet, especially since it doesn’t make sense for a home computer. Recovering lost files will be problematic, and not all programs understand FS.

Most likely, the technology will develop most on servers, but this will not happen soon. If you remember the advent of NTFS, its full implementation took about seven years. More information can be found on the official Microsoft website - https://docs.microsoft.com/ru-ru/windows-server/storage/refs/refs-overview. In the meantime, you can follow new IT technologies on our website, do not forget to subscribe.

Windows 10 supports multiple file systems out of the box. Some are legacy and exist primarily for backwards compatibility, others are modern and have broad applications. This article describes various methods you can use to see what file system your drives are formatted with.

File system is a special way to store and organize your information across different media, including hard drives, solid state drives, USB drives and other devices. It allows you to store, modify, read files and folders for applications and the operating system installed on your computer.

When you format an internal drive or flash drive, you are preparing it for use as storage media for your operating system. During this process, a file system is created. During formatting, all information stored on the disk or partition will be deleted.

Windows 10 supports file systems FAT, FAT32, exFAT, NTFS And ReFS without using additional software.

They have different functions and properties. For example, FAT and FAT32 are legacy file systems. FAT supports a maximum capacity of 4GB, FAT32 supports 32GB. FAT file systems also have limitations on maximum file size. NTFS is the only file system that supports file compression and encryption and has advanced features.

There are several methods you can use to find the file system used on your drives.

To find out the file system on drives in Windows 10, follow these steps.

Open "Conductor" and go to the folder "This computer".

Right-click the drive and select from the context menu "Properties".

In the Properties window, on the General tab, you will see the file system of your disk.

This method is the simplest and fastest.

Alternatively, you can use the Diskpart tool, Disk Management, or PowerShell.

View disk file system using Diskpart

Press the key combination Win + R.

In the Run field, enter " diskpart" and press Enter.

In Diskpart, enter the command list volume.

After running the command, you will see the file system for each drive connected to your computer.

Show the disk file system using Disk Management.

Press Win + X or right-click on the button "Start".

From the WinX menu, select

See Values in the File System column.

Finally, there is another way to determine the file system for each drive connected to your computer using the PowerShell scripting language.

Open PowerShell on behalf of the administrator.

Enter: get-volume and press the Enter key.

For output see values in column FileSystemType.

Now you know that it is very easy to determine the file system for your disks. You can use any method you like best.

Currently, the computer market offers many options for storing huge amounts of personal or corporate information in digital form. Storage devices include internal and external hard drives, USB flash drives, photo/camcorder memory cards, complex RAID systems, etc. Actual documents, presentations, pictures, music, videos, databases, emails are stored as files that can take up a lot of space.

This article provides a detailed description of how information is stored on a storage device.

Any computer file is stored in storage with a given capacity. In fact, each storage is a linear space for reading or reading and writing digital information. Each byte of information in storage has its own offset from the start of storage (address) and refers to this address. The storage can be represented as a grid with a set of numbered cells (each cell represents one byte). Any file that is saved to storage receives these cells.

Typically, computer storage uses a pair of sectors and an offset within the sector to reference any byte of information in the storage. A sector is a group of bytes (usually 512 bytes), the smallest addressable unit of physical storage. For example, 1040 bytes on a hard drive would be referred to as sector #3 and the sector offset is 16 bytes ([sector - 512] + [sector - 512] + ). This scheme is used to optimize storage addressing and use fewer numbers to refer to any piece of information in the storage.

To omit the second part of the address (sector offset), files are typically stored starting at the beginning of the sector and occupying entire sectors (for example, a 10-byte file occupies an entire sector, a 512-byte file also occupies an entire sector, while a 514-byte file the file occupies two whole sectors).

Each file is stored in "unused" sectors and can be read at a known location and size. However, how do we know which sectors are being used and which are not? Where are the file size, position and name stored? These responses are provided by the file system.

In general, a file system is a structured representation of data and a set of metadata that describes the stored data. The file system serves to store the entire storage, and is also part of an isolated storage segment - a disk partition. Typically, the file system manages blocks rather than sectors. File system blocks are groups of sectors that optimize storage addressing. Modern file systems typically use block sizes ranging from 1 to 128 sectors (512-65536 bytes). Files are usually stored at the beginning of a block and occupy entire blocks.

Huge write/delete operations on the file system result in file system fragmentation. Thus, files are not saved as whole units, but are divided into fragments. For example, the storage is entirely occupied by files of about 4 blocks in size (for example, a collection of images). The user wants to save a file that will take 8 blocks and therefore delete the first and last files. By doing this it clears 8 blocks of space, however the first segment is close to the start of the storage and the second is close to the end of the storage. In this case, a file with 8 blocks is split into two parts (4 blocks for each part) and takes up “holes” of free space. Information about both fragments as part of one file is stored in the file system.

In addition to user files, the file system also contains its own parameters (such as block size), file descriptors (including file size, file location, file fragments, etc.), file names, and directory hierarchy. It can also store security information, extended attributes, and other parameters.

To meet various requirements such as performance, stability and storage reliability, a large number of file systems are designed to serve specific user purposes.

Windows file systems

Microsoft Windows OS uses two main file systems: FAT, inherited from the old DOS with its later extension FAT32, and the widely used NTFS file systems. The recently released ReFS file system was developed by Microsoft as a next-generation file system for Windows 8, 10 servers.

FAT (File Allocation Table) is one of the simplest types of file systems. It consists of a file system descriptor sector (boot sector or superblock), a file system block allocation table (called a file allocation table), and simple storage space for files and folders. Files in FAT are stored in directories. Each directory is an array of 32-byte entries, each of which defines files or extended file attributes (such as a long filename). A file entry assigns the first block of a file. Any next block can be found through the block allocation table, using it as a linked list.

The block allocation table contains an array of block descriptors. A value of zero indicates that the block is not used, and a value other than zero refers to the next block of the file or a special value for the end of the file.

The numbers in FAT12, FAT16, FAT32 indicate the number of bits used to enumerate a file system block. This means that FAT12 can use up to 4096 different block references, while FAT16 and FAT32 can use up to 65536 and 4294967296 respectively. The actual maximum number of blocks is even smaller and depends on the file system driver implementation.

FAT12 was used for older floppies. FAT16 (or simply FAT) and FAT32 are widely used for flash memory cards and USB flash drives. The system is supported by mobile phones, digital cameras and other portable devices.

FAT or FAT32 is a file system that is used on Windows-compatible external storage or disk partitions that are smaller than 2 GB (for FAT) or 32 GB (for FAT32). Windows cannot create a FAT32 file system larger than 32 GB (however, Linux supports FAT32 up to 2 TB).

NTFS (New Technology File System) was introduced in Windows NT and is currently the primary file system for Windows. This is the default file system for disk partitions and the only file system that supports 32 GB disk partitions. The file system is quite extensible and supports many file properties including access control, encryption, etc. Each file in NTFS is stored as a file descriptor in the master file table and file contents. The main file table contains all the information about the file: size, distribution, name, etc. The first and last sectors of the file system contain the file system parameters (boot record or superblock). This file system uses 48 and 64-bit values for file references, thereby supporting high-capacity disk storage.

ReFS (Resilient File System) is the latest development from Microsoft, currently available for Windows 8 and 10 servers. The file system architecture is completely different from other Windows file systems and is mainly organized as a B+-tree. ReFS is highly fault-tolerant due to the new features included in the system, namely Copy-on-Write (CoW): no metadata is changed without being copied; data is written to new disk space rather than over existing data. Whenever files are modified, a new copy of the metadata is stored in the free storage space, and then the system creates a link from the old metadata to the newer one. Thus, the system stores a significant number of old backups in different locations, allowing for easy file recovery as long as the storage location is not overwritten.

For information on recovering data from these file systems, visit " Chances for recovery ».

MacOS File Systems

Apple's MacOS operating system uses two file systems: HFS+, an extension to its own HFS file system used on older Macintosh computers, and the recently released APFS.

The HFS+ file system runs on Apple products, including Mac computers, iPods, and Apple X Server products. Advanced server products also use the Apple Xsan file system, a clustered file system derived from StorNext or CentraVision file systems.

This file system stores files and folders and Finder information about directory browsing, window positions, and so on.

Linux File Systems

The open source Linux OS aims to implement, test and use various file system concepts.

The most popular Linux file systems are:

Ext2, Ext3, Ext4- “native” Linux file system. This file system is subject to active development and improvement. The Ext3 file system is simply an extension of Ext2 that uses journal transaction write operations. Ext4 is an additional extension of Ext3, with support for optimized file distribution information (extents) and extended file attributes. This file system is often used as the "root" file system for most Linux installations.
ReiserFS - An alternative Linux file system for storing huge numbers of small files. It has good file search capabilities and allows you to compactly distribute files, storing tails of files or small files along with metadata, so as not to use large blocks of the file system for the same purpose.
XFS is a file system created by SGI and originally used for the company's IRIX servers. The XFS specifications are now implemented on Linux. The XFS file system has excellent performance and is widely used for storing files.
J.F.S.- a file system developed by IBM for the company's powerful computing systems. JFS1 usually stands for JFS, JFS2 is the second version. Currently, this file system is open source and is implemented in most modern versions of Linux.

Concept " hard coupling" used in such operating systems makes most Linux file systems look the same, since the file name is not treated as an attribute of the file and is rather defined as an alias for a file in a specific directory. A file object can be associated with many locations, even replicated from the same directory under different names. This can lead to serious and even insurmountable difficulties in recovering file names after files have been deleted or the file system has become corrupted.

For information on recovering data from these file systems, visit the " " page.

File systems BSD, Solaris, Unix

The most common file system for these operating systems is UFS (Unix File System), also often called FFS (Fast File System).

Currently, UFS (in different versions) is supported by all operating systems of the Unix family and is the main file system of the BSD OS and the Sun Solaris operating system. Modern computer technologies tend to implement replacements for UFS in different operating systems (ZFS for Solaris, JFS and derivative file systems for Unix, etc.).

For information on recovering data from these file systems, visit the " " page.

Clustered file systems

Clustered file systems are used in computer cluster systems. These file systems support distributed storage.

Distributed file systems include:

ZFS- “Zettabyte File System” is a new file system designed for Sun Solaris OS distributed storage.
Apple Xsan- Apple's evolution of CentraVision and later StorNext file systems.
VMFS- “Virtual Machine File System” developed by VMware for its VMware ESX Server.
GFS- Red Hat Linux "Global File System".
JFS1- original (legacy) design of the IBM JFS file system used in older AIX storage systems.

Common properties of these file systems include support for distributed storage, extensibility, and modularity.

For more information on recovering data from these file systems, please visit the " " page.

Why may a smartphone not launch programs from a memory card? How is ext4 fundamentally different from ext3? Why will a flash drive last longer if you format it in NTFS rather than FAT? What is the main problem with F2FS? The answers lie in the structural features of file systems. We'll talk about them.

Introduction

File systems define how data is stored. They determine what limitations the user will encounter, how fast read and write operations will be, and how long the drive will operate without failures. This is especially true for budget SSDs and their younger brothers - flash drives. Knowing these features, you can get the most out of any system and optimize its use for specific tasks.

You have to choose the type and parameters of the file system every time you need to do something non-trivial. For example, you want to speed up the most common file operations. At the file system level, this can be achieved in different ways: indexing will provide fast searches, and pre-reserving free blocks will make it easier to rewrite frequently changing files. Pre-optimizing the data in RAM will reduce the number of required I/O operations.

Such properties of modern file systems as lazy writing, deduplication and other advanced algorithms help to increase the period of trouble-free operation. They are especially relevant for cheap SSDs with TLC memory chips, flash drives and memory cards.

There are separate optimizations for different levels of disk arrays: for example, the file system can support simplified volume mirroring, instant snapshotting, or dynamic scaling without taking the volume offline.

Black box

Users generally work with the file system that is offered by default by the operating system. They rarely create new disk partitions and even less often think about their settings - they simply use the recommended parameters or even buy pre-formatted media.

For Windows fans, everything is simple: NTFS on all disk partitions and FAT32 (or the same NTFS) on flash drives. If there is a NAS and it uses some other file system, then for most it remains beyond perception. They simply connect to it over the network and download files, as if from a black box.

On mobile gadgets with Android, ext4 is most often found in the internal memory and FAT32 on microSD cards. Yabloko does not care at all what kind of file system they have: HFS+, HFSX, APFS, WTFS... for them there are only beautiful folder and file icons drawn by the best designers. Linux users have the richest choice, but you can add support for non-native file systems in both Windows and macOS - more on that later.

Common roots

Over a hundred different file systems have been created, but a little more than a dozen can be considered current. Although they were all developed for their own specific applications, many ended up being related on a conceptual level. They are similar because they use the same type of (meta)data representation structure - B-trees (“bi-trees”).

Like any hierarchical system, a B-tree begins with a root record and then branches down to leaf elements - individual records of files and their attributes, or “leaves.” The main reason for creating such a logical structure was to speed up the search for file system objects on large dynamic arrays - such as multi-terabyte hard drives or even larger RAID arrays.

B-trees require far fewer disk accesses than other types of balanced trees to perform the same operations. This is achieved due to the fact that the final objects in B-trees are hierarchically located at the same height, and the speed of all operations is precisely proportional to the height of the tree.

Like other balanced trees, B-trees have equal path lengths from the root to any leaf. Instead of growing upward, they branch more and grow wider: all branch points in a B-tree store many references to child objects, making them easy to find in fewer calls. A large number of pointers reduces the number of the most time-consuming disk operations - head positioning when reading arbitrary blocks.

The concept of B-trees was formulated back in the seventies and has since undergone various improvements. In one form or another it is implemented in NTFS, BFS, XFS, JFS, ReiserFS and many DBMSs. All of them are relatives in terms of the basic principles of data organization. The differences concern details, often quite important. Related file systems also have a common disadvantage: they were all created to work specifically with disks even before the advent of SSDs.

Flash memory as the engine of progress

Solid-state drives are gradually replacing disk drives, but for now they are forced to use file systems that are alien to them, passed down by inheritance. They are built on flash memory arrays, the operating principles of which differ from those of disk devices. In particular, flash memory must be erased before being written, an operation that NAND chips cannot perform at the individual cell level. It is only possible for large blocks entirely.

This limitation is due to the fact that in NAND memory all cells are combined into blocks, each of which has only one common connection to the control bus. We will not go into details of the page organization and describe the complete hierarchy. The very principle of group operations with cells and the fact that the sizes of flash memory blocks are usually larger than the blocks addressed in any file system are important. Therefore, all addresses and commands for drives with NAND flash must be translated through the FTL (Flash Translation Layer) abstraction layer.

Compatibility with the logic of disk devices and support for commands of their native interfaces is provided by flash memory controllers. Typically, FTL is implemented in their firmware, but can (partially) be implemented on the host - for example, Plextor writes drivers for its SSDs that accelerate writing.

It is impossible to do without FTL, since even writing one bit to a specific cell triggers a whole series of operations: the controller finds the block containing the desired cell; the block is read completely, written to the cache or to free space, then erased entirely, after which it is rewritten back with the necessary changes.

This approach is reminiscent of everyday life in the army: in order to give an order to one soldier, the sergeant makes a general formation, calls the poor fellow out of formation and commands the rest to disperse. In the now rare NOR memory, the organization was special forces: each cell was controlled independently (each transistor had an individual contact).

The tasks for controllers are increasing, since with each generation of flash memory the technical process of its production decreases in order to increase density and reduce the cost of data storage. Along with technological standards, the estimated service life of chips is also decreasing.

Modules with single-level SLC cells had a declared resource of 100 thousand rewrite cycles and even more. Many of them still work in old flash drives and CF cards. For enterprise-class MLC (eMLC), the resource was declared in the range of 10 to 20 thousand, while for regular consumer-grade MLC it is estimated at 3-5 thousand. Memory of this type is actively being squeezed by even cheaper TLC, whose resource barely reaches a thousand cycles. Keeping the lifespan of flash memory at an acceptable level requires software tricks, and new file systems are becoming one of them.

Initially, the manufacturers assumed that the file system was unimportant. The controller itself must service a short-lived array of memory cells of any type, distributing the load between them in an optimal way. For the file system driver, it simulates a regular disk, and itself performs low-level optimizations on any access. However, in practice, optimization varies from device to device, from magical to bogus.

In enterprise SSDs, the built-in controller is a small computer. It has a huge memory buffer (half a gigabyte or more) and supports many data efficiency techniques to avoid unnecessary rewrite cycles. The chip organizes all blocks in the cache, performs lazy writes, deduplicates on the fly, reserves some blocks and clears others in the background. All this magic happens completely unnoticed by the OS, programs and the user. With an SSD like this, it really doesn't matter which file system is used. Internal optimizations have a much greater impact on performance and resource than external ones.

Budget SSDs (and even more so flash drives) are equipped with much less smart controllers. The cache in them is limited or absent, and advanced server technologies are not used at all. The controllers in memory cards are so primitive that it is often claimed that they do not exist at all. Therefore, for cheap devices with flash memory, external methods of load balancing remain relevant - primarily using specialized file systems.

From JFFS to F2FS

One of the first attempts to write a file system that would take into account the principles of organizing flash memory was JFFS - Journaling Flash File System. Initially, this development by the Swedish company Axis Communications was aimed at increasing the memory efficiency of network devices that Axis produced in the nineties. The first version of JFFS supported only NOR memory, but already in the second version it became friends with NAND.

Currently JFFS2 has limited use. It is still mainly used in Linux distributions for embedded systems. It can be found in routers, IP cameras, NAS and other regulars of the Internet of Things. In general, wherever a small amount of reliable memory is required.

A further attempt to develop JFFS2 was LogFS, which stored inodes in a separate file. The authors of this idea are Jorn Engel, an employee of the German division of IBM, and Robert Mertens, a teacher at the University of Osnabrück. LogFS source code is available on GitHub. Judging by the fact that the last change to it was made four years ago, LogFS has not gained popularity.

But these attempts spurred the emergence of another specialized file system - F2FS. It was developed by Samsung Corporation, which accounts for a considerable part of the flash memory produced in the world. Samsung makes NAND Flash chips for its own devices and for other companies, and also develops SSDs with fundamentally new interfaces instead of legacy disk ones. Creating a specialized file system optimized for flash memory was a long overdue necessity from Samsung's point of view.

Four years ago, in 2012, Samsung created F2FS (Flash Friendly File System). Her idea was good, but the implementation turned out to be crude. The key task when creating F2FS was simple: to reduce the number of cell rewrite operations and distribute the load on them as evenly as possible. This requires performing operations on multiple cells within the same block at the same time, rather than forcing them one at a time. This means that what is needed is not instant rewriting of existing blocks at the first request of the OS, but caching of commands and data, adding new blocks to free space and delayed erasing of cells.

Today, F2FS support is already officially implemented in Linux (and therefore in Android), but in practice it does not yet provide any special advantages. The main feature of this file system (lazy rewrite) led to premature conclusions about its effectiveness. The old caching trick even fooled early versions of benchmarks, where F2FS demonstrated an imaginary advantage not by a few percent (as expected) or even by several times, but by orders of magnitude. The F2FS driver simply reported the completion of an operation that the controller was just planning to do. However, if the real performance gain for F2FS is small, then the wear on the cells will definitely be less than when using the same ext4. Those optimizations that a cheap controller cannot do will be performed at the level of the file system itself.

Extents and bitmaps

For now, F2FS is perceived as exotic for geeks. Even Samsung's own smartphones still use ext4. Many consider it a further development of ext3, but this is not entirely true. This is more about a revolution than about breaking the 2 TB per file barrier and simply increasing other quantitative indicators.

When computers were large and files were small, addressing was not a problem. Each file was allocated a certain number of blocks, the addresses of which were entered into the correspondence table. This is how the ext3 file system worked, which remains in service to this day. But in ext4 a fundamentally different addressing method appeared - extents.

Extents can be thought of as extensions of inodes as discrete sets of blocks that are addressed entirely as contiguous sequences. One extent can contain an entire medium-sized file, but for large files it is enough to allocate a dozen or two extents. This is much more efficient than addressing hundreds of thousands of small blocks of four kilobytes.

The recording mechanism itself has also changed in ext4. Now blocks are distributed immediately in one request. And not in advance, but immediately before writing data to disk. Lazy multi-block allocation allows you to get rid of unnecessary operations that ext3 was guilty of: in it, blocks for a new file were allocated immediately, even if it entirely fit in the cache and was planned to be deleted as temporary.

FAT restricted diet

In addition to balanced trees and their modifications, there are other popular logical structures. There are file systems with a fundamentally different type of organization - for example, linear. You probably use at least one of them often.

Mystery

Guess the riddle: at twelve she began to gain weight, by sixteen she was a stupid fatty, and by thirty-two she became fat, and remained a simpleton. Who is she?

That's right, this is a story about the FAT file system. Compatibility requirements provided her with bad heredity. On floppy disks it was 12-bit, on hard drives it was initially 16-bit, and has survived to this day as 32-bit. In each subsequent version, the number of addressable blocks increased, but nothing changed in its essence.

The still popular FAT32 file system appeared twenty years ago. Today it is still primitive and does not support access control lists, disk quotas, background compression, or other modern data optimization technologies.

Why is FAT32 needed these days? Everything is still solely to ensure compatibility. Manufacturers rightly believe that a FAT32 partition can be read by any OS. That's why they create it on external hard drives, USB Flash and memory cards.

How to free up your smartphone's flash memory

microSD(HC) cards used in smartphones are formatted in FAT32 by default. This is the main obstacle to installing applications on them and transferring data from internal memory. To overcome it, you need to create a partition on the card with ext3 or ext4. All file attributes (including owner and access rights) can be transferred to it, so any application can work as if it were launched from internal memory.

Windows does not know how to create more than one partition on flash drives, but for this you can run Linux (at least in a virtual machine) or an advanced utility for working with logical partitioning - for example, MiniTool Partition Wizard Free. Having discovered an additional primary partition with ext3/ext4 on the card, the Link2SD application and similar ones will offer many more options than in the case of a single FAT32 partition.

Another argument in favor of choosing FAT32 is often cited as its lack of journaling, which means faster write operations and less wear on NAND Flash memory cells. In practice, using FAT32 leads to the opposite and gives rise to many other problems.

Flash drives and memory cards die quickly due to the fact that any change in FAT32 causes overwriting of the same sectors where two chains of file tables are located. I saved the entire web page, and it was overwritten a hundred times - with each addition of another small GIF to the flash drive. Have you launched portable software? It creates temporary files and constantly changes them while running. Therefore, it is much better to use NTFS on flash drives with its failure-resistant $MFT table. Small files can be stored directly in the main file table, and its extensions and copies are written to different areas of flash memory. In addition, NTFS indexing makes searching faster.

INFO

For FAT32 and NTFS, theoretical restrictions on the level of nesting are not specified, but in practice they are the same: only 7707 subdirectories can be created in a first-level directory. Those who like to play matryoshka dolls will appreciate it.

Another problem that most users face is that it is impossible to write a file larger than 4 GB to a FAT32 partition. The reason is that in FAT32 the file size is described by 32 bits in the file allocation table, and 2^32 (minus one, to be precise) is exactly four gigs. It turns out that neither a movie in normal quality nor a DVD image can be written to a freshly purchased flash drive.

Copying large files is not so bad: when you try to do this, the error is at least immediately visible. In other situations, FAT32 acts as a time bomb. For example, you copied portable software onto a flash drive and at first you use it without problems. After a long time, one of the programs (for example, accounting or email), the database becomes bloated, and... it simply stops updating. The file cannot be overwritten because it has reached the 4 GB limit.

A less obvious problem is that in FAT32 the creation date of a file or directory can be specified to within two seconds. This is not sufficient for many cryptographic applications that use timestamps. The low precision of the date attribute is another reason why FAT32 is not considered a valid file system from a security perspective. However, its weaknesses can also be used for your own purposes. For example, if you copy any files from an NTFS partition to a FAT32 volume, they will be cleared of all metadata, as well as inherited and specially set permissions. FAT simply doesn't support them.

exFAT

Unlike FAT12/16/32, exFAT was developed specifically for USB Flash and large (≥ 32 GB) memory cards. Extended FAT eliminates the above-mentioned disadvantage of FAT32 - overwriting the same sectors with any change. As a 64-bit system, it has no practically significant limits on the size of a single file. Theoretically, it can be 2^64 bytes (16 EB) in length, and cards of this size will not appear soon.

Another fundamental difference between exFAT is its support for access control lists (ACLs). This is no longer the same simpleton from the nineties, but the closed nature of the format hinders the implementation of exFAT. ExFAT support is fully and legally implemented only in Windows (starting from XP SP2) and OS X (starting from 10.6.5). On Linux and *BSD it is supported either with restrictions or not quite legally. Microsoft requires licensing for the use of exFAT, and there is a lot of legal controversy in this area.

Btrfs

Another prominent representative of file systems based on B-trees is called Btrfs. This FS appeared in 2007 and was originally created in Oracle with an eye to working with SSDs and RAIDs. For example, it can be dynamically scaled: creating new inodes directly on the running system or dividing a volume into subvolumes without allocating free space to them.

The copy-on-write mechanism implemented in Btrfs and full integration with the Device mapper kernel module allow you to take almost instantaneous snapshots through virtual block devices. Pre-compression (zlib or lzo) and deduplication speed up basic operations while also extending the lifetime of flash memory. This is especially noticeable when working with databases (2-4 times compression is achieved) and small files (they are written in orderly large blocks and can be stored directly in “leaves”).

Btrfs also supports full logging mode (data and metadata), volume checking without unmounting, and many other modern features. The Btrfs code is published under the GPL license. This file system has been supported as stable in Linux since kernel version 4.3.1.

Logbooks

Almost all more or less modern file systems (ext3/ext4, NTFS, HFSX, Btrfs and others) belong to the general group of journaled ones, since they keep records of changes made in a separate log (journal) and are checked against it in the event of a failure during disk operations . However, the logging granularity and fault tolerance of these file systems differ.

Ext3 supports three logging modes: closed-loop, ordered, and full logging. The first mode involves recording only general changes (metadata), performed asynchronously with respect to changes in the data itself. In the second mode, the same metadata recording is performed, but strictly before making any changes. The third mode is equivalent to full logging (changes both in metadata and in the files themselves).

Only the last option ensures data integrity. The other two only speed up the detection of errors during the scan and guarantee restoration of the integrity of the file system itself, but not the contents of the files.

Journaling in NTFS is similar to the second logging mode in ext3. Only changes in metadata are recorded in the log, and the data itself may be lost in the event of a failure. This logging method in NTFS was not intended as a way to achieve maximum reliability, but only as a compromise between performance and fault tolerance. This is why people who are used to working with fully journaled systems consider NTFS pseudo-journaling.

The approach implemented in NTFS is in some ways even better than the default in ext3. NTFS additionally periodically creates checkpoints to ensure that all previously deferred disk operations are completed. Checkpoints have nothing to do with recovery points in \System Volume Information\ . These are just service log entries.

Practice shows that such partial NTFS journaling is in most cases sufficient for trouble-free operation. After all, even with a sudden power outage, disk devices do not lose power instantly. The power supply and numerous capacitors in the drives themselves provide just the minimum amount of energy that is enough to complete the current write operation. With modern SSDs, with their speed and efficiency, the same amount of energy is usually enough to perform pending operations. An attempt to switch to full logging would reduce the speed of most operations significantly.

Connecting third-party files in Windows

The use of file systems is limited by their support at the OS level. For example, Windows does not understand ext2/3/4 and HFS+, but sometimes it is necessary to use them. This can be done by adding the appropriate driver.

WARNING

Most drivers and plugins for supporting third-party file systems have their limitations and do not always work stably. They may conflict with other drivers, antiviruses, and virtualization programs.

An open driver for reading and writing ext2/3 partitions with partial support for ext4. The latest version supports extents and partitions up to 16 TB. LVM, access control lists, and extended attributes are not supported.

There is a free plugin for Total Commander. Supports reading ext2/3/4 partitions.

coLinux is an open and free port of the Linux kernel. Together with a 32-bit driver, it allows you to run Linux on Windows from 2000 to 7 without using virtualization technologies. Supports 32-bit versions only. Development of a 64-bit modification was canceled. CoLinux also allows you to organize access to ext2/3/4 partitions from Windows. Support for the project was suspended in 2014.

Windows 10 may already have built-in support for Linux-specific file systems, it's just hidden. These thoughts are suggested by the kernel-level driver Lxcore.sys and the LxssManager service, which is loaded as a library by the Svchost.exe process. For more information about this, see Alex Ionescu’s report “The Linux Kernel Hidden Inside Windows 10,” which he gave at Black Hat 2016.

ExtFS for Windows is a paid driver produced by Paragon. It runs on Windows 7 to 10 and supports read/write access to ext2/3/4 volumes. Provides almost complete support for ext4 on Windows.

HFS+ for Windows 10 is another proprietary driver produced by Paragon Software. Despite the name, it works in all versions of Windows starting from XP. Provides full access to HFS+/HFSX file systems on disks with any layout (MBR/GPT).

WinBtrfs is an early development of the Btrfs driver for Windows. Already in version 0.6 it supports both read and write access to Btrfs volumes. It can handle hard and symbolic links, supports alternative data streams, ACLs, two types of compression and asynchronous read/write mode. While WinBtrfs does not know how to use mkfs.btrfs, btrfs-balance and other utilities to maintain this file system.

Capabilities and limitations of file systems: summary table

File system	Maximum volume size	Limit size of one file	Length of proper file name	Length of the full file name (including the path from the root)	Limit number of files and/or directories	Accuracy of file/directory date indication	Rights dos-tu-pa	Hard links	Symbolic links	Snap-shots	Data compression in the background	Data encryption in the background	Grandfather-ple-ka-tion of data
FAT16	2 GB in 512 byte sectors or 4 GB in 64 KB clusters	2 GB	255 bytes with LFN	—	—	—	—	—	—	—	—	—	—
FAT32	8 TB sectors of 2 KB each	4 GB (2^32 - 1 byte)	255 bytes with LFN	up to 32 subdirectories with CDS	65460	10 ms (create) / 2 s (modify)	No	No	No	No	No	No	No
exFAT	≈ 128 PB (2^32-1 clusters of 2^25-1 bytes) theoretical / 512 TB due to third-party restrictions	16 EB (2^64 - 1 byte)			2796202 in the catalog	10 ms	ACL	No	No	No	No	No	No
NTFS	256 TB in 64 KB clusters or 16 TB in 4 KB clusters	16 TB (Win 7) / 256 TB (Win 8)	255 Unicode characters (UTF-16)	32,760 Unicode characters, up to a maximum of 255 characters per element	2^32-1	100 ns	ACL	Yes	Yes	Yes	Yes	Yes	Yes
HFS+	8 EB (2^63 bytes)	8 EB	255 Unicode characters (UTF-16)	not limited separately	2^32-1	1 s	Unix, ACL	Yes	Yes	No	Yes	Yes	No
APFS	8 EB (2^63 bytes)	8 EB	255 Unicode characters (UTF-16)	not limited separately	2^63	1 ns	Unix, ACL	Yes	Yes	Yes	Yes	Yes	Yes
Ext3	32 TB (theoretically) / 16 TB in 4 KB clusters (due to limitations of e2fs programs)	2 TB (theoretically) / 16 GB for older programs	255 Unicode characters (UTF-16)	not limited separately	—	1 s	Unix, ACL	Yes	Yes	No	No	No	No
Ext4	1 EB (theoretically) / 16 TB in 4 KB clusters (due to limitations of e2fs programs)	16 TB	255 Unicode characters (UTF-16)	not limited separately	4 billion	1 ns	POSIX	Yes	Yes	No	No	Yes	No
F2FS	16 TB	3.94 TB	255 bytes	not limited separately	—	1 ns	POSIX, ACL	Yes	Yes	No	No	Yes	No
BTRFS	16 EB (2^64 - 1 byte)	16 EB	255 ASCII characters	2^17 bytes	—	1 ns	POSIX, ACL	Yes	Yes	Yes	Yes	Yes	Yes

In this article we'll figure it out what features does ReFS provide and why is it better than the NTFS file system?. How to recover data from ReFS disk space. Microsoft's new ReFS file system was originally introduced in Windows Server 2012. It is also included in Windows 10 as part of the Disk Space tool. ReFS can be used for a disk pool. The file system has been improved with the release of Windows Server 2016 and will soon be available in the new version of Windows 10.

What features does ReFS provide and how is it better than the current NTFS system?

Content:

What does ReFS mean?

Short for "Resilient File System", ReFS is a new system based on NTFS. At this stage, ReFS does not offer a comprehensive replacement for NTFS for home users' disk use. The file system has its advantages and disadvantages.

ReFS is intended to solve the main problems of NTFS. It is more resilient to data corruption, handles higher workloads better, and easily scales to very large file systems. Let's look at what this means?

ReFS protects data from corruption

The file system uses checksums for metadata and may also use checksums for file data. When reading or writing a file, the system checks the checksum to ensure it is correct. This enables detection of corrupted data in real time.

ReFS is integrated with the Disk Space feature. If you have configured a mirrored data store, ReFS will allow Windows to detect and automatically repair file system corruption by copying data from another drive. This feature is available in both Windows 10 and Windows 8.1.

If the file system detects damaged data that does not have an alternative copy for recovery, then ReFS will immediately delete such data from the disk. This does not require rebooting the system or unplugging the storage device, as is the case with NTFS.

The need to use the chkdsk utility completely disappears, since the file system is automatically corrected immediately when an error occurs. The new system is also resistant to other types of data corruption. NTFS, when writing file metadata, writes it directly. If there is a power outage or computer crash during this time, you will experience data corruption.

When metadata changes, ReFS creates a new copy of the data and associates the data with a file only after the metadata is written to disk. This eliminates the possibility of data corruption. This function is called copy-on-write; it is also present in other popular Linux OS systems: ZFS, BtrFS, as well as the Apple APFS file system.

ReFS removes some NTFS restrictions

ReFS is more modern and supports much larger volumes and longer file names than NTFS. In the long term, these are important improvements. In the NTFS file system, a file name is limited to 255 characters; in ReFS, a file name can contain up to 32768 characters. Windows 10 allows you to disable the character limit for NTFS file systems, but it is always disabled on ReFS volumes.

ReFS no longer supports short filenames in DOS 8.3 format. On an NTFS volume you can access C:\Program Files\ V C:\PROGRA~1\ to ensure compatibility with older software.

NTFS has a theoretical maximum capacity of 16 exabytes, while ReFS has a theoretical maximum capacity of 262,144 exabytes. Although it doesn't really matter now, computers are constantly evolving.

Which file system is faster ReFS or NTFS?

ReFS was not designed to improve file system performance over NTFS. Microsoft has made ReFS much more efficient in certain cases.

For example, when used with Disk Space, ReFS supports "real-time optimization". Let's say you have a storage pool with two disks, one provides maximum performance, the other is used for capacity. ReFS will always write data to the faster disk, ensuring maximum performance. In the background, the file system will automatically move large chunks of data to slower drives for long-term storage.

In Windows Server 2016, Microsoft has improved ReFS to provide better performance for virtual machine functions. The Microsoft Hyper-V virtual machine takes advantage of these benefits (in theory, any virtual machine can take advantage of ReFS).

For example, ReFS supports block cloning, which speeds up the process of cloning virtual machines and merging checkpoint operations. To create a copy of a virtual machine, ReFS only needs to write the new metadata to disk and provide a link to the existing data. This is because in ReFS, multiple files can point to the same underlying data on disk.

When a virtual machine writes new data to disk, it is written to a different location, but the original virtual machine data remains on disk. This greatly speeds up the cloning process and requires much less disk bandwidth.

ReFS also offers a new feature "rare VDL", which allows ReFS to quickly write zeros to a large file. This greatly speeds up the creation of a new, empty, fixed-size virtual hard disk (VHD) file. In NTFS this operation can take 10 minutes, in ReFS – a few seconds.

Why ReFS can't replace NTFS

Despite a number of advantages, ReFS cannot yet replace NTFS. Windows cannot boot from a ReFS partition and requires NTFS. ReFS does not support NTFS features such as data compression, file system encryption, hard links, extended attributes, data deduplication, and disk quotas. But unlike NTFS, ReFS allows you to perform full disk encryption using BitLocker, including the disk system structures.

Windows 10 does not allow you to format a partition in ReFS; this file system is only available within Disk Space. ReFS protects data used on pools of multiple hard drives from corruption. In Windows Server 2016, you can format volumes using ReFS instead of NTFS. Such a volume can be used to store virtual machines, but the operating system can still only boot from NTFS.

Hetman Partition Recovery allows you to analyze disk space managed by the ReFS file system using a signature analysis algorithm. Analyzing the device sector by sector, the program finds certain byte sequences and displays them to the user. Restoring data from ReFS disk space is no different from working with the NTFS file system:

Download and install the program;
Analyze the physical disk that is included in the disk space;
Select and save the files that need to be restored;
Repeat steps 2 and 3 for all disks included in the disk space.

The future of the new file system is rather unclear. Microsoft may improve ReFS to replace the legacy NTFS in all versions of Windows. At the moment, ReFS cannot be used everywhere and serves only certain tasks.