Slice the apples. What's inside the iOS file system? The Four Most Exciting Features of the New Apple File System
09/26/2017, Tue, 09:50, Moscow time, Text: Vladimir Bakhur
The final version of macOS High Sierra for Apple Mac is now available for download. The list of the most significant changes includes a completely new file system, improved support for 4K, VR and HEVC, new features in the Safari browser and photo editor.
Final update
Apple has officially unveiled the final commercial version of its macOS operating system, tentatively titled High Sierra, for Mac systems. Previously, macOS High Sierra was only available as a beta version at the 2017 WWDC developer conference.
Apple emphasizes that the set of new technologies implemented in the new operating system will provide Mac systems with greater reliability, functionality and speed, and also “lays the foundation for future innovations.”
Among the key core technologies of the new macOS High Sierra, Apple especially highlights the new storage architecture, enhanced video streaming capabilities, and improved GPU efficiency.
Apple File System
The new Apple File System (APFS), which first came to High Sierra to replace the 30-year-old “veteran” of the HFS/HFS+ family of file systems, has a 64-bit architecture and allows you to work with high-capacity flash drives while maintaining backward compatibility for reading and writing from HFS formatted media.
New 64-bit Apple File System
According to the company, the new file system performs most common operations much faster, such as determining the size of folders or copying files. APFS's list of benefits also includes built-in encryption, failover protection, and simplified on-the-go backups.
New codecs
Apple has fully adapted the new High Sierra OS to work with the new HEVC (High Efficiency Video Coding) video codec, also known as H.265. This codec allows you to compress video data without noticeable loss of quality more efficiently than the previous generation of codecs (AVC/H264, AVCHD 2.0) and is aimed at working with 4K video format and even 8K in the future.
The HEVC codec provides a number of advantages over the H.264 standard
According to Apple's own data, at the current stage of development of the HEVC codec, it provides video compression up to 40% stronger than with the H.264 codec, so HEVC video files take up less space without sacrificing quality and are transmitted faster.
Metal 2: advanced graphics and virtuality
The new version of Metal 2 technology integrated into macOS High Sierra has full support for machine learning, external discrete graphics and virtual reality.
Mac users running macOS High Sierra will now be able to experience a variety of VR experiences with the first support for Valve's SteamVR headset and HTC Vive VR headset.
Metal 2 technology supports external graphics and virtual reality
Mac users can now create interactive virtual reality experiences on the new iMac with Retina 5K display and the new iMac Pro launching late 2017, as well as other supported Mac models when connected to an external graphics card. Particular attention is paid to a number of Apple applications for creating visual content, including Final Cut Pro X, Epic Unreal 4 Editor and Unity Editor.
Advanced App Features
In macOS High Sierra, Apple introduced a significantly redesigned Photos app for organizing and editing your photo collections, linked to iCloud Photo Library.
The Photos application has improved functionality
The Photos app features a redesigned side menu, expanded and expanded editing features, including Memories, Moments, Collections, and Years for organizing and curating your favorite photos, People with a facial recognition tool, and Places » with the location of the photo on the world map and search by keywords.
The Markup feature lets you add text, graphics, sketches, and captions to your images. The Enhance feature allows you to enhance the quality of your photo with one click, and numerous filters allow you to automatically change the style and make professional-grade edits using smart sliders. Additional third-party photo enhancements, filters, and textures are available in the Mac App Store.
New features in the Photos app
You can turn Live Photos into looping videos using the Loop effect and then share them on social media. The Pendulum effect lets you play video back and forth, while the Long Exposure effect lets you simulate a long shutter speed on a DSLR camera to blur the texture of water or lengthen the rays of light.
After editing a photo in Photoshop or Pixelmator, the changes are automatically saved in the Photos app's library. After taking a photo or video on your iPhone, the footage automatically goes to the iCloud Photo Library and becomes available for viewing on Mac, iOS devices, Apple TV and PC.
Record FaceTime video calls as Live Photos
The macOS High Sierra operating system has learned to record FaceTime video calls in Live Photo format. After the communication session, both subscribers receive a notification about the created photo, and the Live Photo is saved on the device in the Photos library.
Organizing an iCloud cloud library
iCloud Photo Library provides 5 GB of free storage, with the option of further paid upgrades. With Optimize Mac Storage, all high-resolution photos and videos will be stored exclusively on iCloud, while smaller versions will remain on your Mac.
New functionality for the Mail app
The search experience in the Mail application has been significantly improved and expanded: it has now become faster, more convenient, and has the ability to move the most relevant letters to the top of the list with the “Best Matches” function. The Mail search takes into account the hierarchy of contacts, correspondence read and sent, and a number of other factors. Apple notes that search in Mail is now also self-learning, and works better with frequent use of this feature.
Safari: now with smart protection and without traces
According to Apple's own data, the Safari browser on macOS is again the fastest and outperforms Chrome in most tests. Introduced in macOS High Sierra, Intelligent Tracking Prevention uses machine learning to help Safari recognize who is tracking your online activity and delete the data collected.
Safari browser gets machine learning feature
Safari has also learned how to mute the sound when playing advertising videos. If necessary, launching videos with sound can be enabled by default in individual browser settings, or for individual websites.
For certain sites, you can also configure Safari's zoom level, use geolocation, show notifications, and block content.
Web pages that support Reader view automatically open in Safari without ads, navigation buttons, or other distractions.
Siri: more humane
In the new macOS High Sierra, the timbre and intonation of Siri’s “voice” have become more natural and varied, and semantic accents have become more precise.
Siri continues to improve its artificial intelligence and now, using built-in machine learning, remembers the user's musical tastes when listening to music on Apple Music, and can then suggest something new in the same style.
Enhanced Siri functionality in macOS High Sierra
Now you can ask Siri to play something calm. Siri can also read the description of a song and provide information about new songs and artists.
Compatibility and updates
The new macOS is available now as a free update or direct download from the Mac App Store.
The new macOS High Sierra has similar system requirements as its predecessor, macOS Sierra. Thus, all systems running Sierra can be upgraded to High Sierra.
Apple says that all MacBook and iMac models released after 2009 are fully compatible with the new macOS High Sierra operating system, as well as the MacBook Air platforms released after 2010, and the MacBook Pro, Mac mini and Mac Pro introduced after mid-2010
Denotes Apple's new file system and will eventually be used on all of the company's major platforms. But why does Apple use it? What's so special about it? This article is devoted precisely to this topic.
Apple File System
The abbreviation APFS stands for Apple File System. It replaces the file system and over time this file system will become an integral part of iOS, tvOS, and watchOS - that is, it will be used across the entire line of devices: from computers, iPhone, iPad, Apple TV and to the Apple Watch device.
Currently, iOS 10.3 is the only platform available to everyone that uses the APFS file system.
APFS can be integrated on all platformsApple.
The HFS+ file system appeared in 1998 and is now almost 20 years old. She has become a relic of a bygone era. It was created in the era of Mac computers and modified for use in iPhone, Apple Watch and Apple TV devices.
But a lot has changed over the past 20 years. We have switched from hard drives to SSDs; the software is no longer distributed on CDs, but is simply downloaded. Our devices are now backed up in cloud services.
Optimized for working with flash drives
These days, almost every new device Apple sells runs on a flash drive, including most Mac computers. And APFS was designed to work with flash-based storage devices. Therefore, we will see improvements in parameters such as read and write speeds, and an increase in overall reliability when it comes to storing and transferring information.
main feature: System snapshots and cloning
The two main features of APFS are the ability to take snapshots of the system and the ability to clone data.
A snapshot represents a one-time read-only state of the file system.
Using clones, APFS will be able to create quick copies of the same partition without using additional disk space.
Minimum waiting time
APFS has minimal latency. This means that operations such as launching applications and loading data will be noticeably faster. Which in turn will lead to a reduction in waiting time - we will see less of the spinning wait wheel in iOS and the beach ball icon in Mac OS.
Availableflowing disksections
If you use disk partitions on your Mac, then APFS can make your life a little easier. If one partition runs out of free disk space, APFS can automatically use the free space on another partition because APFS creates special containers around each disk partition.
Encryption is everything
APFS was primarily designed for strong encryption. It supports both single key and multiple encryption keys.
Developments for the future
HFS+ supports 32-bit metadata file. APFS supports 64-bit inode numbers. APFS was designed to work in the future and will perform even better over time as appropriate updates are released.
FirmwareiOS 10.3 gives you back disk space
Switching to APFS has one clear advantage. You have disk space at your disposal - the amount of space saved will depend on how full your disk is. Some managed to get a couple of gigabytes back after updating.
APFS for Mac OS still exists in beta version
iOS already uses the new file system, but for Mac OS the new file system is only available in a very early beta version. And Apple will have to make a lot of effort to convert Mac computers to use APFS. Access to the file system in iOS is closed to outsiders, but Mac OS uses an open, editable file system. In other words, APFS in practice may encounter various edge cases that Apple experts are not even aware of yet. We hope that all these issues will be resolved during the beta testing period. You can try the beta version in action. True, it is not quite ready for everyday use. APFS does not support boot drives, Time Machine, FileVault encryption, or Fusion drives.
Yes, you need to update toiOS 10.3
Currently, iOS 10.3 is the only way to try APFS in action. And despite the fact that this is a completely new file system, it is nevertheless worth switching to. This update has been tested for several weeks by a large number of beta testers and everything speaks in favor of switching to the updated version. The transition itself occurs quite smoothly, quickly and without data loss. However, before updating, we recommend, just in case, to back up your data using the appropriate iTunes application services or iCloud cloud storage (which is recommended in any case).
How do you like itAPFS?
At the last WWDC, Apple introduced a new file system - Apple File System, or APFS. Although HFS+ is used on all Apple products (and will be used for at least another year, APFS is scheduled for release in 2017), HFS+ does not meet the requirements of modern file systems. It is over 30 years old and was created for a time when floppy disks and hard drives were considered the latest technology. To use HFS+ on iOS, tvOS and watchOS, Apple had to add completely third-party functionality that is not supported on macOS: for example, encryption for each individual file.
The new APFS has added so many features that it’s dizzying: optimization for Flash and SSD, protection against data loss during crashes by increasing the number of atomic operations, built-in support for several types of encryption, greatly reduced response time, quick estimation of the size of files and folders - the list goes on and on. In this article, we'll look at four of what we think are the most interesting features of the new file system.
Secure Copy-on-write
First, we'll have to understand a few key terms. Atomic operations are operations that are performed inseparably on a file system or database and cannot be simplified. An atomic operation cannot remain unfinished if it is interrupted. Such an operation is either performed completely or not at all. POSIX-compliant secure atomic file storage is performed as follows - when we work with an application and save a file, data from memory is downloaded to a temporary file on disk. When the application is confident that the data has been downloaded completely and without errors, the application queries the file system about renaming. The renaming operation is atomic; it will either be completed completely or not at all. When renaming, what happens is that the file system moves the temporary file to the save location (say from /var/tmp/Batcave_shopping.pages to ~fedor/Documents/Batcave_shopping.pages) and then deletes the old file.
However, what if we are saving not a file, but a folder or Bundle (for example, .rtfd or .app, also, in essence, folders)? Let's say we are working with the document ~fedor/Documents/AlfredHappyBirthday.rtfd. This is a Bundle, it contains text and several images. When we make changes and save them, the application overwrites the entire Bundle, not just the changed text or image. POSIX requirements do not allow you to rename a folder somewhere where there is already data, so you have to resort to tricks. The document is moved from ~fedor/Documents/ to free up space, then the temporary file in /var/tmp/AlfredHappyBirthday.rtfd is renamed to the empty space. This operation is non-atomic and unsafe. If a system error occurs or the computer loses power during an operation, the document will be lost.
Obviously, such a solution is undesirable. Therefore, APFS introduced a new, POSIX-incompatible atomic safe folder save operation. When an application requests to save a folder, the temporary and permanent files are swapped. This is done by the file system and is an atomic operation.
This is the secure Copy-on-write feature - one of the most important innovations in APFS.
Cloning and snapshots
If you need a copy of a file or folder, you right-click on it and select Duplicate. In HFS+, this means that the file system reads all data and then writes it to free space. This approach is an inefficient use of space and wastes the disk and processor load. In APFS, when you duplicate data - via Duplicate or simply ⌘C, ⌘V - the file system copies only the link to it (the so-called hard link), and the content is not duplicated. This operation is called cloning. Thanks to cloning, a duplicate data or folder appears almost instantly and takes up a negligible amount of free space. If you make changes to the original or clone, the file system will record the changes and redirect the link accordingly, leaving the original piece of data untouched. In APFS, the cloning operation is atomic.
Snapshots is a specific snapshot of a volume that stores information about the data on that volume at a specific point in time. The easiest way to understand how it works is with an example. Let's say we have a volume (let's say a flash drive) with two presentations. We create a Snapshot of this flash drive, it contains two files Joker.key and Riddler.key, the following data blocks belong to the files. The Snapshot itself does not contain data from the flash drive - only what files were on it at the time the snapshot was created and what blocks they belonged to. The peculiarity of Snapshot is that blocks written to it are closed. They cannot be deleted, moved or overwritten. Therefore, for example, if we make changes to Joker.key and delete Riddler.key altogether, the original data blocks will remain intact. Although this approach takes up additional free space, it allows APFS to almost instantly return the volume back in time to the point at which any snapshot was taken.
Space sharing
Space sharing is another very interesting feature of APFS. Once again, the easiest way to understand it is with an example. Let's say you have a laptop with a 256GB SSD installed (it's very possible that you actually have one) and you want to install OS X El Capitan and macOS Sierra beta on it. Alas, Apple doesn't yet allow you to boot the operating system from an APFS volume (or use it for Time Machine, for that matter), so you'll need to create two HFS+ partitions. You launch Disk Utility and you are faced with a dilemma - how big should you make the partitions? You can simply split the drive in half, but what if you don't have enough space to run El Capitan comfortably? Or, on the contrary, will you overdo it and the applications you need won’t fit in Sierra?
APFS solves this problem with Space sharing. On a disk with APFS volumes at a higher level, a container is created containing all the volumes. They can freely and dynamically grow or shrink within such a container, without requiring the user to choose how large they should be. Let's return to our hypothetical laptop. If APFS were used instead of HFS+ for both volumes, both operating systems would report that 256 GB of space is available for each. In this case, the space occupied by files on one system would be inaccessible to another. This solution greatly simplifies the work with multiple volumes and allows you to get the maximum benefit from their use without much difficulty.
Migration from HFS+
Finally, how can we migrate to APFS? The process of switching to a file system is usually difficult, long and not always safe. You have to back up your data to an external drive, erase your computer's hard drive, write a new file system, install the OS, restore your data from the copy and hope that nothing has disappeared from its place.
Fortunately, Apple took care of this. The company's engineers did a lot of work on the migration process. The process consists almost entirely of atomic operations, but the main thing is that data about the new file system is written only to empty space on the HFS+ volume. This means that even if something goes wrong, it will not damage either your data or the structure of your existing file system. And according to Apple engineers, the entire migration takes just a few minutes.
Here we come to the end. We only talked about four functions, but there are many more in APFS. If you want to dive into the details, the official documentation has all the details. The new file system, as we mentioned above, cannot be used for Startup Disk or Time Machine, and will not be supported on OS X Yosemite and older. The official release of APFS is scheduled for 2017.
Yesterday, Apple showed new versions of the operating systems macOS (Sierra) 10.12, iOS 10, tvOS 10, watchOS 3, an application for teaching children programming Swift Playgrounds and new emoji.It would seem that there is nothing interesting. However, Apple still rolled out something fundamental. The most significant development of all mentioned at the presentation is the next-generation Apple File System (APFS) file system in the macOS (Sierra) 10.12 operating system.
Documentation with the main characteristics and description of the file system, which repeats individual functions of the powerful free file system ZFS, was published on the website for developers shortly after the presentation.
Currently, Apple computers use the HFS+ file system, an extended version of HFS (Hierarchical File System, hierarchical file system), created more than 30 years ago. Like its predecessor, HFS+ uses a tree structure called a B* tree to store most of the metadata. Hence the name “hierarchical file system”.
The official introduction of HFS+ took place on January 19, 1998, along with MacOS 8.1. Since 2002, the system has implemented logging to increase the reliability of information storage. Since version OS X 10.3, logging is enabled by default, and it is now possible to work in case-sensitive mode.
Up until OS X 10.7, developers continued to refine HFS+ and implement new features for OS X at the file system level. But the fact remains: HFS was originally developed in the days of floppy disks and spinning hard drives, when file sizes were measured in kilobytes or megabytes. Today, many people work with SSD drives, where millions of files are stored - gigabytes or terabytes of data. The file system has completely different requirements. Instead of reworking the old code, Apple decided to finally write a new file system from scratch.
The next generation APFS file system is still at the stage developer preview, that is, it is not planned to be rolled out into mass use in the near future. An APFS volume cannot currently be used as a boot drive, nor can it be used with Time Machine backup, Fusion Drive, or File Vault encryption. But it is possible for a regular non-bootable volume.
There is still a lot of development and testing ahead, but only then APFS will become Apple's main file system for decades to come.
APFS, unlike HFS+, inherently distinguishes the case of characters in file and folder names, and this feature cannot be disabled. This should be kept in mind by anyone who decides to use APFS.
Main characteristics
The official documentation lists the general characteristics of the APFS file system in comparison with HFS+.Containers and volumes
Container is the main object for storing data in APFS. Containers are usually exactly the same as GUID Partition Table (GPT) entries, and they have their own failure protection and disk space allocation scheme. Each container contains one or more volumes or file systems, each of which has its own namespace, that is, a set of files and directories.APFS does not directly support software RAID, but it can be used with Apple RAID volumes to support Striping (RAID 0), Mirroring (RAID 1), and Concatenation (JBOD).
64-bit inodes
64-bit inodes significantly increase the namespace compared to 32-bit inodes in HFS+. The 64-bit APFS file system supports more than 9 quintillion files on each volume. This should be enough for everyone, as Bill Gates said.Nanosecond timestamps
APFS has significantly increased the accuracy of timestamps. APFS supports timestamping with nanosecond precision. For comparison, in HFS+, time stamps were set with an accuracy of up to a second.Nanosecond timestamps are very important in modern file systems because they help implement atomicity and atomic transactions - one of the main ACID requirements for a transactional system (for example, a DBMS). Atomicity ensures that no transaction is partially committed to the system. Either all of its sub-operations will be performed, or none will be performed.
Failure protection
APFS implements an innovative copy-on-write metadata scheme that Apple calls “Crash Protection.” It ensures that file system changes and log writes remain synchronized if something happens while the write is in progress, such as a power failure.Copy-on-write scheme in ZFS
Sparse files
A file with the “sparse” attribute assumes that it contains blocks of zero bytes that are not stored on the drive, but are implied. HFS+ did not have support for sparse files.Extended Attributes
APFS has built-in support for extended file attributes, which in HFS+ was implemented through the Attributes file, that is, through a B-tree.Encryption
Apple says encryption is a fundamental property that is built into APFS at the file system level. For each volume in an APFS container, one of the encryption models is set: no encryption, single-key encryption, or multiple-key encryption. In the latter case, separate keys are used to encrypt files and metadata. Depending on the hardware, APFS uses AES-XTS or AES-CBC encryption mode.Cloning files and directories
Cloning is an almost instantaneous copying of a file or directory, which does not require additional data storage space. When a clone is modified, the file system records only the data change. This way, the new file system can store many versions of large files while taking up less disk space.Snapshots
Snapshots are read-only snapshots of the file system on a volume. The operating system can use snapshots for a more efficient backup procedure. That is, finally Time Machine will work normally (fast).Of course, APFS is significantly inferior in its capabilities to the 128-bit file system ZFS, which is supported by Linux, FreeBSD and other free operating systems, but on Apple's part this is a step in the right direction.
It is strange that the preliminary documentation does not mention the compression function, which HFS+, by the way, supports.
Apple has been trying for a long time to port ZFS to the OS X system, there was an active discussion about this on the ZFS mailing lists, preliminary snapshots were published for the next version of OS X. Later, an implementation of OpenZFS was made for OS X (O3X) and MacZFX.
The ZFS file system is open source, and Apple may well have borrowed some ideas from the APFS file system. An open source implementation for APFS is not yet ready; Apple plans to document and publish the APFS format in 2017.
The first formal session will take place at WWDC this evening, where the new APFS capabilities will be demonstrated to developers in more detail.
When working with a jailbroken iOS device, problems may arise that can only be solved by modifying the files. And for this you need to know the basic structure of the file system, understand where everything is and which files are responsible for what, where programs and tweaks are installed and how they interact with each other. We'll talk about all this.
Main directories and files
iOS is a UNIX-like operating system and uses a very similar file system structure to UNIX and OS X. The "folder" here is called a "directory", and the file system "grows" from the root /. The ~ sign denotes the user's home directory. In normal mode, this is the directory /var/mobile/, in root user mode - /var/root. Some directories are standard for UNIX systems. This is /boot - here in UNIX the system kernel and RAM disk are located (in iOS the kernel is located in the /System/Library/Caches/com.apple.kernelcaches/kernelcache directory), /etc - settings for low-level services, /tmp - temporary files, /bin - commands to run using the terminal, /mnt - mount point for external file systems (flash drives, etc. are connected here).
The directories that are most interesting to us are /System, /Library and /var. This is where the operating system itself (the first directory), system data (the second), application settings and their data (the third) are stored.
Standard (pre-installed) applications are located in the /Applications directory. The executable files of Cydia, Zeusmos and some other .app programs are also located here, the installation of which requires a jailbreak. There are many more files here than there are icons on the iOS desktop, since some internal services are also located here, separated as separate applications (Facebook service built into iOS, Print Center and some others). On an unjailbroken device, this directory is updated only when the entire firmware is updated, but some applications from Cydia are installed in it, like the Cydia store itself.
Applications downloaded from the App Store are stored in /var/mobile/Containers/Bundle/Application , each in its own subdirectory. The names of these subdirectories are encoded, and in order to understand what kind of application it is, you need to go into the directory and move to the next one. Applications installed from outside the App Store are also usually found here. Each application is assigned a subdirectory in the /var/mobile/Containers/Data/Application directory, within which the application stores its settings and data created during operation. We will look at its structure in more detail later.
System updates are downloaded to the /var/mobile/MobileSoftwareUpdate directory. They can be deleted through the standard Settings application. All wallpapers are stored in /Library/Wallpaper, and system sounds are stored in /System/Library/Audio/UISounds, music and videos from the computer are stored in /var/mobile/Media/iTunes_Control/Music.
The directory /var/mobile/Library/caches/com.saurik.Cydia is especially worth mentioning. This is where tweak deb packages are stored until Springboard is restarted. If you need to download a tweak to your computer, you can get it from here. Also, during installation, tweaks often create additional directories where they store their own data. Usually such directories are mentioned in the tweaks' documentation.
Programs for working with the device’s FS directly
There are several programs for working with the FS of a device after jailbreak.
- iTools is a program for Mac and Windows that allows you to add and delete files to the file system. This is where its functions end. An advantage for many may be the bookmarks bar to the left of the FS viewing window, where you can find programs, tweaks, ringtones or device wallpapers.
- iFunBox - has significantly more features. With its help you can create directories, delete, rename files or export them to a PC. The only thing that is surprising is the lack of search in the FS.
- Filza File Manager - allows you to view file metadata directly on the device, rename, move, delete them, and open many types of files. There is a built-in search for folders and simultaneous work with several files.
- iFile is superior to Filza File Manager in many ways. Some features are available only after paying for the full version of the program, but the interface in the utility is more convenient and understandable.
Of course, you can also work with the file system using the terminal. There is full support for UNIX commands, so managing the FS will be very fast and convenient.
Application directories and sandboxes
Continuation is available only to subscribers
Option 1. Subscribe to Hacker to read all materials on the site
Subscription will allow you to read ALL paid materials on the site within the specified period.
