It's been a year since we looked in on the kernel lockdown patches; that's because things have been fairly quiet on that front since there was a loud and discordant dispute about them back then. But Matthew Garrett has been posting new versions over the last two months; it would seem that the changes that have been made might be enough to tamp down the flames and, perhaps, even allow them to be merged into the mainline.

The idea behind kernel lockdown is to supplement secure boot mechanisms to limit the ability of the root user to cause unverified, potentially malicious code to be run. The most obvious way to do that is to use the kexec subsystem to run a new kernel that has not been vetted by the secure boot machinery, but there are lots of other ways that root can circumvent the intent of (some) secure boot users. While the support for UEFI secure boot has been in the kernel for years, providing a way to restrict the root user after that point has always run aground.

In the real world, text is expressed in many languages using a wide variety of character sets; those character sets can be encoded in a lot of different ways. In the kernel, life has always been simpler; file names and other string data are just opaque streams of bytes. In the few cases where the kernel must interpret text, nothing more than ASCII is required. The proposed addition of case-insensitive file-name lookups to the ext4 filesystem changes things, though; now some kernel code must deal with the full complexity of Unicode. A look at the API being provided to handle encodings illustrates nicely just how complicated this task is.
The Unicode standard, of course, defines "code points"; to a first approximation, each code point represents a specific character in a specific language group. How those code points are represented in a stream of bytes — the encoding — is a separate question. Dealing with encodings has challenges of its own, but over the years the UTF-8 encoding has emerged as the preferred way of representing code points in many settings. UTF-8 has the advantages of representing the entire Unicode space while being compatible with ASCII — a valid ASCII string is also valid UTF-8. The developers implementing case independence in the kernel decided to limit it to the UTF-8 encoding, presumably in the hope of solving the problem without going entirely insane.

The API that resulted has two layers: a relatively straightforward set of higher-level operations and the primitives that are used to implement them. We'll start at the top and work our way down.

BFQ is a proportional-share I/O scheduler available for block devices since the 4.12 kernel release. It associates each process or group of processes with a weight, and grants a fraction of the available I/O bandwidth proportional to that weight. BFQ also tries to maximize system responsiveness and to minimize latency for time-sensitive applications. Finally, BFQ aims at boosting throughput and at running efficiently. A new set of changes has improved BFQ’s performance with respect to all of these criteria. In particular, they increase the throughput that BFQ reaches while handling the most challenging workloads for this I/O scheduler. A notable example is DBENCH workloads, for which BFQ now provides 150% more throughput. These changes also improve BFQ’s I/O control — applications start about 80% more quickly under load — and BFQ itself now runs about 10% faster.

Let’s start with throughput improvements and, to introduce them, let’s examine the main cause of throughput loss with BFQ.

One of the surest signs that the Linux Storage, Filesystem, and Memory-Management (LSFMM) Summit is approaching is the seasonal migration of memory-management developers toward the get_user_pages() problem. This core kernel primitive is necessary for high-performance I/O to user-space memory, but its interactions with filesystems have never been reliable — or even fully specified. There are currently a couple of patch sets in circulation that are attempting to improve the situation, though a full solution still seems distant.
get_user_pages() is a way to map user-space memory into the kernel's address space; it will ensure that all of the requested pages have been faulted into RAM (and locked there) and provide a kernel mapping that, in turn, can be used for direct access by the kernel or (more often) to set up zero-copy I/O operations. There are a number of variants of get_user_pages(), most notably get_user_pages_fast(), which trades off some flexibility for the ability to avoid acquiring the contended mmap_sem lock before doing its work. The ability to avoid copying data between kernel and user space makes get_user_pages() the key to high-performance I/O.

If get_user_pages() is used on anonymous memory, few problems result. Things are different when, as is often the case, file-backed memory is mapped in this way. Filesystems are generally responsible for the state of file-backed pages in memory; they ensure that changes to those pages are written back to permanent storage, and they make sure that the right thing happens when a file's layout on that storage changes. Filesystems are not designed to expect that file-backed pages can be written to without their knowledge, but that is exactly what can happen when those pages are mapped with get_user_pages().

Most of the time, things happen to work anyway. But if an I/O operation writes to a page while the filesystem is, itself, trying to write back changes to that page, data corruption can result. In some cases, having a page unexpectedly become dirty can cause filesystem code to crash. And there is a whole new range of problems that can turn up for filesystems on nonvolatile memory devices, where writing to a page directly modifies the underlying storage. Filesystems implementing this sort of direct access (a mode called "DAX") can avoid some problems by being careful to not move file pages around while references to them exist, but that leads to different kinds of problems if pages mapped with get_user_pages() remain mapped for long periods of time. Naturally, certain subsystems (notably the RDMA layer) do exactly that.

A Linux user's $PATH likely contains well over a thousand different commands that were installed by various packages. It's not immediately obvious which package is responsible for a command with a generic name, like createuser. There are ways to figure it out, of course, but perhaps it would make sense for packages like PostgreSQL, which is responsible for createuser, to give their commands names that are less generic—and more easily disambiguated—such as pg_createuser. But renaming commands down the road has "backward compatibility problems" written all over it, as a recent discussion on the pgsql-hackers mailing list shows.

Someone with the unlikely name of "Fred .Flintstone" started things off with a post complaining that PostgreSQL "pollutes the file system" with generic program names. The post suggested that names either be prefixed with "pg_" or that they become subcommands of a wrapper command, à la Git: postgresql createuser. It is not the first time that the topic has been raised, Andreas Karlsson pointed to this thread from 2008; Tom Lane reached further back and pointed to a discussion in 1999.

E8, the NVMe-oF array supplier, is backing up ThinkParQ’s clustered parallel file system to deliver files faster to high performance computing customers.

ThinkParQ’s product is called BeeGFS, an open source file system, and is an alternative to Lustre and Spectrum Scale, formerly known as GPFS. It has thousands of users worldwide.