Linux Kernel and Linux Foundation

-
Preparing for the realtime future
Unlike many of the previous gatherings of the Linux realtime developers, their microconference at the virtual 2020 Linux Plumbers Conference had a different feel about it. Instead of being about when and how to get the feature into the mainline, the microconference had two sessions that looked at what happens after the realtime patches are upstream. That has not quite happened yet, but is likely for the 5.10 kernel, so the developers were looking to the future of the stable realtime trees and, relatedly, plans for continuous-integration (CI) testing for realtime kernels.
-
Profile-guided optimization for the kernel
One of the many unfortunate consequences of the Covid-19 pandemic was the cancellation of the 2020 GNU Tools Cauldron. That loss turned out to be a gain for the Linux Plumbers Conference, which was able to add a GNU Tools track to host many of the discussions that would have otherwise occurred at Cauldron. In that track, Ian Bearman presented his group's work using profile-guided optimization with the Linux kernel. This technique, which he often referred to as "pogo", is not straightforward to apply to the kernel, but the benefits would appear to justify the effort.
Bearman is the leader of Microsoft's GNU/Linux development-tools team, which is charged with supporting those tools for the rest of the company. The team's responsibilities include ensuring the correctness, performance, and security of those tools (and the programs generated by them). Once upon a time, the idea of Microsoft having a GNU tools team would have raised eyebrows. Now, he said, about half of the instances in the Microsoft cloud are running Linux, making Linux a big deal for the company; it is thus not surprising that the company's cloud group is his team's biggest customer.
There was recently, he said, an internal customer working on a new Linux-based service that asked his team for performance help. After some brainstorming, the group concluded that this would be a good opportunity to use profile-guided optimization; the customer would have control of the whole machine running the service and was willing to build a custom kernel, making it possible to chase performance gains at any level of the system. But there was a small problem in that the customer was unable to provide any code to allow workload-specific testing.
-
Conventions for extensible system calls
The kernel does not have just one system call to rename a file; instead, there are three of them: rename(), renameat(), and renameat2(). Each was added when the previous one proved unable to support a new feature. A similar story has played out with a number of system calls: a feature is needed that doesn't fit into the existing interfaces, so a new one is created — again. At the 2020 Linux Plumbers Conference, Christian Brauner and Aleksa Sarai ran a pair of sessions focused on the creation of future-proof system calls that can be extended when the need for new features arises.
Brauner started by noting that the problem of system-call extensibility has been discussed repeatedly on the mailing lists. The same arguments tend to come up for each new system call. Usually, developers try to follow one of two patterns: a full-blown multiplexer that handles multiple functions behind a single system call, or creating a range of new, single-purpose system calls. We have burned ourselves and user space with both, he said. There are no good guidelines to follow; it would be better to establish some conventions and come to an agreement on how future kernel APIs should be designed.
The requirements for system calls should be stronger, and they should be well documented. There should be a minimal level of extensibility built into every new call, so that there is never again a need to create a renameat2(). The baseline, he said, is a flags argument; that convention is arguably observed for new system calls today. This led to a brief side discussion on why the type of the flags parameter should be unsigned int; in short, signed types can be sign extended, possibly leading to the setting of a lot of unintended flags.
Sarai took over to discuss the various ways that exist now to deal with system-call extensions. One of those is to add a new system call, which works, but it puts a big burden on user-space code, which must change to make use of this call. That includes checking to see whether the new call is supported at all on the current system and falling back to some other solution in its absence. The other extreme, he said, is multiplexers, which have significant problems of their own.
-
Lua in the kernel?
BPF is, of course, the language used for network (and other) customization in the Linux kernel, but some people have been using the Lua language for the networking side of that equation. Two developers from Ring-0 Networks, Lourival Vieira Neto and Victor Nogueira, came to the virtual Netdev 0x14 to present that work. It consists of a framework to allow the injection of Lua scripts into the running kernel as well as two projects aimed at routers, one of which is deployed on 20 million devices.
Neto introduced the talk by saying that it was also based on work from Ana Lúcia de Moura and Roberto Ierusalimschy of the Pontifical Catholic University of Rio de Janeiro (PUC-Rio), which is the home organization of the Lua language. They have been working on kernel scripting since 2008, Neto said, developing the Lunatik framework for Linux. It allows kernel developers to make their subsystems scriptable with Lua and also allows users to load and run their Lua scripts in the kernel.
-
OpenZFS 2.0-RC2 Released With Dozens Of Fixes
Nearly one month ago OpenZFS 2.0 saw its first release candidate while now it's been succeeded by another test candidate in time for some weekend exposure.
OpenZFS 2.0 is a huge update for this open-source ZFS file-system implementation in that it mainlines FreeBSD support alongside Linux, there is Zstd compression support, many performance optimizations, fast clone deletion, sequential resilvering, and a lot of other improvements and new features.
-
New /dev/random Implementation Hits 35th Revision
Going on for more than four years now has been creating a new /dev/random implementation and this Friday marks the 35th revision to this big set of patches that aim for better performance and security.
The code has been through many changes over the years for this new "Linux Random Number Generator" (LRNG).
-
Linux 5.10 To Support AMD SME Hardware-Enforced Cache Coherency
Linux 5.10 is set to support a new feature of AMD Secure Memory Encryption (SME) as part of the Secure Encrypted Virtualization (SEV).
-
Linux 5.9 To Allow Controlling Page Lock Unfairness In Addressing Performance Regression
Following the Linux 5.0 to 5.9 kernel benchmarks on AMD EPYC and it showing the in-development Linux 5.9 kernel regressing in some workloads, bisecting that issue, and that bringing up the issue of the performance regression over page lock fairness a solution for Linux 5.9 has now landed.
[...]
Long-term Linus Torvalds and other upstream developers will be looking at further improving the page lock behavior, but merged today for Linux 5.9 was a short-term solution. The change is allowing a controlled amount of unfairness in the page lock.
-
Notes from an online free-software conference
An online event requires an online platform to host it. The Linux Foundation, which supports LPC in a number of ways, offered a handful of possibilities, all of which were proprietary and expensive. One cannot blame the Linux Foundation for this; the events group there was under great pressure with numerous large events going up in flames. In such a situation, one has to grasp at whatever straws present themselves. We, though, had a bit more time and a strong desire to avoid forcing our attendees onto a proprietary platform, even if the alternative required us to build and support a platform ourselves.
Research done in those early days concluded that there were two well-established, free-software systems to choose from: Jitsi and BigBlueButton. Either could have been made to work for this purpose. In the end, we chose BigBlueButton for a number of reasons, including better-integrated presentation tools, a more flexible moderation system, and a more capable front-end system (though, as will be seen, we didn't use that part).
BigBlueButton worked out well for LPC, but it must be said that this system is not perfect. It's a mixture of highly complex components from different projects glued together under a common interface; its configuration spans literally hundreds of XML files (and some in other formats). It only runs on the ancient Ubuntu 16.04 distribution. Many features are hard to discover, and some are outright footguns: for moderators, the options to exit a meeting (leaving it running) and to end the meeting (thus kicking everybody else out, disposing of the chat session, and more) are adjacent to each other on the menu and look almost identical. Most worryingly, BigBlueButton has a number of built-in scalability limitations.
The FAQ says that no BigBlueButton session should have more than 100 users — a limitation that is certain to get the attention of a conference that normally draws around 600 people. A lot of work was done to try to find out what the real limitations of the platform were; these included automated testing and running a couple of "town hall" events ahead of the conference. In the end, we concluded that BigBlueButton would do the job if we took care not to stress it too hard.
-
September 2020 Linux Foundation Newsletter
-
Open Source Collaboration is a Global Endeavor, Part 2
The Linux Foundation would like to reiterate its statements and analysis of the application of US Export Control regulations to public, open collaboration projects (for example, open source software, open standards, open hardware, and open data) and the importance of open collaboration in the successful, global development of the world’s most important technologies.
Today’s announcement of prohibited transactions by the Department of Commerce regarding WeChat and TikTok in the United States confirms our initial impact analysis for open source collaboration. Nothing in the orders prevents or impacts our communities’ ability to openly collaborate with two valued members of our open source ecosystem, Tencent and ByteDance. From around the world, our members and participants engage in open collaboration because it is open and transparent, and those participants are clear that they desire to continue collaborating with their peers around the world. -
Linux Foundation Certified IT Administrator Exam To Be Launched Soon
-
Linux Foundation launches new entry-level IT certification
If you're Linus Torvalds, you don't need a certification to get a job. People know who you are. But most of us trying to get a start in technology need a certification. Now, The Linux Foundation, the nonprofit, open-source powerhouse organization, and Certiverse, a certification testing startup, have announced they're working on a new entry-level IT certification offering: The Linux Foundation Certified IT Associate (LFCA).
-

- Login or register to post comments
Printer-friendly version- 2854 reads
PDF version
More in Tux Machines
- Highlights
- Front Page
- Latest Headlines
- Archive
- Recent comments
- All-Time Popular Stories
- Hot Topics
- New Members
digiKam 7.7.0 is released
After three months of active maintenance and another bug triage, the digiKam team is proud to present version 7.7.0 of its open source digital photo manager. See below the list of most important features coming with this release.
|
Dilution and Misuse of the "Linux" Brand
|
Samsung, Red Hat to Work on Linux Drivers for Future Tech
The metaverse is expected to uproot system design as we know it, and Samsung is one of many hardware vendors re-imagining data center infrastructure in preparation for a parallel 3D world.
Samsung is working on new memory technologies that provide faster bandwidth inside hardware for data to travel between CPUs, storage and other computing resources. The company also announced it was partnering with Red Hat to ensure these technologies have Linux compatibility.
|
today's howtos
|








.svg_.png)
Content (where original) is available under CC-BY-SA, copyrighted by original author/s.

Recent comments
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago
1 year 11 weeks ago