The purpose of a cpuidle governor is to decide which idle state a CPU should go into when it has no useful work to do; the cpuidle driver then actually puts the CPU into that state. But, at the 2020 Power Management and Scheduling in the Linux Kernel summit (OSPM), Abhishek Goel presented a new cpuidle driver that doesn't actually change the processor's power state at all. Such a driver will clearly save no power, but it can be quite useful as a tool for evaluating and debugging cpuidle policies.

Goel began by saying that this work was motivated by a performance problem encountered with high-latency idle states — deep CPU sleep states that take a long time to restart from. A GPU-oriented workload that generated lots of interrupts was involved; the time between those interrupts was just enough to cause the governor to choose a deep idle state. That created latency which added up over time as the workload progressed. The temporary workaround was to increase the target latency (the expected sleep time) for those idle states by a factor of three to five, biasing the idle-state choice toward the shallower states. It solved the problem, but is "not elegant"; others will undoubtedly find workloads that go wrong in other ways.

Rafael Wysocki interjected to suggest using the pm_qos mechanism instead; its purpose is to address just this sort of latency issue, and he was curious to know why it didn't work. Part of the problem, evidently, is that pm_qos will disable the deeper idle states entirely, but there can be value in having them remain available for the truly long idle periods. Parth Shah added that, on the Power architecture, this is even more true; without getting to those deeper idle states little energy will be saved.

Frequency scaling — adjusting a CPU's operating frequency to save power when the workload demands are low — is common practice across systems supported by Linux. It is, however, viewed with some suspicion in data-center settings, where power consumption is less of a concern and there is a strong emphasis on getting the most performance out of the hardware. At the 2020 Power Management and Scheduling in the Linux Kernel summit (OSPM), Giovanni Gherdovich worried that frequency scaling may be about to go extinct in data centers; he made a plea for improving its behavior for such workloads while there is still time.

He started with a quote from a car-rally driver: "if in doubt, go flat out". This may not actually be the best advice for drivers of motor vehicles, he said, but it may be the right approach for frequency scaling. Users in data centers often switch to the "performance" CPU-frequency governor, which is not frequency scaling. This governor, which simply runs at full speed all the time, reflects one outcome of the tension between energy efficiency and raw performance. Program managers tend to be interested in performance first, and performance is the first thing that customers see. The cost of power usage is only discovered later, resulting in attempts to hack efficiency into a data-center deployment as an afterthought. It would be better to have that efficiency there from the outset, he said.

He asked the audience a question: assume you are a regional bank running an on-premises data center. Which CPU-frequency governor would you choose? The intel_pstate powersave governor would be the smart choice for now. But the intel_pstate performance governor beckons. The "schedutil" governor is the upcoming next generation. Would you pick one of those, or just go with whatever default the distribution picked? The choice is not obvious. Frequency scaling looks like a risk for a data-center user. Can the distribution vendor be trusted to have made the right choice? For distributors, the bank is a customer who must be catered to. Which governor would you set as the default?

As Rafael Wysocki conceded at the beginning of a session at the 2020 Power Management and Scheduling in the Linux Kernel summit (OSPM), the combination of the deadline scheduling class with CPU idle states might seem a little strange. Deadline scheduling is used in realtime settings, where introducing latency by idling the CPU tends to be frowned upon. But there are reasons to think that these two technologies might just be made to work together.
Why would one even bother to try to combine the deadline scheduler and CPU idle states? One should never miss opportunities to save energy, he said. Plus, on some systems, avoiding idle states is not really an option; without them, the CPU will overheat and thermal throttling will kick in. Meanwhile, the combination seems viable to him. In theory, at least, all of the information needed to select idle states is present; the scheduler has work estimates and deadlines for all tasks, and it has the idle-state properties for the CPU. It's just a matter of using that information correctly.

In the 5.7 kernel, he began, the runnable_load_avg signal has been removed in favor of runnable_avg, which is the sum of the "runnable" time of every scheduler entity (either an individual task or a control group containing tasks) in a run queue. The runnable time is defined as the time a task actually spends running, but also the time it spends waiting to run. This change addresses a problem that had been seen in capacity tracking when a task is migrated from one CPU to another.

Specifically, moving a task off of a CPU moves that task's utilization with it, causing that CPU to appear to suddenly have a lot of spare capacity. But if other tasks on the CPU were spending a lot of time waiting to run, that capacity doesn't really exist; the utilization of those tasks was artificially reduced by the fact that they couldn't run as much as they needed to. Including the waiting time prevents that false capacity from appearing when one task moves away, giving the remaining tasks time to expand their actual utilization. The calculation of when a CPU (or set of CPUs) is overloaded now looks at runnable_avg, which must exceed the CPU capacity by a threshold before the scheduler will try to move tasks away.

NUMA balancing is still not using this metric, though, so there is currently a mismatch between normal load balancing and NUMA balancing. That can lead to conflicting decisions at times. It might make sense to change the balancing at the NUMA level, but NUMA nodes can contain a lot of CPUs, and he worries about the impact of summing that many runnable_avg values. He has not started working on this problem, but it's at the top of his list.

Peter Zijlstra noted that developers are still "chasing the fallout" from the changes that have been made so far. Guittot acknowledged that, saying he's not sure if the NUMA issues play into that or not.

Hibernation is normally thought of as a laptop feature — and an old and obsolete laptop feature at that. One does not normally consider it to be relevant in cloud settings. But, at the 2020 Power Management and Scheduling in the Linux Kernel summit (OSPM), Andrea Righi argued that there may actually be a place for hibernation on cloud-based systems if it can be made to work reliably.
The core idea behind hibernation is powering the system down entirely, but restoring it to its previous state, including any running processes, when the system is powered up. To do that, the contents of memory, which will not survive loss of power, must be written to persistent storage before turning things off. The advantage of hibernation is that the system can retain its state indefinitely without power; the cost is a great deal of I/O at both hibernation and resume times.

Hibernation was a hot topic back in 2004, when it was usually known as "software suspend"; see the LWN kernel index entry for software suspend to understand just how hot. Work in this area slowed around 2008, though, when suspend-to-RAM functionality (often just called "suspend") became widely available. Support for hibernation was dropped entirely in Ubuntu 12.04. The Fedora 29 release included an experiment with suspend-then-hibernate functionality, but that "didn't go well" and was dropped. Hibernation mostly seems like a dead topic, he said.