How Meta Patches Linux at Hyperscale
Anyone with a tech clue can patch a Linux server. But, patching thousands of them without any downtime, that’s not easy.
At the Linux Plumbers Conference, the invite-only conference of top Linux kernel developers earlier this month, Meta Linux kernel engineer Breno Leitao explained how Facebook pulls the trick off with its millions of servers around the world.
If you were to use ordinary techniques, Leitao said it would take more than 45 days to roll out a new kernel to all machines. As he put it, “Draining and un-draining hosts is hard.” You can say that again.
That may be fine if it’s a minor update, but if it’s a security patch, that won’t work.
So, Meta uses Kernel Live Patching (KLP) with Red Hat‘s Kpatch, to deliver fast patches. In KLP, you can apply the latest security updates to Linux kernels without rebooting. This maximizes system uptime and availability.