news
Red Hat Promoting Slop for IBM, Oracle Linux-based Security Onion 2.4.210 is Out
-
Red Hat ☛ Practical strategies for vLLM performance tuning [Ed: Red Hat peddling slop]
Performance tuning large language model (LLM) serving frameworks like vLLM is rarely about a single magic flag or configuration. Instead, it's an iterative process that balances hardware constraints, workload characteristics, and user experience goals such as latency and throughput.
This article walks through practical tuning recommendations with a focus on designing meaningful benchmarks and extracting the most performance from vLLM.
-
Red Hat ☛ Serve and benchmark Prithvi models with vLLM on OpenShift
In Scaling Earth and space Hey Hi (AI) models with Red Bait Hey Hi (AI) Inference Server and Red Bait OpenShift AI, we showed the performance benefits of serving inference for the Prithvi-EO model with Red Hat Hey Hi (AI) Inference Server. We demonstrated this using both a standalone setup and a combination of KServe and Knative. Here, we will dive deeper and show how to set up and test both cases. If you are feeling adventurous, you can also try using your own Earth and space model instead of Prithvi.
Let’s dive in!
-
Red Hat ☛ Optimize PyTorch training with the autograd engine
How does PyTorch calculate gradients when you run a
backward()call? The answer is the autograd engine. This tool provides automatic differentiation, which calculates the gradients required for training deep learning models in PyTorch.This article examines how the autograd engine builds and executes the computational graph, manages memory during backpropagation, and optimizes performance. Whether you are a researcher debugging gradient flows or an engineer optimizing training pipelines, understanding the internals of the autograd engine will improve your PyTorch skills.
-
Notebook Check ☛ Oracle Linux-based Security Onion 2.4.210 launches with Onion AI local model support
Security Onion 2.4.210 is back with major improvements for the Onion AI Assistant, as well as local model support for Onion AI. Several system components of this platform that performs threat hunting and enterprise security monitoring were updated. This new version also comes with the usual set of minor tweaks and fixes.
Originally based on Xubuntu 10.04, Security Onion has been around since 2009. This free and open Linux distribution for threat hunting, enterprise security monitoring, and log management is now based on Oracle Linux, and its latest update, labeled 2.4.210, was released yesterday.
The aforementioned update comes with major improvements for the popular new Onion AI Assistant, which is only available for Security Onion Pro customers. Since local model support has been in the list of requirements for many users, the latest software refresh allows the local model with an OpenAI-compatible endpoint to connect to it. For more information concerning Onion AI, those interested should check out this webpage.