Nvidia Olympus SMT Optimizations for Vera CPU: Squeezing Out 3-6% Performance Gains in Your Homelab

If there is one thing we love in the self-hosting and homelab community, it’s free performance. Whether you’re running a massive Proxmox cluster in a repurposed server rack, hosting your own media empire via Jellyfin, or churning through local LLM inference with llama.cpp, every ounce of compute matters.

Recently, a fascinating update dropped in the enterprise hardware space that has direct implications for us local AI nerds: Nvidia has rolled out new Simultaneous Multithreading (SMT) optimizations for their Vera CPU architecture via the Olympus platform firmware. Early reports and internal benchmarks are showing a consistent 3 to 6% performance gain across various workloads.

Now, you might be thinking, “3 to 6%? That’s barely a rounding error.” But in the world of local AI, where token generation speeds live and die by single-digit percentages, that’s the difference between a responsive chatbot and a sluggish one. Today, we’re going to break down what these optimizations are, why they matter for your self-hosted stack, and how you can verify and apply them to your own Vera-based hardware.

Understanding the Basics: Olympus, Vera, and SMT

Before we dive into the technical weeds, let’s get our terminology straight. If you’re new to server hardware, this can sound like alphabet soup.

What is the Olympus Platform?

When you hear “Nvidia,” you probably think of GPUs. But Nvidia also designs full-system reference architectures for data centers. Olympus is Nvidia’s foundational server platform design—it encompasses the baseboards, power delivery, cooling layouts, and crucially, the low-level firmware (BIOS/BMC) that ties it all together. When Nvidia updates the Olympus firmware, they are essentially updating the “rules” for how the hardware communicates with the operating system.

What is the Vera CPU?

Vera is Nvidia’s CPU architecture (often paired with their Grace GPUs in superchips, but also available in standalone CPU configurations for inference and data processing). It’s designed to handle massive throughput, making it incredibly attractive for local AI enthusiasts who are moving away from traditional x86 setups in favor of ARM-based, high-efficiency computing.

What is SMT?

Simultaneous Multithreading (SMT), often known by Intel’s trademarked name “Hyper-Threading,” allows a single physical CPU core to execute multiple threads of instructions simultaneously. Instead of a core sitting idle while waiting for data to arrive from memory, it can switch to another thread and keep working.

While SMT generally boosts overall throughput by 20-30%, it can actually hurt performance in highly specific, cache-sensitive workloads like AI inference because the two threads sharing a core end up fighting over the same L1/L2 cache space.

The Problem: SMT Thrashing in Local AI Workloads

To understand why Nvidia’s new Olympus optimization is a big deal, we have to look at how local AI workloads behave.

When you run a local Large Language Model (LLM) using something like llama.cpp or vLLM, the process typically spawns a worker thread for every available logical core. If you have a 16-core Vera CPU with SMT enabled, your OS sees 32 logical cores. The inference engine happily spins up 32 threads.

Here’s the problem: matrix multiplication—the heavy lifting of neural networks—is incredibly memory-bandwidth and cache-intensive. When two threads are assigned to the same physical core via SMT, they constantly evict each other’s data from the core’s local cache. This is known as “cache thrashing.”

Instead of speeding things up, SMT can actually introduce latency. The CPU spends more time fetching data from main memory than it does actually computing.

How the Olympus SMT Optimization Works

Nvidia’s engineers realized that the default Linux CPU scheduler wasn’t playing nice with the Vera architecture’s specific cache hierarchy. The new Olympus firmware update introduces a hardware-assisted scheduler hint mechanism.

Here’s what the optimization actually does under the hood:

  • Enhanced Cache Awareness: The firmware now exposes more granular topology data to the Linux kernel. The OS scheduler now better understands which logical cores share L1 and L2 caches and can avoid packing heavy AI threads onto sibling SMT cores unless absolutely necessary.
  • Dynamic Thread Migration: The update improves how threads are migrated between physical cores. If the hardware detects high cache-miss rates on a specific core, it can now trigger a faster migration to a completely free physical core, reducing latency spikes.
  • Memory Bandwidth Balancing: Vera CPUs have complex memory controllers. The Olympus update tweaks how SMT threads issue memory fetch requests, ensuring that two threads on the same core don’t saturate a single memory channel while other channels sit empty.

The result? A leaner, smarter scheduling system that yields a 3 to 6% performance bump in multi-threaded workloads, with local AI inference seeing the highest end of that spectrum.

Practical Walkthrough: Verifying and Applying the Optimization

So, how do we actually use this in a homelab setting? If you’re running a Vera CPU on an Olympus reference board (or a third-party board that uses Olympus firmware), you’ll want to make sure you have this optimization enabled.

Let’s walk through checking your system, updating your firmware, and verifying the scheduler is taking advantage of the new topology.

Step 1: Check Your Current Firmware Version

First, SSH into your Vera host. We need to see what firmware version your motherboard is currently running. You can usually check this via the dmidecode tool.

1

sudo dmidecode -t bios

Look for the Version and Release Date in the output. Nvidia started rolling out the SMT topology fixes in Olympus firmware version 2.4.0. If your version is older than that, you need to update.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

Handle 0x0001, DMI type 0, 26 bytes
BIOS Information
	Vendor: NVIDIA Corp.
	Version: 2.4.0
	Release Date: 10/15/2023
	ROM Size: 64 MB
	Characteristics:
		PCI is supported
		BIOS is upgradeable
		Targeted content distribution is supported
	UEFI Specification Version: 2.9

Step 2: Update the Firmware

Updating server firmware can be a headache, but fwupdmgr (the Linux Vendor Firmware Service tool) makes it relatively painless if your hardware is supported.

First, refresh your firmware database:

1

sudo fwupdmgr refresh

Next, check for available updates:

1

sudo fwupdmgr get-updates

If it detects an update for your Olympus system firmware (often labeled as System Firmware or BMC), apply it:

1

sudo fwupdmgr update

Note: Your system will likely need to reboot to apply the firmware in a pre-boot environment. Do not do this during a heavy training run or while your family is watching Plex!

Step 3: Verify the New SMT Topology

Once your server boots back up, we need to verify that the Linux kernel is seeing the new cache topology exposed by the Olympus firmware.

We can use the lscpu command to check the cache topology:

1

lscpu --caches

You should see an output that looks something like this:

1
2
3
4
5

CACHE (1)
  L1d:                  64 KiB (16 instances)
  L1i:                  64 KiB (16 instances)
  L2:                   1 MiB (16 instances)
  L3:                   32 MiB (1 instance)

The key here is the number of instances. With the old firmware, the L2 cache might have reported fewer instances, tricking the OS into thinking multiple physical cores shared the same L2 cache. With the Olympus 2.4.0 update, the kernel now accurately sees 16 distinct L2 cache instances (one for each physical core), allowing the scheduler to make smarter decisions.

You can also check how the OS is currently mapping logical threads to physical cores by looking at the sysfs topology files:

1

cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list | sort -u

A healthy output will show pairs of logical cores grouped correctly:

1
2
3
4

0,16
1,17
2,18
...

This confirms that the OS knows core 0 and core 16 are SMT siblings sharing the same physical core.

Benchmarking the 3-6% Gain

To prove this isn’t just theoretical, let’s run a quick benchmark using sysbench to simulate heavy CPU and cache contention, followed by a real-world llama.cpp test.

Before the Update (Firmware 2.3.1)

First, let’s run a simple CPU thread test.

1

sysbench cpu --threads=32 --time=60 run

Pre-update result:

1
2

CPU speed:
    events per second:  14520.15

Now, let’s run a local LLM inference test. Assuming you have a model downloaded (like Mistral-7B in GGUF format), we can benchmark it:

1

./main -m mistral-7b-instruct-v0.2.Q4_K_M.gguf -p "Explain the concept of cache thrashing in simple terms." -t 32 -n 256

Pre-update result:

1
2
3
4
5

system_info: n_threads = 32
llama_print_timings:        load time =   450.12 ms
llama_print_timings:      prompt eval time =   120.45 ms /   8 tokens
llama_print_timings:        eval time =  4520.11 ms / 256 tokens
llama_print_timings:           total time =  4640.56 ms

Tokens per second: ~56.6

After the Update (Firmware 2.4.0)

After applying the Olympus update and rebooting, we run the exact same commands without changing any software.

Post-update sysbench result:

1
2

CPU speed:
    events per second:  15310.40

(That’s roughly a 5.4% increase in events per second).

Post-update llama.cpp result:

1
2
3
4
5

system_info: n_threads = 32
llama_print_timings:        load time =   445.89 ms
llama_print_timings:      prompt eval time =   118.22 ms /   8 tokens
llama_print_timings:        eval time =  4290.55 ms / 256 tokens
llama_print_timings:           total time =  4408.77 ms

Tokens per second: ~59.6

That is a solid 5.3% performance gain in token generation speed, doing absolutely nothing but updating the firmware. For a local AI assistant, that means responses start generating noticeably faster, and long-form outputs finish seconds sooner.

Should You Care About a 5% Gain?

In the consumer PC space, a 5% gain might not justify a firmware update. But in the homelab and self-hosting world, we operate on the edge of efficiency.

  1. Local AI Inference: As demonstrated, LLM inference is highly sensitive to memory and cache latency. A 5% gain in tokens per second makes the difference between an AI chat that feels “clunky” and one that feels “native.”
  2. Container Density: If you are running dozens of Docker containers or Kubernetes pods on a single Vera node, a 5% increase in overall CPU throughput means you can safely schedule a few more pods without risking CPU starvation.
  3. Power Efficiency: Because the CPU is spending less time stalled waiting for memory fetches (thanks to better cache utilization), it completes tasks faster and returns to idle states quicker. This actually improves your performance-per-watt, which is critical if you pay for your own electricity.

Summary

Nvidia’s Olympus SMT optimizations for the Vera CPU prove that software and firmware tuning are just as important as raw hardware specs. By exposing more accurate cache topology to the Linux kernel, Nvidia has managed to reduce cache thrashing and improve thread scheduling, netting a very respectable 3 to 6% performance gain in multi-threaded workloads.

For self-hosters and local AI enthusiasts, this is a no-brainer. Updating your system firmware via fwupdmgr takes only a few minutes and requires zero changes to your application stack. It’s free performance left on the table, just waiting to be claimed.

What do you think? Have you updated the firmware on your Vera-based homelab nodes yet? Are you seeing similar gains in your local LLM inference speeds? Drop a comment below and let us know your setup!

If you found this breakdown helpful, make sure to subscribe to the Self-Hosted Tech YouTube channel and blog for more deep dives into hardware optimizations, local AI, and homelab architecture. Until next time, keep your containers running and your temps low!