Docker 28, NVIDIA GPUs, and the cgroupns Trap
Plex was pegging the CPU on every transcode. The GTX 1070 sitting in the box was doing nothing. The GPU was passed through to the container correctly, the NVIDIA driver capabilities were set, the device requests were in the compose file. Everything looked right. It wasn’t.
Contents#
The Symptom#
Stream transcoding on Plex was hitting 100% CPU on big-boi (192.168.1.21). This machine has a GTX 1070 specifically for hardware transcoding. Plex supports NVENC/NVDEC out of the box with a Plex Pass; it should be offloading transcodes to the GPU automatically. Instead, every stream was software transcoding and cooking the CPU.
What Looked Correct#
The docker-compose had the right GPU reservation:
plex:
image: linuxserver/plex
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
The NVIDIA container toolkit was installed. The nvidia runtime was registered with Docker. The environment variable was set inside the container:
NVIDIA_DRIVER_CAPABILITIES=compute,video,utility,graphics
The GPU device nodes were present inside the container:
/dev/nvidia0
/dev/nvidiactl
/dev/nvidia-uvm
/dev/nvidia-uvm-tools
The driver was even visible through /proc:
NVRM version: NVIDIA UNIX x86_64 Kernel Module 570.153.02
Everything pointed to a working GPU passthrough. But running nvidia-smi inside the container told a different story:
Failed to initialize NVML: Unknown Error
NVML is the NVIDIA Management Library. If it can’t initialize, nothing that depends on it (including Plex’s hardware transcoder) can discover or use the GPU. The devices are there, the driver is loaded, but the management layer that ties it all together fails silently. Plex sees no GPU and falls back to CPU.
The Actual Problem#
$ docker inspect plex --format '{{.HostConfig.CgroupnsMode}}'
private
Docker 28.x changed the default cgroup namespace mode from host to private. This isolates the container’s view of /sys/fs/cgroup, which is where NVML reads GPU resource information. With a private cgroup namespace, NVML can see the device nodes but can’t read the cgroup hierarchy it needs to initialize. It fails with the unhelpfully vague “Unknown Error.”
The same issue affected the steam-headless container on the same host. Both containers had GPU device requests, both had the nvidia driver loaded, and both were broken.
The Fix#
One line in /etc/docker/daemon.json:
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"default-cgroupns-mode": "host"
}
Then restart Docker and recreate the containers (a restart alone isn’t enough; the cgroupns mode is set at container creation time):
systemctl restart docker
cd /docker/confs/docker-compose-usenet && docker compose up -d --force-recreate plex
cd /docker/confs/steam-headless && docker compose up -d --force-recreate
Setting it at the daemon level instead of per-container means any future GPU container on this host works without needing to remember to add cgroupns: host to every compose file.
Why This Happens#
Docker 28 made cgroupns: private the default for better container isolation. In most cases this is the right call. Containers shouldn’t need to see the host’s cgroup tree. But NVML was built assuming it can read /sys/fs/cgroup to discover GPU resources, and the private namespace hides that. The NVIDIA container toolkit (1.17.x at the time of this fix) doesn’t compensate for it.
This is one of those issues where every individual component is working correctly. Docker is doing what it’s supposed to do (isolating cgroups). The NVIDIA toolkit is passing through devices correctly. The driver is loaded. The environment variables are set. But the interaction between Docker’s namespace isolation and NVML’s cgroup dependency creates a failure that nothing logs clearly.
The nvidia-smi error message, “Failed to initialize NVML: Unknown Error,” is the only clue, and it’s not a great one.
Verifying the Fix#
After recreating the containers:
$ docker inspect plex --format '{{.HostConfig.CgroupnsMode}}'
host
$ docker exec plex nvidia-smi
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.153.02 Driver Version: 570.153.02 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| 0 NVIDIA GeForce GTX 1070 Off | 00000000:03:00.0 Off | N/A |
| 0% 42C P8 7W / 151W | 5MiB / 8192MiB | 0% Default |
+-----------------------------------------+------------------------+----------------------+
Both plex and steam-headless now see the GTX 1070. Plex hardware transcoding works. CPU usage during transcodes dropped from 100% to near zero.