….and I bought an AMD GPU.
It’s the first time I’ve gone non-Nvidia since before 2010 as I’ve been a Nvidia loyalist since the GeForce 256 back in 1999 (was 3dfx prior). But Nvidia GPU pricing on anything with decent VRAM has gotten absurd, even on the used market. I kept hearing that ROCm (Radeon Open Compute) support for inference has finally matured. So I decided to find out for myself.
I picked up the AMD Radeon AI PRO R9700, 32GB. Not cheap, but brand new its cheaper than an RTX 3090 24gb from 2020.
Since this is my first AMD in very long while, going to have to figure out drivers. So let’s get started. First lets gather some info by running the commands below.
lspci -nn | grep -Ei 'VGA|Display|AMD|ATI'lsmod | grep amdgpudmesg | grep -i amdgpu | tail -50
Verify GPU Detection
Prior to card swap, purged my Ubuntu 24.04 system of all Nvidia drivers and tools.
Lets check to see if the card is detected properly
lspci -nn | grep -Ei 'VGA|Display|AMD|ATI'
This command searches the PCI device list for anything matching VGA, Display, AMD, or ATI. In the output below we see the following
03:00.0is the PCI id of the R9700 GPU- Vendor ID
1002confirms AMD Device 7551is how the kernel identifies this GPU generation/device- The two PCI bridge entries show the GPU exposes an internal PCIe switch/bridge topology
01:00.0 PCI bridge: AMD Navi 10 XL Upstream Port of PCI Express Switch02:00.0 PCI bridge: AMD Navi 10 XL Downstream Port of PCI Express Switch03:00.0 VGA compatible controller: AMD Device [1002:7551]03:00.1 Audio device: AMD Device [1002:ab40]
The important takeaway is that Ubuntu already recognizes the R9700 correctly and the in-kernel AMDGPU driver is successfully initializing it.
lsmod | grep amdgpu
This command above shows whether the AMDGPU kernel module is loaded and what other kernel modules depend on it.
amdgpu 21188608 26
This tells us:
amdgpuis loaded- the module occupies roughly 21 MB of kernel module memory
26references/users currently depend on it
dmesg | grep -i amdgpu | tail -50
The command above shows the last 50 kernel messages associated with AMDGPU initialization.
The most important things we see are:
MEM ECC is active.
The GPU’s VRAM ECC capability is active.
VRAM: 32624M
The kernel detects approximately 32 GB of VRAM.
Detected VRAM RAM=32624M, BAR=32768M
Preparing for ROCm Install
First lets make sure that the kernel has created the ROCm compute device
ls -l /dev/kfd
crw-rw---- 1 root render 235, 0 Aug 9 16:20 /dev/kfd
/dev/kfd is the Kernel Fusion Driver device that ROCm/HIP uses to submit compute workloads to the AMD GPU. AMD’s ROCm installation process relies on access through the render/video device groups.
In the output above, you can see that the Kernel Fusion Driver is owned by root with group membership of render. We need to add my user to the render group. Note that I had to reboot to pick up the changes.
sudo usermod -aG render cpaquin
Now we can add the ROCm 7.2.2 package repo, which is two steps. This one below…
sudo mkdir --parents --mode=0755 /etc/apt/keyringswget https://repo.radeon.com/rocm/rocm.gpg.key -O - | \gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null
And this next one to actually add the repo.
sudo tee /etc/apt/sources.list.d/rocm.list <<'EOF'deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/7.2.2 noble maindeb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/graphics/7.2.1/ubuntu noble mainEOF
Now let’s be smart and ensure that the repo was added properly.
cat /etc/apt/sources.list.d/rocm.listdeb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/7.2.2 noble maindeb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/graphics/7.2.1/ubuntu noble main
Now before we install anything, per AMD recommendation, we need to pin the above repo priority. Pinning means settings which repository Ubuntu prefers when the same package is available from multiple repositories, which means prefer the AMD repos over Ubuntu repos for AMD bits and bobbles.
sudo tee /etc/apt/preferences.d/rocm-pin-600 <<'EOF'Package: *Pin: release o=repo.radeon.comPin-Priority: 600EOF
Now we update metadata.
sudo apt update
We can see the repo preference via pinning below.
apt policy rocm
rocm:
Installed: (none)
Candidate: 7.2.2.70202-86~24.04
Version table:
7.2.2.70202-86~24.04 600
600 https://repo.radeon.com/rocm/apt/7.2.2 noble/main amd64 Packages
ROCm Install
First, lets make sure we have the following installed.
sudo apt install python3-setuptools python3-wheel
Now we can install ROCm. It takes a bit so be patient, or don’t.
sudo apt install rocm
Now lets verify that ROCm can see the GPU
rocminfo
Truncated output below
rocminfoROCk module is loaded===================== HSA System Attributes ===================== Runtime Version: 1.18Runtime Ext Version: 1.15System Timestamp Freq.: 1000.000000MHzSig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)Machine Model: LARGE System Endianness: LITTLE Mwaitx: DISABLEDXNACK enabled: NODMAbuf Support: YESVMM Support: YES******* Agent 2 ******* Name: gfx1201 Uuid: GPU-407fcdae2bc6b11d Marketing Name: AMD Radeon AI PRO R9700 Vendor Name: AMD Feature: KERNEL_DISPATCH Profile: BASE_PROFILE Float Round Mode: NEAR Max Queue Number: 128(0x80) Queue Min Size: 64(0x40) Queue Max Size: 131072(0x20000) Queue Type: MULTI Node: 1 Device Type: GPU
And now run amd-smi.
amd-smi+------------------------------------------------------------------------------+| AMD-SMI 26.2.2+671d39a71e amdgpu version: 7.0.0-28 ROCm version: 7.2.2 || VBIOS version: 00158746 || Platform: Linux Baremetal ||-------------------------------------+----------------------------------------|| BDF GPU-Name | Mem-Uti Temp UEC Power-Usage || GPU HIP-ID OAM-ID Partition-Mode | GFX-Uti Fan Mem-Usage ||=====================================+========================================|| 0000:03:00.0 ...Radeon AI PRO R9700 | 1 % 31 °C 0 9/300 W || 0 0 N/A N/A | 3 % 20.0 % 661/32624 MB |+-------------------------------------+----------------------------------------++------------------------------------------------------------------------------+| Processes: || GPU PID Process Name GTT_MEM VRAM_MEM MEM_USAGE CU % ||==============================================================================|| No running processes found |
What we are looking at:
- ROCm 7.2.2 is active.
- The Radeon AI PRO R9700 is correctly identified.
- Temperature is 31 °C.
- Power draw is only 9 W / 300 W at idle.
- Fan is at 20%.
- VRAM usage is 661 MB / 32,624 MB, which is normal for the display/driver stack.
- No uncorrectable ECC errors:
UEC 0. - No compute processes are currently using the GPU.
Test ROCm inside Docker
Can a Docker container access the R9700 through ROCm? Lets find out.
docker run --rm \ --device=/dev/kfd \ --device=/dev/dri \ --group-add 992 \ --group-add 44 \ rocm/rocm-terminal:latest \ rocminfo
What this does:
- starts a temporary AMD ROCm container
- passes
/dev/kfdinto it for ROCm compute - passes
/dev/driinto it for the GPU device nodes - gives the container the needed
video(44) andrender(992) group access - runs
rocminfoinside the container - deletes the container afterward because of
--rm
If it works, we should see the same R9700 identification inside Docker that we saw on the host, including something like:
Pytorch Test
Next we’ll use AMD’s validated PyTorch container for ROCm 7.2 / Ubuntu 24.04 / Python 3.12 / PyTorch 2.9.1. AMD recommends Docker for PyTorch because the image already contains a tested PyTorch+ROCm combination.
docker run --rm \ --device=/dev/kfd \ --device=/dev/dri \ --group-add 992 \ --group-add 44 \ --security-opt seccomp=unconfined \ --ipc=host \ --shm-size 8G \ rocm/pytorch:rocm7.2_ubuntu24.04_py3.12_pytorch_release_2.9.1 \ python3 -c "import torch; print('PyTorch:', torch.__version__); print('ROCm:', torch.version.hip); print('GPU available:', torch.cuda.is_available()); print('GPU:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'NONE')"
This will likely download a fairly large container image the first time.
What we want to see is essentially:
PyTorch: 2.9.1+rocm7.2.0.git7e1940d4ROCm: 7.2.26015-fc0010cf6aGPU available: TrueGPU: AMD Radeon AI PRO R9700
One potentially confusing point: PyTorch keeps the API name torch.cuda even when using AMD ROCm.
torch.cuda.is_available() returning True means PyTorch has a usable GPU backend; on this container that backend is HIP/ROCm, not NVIDIA CUDA. AMD uses this same test in its ROCm PyTorch verification instructions.
So the R9700 is now validated for actual ROCm/PyTorch compute inside Docker. Next Step VLLM and test some models.
