AMD – Chris Paquin

….and I bought an AMD GPU.

It’s the first time I’ve gone non-Nvidia since before 2010 as I’ve been a Nvidia loyalist since the GeForce 256 back in 1999 (was 3dfx prior). But Nvidia GPU pricing on anything with decent VRAM has gotten absurd, even on the used market. I kept hearing that ROCm (Radeon Open Compute) support for inference has finally matured. So I decided to find out for myself.

I picked up the AMD Radeon AI PRO R9700, 32GB. Not cheap, but brand new its cheaper than an RTX 3090 24gb from 2020.

Since this is my first AMD in very long while, going to have to figure out drivers. So let’s get started. First lets gather some info by running the commands below.

			
lspci -nn | grep -Ei 'VGA|Display|AMD|ATI'
lsmod | grep amdgpu
dmesg | grep -i amdgpu | tail -50

Verify GPU Detection

Prior to card swap, purged my Ubuntu 24.04 system of all Nvidia drivers and tools.

Lets check to see if the card is detected properly

lspci -nn | grep -Ei 'VGA|Display|AMD|ATI'

This command searches the PCI device list for anything matching VGA, Display, AMD, or ATI. In the output below we see the following

03:00.0 is the PCI id of the R9700 GPU
Vendor ID 1002 confirms AMD Device
7551 is how the kernel identifies this GPU generation/device
The two PCI bridge entries show the GPU exposes an internal PCIe switch/bridge topology

			
00.0 PCI bridge: AMD Navi 10 XL Upstream Port of PCI Express Switch
00.0 PCI bridge: AMD Navi 10 XL Downstream Port of PCI Express Switch
00.0 VGA compatible controller: AMD Device [1002:7551]
00.1 Audio device: AMD Device [1002:ab40]

The important takeaway is that Ubuntu already recognizes the R9700 correctly and the in-kernel AMDGPU driver is successfully initializing it.

lsmod | grep amdgpu

This command above shows whether the AMDGPU kernel module is loaded and what other kernel modules depend on it.

amdgpu 21188608 26

This tells us:

amdgpu is loaded
the module occupies roughly 21 MB of kernel module memory
26 references/users currently depend on it

dmesg | grep -i amdgpu | tail -50

The command above shows the last 50 kernel messages associated with AMDGPU initialization.

The most important things we see are:

MEM ECC is active.

The GPU’s VRAM ECC capability is active.

VRAM: 32624M

The kernel detects approximately 32 GB of VRAM.

Detected VRAM RAM=32624M, BAR=32768M

Preparing for ROCm Install

First lets make sure that the kernel has created the ROCm compute device

 ls -l /dev/kfd
crw-rw---- 1 root render 235, 0 Aug  9 16:20 /dev/kfd

/dev/kfd is the Kernel Fusion Driver device that ROCm/HIP uses to submit compute workloads to the AMD GPU. AMD’s ROCm installation process relies on access through the render/video device groups.

In the output above, you can see that the Kernel Fusion Driver is owned by root with group membership of render. We need to add my user to the render group. Note that I had to reboot to pick up the changes.

sudo usermod -aG render cpaquin

Now we can add the ROCm 7.2.2 package repo, which is two steps. This one below…

			
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | \
gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null

And this next one to actually add the repo.

			
sudo tee /etc/apt/sources.list.d/rocm.list <<'EOF'
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/7.2.2 noble main
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/graphics/7.2.1/ubuntu noble main
EOF

Now let’s be smart and ensure that the repo was added properly.

			
cat /etc/apt/sources.list.d/rocm.list
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/7.2.2 noble main
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/graphics/7.2.1/ubuntu noble main

Now before we install anything, per AMD recommendation, we need to pin the above repo priority. Pinning means settings which repository Ubuntu prefers when the same package is available from multiple repositories, which means prefer the AMD repos over Ubuntu repos for AMD bits and bobbles.

			
sudo tee /etc/apt/preferences.d/rocm-pin-600 <<'EOF'
Package: *
Pin: release o=repo.radeon.com
Pin-Priority: 600
EOF

		

Now we update metadata.

sudo apt update

We can see the repo preference via pinning below.

 apt policy rocm
rocm:
  Installed: (none)
  Candidate: 7.2.2.70202-86~24.04
  Version table:
     7.2.2.70202-86~24.04 600
        600 https://repo.radeon.com/rocm/apt/7.2.2 noble/main amd64 Packages

ROCm Install

First, lets make sure we have the following installed.

sudo apt install python3-setuptools python3-wheel

Now we can install ROCm. It takes a bit so be patient, or don’t.

sudo apt install rocm

Now lets verify that ROCm can see the GPU

rocminfo

Truncated output below

			
rocminfo
ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.18
Runtime Ext Version:     1.15
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
XNACK enabled:           NO
DMAbuf Support:          YES
VMM Support:             YES
*******                  
Agent 2                  
*******                  
  Name:                    gfx1201                            
  Uuid:                    GPU-407fcdae2bc6b11d               
  Marketing Name:          AMD Radeon AI PRO R9700            
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    1                                  
  Device Type:             GPU                                

		

And now run amd-smi.

			
amd-smi
+------------------------------------------------------------------------------+
| AMD-SMI 26.2.2+671d39a71e    amdgpu version: 7.0.0-28 ROCm version: 7.2.2    |
| VBIOS version: 00158746                                                      |
| Platform: Linux Baremetal                                                    |
|-------------------------------------+----------------------------------------|
| BDF                        GPU-Name | Mem-Uti   Temp   UEC       Power-Usage |
| GPU  HIP-ID  OAM-ID  Partition-Mode | GFX-Uti    Fan               Mem-Usage |
|=====================================+========================================|
| 0000:03:00.0 ...Radeon AI PRO R9700 | 1 %      31 °C   0             9/300 W |
|   0       0     N/A             N/A | 3 %     20.0 %            661/32624 MB |
+-------------------------------------+----------------------------------------+
+------------------------------------------------------------------------------+
| Processes:                                                                   |
|  GPU        PID  Process Name          GTT_MEM  VRAM_MEM  MEM_USAGE     CU % |
|==============================================================================|
|  No running processes found                                                  |

		

What we are looking at:

ROCm 7.2.2 is active.
The Radeon AI PRO R9700 is correctly identified.
Temperature is 31 °C.
Power draw is only 9 W / 300 W at idle.
Fan is at 20%.
VRAM usage is 661 MB / 32,624 MB, which is normal for the display/driver stack.
No uncorrectable ECC errors: UEC 0.
No compute processes are currently using the GPU.

Test ROCm inside Docker

Can a Docker container access the R9700 through ROCm? Lets find out.

			
docker run --rm \
  --device=/dev/kfd \
  --device=/dev/dri \
  --group-add 992 \
  --group-add 44 \
  rocm/rocm-terminal:latest \
  rocminfo

		

What this does:

starts a temporary AMD ROCm container
passes /dev/kfd into it for ROCm compute
passes /dev/dri into it for the GPU device nodes
gives the container the needed video (44) and render (992) group access
runs rocminfo inside the container
deletes the container afterward because of --rm

If it works, we should see the same R9700 identification inside Docker that we saw on the host, including something like:

Pytorch Test

Next we’ll use AMD’s validated PyTorch container for ROCm 7.2 / Ubuntu 24.04 / Python 3.12 / PyTorch 2.9.1. AMD recommends Docker for PyTorch because the image already contains a tested PyTorch+ROCm combination.

			
docker run --rm \
  --device=/dev/kfd \
  --device=/dev/dri \
  --group-add 992 \
  --group-add 44 \
  --security-opt seccomp=unconfined \
  --ipc=host \
  --shm-size 8G \
  rocm/pytorch:rocm7.2_ubuntu24.04_py3.12_pytorch_release_2.9.1 \
  python3 -c "import torch; print('PyTorch:', torch.__version__); print('ROCm:', torch.version.hip); print('GPU available:', torch.cuda.is_available()); print('GPU:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'NONE')"

		

This will likely download a fairly large container image the first time.

What we want to see is essentially:

			
PyTorch: 2.9.1+rocm7.2.0.git7e1940d4
ROCm: 7.2.26015-fc0010cf6a
GPU available: True
GPU: AMD Radeon AI PRO R9700

One potentially confusing point: PyTorch keeps the API name torch.cuda even when using AMD ROCm.

torch.cuda.is_available() returning True means PyTorch has a usable GPU backend; on this container that backend is HIP/ROCm, not NVIDIA CUDA. AMD uses this same test in its ROCm PyTorch verification instructions.

So the R9700 is now validated for actual ROCm/PyTorch compute inside Docker. Next Step VLLM and test some models.

Tag: AMD

The AMD R9700 – Ubuntu 24.04 Quickstart Guide

Verify GPU Detection

Preparing for ROCm Install

ROCm Install

Test ROCm inside Docker

Pytorch Test

Like this: