The shift from srvadmin (OMSA) to iSM (iDRAC Service Module) marks the end of bloated, “in-band” server management. This occurred between Dell 12th gen and Dell 13th gen server.
If you have a 12th Gen Dell Server, you can still leverage Dell srvadmin (Idrac 7). I wrote a post on it here.
While OMSA ran as a heavy suite of services directly on the host, the modern dcism acts as a lightweight bridge that offloads the heavy lifting to the dedicated iDRAC hardware.
For RHEL 10 users, this transition is no longer optional but necessary, trading the aging omreport ecosystem for a more secure, stable, and kernel-friendly management experience.
So in this post we are going to go through the install of DISM (Dell IDRAC Service Module), on RHEL 10, running on a Dell R730.
Set Up the Dell Repository
First we need to setup the official Dell System Update (DSU) utility. Install Dell GPG keys when prompted.
You can generate a full hardware diagnostic bundle directly from the (TSR).
Note that the used Dell R730 that I just purchased on Ebay is actually under hardware support till October of 2026 – so if I need on-site hardware support a Dell technician will be dispatched to my house. I will provide snacks and a static strap.
Since omreport storage is gone, use racadm. With the iSM bridge active, Local RACADM is now your primary interface for your raid controller
racadm storage get pdisks
racadm storage get vdisks
racadm storage get controllers
The “Full Power Cycle” (The “Cold Boot” Tool) – Flea Power Drain
On an R730, if you have a hardware component acting up (like a hung NIC), a standard OS reboot often isn’t enough because the motherboard stays powered.
What it does: It tells the iDRAC to perform a “virtual” AC power pull. The server will shut down, the flea power will be drained, and it will cold-boot. This is a powerful feature to have in a remote RHEL 10 environment.
/opt/dell/srvadmin/iSM/bin/Invoke-FullPowerCycle
Real-Time “Tech Support Report” (TSR)
If you ever need to open a support ticket with Dell, they will ask for a TSR. Previously, you had to log into the iDRAC web UI and wait 10 minutes.
What it does: It triggers the iDRAC to gather every log, firmware version, and hardware event into a .zip file and saves it directly to the iDRAC (or can be pulled to the OS).
The racadm utility (which uses the iSM bridge) is the best way to see why your server has an amber light or a “System Health” warning.
View the System Event Log (SEL): This is the “Hardware Event Viewer.” It shows power supply failures, memory errors, and thermal trips.
/opt/dell/srvadmin/sbin/racadm getsel
Installing perccli
PERCCLI (PowerEdge RAID Controller Command Line Interface) is the specialized successor to the older MegaCLI tool. While RACADM is great for general server health, PERCCLI is the “surgical tool” specifically for your RAID controller (like the H730 in your R730).
Install as shown below
dnf -y install perccli.noarch
Now you can show all controllers
/opt/MegaRAID/perccli/perccli64 show
Show events.
/opt/MegaRAID/perccli/perccli64 /c0 show eventloginfo
Show all disks
/opt/MegaRAID/perccli/perccli64 /c0/eall/sall show
Epilogue
So in a nutshell. Dell srvadmin was replaced with DISM starting on Dell 13th gen servers. You have a couple powerful commands at your disposal with a ton of various ways to run both of them.
Also available is perccli, which is also a nice tool on its own.
Dell OpenManage Server Administrator (OMSA) is Dell’s on-host hardware management and monitoring framework for PowerEdge servers.
It runs inside the operating system and provides direct visibility into system hardware such as RAID controllers, physical and virtual disks, power supplies, fans, temperatures, memory, processors, and chassis health.
OMSA communicates with the server’s iDRAC and hardware controllers to retrieve real-time status and exposes this information through command-line tools like omreport, optional web interfaces, and SNMP for monitoring systems.
It is primarily used for hardware diagnostics, RAID and storage monitoring, fault detection, and health reporting, allowing administrators to verify system integrity and troubleshoot hardware issues without leaving the operating system.
So now that we have an idea of what it is, lets install it.
Note: Our system under test is a Dell R720. I probably should have lead with that.
Installation
First you will need to download the bootstrap script as shown below.
Provides shared libraries and backend support for other OMSA components
⚠️ Usually
Required by other OMSA components; often left disabled unless needed
dsm_sa_datamgrd
Systems Management Data Manager
Collects and maintains hardware inventory and system data
✅ Yes
Required for hardware monitoring (disks, temps, fans, etc.)
dsm_sa_eventmgrd
Systems Management Event Manager
Handles hardware events, alerts, and logs
✅ Yes
Required for alerts and log reporting
dsm_sa_snmpd
OMSA SNMP Agent
Provides SNMP interface for monitoring tools (LibreNMS, Zabbix, etc.)
⚠️ Optional
Needed only if using SNMP monitoring
Service Review
Now let’s review each service in detail, and see how we can leverage each one.
dsm_om_connsvc
First we need to ensure our Firewall allows this traffic. So run the commands below (RHEL), ubuntu may use ufw.
# firewall-cmd --permanent --add-port=1311/tcp
# firewall-cmd --reload
Now in your web browser, navigate to https://viper.lab:1311/ (Note https, and change host name accordingly).
Similar in appearance to the Idrac – you can find system information here, some of which does not appear in the Idrac itself. Everything is read-only.
dsm_om_shrsvc
(OMSA Shared Services) is a supporting service, not a management interface or monitoring agent.
It provides:
Shared libraries
Inter-process communication
Authentication helpers
Common backend utilities
These are required by other OMSA components to function correctly. So nothing really to see here.
dsm_sa_datamgrd.service
This is the service that collects all the data that is displayed in the WebUI (port 1311)
dsm_sa_eventmgrd.service
dsm_sa_snmpd.service is the OMSA component responsible for exposing Dell hardware monitoring data to external systems using the SNMP protocol.
This service does not collect or analyze hardware information itself; instead, it serves as the communication layer that allows remote systems to query or receive notifications about server health. When enabled, it allows OMSA to participate in centralized monitoring environments and enterprise alerting workflows, making it essential in environments where hardware status must be visible outside the server itself.
This service relies entirely on the existing system SNMP configuration, so no need to modify anything if you already have snmp configured and running with your specfic community string.
dsm_sa_snmpd.service
dsm_sa_snmpd.service is the OMSA component responsible for exposing Dell hardware monitoring data to external systems using the SNMP protocol.
It acts as the interface between the data collected internally by OMSA and third-party monitoring platforms such as LibreNMS, Zabbix, or Nagios.
This service does not collect or analyze hardware information itself; instead, it serves as the communication layer that allows remote systems to query or receive notifications about server health. When enabled, it allows OMSA to participate in centralized monitoring environments and enterprise alerting workflows, making it essential in environments where hardware status must be visible outside the server itself.
dsm_sa_snmpd.service acts as the Dell OMSA integration layer between the server’s hardware telemetry and the system’s SNMP stack, allowing Dell-specific health, storage, and sensor data to be exposed through standard SNMP queries.
It does not replace or conflict with existing monitoring agents such as Zabbix or LibreNMS (which I am using) Instead, it complements them by extending the SNMP data they can read
dsm_sa_snmpd provides access to Dell-specific MIBs that Zabbix Agent and LibreNMS Agent (or any other agent you may use) can query via the system’s SNMP daemon.
Basically, dsm_sa_snmpd is the bridge that exposes OMSA’s hardware awareness to your existing monitoring stack. Note that if you are using Zabbix you will need to install additional templates. For example, this one, note that I have not tried it yet.
Ompreport
Omreport is the command-line interface (CLI) frontend for OMSA. It queries the OMSA backend services and formats the results for humans or scripts
Note we have to use the absolute path to run this as the working dir is not in our $PATH.
/opt/dell/srvadmin/sbin/omreport system summary
If you want to add to your $PATH, run the command below
echo 'export PATH=$PATH:/opt/dell/srvadmin/sbin' | sudo tee /etc/profile.d/dell-omsa.sh
source /etc/profile
Example usage below.
omreport system summary
omreport chassis temps
omreport chassis fans
omreport chassis power
Here is an example I found useful. For some reason my raid controller keeps throwing these errors which do not seem to go away despite the fact that both SAS cables are connected (tried switching them) and all disks are healthy.
# omreport system esmlog
Severity : Critical
Date and Time : Sat Jan 31 21:15:18 2026
Description : The storage BP1 SAS B cable is not connected, or is improperly connected.
Severity : Ok
Date and Time : Sat Jan 31 21:15:20 2026
Description : The chassis is closed while the power is off.
Severity : Critical
Date and Time : Sat Jan 31 21:15:21 2026
Description : The storage BP1 SAS A cable is not connected, or is improperly connected.
Enter omconfig.
omconfig
omconfig is the configuration and control interface for Dell OpenManage Server Administrator (OMSA).
Unlike omreport, which is strictly read-only, omconfig is used to change system and OMSA behavior—including alerting, storage actions, and certain hardware-related settings exposed by OMSA.
Omconfig It operates by issuing commands to the OMSA backend services (primarily dsm_sa_datamgrd and dsm_sa_eventmgrd) and applying those changes at the software management layer, not directly to hardware firmware.
Common uses include enabling or disabling alert actions, configuring storage-related behavior, managing notification settings, and triggering maintenance operations such as consistency checks. Because omconfig can modify system behavior and suppress alerts, it should be used carefully—especially on production systems—and is best suited for controlled configuration changes rather than routine monitoring.
Because omconfig can modify system behavior and suppress alerts, it should be used carefully—especially on production systems—and is best suited for controlled configuration changes rather than routine monitoring.
So back to my two SAS cable alerts. Lets leverage omreport and omconfig to diagnose and hopefully ether determine that there is an actual issue, or suppress the alerts.
OMCONFIG and OMREPORT – Real world Usage
First lets take a sneaky-peak at my raid controller.
omreport storage controller
Controller PERC H730 Adapter(Slot 4)
Controller
ID : 0
Status : Ok
Name : PERC H730 Adapter
Slot ID : PCIe Slot 4
State : Ready
Firmware Version : 25.5.9.0001
-----truncated---
According to chatgpt I am getting these errors because the system expects that I have an external enclosure connected. I do not think that is the case, as that seems dumb. More than likely this alert is due to the fact that I am using an H730 (a 13G controller) (and its cables) in an R720 (a 12G server) in Slot 4, which is a PCIe Adapter and not the integrated “mini” version.
So let’s look at the disks and make sure all is well
Here we can see that the PERC H730 adapter does not “See” the connector states as connected, rather just ready
omreport storage connector controller=0
List of Connector(s) on Controller PERC H730 Adapter (Slot 4)
ID : 0
Status : Ok
Name : Connector 0
State : Ready
Connector Type : SAS Port RAID Mode
Termination : Not Applicable
SCSI Rate : Not Applicable
ID : 1
Status : Ok
Name : Connector 1
State : Ready
Connector Type : SAS Port RAID Mode
Termination : Not Applicable
SCSI Rate : Not Applicable
In order to use this 13th gen raid controller in a 12th gen Dell Server I could not use the original SAS cables. Rather I used SFF-8087 to SFF-8643 cables Which can cause the sideband signals (SGPIO / SES) which tell the iDRAC everything is okay which is why we are seeing the errors.
As stated above this is not actually a problem as I can see my disks. Lets do that below.
“The NVIDIA CUDA Toolkit is a software development kit that helps users create GPU-accelerated applications. It includes libraries, compilers, debuggers, and optimization tools”. Since we have an Nvidia GPU we will install it as shown below.
Note: If you installed the nvidia-cuda-toolkit from default ubuntu noble repo, uninstall it first as that we do not want conflicts. Uninstall with “sudo apt purge nvidia-cuda* -y“. More info on ubuntu repo install vs nvidia repo install later in post.
First we wget the Nvidia Cuda Repository pin as shown above. Then we move it to /etc/apt/preferences.d/ (shown below).
This is a configuration file that is used to prioritize packages from the NVIDIA CUDA repository when installing CUDA on a Linux system. Basically it tells apt where to get the Cuda Toolkit.
Installing the Nvidia Cuda toolkit from Ubuntu Vs Nvidia
The key differences between installing the NVIDIA CUDA Toolkit from the Ubuntu repositories vs. NVIDIA’s official website come down to version availability, ease of installation, and compatibility. Here’s are a few differences:
1. Installing from Ubuntu Repositories (apt install nvidia-cuda-toolkit)
Pros:
Easier installation – The package is well-integrated into Ubuntu’s package management (apt), meaning dependency resolution is generally smoother.
More stable – The versions in the repositories are tested against the Ubuntu ecosystem to ensure compatibility.
Automatic updates – Managed through apt, making it easier to keep up with system updates.
Cons:
Often outdated – Ubuntu’s repositories may not have the latest CUDA version, which can be an issue if you need newer features or performance improvements.
Limited driver version control – The version of CUDA available in the Ubuntu repos may not always match the latest NVIDIA drivers, which can lead to compatibility issues.
2. Installing from NVIDIA’s Official Website
Pros:
Latest version – You get the newest CUDA Toolkit, which can include performance improvements, bug fixes, and new features.
Better control over versions – You can install specific versions to match requirements for AI/ML frameworks like TensorFlow or PyTorch.
More driver compatibility – NVIDIA ensures that the toolkit and drivers are aligned, reducing the risk of version mismatches.
Cons:
More manual setup – You have to install dependencies manually, such as the correct NVIDIA driver, and configure paths properly.
Updates are not automatic – Unlike apt, updates need to be managed manually.
“The NVIDIA CUDA Toolkit is a software development kit that helps users create GPU-accelerated applications. It includes libraries, compilers, debuggers, and optimization tools”. Since we have an Nvidia GPU we will install it as shown below.
Note: If you installed the nvidia-cuda-toolkit from default ubuntu noble repo, uninstall it first as that version is probably too old. Uninstall with “sudo apt purge nvidia-cuda* -y“
First we wget the Nvidia Cuda Repository pin as shown above. Then we move it to /etc/apt/preferences.d/ (shown below).
This is a configuration file that is used to prioritize packages from the NVIDIA CUDA repository when installing CUDA on a Linux system. Basically it tells apt where to get the Cuda Toolkit.
Nvidia states that you need to add the following env vars to your .bashrc, however you will also need to add them to your python virtual env — in venv/bin/activate. These variables will be needed during the install process below.
Nvidia System Management interface (nvidia-smi) is a cli command which facilitates management and monitoring of Nvidia GPUs (mainly Tesla, GRID, Quatro, and Titan products). It ships with Nvidia GPU drivers on Linux. It is an extension buildt on the Nvidia Management Library. Official documentation for nvidia-smi can be found here.
The output comprises two tables. The first table provides comprehensive details about all detected GPUs (e.g., one GPU in the provided example), while the second table enumerates the processes actively utilizing the GPUs. Below are detailed explanations of each parameter:
Temp (Temperature): Indicates the GPU core temperature in Celsius. Typically, temperature regulation is managed by data center infrastructure or external cooling solutions. Values like “44°C” are normal operating conditions, but sustained temperatures exceeding 90°C should trigger immediate action to prevent hardware degradation.
Perf (Performance State): Represents the current performance state of the GPU, ranging from P0 (highest performance) to P12 (lowest performance).
Persistence-M (Persistence Mode): Specifies whether the NVIDIA driver remains loaded in memory even in the absence of active processes like nvidia-smi. When “On,” this mode reduces driver load latency for GPU-dependent applications such as CUDA workloads.
Pwr: Usage/Cap (Power Usage/Capacity): Displays the current power draw of the GPU relative to its total power capacity, measured in Watts.
Bus-Id: Represents the PCI bus address of the GPU in the format domain:bus:device.function (hexadecimal). This identifier is critical for targeting specific GPUs in systems with multiple devices.
Disp.A (Display Active): Denotes whether memory on the GPU is allocated for display purposes. An “Off” value signifies no display context is associated with the GPU, making it dedicated to compute tasks.
Memory-Usage: Indicates memory utilization on the GPU, expressed as the amount of memory in use versus total available memory. Machine learning frameworks like TensorFlow may preallocate the full GPU memory capacity upon initialization, irrespective of immediate requirements.
Volatile Uncorr. ECC (Volatile Uncorrected ECC): Tracks uncorrected memory errors since the last driver initialization. Error Correction Code (ECC) is designed to detect and correct memory errors, ensuring data integrity during GPU operations.
GPU-Util (GPU Utilization): Reports the percentage of time over the sample interval during which one or more kernels actively used the GPU.
Compute M. (Compute Mode): Specifies the GPU’s compute mode. In “Default” mode, multiple processes can access the GPU concurrently. Other modes may restrict access to a single process or prohibit access entirely.
GPU (Index): Enumerates the GPUs detected in the system. The index corresponds to the NVML (NVIDIA Management Library) device index, enabling precise identification in multi-GPU environments.
PID (Process ID): Lists the process identifier of applications utilizing GPU resources.
Type: Describes the context of GPU usage—“C” for Compute tasks, “G” for Graphics tasks, and “C+G” for combined Compute and Graphics contexts.
Process Name: Identifies the executable or application utilizing GPU resources.
GPU Memory Usage: Reports the GPU memory utilized by each individual process.
# nvidia-smi --query-gpu=name,temperature.gpu --format=csv
name, temperature.gpu
Tesla T4, 34
Tesla T4, 30
Query PCI Slot
# nvidia-smi --query-gpu=index,name,pci.bus_id --format=csv
index, name, pci.bus_id
0, Tesla T4, 00000000:02:00.0
1, Tesla T4, 00000000:43:00.0
Show Numa Affinity
Non-Uniform Memory Access (numa) is a term used on systems with more than one bus/CPU. In the example below my GPU is installed in the numa node local to CPU0. use the flag “topo -m”
Show Running Stats with dmon
The dmon flag is unsed to show running statistics for one or more CPUs at 1s intervals. Dmon accepts a slew of options which are explained here.
Below are the available base metrics and associated metric letter.
SWITCH
DESCRIPTION
p
Power Usage and Temperature
u
Utilization
c
Proc and Mem Clocks
v
Power and Thermal Violations
m
FB, Bar1 and CC Protected Memory
e
ECC Errors and PCIe Replay errors
t
PCIe Rx and Tx Throughput
The nvidia-smi dmon command is also able to query available GPM (GPU Performance Monitor) metrics as shown in the example below
#nvidia-smi dmon --gpm-metrics <gpmMetric1, gpmMetric2, ... ,gpmMetricN>
<gpmMetricX> R
The table below shows some of the available metrics and associated metric number. A complete list of metrics can be found here.
METRIC
VAR
DESCRIPTION
Graphics Activity
= 1
SM Activity
= 2
SM Occupancy
= 3
Integer Activity
=4
Tensor Activity
= 5
DFMA Tensor Activity
= 6
HMMA Tensor Activity
= 7
IMMA Tensor Activity
= 9
DRAM Activity
= 10
FP64 Activity
= 11
FP32 Activity
= 12
FP16 Activity
= 13
PCIe TX
= 20
PCIe RX
= 21
NVDEC 0-7 Activity
= 30-37
NVOFA 0 Activity
= 50
NVJPG 0-7 Activity
= 40-47
NVLink Total RX
= 60
NVLink Total TX
= 61
NVLink L0-17 RX
= 62, 64, 66, …, 96
NVLink L0-17 TX
= 63, 65, 67, …, 97
Enable persistence mode to reduce initialization overhead and keep the GPU active and running.
# sudo nvidia-smi -pm 1
Enabled persistence mode for GPU 00000000:43:00.0.
All done.
Installing the Nvidia Container Toolkit on Ubuntu 22.04
The NVIDIA Container Toolkit is a set of tools that enables the use of NVIDIA GPUs within Docker and other container runtimes. It allows GPU-accelerated applications to run inside containers by providing the necessary drivers, libraries, and runtime components. Instructions for installing the Nvidia Container Toolkit are below. The official Nvidia doc can be found here, where you can also find guides for installing via DNF/Yum or Zypper
Configure the NVIDIA-Container Toolkit with Containerd
For Ubuntu, the default runtime is containerd. In the example below we configure integration with containerd. Which modifies /etc/containerd/config.toml
You will need an Nvidia Cloud account and and API key, follow the setup guide here to get started.
Then docker login as shown below using your API key as your password
# docker login nvcr.io
Username: $oauthtoken
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
nvidia-ctk
A Container Device Interface (CDI) device is a standard way to manage container hardware access. More specifically it is used to assign GPU to containers through the Nvidia Container Toolkit
Run the command below to generate the CDI specification file
Then run the list command below to see what GPUs were detected.
# nvidia-ctk cdi list
INFO[0000] Found 5 CDI devices
nvidia.com/gpu=0
nvidia.com/gpu=1
nvidia.com/gpu=GPU-1d877ac8-5df1-34b0-4f86-59945e37d2ba
nvidia.com/gpu=GPU-9491a3e6-ea29-ba4e-4403-083244d5575c
nvidia.com/gpu=all
Running a Sample Docker Workload
Run the command below to test to make sure docker is working properly
# sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
The command above should run nvidia-smi once and then exit.
GPU Burn With Docker
Pop a second terminal window, run nvtop and run the command below. You should see load on your GPUs. In the example below, 60 represents the number of seconds to run the test.
# sudo docker run --gpus all --rm oguzpastirmaci/gpu-burn 60
I just recently posted about installing a GPU Power Supply Expansion Board into the Dell T620. I suggest giving that post a read before you continue, as this will be post builds upon what was discussed previously. This post will be a bit of a deep dive into what you should consider when choosing a GPU for your T620.
Ok, so first things first, you need to choose a GPU that is compatible with your T620, as not everything is going to work. Choosing a GPU that is a good fit for your workload, well that comes second.
Let’s start by looking at what Dell tested and certified.
Officially Supported GPUs for the T620
Manufacturer
Model
Heatsink
Power Usage
Nvidia
Tesla C2075
Active
215W
Nvidia
Quadro K4000
Active
80W
Nvidia
Quadro Q6000
Active
204W
Nvidia
Tesla K20A
Active
225W
ATI™ FirePro™
V7800
Active
150W
AMD FirePro
W7000
Active
127.7W
According to the documentation, only the above GPUs are “supported” on the Dell T620. Note that this really means that these are the only cards that Dell certified/tested. It does not mean that there are not any other compatible cards out there. These GPUs are honestly quite old. The only still relevant use case for any of them in this day and age is video transcoding on the quadro card. However, you might be able to do some lightweight AI/ML work on the Tesla K20A (2688 Cuda Cores, Kepler Architecture)
Dell documentation further expands on what is supported. Specifically…
Up to four 300W, full-length, single- or double-wide GPU cards
GPUs with up to 6GB of dedicated DDR5 memory
GPUs that are actively cooled.
No more than 2 power connectors per card
Power connectors located on the back of the card
GPU enablement kit (Power & Cooling Required)
Additionally Dell documentation states the following…
All cards are the same make/model
1000W power supplies are required (technically for systems with more than 2 cards)
You need not to cram your T620 too full with GPUs and Raid Controllers to avoid overheating
What About other GPUs?
So this is where we start to go off the rails, and venture into the unknown. Let’s start with what we know, which is what Dell officially supported and what their documentation says about choosing a GPU
Is 6GB the VRAM Limit?
At the time, Dell only were able to test and certify a handful of GPUs, and none of those GPUs had more than 6GB of VRAM. So it’s very possible that this is why Dell documentation tops out at 6GB as supported. I doubt this is an actual real-world limit.
The T620 was released in 2012 and probably went end-of-life in 2017 – 2019. None of the officially “supported” cards were released after 2013. Available GPUs with more than 6GB VRAM were probably not widely available, and if they were, Dell probably had finished their initial round of GPU certification and moved on to better things. Any GPUs with more than 6GB would probably run hotter and consumed more power, which could be an issue. Additionally there could be conflicts at the firmware/bios level for mapping/managing more than 6GB of VRAM. Who knows? Generically, they probably drew the line at 6GB because “that is what they knew worked”.
What about Power and Cooling?
Adding GPUs to a system will increase cooling and power needs. Replacing existing PSUs, with 1000w PSUs (as recommended by Dell) is very simple and actually pretty cheap. I think I picked up mine for less than $30 USD a pair on ebay. Dual 1000w PSUs was the only supported configuration if you wanted to order a T620 with GPU. However, you still needed to get that power to the GPUs themselves.
This is where the GPU Enablement Kit comes into play. The enablement kit consists of…
Power expansion board
GPU Power Cables
A Fan Gantry (shown below)
T620 Fan Gantry
The fan gantry is not easily found, and they are expensive. You will probably spend more on one of these that you did for your T620. They run about $300+ USD on ebay.
I have found 3 part numbers on line
0VDY5 – fan
8G79K – gantry
2R4DV – Fan + Gantry?
The fan gantry from a T630 will also work as well, albeit with a couple of slight modifications. As you need to cut a bit of metal, took me about 2 minutes to modify mine with some tin snips.
The TOP square I modified a slight bit with tin snips. You will need to do the same for a 4 squares
Note that you will need T620 Fans, as the T630 fan plugs do not line up with the power receiver on the motherboard (hence why you need to modify the gantry)
T620 Fan on the left. T630 Fan on the right. Notice the offset of the power plug is different between the two server models.
Note that the Fan Gantry for the T630 is not any cheaper, it’s just a bit easier to find . The part number for the T620 fan is 0TW71C, which is the same fan that are at the rear of the baffle.
Adding non-Dell proprietary fans to the chassis of the T620 is either much harder or much more expensive, as most of the unused power connectors in the chassis are proprietary. There are also no standard mounting brackets for common fan sizes.
However, there is power to be had. There is connector coming off the drive backplane that provides SATA power to the CDROM. And in theory you could do this…
Male Sata–Female Molex == Male Molex–Female Molex == Fan Controller with 2pin Molex
Now you can add a fan controller and can add some more fans, but you will have to get creative. A few pci slot blower fans? PCI slot Graphics Card Cooler possibly?
Similar Alternative GPUs with Active Cooling
Assuming we need to keep to similar power usage and VRAM, let’s look at a few older, yet not as anemic GPUs.
1. NVIDIA GeForce GTX 1660 Super
VRAM: 6GB GDDR6
TDP: 125W
Cooling: Actively cooled
Performance: Great for basic AI/ML tasks, especially if you are focused on inference or lighter training workloads.
2. NVIDIA RTX A2000 (Professional Card)
VRAM: 6GB GDDR6
TDP: 70W
Cooling: Actively cooled, compact design
Performance: A professional GPU tailored for AI/ML workloads and well-suited for power-constrained environments.
Performance: Suitable for lightweight AI/ML applications and inference tasks, ideal if energy efficiency is a priority.
What about GPUs without External Power
No power supply expansion board?
Let’s look at GPUs that do not require external power. Most PCIe 3.0 slots provide only 25 watts of power, however 75W is the maximum power delivery from a single PCIe 3.0 slot. The Dell T620 has 4 such X16 slots. This alone is appealing and is one of the reasons the Dell T-Series are popular with home labbers. By contract, my newer R630 only provides max of 25W per PCIe slot.
T620 Power Provided Per PCIE Slot
SLOT
POWER
CPU CONNECTION
Slot 1 (x8 lanes)
up to 25W
CPU1
Slot 2 (x16 lanes)
up to 75W
CPU1
Slot 3 (x4 lanes)
up to 25W
Platform Controller Hub
Slot 4 (x16 lanes)
up to 75W
CPU1
Slot 5 (x16 lanes)
up to 75W
CPU2
Slot 6 (x8 lanes)
up to 25W
CPU2
Slot 7 (x16 lanes)
up to 75W
CPU2
GPUs that run on 75W or less
Below is a list of possible GPUs that you can run without additional power requirements, other than what select PCIe slots in the T620 will provide.
NVIDIA GPUs
1. NVIDIA Tesla T4
Performance: Excellent for AI inference and light training.
Power: 70W (fits within PCIe 3.0 constraints).
VRAM: 16GB GDDR6.
Features: Tensor Cores, CUDA support, and FP16/FP32/INT8 capabilities for AI/ML.
Notes: Designed for data centers and supports efficient AI workloads.
Cooling: Passive (gets super hot)
2. NVIDIA Quadro T1000 / T2000
Performance: Entry-level professional GPUs for AI tasks.
Power: 50-75W.
VRAM: 4GB-6GB GDDR6.
Features: CUDA cores and optimized drivers for compute tasks.
Notes: Focused on balanced performance and efficiency.
Cooling: Active
3. NVIDIA GeForce GTX 1650 (Low Profile/Standard)
Performance: Suitable for light AI/ML workloads.
Power: 75W.
VRAM: 4GB GDDR5/GDDR6.
Features: CUDA support for training and inference.
Notes: Ensure you get a variant without external power connectors.
Cooling: Active
4. NVIDIA GeForce GTX 1050 Ti
Performance: Basic AI/ML tasks and experiments.
Power: 75W.
VRAM: 4GB GDDR5.
Features: CUDA cores and basic AI capabilities.
Notes: An older but efficient card for small-scale projects.
Cooling: Active
AMD GPUs
1. AMD Radeon RX 6400
Performance: Entry-level GPU for basic AI workloads.
Power: ~53W.
VRAM: 4GB GDDR6.
Features: RDNA 2 architecture with good efficiency.
Notes: Limited AI-focused features but sufficient for lightweight tasks.
Cooling: Active
2. AMD Radeon Pro WX 3200
Performance: Professional GPU for light compute tasks.
Power: 50W.
VRAM: 4GB GDDR5.
Features: Optimized drivers for professional workloads.
Notes: Reliable for basic AI workloads and professional use.
Cooling: Active
Intel GPUs
1. Intel Arc A310
Performance: Basic GPU for lightweight AI and inference.
Power: ~75W.
VRAM: 4GB GDDR6.
Features: Support for AI frameworks like TensorFlow and PyTorch.
Notes: Newer entry from Intel with decent AI potential.
Cooling: Active
What about Modern GPUs?
Let me start out by saying that it’s possible that many present day GPUs might actually operate without issue in the T620. That being said, what you risk is the bios/lifecycle controller throwing an error as it does not like what it sees happening in its slot.
A bus fatal error was detected on a component at slot 7
Above is the error that I encountered attempting to run a Telsa T4 card. I attempted with several different 16x slots, and the error moved with each slot that I tried. This error usually resulted in an OS kernel panic. In my case, its possible that the Nvidia T4, being passively cooled, could have been part of the problem, the card was too hot to touch. Although temperature could have had nothing to do with it and the card may just not “work” in the T620 (at least not yet, as I continue to troubleshoot)
Regarding newer cards, through my research I have found that a number of folks have had good luck with a few different CPUs (See Reference Section Below). Bottom line, you may get lucky.
Myself, well I am not giving up on getting the Nvidia T4 to run in my T620. I still have a some experimentation to do. Mainly some bios config tweaking and a bit of add on cooling. I also have some ideas regarding the lifecycle controler, but I have yet to test anything.
Troubleshooting/Stability
Below are a few ideas that I have regarding troubleshooting, good luck. I will create a new post at some point in the near future regarding my sucess with the Nvidia T4 GPU.
Slot Disablement – disable unused PCIE slots. PCIE power stability is especially important
“Boot Driver Disabled”. Will investigate this setting. This allow the slot to be visible to the OS, but will not be available as a boot device.
BIOS upgrade – make sure you are running the latest bios (2.9.0)
PCI Slot Selection – choose 16x slot closest to a CPU.
I have seen that many have success in slot 5
Cooling – Make sure your GPU is not getting too hot.
Pull hard disks to improve air flow on the right side of chassis
PCI slot cooler
PCI slot blower
Disable Dell Lifecycle Controller?
Disable Collect System Inventory On Restart (CSIOR)?
Conclusion
Bottom line, these machines were not exactly designed to run with GPUs, and only a handful of GPUs were actually tested and supported on the Dell T620. The list of supported cards is short, and these GPUs are quite old. However, just because its not supported, does not mean that it will not work. It just means it was not tested. Your mileage may vary. I’ve seen a number of post from individuals who are running more modern GPUs the T620 and they do so without issue.
A while back, I wrote a blog post that outlined a process of injecting deprecated storage controller drivers into RHEL 8 via a Driver Update Disk.
In a nutshell, this process allows you to install RHEL 8 on the R710/R610 (11th Gen)
This process worked fine, unless you wanted to yum update your server. Rather than attempt to find a repeatable process of injecting drivers prior to updating, I decided to upgrade my RAID controller to one that was supported.
My 11th generation, R710, came with a Perc H700. Identified below.
# lspci -knn | grep 'RAID bus controller'
03:00.0 RAID bus controller [0104]: Broadcom / LSI MegaRAID SAS 2108 [Liberator] [1000:0079] (rev 05)
Based on feedback from friends and co-workers. I purchased a PERC H330, which I believe ships with the 13th Gen Dell rackmount servers.
These cards can be found for pretty cheap on Ebay. Make sure that you order one with the correct bracket based on your needs (full-height/half-height).
Plus you will need 2 new cables. You are looking for SFF8643 to SFF8087 (mini SAS HD to mini SAS). They are also cheap and can be found on Ebay/Amazon
Output from one of my Dell R710s. Note I did not remove the original card.
# lspci -knn | grep 'RAID bus controller'
03:00.0 RAID bus controller [0104]: Broadcom / LSI MegaRAID SAS 2108 [Liberator] [1000:0079] (rev 05)
05:00.0 RAID bus controller [0104]: Broadcom / LSI MegaRAID SAS-3 3108 [Invader] [1000:005d] (rev 02)
Afterthoughts
I’ve only performed this swap on one of my R710s, I have 2 more as well as 2 R610s. Below are are few things to consider if you are looking to perform a similar upgrade
Take a hard look at your existing server. Do you need to purchase a full sized pci card or a mini card? This may likely be the case on the 1u servers and may be a blocker for you
Remove the old card. The server does not quite like having a raid controller installed without cables.
If you buy a full-sized pci card, make sure you get one with the bracket that you need (full height vs. half height vs. no bracket)
You might want to find a cable with 90 degree connectors for the backplane. Mine fit fine on the 710, but I have heard that the 90 degree connector is best for 1U servers.
You may want to consider upgrading your 11th generation server to an 12th generation Dell, R720/R620s can be purchased relatively cheaply and the RAM and disks in your 11th gen will work in a newer 12th generation. (note that the 11th gen Dell CPUs are socket LGA1366, while the 12th gen Dells utilize socket LGA2011)
