Tag: monitoring

  • RHEL 10 – Install and Configure NUT UPS Tools and Monitor Via Zabbix.

    RHEL 10 – Install and Configure NUT UPS Tools and Monitor Via  Zabbix.

    In my personal lab I do not have a network managed UPS. My rack mount UPS can only be monitored via USB from a directly connected host.

    This guide details the installation and configuration of Network UPS Tools (NUT) to monitor a Tripp Lite USB PDU and prepare it for Zabbix integration.


    Prerequisites & Installation

    Install the necessary NUT packages. RHEL 10 uses the 2.8.x branch, which introduces an “instantiated” service model.

    sudo dnf install nut nut-client nut-server
    

    Directory Structure Setup

    RHEL 10 defaults to /etc/ups, we will use /etc/nut to ensure our configuration files are in the correct location and we will create a symbolic link for legacy compatibility.

    # Move config if it exists in the old /etc/ups location
    sudo mkdir -p /etc/nut
    sudo mv /etc/ups/* /etc/nut/ 2>/dev/null
    sudo ln -s /etc/nut /etc/ups
    

    Configuration Files

    The following files in /etc/nut/ must be configured:

    nut.conf

    Set the operation mode:

    MODE=netserver
    

    NUT (Network UPS Tools) Netserver Mode is a configuration where one machine (the server) connects to a UPS, monitors it, and then broadcasts the power status and alerts (like “on battery”) across the local network

    ups.conf

    Define the hardware driver. Note the exact spelling of the section name [tripplite].

    [tripplite]
        driver = usbhid-ups
        port = auto
        desc = "Tripp Lite USB PDU"
    

    upsd.conf

    Control network listening. To allow remote connections (like from a Zabbix server), you must listen on the host’s IP or all interfaces (0.0.0.0).

    LISTEN 127.0.0.1 3493
    LISTEN 10.1.10.21 3493
    

    upsd.users

    Define a user for Zabbix or remote monitoring:

    [zabbix]
        password = your_secure_password
        upsmon master
    

    USB Permissions (udev)

    Create a rule to allow the nut user to access the USB bus for Tripp Lite devices (Vendor ID 09ae).

    Create file: /etc/udev/rules.d/99-nut-ups.rules and add content shown below.

    ACTION=="add", SUBSYSTEM=="usb", ATTR{idVendor}=="09ae", GROUP="nut", MODE="0660"

    Then apply

    sudo udevadm control --reload-rules && sudo udevadm trigger

    Firewall

    You need to allow inbound port 3493 since we want to be able to poll the ups remotely from a Zabbix host.

    # Open port 3493 permanently
    sudo firewall-cmd --permanent --add-port=3493/tcp
    sudo firewall-cmd --reload

    Service Management

    RHEL 10 uses a specific sequence. You must trigger the “enumerator” to generate the systemd units for your specific PDU name.

    # Force the system to recognize the [tripplite] section
    sudo /usr/libexec/nut-driver-enumerator.sh
    
    # Enable and start the driver instance
    sudo systemctl enable --now nut-driver@tripplite.service
    
    # Start the data server and monitoring daemon
    sudo systemctl enable --now nut-server.service
    sudo systemctl enable --now nut-monitor.service
    

    Manual Check Commands

    Use these commands to verify health:

    CommandPurpose
    upsc -lLists all configured UPS/PDU devices.
    upsc tripplite@localhostDumps all current PDU metrics (voltage, load, etc.).
    systemctl status nut-driver@trippliteChecks if the USB driver is active and running.
    journalctl -u nut-driver@tripplite -fFollow live logs if the USB connection drops.

    Test Remotely

    From another host, install the nut-client. If this step does not work, but the same commands worked locally, you should double check your Firewall settings.

    $ sudo apt install nut-client

    And test

    $ upsc tripplite@10.1.10.21

    Prepare the Zabbix Agent Permissions

    To allow Zabbix to collect this data automatically. Run the two commands below on the host connected to UPS via USB

    # sudo usermod -a -G nut zabbix
    # sudo systemctl restart zabbix-agent

    Create the Zabbix Agent UserParameters

    sudo vi /etc/zabbix/zabbix_agentd.d/nut_tripplite.conf

    Paste the following lines into the file above

    # Discovery rule to find the UPS name
    UserParameter=nut.ups.discovery,upsc -l | awk '{print "{\"{#UPSNAME}\":\"" $1 "\"}"}' | paste -sd "," | sed 's/^/{"data":[/' | sed 's/$/]}/'
    
    # Fetch specific values (e.g., ups.load, output.voltage)
    UserParameter=nut.ups.value[*],upsc $1@localhost $2 | grep -v "Value:"

    Now restart the zabbix agent

    # sudo systemctl restart zabbix-agent

    Now verify that the Zabbix agent returns UPS data

    # zabbix_agentd -t "nut.ups.value[tripplite,ups.load]"
    nut.ups.value[tripplite,ups.load]             [t|24]
    

    Setting up the Zabbix Web UI

    Lets create a simple template for zabbix. Follow the steps shown below.

    Manual Item Creation

    If you just want the most important metrics, go to your Host in Zabbix and create these Items:

    NameTypeKeyType of Information
    PDU LoadZabbix Agentnut.ups.value[tripplite,ups.load]Numeric (float)
    Input VoltageZabbix Agentnut.ups.value[tripplite,input.voltage]Numeric (float)
    Battery ChargeZabbix Agentnut.ups.value[tripplite,battery.charge]Numeric (float)
    PDU StatusZabbix Agentnut.ups.value[tripplite,ups.status]Character

    Configuring SELinux

    If you check the zabbix webui, you are going to see some failures, as we need to configure selinux.

    Note that we configured Zabbix “items” first so that we would be able to capture the failures in the audit log and generate a policy.

    So now that we have failures, we can tell RHEL to look at all recent security denials involving the zabbix_agent and write a “policy” that allows those specific actions.

    # Search the audit log for zabbix-related denials and create a policy module
    grep "zabbix" /var/log/audit/audit.log | audit2allow -M zabbix_nut_final

    Install the Policy

    The previous command created a file called zabbix_nut_final.pp. Now, load it into the kernel:

     semodule -i zabbix_nut_final.pp

    You should now see data returning in the Zabbix WebUI. You may need to wait a few second or re-run the check.

    Screenshot of a Zabbix web interface displaying the item 'Battery Charge' with details on the last check and last value.

  • Centos7/Rhel7 – Collectd Config for Libvirt, Carbon-Cache

    collectd-logo

    The following is the collectd config that I am running on my RHEL 7 kvm hypervisor. This is not meant to be an all inclusive config for collectd, rather I am looking to gather basic performance metrics on my hypervisors and VMs.

    Note that I have disabled selinux, as I am running these hypervisors in my lab. Do not do the same in your production environments.

    
    #setenforce 0
    

    I also edited /etc/selinux/config as shown below. Again, this is for non-prod/test envs. Do not disable in production

    
    # This file controls the state of SELinux on the system.
    # SELINUX= can take one of these three values:
    #     enforcing - SELinux security policy is enforced.
    #     permissive - SELinux prints warnings instead of enforcing.
    #     disabled - No SELinux policy is loaded.
    SELINUX=disabled
    # SELINUXTYPE= can take one of three two values:
    #     targeted - Targeted processes are protected,
    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    #     mls - Multi Level Security protection.
    SELINUXTYPE=targeted 
    

    Also note that I have already configured the EPEL repo.. again, not production.

    EPEL repo installed via the command below

    
    rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    

    First a couple of steps to get us up and running. We need to install packages

    
    # yum -y install collectd collectd-virt.x86_64 collectd-rrdtool.x86_64 collectd-sensors.x86_64 collectd-smart.x86_64 collectd-netlink.x86_64 collectd-ipmi.x86_64
    

    We also need to write our config.

    You can pull my config using the gist below


    #Custom Collectd Config
    LoadPlugin "write_graphite"
    <Plugin "write_graphite">
    <Node "example">
    Host "10.1.0.202"
    Port "2003"
    Prefix "collectd."
    #Postfix ""
    Protocol "tcp"
    #Protocol "udp"
    #LogSendErrors false
    EscapeCharacter "_"
    SeparateInstances true
    StoreRates false
    AlwaysAppendDS false
    </Node>
    </Plugin>
    LoadPlugin cpu
    LoadPlugin load
    LoadPlugin memory
    LoadPlugin processes
    LoadPlugin disk
    <Plugin "disk">
    Disk "sda1"
    Disk "sdb1"
    Disk "sdc1"
    Disk "dm-0"
    Disk "dm-2"
    Disk "dm-3"
    IgnoreSelected true
    </Plugin>
    <LoadPlugin virt>
    Globals false
    </LoadPlugin>
    <Plugin "virt">
    Connection "qemu:///system"
    RefreshInterval 60
    Domain "dom0"
    BlockDevice "name:device"
    InterfaceDevice "name:interface"
    IgnoreSelected true
    HostnameFormat "name"
    </Plugin>
    LoadPlugin interface
    <Plugin interface>
    Interface "lo"
    IgnoreSelected true
    </Plugin>
    Include "/etc/collectd.d"

    view raw

    collectd.conf

    hosted with ❤ by GitHub

    We now need to enable and start collectd

    
    #systemctl enable collect
    #systemctl start collectd 
    

     

  • Install and Configure SNMP on the Asus RT-AC66U Router

    ASUS_RT-AC66U_newsOk first off let me start by saying that this is probably the coolest piece of home computing hardware that I have ever laid my hands on. Yes the setup was easy, and yes the thing is rock solid, and yes wireless range is awesome. But for approx $200 USD you really should not expect anything less.  I’m not going to go into its specs or features, as I’ll leave that to the professionals. Read up on it here.

    Anyway out of the box it supported telnet, but I wanted ssh, so I dropped the default firmware and went with Asuswrt-Merlin. It was at this point I started to explore the Busybox OS and decided I wanted to monitor the device via my HomeLab Zenoss install.

    However, much to my chagrin net-snmp was not installed out of the box.

    So how do you install it you ask? Would you believe via a package manager?

    First, you need to find the package name

    #ipkg list | grep snmp

    Then install the snmp package

    #ipkg install net-snmp

    Then configure it to start at boot time.

    #app_set_enabled.sh net-snmp yes

    In order to configure it, you are going to have to search for the snmp.conf

    #find / -name snmpd.conf

    I found two files and one of them clearly states that you should not edit it directly. The other one does not so this is the one that I modified to include my custom rocommunity. See below.

    rocommunity  lab

    Now I just need to figure out how to allow port 161 udp/tcp on my local LAN segment and I am in business. However, I will probably tackle that tomorrow.

     

    Additional Info (2016)

    Note, you can restart snmp as shown below.

    Stopping:

    admin@RT-AC66U:/tmp/home/root# app_stop.sh net-snmp
    killall: dm2_transmission-daemon: no process killed
    killall: asus_lighttpd: no process killed
    killall: dm2_snarfmaster: no process killed
    killall: dm2_nzbget: no process killed
    killall: dm2_amuled: no process killed
    iptables: No chain/target/match by that name
    iptables: Bad rule (does a matching rule exist in that chain?)

    Starting:

    admin@RT-AC66U:/tmp/home/root# app_set_enabled.sh net-snmp yes
    The field(Enabled) was set “yes” already.
    Restarting the package…

     

    2016 Update – Configuring SNMP via the WebUI

    Figured that I would add an update to this post as it seems that there are plenty of folks who are looking to setup SNMP on their Asus routers. Note that SNMP can now be configured directly from the WebUI.

    In the left pane, click on “Advanced Setting”. Then click on the “SNMP” tab. See example below. Note that the webui does not seem to pick up your configuration if you have configured it via the cli. I have not tested to see if the WebUI overwrites the CLI configuration or if it creates another configuration file.

    asus-4.png

    Additional Resources

    Fatmin: How to Add a Static Route on the Asus RT-AC66U

    Fatmin: Install and Configure SNMP on the Asus RT-AC66U