Tag: centos

  • RHEL 10 – Install and Configure NUT UPS Tools and Monitor Via Zabbix.

    RHEL 10 – Install and Configure NUT UPS Tools and Monitor Via  Zabbix.

    In my personal lab I do not have a network managed UPS. My rack mount UPS can only be monitored via USB from a directly connected host.

    This guide details the installation and configuration of Network UPS Tools (NUT) to monitor a Tripp Lite USB PDU and prepare it for Zabbix integration.


    Prerequisites & Installation

    Install the necessary NUT packages. RHEL 10 uses the 2.8.x branch, which introduces an “instantiated” service model.

    sudo dnf install nut nut-client nut-server
    

    Directory Structure Setup

    RHEL 10 defaults to /etc/ups, we will use /etc/nut to ensure our configuration files are in the correct location and we will create a symbolic link for legacy compatibility.

    # Move config if it exists in the old /etc/ups location
    sudo mkdir -p /etc/nut
    sudo mv /etc/ups/* /etc/nut/ 2>/dev/null
    sudo ln -s /etc/nut /etc/ups
    

    Configuration Files

    The following files in /etc/nut/ must be configured:

    nut.conf

    Set the operation mode:

    MODE=netserver
    

    NUT (Network UPS Tools) Netserver Mode is a configuration where one machine (the server) connects to a UPS, monitors it, and then broadcasts the power status and alerts (like “on battery”) across the local network

    ups.conf

    Define the hardware driver. Note the exact spelling of the section name [tripplite].

    [tripplite]
        driver = usbhid-ups
        port = auto
        desc = "Tripp Lite USB PDU"
    

    upsd.conf

    Control network listening. To allow remote connections (like from a Zabbix server), you must listen on the host’s IP or all interfaces (0.0.0.0).

    LISTEN 127.0.0.1 3493
    LISTEN 10.1.10.21 3493
    

    upsd.users

    Define a user for Zabbix or remote monitoring:

    [zabbix]
        password = your_secure_password
        upsmon master
    

    USB Permissions (udev)

    Create a rule to allow the nut user to access the USB bus for Tripp Lite devices (Vendor ID 09ae).

    Create file: /etc/udev/rules.d/99-nut-ups.rules and add content shown below.

    ACTION=="add", SUBSYSTEM=="usb", ATTR{idVendor}=="09ae", GROUP="nut", MODE="0660"

    Then apply

    sudo udevadm control --reload-rules && sudo udevadm trigger

    Firewall

    You need to allow inbound port 3493 since we want to be able to poll the ups remotely from a Zabbix host.

    # Open port 3493 permanently
    sudo firewall-cmd --permanent --add-port=3493/tcp
    sudo firewall-cmd --reload

    Service Management

    RHEL 10 uses a specific sequence. You must trigger the “enumerator” to generate the systemd units for your specific PDU name.

    # Force the system to recognize the [tripplite] section
    sudo /usr/libexec/nut-driver-enumerator.sh
    
    # Enable and start the driver instance
    sudo systemctl enable --now nut-driver@tripplite.service
    
    # Start the data server and monitoring daemon
    sudo systemctl enable --now nut-server.service
    sudo systemctl enable --now nut-monitor.service
    

    Manual Check Commands

    Use these commands to verify health:

    CommandPurpose
    upsc -lLists all configured UPS/PDU devices.
    upsc tripplite@localhostDumps all current PDU metrics (voltage, load, etc.).
    systemctl status nut-driver@trippliteChecks if the USB driver is active and running.
    journalctl -u nut-driver@tripplite -fFollow live logs if the USB connection drops.

    Test Remotely

    From another host, install the nut-client. If this step does not work, but the same commands worked locally, you should double check your Firewall settings.

    $ sudo apt install nut-client

    And test

    $ upsc tripplite@10.1.10.21

    Prepare the Zabbix Agent Permissions

    To allow Zabbix to collect this data automatically. Run the two commands below on the host connected to UPS via USB

    # sudo usermod -a -G nut zabbix
    # sudo systemctl restart zabbix-agent

    Create the Zabbix Agent UserParameters

    sudo vi /etc/zabbix/zabbix_agentd.d/nut_tripplite.conf

    Paste the following lines into the file above

    # Discovery rule to find the UPS name
    UserParameter=nut.ups.discovery,upsc -l | awk '{print "{\"{#UPSNAME}\":\"" $1 "\"}"}' | paste -sd "," | sed 's/^/{"data":[/' | sed 's/$/]}/'
    
    # Fetch specific values (e.g., ups.load, output.voltage)
    UserParameter=nut.ups.value[*],upsc $1@localhost $2 | grep -v "Value:"

    Now restart the zabbix agent

    # sudo systemctl restart zabbix-agent

    Now verify that the Zabbix agent returns UPS data

    # zabbix_agentd -t "nut.ups.value[tripplite,ups.load]"
    nut.ups.value[tripplite,ups.load]             [t|24]
    

    Setting up the Zabbix Web UI

    Lets create a simple template for zabbix. Follow the steps shown below.

    Manual Item Creation

    If you just want the most important metrics, go to your Host in Zabbix and create these Items:

    NameTypeKeyType of Information
    PDU LoadZabbix Agentnut.ups.value[tripplite,ups.load]Numeric (float)
    Input VoltageZabbix Agentnut.ups.value[tripplite,input.voltage]Numeric (float)
    Battery ChargeZabbix Agentnut.ups.value[tripplite,battery.charge]Numeric (float)
    PDU StatusZabbix Agentnut.ups.value[tripplite,ups.status]Character

    Configuring SELinux

    If you check the zabbix webui, you are going to see some failures, as we need to configure selinux.

    Note that we configured Zabbix “items” first so that we would be able to capture the failures in the audit log and generate a policy.

    So now that we have failures, we can tell RHEL to look at all recent security denials involving the zabbix_agent and write a “policy” that allows those specific actions.

    # Search the audit log for zabbix-related denials and create a policy module
    grep "zabbix" /var/log/audit/audit.log | audit2allow -M zabbix_nut_final

    Install the Policy

    The previous command created a file called zabbix_nut_final.pp. Now, load it into the kernel:

     semodule -i zabbix_nut_final.pp

    You should now see data returning in the Zabbix WebUI. You may need to wait a few second or re-run the check.

    Screenshot of a Zabbix web interface displaying the item 'Battery Charge' with details on the last check and last value.

  • Configure RHEL7/Centos 7 as a Virtualization Host

    smarta

    This is a fresh install of RHEL 7.5

    First install the packages as shown below.

    yum install qemu-kvm libvirt
    

    Now install the additional recommened virtualization packages

    
    # yum install virt-install libvirt-python virt-manager virt-install libvirt-client 
    

    Now restart libvirtd

    
    # systemctl restart libvirtd
    

     
    Now you should be able to launch virt-manager from your remote machine and add a connection to your new virtualization host.

    Special note. Make sure that you disable NetworkManager

    
    # systemctl stop NetworkManager
    # systemctl disable NetworkManager
    
  • RHEL 7 Two-Factor SSH Via Google Authenticator

    650x300xgoogle-authenticator-header.png.pagespeed.gp+jp+jw+pj+js+rj+rp+rw+ri+cp+md.ic.194erIegOZ.jpg

    In this post,  I am going to walk you through the process of installing and configuring two- factor SSH authentication via Google Authenticator. My base system is running a fresh install of RHEL 7.2

    Installation Steps

    The first step on my system was to install autoreconf, automake, and libtool. These packages are required by the bootstrap.sh script that we will need to in a couple more steps.

    # yum -y install autoconf automake libtool

    Now, we are going to install Git.

    #yum -y install git

    One more dependency to knock out. Install pam-devel as shown below.

    # yum -y install pam-devel

    Next, we clone the google-authenticator Git repo. In this example, I am cloning to /root

    # git clone https://github.com/google/google-authenticator.git
    Cloning into 'google-authenticator'...
    remote: Counting objects: 1435, done.
    remote: Total 1435 (delta 0), reused 0 (delta 0), pack-reused 1435
    Receiving objects: 100% (1435/1435), 2.32 MiB | 0 bytes/s, done.
    Resolving deltas: 100% (758/758), done.

    Now change directory as shown below and run bootstrap.sh.

    # cd /root/google-authenticator/libpam
    # ./bootstrap.sh

    Now run the following commands to finalize the module installs.

    # ./configure
    #make
    #make install

    Assuming that you do not run into any errors, the following modules will be installed.

    • /usr/local/lib/security/pam_google_authenticator.so
    • /usr/local/lib/security/pam_google_authenticator.la
    (more…)
  • Redhat 6 Minimal Kickstart Configuration with VMware Tools and Puppet Agent Install

    smartaHere is my small, crude, little Kickstart configuration and post install script that I have up and running in my lab at home. Don’t expect to find anything too fancy here, as this Kickstart was purposefully built to be small and to the point. Here, the point was to spin up a VM, run through a basic install of CentOS/Redhat Linux,  and install VMware Tools along with a Puppet agent.

    Note that this post assumes that you have a working Kickstart server.

    First lets take a look at our kickstart file, CentOS-6.6-x86_64-minimal.ks

    The section directly below kicks off our kickstart ks file. Here we set our root password (no that’s not my hash) and setup our network interface for DHCP. We do a tiny bit of disk partitioning, and setup very simple LVM. Then we choose our packages. As you can see my package list is not at all fancy, I just want to make sure that I have pretty much every package that might need for a lab VM.

    [code language=”bash”]
    # Kickstart file for RHEL 6 Minimal
    # Small Disk

    text
    install
    url –url=http://10.1.0.106/ks/loop/CentOS-6.6-x86_64-bin-DVD1
    lang en_US.UTF-8
    keyboard us
    network –onboot yes –device eth0 –bootproto dhcp –noipv6
    rootpw –iscrypted $6$X/4YYZPN$4Sv.khxXms8N8vRssR/Vl35w/m80FF5P6p7aX0D7EFfD9p734F6tU4kXdcSCoOjPiXLrVxqfKxxxxxxxxxxxq5551
    firewall –disabled
    authconfig –enableshadow –passalgo=sha512
    selinux –permissive
    timezone America/New_York

    # Disk
    bootloader –location=mbr –driveorder=sda –append="crashkernel=auto rhgb"
    zerombr
    clearpart –all –drives=sda
    part pv.1 –grow –size=1
    part /boot –fstype=ext4 –size=1024
    volgroup VolGroup pv.1
    logvol / –fstype=ext4 –name=lv_root –vgname=VolGroup –size=1024 –grow
    logvol swap –name=lv_swap –vgname=VolGroup –size=1024

    #Network
    network –device=eth0 –bootproto=dhcp –nameserver=10.1.0.110

    # Package Selection
    %packages –nobase –excludedocs
    @Base
    @core
    kernel-headers
    wget
    perl
    sysstat
    bind-utils
    tcpdump
    [/code]

    Now let me pause to point out the section below. This is the %pre script that I am using to prompt me for the VM hostname before the install begins. The hostname needs to be set before you install puppet on the VM, otherwise you are going to have to recreate your puppet certificates after you set properly set your hostname post install and reboot.

    [code language=”bash”]
    %pre –log=/root/ks_pre.log
    #change to tty6 to get input
    chvt 6
    exec </dev/tty6 > /dev/tty6

    #Prompt for hostname
    echo "What is my hostname?"
    read NAME
    echo "NETWORKING=yes
    HOSTNAME=${NAME}" > network
    chvt 1
    [/code]

    Now we run a simple post install, along with a custom post install script. It is this script that will install Vmware tools and Puppet. Myself, I prefer keeping most of my code out of the actual Kickstart ks file, however you can always jam all your code into it if you like. You will just need to validate your syntax first, as I have not tested my config this way.

    [code language=”bash”]
    %post –nochroot
    # bring in hostname collected from %pre, then source it
    cp network /mnt/sysimage/etc/sysconfig/network
    . /mnt/sysimage/etc/sysconfig/network
    # force hostname change
    /mnt/sysimage/bin/hostname $HOSTNAME
    #Post Install
    %post –log=/root/ks-post.log
    cd /root
    echo "Getting the post install script – if this takes a long time check network or path"
    wget http://10.1.0.106/ks/scripts/centos-6-postinstall.bash
    echo "Running the post install script"
    /bin/bash centos-6-postinstall.bash
    [/code]

    Ok, so below is the post install script that I am calling in the section above. After a quick modification of my hosts file, I pull down the Puppet installer from my local Puppet server. Next we install the open source VMware tools packages, after creating the required yum repofile.

     

    [code language=”bash”]
    #!/bin/bash
    #centos-6-postinstall.bash

    #Switch to the 6th console and redirect all i/o
    exec < /dev/tty6 > /dev/tty6 2> /dev/tty6
    chvt 6

    # Lets make sure we know who the puppet server is before we get too far
    echo "Adding hosts entry for puppet master"
    echo "10.1.0.115 puppet puppet.lab.localdomain" >> /etc/hosts

    ## Update Via Yum – not doing this for now in order to save time
    #yum -y update
    # Install puppet from local puppet master
    echo "Downloading and running Puppet installer"
    curl -k https://10.1.0.115:8140/packages/current/install.bash | sudo bash
    #Install Open Source VMware Tools
    rpm –import http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-DSA-KEY.pub
    rpm –import http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-RSA-KEY.pub

    echo -e "[vmware-tools]\nname=VMware Tools\nbaseurl=http://packages.vmware.com/tools/esx/5.1latest/rhel6/$HOSTTYPE\nenabled=1\ngpgcheck=1" > /etc/yum.repos.d/vmware-tools.repo

    echo "Installing Vmware Tools"
    yum -y install vmware-tools-esx-nox

    #Minor grub.conf modifications
    sed -i ‘s/rhgb quiet//’ /boot/grub/grub.conf
    sed -i ‘s/hiddenmenu//’ /boot/grub/grub.conf
    sed -i ‘s/timeout=5/timeout=10/’ /boot/grub/grub.conf

    #Kick off first puppet run, for some reason I think you might need to do this twice.
    sleep 5
    echo "Running Puppet for the first time"
    puppet agent –test
    puppet agent –test

    #Tell us we have reached the end
    echo "We have reached the end of the post-install script"
    [/code]

    A couple of additional details to note about the post install script above. I like to modify the grub.conf so that I unhide the menu and increase the time out. I also like to make sure that we disable the Redhat graphical boot screen… I want to make sure its easy to catch any errors or miss-configurations in my kickstarts.
     

  • Getting Started with Git: Creating a Git Repo

    NettutsFetch-1So first off let me start by saying that I know that there is a ton of information out there on how to get started with Git. Heck, when you create your repo in GitLab it spits these instructions right out in front of your nose. However, what I have found is that most instructions tell you what to do to get started with git, however they do not tell you exactly what you are doing. You end up running a few command and then sit back and try to figure out what you actually just did.

     

    That being said, getting started with Git has been the hardest part of the process, as most of us traditional grey-beard sysadmins are not that familiar with code management. When I was first getting started in technology, there were developers and there were sysadmins, and those two worlds were extremely separate. Now, we as we enter the age of DEVOPS and automation these two worlds are once again colliding (like they did in the beginning, but more on that another day).

     

     In my lab I decided to build a stand alone vm for Git and Puppet. After doing some research – and asking others– on how and what to do to get Git up and running with a nice front end web interface, I decided to get started by installing the GitLab Omnibus package for Centos 6. This process was quick, easy, and painless.

     

    Once I had a working webUI up and running it was time to create my first repo. Instead of trying to accomplish this from my Fedora workstaion, I just clicked on the "New Project" button on the GitLab dashboard and created an empty repo called "General_Scripts".

     

    Back on my Fedora workstation I created a new directory in my home dir called git, and inside that directory I created a directory called "General_Scripts" as I had done in the webUI.

     

    Now it was time to use Git.

     

    First off you need to configure a few global Git options. This only needs to be done once.

    #git config –global user.name "Fatmin"

    #git config –global user.email "fatmin@fatmin.com"

     

    Once these global configs are set you can then you can move on to seeding your repo. Here you see my change directories to the Global_Scripts directory and tell Git to initialize this directory as a repo.

    #cd Global_Scripts

    #git init

     

    Now lets create a file and add it to the repo. Here were are going to create a simple README containing a description of my new repo. The instructions do not tell you that you have to put anything in this initial file, however what good is an empty README anyway

    #vi README.md

     

    Now tell git that this file needs to be added to the "General_Scripts" repo that we created a few steps ago.

    #git add README.md

     

    Now lets commit that file with a nice little comment. Commit comments should describe what the file we added or what we changed in an existing file

    #git commit -m "Added README.md"

     

    Now we need to tell our local git command what remote repo we are going to sync to. Note my git repo url is puppet.lab.localdomain, fatmin is my user's namespace, and General_Scripts is my repo.

    #git remote add origin git@puppet.lab.localdomain:/fatmin/General_Scripts.git

     

    Now we need to actually push the local files to the remote repo (origin) in the master branch.

    #git push -u origin master

     

    Now wait a bit and go check out the webUI. You should now see the README.md file in your new remote repo.

    Related articles

    Super Quick Git Guide
    Bashit… Just a Custom Bash Prompt Setup for Git
    Git and GitHub LiveLessons
  • RHEL6: Cool PAM Tricks – Logging Terminal Keystokes

    Pam-original-cooking-spray-72450This is a neat and very useful trick that I learned today. Lets say that you want to be able to monitor and log all keystrokes that are typed as root. This is particularly useful as normally you can only log when a user uses sudo to run a command. If the user has the abilty to become root however,  then they have effectively eluded yourattempts to track their activity. Like Thomas Magnum shaking a tail, they are free to scoot around your island with the top down.

    So how do you stop this from occuring? How to you log all activity and keystrokes made by root without implementing a bloated 3rd party software that will probably cost and arm and a leg? You use PAM you dingbat.
     

    The secret sauce in this security burrito is the pam_tty_audit.so module. Here is how to use it,

    Below is my stock /etc/pam.d/system-auth file

    #%PAM-1.0
    # This file is auto-generated.
    # User changes will be destroyed the next time authconfig is run.
    auth        required      pam_env.so
    auth        sufficient    pam_fprintd.so
    auth        sufficient    pam_unix.so nullok try_first_pass
    auth        requisite     pam_succeed_if.so uid >= 500 quiet
    auth        required      pam_deny.so

    account     required      pam_unix.so
    account     sufficient    pam_localuser.so
    account     sufficient    pam_succeed_if.so uid < 500 quiet
    account     required      pam_permit.so

    password    requisite     pam_cracklib.so try_first_pass retry=3 type=
    password    sufficient    pam_unix.so md5 shadow nullok try_first_pass use_authtok
    password    required      pam_deny.so

    session     optional      pam_keyinit.so revoke
    session     required      pam_limits.so
    session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
    session     required      pam_unix.so

    Now look above and then look below at my modified system-auth file. Note the additonal session entry for pam_tty_audit.so.

    [root@ip-172-31-21-28 pam.d]# cat system-auth
    #%PAM-1.0
    # This file is auto-generated.
    # User changes will be destroyed the next time authconfig is run.
    auth        required      pam_env.so
    auth        sufficient    pam_fprintd.so
    auth        sufficient    pam_unix.so nullok try_first_pass
    auth        requisite     pam_succeed_if.so uid >= 500 quiet
    auth        required      pam_deny.so

    account     required      pam_unix.so
    account     sufficient    pam_localuser.so
    account     sufficient    pam_succeed_if.so uid < 500 quiet
    account     required      pam_permit.so

    password    requisite     pam_cracklib.so try_first_pass retry=3 type=
    password    sufficient    pam_unix.so md5 shadow nullok try_first_pass use_authtok
    password    required      pam_deny.so

    session     optional      pam_keyinit.so revoke
    session     required      pam_tty_audit.so enable=root
    session     required      pam_limits.so
    session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
    session     required      pam_unix.so

    Note that you will need to add the pam_tty_audit.so call to your /etc/pam.d/password-auth as well to ensure that you capture all of root's keystrokes, no matter how they log in.

     

    Related articles

    Much Todo About Linux/RHEL Passwords
    RHEL6 – How to Manually Logout of an Iscsi Disk
    Fun With PAM: Working with pam_cracklib and pam_tally2
  • RHEL6: Composing Custom Auditd Rules for Fun and Profit

    297247-typewriterAuditd gives you the ability to write your own custom audit rules. This functionality allows an administrator to keep a close eye on system calls, file access, and user behavior. This added functionality is especially useful in environments that are requred to adhear to compliance standards that are above and beyond normal standards. Think PCI.

    Once of the simplest rules to add is a watch rule which can be set on files and directories. In the example below we are watching the /etc/passwd file for permission changes (writes and attibute changes specifically). We are creating a custom key to use for organizational purposes.

    [root@ip-172-31-21-28 ~]# auditctl -w /etc/passwd -p wa -k edit_watch

    Here is a cool one – lets audit all binary executions under /usr/bin.

    [root@ip-172-31-21-28 ~]# auditctl -w /usr/bin -p x

    Using the -l option you can list your current audit rules, and using the -s option you can see the current status of the auditd subsystem

    [root@ip-172-31-21-28 ~]# auditctl -l
    LIST_RULES: exit,always watch=/etc/passwd perm=wa key=edit_watch
    LIST_RULES: exit,always dir=/usr/bin (0x8) perm=x

    [root@ip-172-31-21-28 ~]# auditctl -s
    AUDIT_STATUS: enabled=1 flag=1 pid=741 rate_limit=0 backlog_limit=320 lost=0 backlog=0

    Want to get down and dirty with audit rules – you can check out the audit rules man page by running the command below.

    [root@ip-172-31-21-28 ~]# man audit.rules

     

     

     
    Related articles

    RHEL6 – SELinux Troubleshooting II: Electric Boogaloo
    Linux audit files to see who made changes to a file
    A Brief Introduction to auditd