Chris Paquin

AI, Virtualization, Containers, Infrastructure, Linux

Category: Redhat

  • RHEL6: Composing Custom Auditd Rules for Fun and Profit

    297247-typewriterAuditd gives you the ability to write your own custom audit rules. This functionality allows an administrator to keep a close eye on system calls, file access, and user behavior. This added functionality is especially useful in environments that are requred to adhear to compliance standards that are above and beyond normal standards. Think PCI.

    Once of the simplest rules to add is a watch rule which can be set on files and directories. In the example below we are watching the /etc/passwd file for permission changes (writes and attibute changes specifically). We are creating a custom key to use for organizational purposes.

    [root@ip-172-31-21-28 ~]# auditctl -w /etc/passwd -p wa -k edit_watch

    Here is a cool one – lets audit all binary executions under /usr/bin.

    [root@ip-172-31-21-28 ~]# auditctl -w /usr/bin -p x

    Using the -l option you can list your current audit rules, and using the -s option you can see the current status of the auditd subsystem

    [root@ip-172-31-21-28 ~]# auditctl -l
    LIST_RULES: exit,always watch=/etc/passwd perm=wa key=edit_watch
    LIST_RULES: exit,always dir=/usr/bin (0x8) perm=x

    [root@ip-172-31-21-28 ~]# auditctl -s
    AUDIT_STATUS: enabled=1 flag=1 pid=741 rate_limit=0 backlog_limit=320 lost=0 backlog=0

    Want to get down and dirty with audit rules – you can check out the audit rules man page by running the command below.

    [root@ip-172-31-21-28 ~]# man audit.rules

     

     

     
    Related articles

    RHEL6 – SELinux Troubleshooting II: Electric Boogaloo
    Linux audit files to see who made changes to a file
    A Brief Introduction to auditd

  • Mastering Auditd: A Guide to Linux Auditing

    Mastering Auditd: A Guide to Linux Auditing

    Auditd is the userland piece of the RHEL audit tool suite. When its up and running, audit messages sent by the kernel will be send to log files that you have configured. By default, only a small and limited number of messages will be picked up by Auditd; these are mostly messages related to authentication and authorization.

    Auditd has three main config files as shown below

    • /etc/sysconfig/auditd – basic configuration options
    • /etc/audit/auditd.conf – main config file
    • /etc/audit/audit.rules – auditing rules

    Auditd and Syslog:

    Its possible to send audit messages to a syslog. By setting active=yes in /etc/audisp/plugins.d/syslog.conf you can send all your audit messages to syslog. If your system is setup to log to a remote syslog server, then your audit messages will go along for the ride as well. Note that you can also send audit messages to a remote logging server via native audit protocol over TCP. I am not going to go into this option, but I want to make sure that we are aware that it exists.

     

    Looking for Audit Events in All the Wrong Places:

    Auditd includes a handy-dandy tool for searching audit logs. Ausearch. You can check out all your current audit log messages using the command below.

    [root@ip-172-31-21-28 ~]# ausearch -l

    Viewing audit logs in their raw format can be accomplished with the command below

    [root@ip-172-31-21-28 ~]# ausearch --raw

    The -a option allows you to search by audit event ids

    [root@ip-172-31-21-28 ~]# ausearch -a 282

    Auditd also includes ausearch, which allows you to get a quick summary of audit events, rather than trying to view massive audit logs. Usage and output shown below.

    root@ip-172-31-21-28 ~]# aureport
Summary Report
======================
Range of time in logs: 07/17/2014 10:21:36.438 - 07/17/2014 19:52:49.556
Selected time for report: 07/17/2014 10:21:36 - 07/17/2014 19:52:49.556
Number of changes in configuration: 4
Number of changes to accounts, groups, or roles: 24
Number of logins: 20
Number of failed logins: 4
Number of authentications: 75
Number of failed authentications: 3
Number of users: 3
Number of terminals: 18
Number of host names: 19
Number of executables: 14
Number of files: 0
Number of AVC's: 10
Number of MAC events: 20
Number of failed syscalls: 10
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 244
Number of keys: 0
Number of process IDs: 203
Number of events: 1132

    You can also use aureport and ausearch together. Simliar to the powerfull partnership between Batman and Robin, these two tools complement each other in ways that you can only imagine. Check out my sexy bits below.

    [root@ip-172-31-21-28 ~]# ausearch --start today --raw | aureport
Summary Report
======================
Range of time in logs: 07/17/2014 10:21:36.438 - 07/17/2014 20:01:01.911
Selected time for report: 07/17/2014 10:21:36 - 07/17/2014 20:01:01.911
Number of changes in configuration: 4
Number of changes to accounts, groups, or roles: 24
Number of logins: 20
Number of failed logins: 4
Number of authentications: 75
Number of failed authentications: 3
Number of users: 3
Number of terminals: 18
Number of host names: 19
Number of executables: 14
Number of files: 0
Number of AVC's: 10
Number of MAC events: 20
Number of failed syscalls: 10
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 244
Number of keys: 0
Number of process IDs: 205
Number of events: 1144

    Want to know another cool tool that is part of auditd? I know, its a lot to take in at one time, but I am sure that you can handle it. Using autrace you can trace and investigate system calls made by a process. 

    Want to see everything that nslookup is doing? Then run the command below.

    [root@ip-172-31-21-28 ~]# autrace /usr/bin/nslookup google.com

    This will output a pid for you to trace with ausearch.

    Trace complete. You can locate the records with 'ausearch -i -p 3359'

     
    Related articles

    RHEL6- Getting Up Close and Personal With Rsyslog
    Linux audit files to see who made changes to a file

  • RHEL6: Using Advanced Log File Filtering in Rsyslog

    100-8591So by default when you forward logs to a syslog/rsyslog server all the logs end up in the same file (ususally configured to go to the messages file). Sometimes one may prefer to forward logs from a particular server to a separate logfile. I know for a fact that my sometimes friends in our info-sec group prefers this setup.

    While managing such a setup for more than a handful of hosts would probably be a nightmare, its not actaually hard to setup on the most basic level.

    First create a custom filters.conf (or whatever you want to name it) in your /etc/rsyslog.d directory. Below is the file that I created.

    [root@ip-172-31-21-28 rsyslog.d]# cat filters.conf
    :fromhost, isequal, "ip-172-31-25-104.ec2.internal" /var/log/ip-172-31-25-104.ec2.internal/messages
    :fromhost, isequal, "ip-172-31-25-104.ec2.internal" ~

    Note that you will need to include two lines in your file for each host, one with the specific file/location that you want to send your filtered log messages, and a second line that directs rsyslog to discard any messages from the specified hosts once they have been logged to the specified locations. This keeps these messages from ending up in the messages file as well as the file defined in your filter file.

     
    Related articles

    RHEL6 – Introduction to IPtables, Part II
    HomeLab: Basic Syslog Configuration on Cisco Catalyst Devices
    RHEL6- Getting Up Close and Personal With Rsyslog

  • RHEL6: Configuring Encrypted Remote Logging Via Rsyslog

    Picresized_1350512420_mummyRsyslog has the ability to forward/recieve encrypted logs using certificates. I am going to go over a very quick and dirty install and configuration. Since I am again sitting in Starbucks and do not have access to my homelab and its abundant resources, I am once again going to use Amazon EC2 instances of RHEL 6.5 for testing. The two instances that I will use we will call Server1 and Server2. Our goal here is not to build out an entire environment, but rather we are just looking to get this working on its most basic level.

    Lets start with the log reciever server as we need to install a couple of packages. The first provides the certtool utility and the second provides the tls library for rsyslog.

    [root@ip-172-31-21-28 ~]# yum -y install gnutls-utils.x86_64 rsyslog-gnutls

    Also note that we are going to be looking at some documentation on the remote hosts to make this install and config a bit easier. You are probably going to want to install lynx so you can view the html documentation files. On my server1 instance, the rsyslog docs are in the following directory – /usr/share/doc/rsyslog-5.8.10.

    This is the file that you want to take a look at. Below I am using lynx to view it.

    lynx tls_cert_server.html

    Now lets head on over to the sending server and install our packages again.

    [root@ip-172-31-21-28 ~]# yum -y install gnutls-utils.x86_64 rsyslog-gnutls

    Grab these bits from the file and modify to match your system. Then drop it in a file in /etc/rsyslog.d on your rsyslog receiving server.  I am calling my logging.conf

    $ModLoad imuxsock # local messages
    $ModLoad imtcp # TCP listener

    # make gtls driver the default
    $DefaultNetstreamDriver gtls

    # certificate files
    $DefaultNetstreamDriverCAFile /rsyslog/protected/ca.pem
    $DefaultNetstreamDriverCertFile /rsyslog/protected/machine-cert.pem
    $DefaultNetstreamDriverKeyFile /rsyslog/protected/machine-key.pem

    $InputTCPServerStreamDriverAuthMode x509/name
    $InputTCPServerStreamDriverPermittedPeer *.example.net
    $InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode
    $InputTCPServerRun 6514 # start up listener at port 10514

    Now, from the same file as you used above, grab the following giggly-bits and stick them out on the rsyslog sending server. Modify the file to match your needs and drop in /etc/rsyslog.d. To keep things simple I am going to call this file logging.conf as well

    # certificate files – just CA for a client
    $DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem
    # set up the action
    $DefaultNetstreamDriver gtls # use gtls netstream driver
    $ActionSendStreamDriverMode 1 # require TLS for the connection
    $ActionSendStreamDriverAuthMode anon # server is NOT authenticated
    *.* @@(o)server.example.net:10514 # send (all) messages

     

    Now, lets review where we are. Our config files are in place on our sending server and our recieving server. Plus we have installed all required packages. Now lets get on to the keys. This is the confusing part, so I will go through my process one step at a time. Once again you can use the built in documentation to help you get this part right.

    Jump on the recieving server and check out the file below via lynx

    lynx tls_cert_ca.html

    Jump into your newly created keys directory.

    [root@ip-172-31-21-28 ~]# cd /etc/rsyslog-keys/

    And run the command below to generate the private key for your CA (Certficate Authority)


    [root@ip-172-31-21-28 rsyslog-keys]# certtool –generate-privkey –outfile ca-key.pem                                        
    Generating a 2048 bit RSA private key…

    Now we are going to generate a self signed cert from our private key.

    [root@ip-172-31-21-28 rsyslog-keys]# certtool –generate-self-signed –load-privkey ca-key.pem –outfile ca.pem

    You will be asked a bunch of questions but there are only two important ones that are shown below, make sure you answer yes to both of them.

    Does the certificate belong to an authority? (y/N): y
    Will the certificate be used to sign other certificates? (y/N): y

    Now take this key (ca.pem) and copy it onto the sending server in /etc/rsyslog-keys, or wherever you set your config files to look for them.

    Now lets go back to the recieving server so we can genreate a private key.

    [root@ip-172-31-21-28 rsyslog-keys]# certtool –generate-privkey –outfile key.pem –bits 2048
    Generating a 2048 bit RSA private key…

    Now lets generate a request key.

    [root@ip-172-31-21-28 rsyslog-keys]# certtool –generate-request –load-privkey key.pem –outfile request.pem

    And how lets sign the certificate request and generate a certificate.

    certtool –generate-certificate –load-request request.pem –outfile cert.pem –load-ca-certificate ca.pem –load-ca-privkey ca-key.pem

    This one also requires you to answer a bunch of questions. These are the two that have "yes" for the answer.

    Is this a TLS web client certificate? (y/N): y
    Is this also a TLS web server certificate? (y/N): y

    Now at this point you should be able to start rsyslog on each server. Using the logger command on your sending server, you should be able to send a message to your local syslog and see it forward to the sender.

     

     
    Related articles

    RHEL6 – Configuring Apache with TLS/SSL Encryption
    RHEL6- Getting Up Close and Personal With Rsyslog
    Install and Configure Rsyslog Server in CentOS 6.4 / RHEL 6.4

  • Disabling Ctrl+Alt+Delete in RHEL 6

    Ctrl-Alt-Del-Cup-Set

    Let me start off by saying that I am not a fan of disabling ctrl+alt+delete, especially if you do not have physical access to a server.

    Sometimes the old three finger salute is the best and quickest method to reboot a locked and unresponsive Operating System.

    Regardless of this fact, some Info Sec folks think that disabling this functionality is a good idea for security.

    So for their sake, I will show you how to do it.

    First ssh into your server and become root and change directory to what is shown below.

    [root@ip-172-31-22-45 ~]# cd /etc/init

    Now copy the file below exactly as I have illustrated. You do not want to try to modify the original file as it could be overwritten by Upstart.

    [root@ip-172-31-22-45 init]# cp control-alt-delete.conf control-alt-delete.override

    Now modify the line below and change it to run a command that will exit with an exit code of 0.

    exec /sbin/shutdown -r now "Control-Alt-Delete pressed"

    Something like /bin/true will work, although I am sure that you can come up with more creative ideas.

    exec /bin/true

    Now try ctrl+atl+delete, nothing should happen.

  • Password Protecting Grub in RHEL 6

    19552780-illustration-of-cartoon-wormGrub, is the standard boot loader used by each and every Linux type operating system that I can think of. RHEL 6 uses what I guess we are now calling grub 1.o, since grub 2.0 has been released and in use by Fedora for the last few releases. You will also find that grub 2.0 has replaced grub 1.0 in RHEL 7. At some point I plan to explore grub 2 at lenght, but today is not that day (unless something strange happens before I go to bed tonight — you never know).

     

    Anyway – I digress. Below is an excerpt from the current grub.conf that is in use by my EC2 RHEL 6.5 instance. I've made it really tiny to save space.

    # grub.conf generated by anaconda
    #
    # Note that you do not have to rerun grub after making changes to this file
    # NOTICE:  You do not have a /boot partition.  This means that
    #          all kernel and initrd paths are relative to /, eg.
    #          root (hd0)
    #          kernel /boot/vmlinuz-version console=ttyS0 ro root=/dev/vda1
    #          initrd /boot/initrd-[generic-]version.img
    #boot=/dev/vda
    default=0
    timeout=1
    splashimage=(hd0)/boot/grub/splash.xpm.gz
    hiddenmenu
    title Red Hat Enterprise Linux Server (2.6.32-431.17.1.el6.x86_64)
            root (hd0)
            kernel /boot/vmlinuz-2.6.32-431.17.1.el6.x86_64 console=ttyS0 ro root=UUID=05e7b919-2577-40a7-91fb-1ccdede87fc4 rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 xen_blkfront.sda_is_xvda=1 console=ttyS0,115200n8 console=tty0 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_LVM rd_NO_DM rhgb quiet
            initrd /boot/initramfs-2.6.32-431.17.1.el6.x86_64.img

    To configure a password to use for grub you need to use the grub-crypt command. as you can see in the example below, grub-crypt prompts you for your password, although you can also pass the password to grub-crypt in the inital command. Grub-crypt spits out your hashed and salted password.

    [root@ip-172-31-22-45 ~]# grub-crypt
    Password:
    Retype password:
    $6$V0qkcD4b/CouCY/L$.7vrHd.X6MZWzzDxvXdQn3avPwv2rUX80XKuS.Hvh1YqR11ZyINZszvJ/e0qH6VcsFMAVWAPWXtugHHYTzYj90

    Now stick this password in your grub.conf using the format below.

    password –encrypted $6$V0qkcD4b/CouCY/L$.7vrHd.X6MZWzzDxvXdQn3avPwv2rUX80XKuS.Hvh1YqR11ZyINZszvJ/e0qH6VcsFMAVWAPWXtugHHYTzYj90

    You have a couple of options when inserting the password into the grub.conf. You can either stick this entry near the top of the file, which protects each and every boot image that you might be configured to use, or you can stick the entry directly into a particular stanza to protect a specific boot image. For example, if you are building your servers with the option to pxe-boot from grub, or wipe the disk from grub, you might want to protect these options with a password.

    Also note that adding a password to grub does not keep an unauthorized user from being able to boot or reboot a server, rather what it does is protect the grub menu from being edited manually durring a reboot. This keeps nefarious users from being able to break the boot sequence and boot into single-user mode where they can compromise your system.

     
    Related articles

    RHEL6 – Restore Grub on MBR
    Grub
    RHEL6 – Mount: Could Not Find Any Free Loop Device

  • Fun With PAM: Working with pam_cracklib and pam_tally2

    Liberty-bell_13850_smPlugable Authentication Modules, or PAM, is the standard mechanism that most Unix and Linux Operatng Systems use for user credential authentication. By design, PAM is broken out into a number of files, each with a specific purpose. Before you can get started with PAM you need to understand a bit about how PAM configuration files are formatted.  So lets get into that first before we try to bite off anything more.

     

    PAM Config File Standards:

    PAM config files follow a standard format as shown below.

    Rule Types        Control        Module [module arguments]

     

    There are 4 Rule Types

    • auth – used to authenticate users/passwords
    • account – set properties for a user's account
    • password – controls password changes
    • session – sets and controls environmental variables

    And there are 5 Control Types

    • required – a module that a user is required to pass
    • sufficient – a user is not required to pass a sufficient module
    • optional – these modules do not have to be passed sucessfully
    • include – these rules reference other PAM config files
    • requisite – similar to required, but if failed, no further rules are checked

     

    Configuring pam_cracklib:

    Pam_cracklib is used to define password complexity. It has several module arguments that can be used to define password complexity and lenght. Its most common arguments are show below

    • ucredit – when used in the following format (ucredit=-n) requires the defined number of uppercase characters in a password
    • dcredit – when used in the following format (dcredit=-n) requires the defined number of digits in a password
    • ocredit – when used in the following format (ocredit=-n) requires the defined number of other (think symbols) type charaters in a password
    • lcredit – when used in the following format (lcredit=-n) requires the defined number of lower case letters  in a password
    • minclass – defines the minimum number of different character classes that must be present in a password.
    • minlen – defines the minimum required lenght of a password.

    Here's a usage example of the cracklib module from a /etc/pam.d/system-auth file. In this example try_first_pass tells pam to try to use any cached credentials, while retry allows a user to try their password 3 times before the fail this module.

    password    requisite     pam_cracklib.so minlen=8 ocredit=-1 ucredit=-1 try_first_pass retry=3

     

    Configuring pam_tally2:

    Pam_tally2 can be used to lock users after a defined number of failed login attempts. The example below, taken from the system-auth file will lock a user after 3 failed login attempts, will automatically unlock the user after 300 seconds, and will do so quietly, without any notification to the user.

    auth        required        pam_tally2.so deny=3 unlock_time=300 quiet

    The command pam_tally2 can be used to list users with failed logins and can also be used to reset a user's failed login count. See reset example below

    pam_tally2 –reset –u testuser

    Note that pam_tally2 deprecates the faillock module.

     

    Supplementary PAM Configuration Options:

    Want to limit a user to a particular number of concurrent ssh sessions? You can set this up in /etc/security/limits.conf if you are calling the pam_limits.so in your pam configs. Limits.conf provides the example below. Just copy the format and you are off to the races. Remember to remove the #.

    #@student        -       maxlogins       4

     

     
    Related articles

    How to enforce password complexity on Linux
    Secure Linux Servers
    You are not allowed to access to (crontab) because of pam configuration.