How To Check if RHEL/AlmaLinux needs a reboot after an update

Typically you’ll need to reboot a server after an update if the Linux Kernel was updated. It is possible that services need to be restarted.

There is some good information here https://serverfault.com/questions/122178/how-can-i-check-from-the-command-line-if-a-reboot-is-required-on-rhel-or-centos

Using Yum Utilities needs-restarting

Install the needs-restarting utility

sudo dnf install -y yum-utils

Once installed, we can check if we need to reboot with

sudo needs-restarting -r

The -r option only reports if a reboot is required.

If we wanted to automatically check and reboot, we could do

sudo needs-restarting -r || sudo shutdown -r

Alternative way

We could alternatively just check the kernel version and if it is different, manually reboot the machine. Note that there could be a couple cases where the kernel didn’t update, but you still need a reboot, or services needed to be restarted View links below for more information.

LAST_KERNEL=$(rpm -q --last kernel | perl -pe 's/^kernel-(\S+).*/$1/' | head -1)
CURRENT_KERNEL=$(uname -r)

test $LAST_KERNEL = $CURRENT_KERNEL || shutdown -r

View Fiber SFP details in Mikoritk RouterOS

Quick and simple way to check the details on a fiber SFP on a Mikrotik router. Replace sfp1_name with the SFP name or leave out the name and select a number.

/interface ethernet monitor "sfp1_name"

Results

                      name: sfp1
                    status: link-ok
          auto-negotiation: done
               full-duplex: yes
           tx-flow-control: no
           rx-flow-control: no
               advertising: 
  link-partner-advertising: 
        sfp-module-present: yes
               sfp-rx-loss: no
                  sfp-type: SFP-or-SFP+
        sfp-connector-type: LC
       sfp-link-length-9um: 3000m
           sfp-vendor-name: UBNT
    sfp-vendor-part-number: UF-SM-1G-S
         sfp-vendor-serial: FL31F80285729
    sfp-manufacturing-date: 20-02-20
            sfp-wavelength: 1550.32nm
           sfp-temperature: 64C
        sfp-supply-voltage: 3.23V
       sfp-tx-bias-current: 30mA
              sfp-tx-power: -5.254dBm
              sfp-rx-power: -4.1dBm
           eeprom-checksum: good
                    eeprom: 0000: 01 02 03 00 00 00 00 00  00 00 00 .....

How to determine if Ubuntu Needs a Reboot after an update

Typically after a Linux Kernel update, you will want to reboot your machine to take advantage of the new kernel. But how do you know if you need to reboot?

Fortunately, there is a simple way to check.

cat /var/run/reboot-required

If it returns

*** System restart required ***

Then we should reboot the machine.

https://www.cyberciti.biz/faq/how-to-find-out-if-my-ubuntudebian-linux-server-needs-a-reboot/

Using Auditd to monitor changes to Linux

Install and enable auditd with

sudo dnf install auditd
sudo systemctl enable auditd
sudo systemctl start auditd

Add a file or directory to monitor with

auditctl -w /etc/passwd -k password

-w is watch path
-k is a filter key we can use later to search through logs

Now we can search with ausearch

ausearch -k password

Using Preconfigured Rules

There are already some preconfigured rules in /usr/share/audit/sample-rules/

We can copy those to /etc/auditd/rules.d/ and use them.

cd /usr/share/audit/sample-rules/
cp 10-base-config.rules 30-stig.rules 31-privileged.rules 99-finalize.rules /etc/audit/rules.d/
augenrules --load

Note on the 31-privileged.rules file. You’ll need to run the commands in the file which will create a new file. Then we can copy that to “/etc/auditd/rules.d/”

find /bin -type f -perm -04000 2>/dev/null | awk '{ printf "-a always,exit -F path=%s -F perm=x -F auid>=1000 -F auid!=unset -F key=privileged\n", $1 }' > priv.rules
#find /sbin -type f -perm -04000 2>/dev/null | awk '{ printf "-a always,exit -F path=%s -F perm=x -F auid>=1000 -F auid!=unset -F key=privileged\n", $1 }' >> priv.rules
#find /usr/bin -type f -perm -04000 2>/dev/null | awk '{ printf "-a always,exit -F path=%s -F perm=x -F auid>=1000 -F auid!=unset -F key=privileged\n", $1 }' >> priv.rules
#find /usr/sbin -type f -perm -04000 2>/dev/null | awk '{ printf "-a always,exit -F path=%s -F perm=x -F auid>=1000 -F auid!=unset -F key=privileged\n", $1 }' >> priv.rules
#filecap /bin 2>/dev/null | sed '1d' | awk '{ printf "-a always,exit -F path=%s -F perm=x -F auid>=1000 -F auid!=unset -F key=privileged\n", $2 }' >> priv.rules
#filecap /sbin 2>/dev/null | sed '1d' | awk '{ printf "-a always,exit -F path=%s -F perm=x -F auid>=1000 -F auid!=unset -F key=privileged\n", $2 }' >> priv.rules
#filecap /usr/bin 2>/dev/null | sed '1d' | awk '{ printf "-a always,exit -F path=%s -F perm=x -F auid>=1000 -F auid!=unset -F key=privileged\n", $2 }' >> priv.rules
#filecap /usr/sbin 2>/dev/null | sed '1d' | awk '{ printf "-a always,exit -F path=%s -F perm=x -F auid>=1000 -F auid!=unset -F key=privileged\n", $2 }' >> priv.rules

And Copy priv.rules to /etc/audit/rules.d/31-privileged.rules. Overwrite the file there if needed.

cp ./priv.rules /etc/audit/rules.d/31-privileged.rules

Load the rules.

augenrules --load

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/auditing-the-system_security-hardening

DNS Query “sl 0.0.0.0” in Mikrotik DNS Cache

Sounds like this could be from a potential scan. The record is useless as 0.0.0.0 doesn’t go to anything.

https://learn.microsoft.com/en-us/answers/questions/547092/what-is-the-sl(0)-in-dns-logs-as-host-name

We can block this type of behavior by blocking inbound DNS request. Change in-interface to your interface or change to an interface list.

ip firewall filter add chain=input protocol=6 dst-port=53 in-interface=ether1 action=drop
ip firewall filter add chain=input protocol=17 dst-port=53 in-interface=ether1 action=drop

How to Create a Self Signed TLS Certificate in Linux

Here is a quick way to create a self signed certificate in Linux.

Run the following command. Fill out the required info.

openssl req -x509 -sha256 -nodes -days 3652 -newkey rsa:4096 -keyout /etc/pki/tls/private/localhost.key -out /etc/pki/tls/certs/localhost.crt
chmod 400 /etc/pki/tls/private/localhost.key

Now in your Apache or Nginx files, specify the path to the Key and the Certificate.

Note that if you’ll need to add the

https://www.linode.com/docs/guides/create-a-self-signed-tls-certificate/

Backup UISP Application Backup Files with Rsync

UISP runs inside of a docker container. To copy out the backup files we need to use the “docker cp” command.

sudo docker cp unms:/home/app/unms/data/unms-backups ./uisp-backups

This will copy the backups into ./uisp-backups directory.

On an Ubuntu system, docker needs sudo permissions. If you copy the backups with the above command, the backup files will be assigned to the root user and you will not be able to use your normal user to manipulate the files.

You can either add your current user to the Docker group, or change the files owner

sudo chown username:username -R ./uisp-backups/

We can now copy all the automatic backups with rsync

sudo rsync -a ./uisp-backups -e "ssh -p 22" backupuser@backuphost:/backups

You can also automate this with Cron by doing something like

1 1 * * 1 docker cp unms:/home/app/unms/data/unms-backups ~/uisp-backups && rsync -a ~/uisp-backups -e "ssh -p 22" backupuser@backuphost:/backups

Every Monday at 1:01AM, copy the current UISP automatic backups, then use rsync to copy them to a remote server.

This expects that the current user has permissions to call Docker without sudo.

Give Ubuntu User Access to Run Docker?

By default on Debian based systems, Docker needs the sudo command to run. We can add a normal user to the Docker group so we don’t have to.

sudo usermod -aG docker username

Change out the username to your Ubuntu username.

The -a option means append the group to the username. It does not remove the user from current groups.
the -G option means add the specified group.

LibreNMS Devices Disappeared from Web Interface

What do you do when all your devices disappear from the web interface?

Everything still appears to be working. Alerts work.

Running a MySQL command to check if the devices are in the database returns all the devices

mysql -u librenms -p librenms -e 'use librenms ; select hostname,sysName,status from devices where status=1'

If we try going to /addhost we are greeted with an Error You have insuffecient permissions to view this page.

Running ./validate.php returns everything good

Potentially it could be an issue with SElinux or with Apache/NGinx

Running

audit2why < /var/log/audit/audit.log

Doesn’t return anything

No errors pop up in the Logs

Could be something happened with the LibreNMS user.

Test a different LibreNMS user and all the devices show up.

We’ve now isolated the issue to being something with out user.