Ansible Playbook to upgrade Linux Servers (Debian, Ubuntu, RedHat, Fedora, CentOS)

This is an Ansible playbook that can upgrade all your Linux machines! Or at least most of them. No openSUSE support yet.

Copy the playbook below, and put all your servers into an inventory file and run with

ansible-playbook -i hosts.ini master_update.yaml --ask-vault-pass

Couple of notes.

  1. This will do a full update automatically reboot your servers if needed.
  2. There is a special section for RHEL, CentOS 7 servers. If a server is running say CentOS 7, it will default to using YUM instead of DNF.
  3. You need sudo or become: yes to reboot and install upgrades.

Linux OS Upgrade Playbook

---
- name: Linux OS Upgrade
  hosts: all
  gather_facts: yes
  become: yes

  tasks:
    - name: Upgrade Debian and Ubuntu systems with apt
      block: 
        - name: dist-upgrade
          ansible.builtin.apt:
            upgrade: dist
            update_cache: yes 
          register: upgrade_result

        - name: Debain check if reboot is required
          shell: "[ -f /var/run/reboot-required ]"
          failed_when: False
          register: debian_reboot_required
          changed_when: debian_reboot_required.rc == 0
          notify:
            - Reboot server 

        - name: Debian remove unneeded dependencies
          ansible.builtin.apt:
            autoremove: yes
          register: autoremove_result 

        - name: Debian print errors if upgrade failed
          ansible.builtin.debug:
            msg: | 
              Upgrade Result: {{ upgrade_result }}
              Autoremove Result: {{ autoremove_result }}
      when: ansible_os_family == "Debian"
    
    - name: Upgrade RHEL systems with DNF
      block:
        - name: Get packages that can be upgraded with DNF
          ansible.builtin.dnf:
            list: upgrades
            state: latest
            update_cache: yes 
          register: reg_dnf_output_all

        - name: List packages that can be upgraded with DNF
          ansible.builtin.debug: 
            msg: "{{ reg_dnf_output_all.results | map(attribute='name') | list }}"

        - name: Upgrade packages with DNF
          become: yes
          ansible.builtin.dnf:
            name: '*'
            state: latest
            update_cache: yes
            update_only: no
          register: reg_upgrade_ok

        - name: Print DNF errors if upgrade failed
          ansible.builtin.debug:
            msg: "Packages upgrade failed"
          when: reg_upgrade_ok is not defined

        - name: Install dnf-utils
          become: yes
          ansible.builtin.dnf:
            name: 'dnf-utils'
            state: latest
            update_cache: yes
          when: reg_dnf_output_all is defined

      when: ansible_os_family == "RedHat" and not (ansible_distribution_major_version == "7")

    - name: Upgrade legacy RHEL systems with YUM
      block:
        - name: Get packages that can be upgraded with YUM
          ansible.builtin.yum:
            list: upgrades
            state: latest
            update_cache: yes 
          register: reg_yum_output_all
            

        - name: List packages that can be upgraded with YUM
          ansible.builtin.debug: 
            msg: "{{ reg_yum_output_all.results | map(attribute='name') | list }}"

        - name: Upgrade packages with YUM
          become: yes
          ansible.builtin.yum:
            name: '*'
            state: latest
            update_cache: yes
            update_only: no
          register: reg_yum_upgrade_ok

        - name: Print YUM errors if upgrade failed
          ansible.builtin.debug:
            msg: "Packages upgrade failed"
          when: reg_yum_upgrade_ok is not defined
            
        - name: Check legacy RHEL system if a reboot is required
          become: yes
          command: needs-restarting -r
          register: reg_reboot_required
          ignore_errors: yes
          failed_when: false
          changed_when: reg_reboot_required.rc != 0
          notify:
            - Reboot server 
      when: ansible_os_family == "RedHat" and ansible_distribution_major_version == "7"


  handlers:
    - name : Reboot server
      ansible.builtin.reboot:
        msg: "Reboot initiated by Ansible after OS update"
        reboot_timeout: 3600
        test_command: uptime

Helpful links

https://github.com/simeononsecurity/ansible_linux_update/tree/main
https://simeononsecurity.com/guides/automate-linux-patching-and-updates-with-ansible/
https://thenathan.net/2020/07/16/yum-and-dnf-update-and-reboot-with-ansible/

Ansible Playbook for Updating Linux (Debian/Ubuntu)

Video on using Ansible to Update Linux

The three steps to update a machine with Ansible

  1. Create Ansible Inventory/Hosts file
  2. Create Playbook
  3. Run Playbook

Create Inventory

The first thing we need to do is create an inventory file. This will contain a list of our servers along with the credentials.

touch hosts.txt

Now let’s encrypt the file with Ansible Vault.

ansible-vault encrypt hosts.txt

The file is now encrypted. To edit the file, we need to use `ansible-vault edit`.
If you want to, you can configure the hosts.txt file and then encrypt it when you are finished.

ansible-vault edit hosts.txt

Now add some hosts. In this example we add the local Kali machine, because why not. If you have Ubuntu servers, replace debian with ubuntu.

[debian]
kali ansible_host=127.0.0.1 ansible_ssh_user=kali ansible_ssh_port=22 ansible_ssh_password='kali pass' ansible_become_pass='kali sudo pass'

Add as many hosts as you need. For sake of simplicity, we are only adding one, and it is our localhost.

Create Playbook

Create a new playbook.

vi debian_update.yml

Put the following into the playbook. Edit as desired. Change hosts to match the above hosts in the inventory/hosts file.

---
- name: OS update
  hosts: debian
  gather_facts: yes
  become: yes

  tasks:
    - name: dist-upgrade
      ansible.builtin.apt:
        upgrade: dist
        update_cache: yes
      register: upgrade_result

    - name: Check if a reboot is required
      ansible.builtin.stat:
        path: /var/run/reboot-required
        get_checksum: no
      register: reboot_required_file

    - name: Reboot the server (if required).
      ansible.builtin.reboot:
      when: reboot_required_file.stat.exists
      register: reboot_result

    - name: Remove unneeded dependencies
      ansible.builtin.apt:
        autoremove: yes
      register: autoremove_result

    - name: Print errors if upgrade failed
      ansible.builtin.debug:
        msg: |
          Upgrade Result: {{ upgrade_result }}
          Reboot Result: {{ reboot_result }}
          Autoremove Result: {{ autoremove_result }}

A couple of notes

  1. On the 3rd line it defines which group to run this playbook against. In this case debian.
  2. This will check if a reboot is needed and reboot the machine. Reboots are usually needed when the kernel is updated
  3. The 5th line contains `become: yes` this means that the playbook will use sudo. You can specify the sudo password in the hosts file `ansible_become_pass=sudopass` or with the -k or –ask-become options
  4. The update and reboot are natively built into Ansible. Hence the ansible.builtin.

Run Playbook

Now that we have our inventory and playbook, we can upgrade our machines.

ansible-playbook debian_update.yml -i hosts.ini --ask-vault-password

Tip! If you have not specified a “ansible_ask_become” password (that is the sudo password), you can specify it with the -k or –ask-become options.

Mongo “illegal hardware instruction mongo” on Linux

While trying to install and run mongo on Kali Linux, I encountered the following error.

zsh: illegal hardware instruction mongo

Using a Bash shell it returns the following instead.

Illegal instruction

It appears that the issue is from running mongo in a virtual machine.

https://www.mongodb.com/community/forums/t/mongodb-community-5-0-12-illegal-instruction-core-dumped-ubuntu-20-04-5-lts/204332

https://askubuntu.com/questions/699077/how-to-enable-avx2-extensions-on-a-ubuntu-guest-in-virtualbox-5

Resolution? Run on bare metal or find a way to enable avx2 in the virtual machine.

Hardening SNMP on Debian

Hardening SNMP on Debian by disabling SNMP v1 and v2c, and configuring SNMP v3.

Modify /etc/snmp/snmpd.conf

First we’ll want to open up the /etc/snmp/snmpd.conf file and comment out all lines that begin with

  • rocommunity
  • view
  • rouser authPriv <– “This may be the last line by default, we don’t need it”

Alternatively, you can copy and paste the following sed commands instead of manually editing the file.

sudo sed -i 's/^rocommunity/# rocommunityc/g' /etc/snmp/snmpd.conf
sudo sed -i 's/^view/# view/g' /etc/snmp/snmpd.conf
sudo sed -i 's/^rouser authPriv/# rouser authPriv/g' /etc/snmp/snmpd.conf

Create SNMP v3 User

We can create a SNMP v3 user with the following command. There it will ask you for the username and passwords.

sudo net-snmp-create-v3-user -ro -a SHA-512 -x AES

You may receive an error about not being able to touch /snmp/snmpd.conf. I am not sure why Debian is attempting to create that file. Take the “rouser snmpuser” line and add it to the end of the /etc/snmp/snmpd.conf config.

Debian SNMP Error

Now we can start SNMPD

sudo systemctl start snmpd

Troubleshooting

My created user is not working! This could result from two different issues.

  1. It appears that Debian/SNMP doesn’t like pass phrases with special characters. You can try using a different password or escaping the special characters in “/var/lib/snmp/snmpd.conf” file before starting SNMPD.
  2. The user didn’t get added to /etc/snmp/snmpd.conf To fix, add “rouser snmpuser” (Change snmpuser to your snmp username) to the bottom of the config file.

Install RX 580 Mining Drivers on Debian Based Distributions

Use wget to download AMD drivers.

wget https://drivers.amd.com/drivers/linux/amdgpu-pro-20.45-1164792-ubuntu-20.04.tar.xz --referer https://support.amd.com

Extract archive.

tar xf amdgpu-pro-20.45-1164792-ubuntu-20.04.tar.xz

Change directory

cd amdgpu-pro-20.45-1164792-ubuntu-20.04

Install AMD Drivers

./amdgpu-pro-install -y --opencl=legacy,rocm --headless

If you run into issues with it saying “Unsupported DEB based OS” Refer to the following article.

Unsupported DEB-based OS: /etc/os-release ID ‘kali’

Delete SNMPv3 User on Linux

Don’t know if this is the recommended way to delete a user, but it seems to work.

sudo service snmpd stop

Open up the snmpd.conf file in /var/lib and find the line with the SNMP user and delete the line

sudo vi /var/lib/snmp/snmpd.conf

The above file may be in the following location on RPM based systems.

sudo vi /var/lib/net-snmp/snmpd.conf

Save, exit, and start snmpd

sudo service snmpd start

These steps work for Ubuntu, but should work for any Debain based distro as well as CentOS, Fedora, RedHat etc.

Install dig on Ubuntu, Debian or Kali Linux

install dig
Help options for dig


Dig is a DNS lookup utility.  It is included in most Linux distributions by default, but if it isn’t you can easily install dig with the following command.

The dig utility is apart of the dnsutils package

sudo apt-get install dnsutils -y

After it is installed, we can verify that it is working with

dig -v

For more information on how to use dig, refer to the following link.

https://www.howtogeek.com/663056/how-to-use-the-dig-command-on-linux/

The following is copied and pasted from the dig man page.

NAME
       dig - DNS lookup utility

SYNOPSIS
       dig [@server] [-b address] [-c class] [-f filename] [-k filename] [-m] [-p port#] [-q name]
           [-t type] [-v] [-x addr] [-y [hmac:]name:key] [[-4] | [-6]] [name] [type] [class]
           [queryopt...]

       dig [-h]

       dig [global-queryopt...] [query...]

DESCRIPTION
       dig is a flexible tool for interrogating DNS name servers. It performs DNS lookups and
       displays the answers that are returned from the name server(s) that were queried. Most DNS
       administrators use dig to troubleshoot DNS problems because of its flexibility, ease of use
       and clarity of output. Other lookup tools tend to have less functionality than dig.

       Although dig is normally used with command-line arguments, it also has a batch mode of
       operation for reading lookup requests from a file. A brief summary of its command-line
       arguments and options is printed when the -h option is given. Unlike earlier versions, the
       BIND 9 implementation of dig allows multiple lookups to be issued from the command line.

       Unless it is told to query a specific name server, dig will try each of the servers listed
       in /etc/resolv.conf. If no usable server addresses are found, dig will send the query to the
       local host.

       When no command line arguments or options are given, dig will perform an NS query for "."
       (the root).

       It is possible to set per-user defaults for dig via ${HOME}/.digrc. This file is read and
       any options in it are applied before the command line arguments. The -r option disables this
       feature, for scripts that need predictable behaviour.

       The IN and CH class names overlap with the IN and CH top level domain names. Either use the
       -t and -c options to specify the type and class, use the -q the specify the domain name, or
       use "IN." and "CH." when looking up these top level domains.

SIMPLE USAGE
       A typical invocation of dig looks like:

            dig @server name type

       where:

       server
           is the name or IP address of the name server to query. This can be an IPv4 address in
           dotted-decimal notation or an IPv6 address in colon-delimited notation. When the
           supplied server argument is a hostname, dig resolves that name before querying that name
           server.

           If no server argument is provided, dig consults /etc/resolv.conf; if an address is found
           there, it queries the name server at that address. If either of the -4 or -6 options are
           in use, then only addresses for the corresponding transport will be tried. If no usable
           addresses are found, dig will send the query to the local host. The reply from the name
           server that responds is displayed.

       name
           is the name of the resource record that is to be looked up.

       type
           indicates what type of query is required — ANY, A, MX, SIG, etc.  type can be any valid
           query type. If no type argument is supplied, dig will perform a lookup for an A record.