Hacking Hashed SSH known_hosts file

On Ubuntu, by default, the hosts in .ssh/known_hosts, are hashed. This can theoretically help with security. If an attacker compromises a hosts, they will not be able to tell the IP addresses of other hosts in the known_hosts file.

https://security.stackexchange.com/questions/56268/ssh-benefits-of-using-hashed-known-hosts

Anatomy of the hashed known_hosts

Here is an example of a hashed entry in the known_hosts file.

|1|ma8KL2XrNYkNnknf68N4IuZ+c+I=|PmR+n2i0/epUGZZh2S+LB6OaowQ= ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEjqG8/el8c669FxcvEw5mMfDRTDxsjgLiz44dCTtchs

There are three main parts.

The first part ma8KL2XrNYkNnknf68N4IuZ+c+I= is the salt to use.

PmR+n2i0/epUGZZh2S+LB6OaowQ= This is our hashed IP address/hostname

ssh-ed25519 is the key type

AAAAC3NzaC1lZDI1NTE5AAAAIEjqG8/el8c669FxcvEw5mMfDRTDxsjgLiz44dCTtchs Is the public SSH key of the remote host.

SSH-KEYSCAN

We can use ssh-keyscan to check the keys of hosts. The -t ssh-ed25519 option only shows ed25519 keys. Remove or change to show all key types e.g. RSA/DSA

For example:

┌──(kali㉿localhost)-[~]
└─$ ssh-keyscan -t ssh-ed25519 localhost
# localhost:22 SSH-2.0-OpenSSH_9.6p1 Debian-3
localhost ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEjqG8/el8c669FxcvEw5mMfDRTDxsjgLiz44dCTtchs

We can compare the SSH public key with the one in our known_hosts file to verify we have the correct host.

As a side note, we can also use the -H option to show us a hashed version. The salt changes each time it is run, so it is not useful for comparing the hashed IP address.

Example:

┌──(kali㉿localhost)-[~]
└─$ ssh-keyscan -H -t ssh-ed25519 localhost
# localhost:22 SSH-2.0-OpenSSH_9.6p1 Debian-3
|1|j2j9iv/GkPfnG9Yv4WzJsy/L1pc=|wethKgsGBH0Mi+rFW3zSNSWiGso= ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEjqG8/el8c669FxcvEw5mMfDRTDxsjgLiz44dCTtchs

Hacking known_hosts hashes

There are a few different techniques that can be used to identify known hosts IP addresses even if they are hashed.

https://stackoverflow.com/questions/427979/how-do-you-extract-ip-addresses-from-files-using-a-regex-in-a-linux-shell

Search through bash history

history | egrep '([0-9]{1,3}\.){3}[0-9]{1,3}'

Example output

┌──(kali㉿localhost)-[~]
└─$ history | egrep '([0-9]{1,3}\.){3}[0-9]{1,3}' | head -n2
1 ssh kali@localhost

Check if SSH Public Key is on Shodan

Since the SSH public key is um, well, public, we can search for it on Shodan to see if it’s a known public server. https://www.shodan.io

Copy the public ssh key from the known_hosts file. It is the last portion of the line i.e.
AAAAC3NzaC1lZDI1NTE5AAAAIEjqG8/el8c669FxcvEw5mMfDRTDxsjgLiz44dCTtchs

|1|ma8KL2XrNYkNnknf68N4IuZ+c+I=|PmR+n2i0/epUGZZh2S+LB6OaowQ= ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEjqG8/el8c669FxcvEw5mMfDRTDxsjgLiz44dCTtchs

Brute force

Since the address space for IPv4 is fairly small, and the private IP address space even smaller, brute forcing all the addresses is perfectly feasible.

Here is a quick example on how you would hash an IP address. Commands are taken from the above Stack Exchange link.

Take the salt and put it into a variable

key=`echo j2j9iv/GkPfnG9Yv4WzJsy/L1pc= | base64 -d | xxd -p`

Next we can run the following command to hash the result. The IP (127.0.0.1) is where we would want to enumerate the IP address.

echo -n "127.0.0.1" | openssl sha1 -mac HMAC -macopt hexkey:$key|awk '{print $2}' | xxd -r -p | base64

The output is PmR+n2i0/epUGZZh2S+LB6OaowQ= which is the correct hash.

Automating should be fairly simple.

A note on SSH ports. If the host is using a non standard ssh port, you will need to update the above command with the port, but the address needs to be wrapped in square brackets []

echo -n "[127.0.0.1]:2222" | openssl sha1 -mac HMAC -macopt hexkey:$key|awk '{print $2}' | xxd -r -p | base64

Use ssh-keyscan

A final way we can discover known-hosts, is by using ssh-keyscan. The man page says the following

ssh-keyscan is a utility for gathering the public SSH host keys of a number of hosts. It was designed to aid in building and verifying ssh_known_hosts files

ssh-keyscan uses non-blocking socket I/O to contact as many hosts as possible in parallel, so it is very efficient. The keys from a domain of 1,000 hosts can be collected in tens of seconds, even when some of those hosts are down or do not run sshd(8). For scanning, one does not need login access to the machines that are being scanned, nor does the scanning process involve any encryption.

Hosts to be scanned may be specified by hostname, address or by CIDR network range (e.g. 192.168.16/28). If a network range is specified, then all addresses in that range will be scanned.

This makes it super convenient to do a network scan using ssh-keyscan and then compare the public ssh keys with those in the known_hosts file.

Example:

ssh-keyscan 192.168.0.0/16

To scan all private IP ranges (RFC1912), we just run the scan with all three IP ranges

ssh-keyscan 192.168.0.0/16
ssh-keyscan 172.12.0.0/12
ssh-keyscan 10.0.0.0/8

Check for backdoored version of xz (CVE-2024-3094) (Ansible/Bash)

Info on the xc backdoor

https://www.openwall.com/lists/oss-security/2024/03/29/4

https://tukaani.org/xz-backdoor/

https://www.tenable.com/blog/frequently-asked-questions-cve-2024-3094-supply-chain-backdoor-in-xz-utils

Kostas on Twitter posted a helpful one-liner to check the xz version without running the actual command.

https://twitter.com/kostastsale/status/1773890846250926445

Versions 5.6.0 and 5.6.1 are backdoored.

Bash one liner

The following Bash commands were taken and modified from the above Twitter link

Here is a one liner that will check the version of xz binaries and return if they are safe or vulnerable. You’ll need to run this in a Bash shell. May have issues in sh.

for xz_p in $(type -a xz | awk '{print $NF}' ); do  if ( strings "$xz_p" | grep "xz (XZ Utils)" | grep '5.6.0\|5.6.1' ); then echo $xz_p Vulnerable; else echo $xz_p Safe ; fi ; done 

Ansible Playbooks

Here are two different Ansible Playbooks to check if the xz package(s) are backdoored.

This one uses the above Bash commands to check the xz binaries.

---
- name: Check if XZ tools are compromised
# https://twitter.com/kostastsale/status/1773890846250926445
  hosts: all

  tasks: 
    - name: Run Bash command
      shell : 
        for xz_p in $(type -a xz | awk '{print $NF}' ); do 
          if ( strings "$xz_p" | grep "xz (XZ Utils)" | grep '5.6.0\|5.6.1' ); 
            then echo $xz_p Vulnerable!; 
          else 
            echo $xz_p Safe ; 
          fi ; 
        done
      args: 
        executable: /bin/bash
      register: result

    - name: Show output
      ansible.builtin.debug:
        msg: "{{ result.stdout_lines }}"

The following playbook uses the package manager to check the xz version. On RHEL/Fedora this is the xc package. On Debian/Ubuntu, it is part of the liblzma5 package.

---
- name: Check if XZ tools are compromised
  hosts: all

  tasks:
    - name: Collect package info
      ansible.builtin.package_facts:
        manager: auto

    - name: Check if liblzma5 is vulnerable (Ubuntu/Debian)
      ansible.builtin.debug:
        msg: "Installed version of liblzma5/xz: {{ ansible_facts.packages['liblzma5'] | map(attribute='version') | join(', ') }} Vulnerable!"
      when: ('liblzma5' in ansible_facts.packages) and (ansible_facts.packages['liblzma5'][0].version.split('-')[0] is version('5.6.0', '==') or ansible_facts.packages['liblzma5'][0].version.split('-')[0] is version('5.6.1', '=='))

    - name: Check if xz is vulnerable (RHEL/Fedora/Rocky/Alma)
      ansible.builtin.debug:
        msg: "Installed version of xz: {{ ansible_facts.packages['xz'] | map(attribute='version') | join(', ') }} is vulnerable"
      when: ('xz' in ansible_facts.packages) and (ansible_facts.packages['xz'][0].version is version('5.6.0', '==') or ansible_facts.packages['xz'][0].version is version('5.6.1', '=='))

How to Disable the Bandwidth Server on Mikrotik/RouterOS

The Bandwidth test tool can be helpful to test speed between Mikrotik routers. But you can disable it if you don’t need it.

From Winbox

From Winbox click on Tools > BTest Server > Disable > OK

From Command Line

From the command line you can disable the bandwidth server by running the following command.

/tool/bandwidth-server/set enabled=no

If you are still on RouterOS 6.x use

/tool bandwidth-server set enabled=no

Enable Bandwidth test

If you need to enable the bandwidth server again, just change enabled=no to enabled=yes

/tool bandwidth-server set enabled=yes

https://grohler.com/disable-mikrotik-bandwidth-btest-server/

Handling Spaces in File Names on Linux

Using ls to parse file names is not recommended for multiple reasons

https://mywiki.wooledge.org/ParsingLs

Let’s say we have a directory with two files in it.

Helloworld.txt
Hello, world.txt

Now we want to loop over the files. If we use ls in our for loop,

for file in $(ls); do echo "$file" ; done

We receive the following output

Hello,
world.txt
Helloworld.txt

The space in “Hello, world.txt” is translated as a new line. This could break our script.

Here is a better way

for file in * ; do echo "$file" ; done

Helpful links

https://mywiki.wooledge.org/BashPitfalls

Harden SSH for AlmaLinux 9 (RHEL, Fedora)

These steps are taken from the following link. They have other guides for hardening Ubuntu, Debian etc.

https://www.sshaudit.com/hardening_guides.html#rocky9

You will need to become the root user, use either su – or sudo -i

First we need to regenerate the RSA and ED25519 keys

rm /etc/ssh/ssh_host_*
ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_rsa_key -N ""

Next, remove the small Diffie-Hellman moduli. The moduli file contains prime numbers and generators. Removing the smaller numbers should help increase security as it makes attempting to factor the private keys harder.

awk '$5 >= 3071' /etc/ssh/moduli > /etc/ssh/moduli.safe
mv /etc/ssh/moduli.safe /etc/ssh/moduli

We can now specify which key exchange, ciphers, and algorithms to use.

Add the following to “/etc/crypto-policies/back-ends/opensshserver.config”

# Restrict key exchange, cipher, and MAC algorithms, as per sshaudit.com
# hardening guide.
KexAlgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,gss-curve25519-sha256-,diffie-hellman-group16-sha512,gss-group16-sha512-,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha256

Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

MACs hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-128-etm@openssh.com

HostKeyAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256

RequiredRSASize 3072

CASignatureAlgorithms sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256

GSSAPIKexAlgorithms gss-curve25519-sha256-,gss-group16-sha512-

HostbasedAcceptedAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-256

PubkeyAcceptedAlgorithms sk-ssh-ed25519-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256

Finally, restart the ssh server

systemctl restart sshd

Other helpful links

https://www.ssh.com/academy

https://www.redhat.com/en/blog/primes-parameters-and-moduli

https://security.stackexchange.com/questions/79043/is-it-considered-worth-it-to-replace-opensshs-moduli-file

Configuring Firewalld with Ansible

We’ll be using Ansible to change and maintain our firewall settings on a server.

The playbook will do the following.

  1. Set the default zone to drop (Drops all external traffic to server)
  2. Set a zone for internal access
  3. Allow access from RFC1918 addresses to internal zone (Any local IP address will be able to access the server)
  4. Enable the services and ports specified in the vars section
  5. Disable the services listed in firewall_disable_services variable

Modify the variables as needed for your server(s). You can also add or move the variables to the inventory or host_vars files.

If you need to create an inventory file, refer to the first part of this post

BE CAREFUL CHANGING FIREWALL SETTINGS!!! IMPROPER SETTINGS COULD RENDER THE SERVER INACCESSIBLE!!!

Playbook for firewalld

Change the variables under the vars section

---
- name: Configure firewalld
  hosts: rhel
  gather_facts: yes
  become: yes

  vars: 
    firewall_allowed_ips:
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 192.168.0.0/16
    firewall_allowed_services:
      - ssh
      - https
      - snmp
    firewall_allowed_ports:
      - "2222/tcp"
    firewall_disable_services:
      - cockpit
      - dhcpv6-client
      - mdns
      - samba-client

  tasks: 
  - name: Set default zone to drop
    ansible.builtin.command: firewall-cmd --set-default-zone=drop
    register: default_zone_set
    changed_when:
      - '"ZONE_ALREADY_SET" not in default_zone_set.stderr'

  - name: Enable and allow access to internal zone from RFC1918 addresses
    ansible.posix.firewalld:
      source: "{{ item }}"
      zone: internal
      permanent: true
      immediate: true
      state: enabled
    with_items: "{{ firewall_allowed_ips }}"

  - name: Disable unused services for internal zone
    ansible.posix.firewalld:
      service: "{{ item }}"
      zone: internal
      permanent: true
      immediate: true
      state: disabled
    with_items: "{{ firewall_disable_services }}"


  - name: Set services for internal zone
    ansible.posix.firewalld:
      service: "{{ item }}"
      zone: internal
      permanent: true
      immediate: true
      state: enabled
    with_items: "{{ firewall_allowed_services }}"

  - name: Set custom ports for internal zone
    ansible.posix.firewalld:
      port: "{{ item }}"
      zone: internal
      permanent: true
      immediate: true
      state: enabled
    with_items: "{{ firewall_allowed_ports }}"

Helpful links

https://docs.ansible.com/ansible/latest/collections/ansible/posix/firewalld_module.html#parameter-source

https://stackoverflow.com/questions/51563643/how-to-change-firewalld-zone-using-ansible

https://www.middlewareinventory.com/blog/ansible-firewalld/

Ansible Playbook for Updating Mikrotik RouterOS

This playbook is for updating Mikrotik routers. It will update both the RouterOS version and the firmware.

The playbook executes in the following order.

  1. Check for RouterOS Updates
  2. Update RouterOS (Router will reboot if there is an update)
  3. Sleep 120 seconds to allow the router(s) to boot up
  4. Check current firmware version, and if there is an available upgrade
  5. Update firmware
  6. Reboot router to apply firmware upgrade

This playbook attempts to be smart and will not reboot a router if there is not an update available. Routers that have updates available will reboot twice. Once to apply the RouterOS version, and the second time to apply the firmware.

Prerequisites

You should already have an inventory file and the Ansible RouterOS collection installed. If not, check out the following post.

Setup Ansible host file and RouterOS collection

Playbook

Here is the playbook.
A quick command syntax note, RouterOS 7 and newer typically use slashes / between commands. i.e. /system/package/update/install. Older versions of RouterOS have spaces in the command path i.e. /system package update install Since this still works on newer versions, we use it here.

---
- name: Mikrotik RouterOS and Firmware Upgrades
  hosts: routers
  gather_facts: false
  tasks:

# Update RouterOS version.  Mikrotik update/install command automatically reboots the router
  - name: Check for RouterOS updates
    community.routeros.command:
      commands:
        - /system package update check-for-updates
    register: system_update_print

  - name: Update RouterOS version
    community.routeros.command:
      commands:
        - /system package update install
    when: system_update_print is not search('System is already up to date')

# Check if firmware needs an upgrade, upgrade and reboot.
  - name: Sleeping for 120 seconds.  Giving time for routers to reboot.
    ansible.builtin.wait_for:
      timeout: 120
    delegate_to: localhost
      
  - name: Check Current firmware
    community.routeros.command:
      commands:
        - ':put [/system routerboard get current-firmware]'
    register: firmware_current

  - name: Check Upgrade firmware 
    community.routeros.command:
      commands:
        - ':put [/system routerboard get upgrade-firmware]'
    register: firmware_upgrade

  - name: Upgrade firmware
    community.routeros.command:
      commands:
        - ':execute script="/system routerboard upgrade"'
    when: firmware_current != firmware_upgrade

  - name: Wait for firmware upgrade and then reboot
    community.routeros.command:
      commands:
        - /system routerboard print
    register: Reboot_Status
    until: "Reboot_Status is search(\"please reboot\")"
    notify:
      - Reboot Mikrotik
    retries: 3
    delay: 15
    when: firmware_current != firmware_upgrade

  handlers:
    - name : Reboot Mikrotik
      community.routeros.command:
        commands:
          - ':execute script="/system reboot"'

Run the playbook with

ansible-playbook -i routers.ini mikrotik_update.yaml

Change routers.ini to your router inventory.
mikrotik_update.yaml to whatever you end up calling the playbook.

Setup Ansible and Mikrotik RouterOS

https://docs.ansible.com/ansible/devel/collections/community/routeros/

https://github.com/ansible-collections/community.routeros

Install the RouterOS collection.

ansible-galaxy collection install community.routeros

Create inventory

vi inventory.ini
[routers]
mikrotik ansible_host=192.168.88.1

[routers:vars]
ansible_connection=ansible.netcommon.network_cli
ansible_network_os=community.routeros.routeros
ansible_user=admin
ansible_ssh_pass=
ansible_ssh_port=22

If you are using a custom SSH port, be sure that ansible-pylibssh is installed.

pip install ansible-pylibssh

Simple Playbook

This simple playbook will print the system resources. Playbook is taken from here.

---
- name: RouterOS test with network_cli connection
  hosts: routers
  gather_facts: false
  tasks:

  - name: Gather system resources
    community.routeros.command:
      commands:
        - /system resource print
    register: system_resource_print

  - name: Show system resources
    debug:
      var: system_resource_print.stdout_lines

  - name: Gather facts
    community.routeros.facts:

  - name: Show a fact
    debug:
      msg: "First IP address: {{ ansible_net_all_ipv4_addresses[0] }}"