Here are the commands you’ll need to harden SSH on your Mikrotik Routers. It looks like it still can use SSH-RSA, but it does get rid of most of the weaker crytpo algorithms.
In this example, the server is already using Let’s Encrypt to create the certificate for a LibreNMS server. So all we are doing is copying the certificate to a Grafana directory, putting the correct permissions on it, and updating the Grafana config file to use the certificate.
Steps
Copy Certificate to Grafana Directory
Configure Grafana Config File
Automate Certificate Copy to Grafana Directory
Copy Certificate files
In the following commands, change librenms.incredigeek.com to the directory that Let’s Encrypt is using for your fully qualified domain name (FQDN). Usually it is just your FQDN, but could also have -0001 or something appended to the end.
cp -f /etc/letsencrypt/live/librenms.incredigeek.com/privkey.pem
/etc/grafana/
cp -f /etc/letsencrypt/live/librenms.incredigeek.com/fullchain.pem /etc/grafana/
chown root:grafana /etc/grafana/*.pem
chmod 640 /etc/grafana/*.pem Enable grafana on system bootup
In the above, we are copying the privkey.pem and fullchain.pem to /etc/grafana. We are then setting the correct owner/permissions on the files so that the Grafana service can read the certificate.
Configure Grafana Config File
This is super easy. Open up the Grafana config file in /etc/grafana.ini
vi /etc/grafana.ini
Find the following variables and configure them like so
You should now have a working SSL certificate for the site.
Automate Certificate Copy
Let’s Encrypt certificates need to be updated frequently. This means that we should automate the above steps to avoid any down time. After all, a monitoring tool with down time defeats the purpose of monitoring.
We’ll need to create a root crontab
sudo crontab -e
Add the following changing out the FQDN to your FQDN.
This is set to run once a month. Change if desired. Also change out librenms.incredigeek.com with your FQDN.
Note about domain name and IP addresses. Let’s Encrypt will not create a certificate for an IP address. You should be using a domain name instead (i.e. networkmonitoring.yourdomain.com) If the certificate is installed, and you access it via the IP address, you will receive a HTTPS error in your browser.
LibreNMS uses fping to check if devices are up or not. So if something is broken with fping, say a SELinux permission, you can receive the “Could not ping” error, while trying to add a new device.
LibreNMS unable to ping device
First we need to verify that fping is working. SSH into the LibreNMS server and try pinging an address.
fping 192.168.1.20
There was an issue with fping working if ipv6 was disabled. If fping is not working at all, check out this thread.
If you get an alive or unreachable message, then we know fping is working and can move on to the next stage of troubleshooting.
If you are using SELinux, then there is a good chance the problems has to do with that. You can try rerunning all the SELinux commands from the install guide. Note that it has a specific portion for fping.
Now we have verified that the issue is SELinux permissions related. We can create a module to allow it.
audit2allow -a -M fping_http < /var/log/audit/audit.log
And apply the module with
semodule -i fping_http.pp
You may need to do this a couple times. Check the audit log again to see if anything new shows up. Notice the slight difference in this error compared to the above error.
audit2allow -a -M node_http < /var/log/audit/audit.log
semodule -i node_http.pp
Not sure that is the best way to fix the problem. But it appears that SELinux is keeping Apache “httpd” from running fping which is why we need to create and load the modules.
Renaming of 192.168.1.20 failed . Does your web server have permission to modify the rrd files?
If you are getting the above error while trying to rename a device in LibreNMS, you may need to rerun some of the SELinux commands from the installation.
SSH into the server and run
restorecon -RFvv /opt/librenms
Now try renaming the device. Note that you can’t rename the device if the name/ip to a name that is being used by a different device.
If you continue to have issues, check the permissions from the installation guide (Official guide here)
Note that “mymodule.pp” will replace any previous “mymodule.pp”. If your needing to create multiple modules/allow multiple exceptions, you can change the name of each module.
You can also add the rules together then manually compile it. Refer to the first link for more details.
Create docker compose file with the following options
vi podman-compose.yml
Change the TZ to your time zone. If you have issues with the graphs, most likely something is off with the time zone between this container and Grafana/LibreNMS server