LibreNMS Error Running ./validate.php – Database: incorrect column…

LibreNMS error while running ./validate.php

Recently I started getting the following error while running ./validate.php.

$ sudo -u librenms ./validate.php
====================================
Component | Version
--------- | -------
LibreNMS  | 21.8.0-41-g0a76ca4
DB Schema | 2021_08_26_093522_config_value_to_medium_text (217)
PHP       | 7.4.20
Python    | 3.6.8
MySQL     | 10.5.10-MariaDB
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================

[OK]    Composer Version: 2.1.6
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[FAIL]  Database: incorrect column (notifications/datetime)
[FAIL]  Database: incorrect column (users/created_at)
[FAIL]  We have detected that your database schema may be wrong, please report the following to us on Discord (https://t.libren.ms/discord) or the community site (https://t.libren.ms/5gscd):
        [FIX]:
        Run the following SQL statements to fix.
        SQL Statements:
         SET TIME_ZONE='+00:00';
         ALTER TABLE `notifications` CHANGE `datetime` `datetime` timestamp NOT NULL DEFAULT '1970-01-02 00:00:00' ;
         ALTER TABLE `users` CHANGE `created_at` `created_at` timestamp NOT NULL DEFAULT '1970-01-02 00:00:01' ;

Reading online it sounds like some of the recent changes are causing the issue. Looks fairly easy to resolve though.

First we’ll need to get a MySQL prompt. We’ll do that by running

mysql -u librenms -p librenms

It’ll ask use for the librenms user’s mysql password.

Once we have the MySQL prompt we can just copy and paste the commands in.

SET TIME_ZONE='+00:00';
ALTER TABLE `notifications` CHANGE `datetime` `datetime` timestamp NOT NULL DEFAULT '1970-01-02 00:00:00' ;
ALTER TABLE `users` CHANGE `created_at` `created_at` timestamp NOT NULL DEFAULT '1970-01-02 00:00:01' ;

Type ‘quit’ to exit MySQL and lets run the validate script again.

sudo -u librenms ./validate.php

Everything should check out OK.

LibreNMS Error – port_groups Base table or view already exists

port_groups issue upgrading LibreNMS

The above error can be resolved by dropping the port_groups table in mysql. You may double check that there is no information in the table. I checked this instance and it didn’t have any data in the table so I dropped it with

drop table port_groups;

I then ran the lnms command again and it created the table and started migrating.

Show Down Devices From SSH – LibreNMS

Fortunately, it is super easy to get a list of down devices in LibreNMS. All we need to do is a dump of the devices from MySQL.

You can use the following command to print which devices are down. Should be the same devices that show down on your dashboard.

mysql -u librenms -p librenms -e 'use librenms ; select hostname,sysName,status from devices where status=0'

Tip: You can change 0 to 1 to see all up devices.

Show down devices via SSH in LibreNMS

Migrate LibreNMS to new Server

Some quick notes on manually moving LibreNMS to a new server.  For more detailed instructions you can refer to the following post

Recovering LibreNMS from crashed XenServer VM

Steps

  1. Backup /opt/librenms file
  2. Backup librenms database
  3. Install LibreNMS on new server
  4. Import /opt/librenms files
  5. Import librenms database

Backing up LibreNMS files

LibreNMS stores all it’s files in /opt/librenms
Quick and easy way to back up the LibreNMS files is to use tar.

tar zcvf librenms_backup.tgz /opt/librenms

Migrating Database

I believe you can copy the whole /var/lib/mysql directory.  Should be able to use the above command.  If not you can do a mysqldump and then import it on the new system.

Cannot load Zend OPcache – it was already loaded

Started getting the following error while running ./validate.php for LibreNMS

Cannot load Zend OPcache - it was already loaded

Looks like the problem arises out of PHP trying to load two ini files for OPcache. Was trying to enable OPcache for LibreNMS so I created an opcache.ini file and put the settings in it. I missed the default 10-opcache.ini file.

Moving all the settings into the 10-opcache.ini file and deleting the created opcache.ini file resolved the issue for me.

https://unix.stackexchange.com/questions/253448/php-7-install-throws-cannot-load-zend-opcache-it-was-already-loaded-error/253484

LibreNMS upgrade PHP from 7.2 to 7.4 – CentOS 7

First you’ll need to install the remi repo

sudo wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo wget http://rpms.remirepo.net/enterprise/remi-release-7.rpm
rpm -Uvh remi-release-7.rpm epel-release-latest-7.noarch.rpm

You’ll need to install yum-utils, disable all old php versions, and enable the remi-php74

sudo yum install -y yum-utils
sudo yum-config-manager --disable remi-php56
sudo yum-config-manager --disable remi-php71
sudo yum-config-manager --disable remi-php72
sudo yum-config-manager --disable remi-php73
sudo yum-config-manager --enable remi-php74

If you run into issues you may want to see if you have the webtatic repo installed. I had to disable it to get php to update.

vi /etc/yum.repos.d/webtatic
[webtatic]
name=Webtatic Repository EL7 - $basearch
baseurl=https://repo.webtatic.com/yum/el7/$basearch/
mirrorlist=https://mirror.webtatic.com/yum/el7/$basearch/mirrorlist
failovermethod=priority
enabled=0   <- Disabled by changing to a 0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-webtatic-el7

And running the yum update should update it to php 7.4

sudo yum update

More information available here.

LibreNMS update to Python 3

You may get the following alert in LibreNMS. Basically you need to install python 3 to keep things up to date.

Python 3 is required to run LibreNMS as of May, 2020. You need to install Python 3 to continue to receive updates. If you do not install Python 3 and required packages, LibreNMS will continue to function but stop receiving bug fixes and updates.

Install Python 3

Install Python 3 with yum, or apt if you are on a Debian based distro.

sudo yum install python3
sudo pip3 install -r /opt/librenms/requirements.txt

Verify LibreNMS is updated and working

Run the following commands to make sure that LibreNMS is working correctly and is up to date.

cd /opt/librenms
sudo ./validate.php
sudo ./daily.sh

LibreNMS ./validate.sh – Some folders have incorrect file permissions, this may cause issues.

Helpful article here. https://wifitechtalk.com/librenms-permissions-error/

When running the following command,

sudo /opt/librenms/validate.sh

I’ve been getting the following errors.

[FAIL]  Some folders have incorrect file permissions, this may cause issues.
    [FIX]: 
    sudo chown -R librenms:librenms /opt/librenms
    sudo setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
    sudo chmod -R ug=rwX /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
    Files:
     /opt/librenms/storage/framework/views/adc52b677409cdba8d8e89dc

You can run the commands and fix the problem, but they pop up later and it does the same thing.

Create Script to fix permissions

Work around is to have a script run these commands every few minutes.

vi /root/librenms_fix_permissions.sh

Add the following

#!/bin/bash 
sudo chown -R librenms:librenms /opt/librenms 
sudo setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/ 
sudo chmod -R ug=rwX /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/

Make the script executable

sudo chmod +x librenms_fix_permissions.sh

Add to Crontab

sudo crontab -e 

Add the following entry to Cron to run every 30 minutes

*/30 * * * * /root/librenms_fix_permissions.sh

LibreNMS – Daily.sh failed

All the following commands are run from the “/opt/librenms” directory

$ sudo ./daily.sh 
Re-running /opt/librenms/daily.sh as librenms user
Updating to latest codebase                        FAIL
error: Your local changes to the following files would be overwritten by merge:
        html/js/lang/de.js
        html/js/lang/en.js
Please, commit your changes or stash them before you can merge.
...
$

Checking with validate.php shows that you can run githup-remove to fix it

$ sudo ./validate.php
[WARN]  Your install is over 24 hours out of date, last update: Mon, 23 Feb 2020 05:43:12 +0000
         [FIX]: 
         Make sure your daily.sh cron is running and run ./daily.sh by hand to see if there are any errors.
 [WARN]  Your local git contains modified files, this could prevent automatic updates.
         [FIX]: 
         You can fix this with ./scripts/github-remove
         Modified Files:
          html/js/lang/de.js
...
$

Run “sudo ./scripts/github-remove -d”

$ sudo ./scripts/github-remove -d
Are you sure you want to delete all modified and untracked files? [y/N] y
$

You should also be able to remove the files individually if the above command does not work.

Run validate again to make sure it checks out good.

Now run ./daily.sh