Repairing Fedora grub from chroot

Notes from repairing a Fedora drive.

Mount system in chroot.

If the system is a raid drive and your not able to access it refer to this post.  May just need to install the raid utilities.

For mounting the chroot environment refer to this post

Repairing grub

yum install grub2-efi-*

Install grub.  Change /sda to your drive, may need to specify the efi partition.

grub2-install /dev/sda

If your boot and efi partitions are mounted.

grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

Reboot.

DNF/YUM not working in chroot environment

The reason is probably because the chrooted environment can’t resolve DNS.

Test it with

ping incredigeek.com

If it is not resolving, edit “/etc/resolv.conf” and change/add your nameserver.  Or just replace everything in it with

echo "nameserver 4.2.2.2" > /etc/resolv.conf

It should now be able to resolve and you should be able to use yum, or dnf.

yum update

Add Self Signed SSL certificate to LibreNMS in CentOS

Install mod_ssl

yum install mod_ssl -y

Create Directory for SSL key.

mkdir /etc/ssl/key
chmod 700 /etc/ssl/key

Create certificate.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/key/localhost.key -out /etc/ssl/certs/localhost.crt

Fill out the info or what is applicable.

Now edit the LibreNMS Apache config file /etc/httpd/conf.d/librenms.conf

All you have to do is add the following three lines under the VirtualHost and change *:80 to *:443.

SSLEngine on
SSLCertificateFile /etc/ssl/certs/localhost.crt
SSL CertificateKeyFile /etc/ssl/key/localhost.key

So when your finished the file should look like this.

<VirtualHost *:443>
 DocumentRoot /opt/librenms/html/
 ServerName server_hostname_or_IP
 SSLEngine on
 SSLCertificateFile /etc/ssl/certs/localhost.crt
 SSLCertificateKeyFile /etc/ssl/key/localhost.key
 CustomLog /opt/librenms/logs/access_log combined
 ErrorLog /opt/librenms/logs/error_log
 AllowEncodedSlashes NoDecode
 <Directory "/opt/librenms/html/">
 Require all granted
 AllowOverride All
 Options FollowSymLinks MultiViews
 </Directory>
</VirtualHost>

Don’t forget to allow https/port 443 traffic through the firewall.  Guide here

If you have any issues, you may need to chmod the key and crt file.

chmod 644 /etc/ssl/certs/localhost.crt
chmod 644 /etc/ssl/key/localhost.key

You should now be able to access LibreNMS using https.  Note, you’ll need to allow an exception in your browser for your self signed certificate.

https://LibreNMS_IP_Address

Recovering LibreNMS from crashed XenServer VM

Had a LibreNMS instance crash, or the VM crashed, not bootable anymore.  Was able to boot it up on a CentOS iso with rescue mode, which gave me access to the files.  So the idea is to manually copy off the LibreNMS files and LibreNMS database and import them to a new LibreNMS instance

Issues

  • Need to access the the system files
  • MySQL doesn’t start in a chroot environment, so no way to do a mysqldump —  (Has to do with systemv or something)

Steps

  1. Create new LibreNMS VM with a new instance of LibreNMS installed
  2. Gain access to the crashed system
  3. Copy over LibreNMS MySQL databases to new LibreNMS instance
  4. Copy over LibreNMS files “/opt/librenms” to new LibreNMS instance
  5. Clean up.  Set users on directories, check SELinux etc.

1. Installing New LibreNMS VM

  • Guide for Ubuntu/Debian distro’s here
  • Guide for for Fedora/CentOS/RedHat here

2. Gaining access to crashed VM

There can be a couple of ways to gain access to a crashed XenServer VM.  One of the easiest ways is to boot up in recovery mode and go through the installers rescue mode.  Guide to boot up XenServer VM in recovery mode here.

The installer rescue mode should detect the OS and mount everything.  If not you should be able to mount the root partition manually.

Once booted up, you’ll need to enable network access if your going to use sftp or scp to copy files.  There are a few different ways to do this

  • Run “dhclient” to pull an address via DHCP
  • Set a static IP address
    • Guide for Ubuntu/Debian distro’s here
    • Guide for fedora/CentOS/RedHat here

3.Copy LibreNMS Mysql Database

Backup the LibreNMS MySQL database directory

tar czvf librenms_mysql.tgz /var/lib/mysql

Use scp or sftp to copy it to the new LibreNMS instance

scp librenms_mysql.tgz user@new_LibreNMS_ip

Now on the new LibreNMS instance we need to run the following few commands

systemctl stop mariadb
rm -rf /var/lib/mysql/*
tar xzvf librenms_mysql.tgz -C /
chown -R mysql:mysql /var/lib/mysql

4. Copy LibreNMS directory

Backup the LibreNMS directory

tar czvf librenms.tgz /opt/librenms

Use scp or sftp to copy it to the new LibreNMS instance

scp librenms.tgz user@new_LibreNMS_ip

Now on the new LibreNMS instance we need to run the following few commands

rm -rf /opt/librenms/*
tar xzvf librenms.tgz -C /
chown -R librenms:librenms /opt/librenms

5. Clean up

Disable SELinux if you have not already.  Guide here

Restart apache, and start mysql.  If your on Ubuntu, the services are named apache2 and mysql

systemctl restart httpd
systemctl restart mariadb

That should get it working, if not try a reboot.

Special notes

The whole MySQL directory needs to be copied, there are innodb files that will keep MySQL from starting if they are not copied.

There is some good info here
https://stackoverflow.com/questions/1795176/how-to-change-mysql-data-directory

Install LibreNMS on CentOS

Install prerequisites

yum install -y epel-release 
rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
yum install -y policycoreutils-python cronie fping git httpd ImageMagick jwhois mariadb mariadb-server mtr MySQL-python net-snmp net-snmp-utils nmap php71w php71w-cli php71w-common php71w-curl php71w-gd php71w-mcrypt php71w-mysql php71w-process php71w-snmp php71w-xml php71w-zip python-memcached rrdtool

Setup LibreNMS user

useradd librenms -d /opt/librenms -M -r
usermod -a -G librenms apache

Install LibreNMS

cd /opt
git clone https://github.com/librenms/librenms.git librenms

Configure MySQL

Start MySQL

systemctl start mariadb
mysql -u root

Create the Database and database user. Change password to a different password

CREATE DATABASE librenms CHARACTER SET utf8 COLLATE utf8_unicode_ci;
CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON librenms.* TO 'librenms'@'localhost';
FLUSH PRIVILEGES;
exit

Disable MySQL strict mode

vi /etc/my.cnf

Add the following in the [mysqld] section

innodb_file_per_table=1
sql-mode=""
lower_case_table_names=0

Enable and restart mariadb i.e.(MySQL)

systemctl enable mariadb
systemctl restart mariadb

Configure PHP.  Set Timezone

You can go here to figure out which time zone you need.

vi  /etc/php.ini

Find the [Date] Section and change the timezone

Example:

date.timezone = America/New_York

Configure Apache

vi /etc/httpd/conf.d/librenms.conf

Add the following, change the server name.

<VirtualHost *:80>
  DocumentRoot /opt/librenms/html/
  ServerName  librenms.example.com

  AllowEncodedSlashes NoDecode
  <Directory "/opt/librenms/html/">
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews
  </Directory>
</VirtualHost>

Enable and restart apache

systemctl enable httpd
systemctl restart httpd

Allow fping

cd /opt/librenms
vi http_fping.tt

Add the following

module http_fping 1.0;

require {
type httpd_t;
class capability net_raw;
class rawip_socket { getopt create setopt write read };
}

#============= httpd_t ==============
allow httpd_t self:capability net_raw;
allow httpd_t self:rawip_socket { getopt create setopt write read };

Configure SELinux

You can disable SELinux completely in the /etc/selinux/config file or run these commands to make LibreNMS and SELinux play together.

semanage fcontext -a -t httpd_sys_content_t '/opt/librenms/logs(/.*)?'
semanage fcontext -a -t httpd_sys_rw_content_t '/opt/librenms/logs(/.*)?'
restorecon -RFvv /opt/librenms/logs/
semanage fcontext -a -t httpd_sys_content_t '/opt/librenms/rrd(/.*)?'
semanage fcontext -a -t httpd_sys_rw_content_t '/opt/librenms/rrd(/.*)?'
restorecon -RFvv /opt/librenms/rrd/
setsebool -P httpd_can_sendmail=1
checkmodule -M -m -o http_fping.mod http_fping.tt
semodule_package -o http_fping.pp -m http_fping.mod
semodule -i http_fping.pp

Configure Firewall

firewall-cmd --zone public --add-service http
firewall-cmd --permanent --zone public --add-service http
firewall-cmd --zone public --add-service https
firewall-cmd --permanent --zone public --add-service https

Configure snmpd

cp /opt/librenms/snmpd.conf.example /etc/snmp/snmpd.conf

Edit the snmpd file and change RANDOMSTRINGGOESHERE to your community string

vi /etc/snmp/snmpd.conf

Enable and restart snmpd

curl -o /usr/bin/distro https://raw.githubusercontent.com/librenms/librenms-agent/master/snmp/distro
chmod +x /usr/bin/distro
systemctl enable snmpd
systemctl restart snmpd

Setup Cron Job and Cycle logs

cp /opt/librenms/librenms.nonroot.cron /etc/cron.d/librenms
cp /opt/librenms/misc/librenms.logrotate /etc/logrotate.d/librenms

Set Permisions

chown -R librenms:librenms /opt/librenms
setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs
setfacl -R -m g::rwx /opt/librenms/rrd /opt/librenms/logs

Finish Install

Finish the install by

http://your-server-ip/install.php

Change timezone in CentOS, Fedora, RedHat

The file /etc/localtime is a symbolic link to the timezone.  All the timezones are listed in /usr/share/zoneinfo/

Replace America/New_York with the appropriate timezone.

ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime

You can view the current time zone with the following command

timedatectl

or with date

date +"%Z %z"

Install SSH Server on Linux (Debian, Ubuntu, Fedora, CentOS, RedHat)

Debian / Ubuntu

sudo apt-get install -y openssh-server

RPM based Distros, Fedora / CentOS / RedHat

sudo dnf install -y openssh-server

or use yum

sudo yum install -y openssh-server

Start ssh service

sudo systemctl start sshd

By default the SSH service should start when the system starts, but if not try the following command to enable the service on boot up.

Debian / Ubuntu

systemctl enable ssh

Fedora, CentOS, RedHat

systemctl enable sshd

Change SSH port

Not necessary, but it is a good idea to change the default ssh port.  To change the port edit the sshd file.

vi /etc/ssh/sshd_config

If you change the port, you’ll need to allow it in the firewall (firewalld, iptables) and if SELinux is enabled, semanage.

Allow SSH access from a specific host using hosts.allow and hosts.deny on Linux

This is just a quick write on the hosts.allow and deny files.  You can lookup “spawn” and/or “twist” for some advanced usage.

 

So to limit an IP address, or a IP range access to SSH, do the following

Deny all incoming request for SSH

Edit the “hosts.deny” file

vi /etc/hosts.deny

add the following line

sshd : ALL

Now edit “hosts.allow” and allow the client IP, or IP range to access SSH

vi /etc/hosts.allow

add the following line to allow a single IP

sshd : 192.168.1.182

If you want to allow the whole subnet, then replace the above line with this one

sshd : 192.168.1.

hosts.allow overrides hosts.deny.  So you deny everything and then allow exceptions.

How to Install ZenPack on Zenoss 5.1

To install a ZenPack on Zenoss 5.1 and later, you are going to need to stop the zenoss services, restart a couple services that are needed to install the ZenPack, install the ZenPack, and then restart the Zenoss service.

Stop Zenoss.Core :

serviced service stop 

Make sure the service is stopped by running the following command.  When it reports back the Zenoss is Stopped, then continue.

 if ( serviced service status Zenoss.Core | awk '{print $3}' | grep -q Stopped) ; then echo "Zenoss is not running!" ; else echo "Zenoss is not Stopped!" ; fi

Or you can run this command, just make sure it says it is stopped.

serviced service status zenoss.core

Create a Snapshot

serviced service snapshot Zenoss.core

Start the following three services

serviced service start Infrastructure
zeneventserver
Zope


serviced service start Infrastructure
serviced service start zeneventserver
serviced service start Zope

Change directory to tmp, make sure you put the zenpack in /tmp

cd /tmp

Install the Zenpack

serviced service run zope zenpack-manager install *.egg

Example:

serviced service run zope zenpack-manager install ZenPacks.zenoss.MikroTik-1.1.1.egg

Restart the Zenoss Service

serviced service restart zenoss.core

Log into Zenoss and check it.