UNMS is starting… Repairing UNMS

UNMS is starting

Checking log files.

Log files are located in /home/unms/data/logs

larry@localhost:~$ sudo tail -f /home/unms/data/logs/unms.b5a3713b02f71e06fb3a84a3a9a75c558.log
{"msg":"setting permissions on /home/app/unms/data/control","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}
{"msg":"Linking /home/app/unms/public/site-images -> /home/app/unms/data/images","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}
{"msg":"Linking /home/app/unms/public/firmwares -> /home/app/unms/data/firmwares","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}
{"msg":"Stepping down from root: su-exec \"/usr/local/bin/docker-entrypoint.sh\" \"index.js\"","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}
{"msg":"Running docker-entrypoint index.js","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}
{"msg":"Version: 1.1.6+e6ec4747d.2020-03-13T10:52:14+01:00","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}
{"msg":"Waiting for database containers","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}
{"msg":"psql: fe_sendauth: no password supplied","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}
{"msg":"nc: bad address 'unms-redis'","name":"unknown","hostname":"unknown","pid":0,"v":0,"level":30,"tag":"unms","time":"2020-04-17T16:28:35+00:00"}

Stop UNMS

sudo ~unms/app/unms-cli stop

Start UNMS

sudo ~unms/app/unms-cli start

Fix Redis AOF

sudo ~unms/app/unms-cli fix-redis-aof

Running the above command resolved the problem and let UNMS start working.

larry@localhost:~$ sudo ~unms/app/unms-cli stop
larry@localhost:~$ sudo ~unms/app/unms-cli fix-redis-aof
Creating network "unms_internal" with the default driver
Creating network "unms_public" with the default driver
Creating unms-fluentd
The AOF appears to start with an RDB preamble.
Checking the RDB preamble to start:
[offset 0] Checking RDB file --fix
[offset 26] AUX FIELD redis-ver = '5.0.5'
[offset 40] AUX FIELD redis-bits = '64'
[offset 52] AUX FIELD ctime = '1587130996'
[offset 67] AUX FIELD used-mem = '276217328'
[offset 83] AUX FIELD aof-preamble = '1'
[offset 85] Selecting DB ID 0
[offset 80304063] Checksum OK
[offset 80304063] \o/ RDB looks OK! \o/
[info] 3307 keys read
[info] 0 expires
[info] 0 already expired
RDB preamble is OK, proceeding with AOF tail…
0x 79b7264: Expected prefix '*', got: '
AOF analyzed: size=127631360, ok_up_to=127627876, diff=3484
This will shrink the AOF from 127631360 bytes, with 3484 bytes, to 127627876 bytes
Continue? [y/N]: Successfully truncated AOF
larry@localhost:~$ sudo ~unms/app/unms-cli start

On the forums they seemed to recommend stopping the service and running

sudo ~unms/app/unms-cli rewrite-redis-aof

I was getting the following error when trying to run this command. Running the above command worked though.

larry@localhost:~$ sudo ~unms/app/unms-cli stop
larry@localhost:~$ sudo redis-check-aof --fix ~unms/data/redis/appendonly.aof
The AOF appears to start with an RDB preamble.
Checking the RDB preamble to start:
[offset 0] Checking RDB file --fix
--- RDB ERROR DETECTED ---
[offset 9] Can't handle RDB format version 9
[additional info] While doing: start
[additional info] Reading type 0 (string)
[info] 0 keys read
[info] 0 expires
[info] 0 already expired
RDB preamble of AOF file is not sane, aborting.
larry@localhost:~$ 

https://community.ui.com/questions/ERROR-UNMS-is-starting-This-should-not-take-more-than-a-minute-/1010b107-dfd4-4c0f-b667-2f3cf4ef713b
https://community.ui.com/questions/UNMS-is-broken-after-upgrade-from-13-x-to-14-x/411ec85e-7a1b-4d17-bb12-34e449b4ae5a

MongoDB – dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)

Ran into an issue when trying to figure out some problems with UniFi and UniFi-Video. Upgrading both to the latest version was causing problems, because they both needed different versions of MongoDB.

Was getting the following error when trying to install MongoDB. Think I was trying to install Mongo following the instructions on their site and then it caused issues with apt.

dpkg: error processing archive /var/cache/apt/archives/mongodb-org-server_4.2.5_amd64.deb (--unpack):
trying to overwrite '/usr/bin/mongod', which is also in package mongodb-server-core 1:3.6.3-0ubuntu1.1
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)

Removed the following file

rm /etc/apt/sources.list.d/mongodb-org-4.2.list

and was able to use apt again to install MongoDB. Didn’t end up solving my problem because I technically needed two versions, but at least apt was being nice again. Ended up installing the UniFi controller docker container and can run both on the same server that way.

Installing UniFi controller in Docker Container

Instructions followed from here. Some changes were made
Commands work on Ubuntu 16.04

Install Docker

sudo apt install -y docker docker.io
systemctl enable docker
systemctl start docker

Install UniFi

sudo docker pull jacobalberty/unifi:latest

Setup Docker Image

The following command sets up a container which we can later manipulate to start and stop the “service”
You can specify where you want the UniFi files to reside if desired.

sudo docker run -d --init --restart=unless-stopped --name=unifi-controller --net=host --volume=/docker/unifi:/var/lib/unifi -p 8080:8080/tcp -p 8081:8081/tcp -p 8443:8443/tcp -p 8843:8843/tcp -p 8880:8880/tcp -p 8883:8883/tcp -p 3478:3478/udp jacobalberty/unifi:latest

Docker commands

List docker containers

sudo docker container list
367c7a1465ec        jacobalberty/unifi:latest   "/usr/loca/bin/dock…" 15 minutes ago      Up 14 minutes (healthy) unifi-controller

List docker images on system

sudo docker images
jacobalberty/unifi latest baebbe301633 9 days ago 711MB

Stop container. Also stops the UniFi service. Change the ID to your container ID.

sudo docker stop 367c7a1465ec

Other notes

When setting up the Docker image, the directory specified was “/docker/unifi” so all the UniFi files are in there and it looks like if you manipulate the files, it makes the changes fine. At least for setting up the SSL certificates.

Change Static IP on AirFiber over SSH

Log into device

SSH into AirFiber

ssh ubnt@192.168.1.20

Modify config file

Open up the “/tmp/system.cfg” configuration file

vi /tmp/system.cfg

Find the line that has the ip address and modify the address.
Note that you’ll need to hit i to enter text if you are using vi

...
netconf.3.hwaddr.status=disabled
netconf.3.ip=192.168.1.10   <-- Change IP here
netconf.3.mtu=1500 
...

Save file by hitting the esc key then type :x and then hit Enter

Save changes and reboot

Now run the following command to write the changes to the device. Once it comes back up it should have the new IP address from the config file.

cfgmtd -f /tmp/system.cfg -w && reboot

How To Fix no matching cipher found. Their offer: aes128-cbc,3des-cbc…

When trying to SSH to older devices like a Ubiquiti Bullet2, you may receive an error saying

no matching cipher found. Their offer: aes128-cbc,3des-cbc,aes256-cbc,twofish256-cbc,twofish-cbc,twofish128-cbc,blowfish-cbc

The work around is to manually specify the cipher with the “-c” option. You will also probably need to specify the KexAlgorithm “Key Exchange Algorithm”

ssh -c aes128-cbc -oKexAlgorithms=+diffie-hellman-group1-sha1 admin@192.168.1.20

You can see what ciphers SSH supports by running “ssh -Q cipher”

Example output

ssh -Q cipher
3des-cbc
aes128-cbc
aes192-cbc
aes256-cbc
rijndael-cbc@lysator.liu.se
aes128-ctr
aes192-ctr
aes256-ctr
aes128-gcm@openssh.com
aes256-gcm@openssh.com
chacha20-poly1305@openssh.com

More information
https://www.openssh.com/legacy.html

UNMS – Set user as Super Admin from command line

The following commands can be run from a ssh session to your UNMS instance.

Show Current Users

sudo ~unms/app/unms-cli set-superadmin

Should get something like the following

            UNMS Users
 ┌─────────┬───────────┬───────────────────────────┬──────────────┬────────────┐
 │ (index) │ username  │           email           │     role     │ two-factor │
 ├─────────┼───────────┼───────────────────────────┼──────────────┼────────────┤
 │    0    │  'larry'  │  'larry@incredigeek.com'  │   'admin'    │   false    │
 │    1    │   'bob'   │   'bob@incredigeek.com'   │   'admin'    │   false    │
 └─────────┴───────────┴───────────────────────────┴──────────────┴────────────┘

Change User to Super Admin

To change a user to a Super Admin, add “–username username” to the above command.

Example:

sudo ~unms/app/unms-cli set-superadmin --username bob
            UNMS Users
 ┌─────────┬───────────┬───────────────────────────┬──────────────┬────────────┐
 │ (index) │ username  │           email           │     role     │ two-factor │
 ├─────────┼───────────┼───────────────────────────┼──────────────┼────────────┤
 │    0    │  'larry'  │  'larry@incredigeek.com'  │   'admin'    │   false    │
 │    1    │   'bob'   │   'bob@incredigeek.com'   │ 'superadmin' │   false    │
 └─────────┴───────────┴───────────────────────────┴──────────────┴────────────┘

Note that there does not appear to be a way to

UniFi Cameras – configure static IP or DHCP from command line

The UniFi cameras are different then the AirMax line in how the configuration works. There is a “/tmp/system.cfg” file, but it does not contain most of the configuration option. In the default directory there are some files which appear to handle the device configuration.

The network configuration file is “ubnt_networkd.conf”

Edit the file by opening it up in vi or a different text editor.

Static IP config

{
     "cfgver": 2,
     "dhcp": {
         "fallbackIP": "192.168.1.20",
         "fallbackNetmask": "255.255.255.0",
         "status": 0  <-- 0=Static 1=DHCP
     },
     "dns": {
         "ns1": {
             "ip": "8.8.8.8",
             "status": 1
         },
         "ns2": {
             "ip": "4.2.2.2",
             "status": 1
         }
     },
     "routes": {
         "gateway": "192.168.5.1"
     },
     "statik": {
         "ip": "192.168.5.11",
         "netmask": "255.255.255.0"
     }
}

If you want to set the static address, you should be able to edit the settings in this config and leave DHCP status disabled.

DHCP config

To set the Camera to use DHCP, under dhcp change “”status”: 0″ to “”status”: 1″

{
     "cfgver": 2,
     "dhcp": {
         "fallbackIP": "192.168.1.20",
         "fallbackNetmask": "255.255.255.0",
         "status": 1
     },
     "dns": {
         "ns1": {
             "ip": "8.8.8.8",
             "status": 1
         },
         "ns2": {
             "ip": "4.2.2.2",
             "status": 1
         }
     },
     "routes": {
         "gateway": "192.168.5.1"
     },
     "statik": {
         "ip": "192.168.5.11",
         "netmask": "255.255.255.0"
     }
}

After the settings are applied the camera will use use DHCP regardless of the statik ip settings.

Save changes

Use the following command to write the changes to persistent storage

ubnt_cmd_persist.sh

And reboot the camera.

reboot

Camera should pull a DHCP address when it boots up

Ubiquiti set DHCP from command line

Edit the “/tmp/system.cfg”

Change the line that contains “dhcpc.status=disabled” to

dhcpc.status=enabled

add the following lines

dhcpc.1.devname=br0         
dhcpc.1.fallback=192.168.1.20
dhcpc.1.fallback_netmask=255.255.255.0
dhcpc.1.status=enabled

Save and exit and save changes

/usr/etc/rc.d/rc.softrestart save

Log into the GUI and verify everything looks correct. Under the network tab it was still showing that it had a static address, although it pulled a DHCP IP.

Change Channel Width on Ubiquiti AC CPE Over SSH

SSH into CPE

ssh ubnt@192.168.1.20

open up the “/tmp/system.cfg” file

vi /tmp/system.cfg

Find the “radio.1.chanbw=” line and change to the following

radio.1.chanbw=10

Note: it may say “radio.1.chanbw=0” that looks to be the default and works for 20mhz

exit file with

esc + :wq + Return or Enter

Save settings with

/usr/etc/rc.d/rc.softrestart save 

Change AP to 10Mhz and the CPE should reconnect.

Ubiquiti – Renewing DHCP client lease from command line

AirOS uses udhcpc for the DHCP client on Ubiquiti Radios. To renew the DHCP address you can kill the udhcpc process and it’ll automatically restart and get a new address.

Renew DHCP lease

Kill udhcpc with the following command.

killall udhcpc

Other info

Print info about the DHCP lease. May need to change “info.br1” to “info.eth0” or some other interface.

cat /etc/udhcpc/info.eth0 

Example output.

XW.v6.2.0# cat /etc/udhcpc/info.br1 
u_interface="br1"
u_broadcast=""
u_subnet="255.255.255.0"
u_ip="10.93.0.10"
u_router="10.93.0.1"
u_dns="8.8.8.8 1.1.1.1"
u_hostname=""
u_serverid="10.93.0.1"
u_domain=""
u_leasetime=600
u_timestamp="1143249941"
u_started=1142593
u_pid=936
XW.v6.2.0# 

Command arguments that udhcp is run with. Info was collected by running the “ps | grep udhcp” command. Note that the interface “eth0” can be different if the device is in bridge mode.

/sbin/udhcpc -f -i eth0 -s /etc/udhcpc/udhcpc -p /var/run/udhcpc.eth0.pid -h device_name

udhcpc help output

XW.v6.2.0# udhcpc --help
BusyBox v1.24.2 (2019-07-03 11:13:35 EEST) multi-call binary.

Usage: udhcpc [-fbqvRB] [-t N] [-T SEC] [-A SEC/-n]
        [-i IFACE] [-s PROG] [-p PIDFILE]
        [-oC] [-r IP] [-V VENDOR] [-F NAME] [-x OPT:VAL]... [-O OPT]...

        -i,--interface IFACE    Interface to use (default eth0)
        -s,--script PROG        Run PROG at DHCP events (default /usr/share/udhcpc/default.script)
        -p,--pidfile FILE       Create pidfile
        -B,--broadcast          Request broadcast replies
        -t,--retries N          Send up to N discover packets (default 3)
        -T,--timeout SEC        Pause between packets (default 3)
        -A,--tryagain SEC       Wait if lease is not obtained (default 20)
        -n,--now                Exit if lease is not obtained
        -q,--quit               Exit after obtaining lease
        -R,--release            Release IP on exit
        -f,--foreground         Run in foreground
        -b,--background         Background if lease is not obtained
        -S,--syslog             Log to syslog too
        -r,--request IP         Request this IP address
        -o,--no-default-options Don't request any options (unless -O is given)
        -O,--request-option OPT Request option OPT from server (cumulative)
        -x OPT:VAL              Include option OPT in sent packets (cumulative)
                                Examples of string, numeric, and hex byte opts:
                                -x hostname:bbox - option 12
                                -x lease:3600 - option 51 (lease time)
                                -x 0x3d:0100BEEFC0FFEE - option 61 (client id)
        -F,--fqdn NAME          Ask server to update DNS mapping for NAME
        -V,--vendorclass VENDOR Vendor identifier (default 'udhcp VERSION')
        -C,--clientid-none      Don't send MAC as client identifier
        -v                      Verbose
Signals:
        USR1    Renew lease
        USR2    Release lease
XW.v6.2.0# 

Note that you can also send a signal to the PID of udhcpc and have it renew the address. To do that use the following command, replacing pidofudhcpc to pid of udhcp.

kill -s USR1 pidofudhcpc

or use the following command to find the pid for you.

kill -s SIGUSR1 $(pgrep udhcpc)