LineageOS Default Network Connections

This was a fairly simple test to see what network connections a fresh LineageOS install on a Google Pixel 5 makes. During the initial setup, GPS was disabled. After we set it up and got a base line, we turned GPS on to see what DNS requests it made.

Testing methodology.

  • A Computer was used as a Hotspot running both Network Miner 2.8 and Wireshark to log all network request
  • Pixel 5 was installed with the latest version of LineageOS 20 (August 2023)
  • Setup was completed without connecting to WiFi or a cellular network
  • There was no SIM card in while installing, setting up, or testing
  • After setup was complete, WiFi was connected to the computer running the Hotspot
  • After a base line was logged, we turned on GPS
  • GApps were not installed.

Fresh Install Network Requests

After setup was complete, we connected the Pixel 5 to the PC running NetworkMiner and Wireshark. It immediately made a handful of requests to the following 5 domain names

  1. www.google.com
  2. connectivitycheck.gstatic.com
  3. time.android.com
  4. g.co
  5. firebaseinstallations.googleapis.com

connectivitycheck.gstatic.com is used to detect if the current network has internet and also to detect if there is a captive portal that you need to log into.

time.android.com would be to check the time and make sure it is correct.

Not sure what the extra 3 are used for. It is possible that firebaseinstallations.googleapis.com is used for the Android System Intelligence, or some other app that comes by default on LineageOS.

The following NetworkMiner screenshot shows all the IP addresses that were returned for the DNS queries. Note that a DNS query can return multiple IP addresses for a domain name, and then the device only use one of those IP addresses to transmit traffic.

A couple of normal network broadcast, multicast, and gateway addresses are blurred out as they are normal for devices on a local network.

Here is a Wireshark screenshot for all the DNS requests.

Total bandwidth sent and received for each IP

Using Wireshark, we were able to get a total amount of data sent and received for each of the domains.

  1. www.google.com – 12.976 KiB
  2. connectivitycheck.gstatic.com – 1.497 KiB
  3. time.android.com 270 bytes
  4. g.co – 21.883 KiB,
  5. firebaseinstallations.googleapis.com – 16.225 KiB
  6. Total for Pixel 5 – 52.851 KiB

Turning on GPS

Turning on GPS immediately led to a connection to xtrapath5.xboxprod.izatcloud.net.

The four lines are just different IP’s for the same domain.

From the Location settings, we can toggle on or off the “Use assisted GPS”.

The settings say the following about Assisted GPS “Download satellite assistance data from the internet which can greatly improve the GPS startup performance. For emergency calls, assisted GPS is always allowed”

Essentially, it will download some files that help your phone find satellites faster which will get you a faster GPS lock. Without it, it can take awhile to find your position.

During the initial setup (First screenshot below), you can toggle on/off Assisted GPS. By default, Android System Intelligence and the Browser are allowed to use Location.

Hopefully that is a helpful overview of the default LineageOS network connections and what some of them are used for.

Hardening Mikrotik RouterOS

https://wiki.mikrotik.com/wiki/Manual:Securing_Your_Router

Things to harden

  • Delete default admin user
  • Disable unused services and whitelist IP’s
  • Secure SSH
  • DNS

Delete default admin user

Before deleting the default admin user, create your own user account.

/user/add name=MyUsername group=full password=mylongsecurepassword

Note: running /user/add will prompt you for the rest of the options.

Delete the default admin user with

/user remove admin

We want to delete the default admin user for two reasons. 1. There is no default password for this user. 2. It is a default username which means it will be targeted for brute force attacks.

Consider using the /users/groups for more granular control.

Disable unused services

In the following, we disabled all services except SSH and Winbox. We also limit access to those services only from private “RFC 1918” IP addresses. Customize as needed.

/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set www-ssl tls-version=only-1.2
set ssh address="set winbox address="192.168.0.0/16,172.16.0.0/12,10.0.0.0/8"
set api disabled=yes
set winbox address="set winbox address="192.168.0.0/16,172.16.0.0/12,10.0.0.0/8"
set api-ssl disabled=yes tls-version=only-1.2

for www-ssl and api-ssl, tls-version is not a required argument, but you may consider using it if you need the API or Webfig.

Secure SSH

/ip/ssh/set strong-crypto=yes allow-none-crypto=no always-allow-password-login=no host-key-size=4096

And regenerate the SSH host key. It will prompt for a [y/N], hit y to regenerate.

/ip/ssh/regenerate-host-key 

DNS

Unless your device is being used as a DNS resolver, it is best to disable the “Allow Remote Request”

ip dns/set allow-remote-requests=no

If you do need it enabled, then be sure to add some firewall rules to keep your router from being used in amplification attacks.

add action=drop chain=input dst-port=53 in-interface-list=WAN protocol=udp

You can configure interface lists in /interface/list or Interface -> Interface List in the gui

Or you can change to in-interface and specify the WAN interface directly. You could also set it to !LAN if you have a LAN interface list set up.

Named Error “option ‘dnssec-enable’ no longer exists”

After a recent update, the named service stopped working. When manually trying to restart the service it would just fail.

Running the named-checkconf command

sudo /usr/bin/named-checkconf -z /etc/named.conf

Returned the following error.

/etc/named.conf:37: option 'dnssec-enable' no longer exists

Looks like what happened is they removed the dnssec-option, but enabled it by default

The dnssec-enable option has been obsoleted and no longer has any effect. DNSSEC responses are always enabled if signatures and other DNSSEC data are present. [GL #866]

https://bind9.readthedocs.io/en/v9_16/notes.html#id118

So the option is invalid now and not needed.

To fix the issue, simply open up the /etc/named.conf file and delete the line that has ‘dnssec-enable’

Enable Syslog for PowerDNS Recursor

  1. Enable Logging in PowerDNS Recursor Config
  2. Edit Systemd Unit File for PowerDNS to Allow Syslog
  3. Enable Logging in rsyslog Config File

The following links were helpful in setting things up.

https://doc.powerdns.com/recursor/running.html
https://www.reddit.com/r/linuxadmin/comments/9lc4jl/logging_queries_in_pdnsrecursor/

Enable logging in PowerDNS Recursor Config

First we need to find the line that says “disable-syslog” and uncomment/change it to

disable-syslog=no

Next find the line that says “quiet” and uncomment/change it to

quiet=no

Some other lines you may want to check and change

logging-facality=1
loglevel=6

Edit Systemd Unit File for PowerDNS to allow Syslog

Next we need to modify the Systemd unit file to allow PowerDNS Recursor to log to syslog.

systemctl edit --full pdns-recursor.service

On the ExecStart Line, remove the part that says

--disable-syslog

The resulting line should look something like

[Service]
ExecStart=/usr/sbin/pdns_recursor --socket-dir=%t/pdns-recursor --socket-dir=%t/pdns-recursor --daemon=no --write-pid=no --log-timestamp=no

Save the file.

Enable Logging in rsyslog Config File

Edit the rsyslog file

sudo vim /etc/rsyslog.conf

Add the following line

local1.*        /var/log/pdns_recursor.log

This should now log all of the PowerDNS Recursor log info to “/var/log/pdns_recursor.log”

Restart the rsyslog and PowerDNS Recursor service

sudo systemctl restart rsyslog
sudo systemctl restart pdns-recursor

You should now see DNS request in the log file.

tail /var/log/pdns_recursor.log

They should also show up in the “/var/log/messages”

Troubleshooting DNS CPU Usage on Mikrotik Router

Problem : Lots of CPU utilization. Profile shows a good bit of it is DNS related.

DNS eating CPU on Router

The router is setup to allow DNS to pass through to web servers so rDNS and other records can be looked up and resolved. This is a specific IP block that gets it’s addresses from the router. The firewall rules explicitly allow this address range. We’ll say 192.168.88.0/24, and blocks everything else. This works for the web servers. But why are we still getting a bunch of CPU utilization with DNS?

As it turns out, the firewall rule that allows the server address range also includes routers own address! So we have unintentionally whitelisted DNS access to our router.

To resolve the issue we can add another firewall rule that explicitly blocks DNS traffic to the routers IP address. We are using two rules, one to block TCP and the other UDP.

ip firewall filter add chain=input dst-address=192.168.88.1 protocol=6 dst-port=53 in-interface-list=WAN action=drop
ip firewall filter add chain=input dst-address=192.168.88.1 protocol=17 dst-port=53 in-interface-list=WAN action=drop

Rules 6 & 7 are the two new rules we just applied. 14 & 15 block input to the router, however rules 8 & 9 inadvertently allowed access to the router’s public IP.

Firewall Rules for Router

The Result? Our CPU usage dropped!

CPU Usage dropped after adding DNS firewall rules.

Quite dramatically too as the following LibreNMS screenshot shows.

LibreNMS CPU graph showing the overall CPU utilization improvement

For more information about DNS Amplification attacks, refer to the following links.

https://ask.wireshark.org/question/6865/dns-amplification-attack/
https://security.stackexchange.com/questions/237127/why-would-hackers-attack-a-dns-server-with-a-dos

Named Debugging Levels

In the following command, x should be the debug level number

named -g -d x

Example,

named -g -d 3

Following info taken from here.
https://docstore.mik.ua/orelly/networking/dnsbind/ch12_01.htm

12.1.1 What Information Is at Each Level?

Here is a list of the information that each debugging level will give. The debugging information is cumulative; for example, level 2 includes all level 1’s debugging information. The data are divided into the following basic areas: starting up, updating the database, processing queries, and maintaining zones. We won’t cover updating the name server’s internal database – problems always occur elsewhere. However, what the name server adds or deletes from its internal database can be a problem, as you’ll see in Chapter 13, Troubleshooting DNS and BIND .

Level 1

The information at this level is necessarily brief. Name servers can process lots of queries, which can create lots of debugging output. Since the output is condensed, you can collect data over long periods. Use this debugging level for basic startup information and for watching query transactions. You’ll see some errors logged at this level, including syntax errors and DNS packet formatting errors. This level will also show referrals.

Level 2

Level 2 provides lots of useful stuff: it lists the IP addresses of remote name servers that are used during a lookup, along with their round trip time values; it calls out bad responses; and it tags a response as to which type of query it is answering, a SYSTEM (sysquery) or a USER query. When you are tracking down a problem with a secondary server loading a zone, this level shows you the zone values – serial number, refresh time, retry time, expire time, and time left – as the secondary checks if it is up-to-date with its master.

Level 3

Level 3 debugging becomes much more verbose because it generates lots of messages about updating the name server database. Make sure you have enough disk space if you are going to collect debugging output at level 3 or above. At level 3, you’ll also see: duplicate queries called out, system queries generated (sysquery), the names of the remote name servers used during a lookup, and the number of addresses found for each server.

Level 4

Use level 4 debugging when you want to see the query and response packets received by the name server. This level also shows the credibility level for cached data.

Level 5

There are a variety of messages at level 5, but none of them are particularly useful for general debugging. This level includes some error messages, for example, when a malloc() fails, and a message when the name server gives up on a query.

Level 6

Level 6 shows you the response sent to the original query.

Level 7

Level 7 shows you a few configuration and parsing messages.

Level 8

There is no significant debugging information at this level.

Level 9

There is no significant debugging information at this level.

Level 10

Use level 10 debugging when you want to see the query and response packets sent by the name server. The format of these packets is the same format used in level 4. You wouldn’t use this level very often, since you can see the name server response packet with nslookup .

Level 11

There are only a couple of debugging messages at this level, and they are in seldom-traversed code.

UBNTMOD check range of IP addresses and see if they resolve DNS

The following uses the ubntmod.sh script to check a device then report if it is resolving DNS or not. ip=”192.168.1.” specifies the first part of the ip, the “for ((i=1; i<=254;i++))” tells it to go from 192.168.1.1 – 192.168.1.254, change the beginning and ending number if you want to change the ip range.

ip="192.168.1." && for ((i=10; i<=30;i++)); do if ( fping ${ip}$i -r1 | grep alive); then ./ubntmod.sh -i ${ip}${i} -e ; else echo ${ip}$i not alive; fi ; done 

Broken out for easier reading.

ip="192.168.1." 
for ((i=10; i<=30;i++))
do
if ( fping ${ip}$i -r1 | grep alive); then
./ubntmod.sh -i ${ip}${i} -e
else echo ${ip}$i not alive
fi
done

If the script is able to log into the device and resolve DNS you should get

192.168.1.1 Resolves DNS

Check if Mikrotik is an Open DNS Resolver

https://www.openresolver.com

You can test if a router is acting as an open DNS resolver by running the following command from a Linux terminal. If you need to install dig, refer to here for Debian/Ubuntu and here for RPM/CentOS/Fedora Distros.

Replace 192.168.88.1 with the host you want to test against.

dig +short test.openresolver.com TXT @192.168.88.1

If you receive the following

"open-resolver-detected"

The router is acting as an open resolver.

If you get

;; connection timed out; no servers could be reached

Then you are unable to use that router to resolve DNS.

Example running the command against a Mikrotik router with Remote DNS turned on Then adding a firewall rule to block unwanted request.

bob@localhost:~$ dig +short test.openresolver.com TXT @192.168.88.1
"open-resolver-detected"
bob@localhost:~$ 
<<-- Put firewall rule on router -->>
bob@localhost:~$ dig +short test.openresolver.com TXT @192.168.88.1
;; connection timed out; no servers could be reached  
bob@localhost:~$ 

Extra notes

If you have firewall rules allowing your IP address to use the router for DNS, then the above command to test will show it as an Open Resolver. Ideally you would want a connection from the outside to test. Or you can use this link and test it from the website. https://www.openresolver.com

cPanel unable to access locally hosted domains

Check and verify that DNS is not being blocked upstream by a firewall. Behavior is weird, the server can get out to the internet, access to the servers IP address is available, but can’t ping domains that are locally hosted. Are also unable to ping the domains from the internet in.

To resolve the issue either disable the DNS firewall rules, or better yet add some rules to allow access to the cPanel server.

Install dig on Ubuntu, Debian or Kali Linux

install dig
Help options for dig


Dig is a DNS lookup utility.  It is included in most Linux distributions by default, but if it isn’t you can easily install dig with the following command.

The dig utility is apart of the dnsutils package

sudo apt-get install dnsutils -y

After it is installed, we can verify that it is working with

dig -v

For more information on how to use dig, refer to the following link.

https://www.howtogeek.com/663056/how-to-use-the-dig-command-on-linux/

The following is copied and pasted from the dig man page.

NAME
       dig - DNS lookup utility

SYNOPSIS
       dig [@server] [-b address] [-c class] [-f filename] [-k filename] [-m] [-p port#] [-q name]
           [-t type] [-v] [-x addr] [-y [hmac:]name:key] [[-4] | [-6]] [name] [type] [class]
           [queryopt...]

       dig [-h]

       dig [global-queryopt...] [query...]

DESCRIPTION
       dig is a flexible tool for interrogating DNS name servers. It performs DNS lookups and
       displays the answers that are returned from the name server(s) that were queried. Most DNS
       administrators use dig to troubleshoot DNS problems because of its flexibility, ease of use
       and clarity of output. Other lookup tools tend to have less functionality than dig.

       Although dig is normally used with command-line arguments, it also has a batch mode of
       operation for reading lookup requests from a file. A brief summary of its command-line
       arguments and options is printed when the -h option is given. Unlike earlier versions, the
       BIND 9 implementation of dig allows multiple lookups to be issued from the command line.

       Unless it is told to query a specific name server, dig will try each of the servers listed
       in /etc/resolv.conf. If no usable server addresses are found, dig will send the query to the
       local host.

       When no command line arguments or options are given, dig will perform an NS query for "."
       (the root).

       It is possible to set per-user defaults for dig via ${HOME}/.digrc. This file is read and
       any options in it are applied before the command line arguments. The -r option disables this
       feature, for scripts that need predictable behaviour.

       The IN and CH class names overlap with the IN and CH top level domain names. Either use the
       -t and -c options to specify the type and class, use the -q the specify the domain name, or
       use "IN." and "CH." when looking up these top level domains.

SIMPLE USAGE
       A typical invocation of dig looks like:

            dig @server name type

       where:

       server
           is the name or IP address of the name server to query. This can be an IPv4 address in
           dotted-decimal notation or an IPv6 address in colon-delimited notation. When the
           supplied server argument is a hostname, dig resolves that name before querying that name
           server.

           If no server argument is provided, dig consults /etc/resolv.conf; if an address is found
           there, it queries the name server at that address. If either of the -4 or -6 options are
           in use, then only addresses for the corresponding transport will be tried. If no usable
           addresses are found, dig will send the query to the local host. The reply from the name
           server that responds is displayed.

       name
           is the name of the resource record that is to be looked up.

       type
           indicates what type of query is required — ANY, A, MX, SIG, etc.  type can be any valid
           query type. If no type argument is supplied, dig will perform a lookup for an A record.