« Raspberry Pi | Main | Tsunami »

Saturday, August 17, 2019

About to update to RedHat Enterprise Linux or CentOS 7.7? Beware!


There's a couple of caveats before upgrading to RHEL / CentOS 7.7.

1: Upgrading to latest RHEL 7 PCP package pcp-4.3.2-2.el7 gives several issues.

During installation/upgrade of pcp-selinux:

    Updating / installing...
    1:pcp-selinux-4.3.2-2.el7 ################################# [100%]
Failed to resolve allow statement at /etc/selinux/targeted/tmp/modules/400/pcpupstream/cil:83
semodule: Failed!

Followed by lots of selinux errors logged.

Workaround is to install 7.7's selinux-policy first

    yum update selinux-policy

or, if you've already updated,

    yum reinstall pcp-selinux

Documented in RedHat Bugzilla entry 1714101 

2: bind is updated from version 9.9 to 9.11. This breaks configurations in which the same zone file is defined in several views.

The solution is to use bind 9.11's "in-view xxx" in subsequent definitions of the same zone in different views.

The fix is described in detail here.



Posted by Phil at 7:32 PM
Edited on: Saturday, September 14, 2019 3:47 PM
Categories: IT, Software

Saturday, April 27, 2019

A Raspberry Pi Stratum 1 NTP Server


Overview

The following instructions are how to make a cheap Pulse Per Second (PPS) disciplined Stratum 1 NTP Time server using one of the Raspberry Pi U-blox M8Q based GPS boards sold by Uputronics.

Our basic requirement is for an NTP server which will work standalone without connectivity to other NTP servers, so if a datacentre loses connectivity its servers will still have a stable time source.

As we’re only interested in time and not height, motion, or location, all we’re interested in is the NMEA xxZDA and xxRMC sentences and the highly accurate PPS signal from the GPS module, both of which we feed straight into the ntp daemon. The xxZDA sentences (which give us full 4-digit years) are not output by default by the M8Q so we have to enable them at boot time.

Many guides on the net use gpsd to feed ntp via shared memory. That’s an additional overhead and complexity, especially if we want to enable Galileo reception (gpsd initialises the U-blox the way it sees fit). I prefer to let the NTP daemon read the NMEA stream from the GPS receiver itself, avoiding the middle man.

In this guide we’ve chosen to use NMEA output from the GPS module via the ntp Generic GPS Receiver driver 20 ( https://www.eecis.udel.edu/~mills/ntp/html/drivers/driver20.html ) and the PPS signal via driver 22 ( https://www.eecis.udel.edu/~mills/ntp/html/drivers/driver22.html ).

You will need a Raspberry Pi 4B, the Uputronics Raspberry Pi+ GPS Expansion Board and a suitable GPS antenna.

This guide assumes that you’re using Raspbian Buster Lite or later. Download and write this to an SD card.

See https://www.raspberrypi.org/documentation/installation/installing-images/windows.md

Create a file named ssh in the boot folder after burning the MicroSD card.

Attach the Uputronics Raspberry Pi+ GPS Expansion Board to the Pi, insert the SD card, connect the antenna and network cable and boot the Pi up. Either connect locally or via SSH to the Pi. If you can’t SSH in and don’t have a monitor see this

https://www.raspberrypi.org/documentation/configuration/wireless/headless.md

Follow the instructions carefully if you miss steps things won’t work.

The Uputronics board has u-blox firmware 3.01 on it, dated 2016.

The week number rollover is set to 1867 (October 2015). All transmitted week numbers are mapped to the ~19.5 year period between week 1867 and week 2990 (April 2035).

A Note About Accuracy

In theory, GPS-based time receivers can give a very high accuracy, with the PPS (Pulse per Second) signal being accurate to within 10ns.

However, fix data from gpsctl shows:


    
pi@ntp2:~ $ /usr/local/bin/gpsctl -Q fix
Time (UTC): 2019-04-18 15:31:27 (yyyy-mm-dd hh:mm:ss)
Latitude: 52.05994980 N
Longitude: 2.72698960 W
Altitude: 198.789 feet
Motion: 0.338 mph at 53.114 degrees heading
Satellites: 5 used for computing this fix
Accuracy: time (39 ns), height (+/-18.199 feet), position (+/-103.911 feet), heading(+/-8.230 degrees), speed(+/-0.132 mph)

Note the 39ns time accuracy from that fix. The more satellites, the better.

Errors are also caused by signal delay (4ns per foot) in the cable from the aerial to the GPS receiver, etc.

So we’d be lucky to get 100ns accuracy from the PPS pulse.

Add to that the processing overheads of the PPS interrupt, and processor clock jitter in the Raspberry Pi, and overheads transferring and decoding the NMEA sentences. The jitter on the PPS signal is less than 5 microseconds.

The output from the ntp NMEA driver without PPS correction can have a jitter of a few milliseconds, and an offset from real time which has to be tweaked manually to get within 5ms of the correct time.

NTP sources on the internet can show offsets / jitter of over 5ms, local LAN 50us or more.

On the Pi 4B we can expect an average jitter of less than 1 microsecond.

Required Components

I sourced my components from the Pi Hut unless noted elsewhere:

1 Raspberry Pi Model 4B 1GB RAM

1 8GB or larger micro SD card

(a Transcend High Endurance 32GB micro SD card would be better choice than a generic one for longevity)

1 Uputronics GPS Hat

1 Pi Hut Pi 4 GPS case

1 Raspberry Pi 4 power supply

1 GPS SMA Antenna

(optional) 1 SMA-male to TNC-female (or BNC, as needed) adaptor to connect to existing GPS aerial (from Amazon or eBay)

Prerequisite Settings

Login as pi / raspberry, and immediately change the password from the default

passwd

sudo raspi-config

Advanced Options

Expand filesystem

Interfacing Options

SSH -> Would you like the SSH server to be enabled – Yes (Recommended)

I2C -> Would you like the ARM I2C interface to be enabled? - Yes

Serial -> Login Shell (no) Hardware (yes) (Optional)

Quit but no need to reboot at this point.

for i in systemd-timesyncd avahi-daemon alsa-state bluetooth triggerhappy hciuart rng-tools autologin getty;

do

sudo systemctl disable $i.service; sudo systemctl stop $i.service;

done

sudo systemctl disable serial-getty@ttyAMA0.service

sudo systemctl mask serial-getty@ttyAMA0.service

sudo apt install pps-tools ntp wiringpi cpufrequtils i2c-tools

sudo apt purge bluez bluez-firmware wpasupplicant (assuming no wifi required)

sudo apt update

sudo apt full-upgrade

sudo nano /boot/config.txt and add at the bottom :

dtoverlay=disable-bt
dtoverlay=disable-wifi
dtoverlay=pps-gpio,gpiopin=18

Make sure that enable_uart=1 is set in /boot/config.txt

sudo nano /boot/cmdline.txt and add at the end

nohz=off

sudo nano /etc/modules and add at the bottom :

pps-gpio
  

Save and Quit Nano.

sudo nano /etc/udev/rules.d/80-gps-to-ntp.rules (needed for refclock 20 in ntp)

# Change MODE of ttyAMA0 ao it is readable by NTP and provide

# a symlink to /dev/gps0
KERNEL=="ttyAMA0", SYMLINK+="gps0", MODE="0666"
# Symlink /dev/pps0 to /dev/gpspps0
KERNEL=="pps0", SYMLINK+="gpspps0", MODE="0666"

and then sudo reboot


Enabling Galileo Satellites and Setting Stationary Mode

Stationary mode gives a faster GPS fix.

See Tom Dilatush’s gpsctl:

http://www.jamulblog.com/2017/11/paradise-ponders-gpsctl-functionally.html

I’ve customised it further to enable the use of a config file and put it in my GitHub repo:

https://github.com/philrandal/gpsctl

Download and build gpsctl.

cd ~
wget https://github.com/philrandal/gpsctl/archive/master.zip
unzip master.zip
mv gpsctl-master gpsctl
cd gpsctl
./build.sh
sudo cp /home/pi/gpsctl/gpsctl /usr/local/bin
sudo cp /home/pi/gpsctl/etc/gpsctl.conf /etc/gpsctl.conf

Individual settings can be tweaked in /etc/gpsctl.conf.

To set port speed to 115200 baud, enable Galileo satellites, set stationary mode, tweak antenna delays, PPS timing, etc:

/usr/local/bin/gpsctl -a -B 115200 --configure_for_timing -vv

To reset the device to its defaults

/usr/local/bin/gpsctl -a --reset -vv

To view info:

/usr/local/bin/gpsctl -a -Q satellites

/usr/local/bin/gpsctl -a -Q config

/usr/local/bin/gpsctl -a -Q fix

Note that these commands can only be run when gpsd / ntp are not using /dev/ttyAMA0

Example /etc/gpsctl.conf which configures the U-blox M8Q for this environment:

    
# # example gpsctl.conf which enables Galileo as in --galileo parameter
#
[gpsctl]
port = /dev/serial0
# sync method: ASCII = 1, NMEA = 2, UBX = 3
sync method = 3
verbosity = 0
[NMEA]
enabled = true
version = 41
GGA = off
GLL = off
GSA = off
GSV = off
RMC = on
VTG = off
GRS = off
GST = off
ZDA = on
[GPS]
enabled = yes
minimum channels=8
maximum channels=16
[SBAS]
enabled = no
minimum channels=1
maximum channels=3
[Galileo]
enabled = yes
minimum channels=4
maximum channels=8
[Beidou]enabled = no
minimum channels=8
maximum channels=16
[IMES]
enabled = no
minimum channels=0
maximum channels=8
[QZSS}
enabled = no
minimum channels=0
maximum channels=3
[GLONASS]
enabled = yes
minimum channels=8
maximum channels=14
[Navigation Engine]
# Dynamic model: Portable = 0, Stationary = 2, Pedestrian = 3, Automotive = 4,
# Sea = 5, Air1G = 6, Air2G = 7, Air4G = 8, Watch = 9
Dynamic model = 2
# Fix mode: 2D only = 1, 3D only = 2, auto 2D/3D = 3
Fix mode = 3
Fixed altitude (2D) = 0.00 meters
Fixed altitude variance (2D) = 1.0000 meters^2
Minimum elevation = 5 degrees
Position DoP mask = 10.0
Time DoP mask = 10.0
Position accuracy mask = 100 meters
Time accuracy mask = 300 meters
Static hold threshold = 0 cm/s
Dynamic GNSS timeout = 60 seconds
Threshold above C/No = 0 satellites
C/No threshold = 0 dBHz
Static hold max distance = 0 meters
# UTC Standard: AutoUTC = 0, USNO_UTC = 3, GLONASS_UTC = 6, BeiDou_UTC = 7
UTC standard = 3
[Time Pulse]
# the nanoseconds / microseconds after the numbers are just reminders,
# they don't mean anything to the config parserAntenna cable delay = 56 nanoseconds
RF group delay = 20 ns
Unlocked pulse period = 1000000 microseconds
Unlocked pulse length = 0
Locked pulse period = 1000000 microseconds
Locked pulse length = 500000 microseconds
User configurable delay = 0

To run gpsctl at system startup,

sudo cp /home/pi/gpsctl/etc/systemd/system/ublox-init.service /etc/systemd/system/ublox-init.service

or

nano /etc/systemd/system/ublox-init.service

and add the following contents

[Unit]
Description=u-blox initialisation
Before=gpsd.service
Before=ntp.service
[Service]
Type=oneshot
Environment="PARAMS=-q -a -B 115200 --configure_for_timing"
EnvironmentFile=-/etc/default/gpsctl
ExecStart=/bin/bash '/usr/local/bin/gpsctl ${PARAMS}'
[Install]
WantedBy=multi-user.target

    

This will enable Galileo satellites, set stationary mode, configure comms at 115200 baud, and restrict NMEA output to RMC and ZDA records.

To use the default baud rate of 9600, create /etc/default/gpsctl with the contents

PARAMS=-q --configure-for-timing

Then

sudo systemctl enable ublox-init.service

sudo systemctl daemon-reload


    

Verifying that PPS Is Working

Ensure the GPS has a lock and the Green PPS LED on the Uputronics Pi+ GPS Expansion Board is blinking once a second.

lsmod | grep pps

Output should be similar to :

pps_gpio                3089  1pps_
core                8606  4 pps_gpio

dmesg | grep pps

Output should be similar to :

[    2.735586] pps_core: LinuxPPS API ver. 1 registered
[ 2.738121] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 2.763842] pps pps0: new PPS source pps@12.-1
[ 2.766361] pps pps0: Registered IRQ 169 as PPS source

    

This indicates that the PPS Module is loaded.

sudo ppstest /dev/pps0

Output should be similar to:

trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
source 0 - assert 1418933982.998042450, sequence: 970 - clear  0.000000000, sequence: 0
source 0 - assert 1418933983.998045441, sequence: 971 - clear  0.000000000, sequence: 0

    

(Press CTRL+C to quit). This indicates that the PPS Module is working.

Enabling PPS/ATOM Support in NTPD

The supplied version of NTPD on the Raspberry Pi in Raspbian Stretch 2018-11-13 and later supports PPS so there is no need to roll your own NTP.

You need to pick a few local NTP servers to use. The easiest way to do this is pick your region:

https://support.ntp.org/bin/view/Servers/NTPPoolServers

Select your region then you get a list of the country servers. E.g for the UK its uk.pool.ntp.org.

Type:

dig uk.pool.ntp.org

You will get four IP’s back:

;; &lt;&lt;&gt;&gt; DiG 9.10.3-P4-Raspbian &lt;&lt;&gt;&gt; +answer uk.pool.ntp.org
;; global options: +cmd
;; Got answer:
;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 51647
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
;; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;uk.pool.ntp.org. IN A
;; ANSWER SECTION:
uk.pool.ntp.org. 24 IN A 193.150.34.2
uk.pool.ntp.org. 24 IN A 176.58.109.199
uk.pool.ntp.org. 24 IN A 195.195.221.100
uk.pool.ntp.org. 24 IN A 185.53.93.157


sudo nano /etc/ntp.conf

Add

# PPS Driver
server 127.127.22.0 minpoll 4 maxpoll 4
fudge 127.127.22.0 time1 +0.000000 flag3 0 refid PPS
#flag3 Controls the kernel PPS discipline: 0 for disable (default), 1 for enable.
#time1 PPS time offset
# use local clock at stratum 10 if no others available
tos orphan 10
tos mindist 0.002
# NMEA driver (/dev/gps0 and /dev/gpspps0)
# 115200 baud, RMC and ZDA messages
server 127.127.20.0 mode 89 minpoll 4 maxpoll 4 iburst prefer
fudge 127.127.20.0 flag1 0 flag2 0 flag3 0 time2 0.050 refid GPS stratum 2

# 9600 baud, RMC and ZDA
server 127.127.20.0 mode 89 minpoll 4 maxpoll 4 iburst prefer
fudge 127.127.20.0 flag1 0 flag2 0 flag3 0 time2 0.043 refid GPS stratum 2
#flag1 Disable PPS signal processing if 0 (default); enable PPS signal processing if 1.
#flag2 If PPS signal processing is enabled, capture the pulse on the rising edge if 0 (default); capture on the falling edge if 1.
#flag3 If PPS signal processing is enabled, use the ntpd clock discipline if 0 (default); use the kernel discipline if 1.
#time1 PPS time offset
#time2 NMEA time offset
#mode bit 0 - process $GPMRC (value = 1)
# bit 1 - process $GPGGA (value = 2)
# bit 2 - process $GPGLL (value = 4)
# bit 4 - process $GPZDA or $GPZDG (value = 8)
# bits 4/5/6 - select serial bitrate (0 for 4800 - the default, 16 for 9600, 32 for 19200, 48 for 38400, 64 for 57600, 80 for 115200)
# mode 25 = process only xxRMC and xxZDA NMEA records at 9600 baud
  

(Note that I used “fudge 127.127.20.0 time2 0.043 …” to adjust the GPS time according to the other NTP servers. You can try some higher/lower values to have the offset of your NMEA driver compared to the offsets of those other NTP servers very small. We’ll cover that later.)

We want to make sure that the second reported by the NMEA driver is the second that the last PPS pulse referred to, so it needs to be within a few hundred milliseconds of the correct time so that NTP does the right thing. This is why it’s desirable to mark another NTP server as “prefer” to help the NTP daemon get it right.

Note that we use NTP PPS discipline, not kernel PPS (which in my testing results in warnings from ntp that kernel PPS discipline isn’t supported).

Comment out all the pool lines.

Add the servers from the dig command, or use servers of your choice) with the top one saying prefer on it (example only, don't all use these IP addresses):

server 176.58.109.199 iburst prefer
server 195.195.221.100 iburst
server 185.53.193.157 iburst
# By default, exchange time with everybody, but don't allow configuration.
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict -6 ::1
restrict 10.0.0.0 mask 255.0.0.0
restrict 172.16.0.0 mask 255.240.0.0
restrict 192.168.0.0 mask 255.255.0.0
# Drift file etc.
driftfile /var/lib/ntp/ntp.drift

    

Note You MUST add a preferred server or PPS doesn’t work.

Save and close nano.

sudo systemctl restart ntp.service

After a few minutes run

ntpq –p

If you get oPPS(0) this indicates source selected, Pulse Per Second (PPS) used and everything is working.

pi@ntp2:~ $ ntpq -pn
remote refid st t when poll reach delay offset jitter
==============================================================================
o127.127.22.0 .PPS. 0 l 6 16 377 0.000 -0.002 0.001
*127.127.20.0 .GPS. 1 l 5 16 377 0.000 -0.323 1.052
+195.195.221.100 .GPS. 1 u 49 64 377 18.307 0.190 0.127
+193.150.34.2 85.199.214.101 2 u 56 64 377 7.953 0.067 0.114
  

And then

pi@ntp2:~ $ ntpq -csysinfo
associd=0 status=0115 leap_none, sync_pps, 1 event, clock_sync,
system peer: PPS(0)
system peer mode: client
leap indicator: 00
stratum: 1
log2 precision: -21
root delay: 0.000
root dispersion: 2.015
reference ID: PPS
reference time: e0601d81.248b4b37 Tue, Apr 16 2019 10:23:13.142
system jitter: 0.000477
clock jitter: 0.003
clock wander: 0.001
broadcast delay: -50.000
symm. auth. delay: 0.000

If you aren’t seeing the settings its possible the NTP server is picking up the NTP information via DHCP which is overriding your settings above. Do this :

rm /etc/dhcp/dhclient-exit-hooks.d/ntp

rm /var/lib/ntp/ntp.conf.dhcp

At this point you have a NTP server which will use an external time source and use your local PPS to discipline it.

Note: Once you've configured ntp to use your GPS source, you'll need to stop the ntp service before running gpsctl, otherwise they'll both be trying to access ttyAMA0 at the same time.

GPS Offset Tuning

Your PPS time is going to be more accurate than NTP pool servers. Unless you have specialised equipment and local LAN GPS / DCF77 / MSF PPS sources to calibrate against, it’s not really possible to determine the appropriate PPS offset. It’s likely to be in the order of a few microseconds at most ( htt://lists.ntp.org/pipermail/questions/2011-September/030338.html ).

The PPS is a precise signal with around 10 ns of jitter. On the other hand, the offset of the GPS serial data output (GPSD) generally will have much more variation than the PPS because of the variables involved in sending data over an asynchronous serial port. However, the GPSD offset can be reduced somewhat by adjusting the GPSD reference clock fudge parameter time2 in the ntp.conf file.

The initial value used in our ntp.conf file is 43ms (0.043s). This means that the GPS ZDA packet arrives in the NTP daemon approximately 43ms after the PPS pulse corresponding to it. On our test device, we see a jitter of under 3ms on the GPS_NMEA clock.

We might need to adjust the GPS ntp.conf time2 (offset) value to get the NMEA time offset low enough that the offset isn’t great enough to confuse NTP as to which second the PPS pulse referred to.

This option can be used to compensate for a constant error. The specified offset (in seconds) is applied to all samples produced by the reference clock. The default is 0.000s.

Start with these ntp.conf settings, they should get you close enough to get everything working properly:

sudo nano /etc/ntp.conf

#kernel-mode PPS
server 127.127.22.0 minpoll 4 maxpoll 4
fudge 127.127.22.0 time1 +0.000 flag3 0 refid PPS
tos mindist 0.002
#GPS (NMEA)
server 127.127.20.0 mode 89 minpoll 4 maxpoll 4 iburst prefer
fudge 127.127.20.0 flag1 0 flag3 0 time2 0.043 refid GPS stratum 2

Add these lines at the top of ntp.conf:

statsdir /var/log/ntpstats/
statistics peerstats
filegen peerstats file peerstats type day enable

    

This enables logging of the peer server statistics.

To calculate the GPS offset we must disable GPS by placing a noselect in the ntp.conf GPS line. We'll run the time server for a few hours and then compare the ntpq -p GPS offset to the average public time server offset. For accurate tuning use a bunch of known-good Stratum 1 servers in ntp.conf.

sudo nano /etc/ntp.conf

#PPS
server 127.127.22.0 minpoll 4 maxpoll 4
fudge 127.127.22.0 time1 +0.000 flag3 0 refid PPS
tos mindist 0.002
#GPS (NMEA)
server 127.127.20.0 mode 89 minpoll 4 maxpoll 4 iburst prefer noselect
fudge 127.127.20.0 flag1 0 flag3 0 time2 0.043 refid GPS stratum 2

sudo systemctl stop ntp.service

sudo rm –f /var/log/ntpstats/*

sudo systemctl start ntp.service

Start ntpd and let it run for at least four hours. Periodically check progress with "ntpq -p" and wait until change has settled out.

Look for the row with GPS_NMEA(0) and refid GPSD. The offset values probably will be different for each

query. Note that the ntpq offsets are in milliseconds, but the peerstats file offsets and NTP’s time2 parameter are in seconds.

Calculate the average GPS offset (in seconds) using this script:

sudo nano ~/nmeaoffset

#!/bin/sh
#
# Generate an estimate of your GPS's offset from a peerstats file
#
awk '
/127\.127\.20\.0/ { sum -= $5 ; cnt++; }
END { printf("%.6f\n", sum / cnt); }
' </var/log/ntpstats/peerstats

    

Then sudo chmod +x ./nmeaoffset

Run by typing

pi@ntp2:~ $ ./nmeaoffset

-0.001212
  

That’s within a few milliseconds, close enough for anyone as PPS is going to do its magic to give us accuracy within a few microseconds.

Adjust the "time2" value for the GPS source of your ntp.conf by adding the average offset from above.

sudo systemctl stop ntp.service

sudo rm –f /var/log/ntpstats/*

sudo systemctl start ntp.service

Repeat the procedure above until -5ms < offset < +5ms (or under 10ms, if that’s OK with you). When you’re done, remove the noselect from the server 127.127.20.0 line in ntp.conf.

If you decide to recalculate the average offset using the above procedures, wait at least another day or two.

Avoid unnecessarily changing the time2 value. A typical value for the Adafruit GPSD driver is +0.534 s when using its default 4800 baud interface and in the HAB Supplies/Uputronics GPSD it’s +0.043 s when using 115200 baud and the gpsctl --config_for_timing tweaks.

To save wear and tear on the SD card, comment out the statistics line in /etc/ntp.conf when done.

Automatically updating GPS leap seconds

These semi-annual changes will be made no later than 1 June and 1 December of each year to indicate what action (if any) is to be taken on 30th June and 31st December, respectively.

In Buster, the default ntp.conf points LeapFile at the tzdata-provided leap-seconds.list. That's fine, as long as you auto apply system updates via cron or manually on a regular basis.

Automatically updating Raspbian

To automatically update Raspbian on a schedule, follow these steps:

sudo apt install unattended-upgrades mailutils

sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

Edit to include these sources (remove all others)

    
"origin=Raspbian,codename=${distro_codename},label=Raspbian";
"origin=Raspberry Pi Foundation,codename=${distro_codename},label=Raspberry Pi Foundation";
and add
Unattended-Upgrade::Automatic-Reboot "true";
sudo nano /etc/apt/apt.conf.d/20auto-upgrades
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "1";
APT::Periodic::Verbose "1";
APT::Periodic::AutocleanInterval "7";


sudo dpkg-reconfigure --priority=low unattended-upgrades

Set the systemctl apt timer to fire at 2AM daily

sudo systemctl edit apt-daily-upgrade.timer
[Timer]
OnCalendar=
# OnCalendar=DayOfWeek Year-Month-Day Hour:Minute:Second
# see https://wiki.archlinux.org/index.php/Systemd/Timers
OnCalendar=02:00
RandomizedDelaySec=0

To test,

sudo unattended-upgrade -d -v --dry-run

Static IP and Hostname

If you want to fix your LAN IP you do it by amending /etc/dhcpcd.conf adding the following lines (adjust to suit your environment):

# It is possible to fall back to a static IP if DHCP fails:
# define static profile
profile static eth0
static ip_address=192.168.1.7/24
static routers=192.168.1.254
static domain_name_servers=8.8.8.8 8.8.4.4
# fallback to static profile on eth0
interface eth0
fallback static_eth0

    

This way DHCP will work if plugged into a client switch port, but static IP will default when no DHCP is available.

Great for testing the box on one’s desktop before going live.

Amend your hostname by editing /etc/hostname and then adding the below to /etc/hosts.

i.e if you call your machine ‘ntp’ fix the /etc/hosts 127.0.1.1 line:

127.0.1.1 ntp
  

Further Reading

The original text this document was based on can be found at https://ava.upuaut.net/?p=951

Updated to include information from comments at https://blog.webernetz.net/ntp-server-via-gps-on-a-raspberry-pi/

David Taylor’s website https://satsignal.eu/ntp/Raspberry-Pi-NTP.html goes into much further detail about the process above and covers graphing, remote access monitoring.

Rob Robinette has a good write up on Pi-based NTP servers

https://robrobinette.com/pi_GPS_PPS_Time_Server.htm

Whitham D. Reeve’s GpsNtp-Pi Installation and Operation Guide

http://www.reeve.com/RadioScience/Raspberry%20Pi/GpsNtp-Pi.htm

Rich Laager’s Raspberry Pi 3 Stratum 1 NTP Server has the best stuff on offsets

https://coderich.net/2016/11/21/raspberry-pi-3-stratum-1-ntp-server/

Tech Solvency’s similar setup

https://www.techsolvency.com/ntp/systems/tackleberry-uputronics/

Gary Miller’s GPSD Time Service HOWTO

http://www.catb.org/gpsd/gpsd-time-service-howto.html

Jack Zimmerman’s Raspberry PI NTP Server LCD Display

https://github.com/jacken/Raspberry-Pi-ntp-server-LCD-display

Jack Zimmerman’s Pi U-Blox Stationary Mode

https://github.com/jacken/Raspberry-Pi-U-Blox-Stationary-Mode

Tom Dilatush’s gpsctl

http://www.jamulblog.com/2017/11/paradise-ponders-gpsctl-functionally.html

My improved gpsctl with added .conf file support GitHub repo

https://github.com/philrandal/gpsctl

u-blox M8 Receiver Description (the bible)

https://www.u-blox.com/sites/default/files/products/documents/u-blox8-M8_ReceiverDescrProtSpec_%28UBX-13003221%29_Public.pdf

GNSS Firmware 3.01 for u-blox M8

https://www.u-blox.com/sites/default/files/GNSS-FW3.01_ReleaseNotes_%28UBX-16000319%29_Public.pdf

Postscript, July 15th, 2019

You can enable either BeiDou satellites, or Glonass, but not both with gpsctl. You'll get an error if you try. Section 4.2 of the u-blox m8 receiver description manual has a hint as to why.

From version 1.5 of gpsctl, enabling either BeiDou or Galileo automatically enables NMEA 4.1.

Postscript, January 30th, 2020

Updated for the Raspberry Pi 4, used cpufrequtils to set processor speed, added nohz=off, fixed numerous typos, added automatic updating howto, and added a note about making sure that ntpd is not running when you run gpsctl.

Postscript, June 21st, 2020

Released gpsctl v1.9. Tried to fix an issue with gpsctl not initialising the u-blox module via systemd at startup. Still not convinced that it is right.

Added configure-rv3028.sh script to properly configure the rv3028 RTC chip on the new (June 2020) Uputronics boards.

Without this step I got lots of

    kernel: [  156.227408] rtc-rv3028 1-0052: Voltage low, data is invalid.

messages in /var/log/messages on boot.

To configure Raspberry Pi OS to use the rv3028,

sudo nano /boot/config.txt

Add the line

dtoverlay=i2c-rtc,rv3028

at the end.

Remove the fake hardware clock:

sudo apt-get -y remove fake-hwclock
sudo update-rc.d -f fake-hwclock remove
sudo systemctl disable fake-hwclock

Run sudo nano /lib/udev/hwclock-set and comment out these three lines:

#if [ -e /run/systemd/system ] ; then
# exit 0
#fi

Also comment out the two lines

/sbin/hwclock --rtc=$dev --systz --badyear

and

/sbin/hwclock --rtc=$dev --systz

Configure the rv3028 by running

sudo configure-rv3028.sh

from the gpsctl archive.

Then,

sudo hwclock -w -v

to set the clock for the first time.

To debug, sudo systemctl disable ntp, then sudo reboot

After the reboot, sudo hwclock -r -v should show the retained time.

Then sudo systemctl enable ntp --now to enable and start ntp.

Postscript, October 17th, 2020

Due to the repeated unreliabilty of gpsctl's automatic baud rate synchronisation, I've reverted to using 9600 baud and not messing with the baud rate at all. The latest version in the github master branch allows you to specify the gpsctl parameters for ublox-init.service in /etc/default/gpsctl. e.g.

PARAMS=-q --configure-for-timing

If /etc/default/gpsctl doesn't exist, the behaviour will be as before.

Any edits of ublox-init.service or /etc/default/gpsctl require you to run sudo systemctl daemon-reload



Posted by Phil at 10:26 AM
Edited on: Friday, May 07, 2021 2:22 PM
Categories: IT, Raspberry Pi, Software

Monday, April 29, 2013

Migrating Nagios Configuration from Nagmin to Check_MK's WATO


When I first set up a Nagios server, many years ago, in the days of Nagios 1.x, the best configuration tool I could find was Fred Reimers' Nagmin. That has since turned into abandonware, but there is a fork, NagminV, under development.

I'd patched Nagmin to support Nagios 2.x and 3.x, and added a few fields to its database, but it was still buggy and quirky.

So, after I'd installed Mathias Kettner's Check_MK for its livestatus broker module for use with PNP4Nagios, I started investigating Check_MK's broader features.

What soon caught my eye was Check_MK's use of rulesets based on host tags. The temptation of editing text files (python scripts in disguise) was too great for me, so I started converting my Nagios service checks into Check_MK format.

These are very rough notes, proceed with caution. Back up everything first!!!

First thing to do was get Check_MK's agent on all our hosts.

Then, to list all of them in /etc/check_mk/main.mk:

# don't generate host config yet
# comment this out when Nagmin is decommissioned
generate_hostconf = False
all_hosts = [
host1,
host2,
]

Then I added the obvious tags, win, linux, etc, and created config files for legacy checks in /etc/check_mk/conf.d, until eventually all service checks were defined in Check_MK and not in Nagmin.

After each check was migrated to Check_MK, I'd run

check_mk -U
nagios -v /etc/nagios/nagios.cfg

The first command generates a new Nagios config in /etc/nagios/check_mk.d/check_mk_objects.cfg

The second validates the resulting config. Here I'd find duplicate definitions, reminding me to delete them from Nagmin.

So, after a while plodding away at this - in my case this meant over a year of coexistence - all that was left in Nagmin was hosts, host groups, contacts, contact groups, and timeperiods.

Time to abandon Nagmin and get serious.

Do not, under any circumstances, try to generate a Nagios config using Nagmin again. It will always fail!

Comment out the Command.cfg include in nagios.cfg

#cfg_file=/etc/nagios/Command.cfg

Try validating the nagios config again, with nagios -v. If it fails, you've forgotten to define some commands as legacy checks

Rinse, repeat until you've got it right.

Check the contents of /etc/nagios/Services.cfg - if it contains any service definitions, you've forgotten something.

Do the same with all the service-related .cfg files; if things fail, change your config to use Check_MK's default service templates.

Get your custom Time Periods into WATO - check_mk has a default, hidden timeperiod 24X7 (Nagmin's is 24x7, case matters), and comment out TimePeriods.cfg in nagios.cfg. Revalidate.

And so on till you're left only with hosts, host groups, contacts, and contact groups from the original Nagmin.

So now was the time to take the leap and use Check_MK's WATO to configure nagios.

I created contacts and contact groups manually in WATO. There's no conflict with your existing Nagios config until you associate a contact group with hosts and/or services in WATO. That was the last thing I did.

Now comes the fun bit.

We need to import our hosts into WATO.

Run the attached hosts.py which will generate a list of hosts in the format hostname;alias;parents;ipaddress

python hosts.py >wato.csv

Our import script requires a file containing

wato folder;hostname;alias;parents;ipaddress;tags

where tags is a list of check_mk tags separated by |

So, we'll have to manually edit it. I didn't use WATO folders so just preceded each line with a ;

Tags are the fun bits. By now you may have been using them in your check_mk config. WATO comes with some predefined ones, which we need to list in our import file (wato.csv).

You'll need one each of the 'agent', 'criticality', and 'networking' tags. As well as your custom tags, which should also be entered into WATO.

Download and edit my_wato_import.py adding your tag definitions into tagz, run:

python my_wato_import.py wato.csv

and watch the output scroll by. If there are any errors reported, then you've forgotten to add a tag definition to tagz, or misspelt a tag in wato.csv. Rinse and repeat until all's well.

Set testing = False near the top of the script, and run again.

Comment out generate_hostconf = False in your main.mk, and check_mk -U

Comment out Host.cfg in /etc/nagios/nagios.cfg

Validate your nagios configuration. It should be OK.

A look in WATO will show all your hosts, with appropriate tags.

Hidden away in /usr/share/doc/check_mk/treasures is a script wato_host_svc_groups.py

I ran this against my original Nagmin Hosts.cfg file, which produced output which was easily massaged into the form needed for WATO. I could have amended the script to produce output of the following form, but a few regular-expression search and replaces got me there quickly enough.

host_groups = [
( 'group1', []. ['server1', 'server2']),
( 'group2', []. ['server3', 'server3']),
]

Place that code into /etc/check_mk/conf.d/wato/rules.mk and create the host groups (with descriptions) in WATO. Before applying changes in WATO, run check_mk -U

Comment out the Hostgroup.cfg include in nagios.cfg, and validate config once more.

#cfg_file=/etc/nagios/HostGroup.cfg

Now, if you've done everything properly, the Nagios config validation will succeed, and on a restart of Nagios your host groups will be there as before.

That just leaves Contacts, Contact Groups and notifications.

I'll leave that as an exercise for the reader. Hint: don't try any of the above until you've figured out how to apply contact groups to hosts and services.

And you'll also need to adjust host and service check intervals and retries in WATO too, otherwise everything gets polled every minute, which probably isn't what you want.



Posted by Phil at 9:59 PM
Edited on: Wednesday, April 01, 2015 9:24 PM
Categories: IT, Software

Thursday, September 02, 2010

Installing Exchange 2010 Service Pack 1


I had a few hassles installing Microsoft Exchange 2010 SP1 today.

I guess if things ran smoothly all the time, I wouldn't have a job, but....

All worked perfectly yesterday on our virtualised test setup, but in real life, it is never that simple.

The Exchange Team have provided a handy list of prerequisites and known problems with SP1 in their "Exchange 2010 SP1 FAQ and Known Issues" blog post. Read it carefully.

On one of our CAS servers, I got an error warning me of a failed installation. Following the instructions in "How does Exchange 2007 setup know to resume a failed setup?", I determined that there had been a previous error installing the Admin Tools. Clobbering the registry entry (the one with the install value) enabled the install to proceed.

After the install, an attempt to start the Management Console failed. Turning again to the MS Exchange Team's blog provided the solution, in "Troubleshooting Exchange 2010 Management Tools startup issues". In our case, the Service Pack install had stripped the WSMan module entry from the global modules section of C:\Windows\System32\Inetsrv\config\ApplicationHost.config. Adding it back in after the SP1 install and restarting the default web site, that issue was solved. We had to do this on all our Exchange 2010 boxes.

Next error came moving the active database from one DAG member server to another. Corrupt index. After a quick Google I discovered that restarting the Windows Search (Exchange) service on the Mailbox servers was enough to fix this issue. I guess a reboot after the Service Pack install would have had the same effect, even though the Service Pack install didn't request one.

It is all working fine now (fingers crossed).



Posted by Phil at 9:07 PM
Edited on: Thursday, September 02, 2010 9:50 PM
Categories: Comment, Software

Friday, August 11, 2006

Active Virus Shield


AOL is widely regarded as an evil empire, second only to Microsoft, but this week they redeemed themselves somewhat by releasing a free Windows antivirus program, Active Virus Shield, based on Kapersky Labs' Personal Antivirus.

I've given it a whirl here and so far I'm impressed. Kapersky's Virus scanning engine is one of the best, and they are renowned for the speed of their responses to new malware. So far I'm seeing several pattern updates a day, which is what I'd expect from any proper antivirus vendor.

It's staying on my PC for a while. It may yet replace Alwil's Avast! here, but I have a certain irrational fondness for products whose support forums give the users direct contact with the developers.

Postscript, September 5th

Independent research confirms Active Virus Shield as being one of the best antivirus products. There's commentary and discussion over at cybernetnews.com.



Posted by Phil at 11:00 PM
Edited on: Tuesday, September 05, 2006 11:56 AM
Categories: Computer Security, Software

Thursday, June 08, 2006

Mentioned in Dispatches


Thanks Blake, for your kind mention in your weblog. I'm honoured!



Posted by Phil at 9:50 PM
Edited on: Thursday, June 08, 2006 9:52 PM
Categories: Software, Waffle

Saturday, April 30, 2005

50,000,000


Congratulations to the Mozilla Firefox team on their 50 millionth download.

50 million downloads counted

I was a bit slow with the screenshot there :-)

Posted by Phil at 12:20 PM
Edited on: Wednesday, April 01, 2015 9:59 PM
Categories: Software

Monday, September 20, 2004

I'm a Firefoxer


My humblest apologies to Prodigy...

If you're bored you might want to check out my other blog at SpreadFirefox.

Posted by Phil at 9:36 PM
Edited on: Saturday, April 23, 2005 10:20 AM
Categories: Software

Tuesday, January 14, 2003

Favourite Software


Just because you asked (or didn't), here's my favourite software, in no particular order:

Mozilla, the most standards-compliant web browser out there.

Audacity, for sound file editing.

Gnucleus, a peer-to-peer filesharing client completely free of spyware.

The GIMP, the graphics program which doesn't cost an arm and a leg. Installing it on windows is a bit involved, however.

OpenOffice, who needs Microsoft's bloatware when this will do nicely for most people?

All the above are open source and free to use wherever you like.

ConText, a lean and mean text editor. Free, no source included.

IrfanView, a great image viewer. Free for personal use, no source included.

Spybot Search and Destroy, the best SpyWare/AdWare cleaner.

So tell me, who needs to pay for software?

Posted by Phil at 9:03 PM
Edited on: Saturday, April 23, 2005 10:22 AM
Categories: Software