Phil's Occasional Blog : April 2019

« July 2019 | Main | January 2016 »

Saturday, April 27, 2019

A Raspberry Pi Stratum 1 NTP Server

Overview

The following instructions are how to make a cheap Pulse Per Second (PPS) disciplined Stratum 1 NTP Time server using one of the Raspberry Pi U-blox M8Q based GPS boards sold by Uputronics.

Our basic requirement is for an NTP server which will work standalone without connectivity to other NTP servers, so if a datacentre loses connectivity its servers will still have a stable time source.

As we’re only interested in time and not height, motion, or location, all we’re interested in is the NMEA xxZDA and xxRMC sentences and the highly accurate PPS signal from the GPS module, both of which we feed straight into the ntp daemon. The xxZDA sentences (which give us full 4-digit years) are not output by default by the M8Q so we have to enable them at boot time.

Many guides on the net use gpsd to feed ntp via shared memory. That’s an additional overhead and complexity, especially if we want to enable Galileo reception (gpsd initialises the U-blox the way it sees fit). I prefer to let the NTP daemon read the NMEA stream from the GPS receiver itself, avoiding the middle man.

In this guide we’ve chosen to use NMEA output from the GPS module via the ntp Generic GPS Receiver driver 20 ( https://www.eecis.udel.edu/~mills/ntp/html/drivers/driver20.html ) and the PPS signal via driver 22 ( https://www.eecis.udel.edu/~mills/ntp/html/drivers/driver22.html ).

You will need a Raspberry 3 B+, the Uputronics Raspberry Pi+ GPS Expansion Board and a suitable GPS antenna.

This guide assumes that you’re using Raspbian Stretch Lite November 2018 released 2018-11-13 image, or later. Download and write this to an SD card.

See https://www.raspberrypi.org/documentation/installation/installing-images/windows.md

Create a file named ssh in the boot folder after burning the MicroSD card.

Attach the Uputronics Raspberry Pi+ GPS Expansion Board to the Pi, insert the SD card, connect the antenna and network cable and boot the Pi up. Either connect locally or via SSH to the Pi. If you can’t SSH in and don’t have a monitor see this

https://www.raspberrypi.org/documentation/configuration/wireless/headless.md

Follow the instructions carefully if you miss steps things won’t work.

The Uputronics board has u-blox firmware 3.01 on it, dated 2016.

The week number rollover is set to 1867 (October 2015). All transmitted week numbers are mapped to the ~19.5 year period between week 1867 and week 2990 (April 2035).

A Note About Accuracy

In theory, GPS-based time receivers can give a very high accuracy, with the PPS (Pulse per Second) signal being accurate to within 10ns.

However, fix data from gpsctl shows:

pi@ntp2:~ $ /usr/local/bin/gpsctl -Q fix

Time (UTC): 2019-04-18 15:31:27 (yyyy-mm-dd hh:mm:ss)

Latitude: 52.05994980 N

Longitude: 2.72698960 W

Altitude: 198.789 feet

Motion: 0.338 mph at 53.114 degrees heading

Satellites: 5 used for computing this fix

Accuracy: time (39 ns), height (+/-18.199 feet), position (+/-103.911 feet), heading(+/-8.230 degrees), speed(+/-0.132 mph)

Note the 39ns time accuracy from that fix. The more satellites, the better.

Errors are also caused by signal delay (4ns per foot) in the cable from the aerial to the GPS receiver, etc.

So we’d be lucky to get 100ns accuracy from the PPS pulse.

Add to that the processing overheads of the PPS interrupt, and processor clock jitter in the Raspberry Pi, and overheads transferring and decoding the NMEA sentences. The jitter on the PPS signal is less than 5 microseconds.

The output from the ntp NMEA driver without PPS correction can have a jitter of a few milliseconds, and an offset from real time which has to be tweaked manually to get within 5ms of the correct time.

NTP sources on the internet can show offsets / jitter of over 5ms, local LAN 50us or more.

On the Pi 3B+ we can expect an average jitter of less than 1 microsecond.

Required Components

I sourced my components from ModMyPi (now taken over by the Pi Hut) unless noted elsewhere:

1 Raspberry Pi Model 3B+

1 8GB or larger micro SD card

(a Transcend High Endurance 32GB micro SD card would be better choice than a generic one for longevity)

1 Uputronics GPS Hat

1 ModMyPi Pi GPS case

1 Raspberry Pi 3 power supply

1 GPS SMA Antenna

(optional) 1 SMA-male to TNC-female (or BNC, as needed) adaptor to connect to existing GPS aerial (from Amazon or eBay)

Prerequisite Settings

passwd

sudo raspi-config

7 Advanced Options

A1 Expand filesystem

5 Interfacing Options

P2 SSH -> Would you like the SSH server to be enabled – YES (Recommended)

P6 Serial -> Login Shell (no) Hardware (yes) (Optional)

Quit but no need to reboot at this point.

sudo systemctl disable hciuart

sudo systemctl disable serial-getty@ttyAMA0.service

sudo systemctl mask serial-getty@ttyAMA0.service

sudo apt install pps-tools ntp setserial wiringpi

sudo apt purge bluez bluez-firmware wpasupplicant (assuming no wifi required)

sudo apt update

sudo apt full-upgrade

sudo nano /boot/config.txt and add at the bottom :

dtoverlay=pi3-disable-bt
dtoverlay=pi3-disable-wifi
dtoverlay=pps-gpio,gpiopin=18

Make sure that enable_uart=1 is set in /boot/config.txt

sudo nano /etc/modules and add at the bottom :

pps-gpio

Save and Quit Nano.

sudo nano /etc/udev/rules.d/80-gps-to-ntp.rules (needed for refclock 20 in ntp)

# Change MODE of ttyAMA0 so it is readable by NTP and provide

# a symlink to /dev/gps0

KERNEL=="ttyAMA0", SUBSYSTEM=="tty", DRIVER=="", SYMLINK+="gps0", MODE="0666"

# Symlink /dev/pps0 to /dev/gpspps0

KERNEL=="pps0", SUBSYSTEM=="pps", DRIVER=="", SYMLINK+="gpspps0", MODE="0666"

and then sudo reboot

Enabling Galileo Satellites and Setting Stationary Mode

Stationary mode gives a faster GPS fix.

See Tom Dilatush’s gpsctl:

http://www.jamulblog.com/2017/11/paradise-ponders-gpsctl-functionally.html

I’ve customised it further to enable the use of a config file and put it in my GitHub repo:

https://github.com/philrandal/gpsctl

Download and build gpsctl.

cd ~
wget https://github.com/philrandal/gpsctl/archive/master.zip
unzip gpsctl-master.zip
mv gpsctl-master gpsctl
cd gpsctl
./build.sh
sudo cp /home/pi/gpsctl/gpsctl /usr/local/bin
sudo cp /home/pi/gpsctl/etc/gpsctl.conf /etc/gpsctl.conf

Individual settings can be tweaked in /etc/gpsctl.conf.

To set port speed to 115200 baud, enable Galileo satellites, set stationary mode, tweak antenna delays, PPS timing, etc:

/usr/local/bin/gpsctl -a -B 115200 --configure_for_timing -vv

To reset the device to its defaults

/usr/local/bin/gpsctl -a --reset -vv

To view info:

/usr/local/bin/gpsctl –a –Q satellites

/usr/local/bin/gpsctl –a –Q config

/usr/local/bin/gpsctl –a –Q fix

Note that these commands can only be run when gpsd / ntp are not using /dev/ttyAMA0

Example /etc/gpsctl.conf which configures the U-blox M8Q for this environment:

    
#
# example gpsctl.conf which enables Galileo as in --galileo parameter

[gpsctl]

port = /dev/serial0

# sync method: ASCII = 1, NMEA = 2, UBX = 3

sync method = 3

verbosity = 0

[NMEA]

enabled = true

version = 41

GGA = off

GLL = off

GSA = off

GSV = off

RMC = on

VTG = off

GRS = off

GST = off

ZDA = on

[GPS]

enabled = yes

minimum channels=8

maximum channels=16

[SBAS]

enabled = no

minimum channels=1

maximum channels=3

[Galileo]

enabled = yes

minimum channels=4

maximum channels=8

[Beidou]enabled = no

minimum channels=8

maximum channels=16

[IMES]

enabled = no

minimum channels=0

maximum channels=8

[QZSS]

enabled = no

minimum channels=0

maximum channels=3

[GLONASS]

enabled = yes

minimum channels=8

maximum channels=14

[Navigation Engine]

# Dynamic model: Portable = 0, Stationary = 2, Pedestrian = 3, Automotive = 4,

# Sea = 5, Air1G = 6, Air2G = 7, Air4G = 8, Watch = 9

Dynamic model = 2

# Fix mode: 2D only = 1, 3D only = 2, auto 2D/3D = 3

Fix mode = 3

Fixed altitude (2D) = 0.00 meters

Fixed altitude variance (2D) = 1.0000 meters^2

Minimum elevation = 5 degrees

Position DoP mask = 10.0

Time DoP mask = 10.0

Position accuracy mask = 100 meters

Time accuracy mask = 300 meters

Static hold threshold = 0 cm/s

Dynamic GNSS timeout = 60 seconds

Threshold above C/No = 0 satellites

C/No threshold = 0 dBHz

Static hold max distance = 0 meters

# UTC Standard: AutoUTC = 0, USNO_UTC = 3, GLONASS_UTC = 6, BeiDou_UTC = 7

UTC standard = 3

[Time Pulse]

# the nanoseconds / microseconds after the numbers are just reminders,

# they don't mean anything to the config parser

Antenna cable delay = 56 nanoseconds

RF group delay = 20 ns

Unlocked pulse period = 1000000 microseconds

Unlocked pulse length = 0

Locked pulse period = 1000000 microseconds

Locked pulse length = 500000 microseconds

User configurable delay = 0

To run gpsctl at system startup,

sudo cp /home/pi/gpsctl/systemd/ublox-init.service /etc/systemd/system/ublox-init.service

nano /etc/systemd/system/ublox-init.service

and add the following contents

[Unit]
Description=u-blox initialisation

Before=gpsd.service

Before=ntp.service

[Service]

Type=oneshot

ExecStart=/usr/local/bin/gpsctl -q -a -B 115200 --configure_for_timing

[Install]

WantedBy=multi-user.target

This will enable Galileo satellites, set stationary mode, configure comms at 115200 baud, and restrict NMEA output to RMC and ZDA records.

Then

sudo systemctl enable ublox-init.service

sudo systemctl daemon-reload

For maximum stability and ntp performance, add these lines to the end of /etc/rc.local, before the exit 0 line

echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor

Verifying that PPS Is Working

Ensure the GPS has a lock and the Green PPS LED on the Uputronics Pi+ GPS Expansion Board is blinking once a second.

lsmod | grep pps

Output should be similar to :

pps_gpio                3089  1pps_
core                8606  4 pps_gpio

dmesg | grep pps

Output should be similar to :

[    2.735586] pps_core: LinuxPPS API ver. 1 registered
[ 2.738121] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>

[ 2.763842] pps pps0: new PPS source pps@12.-1

[ 2.766361] pps pps0: Registered IRQ 169 as PPS source

This indicates that the PPS Module is loaded.

sudo ppstest /dev/pps0

Output should be similar to:

trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...

source 0 - assert 1418933982.998042450, sequence: 970 - clear  0.000000000, sequence: 0

source 0 - assert 1418933983.998045441, sequence: 971 - clear  0.000000000, sequence: 0

(Press CTRL+C to quit). This indicates that the PPS Module is working.

Enabling PPS/ATOM Support in NTPD

The supplied version of NTPD on the Raspberry Pi in Raspbian Stretch 2018-11-13 and later supports PPS so there is no need to roll your own NTP.

You need to pick a few local NTP servers to use. The easiest way to do this is pick your region:

https://support.ntp.org/bin/view/Servers/NTPPoolServers

Select your region then you get a list of the country servers. E.g for the UK its uk.pool.ntp.org.

Type:

dig uk.pool.ntp.org

You will get four IP’s back:

;; &lt;&lt;&gt;&gt; DiG 9.10.3-P4-Raspbian &lt;&lt;&gt;&gt; +answer uk.pool.ntp.org
;; global options: +cmd

;; Got answer:

;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 51647

;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:

; EDNS: version: 0, flags:; udp: 4000

;; QUESTION SECTION:

;uk.pool.ntp.org. IN A

;; ANSWER SECTION:

uk.pool.ntp.org. 24 IN A 193.150.34.2

uk.pool.ntp.org. 24 IN A 176.58.109.199

uk.pool.ntp.org. 24 IN A 195.195.221.100

uk.pool.ntp.org. 24 IN A 185.53.93.157

sudo nano /etc/ntp.conf

Add

# PPS Driver
server 127.127.22.0 minpoll 4 maxpoll 4

fudge 127.127.22.0 time1 +0.000000 flag3 0 refid PPS

#flag3 Controls the kernel PPS discipline: 0 for disable (default), 1 for enable.

#time1 PPS time offset

tos mindist 0.002

# NMEA driver (/dev/gps0 and /dev/gpspps0)

server 127.127.20.0 mode 89 minpoll 4 maxpoll 4 iburst prefer

fudge 127.127.20.0 flag1 0 flag2 0 flag3 0 time2 0.043 refid GPS stratum 2

#flag1 Disable PPS signal processing if 0 (default); enable PPS signal processing if 1.

#flag2 If PPS signal processing is enabled, capture the pulse on the rising edge if 0 (default); capture on the falling edge if 1.

#flag3 If PPS signal processing is enabled, use the ntpd clock discipline if 0 (default); use the kernel discipline if 1.

#time1 PPS time offset

#time2 NMEA time offset

#mode bit 0 - process $GPMRC (value = 1)

# bit 1 - process $GPGGA (value = 2)

# bit 2 - process $GPGLL (value = 4)

# bit 4 - process $GPZDA or $GPZDG (value = 8)

# bits 4/5/6 - select serial bitrate (0 for 4800 - the default, 16 for 9600, 32 for 19200, 48 for 38400, 64 for 57600, 80 for 115200)

# mode 89 = process only xxRMC and xxZDA NMEA records at 115200 baud

(Note that I used “fudge 127.127.20.0 time2 0.043 …” to adjust the GPS time according to the other NTP servers. You can try some higher/lower values to have the offset of your NMEA driver compared to the offsets of those other NTP servers very small. We’ll cover that later.)

We want to make sure that the second reported by the NMEA driver is the second that the last PPS pulse referred to, so it needs to be within a few hundred milliseconds of the correct time so that NTP does the right thing. This is why it’s desirable to mark another NTP server as “prefer” to help the NTP daemon get it right.

Note that we use NTP PPS discipline, not kernel PPS (which in my testing results in warnings from ntp that kernel PPS discipline isn’t supported).

Comment out all the pool lines.

Add the servers from the dig command, or use servers of your choice) with the top one saying prefer on it (example only, don't all use these IP addresses):

server 176.58.109.199 iburst prefer
server 195.195.221.100 iburst

server 185.53.193.157 iburst

# By default, exchange time with everybody, but don't allow configuration.

restrict default kod nomodify notrap nopeer noquery

restrict -6 default kod nomodify notrap nopeer noquery

# Local users may interrogate the ntp server more closely.

restrict 127.0.0.1

restrict -6 ::1

restrict 10.0.0.0 mask 255.0.0.0

restrict 172.16.0.0 mask 255.240.0.0

restrict 192.168.0.0 mask 255.255.0.0

# Drift file etc.

driftfile /var/lib/ntp/ntp.drift

Note You MUST add a preferred server or PPS doesn’t work.

Save and close nano.

sudo systemctl restart ntp.service

After a few minutes run

ntpq –p

If you get oPPS(0) this indicates source selected, Pulse Per Second (PPS) used and everything is working.

pi@ntp2:~ $ ntpq -pn
remote refid st t when poll reach delay offset jitter

==============================================================================

o127.127.22.0 .PPS. 0 l 6 16 377 0.000 -0.002 0.001

*127.127.20.0 .GPS. 1 l 5 16 377 0.000 -0.323 1.052

+195.195.221.100 .GPS. 1 u 49 64 377 18.307 0.190 0.127

+193.150.34.2 85.199.214.101 2 u 56 64 377 7.953 0.067 0.114

And then

pi@ntp2:~ $ ntpq -csysinfo
associd=0 status=0115 leap_none, sync_pps, 1 event, clock_sync,

system peer: PPS(0)

system peer mode: client

leap indicator: 00

stratum: 1

log2 precision: -21

root delay: 0.000

root dispersion: 2.015

reference ID: PPS

reference time: e0601d81.248b4b37 Tue, Apr 16 2019 10:23:13.142

system jitter: 0.000477

clock jitter: 0.003

clock wander: 0.001

broadcast delay: -50.000

symm. auth. delay: 0.000

If you aren’t seeing the settings its possible the NTP server is picking up the NTP information via DHCP which is overriding your settings above. Do this :

rm /etc/dhcp/dhclient-exit-hooks.d/ntp

rm /var/lib/ntp/ntp.conf.dhcp

At this point you have a NTP server which will use an external time source and use your local PPS to discipline it.

GPS Offset Tuning

Your PPS time is going to be more accurate than NTP pool servers. Unless you have specialised equipment and local LAN GPS / DCF77 / MSF PPS sources to calibrate against, it’s not really possible to determine the appropriate PPS offset. It’s likely to be in the order of a few microseconds at most ( http://lists.ntp.org/pipermail/questions/2011-September/030338.html ).

The PPS is a precise signal with around 10 ns of jitter. On the other hand, the offset of the GPS serial data output (GPSD) generally will have much more variation than the PPS because of the variables involved in sending data over an asynchronous serial port. However, the GPSD offset can be reduced somewhat by adjusting the GPSD reference clock fudge parameter time2 in the ntp.conf file.

The initial value used in our ntp.conf file is 43ms (0.043s). This means that the GPS ZDA packet arrives in the NTP daemon approximately 43ms after the PPS pulse corresponding to it. On our test device, we see a jitter of under 3ms on the GPS_NMEA clock.

We might need to adjust the GPS ntp.conf time2 (offset) value to get the NMEA time offset low enough that the offset isn’t great enough to confuse NTP as to which second the PPS pulse referred to.

This option can be used to compensate for a constant error. The specified offset (in seconds) is applied to all samples produced by the reference clock. The default is 0.000s.

Start with these ntp.conf settings, they should get you close enough to get everything working properly:

sudo nano /etc/ntp.conf

#kernel-mode PPS
server 127.127.22.0 minpoll 4 maxpoll 4

fudge 127.127.22.0 time1 +0.000 flag3 0 refid PPS

tos mindist 0.002

#GPS (NMEA)

server 127.127.20.0 mode 89 minpoll 4 maxpoll 4 iburst prefer

fudge 127.127.20.0 flag1 0 flag3 0 time2 0.043 refid GPS stratum 2

Add these lines at the top of ntp.conf:

statsdir /var/log/ntpstats/
statistics peerstats

filegen peerstats file peerstats type day enable

This enables logging of the peer server statistics.

To calculate the GPS offset we must disable GPS by placing a noselect in the ntp.conf GPS line. We'll run the time server for a few hours and then compare the ntpq -p GPS offset to the average public time server offset. For accurate tuning use a bunch of known-good Stratum 1 servers in ntp.conf.

sudo nano /etc/ntp.conf

#PPS
server 127.127.22.0 minpoll 4 maxpoll 4

fudge 127.127.22.0 time1 +0.000 flag3 0 refid PPS

tos mindist 0.002

#GPS (NMEA)

server 127.127.20.0 mode 89 minpoll 4 maxpoll 4 iburst prefer noselect

fudge 127.127.20.0 flag1 0 flag3 0 time2 0.043 refid GPS stratum 2

sudo systemctl stop ntp.service

sudo rm –f /var/log/ntpstats/*

sudo systemctl start ntp.service

Start ntpd and let it run for at least four hours. Periodically check progress with "ntpq -p" and wait until change has settled out.

Look for the row with GPS_NMEA(0) and refid GPSD. The offset values probably will be different for each

query. Note that the ntpq offsets are in milliseconds, but the peerstats file offsets and NTP’s time2 parameter are in seconds.

Calculate the average GPS offset (in seconds) using this script:

sudo nano ~/nmeaoffset

#!/bin/sh
#

# Generate an estimate of your GPS's offset from a peerstats file

awk '

/127\.127\.20\.0/ { sum -= $5 ; cnt++; }

END { printf("%.6f\n", sum / cnt); }

' </var/log/ntpstats/peerstats

Then sudo chmod +x ./nmeaoffset

Run by typing

pi@ntp2:~ $ ./nmeaoffset

-0.001212

That’s within a few milliseconds, close enough for anyone as PPS is going to do its magic to give us accuracy within a few microseconds.

Adjust the "time2" value for the GPS source of your ntp.conf by adding the average offset from above.

sudo systemctl stop ntp.service

sudo rm –f /var/log/ntpstats/*

sudo systemctl start ntp.service

Repeat the procedure above until -5ms < offset < +5ms (or under 10ms, if that’s OK with you). When you’re done, remove the noselect from the server 127.127.20.0 line in ntp.conf.

If you decide to recalculate the average offset using the above procedures, wait at least another day or two.

Avoid unnecessarily changing the time2 value. A typical value for the Adafruit GPSD driver is +0.534 s when using its default 4800 baud interface and in the HAB Supplies/Uputronics GPSD it’s +0.043 s when using 115200 baud and the gpsctl --config_for_timing tweaks.

To save wear and tear on the SD card, comment out the statistics line in /etc/ntp.conf when done.

Automatically updating GPS leap seconds

These semi-annual changes will be made no later than 1 June and 1 December of each year to indicate what action (if any) is to be taken on 30th June and 31st December, respectively

Make a cron script for auto-downloading leap seconds file (based on https://developers.redhat.com/blog/2017/02/22/how-to-build-a-stratum-1-ntp-server-using-a-raspberry-pi/ )

nano /usr/local/bin/leap-seconds.sh

#!/bin/sh
/usr/bin/wget –q –N https://www.ietf.org/timezones/data/leap-seconds.list -O /var/lib/ntp/leap-seconds.list

/usr/sbin/service ntp restart &> /dev/null

Once this is done, make sure it is executable:

sudo chmod a+x /usr/local/bin/leap-seconds.sh

Now, create an entry in root’s crontab to check on the 7th June and the 7th December:

sudo crontab -e

0 0 07 6,12 * /usr/local/bin/leap-seconds.sh

Lastly, modify the “/etc/ntp.conf” configuration file once again, and add this near the top of the file:

# leap seconds file
leapfile /var/lib/ntp/leap-seconds.list

Static IP and Hostname

If you want to fix your LAN IP you do it by amending /etc/dhcpcd.conf adding the following lines (adjust to suit your environment):

# It is possible to fall back to a static IP if DHCP fails:
# define static profile

profile static eth0

static ip_address=192.168.1.7/24

static routers=192.168.1.254

static domain_name_servers=8.8.8.8 8.8.4.4

# fallback to static profile on eth0

interface eth0

fallback static_eth0

This way DHCP will work if plugged into a client switch port, but static IP will default when no DHCP is available.

Great for testing the box on one’s desktop before going live.

Amend your hostname by editing /etc/hostname and then adding the below to /etc/hosts.

i.e if you call your machine ‘ntp’ fix the /etc/hosts 127.0.1.1 line:

127.0.1.1 ntp

April 2019
Mon	Tue	Wed	Thu	Fri	Sat	Sun
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30

April 2019

Saturday, April 27, 2019

A Raspberry Pi Stratum 1 NTP Server

Overview

A Note About Accuracy

Required Components

Prerequisite Settings

Enabling Galileo Satellites and Setting Stationary Mode

Verifying that PPS Is Working

Enabling PPS/ATOM Support in NTPD

GPS Offset Tuning

Automatically updating GPS leap seconds

Static IP and Hostname

Further Reading