Building a Stratum 1 NTP Server with a Raspberry Pi 4 and Adafruit Ultimate GPS Hat
I’m going to be honest with you. There’s a lot of posts on the internet on how to do this, but there’s a lot of misinformation out there to. My goal in this post is to give you what you need to get this setup, explain why you need to do what you need to do, and give you the tools you’ll need to to study up if you want to know more. In the past week, I’ve been troubleshooting a GPS hat and I’ve built an NTP server on a fresh Raspbian build at least 5 times to perfect this, and yesterday I ran through it again writing down the steps as I went. That’s where this how-to came from.
There’s two trains of thought on how to do this post-RPi2; software UART or hardware UART. Software UART is if you still want to use bluetooth on the RPi. I don’t know why you would want to use bluetooth on your NTP server, but some folks might want to. This will not be for them. I’m in the hardware UART crowd.
The Ultimate GPS hat delivers its data over a serial port to GPIO pins 14 and 15 at 9600 bps. On the RPi2, this went directly to the hardware UART, but on the RPi 3 and 4, the hardware UART is taken up by the bluetooth subsystem, and the serial port for pins 14 and 15 is emulated in software. Luckily, we can disable bluetooth and use the hardware UART for the GPS. So, let’s get started.
Parts List and Cost:
Raspberry Pi 4 2GB: $34.99 (The 1GB version will work fine, I just had the 2GB version laying around)
Adafruit Industries Ultimate GPS HAT: $44.99
Waterproof GPS Active Antenna 28dB Gain: $14.99 (Optional, but you’ll really need this for best results)
Raspberry Pi 4 Power Supply with ON/Off Switch: $9:99
SanDisk Ultra 32GB microSDHC: $7.99
Total Cost: $112.95 (you can usually find things cheaper too). Compare this to commercial NTP servers that range from $1500 to $5000+
I’m going to assume that this is a fresh build and you’re doing this headless and over wifi. I also assume you’re using Raspbian. The build I used for this how-to was 2020-02-13-raspbian-buster-lite, but unless something major changes with Raspbian, it should work with any build of Raspbian. I also use apt rather than apt-get because I like the little progress bar at the bottom of the screen, but you can use apt-get if you like. I also use nano as my editor. If you’re one of those people that’s into self harm and you want to use vi, more power to you. Without further ado…
1. Burn the Raspbian image to an SD card
2. Add a blank ssh file and your wpa_supplicant.conf to /boot. (standard stuff for headless RPi access)
3. Put a battery in the hat, attach the hat, plug in your SD card, and either attach an external active GPS antenna or just put the whole RPi outside and power it up.
When I was writing out the instructions for this yesterday, I just plugged my RPi4 in and set it on my back patio for about 30 minutes. You really can’t do the later steps until the GPS has a fix. You’ll know it has a fix when the red LED stops blinking once per second and starts giving you a brief flash every 15 seconds.
While we’re waiting for a fix, go ahead and ssh to raspberrypi.local using your favorite terminal program (I’m on a Mac, so I use terminal, if you’re on Windows, I recommend Putty). Let’s get this thing updated while waiting…
sudo apt update
sudo apt upgrade
Once we have a GPS fix, we’ll move forward. The first thing we want to do is disable the console getty programs. We’ll be wanting to use /dev/ttyAMA0, and they’re currently using them. While we’re at it, we’re also going to disable the hciuart service, as it usually attempts to talk to the UART.
sudo systemctl stop serial-getty@ttyAMA0.service
sudo systemctl disable serial-getty@ttyAMA0.service
sudo systemctl disable hciuart
Even though we’ve stopped the console from starting, we need to stop the kernel from trying to use it. We edit /boot/cmdline.txt and remove the console.
sudo nano /boot/cmdline.txt
remove this: console=serial0,115200
Now we’ll need to actually disable bluetooth and take over the hardware UART. This will allow us to use /dev/ttyAMA0 for our GPS. While we’re in here, we’re also going to enable the PPS pin, which is GPIO pin 4 and disable power saving. We’re just doing this now so we don’t have to reboot again later.
sudo nano /boot/config.txt
#At the bottom of the file, add the following:# Use the /dev/ttyAMA0 UART GPS instead of Bluetooth
dtoverlay=disable-bt# enable GPS PPS
dtoverlay=pps-gpio,gpiopin=4# Disable power saving
nohz=off
We also need to clean up one more thing before we move on. DHCP can be configured to deliver NTP server info on some networks, but that doesn’t work very well with NTP servers themselves. We want to make sure that this doesn’t interfere with us, so we’ll disable it. If we don’t, it could cause your ntp.conf file to be edited or ignored completely.
sudo rm /etc/dhcp/dhclient-exit-hooks.d/ntp
sudo rm /lib/dhcpcd/dhcpcd-hooks/50-ntp.confsudo nano /etc/dhcp/dhclient.conf
In the “request” block, remove dhcp6.sntp-servers and ntp-servers
Delete the highlighted options
Finally, we want to change our CPU scaling governor settings to keep the CPU set to the maximum speed for continuous usage. Normally enabling power saving features is a good thing: it saves you power. But when your CPU changes power saving modes, the impact on PPS timing is noticeable. For some reason the NO_HZ kernel mode has a similar bad effect on timekeeping. We disabled nohz earlier in the /boot/config.txt file and to change the scaling governor we need to do the following:
sudo nano /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
replace ondemand with performance
Now that we’ve edited those files, removed the DHCP configurations, and set our performance level, we need to reboot.
sudo reboot
Give it a couple minutes, then SSH back in and let’s check to see if we have communication from the GPS on /dev/ttyAMA0
sudo cat /dev/ttyAMA0
You should see something like this:
Great, we have communication between the RPi and the GPS Hat! Awesome! Now, let’s add some tools to make this whole thing work.
sudo apt install gpsd gpsd-clients python-gps pps-tools ntp
GPSD is the service we’re going to use to decode the NMEA data coming from the GPS. Before it will work, we need to edit it’s configuration file. You’ll want the options to match the options below. We’re not using USB GPS, so we can turn that off, the devices are the /dev/ttyAMA0 that is the UART we stole from bluetooth, and the /dev/pps0 that we requested earlier for pin 4 in the /boot/config.txt file, and the -n option tells GPSD to start talking to the GPS device on startup and not to wait for the first client to attach.
sudo nano /etc/default/gpsd
START_DAEMON=”true”
USBAUTO=”false”
DEVICES=”/dev/ttyAMA0 /dev/pps0″
GPSD_OPTIONS=”-n”
After we save the config file, we need to restart the gpsd service so it can pick up the config.
sudo systemctl restart gpsd
Once gpsd is restarted, we’ll run gpsmon to see how we’re looking.
gpsmon
You should see something like this:
YAY! Your GPS is now passing data and GPSD is processing that data properly, but this is only half the battle. You should see PPS offsets in the gpsmon window, but to verify we have good communication on /dev/pps0, we run the following command.
sudo ppstest /dev/pps0
The output should look something like this:
Great, we have a good PPS signal. The GPS is working, PPS is working, now all that’s left to do is to edit the ntp.conf file and add some pretty important stuff. Before we do that, I want to explain a few things as to why we’re going to do what we do.
NTP gets precise time from GPSD via a shared memory driver. That shared memory driver uses the magic pseudo-IP address of 127.127.28.X. 127.127.28.0 identifies unit 0 of the ntpd shared-memory driver (NTP0); 127.127.28.1 identifies unit 1 (NTP1). Unit 0 is used for in-band message timestamps and unit 1 for the (more accurate, when available) time derived from combining in-band message timestamps with the out-of-band PPS synchronization pulse. Splitting these notifications allows ntpd to use its normal heuristics to weight them.
Different units – 2 (NTP2) and 3 (NTP3), respectively – must be used when gpsd is not started as root. We’ve told our GPS HAT to put PPS time on GPIO pin 4, so will also use unit 2 (NTP2) for the PPS time correction. You can verify this by running the command ntpshmmon and it will show you that NTP2 is our primary shared memory clock source. Run that command as sudo, and you should see NTP0 and NTP1 show up as well.
Another thing to note is that even though you’re building a highly accurate GPS based stratum 1 NTP server, you’re going to want need than one time source. If something happens to the GPS, the antenna breaks, or something else, it’s best to have a few sources and let NTP handle the rest. I recommend adding servers that are close to you, and having a few of them available.
Now, we’ll get on with editing the ntp.conf file and adding a few NTP servers as well as a log file, our PPS reference and our GPS reference. Your ntp.conf file should look something like this:
sudo nano /etc/ntp.conf
# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help
driftfile /var/lib/ntp/ntp.drift
logfile /var/log/ntp.log# Leap seconds definition provided by tzdata
leapfile /usr/share/zoneinfo/leap-seconds.list# Enable this if you want statistics to be logged.
statsdir /var/log/ntpstats/statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable# You do need to talk to an NTP server or two (or three).
#server ntp.your-provider.example
server time.nist.gov iburst minpoll 5 maxpoll 5
server tick.usno.navy.mil iburst minpoll 5 maxpoll 5
server 0.us.pool.ntp.org iburst minpoll 5 maxpoll 5
server 1.us.pool.ntp.org iburst minpoll 5 maxpoll 5
server 2.us.pool.ntp.org iburst minpoll 5 maxpoll 5
server 3.us.pool.ntp.org iburst minpoll 5 maxpoll 5# GPS PPS reference (NTP2)
server 127.127.28.2 minpoll 4 maxpoll 4 prefer
fudge 127.127.28.2 refid PPS# GPS Serial data reference (NTP0)
server 127.127.28.0 minpoll 4 maxpoll 4
fudge 127.127.28.0 time1 0.500 refid GPS# pool.ntp.org maps to about 1000 low-stratum NTP servers. Your server will
# pick a different set every time it starts up. Please consider joining the
# pool: <http://www.pool.ntp.org/join.html>
#pool 0.debian.pool.ntp.org iburst
#pool 1.debian.pool.ntp.org iburst
#pool 2.debian.pool.ntp.org iburst
#pool 3.debian.pool.ntp.org iburst# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
# details. The web page <http://support.ntp.org/bin/view/Support/AccessRestrictions>
# might also be helpful.
#
# Note that “restrict” applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.# By default, exchange time with everybody, but don’t allow configuration.
restrict -4 default kod notrap nomodify nopeer noquery limited
restrict -6 default kod notrap nomodify nopeer noquery limited# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1# Clients from this (example!) subnet have unlimited access, but only if
# cryptographically authenticated.
#restrict 192.168.123.0 mask 255.255.255.0 notrust# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255
As you can see, I’ve added a log file at the top to send logs to /var/log/ntp.log. I also enabled the statsdir to /var/log/ntpstats/. I added 6 NTP servers to make sure I had quite a bit of redundancy, but you can add however many you like. I’d suggest a minimum of 3. I added the minpoll 5 and maxpoll 5 because by default, ntp polls remote servers every 64 seconds, but Linux by default only keeps an ARP table entry for 60 seconds. If the ARP table has flushed the entry for a remote peer or server then when the NTP server sends a request to the remote server an entire ARP cycle will be added to the NTP packet round trip time (RTT). This will throw off the time measurements to servers on the local lan. On a RaspberryPi ARP has been shown to impact the remote offset by up to 600 uSec in some rare cases. The solution is the same for both ntpd and chronyd, add the “maxpoll 5″ command to any ‘server” or “peer directive. This will cause the maximum polling period to be 32 seconds, well under the 60 second ARP timeout.
Next we added our GPS data, first the PPS reference using the server 127.127.28.2, and we’re making PPS our preferred server. Next, we added the GPS signal from 127.127.28.0. We’re fudging that one by 500 ms as a start because in my experience, the GPS signal is usually around 500ms off. This will need to be tuned for it to be accurate. More information on that later.
Finally, I remarked out all the standard debian pools, as we’re using our own servers. You can leave them in if you want instead of using your own servers.
Now we need to restart the ntp service for it to pick up the config.
sudo systemctl restart ntp
Once NTP restarts, we can check the status by using the ntpq -p (or -pn if you don’t want name resolution) command.
ntpq -p
It will take a few moments for NTP to connect to the servers in your list and sort things out. You’re looking for the little space just to the left of the name or IP.
(blank) Discarded as not valid
x Discarded by the intersection algorithm as a falseticker
– Discarded by the cluster algorithm as an outlier
+ Included by the combining algorithm
# Backup time source
* System peer (This is what we’re looking for)
o Indicates a PPS peer whose driver support is directly compiled into ntpd (NA for us)
Ultimately, you’ll end up with something that looks like this:
And now, you have a working Stratum 1 NTP Server! Your next steps should be to go ahead and configure your RPi properly by setting localization options, giving it a static IP (you’ll definitely want to do this if you’re making an NTP server), and anything else I’ve completely skipped over in the making of this how-to, especially if you haven’t done this already. I would not recommend making this a public server as legacy NTP has some security issues with it. There is a hardened version of NTP called NTPSec that is available for Raspbian, but I haven’t gotten around to messing with it yet. I would assume that the steps would be the same though.
Update: I just installed NTPSec. It removes NTP, and the ntp.conf file looks like it lives at /etc/ntpsec/ntp.conf now, plus the service is obviously called ntpsec rather than ntp. If you want to make your sever publicly available, I’d suggest using ntpsec rather than regular ntp.
Remember earlier where I said that we were fudging the GPS signal by 500ms, but it needed to be tuned? Yeah, well that’s a world all it’s own. As it sits, the time you will receive right now will be just fine, but it you want more accurate time, you can fiddle around with it and tune things to become incredibly accurate. Here’s a link you can use to learn about that tuning. as it’s something that’s a little too deep to get into in this post. There’s more info in the references below as well about tuning,
There is one tool that comes with gpsd, it’s called ntpoffset. It’s mentioned in the link above and can be found in /usr/share/doc/gpsd-clients/examples/ for those of you that want to play with it (check out the README in that directory too) . If you’re going to try to tune this thing, I would recommend removing the 500ms fudge and letting it settle to get an accurate offset number, at least a day to be safe. I’m doing that right now myself. If you don’t mind, please let me know in the comments what your offset comes out to and how long you let it settle for before running the ntpoffset script. You won’t have to create the directory or chown it as said in the tuning link above, it will already be there for you. Just run the script and let me know your offset. Also, check periodically to see if your offset is changing. The script gives you an AVERAGE, and it’s probably going to change. Remember, once you set the offset (the fudge), that offset is going to change by number of your fudge, so if you set it to 0.500, then let it run for a day and the actual offset needed to be 0.540, the script is going to tell you it’s now -40.xxx. If you set it to 0.500 and let it run for a bit, then changed it to 0 and let it run for, say a couple hours and the offset is really 0.540, the ntpoffset script is going to spit out something like -220.xxx. In the image below (which it’s incorrect because I removed the fudge from the config and let it run for a short while, so it’s lower than it should be), you’ll see that the number is -465.353, so my fudge time1 number would be 0.465. In the live ntpq screen, it would be 0.553. If your offset is positive, say if ntpoffset gives you 465.353 (no negative sign), then your fudge time1 would be -0.465. Got it? Told you that it’s a world all it’s own…
I hope this help clear up some of the confusion out there and help some folks out. Be sure to check out the references below. I couldn’t have done this without them.
Resources:
BIPM 2018 Annual Report: Scroll to page 65 for the Time Dissemination Services section. This contains the NTP servers of the National Metrology Laboratories of countries around the world and is a great resource for other stratum 1 NTP servers, many of which are updated directly from atomic sources.
NTPSec.org
References:
Steve Friedl’s Unixwiz.net Tech Tips: Building a GPS Time Server with the Raspberry Pi 3
Gary E. Miller and Eric S. Raymond: GPSD Time Service HOWTO
David Taylor @ Satsignal.eu: The Raspberry Pi as a Stratum-1 NTP Server
Ax0n’s Den: Stratum-1 NTP Server
Adafruit: Adafruit Ultimate GPS HAT for Raspberry Pi