[Fixed] NTP on VMs

[MartinR]

My public facing workstation is running CentOS 6.8. For some time now I have had a couple of VMs running in the background, one at C7 and one running Fedora 23. Each evening when I shut down the workstation it suspends the VMs, and in the morning they restart happily enough. However after the Christmas/New Year break I realised that time had slipped backwards several weeks on the VMs. I narrowed it down to NTP failing, which in turn was due to the host's firewall. I duly opened the port and all seemed fine until the following morning when time had slipped back half a day again. The client NTP daemons were showing zero for the reach and a hyphen for the when. Even after a couple of hours (poll had got up to 1024 by this stage) they hadn't resynced. I manually restarted them, but the following day the same scenario presented itself. I've now modified ntp.conf as follows:

Code: Select all

  ...
server 192.168.122.1 minpoll 3 maxpoll 6
  ...
# Allow for large steps when the host is shutdown or rebooted.
#
tinker panic 0

This morning both had frozen time overnight. The update job contains ssh <name> date, ssh <name> yum -y update (dnf for the F23 VM) and ssh <name> sync ; sync. Even after the update NTP was still frozen. When I ssh'ed in interactively (about 20 minutes after bootup) NTP on the C7 VM then started reaching the host, and in due course jumped back to the present; the Fedora machine also started reaching the hist, but refused to sync and needed a daemon restart.

All three machines (host + clients) are updated daily.

This is a test/development machine, not a server, so the priority is low, more of an annoyance. Does anyone have any ideas on kicking NTP on the VMs into life? Is there a way to make the F23 VM jump when time changes significantly?

Thanks,
Martin

[aks]

By default ntp/chrony installations will not re-sync if they are to far out - it is one of the config. options (to determine the range), but can not remember which. Set the date/time on each VM (observing TZ differences if applicable) on each VM. Get them to sync to the ntp server, suspend (or whatever) and see if they come back fairly close to each other.
I usually have been 1 and 3 ntp servers (depending on what my needs are) that connect to the Internet for time and then all vms connect to that one. Seems to work quite well.

[MartinR]

The option you are thinking of is probably -g, but significantly

however, this can happen only once. If the threshold is exceeded after that, ntpd will exit ... See the tinker command for other options.

which is why I set tinker panic 0, see ntp_misc(5)

Spedifies[sic] the panic threshold in seconds with default 1000 s. If set to zero, the panic sanity check is disabled and a clock offset of any value will be accepted.

The host machine comes up synchronised to the CentOS default pool, the problem is with the VMs synchronising to their host.

My main machines do as you suggest and have three servers. One (time1, which is a CNAME defined in my DNS) talks to three intranet sources. Time1, along with time2 and time3 form a peer group, each with an appropriate tos orphan setting. All my other machines take time from these three. Main system machines rarely go down (I hope!) and VMs have a shorter life than their servers. My workstation though is different since it is shutdown each evening, hence the problems.

If I can't fix it I may just include a ssytemctl restart ntpd in the daily update check. A cludge, but as I said this is an annoyance rather than critical!

[MartinR]

For general information, I've fixed this problem by a restart each day. The enclosed script is a bit of a hack, but it works. Just dump it in ~/bin and assuming that your path includes ~/bin just run the script. You do need to have a ~/bin on the VMs though. The code makes two assumptions:

• * The virtual bridge is called virbr0, but you can change this in the header.
  * You have nmap installed on the host machine.

Code: Select all

#!/bin/sh
#
#	Daily tasks for internet workstation and its VMs.
#
#	The script will scan all remote machines attached to a specified
#	bridge.  this script is copied to /root/bin on each machine and
#	run.  First NTP is restarted to synchronise, then the date and NTP
#	status are reported before attempting to perform a yum/dnf update.
#
#	All option switches are for internal use only.
#
#	Define the bridge name:
bridge=virbr0
#
#==============================================================================

function tasks {
	echo ""
	echo "=============================================================="
	echo "Updating $(hostname) - $(cat /etc/system-release)"
	echo ""
	if [[ $1 != "-l" ]]
	then
		date
		systemctl restart ntpd
		printf "Sleeping ..."
		sleep 1m
	fi
	date
	ntpq -p
	echo ""
	if command -vp dnf >/dev/null
	then
		dnf -y update
	else
		yum -y update
	fi
}

if [[ $1 != "-r" ]]
then
	tasks -l
	ipnet=$( ip addr show virbr0 | awk ' $1 == "inet" {print $2} ' )
	ipaddr=${ipnet%%/*}
	vm_list=$( nmap -sn -n $ipnet | awk '
		$5 == mynode { next }
		$1$2$3 == "Nmapscanreport" {print $5}
	' mynode=$ipaddr - )
	for vnode in $vm_list
	do
		scp $0 $vnode:$0
		ssh $vnode $0 -r
	done
else
	tasks
fi