I have a security camera which synchronises its time to an NTP server. Recently I've noticed that the log is full of no reply received in time errors. I've tried various NTP servers, including Virgin's own, all with the same result. Is Virgin somehow throttling NTP?
Here's what I get when I run ntpq on it after a few hours.
PS C:\tools\ntp\bin> ntpq -p
remote refid st t when poll reach delay offset jitter
+22.214.171.124 126.96.36.199 2 u 104 128 377 19.857 -2.635 1.459
+ntp.teaparty.ne 188.8.131.52 2 u 58 128 377 17.855 -3.168 1.335
*188-39-213-7.st .GPS. 1 u 61 128 377 24.405 -3.756 2.694
-184.108.40.206 (a 220.127.116.11 2 u 108 128 377 23.140 -1.410 0.824
-thuis.bentware. 18.104.22.168 2 u 119 128 377 27.444 -2.253 0.439
The daemon is polling several ntp servers from uk.pool.ntp.org once every 128 seconds.
A reachability statistic of 377 means there have been no packets dropped from these servers in the last 8 polls of the NTP servers.
Note that I did drop one packet later on as can be seen here
Checking current status of NTP service with ntpq -p
remote refid st t when poll reach delay offset jitter
+22.214.171.124 126.96.36.199 2 u 45 64 377 19.665 -2.454 1.371
+ntp.teaparty.ne 188.8.131.52 2 u - 64 377 17.492 -2.785 2.213
*188-39-213-7.st .GPS. 1 u 65 64 357 24.504 -3.035 0.367
-184.108.40.206 (a 220.127.116.11 2 u 42 64 377 23.077 -1.405 0.622
-thuis.bentware. 18.104.22.168 2 u 60 64 377 27.394 -1.431 0.333
(Auto-Refresh every 10s --- CTRL+C to Cancel)
However this dropped off and the reachability statistic returned to 377 later on.
Only use Helpful answer if your problems been solved.
I use Cisco VIRL for my job which needs to contact NTP servers as part of the checks. Unless I run a VPN these requests are blocked by Virgin Media's network.
I think this change was made about 12 months ago because it previously used to work fine. Kind of irritating.
I always have a problem with the assumptions inherent in statements such as these.
There are actually a number of reasons why the connection might fail, and they're not all down to deliberate action by Virgin Media.
That said, I'd like to get to help get to the bottom of this, and if it is an issue on VM's side - try to get it resolved.
Are you able to run a tracert to the NTP servers?
Once without the VPN Once with the VPN
So we can have a look and see if there are any issues with the route.
While I appreciate the reply, not everyone out there who posts on these forums is a know-nothing know-it-all who always blames Virgin Media, and I did a bunch of troubleshooting before coming to my conclusion. I work as a network engineer at an ISP/MSP that is larger than Virgin Media (hence why I use VIRL), and while I've worked in this industry long enough to never rule anything out, I am very confident that Virgin Media are blocking the port, or otherwise interfering. Given that this started happening around the time there were a bunch of NTP attacks in the news, I was not too surprised that NTP started (seemingly) being blocked.
I do not believe it is a routing issue in the VM/LG network, as it is perfectly possible to reach the NTP servers via ping, and tracert shows no issues either (see example below to one of the main pool.ntp.org servers). It seems very apparent that at least this part of the Virgin Media network I am on is interfering with port 123, and I believe this issue first started around October/November 2017, though I may be mistaken.
Tracing route to 22.214.171.124 over a maximum of 30 hops
1 1 ms 1 ms 1 ms 192.168.0.1 2 9 ms 9 ms 7 ms 10.48.68.1 3 13 ms 10 ms 10 ms perr-core-2b-xe-1110-0.network.virginmedia.net [126.96.36.199] 4 * * * Request timed out. 5 * * * Request timed out. 6 16 ms 17 ms 16 ms m686-mp2.cvx1-b.lis.dial.ntli.net [188.8.131.52] 7 17 ms 15 ms 15 ms 184.108.40.206 8 18 ms 17 ms 15 ms h185-145-200-80.reverse.clouvider.net [220.127.116.11] 9 19 ms 26 ms 20 ms h185-145-200-81.reverse.clouvider.net [18.104.22.168] 10 22 ms 17 ms 19 ms 22.214.171.124
Pinging 126.96.36.199 with 32 bytes of data: Reply from 188.8.131.52: bytes=32 time=17ms TTL=55 Reply from 184.108.40.206: bytes=32 time=22ms TTL=55 Reply from 220.127.116.11: bytes=32 time=25ms TTL=55 Reply from 18.104.22.168: bytes=32 time=20ms TTL=55
Ping statistics for 22.214.171.124: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 17ms, Maximum = 25ms, Average = 21ms
The following resolves the issue instantly with no configuration changes:
Using another ISP such as BT. Running over a VPN on the VM network. Tethering using Vodafone/Three.
The following has no impact:
Disabling all firewalls. Changing NTP servers (except to ntp.virginmedia.com which resolves the issue. time.windows.com also seems to work, so possible whitelisting). pool.ntp.org and nist.time.gov are blocked.
The moment I move off from the Virgin Media network, the issue magically resolves itself.
When using a non-VM NTP server aside from the Windows one, it will not sync and gets stuck in the INIT state.
Edit: I also tested uk.pool.ntp.org which works. The main pool.ntp.org does not. It would be interesting to see if your daemon stops working if you point it elsewhere.
You may believe that you've narrowed it down to being a network level block by Virgin, however pool.ntp.org works perfectly over Virgin for me, as does the IP address you've pinged in your response, 126.96.36.199.
Following is a paste from my local development server (over VIVID350):
Given that this topic was created due to another customer experiencing NTP issues, seemingly the same very specific and somewhat obscure error, one should at least be willing to entertain the idea that something has been configured either by accident or design somewhere that is causing issues with NTP for at least parts of the Virgin Media network. Your post comes dangerously close to "it's working for me so therefore it must be working everywhere", which ironically enough would be a far bigger assumption than anything I have suggested.
I submit that doing a large amount of troubleshooting to eliminate essentially every other possible cause is not jumping to conclusions. If you want to be really pedantic, then fine. "Based on my troubleshooting listed below, it looks like Virgin Media may be interfering with NTP on my segment of the network, as I seem to have checked all the other likely causes". Is that sufficiently non-absolute wording?
Tim did not have the luxury of knowing what troubleshooting I had performed when making the original response, but given the large body of checks I have done, I am confident there is a lot of evidence to make my claim a very strong hypothesis. At the end of the day, I do get paid well to troubleshoot a very large network as quickly and accurately as possible. I may very well be wrong on this occasion (and others), but I do make my living by being right, and having some idea about what I'm talking about.