Latency and Packet Loss Spikes for Months, Are These Power Levels Normal?

[StillHoping]

Hey everyone, for the last few months we have been having very high packet loss and latency spikes (ranging from only a few seconds to a few hours at a time). For a quick example here is the connection today to OpenDNS. With average latency of around 900ms during the window (the graph is for 3 hours).

Before and after will be okay with latency of between 10 and 21ms. Local traffic is completely fine. The box is a SuperHub 3 and has been running in Modem Mode for years.

I have been trying everything and speaking to Virgin Media but as the issue is intermittent they are not interested in investigating. One final thought I have is that it might be related to power levels. I don't really know anything but I feel the T3 timeouts look a bit high. What do people think? This was also during a period of the connection working, will try and capture it during a spike next time. Also, thank you so much for any time you spent on this. It has been driving me insane.

Downstream bonded channels

Channel Frequency (Hz) Power (dBmV) SNR (dB) Modulation Channel ID

1	331000000	5.4	38	256 qam	25
2	339000000	5.5	38	256 qam	26
3	347000000	5.5	38	256 qam	27
4	355000000	5	38	256 qam	28
5	363000000	5.1	38	256 qam	29
6	371000000	5	38	256 qam	30
7	379000000	5.5	38	256 qam	31
8	387000000	5.5	38	256 qam	32
9	395000000	5.4	38	256 qam	33
10	403000000	5.6	38	256 qam	34
11	411000000	5.5	38	256 qam	36
12	419000000	5.1	38	256 qam	37
13	427000000	5.1	38	256 qam	38
14	435000000	4.3	38	256 qam	39
15	443000000	4.1	37	256 qam	35
16	451000000	4.1	38	256 qam	40
17	459000000	3.9	37	256 qam	41
18	467000000	3.7	37	256 qam	42
19	475000000	3.4	37	256 qam	43
20	483000000	2.7	37	256 qam	44
21	491000000	2.7	37	256 qam	45
22	499000000	2.5	37	256 qam	46
23	507000000	2.2	37	256 qam	47
24	515000000	2.5	38	256 qam	48

Downstream bonded channels

Channel Locked Status RxMER (dB) Pre RS Errors Post RS Errors

1	Locked	38.9	12	4
2	Locked	38.6	18	3
3	Locked	38.6	10	0
4	Locked	38.6	0	0
5	Locked	38.6	15	0
6	Locked	38.9	15	0
7	Locked	38.6	10	0
8	Locked	38.6	7	0
9	Locked	38.6	6	0
10	Locked	38.9	5	0
11	Locked	38.6	5	0
12	Locked	38.6	0	0
13	Locked	38.6	5	0
14	Locked	38.6	1	0
15	Locked	37.3	5	0
16	Locked	38.6	6	0
17	Locked	37.6	8	0
18	Locked	37.6	8	0
19	Locked	37.6	5	0
20	Locked	37.6	5	0
21	Locked	37.6	4	0
22	Locked	37.6	23	0
23	Locked	37.6	2	0
24	Locked	38.6	7	0

 

Upstream bonded channels

Channel Frequency (Hz) Power (dBmV) Symbol Rate (ksps) Modulation Channel ID

1	49600000	49	5120	64 qam	1
2	36600000	48.8	5120	64 qam	3
3	23600068	48	5120	16 qam	5
4	30100000	48.3	5120	32 qam	4
5	43100000	48.8	5120	64 qam	2

Upstream bonded channels

Channel Channel Type T1 Timeouts T2 Timeouts T3 Timeouts T4 Timeouts

1	ATDMA	0	0	4	0
2	ATDMA	0	0	2	0
3	ATDMA	0	0	10	0
4	ATDMA	0	0	3	0
5	ATDMA	0	0	4	0

[Beth_G]

Hi StillHoping

Thanks for your post, welcome to the Community Forums! Sorry to hear you're having trouble with your broadband service.

We've struggled to locate the backend of your services from here to be able to look into this more closely, however I've dropped you a message so I can take your account details and look on our side. We can run further diagnostic checks from there.

Thanks

[Beth_G]

Hi StillHoping

Thanks for providing your account details via PM.

I've had a look and your signal levels all look fine from here, however we haven't been able to run further diagnostic checks as it looks like you're in modem mode. 

If possible, can you please hop out of modem mode and into router mode so we can run further tests for you?

Thanks😊

[legacy1]

I been very disappointed if you don't find anything wrong in router mode😞 

[StillHoping]

Hi @Beth_G, thank you for having a look. We will look at switching over to the router mode of the super hub 3 so you can run more complete tests after the Christmas period is over.

However I have had the opportunity to set up a more complete testing suite on our side and after analysing some local traffic I can replicate the latency spike. Here is an example of a persistent latency issue that we saw above.

And here is a triggered latency issue for a few minutes:

The latter of which occurred while continuously uploading to the internet (I have tested this locally and there is no latency impact when copying to/from a local server at 350Mbps).

After which I can across this buffer bloat test showing just how extreme the increase in latency is while uploading:
https://www.waveform.com/tools/bufferbloat?test-id=52c72849-475c-406f-bca6-35f4b779493c

We are on the M500 package (so in theory 52Mbps upload) but something we see only on occasion.

In theory SQM (which we have and is "automagically" enabled) should help but with such varying bandwidth it seems pretty difficult to solve. Maybe something like `autorate-ingress` in openwrt would help but this is definitely a plaster to cover a bigger upstream problem.

[Martin_N]

Thank you for that information. If you can please let us know when you have been able to take the hub out of modem mode. 

^Martin

[StillHoping]

Thank you @Martin. I will change it on Tuesday (working hours) for you to have a look at (unless Saturday/Sunday/Monday works in which case let me know).

Just coming back to share some additional latency graphs from today which has had two particular bad periods (during which time there was limited local traffic on the network). The below graphs are for Facebook but true for any outbound traffic.

Also would encourage anyone who is facing similar issues to use SmokePing for long term monitoring (with a very low polling time, we are using 3 pings every minute).

[StillHoping]

The hub has been switched to router mode. Thank you for looking into it and please let me know when you are finished!

[Beth_G]

Hi StillHoping

Thanks so much for popping into router mode for us. We've had a look at the data across Samknows and our internal systems and we can't seem to see any issues at all. All your power levels are in spec. Your speeds to the Hub have been very consistent and there's been no disconnections logged over the past week.

We really appreciate you providing the data recorded from SmokePing, however would you please be able to start a Broadband Quality Monitor graph so we can monitor your broadband performance that way?

Let us know 😊

[StillHoping]

Hi Beth_G, thank you for investigating. We switched back to our router and we can't use BQM as it refuses to respond to ICMP from WAN.

However we have been experiencing very high latency today and I am switching back to the router as we speak. If you could investigate that would be appreciated.

For some quick SmokePing comparisons (Google):

Router:

The modem has a very high latency for some reason (connecting to 192.168.100.1):

This actually makes me think this might be something with the Superhub 3. Let me know if you have any thoughts.