Hey everyone, for the last few months we have been having very high packet loss and latency spikes (ranging from only a few seconds to a few hours at a time). For a quick example here is the connection today to OpenDNS. With average latency of around 900ms during the window (the graph is for 3 hours).Before and after will be okay with latency of between 10 and 21ms. Local traffic is completely fine. The box is a SuperHub 3 and has been running in Modem Mode for years.I have been trying everything and speaking to Virgin Media but as the issue is intermittent they are not interested in investigating. One final thought I have is that it might be related to power levels. I don't really know anything but I feel the T3 timeouts look a bit high. What do people think? This was also during a period of the connection working, will try and capture it during a spike next time. Also, thank you so much for any time you spent on this. It has been driving me insane.Downstream bonded channelsChannel Frequency (Hz) Power (dBmV) SNR (dB) Modulation Channel ID13310000005.438256 qam2523390000005.538256 qam2633470000005.538256 qam274355000000538256 qam2853630000005.138256 qam296371000000538256 qam3073790000005.538256 qam3183870000005.538256 qam3293950000005.438256 qam33104030000005.638256 qam34114110000005.538256 qam36124190000005.138256 qam37134270000005.138256 qam38144350000004.338256 qam39154430000004.137256 qam35164510000004.138256 qam40174590000003.937256 qam41184670000003.737256 qam42194750000003.437256 qam43204830000002.737256 qam44214910000002.737256 qam45224990000002.537256 qam46235070000002.237256 qam47245150000002.538256 qam48Downstream bonded channelsChannel Locked Status RxMER (dB) Pre RS Errors Post RS Errors1Locked38.91242Locked38.61833Locked38.61004Locked38.6005Locked38.61506Locked38.91507Locked38.61008Locked38.6709Locked38.66010Locked38.95011Locked38.65012Locked38.60013Locked38.65014Locked38.61015Locked37.35016Locked38.66017Locked37.68018Locked37.68019Locked37.65020Locked37.65021Locked37.64022Locked37.623023Locked37.62024Locked38.670 Upstream bonded channelsChannel Frequency (Hz) Power (dBmV) Symbol Rate (ksps) Modulation Channel ID14960000049512064 qam123660000048.8512064 qam332360006848512016 qam543010000048.3512032 qam454310000048.8512064 qam2Upstream bonded channelsChannel Channel Type T1 Timeouts T2 Timeouts T3 Timeouts T4 Timeouts1ATDMA00402ATDMA00203ATDMA001004ATDMA00305ATDMA0040

Hi StillHoping Thanks for your post, welcome to the Community Forums! Sorry to hear you're having trouble with your broadband service. We've struggled to locate the backend of your services from here to be able to look into this more closely, however I've dropped you a message so I can take your account details and look on our side. We can run further diagnostic checks from there. Thanks

Hi StillHoping Thanks for providing your account details via PM. I've had a look and your signal levels all look fine from here, however we haven't been able to run further diagnostic checks as it looks like you're in modem mode. If possible, can you please hop out of modem mode and into router mode so we can run further tests for you? Thanks😊

I been very disappointed if you don't find anything wrong in router mode😞

Hi @Beth_G, thank you for having a look. We will look at switching over to the router mode of the super hub 3 so you can run more complete tests after the Christmas period is over.However I have had the opportunity to set up a more complete testing suite on our side and after analysing some local traffic I can replicate the latency spike. Here is an example of a persistent latency issue that we saw above.And here is a triggered latency issue for a few minutes:The latter of which occurred while continuously uploading to the internet (I have tested this locally and there is no latency impact when copying to/from a local server at 350Mbps).After which I can across this buffer bloat test showing just how extreme the increase in latency is while uploading:https://www.waveform.com/tools/bufferbloat?test-id=52c72849-475c-406f-bca6-35f4b779493cWe are on the M500 package (so in theory 52Mbps upload) but something we see only on occasion. In theory SQM (which we have and is "automagically" enabled) should help but with such varying bandwidth it seems pretty difficult to solve. Maybe something like `autorate-ingress` in openwrt would help but this is definitely a plaster to cover a bigger upstream problem.

Thank you for that information. If you can please let us know when you have been able to take the hub out of modem mode. ^Martin

StillHoping

Tuning in

2 years ago

Latency and Packet Loss Spikes for Months, Are These Power Levels Normal?

Hey everyone, for the last few months we have been having very high packet loss and latency spikes (ranging from only a few seconds to a few hours at a time). For a quick example here is the connection today to OpenDNS. With average latency of around 900ms during the window (the graph is for 3 hours).

Before and after will be okay with latency of between 10 and 21ms. Local traffic is completely fine. The box is a SuperHub 3 and has been running in Modem Mode for years.

I have been trying everything and speaking to Virgin Media but as the issue is intermittent they are not interested in investigating. One final thought I have is that it might be related to power levels. I don't really know anything but I feel the T3 timeouts look a bit high. What do people think? This was also during a period of the connection working, will try and capture it during a spike next time. Also, thank you so much for any time you spent on this. It has been driving me insane.

Downstream bonded channels

Channel Frequency (Hz) Power (dBmV) SNR (dB) Modulation Channel ID

1	331000000	5.4	38	256 qam	25
2	339000000	5.5	38	256 qam	26
3	347000000	5.5	38	256 qam	27
4	355000000	5	38	256 qam	28
5	363000000	5.1	38	256 qam	29
6	371000000	5	38	256 qam	30
7	379000000	5.5	38	256 qam	31
8	387000000	5.5	38	256 qam	32
9	395000000	5.4	38	256 qam	33
10	403000000	5.6	38	256 qam	34
11	411000000	5.5	38	256 qam	36
12	419000000	5.1	38	256 qam	37
13	427000000	5.1	38	256 qam	38
14	435000000	4.3	38	256 qam	39
15	443000000	4.1	37	256 qam	35
16	451000000	4.1	38	256 qam	40
17	459000000	3.9	37	256 qam	41
18	467000000	3.7	37	256 qam	42
19	475000000	3.4	37	256 qam	43
20	483000000	2.7	37	256 qam	44
21	491000000	2.7	37	256 qam	45
22	499000000	2.5	37	256 qam	46
23	507000000	2.2	37	256 qam	47
24	515000000	2.5	38	256 qam	48

Downstream bonded channels

Channel Locked Status RxMER (dB) Pre RS Errors Post RS Errors

1	Locked	38.9	12	4
2	Locked	38.6	18	3
3	Locked	38.6	10	0
4	Locked	38.6	0	0
5	Locked	38.6	15	0
6	Locked	38.9	15	0
7	Locked	38.6	10	0
8	Locked	38.6	7	0
9	Locked	38.6	6	0
10	Locked	38.9	5	0
11	Locked	38.6	5	0
12	Locked	38.6	0	0
13	Locked	38.6	5	0
14	Locked	38.6	1	0
15	Locked	37.3	5	0
16	Locked	38.6	6	0
17	Locked	37.6	8	0
18	Locked	37.6	8	0
19	Locked	37.6	5	0
20	Locked	37.6	5	0
21	Locked	37.6	4	0
22	Locked	37.6	23	0
23	Locked	37.6	2	0
24	Locked	38.6	7	0

Upstream bonded channels

Channel Frequency (Hz) Power (dBmV) Symbol Rate (ksps) Modulation Channel ID

1	49600000	49	5120	64 qam	1
2	36600000	48.8	5120	64 qam	3
3	23600068	48	5120	16 qam	5
4	30100000	48.3	5120	32 qam	4
5	43100000	48.8	5120	64 qam	2

Upstream bonded channels

Channel Channel Type T1 Timeouts T2 Timeouts T3 Timeouts T4 Timeouts

1	ATDMA	4
2	ATDMA	2
3	ATDMA	10
4	ATDMA	3
5	ATDMA	4

Beth_G
Forum Team
2 years ago
Hi StillHoping

Thanks for your post, welcome to the Community Forums! Sorry to hear you're having trouble with your broadband service.

We've struggled to locate the backend of your services from here to be able to look into this more closely, however I've dropped you a message so I can take your account details and look on our side. We can run further diagnostic checks from there.

Thanks
Beth_G
Forum Team
2 years ago
Hi StillHoping

Thanks for providing your account details via PM.

I've had a look and your signal levels all look fine from here, however we haven't been able to run further diagnostic checks as it looks like you're in modem mode.

If possible, can you please hop out of modem mode and into router mode so we can run further tests for you?

Thanks😊
legacy1
Alessandro Volta
2 years ago
I been very disappointed if you don't find anything wrong in router mode😞
StillHoping
Tuning in
2 years ago
Hi @Beth_G, thank you for having a look. We will look at switching over to the router mode of the super hub 3 so you can run more complete tests after the Christmas period is over.
However I have had the opportunity to set up a more complete testing suite on our side and after analysing some local traffic I can replicate the latency spike. Here is an example of a persistent latency issue that we saw above.
And here is a triggered latency issue for a few minutes:
The latter of which occurred while continuously uploading to the internet (I have tested this locally and there is no latency impact when copying to/from a local server at 350Mbps).
After which I can across this buffer bloat test showing just how extreme the increase in latency is while uploading:
https://www.waveform.com/tools/bufferbloat?test-id=52c72849-475c-406f-bca6-35f4b779493c
We are on the M500 package (so in theory 52Mbps upload) but something we see only on occasion.

In theory SQM (which we have and is "automagically" enabled) should help but with such varying bandwidth it seems pretty difficult to solve. Maybe something like `autorate-ingress` in openwrt would help but this is definitely a plaster to cover a bigger upstream problem.
- Martin_N
  Forum Team
  2 years ago
  Thank you for that information. If you can please let us know when you have been able to take the hub out of modem mode.
  
  ^Martin
  - StillHoping
    Tuning in
    2 years ago
    Thank you @Martin. I will change it on Tuesday (working hours) for you to have a look at (unless Saturday/Sunday/Monday works in which case let me know).
    
    Just coming back to share some additional latency graphs from today which has had two particular bad periods (during which time there was limited local traffic on the network). The below graphs are for Facebook but true for any outbound traffic.
    
    Also would encourage anyone who is facing similar issues to use SmokePing for long term monitoring (with a very low polling time, we are using 3 pings every minute).
StillHoping
Tuning in
2 years ago
Testing over the weekend and it does look like there is something happening on the hub occasionally which causes periods of slow packet processing and/or packets drops.
It looks similar to this issue where a hub replacement solved the issue: https://community.virginmedia.com/t5/Networking-and-WiFi/Hub3-in-modem-mode-ping-spikes-and-packet-loss/td-p/5384022/page/2

However the good news it that whenever latency to the modem increases, it can just be restarted and it will be okay until the process on the SuperHub 3 starts again (and then it is just a reboot away from being okay). This is frustrating but at least there is a workable solution in the meantime.
- Steven_L
  Forum Team
  2 years ago
  Thanks for coming back to us StillHoping, I can see that there is an SNR fault logged in your area at the moment. This has been open since the 4th January and has a current estimated fix time of 12.30 on Thursday 11th January.
  
  Kind Regards,
  
  Steven_L
  - jpeg1
    Alessandro Volta
    2 years ago
    The data you gave in your first post showed QAM level errors on the upstream, and many T3 errors. I'm slightly surprised that these weren't commented on.
    It would be helpful if you posted the Hub data again when the reported fault has been cleared.