Hub 3 Modem - High Latency, Dropped Packets, Loss of Sync, T3 Timeouts

[katspike]

I'm using my Hub 3 in Modem mode. It’s been working for several years with no noticeable issues… but for the last week, it started disconnecting around the same time every day. Running tests via the VM website do not reveal any errors. However, my network log shows warnings and critical errors and the Broadband Quality Monitor (BQM) shows high latency and dropped packets.

Every day, the network log shows the following warnings and critical errors between 10 and 12noon, and then at around 4pm. My family uses Internet heavily in the evening, but not as much at midday when it fails. It seems to get worse after each attempt by VM customer service to fix it.

Network Log Summary:

“Critical   - SYNC Timing Synchronization failure - Loss of Sync”

“Warning - RCS Partial Service

"Critical   - No Ranging Response received T3 timeout"

Plus a critical error from 1970 ?!

(See here for an explanation of these errors)

All internet connectivity is lost, and the hub tries to renegotiate channels, sometimes several times a day. The rest of the time, the network is good - even at night when there’s a lot of streaming and gaming activity.

I've read many posts with similar problems, but not many solutions, and everyone's setup is different. I need some help to identify the cause of my issue.  Really appreciate any advice you may have.

Here's a diagram of my setup showing all wired devices:

My Setup: 

• HUB 3.0 (Modem Mode)
• 350M Oomph Plan (broadband, landline, mobile)
• Daytime use: 1-2 adults, W.F.H - video meetings, streaming podcasts, web development
• Night-time use 2 adults, + 2 teens - a mix of video streaming and / or X-box or PS4
• Router is configured as DHCP+NAT
• Daisy-chain: Modem > Router > TP-Link 16 port Gigabit Switch > Cat 5e patch panel > Netgear Gigabit Switches > Airport Express bridge mode wireless networks
• Cat5e / Cat6 ethernet throughout house
• Problem occurs on both Wired and Wireless clients
• Line was installed in 2017

Total Connected (not all used at once):

 20  wired devices  (maximum 12 in concurrent use).
 10  wireless devices
 48  zigbee smart bulbs, sensors, switches and sockets (Hue)

Broadband Quality Monitor (BQM): 

Live Link

Issue:

Some outages are very short, and the frequency of occurrence varies, but it usually fails and reboots itself between 10am and 11am - even though demand is low and not many devices are being used at that time.

 Attempts to rectify: 

• VM have run several tests remotely and on-site
• VM Engineer removed old Tivo splitter and replaced connections - he only considered the Hub - he did not visually inspected anything connected to it, including the coax cable that runs up the side of the house. I live right on the coast where salt-water corrosion is a major issue.
• Confirmed all connections are tight
• VM Engineer saw my BQM, but did not look at network logs
• The modem and router are fan-cooled but temperature can still reach 30 deg C.
• Power Off / On Modem for 5 mins 
• Power off all devices several hours
• Reset Modem
• Disconnected all connected equipment and changed to Router mode
• VM changed frequency and increased something (they weren’t specific)
• Reduced the amount of devices connecting to the network - just 2 mobiles, 1 laptop and Apple TV - still failed
• Increased cooling
• Another engineer is booked

Possible Correlation:

The problem seemed to start around the same time that I added a PoE wireless outdoor AP (Access Point). But when I disconnected the AP it still failed, so I don’t think it’s the cause.

Around this time my neighbour also got Virgin Media installed, but VM say that is not related (I remain sceptical).

The large green junction cabinet on the street nearby has a door that keeps falling off. I have booked an engineer to fix via 0330 333 0444

[Adduxi]

Firstly a simple answer - 1970.  The Unix epoch (or Unix time or POSIX time or Unix timestamp) is the number of seconds that have elapsed since January 1, 1970.  Any UNIX based kit when it can't get a time signal reverts to the epoch date until the timestamp is updated by an NTP server.

Second - I think the PoE is a red herring as I have these and never have had issues since installing.

Third - A new neighbour connection shouldn't matter.

Fourth - The cabinet could be a problem as our area had this issue when it rained as the cabinet was flooding!  Thankfully that was sorted quickly.

However I don't think it's it either as it seems to be time related.  Is there any equipment in your house or nearby that is running over these time periods?  Could be interference?  

[katspike]

Thank you @Adduxi for reading through all of that and providing some helpful insight.

Interference does seem to be the most likely cause. My adjacent neighbours can’t think of anything that could be related (apart from heating a coffee pot on a hob?) ... and in my own household, it still happens when I disconnect all peripherals and minimise demand.

What sort of device would cause this interference? Something in my home or on the fibre network outside my home? 

When I switched the Hub from Modem mode to default Router mode, the problems continued into the night. So I called VM, and they "changed the frequency". I'm not sure what that means exactly but it is now more unstable than ever. However, there's a big storm outside, so it may well be related to the broken door on the nearby junction cabinet.

[katspike]

... alternatively, could it be related to the electrical wiring or overheating? Here's a diagram of my surge protection and cooling equipment 

[katspike]

Is there a risk of the hub 3 getting hacked (in modem mode) and would it be better if it was https rather than http (as recommended here)?

Why?.... I've been struggling with high latency, packet loss, T3 timeouts etc. for two weeks. VM customer support cannot find any fault on the line or coax connections.

I've been monitoring Network Logs, weather, power consumption etc. and via BQM, and although it happens randomly, there does seem to be some consistency at certain times. See my graphs where I've labelled critical errors and attempted remedies. 

See my other post for more info about my setup

My neighbours are not knowingly doing anything that could cause interference and I'm running out of possible causes. So, could the hub be vulnerable because the admin settings are not protected by https?

[legacy1]

---

@katspike wrote:

Is there a risk of the hub 3 getting hacked (in modem mode)

---

nope

[Adduxi]

Try to keep to the one post.  It's against the Forum rules to have multiple posts for the same issue.

I've been using Modem mode ever since VM have been supplying me with Hubs and have never even thought of my neighbours hacking my modem  😉 

[katspike]

Ok, but there's a lot of hypothesis in my other post, and not many answers, so I was just trying to narrow it down.

I did immediately change all passwords when the hub was first set up, and have now changed them again, but the problems persist, so how can you be sure it's safe on http?

Is there anything in the network log that would indicate suspicious activity?

Network Log

Time Priority Description

02/11/2021 08:41:52	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
02/11/2021 08:15:28	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
02/11/2021 08:13:19	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 23:50:52	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 21:24:55	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 17:30:40	critical	Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 17:30:37	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 17:30:34	critical	Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 17:25:6	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 17:24:49	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 15:42:24	critical	Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 14:43:10	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 14:43:10	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 14:12:21	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 14:11:36	critical	Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 14:08:58	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 14:07:26	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
01/11/2021 09:50:10	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;

[katspike]

Upstream bonded channels

Channel Frequency (Hz) Power (dBmV) Symbol Rate (ksps) Modulation Channel ID

1	53700000	45.3	5120	64 qam	1
2	46200000	45	5120	64 qam	2
3	32599956	44.8	5120	64 qam	4
4	39399917	44.8	5120	64 qam	3

Upstream bonded channels

Channel Channel Type T1 Timeouts T2 Timeouts T3 Timeouts T4 Timeouts

1	ATDMA	0	0	2	0
2	ATDMA	0	0	3	0
3	ATDMA	0	0	0	0
4	ATDMA	0	0	0	0

Downstream bonded channels

Channel Frequency (Hz) Power (dBmV) SNR (dB) Modulation Channel ID

1	330750000	6	40	256 qam	25
2	202750000	4.8	40	256 qam	9
3	210750000	4.8	40	256 qam	10
4	218750000	4.6	40	256 qam	11
5	226750000	4.6	40	256 qam	12
6	234750000	4.5	40	256 qam	13
7	242750000	4.3	40	256 qam	14
8	250750000	4	40	256 qam	15
9	258750000	4	40	256 qam	16
10	266750000	4	40	256 qam	17
11	274750000	4	40	256 qam	18
12	282750000	4.3	40	256 qam	19
13	290750000	4.6	40	256 qam	20
14	298750000	5.1	40	256 qam	21
15	306750000	5.5	40	256 qam	22
16	314750000	5.5	40	256 qam	23
17	322750000	5.6	40	256 qam	24
18	338750000	6	40	256 qam	26
19	346750000	5.9	40	256 qam	27
20	354750000	5.8	40	256 qam	28
21	362750000	5.6	40	256 qam	29
22	370750000	5.5	40	256 qam	30
23	378750000	5.5	40	256 qam	31
24	386750000	5.4	40	256 qam	32

Downstream bonded channels

Channel Locked Status RxMER (dB) Pre RS Errors Post RS Errors

1	Locked	40.9	31104	2831
2	Locked	40.3	51626	9763
3	Locked	40.9	51490	8407
4	Locked	40.9	51869	8095
5	Locked	40.9	55165	9345
6	Locked	40.3	54210	9135
7	Locked	40.9	49197	7679
8	Locked	40.3	47744	6972
9	Locked	40.9	46572	7358
10	Locked	40.3	49398	8648
11	Locked	40.3	45524	8520
12	Locked	40.3	41231	6478
13	Locked	40.9	38043	4885
14	Locked	40.9	35702	3876
15	Locked	40.3	33870	3345
16	Locked	40.9	34047	3381
17	Locked	40.9	33540	3536
18	Locked	40.9	28727	2748
19	Locked	40.9	28734	3184
20	Locked	40.9	28757	3166
21	Locked	40.9	29995	2634
22	Locked	40.9	30013	2844
23	Locked	40.9	30441	2675
24	Locked	40.3	28548	2303

[Beth_G]

Hi katspike,

Thank you for your post and for including all of those details for us, that's very helpful. I'm really sorry for all the issues you've been experiencing.

I've been able to locate your account and can see that there are some obvious signal issues that need putting right by an engineer - I can see that you do have a visit scheduled, so please do let us know how it goes and we can go from there if the issues do persist.

Kind regards

Beth