VM CORE NETWORK FAILURE {URGANT}

[RR-IT-GUY]

Good afternoon all.

I want to bring to the attention of Virgin Media ASAP that there is an issue with the core network configuration.

It has been brought to my attention by a member of the Thinkbroadband forum that the "BGP is incorrectly configured and rather than going via Liberty Global /Aorta some traffic is going via LINX direct to VMO2" 

This poses an issue since there is a lack of capacity through this routing.

https://www.thinkbroadband.com/news/9195-bqm-showing-peak-time-latency-issues-for-virgin-media-users

This is causing issues with the core network within the last few days.

Please refer to the article attached.

MY BMQ's for reference

LIVE BMQ 

8/04/2022 

09/04/2022 

10/04/2022 

11/04/2022 

12/04/2022 

Kind Regards

RR_THE_IT_GUY

[RR-IT-GUY]

for the sake of this I'll do it a 3rd time

Downstream bonded channels

Channel Frequency (Hz) Power (dBmV) SNR (dB) Modulation Channel ID

1	267000000	6.6	40	256 qam	17
2	275000000	6.5	40	256 qam	18
3	283000000	6.5	40	256 qam	19
4	291000000	6.5	40	256 qam	20
5	299000000	6.6	40	256 qam	21
6	307000000	6.5	40	256 qam	22
7	315000000	6.5	40	256 qam	23
8	323000000	6.5	40	256 qam	24
9	331000000	6.5	40	256 qam	25
10	339000000	6.5	40	256 qam	26
11	347000000	6.3	40	256 qam	27
12	355000000	6	40	256 qam	28
13	363000000	5.9	40	256 qam	29
14	371000000	5.8	40	256 qam	30
15	379000000	5.9	40	256 qam	31
16	387000000	5.9	40	256 qam	32
17	395000000	6	40	256 qam	33
18	403000000	6	40	256 qam	34
19	411000000	5.9	38	256 qam	35
20	419000000	5.5	40	256 qam	36
21	427000000	5.5	40	256 qam	37
22	435000000	5.4	38	256 qam	38
23	443000000	5.4	38	256 qam	39
24	451000000	5.3	40	256 qam	40

Downstream bonded channels

Channel Locked Status RxMER (dB) Pre RS Errors Post RS Errors

1	Locked	40.3	111	0
2	Locked	40.3	121	0
3	Locked	40.3	112	0
4	Locked	40.3	133	0
5	Locked	40.3	121	0
6	Locked	40.9	129	0
7	Locked	40.3	131	0
8	Locked	40.9	112	0
9	Locked	40.3	142	0
10	Locked	40.3	94	0
11	Locked	40.9	115	0
12	Locked	40.3	205	0
13	Locked	40.9	118	0
14	Locked	40.3	130	0
15	Locked	40.3	115	0
16	Locked	40.3	114	0
17	Locked	40.3	148	0
18	Locked	40.9	113	0
19	Locked	38.6	129	0
20	Locked	40.3	98	0
21	Locked	40.3	122	0
22	Locked	38.9	133	0
23	Locked	38.9	118	0
24	Locked	40.3	119	0

Upstream bonded channels

Channel Frequency (Hz) Power (dBmV) Symbol Rate (ksps) Modulation Channel ID

1	60299988	46.5	5120	64 qam	1
2	46200000	45.3	5120	64 qam	3
3	39400000	45	5120	64 qam	4
4	53699880	45.8	5120	64 qam	2

Upstream bonded channels

Channel Channel Type T1 Timeouts T2 Timeouts T3 Timeouts T4 Timeouts

1	ATDMA	0	0	3	0
2	ATDMA	0	0	3	0
3	ATDMA	0	0	1	0
4	ATDMA	0	0	1	0

General Configuration

Network access	Allowed
Maximum Number of CPEs	1
Baseline Privacy	Enabled
DOCSIS Mode	Docsis30
Config file	cmreg-vmdg505-bbt057-b.cm

Primary Downstream Service Flow

SFID	20700
Max Traffic Rate	117000047
Max Traffic Burst	42600
Min Traffic Rate	0

Primary Upstream Service Flow

SFID	20699
Max Traffic Rate	10500047
Max Traffic Burst	16320
Min Traffic Rate	0
Max Concatenated Burst	16320
Scheduling Type	BestEffort

[RR-IT-GUY]

Refresh data

Network Log

Time Priority Description

03/05/2022 00:40:15	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
02/05/2022 16:47:17	Error	DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
02/05/2022 06:29:23	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
30/04/2022 15:18:42	notice	LAN login Success;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
29/04/2022 18:49:33	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
29/04/2022 04:47:16	Error	DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
28/04/2022 04:28:33	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
26/04/2022 12:58:58	Error	DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
24/04/2022 16:54:1	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
24/04/2022 13:23:53	notice	LAN login Success;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/04/2022 22:31:45	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/04/2022 17:08:35	notice	LAN login Success;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/04/2022 02:02:40	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
22/04/2022 21:45:45	notice	LAN login Success;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
21/04/2022 13:45:40	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
21/04/2022 11:20:1	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
21/04/2022 07:35:35	Error	DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
20/04/2022 23:32:34	notice	LAN login Success;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
17/04/2022 23:21:43	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
17/04/2022 21:27:15	Error	DHCP RENEW WARNING - Field invalid in response v4 option;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;

[Sephiroth]

My BQM is the same as everyone else's but I have no performance issues. There may well be people on this thread who think the BQM mountain is the cause of their problem when it isn't. The OP introdced the SNR matter so I've just picked up on it. Perfectly reasonable.

[RR-IT-GUY]

Yes i Know i have massively high pre RS errors, no intension in restarting until thursday as that's when my new speed tier is "sent" to the hub config

[Sephiroth]

@RR_IT_GUY

Thanks for posting the stats.  They are fine.  They don't show massively high Pre-RS errors at all - in any case they are corrected in flight.

So your problems are something else.  It's not clear to me whether or not they are confined to the BQM mountain times.

[RR-IT-GUY]

Thanks for confirming, but I was well aware, had SNR issues last year, just got over that

Like I said, spoke to my contact and the problem is transit and peering at a core network level "LINX overloaded"

All the problems will go away after action is taken at a network level.

Once my package change is processed on Thursday (requested last month out of cooling off period), I will be investigating and requesting investigations until the problem is solved. 

Last year it took over 20 engineers to get a basic issue causing massive disruption in the evenings fixed. 

[bentnose]

Another BQM to backup the issue.

Will be leaving virgin if they dont acknowledge this soon. 

https://www.thinkbroadband.com/broadband/monitoring/quality/share/d22955eaeedd06546247e16876e01a826fa39eed-03-05-2022

[bentnose]

To follow up to this, my issues are ping exceeding 300ms regularly online gaming which is unplayable. Lots of packet loss

[IanSR]

---

@RR-IT-GUY wrote:

Thanks for confirming, but I was well aware, had SNR issues last year, just got over that

Like I said, spoke to my contact and the problem is transit and peering at a core network level "LINX overloaded"

All the problems will go away after action is taken at a network level.

Once my package change is processed on Thursday (requested last month out of cooling off period), I will be investigating and requesting investigations until the problem is solved. 

Last year it took over 20 engineers to get a basic issue causing massive disruption in the evenings fixed. 

---

Yeah the problem isn't our data route out, it's the return leg that is the problem where stuff is being routed via the LINX network peering (or whatever it's called).  Virgin's excuse that it is and error at ThinkBroadband is horse manure as I've been monitoring both my dad's connection (BT Internet) and our gaming server (Telehouse London) and neither of them exhibit the packet loss or 8pm hump.

I will stream our race series tonight here https://www.youtube.com/channel/UCLcacTvNAHiLo326iW3vppA
It doesn't schedule live events because OBS streams via Restream so it's done automatically, just goto that url from about 7.45pm and you'll see the live event, on my screen lower left is will be the communication meters as follows:

:L = Latency (my ping to server, usually sticks around 55, will go up once the 8pm hump starts to around 100, anything over 100 is yellow and bad).

Q = Quality (how many packets are being lost on route from the server 17 miles away from me as the crow flies, to my PC, the scale is 0 to 50%, if it gets to 50% packet loss, I get disconnected from the server, during the last few weeks this has been skipping between 10 and 30%, our game is old, so any packet loss is bad there is no error correction built in, when the Q meter goes up into the yellow, you'll see everybody else warping, but they swear I am not (which is why to me this is only the return route being affected)).

S = sync, not to be too worried about, this only comes on if there are issues with the PC or Server, say either is hanging or "Not Responding" like windows does.

Also, the two other guys in our group who are also on Virgin Media, has experienced exactly the same issues for around a month too, everybody else is either BT based or in a different country and doesn't experience the8pm hump.

Come on Virgin, fix your crap.

[conman33158]

---

@Adduxi wrote:

---

@conman33158 wrote:

<snip>   Roll on tomorrow, my new BT hub arrived this morning but at least when the engineer leaves I can connect my Asus straight into the ONT and setup using PPPOE, no more modem mode from the crappy Virgin Hubs 🙂

---

Unless you need the BT SmartHub 2 for a phone line, that should work fine.  I believe the BT SmartHub 2 does not allow ping responses for a BQM, or a change of DNS.

---

I won't be using the phone mate, didn't opt for their DV handset or phone line this time around, should be a simple process switching my network over, maybe a few tweaks on the MTU settings on the Asus as using the Asuswrt-Merlin firmware so no restrictions 🙂