VM CORE NETWORK FAILURE {URGANT}

[RR-IT-GUY]

Good afternoon all.

I want to bring to the attention of Virgin Media ASAP that there is an issue with the core network configuration.

It has been brought to my attention by a member of the Thinkbroadband forum that the "BGP is incorrectly configured and rather than going via Liberty Global /Aorta some traffic is going via LINX direct to VMO2" 

This poses an issue since there is a lack of capacity through this routing.

https://www.thinkbroadband.com/news/9195-bqm-showing-peak-time-latency-issues-for-virgin-media-users

This is causing issues with the core network within the last few days.

Please refer to the article attached.

MY BMQ's for reference

LIVE BMQ 

8/04/2022 

09/04/2022 

10/04/2022 

11/04/2022 

12/04/2022 

Kind Regards

RR_THE_IT_GUY

[moonies]

 

 

 

 

Surely this is becoming illegal practice and should be reported to the proper people? 2+ weeks more images if we wanna go there btw. I want my months money back and I want out my contract right now 

[conman33158]

Same every night without fail. Thankfully I get BT Full Fibre 900 installed next Wednesday the (4th May). If I ever come back to Virgin, with a bit of luck they will have sorted out their network by then. I just pity some of you guys that are still stuck in contract as I've been there myself in the past and it's not a nice place to be, especially when the ISP insist there's nothing wrong 😞

[IKbrunel]

[Rwilliams100]

Portsmouth here

[Darkman]

Identical looking peak in the last three screenshots. Still nothing from VM I'm guessing?

[jgtimperley]

---

@moonies wrote:

Here we go again boys.

surely this is breach of contract? Losing money daily 

---

Whatever is going on at just before 1am is something specific to your line. It isn't appearing on my graph, or anyone else's for that matter.

Genuinely out of interest, I'm not trying to troll you here, but how are you losing money from this particular fault (the inbound LINX port congestion, resulting in increased peak time latency to anything coming in over LINX from around 7pm to 10pm). I've been running without my L2TP tunnel for the past few days, and honestly haven't noticed any issues, other than the hump in the graph and presumably rubbish speedtest results from TBB (although I haven't had need to run one). 

Of course, anything taking the LINX route will be affected, but pretty much any traffic I can think of testing doesn't take that route - that is of course just the traffic that I'm using, so your use-case could well be different.

[jgtimperley]

As a more general reply, it may help to provide specific examples of what is being affected by this fault, in addition to BQM graphs and TBB speedtests. Is there a particular VoIP provider that sees Virgin through LINX for example, or a gaming server that has traffic coming back in through LINX, or a card processing website that doesn't work. I've also seen many other posts that show underlying issues not related to this fault, e.g. packet loss in the middle of the day, disconnections at a local level rather than this peering fault, so it's worth confirming that any issues are due to traffic coming back in over LINX.

Not only will having this understanding help the discussion here, but it will also avoid disappointment if/when the LINX peering issue is fixed, but customers with other faults are still having issues.

[Adduxi]

---

@jgtimperley wrote:

---

<snip>  

Of course, anything taking the LINX route will be affected, but pretty much any traffic I can think of testing doesn't take that route - that is of course just the traffic that I'm using, so your use-case could well be different.

---

Interesting. and I'm no expert, but going just by the Host names, it looks like my BT traffic goes via Linx and is not affected.  However the VM route has no mention of Linx and is affected, along with the Belfast core router.?

BT Tracert 

Tracing route to thinkbroadband.com [80.249.106.141]
over a maximum of 30 hops:

1 2 ms <1 ms 1 ms 192.168.1.1
2 46 ms 6 ms 5 ms 172.16.19.46
3 * * * Request timed out.
4 19 ms 19 ms 19 ms 62.172.102.68
5 21 ms 23 ms 21 ms peer8-et3-0-6.telehouse.ukcore.bt.net [62.172.103.133]
6 20 ms 20 ms 19 ms linx-gw2.thdo.ncuk.net [195.66.236.240]
7 22 ms 21 ms 21 ms po4-31.core-rs4.thdo.ncuk.net [80.249.97.85]
8 22 ms 22 ms 21 ms ip106-141.thdo.ncuk.net [80.249.106.141]

VM Tracert

Tracing route to thinkbroadband.com [80.249.106.141]
over a maximum of 30 hops:

1 2 ms 1 ms <1 ms 192.168.1.1
2 7 ms 5 ms 6 ms 172.16.19.46
3 16 ms 10 ms 10 ms belf-core-2a-xe-305-0.network.virginmedia.net [213.105.207.165]
4 19 ms 19 ms 19 ms 62.172.102.68
5 * * * Request timed out.
6 32 ms 26 ms 26 ms 86.85-254-62.static.virginmediabusiness.co.uk [62.254.85.86]
7 26 ms 28 ms 26 ms ae-8.a02.londen12.uk.bb.gin.ntt.net [129.250.66.101]
8 32 ms 27 ms 29 ms ae-0.r21.londen12.uk.bb.gin.ntt.net [129.250.3.214]
9 31 ms 33 ms 26 ms ae-1.a01.londen12.uk.bb.gin.ntt.net [129.250.2.185]
10 28 ms 26 ms 27 ms 192.80.16.146
11 27 ms 27 ms 26 ms po4-31.core-rs4.thdo.ncuk.net [80.249.97.85]
12 22 ms 22 ms 21 ms ip106-141.thdo.ncuk.net [80.249.106.141]

[IKbrunel]

[jgtimperley]

---

@Adduxi wrote:

---

@jgtimperley wrote:

---

<snip>  

Of course, anything taking the LINX route will be affected, but pretty much any traffic I can think of testing doesn't take that route - that is of course just the traffic that I'm using, so your use-case could well be different.

---

Interesting. and I'm no expert, but going just by the Host names, it looks like my BT traffic goes via Linx and is not affected.  However the VM route has no mention of Linx and is affected, along with the Belfast core router.?

[...]

---

A couple of points that will hopefully help your understanding.

With LINX being (a number of) peering LANs, it will just be Virgin’s port that is of relevance here. BT and everyone else who chooses to have a port there will have their own port, so we’re only seeing the congestion on Virgin’s port. 

Traceroute only shows you the path in one direction. You need to see a traceroute in the other direction to see the full path end to end. A good example of this is RR's post that shows the route between Virgin and TBB (outbound from a Virgin perspective) is via Aorta/NTT, but the route back from TBB to Virgin is via LINX (linx-gw1.router.ntli.net). The LINX port is where the congestion is occurring.