IPv6 support on Virgin media

Lol

 Fixed broadband providers like VM have an two orders of magnitude more data flowing through their networks. VM have almost 1Tbps of interconnect bandwidth through LINX for example.

Scaling is really non trivial at this scale.

The DS-Lite with public IPv4 address is an interesting one - it actually makes the problem a lot simpler.  Instead of needing a stateful CGNAT (that remembers the port and maps it to a client IP), the edge router just needs to (de)encapsulate the v4 packet in its v6 wrapper and send it on its way.  That's stateless, so can be done much more efficiently - could be done at line rate with suitable hardware. At this point the ISP network is v6 only, but customers have full v4 routing so it shouldn't break any apps.  Unlike DNS64 and friends it's completely transparent to the end user.

It doesn't solve the v4 exhaustion problem, but that could be done opportunistically (eg new customers don't get a public v4, or put lower tier customers on private v4, or some other criteria).

---

impromptu wrote:

The DS-Lite with public IPv4 address is an interesting one - it actually makes the problem a lot simpler.  Instead of needing a stateful CGNAT (that remembers the port and maps it to a client IP), the edge router just needs to (de)encapsulate the v4 packet in its v6 wrapper and send it on its way.  That's stateless, so can be done much more efficiently - could be done at line rate with suitable hardware. At this point the ISP network is v6 only, but customers have full v4 routing so it shouldn't break any apps.  Unlike DNS64 and friends it's completely transparent to the end user.

It doesn't solve the v4 exhaustion problem, but that could be done opportunistically (eg new customers don't get a public v4, or put lower tier customers on private v4, or some other criteria).

 

---

The thing is your forgetting that to be able to communicate you have to be able to deal with the return traffic.

There's no such thing as stateless NAT - The individual customers won't have an IPv4 address to themselves, to when that traffic comes back, unless you've stored a table telling you about the outbound traffic state, then there's no way to route the replies back to the correct customers IPv4 address.

Tim

I was referring to scale in terms of the number of end-point devices being supported under GCNAT as opposed to raw bandwidth. Both are different challenges in scale.

---

ravenstar68 wrote:

---

impromptu wrote:

The DS-Lite with public IPv4 address is an interesting one - it actually makes the problem a lot simpler.  Instead of needing a stateful CGNAT (that remembers the port and maps it to a client IP), the edge router just needs to (de)encapsulate the v4 packet in its v6 wrapper and send it on its way.  That's stateless, so can be done much more efficiently - could be done at line rate with suitable hardware. At this point the ISP network is v6 only, but customers have full v4 routing so it shouldn't break any apps.  Unlike DNS64 and friends it's completely transparent to the end user.

It doesn't solve the v4 exhaustion problem, but that could be done opportunistically (eg new customers don't get a public v4, or put lower tier customers on private v4, or some other criteria).

 

---

The thing is your forgetting that to be able to communicate you have to be able to deal with the return traffic.

There's no such thing as stateless NAT - The individual customers won't have an IPv4 address to themselves, to when that traffic comes back, unless you've stored a table telling you about the outbound traffic state, then there's no way to route the replies back to the correct customers IPv4 address.

Tim

---

no, he's suggesting the use of DS-lite, but instead of your ipv4 session being behind CGNAT, your public ipv4 address is basically tunneled to you over ipv6 instead. this is why it would be stateless as there is no nat...

That would allow virgin to run an internal ipv6 network but still give out public IP addresses to those who want or need it...

Not convinced that's what virgin will want to do, but it's an interesting proposal...

I'm also worried that I'll be looking for a new router as I don't know whether my current pfsense box will support all this tunneling stuff..

Except the dual stack lite RFC 6333 specifies that NAT be carried out on the AFTR elements.

Here's the introduction to the RFC

1. Introduction

   The common thinking for more than 10 years has been that the
   transition to IPv6 will be based solely on the dual-stack model and
   that most things would be converted this way before we ran out of
   IPv4.  However, this has not happened.  The IANA free pool of IPv4
   addresses has now been depleted, well before sufficient IPv6
   deployment had taken place.  As a result, many IPv4 services have to
   continue to be provided even under severely limited address space.

   This document specifies the Dual-Stack Lite technology, which is
   aimed at better aligning the costs and benefits in service provider
   networks.  Dual-Stack Lite will enable both continued support for
   IPv4 services and incentives for the deployment of IPv6.  It also
   de-couples IPv6 deployment in the service provider network from the
   rest of the Internet, making incremental deployment easier.

   Dual-Stack Lite enables a broadband service provider to share IPv4
   addresses among customers by combining two well-known technologies:
   IP in IP (IPv4-in-IPv6) and Network Address Translation (NAT).

   This document makes a distinction between a dual-stack-capable and a
   dual-stack-provisioned device.  The former is a device that has code
   that implements both IPv4 and IPv6, from the network layer to the
   applications.  The latter is a similar device that has been
   provisioned with both an IPv4 and an IPv6 address on its
   interface(s).  This document will also further refine this notion by
   distinguishing between interfaces provisioned directly by the service
   provider from those provisioned by the customer.



Durand, et al.               Standards Track                    [Page 3]

 
RFC 6333                     Dual-Stack Lite                 August 2011


   Pure IPv6-only devices (i.e., devices that do not include an IPv4
   stack) are outside of the scope of this document.

   This document will first present some deployment scenarios and then
   define the behavior of the two elements of the Dual-Stack Lite
   technology: the Basic Bridging BroadBand (B4) element and the Address
   Family Transition Router (AFTR) element.  It will then go into
   networking and NAT-ing considerations.

Note that the idea behind dual stack lite is the fact that IPv6 deployment is nowhere near as complete as it should have been, it was hoped (which is something I mentioned earlier, even though I had not read this document before today), that we would have dual stacked IPv6 and IPv4 BEFORE we ran out of usable IPv4 space.

Because that hasn't happened, we now have to look at how to retain IPv4 access for and users while still growing the internet as a whole.

In short DS-Lite now requires NAT on the AFTR element.

While it does discuss port forwarding at the AFTR it also suggests that ISP's may choose not to forward some of the well known service ports to end users (i.e. Ports 0-1023 e.g. port 80 TCP)  Note that this won't affect outbound connections but will make it impossible to run web servers or personal mail servers.

So talking about Dual Stack-Lite with no NAT is to put it simply - a pipe dream.

Tim

I was reading through the RFC's the other day too.

I think that the RFC was written with the view that you would only use it when you needed it, and as such you should need CGNAT.  It doesn't seem to explicitly state that you cant use the IPv4 end as a single point per user.

AS5089 has about 26 million IPv4's (If my math is correct) under the "Virgin Media" name. I am not going to go through all the SUB AS's to see what looks customer-ish, someone else here probably knows how to pull that info from the database with ease and parse it (I would have to do it one by one by hand). 

They say they have 5.9 million cable customers, and 3.1 million mobile customers (hard for me to believe that one).  So that's 9.1 million IP's at a minimum.  Mobile customers have probably been under CGNAT since the start so lets cut that 3.1 million down.  Europol says some providers have thousands of people per IP, but lets say that virgin limits us to 250 people per IP...  So that knocks mobile down to 12,000 users..  Ignorable levels.  Their old TV boxes could have a public facing IP I think?  The new V6 box is now behind your NAT router (Designed to save on the v4 assignment?).  They don't give nearly enough stats to even take a rough guess on this.  But lets just take their numbers and shoot for 8 million.  So 8+6= 14 million IP addresses. 

I have a hard time seeing how they are out of IPv4 addresses.  It was only a couple of years ago that they suggested they were a really long way off from even needing to worry about IPv4, the V6 box should have reduced that worry by a huge amount, and now all of the sudden they want to add CGNAT.

I am watching the IPv6 transition with a little interest.

Yesterday, I got put on a new CMTS with a new with a change of public IPv4 address and hop 2 (10.* IPv4) address. CMTS MAC identifies as Cisco, which it was before too. No such IPv6 activity reported by my CM though. Although this sounds obvious that it might be some sort of congestion clean-up/re-seg of some-sort, I was having absolutely no bandwidth/latency issues on 350meg even at very busy times so I call this move peculiar on my part. Would this sort of activity be evident of a transition?

They have a lot of announcements that look like this:

62.30.0.0/15 	Virgin Media Limited
62.30.0.0/16 	Virgin Media Limited
62.31.0.0/16 	Virgin Media Limited 

which is 65k IPs, not 131k. If you remove all of the overlapping announcements then I make it about 8.3 million IPs. bgp.he.net says "IPs Originated (v4): 9,471,488" including all of the customer prefixes, so that looks about right.

Comparing that 8.3 million figure vs your 14 million estimate... yeah. You didn't even account for infrastructure address use or allocation inefficiencies and you still ended up with an estimate that was 1.7x higher than the address space they have available. And you wonder why they want CGNAT?

---

ravenstar68 wrote:

So talking about Dual Stack-Lite with no NAT is to put it simply - a pipe dream.

---

It's still NAT, but stateless 1:1 NAT.

For instance, imagine VM allocated everyone an address in 10.0.0.0/8.  There would be nearly enough addresses to go round there.  They could issue that to their customers, who would all have a unique IP.  However let's say they also owned the public IP space 99.0.0.0/8.  They could NAT 99.x.y.z to 10.x.y.z and it every customer would have a unique public IP address that they could be reached by.  IPv4 apps still work fine. Further, let's say instead of 99.0.0.0/8 they own 2^16 random public /24 subnets - by using 10.0.0.0/8 they can keep a uniform internal address space for management purposes, NATting them externally so users still get public IPs.

In the v6 case, a customer can be given v6 address of ab:cd:ef::10.x.y.z for their v4 packets (with whatever scheme for embedding the 32 bit v4 inside the v6 address).  Their CPE NATs 192.168.1.0/24 to v6 ab:cd:ef::10.x.y.z and sends that over the native v6 network.  The AFTR receives this, NATs it to 99.x.y.z and sends it out to the v4 internet.  Apart from the v4 NAT in the CPE, this path is stateless (each point just needs a general mapping rule, not a memory of what happened recently) and reversible.

I'm not saying it will happen like that, or even that routers can do it like this (I don't know), but that it would make the CGNAT easier if it did.

Once you have a scheme like this, you can start reallocating addresses according to marketing-driven policy.  People on the lowest tier might be a target here - they don't push as much data so aren't as much work for the CGNAT.  So the AFTR knows that when a packet from ab:cd:ef::10.1.y.z comes in, instead of doing the 1:1 NAT it needs to do a full stateful NAT behind a shared IP.  If the customer pays the extra £3 a month for the gaming bundle, they instead get a put in a pool with their own static public IP.

All hypothetical castles in the air of course, but just imagining how it could work.

---

Dagger2 wrote:

 

Spoiler

They have a lot of announcements that look like this:

62.30.0.0/15 	Virgin Media Limited
62.30.0.0/16 	Virgin Media Limited
62.31.0.0/16 	Virgin Media Limited 

which is 65k IPs, not 131k. If you remove all of the overlapping announcements then I make it about 8.3 million IPs. bgp.he.net says "IPs Originated (v4): 9,471,488" including all of the customer prefixes, so that looks about right.

Comparing that 8.3 million figure vs your 14 million estimate... yeah. You didn't even account for infrastructure address use or allocation inefficiencies and you still ended up with an estimate that was 1.7x higher than the address space they have available. And you wonder why they want CGNAT?

 

---

Perhaps my understand of networks is just quite poor then, but why should any of the internal infrastructure (not connected to a external provider) use a public IP?  I know a lot of the Microsoft switches for years now would show their private IP because the internal network didn't have Public IPv4's attached to them.  VM should admin all of their equipment using a 10.0.0.0/8 assignments.  Allocation inefficiencies will be high, BUT, 4over6 could be used to localize a lot of the exit points and help get rid of much of that wastage right?  Sure they will need other servers (DNS/MAIL/Whatever Else) but in the scheme of things I would think that would be quite small, like in the 100's of IP's.

If your telling me that 26 million IP's can't allow a ISP to run more than than about 6 million customers then that makes me really scratch my head about how poorly these things are working.

Okay, I see what your saying now.

My guess on the 14 million was making the assumption that the STB received a public IP (does it, did it?).  Even if it does, the old STB should work fine under CGNAT (from what I know of them).

And what about the mobile network, are those CGNAT or not (I know EE is, but my Three sims aren't), I don't have a virgin mobile so cant check.

They did say in the past they wouldn't look to deploy IPv6 until IPv4 ran out, to me the two were separate, but If your saying that they have ~9 million to spread around mobile (non cgnat) and home broadband then that would mean they have ran out of breathing room.

Although it's useful to know roughly how many IPv4 addresses Virgin has available, I doubt that it matters at all in terms of deployments, because the vast majority are sure to be sold off while the market price is high --- currently around $18 per single address, I read.  The attraction is irresistible, and cashing in on it is made somewhat urgent by knowing that the price will plummet once IPv6 is the majority protocol and IPv4-based companies start bringing up IPv6 in panic.

Unlike many non-ISP companies out there, all ISPs know that IPv6 is creeping up on them even if they haven't yet deployed it to customers, and for an ISP it is not a viable option not to deploy if they want to stay relevant.  Alas for an ISP, running dual stacks internally is about as welcome as a hole in the head, so they also know that their long-term direction is towards IPv6-only internally plus IPv4 gateways at the edge for backwards compatibility.

There will be exceptions to this appealing organization of course.  Quite a few ISP businesses fill a specialist niche rather than the mass public one, and some will find a good role supporting the long tail and extra costs of IPv4.  Not the mass public ISPs though --- it's far too painful and costly in terms of both manpower and equipment for them to do so.  Even worse for the medium term, it limits the speed at which they can evolve.

And so, while I like having numbers and stats on everything, I think that the number of IPv4 addresses at Virgin's disposal will mainly determine the extra profit that will appear on their ledger from the sell-off of excess IPv4 address blocks.  An estimate of the IPv4 addresses that they will need overall is probably well known to them (current session stats minus the number of IPv6-capable destinations), but how many they will need to satisfy those IPv4 users who can neither move to IPv6 services nor use tunneled IPv4 is an extremely hard estimation to make with any confidence.

One thing is certain though --- the number of people unavoidably tied to native IPv4 can only decrease with time, and that is probably a very welcome realization for Virgin.  It means that they can err on the side of retaining fewer rather than too many IPv4 addresses, because time will heal any miscalculation.

Another interesting conclusion is that Virgin is probably quite eager to release IPv6 onto us so that they can know with accuracy how many IPv4 addresses they definitely cannot sell off at this point in time.  Factor in a safety margin and the rest of their IPv4 blocks are pure profit.  Clearly the earlier they know their numbers the better. :-)

Morgaine.