Hub 3 / Compal CH7465-LG (TG2492LG) and CGNV4 Latency Cause

[Datalink]

Good Day Ladies and Gentlemen,

Greetings from the other side of the pond, so to speak.  Over the last few weeks I've been perusing various user forums across North America and Europe for issues related to Intel Puma 6 modem latency.  Of those forums, your Hub 3 stands out as yet another Puma 6 based modem where users see continuous latency no matter what site is used or what online game is played. Considering all of the problems that are on the go, the following information should be of interest to all Hub 3, Compal CH7465-LG and Hitron CGNV4 modem users.  There is much more to post regarding this, so this is a start, to alert VM users as to the real cause of the latency and hopefully engage the VM engineering staff, via the forum staff, with Arris.  I am surprised to see that there has been no mention on this board of users from other ISPs who are suffering the exact same issues with their modems, so, this may come as a surprise to some, and possibly old news to others.

So, the short story ........

The Hub 3 / Compal CH7465-LG (TG2492LG) & Hiton CGNV4 modems are Intel Puma 6 / 6 Media Gateway (MG) based modems.  These modems exhibit high latency to the modem and high latency thru the modem.  The latency affects all IPV4 and IPV6 protocols, so it will be seen on every internet application and game.  The basic cause is the processing of the data packets thru a CPU software based process instead of thru the hardware processor / accelerator.  It appears that a higher priority task runs periodically, causing the packet processing to halt, and then resume.  This is observed as latency in applications and in ping tests to the modem and beyond.  For the last several weeks, Hitron, along with Intel and Rogers Communications in Canada have been addressing the latency issue within the Hitron CGNxxx series modems.  To date, only the IPV4 ICMP latency has been resolved.  Although this is only one protocol, it does show that a Puma 6MG modem is capable of using the hardware processor / accelerator with good results.  Currently Rogers is waiting for further firmware updates from Hitron which should include an expanded list of resolved protocol latency issues.  For Arris modems, "Netdog" an Arris engineer indicated last week that Arris was onboard to address the issue for the Arris SB6190 modem.  That should be considered as good news for any Arris modem (read Hub 3) user as Arris should be able to port those changes over to other Puma 6/6MG modems fairly quickly.  This is not a trivial exercise and will probably take several weeks to accomplish.  Note that there is no guarantee at this point that it is possible to shift all packet processing to the hardware processor / accelerator without suffering from any packet loss side effects.  Time will tell if all of the technical issues can be resolved with the current hardware included in the Puma 6/6MG chipset.  Last night, Netdog loaded beta firmware on selected test modems on the Comcast Communications network.  As this was only done last night, it's too soon to tell what this version resolves and if it was successful or not.  Netdog has contacts with staff at Comcast, Rogers, Charter and Cox Communications to fan out beta versions and modifications for testing.  I'd say its time to add Virgin Media and/or Liberty Global to that group as well.

Recent activity:

Approx three weeks ago a DSLReports user, xymox1 started a thread where he reported high latency to an Arris SB6190 and illustrated that with numerous MultiPing plots.  This is the same latency that I and other users with Rogers communications have been dealing with for months so it came as no surprise.  As well as reporting via that thread, xymox1 took it upon himself to email several staff members at Arris, Intel, Cablelabs and others.  The result of that campaign was Netdog's announcement, last week, that Arris was fully engaged at resolving the issue.  That has led to last nights release of beta firmware, although as I indicated its too early to determine what the beta firmware resolves, if anything.

The original thread that xymox1 started is here:

https://www.dslreports.com/forum/r31079834-ALL-SB6190-is-a-terrible-modem-Intel-Puma-6-MaxLinear-mis...

Yesterday, DSLReports issued a news story covering the thread:

https://www.dslreports.com/shownews/The-Arris-SB6190-Modem-Puma-6-Chipset-Have-Some-Major-Issues-138...

Today, Arris responded:

https://www.dslreports.com/shownews/Arris-Tells-us-Its-Working-With-Intel-on-SB6190-Puma6-Problems-1...

That response was also picked by Multichannel.com

http://www.multichannel.com/news/distribution/intel-arris-working-firmware-fix-sb6190-modem/409379

This is more news likely to appear in the next few days as additional tech and news staff pick up on this issue.

Hub 3 observations:

Like many others using a Puma 6/6MG modem, Hub 3 users are experiencing latency when they ping the modem, or ping a target outside of the home, game online or use low latency applications.  The common misconception is that this is Buffer Bloat. It's not. Its most likely a case of the packet processing stopping while the CPU processes a higher priority task.  The packet processing is done via the CPU no matter what mode the modem is operating in, modem mode or router mode and no matter what IPV4 or IPV6 protocol is used.  Normally, the latency is just that, latency.  The exception are UDP packets. In this case there is latency and packet loss.  The result of that is delayed and failed DNS lookups, and poor game performance for games that use UDP for player/server comms or player/player comms.

Can this be fixed?

So far, it appears that the answer is yes.  Rogers Communications issued beta firmware to a small group of test modems in October.  This version shifted the IPV4 ICMP processing from the CPU to the hardware processor / accelerator, resulting in greatly improved performance in ping latency.  At the present time we are waiting for the next version firmware which should shift other protocols over to the hardware processor / accelerator.  That can be seen in the following post:

http://communityforums.rogers.com/t5/forums/forumtopicpage/board-id/Getting_connected/message-id/369...

The details and results of last nights beta release to the Comcast group have yet to be seen.

At this point there is enough reading to keep most staff and users busy.  My intention is to post some of the history leading up to this point and instructions on how to detect the latency and packet loss.  This is not thru the use of a BQM.  I had hoped to post this all at once but events are moving much faster than I had thought they would.  For now this should suffice to get the ball rolling.

Below is a link to a post with a couple of HrPing plots from my 32 channel modem to the connected CMTS.  This shows the latency that is observed and reflects what others have posted in this forum using Pingplotter and HrPing.

https://www.dslreports.com/forum/r31106550-

HrPing is one of the freebie applications that can be used to monitor the latency to and thru the modem. 

Pingplots with Pingplotter which show the latency from my modem to the CMTS can be found in the first two to three rows of my online image library at Rogers Communications, located below.  They are essentially what the BQM would look like if you were able to zoom into the plot to the point where you could see the individual ping spikes.  Those ping spikes are common to Puma 6 and Puma 6MG modems.

http://communityforums.rogers.com/t5/media/gallerypage/user-id/829158

 [MOD EDIT: Subject heading changed to assist community]

[OllyC]

Also just to show how the performance has continued 10 hours later today with the Super Hub 2.0 running in modem mode:

It's a night and day difference.

[nallar]

No, there's no way to avoid the terrible latency connecting to the internet through a hub 3.

The discussion there was about communications between local devices.

[OllyC]

Ahh gotcha I thought that may be the case.

[mini_beest]

Oh my experience with Virgin has been a painful one for the last 10 days.

I got a good deal with Virgin, hence the reason I left Sky. But after reading this forum & all the horror stories others have had with the Hub 3, alarm bells have been ringing. So I've asked Virgin to replace the Hub 3 I would have on order for the 2ac before the installation takes place.

To cut a long story short, I've been told by Virgin "That they do not have any Hub 2ACs availble","We do but you're not an existing customer","We do, but your 200mb broadband will not work on the 2AC" (Which I know is a lie because I was with that hub & that package with Virgin before moving to Sky last year) I've even had issues with the less than helpful Virgin Media store staff, who basically told me there was no issues & I had to ring customer relations.

After making my blood boil more than it should, I've decided to cancel my deal with Virgin. But, only then have I been offered the SuperHub 2AC I wanted in the first place!

No fault on the customer relations team who have been helpful and tried to keep my business, but it's the other departments which have badly let Virgin down and I will think twice before moving back to them in the future.

[OllyC]

It really is horrendous the experiences people are having...

I cannot tell you how much of my own time its taken to call, chase up, investigate and test all of the faults I've had but in my own experience the "experts" even at the Tier 2 tech level support and higher level customer service haven't had a clue on how to solve the issues I've had, I've pretty much researched and attempted to solve all of them on my own and even passed the information I find back on to Virgin and asked for it to be relayed as high as it can so that everyone else effected can be helped!

The amount of misinformation and in a lot of cases flat out LIES being told is unbelievable, so many times its been up to me to prove extensively that I'm experiencing issues as usually they claim the service is running fine and then even when you find the fault and possible solution you're told you can't do it or have it and are stuck in the bad situation.

I'm currently trying to talk to people as high up within Virgin as possible not only to relay the terrible customer service experiences that I've had from various departments but also to get the relevant departments to understand each of the issues I've faced and how to resolve them so others don't have to go through the same.

[shanematthews]

---

@mini_beest wrote:

Oh my experience with Virgin has been a painful one for the last 10 days.

I got a good deal with Virgin, hence the reason I left Sky. But after reading this forum & all the horror stories others have had with the Hub 3, alarm bells have been ringing. So I've asked Virgin to replace the Hub 3 I would have on order for the 2ac before the installation takes place.

To cut a long story short, I've been told by Virgin "That they do not have any Hub 2ACs availble","We do but you're not an existing customer","We do, but your 200mb broadband will not work on the 2AC" (Which I know is a lie because I was with that hub & that package with Virgin before moving to Sky last year) I've even had issues with the less than helpful Virgin Media store staff, who basically told me there was no issues & I had to ring customer relations.

After making my blood boil more than it should, I've decided to cancel my deal with Virgin. But, only then have I been offered the SuperHub 2AC I wanted in the first place!

No fault on the customer relations team who have been helpful and tried to keep my business, but it's the other departments which have badly let Virgin down and I will think twice before moving back to them in the future.

---

Well Hub2's are a finite resource at this point. they won't just be offered to everyone who asks for them as they would just immediately exhaust what little stock they have, fairly sensible use of resources to be honest, some people will have read this, and would just ask for a hub2 despite not doing anything that would be majorly impacted by having a hub3, in essence they want to make sure only the people who truely need them get them rather than wasting them on people who wouldn't benefit from them, its a resource thats getting more depleted by the day

[funkyphil]

Ok now this is pretty random but thought i`d share it with you guys! I have been suffering really badly with latency issues for close on 2 months with my superhub 3. I was running windows 10 that was until the other day when my computer decided too spit its dummy out and crash on me.I tried everything to reboot it without success so had to do a factory reset back to window`s 8.1 Here`s the funny thing after reinstalling everything (except windows 10) touch wood my latency issues seem to have subsided! Coincidence? Still show`s blips on latency mon but no where near how it was. 6 windows open in chrome (4 playing youtube video`s) and very little in the way of popping crackling like i had before! Maybe it was a bad driver installed on my pc that was causing the issue all along? Yes i know the superhub 3 is to blame for 99% of the issues us guys have had on here but so far i`m much happier! Here`s hoping they sort the superhub 3 in the very near future.

[OllyC]

Have you ran a Quality Monitor on your line through www.thinkbroadband.com?

You may have had issues with your PC before and the changes have helped but you will almost certainly have the latency issue that this thread is about whilst using the Hub 3 in its current state.

[funkyphil]

Just run the test the monitor is completely empty! Whats that? 

[mannakin]

You need to enable your router to respond to IMCP pings