cancel
Showing results for 
Search instead for 
Did you mean: 

Hub 3 / Compal CH7465-LG (TG2492LG) and CGNV4 Latency Cause

Datalink
Up to speed

Good Day Ladies and Gentlemen,

Greetings from the other side of the pond, so to speak.  Over the last few weeks I've been perusing various user forums across North America and Europe for issues related to Intel Puma 6 modem latency.  Of those forums, your Hub 3 stands out as yet another Puma 6 based modem where users see continuous latency no matter what site is used or what online game is played. Considering all of the problems that are on the go, the following information should be of interest to all Hub 3, Compal CH7465-LG and Hitron CGNV4 modem users.  There is much more to post regarding this, so this is a start, to alert VM users as to the real cause of the latency and hopefully engage the VM engineering staff, via the forum staff, with Arris.  I am surprised to see that there has been no mention on this board of users from other ISPs who are suffering the exact same issues with their modems, so, this may come as a surprise to some, and possibly old news to others.

So, the short story ........

The Hub 3 / Compal CH7465-LG (TG2492LG) & Hiton CGNV4 modems are Intel Puma 6 / 6 Media Gateway (MG) based modems.  These modems exhibit high latency to the modem and high latency thru the modem.  The latency affects all IPV4 and IPV6 protocols, so it will be seen on every internet application and game.  The basic cause is the processing of the data packets thru a CPU software based process instead of thru the hardware processor / accelerator.  It appears that a higher priority task runs periodically, causing the packet processing to halt, and then resume.  This is observed as latency in applications and in ping tests to the modem and beyond.  For the last several weeks, Hitron, along with Intel and Rogers Communications in Canada have been addressing the latency issue within the Hitron CGNxxx series modems.  To date, only the IPV4 ICMP latency has been resolved.  Although this is only one protocol, it does show that a Puma 6MG modem is capable of using the hardware processor / accelerator with good results.  Currently Rogers is waiting for further firmware updates from Hitron which should include an expanded list of resolved protocol latency issues.  For Arris modems, "Netdog" an Arris engineer indicated last week that Arris was onboard to address the issue for the Arris SB6190 modem.  That should be considered as good news for any Arris modem (read Hub 3) user as Arris should be able to port those changes over to other Puma 6/6MG modems fairly quickly.  This is not a trivial exercise and will probably take several weeks to accomplish.  Note that there is no guarantee at this point that it is possible to shift all packet processing to the hardware processor / accelerator without suffering from any packet loss side effects.  Time will tell if all of the technical issues can be resolved with the current hardware included in the Puma 6/6MG chipset.  Last night, Netdog loaded beta firmware on selected test modems on the Comcast Communications network.  As this was only done last night, it's too soon to tell what this version resolves and if it was successful or not.  Netdog has contacts with staff at Comcast, Rogers, Charter and Cox Communications to fan out beta versions and modifications for testing.  I'd say its time to add Virgin Media and/or Liberty Global to that group as well.

Recent activity:

Approx three weeks ago a DSLReports user, xymox1 started a thread where he reported high latency to an Arris SB6190 and illustrated that with numerous MultiPing plots.  This is the same latency that I and other users with Rogers communications have been dealing with for months so it came as no surprise.  As well as reporting via that thread, xymox1 took it upon himself to email several staff members at Arris, Intel, Cablelabs and others.  The result of that campaign was Netdog's announcement, last week, that Arris was fully engaged at resolving the issue.  That has led to last nights release of beta firmware, although as I indicated its too early to determine what the beta firmware resolves, if anything.


The original thread that xymox1 started is here:

https://www.dslreports.com/forum/r31079834-ALL-SB6190-is-a-terrible-modem-Intel-Puma-6-MaxLinear-mis...


Yesterday, DSLReports issued a news story covering the thread:

https://www.dslreports.com/shownews/The-Arris-SB6190-Modem-Puma-6-Chipset-Have-Some-Major-Issues-138...


Today, Arris responded:

https://www.dslreports.com/shownews/Arris-Tells-us-Its-Working-With-Intel-on-SB6190-Puma6-Problems-1...


That response was also picked by Multichannel.com

http://www.multichannel.com/news/distribution/intel-arris-working-firmware-fix-sb6190-modem/409379

This is more news likely to appear in the next few days as additional tech and news staff pick up on this issue.


Hub 3 observations:

Like many others using a Puma 6/6MG modem, Hub 3 users are experiencing latency when they ping the modem, or ping a target outside of the home, game online or use low latency applications.  The common misconception is that this is Buffer Bloat. It's not. Its most likely a case of the packet processing stopping while the CPU processes a higher priority task.  The packet processing is done via the CPU no matter what mode the modem is operating in, modem mode or router mode and no matter what IPV4 or IPV6 protocol is used.  Normally, the latency is just that, latency.  The exception are UDP packets. In this case there is latency and packet loss.  The result of that is delayed and failed DNS lookups, and poor game performance for games that use UDP for player/server comms or player/player comms.


Can this be fixed?

So far, it appears that the answer is yes.  Rogers Communications issued beta firmware to a small group of test modems in October.  This version shifted the IPV4 ICMP processing from the CPU to the hardware processor / accelerator, resulting in greatly improved performance in ping latency.  At the present time we are waiting for the next version firmware which should shift other protocols over to the hardware processor / accelerator.  That can be seen in the following post:

http://communityforums.rogers.com/t5/forums/forumtopicpage/board-id/Getting_connected/message-id/369...

The details and results of last nights beta release to the Comcast group have yet to be seen.

At this point there is enough reading to keep most staff and users busy.  My intention is to post some of the history leading up to this point and instructions on how to detect the latency and packet loss.  This is not thru the use of a BQM.  I had hoped to post this all at once but events are moving much faster than I had thought they would.  For now this should suffice to get the ball rolling.

Below is a link to a post with a couple of HrPing plots from my 32 channel modem to the connected CMTS.  This shows the latency that is observed and reflects what others have posted in this forum using Pingplotter and HrPing.

https://www.dslreports.com/forum/r31106550-

HrPing is one of the freebie applications that can be used to monitor the latency to and thru the modem. 

Pingplots with Pingplotter which show the latency from my modem to the CMTS can be found in the first two to three rows of my online image library at Rogers Communications, located below.  They are essentially what the BQM would look like if you were able to zoom into the plot to the point where you could see the individual ping spikes.  Those ping spikes are common to Puma 6 and Puma 6MG modems.

http://communityforums.rogers.com/t5/media/gallerypage/user-id/829158

 

 

 [MOD EDIT: Subject heading changed to assist community]

4,478 REPLIES 4,478

Well, I can't disagree with the fact that whatever is going on it is seriously out of order and a total disgrace. By the way I've written to Watchdog about it (there's an online link here). I suggest as many people as possible do the same. VM will not like it if this goes public - lots of young gamers out there, maybe they'll actually do something.


@Scouts wrote:

Well, I can't disagree with the fact that whatever is going on it is seriously out of order and a total disgrace. By the way I've written to Watchdog about it *there's an online link). I suggest as many people as possible do the same. VM will not like it if this goes public - lots of young gamers out there!


Well, there were couple articles in big online techy press about it already. Not sure how "public" they want to wait for - front page of Guardian? 😉

Also, a lot of affected people don't even realise they are. E.g. for example I was told by somebody "I am still good at CS", not realising that they are good, sure, but can be *better*. The way problem manifests with some games (because its periodic spikes) - you don't see any obvious rubberbanding or something like that, its just like something *slightly bumping your arm every minute*. If you happen to shoot at same moment, you'll miss the shoot - but unless you know what went behind the scenes you will just think "ah, I just didn't aim well or it was bad luck".

 

Rang up tech support on Thursday for laughs and giggles.  Not aware of the issue, she basically advised me to do a reset of the modem by using a paperclip in the reset button.  Magic formula was to hold it in longer than a minute, not less.

I have also written to watchdog via the online link.
Suggest everyone does the same!

https://ssl.bbc.co.uk/programmes/b006mg74/contact


@bambamboyo wrote:
I have also written to watchdog via the online link.
Suggest everyone does the same!

https://ssl.bbc.co.uk/programmes/b006mg74/contact

Watchdog did a piece on VM last month, they had their usual corporate answers but no real promises.  Get on Twitter and make their social media life hell.


@steelysteph wrote:

@bambamboyo wrote:
I have also written to watchdog via the online link.
Suggest everyone does the same!

https://ssl.bbc.co.uk/programmes/b006mg74/contact

Watchdog did a piece on VM last month, they had their usual corporate answers but no real promises.  Get on Twitter and make their social media life hell.


Eh, they won't care, besides all you're doing is making the job of the guy who has to run the account a pain, it won't actually affect anyone else in VM, nor will it suddenly make any massive differences in customer retention that the hub 3 isn't already doing, at the end of the day you can whine at watchdog all you like, they can't do anything about it and at this stage VM can't really do much about it, yes people will chime in with "but they could do x" to which i tell them to read this thread and find the answers as to why they won't, most hub 3 owners won't care about or event notice the latency issues, especially if they have never had a hub 2, my mother has a connection on a hub 3 and it works fine for her and the chromecast my sister has connected to it, they don't game so they don't have any massive issues 😛

Not many people are aware of the issue using it as they brush off the terrible performance as "oh its the internet".  What we have found over on DSLR is people say it does not bother them UNTIL they swap out to  Broadcom. Then UNIVERSALLY they go "Oh wow, I had no idea, this is much better".

Now of course the 0-day DoS attack would be a different story as suddenly nothing works. And does not come back. And no one can fix it. And its on the news as all of the UK and Canada and others are completely off line. Pumageddon.

People may notice that, hahaha..

If you guys dont know this, you can use the 0-day against yourself for testing. It also works from the LAN side. https://github.com/nallar/Puma6Fail/releases

The Intel Puma 5/6/7 is a ticking time bomb and so far we have all been lucky.


@shanematthews wrote:

@steelysteph wrote:

@bambamboyo wrote:
I have also written to watchdog via the online link.
Suggest everyone does the same!

https://ssl.bbc.co.uk/programmes/b006mg74/contact

Watchdog did a piece on VM last month, they had their usual corporate answers but no real promises.  Get on Twitter and make their social media life hell.


Eh, they won't care, besides all you're doing is making the job of the guy who has to run the account a pain, it won't actually affect anyone else in VM, nor will it suddenly make any massive differences in customer retention that the hub 3 isn't already doing, at the end of the day you can whine at watchdog all you like, they can't do anything about it and at this stage VM can't really do much about it, yes people will chime in with "but they could do x" to which i tell them to read this thread and find the answers as to why they won't, most hub 3 owners won't care about or event notice the latency issues, especially if they have never had a hub 2, my mother has a connection on a hub 3 and it works fine for her and the chromecast my sister has connected to it, they don't game so they don't have any massive issues 😛


Shane, if I am correct, this is not the first time you have responded to a user suggesting people start making a fuss in other places other than the VM thread.  You have made the same points multiple times, we hear you.

This does not mean paying customers "making a noise" e.g. through a tv programme such as BBC Watchdog, is not a valid, and potentially good thing to do.  

Yes, I agree that ultimately VM did not design the chip; however, my contract is not with Intel, it is with VM, and it is VM that forced me (yes, told me I had no choice) to replace my perfectly working SU2 with one they know has major issues.  I therefore feel it is perfectly reasonably that when I complain, I am complaining about the service VM provide/the contract I pay for.  It is for VM to deal with their own supply chain.

I therefore believe people raising their complaint through BBC Watchdog is a GOOD thing.

The more noise the better!

These days the only way to make a corporate entity do anything towards making customers lives better, is to complain enough till it becomes a public issue. Then they start caring enough about public opinion hurting the bottom-line.

If you say nothing, they will continue to do nothing about it. More so I think that the "a stick your head in and attitude" of some people on this thread is not very helpful at all.

Even if the complainers do not get their desired outcome they would have raised the profile of the issue, potentially speeding up its resolution in some form or other.


@JohnE4U wrote:

@shanematthews wrote:

@steelysteph wrote:

@bambamboyo wrote:
I have also written to watchdog via the online link.
Suggest everyone does the same!

https://ssl.bbc.co.uk/programmes/b006mg74/contact

Watchdog did a piece on VM last month, they had their usual corporate answers but no real promises.  Get on Twitter and make their social media life hell.


Eh, they won't care, besides all you're doing is making the job of the guy who has to run the account a pain, it won't actually affect anyone else in VM, nor will it suddenly make any massive differences in customer retention that the hub 3 isn't already doing, at the end of the day you can whine at watchdog all you like, they can't do anything about it and at this stage VM can't really do much about it, yes people will chime in with "but they could do x" to which i tell them to read this thread and find the answers as to why they won't, most hub 3 owners won't care about or event notice the latency issues, especially if they have never had a hub 2, my mother has a connection on a hub 3 and it works fine for her and the chromecast my sister has connected to it, they don't game so they don't have any massive issues 😛


Shane, if I am correct, this is not the first time you have responded to a user suggesting people start making a fuss in other places other than the VM thread.  You have made the same points multiple times, we hear you.

This does not mean paying customers "making a noise" e.g. through a tv programme such as BBC Watchdog, is not a valid, and potentially good thing to do.  

Yes, I agree that ultimately VM did not design the chip; however, my contract is not with Intel, it is with VM, and it is VM that forced me (yes, told me I had no choice) to replace my perfectly working SU2 with one they know has major issues.  I therefore feel it is perfectly reasonably that when I complain, I am complaining about the service VM provide/the contract I pay for.  It is for VM to deal with their own supply chain.

I therefore believe people raising their complaint through BBC Watchdog is a GOOD thing.


Well i'm not stopping anyone from raising it, just pointing out that nobody will care and it won't change anything, VM will already have things in motion and nothing any of us can say will change the timeframe for that, i mean if you want to vent then feel free, personally i'll just funnel the time i would spend complaining about something i have no control over in to doing other things, ultimately watchdog doesn't even have any actual power to change anything and they won't prevent people from signing up with VM and as pointed out, most people won't even be aware that there is an "issue", i mean its a noble cause but VM already plan for bad press with their marketing and will just increase advertising to counter the bad press