Menu
Reply
  • 10.92K
  • 237
  • 2.01K
Superuser
Superuser
1,037 Views
Message 1 of 74
Flag for a moderator

BN37Nx brig15 - suspiciously like downstream utilisation during last week?

SK monitor for last 7 days is indicating downstream degradation during evening peaks (approx 4pm - 11pm) which I can't correlate with any excessive downloads on my own connection:

Screenshot 2014-10-09 14.00.50.png

 

and for the previous 4 weeks prior to above:

 

Screenshot 2014-10-09 14.12.16.png

 

I can provide the detail "sample" figures for both the above examples but suspect your network monitors are already indicating this trend.  Question is I suspect it hasn't yet breached your thresholds?   Smiley Tongue

 

Presumably I'm still on cable 706 via brig15?

Regards Tony
"Life is a Binary Inspired Turing Computed Hologram"(don't PM or @Mention me - in case ignoring you offends)
0 Kudos
Reply

Helpful Answers
  • 13.01K
  • 371
  • 1.06K
Moderator
Moderator
356 Views
Message 70 of 74
Flag for a moderator
Helpful Answer

Re: F003396251 status closed? -BN37Nx brig19 - suspiciously like downstream utilise...

Hi Tony,

 

As anticipated F003396251 was closed yesterday, with in fairness downstreams looking somewhat happier over the past few days, hopefully this was down to load balancing and not the weather or the local student population going to bed earlySmiley Wink

 

Touch wood the trend continues.

 

If I come across any 'load balancing guides' for Arris I'll be sure to let you knowSmiley Wink

 

Take care

 

 

 

 

 

 

Ralph_R
Forum Moderator

The do's and don'ts. Keep the community welcoming for all. Follow the house rules



All Replies
  • 10.92K
  • 237
  • 2.01K
Superuser
Superuser
1,022 Views
Message 2 of 74
Flag for a moderator

Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?

I'm fairly confident this hasn't breached VM's threshold but perhaps a better perspective can be seen by comparing the previous 6 months hourly average with the last weeks hourly average.

 

We don't actually know VM's utilisation threshold  but expect the downstream (and thus possibly upstream) to drop during peak periods anyway.  The question is by how much and for how long?

Personally I use two empirical metrics:

1. If I obtain 75% of headline speed at any time of day then I consider that is not worth commenting/investigating further.

2. If it falls to(or below) the next lower service tier headline speed during any hour of the day (on average during a complete week) then I'd expect VM to already be taking some remedial action.

 

Since I'm only on lowest 50/3 tier then I use my previous legacy 30/2 tier as comparison and thus derive 37mbps and 30mbps respectively for 1 & 2 above downstream data rates.

I accept this is debatably a far more liberal degradation than most/all other users would readily accept (who would more likely be looking at say >95% and >75% respectively?).

 

That said here's approx 6 months(hourly average) from March27th to 2ndOctober 2014:

chart_1412918902.png

From which we can see that 9pm low point was still 96% of headline speed.

Whereas during last week from 2ndOct to 9thOct:

chart_1412918452.png 

We see that downstream dropped to (and below) 80% for at least 2hrs during peak time and almost to the lower (previous) 30mbps tier headline rate on 1 hourly (9pm) sample period!

 

Just interested to see how close that weeks (average) downstream results came to VM's escalation threshold?  

Of course within the next 5days when Level2 manage to respond then another weeks average may have already improved anyway! Smiley Wink

Regards Tony
"Life is a Binary Inspired Turing Computed Hologram"(don't PM or @Mention me - in case ignoring you offends)
0 Kudos
Reply
  • 10.92K
  • 237
  • 2.01K
Superuser
Superuser
1,008 Views
Message 3 of 74
Flag for a moderator

Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?

Still testing/monitoring my SH2AC and noticed some interruptions to SK hourly sampling (which has connected via SH2AC in RouterModem on my Lan topology) but still further indications of downstream utilisation during evening peak hours:

 

10Oct - 15Oct inclusive  (you can see no samples after midday 13th until- 6am on 14th)

Screenshot 2014-10-15 07.23.49.png

 

While the upstream doesn't drop below 3mbps (tier headline minima) it does vary erratically above that.

 

The TBB monitor doesn't display any significant peak increases (normally associated with upstream utilisation) - Here's a snapshot for the 14th that dropped to around 37mbps at 9pm:

My Broadband Ping - brig15-50/3BN37N-SH2ACRM

 

 

Also another Hove user has posted with an alleged utilisation issue (but no further details of CMTS,  Fault Reference or more exact postcode for comparison) but was told by visiting tech it was an area UBR issue?.  (Ref > how-do-I-find-out-when-my-local-UBR-is-to-be-upgraded < )

Other posts in recent weeks (albeit BN1/BN2) have indicated area downstream utilisation issues and while I can't be sure whether that's related to brig15, I do know that load can be balanced from same chassis across several nodes across disparate major postcodes (eg brig18 on a BN3 5xx and BN2 6xx node).

 

For those rather tenuous reasons I suspect downstream utilisation on at least one of brig15 linecards? (Although equally it may be eqam/combiner?)  

 

 

Regards Tony
"Life is a Binary Inspired Turing Computed Hologram"(don't PM or @Mention me - in case ignoring you offends)
0 Kudos
Reply
  • 10.92K
  • 237
  • 2.01K
Superuser
Superuser
997 Views
Message 4 of 74
Flag for a moderator

Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?

Had a time wasting 7min19sec with offshore agent who clearly couldn't appreciate I was detailing a downstream reduction only during Peak hours and unilaterally rebooted my SH2AC (with magic hits on both wired and wireless while refusing to elaborate on what "changes he allegedly was making")! Grrr...

Got quickly shot of his time wasting and transferred me to "Sally" in CR(Sheffield) who valiantly attempted to Xfer me to Swansea (who were obviously too busy at the time....)!

 

Sally at least re-negotiated my BB+TV+loyalty discount so my grateful thanks to her and despite her offers to ring me back once she managed to get Swansea I decided to re-attempt later in day myself.

Most efficient, pleasant AND more importantly, clearly understandable lady!

 

So some poor unsuspecting Level2 in Swansea might be getting a phone call later!  Smiley Tongue

 

 

EDIT: Update - just noticed Nicola's reply on an0ther users thread that I hijacked to confirm there's utilisation but not yet at escalation threshold so I'll not bother Swansea with a phone call yet.

ForumTeam   heaves a sigh of relief!  Smiley Tongue

Regards Tony
"Life is a Binary Inspired Turing Computed Hologram"(don't PM or @Mention me - in case ignoring you offends)
0 Kudos
Reply
  • 10.92K
  • 237
  • 2.01K
Superuser
Superuser
984 Views
Message 5 of 74
Flag for a moderator

OT Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?

OK Hopefully a final update for at least a month (or two).

I've marked the thread answered (can I do that with my own post? Someone else got recently banned for flagging his own posts as answers. I recall? Smiley Tongue )

Basically based on Nicola's response in the other thread as I suspect even though the downstream utilisation will worsen in coming months (and probably over Xmas) I've every confidence it will eventually result in a Networks/Planning Fault reference (albeit probably with a Easter/Summer 2015 resolution date! 

 

So until the service degrades it's marked as answered so as not to distract ForumTeam from more important priorities! 

So why am I still so happy about this potentially future degraded service?

 

<OFF TOPIC>: TL;DR  or unless you're specifically interested in my personal experiences of re-negotiating my own bundle/contract?

OK to set the scene a couple of months ago I rang 0800 sales number to explore what VM could offer me in expanding my bundle to include Mobile Data (I spend 2 out of 3 weeks on average in a Selsey beach hut with no cabled service nor ADSL/VDSL alternatives). 

The "offshore" agent appeared quite enthused and said in addition to the Mobile data connectivity she could actually enhance my existing bundle to 152/12 BB for £6 pcm less than I was paying!

Too good to be true? Well she refused point blank to send any written confirmation prior to "signing me up" so naturally I declined her verbal (albeit "recorded") offer!

 

Now my usage profile actually doesn't require much more than a 10mbps tier anyway but as stated when my loyalty discount terminated on 18Sept then Sheffield Sally actually offered me the same deal with Big Kahuna including the Tivo (w/o any installation charge) at again £6 pcm less than I was paying previously!

 

Ahh what's the catch you ask.  Yes another 12month contract but I've been(and will probably die) with VM for last 15yrs, so 12mths (including an inevitable annual 2015 price rise)  doesn't exactly deter me in the long term.

 

More importantly having chatted to Sally this morning (while waiting for those d*mn lazy ForumTeamers to pick up their phone in Swansea, Smiley Wink ) I subsequently received two VM emails advising me  of a hardcopy contract in post in 3 days time and details of my upgrade within the hour!  (So why didn't the offshore agent tell me that 2 months previously because you would have got a happy customer a lot earlier?)

 

Well when we see lot's of negatives on this forum about how users were allegedly "mis-sold" their bundle and received  "unexpected" charges, then one can be forgiven for being somewhat cynical and/or questioning about what "alleged" sharp practises VM are employing?

 

So when I looked at the "Bundle details" that actually stated my BK Bundle package would be £6 MORE than I agreed on telephone then I thought: "Aha! Got yer VM" (Naturally with agents permission I had my auto recording device enabled)

So I thought I'd ring again this pm and see how they wriggle out of that clearly "mis-sold" package followed by some determined complaints, threats to OFT/OfCom/Brighton evening echo/voluntary adjudication etc etc....   Smiley Wink

 

Off course none of that "heavy handed approach" was necessary Smiley Wink because speaking to another courteous and patient agent called Jade in CR this pm, it transpires that the Bundle change email currently never does list the "Loyalty discount"....   BUT the forthcoming hardcopy contract WILL list the loyalty discount!!!!!    Duhhhh! why make a "rod for your own back" VM process management?

Furthemore Jade assured me that since Sally had noted on my customer notes that the Package price was actually EXACTLY as per our am telecon, THEN EVEN IF IT WORKED OUT MORE EXPENSIVE THEN VM WOULD HAVE TO HONOUR THE LOWER VERBAL QUOTE!!!!

 

So both (UK) agents (were recorded and) singing from same hymn sheet, which means what?

Am I gullible, was I just lucky and bucked the statistical trend? Did being a (GOLD) long term customer alter the contract? Did having a multi service bundle affect it? ....or....

Perhaps a UK CR(ala Retentions) agent is just consistently more (understandable), flexible and knowledgable about the products they sell than an offshore CS agent!

 

That's only the 4th time in 15yrs I've felt the the need to ring Retentions (and perversely the idiot Offshore TS agent this morning did me a favour by starting to digress in a time wasting NON-TECHNICAL explanation that forced me to pursue that Retentions route to start with!) and they've(UK) always satisfied my questions so why can't offshore measure up to the same consistent standards as the UK !!!!  Smiley Wink

 

Apart from that and wearing my cynical "hat" again, then yes my 152 tier upgrade might not happen tomorrow, the current downstream utilisation may degrade my headline speeds anyway, my Tivo may not get installed on 3rdNovember and VM may slip in a few 100 quid extra on my first amended bill.....

 

Watch this space......   or not...   Smiley Tongue

 

Regards Tony
"Life is a Binary Inspired Turing Computed Hologram"(don't PM or @Mention me - in case ignoring you offends)
0 Kudos
Reply
  • 13.01K
  • 371
  • 1.06K
Moderator
Moderator
963 Views
Message 6 of 74
Flag for a moderator

Re: OT Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?

Hi Tony,

 

Sorry we've not got to your thread a little sooner, I just thought I'd confirm here what Nicola has mentioned previously, in that your cable is showing the noted spikes in peak time utilisation, though these levels, as you have already surmised, remain a fair way off that we can escalate at the moment.

 

I can see though that you've received your broadband tier upgrade and eagerly await the next instalment on how the rest of your bundle goes.

 

I know you'll keep us posted Smiley Wink

Ralph_R
Forum Moderator

The do's and don'ts. Keep the community welcoming for all. Follow the house rules


  • 10.92K
  • 237
  • 2.01K
Superuser
Superuser
934 Views
Message 7 of 74
Flag for a moderator

Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?


Ralph_R wrote:

.... in that your cable is showing the noted spikes in peak time utilisation, though these levels, as you have already surmised, remain a fair way off that we can escalate at the moment......


Thanks for that Ralph and here's the update as downstream utilisation appears to have increased significantly in last 3 weeks!   

 

First key dates:

  • 150/12 upgrade on 17th Oct 
  • Tivo install circa 4pm on Mon 3/11  Note that another Tivo installed on my cable segment 24 houses away (thats my house number + 42) a couple of hrs previous. While I wouldn't expect that to affect BB utilisation then potentially sharing the same combiner/eqam port might be relevant.

Results:

 

 Hourly average for week 17/10 - 23/10

Screenshot 2014-11-05 06.12.59.png

Screenshot 2014-11-05 06.13.25.png

Screenshot 2014-11-05 06.13.59.png

 

Hourly average for week 24/10 - 30/10

Screenshot 2014-11-05 06.24.05.png

Screenshot 2014-11-05 06.24.22.png

Screenshot 2014-11-05 06.24.54.png

Hourly average for (PART) week 31/10 - 5/11

Screenshot 2014-11-05 06.37.10.png

Screenshot 2014-11-05 06.37.45.png

Screenshot 2014-11-05 06.38.10.png

Interim Conclusion:

It's the trend over those 3 weeks that concerning(?) me.  If that continues/worsens (rather than just a half term blip) then this node is going to look very sick over Xmas (VM work moratorium) period! 

It's of course entirely possible that my neighbour up the road is thrashing his new Big Kahuna bundle and downloading the internet 24x7 this current week:

Last 2 days by hour

Screenshot 2014-11-05 06.51.02.png

Screenshot 2014-11-05 06.51.19.png

Screenshot 2014-11-05 06.51.40.png

The downstream was actually consistently lower than any one day sample of my previous 50 Tier since March!

I'm not totally convinced this is entirely down to utilisation on my segment as it's hard to explain the following - 

 

Misc/Circumstantial/Co-incidental Observations:

 

  •  Around 4am-5am on Tuesday 4th my Tivo spontaneously rebooted! 
  • A couple of hours later at 7:18 am same day my SH2AC threw a couple of Sync errors, half a dozen MDD's and rebooted itself.

I assume NOC/Network were "tweaking" the brig15/EQAM servicing my cable segment at the time and these two incidents weren't just co-incidental unrelated hiccups?

 

Curiously my SH2AC downstream then came up with "zero" RXMer on one of the 8 downstream channels! I just rebooted again today and normal function resumed, so I'm assuming that's more like a Hub Tuner/firmware anomaly than co-lateral result of any ongoing Network maintenance activity?

 

QUESTION for ForumTeam:  

Considering all the above, then is downstream utilisation on my brig15 BN37Nx node/cable segment showing a similar trend degradation from your stats and assuming this continues how long before you suspect it will breach your thresholds?  I appreciate you can't disclose your threshold but can you provide a % increase in utilisation in last 3 weeks instead?

Or should I/we be looking also for other possible contributing issues on my connection as well?

 

Regards Tony
"Life is a Binary Inspired Turing Computed Hologram"(don't PM or @Mention me - in case ignoring you offends)
0 Kudos
Reply
  • 10.92K
  • 237
  • 2.01K
Superuser
Superuser
927 Views
Message 8 of 74
Flag for a moderator

UPDATE Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?


horseman wrote:......

QUESTION for ForumTeam:  

Considering all the above, then is downstream utilisation on my brig15 BN37Nx node/cable segment showing a similar trend degradation from your stats and assuming this continues how long before you suspect it will breach your thresholds?  ......

Or should I/we be looking also for other possible contributing issues on my connection as well?

 


I suspect I've answered my own question and @ForumTeam will confirm that utilisation is still no where need escalation threshold! :

Screenshot 2014-11-05 08.49.17.png

as we can see above - after the SH2AC reboot at 05:22am this morning then subsequent d/s rate has returned to normal?

 

If we look at the corresponding ICMP TBB/BQM:

Screenshot 2014-11-05 08.49.17.png

 

Which shows no sign of increased latency trend then this "appears" to indicate a potential "non optimal" (TCP) buffering issue with Hub which had only c 1.5days uptime since last previous reboot and this morning's reboot! 

 

So it seems like I now need to correlate system uptime, downstream data rate/utilisation, along with any (premature)DHCP RENEW and/or network maintenance resets with the weekly average hourly downstream results.

 

Once Level2 have confirmed(or otherwise) the actual utilisation trend hasn't significantly worsened  then I'll consider marking the thread "answered" again and revert to my original intent of only monthly updates.

 

I also suspect Level2 will not be able to comment on my (circumstantial) non-optimal buffer bloat (sizing) hypothesis and request I continue addressing that separately on NHBB Trials forum anyway?

 

Regards Tony
"Life is a Binary Inspired Turing Computed Hologram"(don't PM or @Mention me - in case ignoring you offends)
0 Kudos
Reply
  • 10.92K
  • 237
  • 2.01K
Superuser
Superuser
888 Views
Message 9 of 74
Flag for a moderator

WEEKLY UPDATE Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?

I've stopped effort on stress testing SH2AC (until V10 arrives probably after Xmas change moratorium) but confined the results to date to my > NBHH Trials thread <

 

Meantime I'm monitoring (effects of) utilisation on my cable segment (crudely) as follows:

 

Weekly hourly average DOWNSTREAM:

Screenshot 2014-11-10 09.49.06.png

Below 85% of headline speed for all the peak period. [See Note *1, 2i]

 

Weekly hourly average UPSTREAM:

Screenshot 2014-11-10 10.49.21.png

 Not dropped below 85% of headline rate during peak period [See Note *1, 2i)]

 

Weekly hourly average UDP LATENCY:

Screenshot 2014-11-10 10.52.05.png

Not risen above threshold during peak period (see Note 1, 3]

 

Weekly hourly average RTP JITTER UPSTREAM

Screenshot 2014-11-10 10.59.03.png

Not risen above threshold during peak period [See Note 1, 4]

 

 

Conclusion:

Of course this doesn't/won't correlate exactly with @ForumTeam  brig15/cable segment (7060?) utilisation stats nor their escalation threshold which I suspect is somewhere around 85% sustained during at least a weeks hourly average during peak times.

 

On that basis the degradation(on Level2's utilisation stats) over the last 3 weeks since Level2's last response will undoubtedly mean it is still no where near the escalation threshold yet! Smiley Tongue

 

However it would be interesting to know whether it has subjectively increased (or not) even if any relative percentage increases still cannot be disclosed?

 

 

[*]Notes - Explanation(tl;dr):

Of course irrespective of actual utilisation peaks,averages etc from the CMTS/network/node stats the important thing to the user is the actual perceived performance metrics.   

 

So here's my personal interpretation:

1. I'm looking at peak time between 4pm and 11pm (ignoring the fact the weekend peaks are different) between 3 and 9th November inclusive as an hourly average for that week.

2. Assuming two data rate limits:

i) Abitrarily using 85% of headline speed for D/S & U/S hourly average over a week which will be ~130mbps and 10.2mbps resp. (these would only be flagged if they were below their limit for peak period duration respectively)

ii) Similarly using 85% reduction in headline which will be 12mbps and 1.8mbps resp which would be flagged if these levels weren't achieved for 15% of the peak period duration.

3. For UDP Latency then I arbitrarily look for periods where the peaks are triple the minima and flag if they are more than maxima during  the peak period. In example above the minima is 18ms and maxima would therefore be 54ms.

4. For RTP UPSTREAM JITTER then I'm using same ratio again yielding 1.6ms and 5.2ms resp.

5. For brevity the RTP DOWNSTREAM JITTER is omitted as implemented interleaving will normally reduce this anyway until downstream utilisation is significantly far worse anyway.

 

All the above roughly gives me a cost & performance comparison with my original 10/1 Tier a couple of years ago. (Since perversely the cost hasn't risen more than normal annual price increments despite the upgrades and my limited usage was adequate at those original tier rates!)

Regards Tony
"Life is a Binary Inspired Turing Computed Hologram"(don't PM or @Mention me - in case ignoring you offends)
0 Kudos
Reply
  • 13.01K
  • 371
  • 1.06K
Moderator
Moderator
874 Views
Message 10 of 74
Flag for a moderator

Re: WEEKLY UPDATE Re: BN37Nx brig15 - suspiciously like downstream utilisation during last week?

Morning Tony,

 

Thanks for all the data and analysis, looks like utilisation has been on the climb ( it not you driving this Tony is it Smiley Wink) and has been breaching network thresholds this week, I've raised this over to them for their consideration.

 

As long as the utilisation doesn't quieten down again over the next day or two we should get a fault reference and possibly an idea of what's planned to mitigate the problem.

 

Thanks

 

 

Ralph_R
Forum Moderator

The do's and don'ts. Keep the community welcoming for all. Follow the house rules