Menu
Reply
  • 98
  • 6
  • 28
andybundy
Up to speed
104 Views
Message 1 of 3
Flag for a moderator

Waterlooville/Crookhorn - PO7 8.. Continued Outage after unannounced Maint. Work

Posting on behalf of my Mother as she STILL has no TV or broadband since Thursday.

Dateline 8th December 2016, 0900hrs.  TV and broadband cease to work.  Around 5 minutes later a card is dropped through letter box announcing that services would be off for no longer than four hours, and stating a final finish of 6pm, and giving phone numbers to ring.

Phone service resumed around 1400hrs, but as of now, 1045 on Sunday 11th December, TV and b/band are still not working despite considerable numbers of calls to 150, the majority of agents who have no idea, and no apparent ability to search the change/fault management system and therefore decide it is a fault, and assign the problem to the usual fault channel which means that it will get fixed eventually, at a time when you are least available or able to stay home!

This leaves us with numerous questions:

1.  Why was the work not announced to local residents before it actually started?  Surely it must have been planned, as any equipment would have had to have been ordered, engineers assigned, etc, and I'm sure this is not all planned on the day.
2.  Why did all the provided numbers, supposedly intended to take customers directly to people involved in the project, all redirect to the usual 150 staff - most of whom had no idea apart from saying 'oh, the PO7 8** problem.
3.  Why did the engineers simply walk off the job at the end of the day, without tagging potentially affected accounts, without updating customers, and with connections not working (and please don't tell me the equipment cannot be queried to make sure all paid-for connections are indeed working).
4.  Given the problem was caused BY VM engineers - why is resolving the problem being left to normal maintenance staff who likely enough have no idea what upgrade work has been carried out, and who potentially may not even be trained (yet) on the new equipment?

We've learned that Sky have expressed some surprise at the number of new applications that postcode all in one weekend, which means at least TWO customers have already thrown in the towel and gone over to the 'dark side'.  Indeed, when my Mother expressed to the customer service agents her intent to look elsewhere, the rep was less than interested.

As a former change manager, in charge of assessing and reviewing planned activities much larger in scale than this, I can only class this as a grade 'a' **** up, most likely caused by just too much Christmas cheer.

Somebody in VM needs to review this incident, implement proper change management, and review testing and validation plans for each work activity which causes an outage.  For sure, such things really ought NOT be done during the day, and equally there should be an approach which causes less of an outage.

The old adage about pennies - "look after the pennies, and the pounds will look after themselves" applies to customers.  Lose one customer, shame on you.  Lose more customers, start looking for heads (to roll).

AndyBundy
-----------------
I don't work for VM. I am medically 'retired' after working for a large multi-national Telco and have spent fifteen years in WAN/LAN IP Networking, Network Management and IT Support.
0 Kudos
Reply
  • 5.06K
  • 180
  • 309
Forum Team
Forum Team
49 Views
Message 2 of 3
Flag for a moderator

Re: Waterlooville/Crookhorn - PO7 8.. Continued Outage after unannounced Maint. Work

Hello andybundy,

 

Thank you for posting in the community and I'm very sorry to learn that your mother was experiencing a loss of TV and Broadband.

 

Depending on whether there's a fault in the area or if it's planned maintenance work it can sometimes affect a large amount of people so it can be difficult to contact everyone. We do try our best to inform people if we think the problem is going to take longer than expected and we also contact people to check whether the issue has been resolved but again this can be a significant amount of people so we also rely on our customers to let us know if there's any problems.

 

I'm unable to find the account as you have mentioned it's your mother who was being affected but please let me know if there's any further problems and I'll investigate it further.

 

Kind regards

 

 

Rose
Forum Team


New around here? To find out more about the Community check out our Getting Started guide


0 Kudos
Reply
  • 98
  • 6
  • 28
andybundy
Up to speed
39 Views
Message 3 of 3
Flag for a moderator

Re: Waterlooville/Crookhorn - PO7 8.. Continued Outage after unannounced Maint. Work

Apologies for not getting back on this one.  I can confirm that my mother has been suitably 'looked after' by the retentions team, especially after the third attempt by Sky to get her VM telephone number released so she could switch over accordingly - seems that is the only way to get any attention, let alone timely or informed repair out of VM on some occasions.

The leaflet, and the engineers, clearly stated that one of the local green boxes specific to that one postcode was being upgraded as part of a planned programme - not a repair - not last minute - unlike the card which landed on her doorstep five minutes after everything stopped working.  I know the work was limited because at one point there was a recorded message referring to the postcode right down to the six character level - with 15-30 houses.

As my footer mentions, I am/was (a little weird as I'm still employed but unable to work) as a change manager for a VERY large telco, by which to say one with millions of US domestic customers and thousands of accounts world-wide from $5million and higher customers.  Said company also owns much of the cross ocean fibre linking the world together.  I've worked in both the (bizarrely) larger US change management arena, out of hours, reviewing and approving high level changes, and subsequently internationally across dozens of different countries.

In whatever environment, a 'change' which is the generic term for an activity such as a hardware upgrade, requires:

Ownership/Justification/Cost-Benefit Analysis.
Proper planning involving all stakeholders, including network designers, engineers, call centers and account managers
Test plans to be executed before, during and after the change.  Backout plans if things don't work, and test plans to be executed again to make sure everything is as it was.
Agreement from the Change Management Board to proceed.

In this case, it was not a fault.  Numbers were provided should service not resume, but nobody answering them had a clue.  Nobody working the change came around to check the necessarily limited number of customers, nor were the (quite impressive) array of monitoring tools employed to ensure that all affected customers were re-connected.  Either way, once called in the following day (my Mum is very patient) it should have been prioritised and passed back to the team working the project, instead it was passed around half the company and allocated as a fault, with a five day wait.  That it was not a fault, and caused by your engineers was explained, and ignored.

All in all, in my world, a massively failed change, and one which would have ensured the 'guilty parties' were grilled slowly over the coals of a Vice-President level outage review to ensure in painful detail that the causes were understood, lessons learned and evidence of documented processes handed over to prove that.

We live in a world which demands more, and there is never a good time to perform such upgrades, though in our cases we were generally obliged to do them in the evenings, overnight and preferably over the weekend.  There is no reason, if this work is properly planned, that customers cannot be informed much earlier, and if a threshold number express a problem with the time, then a reschedule considered.

Even the most demanding of customer, like myself, will forgive a fault/outage caused by accident or the proverbial JCB (enemy of fibre cables the world over), but if you continue to make such avoidable errors, and then either hide or lie about them, and not put things right ASAP, customers will leave.  We were indeed amused to learn that in my Mum's postcode, Sky were surprised to see three new applications in one morning.  After my Mum explained why, I hear it took them a while to stop laughing.  If the remaining customers who were not so badly affected had the level of understanding that I've documented above as to how such work SHOULD be planned and executed, you would lose the entire lot.

Luckily, my Mum really did not want to leave VM (but had felt no choice) and was offered a considerable incentive over the next two years by way of an apology for that outage, as well as the vast and continuing number of problems her small area keeps experiencing.

While I am sure that advice from a simple lowly customer will always be ignored, I would recommend  anybody in the IT business to have an awareness of ITIL (IT Infrastructure Library) which is a comprehensive and all encompassing set of guidelines and processes written to define and support the management of every part of an IT-based operation.  If VM is not already using it, then now might be the time to research it.  It's "Made in England" and good enough to be adopted by companies all over the world, from small to massive, and it works.

AndyBundy
-----------------
I don't work for VM. I am medically 'retired' after working for a large multi-national Telco and have spent fifteen years in WAN/LAN IP Networking, Network Management and IT Support.
0 Kudos
Reply