There are problems right across the Portsmouth/PO area. They might (or might not) be affecting small areas on an individual basis, but if you search this forum you'll see that Portsmouth and PO comes up an awful lot. This could indicate a wider problem with old infrastructure that VM needs to update/replace. Or it could be not enough capacity for all the customers that they continue to sign up despite no longer being able to deliver an effective service at peak times. More likely it's both.
They claim that they don't throttle, but I can't believe it's coincidental that between 6pm and 12am everyone's speed in overutilised areas just happen to settle around the 5Mbs mark in our area. And I don't believe its coincidental that it matches the best speed you're likely to get on BT, Sky, Talk Talk non-fibre based services. Currently there's no realistic alternative in our area to VM. When there is hopefully things will change or I'll change, but there's no incentive for VM to fix the problems dogging them in the PO postcode.
Just to evidence this, I'm affected by a utilisation fault that's been ongoing since October 2015 with peak time performance around 5Mbs. It was "reviewed" on 29 March and nothing was done to address the problem. The next review date is 29 December! Other people in the PO postcode are being told the same thing.
You have two choices. Like it or lump it.
"Cable broadband is typically faster than copper-based services, but a significant minority of users experience severe slowdowns in peak times." — Ofcom, UK Home Broadband Performance, November 2016