Sorry guys, but its still happening, causing severe usability issues.
This is what I see from a constant ping to google dns when the issue occurs:

No heavy activity at the time. Normal usage according to the SNMP monitor of the external interface of my router:

Pings are fine to the router locally:

Router CPU showing no issues:

But pings to lutn-core-2a-xe-7113-0.network.virginmedia.net shows my problem perfectly.

This is causing bigger and bigger problems over time. Its becoming more and more unreliable as the problem continues and seems to be getting worse. This is clearly not my end and If this cannot be resolved I will need to start looking at other solutions.