My guess is its a congestion issue and I too have suffered with the issue in the past, two years went by and a lot of publicity on the issue but not easy one to fix, in the end left went to a different provider.
Unfortunately it doesn't look like Virgin give a toss about providing minimum service requirements, they just let customers go if they're not happy.
Its a case of they don't have to give you anything, you just don't have to stick around and get nothing:)