The most obvious cause of OnDemand issues only at certain times would be internet speeds slowing down, although I have some doubts that's the main reason in your case.
When you next have problems, there's 2 things you can try...
1: If you're not recording anything and are able, try restarting your box - Home > Help & Settings > Clear or restart my box. I'm doubtful, if you say this error self-clears anyway (and it will interrupt any recordings), but it's something to rule out.
2: Regardless of whether you're recording or not... Home > Help & Settings > Help > (look along the top now) > TV Speed test.
I'm a Very Insightful Person, I'm here to share knowledge, I don't work for Virgin Media. Learn more
Have I helped? Click Mark as Helpful Answer or use Kudos to say thanks