Yes you can have the broadband service without taking any other service, but the saving might not be much as they discount prices when you bundle services. Call them on 150 (from a VM phone) or 0345 454 1111 and take the option "thinking of leaving us" to get to the retentions team who might be able to offer you a better deal on your existing package or can help you cancel the TV part of your package.
Disclaimer - I don't work for Virgin Media. I'm just another VM user trying to help out so my answers may be wrong If my answer solves your problem please mark it as helpful as it may help others My setup: Vivid 200 Optical fibre with Superhub 2 in modem mode connected to a Netgear R7000 router. Telewest/VM user since 2001.