I have had the Xbox Series X since launch, and have Virgin M350 Fibre. Ever since I set the console up (over Wifi), all other devices seem to struggle with speeds, some even dropping connection altogether intermittently. I have quite a lot of devices attached, but also did before getting the Series X so can't see that being the main issue.
I have tried using a powerlink to connect the Xbox to the Hub directly before, but had issues for about 6 months and it turns out the powerlink was making the Hub trip and cutting the whole broadband off, so connecting like that is not an option (if that would help, that is).
I've still got my Xbox One and One X and can run them the same time as my Series X, which is my main console and used most days, all on the Wi-Fi (5 GHz) without any problems, as well as there being a PS4 on the network (2.4 GHz) and many other devices streaming, homeworking etc. on both bands. However, I use a good third-party router (Asus AC86U) and not the Super Hub for router duties, so that makes a difference.