It sounds like the connection between the HUB and Google Nest is dropping from 1Gbps to 100Mbps.
Is it a direct wired connection between the two devices (No power line adapters in between)?
Could you try a new Ethernet cable between the HUB and Google Nest?