Something funky with upstream latency

It sounds like you're experiencing an unusual latency issue, and you've done some impressive troubleshooting already! Here are some thoughts and suggestions to help you dig deeper:

Observations:

1. Latency Improvement with StarTrinity CST:
   • The fact that running a 1Mb stream improves latency suggests that the issue might be related to how Virgin Media's network prioritizes or handles idle connections. Some ISPs implement traffic management policies that could deprioritize idle connections, leading to higher latency.
2. Ethernet Unplugging Fix:
   • Unplugging the Ethernet cable and seeing an immediate improvement might indicate a temporary issue with your router's connection to the ISP or a problem with how the router handles idle states.
3. Time Drift and Offset Calculations:
   • Your detailed analysis of the time drift and offsets is insightful. It might point to synchronization issues between your devices and the network.

Suggestions:

1. Router Settings:
   • Check your router's Quality of Service (QoS) settings. If QoS is enabled, it might be prioritizing certain types of traffic over others. Try disabling or adjusting these settings to see if it impacts latency.
2. Firmware Update:
   • Ensure your router's firmware is up-to-date. Sometimes, firmware updates address bugs or improve performance.
3. ISP Traffic Management:
   • Contact Virgin Media to inquire about any traffic management policies that might be affecting your connection. They might deprioritize certain types of traffic during peak times.
4. Continuous Monitoring:
   • Use tools like StarTrinity CST or ThinkBroadband's Broadband Quality Monitor to continuously track latency and identify patterns. This data can be helpful when discussing the issue with Virgin Media.
5. Alternative DNS:
   • Switch to a different DNS provider (e.g., Google DNS or Cloudflare) to see if it impacts latency. Sometimes, DNS resolution issues can contribute to delays.
6. Test with a Different Device:
   • If possible, test the connection with a different device or router to rule out hardware issues on your side.
7. Escalate to Virgin Media:
   • If the issue persists, escalate it to Virgin Media's technical support team. Provide them with the data you've gathered, including the ThinkBroadband link and your observations.

This sounds like a classic case of bufferbloat or an issue with VM’s traffic management. A few things to consider:

1. Bufferbloat & Idle Line Behavior

• When your connection is idle, latency spikes could be due to excessive buffering in your upstream queue.
• Running StarTrinity CST at 1Mb may be keeping the line "active" enough to avoid the worst effects of VM’s queue management.
• A temporary fix like unplugging your Ethernet cable suggests that the modem/router is resetting its internal queues or clearing a problematic buffer.

2. VM Traffic Shaping or QoS Mechanism

• Some ISPs apply traffic prioritization, which might deprioritize lower-bandwidth traffic but improve under moderate load.
• VM has a history of latency issues when certain types of traffic don’t keep the upstream engaged.

3. Your Latency Calculation

• Your offset adjustment seems logical; you’re accounting for time drift, so it looks like your TX latency is unusually high when idle.
• A 1Mb stream shouldn’t be reducing latency unless small packets are getting stuck behind larger ones in a queue—again, bufferbloat symptoms.

4. Tests & Fixes

• Try running a constant low-bandwidth ping (like 64-byte ICMP pings every 1s) and see if it stabilizes things.
• Use a QoS solution like Cake or fq_codel on a router if you can (if VM Hub is in modem mode).
• Check if VM has maintenance or congestion reports in your area.
• If unplugging the Ethernet temporarily fixes it, that suggests a local queuing issue rather than an external routing problem.

Final Take

It’s likely a combination of bufferbloat and VM’s traffic shaping. Running CST or any persistent small upload keeps things in check because it prevents excessive queue buildup. If VM is shaping traffic weirdly, you might have to keep applying these workarounds.

Have you checked with VM if they acknowledge any traffic management changes?

One thing I did not test was ping to 192.168.100.1 so if it happen again I test that

This would not be a Bufferbloat as my connection was idle there is another bit of hardware before it get to my QoS/BQM hardware so that would suggest its not my end. But maybe the ingress buffer on the hub was looping and that running a 1Mb CST some how made latency better? But a unplug making the port link down reset it?