Hub 4 modem mode: regular issues with SSL traffic

[heliosfa]

Hello,

For the past decade or so I have been running a Virgin connection with the hub in modem mode connected to a pfsense firewall. For the most part, my setup has was working fine on the 200 Mb/s service with a Hub 2ac for a fair few years. I upgraded on the 18th of January to the 500 Mb/s service with a Hub 3 and this was also working fine. The problems started on the 26th of January when I was upgraded to Gig1 with a Hub 4 as a Volt benefit.

Since then, I am having a regular issues with SSL traffic to *some* destinations breaking when the Hub 4 is in modem mode. In Firefox, this manifests as the browser hanging on "Performing TLS Handshake" until it times out. In Chrome it hangs on "Establishing Secure Connection".

Some of the problem sites include:

• https://www.bbc.co.uk/news
• https://www.hobbycraft.co.uk
• https://community.virginmedia.com
• https://www.netflix.com

When the problem occurs, I also have issues tunneling traffic (e.g. remote desktop) over SSH. Other websites (e.g. https://old.reddit.com and https://www.speedtest.net ) and anything routed over my Hurricane Electric IPv6 tunnel continue to work absolutely fine. Online game traffic (CS:Go, Tarkov, etc.) also seems to be unaffected. Speed tests continue to report the full expected speed.

Pings to the problem sites continue to work. Even the redirect from news.bbc.co.uk to www.bbc.co.uk/news works, the browser just hangs after the redirect.

Sometimes the problem is transient and resolves itself after a few minutes. Other times it takes a reboot of the Hub 4 to resolve. Rebooting the firewall does not resolve the issue and the issue occurs with other devices (including an alternative router and a straight Windows desktop) in place of the firewall.

Packet captures show that the traffic is correctly leaving the firewall's WAN interface.

Also, the admin interface for the Hub 4 at 192.168.100.1 stops responding within an hour or two of rebooting the hub. Ping to the hub also become sporadic with only one in every ten or 20 pings eliciting a response. It takes a reboot of the Hub 4 to regain access to the admin interface.

The issues (including the admin interface one) do not occur if I introduce a double-NAT scenario with the Hub 4 in "normal" mode with the firewall in the DMZ. Obviously this is not an ideal long-term solution.

Has anyone else come across this? is it a "feature" of the Hub 4 or have I just got a bad one?

I am dreading trying to explain this to support over the phone as it is an intermittent issue and involves modem mode.

[legacy1]

But you do get a new IP from modem mode to router mode when you do double-NAT so its likely a change in IP thats done it.

Connect a PC to the hub in modem mode and check if your not in a subnet you was in in modem mode with the pfsense.

[BaldrickBravo]

---

@legacy1 wrote:
But you do get a new IP from modem mode to router mode when you do double-NAT so its likely a change in IP thats done it.

---

^^ I doubt this to be the case.

[BaldrickBravo]

It's TLS, not SSL. Well, I hope it is TLS.

Have you got Snort/Suricatta running on pfSense? Do you have any other pfSense packages installed?
How are your upstream/downstream power/SNR levels and post-RS errors looking? If they are healthy and there are no ranging response timeouts etc. in the hub's logs, I would have a look at pfSense's system logs. And probably get into capturing some packets, both on the Firewalls WAN connection and the client computer. Could probably yield some clues as to what is happening.

I would also look at testing the maximum MTUs between yourself and some of the websites you have been trying to visit of TLS. (You can do this with flags to ping (under Linux) that set a "don't fragment" and the payload size).

Have you set MSS/MTU values at all on your Firewall's VM WAN connection, or left them at default?

[legacy1]

---

@BaldrickBravo wrote:

---

@legacy1 wrote:
But you do get a new IP from modem mode to router mode when you do double-NAT so its likely a change in IP thats done it.

---

^^ I doubt this to be the case.

---

So why would double-NAT work as said by heliosfa?

[heliosfa]

---

@BaldrickBravo wrote:

It's TLS, not SSL. Well, I hope it is TLS.

Have you got Snort/Suricatta running on pfSense? Do you have any other pfSense packages installed?
How are your upstream/downstream power/SNR levels and post-RS errors looking? If they are healthy and there are no ranging response timeouts etc. in the hub's logs, I would have a look at pfSense's system logs. And probably get into capturing some packets, both on the Firewalls WAN connection and the client computer. Could probably yield some clues as to what is happening.

I would also look at testing the maximum MTUs between yourself and some of the websites you have been trying to visit of TLS. (You can do this with flags to ping (under Linux) that set a "don't fragment" and the payload size).

Have you set MSS/MTU values at all on your Firewall's VM WAN connection, or left them at default?

---

pfsense is version 2.5.2 with pfBlocjerNG and openvpn-client-export as extra packages. I am not running Snort/Suricatta.

As stated in the OP, I have already been running packet captures - responses from the far end just stop during TLS handshake. This manifests on packet captures on the host, pfsense's WAN interface and via a network tap inserted between the hub and pfsesne.

I have just rebooted the hub to get these:

Upstream power ranges from 40.52 dBmV to 43.77 dBmV across the four channels.

Downstream power ranges from 0.7 dBmV to 3.4 dBmV across the 31 DOCSIS 3.0 downstream channels. RxMER ranges 40.3 dB to 40.9 dB.

The DOCSIS 3.1 channel has a RxMER of 43 dB and a PLC Power of 2.9 dBmV.

I have not seen any post-RS errors, but as the interface packs up quite soon after reboot I am unable to see exactly what is going on when the issue is occuring. There is nothing in the Networklog of interest - the only thing is a "No Ranging Response received - T3 time-out;"  and a "MIMO Event MIMO: Stored MIMO=-1 post cfg file MIMO=-1" associated with each reboot (the t3 timeout has a 1970 timestamp each time)

MSS/MTU are at default except for the HE gif interface being set to 1470. I will check MTUs between myself and some of the problem sites when they are working and when it breaks.

As mentioned, this ONLY happens in modem mode and occurs on different devices.

[legacy1]

Change the WAN MAC on your pfsense router and get a new IP.

[BaldrickBravo]

Because the IP address is arbitrary.

In my experiments I haven't seen any routing changes as a result of spoofing the hub MAC address or using the FW WAN port's native MAC address. The IP address provided is in the same subnet and traceroutes look identical.

[BaldrickBravo]

---

@heliosfa wrote:

As stated in the OP, I have already been running packet captures - responses from the far end just stop during TLS handshake. This manifests on packet captures on the host, pfsense's WAN interface and via a network tap inserted between the hub and pfsesne.

Puzzling.

You don't see any resets? Or re-transmissions?

I'm wondering if it is a generic problem that is more broken with TLS.
Are you able to run iperf3? It will show the number of packets re-transmitted in the output.

Do you have a BQM setup?

[legacy1]

---

@BaldrickBravo wrote:

Because the IP address is arbitrary.

---

In a perfect world yes but we don't