CF23 Internet stability and packet loss

[RedDragonTaff]

Hi,

I am at the end of my tether.  Since the 14 Sept 20 the VM network has really gone bad.  Even today this is still ongoing.  All other levels of communication Facebook, Customer Support 150/151 has fallen on deaf ears.

So tonight here's what I see on the Hub:

Network Log:
27/09/2020 21:35:13 GMT 27/09/2020 21:35:13 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:34:59 GMT 27/09/2020 21:34:59 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:34:37 GMT 27/09/2020 21:34:37 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:34:19 GMT 27/09/2020 21:34:19 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:34:00 GMT 27/09/2020 21:34:00 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:33:39 GMT 27/09/2020 21:33:39 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:33:20 GMT 27/09/2020 21:33:20 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:33:00 GMT 27/09/2020 21:33:00 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:32:40 GMT 27/09/2020 21:32:40 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:32:20 GMT 27/09/2020 21:32:20 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:32:01 GMT 27/09/2020 21:32:01 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:31:44 GMT 27/09/2020 21:31:44 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:31:21 GMT 27/09/2020 21:31:21 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:31:13 GMT 27/09/2020 21:31:13 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:31:03 GMT 27/09/2020 21:31:03 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:30:27 GMT 27/09/2020 21:30:27 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:30:18 GMT 27/09/2020 21:30:18 GMT Error (4) 68010302 DHCP WAN IP - removed
27/09/2020 21:30:04 GMT 27/09/2020 21:30:04 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:29:45 GMT 27/09/2020 21:29:45 GMT Warning (5) 84020200 Lost MDD Timeout
27/09/2020 21:29:38 GMT 27/09/2020 21:29:38 GMT Warning (5) 84020200 Lost MDD Timeout

Downstream:
DS-1 DS-2 DS-3 DS-4 DS-5 DS-6 DS-7 DS-8
Frequency (Hz) 139000000 147000000 155000000 163000000 N/A 179000000 187000000 195000000
Lock Status(QAM Lock/FEC Sync/MPEG Lock) Locked Locked Locked Locked Unlocked Locked Locked Locked
Channel ID 1 2 3 4 N/A 6 7 8
Modulation 256QAM 256QAM 256QAM 256QAM Unknown 256QAM 256QAM 256QAM
Symbol Rate (Msym/sec) 6.952000 6.952000 6.952000 6.952000 N/A 6.952000 6.952000 6.952000
Interleave Depth I=12
J=17 I=12
J=17 I=12
J=17 I=12
J=17 N/A I=12
J=17 I=12
J=17 I=12
J=17
Power Level (dBmV) -2.30 -2.44 -2.64 -2.56 N/A -2.69 -2.95 -2.87
RxMER (dB) 36.84 37.09 36.84 36.84 N/A 38.26 38.26 36.39
Pre RS Errors 327 319 297 276 N/A 295 302 966
Post RS Errors 289 287 297 275 N/A 295 302 966

Upstream:
US-1 US-2 US-3 US-4
Channel Type 2.0 2.0 2.0 2.0
Channel ID 1 4 2 3
Frequency (Hz) 60300000 39400000 53700000 46200000
Ranging Status Success Success Success Success
Modulation 64QAM 64QAM 64QAM 64QAM
Symbol Rate (Sym/sec) 5120000 5120000 5120000 5120000
Mini-Slot Size 2 2 2 2
Power Level (dBmV) 46.50 45.50 46.00 45.75
T1 Timeouts 0 0 0 0
T2 Timeouts 0 0 0 0
T3 Timeouts 1 0 0 0
T4 Timeouts 0 0 0 0

Upstream Burst:
Req
(1) Init Maint
(3) Per Maint
(4) Adv Short
(9) Adv Long
(10) Adv UGS
(11)
Modulation Type 16QAM QPSK 16QAM 64QAM 64QAM 16QAM
Differential Encoding OFF OFF OFF OFF OFF OFF
Preamble Length 56 640 384 104 104 104
Preamble Value Offset 652 0 0 716 716 716
FEC Error Correction (T) 0 5 5 10 16 5
FEC Codeword Information Bytes (K) 16 34 34 81 223 86
Maximum Burst Size 0 0 0 3 255 255
Guard Time Size 8 48 48 8 8 8
Last Codeword Length Fixed Fixed Fixed Shortened Shortened Shortened
Scrambler On/Off ON ON ON ON ON ON

Attached is the BQM graphs daily since 14th Sept.  Can someone initiate an investigation please?

Thanks

AL

[MOD EDIT: Personal and private information has been removed from this post. Please do not post personal or private information in your public posts. Please review the Forum Guidelines]

[RedDragonTaff]

Update:

Network Log

Time Priority Description

23/06/2021 11:02:43	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:13	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:11	Warning!	Lost MDD Timeout;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:7	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:7	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:7	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:59:54	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;

[RedDragonTaff]

Upstream bonded channels

Channel Frequency (Hz) Power (dBmV) Symbol Rate (ksps) Modulation Channel ID

1	32600000	42.3	5120	64 qam	5
2	53700000	43.3	5120	64 qam	2
3	39400000	42.5	5120	64 qam	4
4	46200000	42.8	5120	64 qam	3

Upstream bonded channels

Channel Channel Type T1 Timeouts T2 Timeouts T3 Timeouts T4 Timeouts

1	ATDMA	0	0	1	0
2	ATDMA	0	0	1	0
3	ATDMA	0	0	0	0
4	ATDMA	0	0	0	3

Downstream bonded channels

Channel Frequency (Hz) Power (dBmV) SNR (dB) Modulation Channel ID

1	139000000	8.5	40	256 qam	1
2	147000000	8.5	40	256 qam	2
3	155000000	8.6	40	256 qam	3
4	163000000	8.6	40	256 qam	4
5	171000000	8.9	40	256 qam	5
6	179000000	8.8	40	256 qam	6
7	187000000	8.6	40	256 qam	7
8	195000000	8.5	40	256 qam	8
9	203000000	8.5	40	256 qam	9
10	211000000	8.4	40	256 qam	10
11	219000000	8.3	40	256 qam	11
12	227000000	8	40	256 qam	12
13	235000000	8	40	256 qam	13
14	243000000	8	40	256 qam	14
15	251000000	8	40	256 qam	15
16	259000000	8	40	256 qam	16
17	267000000	7.5	40	256 qam	17
18	275000000	6.8	40	256 qam	18
19	283000000	8.3	40	256 qam	19
20	291000000	9	40	256 qam	20
21	299000000	9.3	40	256 qam	21
22	307000000	9.3	40	256 qam	22
23	315000000	9.5	40	256 qam	23
24	323000000	9.5	40	256 qam	24

Downstream bonded channels

Channel Locked Status RxMER (dB) Pre RS Errors Post RS Errors

1	Locked	40.3	17	0
2	Locked	40.3	6	0
3	Locked	40.9	6	0
4	Locked	40.9	5	0
5	Locked	40.9	17	0
6	Locked	40.3	19	0
7	Locked	40.3	21	0
8	Locked	40.3	18	0
9	Locked	40.9	19	0
10	Locked	40.3	5	0
11	Locked	40.3	5	0
12	Locked	40.3	5	0
13	Locked	40.9	6	0
14	Locked	40.3	6	0
15	Locked	40.3	5	0
16	Locked	40.9	5	0
17	Locked	40.3	13	0
18	Locked	40.3	1	0
19	Locked	40.3	4	0
20	Locked	40.9	5	0
21	Locked	40.3	17	0
22	Locked	40.3	17	0
23	Locked	40.3	19	0
24	Locked	40.3	15	0

[RedDragonTaff]

I am hopeful that this port/tap I am on now is good.  As of just after 12:00 noon

Power level wise I seem happier, and over these past 30mins seem stable.

Upstream power 42.8 dBmV
Downstream power 8 dBmV

Very little or next to none signal degradation even with over 10 year old cabling so SNR is good. 

Hopefully we can keep it that way.  I will of course monitor over the days to come.  Third time lucky (maybe)

[lotharmat]

Are the upstream timeouts historic? or are they since they changed tap?

[David_Bn]

We'll be on hand to support you further if required @RedDragonTaff

Kindest regards,

David_Bn

[RedDragonTaff]

>Are the upstream timeouts historic? or are they since they changed tap?

Historic when I was getting 50+ dBmV upstream power.

[lotharmat]

Phew!!!

[RedDragonTaff]

Visit 3 Day 0: Senior Engineer.

BQM:

N.B. Engineers visit and work was concluded just before noon.

Note: already posted these network logs but nothing new popped in the list post noon, so we don't know from the network logs on the router why we had router reboots/sync issues after that:

Network Log

Time Priority Description

23/06/2021 11:02:43	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:13	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:11	Warning!	Lost MDD Timeout;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:7	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:7	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 11:00:7	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:59:54	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:31:26	Warning!	Lost MDD Timeout;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:31:21	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:31:20	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:31:20	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:31:20	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:29:30	critical	No Ranging Response received - T3 time-out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:29:25	critical	Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:29:23	Warning!	Lost MDD Timeout;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:29:19	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:28:53	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:28:53	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:28:52	Warning!	RCS Partial Service;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;
23/06/2021 10:28:51	critical	SYNC Timing Synchronization failure - Loss of Sync;CM-MAC=**:**:**:**:**:**;CMTS-MAC=**:**:**:**:**:**;CM-QOS=1.1;CM-VER=3.0;

[RedDragonTaff]

Upstream and Downstream power levels where as reported yesterday and haven't dropped so from that point of view the signal levels are steady.

Upstream bonded channels

Channel Frequency (Hz) Power (dBmV) Symbol Rate (ksps) Modulation Channel ID

1	32600000	43.3	5120	64 qam	5
2	53700000	44.3	5120	64 qam	2
3	39400000	43.5	5120	64 qam	4
4	46200000	43.8	5120	64 qam	3

Upstream bonded channels

Channel Channel Type T1 Timeouts T2 Timeouts T3 Timeouts T4 Timeouts

1	ATDMA	0	0	0	0
2	ATDMA	0	0	0	0
3	ATDMA	0	0	0	0
4	ATDMA	0	0	0	0

Downstream bonded channels

Channel Frequency (Hz) Power (dBmV) SNR (dB) Modulation Channel ID

1	203000000	7.4	40	256 qam	9
2	211000000	7.3	40	256 qam	10
3	219000000	7.1	40	256 qam	11
4	227000000	7	40	256 qam	12
5	235000000	7	40	256 qam	13
6	243000000	7	40	256 qam	14
7	251000000	7	40	256 qam	15
8	259000000	6.9	40	256 qam	16
9	267000000	6.4	40	256 qam	17
10	275000000	5.5	40	256 qam	18
11	283000000	7.4	40	256 qam	19
12	291000000	8	40	256 qam	20
13	299000000	8.1	40	256 qam	21
14	307000000	8.3	40	256 qam	22
15	315000000	8.5	40	256 qam	23
16	323000000	8.5	40	256 qam	24
17	331000000	8.6	40	256 qam	25
18	371000000	9.1	40	256 qam	26
19	379000000	9.5	40	256 qam	27
20	387000000	7.9	40	256 qam	28
21	395000000	7	40	256 qam	29
22	403000000	7.4	40	256 qam	30
23	411000000	7.8	40	256 qam	31
24	419000000	7.9	40	256 qam	32

Downstream bonded channels

Channel Locked Status RxMER (dB) Pre RS Errors Post RS Errors

1	Locked	40.9	6	0
2	Locked	40.3	4	0
3	Locked	40.3	7	0
4	Locked	40.9	5	0
5	Locked	40.3	4	0
6	Locked	40.3	5	0
7	Locked	40.3	5	0
8	Locked	40.9	4	0
9	Locked	40.3	4	0
10	Locked	40.3	0	0
11	Locked	40.3	0	0
12	Locked	40.3	4	0
13	Locked	40.9	7	0
14	Locked	40.3	18	0
15	Locked	40.3	18	0
16	Locked	40.3	7	0
17	Locked	40.3	0	0
18	Locked	40.9	0	0
19	Locked	40.3	0	0
20	Locked	40.3	4	0
21	Locked	40.9	0	0
22	Locked	40.9	5	0
23	Locked	40.9	5	0
24	Locked	40.3	5	0

[RedDragonTaff]

Can I ask a technical question about Downstream Power Level (dBmV)?  What is the spec range for acceptable power levels?  i.e. I know from yesterday the engineer said upstream power levels have a max of 51 dBmV before the router get upset.

I now want to know the upper limit on Downstream?  I am wondering if I am now too high on Downstream?