Reply
Settled in nicely II
blaise
Posts: 28
Registered: ‎09-09-2011

"Corrupted" downloads (since R36?)

Hi,

 

I am a user of the super hub, didn't have much trouble with it until about 3 days ago. I think this may coincide with the date of the upgrade the new R36 firmware?

 

Anyway, for the last 3 days I have been experiencing corruption / interruption in the downloads of "large" files (i.e more than 2 Megs). It sometimes takes 5 attempts in chrome to download a file more than 3Mo from google (sad, isn't it?).

 

Also it has been impossible to upgrade my linux distribution as the number of faulty downloads is just too high, many files don't have the right md5 checksum, not even some of the file lists sometimes.

 

Did that in a terminal as a test:

~$ wget http://mirror1.ku.ac.th/ubuntu-security/pool/universe/f/fonts-horai-umefont/fonts-horai-umefont_434-...

[...]

2012-07-23 22:22:33 (907 KB/s) - `fonts-horai-umefont_434-1_all.deb' saved [45421504/45421504]

~$ wget http://mirror1.ku.ac.th/ubuntu-security/pool/universe/f/fonts-horai-umefont/fonts-horai-umefont_434-...

[...]

2012-07-23 22:26:30 (1.11 MB/s) - `fonts-horai-umefont_434-1_all.deb.1' saved [45421504/45421504]

~$ diff fonts-horai-umefont_434-1_all.deb.1 fonts-horai-umefont_434-1_all.deb

Binary files fonts-horai-umefont_434-1_all.deb.1 and fonts-horai-umefont_434-1_all.deb differ

 

i.e. downloaded the same file twice, and compared the resulting files, the two files are different...

...which of course should not happen as these two files are supposed to be exactly the same.

 

Been trying with different servers, etc. Always the same problem.

 

Tried the pinhole reset already, btw. Did not change anything.

 

Hope you will fix this soon, as currently this internet connection is a bit useless.

 

Please let me know if you need any specific details.

 

Best regards,

Blaise

Please use plain text.
Settled in nicely II
blaise
Posts: 28
Registered: ‎09-09-2011

Re: "Corrupted" downloads (since R36?)

... Interestingly, it seems the corrupted bytes are always the last byte of a word. Don't know if that will help you pinpoint the problem, though.

 

If we analyse in more details the first 16MB of the two downloaded files to see where they differ:

~$ hexdump -C fonts-horai-umefont_434-1_all.deb.1 | head -n 1000000 > /tmp/plop1
~$ hexdump -C fonts-horai-umefont_434-1_all.deb | head -n 1000000 > /tmp/plop0
~$ diff /tmp/plop0 /tmp/plop1
46399c46399
< 000b53e0  71 ef ce f4 f7 69 6c 7a  6e 36 b1 38 43 be 16 9c  |q....ilzn6.8C...|
---
> 000b53e0  71 ef ce f5 f7 69 6c 7a  6e 36 b1 38 43 be 16 9c  |q....ilzn6.8C...|
304891c304891
< 004a6fa0  3f 52 be a7 7c 4b 79 91  f2 3c e5 48 ca a3 94 fb  |?R..|Ky..<.H....|
---
> 004a6fa0  3f 52 be a7 7c 4b 79 91  f2 3c e5 49 ca a3 94 fb  |?R..|Ky..<.I....|
358079c358079
< 00576be0  8d 76 b0 12 96 d7 99 6f  cc 09 65 64 61 0f 2c f2  |.v.....o..eda.,.|
---
> 00576be0  8d 76 b0 13 96 d7 99 6f  cc 09 65 64 61 0f 2c f2  |.v.....o..eda.,.|
448740c448740
< 006d8e30  8f 42 88 b1 f1 36 98 64  cf 6f 93 10 5c 98 23 2e  |.B...6.d.o..\.#.|
---
> 006d8e30  8f 42 88 b5 f1 36 98 64  cf 6f 93 10 5c 98 23 2e  |.B...6.d.o..\.#.|
484598c484598
< 00764f70  bc f6 96 9b 9e 7e f4 3f  9f e0 3f e4 27 45 ec 3c  |.....~.?..?.'E.<|
---
> 00764f70  bc f6 96 9a 9e 7e f4 3f  9f e0 3f e4 27 45 ec 3c  |.....~.?..?.'E.<|
813789c813789
< 00c6adf0  19 05 6d 2a 7a df c7 85  70 a2 24 80 d5 b7 d5 df  |..m*z...p.$.....|
---
> 00c6adf0  19 05 6d 2f 7a df c7 85  70 a2 24 80 d5 b7 d5 df  |..m/z...p.$.....|
882250c882250
< 00d764c0  73 0b ec 27 1f 77 be 07  bb 4f 21 7b fe b5 d9 4e  |s..'.w...O!{...N|
---
> 00d764c0  73 0b ec 23 1f 77 be 07  bb 4f 21 7b fe b5 d9 4e  |s..#.w...O!{...N|
925990c925990
< 00e212a0  e8 74 f0 8a b5 6d 92 b6  6c f5 aa 11 36 a6 58 1b  |.t...m..l...6.X.|
---
> 00e212a0  e8 74 f0 8e b5 6d 92 b6  6c f5 aa 11 36 a6 58 1b  |.t...m..l...6.X.|

 

... yes, on average one error every 2MB, not great. Note that it appears to be always the last byte of the word which is affected, and only two differents bits of that last byte (I am basing my conclusions on a little bit more than just these 2 files: all the files I have tested seem to be affected in the same way).

I have to say that in my 12 years as an active internet user, I have never experienced such a high rate of errors on a network connection.

 

... Well, I guess there must be a first for everything. I certainly hope you will fix this soon.

Please use plain text.
Settled in nicely II
blaise
Posts: 28
Registered: ‎09-09-2011

Re: "Corrupted" downloads (since R36?)

... I should probably add -- in case someone asks, you never know, maybe someone at Virgin still read these forums -- that I ruled out the hypothesis of _my_ kit being faulty by using 3 different computers connected to the super-duper-wonderful; one of them connected with a good ol' ethernet cable, the two other through wifi. All computers of course had the same issue with large files being corrupted.

 

So... it appears the issue is somewhere between the marvelous, and the internet (inclusive). And as I still can't believe - even though sometimes you may wonder - that the internet has fallen to generalised corruption, I am afraid the corrupting spirit is somewhere in the Virgin part of the loop.

 

I wouldn't want to accuse the fantastic, of course. Especially since it has so recently been "upgraded" with a brand new firmware that gives it a shiny new web interface.

 

Even though I have to admit that the new numbers you find in the (shiny!) network status page are rather suspicious...

 

Downstream Channels Lock Status Channel ID Frequency Modulation Rx Power SNR Pre RS Errors Post RS Errors

Locked 58 307000000 Hz QAM256 8.1 dBmV 42.5 dB 233 0

Locked 57 299000000 Hz QAM256 8.1 dBmV 42.6 dB 162 0

Locked 59 315000000 Hz QAM256 7.9 dBmV 40.8 dB 169 0

Locked 60 323000000 Hz QAM256 7.8 dBmV 41.9 dB 208 0

 

I wonder what this long column of "0" could mean... could it be just a "display bug" as a guru of Virgin seem to be implying, or just an actual bug that seem to indicate that the precious has just lost its ability to detect errors or do error correction?

 

Nooooo... Such an hypothesis is simply outrageous! It is simply unthinkable that the talented engineers working at Virgin could have let an update through, affecting millions of customers, without noticing such a catastrophic flaw.

 

Please let me know what is the actual problem then, dear Virgin engineers -- or even better: fix it! -- because I am not really in the mood to patch my browser to do the error correction your device is supposed to do -- and had been doing just fine until a few days ago, just like every other internet connection device I have been using in the last 12 years.

 

Currently the "internet connection" you provide me with does not allow me to accomplish basic browsing tasks. I should thus probably point out that as you are not fulfilling your part of the contract, I don't have to fulfill mine.

 

Thanks in advance,

Blaise

Please use plain text.
A regular II
John_Gray
Posts: 125
Registered: ‎02-06-2010

Re: "Corrupted" downloads (since R36?)

> the tools to deal with that problem actually exist since the 19th century

 

Really?  Used by Babbage, I presume!

 

I wonder whether the percentage of people on this BBS who know what a C compilerdois, let alone how to use it, would be above 2%?

Please use plain text.
Settled in nicely II
blaise
Posts: 28
Registered: ‎09-09-2011

Re: "Corrupted" downloads (since R36?)

As strange as it may appear nowadays, there used to exist (and even still exist nowadays!) a digital transmission channel that largely predates internet. It was called the telegraph. Oh, glorious times!

 

In any case it was confronted with the same basic problems that can happen on a network link, i.e. loss of data, because it was a network link. So some clever mathematicians devised some clever - but highly impractical for the time since it involved doing a lot of computation - algorithms to transmit redundant information, allowing the operators to correct the data. I don't think it was much used though, most of the work was theoretical and did not necessarily involve practical implementation. You can read relevant works from Fourrier, Euler and others, or even Gallois (a stupid 20 year old kid that decided to spend the night before the duel that would take its live writing a mathematical thesis where all the foundations for - among other things - modern error correction algorithm were already present, almost 200 years ago).

 

You can also read relevant work from Turing and Shannon, that formalised the domain and presented practical algorithms for the then appearing "computing machines", before the 1950s.

 

Just to say that redundancy, error correction codes, and all that, are nothing new.

 

By the way, did you know that IP and TCP - the protocols used to incorporate your internet data - did not incorporate error correction mecanisms, because it was already considered a trivial problem at the time, that could easily be handled by the measly "transport" layer of the network protocol?

 

Well, Virgin is in charge of that "transport layer" between "the internet" and your home, I have no idea what protocol they use for that and frankly, it is none of my concern.

What is my concern is that they do corrupt the data I am downloading from the internet, data that was obviously corrupted by their transport layer, or maybe their "super winderful hub"; so obviously they do not fulfill their basic obligations as an ISP.

 

They have to solve that problem soon, because otherwise I am afraid they risk losing their licence if the relevant authorities become aware of the current situation. They wouldn't want to suddenly lose that wonderful source of money, and have to pay millions in compensation?

 

 

Regarding that C program... I was not pretending to provide an efficient, useful or even usable solution to a problem that Virgin still refuse to acknowledge. This is a "Virgin" support forum, they are supposed to be the ones providing the support, remember? I hope that this "rosetta stone" algorithm will nonetheless please them, and not exceed their level of mathematical knowledge. If they wish (and have the time), they can incorporate it in a nice shiny tool that they can then provide to the "very few" users affected by this issue.

 

You are probably right about your 2% though :-)

 

And btw, I used this program successfully to upgrade my linux distribution yesterday, so not a total waste of time!

 

Please use plain text.
Settled in nicely II
EdwardR
Posts: 40
Registered: ‎19-07-2012

Re: "Corrupted" downloads (since R36?)

You are forgetting one thing. You are asking people to download a C compiler. Chances are it will be distributed in a compressed format, which is susceptible to corruption...you get what I mean. They aren't able to download and create an application for error correction, because of errors..LOL.
Please use plain text.
Forum Manager
Mark_Wilkin
Posts: 3,136
Registered: ‎04-08-2009

Re: "Corrupted" downloads (since R36?)

[ Edited ]

Thanks for the details on that I'll pass that along to the firmware team and see if they can replicate that problem. Can you try downloading the same files with IP Flood Detection turned off and then the Firewall turned off completely? That's in the advanced settings>security>firewall section. Can you also try putting your Super Hub into modem mode and check if you see this issue there as well.

 

Thanks 


Mark Wilkin
Help & Support Forum Manager

Seen a useful post?Problem Solved?Fault with your service?
Please use plain text.
Settled in nicely II
bertrumb
Posts: 31
Registered: ‎29-05-2011

Re: "Corrupted" downloads (since R36?)

[ Edited ]

I have noticed corrupt downloads on both my Mac and PC since the R36 upgrade although it seems to be intermittant when I use Firefox on either machine.

 

Just turned IP Flood Detection off and it made no difference. Also turing the Firewall off didn't help.

 

Can we get a rollback to R30 ?

Please use plain text.
Settled in nicely II
blaise
Posts: 28
Registered: ‎09-09-2011

Re: "Corrupted" downloads (since R36?)

I certainly would appreciate the rollback myself, this situation with the corrupt downloads is driving mental.

But in any case, you cannot do the rollback yourself, someone from Virgin has to do it for you; and in my opinion they are very unlikely to do so:

 - As far as I am aware they have never rolled back a firmware

 - Not sure they would be able to do so even if they wanted to

 - They probably have not tested this procedure, so it is likely to fail and brick your superhub, making you lose for a few days the limited internet you have left

 - That would probably be their very last resort...

 - ... and currently the problem has not even been officially acknowledged.

So in any case don't expect them to do so. They would probably send you two new super hubs and few technicians visit before they consider rolling back the firmware.

 

For me, disabling the "IP Flood Detection" fixed the issue with internet disconnecting, but it did not fix the corruption - hence this thread. I tried to disable the firewall as well, and it seemed to have lowered the rate of corruption (and don't ask me why, it seemed to have fixed an issue with chrome, file downloads were getting interrupted randomly before, and not after)... but it certainly did not fix the issue.

Please use plain text.
Settled in nicely II
bertrumb
Posts: 31
Registered: ‎29-05-2011

Re: "Corrupted" downloads (since R36?)

This is now evident when updating apps on my phone, I now get a cycle of downloading then back to waiting before downloading again. Will not update at all now.

Going to try a factory reset tonight to see if that helps.

These corruptions are via wired and wireless.
Please use plain text.