Stalled downloads

Message boards : Number crunching : Stalled downloads

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7

AuthorMessage
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 92674 - Posted: 30 Mar 2020, 23:30:47 UTC - in response to Message 92659.  

"The techs" would be the ones responsible for the entire UW campus. I'm sure they see so many such exceptions that they just pat themselves on the back and say "I'm sure glad I'm protecting the network with that great tool I installed".
Rosetta Moderator: Mod.Sense
ID: 92674 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mad_Max

Send message
Joined: 31 Dec 09
Posts: 206
Credit: 19,197,372
RAC: 5,172
Message 93148 - Posted: 3 Apr 2020, 2:13:04 UTC - in response to Message 92542.  
Last modified: 3 Apr 2020, 2:19:38 UTC

So maybe allow repeated attempts to download for a maximum 24hrs, else abort? But could be a loss of connectivity at the user end, or at the project end?


Yes, some means for BOINC to give up is required. Perhaps it should wait even longer if the file is "particularly large" (intentionally left for others to define). The other issue is that in the case here at R@h the problems seemed to be with small files, but the small file was required to run a given WU. That WU might have had many GBs of other required files that came down ok. So the actual size of the particular file having problems is not really the only consideration. If you abort the WU, all of the WU files are being lost (unless other WUs your machine has on-board use them as well).

I should think the BOINC Manager can see the difference between a lack of connectivity being the cause, and a dropped frame. So that info. could be used. If there was a loss of connectivity, then perhaps you double the maximum timeout.

...all good discussion for BOINC message boards.


Connectivity detection is also partially broken in BOINC clients:
If BOINC client can not download file or contact project server it contacts "reference site" which is google.com for last few years.
If it get "HTTP 200 - OK" response from google - all work as expected: BOINC understands there are problem with particular project but Internet connection is OK and proceeds with other projects.

But if google return any other response(not "200 OK") - BOINC stupidly interprets it as connectivity issues (no internet connection) and temporarily ceases all internet activity and increase "back-off" timer each time.
And Google often returns other codes - because a lot of such similar automated requests from thousands of computers around the world running BOINC often trigger the anti-bot filters of Google (it blocks such requests and offers to solve CAPTCHA to continue work). BOINC of course can not understand it and react to this by declaring "connectivity issues".

Although the fact that it was able to get an error message from Google (no matter which and for what reason) means the exactly opposite - that the Internet connection is working OK.
ID: 93148 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Klimax

Send message
Joined: 27 Apr 07
Posts: 35
Credit: 1,712,432
RAC: 0
Message 93194 - Posted: 3 Apr 2020, 9:25:20 UTC - in response to Message 93148.  

So maybe allow repeated attempts to download for a maximum 24hrs, else abort? But could be a loss of connectivity at the user end, or at the project end?


Yes, some means for BOINC to give up is required. Perhaps it should wait even longer if the file is "particularly large" (intentionally left for others to define). The other issue is that in the case here at R@h the problems seemed to be with small files, but the small file was required to run a given WU. That WU might have had many GBs of other required files that came down ok. So the actual size of the particular file having problems is not really the only consideration. If you abort the WU, all of the WU files are being lost (unless other WUs your machine has on-board use them as well).

I should think the BOINC Manager can see the difference between a lack of connectivity being the cause, and a dropped frame. So that info. could be used. If there was a loss of connectivity, then perhaps you double the maximum timeout.

...all good discussion for BOINC message boards.


Connectivity detection is also partially broken in BOINC clients:
If BOINC client can not download file or contact project server it contacts "reference site" which is google.com for last few years.
If it get "HTTP 200 - OK" response from google - all work as expected: BOINC understands there are problem with particular project but Internet connection is OK and proceeds with other projects.

But if google return any other response(not "200 OK") - BOINC stupidly interprets it as connectivity issues (no internet connection) and temporarily ceases all internet activity and increase "back-off" timer each time.
And Google often returns other codes - because a lot of such similar automated requests from thousands of computers around the world running BOINC often trigger the anti-bot filters of Google (it blocks such requests and offers to solve CAPTCHA to continue work). BOINC of course can not understand it and react to this by declaring "connectivity issues".

Although the fact that it was able to get an error message from Google (no matter which and for what reason) means the exactly opposite - that the Internet connection is working OK.

I think it is work around for various captive portals and security appliances/gateways that might block connection with custom error page.
ID: 93194 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 4 · 5 · 6 · 7

Message boards : Number crunching : Stalled downloads



©2022 University of Washington
https://www.bakerlab.org