Problen downloading work tasks.

Message boards : Number crunching : Problen downloading work tasks.

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 6 · Next

AuthorMessage
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1695
Credit: 18,134,466
RAC: 24,304
Message 107819 - Posted: 17 Dec 2022, 1:07:13 UTC - in response to Message 107817.  

Edit: Reading this thread again from the top, I'm suspicious of whether you're running Norton and have Secure VPN turned on. I have it on 3 or 4 of my PCs and it's fine on all but one, which is constantly restarting Secure VPN and it's knocking out our email and access to web pages in a really weird way. Turning it off and turning off Auto VPN is an immediate solution. No idea why it's happening on that one but not all the others.
Given the number of posts, and the fact that while there is work being dished out it is well below the usual levels this is more than just an issue with some systems & as the Server Staus page does show the download server as not running, the project at the very least should restart the download server and see if that resolves the problem (with both the downloads and the reporting of the server status).
Grant
Darwin NT
ID: 107819 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
hadron

Send message
Joined: 4 Sep 22
Posts: 68
Credit: 1,561,654
RAC: 273
Message 107820 - Posted: 17 Dec 2022, 1:33:03 UTC - in response to Message 107811.  

I would say it's a pure Linux problem. I have no problems on my Windows computers. Only on my Linux computer does it not want to download the rest.

Greetings from Daniel

I don't believe you, not for one nanosecond. When the server status page says the download server isn't running, it isn't running, and no one is getting any tasks sent down to them.
ID: 107820 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1695
Credit: 18,134,466
RAC: 24,304
Message 107821 - Posted: 17 Dec 2022, 3:12:44 UTC - in response to Message 107820.  

When the server status page says the download server isn't running, it isn't running, and no one is getting any tasks sent down to them.
Even though all the other stats show that they are.
Grant
Darwin NT
ID: 107821 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
hadron

Send message
Joined: 4 Sep 22
Posts: 68
Credit: 1,561,654
RAC: 273
Message 107822 - Posted: 17 Dec 2022, 3:53:52 UTC - in response to Message 107821.  

When the server status page says the download server isn't running, it isn't running, and no one is getting any tasks sent down to them.
Even though all the other stats show that they are.

Perhaps you can tell me how to get around all these 403 errors then, because that is all I've been seeing since this mess began.
ID: 107822 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
pututu

Send message
Joined: 12 Jun 16
Posts: 5
Credit: 10,028,325
RAC: 0
Message 107823 - Posted: 17 Dec 2022, 4:32:17 UTC - in response to Message 107822.  

When the server status page says the download server isn't running, it isn't running, and no one is getting any tasks sent down to them.
Even though all the other stats show that they are.

Perhaps you can tell me how to get around all these 403 errors then, because that is all I've been seeing since this mess began.


I think you should try to run this under windows instead of linux.

Somehow it works in my windows but not linux.
ID: 107823 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 193
Credit: 6,516,301
RAC: 7,271
Message 107824 - Posted: 17 Dec 2022, 6:31:17 UTC - in response to Message 107823.  

I think you should try to run this under windows instead of linux.

Somehow it works in my windows but not linux.


My little computer runs Windows 10 and it did download a bunch of Rosetta tasks this afternoon. But this big fast computer with 64 Gigabytes of RAM which ran Rosetta perfectly until very recently, suddenly stopped receiving Rosetta tasks perhaps a week ago when there were no tasks available. And a day or two ago, I got these tasks to tease me, but they resist downloading. They are attached to the same router to my 75 Megabit/second Internet connection.

Fri 16 Dec 2022 11:31:45 PM EST | Rosetta@home | Started download of rb_12_15_464343_459634_ab_t000__robetta.200.9mers.index.gz
Fri 16 Dec 2022 11:31:48 PM EST |  | Project communication failed: attempting access to reference site
Fri 16 Dec 2022 11:31:48 PM EST | Rosetta@home | Temporarily failed download of rb_12_15_464343_459634_ab_t000__robetta.200.9mers.index.gz: transient HTTP error
Fri 16 Dec 2022 11:31:48 PM EST | Rosetta@home | Backing off 05:20:10 on download of rb_12_15_464343_459634_ab_t000__robetta.200.9mers.index.gz
Fri 16 Dec 2022 11:31:49 PM EST |  | Internet access OK - project servers may be temporarily down.


What is there about Unix/Linux that keeps getting HTTP errors on downloading tasks, when ClimatePrediction, Einstein, MilkyWay, Universe, and WorldCommunityGrid do not have trouble? And Rosetta used to be able to download tasks too. I can obviously send and receive from the Rosetta web site. I can send and receive from the download server as well. It is kinda slow, but it works. Other sites are faster.

$ ping -c 5 boinc-files.bakerlab.org   <---<<<
PING boinc-files.bakerlab.org (128.95.160.135) 56(84) bytes of data.
64 bytes from fw2.ipd.uw.edu (128.95.160.135): icmp_seq=1 ttl=50 time=74.1 ms
64 bytes from fw2.ipd.uw.edu (128.95.160.135): icmp_seq=2 ttl=50 time=73.10 ms
64 bytes from fw2.ipd.uw.edu (128.95.160.135): icmp_seq=3 ttl=50 time=73.9 ms
64 bytes from fw2.ipd.uw.edu (128.95.160.135): icmp_seq=4 ttl=50 time=75.5 ms
64 bytes from fw2.ipd.uw.edu (128.95.160.135): icmp_seq=5 ttl=50 time=74.5 ms

--- boinc-files.bakerlab.org ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4002ms
rtt min/avg/max/mdev = 73.906/74.374/75.458/0.647 ms

$ ping -c 5 www.facebook.com  <---<<<
PING star-mini.c10r.facebook.com (31.13.71.36) 56(84) bytes of data.
64 bytes from edge-star-mini-shv-01-lga3.facebook.com (31.13.71.36): icmp_seq=1 ttl=58 time=6.56 ms
64 bytes from edge-star-mini-shv-01-lga3.facebook.com (31.13.71.36): icmp_seq=2 ttl=58 time=7.27 ms
64 bytes from edge-star-mini-shv-01-lga3.facebook.com (31.13.71.36): icmp_seq=3 ttl=58 time=6.32 ms
64 bytes from edge-star-mini-shv-01-lga3.facebook.com (31.13.71.36): icmp_seq=4 ttl=58 time=7.73 ms
64 bytes from edge-star-mini-shv-01-lga3.facebook.com (31.13.71.36): icmp_seq=5 ttl=58 time=6.72 ms

--- star-mini.c10r.facebook.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 6.317/6.918/7.729/0.514 ms


ID: 107824 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 107827 - Posted: 17 Dec 2022, 11:45:22 UTC
Last modified: 17 Dec 2022, 12:15:12 UTC

Well I have both Microsoft and Linux here ,
Both my Linux and MS systems have downloads in `project backoff` and `transient http error` etc
And on my tasks page they show as `in progress` Have we got dumped with a heap of Ghost tasks that may never download
time for an experiment and abort the lot from one system and see if I can get some new ones
don't have any vpn or NortOnYourLife softly ware and I have bashed the retry buttons often .
I do see that the tasks in progress has gone up on the server status page,
so some systems out there are getting work , just not me/us whatever I/we do , including reboot system , so the problem is sporadic
So , some of us get work some don't , now all we need is someone back at the Lab to work out why and do something about it
Hmmm , this being the modern Rosetta it may get fixed by next year (its not that far away) . . . .

Edit , after aborting all tasks from the download que from one system and press update I still get

17/12/2022 12:01:38 | Rosetta@home | update requested by user
17/12/2022 12:01:44 | Rosetta@home | Sending scheduler request: Requested by user.
17/12/2022 12:01:44 | Rosetta@home | Not requesting tasks: some download is stalled
17/12/2022 12:01:46 | Rosetta@home | Scheduler request completed
17/12/2022 12:01:46 | Rosetta@home | Project requested delay of 31 seconds

even though I have no tasks waiting / stuck to download on that system
The aborted downloads now show as `error while downloading`
It was worth a try
I hate zombie tasks nnnn
ID: 107827 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 395
Credit: 12,213,138
RAC: 9,763
Message 107828 - Posted: 17 Dec 2022, 11:56:06 UTC - in response to Message 107813.  

Regardless of the Server Staus page, work is being sent out.
The Top Computers list shows systems with recently allocated work, and the graphs also show work going out over the last few days. However it does appear to be at a lower rate than usual, so a reset of the download server would probably not go astray.



Yes, the Scheduler is allocating work and therefore the system sees it as “In Progress” but the Download Server is down and therefore the users are not getting the work whether they be Windows of Linux.

The Ready to Send queue is at 29,000 and should stay that way until the Queued figure on the front page drops to zero, the fact that it is steady at 6,340,000 or thereabouts shows that no work is making it to the volunteers.
ID: 107828 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 193
Credit: 6,516,301
RAC: 7,271
Message 107829 - Posted: 17 Dec 2022, 13:15:31 UTC - in response to Message 107828.  

Yes, the Scheduler is allocating work and therefore the system sees it as “In Progress” but the Download Server is down and therefore the users are not getting the work whether they be Windows of Linux.


That makes good sense, but i checked my Windows box and it downloaded a coupla dozen Rosetta tasks between 2:57 and 7:43PM yesterday; Three of them are currently running and the rest are in queue on my Windows machine. There may have been some that came in earlier. One has already completed successfully.
Task 1510540883
Name 	KC_13mer_13_hallucinated_51_25_SAVE_ALL_OUT_2937287_27_0
Workunit 	1344007211
Created 	16 Dec 2022, 16:19:25 UTC
Sent 	16 Dec 2022, 19:57:07 UTC
Report deadline 	19 Dec 2022, 19:57:07 UTC
Received 	17 Dec 2022, 12:54:48 UTC
Server state 	Over
Outcome 	Success
Client state 	Done
Exit status 	0 (0x00000000)
Computer ID 	5958977
Run time 	7 hours 58 min 39 sec
CPU time 	7 hours 56 min 27 sec
Validate state 	Valid


On this, my Linux box, those 22 files are still there, waiting for the backoff to expire, which will be in a few minutes.

Nope: while typing this, the backoff has increased. I manually poked every one of them and now they all must wait about five hours before it will try again. Grrrr.
ID: 107829 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 395
Credit: 12,213,138
RAC: 9,763
Message 107830 - Posted: 17 Dec 2022, 13:20:10 UTC - in response to Message 107829.  

Yes, the Scheduler is allocating work and therefore the system sees it as “In Progress” but the Download Server is down and therefore the users are not getting the work whether they be Windows of Linux.


That makes good sense, but i checked my Windows box and it downloaded a coupla dozen Rosetta tasks between 2:57 and 7:43PM yesterday; Three of them are currently running and the rest are in queue on my Windows machine. There may have been some that came in earlier. One has already completed successfully.
Task 1510540883
Name 	KC_13mer_13_hallucinated_51_25_SAVE_ALL_OUT_2937287_27_0
Workunit 	1344007211
Created 	16 Dec 2022, 16:19:25 UTC
Sent 	16 Dec 2022, 19:57:07 UTC
Report deadline 	19 Dec 2022, 19:57:07 UTC
Received 	17 Dec 2022, 12:54:48 UTC
Server state 	Over
Outcome 	Success
Client state 	Done
Exit status 	0 (0x00000000)
Computer ID 	5958977
Run time 	7 hours 58 min 39 sec
CPU time 	7 hours 56 min 27 sec
Validate state 	Valid


On this, my Linux box, those 22 files are still there, waiting for the backoff to expire, which will be in a few minutes.

Nope: while typing this, the backoff has increased. I manually poked every one of them and now they all must wait about five hours before it will try again. Grrrr.


Then I apologise for my misinformation, in my defence I don't have a Windows box so I was working from first principles :-)
ID: 107830 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
UBT - wbiz

Send message
Joined: 5 Feb 21
Posts: 6
Credit: 968,608
RAC: 80
Message 107831 - Posted: 17 Dec 2022, 13:25:19 UTC

Cleaner has pulled the download server's plug out to use the vacuum cleaner.
ID: 107831 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
hadron

Send message
Joined: 4 Sep 22
Posts: 68
Credit: 1,561,654
RAC: 273
Message 107833 - Posted: 17 Dec 2022, 16:54:55 UTC - in response to Message 107827.  
Last modified: 17 Dec 2022, 16:57:30 UTC


Edit , after aborting all tasks from the download que from one system and press update I still get

17/12/2022 12:01:38 | Rosetta@home | update requested by user
17/12/2022 12:01:44 | Rosetta@home | Sending scheduler request: Requested by user.
17/12/2022 12:01:44 | Rosetta@home | Not requesting tasks: some download is stalled
17/12/2022 12:01:46 | Rosetta@home | Scheduler request completed
17/12/2022 12:01:46 | Rosetta@home | Project requested delay of 31 seconds

even though I have no tasks waiting / stuck to download on that system
The aborted downloads now show as `error while downloading`
It was worth a try
I hate zombie tasks nnnn

Restart your boinc client to clear any "stalled downloads".

It won't help in getting the downloads to complete, of course, but at least it gets rid of one small problem.
ID: 107833 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 193
Credit: 6,516,301
RAC: 7,271
Message 107834 - Posted: 17 Dec 2022, 17:27:28 UTC - in response to Message 107833.  

Restart your boinc client to clear any "stalled downloads".

It won't help in getting the downloads to complete, of course, but at least it gets rid of one small problem.


OK: I aborted the downloads that would not download for a couple of days.
I restarted the boinc-client.

Now I have 14 new tasks that will not download. Same symptoms as before. I still have connectivity to the download server. What small problem did it remove?

Sat 17 Dec 2022 12:14:08 PM EST | Rosetta@home | Started download of KC_12mer_12_hallucinated_84_99.zip
Sat 17 Dec 2022 12:14:08 PM EST | Rosetta@home | Started download of KC_13mer_13_hallucinated_112_231.zip
Sat 17 Dec 2022 12:14:09 PM EST |  | Internet access OK - project servers may be temporarily down.
Sat 17 Dec 2022 12:14:09 PM EST | Rosetta@home | Temporarily failed download of KC_12mer_12_hallucinated_84_99.zip: transient HTTP error
Sat 17 Dec 2022 12:14:09 PM EST | Rosetta@home | Backing off 00:07:23 on download of KC_12mer_12_hallucinated_84_99.zip
Sat 17 Dec 2022 12:14:09 PM EST | Rosetta@home | Temporarily failed download of KC_13mer_13_hallucinated_112_231.zip: transient HTTP error
Sat 17 Dec 2022 12:14:09 PM EST | Rosetta@home | Backing off 00:06:38 on download of KC_13mer_13_hallucinated_112_231.zip

$ ping -c 5 boinc-files.bakerlab.org
PING boinc-files.bakerlab.org (128.95.160.134) 56(84) bytes of data.
64 bytes from fw1.ipd.uw.edu (128.95.160.134): icmp_seq=1 ttl=50 time=71.8 ms
64 bytes from fw1.ipd.uw.edu (128.95.160.134): icmp_seq=2 ttl=50 time=71.1 ms
64 bytes from fw1.ipd.uw.edu (128.95.160.134): icmp_seq=3 ttl=50 time=72.6 ms
64 bytes from fw1.ipd.uw.edu (128.95.160.134): icmp_seq=4 ttl=50 time=71.6 ms
64 bytes from fw1.ipd.uw.edu (128.95.160.134): icmp_seq=5 ttl=50 time=70.6 ms

--- boinc-files.bakerlab.org ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 70.604/71.538/72.599/0.695 ms


ID: 107834 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
hadron

Send message
Joined: 4 Sep 22
Posts: 68
Credit: 1,561,654
RAC: 273
Message 107835 - Posted: 17 Dec 2022, 20:00:27 UTC - in response to Message 107834.  

Restart your boinc client to clear any "stalled downloads".

It won't help in getting the downloads to complete, of course, but at least it gets rid of one small problem.


OK: I aborted the downloads that would not download for a couple of days.
I restarted the boinc-client.

Now I have 14 new tasks that will not download. Same symptoms as before. I still have connectivity to the download server. What small problem did it remove?

It got rid of the problem that, after you aborted the other tasks, you weren't getting any new ones because of some "stalled" download.
I did say it wouldn't solve the problem of not being able to download the new tasks, didn't I?
ID: 107835 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 193
Credit: 6,516,301
RAC: 7,271
Message 107836 - Posted: 17 Dec 2022, 20:30:57 UTC - in response to Message 107835.  

It got rid of the problem that, after you aborted the other tasks, you weren't getting any new ones because of some "stalled" download.
I did say it wouldn't solve the problem of not being able to download the new tasks, didn't I?


So all that exercise did was replace an old set of undownloadable tasks with a new set of undownloadable tasks.
ID: 107836 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
hadron

Send message
Joined: 4 Sep 22
Posts: 68
Credit: 1,561,654
RAC: 273
Message 107838 - Posted: 17 Dec 2022, 23:18:50 UTC - in response to Message 107836.  

It got rid of the problem that, after you aborted the other tasks, you weren't getting any new ones because of some "stalled" download.
I did say it wouldn't solve the problem of not being able to download the new tasks, didn't I?


So all that exercise did was replace an old set of undownloadable tasks with a new set of undownloadable tasks.

Mother of God, Jean-David, you said you aborted the old tasks, and then couldn't get any new ones because of stalled downloads.

I merely told you how to get rid of those "stalled downloads." What more do you want?
ID: 107838 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 107839 - Posted: 18 Dec 2022, 0:40:49 UTC - in response to Message 107838.  

It got rid of the problem that, after you aborted the other tasks, you weren't getting any new ones because of some "stalled" download.
I did say it wouldn't solve the problem of not being able to download the new tasks, didn't I?


So all that exercise did was replace an old set of undownloadable tasks with a new set of undownloadable tasks.

Mother of God, Jean-David, you said you aborted the old tasks, and then couldn't get any new ones because of stalled downloads.

I merely told you how to get rid of those "stalled downloads." What more do you want?

Errm , That would be tasks that download and crunch without any bother ;-)
ID: 107839 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 193
Credit: 6,516,301
RAC: 7,271
Message 107840 - Posted: 18 Dec 2022, 1:08:39 UTC - in response to Message 107838.  

Mother of God, Jean-David, you said you aborted the old tasks, and then couldn't get any new ones because of stalled downloads.

I merely told you how to get rid of those "stalled downloads." What more do you want?


I know I could not get any new ones, but only because the new ones stalled just as did the old ones. I do not think I ever said why, because I have no idea why.

How many days does it take for them to fix the download server so it will serve tasks to Linux machines? Somehow it serves Windows machines. And all five of my other projects serve Linux machines too. And Rosetta used to work too.
ID: 107840 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
vdquang

Send message
Joined: 29 Sep 06
Posts: 9
Credit: 1,572,211
RAC: 636
Message 107841 - Posted: 18 Dec 2022, 1:49:18 UTC - in response to Message 107840.  

I use an Windows 7 (64-bit) machine and BOINC with the virtual box. And I cannot get rid of those "stalled downloads". What to do now?
ID: 107841 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 193
Credit: 6,516,301
RAC: 7,271
Message 107842 - Posted: 18 Dec 2022, 2:05:34 UTC - in response to Message 107841.  

I use an Windows 7 (64-bit) machine and BOINC with the virtual box. And I cannot get rid of those "stalled downloads". What to do now?


I have a machine running Windows 10 (64-bit) without the virtual box, and it has no trouble getting downloads.
What version of the boinc client is your machine using?

My Linux machine is using 7.20.2 and it does not work, though it does for all 5 of my other projects.
Trouble is, my Windows machine is running the same version. But if yuor Windows 7 machine is using an older client, this may be the problem.

I am, by no means, a Windows expert.
ID: 107842 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 . . . 6 · Next

Message boards : Number crunching : Problen downloading work tasks.



©2024 University of Washington
https://www.bakerlab.org