Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 15 · 16 · 17 · 18 · 19 · 20 · 21 . . . 28 · Next

AuthorMessage
Profile shanen
Avatar

Send message
Joined: 16 Apr 14
Posts: 187
Credit: 11,472,979
RAC: 6,469
Message 87371 - Posted: 25 Sep 2017, 19:01:38 UTC

Eh? Seems to be some problem with the Message boards, too. The listing showed this thread had fresh content, but I couldn't see it no matter how I searched. All I could see were various old posts. Then I decided to add this comment, and the new messages suddenly became visible?

Unfortunately those new messages didn't help me understand what is going on. It would help if it were more clear which of the posters might be supposed to know what is going on and which ones are just describing symptoms.

Based on what I can see, I'll just say "It's back." Work units are stuck in the "Uploading" state.

Of course the first thing I did when I noticed the return of the symptom was to visit the server status page. Not sure how to interpret the right side, but the left side seems to be saying that all of the servers and daemons are nominal.
#1 Freedom = (Meaningful - Constrained) Choice{5} != (Beer^3 | Speech)
ID: 87371 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1016
Credit: 3,974,123
RAC: 310
Message 87372 - Posted: 25 Sep 2017, 19:09:40 UTC

Some daemons crashed unexpectedly sometime this weekend. They are back up and running now. May have been a network glitch. Seems ok so far.
ID: 87372 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile shanen
Avatar

Send message
Joined: 16 Apr 14
Posts: 187
Credit: 11,472,979
RAC: 6,469
Message 87373 - Posted: 25 Sep 2017, 19:11:31 UTC - in response to Message 87371.  

So then I thought I'd look at the older discussion. Wanted to compare symptoms, see how long it was, and if anyone had added any explanatory material since I last looked at it. Can't even find that discussion.

I did notice one thing that might be new this time, but maybe I'm wasting keystrokes to note that the "Avg. work done" as displayed in the manager slumped and froze a day or two before I noticed the stuck in "Uploading" units. Now it's jumped back up to a more normal value. Might indicate this is a new problem with a similar symptom.
#1 Freedom = (Meaningful - Constrained) Choice{5} != (Beer^3 | Speech)
ID: 87373 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile shanen
Avatar

Send message
Joined: 16 Apr 14
Posts: 187
Credit: 11,472,979
RAC: 6,469
Message 87374 - Posted: 25 Sep 2017, 19:14:54 UTC - in response to Message 87372.  

Hmm... Posted between my comments, but I just checked and I can confirm that the tasks are still stuck in the Uploading status and Retry Now fails to upload the results. The delay time until it gives up is shorter than my memory of last time.
#1 Freedom = (Meaningful - Constrained) Choice{5} != (Beer^3 | Speech)
ID: 87374 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1016
Credit: 3,974,123
RAC: 310
Message 87375 - Posted: 25 Sep 2017, 20:07:42 UTC

Oh, I'm sorry, this is likely another issue. I'll take a look.
ID: 87375 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1016
Credit: 3,974,123
RAC: 310
Message 87376 - Posted: 25 Sep 2017, 20:11:08 UTC

Hmmm, looks like something weird was/is going on with our filesystem. I had to create/touch the file upload handler log file for one of our web servers. I'll talk to our sys admin about this to see what might have caused this.
ID: 87376 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1016
Credit: 3,974,123
RAC: 310
Message 87377 - Posted: 25 Sep 2017, 20:30:14 UTC

Are you still seeing upload issues?
ID: 87377 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 980
Credit: 22,059,531
RAC: 14,140
Message 87378 - Posted: 25 Sep 2017, 21:57:03 UTC - in response to Message 87376.  

Hmmm, looks like something weird was/is going on with our filesystem. I had to create/touch the file upload handler log file for one of our web servers. I'll talk to our sys admin about this to see what might have caused this.

As shown in my error log above. Some flag got set on the file during the crash? Often happens.

Just done a retry and everything uploaded immediately. Quick check and they've all validated too.

I can also see Android tasks showing on the server status page and a quick update has brought some tasks down.

All looking good at my end now. Many thanks.
ID: 87378 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1016
Credit: 3,974,123
RAC: 310
Message 87379 - Posted: 25 Sep 2017, 23:38:22 UTC

Good to hear.
ID: 87379 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 980
Credit: 22,059,531
RAC: 14,140
Message 87380 - Posted: 26 Sep 2017, 0:19:58 UTC - in response to Message 87371.  

Eh? Seems to be some problem with the Message boards, too. The listing showed this thread had fresh content, but I couldn't see it no matter how I searched. All I could see were various old posts. Then I decided to add this comment, and the new messages suddenly became visible?

The new version of the message board uses pages, this being page 4 of the thread. Confused me at the start too. Is that it?
ID: 87380 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Erich56

Send message
Joined: 11 Jan 16
Posts: 8
Credit: 554,938
RAC: 43
Message 87406 - Posted: 29 Sep 2017, 4:51:48 UTC

Since yesterday ( Sept. 28, 2017), no new Rosetta tasks can be downloaded. What's the reason for this?
ID: 87406 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
JohnH

Send message
Joined: 25 Mar 13
Posts: 40
Credit: 1,764,496
RAC: 892
Message 87407 - Posted: 29 Sep 2017, 5:55:41 UTC - in response to Message 87406.  

Me too ... according to server status Tasks ready to send 9989 but I don't think that number is changing
ID: 87407 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
JohnH

Send message
Joined: 25 Mar 13
Posts: 40
Credit: 1,764,496
RAC: 892
Message 87408 - Posted: 29 Sep 2017, 7:14:37 UTC - in response to Message 87407.  

Tasks ready to send 9983 now so it has changed but not by much ... I'm still not getting new units though.
9/29/2017 8:09:24 AM | Rosetta@home | Sending scheduler request: To fetch work.
9/29/2017 8:09:24 AM | Rosetta@home | Requesting new tasks for CPU
9/29/2017 8:09:25 AM | Rosetta@home | Scheduler request completed: got 0 new tasks
9/29/2017 8:09:25 AM | Rosetta@home | No tasks sent
ID: 87408 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Erich56

Send message
Joined: 11 Jan 16
Posts: 8
Credit: 554,938
RAC: 43
Message 87409 - Posted: 29 Sep 2017, 7:17:46 UTC - in response to Message 87408.  

... No tasks sent
same here, still :-(

I have now switched to another project.
ID: 87409 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 980
Credit: 22,059,531
RAC: 14,140
Message 87412 - Posted: 29 Sep 2017, 12:05:44 UTC

I thought there'd be more comments on the lack of any Mini-Rosetta and Rosetta tasks for a while. The only availability is for Android tasks
ID: 87412 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
JohnH

Send message
Joined: 25 Mar 13
Posts: 40
Credit: 1,764,496
RAC: 892
Message 87415 - Posted: 29 Sep 2017, 15:05:57 UTC - in response to Message 87412.  

Sid - where do you see that?
ID: 87415 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
JohnH

Send message
Joined: 25 Mar 13
Posts: 40
Credit: 1,764,496
RAC: 892
Message 87416 - Posted: 29 Sep 2017, 15:28:26 UTC - in response to Message 87415.  
Last modified: 29 Sep 2017, 15:33:55 UTC

Oh I see it's here in Tasks by Application of Server Status
Rosetta 0 9270 7.58 (0.51 - 43.16) 919
Rosetta Mini 0 233999 4.98 (0.34 - 188.04) 13152
Rosetta Mini for Android 9983 22726 1.27 (0.35 - 25.21) 1851
ID: 87416 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1016
Credit: 3,974,123
RAC: 310
Message 87418 - Posted: 29 Sep 2017, 17:42:21 UTC

There is a lull in jobs being submitted by Baker lab researchers. I sent out an alert that the queue is nearly empty. There should still be Robetta jobs trickling in from the public structure prediction server.
ID: 87418 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 980
Credit: 22,059,531
RAC: 14,140
Message 87454 - Posted: 4 Oct 2017, 19:48:42 UTC - in response to Message 87416.  

Oh I see it's here in Tasks by Application of Server Status
Rosetta 0 9270 7.58 (0.51 - 43.16) 919
Rosetta Mini 0 233999 4.98 (0.34 - 188.04) 13152
Rosetta Mini for Android 9983 22726 1.27 (0.35 - 25.21) 1851

You got it - a nice piece of useful extra info on that server status page.

Sorry for the delay replying. I was in Madrid last week and returned for some brief (scheduled and successful) hospital time this week Everything looking good, with a new Mini Rosetta version now out
ID: 87454 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 980
Credit: 22,059,531
RAC: 14,140
Message 87468 - Posted: 8 Oct 2017, 21:52:44 UTC

Mini Rosetta tasks have just run out again
Remote daemon status as of 8 Oct 2017, 21:05:06 UTC
Tasks by application
Application Unsent In progress Runtime of last 100 tasks in hours: average, min, max Users in last 24 hours
Rosetta 0 1516 8.22 (0.56 - 34.46) 211
Rosetta Mini 9 379506 5.92 (0.34 - 92.36) 12257
Rosetta Mini for Android 9993 25624 1.28 (0.37 - 50.9) 1867

ID: 87468 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 15 · 16 · 17 · 18 · 19 · 20 · 21 . . . 28 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2019 University of Washington
http://www.bakerlab.org