Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 126 · 127 · 128 · 129 · 130 · 131 · 132 . . . 279 · Next

AuthorMessage
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1498
Credit: 14,743,955
RAC: 16,793
Message 103052 - Posted: 30 Oct 2021, 0:15:52 UTC

Just checked my Tasks and a few from the 29th have come through, but the number of Pendings is still almost triple the number of Valids.
Hopefully the life signs will continue to improve as the day goes on.
Grant
Darwin NT
ID: 103052 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2003
Credit: 38,610,491
RAC: 18,020
Message 103053 - Posted: 30 Oct 2021, 0:36:04 UTC - in response to Message 103052.  

Just checked my Tasks and a few from the 29th have come through, but the number of Pendings is still almost triple the number of Valids.
Hopefully the life signs will continue to improve as the day goes on.

Yeah, another look and I'm not buying my idea either tbh. Updated to 243k backlog - higher still, not lower.
A watched pot never boils - I'll look again tomorrow
ID: 103053 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1498
Credit: 14,743,955
RAC: 16,793
Message 103055 - Posted: 30 Oct 2021, 9:15:19 UTC

Luckily the Rosetta graphs also show the Validation numbers.
It looks like the Validators have been having issues for a while now. Generally they've been averaging a backlog of around 600 or so. But since Wednesday of last week, there have been periods where they've been falling behind, then catching up. The amount they fall behind each time getting larger until they came good for a couple of days from late Sunday.
Then they stared falling behind again, more and more each time until the present huge backlog.



Compare that to over the last year.


Grant
Darwin NT
ID: 103055 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1225
Credit: 13,875,753
RAC: 2,978
Message 103057 - Posted: 30 Oct 2021, 12:38:13 UTC - in response to Message 103046.  

A task running MUCH longer than the expected 8 hours:

aaab_nNMALA_pp-SAR_pp-mPPS-BGLY_pp_2_2245795_6_1

https://boinc.bakerlab.org/rosetta/result.php?resultid=1441862159

2 days, 8 hours, 32 minutes so far

rosetta python 1.03 vbox64

This is elapsed time, not the much shorter CPU time.

Now aborted after 3 days and 20 hours elapsed, less than 10 minutes CPU time.

The python tasks need a major improvement in how they detect tasks taking too long to run,

Could the current validator be written in Python, and having this same problem?
ID: 103057 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1225
Credit: 13,875,753
RAC: 2,978
Message 103064 - Posted: 30 Oct 2021, 23:00:13 UTC

Rosetta@Home has a problem with how you recover after losing your password.

The line where it asks you to enter your email address will not allow you to enter anything unless toy first click in the right half of the line and make the box appear that you need to put the email address inside.
ID: 103064 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2003
Credit: 38,610,491
RAC: 18,020
Message 103065 - Posted: 31 Oct 2021, 3:41:56 UTC - in response to Message 103053.  

Just checked my Tasks and a few from the 29th have come through, but the number of Pendings is still almost triple the number of Valids.
Hopefully the life signs will continue to improve as the day goes on.

Yeah, another look and I'm not buying my idea either tbh. Updated to 243k backlog - higher still, not lower.
A watched pot never boils - I'll look again tomorrow

Not getting any better - in fact much worse.
I've sent another nudge with a request for a timescale.

Combined with my entire email provider being down for 3 consecutive days, this is not what I want to see...
ID: 103065 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
TSD

Send message
Joined: 10 Oct 08
Posts: 7
Credit: 2,189,714
RAC: 0
Message 103067 - Posted: 31 Oct 2021, 17:14:53 UTC

As usual there is no information about what is happening. I don't know what I am doing here.

I'm running Folding@Home now.
ID: 103067 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
sgaboinc

Send message
Joined: 2 Apr 14
Posts: 282
Credit: 208,966
RAC: 0
Message 103068 - Posted: 31 Oct 2021, 17:19:13 UTC
Last modified: 31 Oct 2021, 17:19:27 UTC

validator not running?
I've a bunch of tasks that has not been validated for a few days

https://boinc.bakerlab.org/rosetta/server_status.php
Workunits waiting for validation 396840

Seem to be increasing.
even though the server status page seem to say the validator is running.
ID: 103068 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 376
Credit: 10,765,034
RAC: 6,825
Message 103069 - Posted: 31 Oct 2021, 17:24:54 UTC - in response to Message 103067.  

As usual there is no information about what is happening. I don't know what I am doing here.

I'm running Folding@Home now.


Given that it’s now Sunday evening and the people who fix it work a normal working week I am not surprised that they are not posting updates every five minutes.

Are you so desperate for credits that a problem that does not stop you from processing, just delays the credits from going to your account, sends you running to another project?

Sorry, I just don’t see the emergency.
ID: 103069 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
TSD

Send message
Joined: 10 Oct 08
Posts: 7
Credit: 2,189,714
RAC: 0
Message 103070 - Posted: 31 Oct 2021, 17:39:54 UTC - in response to Message 103069.  

Given that it’s now Sunday evening and the people who fix it work a normal working week I am not surprised that they are not posting updates every five minutes.


I would be happy if there were updates every five weeks.

Are you so desperate for credits that a problem that does not stop you from processing, just delays the credits from going to your account, sends you running to another project?


I don't care about credits. Credits means nothing. Credits is just a digit on my screen.
ID: 103070 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2003
Credit: 38,610,491
RAC: 18,020
Message 103071 - Posted: 31 Oct 2021, 19:08:02 UTC - in response to Message 103067.  

As usual there is no information about what is happening. I don't know what I am doing here.

I'm running Folding@Home now.

Presumably, if you aren't interested in credits, you're downloading, running and returning tasks without any restriction and everything's great.
Lots of articles get posted here on everything the project's doing when they issue papers about them.
I don't think they've ever been doing as much work as they currently are.
ID: 103071 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1867
Credit: 8,208,539
RAC: 7,163
Message 103072 - Posted: 31 Oct 2021, 19:08:20 UTC - in response to Message 103070.  

I am not surprised that they are not posting updates every five minutes.


I would be happy if there were updates every five weeks.


I would be happy if there were updates every five months
ID: 103072 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 376
Credit: 10,765,034
RAC: 6,825
Message 103073 - Posted: 31 Oct 2021, 22:36:42 UTC - in response to Message 103070.  

Given that it’s now Sunday evening and the people who fix it work a normal working week I am not surprised that they are not posting updates every five minutes.


I would be happy if there were updates every five weeks.

Are you so desperate for credits that a problem that does not stop you from processing, just delays the credits from going to your account, sends you running to another project?


I don't care about credits. Credits means nothing. Credits is just a digit on my screen.


Then why move to another project just because your credits are delayed? I seriously do not understand the reasoning.
ID: 103073 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1498
Credit: 14,743,955
RAC: 16,793
Message 103085 - Posted: 1 Nov 2021, 20:20:16 UTC

Well, for a while there the Validation backlog started to reduce (slowly, but it was reducing), but now it's back on it's way up again.


Grant
Darwin NT
ID: 103085 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 376
Credit: 10,765,034
RAC: 6,825
Message 103086 - Posted: 2 Nov 2021, 1:39:16 UTC

I’m surprised that the drop is so small - my backlog has halved and tomorrow’s credits are looking good :-)
ID: 103086 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1498
Credit: 14,743,955
RAC: 16,793
Message 103087 - Posted: 2 Nov 2021, 2:57:11 UTC - in response to Message 103086.  

I’m surprised that the drop is so small - my backlog has halved and tomorrow’s credits are looking good :-)
It looks like just after my last post that they got the Validators working again.
The backlog is now half of what is was, so with a bit of luck things should be back to normal in about another 3-4 hours or so.

Latest graph.

Grant
Darwin NT
ID: 103087 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1498
Credit: 14,743,955
RAC: 16,793
Message 103088 - Posted: 2 Nov 2021, 8:02:41 UTC

And the backlog has cleared.
Grant
Darwin NT
ID: 103088 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1498
Credit: 14,743,955
RAC: 16,793
Message 103091 - Posted: 2 Nov 2021, 21:21:48 UTC




That's what we like to see.
Grant
Darwin NT
ID: 103091 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Paddles

Send message
Joined: 15 Mar 15
Posts: 11
Credit: 4,986,277
RAC: 4,255
Message 103101 - Posted: 4 Nov 2021, 10:43:59 UTC - in response to Message 103057.  

A task running MUCH longer than the expected 8 hours:

aaab_nNMALA_pp-SAR_pp-mPPS-BGLY_pp_2_2245795_6_1

https://boinc.bakerlab.org/rosetta/result.php?resultid=1441862159

2 days, 8 hours, 32 minutes so far

rosetta python 1.03 vbox64

This is elapsed time, not the much shorter CPU time.

Now aborted after 3 days and 20 hours elapsed, less than 10 minutes CPU time.

The python tasks need a major improvement in how they detect tasks taking too long to run,

Could the current validator be written in Python, and having this same problem?


Reassuring it's not just me. I've had a couple of the vbox tasks do that. I just aborted a task that had been running for 2d 12 hours elapsed, but only about 5 minutes of CPU. Supposedly was 99.8% complete but I think it was "99.6% complete" a day ago, and has gone a past deadline. (https://boinc.bakerlab.org/rosetta/result.php?resultid=1443593989

In this situation, is aborting it the most useful thing to do? I'm not really worried about credits or losing them - just want the CPU time to be doing something useful. Is letting it go on long past deadline still useful to someone, or should I manually abort tasks that seem to be wandering aimlessly so that the processing slot can go to another task?
ID: 103101 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1498
Credit: 14,743,955
RAC: 16,793
Message 103105 - Posted: 4 Nov 2021, 23:45:33 UTC - in response to Message 103101.  

In this situation, is aborting it the most useful thing to do?
IMHO- yep.
For normal Rosetta tasks the default Target CPU time is 8 hours. There is a watchdog timer that kicks in after 10 hours if it's not completed within that initial 8 hours- so 18hrs all up. From the few Python results i've seen they generally run for less than 8 hours anyway.
So if it's not done after a couple of days, i'd abort it- or possibly exit BOINC & restart & see if it then finishes off by itself within a few minutes of restarting. If not, then abort.
Grant
Darwin NT
ID: 103105 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 126 · 127 · 128 · 129 · 130 · 131 · 132 . . . 279 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org