Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 55 · 56 · 57 · 58 · 59 · 60 · 61 . . . 309 · Next

AuthorMessage
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2141
Credit: 41,526,036
RAC: 10,392
Message 97498 - Posted: 22 Jun 2020, 18:46:38 UTC - in response to Message 97493.  

It's been an excellent run for the last few months, but no tasks available for download

Thanks. I wish they would tell us whether it is an operational glitch with the servers, or a longer-term issue with supply.
We need to manage our machines too, if we are to do the work effectively.

The reaction to a lack of new tasks coming down always amuses me.
As far as I can make out there are still over 500k un-returned tasks, so it's an early-warning notice of about a day to the project, not users.
Users (in total) can be running those 500k tasks quite merrily until the project has more tasks to be worked on.
Rosetta won't create meaningless tasks just to keep users active - that was Seti's job and even they got fed up of doing it <cough>

Users can do any number of things:
We can temporarily increase runtimes to eke out the remaining tasks we have
We can download tasks from backup projects we set up for just this eventuality (we did that years ago, right?)
We can complete existing tasks and wait for meaningful work to become available again, knowing that we offer our capacity without any guarantee tasks will always be available to fill it.
We can use the downtime to clean and check PCs for the first time in years so that when new tasks arrive they'll run cleaner/cooler/more productively (I did this last week before I switched two furloughed PCs back on to great effect)

Users can avoid:
Whining about it, knowing that any shortfall in productivity will affect the project vastly more than any 1 user (or all users put together)

As far as managing our machines goes, I know it's been fashionable to pare offline caches to the bone, and that makes sense if there are problems running work or in meeting deadlines, but I've never done that.
As long as my cache is below (2 days, minus runtime, minus a margin for variability) I'm good and I've got time to react to short-term availability issues at the project.
So, anything below 1.5 days total works for my 24/7 PCs, 1.1 for my 24/7 remote PC and 0.6 for my remote work PC.
I set that up 3 or 4 years ago and adjusted my remote PCs when I finally got back to work last week.
Being out of tasks at Rosetta, short or long-term, makes no difference to those settings, so I'm not particularly waiting for any official message here outside of a casual interest
ID: 97498 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 97499 - Posted: 22 Jun 2020, 18:52:35 UTC - in response to Message 97498.  

I will be out of work on a few cores by this evening.
I am quite capable of managing it.
ID: 97499 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1234
Credit: 14,338,560
RAC: 2,014
Message 97500 - Posted: 22 Jun 2020, 19:09:52 UTC - in response to Message 97498.  

It's been an excellent run for the last few months, but no tasks available for download

Thanks. I wish they would tell us whether it is an operational glitch with the servers, or a longer-term issue with supply.
We need to manage our machines too, if we are to do the work effectively.

The reaction to a lack of new tasks coming down always amuses me.
As far as I can make out there are still over 500k un-returned tasks, so it's an early-warning notice of about a day to the project, not users.
Users (in total) can be running those 500k tasks quite merrily until the project has more tasks to be worked on.
Rosetta won't create meaningless tasks just to keep users active - that was Seti's job and even they got fed up of doing it <cough>
[snip]

Before that, Predictor@home did even worse - after the project team split up and they lost the two members who could create useful new workunits, the kept the project running for a few more months by repeatedly increasing the number of times unfinished workunits could fail before they were no longer had tasks sent out. Some of them had failed over 30 times before the project shut down.
ID: 97500 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2141
Credit: 41,526,036
RAC: 10,392
Message 97503 - Posted: 22 Jun 2020, 21:26:58 UTC - in response to Message 97499.  
Last modified: 22 Jun 2020, 21:35:10 UTC

I will be out of work on a few cores by this evening.
I am quite capable of managing it.

I'd have thought that already - I was a bit surprised you suggested otherwise.
Still worth going through the stages for any newcomers.

Two other things:
UW are UTC -9 I think, so more time to get a solution together
Any switch of tasks to other projects creates a 'debt' with Boinc for Rosetta, which will be 'repaid' once tasks become available again

Edit: Just noticed my work PC completed all Rosetta tasks already and buffers starting to be filled on two other machines ahead of completing remaining Rosetta tasks. All going as expected
ID: 97503 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 97507 - Posted: 23 Jun 2020, 1:15:47 UTC - in response to Message 97503.  

Edit: Just noticed my work PC completed all Rosetta tasks already and buffers starting to be filled on two other machines ahead of completing remaining Rosetta tasks. All going as expected

You are lucky, or maybe are getting re-sends. One of my machines is starting to going dry. That is my concern. They may come back immediately, or not. You don't know. Good luck.
ID: 97507 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2141
Credit: 41,526,036
RAC: 10,392
Message 97521 - Posted: 23 Jun 2020, 8:20:03 UTC - in response to Message 97507.  

Edit: Just noticed my work PC completed all Rosetta tasks already and buffers starting to be filled on two other machines ahead of completing remaining Rosetta tasks. All going as expected

You are lucky, or maybe are getting re-sends. One of my machines is starting to going dry. That is my concern. They may come back immediately, or not. You don't know. Good luck.

Sorry, I missed out the important words "with WCG tasks", not Rosetta - completely changing the meaning of that edit...

And since then, a 2nd of my 4 PCs has run out of Rosetta tasks.

And since then, it looks like 70k+ tasks have been issued, increasing in progress tasks, 6 of which I grabbed for 1 PC and none for the others, and they've all been gobbled up.
So something is happening, but insufficient to meet demand yet. Fingers crossed for later today.
ID: 97521 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
gw666

Send message
Joined: 30 Apr 20
Posts: 2
Credit: 387,116,939
RAC: 0
Message 97525 - Posted: 23 Jun 2020, 10:40:24 UTC

It seems I don't get new tasks, this is a typical message on one of my machines:
23-Jun-2020 10:58:33 [Rosetta@home] Scheduler request completed: got 0 new tasks
23-Jun-2020 10:58:33 [Rosetta@home] No tasks sent
23-Jun-2020 11:04:08 [Rosetta@home] Sending scheduler request: To fetch work.
23-Jun-2020 11:04:08 [Rosetta@home] Requesting new tasks for CPU
23-Jun-2020 11:04:11 [Rosetta@home] Scheduler request completed: got 0 new tasks
23-Jun-2020 11:04:11 [Rosetta@home] No tasks sent
23-Jun-2020 11:17:48 [Rosetta@home] Sending scheduler request: To fetch work.
23-Jun-2020 11:17:48 [Rosetta@home] Requesting new tasks for CPU
23-Jun-2020 11:17:51 [Rosetta@home] Scheduler request completed: got 0 new tasks
23-Jun-2020 11:17:51 [Rosetta@home] No tasks sent
23-Jun-2020 12:16:33 [Rosetta@home] Sending scheduler request: To fetch work.
23-Jun-2020 12:16:33 [Rosetta@home] Requesting new tasks for CPU
23-Jun-2020 12:16:37 [Rosetta@home] Scheduler request completed: got 0 new tasks
23-Jun-2020 12:16:37 [Rosetta@home] No tasks sent

This machine has 32 cores and only one is occupied at the moment.
ID: 97525 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1234
Credit: 14,338,560
RAC: 2,014
Message 97529 - Posted: 23 Jun 2020, 13:59:52 UTC - in response to Message 97525.  

gw666,

That has been typical for the last few days,

It's a good reason to add another BOINC project on your computer. I like World Community Grid, since their Open Pandemics subproject is currently working on COVID-19.

https://join.worldcommunitygrid.org?recruiterId=480838
ID: 97529 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 97535 - Posted: 23 Jun 2020, 20:04:54 UTC - in response to Message 97507.  

Edit: Just noticed my work PC completed all Rosetta tasks already and buffers starting to be filled on two other machines ahead of completing remaining Rosetta tasks. All going as expected

You are lucky, or maybe are getting re-sends. One of my machines is starting to going dry. That is my concern. They may come back immediately, or not. You don't know. Good luck.


Why is it of concern? I look at it as a good thing that we completed the work load. Now there's little chunks coming in, they get taken immediately. Best for the science to get done rapidly than they have to wait until we finish it. I assume they've reached a key point in the research.
ID: 97535 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile yoerik
Avatar

Send message
Joined: 24 Mar 20
Posts: 128
Credit: 169,525
RAC: 0
Message 97539 - Posted: 23 Jun 2020, 21:28:54 UTC - in response to Message 97535.  

Edit: Just noticed my work PC completed all Rosetta tasks already and buffers starting to be filled on two other machines ahead of completing remaining Rosetta tasks. All going as expected

You are lucky, or maybe are getting re-sends. One of my machines is starting to going dry. That is my concern. They may come back immediately, or not. You don't know. Good luck.


Why is it of concern? I look at it as a good thing that we completed the work load. Now there's little chunks coming in, they get taken immediately. Best for the science to get done rapidly than they have to wait until we finish it. I assume they've reached a key point in the research.

can't speak for other users, but I know for me that I'm less concerned about the shortage overall - moreso concerned by the continued lack of communication with volunteers since I joined in late march, 2020.
ID: 97539 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2141
Credit: 41,526,036
RAC: 10,392
Message 97549 - Posted: 24 Jun 2020, 1:38:57 UTC - in response to Message 97521.  

And since then, it looks like 70k+ tasks have been issued, increasing in progress tasks, 6 of which I grabbed for 1 PC and none for the others, and they've all been gobbled up.
So something is happening, but insufficient to meet demand yet. Fingers crossed for later today.

Some time between 23:00 UTC and 01:25 UTC, in progress tasks increased by 130k with 30k more showing ready to send
Will have to wait until the front page updates to be sure, but it looks like a healthy batch of work has now become available
ID: 97549 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 97564 - Posted: 24 Jun 2020, 13:40:32 UTC - in response to Message 97549.  
Last modified: 24 Jun 2020, 13:41:02 UTC

And since then, it looks like 70k+ tasks have been issued, increasing in progress tasks, 6 of which I grabbed for 1 PC and none for the others, and they've all been gobbled up.
So something is happening, but insufficient to meet demand yet. Fingers crossed for later today.

Some time between 23:00 UTC and 01:25 UTC, in progress tasks increased by 130k with 30k more showing ready to send
Will have to wait until the front page updates to be sure, but it looks like a healthy batch of work has now become available


Looks more like small batches which are gobbled immediately, it could mean they're doing finishing touches to a study?

And the set of 12 tasks I got an hour ago (which would be 12:40pm UTC 24th June) had a download error, permanent HTTP error on one of the data files.
ID: 97564 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
KWSN Ekky Ekky Ekky

Send message
Joined: 3 Apr 20
Posts: 9
Credit: 5,062,511
RAC: 0
Message 97565 - Posted: 24 Jun 2020, 15:15:36 UTC

"Deadlines for Rosetta@home tasks are 3 days from the time they are issued. "
Is there any sesible reason for sauch a short time?
Spending lots of time crunching is pointless if a task is "Completed, too late to validate". Seems more like an insult to people donating their time to then slap them in the face.
ID: 97565 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 97566 - Posted: 24 Jun 2020, 15:27:52 UTC - in response to Message 97565.  
Last modified: 24 Jun 2020, 15:29:08 UTC

"Deadlines for Rosetta@home tasks are 3 days from the time they are issued. "
Is there any sesible reason for sauch a short time?
Spending lots of time crunching is pointless if a task is "Completed, too late to validate". Seems more like an insult to people donating their time to then slap them in the face.


Perhaps the scientists don't want to wait two weeks to get the answers they need? That seems perfectly reasonable to me. A task takes 8 hours, so 3 days is plenty of time.

Your Boinc client shouldn't be downloading more than it can get done. It should learn how often your computer runs, what other projects you're doing, etc.

But the best way is to set your cache to a small value - I have mine on 0.13 + 0.13 days (which is 3-6 hours). There's no need to queue lots of tasks.
ID: 97566 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
KWSN Ekky Ekky Ekky

Send message
Joined: 3 Apr 20
Posts: 9
Credit: 5,062,511
RAC: 0
Message 97567 - Posted: 24 Jun 2020, 16:15:28 UTC - in response to Message 97566.  

The Rosetta deadlines are beyond silly.
People are crunching and losing time after time. I am not complaining about work that take "two weeks" but three days. I keep one alleged day of work but when Rosetta's unrealistic times are always vastly shorter than the actual time it takes, it is ridiculous. Then to reject a result because it is out of such a daft short time is to be frankly insulting.
I do note the very large number of people who have left and continue to leave the project and it does not surprise me in the least. It has to be time to rething some things and that includes the three day time limit.
ID: 97567 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 97568 - Posted: 24 Jun 2020, 17:00:25 UTC - in response to Message 97567.  
Last modified: 24 Jun 2020, 17:04:43 UTC

The Rosetta deadlines are beyond silly.
People are crunching and losing time after time. I am not complaining about work that take "two weeks" but three days. I keep one alleged day of work but when Rosetta's unrealistic times are always vastly shorter than the actual time it takes, it is ridiculous. Then to reject a result because it is out of such a daft short time is to be frankly insulting.
I do note the very large number of people who have left and continue to leave the project and it does not surprise me in the least. It has to be time to rething some things and that includes the three day time limit.


I don't understand what you're getting at. You receive a task that takes 8 hours to complete, and you have to send it back in 3 days (9 times longer than it takes to do it). How can that possibly cause you not to get them done in time?

The predicted times are not "unrealistic", they're precisely correct. In fact they're somehow limited to 8 or 12 hours. If yours take longer than that, there's something very wrong with your computer.

And it's not a daft short time, it's presumably because the scientists need those results back quickly so they can work on the next stage.

I've looked at your computers, and two of them are working just fine, all tasks were validated fine, and done in exactly 8 hours each. Only the i5-8400 is playing up, it's taking 24 hours to do a task, but only doing 8 hours of processing. Is your CPU doing something else aswell? Is it overheating? Have you limited Boinc to use 33% of CPU time? If the latter, change that to 100% and put the cores down to 33% instead. That will run less tasks at once, but at full speed.
ID: 97568 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Daedalus

Send message
Joined: 1 Aug 08
Posts: 39
Credit: 10,107,661
RAC: 56
Message 97573 - Posted: 24 Jun 2020, 19:22:37 UTC

The BOINC manager is not that good at predicting my compute time and anyway i am not home on weekends so i always have some tasks due to finish a day my computer will be off. So at the end of the week i have to cancel a batch. Systematically.

I understand they want their results fast but i cannot always provide.
ID: 97573 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1234
Credit: 14,338,560
RAC: 2,014
Message 97574 - Posted: 24 Jun 2020, 19:33:20 UTC - in response to Message 97573.  

The BOINC manager is not that good at predicting my compute time and anyway i am not home on weekends so i always have some tasks due to finish a day my computer will be off. So at the end of the week i have to cancel a batch. Systematically.

I understand they want their results fast but i cannot always provide.

Have you tried setting No new tasks about the time any more tasks downloaded will not finish in time?
ID: 97574 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2141
Credit: 41,526,036
RAC: 10,392
Message 97576 - Posted: 24 Jun 2020, 19:59:59 UTC - in response to Message 97565.  

"Deadlines for Rosetta@home tasks are 3 days from the time they are issued."
Is there any sensible reason for such a short time?
Spending lots of time crunching is pointless if a task is "Completed, too late to validate". Seems more like an insult to people donating their time to then slap them in the face.

Not sure if you count it as a sensible reason, but around 6,000 people are dying in the world of CV19 every day for the last few months.
The quicker they come back, the quicker the next iteration of tasks can go out again.

You're not spending a lot of time crunching tasks if you can't get an 8hr task back within 3 days. The average turnaround time at this project is 1.3 days, including all delays - a very large number inside 24hrs.
Either you're machine isn't working on them much or your buffer of tasks is too high to work through them by the deadline - or a combination of the two.

Especially while tasks are in short supply, you should reduce your offline cache of tasks by a margin that allows you to complete them on time instead of just waiting in a queue, because many others have run out of work and would complete them inside half a day.

You picked the wrong day to say you've got too many tasks to complete within the deadline...
ID: 97576 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Daedalus

Send message
Joined: 1 Aug 08
Posts: 39
Credit: 10,107,661
RAC: 56
Message 97577 - Posted: 24 Jun 2020, 20:00:14 UTC - in response to Message 97574.  
Last modified: 24 Jun 2020, 20:02:05 UTC


Have you tried setting No new tasks about the time any more tasks downloaded will not finish in time?


Yes but it doesn't work out well enough. Reducing the cache helped better but i still have to cancel some. It doesn't help that i sometimes stop to crunch for Folding. :)
ID: 97577 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 55 · 56 · 57 · 58 · 59 · 60 · 61 . . . 309 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org