Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 230 · 231 · 232 · 233 · 234 · 235 · 236 . . . 257 · Next

AuthorMessage
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 91
Credit: 4,370,591
RAC: 2,745
Message 106986 - Posted: 27 Sep 2022, 14:07:22 UTC - in response to Message 106976.  

Are they taking 8 hours?
That is the initial Estimated completion time, regardless of your set Target CPU time.

If they are taking 8 hours (for those using the project default), then it must have just been the systems that first returned them were set to 3 hours.


I do not know, but the four I have running (these are the first four I got) say they are 50% done and have about 4 hours to go.
ID: 106986 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,138,291
RAC: 10,433
Message 106987 - Posted: 27 Sep 2022, 14:22:09 UTC - in response to Message 106986.  

Are they taking 8 hours?
That is the initial Estimated completion time, regardless of your set Target CPU time.

If they are taking 8 hours (for those using the project default), then it must have just been the systems that first returned them were set to 3 hours.
I do not know, but the four I have running (these are the first four I got) say they are 50% done and have about 4 hours to go.
Are they using a full CPU core? I had some takinga lotlongerthan 3hours,but the CPutiewasalways exactly3 hours. usingBoinctaskswilltellyou yourcputime.

Excusethe lackofspaces,mykeyboardwasjust viciouslyattackedby an African Grey parrotwithout provocation.
ID: 106987 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 91
Credit: 4,370,591
RAC: 2,745
Message 106990 - Posted: 27 Sep 2022, 19:33:54 UTC - in response to Message 106987.  

I do not know, but the four I have running (these are the first four I got) say they are 50% done and have about 4 hours to go.

Are they using a full CPU core? I had some takinga lotlongerthan 3hours,but the CPutiewasalways exactly3 hours. usingBoinctaskswilltellyou yourcputime.


What do you mean, Are they using a full CPU core? My machine has 16 cores of which I permit 8 of them to be used by the Boinc tasks, and each of the projects is allowed up to four cores. So, I would say yes, they are using a full core.

Task            Work unit	Sent	                    Time reported          Status	        Run time  CPU time  Credit Application

1494624336 	1329099049 	27 Sep 2022, 9:59:08 UTC 27 Sep 2022, 18:54:13 UTC Completed,validated 	28,888.37 28,678.61 662.09  Rosetta v4.20
x86_64-pc-linux-gnu
1494624082 	1329098776 	27 Sep 2022, 9:59:08 UTC 27 Sep 2022, 18:54:13 UTC Completed,validated 	28,479.40 28,275.87 587.82  Rosetta v4.20
x86_64-pc-linux-gnu
1494624396 	1329099169 	27 Sep 2022, 9:59:08 UTC 27 Sep 2022, 18:54:13 UTC Completed,validated 	29,307.91 29,079.61 647.73  Rosetta v4.20
x86_64-pc-linux-gnu
1494623988 	1329098775 	27 Sep 2022, 9:59:08 UTC 27 Sep 2022, 18:54:13 UTC Completed,validated 	28,924.58 28,702.42 594.36  Rosetta v4.20
x86_64-pc-linux-gnu

ID: 106990 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,138,291
RAC: 10,433
Message 106991 - Posted: 27 Sep 2022, 19:43:53 UTC - in response to Message 106990.  
Last modified: 27 Sep 2022, 19:47:22 UTC

No Rosetta project uses more than one core. If you're only running 8 Rosettas and nothing else, then yes they're getting whole core each. Mine on a variety of computers are all running around 3 hours each. They only take longer if the CPU is overloaded with other things (I also run Folding at Home.)

All my hallucinated tasks take 3 hours (+/- 10 minutes) and my RB ones 8 hours. Not sure why your hallucinated take 8 hours. The only difference I can see is you run Linux. Maybe you got a different batch of work to the Windows users? Or somebody forgot to change the timer for Linux?
ID: 106991 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 91
Credit: 4,370,591
RAC: 2,745
Message 106992 - Posted: 27 Sep 2022, 20:20:53 UTC - in response to Message 106991.  


No Rosetta project uses more than one core. If you're only running 8 Rosettas and nothing else, then yes they're getting whole core each. Mine on a variety of computers are all running around 3 hours each. They only take longer if the CPU is overloaded with other things (I also run Folding at Home.)


I never said they did use more than one core each work unit. AFAIK, only MilkyWay can take more than one core each. I have it set so those tasks get a total of four cores.
I try to run all 8 cores, but I allow only 4 cores for any one project.

If your task take only three hours each, you have a faster machine than mine. I thought they were set up, by default, to take about 8 hours each. So even if your machine is over twice as fast as mine, then each task should still take about 8 hours, and your machine should just do that much more work.

All my hallucinated tasks take 3 hours (+/- 10 minutes) and my RB ones 8 hours. Not sure why your hallucinated take 8 hours. The only difference I can see is you run Linux. Maybe you got a different batch of work to the Windows users? Or somebody forgot to change the timer for Linux?


I have no idea what an hallucinated task is. I do not believe I run any of those, and I would have no idea how long they would take. My machine runs Red Hat Enterprise Linux release 8.6 (Ootpa), so it should get no Windows work. I do have a little Windows machine and it may be doing what you say. But it does not do much.

What timer would they have forgotten to set?
ID: 106992 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,138,291
RAC: 10,433
Message 106993 - Posted: 27 Sep 2022, 20:43:32 UTC - in response to Message 106992.  

No Rosetta project uses more than one core. If you're only running 8 Rosettas and nothing else, then yes they're getting whole core each. Mine on a variety of computers are all running around 3 hours each. They only take longer if the CPU is overloaded with other things (I also run Folding at Home.)
I never said they did use more than one core each work unit. AFAIK, only MilkyWay can take more than one core each. I have it set so those tasks get a total of four cores.
I try to run all 8 cores, but I allow only 4 cores for any one project.
Ah, I misunderstood when you said "each of the projects is allowed up to four cores", I read it as tasks, not projects.

If your task take only three hours each, you have a faster machine than mine. I thought they were set up, by default, to take about 8 hours each. So even if your machine is over twice as fast as mine, then each task should still take about 8 hours, and your machine should just do that much more work.
I have slow and fast machines, and they all take 3 hours, they used to take 8, but the recent tasks seem to be shorter.

I have no idea what an hallucinated task is.
The last batch have names like "hallu_14_hallucinated_7_1_best_SAVE_ALL_OUT_2917450_982_0"

I do not believe I run any of those, and I would have no idea how long they would take. My machine runs Red Hat Enterprise Linux release 8.6 (Ootpa), so it should get no Windows work. I do have a little Windows machine and it may be doing what you say. But it does not do much.
It's quite possible they sent different types of task to Windows and Linux machines.

What timer would they have forgotten to set?
The default (if you haven't picked your own runtime in your account settings) of how long each task takes. For some reason they like them all to take x hours no matter how fast or slow your computer is. It has always been usually 8, but sometimes there are shorter ones. Anything with hallucinated in the name seems to be 3 hours.
ID: 106993 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 91
Credit: 4,370,591
RAC: 2,745
Message 106994 - Posted: 27 Sep 2022, 22:20:31 UTC - in response to Message 106993.  

I have slow and fast machines, and they all take 3 hours, they used to take 8, but the recent tasks seem to be shorter.

I have no idea what an hallucinated task is.

The last batch have names like "hallu_14_hallucinated_7_1_best_SAVE_ALL_OUT_2917450_982_0"


OK: all of my current jobs are hallu_14

Tue 27 Sep 2022 05:59:08 AM EDT | Rosetta@home | Requesting new tasks for CPU
Tue 27 Sep 2022 05:59:09 AM EDT | Rosetta@home | Scheduler request completed: got 39 new tasks
Tue 27 Sep 2022 05:59:09 AM EDT | Rosetta@home | Project requested delay of 31 seconds
Tue 27 Sep 2022 05:59:11 AM EDT | Rosetta@home | Started download of hallu_14_hallucinated_180_4_best.zip
Tue 27 Sep 2022 05:59:11 AM EDT | Rosetta@home | Started download of hallu_14_hallucinated_180_4_best.flags
Tue 27 Sep 2022 05:59:13 AM EDT | Rosetta@home | Finished download of hallu_14_hallucinated_180_4_best.zip
Tue 27 Sep 2022 05:59:13 AM EDT | Rosetta@home | Finished download of hallu_14_hallucinated_180_4_best.flags
...
What timer would they have forgotten to set?

The default (if you haven't picked your own runtime in your account settings) of how long each task takes. For some reason they like them all to take x hours no matter how fast or slow your computer is. It has always been usually 8, but sometimes there are shorter ones. Anything with hallucinated in the name seems to be 3 hours.


It is still 8 hours on my machine.

Computer 5910575

Created 	14 Nov 2020, 17:25:40 UTC
Total credit 	1,556,326
Average credit 	753.42

CPU type 	GenuineIntel
Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz [Family 6 Model 85 Stepping 7]
Number of processors 	16
Operating System 	Linux Red Hat Enterprise Linux
Red Hat Enterprise Linux 8.6 (Ootpa) [4.18.0-372.26.1.el8_6.x86_64|libc 2.28]
BOINC version 	7.20.2
Memory 	 63772.82 MB
Cache 	                16896 KB
Swap space 	        15992 MB
Total disk space   488.04 GB
Free Disk Space 	     481 GB
Measured floating point speed 6580.07 million ops/sec
Measured integer speed 	      30457.57 million ops/sec
Average upload rate 	             182.14 KB/sec
Average download rate 	20051.93 KB/sec
Average turnaround time 	1 days

ID: 106994 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 1902
Credit: 35,032,850
RAC: 3,911
Message 106996 - Posted: 28 Sep 2022, 0:02:32 UTC - in response to Message 106955.  
Last modified: 28 Sep 2022, 0:03:49 UTC

I haven't been here for while. No idea why not - just forgot I suppose.
So I've only just realised Python tasks are exhausted and I haven't seen any Rosetta 4.20 tasks for as long or longer.

About time I asked the question, which I'll look to do in the next couple of days.

First, I'm not responsible for hurrying along the Rosetta 4.20 "hallucinated" tasks today.
Tbh, It was their arrival that reminded me to send an email about Pythons etc, which I've now done.
Fwiw, all my Rosetta tasks are running 8hrs, not 3 hours like for some people. No idea what's going on there.
But I now have a response, which I think is pretty useful and I hope you do too.

There are plans for more python tasks, but we are still working on creating the new disk image for the VM.
This image will likely be larger in size since we hope to run some of the latest deep-learning based methods for protein design and validation.

Computing resources are still very much in high demand within the Baker Lab and IPD so Rosetta@home will definitely help once we can get the latest software set up and running.

Also, the standard rosetta application is currently being worked on for an update.
The IPD has hired a software developer specifically to help expedite this.

I am not sure about timelines at the moment but hopefully there should be updates to both projects/apps within the next month or so.

David Kim

ID: 106996 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 1902
Credit: 35,032,850
RAC: 3,911
Message 106997 - Posted: 28 Sep 2022, 0:28:14 UTC - in response to Message 106996.  

On Python tasks, we've already had issues on memory and disk demands and tweaked our setups as best we can and prevailed on the project to pare those demands to the bone.
Hopefully the larger VM disk image doesn't impact that further and tasks will run in the same footprint we've become accustomed to, but there's no guarantee.
What we can say is it won't get smaller, so we'll have to continue to understand that our older and smaller machines will become less able to contribute over time.

On the continuing need for our volunteer capacity, it sounds like work here will continue to be needed, though whether that's at the current rate or more or less remains to be seen.
Some will continue to think we're being strung along, but I chose not to engage with recent paranoia and I'll continue to be available if we're needed.
This is backed up by mention of hiring someone to work on the Rosetta application - for the first time and you don't do that if you're planning to mothball the project.

Reading literally, I'll hope to see something at the end of October or more likely the beginning of November, so let's see how that timeline goes.
ID: 106997 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,138,291
RAC: 10,433
Message 106999 - Posted: 28 Sep 2022, 10:54:10 UTC

Good to hear they are working hard on new stuff for 4.2 and Python. After watching David Baker's video at TED posted by Kotenok it seems a very worthwhile cause.

Only newer CPUs can run Python due to the AVX instruction being required. The three I have that will run it will take 64GB, 128GB, and 128GB RAM, so if it needs it I'll buy it. They currently only have 48GB, 64GB, 32GB. My antiques that will only take 8GB RAM have a CP{U incapable of Python anyway.
ID: 106999 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
miyamotomusasi

Send message
Joined: 29 Sep 22
Posts: 10
Credit: 0
RAC: 0
Message 107005 - Posted: 29 Sep 2022, 10:37:22 UTC

I'd like to hit the like button on your post. Of course, writing is the best, but writing with your precious time is even better. I wish you all the best in the future. I will guide you to my site too :) 메이저사이트
ID: 107005 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 1902
Credit: 35,032,850
RAC: 3,911
Message 107011 - Posted: 30 Sep 2022, 0:32:18 UTC - in response to Message 106997.  

Reading literally, I'll hope to see something at the end of October or more likely the beginning of November, so let's see how that timeline goes.

I did also mention that any tasks they want to push through before any new version of Rosetta or Python is available will be welcome, so it was pleasing to see another small batch of "hallucinated" tasks come through today.
No, I don't think my comment made the slightest difference in receiving those new tasks, but seeing almost all of them downloaded already backs up the point I made
ID: 107011 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 202
Credit: 23,702,139
RAC: 5,103
Message 107012 - Posted: 30 Sep 2022, 1:13:58 UTC

What a lot of R4.2 work - or am I hallucinate`ing .
Nice to know that rosetta may have work ready when I need to switch the `heaters` on :-)

spam , SPAM , SPAM!!! they have been busy today , sorry , I did`nt read your post correctly .

Parrots , Long ago when I was small and not as worldly wise as I am now {aka = stupid kid} I ended up with thirty gerbil`s , wow can they breed , worse than humans .
what would 8 billion gerbils do to a planet , chew it into little piece`s and shit all over it , hmmm . . . . .
ID: 107012 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,138,291
RAC: 10,433
Message 107013 - Posted: 30 Sep 2022, 10:36:51 UTC - in response to Message 107011.  

I did also mention that any tasks they want to push through before any new version of Rosetta or Python is available will be welcome, so it was pleasing to see another small batch of "hallucinated" tasks come through today.
No, I don't think my comment made the slightest difference in receiving those new tasks, but seeing almost all of them downloaded already backs up the point I made
Some of the hallucinate ones said "test", so something new is happening.
ID: 107013 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,138,291
RAC: 10,433
Message 107014 - Posted: 30 Sep 2022, 10:40:33 UTC - in response to Message 107012.  

What a lot of R4.2 work - or am I hallucinate`ing .
Nice to know that rosetta may have work ready when I need to switch the `heaters` on :-)
I am, but I smoke things I'm not supposed to in this country.

Parrots , Long ago when I was small and not as worldly wise as I am now {aka = stupid kid} I ended up with thirty gerbil`s , wow can they breed , worse than humans .
what would 8 billion gerbils do to a planet , chew it into little piece`s and shit all over it , hmmm . . . . .
I have 20 parrots and they are too fussy, difficult to make them breed. If they all bred properly, I could make £30,000 British / $32,500 USA per year.
ID: 107014 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Wolfpac

Send message
Joined: 11 Aug 07
Posts: 2
Credit: 27,991,308
RAC: 1,167
Message 107019 - Posted: 1 Oct 2022, 23:37:17 UTC

Was thinking about chaning my rigs over to a new project to see, lo an behold Rosetta starts getting work again! :D
ID: 107019 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 1902
Credit: 35,032,850
RAC: 3,911
Message 107020 - Posted: 2 Oct 2022, 1:06:16 UTC - in response to Message 107019.  

Was thinking about changing my rigs over to a new project to see, lo and behold Rosetta starts getting work again! :D

Yes! 3rd batch in about a week - this one is a little bigger than the last two, I think.
Not sure what's going on!
ID: 107020 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1380
Credit: 13,693,695
RAC: 247
Message 107021 - Posted: 2 Oct 2022, 1:38:18 UTC - in response to Message 107020.  

Yes! 3rd batch in about a week - this one is a little bigger than the last two, I think.
Way bigger.
One of (if not the) biggest in the last 2 years- certainly the biggest in the last 12 months.
Around 850,000 or so (120k in progress, 725k in the queue).
Grant
Darwin NT
ID: 107021 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
srettie
Volunteer moderator
Project developer
Project scientist

Send message
Joined: 10 Jul 17
Posts: 4
Credit: 50,961
RAC: 0
Message 107022 - Posted: 2 Oct 2022, 5:34:33 UTC - in response to Message 107021.  

Hello, these are from me. I thought I'd double check how things looked on the forums after submitting such a massive run.

This is part of a massive sampling of small cyclic peptides generated using some pretty cool tweaks to Alphafold 2. The term hallucinated refers to protein structures generated using this method: https://www.nature.com/articles/s41586-021-04184-w

These jobs are generating energy landscapes for structurally diverse cluster centers. I'm super excited about these results and really appreciate the incredible amount of computation available from Rosetta@home. The info we get from this will help us put together a massive set of peptide backbones for binder design projects in the future.

I hope to be submitting more... Assuming I'm not breaking any rules by monopolizing this resource.

Thank you all!

-Stephen
ID: 107022 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1380
Credit: 13,693,695
RAC: 247
Message 107023 - Posted: 2 Oct 2022, 5:50:59 UTC - in response to Message 107022.  

I hope to be submitting more... Assuming I'm not breaking any rules by monopolizing this resource.
Nope.
Many of the systems that used to process Rosetta work have been moved to other projects due to the lack of Rosetta 4.20 work here for some time now. And due to the system requirements of Python Tasks, very few systems were capable of processing them, and because of issues with the application/ some Tasks many others that could have processed them chose not to because of the manual intervention often required to sort out those issues.

The more Rosetta 4.20 work there is, then the more people that will come to process it, and the sooner it will be returned.
Grant
Darwin NT
ID: 107023 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 230 · 231 · 232 · 233 · 234 · 235 · 236 . . . 257 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2022 University of Washington
https://www.bakerlab.org