Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 82 · 83 · 84 · 85 · 86 · 87 · 88 . . . 309 · Next

AuthorMessage
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100795 - Posted: 22 Mar 2021, 2:05:24 UTC - in response to Message 100794.  

Basically: at the end of each chunk the task decides whether it thinks it has time to complete another one without going over the target run time.


This is really interesting. So what happens in the event that a task ends early and has a chunk left over? Does it get added to a different task? It seems like, at some time or another, you would have a task which is mainly composed of "orphan" chunks.
ID: 100795 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 398
Credit: 12,294,748
RAC: 6,222
Message 100798 - Posted: 22 Mar 2021, 9:56:40 UTC - in response to Message 100795.  

Basically: at the end of each chunk the task decides whether it thinks it has time to complete another one without going over the target run time.


This is really interesting. So what happens in the event that a task ends early and has a chunk left over? Does it get added to a different task? It seems like, at some time or another, you would have a task which is mainly composed of "orphan" chunks.


My understanding is that Work units consist of a protein chain in an initial state. Each task within the work unit then takes a random seed value which determines where and how to start folding that protein in the search for the lowest energy configuration so there are effectively a near infinite number of tasks that can be performed.

When the work unit is returned any promising configurations found can be used as the starting point for another work unit or a particularly good configuration can be accepted as a working model for the protein.

I would be interested in any corrections to this understanding from those that know.
ID: 100798 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 100799 - Posted: 22 Mar 2021, 10:41:25 UTC - in response to Message 100795.  

It doesn’t matter. The aim is not to examine every possibility: given the unfathomably large search space, it cannot be. With hundreds of thousands of work units in each batch, it is statistically insignificant whether any individual task completes N or N⁠+⁠1 models. The probability that the ‘orphan’ is the one that will cure all the world’s ills is negligible.

What I imagine does happen is that if any regions of the search space look ‘interesting’ they will be studied more closely in a subsequent batch of work. That is one of the reasons the task deadlines are so short: the results of one batch are analysed rapidly to guide the choice of parameters for the next.
ID: 100799 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1233
Credit: 14,338,560
RAC: 2,014
Message 100802 - Posted: 22 Mar 2021, 16:13:50 UTC - in response to Message 100798.  

[snip]

My understanding is that Work units consist of a protein chain in an initial state. Each task within the work unit then takes a random seed value which determines where and how to start folding that protein in the search for the lowest energy configuration so there are effectively a near infinite number of tasks that can be performed.

When the work unit is returned any promising configurations found can be used as the starting point for another work unit or a particularly good configuration can be accepted as a working model for the protein.

I would be interested in any corrections to this understanding from those that know.

Some of them are like that. Some are, instead, one step each from a list of starting points.

Some are two proteins, to see if these two will bind together.

I'm sure that there are more varieties I don't yet know about.
ID: 100802 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 398
Credit: 12,294,748
RAC: 6,222
Message 100803 - Posted: 22 Mar 2021, 16:54:49 UTC - in response to Message 100802.  

[snip]

My understanding is that Work units consist of a protein chain in an initial state. Each task within the work unit then takes a random seed value which determines where and how to start folding that protein in the search for the lowest energy configuration so there are effectively a near infinite number of tasks that can be performed.

When the work unit is returned any promising configurations found can be used as the starting point for another work unit or a particularly good configuration can be accepted as a working model for the protein.

I would be interested in any corrections to this understanding from those that know.

Some of them are like that. Some are, instead, one step each from a list of starting points.

Some are two proteins, to see if these two will bind together.

I'm sure that there are more varieties I don't yet know about.


Thanks, I’m learning slowly :-)
ID: 100803 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100804 - Posted: 23 Mar 2021, 2:08:32 UTC - in response to Message 100799.  

given the unfathomably large search space


Okay, scratch the infinite monkey approach then.

Where does the excess potential energy go?
ID: 100804 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FrankMeade

Send message
Joined: 8 Mar 20
Posts: 1
Credit: 11,549,660
RAC: 0
Message 100823 - Posted: 24 Mar 2021, 15:52:59 UTC

I am getting a little sick of the "waiting for memory" practice of suspending computation of a task when there is no shortage of available memory
ID: 100823 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 398
Credit: 12,294,748
RAC: 6,222
Message 100824 - Posted: 24 Mar 2021, 16:19:36 UTC - in response to Message 100823.  

I am getting a little sick of the "waiting for memory" practice of suspending computation of a task when there is no shortage of available memory


Is it restricted to certain computer(s) and / or certain project(s)?
ID: 100824 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 100825 - Posted: 24 Mar 2021, 16:43:29 UTC - in response to Message 100823.  

Is this happening when BOINC is trying to switch between projects to satisfy resource share settings? Rosetta tasks allocate hundreds of megabytes each; if they’re being kept in memory when something else is trying to run, there may well be a shortage. Try deselecting Computing preferences » Leave non-GPU tasks in memory while suspended.

Aside: Running the benchmarks on this 3900X will get it earning the right amount of credit. (There may be other machines that need it too; I didn’t go through them all…)
ID: 100825 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100827 - Posted: 25 Mar 2021, 1:51:46 UTC - in response to Message 100825.  

Is this happening when BOINC is trying to switch between projects to satisfy resource share settings?


I see it, on multiple machines, and I'm only running Rosetta.

It's peculiar. There's clearly enough free main memory to run a project, not to mention the swap. Diddling with the values in preferences doesn't seem to wake it up, either. It just goes away on its own.

I've written it off to a cost of doing business, but if there is a fix for it I wouldn't mind at all.
ID: 100827 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 100828 - Posted: 25 Mar 2021, 2:00:07 UTC - in response to Message 100827.  

I see it, on multiple machines, and I'm only running Rosetta.

It's peculiar. There's clearly enough free main memory to run a project, not to mention the swap.
Several of your machines have only 8 GB memory. This isn't enough for 12 cores.
You need at least 1 GB/core to run Rosetta (I usually have 32 GB on my 12-core machines, and more on the larger ones).
Just looking at the "free" memory doesn't do it. Rosetta (and all other BOINC projects too) need to reserve a certain amount to run.
They won't leave home without it.
ID: 100828 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1725
Credit: 18,378,164
RAC: 20,578
Message 100831 - Posted: 25 Mar 2021, 8:09:00 UTC - in response to Message 100828.  
Last modified: 25 Mar 2021, 8:13:20 UTC

I see it, on multiple machines, and I'm only running Rosetta.

It's peculiar. There's clearly enough free main memory to run a project, not to mention the swap.
Several of your machines have only 8 GB memory. This isn't enough for 12 cores.
You need at least 1 GB/core to run Rosetta (I usually have 32 GB on my 12-core machines, and more on the larger ones).
And 32GB for 24 cores isn't enough either if you want to use all of them to do Rosetta work.
I generally allow 1.3GB of RAM per Task- you need to leave enough for the Operating System & and any other programmes that might be running as well (with huge core/thread count systems (32+), you'd probably be OK with 800MB or even less per core as you don't often get many high RAM requirement Tasks (2GB+) at any given time).



I've written it off to a cost of doing business, but if there is a fix for it I wouldn't mind at all.
Just adding more RAM won't necessarily fix it- you need to allow BOINC to actually use what is there.
In your Account settings, Computing preferences, Memory
    When computer is in use, use at most 95 %
When computer is not in use, use at most 95 %
Leave non-GPU tasks in memory while suspended
Make sure Leave non-GPU.. is Not selected.
Works for me.
Grant
Darwin NT
ID: 100831 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2140
Credit: 41,518,559
RAC: 10,612
Message 100836 - Posted: 25 Mar 2021, 18:39:20 UTC

FWIW

Scheduler down
Project down for maintenance

Just got a 1hr delay after an update attempt
ID: 100836 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100837 - Posted: 26 Mar 2021, 3:00:37 UTC - in response to Message 100828.  

Several of your machines have only 8 GB memory. This isn't enough for 12 cores.

You need at least 1 GB/core to run Rosetta


I had noticed that the 12 core host was lagging, so I have ordered another 8gb stick for that one. That's the only 12 core machine. There's one other machine with 4 cores and 4gb (nominally) of memory, but that one can't be expanded; just have to live with it as it is.
ID: 100837 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100838 - Posted: 26 Mar 2021, 3:04:17 UTC - in response to Message 100831.  

Make sure Leave non-GPU.. is Not selected.
Works for me.


Heh. I had it on Not Selected, then Selected per the advice of someone else in another thread, now it is back to Not Selected again. Let's see if that helps.
ID: 100838 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,214,047
RAC: 1,450
Message 100843 - Posted: 26 Mar 2021, 13:24:43 UTC - in response to Message 100837.  

Several of your machines have only 8 GB memory. This isn't enough for 12 cores.

You need at least 1 GB/core to run Rosetta


I had noticed that the 12 core host was lagging, so I have ordered another 8gb stick for that one. That's the only 12 core machine. There's one other machine with 4 cores and 4gb (nominally) of memory, but that one can't be expanded; just have to live with it as it is.


There are lots of other Boinc Projects that have much smaller ram requirements and would love your help too!!
ID: 100843 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100845 - Posted: 26 Mar 2021, 23:09:57 UTC - in response to Message 100843.  


There are lots of other Boinc Projects that have much smaller ram requirements and would love your help too!!


I'm sure that all of them would love my help, cuzz. But if there's one thing that you learn as an American living in the Philippines, it's that there is always a lot more love of help than there is of you.
ID: 100845 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,214,047
RAC: 1,450
Message 100846 - Posted: 26 Mar 2021, 23:33:29 UTC - in response to Message 100845.  


There are lots of other Boinc Projects that have much smaller ram requirements and would love your help too!!


I'm sure that all of them would love my help, cuzz. But if there's one thing that you learn as an American living in the Philippines, it's that there is always a lot more love of help than there is of you.


I totally agree with that statement!! My though was about you being able to use all of your cpu cores at another project instead of just one or two here at Rosetta.
ID: 100846 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100850 - Posted: 27 Mar 2021, 23:13:58 UTC - in response to Message 100846.  


My though was about you being able to use all of your cpu cores at another project instead of just one or two here at Rosetta.


Rosetta is a nice little hobby for me right now. I'm learning new things and the subject matter is congruent with my son's PhD project, so there's that. At some point I suppose those benefits will plateau and, when it does, I will look for new roses to smell. Is there one project that stands out for you as a recommendation?
ID: 100850 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 100851 - Posted: 28 Mar 2021, 0:37:55 UTC - in response to Message 100850.  

Is there one project that stands out for you as a recommendation?

The new trRosetta should be even better.
ID: 100851 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 82 · 83 · 84 · 85 · 86 · 87 · 88 . . . 309 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org