Rosetta Just swamp my computer

Message boards : Number crunching : Rosetta Just swamp my computer

To post messages, you must log in.

AuthorMessage
hangint3n

Send message
Joined: 23 Mar 20
Posts: 8
Credit: 1,958,078
RAC: 0
Message 98843 - Posted: 8 Sep 2020, 0:47:02 UTC

Rosetta just swamped the memory and swap of my computer, thus causing it to freeze up? I'm running a AMD64 Ryzen 5 1800X with 32Gb of ram. Anyone else run into this?


===
hangint3n
ID: 98843 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Keith Myers
Avatar

Send message
Joined: 29 Mar 20
Posts: 95
Credit: 289,903
RAC: 0
Message 98844 - Posted: 8 Sep 2020, 0:55:00 UTC - in response to Message 98843.  

Rosetta just swamped the memory and swap of my computer, thus causing it to freeze up? I'm running a AMD64 Ryzen 5 1800X with 32Gb of ram. Anyone else run into this?


===
hangint3n

Rosetta has dodgy tasks quite often. I just ran into one that grabbed all available memory and swap every ten minutes and thus freezing the computer for a minute until the task released the memory.
Well it was this Rosetta task kp8RjDVk_fold_and_dock_SAVE_ALL_OUT_1009390_201. It is grabbing all the memory and the swap file every five minutes or so.
ID: 98844 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
hangint3n

Send message
Joined: 23 Mar 20
Posts: 8
Credit: 1,958,078
RAC: 0
Message 98848 - Posted: 8 Sep 2020, 1:01:16 UTC - in response to Message 98844.  

Rosetta just swamped the memory and swap of my computer, thus causing it to freeze up? I'm running a AMD64 Ryzen 5 1800X with 32Gb of ram. Anyone else run into this?


===
hangint3n

Rosetta has dodgy tasks quite often. I just ran into one that grabbed all available memory and swap every ten minutes and thus freezing the computer for a minute until the task released the memory.
Well it was this Rosetta task kp8RjDVk_fold_and_dock_SAVE_ALL_OUT_1009390_201. It is grabbing all the memory and the swap file every five minutes or so.


So if I do nothing it will release the memory? How long does this take? I'm using a Linux OS not Windows.
ID: 98848 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Keith Myers
Avatar

Send message
Joined: 29 Mar 20
Posts: 95
Credit: 289,903
RAC: 0
Message 98849 - Posted: 8 Sep 2020, 1:05:13 UTC - in response to Message 98848.  

Rosetta just swamped the memory and swap of my computer, thus causing it to freeze up? I'm running a AMD64 Ryzen 5 1800X with 32Gb of ram. Anyone else run into this?


===
hangint3n

Rosetta has dodgy tasks quite often. I just ran into one that grabbed all available memory and swap every ten minutes and thus freezing the computer for a minute until the task released the memory.
Well it was this Rosetta task kp8RjDVk_fold_and_dock_SAVE_ALL_OUT_1009390_201. It is grabbing all the memory and the swap file every five minutes or so.


So if I do nothing it will release the memory? How long does this take? I'm using a Linux OS not Windows.

It released the memory back to my static level after about a minute. Then ten minutes later it would repeat. Someone replied a similar post had seen the same issue. So I decided the task was bad and aborted it.
Using Ubuntu 20.04.1 OS.
https://boinc.bakerlab.org/rosetta/forum_thread.php?id=12554&postid=98814
ID: 98849 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Keith Myers
Avatar

Send message
Joined: 29 Mar 20
Posts: 95
Credit: 289,903
RAC: 0
Message 98852 - Posted: 8 Sep 2020, 1:55:41 UTC

Well I just decided to stop Rosetta. During these wild memory excursions that stole all my memory and froze my system for a minute, I discovered that the GPUGrid tasks that were running at the same time errored out. And the error message matched what I had been seeing on the very rare occasion when a GPUGrid task failed. So cause and effect indicates that the Rosetta tasks running were the culprit. So have set NNT on Rosetta and will run without them for a while and see if I stop getting those random errors.
ID: 98852 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
MarkJ

Send message
Joined: 28 Mar 20
Posts: 72
Credit: 25,010,478
RAC: 27
Message 98875 - Posted: 8 Sep 2020, 10:10:01 UTC - in response to Message 98843.  
Last modified: 8 Sep 2020, 10:36:50 UTC

Rosetta just swamped the memory and swap of my computer, thus causing it to freeze up? I'm running a AMD64 Ryzen 5 1800X with 32Gb of ram. Anyone else run into this?


===
hangint3n

I spotted a bunch that said waiting for memory. When I checked the other tasks running's properties one said it was using 43GB. I have a few that have crashed as well. Machine hasn't yet. This is on a Ryzen 3900X with 64GB. Swap is up to about 1/3rd so its using that as well. In your BOINC settings check what you have in your memory settings for "in use" and "when not in use". Default values are 60% and 90%. I wouldn't suggest going over 90%.

The tasks were named fold_and_dock. I've aborted all of them and allowed the other tasks to run.
BOINC blog
ID: 98875 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Keith Myers
Avatar

Send message
Joined: 29 Mar 20
Posts: 95
Credit: 289,903
RAC: 0
Message 98901 - Posted: 8 Sep 2020, 23:08:55 UTC - in response to Message 98875.  

In your BOINC settings check what you have in your memory settings for "in use" and "when not in use". Default values are 60% and 90%. I wouldn't suggest going over 90%.

I have use 94% of the cpus and 90% of the memory when not in use. 85% when in use. 32 cores and 32GB of memory.

Since everyone is getting all these buggy tasks, I simply will not run Rosetta until it all gets sorted out. Then might come back or not. Plenty of work available for my other projects. TN-Grid and GPUGrid are doing Covid-19 or precursor work equivalent to what Rosetta does.
ID: 98901 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
MichaelD

Send message
Joined: 16 Dec 18
Posts: 1
Credit: 5,411,289
RAC: 0
Message 98924 - Posted: 9 Sep 2020, 8:40:03 UTC - in response to Message 98844.  

Yes, I'm seeing this issue too in the last couple of days. Some Rosetta WUs are very large in size? I have a number of tasks where the task properties say the 'working set size' is upto 39.76GB of memory (not disk) to run. So smaller PC systems would probably still download the WU and never run.

Not that I'm having any issues crunching this (with my 64GB system) but not as many WUs can run in parallel, and I did find that I had to re-tune my preferences since it consumed most of the working memory and the OS became unstable.

Regs,
Michael
ID: 98924 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
hangint3n

Send message
Joined: 23 Mar 20
Posts: 8
Credit: 1,958,078
RAC: 0
Message 98933 - Posted: 9 Sep 2020, 20:08:27 UTC

What I find annoying is that even though I've got limits set in my options settings. Rosetta still uses all my memory. If this continues to be the case I'll have to stop participating in the project. I can't have my desktop compromised and then having to hard reboot my system.
ID: 98933 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
dduggan47

Send message
Joined: 18 Sep 05
Posts: 12
Credit: 3,320,795
RAC: 1,991
Message 98952 - Posted: 11 Sep 2020, 1:19:44 UTC - in response to Message 98933.  

I can't cope with the memory problem either. Before I was sure what the culprit was my machine twice totally locked up for several minutes and I had to cold boot. I'll check back at some point but NNT for now. Sigh.
ID: 98952 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1484
Credit: 14,651,336
RAC: 13,496
Message 98953 - Posted: 11 Sep 2020, 1:27:37 UTC - in response to Message 98952.  

I can't cope with the memory problem either. Before I was sure what the culprit was my machine twice totally locked up for several minutes and I had to cold boot. I'll check back at some point but NNT for now. Sigh.
Most of those Tasks are long gone & very few (if any) are being re-issued. It's just your large cache that has resulted in it occurring now.
Grant
Darwin NT
ID: 98953 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dave

Send message
Joined: 15 Mar 20
Posts: 3
Credit: 2,610,174
RAC: 0
Message 98976 - Posted: 12 Sep 2020, 7:23:45 UTC - in response to Message 98843.  

yes
ID: 98976 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
James C. Owens

Send message
Joined: 7 Apr 18
Posts: 3
Credit: 22,546,943
RAC: 12,428
Message 99260 - Posted: 8 Oct 2020, 4:35:52 UTC

Seeing the same thing happen again now on three different Windows hosts. I set up a memory cgroup (systemd slice essentially) a while back on my main Linux box to enforce memory limits.

This is not good. The rosetta applications should enforce the memory limits and not present memory leaks that go over the set limits in BOINC.
ID: 99260 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Rosetta Just swamp my computer



©2024 University of Washington
https://www.bakerlab.org