Too much Virtual memory is being used

Message boards : Number crunching : Too much Virtual memory is being used

To post messages, you must log in.

AuthorMessage
Profile GreenBandit

Send message
Joined: 23 Dec 05
Posts: 5
Credit: 3,979,681
RAC: 0
Message 42401 - Posted: 21 Jun 2007, 12:27:13 UTC

One of my lo-spec machines that I run Rosetta on (while working) has recently (sometime in the last few weeks) started being extremely slow when task switching due to large amounts of VM being used by Rosetta applications.
Athlon 2400+ with 768mB RAM, fixed swap file of 1150MB. BOINC manager = 5.10.7 (same symptoms did happen with 5.8.16)
When the slow task switching occurs there are always a number of Rosetta applications that BOINC manager says are "Waiting for memory".
Task manager is currently reporting that there are a total of 3 Rosetta application left in memory even though I have set:-
applications to be removed from memory when suspended,
max page file to be used = 5%.
Each of the 3 Rosetta applications is using over 15% (i.e. over 150MB) of the swap file!!??
I have just increased the amount of memory to be used (idle & in use) from 10% to 50% but I still have one Rosetta application using 170MB of VM.
Can anyone help ?

ID: 42401 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 42405 - Posted: 21 Jun 2007, 13:38:26 UTC

I would be surprised to hear that you got any Rosetta work done using 10% of real memory on a 768MB machine. Most tasks take 110-140MB of real memory to run. The project minimum is a 256MB machine. And the thought is that Rosetta won't consume the entire machine.

Rosetta has two task groups. Some for machines <512MB of memory and others for machines with more. But I'm uncertain how sophisticated BOINC's decisions are as to whether the work fetch will look at the fraction that you allow BOINC to use. So, perhaps you have now been sent some tasks intended for a 512MB machine and they were trying to run in 10% of your memory. I am also uncertain if BOINC does scheduling on memory per CPU, or just overall.

Since it is a low spec machine, and it sounds like you are trying to reserve as much memory as possible for other activity, I would suggest that you set it up to only use one CPU (if it is not hyperthreaded or does not have more then one CPU this would be irrelevant), and to only run while the machine is not in use. And then it should be able to run well in the 50% of memory setting you have now. And you will probably just have to accept that it will be a bit sluggish at first as you sit down to use it for other things.

I see your machines are hidden. Could you explain more about the host in question? Is it Linux? or Windows? Are there other BOINC projects running on it as well?
Rosetta Moderator: Mod.Sense
ID: 42405 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile GreenBandit

Send message
Joined: 23 Dec 05
Posts: 5
Credit: 3,979,681
RAC: 0
Message 42432 - Posted: 22 Jun 2007, 9:18:20 UTC - in response to Message 42405.  
Last modified: 22 Jun 2007, 9:25:48 UTC

To answer the questions put to me:-
Rosetta & Seti are running, Windows 2000 SP4 is used, CPU is a socket A Athlon 2400+ running at the std 2000MHz, HT not supported.
I've unhidden the computers, so at least they can be seen now.

Regarding the memory being used, Real memory present=768MB, VM present = 1150MB(fixed), Rosetta 5.68 memory usage (as reported by Task Manager) = 121,572k of real memory (honest - it really is "over 10% of real memory on a 768MB machine") & 203,988MB of VM. The limits that I have are 50% of memory while in use (PC is normally in use during the day), and 5% of swap file. So it seems that the current swap file usage limit of 57.5MB is being very much exceeded.

I can put up with a small amount of slow task switching (I consider a small amount as up to 5 seconds), but some times a task switch has taken 20 seconds when there are 3 or so Rosetta tasks all in memory all using over 150MB of VM. On several occasions, even though I am only using M/S outlook and Word, the PC has given a "low resource error". I know I could get rid of this by allowing a variable size swap file, but I am trying to keep HDD fragmentation to a minimum.

This is what I don't understand, even though they have been set to be removed from memory when suspended, how is it that I had 3 tasks in memory all using more than the VM limit that I have set.

Today things are running OK (in terms of task switch times), but I still seem to have more VM resources being used than the limits I set and if I can't control the resources being used I am concerned that I may get other problems when I try to remotely access the PC from home.

Is is possible that the trigger for multiple Rosetta application all waiting for memory and all using >150MB of VM was triggered by me setting the allowed amout of real memory too low?
ID: 42432 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile anders n

Send message
Joined: 19 Sep 05
Posts: 403
Credit: 537,991
RAC: 0
Message 42433 - Posted: 22 Jun 2007, 9:57:58 UTC - in response to Message 42432.  


Is is possible that the trigger for multiple Rosetta application all waiting for memory and all using >150MB of VM was triggered by me setting the allowed amout of real memory too low?


Yes I would say that it is very possibel.

Have you thought about increasing your VM?

Anders n
ID: 42433 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile GreenBandit

Send message
Joined: 23 Dec 05
Posts: 5
Credit: 3,979,681
RAC: 0
Message 42437 - Posted: 22 Jun 2007, 11:56:47 UTC - in response to Message 42433.  
Last modified: 22 Jun 2007, 12:47:15 UTC


Is is possible that the trigger for multiple Rosetta application all waiting for memory and all using >150MB of VM was triggered by me setting the allowed amout of real memory too low?


Yes I would say that it is very possibel.

Have you thought about increasing your VM?

Anders n



I did think about this, but I am after both a solution to the specific problem on this machine AND a general solution to ensuring that BOINC does not resource-hog on my other machines. So while increasing the size of the VM would probably remove the Windows "....low on virtual memory..." error, it would not get round the problem of why 450MB out of a 1150MB swap file was being used by Rosetta when the limit that I put on VM usage was 10%.

If it really was "setting the allowed amout of real memory too low" that caused the hogging of VM by 3 instances of Rosetta then I will probably not get any more critical errors, but it still does not explain why almost 204MB of VM is being used by Rosetta when a 5% limit of a 1150MB swap file is only 57.5MB.
ID: 42437 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 42441 - Posted: 22 Jun 2007, 14:15:54 UTC - in response to Message 42432.  
Last modified: 22 Jun 2007, 14:26:50 UTC

Is is possible that the trigger for multiple Rosetta application all waiting for memory and all using >150MB of VM was triggered by me setting the allowed amout of real memory too low?


Just to clarify, no there is nothing on your end to do that would cause a task to require more memory. "trigger a task to wait for memory" yes. "Trigger a task to use >150MB" no, if it requires >150MB then it will wait until BOINC's use of memory is below that (at least if BOINC is working as it should).

BOINC is in control of the resources, not Rosetta. You might consider opening a thread on the BOINC boards.

It would seem you have at least two issues with how BOINC is allocating resources. One is that the tasks do not appear to be removed from memory while suspended. The other is that tasks seem to be able to exceed the limits you've set.

...just to review, these BOINC settings are viewed in your General Preferences. And the preferences are customizable for up to 4 different locations, and each host has a location associated with it which can been seen in the messages tab up at the top from when BOINC was starting up. The settings a given PC uses correspond to the location the PC is configured to reside at.

So I just wanted to review that, because if your machine thinks it is at "school" and you are setting preferences for "home", then the changes will appear to be ignored.
Rosetta Moderator: Mod.Sense
ID: 42441 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile GreenBandit

Send message
Joined: 23 Dec 05
Posts: 5
Credit: 3,979,681
RAC: 0
Message 42445 - Posted: 22 Jun 2007, 14:49:37 UTC
Last modified: 22 Jun 2007, 14:50:26 UTC

Unfortunately for me, the PC in question (368896) is picking up on the correct location, but it was a good check to make.

I think this is now closed as far as Rosetta is concerned as I will be posting the "not removed from memory while suspended" and "exceeding VM limits" issues on BOINC boards.

Mod.Sense & Anders n, thanks for the help     :-)
ID: 42445 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Too much Virtual memory is being used



©2024 University of Washington
https://www.bakerlab.org