Message boards : Number crunching : Larger Memory Models
Author | Message |
---|---|
Paul Send message Joined: 29 Oct 05 Posts: 193 Credit: 65,943,948 RAC: 4,154 ![]() |
Is there a way to request work units that require large memory models? I now have a few systems with more than 1GB RAM per core and would like to see that RAM put to good use. Currently the largest work unit I have is using about 650MB and most are using less than 400MB. Thx! Paul ![]() |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
Not at this time. But, with the new servers in place, that sort of change should now be more approachable than it was previously. Rosetta Moderator: Mod.Sense |
Paul Send message Joined: 29 Oct 05 Posts: 193 Credit: 65,943,948 RAC: 4,154 ![]() |
I looked at one of my machines this morning & it had at least 10 processes using more than 1GB of RAM each! I thought the project team just started throwing me larger processes. This is the fist time I have seen my machine using more than 32GB of RAM. I love to see better resource utilization. Thx! Paul ![]() |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
That was more of a coincidence. However, on occasion, there are tasks that require more memory, and rather than allocating them to a specific subgroup, the Project Team was able to precent them from going out to systems with low memory. With the new servers, they may be able to make the scheduler more proactive in assigning such tasks. Rosetta Moderator: Mod.Sense |
Sam Send message Joined: 27 Dec 10 Posts: 7 Credit: 8,904,341 RAC: 0 |
I would love to volunteer my machines to have the resource intensive work units =D Will this be possible on the new servers? |
![]() Volunteer moderator Project administrator Project developer Project scientist Send message Joined: 1 Jul 05 Posts: 1018 Credit: 4,334,829 RAC: 0 |
This is my fault. I was asked by someone in the lab to increase the length limit on Robetta, our publicly available structure prediction server, so I did but since realized the memory issue. I've reverted the length limit until we can figure out a good approach to handle them. Sorry. There may still be some jobs left over. These are medically relevant targets. |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
I caught these tasks using over 1GB of memory on Windows 7. pq170720_Sh29_1_DHR09_000603_0050_C_0001... D2D2F222xtal_AB_5_3gaf_3s55_0007_0001.pdb_unaD2D2F222_17_07_19_36_34_globalDocking_5_SAVE_ALL_OUT... Rosetta Moderator: Mod.Sense |
![]() Send message Joined: 1 Dec 05 Posts: 1958 Credit: 9,099,368 RAC: 978 ![]() |
It's a problem to introduce, in user profile's, a button with something like "Permit wus with high usage memory"?? Every user can decide or not to accept these wus. |
![]() Volunteer moderator Project administrator Project developer Project scientist Send message Joined: 1 Jul 05 Posts: 1018 Credit: 4,334,829 RAC: 0 |
On the list of things to do. great idea. I'll remind/alert the lab about the memory issues. thanks. |
Juha Send message Joined: 28 Mar 16 Posts: 13 Credit: 705,034 RAC: 0 |
There is no need to introduce a button for tasks with high memory usage. BOINC server already handles this out of the box. The server sends only such tasks to a host that have rsc_memory_bound less than the memory limit set in host's/user's computing preferences. If user doesn't want memory filled with tasks all he or she needs to do is lower the memory limit in computing preferences. That does mean that the project needs to fill rsc_memory_bound accurately. The few tasks that I sampled had rsc_memory_bound=1e8 bytes but actual memory usage was 300-400 MB. Naughty. |
![]() ![]() Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
|
Trotador Send message Joined: 30 May 09 Posts: 108 Credit: 291,214,977 RAC: 3,765 ![]() |
The amount of memory used by each WU type varies widely as it depends on the simulation programmed by the scientific that created it. Now, I'm seeing up to 1.3 Gb in many "yh170802_" units, close to 1GB in "JWdesRMf_closed_" units and around 200Mb in "A211_" ones |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
Now, I'm seeing up to 1.3 Gb in many "yh170802_" units, Interesting. I never bothered to look before. But BoincTasks shows a "yh17802" using 1329 MB of real memory, and 1508 MB of virtual memory. I suppose the "real memory" is the one to pay attention to for DRAM purposes. Fortunately, with 32 GB of main memory, it has not been an issue. |
Trotador Send message Joined: 30 May 09 Posts: 108 Credit: 291,214,977 RAC: 3,765 ![]() |
New king :) "JWC6-62dock-new..." up to 1,5Gb RAM occupation |
![]() ![]() Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
|
yhsia Volunteer moderator Project developer Project scientist Send message Joined: 21 May 16 Posts: 2 Credit: 97,172 RAC: 0 |
Hi, I'm the culprit behind the "yh..." jobs! I apologize if it's causing any issues, I have set rsc_memory_bound to be 2gb, but I will set it higher in the future to make sure I don't top out anyone's computers. These jobs are larger symmetrical proteins, we are testing a computational method to only "fold" a portion of the larger structure. Even though we only modify a portion, there are just so many atoms there that it seems to eat up a lot of memory. Thanks for your precious resources though! Please do let us/me know if anything we do is causing problems. |
LarryMajor Send message Joined: 1 Apr 16 Posts: 22 Credit: 31,533,212 RAC: 0 |
As others have mentioned, some of us have enough memory to run 2Gb jobs, and will gladly do it. If it works better for you guys, it might be worth designing a way for us to request such jobs. |
![]() ![]() Send message Joined: 16 Oct 05 Posts: 711 Credit: 26,694,507 RAC: 0 |
|
![]() Send message Joined: 1 Dec 05 Posts: 1958 Credit: 9,099,368 RAC: 978 ![]() |
There is no need to introduce a button for tasks with high memory usage. BOINC server already handles this out of the box. The server sends only such tasks to a host that have rsc_memory_bound less than the memory limit set in host's/user's computing preferences. If user doesn't want memory filled with tasks all he or she needs to do is lower the memory limit in computing preferences. I continue to prefer the idea of button in profile's page... |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
There is no need to introduce a button for tasks with high memory usage. BOINC server already handles this out of the box. The server sends only such tasks to a host that have rsc_memory_bound less than the memory limit set in host's/user's computing preferences. If user doesn't want memory filled with tasks all he or she needs to do is lower the memory limit in computing preferences. I think the profile page is a much better place for it too. Lowering the memory limit in computing preferences is a very clumsy way to do it, and most people would never think of it. Furthermore, it would apply to all projects you run, not just Rosetta. I am all for using more memory too if some group of tasks can be run more efficiently that way. 3 GB would work for me. And I could do more if necessary by using an app_config.xml to limit the number run concurrently. |
Message boards :
Number crunching :
Larger Memory Models
©2024 University of Washington
https://www.bakerlab.org