Extreme RAM usage

Message boards : Number crunching : Extreme RAM usage

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Gabriel

Send message
Joined: 9 Aug 07
Posts: 1
Credit: 81,093
RAC: 0
Message 64105 - Posted: 20 Nov 2009, 14:36:02 UTC

Dear Rosetta

Is it really necessarily for the application to use more than 300MB of RAM for each unit?
I'm running this on a weak computer (my netbook when it's hooked up to some juice) and sometimes I can only run one unit at a time, due to it requesting more RAM.

Since most netbooks only sport 1GB of RAM it's really a lot that it use up more than 1/3.

What I'm asking, isn't it possible to optimize the code/change something, so it doesn't require those amounts of RAM?
ID: 64105 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 64107 - Posted: 20 Nov 2009, 16:36:50 UTC

Optimization both of runtime and memory usage is sort of a continuous cycle with Rosetta@home. Over time, the program has more and more protocols added to it. This allows it to make good predictions on more types of proteins and handle a greater variety of situations.

The typical pattern is that a new protocol is devised and studied. And then if it proves to do the job well, then it is refined with time to run more efficiently.

Having said that, the project increased the recommended minimum amount of memory to 512MB earlier this year. As you point out, your machine has twice that, but you also have two CPUs. So you are essentially at the minimum size recommended. They made this increase because they felt they had reached a point where they would not be able to continue improving the program and live within the smaller memory space. It was not a decision taken lightly.

Your contribution is valued and it would certainly be desirable for many reasons to run in a smaller memory footprint. However, they just hasn't been possible.

It is starting to sound like some of the recent proteins being run are consuming more memory then has been typical for the past few months. So it is entirely likely this is a temporary situation that will return to normal when new tasks are released in the future.

Otherwise, one option for you, since your machine ends up only running one Rosetta task anyway, would be to find another project that runs with less memory and establish an equal resource share between the two. Over time, this should typically end up running one task from each project, a high memory task from Rosetta and a low memory task from the other project, thus keeping both CPUs contributing to research. The BOINC scheduler isn't perfect and it doesn't think of a 50/50 resource share as meaning one of each should run all the time, but it should get you pretty close over the course of a few days.
Rosetta Moderator: Mod.Sense
ID: 64107 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,694,507
RAC: 0
Message 64111 - Posted: 21 Nov 2009, 2:28:18 UTC

I like watching the 4GB of RAM I paid for getting used for something useful :)
ID: 64111 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1894
Credit: 8,767,285
RAC: 10,641
Message 64113 - Posted: 21 Nov 2009, 10:53:58 UTC - in response to Message 64111.  

I like watching the 4GB of RAM I paid for getting used for something useful :)


Use Vista or Win7 then, they automatically use or release ram as it needs it or other programs need it.
ID: 64113 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,694,507
RAC: 0
Message 64122 - Posted: 21 Nov 2009, 20:55:14 UTC - in response to Message 64113.  

I like watching the 4GB of RAM I paid for getting used for something useful :)


Use Vista or Win7 then, they automatically use or release ram as it needs it or other programs need it.


I do.
ID: 64122 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1225
Credit: 13,853,229
RAC: 1,829
Message 64165 - Posted: 23 Nov 2009, 22:38:59 UTC - in response to Message 64122.  

I like watching the 4GB of RAM I paid for getting used for something useful :)


Use Vista or Win7 then, they automatically use or release ram as it needs it or other programs need it.


I do.


So the minirosetta problem with using so much RAM that non-BOINC programs run very slowly, and NOT turning it loose when needed, has been fixed? I have more RAM available on my machine, but some time ago had to limit BOINC to using 40% of the RAM to allow non-BOINC programs to run well. I use 64-bit Vista SP2. Minirosetta WAS running with a memory leak that apparantly made it use significantly more RAM than Windows Task Manager would report.
ID: 64165 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1894
Credit: 8,767,285
RAC: 10,641
Message 64170 - Posted: 24 Nov 2009, 10:16:27 UTC - in response to Message 64165.  
Last modified: 24 Nov 2009, 10:16:38 UTC

I like watching the 4GB of RAM I paid for getting used for something useful :)


Use Vista or Win7 then, they automatically use or release ram as it needs it or other programs need it.


I do.


So the minirosetta problem with using so much RAM that non-BOINC programs run very slowly, and NOT turning it loose when needed, has been fixed? I have more RAM available on my machine, but some time ago had to limit BOINC to using 40% of the RAM to allow non-BOINC programs to run well. I use 64-bit Vista SP2. Minirosetta WAS running with a memory leak that apparantly made it use significantly more RAM than Windows Task Manager would report.


That I can't answer, Rosetta will not run on a Windows machine for me at all. I can run it just fine on a Linux machine but for some unknown reason Rosetta will just not work on any of my Windows pc's. And believe me we have all tried to make it work, it just will not.
ID: 64170 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1225
Credit: 13,853,229
RAC: 1,829
Message 64246 - Posted: 27 Nov 2009, 17:12:56 UTC - in response to Message 64170.  
Last modified: 27 Nov 2009, 17:25:57 UTC

I like watching the 4GB of RAM I paid for getting used for something useful :)


Use Vista or Win7 then, they automatically use or release ram as it needs it or other programs need it.


I do.


So the minirosetta problem with using so much RAM that non-BOINC programs run very slowly, and NOT turning it loose when needed, has been fixed? I have more RAM available on my machine, but some time ago had to limit BOINC to using 40% of the RAM to allow non-BOINC programs to run well. I use 64-bit Vista SP2. Minirosetta WAS running with a memory leak that apparantly made it use significantly more RAM than Windows Task Manager would report.


That I can't answer, Rosetta will not run on a Windows machine for me at all. I can run it just fine on a Linux machine but for some unknown reason Rosetta will just not work on any of my Windows pc's. And believe me we have all tried to make it work, it just will not.


Which version of Windows? 32-bit or 64-bit? Which version of BOINC? How much memory? How many CPU cores? I've been able to get rosetta or minirosetta to run on two of my Windows machines for some time (haven't tried it on the third one) but have stopped running it on the one with the least memory.
ID: 64246 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1225
Credit: 13,853,229
RAC: 1,829
Message 64247 - Posted: 27 Nov 2009, 17:15:28 UTC - in response to Message 64170.  
Last modified: 27 Nov 2009, 17:16:05 UTC

(duplicate)
ID: 64247 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1894
Credit: 8,767,285
RAC: 10,641
Message 64252 - Posted: 28 Nov 2009, 12:33:23 UTC - in response to Message 64246.  

I like watching the 4GB of RAM I paid for getting used for something useful :)


Use Vista or Win7 then, they automatically use or release ram as it needs it or other programs need it.


I do.


So the minirosetta problem with using so much RAM that non-BOINC programs run very slowly, and NOT turning it loose when needed, has been fixed? I have more RAM available on my machine, but some time ago had to limit BOINC to using 40% of the RAM to allow non-BOINC programs to run well. I use 64-bit Vista SP2. Minirosetta WAS running with a memory leak that apparantly made it use significantly more RAM than Windows Task Manager would report.


That I can't answer, Rosetta will not run on a Windows machine for me at all. I can run it just fine on a Linux machine but for some unknown reason Rosetta will just not work on any of my Windows pc's. And believe me we have all tried to make it work, it just will not.


Which version of Windows? 32-bit or 64-bit? Which version of BOINC? How much memory? How many CPU cores? I've been able to get rosetta or minirosetta to run on two of my Windows machines for some time (haven't tried it on the third one) but have stopped running it on the one with the least memory.


32 bit dual core, even quad core once, single core too. Always Windows XP Pro and at least 1 gig of ram. The problem is that the files just won't download to my machine, the Admins even tried to help, all other Boinc projects I have tried work just fine, Rosie just won't. As I said I even put up a Linux machine and it worked just fine so I now the files can come thru they just won't. No it is not a firewall issue, no it isn't a file issue, we even tried zipping them and even that didn't work. Many MANY different version of Rosie over the years, no luck with Windows for me!! Anything you can think of we tried and tried again. I even took a laptop to McDonalds and it didn't work!! No I don't do anything special or weird, it just doesn't like me. Ubuntu Linux works just fine and I may put up another machine or two and crunch that way again one day, I don't know right now. Most of my machines are crunchers only so one OS over another is fairly immaterial. Right now I am happy where I am, Malaria but the gpu's are on Collatz. Every once in a while I think maybe I will do Rosie again, but not right now. I am swamped at work for the next 3 months or so and am kind of just maintaining at home. Small things get done like adding a hard drive to my server but new projects are just on hold.
ID: 64252 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,694,507
RAC: 0
Message 64263 - Posted: 28 Nov 2009, 17:19:08 UTC - in response to Message 64252.  

I like watching the 4GB of RAM I paid for getting used for something useful :)


Use Vista or Win7 then, they automatically use or release ram as it needs it or other programs need it.


I do.


So the minirosetta problem with using so much RAM that non-BOINC programs run very slowly, and NOT turning it loose when needed, has been fixed? I have more RAM available on my machine, but some time ago had to limit BOINC to using 40% of the RAM to allow non-BOINC programs to run well. I use 64-bit Vista SP2. Minirosetta WAS running with a memory leak that apparantly made it use significantly more RAM than Windows Task Manager would report.


That I can't answer, Rosetta will not run on a Windows machine for me at all. I can run it just fine on a Linux machine but for some unknown reason Rosetta will just not work on any of my Windows pc's. And believe me we have all tried to make it work, it just will not.


Which version of Windows? 32-bit or 64-bit? Which version of BOINC? How much memory? How many CPU cores? I've been able to get rosetta or minirosetta to run on two of my Windows machines for some time (haven't tried it on the third one) but have stopped running it on the one with the least memory.


32 bit dual core, even quad core once, single core too. Always Windows XP Pro and at least 1 gig of ram. The problem is that the files just won't download to my machine, the Admins even tried to help, all other Boinc projects I have tried work just fine, Rosie just won't. As I said I even put up a Linux machine and it worked just fine so I now the files can come thru they just won't. No it is not a firewall issue, no it isn't a file issue, we even tried zipping them and even that didn't work. Many MANY different version of Rosie over the years, no luck with Windows for me!! Anything you can think of we tried and tried again. I even took a laptop to McDonalds and it didn't work!! No I don't do anything special or weird, it just doesn't like me. Ubuntu Linux works just fine and I may put up another machine or two and crunch that way again one day, I don't know right now. Most of my machines are crunchers only so one OS over another is fairly immaterial. Right now I am happy where I am, Malaria but the gpu's are on Collatz. Every once in a while I think maybe I will do Rosie again, but not right now. I am swamped at work for the next 3 months or so and am kind of just maintaining at home. Small things get done like adding a hard drive to my server but new projects are just on hold.


That's weird?
Could it be an antivirus problem? Have clean-installed an XP machine with nothing on it but the drivers and tried running rosie?
ID: 64263 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Paul

Send message
Joined: 29 Oct 05
Posts: 193
Credit: 65,754,624
RAC: 1,594
Message 64269 - Posted: 28 Nov 2009, 23:32:29 UTC - in response to Message 64263.  

What exactly happens when you try to install or run? Does the installer fail? I guess I don't understand where things break down. After installing BOINC on about 25 XP systems, the thought goes to something like AV or group policy issue with Win XP in your environment.

More information on the failure process would help.

thx
Thx!

Paul

ID: 64269 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1894
Credit: 8,767,285
RAC: 10,641
Message 64279 - Posted: 29 Nov 2009, 13:44:53 UTC - in response to Message 64269.  

What exactly happens when you try to install or run? Does the installer fail? I guess I don't understand where things break down. After installing BOINC on about 25 XP systems, the thought goes to something like AV or group policy issue with Win XP in your environment.

More information on the failure process would help.
thx


I am doing this from memory but here goes...I install XP as a brand new install, install Avast AV, malwarebytes.org software, then install Boinc. Boinc installs just fine and attaches to Rosetta too but during the downloading of the Rosetta stuff the largest files just stops near the end of the download and will not finish. As I said the Admin and I worked on it for a couple of weeks and could not solve it. I finally tried Ubuntu Linux and it worked just fine. I tried it on several machines and even did a plain XP install with nothing else and it still would not work. I am not going to try again anytime soon so this is a non-issue for me, at least right now.
ID: 64279 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Paul

Send message
Joined: 29 Oct 05
Posts: 193
Credit: 65,754,624
RAC: 1,594
Message 64283 - Posted: 29 Nov 2009, 17:19:03 UTC - in response to Message 64279.  

It sounds like AV but I don't get it. I run Avast on several systems without the issue.

Can you try it without Avast?

Thx!

Paul

ID: 64283 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 115,751,398
RAC: 58,222
Message 64287 - Posted: 29 Nov 2009, 23:19:15 UTC

I have Avast on around 20 machines running XP, Vista and 7 and all are fine, but I've never used malwarebytes software so I'd look at that as the most likely problem. If it isn't that then possibly permissions on the data folder, but I'd go for malwarebytes initially. I wouldn't even install avast on dedicated crunchers though (or i might install it but I wouldn't leave it running!).
ID: 64287 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nickster

Send message
Joined: 27 Nov 09
Posts: 4
Credit: 8,563
RAC: 0
Message 64289 - Posted: 29 Nov 2009, 23:50:58 UTC - in response to Message 64287.  

I have Avast on around 20 machines running XP, Vista and 7 and all are fine, but I've never used malwarebytes software so I'd look at that as the most likely problem. If it isn't that then possibly permissions on the data folder, but I'd go for malwarebytes initially. I wouldn't even install avast on dedicated crunchers though (or i might install it but I wouldn't leave it running!).


I agree - I had a lot problems with malwarebytes. I use avast home with Boinc 6.10.18 (Vista SP1) for 4 different Boinc projects and have never had an issue...
ID: 64289 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1894
Credit: 8,767,285
RAC: 10,641
Message 64301 - Posted: 30 Nov 2009, 10:19:42 UTC - in response to Message 64283.  

It sounds like AV but I don't get it. I run Avast on several systems without the issue.

Can you try it without Avast?


My next to last sentence "I tried it on several machines and even did a plain XP install with nothing else and it still would not work."

All you guys can quit trying to get me to try anything right now, I am happy and have some goals I am striving for where I am, I am not switching right now. My last sentence says "I am not going to try again anytime soon so this is a non-issue for me, at least right now."

I do appreciate all your help but as I have said I have been thru all this before, it just doesn't work for me. I have been working with pc's for over 30 years, I fix pc's as a hobby. I am NOT a programmer or anything like that, just a hobbyist but the Admins here and I have tried LITERALLY EVERYTHING and it just doesn't work! They even emailed me the files and it still won't work for me. I have always thought it was a Comcast thing but can't prove that, I do know others have Comcast too and have no problems. I have become a Linux user as a result, although I currently have no Linux pc's up and running. I have even become a guy that sets up pc's at work for the kids that are trying to be adopted, putting Linux on all of the pc's that have been donated for them.
ID: 64301 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Paul

Send message
Joined: 29 Oct 05
Posts: 193
Credit: 65,754,624
RAC: 1,594
Message 64315 - Posted: 1 Dec 2009, 2:34:34 UTC - in response to Message 64301.  

Thanks for doing your part. All of us have run into weird issues and we all hate it when things don't work the way we expect them to work.

I will stop "helping" for now.

If and when you are ready to try again, let us know. You have a whole community of folks willing to share past experiences with you.

My Linux system runs great and I LOVE it. That system never requires a reboot, never reboots on its own, never has any issues.

thanks for crunching.
Thx!

Paul

ID: 64315 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 64317 - Posted: 1 Dec 2009, 6:41:31 UTC

.
Rosetta Moderator: Mod.Sense
ID: 64317 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 64318 - Posted: 1 Dec 2009, 6:41:47 UTC

.
Rosetta Moderator: Mod.Sense
ID: 64318 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Number crunching : Extreme RAM usage



©2024 University of Washington
https://www.bakerlab.org