Huge RAM usage by some of latest WUs

Message boards : Number crunching : Huge RAM usage by some of latest WUs

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Fairbrother Family

Send message
Joined: 8 May 17
Posts: 1
Credit: 158,773
RAC: 0
Message 91788 - Posted: 26 Feb 2020, 7:36:44 UTC - in response to Message 91696.  

These are likely jobs that are modeling the Spike complex (http://new.robetta.org/results.php?id=15652) of 2019-nCoV_S, the corona virus. The genome has been sequenced and there is a mad rush to determine structures for possible drug targets.

We are collaborating with a number of different research groups to model corona virus proteins that may be possible drug targets, including the NIH/NAIAD and SSGCID https://www.ssgcid.org/.



we really really must get the word out about Rosetta and the BOINC program
average citizens can actually make a big difference in the cause of curing the corona virus and other viruses
this should be on international news!
ID: 91788 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mad_Max

Send message
Joined: 31 Dec 09
Posts: 207
Credit: 23,369,892
RAC: 10,860
Message 91815 - Posted: 1 Mar 2020, 3:47:25 UTC - in response to Message 91760.  
Last modified: 1 Mar 2020, 3:51:37 UTC


Isn't the Internet bandwidth the same? With multi-threaded you run fewer work units at a time, but you download/upload correspondingly more often.
No, with R@H it works other way - almost all WUs here runs exactly same time (8 hours by current default) regardless of how powerful CPU you are using. E.g. if you replace CPU by other model which is 2 times faster(per each core basis) you do not complete 2x quantity of R@H WU. Instead each WU on faster CPU will be running about same time as on slower (still ~8 hours) but will produce more useful results (they called models or decoys - variants of possible protein configuration) from the same input data files downloaded from server.
Same apply for multi-threaded app if it will be ever available for R@H: increasing number of threads per WU will not reduce runtime of one WU, it just "squeeze out" more scientifically useful results from same starting WU data. So using MT app will reduce total number of WUs downloaded, stored on disk and loaded into the RAM few folds (based on how many threads each WU is using).

I think the only real saving is memory. Most multi-threaded projects now allow you to select how many threads (cores) you want to use on a single work unit. I usually select "1" or "2", since that is usually more efficient. Most MT projects run less efficiently the more threads you use. I am not sure why that is the case, but it is said that on some of them, one thread may finish early before the others, and have nothing to do. There may be other reasons.

I usually have plenty of memory, though having a choice is nice. But I expect that not all tasks are suitable for MT.
Yes from CPU side a lot of single threaded WU running in parallel independently usually is most effective variant, MT apps usually a slightly less effective CPU wise. But swarms of single threaded tasks is a waste of all other resources - RAM, bandwidth and both disk space and disk usage. As each R@H WU unpack own copy of main Rosetta database to work with which is about ~ 500 MB and ~4000 files plus 400 folder per each running WU ( and it keeps growing as more data added to DB) - you can find it in "minirosetta_database" subfolder of each BOINC "slot" folder occupied by R@H WU.

Yes, MT app may be not optimal for simple and "small" tasks but can be very useful for huge tasks like this "corona virus" simulations described above which needs a lot of RAM and CPU power per each WU.
ID: 91815 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2

Message boards : Number crunching : Huge RAM usage by some of latest WUs



©2024 University of Washington
https://www.bakerlab.org