Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 139 · 140 · 141 · 142 · 143 · 144 · 145 . . . 276 · Next

AuthorMessage
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5662
Credit: 5,701,869
RAC: 2,154
Message 103613 - Posted: 30 Nov 2021, 14:47:25 UTC - in response to Message 103612.  
Last modified: 30 Nov 2021, 14:50:22 UTC

Falconet has the right answer. But I would only add that it is the project that sets the memory requirements, not BOINC.
If they say more, then BOINC just obeys. (The memory isn't released, it is just reserved.)

And adding a second BOINC instance gives you another bite at the apple. One BOINC instance doesn't see what the other one is doing.
So if the pythons ever do require more memory to run, that could cause problems. But we are a long way from that at the moment.




I thought there was a memory limiter command?

I have stopped with Python as it is killing LHC ATLAS from running.

How do you set up another instance? This is really new territory for me.
ID: 103613 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103614 - Posted: 30 Nov 2021, 17:13:20 UTC - in response to Message 103613.  

I thought there was a memory limiter command?

I have stopped with Python as it is killing LHC ATLAS from running.

How do you set up another instance? This is really new territory for me.

There are memory limits set by the project (probably both maximum and minimum). You might be able to change them, but I have not looked into it.

Yes, I have two BOINC instances on three Ubuntu machines. I use them as necessary, not just for Rosetta.
The main difference for Windows is that you have to start up the second instance manually to make the icon visible, though I did manage to get it to work via Task Scheduler.
But that was on Win7. I have not tried it on Win10, and it may be different.
ID: 103614 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Thomas Köller

Send message
Joined: 27 Nov 21
Posts: 1
Credit: 293,470
RAC: 0
Message 103615 - Posted: 30 Nov 2021, 17:39:09 UTC

On the computing preferences page, I cannot remove the check mark next to 'Suspend GPU computing when computer is in use'. Or, to be more precise, I can but it reappears as soon as I save my changes. Why is that?
ID: 103615 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103616 - Posted: 30 Nov 2021, 18:00:01 UTC - in response to Message 103615.  

On the computing preferences page, I cannot remove the check mark next to 'Suspend GPU computing when computer is in use'. Or, to be more precise, I can but it reappears as soon as I save my changes. Why is that?
That is a bit curious. I do not see it here. Maybe a blocker of some sort on your web browser? I would try a different one.
ID: 103616 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Humphrey

Send message
Joined: 26 Jul 18
Posts: 5
Credit: 3,123,818
RAC: 10,313
Message 103617 - Posted: 30 Nov 2021, 18:06:14 UTC - in response to Message 103611.  

Read my post. I hope it helps.

https://boinc.bakerlab.org/rosetta/forum_thread.php?id=6893&postid=103572


It does - thanks. Isn't it usual for a program to reserve memory only when it's needed?
ID: 103617 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5662
Credit: 5,701,869
RAC: 2,154
Message 103618 - Posted: 30 Nov 2021, 18:23:05 UTC - in response to Message 103614.  

I thought there was a memory limiter command?

I have stopped with Python as it is killing LHC ATLAS from running.

How do you set up another instance? This is really new territory for me.

There are memory limits set by the project (probably both maximum and minimum). You might be able to change them, but I have not looked into it.

Yes, I have two BOINC instances on three Ubuntu machines. I use them as necessary, not just for Rosetta.
The main difference for Windows is that you have to start up the second instance manually to make the icon visible, though I did manage to get it to work via Task Scheduler.
But that was on Win7. I have not tried it on Win10, and it may be different.



Is there a dummies page with a simple explantion to set this up?
ID: 103618 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103620 - Posted: 30 Nov 2021, 18:35:42 UTC - in response to Message 103618.  

Is there a dummies page with a simple explantion to set this up?

With BOINC, it only gets worse.
https://boinc.berkeley.edu/wiki/Client_configuration#Options
ID: 103620 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 172
Credit: 5,657,330
RAC: 3,362
Message 103623 - Posted: 30 Nov 2021, 19:14:35 UTC - in response to Message 103597.  

project? I have never seen that before.
Most of us tried to use max_concurrent and then got buried in tons of tasks we could never complete by their deadlines.


Project_max_current will limit the total number of work units running for all projects.

But either one of them can cause the problem of excessive downloads.


Well, I use only the <project_max_concurrent> not <max_concurrent>,
and then only in the app_config.xml file in the

/var/lib/boinc/projects/boinc.bakerlab.org_rosetta directory.

[/var/lib/boinc/projects/boinc.bakerlab.org_rosetta]$ cat app_config.xml 
<app_config>
   <project_max_concurrent>3</project_max_concurrent>
</app_config>


I use similar app_config.xml files in the project directories for my other projects as well (with different limits).
localhost:jeandavid8[/var/lib/boinc/projects]$ ls -l

 16384 Nov 30 06:20 boinc.bakerlab.org_rosetta
 12288 Nov 29 16:53 climateprediction.net
 24576 Nov 29 23:13 universeathome.pl_universe
 40960 Nov 30 13:15 www.worldcommunitygrid.org

I am running
boinc-client-7.16.11-3.el8.x86_64
that is the most up-to-date one for this machine and OS. The OS is up-to date as well.
Computer 5910575

CPU type 	GenuineIntel
Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz [Family 6 Model 85 Stepping 7]
Number of processors 	16
Operating System 	Linux Red Hat Enterprise Linux
Red Hat Enterprise Linux 8.5 (Ootpa) [4.18.0-348.2.1.el8_5.x86_64|libc 2.28 (GNU libc)]
BOINC version 	7.16.11
Memory 	         63902.16 MB
Cache 	            16896 KB
Swap space 	    15992 MB
Total disk space   117.21 GB
Free Disk Space     92.03 GB

ID: 103623 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Falconet

Send message
Joined: 9 Mar 09
Posts: 350
Credit: 1,017,068
RAC: 357
Message 103624 - Posted: 30 Nov 2021, 19:20:00 UTC

Pythons are back and I received 5 on my 8 GB laptop which previously received none.
ID: 103624 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5662
Credit: 5,701,869
RAC: 2,154
Message 103626 - Posted: 30 Nov 2021, 19:43:39 UTC - in response to Message 103620.  

Is there a dummies page with a simple explantion to set this up?

With BOINC, it only gets worse.
https://boinc.berkeley.edu/wiki/Client_configuration#Options


I've seen something else elsewhere. I'll look that up this weekend.
ID: 103626 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5662
Credit: 5,701,869
RAC: 2,154
Message 103627 - Posted: 30 Nov 2021, 19:45:55 UTC - in response to Message 103623.  

project? I have never seen that before.
Most of us tried to use max_concurrent and then got buried in tons of tasks we could never complete by their deadlines.


Project_max_current will limit the total number of work units running for all projects.

But either one of them can cause the problem of excessive downloads.


Well, I use only the <project_max_concurrent> not <max_concurrent>,
and then only in the app_config.xml file in the

/var/lib/boinc/projects/boinc.bakerlab.org_rosetta directory.

[/var/lib/boinc/projects/boinc.bakerlab.org_rosetta]$ cat app_config.xml 
<app_config>
   <project_max_concurrent>3</project_max_concurrent>
</app_config>


I use similar app_config.xml files in the project directories for my other projects as well (with different limits).
localhost:jeandavid8[/var/lib/boinc/projects]$ ls -l

 16384 Nov 30 06:20 boinc.bakerlab.org_rosetta
 12288 Nov 29 16:53 climateprediction.net
 24576 Nov 29 23:13 universeathome.pl_universe
 40960 Nov 30 13:15 www.worldcommunitygrid.org

I am running
boinc-client-7.16.11-3.el8.x86_64
that is the most up-to-date one for this machine and OS. The OS is up-to date as well.
Computer 5910575

CPU type 	GenuineIntel
Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz [Family 6 Model 85 Stepping 7]
Number of processors 	16
Operating System 	Linux Red Hat Enterprise Linux
Red Hat Enterprise Linux 8.5 (Ootpa) [4.18.0-348.2.1.el8_5.x86_64|libc 2.28 (GNU libc)]
BOINC version 	7.16.11
Memory 	         63902.16 MB
Cache 	            16896 KB
Swap space 	    15992 MB
Total disk space   117.21 GB
Free Disk Space     92.03 GB



So? How's that working out on Python? That might be the solution to limit them.
ID: 103627 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Falconet

Send message
Joined: 9 Mar 09
Posts: 350
Credit: 1,017,068
RAC: 357
Message 103628 - Posted: 30 Nov 2021, 19:59:39 UTC - in response to Message 103624.  

Pythons are back and I received 5 on my 8 GB laptop which previously received none.


Looks like I can run 2 Pythons plus 3 MCM tasks. So 3 threads are idle.
ID: 103628 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 103629 - Posted: 30 Nov 2021, 20:03:20 UTC
Last modified: 30 Nov 2021, 20:15:04 UTC

PYTHON MINI ARE IN
names like . aagb-AGLY......
`task` - `properties` gives
working set size 2.79GB
virtual memory size 99MB
progressing @ 12.6% per hour
that's a lot better :)

edit
Hellow Falconet, we were typing at the same time :)
another edit
got some cosmology@hum docker tasks to finish off then find out how many Python mini fits into 32GB
yet another idiot, oops, I ment edit
front page news - Total queued jobs: 2,197,433
ID: 103629 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 103631 - Posted: 30 Nov 2021, 22:01:16 UTC
Last modified: 30 Nov 2021, 22:04:28 UTC

Had to reboot it coz it cant handle 15 , changed `use cpu` settings and got 11 running so far
but its thrashing the disk when it starts them
only 20GB memory in use
100GB disk space in use by rosetta
ID: 103631 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 172
Credit: 5,657,330
RAC: 3,362
Message 103632 - Posted: 30 Nov 2021, 22:52:16 UTC - in response to Message 103627.  

Well, I use only the <project_max_concurrent> not <max_concurrent>,
and then only in the app_config.xml file in the

/var/lib/boinc/projects/boinc.bakerlab.org_rosetta directory.

[/var/lib/boinc/projects/boinc.bakerlab.org_rosetta]$ cat app_config.xml
<app_config>
<project_max_concurrent>3</project_max_concurrent>
</app_config>


So? How's that working out on Python? That might be the solution to limit them.

I have no idea.

Mon 29 Nov 2021 01:31:22 AM EST | Rosetta@home | Message from server: VirtualBox is not installed

I do no have VirtualBox, so I cannot run them.
ID: 103632 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5662
Credit: 5,701,869
RAC: 2,154
Message 103633 - Posted: 30 Nov 2021, 23:18:50 UTC - in response to Message 103632.  
Last modified: 30 Nov 2021, 23:20:32 UTC

Well, I use only the <project_max_concurrent> not <max_concurrent>,
and then only in the app_config.xml file in the

/var/lib/boinc/projects/boinc.bakerlab.org_rosetta directory.

[/var/lib/boinc/projects/boinc.bakerlab.org_rosetta]$ cat app_config.xml
<app_config>
<project_max_concurrent>3</project_max_concurrent>
</app_config>


So? How's that working out on Python? That might be the solution to limit them.

I have no idea.

Mon 29 Nov 2021 01:31:22 AM EST | Rosetta@home | Message from server: VirtualBox is not installed

I do no have VirtualBox, so I cannot run them.




hmm...ok...well maybe after they load up the project again I will try that.
ID: 103633 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 103634 - Posted: 1 Dec 2021, 0:58:23 UTC
Last modified: 1 Dec 2021, 0:59:52 UTC

Well I gave the confuzer a while to get its act together with 11 wu, then tried to increase it to twelve cpu`s
it don`t want to play, BM put it on hold with ` waiting for memory` [20GB in actual use]
So that's my lot with a 16 cpu opteron and 32GB mem, using about 75< 80% cpu
a lot better than the 4 wu it ran with big pythons.

Now then what can I meddle with next . . . .
How about going to `vbox64_mt` and setting the default cpu count to two and then half the default run time
that would offset the memory use and use more cpu`s , hmm ;)
ID: 103634 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1224
Credit: 13,847,051
RAC: 1,945
Message 103635 - Posted: 1 Dec 2021, 1:03:17 UTC - in response to Message 103634.  
Last modified: 1 Dec 2021, 1:08:29 UTC

Well I gave the confuzer a while to get its act together with 11, then tried to increase it to twelve cpu`s
it don`t want to play, BM put it on hold with ` waiting for memory` [20GB in actual use]
So that's my lot with 16 cpu opteron and 32GB mem, using about 75< 80% cpu
a lot better than the 4 wu it ran with big pythons.

Now then what can I meddle with next . . . .
How about going to `vbox64_mt` and setting the default cpu count to two and then half the default run time
that would offset the memory use and use more cpu`s , hmm ;)

You'd better check if Oracle provides vbox64_mt and whether the Python tasks are able to use it before doing much with that.

The Python tasks now only reserve 2.79GB of memory each, at least for Windows 10, so the project staff HAS found a way to control the amount of memory reserved.
ID: 103635 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1481
Credit: 14,608,684
RAC: 15,528
Message 103638 - Posted: 1 Dec 2021, 6:46:39 UTC - in response to Message 103635.  

The Python tasks now only reserve 2.79GB of memory each, at least for Windows 10, so the project staff HAS found a way to control the amount of memory reserved.
And if they bring it down to 1.5GB or so then everything should be OK.

Would be nice if we were to get some more Rosetta 4.20 Tasks as well.
Grant
Darwin NT
ID: 103638 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jonathan

Send message
Joined: 4 Oct 17
Posts: 43
Credit: 1,337,472
RAC: 7
Message 103639 - Posted: 1 Dec 2021, 10:42:31 UTC - in response to Message 103638.  

The created VMs still have the same Hard Disk and RAM set up as before. 8Gb HD and 6Gb RAM.
All tasks I had running had to be aborted. They weren't using any cpu cycles.

Is anyone successfully completing one?
ID: 103639 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 139 · 140 · 141 · 142 · 143 · 144 · 145 . . . 276 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org