Not getting any python work

Message boards : Number crunching : Not getting any python work

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 9 · Next

AuthorMessage
Profile G.L.I.S.
Avatar

Send message
Joined: 25 Dec 08
Posts: 23
Credit: 1,170,926
RAC: 0
Message 103262 - Posted: 14 Nov 2021, 15:34:56 UTC

Try:
Close BOINC client. Put a file called Sheduler projects.txt, in the boinc.bakerlab.org_rosetta folder. Write it:
Sheduling priority WUs: rah_make_work_rosetta_python_projects
Save and update.
Pre-emptively add BOINC space to the disk where the same is installed, via Options -> Processing preferences -> Disk and memory.
At least 30GB, otherwise the WUs won't go down (and could download Rosetta's wus).
If the space is insufficient the project will warn.
---------------------------------------------------------------------------------------------
I also tried to edit the app_config.xml file, for a Ryzen 3 3100 (4C / 8T), 16GB RAM as follows:

<app_config>
<app>
<name>rosetta_python_projects</name>
<avg_concurrent>3</avg_concurrent>
<max_concurrent>3</max_concurrent>
</app>
<app_version>
<app_name>rosetta_python_projects_v1.03</app_name>
<plan_class>vbox_64_mt</plan_class>
<avg_ncpus>2</avg_ncpus>
<max_ncpus>2</max_ncpus>
</app_version>
</app_config>

The purpose was to test the possibility of multithreading, but the file:
vboxwrapper_26203_windows_x86_64.exe (981 KB) does not allow this.
Perhaps because it is incomplete.
Another file from another project allows it and works: vboxwrapper_26200_windows_x86_64.exe (1452 KB).
I also tried to replace it, but it is automatically restored from the project.

However, on the Ryzen 3100, it processes a maximum of 2 wus at a time, while on the Ryzen 3600 (32GB RAM), the wus processed are 4.
Any changes I have tried to the app-config.xml file are totally ignored, even when restarting the BOINC client.

The amount of RAM is not a problem.

Byez
ID: 103262 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mmonnin

Send message
Joined: 2 Jun 16
Posts: 51
Credit: 9,212,722
RAC: 5,067
Message 103266 - Posted: 14 Nov 2021, 17:23:54 UTC - in response to Message 103165.  

<?xml version="1.0"?>

-<app_config>


-<app>

<name>rosetta 4.20</name>

<max_concurrent>1</max_concurrent>

</app>

</app_config>

But BOINC sends out multiple tasks using as many cores as it wants to.
I have no restrictions set anymore.



I've read on these forums that there's a bug regarding the max concurrent option. Instead of limiting the number of tasks, it downloads too many or something like that.
Also, I think the Rosetta 4.20 app name is just "rosetta".


The bug, I thought it was in app_config, but that was many discussions ago
You might be on to something with the 4.2
I'll remove that number and see, because over in the BOINC Berkley someone had me use a fake name for RAH and it came back as just Rosetta.


There is no bug.
max concurrent only specifies how many to run, not how many to download.
W/o any project references to select apps there is no way to only get non-vbox tasks at rosetta. You can always abort then, Rosetta does not penalize downloads for high abort counts.
ID: 103266 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mmonnin

Send message
Joined: 2 Jun 16
Posts: 51
Credit: 9,212,722
RAC: 5,067
Message 103268 - Posted: 14 Nov 2021, 17:45:59 UTC - in response to Message 103262.  

Try:
Close BOINC client. Put a file called Sheduler projects.txt, in the boinc.bakerlab.org_rosetta folder. Write it:
Sheduling priority WUs: rah_make_work_rosetta_python_projects
Save and update.
Pre-emptively add BOINC space to the disk where the same is installed, via Options -> Processing preferences -> Disk and memory.
At least 30GB, otherwise the WUs won't go down (and could download Rosetta's wus).
If the space is insufficient the project will warn.
---------------------------------------------------------------------------------------------
I also tried to edit the app_config.xml file, for a Ryzen 3 3100 (4C / 8T), 16GB RAM as follows:

<app_config>
<app>
<name>rosetta_python_projects</name>
<avg_concurrent>3</avg_concurrent>
<max_concurrent>3</max_concurrent>
</app>
<app_version>
<app_name>rosetta_python_projects_v1.03</app_name>
<plan_class>vbox_64_mt</plan_class>
<avg_ncpus>2</avg_ncpus>
<max_ncpus>2</max_ncpus>
</app_version>
</app_config>

The purpose was to test the possibility of multithreading, but the file:
vboxwrapper_26203_windows_x86_64.exe (981 KB) does not allow this.
Perhaps because it is incomplete.
Another file from another project allows it and works: vboxwrapper_26200_windows_x86_64.exe (1452 KB).
I also tried to replace it, but it is automatically restored from the project.

However, on the Ryzen 3100, it processes a maximum of 2 wus at a time, while on the Ryzen 3600 (32GB RAM), the wus processed are 4.
Any changes I have tried to the app-config.xml file are totally ignored, even when restarting the BOINC client.

The amount of RAM is not a problem.

Byez


Your app_config doesn't work because that is the wrong plan_class
From the applications page: 1.03 (vbox64)
Replace vbox_64_mt with vbox64
Although that probably doesn't mean that the app will use 2 threads. BOINC may dedicate 2 threads but thats different from the app using 2 threads.
app_config.xml just needs to be reread. The client does not need restarted to pick up the changes.
The messages/log will show you what is incorrect with the app name and/or plan_class.
ID: 103268 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 70
Message 103273 - Posted: 14 Nov 2021, 18:05:52 UTC - in response to Message 103262.  
Last modified: 14 Nov 2021, 18:11:16 UTC

Sheduler projects.txt <-- you sure about this name? Not Scheduler?

AND

As I have said many many times, you can not use rosetta python projects as a name.

I used your name line to change something in a test app_config and got this:
Rosetta@home: Notice from BOINC
Your app_config.xml file refers to an unknown application 'rosetta_python_projects'. Known applications: None



Try:
Close BOINC client. Put a file called Sheduler projects.txt, in the boinc.bakerlab.org_rosetta folder. Write it:
Sheduling priority WUs: rah_make_work_rosetta_python_projects
Save and update.
Pre-emptively add BOINC space to the disk where the same is installed, via Options -> Processing preferences -> Disk and memory.
At least 30GB, otherwise the WUs won't go down (and could download Rosetta's wus).
If the space is insufficient the project will warn.
---------------------------------------------------------------------------------------------
I also tried to edit the app_config.xml file, for a Ryzen 3 3100 (4C / 8T), 16GB RAM as follows:

<app_config>
<app>
<name>rosetta_python_projects</name>
<avg_concurrent>3</avg_concurrent>
<max_concurrent>3</max_concurrent>
</app>
<app_version>
<app_name>rosetta_python_projects_v1.03</app_name>
<plan_class>vbox_64_mt</plan_class>
<avg_ncpus>2</avg_ncpus>
<max_ncpus>2</max_ncpus>
</app_version>
</app_config>

The purpose was to test the possibility of multithreading, but the file:
vboxwrapper_26203_windows_x86_64.exe (981 KB) does not allow this.
Perhaps because it is incomplete.
Another file from another project allows it and works: vboxwrapper_26200_windows_x86_64.exe (1452 KB).
I also tried to replace it, but it is automatically restored from the project.

However, on the Ryzen 3100, it processes a maximum of 2 wus at a time, while on the Ryzen 3600 (32GB RAM), the wus processed are 4.
Any changes I have tried to the app-config.xml file are totally ignored, even when restarting the BOINC client.

The amount of RAM is not a problem.

Byez
ID: 103273 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tullio

Send message
Joined: 10 May 20
Posts: 62
Credit: 612,922
RAC: 31
Message 103274 - Posted: 14 Nov 2021, 18:20:08 UTC - in response to Message 102936.  

In QuChemPedIA@home when I get a "unmanageable task" error I simply exit the Boinc Manager and then restart it. This works most of the time.
Tullio
ID: 103274 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile G.L.I.S.
Avatar

Send message
Joined: 25 Dec 08
Posts: 23
Credit: 1,170,926
RAC: 0
Message 103277 - Posted: 14 Nov 2021, 18:34:59 UTC - in response to Message 103274.  

sorry for the grammatical error, it is corrected by adding the 'c' :(

However, only wus python comes down with the .txt file in question
ID: 103277 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile G.L.I.S.
Avatar

Send message
Joined: 25 Dec 08
Posts: 23
Credit: 1,170,926
RAC: 0
Message 103278 - Posted: 14 Nov 2021, 18:54:51 UTC

The first few times he also answered me with the same sentence, but now ...

[/url]
ID: 103278 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1361
Credit: 13,624,788
RAC: 0
Message 103289 - Posted: 14 Nov 2021, 22:12:40 UTC - in response to Message 103266.  

There is no bug.
max concurrent only specifies how many to run, not how many to download.
There is a bug that has been reported by many people across several projects.
max_concurrent is supposed to only specify how many to run. But for some reason it does affect the number downloaded- deadlines & cache settings are completely ignored and the Mananger just keeps requesting new work as soon as the Scheduler contact delay expires.
Grant
Darwin NT
ID: 103289 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile G.L.I.S.
Avatar

Send message
Joined: 25 Dec 08
Posts: 23
Credit: 1,170,926
RAC: 0
Message 103290 - Posted: 14 Nov 2021, 22:44:49 UTC - in response to Message 103268.  
Last modified: 14 Nov 2021, 23:03:50 UTC


Your app_config doesn't work because that is the wrong plan_class
From the applications page: 1.03 (vbox64)
Replace vbox_64_mt with vbox64
Although that probably doesn't mean that the app will use 2 threads. BOINC may dedicate 2 threads but thats different from the app using 2 threads.
app_config.xml just needs to be reread. The client does not need restarted to pick up the changes.
The messages/log will show you what is incorrect with the app name and/or plan_class.
Perhaps the project, need an add program:

https://docs.docker.com/toolbox/

If the project accelerates, the science will gain.
ID: 103290 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 70
Message 103309 - Posted: 15 Nov 2021, 19:11:55 UTC - in response to Message 103273.  

Where does this text go?
Scheduling priority WUs: rah_make_work_rosetta_python_projects


Sheduler projects.txt <-- you sure about this name? Not Scheduler?

AND

As I have said many many times, you can not use rosetta python projects as a name.

I used your name line to change something in a test app_config and got this:
Rosetta@home: Notice from BOINC
Your app_config.xml file refers to an unknown application 'rosetta_python_projects'. Known applications: None



Try:
Close BOINC client. Put a file called Sheduler projects.txt, in the boinc.bakerlab.org_rosetta folder. Write it:
Sheduling priority WUs: rah_make_work_rosetta_python_projects
Save and update.
Pre-emptively add BOINC space to the disk where the same is installed, via Options -> Processing preferences -> Disk and memory.
At least 30GB, otherwise the WUs won't go down (and could download Rosetta's wus).
If the space is insufficient the project will warn.
---------------------------------------------------------------------------------------------
I also tried to edit the app_config.xml file, for a Ryzen 3 3100 (4C / 8T), 16GB RAM as follows:

<app_config>
<app>
<name>rosetta_python_projects</name>
<avg_concurrent>3</avg_concurrent>
<max_concurrent>3</max_concurrent>
</app>
<app_version>
<app_name>rosetta_python_projects_v1.03</app_name>
<plan_class>vbox_64_mt</plan_class>
<avg_ncpus>2</avg_ncpus>
<max_ncpus>2</max_ncpus>
</app_version>
</app_config>

The purpose was to test the possibility of multithreading, but the file:
vboxwrapper_26203_windows_x86_64.exe (981 KB) does not allow this.
Perhaps because it is incomplete.
Another file from another project allows it and works: vboxwrapper_26200_windows_x86_64.exe (1452 KB).
I also tried to replace it, but it is automatically restored from the project.

However, on the Ryzen 3100, it processes a maximum of 2 wus at a time, while on the Ryzen 3600 (32GB RAM), the wus processed are 4.
Any changes I have tried to the app-config.xml file are totally ignored, even when restarting the BOINC client.

The amount of RAM is not a problem.

Byez
ID: 103309 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 70
Message 103310 - Posted: 15 Nov 2021, 19:13:11 UTC

Any idea what this means?
[work_fetch] No project chosen for work fetch
ID: 103310 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile G.L.I.S.
Avatar

Send message
Joined: 25 Dec 08
Posts: 23
Credit: 1,170,926
RAC: 0
Message 103315 - Posted: 15 Nov 2021, 21:15:00 UTC - in response to Message 103309.  

Where does this text go?
Scheduling priority WUs: rah_make_work_rosetta_python_projects



On a file (.txt), into Rosetta's directory


ID: 103315 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1361
Credit: 13,624,788
RAC: 0
Message 103316 - Posted: 15 Nov 2021, 21:26:21 UTC - in response to Message 103310.  

Any idea what this means?
[work_fetch] No project chosen for work fetch
To me it would imply that there were no projects for the Scheduler to look for work from.
Grant
Darwin NT
ID: 103316 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 70
Message 103322 - Posted: 15 Nov 2021, 23:36:44 UTC - in response to Message 103316.  

Any idea what this means?
[work_fetch] No project chosen for work fetch
To me it would imply that there were no projects for the Scheduler to look for work from.


Which is odd..being there is 4.2 work.
Or that's a fancy way of that option saying it can't load any more work since the queue is full?
ID: 103322 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 70
Message 103323 - Posted: 15 Nov 2021, 23:37:38 UTC - in response to Message 103315.  

Where does this text go?
Scheduling priority WUs: rah_make_work_rosetta_python_projects



On a file (.txt), into Rosetta's directory





Yeah ok...but where in the file with all the other text? top of the pile or what?
ID: 103323 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1361
Credit: 13,624,788
RAC: 0
Message 103327 - Posted: 16 Nov 2021, 0:42:07 UTC - in response to Message 103322.  

Any idea what this means?
[work_fetch] No project chosen for work fetch
To me it would imply that there were no projects for the Scheduler to look for work from.


Which is odd..being there is 4.2 work.
Or that's a fancy way of that option saying it can't load any more work since the queue is full?
The way i read it is that it can't even find any projects to even request work from. I'd suggest undoing whatever changes you've made recently, exit & restart BOINC & see what messages then come through.
If the queue was full, it would say so.
Grant
Darwin NT
ID: 103327 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 70
Message 103333 - Posted: 16 Nov 2021, 7:34:51 UTC - in response to Message 103327.  

Any idea what this means?
[work_fetch] No project chosen for work fetch
To me it would imply that there were no projects for the Scheduler to look for work from.


Which is odd..being there is 4.2 work.
Or that's a fancy way of that option saying it can't load any more work since the queue is full?
The way i read it is that it can't even find any projects to even request work from. I'd suggest undoing whatever changes you've made recently, exit & restart BOINC & see what messages then come through.
If the queue was full, it would say so.


It's a fresh install.
Other than that text for python there is nothing.
ID: 103333 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Chooka
Avatar

Send message
Joined: 25 Jan 17
Posts: 6
Credit: 8,715,933
RAC: 0
Message 103334 - Posted: 16 Nov 2021, 8:44:26 UTC

I see I'm not the only one who wished there was an option in the preferences as to which work units I want!

I have a Virtual Box installed but even when it's closed, BOINC downloads Python wu's. These are MASSIVE memory hogs at around 7.6gig each. As a result, I have both standard & Python running but w/u's queued up due to insufficient memory. (I have 32gig)

If I had a choice, I'd only enable the standard wu's instead of the Python one's but there's no way to toggle one or the other. To be honest, I aborted the work and moved onto another project. That's not a win for Rosetta@Home.
I guess I could just let my pc's crunch as much as possible and leave it be. I'd still like to see an option in the preferences.

ID: 103334 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1886
Credit: 6,099,222
RAC: 62
Message 103342 - Posted: 16 Nov 2021, 12:26:29 UTC - in response to Message 103334.  
Last modified: 16 Nov 2021, 12:28:10 UTC

I see I'm not the only one who wished there was an option in the preferences as to which work units I want!

I have a Virtual Box installed but even when it's closed, BOINC downloads Python wu's. These are MASSIVE memory hogs at around 7.6gig each. As a result, I have both standard & Python running but w/u's queued up due to insufficient memory. (I have 32gig)

If I had a choice, I'd only enable the standard wu's instead of the Python one's but there's no way to toggle one or the other. To be honest, I aborted the work and moved onto another project. That's not a win for Rosetta@Home.
I guess I could just let my pc's crunch as much as possible and leave it be. I'd still like to see an option in the preferences.


The standard Rosetta tasks also reserve 8gb of ram for each task, one way around this is a simple app_config file that limits the number of tasks running per project, like this:

<app_config>
<project_max_concurrent>1</project_max_concurrent>
</app_config>

Put it in the Project folder and then tell Boinc to reread the config files and you are good to go. It's also useful for running 3 tasks of this project and 5 of that project on an 8 core cpu. You do need to be careful as each Project will download work thinking it's for all 8 cores so it's best to limit the cache size accordingly otherwise you could easily run into task deadline issues.

But it does not do anything about not getting Python over regular Rosetta tasks, that's a Project issue and they haven't done anything yet.
ID: 103342 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile G.L.I.S.
Avatar

Send message
Joined: 25 Dec 08
Posts: 23
Credit: 1,170,926
RAC: 0
Message 103344 - Posted: 16 Nov 2021, 12:53:45 UTC - in response to Message 103323.  

Yeah ok...but where in the file with all the other text? top of the pile or what?


I don't think I fully understand the question, however, to recap
In addition to the text file named 'Schduler projects', the content of which will be: Scheduling priority WUs: rah_make_work_rosetta_python_projects

It will need to be integrated with an 'app_config.xml' file with dedicated content, such as:
<app_config>
<app>
<name>rosetta_python_projects</name>
<avg_concurrent>4</avg_concurrent>
<max_concurrent>4</max_concurrent>
</app>
<app_version>
<app_name>rosetta_python_projects_v1.03</app_name>
<plan_class>vbox_64</plan_class>
<avg_ncpus>1</avg_ncpus>
<max_threads>1</max_threads>
<max_mem_usage>14592</max_mem_usage>
</app_version>
</app_config>

This content is just an example! (to 16GB RAM system)
ID: 103344 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 9 · Next

Message boards : Number crunching : Not getting any python work



©2022 University of Washington
https://www.bakerlab.org