Not getting any python work

Message boards : Number crunching : Not getting any python work

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · Next

AuthorMessage
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1828
Credit: 106,960,362
RAC: 4,216
Message 103411 - Posted: 18 Nov 2021, 19:21:29 UTC
Last modified: 18 Nov 2021, 19:22:40 UTC

According to Jord's post in this thread, being able to choose sub projects is not part of the standard BOINC code.

https://boinc.berkeley.edu/forum_thread.php?id=14462&postid=106111#106111
ID: 103411 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Michael Goetz
Avatar

Send message
Joined: 17 Jan 08
Posts: 12
Credit: 158,189
RAC: 0
Message 103412 - Posted: 18 Nov 2021, 19:45:25 UTC - in response to Message 103411.  

According to Jord's post in this thread, being able to choose sub projects is not part of the standard BOINC code.

https://boinc.berkeley.edu/forum_thread.php?id=14462&postid=106111#106111


With all due respect that Jord is *normally* due, this time he's simply plain wrong.

The app selection is different at each project (because they have different apps), so you do have to write a small amount of code to fill in the details, but the framework and the prototype code for app selection is in the file html/project.sample/project_specific_prefs.inc. It's here in the git repository: https://github.com/BOINC/boinc/blob/master/html/project.sample/project_specific_prefs.inc

Jord should have known better. He was looking in the wrong place.
Want to find one of the largest known primes? Try PrimeGrid. Or help cure disease at WCG.

ID: 103412 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Admin
Project administrator

Send message
Joined: 1 Jul 05
Posts: 4805
Credit: 0
RAC: 0
Message 103413 - Posted: 18 Nov 2021, 19:47:35 UTC
Last modified: 18 Nov 2021, 19:52:49 UTC

The scheduler was modified to flag hosts that produce specific VM box errors in the stderr output:

"VM failed to start"
"execv: Permission denied"
"ERROR: VBoxManage list hostinfo failed"

If flagged, the host will no longer get VM jobs.

This was a quick but dirty fix to prevent sending VM jobs to such hosts.
ID: 103413 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Michael Goetz
Avatar

Send message
Joined: 17 Jan 08
Posts: 12
Credit: 158,189
RAC: 0
Message 103415 - Posted: 18 Nov 2021, 20:22:25 UTC - in response to Message 103413.  

The scheduler was modified to flag hosts that produce specific VM box errors in the stderr output:

"VM failed to start"
"execv: Permission denied"
"ERROR: VBoxManage list hostinfo failed"

If flagged, the host will no longer get VM jobs.

This was a quick but dirty fix to prevent sending VM jobs to such hosts.


So the only way to fix such a host would be to detach from RAH, wait until all the tasks for that host have been purged from the database, delete the host, and then reattach, thus creating a new host? Or do the the dual-installation hack.

Anyway, thanks for this information. At least some of us can stop trying now.
Want to find one of the largest known primes? Try PrimeGrid. Or help cure disease at WCG.

ID: 103415 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 47
Message 103418 - Posted: 18 Nov 2021, 21:31:47 UTC - in response to Message 103412.  
Last modified: 18 Nov 2021, 21:39:17 UTC

According to Jord's post in this thread, being able to choose sub projects is not part of the standard BOINC code.

https://boinc.berkeley.edu/forum_thread.php?id=14462&postid=106111#106111


With all due respect that Jord is *normally* due, this time he's simply plain wrong.

The app selection is different at each project (because they have different apps), so you do have to write a small amount of code to fill in the details, but the framework and the prototype code for app selection is in the file html/project.sample/project_specific_prefs.inc. It's here in the git repository: https://github.com/BOINC/boinc/blob/master/html/project.sample/project_specific_prefs.inc

Jord should have known better. He was looking in the wrong place.



You can't select it in BOINC (at least not that I have ever seen), however....the project website can allow you to choose which project you want (LHC@Home, PrimeGrid, WCG, etc.) RAH chose not to do this. How hard can it be to write some code to interact between the web and your project server? Apparently very hard.
Since even the script that was written has no effect on getting python work.
I see 29,000+ new 4.2 so I guess my computer will get a bite of that stuff.
But I don't see it getting python at all. (It's only downloading 4.2) despite removing the project and reattaching. I give up. The server hates me.
ID: 103418 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Michael Goetz
Avatar

Send message
Joined: 17 Jan 08
Posts: 12
Credit: 158,189
RAC: 0
Message 103420 - Posted: 18 Nov 2021, 22:04:43 UTC - in response to Message 103418.  

You can't select it in BOINC (at least not that I have ever seen), however....the project website can allow you to choose which project you want (LHC@Home, PrimeGrid, WCG, etc.) RAH chose not to do this. How hard can it be to write some code to interact between the web and your project server? Apparently very hard.


In fact, it's trivially easy. The scheduler supports app selection out of the box, and the code I referenced above is the template for adding app selection to the web server's project preferences page. The interaction between the two is automatic. It's written into the scheduler code.

If you're only selecting apps, (and not the more complex stuff like GPU classes, multithreading, and so forth) this is something that anyone who knows PHP could set up in an hour or two. Much less if they've done it before. But longer if you want to, you know, test it :P

Give me the app_id numbers of the Rosetta, Rosetta Mini, and rosetta python project apps (they're probably 1, 2, and something higher than3), and I can give you a drop in, customized project_specific_prefs.inc file that would give you working app selection here.

By tomorrow.

It's that simple.

This is a serious offer. I even promise to write the code *after* I have my morning coffee. There's fewer bugs when I'm caffeinated.

(Okay, there IS a possibility that Rosetta's scheduler code hasn't been updated in 15 years, and might possibly be too old to support app selection. I'm hypothesizing, however. The oldest scheduler code I've ever looked at supported app selection, but Rosetta's older than that. So it's possible it would need an updated scheduler. Otherwise, it's trivial. And certainly it will work for any project created in the last 10 or so years.)
Want to find one of the largest known primes? Try PrimeGrid. Or help cure disease at WCG.

ID: 103420 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Michael Goetz
Avatar

Send message
Joined: 17 Jan 08
Posts: 12
Credit: 158,189
RAC: 0
Message 103423 - Posted: 18 Nov 2021, 22:33:45 UTC - in response to Message 103420.  

After actually going to write that code, it turns out to be even simpler. Or more complex. It depends on the age of specific files on the webserver.

If you have BOINC web server code from the last 4 years or so, you don't have to write any code at all. Near the beginning of html/project/project.inc is a single definition:

define('APP_SELECT_PREFS', false);
    // user can choose which apps to run


Change "false" to "true" and the project selection should appear on the webpage.

That's it.

If, however, the webserver code is older, and doesn't use that define, then the file html/project/project_specific_prefs.inc needs to be customized to add the selections. It's really easy to do, as I mentioned, but I can't do it blind without seeing the other include files on the server. They've changed over the years and the functions need to be written differently depending on what you already have.
Want to find one of the largest known primes? Try PrimeGrid. Or help cure disease at WCG.

ID: 103423 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1636
Credit: 6,462,241
RAC: 35
Message 103424 - Posted: 18 Nov 2021, 22:54:13 UTC - in response to Message 103420.  

Give me the app_id numbers of the Rosetta, Rosetta Mini, and rosetta python project apps (they're probably 1, 2, and something higher than3), and I can give you a drop in, customized project_specific_prefs.inc file that would give you working app selection here.

By tomorrow.

It's that simple.

This is a serious offer. I even promise to write the code *after* I have my morning coffee. There's fewer bugs when I'm caffeinated.


I hope that someone of R@H team will contact you
ID: 103424 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 47
Message 103431 - Posted: 19 Nov 2021, 8:10:12 UTC - in response to Message 103424.  

I hope that someone of R@H team will contact you[/quote]

RAH is not interested in this kind of thing or they would have done something on their end.
They have never ever done anything other than let what they built stand. Never modify, never do anything.
No one monitors these forums now, no one responds to messages unless you win the lottery.
It is up to those out here that have the know how to solve the problems.
ID: 103431 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1636
Credit: 6,462,241
RAC: 35
Message 103432 - Posted: 19 Nov 2021, 8:31:25 UTC - in response to Message 103431.  

RAH is not interested in this kind of thing or they would have done something on their end.
They have never ever done anything other than let what they built stand. Never modify, never do anything.
No one monitors these forums now, no one responds to messages unless you win the lottery.
It is up to those out here that have the know how to solve the problems.


I'm trying to contact R@H on Twitter....
ID: 103432 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 47
Message 103434 - Posted: 19 Nov 2021, 16:59:56 UTC - in response to Message 103432.  

RAH is not interested in this kind of thing or they would have done something on their end.
They have never ever done anything other than let what they built stand. Never modify, never do anything.
No one monitors these forums now, no one responds to messages unless you win the lottery.
It is up to those out here that have the know how to solve the problems.


I'm trying to contact R@H on Twitter....


They don't pay attention to that either.
I hit them up there when they had a big outage.
Got 0 back.
ID: 103434 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 47
Message 103435 - Posted: 19 Nov 2021, 17:01:21 UTC - in response to Message 103415.  
Last modified: 19 Nov 2021, 17:05:40 UTC

The scheduler was modified to flag hosts that produce specific VM box errors in the stderr output:

"VM failed to start"
"execv: Permission denied"
"ERROR: VBoxManage list hostinfo failed"

If flagged, the host will no longer get VM jobs.

This was a quick but dirty fix to prevent sending VM jobs to such hosts.


So the only way to fix such a host would be to detach from RAH, wait until all the tasks for that host have been purged from the database, delete the host, and then reattach, thus creating a new host? Or do the the dual-installation hack.

Anyway, thanks for this information. At least some of us can stop trying now.



Oh..I must have had one of those then.
And you don't find a way to send back to these hosts again or notify them that they had a problem that needs to be fixed? You couldn't get a bot program to send a automated message?

You managed to block my system then and then you can't keep the 4.2 filled?
I think I have had enough after 15 years doing work for you guys.
One less system won't affect you.
ID: 103435 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 47
Message 103436 - Posted: 19 Nov 2021, 17:03:17 UTC

Look guys, 15 years here and then I get blocked by something really stupid on their end?
And they don't communicate anymore, they don't fix anything or try anything new?
They can't keep their basic queue filled?

This side of their project has gone south.
So I am moving on somewhere else.

Was nice knowing you Rosie.
ID: 103436 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile G.L.I.S.
Avatar

Send message
Joined: 25 Dec 08
Posts: 23
Credit: 1,170,926
RAC: 0
Message 103438 - Posted: 19 Nov 2021, 17:27:04 UTC - in response to Message 103436.  
Last modified: 19 Nov 2021, 17:48:15 UTC

@ Greg_BE
I'm sorry that wus python cannot be downloaded.
You can try to edit the file I posted, called 'Scheduler projects.txt' as follows (copy and paste) :

Scheduling_priority_to_rah_make_work_rosetta_python_projects_work_units
Server computer name=bwsrv1

Of course I can't guarantee anything about its actual effectiveness, it's just an attempt ...

P.S. the 'app_config.xml' file it is still necessary in any case.

I forgot (I took it for granted) ...
Virtualization must be enabled in the BIOS (CMOS):
Intel VT-x or AMD-V depending on the processor installed.

With the securable app https://securable.it.uptodown.com/windows/download
verify the effective operation of virtualization. Seeing it enabled in the task manager CPU performance is not enough.

If everything is ok, it is complete:
ID: 103438 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Kissagogo27

Send message
Joined: 31 Mar 20
Posts: 70
Credit: 1,772,317
RAC: 656
Message 103440 - Posted: 19 Nov 2021, 21:49:05 UTC

you can get all the Gibson Reseach Corporation Freeware from the original website ; )


https://www.grc.com/freepopular.htm
ID: 103440 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1364
Credit: 13,624,788
RAC: 0
Message 103442 - Posted: 19 Nov 2021, 23:28:04 UTC - in response to Message 103413.  

The scheduler was modified to flag hosts that produce specific VM box errors in the stderr output:

"VM failed to start"
"execv: Permission denied"
"ERROR: VBoxManage list hostinfo failed"

If flagged, the host will no longer get VM jobs.

This was a quick but dirty fix to prevent sending VM jobs to such hosts.
Does it send a message to that host's BOINC Manager Tab (or at least leave one in the event log) stating what has happened?
We've already got some Hosts that could do work Python work, and now can't, and have been struggling to find the cause. Has this patch been out for several weeks now? If so, it would explain why it has been impossible for previously Python capable hosts to now no longer get any work.

At the very least the Scheduler needs to provide a response as to why it's not providing work- this should be the case for any work. Such as now with the shortages of Rosetta 4.20 work instead of the present completely uninformative response,
20/11/2021 05:49:56 | Rosetta@home | Scheduler request completed: got 0 new tasks
20/11/2021 05:49:56 | Rosetta@home | No tasks sent

it would be much better to see something along the line of
20/11/2021 05:49:56 | Rosetta@home | Scheduler request completed: got 0 new tasks
20/11/2021 05:49:56 | Rosetta@home | No tasks sent
20/11/2021 05:49:56 | Rosetta@home | No Rosetta 4.20 Tasks available.
20/11/2021 05:49:56 | Rosetta@home | Python work is available, but your system doesn't have VirtualBox installed
or

20/11/2021 05:49:56 | Rosetta@home | Scheduler request completed: got 0 new tasks
20/11/2021 05:49:56 | Rosetta@home | No tasks sent
20/11/2021 05:49:56 | Rosetta@home | No Rosetta 4.20 Tasks available.
20/11/2021 05:49:56 | Rosetta@home | Python work is available, but your system can't process it due to previous VirtualBox errors

Grant
Darwin NT
ID: 103442 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5550
Credit: 5,554,708
RAC: 47
Message 103443 - Posted: 20 Nov 2021, 8:37:09 UTC - in response to Message 103438.  
Last modified: 20 Nov 2021, 8:38:13 UTC

@ Greg_BE
I'm sorry that wus python cannot be downloaded.
You can try to edit the file I posted, called 'Scheduler projects.txt' as follows (copy and paste) :

Scheduling_priority_to_rah_make_work_rosetta_python_projects_work_units
Server computer name=bwsrv1

Of course I can't guarantee anything about its actual effectiveness, it's just an attempt ...

P.S. the 'app_config.xml' file it is still necessary in any case.

I forgot (I took it for granted) ...
Virtualization must be enabled in the BIOS (CMOS):
Intel VT-x or AMD-V depending on the processor installed.

With the securable app https://securable.it.uptodown.com/windows/download
verify the effective operation of virtualization. Seeing it enabled in the task manager CPU performance is not enough.

If everything is ok, it is complete:


I'm no longer crunching for this project. And I will never get any python work based on what I quoted from ADMIN in my post here.
He pissed me off with that cheap and dirty way to solve a problem that he is to lazy to fix or find out the cause of. I gave this project 15 years. And I have watched how it has degraded over time. And if he can't keep the queue full of stuff I can crunch, then forget it. I will let WCG and FAH work on protein stuff, I am also running QuChem with Vbox. That's my biological stuff. I have other stuff I run as well.
ID: 103443 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Admin
Project administrator

Send message
Joined: 1 Jul 05
Posts: 4805
Credit: 0
RAC: 0
Message 103473 - Posted: 22 Nov 2021, 20:05:35 UTC - in response to Message 103442.  

That's a great suggestion. I'll add that functionality.
ID: 103473 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Falconet

Send message
Joined: 9 Mar 09
Posts: 343
Credit: 987,274
RAC: 0
Message 103477 - Posted: 22 Nov 2021, 21:31:58 UTC - in response to Message 103473.  

That's a great suggestion. I'll add that functionality.

The ability to choose which apps a user wants to run would also be appreciated.
ID: 103477 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1828
Credit: 106,960,362
RAC: 4,216
Message 103478 - Posted: 22 Nov 2021, 21:41:15 UTC - in response to Message 103473.  

That's a great suggestion. I'll add that functionality.


Please can you post back when you have implemented something so we can check our idle machines - I've got a few that aren't getting any Rosetta tasks at the moment and I have no idea why.
ID: 103478 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · Next

Message boards : Number crunching : Not getting any python work



©2022 University of Washington
https://www.bakerlab.org