Posts by mikey

1) Message boards : Number crunching : New jobs in Rosetta Python projects (Message 108890)
Posted 1 Mar 2024 by mikey
Post:
I added the VM Extension pack, but still having problems. The WUs will now at least run for a while (hours in most cases) but then still error out. Anyone figured out what else to do?


How many are you running at a time, I only run 1 at a time
2) Message boards : Number crunching : New jobs in Rosetta Python projects (Message 108845)
Posted 15 Feb 2024 by mikey
Post:
Anyone else running these fine in Vbox under Linux? what config are you running?


Don't know if it's needed here but in both LinuxMint and Windows the VBOX Extension Pack is not installed by default, I use both OS's as well and had to install it to get the LHC and LHC-dev tasks to run and get credits. On my Ubuntu Linux box it was installed by default.
3) Message boards : Number crunching : New jobs in Rosetta Python projects (Message 108808)
Posted 24 Jan 2024 by mikey
Post:
I know Rosetta hasn't ever done this but these new Python tasks take 7.45gb of memory for EACH task on my Windows pc's, i have pc's with anything from 6 to 32 cpu cores but not a one of them has enough memory to fill every cpu core with a Python task and not lock up the pc to the point only a reboot and reducing the physical number of tasks I can run at Rosetta. Normally it's not a problem with anything EXCEPT the Python tasks


Than limit the amount of concurrent python tasks using an app_confil.xml. This can be even different for each of your computers, depending on the amount of memory.


I DO and said that but the problem is as I said when I get the normal or beta tasks I will want to run as many at a time as I can and I can't do that with the current Prefences setup.
I know I could use an app_config.xml file to limit this or that type of task but have forgotten how to do that and the settings are all project specific and I don't remember how to that here and besides newbies will have no clue and just give up and go elsewhere and that's NOT what anyone wants. PrimeGrid has an elaborate prefences setup that even includes number of cpu's per task in additionto the type of task and how many of that type of task to run so it IS possible to have more and better choices.

Honestly I wish the Developers would just hard code it into the Server coding which would fix places like Gerasim who have totally borked the settings, but that's a whole other problem, as well as help out places like Rosetta. YES Projects should be able to disable the defaults as they do now but sameness can be an advantage for some things.,
4) Message boards : Number crunching : New jobs in Rosetta Python projects (Message 108798)
Posted 23 Jan 2024 by mikey
Post:
We will be releasing new jobs using the Rosetta Python projects application with RoseTTAFold All-Atom, our state-of-the-art deep learning based application that predict the structures of proteins in complex with ligands such as DNA and other biological molecules and also models post-translational modifications. As a reminder, this application will be released in the form of virtual machine. To help us with testing and gathering performance statistics, we ask that you do not abort related work units.


I know Rosetta hasn't ever done this but these new Python tasks take 7.45gb of memory for EACH task on my Windows pc's, i have pc's with anything from 6 to 32 cpu cores but not a one of them has enough memory to fill every cpu core with a Python task and not lock up the pc to the point only a reboot and reducing the physical number of tasks I can run at Rosetta. Normally it's not a problem with anything EXCEPT the Python tasks so what I'm asking is you add an optional setting to run the Python tasks, be it thru a choice box or thru a box that says ie 'allow beta tasks' that we can then select. The problem with the 2nd option alone is that people can't choose to not run the regular Rosetta tasks while they wait for any Python tasks to come their way so they could easily end up with the same problem I'm having with lack of memory in my desktops.

A simple set of type of tasks choice boxes would make things a whole lot easier on the users end and not really change anything beyond the initial setup for you Admins on your end.
This page shows what I am talking about but most Projects have something similar to it:
https://escatter11.fullerton.edu/nfs/prefs.php?subset=project

In case the link doesn't work, it's to my Preferences page at NFS, here is a copy of one section of it:

Home
Resource share 1
Use CPU
Run only the selected applications 14e Lattice Sieve (lasieved) - uses less than 0.5 GB memory, work may be infrequently available: yes
15e Lattice Sieve for smaller numbers (lasievee_small) - small numbers, uses up to 0.8 GB memory: no
15e Lattice Sieve (lasievee) - medium numbers, uses up to 1 GB memory: no
16e Lattice Sieve for smaller numbers (lasievef_small) - large numbers, uses up to 1 GB memory: no
16e Lattice Sieve V5 (lasieve5f) - largest numbers, uses up to 1.25 GB memory: no
If no work for selected applications is available, accept work from other applications? no

Thank you for reading this and I hope you can discuss and implement this soon .
Thank you for you time
mikey
5) Message boards : Number crunching : Rosetta Beta 6.00 (Message 108777)
Posted 22 Dec 2023 by mikey
Post:
I use an APP_CONFIG to run all my projects. I can set to run just 1 beta WU, never tried to set max tasks to 0.
Right now i'm running Milky Way until Beta is more stable.

<max_concurrent>Zero</max_concurrent> = <max_concurrent>Infinity</max_concurrent>


I don't know what your settings are but I'm getting nearly 100% valid tasks here on my both Windows and Linux pc;s.

State: All (2097) · In progress (576) · Validation pending (0) · Validation inconclusive (0) · Valid (1514) · Invalid (0) · Error (7)
Application: All (2097) · Rosetta (102) · Rosetta Beta (1995) · Rosetta Mini (0) · rosetta python projects (0)
6) Message boards : Rosetta@home Science : Unable to get any work (Message 108734)
Posted 2 Dec 2023 by mikey
Post:
Hi there. Apologies if this is posted in the wrong area, but I am unable to get work.
I bought this new computer about a month ago. it has an Intel(R) Arc(TM) A770 Graphics card and a 13th Gen Intel(R) Core(TM) i7-13700F processor.
I hope you can help.
I've tried attaching to several boinc projects, and I'm not getting any work.
Is this just bad luck or is something else wrong?


Try PrimeGrid and Einstein, Asteroids also has cpu and gpu tasks but I don't know if they can use your Intel Arc gpu or not
7) Message boards : Number crunching : Ralph test spotted (Message 108654)
Posted 1 Nov 2023 by mikey
Post:
Not really necessary, don't we all have large SSDs?


I believe the idea is a Squid thing like LHC crunchers use to download tasks to a single pc then send them out to multiple pc's and then return them from the original pc that downloaded them thereby bypassing one of the security features built into Boinc that is that the pc that downloads a task must be the one that returns it back to the project.
8) Message boards : Cafe Rosetta : Happy 2023 (Message 107909)
Posted 1 Jan 2023 by mikey
Post:
Happy new year.
A lot of wus to everyone!!
Let's do science


Well said!! I certainly hope so too!!
9) Message boards : Cafe Rosetta : Antibody That Can Take Out All COVID-19 Variants (Message 106877)
Posted 7 Sep 2022 by mikey
Post:
https://www.prevention.com/health/a41092334/antibody-neutralize-covid-variants/

This is in a mice test ONLY so far so it's VERY early days but it may help cut down on all the boosters they are putting out
10) Message boards : Number crunching : Problems and Technical Issues with Rosetta@home (Message 106673)
Posted 29 Jul 2022 by mikey
Post:
I'm a bit mystified why my RAC keeps diving despite 9 validated tasks in the last whatever time period it is allowed to show.

That means that you are not returning validated tasks as fast as you were earlier in the month the RAC is for.

If you can't download tasks, you can't complete and return them.

The tasks that are shown is usually for a shorter period, such as a week.



I see the problem now, LHC had a big batch of use all (15 in my case)cpu tasks.



Correction: It's Milkyway with 15 out 16 CPUs for 3-4 minutes per task. 83 tasks in queue. But I switch applications every few hours so everyone has their turn.


Try an app_config.xml file in each Project folder that looks like this:

<app_config>
<project_max_concurrent>2</project_max_concurrent>
</app_config>

Then change the number, 2 in this case, to reflect how many tasks you want that project to run at a time. Keep the cache settings in mind as you do this and you can run 2 tasks from this project, 1 from that project and 12 from that project over there all at the same time day after day after day.
11) Message boards : Number crunching : Not getting any python work (Message 103368)
Posted 17 Nov 2021 by mikey
Post:
The standard Rosetta tasks also reserve 8gb of ram for each task,
No they don't.
They request & release RAM as required- none is reserved. And apart from a batch of faulty Tasks some time back, the most i have seen used by a single Task was around 4GB. Generally the highest is around 1.3GB.
The current batch of work are using between 700MB & 1GB each.

Please stop making thing up, it's not helpful.


I didn't 'make it up' it was my misunderstanding that the 4.20 tasks 'request & release ram as required'. I have only ever looked in the properties of a running task and see what it says so was going by that.
12) Message boards : Number crunching : Problems and Technical Issues with Rosetta@home (Message 103367)
Posted 17 Nov 2021 by mikey
Post:
I am out, until someone can confirm that the pythons are working correctly, and I can select them.


Part one of that is no problem.
I returned at least two proper results before the one I was complaining about.
Unfortunately I can't show them off anymore, as the db seems to have taken them out by now.
vbox can be a hassle on every project, and I encountered similar problems elsewhere.
Dealing with projects that have nothing except vbox apps is one thing, but Rosetta isn't one of those.
So the possibility to unselect those monsters should obviously and absolutely be existing!


I can confirm that the Python tasks work just fine on my Windows 11 beta testing laptop, I am running ver 6.1.26 of VBox and ver 7.16.20 of Boinc:

1450357706 1293408677 4767328 17 Nov 2021, 0:09:28 UTC 20 Nov 2021, 0:09:28 UTC In progress --- --- --- rosetta python projects v1.03 (vbox64)
windows_x86_64
1450297317 1293408267 4767328 17 Nov 2021, 0:09:28 UTC 20 Nov 2021, 0:09:28 UTC In progress --- --- --- rosetta python projects v1.03 (vbox64)
windows_x86_64
1450357611 1293408634 4767328 17 Nov 2021, 0:08:51 UTC 17 Nov 2021, 12:13:59 UTC Cancelled by server 0.00 0.00 --- rosetta python projects v1.03 (vbox64)
windows_x86_64
1450286960 1293407997 4767328 17 Nov 2021, 0:08:03 UTC 20 Nov 2021, 0:08:03 UTC In progress --- --- --- rosetta python projects v1.03 (vbox64)
windows_x86_64
1450357628 1293408478 4767328 17 Nov 2021, 0:08:03 UTC 20 Nov 2021, 0:08:03 UTC In progress --- --- --- rosetta python projects v1.03 (vbox64)
windows_x86_64
1449564483 1293454887 4767328 15 Nov 2021, 23:25:48 UTC 17 Nov 2021, 12:12:47 UTC Completed and validated 22,488.82 20,606.27 185.74 rosetta python projects v1.03 (vbox64)
windows_x86_64
1449564347 1293454813 4767328 15 Nov 2021, 23:25:48 UTC 18 Nov 2021, 23:25:48 UTC In progress --- --- --- rosetta python projects v1.03 (vbox64)
windows_x86_64
1449486504 1293416591 4767328 14 Nov 2021, 3:21:18 UTC 17 Nov 2021, 5:45:55 UTC Completed and validated 34,804.85 31,782.95 441.35 rosetta python projects v1.03 (vbox64)
windows_x86_64
1449487643 1293416839 4767328 14 Nov 2021, 3:20:38 UTC 16 Nov 2021, 13:32:46 UTC Completed and validated 40,661.08 40,217.94 515.61 rosetta python projects v1.03 (vbox64)
windows_x86_64
1449487583 1293416824 4767328 14 Nov 2021, 3:20:38 UTC 16 Nov 2021, 19:56:35 UTC Completed and validated 22,768.22 20,693.41 186.15 rosetta python projects v1.03 (vbox64)
windows_x86_64
13) Message boards : Number crunching : Not getting any python work (Message 103342)
Posted 16 Nov 2021 by mikey
Post:
I see I'm not the only one who wished there was an option in the preferences as to which work units I want!

I have a Virtual Box installed but even when it's closed, BOINC downloads Python wu's. These are MASSIVE memory hogs at around 7.6gig each. As a result, I have both standard & Python running but w/u's queued up due to insufficient memory. (I have 32gig)

If I had a choice, I'd only enable the standard wu's instead of the Python one's but there's no way to toggle one or the other. To be honest, I aborted the work and moved onto another project. That's not a win for Rosetta@Home.
I guess I could just let my pc's crunch as much as possible and leave it be. I'd still like to see an option in the preferences.


The standard Rosetta tasks also reserve 8gb of ram for each task, one way around this is a simple app_config file that limits the number of tasks running per project, like this:

<app_config>
<project_max_concurrent>1</project_max_concurrent>
</app_config>

Put it in the Project folder and then tell Boinc to reread the config files and you are good to go. It's also useful for running 3 tasks of this project and 5 of that project on an 8 core cpu. You do need to be careful as each Project will download work thinking it's for all 8 cores so it's best to limit the cache size accordingly otherwise you could easily run into task deadline issues.

But it does not do anything about not getting Python over regular Rosetta tasks, that's a Project issue and they haven't done anything yet.
14) Message boards : Number crunching : High number of invalid tasks (Message 103341)
Posted 16 Nov 2021 by mikey
Post:
An update:
After switching back to running RAH on just one core, all invalids are gone.
I now have 27 Consecutive valid tasks.

@mikey, it looks like you're spot on with the memory requirements.
The other 7 cores now run FAH and WCG and because i only have 16gb on this system, just 1 runs RAH.


And that's exactly why I also limit Rosetta tasks to one at a time on all my machines, most only have 16gb of ram anyway.
15) Message boards : Number crunching : Excessive workunit fetch (Message 103340)
Posted 16 Nov 2021 by mikey
Post:
On macOS those python VB tasks take much much less than 8 Gb (I would 13 Gb free when it was running) + as I explained, I had limited them and only 1 was running at a time (the rest was regular rosetta).


The regular Rosetta tasks also reserve 8gb of ram per task, so if you are trying to run 20 tasks total, or near that and all of them are Rosetta tasks your machine doesn't have enough memory. Yes I know Mac's use less memory but I don't remember how much, you can check by clicking on a running task in the Boinc Manager and then in the left hand panel click on Properties and it will tell you how much the Virtual Memory Size is and the Working set size is, you care about the one that shows the most memory.
16) Message boards : Number crunching : Excessive workunit fetch (Message 103325)
Posted 16 Nov 2021 by mikey
Post:
I'm not sure if you mean "don't bother too much and change project for now" or "there are useful tips in that other thread" ? I had a (very) quick look and it's long and complicated, not even sure if the problems they mention are similar ?

In any case it'll have to wait, for now I have dropped all those many tasks I had (running or not) and I'm focusing on other projects for the moment.

Thanks !


Since each task here at Rosetta takes 8gb of ram and you only have 40gb of ram in that pc you need to limit the cache and extra cache settings on this pc and limit the total number of task you run at one time to no more than 4 at one time and depending on what else you do with that pc fewer than that. It seems to me with 20 cpu's in that pc and a cache size of even 1 day could mean 120 tasks if it used all the cpu cores at once. Of course you can't use 20 cores at once as your pc would be swapping memory like crazy and destroying your hard drives and pushing your memory to it's limit in the process.

One key thing, to me you said. it that it was only advancing at 0.01% even after 48 hours of crunching, that means you are trying to run to many tasks at the same time, again at 8gb of ram per task you can only 4 at a time max because the pc itself uses some overhead with Windows using more than most Linux versions.
17) Message boards : Number crunching : Excessive workunit fetch (Message 103191)
Posted 12 Nov 2021 by mikey
Post:
Hi,

I have the exact same problem : I was forced to limit the python tasks to 1 at a time using app_config (recommended in another thread on this forum) because I would get many problems with those tasks on my iMac (with lots of resources : core i9 + 40 Gb RAM, but they stall / don't stop to crunch after almost 2 days and go over the deadline, ... ) and now it downloaded almost 1000 tasks with 2 days deadline (regular rosetta, not python where it only got 9 waiting to crunch).

So this is "not rosetta's fault" ? I never got such a problem on any other project and it is not the first time I limit number of tasks (I used to do it on LCH back in the days when they didn't have much options in project setup).

And rosetta offers no setup whatsoever with project selection.

I feel much more that *it is* "rosetta's fault" actually.


Go back thru your logs if you still have them and see how much work Boinc requested from Rosetta and compare it to how much you actually received, it sounds like Boinc asked for ALOT more than it should have and Rosetta responded. Boinc Projects don't decide on how much work to give you they respond to a request by the Boinc Manager and then try to give you as much as you ask for. You running an app_config file to limit the number of tasks you run at one time does not currently fit into the calculations when Boinc asks for work, it uses the idea that every single one of the cpu cores you allow Boinc to use will be crunching the tasks the project sends you, THEN the app_config files takes over and does what you tell it to do.
18) Message boards : Cafe Rosetta : What is your Distributed Computing Story? (Message 103146)
Posted 10 Nov 2021 by mikey
Post:
I started crunching when my sons friend should me his computer with Seti running on it in the late 80's and as they say the 'rest is history'. I too have joined most Boinc Projects over the years and am now hunting hours at each project by getting new and bigger badges at wuprop https://wuprop.boinc-af.org/ that tracks the actual hours your computer crunches for each application at each project. I also did the Bitcoin Utopia Boinc Project for a bit even buying a few Asic miners to get even more Bitcoins that were then donated to other Boinc Projects and other things the Admin thought was worthwhile. Crunching for Bitcoin Utopia gave me the credits to rise to #3 in the World and I'm very proud of that achievement. I am starting to fall behind though as the new gpu's have such astronomical costs despite me growing from a single pc to 17+ pc's that comprise my Farm. I have also started using Raspberry Pi's due to their low cost compared to more traditional pc's but they are not nearly as much fun to watch crunch as a 16 core pc going thru tasks like no tomorrow!! Long ago I realized that the screen savers most Projects have use cpu cycles that can be used for crunching if you don't use them so for years I have used the power button to turn off the monitors.
19) Message boards : Number crunching : problem with app_config.xml (Message 103144)
Posted 9 Nov 2021 by mikey
Post:
Fortunately Rosetta does not punish us for aborting the tasks we don't want to run so feel free to abort all the regular tasks and keep trying to get the Python tasks you do want.
For F*&^% sake don't.

kotenok2000- If you don't want to process a particular type of Task for a particular project, and that project doesn't give you the option to choose between Tasks types, then better to not do any work for that project than download work just to abort it.



To process only a particular work type requires the use of app_info.xml where you need to put in the all the details of the project, the work type, the application, the files needed by the application etc. And you will need to manually change that & download & install the new application & files when a new application comes out.
This is what's known as Anonymous Platform- and as far as i know, Rosetta doesn't support it.


So, if you only want to do only Python work, and aren't prepared to do any of the Rosetta 4.20 work, then i suggest you detach from Rosetta as it's not a suitable project for you.


It's about priorities, and we all make choices all day long so making them in Boinc is okay too. I UNDERSTAND where you are coming from but the guy only wants to run Python tasks, like I did awhile back, and I gave him an option....MOST people running Rosetta don't care what kind of tasks they get and just love running them. YES it WOULD be a problem if everyone started aborting regular tasks to only get Python tasks but for those people that REALLY care it doesn't make a blip in the resends. But if it really came to that the Project would take the action of penalizing us for aborted tasks and to most people it would make any difference at all.
20) Message boards : Number crunching : High number of invalid tasks (Message 103131)
Posted 8 Nov 2021 by mikey
Post:
Lately i have a high number of validation errors.
Like this one:
https://boinc.bakerlab.org/rosetta/result.php?resultid=1446981031
Nothing changed on my end, and the system is running fine and stable as always.
This system also runs a number of WCG tasks without any problems or errors.
Also the reported runtime of 2 minutes is not correct.
This task ran 4+ hours before being invalidated.
It's a win10 pro system with an amd ryzen 7 2700 with 16Gb of memory.
It also runs FAH tasks GPU and CPU without any roblems.

Anyone else have a growing number of validation errors?


Each Rosetta task blocks off 8gb of memory for itself, so the problem could be your other tasks, and whatever else you do with the pc, trying to use more than the remaining 8gb and if they also push the envelope then they are both trying to use the same blocks of memory and invalid units are inevitable. For me on my Win10/11 pc the WCG Africa Rainfall tasks use 5gb each, I don't run FAH so don't know how much memory it uses.


Next 20



©2024 University of Washington
https://www.bakerlab.org