Looks like more Rosetta 4.2 work.
Received 4 west_nile tasks.

---

One more data point:
aagb-SAR-PHE-ACHC13T-mB3PHG_pp_2_2704977_1_1
According to task manager, VirtualBox Global Interface uses 0.5MB, VirtualBox Interface uses 2.3 MB, VirtualBox Headless Frontend uses 28.8, two other instances of VirtualBox Headless Frontend both use less than 1MB.

Still can only get 1 Rosetta Python to run despite me actually have more than 7GB available (my settings mean that less than 6GB is available to BOINC). I guess these tasks really want 6GB of available RAM to run. The most I can do in theory is indeed 2.

Why do these tasks have such absurd RAM requirements? Can anyone point me to a Python task that actually used more than 1GB of ram? I have 4 WCG tasks running, one ARP & three MCM, one Rosetta Python task, one YouTube video running, two chrome tabs open, and a generous smacking of background processes, I still have 7GB of free RAM.

nevermind

Nevermind.

I literally just got 4.20 work. All west_nile except for 1 zika. Guess we're working the flaviviruses.
Admin made an appearance on the forums - https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14771&postid=103413

---

I have lots of disk space, they download ok, on a 32GB system 16 cpu, it usualy runs three, once it ran fore for a while,
{in BM memory is set to use 99% when active, 100% when idle}

When you've next got 3 (or even 4) running at a time, could you please check in Windows Task manager to see just how much RAM is actually being used by those Tasks & Vbox in total?
Completed Tasks are reporting less than 100MB being used, Tomcat's posts show that only a few hundred MB is being used (if that) by the Tasks & Vbox combined for a single Task when running. If that is the case, the 7.5GB+ minimum requirement is beyond excessive & counter productive. 750MB would appear to be more than enough.

---

Grant
Darwin NT

Have any extra Python Tasks started up with the extra storage space made available?

All the tasks got downloaded when there was at least 280GB of free disc space. Still, only one runs at a time with all the other ones waiting. Sometimes, BOINC will pause a task to start a new one.

Oh well, it was worth a try. Might as well put things back the way they were
Thanks for your efforts.



If more people with running Python Tasks could let us know just how much RAM Vbox & the Tasks are actually using (amount of RAM and number of Tasks), maybe we can get the minimum RAM requirement changed to something more realistic, and get more work done & make better use of the resources people have made available.
And it might help with the issue of some Tasks sitting there & not running, and old VMs accumulating & needing to be cleared out manaully.

---

Grant
Darwin NT

If more people with running Python Tasks could let us know just how much RAM Vbox & the Tasks are actually using (amount of RAM and number of Tasks), maybe we can get the minimum RAM requirement changed to something more realistic, and get more work done & make better use of the resources people have made available.

To help answer this, I have been running 12 pythons on a Ryzen 3900X for an hour. This is an Ubuntu 20.04.3 machine with 96 GB of memory.
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=6155880
These are the only BOINC tasks running.

BoincTasks shows the memory reserved per work unit:
https://postimg.cc/hzKhjPrz

free shows the amount of memory used and available:
https://postimg.cc/MMtHj2Ps

top shows the amount of memory used per work unit as a percentage of the total:
https://postimg.cc/m1DDYXKT

I hope this helps you to convince them that they can reduce the amount that they require.
They have other problems too, but if people can't run them, they will never find them.

---

To help answer this, I have been running 12 pythons on a Ryzen 3900X for an hour. This is an Ubuntu 20.04.3 machine with 96 GB of memory.
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=6155880

Exclent.
I'm not familiar with LINUX, so if you could help me out further that would be great.

BoincTasks shows the memory reserved per work unit:
https://postimg.cc/hzKhjPrz

So each Python Task reserves 7,629.39GB RAM (so 7.7GB is close enough).

free shows the amount of memory used and available:
https://postimg.cc/MMtHj2Ps

So out of 96GB of total system RAM, only just over 20GB is actually in use- for the entire system (OS, BOINC, all other running programmes, services, etc, etc).

top shows the amount of memory used per work unit as a percentage of the total:
https://postimg.cc/m1DDYXKT

So with the current batch of Python work, the most a single Task is actually using is only 1.5% of the total available system RAM. Let's say 1.7% to give it a nice large safety margin.
So that should give us 1.64GB (96492.28MB * 1.7%) is all that's really needed per Task to run (the actual in use value being less than 1.5GB). Which is only a little over 300MB more than the minimum per Task processed RAM requirement i advocate for Rosetta 4.20 Tasks to avoid running in to memory limitation issues.

So they can reduce their minimum RAM requirement by 6.06GB, with no adverse effects (and from what little reading up i've done on VBox, it will ask for more RAM from the Host OS above & beyond a VMs minimum requirement setting if it needs it anyway, if it can't get it, then that's when Virtual Memory comes in to play).

I hope this helps you to convince them that they can reduce the amount that they require.

We can only hope...

They have other problems too, but if people can't run them, they will never find them.

I suspect that many of those issues could be related to it hitting that excessive minimum RAM requirement. Get rid of that, and there's a chance quite a few of those other issues will go away as well. Only one way to find out.


Thanks for your efforts.

---

Grant
Darwin NT

You are welcome. I think your analysis is correct of the Linux part.

But they must have picked such a large amount for a reason (or at least I hope they had a reason).
So I would allow them 4 GB just to get the ball rolling, and reduce that later if necessary.
But maybe they aren't Linux experts either (not that I am, but I can use it).

Lots of luck.

---

I have lots of disk space, they download ok, on a 32GB system 16 cpu, it usualy runs three, once it ran fore for a while,
{in BM memory is set to use 99% when active, 100% when idle}

When you've next got 3 (or even 4) running at a time, could you please check in Windows Task manager to see just how much RAM is actually being used by those Tasks & Vbox in total?
Completed Tasks are reporting less than 100MB being used, Tomcat's posts show that only a few hundred MB is being used (if that) by the Tasks & Vbox combined for a single Task when running. If that is the case, the 7.5GB+ minimum requirement is beyond excessive & counter productive. 750MB would appear to be more than enough.

I had no 4.2 work , so RPP got the lot to play with [no other projects running, qued etc]
At that time I did, and if I was`nt wrong , the largest was 115MB
A VM built in 750MB would plenty from anything I have seen, one gerbil bite if they want to be wild ;)
By the look of it the RPP tasks actual use is the smallest amount of memory of any R@H task in years.

Looks like more Rosetta 4.2 work.
Received 4 west_nile tasks.

And it looks like that batch is done.
Back to Python only again (apart from the odd RB Task or resends).

---

Grant
Darwin NT

I give up on trying to get Python
Your app_config.xml file refers to an unknown application 'rosetta_python_projects'. Known applications: 'rosetta'

That's from that other guy's script in the python thread.
If they can't keep 4.2 running then I'm dumping this project after 15 years of crunching.

I give up on trying to get Python
Your app_config.xml file refers to an unknown application 'rosetta_python_projects'. Known applications: 'rosetta'

That's from that other guy's script in the python thread.
If they can't keep 4.2 running then I'm dumping this project after 15 years of crunching.

That message is because you haven't received a Python task yet.

---

I give up on trying to get Python
Your app_config.xml file refers to an unknown application 'rosetta_python_projects'. Known applications: 'rosetta'

That's from that other guy's script in the python thread.
If they can't keep 4.2 running then I'm dumping this project after 15 years of crunching.

That message is because you haven't received a Python task yet.

Falconet, I won't get any anway..they blocked my system.
Go look in my thread, I just responded back to Admin about a quick and dirty fix to a problem they found.
Rather than try to communicate with "volunteers" or make a thread about the problem, they just made a quick and dirty script to block systems that kicked back certain errors.

It seems my system kicked back one of those errors, so I am blocked.
They can' keep 4.2 filled, so it is rather pointless to keep pinging their scheduler for work that is not there and work I can not access.

I reattached Rosetta, after I no longer got any WUs.
Not only on my Android device, but also my intel x86 units.

Could it be that the Python tasks are only for teaching their younger students how to create tasks?

So far, Rosetta@Home has not mentioned any other purpose for them.

By the way, I tried a Google search for vbox64 instructions, to see how hard it would be to change the amount of reserved memory. I found instructions, but they did not mention the amount of reserved memory. Some of you could ask Oracle to add a feature for controlling the amount of reserved memory.

Could it be that the Python tasks are only for teaching their younger students how to create tasks?

The only thing that I recall is that one of their experimenters was using them, and I don't know for what purpose.
It was not exactly a ringing endorsement that they would be widely used as a basis for new work, as we might hope.

If anyone can find a mention of a use, maybe they could post it.

---

Hi guys,

I do not know if this is the right thread but I have a problem for two days - I do not download the usual Rosetta units. I guess that this means a definitive transition to units under the Virtual Box? And here comes my problem- I installed BOINC from Virtual Box and downloaded me about 20 units. Unfortunately, it counts only 4 at a time and 16 I have as ready to start. I have absolutely all the resources allocated to Rosetta, 12 threads and 18GB of free RAM. What can I do to make it count on all 12 threads?

Since now each process is a virtual machine I checked in the Virtual Box - each machine has reserved 6GB so it would be correct: 6x4 = 24GB and the next one does not fit to allocate because it uses the system and my other apps. Why as much as 6GB per Linux image? Can you set somewhere globally at 4GB per machine? This is some kind of massacre that now you need 128GB of ram to use all threads on a stupid 5600X ...

Thank you in advance for your help.
Mirek

---

Hi guys,

I do not know if this is the right thread but I have a problem for two days - I do not download the usual Rosetta units. I guess that this means a definitive transition to units under the Virtual Box? And here comes my problem- I installed BOINC from Virtual Box and downloaded me about 20 units. Unfortunately, it counts only 4 at a time and 16 I have as ready to start. I have absolutely all the resources allocated to Rosetta, 12 threads and 18GB of free RAM. What can I do to make it count on all 12 threads?

Since now each process is a virtual machine I checked in the Virtual Box - each machine has reserved 6GB so it would be correct: 6x4 = 24GB and the next one does not fit to allocate because it uses the system and my other apps. Why as much as 6GB per Linux image? Can you set somewhere globally at 4GB per machine? This is some kind of massacre that now you need 128GB of ram to use all threads on a stupid 5600X ...

Thank you in advance for your help.
Mirek

The Python Rosetta@Home tasks now available reserve 7.45 GB of memory each, so you'll need that much memory per task plus a gigabyte or two for things outside the tasks. The tasks actually use closer to 100 MB, though.

The usual Rosetta tasks are now in high demand from computers that don't have that much memory available, so you'll get rather few of them.

You can join into the effort to persuade the Oracle company that provides VirtualBox to add a few features such as the ability to set much lower amounts of memory reserved.

I was able to increase the number of the Python tasks that would run on my computer at the same time from 1 to 3 by increasing the fraction of the total memory BOINC could use, and then also increasing the fraction of the total disk space BOINC could use. It still won't use all 8 of the virtual cores I allow BOINC to use, since the computer has only 32 GB of main memory. I'm thinking of buying more memory is this problem lasts long enough, but I'm not in a hurry to do so.

I've always had this problem, but it's far worse with python tasks.

aaam-NHM_pp-mTIQ_pp-SAR-AMACBEN2_pp_6_2539600_1_0

Instead of running this task, BOINC keeps downloading new Rosetta tasks and running those. Looking at all the tasks on that computer and you'll see this has nothing to do with queue size. It's almost as if BOINC periodically forgets a task exists. It's worse with Python tasks because BOINC will download the same number of tasks as the available threads but will be able to run at most 2 at a time. It's thus far more common to get clogs and "forgotten" tasks.