Oooh I got one!

https://ibb.co/T29Jn4y

---

That completed without any issues: https://boinc.bakerlab.org/rosetta/result.php?resultid=1415112699

---

Peak working set size 57.25 MB
Peak swap size 55.54 MB
Peak disk usage 7,947.27 MB

not so small on disk ^^

I finished one four days ago. BoincTasks reported the memory usage at almost 8 GB.

Rosetta@home 1.03 rosetta python projects (vbox64) afbr-PTAMBA-BEN3_pp-mNIP-TZR_pp_0_1397965_85_1 01:15:06 (01:14:23) 8/4/2021 6:40:46 PM 8/4/2021 6:42:50 PM 99.05 Reported: OK + Ryzen3900X-1 3286.57 MB 7629.39 MB

---

I'm thinking of doing another PC build to run the Rosetta Python units, specifically so it can be loaded with a lot of RAM. Is there a minimum CPU generation or OS that is required to run it? I have a pair of Ryzen 3950 units that run Linux, and are significantly constrained by 32GB of RAM, and they can get the Python work units. I also have an E3-1245v3 (Intel Haswell Xeon) with 32GB of RAM under Windows 7, but it only has picked up the normal Rosetta work units so far, even though it otherwise has the RAM for two work units.

My plan was to get a Haswell or Broadwell-based client set up under Linux with around 300-500GB of RAM, but I don't want to go through the trouble if it won't run the Python units.

I doubt we'll be out of normal Rosetta units for a while yet because I believe the Robetta server takes requests in from around the world and distributes them on r@h. I wouldn't expect all those users to start using trrosetta yet, but I might well be wrong. Would be good to hear from someone in the project.

The problem is the selection of tasks to queue for delivery to us users.

There may be millions of tasks waiting but there are only ever 29k tasks queued and that can (and have been) be skewed to be all python.

Hopefully they don't switch exclusively to vbox - or give users the ability to choose workunit types. There are doubtlessly countless machines that do not have virtual box installed and likely will not do so as they simply sit and crunch with users not checking the forums.

How much memory does each WU consume with these new python WUs? Do they finally use SSE/Avx? I don't think I have gotten one yet.
thanks

If my machine are anything to go be, if you don’t have box installed it won’t download python tasks.

The problem is the selection of tasks to queue for delivery to us users.

There may be millions of tasks waiting but there are only ever 29k tasks queued and that can (and have been) be skewed to be all python.

Mixing them all up shows that they don't understand their cruncher base, nor listen to them (no great surprise there).
I would choose all pythons on one machine, and the regular ones on another. That way you can make the appropriate memory available.
As it is, it is overkill for some work units, and not enough for others.

I was hoping for better, but have some Plan B's in mind.

---

Memory usage is a rounded 8 GB or 7.63 GB

Current server load Unsent In Progress
Rosetta 10653 344270
rosetta python projects 14759 13148

After doing some homework, I think it would be more cost-effective to bump both of my Ryzen 3950x machines from 32GB to 128GB, and tune the <max_concurrent> setting to allocate the first 32GB of RAM for non Python work units, instead of building a Haswell or Broadwell-based machine specific to the task. For my x470D4U board, it will unfortunately mean that my ECC RAM speed will drop from 2666 MHz (1-2 sticks) to 2400 MHz (4 sticks), but it's not a terrible decrease. If Rosetta's future is (a lot) more RAM, then I'll meet them where they are going.

I have a pair of Ryzen 3950 units that run Linux, and are significantly constrained by 32GB of RAM,

To avoid memory issues when running Rosetta Tasks, you need to allow roughly 1.3GB of RAM per Task being run. So you'd need 42GB RAM minimum to run Rosetta tasks on all cores & threads of that system.
Given that's a dual memory channel CPU, you'd be looking at 2*32GB DIMMs to make use of all cores & threads for Rosetta 4.20 work.
If running VirtualBox Tasks, even with the motherboard maxed out on RAM it wouldn't be possible to make use of all available cores/threads for VritualBox Tasks.

---

Grant
Darwin NT

I have a pair of Ryzen 3950 units that run Linux, and are significantly constrained by 32GB of RAM,

To avoid memory issues when running Rosetta Tasks, you need to allow roughly 1.3GB of RAM per Task being run. So you'd need 42GB RAM minimum to run Rosetta tasks on all cores & threads of that system.
Given that's a dual memory channel CPU, you'd be looking at 2*32GB DIMMs to make use of all cores & threads for Rosetta 4.20 work.
If running VirtualBox Tasks, even with the motherboard maxed out on RAM it wouldn't be possible to make use of all available cores/threads for VritualBox Tasks.

We don't know how long Python will last or what is in the pipeline for Vbox work after Python is done, so I personally would not go through all the fuss of upgrading and reconfiguring my system just for Python. Besides I don't know of any command that lets you isolate Python and reject 4.20 work. The scheduler spits out both. So how many Python and how many 4.20 tasks will you get? That is unknown.

At least they are a bit consistent, and are giving me four pythons with each update request. I can handle that nicely on this 64 GB machine.
Maybe they measure how much memory you have and send accordingly?

I have other machines with 48 GB memory, so they would work too if they can get the MD5 error fixed. We will see.

---

And I am now running four more pythons on a Ryzen 3900X with 48 GB of memory (with 24 virtual cores), with two more in the buffer.
But there aren't any on two Ryzen 3950X with 48 GB of memory (with 32 virtual cores).

That may be just the luck of the draw, but I am beginning to think that they calculate how much memory you have per core.

---

That may be just the luck of the draw, but I am beginning to think that they calculate how much memory you have per core.

If so, it would be nice if they did that with the Rosetta 4.20 Tasks as well.
Save a lot of grief with large RAM requirement Tasks on hosts with low amounts of RAM per core/thread.

---

Grant
Darwin NT

If so, it would be nice if they did that with the Rosetta 4.20 Tasks as well.
Save a lot of grief with large RAM requirement Tasks on hosts with low amounts of RAM per core/thread.

Actually, the 4.20 Rosettas are more predictable now. They seem to run around 1 GB, more or less. I don't see the large variations that they had before, when you could go from 3 TB down to 300 MB.

Maybe that is due to the new way they are pre-processing them now with the AI stuff, if that is what they are doing. They don't tell us.

---

But there aren't any on two Ryzen 3950X with 48 GB of memory (with 32 virtual cores).

I now have four pythons on one of the 3950X and five on the other, in the buffer.
So I have enough memory for them to download. Maybe they limit how many run at a time, or at least BOINC does I hope.

---

But there aren't any on two Ryzen 3950X with 48 GB of memory (with 32 virtual cores).

I now have four pythons on one of the 3950X and five on the other, in the buffer.
So I have enough memory for them to download. Maybe they limit how many run at a time, or at least BOINC does I hope.

I have an AMD Ryzen 7 4800 series cpu with 8/16 cores and 16gb of ram and Rosetta tried to run 8 Pyton units at one time, I have Boinc set to only use 50% of the cpu's. Of course everything slowed to a crawl so I made it so Rosetta only runs 1 task at a time and things are working but the 6 hour time estimate to run the tasks is not even close as I'm seeing closer to 24 hours to finish a task. BUT the Rosetta website says I'm only using 8k to 16k seconds to run a task so something is still not right.

Have you done the following:

1) checked you vbox environment for any remains of tasks completed or aborted? Occasionally you have to manually clean your vbox. ( which might explain my other thread as I just checked my vbox)

2) have you checked to make sure your version is up to date for the main program and have you also checked your extensions.

If all this is ok...I don't have answers other than what Mikey and the others go on about with memory.
I had something similar happen with ATLAS long ago when I started there (long run times and failures). But I would refer you to my discussion in the other thread so as to not clog this thread.

What machine are we talking about anyway? You have 5 computers active in the last 30 days, only 1 is running Rosetta and at the moment you are running just 4.2 tasks like me.

And what was the last Python you completed successfully?