I had a quick look at some slot directories after getting the 4.15 Rosetta app on a Raspberry Pi today.

It looks to me, and I could be wrong, the app unzips the database and other files into the slot directory when it first starts. Around 1.5 to 1.8GB of files, for each and every task start. This is going to be a killer on the Raspberry Pi’s and other SD card based systems. They don’t support trim and have minimal wear levelling, not to mention they are much smaller capacities (8 or 16GB cards are common) so have much less spare blocks than say a 256GB SSD. It’s also time consuming doing this for each task start.

If I recall from a couple of years ago GPUgrid used to do the same thing, except they were copying CUDA libs which were also huge. I believe BOINC has the ability to have static files in the project directory that can be sym-linked from the slot directory which is a heck of a lot more efficient than copying them every time. This of course is only suitable if the files are not being updated by the task. Maybe the app developers can look into this.

---

MarksRpiCluster

I had a quick look at some slot directories after getting the 4.15 Rosetta app on a Raspberry Pi today.

It looks to me, and I could be wrong, the app unzips the database and other files into the slot directory when it first starts. Around 1.5 to 1.8GB of files, for each and every task start. This is going to be a killer on the Raspberry Pi’s and other SD card based systems. They don’t support trim and have minimal wear levelling, not to mention they are much smaller capacities (8 or 16GB cards are common) so have much less spare blocks than say a 256GB SSD. It’s also time consuming doing this for each task start.

If I recall from a couple of years ago GPUgrid used to do the same thing, except they were copying CUDA libs which were also huge. I believe BOINC has the ability to have static files in the project directory that can be sym-linked from the slot directory which is a heck of a lot more efficient than copying them every time. This of course is only suitable if the files are not being updated by the task. Maybe the app developers can look into this.

Hi,
I would rename this thread to something about Raspberry Pi.
You have two good points and this kind of i/o optimization may also be good for cellphone versions too.

Tom

---

Help, my tagline is missing..... Help, my tagline is......... Help, m........ Hel.....

I would rename this thread to something about Raspberry Pi.
You have two good points and this kind of i/o optimization may also be good for cellphone versions too.

Tom

It’s not specific to the Pi’s but obviously they would benefit greatly from it as SD cards are a slow storage device. Hopefully one of the mods will pass the idea on to the developers.

I’m still not sure why the project released the Rosetta 4.15 app for aarch64. Maybe the Rosetta for Portable Devices wasn’t providing enough scientific data. I’d probably remove it, but I don’t know all the facts/reasoning behind it.

---

MarksRpiCluster

Would this also help reduce RAM requirements? If multiple Rosetta threads each need the same file loading into RAM, does that memory get loaded once or multiple times separately?

---

A lot of the stuff remains in RAM, and doesn't fully get written until a checkpoint has reached.
I would recommend buying a cheap $18 SSD (64GB) that would still give you enough writes to do Boinc for the next 5 or 10 years 24/7 before wear leveling becomes an issue.

Yes, it is a very good idea.
And BOINC have right tools to do so.
And it very useful not only for SD cards devices but for regular computer with BOINC installed on HDD too. Starting a lot of R@H WUs (on many cores/threads CPUs) from HDD is a very slow process due to multiple databases unpacking and often cause errors and WU failures.

Also it can save a lot of disk space on SSDs: each running R@H WUs consume about 1 GB of disk space 95-99% of which used by copies of a same database (~960 MB, 5300 files in latest revision) common for all WUs.

But it was proposed many times for few years already (I also wrote about this several times) and nothing done on the issue.

And finally this optimization was implemented! In Rosetta v. 4.20 and later: https://boinc.bakerlab.org/rosetta/forum_thread.php?id=12554&postid=95678#95678

And it works fine and cut down disk space usage and disk I/O load by at least an order of magnitude! (> 10 times)