Hello!

To do my tiny bit during the COVID nightmare I decided to fire up some VMs exclusively to run Rosetta. I also plan on running it on (capable) local PCs too (they already crunch other stuff).

Because you pay for EVERYTHING when running a VM then it made sense to me to use a filesystem that could make things "nicer". In steps BTRFS with compression and deduplication. An initial 10 GB disk filled (though in hindsight it probably didn't), until deduped and compressed down to ~800 MB.

Unfortunately, this didn't actually help much.

[Rosetta@home] Rosetta needs 582.77MB more disk space.  You currently have 1133.85 MB available and it needs 1716.61 MB.

Hmm... This number makes no sense in any context.

$ df -hl
$ df -hl
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       9.6G  2.2G  7.4G  23% /
devtmpfs         16G     0   16G   0% /dev
tmpfs            16G     0   16G   0% /dev/shm
tmpfs           3.2G  968K  3.2G   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            16G     0   16G   0% /sys/fs/cgroup
/dev/sda15      105M  3.9M  101M   4% /boot/efi
/dev/loop0       99M   99M     0 100% /snap/google-cloud-sdk/128
/dev/loop1       69M   69M     0 100% /snap/lxd/14804
/dev/loop2       55M   55M     0 100% /snap/core18/1705
/dev/loop3       28M   28M     0 100% /snap/snapd/7264
/dev/sdb         20G  736M   19G   4% /var/lib/boinc-client/slots
tmpfs           3.2G     0  3.2G   0% /run/user/1001

$ sudo btrfs de u /var/lib/boinc-client/slots
/dev/sdb, ID: 1
   Device size:            10.00GiB
   Device slack:              0.00B
   Data,single:             4.01GiB
   Metadata,RAID1:        256.00MiB
   Metadata,DUP:          512.00MiB
   System,RAID1:           32.00MiB
   System,DUP:             16.00MiB
   Unallocated:             5.20GiB

/dev/sdc, ID: 2
   Device size:            10.00GiB
   Device slack:              0.00B
   Metadata,RAID1:        256.00MiB
   System,RAID1:           32.00MiB
   Unallocated:             9.72GiB

$ sudo btrfs fi u /var/lib/boinc-client/slots
Overall:
    Device size:		  20.00GiB
    Device allocated:		   5.09GiB
    Device unallocated:		  14.91GiB
    Device missing:		     0.00B
    Used:			 729.63MiB
    Free (estimated):		  18.31GiB	(min: 10.85GiB)
    Data ratio:			      1.00
    Metadata ratio:		      2.00
    Global reserve:		   4.91MiB	(used: 0.00B)

Data,single: Size:4.01GiB, Used:630.63MiB (15.37%)
   /dev/sdb	   4.01GiB

Metadata,RAID1: Size:256.00MiB, Used:12.64MiB (4.94%)
   /dev/sdb	 256.00MiB
   /dev/sdc	 256.00MiB

Metadata,DUP: Size:256.00MiB, Used:36.84MiB (14.39%)
   /dev/sdb	 512.00MiB

System,RAID1: Size:32.00MiB, Used:16.00KiB (0.05%)
   /dev/sdb	  32.00MiB
   /dev/sdc	  32.00MiB

System,DUP: Size:8.00MiB, Used:0.00B (0.00%)
   /dev/sdb	  16.00MiB

Unallocated:
   /dev/sdb	   5.20GiB
   /dev/sdc	   9.72GiB

$ sudo du -sh /var/lib/boinc-client
8.4G	/var/lib/boinc-client

As you can see I even tried adding another 10GB but it made no difference. The client has no limits set on it (i.e. it can use 100% of all disks available to it).

I also tried the latest build from the Ubuntu PPA and restarting but it made no difference at all. To be honest, I didn't expect it to.

Because of this disk space weirdness, the best I could achieve is 8 instances of R@H (8 CPUs). Adding anything more is a waste of time. I could probably force additional BOINC instances onto additional partitions, but... :-/

So, before blaming BTRFS, which I now run on all new machines... help?

For those unaware: BTRFS is expected to replace EXT2,3,4 so apps will need to work with it.

---

I realised after posting that the fault will be with BOINC, not R@H. Still looking at it.

$ sudo du -sh /var/lib/boinc-client
8.4G	/var/lib/boinc-client

Turns out the numbers just might make sense.

Data directory: /var/lib/boinc-client

Indeed. It would appear that BOINC looks at its data directory usage and compares it to do the amount available on that particular disk. Given that I mounted the slots directory on a difference disk, BOINC had a brain fart.

Moving the mount to /var/lib/boinc-client, allows BOINC to correctly calculate disk space details.

We'll see if anyone is interested in fixing this over at the BOINC bug tracker.

I don't know if it matters in your use case, but the new version of Rosetta being tested over on the Ralph (Rosetta Alpha) project has a fix that drastically cuts disk usage, sometimes by a factor of 10. It should be released to the general public pretty soon if testing goes well. Currently on v4.20 in testing.

Hello!

I just noticed this today. The disk space usage drastically reduced very recently, and I see I've received the newer application. From ~10GB to ~1GB. I checked what was going on with compsize and the 1GB is the native size (before compression, etc.). That's quite a reduction!

Very nice. It looks like it was released the same day you posted. Just-in-time releases. ;-)

---

Yes, the change to run in less disk space is part of the new v4.20 application version.

---

Rosetta Moderator: Mod.Sense