Message from server:rosetta python projects needs 19073.49MB more disk space

Message boards : Number crunching : Message from server:rosetta python projects needs 19073.49MB more disk space

To post messages, you must log in.

AuthorMessage
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104682 - Posted: 6 Feb 2022, 20:39:13 UTC

Time to start my own thread on this problem,
This thread is for when you get disk space messages in event log like this even on computers that can run python work units ok

Message from server:Rosetta needs 1907.35MB more disk space.You currently have 0.00 MB available and it needs 1907.35 MB. - This is a rare one
Message from server:rosetta python projects needs 19073.49MB more disk space.You currently have 0.00 MB available and it needs 19073.49 MB.

This is on a twin xeon 48cpu/thread system with a 500GB ssd dedicated to boinc and boinc is the only thing instaled on it [drive `F`]
A question is , do systems with less than 16 cpu`s get the "disk space" messages like these ?
one of my other crunchers a 16 cpu opteron it gets them with plenty of ram and disk space

these are the two variations of the message that i have had and have written about it in other threads
that are now lost somewhere in among all the other threads and posts {ok I know I can find them if I try}
The values for `needs xxx MB more` and `you currently have xxx MB` can vary a lot in a small space of time
The end of the "Rosetta 1907.35MB" or "rosetta python projects 19073.49MB" part of the message`s are always the same
Even if you have plenty of gigabytes of disk space "available" in the `disk` tab of boinc manager

There does not seem to be anything i can do about it
And to try and fix the problem i have recently fitted a 500GB Sata SSD and instaled only boinc on it, so it has exclusive use of the disk
It was running on a sata3 mechanical drive [ C ]
Disapointingly it did nothing to cure the messages, yet.
It will also speed up work unit loading times in case that had anything to do with it [or anything else], no affect ,yet.
And if boinc eats the disk to death loading the vdi files to the slots folder on starting a work unit I will see how long a 870 evo ssd lasts

I have tried various settings in boinc manager including unticking all the setting boxes for disk space use in the hope to give unlimited use of the disk to boinc.
now , today , i am trying settings of `just chuck everything at it` I dont usualy bother with the web settings though have done both to be shure i have not missed anything
{ I know that local settings should override web settings , I just did it for the hell of it}
and have set local and web prefs to :-

Use no more than 500 GB . . all of the disk, more than is realy available after formating
Leave at least 1 GB free . . Default setting of little importance
Use no more than 100 % of total . . seems a good idea

just to see what happens.
it now states "max disk usage: 464GB" in event log and i will see what happens.
boinc will only use what it needs to

What i would like you to do is post here how much disk space you have `available to boinc` [in the boinc manager pie chart]
How many gigabytes Rosetta is using
and how many cpu/threads you have
when you get the "disk space" message
and anything else that may be of use to find a fix for this.

but dont copy/paste long log file outputs just snipets will do , a few lines will do
ID: 104682 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1484
Credit: 14,632,255
RAC: 12,475
Message 104687 - Posted: 6 Feb 2022, 22:47:16 UTC

System with 6c/12t, but only 3 Tasks presently running.
I only do Rosetta 4.20 work.
1TB SSD

Computing preferences settings
Disk
Use no more than 20 GB
  Leave at least  2 GB free
Use no more than 60 % of total

BOINC Manager disk tab
               Used by BOINC:   2.02 GB
    Free, available to BOINC:  17.98 GB
Free, not available to BOINC: 783.16 GB
      Used by other programs: 127.35 GB

                Rosetta@home:   1.58 GB
The most i have ever seen Rosetta@home use is 2.5GB on my other system (which was attached to BOINC before Rosetta 4.20 was released & while Rosetta Mini was still running, so it still has those older executables & support files on it).
Grant
Darwin NT
ID: 104687 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104690 - Posted: 7 Feb 2022, 1:52:11 UTC
Last modified: 7 Feb 2022, 2:08:35 UTC

Thanks for posting
The rosetta 4.2 play nice and don't do my head in
It shure is the pythons that eat everything
in the few hours since changing the settings I have not had any new `message from server`
it will be a few days before I will trust its fixed
on these systems the rosetta folder itself is 11GB the vdi files are most of that

one thing I have found is that with the settings of :-
Use no more than 500 GB
Leave at least 1 GB free
Use no more than 100 % of total
when saving those settings in boinc manager the "% of total" box unticks itself and the "100%" disappears and the box greys out, and I did this on two systems to see if it was realy happening
so now I have gone and unticked the `leave at least` box and that 1GB was added to the usable disk space, so now its 465GB,
on the web settings it did no do that, so I changed it to match anyway [not that it matters]
So it seems the only setting that needs to be set is the `use no more than` and the other two can be unticked safely to get most possible disk space usable for boinc.
O I do like to meddle :-)
even if I do end up doing my head in with it . . .

And now [only running rosetta] for the first time the xeon system is running 43 tasks [set to use 90% of cpu`s] using 77284MB of ram and an incredible 335GB of disk space with 130GB free.
my bwain hurtzzz ;-),
ID: 104690 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tullio

Send message
Joined: 10 May 20
Posts: 63
Credit: 630,125
RAC: 0
Message 104697 - Posted: 7 Feb 2022, 17:06:03 UTC
Last modified: 7 Feb 2022, 17:07:25 UTC

I have an AMD Ryzen 5 1400 CPU with 8 processors and 24 GB RAM, plus a GTX 1060 GPU Board, rarely used . It runs most World Community Grid tasks plus a Linux Virtual Machine with 8 GB RAM and 30 GB disk, running SuSE Linux Tumbleweed with a 5.16.4 kernel frequently updated. If I try to run rosetta python tasks it iells me that it needs 19 GB of disk space and I have only 10 free for BOINC, so I don't get any, but I get Rosetta 4.20 tasks when available.
Tullio
ID: 104697 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104710 - Posted: 7 Feb 2022, 20:19:52 UTC
Last modified: 7 Feb 2022, 20:48:58 UTC

Well I have had only one day on the new settings and no disk space messages , still not trusting that its fixed for me.
Rosetta 4.2 is user and system friendly and uses relatively small amount of disk and ram [normaly , unlike the `horns` tasks from last year]
Pythons eat everything :()
With 8cpu`s and 24GB ram you should get some tasks, how much space is available to boinc ?
So what size is the partition , presumably `home` that you installed boinc in.
What is the total size of the drive and partition layout ?
Do you have any unpartitioned space on the drive you could extend into
If all is full it is a problem that can be fixed if you have another unused hard drive , even a small one that you can put in and extend `home` onto it without moving partitions about.
Without knowing what size of partition or drive size you have i am stuck

What I have done that workz so far on windows systems with only one partition on the disk where boinc is installed to maximize available disk space
in bionc managers disk tab try this setting :-
Use no more than 500 GB . . [the total size of the disk its on , this also works on my other system that only has a 250GB disk , the event log states 194GB available]
Leave at least ## GB free . . [untick this box not needed]
Use no more than ## % of total . . [untick this box not needed]

And by the look of it I don't need to mess with the web settings anyway
Boinc will only use what it needs
This is a `work in progress` for me as well.
ID: 104710 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104722 - Posted: 8 Feb 2022, 2:20:15 UTC

I put those settings into my Linux Mint19 cruncher to see what happened and the result is the same , should work on suse
Its only got a old 80GB total disk size and a Q9550 to play with and the `home` partition is only 52GB and boinc can see and use all of it if needed.
ID: 104722 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104879 - Posted: 17 Feb 2022, 21:49:10 UTC
Last modified: 17 Feb 2022, 22:10:33 UTC

Time to wrap up this thread with a fix for the problem that has worked for the last week with no `Messages` .
Pythons eat everything :()
In bionc managers disk tab try this setting :-

Use no more than - 500 GB . . [the total size of my disk its on , this is so I can run 45 work units together , [I have now seen Rosetta using up to 352GB of disk and 80GB of RAM] don't worry about setting this BIG.
Leave at least ## GB free . . [untick this box not needed]
Use no more than ## % of total . . [untick this box not needed]

You don't need to mess with web settings.
Boinc will only use what disk space it needs
This works for me , so try it if you get "Message from server" problems.

And if you have a system that can run more than that `do the math` . . . .
And thank you to Grant(SSSF) for encouraging me to try other settings till I found something that works.
ID: 104879 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5664
Credit: 5,705,524
RAC: 1,705
Message 105109 - Posted: 21 Feb 2022, 20:06:18 UTC

.clair. its a bug.
The task will complete ok.

I had the same thing. I told you guys about it in the first thread.
I was told to go to Github and report, I did.
I talked with one specific German and he said he would look into it after I showed him the screenshot of everything.

Your drive is set how in the preferences? Everything unchecked except leave x GB free?

I have 500GB and I set preferences for leave 2GB free and after partion reserve space I have 465GB free.
Both Python and SiDock coughed up disk errors, but completed ok as far as I know.
I was running 15 Python and cycling every hour to 15 SiDock tasks.
It's not specific to Vbox since SiDock is not Vbox.
So its something weird in BOINC.
ID: 105109 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 105215 - Posted: 25 Feb 2022, 21:42:01 UTC

To arrive at the settings I use , In my experimentations I found things like if I set :-
Use no more than - 500 GB 
Leave at least - 0 GB free
Use no more than - 100 % of total

The "Leave at least" and "Use no more than"
Boxes untick themselves when saving leaving the values boxes blank , od behaviour ,
It will let me go as far as "Leave at least - 0.01 GB free" but rejects 0.001 and then unticks the boxes itself and blanks the box when saving .
The % box can go blank at times for some reason
In the long run It works ,
If you want to have something in all three boxes this setting also works and gets the maximum use of the disk with all three boxes used , if you [whoever you are that is reading this] insist on it .
Use no more than - 500 GB - Just set this to something BIG
Leave at least - 0.01 GB free
Use no more than - 99.9 % of total

Each python task wants {rather than actualy using} about 8GB of disk space . don't forget the Rosetta project folder is nearly 12GB and that is part of the "used by boinc" calculation .
Allow a big bit for any other boinc projects you are running and that should work .
On one of my systems with 45 python tasks running Rosetta/boinc was using 357GB of disk space ,
and 86GB of memory .
Pythons eat everything :()
ID: 105215 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5664
Credit: 5,705,524
RAC: 1,705
Message 105216 - Posted: 25 Feb 2022, 23:29:50 UTC - in response to Message 105215.  

Out of all those option just use this: Use no more than - 500 GB
Uncheck the other two as they just give you other options, but logically should not be combined with the first as using them would counteract what you said in the first box.

If you look through the discussion in the problems thread, you will see I was told to use the first box, another person I think is using the percent.

The problem is with leaving at least 0 GB free. We see it as unlimited, the program has a default value that is not large enough for what we do here. This is why you need to use the Use no more than setting on it own.

This is a already known issue in BOINC which they have not corrected or forgot to correct.
It dates back a couple of years.
But they are aware of it again.
ID: 105216 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 108399 - Posted: 30 May 2023, 20:44:28 UTC
Last modified: 30 May 2023, 21:21:04 UTC

Oh ,
well there you go ,
I can still post in my thread ,
so I will bump it up the forum so it can be seen as the problem is still lurking in the config files .
ID: 108399 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Message from server:rosetta python projects needs 19073.49MB more disk space



©2024 University of Washington
https://www.bakerlab.org