Message: Not enough disk space when there is enough space

Message boards : Number crunching : Message: Not enough disk space when there is enough space

To post messages, you must log in.

AuthorMessage
ChristianB
Avatar

Send message
Joined: 20 Oct 05
Posts: 10
Credit: 178,513
RAC: 0
Message 55256 - Posted: 24 Aug 2008, 8:48:34 UTC

Hi,

I recently have a problem with rosetta which might be a bug at server side. I don't know how to debug this further so I want to ask if some project admin can give it a try. Here is what happens:

OS: Kubuntu 8.04 (Hardy Heron)
Client: 6.2.15 (but was there before too)
Projects:
rosetta@home: no work
seti@home: 1 task
orbit@home: no work

Message at client startup: Preferences limit disk usage to 0.47GB (means 470 MB right?)
At this moment BOINC uses 47.00MB on disk (projects use 31.70MB therefrom) and the client shows in disk tab: used by BOINC: 44.52MB and used by projects: 33.2MB the differences are ok. The actual disk size: 14GB free, 49.2GB total, 72% used, 28% free, BOINC uses 0.32% of free space. But what happens now is somehow strange:

The system is low on work and requests work from rosetta@home (3789 seconds) but gets the reply: There was work but you don't have enough disk space allocated. An additional 47 MB is needed.

There is enough disk space but rosetta is still complaining. The last time I had this error I increased the disk usage by setting a higher limit in local preferences. This shouldn't be necessary because BOINC already can use 470MB at the moment.

I already enabled sched_op_debug but the only useful info that appeared was the rosetta server version (601).

Here are some lines from the scheduler request file:
<working_global_preferences>
 <global_preferences>
   <disk_max_used_gb>0.500000</disk_max_used_gb>
   <disk_max_used_pct>80.000000</disk_max_used_pct>
   <disk_min_free_gb>1.000000</disk_min_free_gb>
 </global_preferences>
</working_global_preferences>
<disk_usage>
   <d_boinc_used_total>49059954.000000</d_boinc_used_total>
   <d_boinc_used_project>18298242.000000</d_boinc_used_project>
</disk_usage>


and here is the reply:
<scheduler_version>601</scheduler_version>
<master_url>https://boinc.bakerlab.org/rosetta/</master_url>
<request_delay>242.400000</request_delay>
<message priority="high">No work sent</message>
<message priority="high">There was work but you don't have enough disk space allocated.</message>
<message priority="high">An additional 47 MB is needed.</message>


I think it must be a server issue but rosetta is the only project where I recognized this (maybe the server version is too old but there is no way to say what revision version 601 is).

BOINC Doc | Team-Site | BOINC-Podcast
ID: 55256 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
DJStarfox

Send message
Joined: 19 Jul 07
Posts: 145
Credit: 1,239,073
RAC: 81
Message 55346 - Posted: 28 Aug 2008, 4:38:52 UTC - in response to Message 55256.  

Update your global preferences to allow 0.54 GB of disk space. Update the Project. See if that helped.
ID: 55346 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
ChristianB
Avatar

Send message
Joined: 20 Oct 05
Posts: 10
Credit: 178,513
RAC: 0
Message 55361 - Posted: 28 Aug 2008, 18:02:03 UTC

As I already said:
There is enough disk space but rosetta is still complaining. The last time I had this error I increased the disk usage by setting a higher limit in local preferences. This shouldn't be necessary because BOINC already can use 470MB at the moment.

I could do that (and had done it in the past) but its just a workaround, no solution, there must be a problem in calculating the needed space on the server. I just want to check where it is.

BOINC Doc | Team-Site | BOINC-Podcast
ID: 55361 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
DavidL

Send message
Joined: 17 Dec 05
Posts: 2
Credit: 6,805,660
RAC: 3,616
Message 55372 - Posted: 29 Aug 2008, 1:12:41 UTC - in response to Message 55361.  

As I already said:
There is enough disk space but rosetta is still complaining. The last time I had this error I increased the disk usage by setting a higher limit in local preferences. This shouldn't be necessary because BOINC already can use 470MB at the moment.

I could do that (and had done it in the past) but its just a workaround, no solution, there must be a problem in calculating the needed space on the server. I just want to check where it is.


I've got a somewhat similar problem - I get messages that Rosetta needs more disk space, but every time it attempts to get work, it's asking for more work than it did the last time, thus needing more disk space, so that even if I've freed up enough to satisfy an earlier complaint, now it needs more. Is there any way to get Rosetta to ask for less work?

ID: 55372 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 55380 - Posted: 29 Aug 2008, 13:36:01 UTC - in response to Message 55372.  

Is there any way to get Rosetta to ask for less work?


Yes, let it complete tasks. Sounds like your debt level to Rosetta is increasing faster then it is getting work.

You might want to increase the Rosetta preference for WU runtime. But do so gradually. Maybe bumping it 2 notches each day. That way Rosetta will only need one task per CPU to keep it busy all day long (if you get to the highest runtime preference), so overall it should be less disk space used by the project.

Rosetta Moderator: Mod.Sense
ID: 55380 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5664
Credit: 5,711,666
RAC: 764
Message 55389 - Posted: 29 Aug 2008, 17:32:54 UTC - in response to Message 55380.  

Is there any way to get Rosetta to ask for less work?


Yes, let it complete tasks. Sounds like your debt level to Rosetta is increasing faster then it is getting work.

You might want to increase the Rosetta preference for WU runtime. But do so gradually. Maybe bumping it 2 notches each day. That way Rosetta will only need one task per CPU to keep it busy all day long (if you get to the highest runtime preference), so overall it should be less disk space used by the project.



don't forget to check your preferences for how many extra days of work you want to download as well. reducing the number of days of extra work also reduces disk space.
ID: 55389 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
DavidL

Send message
Joined: 17 Dec 05
Posts: 2
Credit: 6,805,660
RAC: 3,616
Message 55393 - Posted: 29 Aug 2008, 22:41:55 UTC - in response to Message 55389.  
Last modified: 29 Aug 2008, 22:43:05 UTC

Is there any way to get Rosetta to ask for less work?


Yes, let it complete tasks. Sounds like your debt level to Rosetta is increasing faster then it is getting work.

You might want to increase the Rosetta preference for WU runtime. But do so gradually. Maybe bumping it 2 notches each day. That way Rosetta will only need one task per CPU to keep it busy all day long (if you get to the highest runtime preference), so overall it should be less disk space used by the project.



don't forget to check your preferences for how many extra days of work you want to download as well. reducing the number of days of extra work also reduces disk space.


Thanks. It has gotten to the point where I can't download any new work at all, and am unable to do Rosetta tasks, even when I don't get any new work for the other projects I'm crunching for. So it's not a matter of letting Rosetta finish tasks. Unfortunately, it was already set for only having a 1/4 extra day of work. I doubt setting that to 0 will have much impact, but I'll try. I also did bump up the Rosetta preference to 120. I'll see if that helps.
ID: 55393 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
ChristianB
Avatar

Send message
Joined: 20 Oct 05
Posts: 10
Credit: 178,513
RAC: 0
Message 55417 - Posted: 31 Aug 2008, 13:33:27 UTC

Could we please come back to my problem with rosetta? Have you looked into the issue Mod.Sense?

BOINC Doc | Team-Site | BOINC-Podcast
ID: 55417 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 55420 - Posted: 31 Aug 2008, 15:28:58 UTC

Django Fett, when you download a task, BOINC is assuring there is enough available disk for the task to run. Rosetta's tasks are sent with an indication that the maximum output file could reach a size of 25,000,000. So, in addition to any additional files that must be downloaded to process the task, there would need to be another 25MB for the task to run.

The actual output size produced by a Rosetta task is generally about 100k, but some tasks produce more then others, and the 25MB is the point at which the BOINC client should halt the execution of the task due to exceeding it's bounds.

The BOINC client also attempts to allocate disk space to the various projects you attach to based on their resource shares, and the nature of whether their files would require download again if the client deleted them. You said you are running three projects. What are the resource shares between them?

The client application has been enhancing this balancing act between projects, and you are running a rather new client, but not the 6.2.18 nor the 5.10.45 that are currently recommended. Have you tried running one of the recommended versions?

Here is some info. on how BOINC allocates disk space on the client.

Rosetta Moderator: Mod.Sense
ID: 55420 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
ChristianB
Avatar

Send message
Joined: 20 Oct 05
Posts: 10
Credit: 178,513
RAC: 0
Message 55428 - Posted: 31 Aug 2008, 18:06:11 UTC - in response to Message 55420.  

The BOINC client also attempts to allocate disk space to the various projects you attach to based on their resource shares, and the nature of whether their files would require download again if the client deleted them. You said you are running three projects. What are the resource shares between them?

That could be part of the problem. The resource share was:
- Orbit@home: 1000 (suspended)
- Rosetta@home: 100 (0 tasks)
- SETI@home: 1 (4 tasks)

After detaching from orbit@home rosetta now gets 99% of available space (according to the info you pointed me to) but the message still comes back from the server. There is 470MB available for BOINC and it uses 30MB thereof so it's enough free space for at least one rosetta task. (It's the host# 673574). The work cache is set to 1 day for this host.

I have a Vista Laptop also running rosetta with some different settings and the latest client (6.2.18). The preferences limit the disk usage to 930 MB and BOINC uses 350 thereof. 320MB for rosetta alone with ~70 tasks (host# 776261). This host is attached to two projects with the ressource shares:
- lhc@home: 1000 (suspended@90.91%)
- rosetta@home: 100 (~70 tasks@9.09%)

The work cache is set to 3 days + 3 days additional work.

The client application has been enhancing this balancing act between projects, and you are running a rather new client, but not the 6.2.18 nor the 5.10.45 that are currently recommended. Have you tried running one of the recommended versions?

6.2.15 is still recommended for Linux x86.

BOINC Doc | Team-Site | BOINC-Podcast
ID: 55428 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Message: Not enough disk space when there is enough space



©2024 University of Washington
https://www.bakerlab.org