Message boards : Number crunching : Disk usage limit exceeded
Author | Message |
---|---|
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 188 Credit: 6,430,914 RAC: 5,659 |
I get this error message with this task. Task 1531200508 Peak working set size 356.58 MB Peak swap size 427.38 MB Peak disk usage 2,437.40 MB <core_client_version>7.20.2</core_client_version> <![CDATA[ <message> Disk usage limit exceeded</message> [code]<stderr_txt> command: ../../projects/boinc.bakerlab.org_rosetta/rosetta_beta_6.00_x86_64-pc-linux-gnu @yT1NLvkg_flpp3_16_binder_15_cyclic_152_dldesign_0_cycle2.cyclic_redesigned.r1_0001.flags -nstruct 10000 -cpu_run_time 28800 -boinc:max_nstruct 20000 -checkpoint_interval 120 -database minirosetta_database -in::file::zip minirosetta_database.zip -boinc::watchdog -boinc::cpu_run_timeout 36000 -run::rng mt19937 Using database: database_0f7f01a1b07/database </stderr_txt> ]]> My machine is like this. Computer 5910575 Computer information CPU type GenuineIntel Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz [Family 6 Model 85 Stepping 7] Number of processors 16 Operating System Linux Red Hat Enterprise Linux Red Hat Enterprise Linux 8.7 (Ootpa) [4.18.0-425.13.1.el8_7.x86_64|libc 2.28] BOINC version 7.20.2 Memory 128345.24 MB Cache 16896 KB Swap space 15992 MB Total disk space 488.04 GB Free Disk Space 473.91 GB With that much free disk space, and the relatively modest amount used by the task, what are they complaining about? |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1680 Credit: 17,850,239 RAC: 22,492 |
With that much free disk space, and the relatively modest amount used by the task, what are they complaining about?It doesn't matter how much disk space you have, if you don't let BOINC use it. And it could just be a hard coded disk usage limit in the Rosetta application that got exceeded. Grant Darwin NT |
Conan Send message Joined: 11 Oct 05 Posts: 150 Credit: 4,198,375 RAC: 1,306 |
It should not of used so much disk space. I run Fedora Linux and Rosetta Beta work units only use less than 30 MB of disk space and 1.6 GB to 1.8 GB of RAM. This has been consistent for nearly all beta work units. They use so much memory that my 32 GB RAM on a 24 thread computer gets almost 100% used as well as most of my 8.6 GB Swap space. Boinc actually limits running threads so it can keep going. Anyway must work as no errors. Conan |
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 188 Credit: 6,430,914 RAC: 5,659 |
With that much free disk space, and the relatively modest amount used by the task, what are they complaining about? But I DO let Boinc use it; At least 420 GBytes of it. I have entire partition on one of my hard drives where all the Boinc stuff is and it has 420 GBytes allocated. And only about 11 GBytes are being used at the moment. I notice that the other user who attempted to run this task failed the same way I did. Him: Task 1531192066 Name yT1NLvkg_flpp3_16_binder_15_cyclic_152_dldesign_0_cycle2.cyclic_redesigned.r1_0001_SAVE_ALL_OUT_2959796_604_0 Me: Task 1531200508 Name yT1NLvkg_flpp3_16_binder_15_cyclic_152_dldesign_0_cycle2.cyclic_redesigned.r1_0001_SAVE_ALL_OUT_2959796_604_1 I have two running at the moment and they are not hogging memory: $ ps -F 516894 538315 UID PID PPID C SZ RSS PSR STIME TTY STAT TIME CMD boinc 516894 18695 94 111050 372104 8 01:57 ? RNl 449:17 ../../projects/boinc.bakerlab.org_rosetta/rosetta_beta_6.00_x86_64-pc boinc 538315 18695 98 110291 368636 14 08:56 ? RNl 54:50 ../../projects/boinc.bakerlab.org_rosetta/rosetta_beta_6.00_x86_64-pc |
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 188 Credit: 6,430,914 RAC: 5,659 |
It should not of used so much disk space. I am running Red Hat Enterprise Linux release 8.7 (Ootpa). My machine is ID: 5910575. My machine has 128 GBytes of RAM, so I am not running out of that. I am using very little swap space, and not thrashing. $ free -hw total used free shared buffers cache available Mem: 125Gi 7.4Gi 3.0Gi 123Mi 430Mi 114Gi 116Gi Swap: 15Gi 47Mi 15Gi |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
A disk usage limit can be set by the project via <rsc_disk_bound> (in bytes). It is thought to protect the disk from being filled by endless loops. You may check client_state.xml for the current values. If the error happens regularly the project people may need to increase the value. Otherwise it may indicate a program error. |
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 188 Credit: 6,430,914 RAC: 5,659 |
A disk usage limit can be set by the project via <rsc_disk_bound> (in bytes). Which one? There are many different ones. this is a sort -u of them. <rsc_disk_bound>100000000.000000</rsc_disk_bound> <rsc_disk_bound>104857600.000000</rsc_disk_bound> <rsc_disk_bound>160000000.000000</rsc_disk_bound> <rsc_disk_bound>2000000000.000000</rsc_disk_bound> <rsc_disk_bound>20000000.000000</rsc_disk_bound> <rsc_disk_bound>524288000.000000</rsc_disk_bound> <rsc_disk_bound>75000000.000000</rsc_disk_bound> |
Jean-David Beyer Send message Joined: 2 Nov 05 Posts: 188 Credit: 6,430,914 RAC: 5,659 |
I think I found it: <app_name>rosetta_beta</app_name> <version_num>600</version_num> <rsc_fpops_est>80000000000000.000000</rsc_fpops_est> <rsc_fpops_bound>100000000000000000.000000</rsc_fpops_bound> <rsc_memory_bound>3500000000.000000</rsc_memory_bound> <rsc_disk_bound>2000000000.000000</rsc_disk_bound> They allow more RAM than disk? |
majortotositetop1 Send message Joined: 22 May 23 Posts: 4 Credit: 0 RAC: 0 |
|
majortotositetop1 Send message Joined: 22 May 23 Posts: 4 Credit: 0 RAC: 0 |
|
Sid Celery Send message Joined: 11 Feb 08 Posts: 2124 Credit: 41,228,077 RAC: 11,048 |
I think I found it: Yeah, each individual task sets its own requirement, so it'll come up multiple times in the client state file But didn't we have this before and found it to be a problem with Boinc rather than Rosetta? I used to put no limit (blank) on maximum disk size set aside for Boinc, but it ended up needing a high set figure because of some kind of miscalculation, so if I put 500Gb then tasks went through ok, even if the unlimited disk space turned out to be more than that. I may be misremembering how we resolved this, but it was something to do with that, however nonsensical the solution seemed to be. |
.clair. Send message Joined: 2 Jan 07 Posts: 274 Credit: 26,399,595 RAC: 0 |
It waz like this If you want to have something in all three boxes in boinc manager this setting also works and gets the maximum use of the disk with all three boxes used , if you [whoever you are that is reading this] insist on it . Use no more than - 500 GB - Just set this to something BIG Leave at least - 0.01 GB free Use no more than - 99.9 % of total from my thread from February 2022 when we were dealing with greedy python`s ;-) https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14903&postid=105215 I went as far as putting those settings in my web preferences so that any new computer I attach to ANY project gets them automatically , problem fixed PERMANTLY . until next time something screws up . . . . After all this is BOINC . nnnnnn......... |
Sid Celery Send message Joined: 11 Feb 08 Posts: 2124 Credit: 41,228,077 RAC: 11,048 |
It waz like this I must've read that at the time and did much the same. 500Gb 1Gb <blank> |
Message boards :
Number crunching :
Disk usage limit exceeded
©2024 University of Washington
https://www.bakerlab.org