Please remove Virtualbox as a dependency.

Message boards : Number crunching : Please remove Virtualbox as a dependency.

To post messages, you must log in.

Previous · 1 · 2 · 3

AuthorMessage
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,537,480
RAC: 776
Message 106317 - Posted: 28 May 2022, 20:22:53 UTC - in response to Message 106308.  

The new Python application does more than take 3gb of memory per task. It also chews up your disk drives by checkpointing too frequently. Most of the problems and stalls are related to checkpointing. A checkpoint option on the preferences would help.
My SSDs and NVMEs don't mind.
ID: 106317 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 3
Message 106321 - Posted: 28 May 2022, 23:53:49 UTC - in response to Message 106308.  

No news??

Crickets. I did not expect any response. dcdc has been quite responsive in the past, but the developers can't be bothered.
The new Python application does more than take 3gb of memory per task. It also chews up your disk drives by checkpointing too frequently. Most of the problems and stalls are related to checkpointing.

>> A checkpoint option on the preferences would help.<<

You can set checkpoint interval in boinc manager prefs to anything you like
such as 3600 seconds is once per hour , etc .
ID: 106321 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tullio

Send message
Joined: 10 May 20
Posts: 63
Credit: 630,125
RAC: 0
Message 106326 - Posted: 29 May 2022, 7:46:50 UTC - in response to Message 106314.  

https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5594
ID: 106326 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 373
Credit: 10,598,406
RAC: 8,429
Message 106327 - Posted: 29 May 2022, 11:32:16 UTC - in response to Message 106326.  

https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5594


Sadly, that is very specific to lhc requiring the use of CERN utilities and the project to set up additional servers to provide additional functionality.
ID: 106327 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1847
Credit: 7,994,764
RAC: 8,835
Message 106335 - Posted: 31 May 2022, 9:32:14 UTC - in response to Message 106321.  

You can set checkpoint interval in boinc manager prefs to anything you like
such as 3600 seconds is once per hour , etc .


I think that vms don't care about checkpoint.
ID: 106335 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 114,435,336
RAC: 56,017
Message 106482 - Posted: 7 Jul 2022, 8:40:08 UTC

Seeing all tasks saying "Running" whilst having been sat at 0% for a day and needing aborting, and then having the machine blacklisted from running more VBox tasks is getting tedious. The VirtualBox tasks aren't fit to be released. VirtualBox multi-attach would presumably solve this by reducing disk usage massively, especially on machines with more cores.
ID: 106482 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1847
Credit: 7,994,764
RAC: 8,835
Message 106483 - Posted: 7 Jul 2022, 11:55:34 UTC - in response to Message 106482.  
Last modified: 7 Jul 2022, 11:55:57 UTC

The VirtualBox tasks aren't fit to be released. VirtualBox multi-attach would presumably solve this by reducing disk usage massively, especially on machines with more cores.

Lhc_dev project is testing largely this new wrapper.
And seems it works!
ID: 106483 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,537,480
RAC: 776
Message 106484 - Posted: 7 Jul 2022, 12:19:58 UTC - in response to Message 106482.  

Seeing all tasks saying "Running" whilst having been sat at 0% for a day and needing aborting, and then having the machine blacklisted from running more VBox tasks is getting tedious. The VirtualBox tasks aren't fit to be released. VirtualBox multi-attach would presumably solve this by reducing disk usage massively, especially on machines with more cores.
Not seeing that here, 95% of them work ok.
ID: 106484 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 114,435,336
RAC: 56,017
Message 106485 - Posted: 7 Jul 2022, 14:07:32 UTC - in response to Message 106484.  

I'm only seeing it on 8-core machines. The quad-core and older machines are all fine.
ID: 106485 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,537,480
RAC: 776
Message 106486 - Posted: 7 Jul 2022, 14:23:00 UTC - in response to Message 106485.  
Last modified: 7 Jul 2022, 14:23:55 UTC

I'm only seeing it on 8-core machines. The quad-core and older machines are all fine.
How much RAM are you using? I find things go belly up if you run short of it.

I'm only running it on a 6 core at the moment, the only other one I can run it on is 24 core (which I usually leave off Python due to it being sluggish and I use it for other things), I'll try that one to see what happens.
ID: 106486 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,537,480
RAC: 776
Message 106487 - Posted: 7 Jul 2022, 14:48:43 UTC
Last modified: 7 Jul 2022, 14:49:34 UTC

They all got to 2% just fine on my 24 core Ryzen 9 3900XT.

But I aborted them there because Virtualbox on a PC you're trying to use for anything else is preposterous. Even the mouse cursor is more sluggish than a 286! I don't know what Oracle are playing at but the whole concept of virtual machines is flawed. I only run it on Boinc only machines.
ID: 106487 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 114,435,336
RAC: 56,017
Message 106488 - Posted: 7 Jul 2022, 19:19:16 UTC - in response to Message 106486.  

32GB in the two that have issues.
ID: 106488 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 114,435,336
RAC: 56,017
Message 106489 - Posted: 7 Jul 2022, 19:19:18 UTC - in response to Message 106486.  
Last modified: 7 Jul 2022, 19:19:55 UTC

..
ID: 106489 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,537,480
RAC: 776
Message 106490 - Posted: 7 Jul 2022, 20:49:12 UTC - in response to Message 106488.  

32GB in the two that have issues.
Should be ok, you need 3GB per task. I had a 6 core 16GB machine and it would run 5 then refuse to start a 6th. Mind you it never went wrong, it just didn't fill the cores.
ID: 106490 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,537,480
RAC: 776
Message 106491 - Posted: 7 Jul 2022, 20:52:12 UTC
Last modified: 7 Jul 2022, 20:52:42 UTC

I only aborted 1 of the last 70 tasks due to sticking.
ID: 106491 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1847
Credit: 7,994,764
RAC: 8,835
Message 107832 - Posted: 17 Dec 2022, 15:12:44 UTC

Pyinstaller new version, 5.7 with a lot of bugfix and new features, for example
Support building native ARM applications for Windows. If PyInstaller is ran on an ARM machine with an ARM build of Python, it will prodice an ARM application.

ID: 107832 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3

Message boards : Number crunching : Please remove Virtualbox as a dependency.



©2024 University of Washington
https://www.bakerlab.org