Message boards : Number crunching : Multiattach mode disk images
Previous · 1 · 2 · 3 · Next
Author | Message |
---|---|
[VENETO] boboviz Send message Joined: 1 Dec 05 Posts: 1994 Credit: 9,563,789 RAC: 6,764 |
Now also CMS@Home and Test4Theory have new versions with the new wrapper (in test on boinc lhc-dev site) New wrapper is now in production with new Atlas@Home app. When Rosetta? |
.clair. Send message Joined: 2 Jan 07 Posts: 274 Credit: 26,399,595 RAC: 0 |
Now also CMS@Home and Test4Theory have new versions with the new wrapper (in test on boinc lhc-dev site) About the same time I win the Lottery . . . . |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
The steps below are for the impatient volunteers who want to test the differencing image method. Use on own risk. Expect a higher error rate. The steps are written for Linux but can be used for Windows if the Windows vboxwrapper is used: https://boinc.berkeley.edu/dl/vboxwrapper_26204_windows_x86_64.zip 1. Set Rosetta to "no new tasks" and finish/report all Rosetta work 2. Reset Rosetta and close BOINC 3. Change the working directory to to "<BOINC_data_directory>/projects/boinc.bakerlab.org_rosetta/" 4. Downlod the following files: https://boinc.berkeley.edu/dl/vboxwrapper_26204_x86_64-pc-linux-gnu https://boinc-files.bakerlab.org/rosetta/download/boinc_app_v4 https://boinc-files.bakerlab.org/rosetta/download/vbox_job_v4.xml https://boinc-files.bakerlab.org/rosetta/download/26/AIMNet_vm_v2.vdi.gz 5. Set executable flags: chmod +x vboxwrapper_26204_x86_64-pc-linux-gnu chmod +x boinc_app_v4 Extract AIMNet_vm_v2.vdi from AIMNet_vm_v2.vdi.gz rename "AIMNet_vm_v2.vdi" to "AIMNet_vm_v2.vdi.bak" run "vboxmanage clonemedium AIMNet_vm_v2.vdi.bak AIMNet_vm_v2.vdi" (Important: this creates a new UUID for AIMNet_vm_v2.vdi) chmod +r AIMNet_vm_v2.vdi move AIMNet_vm_v2.vdi.bak to a different directory 6. Create the following "app_info.xml" <app_info> <app> <name>rosetta_python_projects</name> <user_friendly_name>rosetta python projects</user_friendly_name> <non_cpu_intensive>0</non_cpu_intensive> </app> <file_info> <name>vboxwrapper_26204_x86_64-pc-linux-gnu</name> <executable/> </file_info> <file_info> <name>boinc_app_v4</name> <executable/> </file_info> <file_info> <name>vbox_job_v4.xml</name> </file_info> <file_info> <name>AIMNet_vm_v2.vdi</name> </file_info> <app_version> <app_name>rosetta_python_projects</app_name> <version_num>103</version_num> <avg_ncpus>1.000000</avg_ncpus> <plan_class>vbox64</plan_class> <api_version>7.7.0</api_version> <file_ref> <file_name>vboxwrapper_26204_x86_64-pc-linux-gnu</file_name> <main_program/> </file_ref> <file_ref> <file_name>boinc_app_v4</file_name> <open_name>shared/boinc_app</open_name> <copy_file/> </file_ref> <file_ref> <file_name>vbox_job_v4.xml</file_name> <open_name>vbox_job.xml</open_name> </file_ref> <file_ref> <file_name>AIMNet_vm_v2.vdi</file_name> </file_ref> <dont_throttle/> </app_version> </app_info> 7. Modify vbox_job_v4.xml <vbox_job> <memory_size_mb>6144</memory_size_mb> <os_name>Debian_64</os_name> <enable_shared_directory/> <enable_remotedesktop/> <enable_vm_savestate_usage/> <disable_automatic_checkpoints/> <multiattach_vdi_file>AIMNet_vm_v2.vdi</multiattach_vdi_file> </vbox_job> 8. Restart BOINC and get fresh work (start with a small work buffer) <edit> May be better to add this to the file_ref section of "app_info.xml" instead of just the filename line: <file_ref> <file_name>AIMNet_vm_v2.vdi</file_name> <open_name>vm_image.vdi</open_name> </file_ref> </edit> |
[VENETO] boboviz Send message Joined: 1 Dec 05 Posts: 1994 Credit: 9,563,789 RAC: 6,764 |
The steps below are for the impatient volunteers who want to test the differencing image method. You-are-great!!! But i prefer an OFFICIAL app from the project. |
[VENETO] boboviz Send message Joined: 1 Dec 05 Posts: 1994 Credit: 9,563,789 RAC: 6,764 |
BOINC recently published vboxwrapper 26204 which introduces the code to deal with differencing images. LHC-Dev project is testing the 26205 version, with some bugfix Waiting for Rosetta@Home.... |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
Waiting for Rosetta@Home.... There are not that many left. Total queued jobs: 244,472 Given their glacial (or continental drift) rate of progress, I expect they would wait until the next series to introduce any changes. If there is a next series at all. |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
Based on the experience from LHC@home the suggestion from post https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14930&postid=106452 needs to be slightly refined. Get vboxwrapper 26205 instead of 26204. As long as 26205 is not available on https://boinc.berkeley.edu/dl/ get it from LHC@home Linux version: http://lhcathome.cern.ch/lhcathome/download/vboxwrapper_26205_x86_64-pc-linux-gnu Windows version: http://lhcathome.cern.ch/lhcathome/download/vboxwrapper_26205_windows_x86_64.exe Replace "vboxwrapper_26204" with "vboxwrapper_26205" in app_info.xml The tag "<open_name>vm_image.vdi</open_name>" at the bottom of the previous post is now obsolete and shouldn't be used. Volunteers waiting for an official release from Rosetta may wait forever. |
kotenok2000 Send message Joined: 22 Feb 11 Posts: 259 Credit: 484,884 RAC: 193 |
I get this error 10/08/2022 20:51:07 | Rosetta@home | [error] Application file AIMNet_vm_v2.vdi missing signature 10/08/2022 20:51:07 | Rosetta@home | [error] BOINC cannot accept this file I'm running boinc 7.20.2 My app_info.xml is <app_info> <app> <name>rosetta_python_projects</name> <user_friendly_name>rosetta python projects</user_friendly_name> <non_cpu_intensive>0</non_cpu_intensive> </app> <file_info> <name>vboxwrapper_26205_windows_x86_64.exe</name> <executable/> </file_info> <file_info> <name>boinc_app_v4</name> <executable/> </file_info> <file_info> <name>vbox_job_v4.xml</name> </file_info> <file_info> <name>AIMNet_vm_v2.vdi</name> </file_info> <app_version> <app_name>rosetta_python_projects</app_name> <version_num>103</version_num> <avg_ncpus>1.000000</avg_ncpus> <plan_class>vbox64</plan_class> <api_version>7.7.0</api_version> <file_ref> <file_name>vboxwrapper_26205_windows_x86_64.exe</file_name> <main_program/> </file_ref> <file_ref> <file_name>boinc_app_v4</file_name> <open_name>shared/boinc_app</open_name> <copy_file/> </file_ref> <file_ref> <file_name>vbox_job_v4.xml</file_name> <open_name>vbox_job.xml</open_name> </file_ref> <file_ref> <file_name>AIMNet_vm_v2.vdi</file_name> <open_name>vm_image.vdi</open_name> </file_ref> <dont_throttle/> </app_version> </app_info> |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
Remove this line from your app_info.xml <open_name>vm_image.vdi</open_name> Backup the following files from your Rosetta project directory: vboxwrapper_26205_windows_x86_64.exe AIMNet_vm_v2.vdi (the already cloned one) app_info.xml vbox_job_v4.xml (the already modified one) boinc_app_v4 Disable the client's network access Reset the Rosetta project (this removes the app_version you got from the project; must be done to avoid getting mixed versions) Shutdown the client Restore the backed up files Check if executable flags are set for vboxwrapper and boinc_app_v4 Start the client Enable network access |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
I am getting download errors https://boinc.bakerlab.org/rosetta/result.php?resultid=1492609800 and computing errors https://boinc.bakerlab.org/rosetta/result.php?resultid=1492609810 |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
I suspect that you didn't reset the project. If so, in client_state.xml there's still a signature of the original app_version files. A project reset removes them. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
Thanks, but a reset did not fix it. I still get the download errors. I am wondering whether it is because I downloaded the files on a Windows machine, and then transferred them to my Ubuntu machine. Maybe they picked up the wrong permissions or some other attributes? What I am going to do (probably not today) is to totally detach from all BOINC projects, and then download the files again directly onto the Ubuntu machine. It should work. |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
Files listed in app_info.xml must be present in Rosetta's project dir before the BOINC client starts (including app_info.xml itself). My guess would be that after the project reset your client contacted the server and got an app_version update from there. If this happened your client_state.xml will contain <signature_required/> tags related to Rosetta files you mentioned in app_info.xml (like AIMNet_vm_v2.vdi). These would be wrong. In addition the client must be restarted to read app_info.xml. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
Files listed in app_info.xml must be present in Rosetta's project dir before the BOINC client starts (including app_info.xml itself). That could be it. But how do you prevent BOINC from starting? There is no option for that in the GUI. You may be referring to a command line option. |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
Pause BOINC's network activity, pause Rosetta or do both before you reset the project and shutdown BOINC. Then copy the required files back before you restart BOINC. See: https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14930&postid=106758 Using the "anonymous platform" is a rather old method, hence not perfectly user friendly. But once it is correctly set up it is as reliable as an app_version from the server. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
Pause BOINC's network activity, pause Rosetta or do both before you reset the project and shutdown BOINC. We are getting closer, but the question is how do you shutdown BOINC? There is no command for it in the GUI. The only commands are for pausing BOINC and shutting down the network (no new tasks). (Would a reboot do it?) I get the following error message at Step 5, $ sudo vboxmanage clonemedium AIMNet_vm_v2.vdi.bak AIMNet_vm_v2.vdi So the original .vdi file apparently still exists. But one thing I did learn is that you can't use VirtualBox installed from a downloaded .deb package. You have to use VirtualBox installed via "sudo apt install virtualbox". Otherwise, the "vboxmanage" command will not run at all. |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
how do you shutdown BOINC? In your BOINC Manager (GUI) isnt' there a menu entry like "File -> Shutdown the currently connected client". (Sorry I'm running the German GUI, hence don't know the exact term) On Linux you may run "[sudo] systemctl stop boinc-client". I get the following error message at Step 5, The file isn't the problem. Open your VirtualBox GUI -> Media Manager and remove the disk entry there (it shouldn't be connected to a VM). You will be asked if you also want to remove the file. Don't remove the file. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
On Linux you may run "[sudo] systemctl stop boinc-client".Thanks. That is what I needed for that part. On Windows, they have "Exit BOINC" in the GUI, which is probably what you wanted there. Open your VirtualBox GUI -> Media Manager and remove the disk entry there (it shouldn't be connected to a VM).There is no disk (or file) at all in VBox Media Manager. And I am getting the same error messages as before. $ sudo vboxmanage clonemedium AIMNet_vm_v2.vdi.bak AIMNet_vm_v2.vdi It could be a permissions problem. I install BOINC using "sudo apt install boinc", which gets it from Ubuntu I suppose. I don't try to compile it myself, so it is not in the "Home" directory, which may be a difference. I will give it a rest for a while, and try again later. Thanks for you input. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
But I did reboot, then a reset, then stopped BOINC and deleted the "AIMNet_vm_v2.vdi" file so that I could try again. I then ran the "sudo vboxmanage clonemedium AIMNet_vm_v2.vdi.bak AIMNet_vm_v2.vdi" command, and got: $ sudo vboxmanage clonemedium AIMNet_vm_v2.vdi.bak AIMNet_vm_v2.vdi So it does think that a hard disk already exists. However, it still does not show up in VirtualBox Manager. There may be something wrong with the VirtualBox installation. That is beyond me for now. |
computezrmle Send message Joined: 9 Dec 11 Posts: 63 Credit: 9,680,103 RAC: 0 |
VERR_ALREADY_EXISTS tells you there's a disk registered with this name/UUID Try vboxmanage list hdds if AIMNet_vm_v2.vdi is in the list, try vboxmanage closemedium UUID In addition, check the owner/group of the files. Compare that with files from other projects. <edit> If you run vboxmanage commands or if you use the GUI, use the account of your BOINC user as each user on your box has it's own vbox environment. </edit> |
Message boards :
Number crunching :
Multiattach mode disk images
©2024 University of Washington
https://www.bakerlab.org