Message boards : Number crunching : Problems and Technical Issues with Rosetta@home
Previous · 1 . . . 123 · 124 · 125 · 126 · 127 · 128 · 129 . . . 309 · Next
Author | Message |
---|---|
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1725 Credit: 18,417,319 RAC: 20,286 |
That means you will get no work at all, even though there are just under 13 million Rosetta 4.20 Tasks available.How can I make it stop? I have been a long time supporter of RH, but will disconnect from the project if it keeps wasting my bandwidth with 2GB failed downloads. Grant Darwin NT |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1725 Credit: 18,417,319 RAC: 20,286 |
I received an odd message today:That was sorted out yesterday. It started on the 18th & when it did, it stopped those without VirtualBox from getting any Rosetta work- even that which doesn't need VirtualBox. Once they fixed it, there were no more notices about a lack of VirtualBox in Event logs, and work started flowing again- as can be seen in the Graph below. In particular, the big dip on the 18th. Grant Darwin NT |
Admin Project administrator Send message Joined: 1 Jul 05 Posts: 5144 Credit: 0 RAC: 0 |
There is a researcher in the group that has submitted a large batch of VM jobs that requires Virtual Box. More information is provided at https://boinc.berkeley.edu/wiki/VirtualBox. This is the first large batch of "rosetta_python_projects" VM jobs but we hope to have more in the future. Since this is our first attempt at using BOINC's VM option, we will undoubtedly run into issues as we scale up. Please be patient as we try to resolve these issues. Our standard rosetta application is still being used but the number of available jobs can fluctuate as it has before depending on the computing demand from our researchers. The VM app feature allows researchers much more flexibility to run scientific software that otherwise would not be possible on R@h. For example, the "rosetta_python_projects" app allows researchers to run deep learning based structure prediction pipelines including RoseTTAFold and AlphaFold2, and other python based sofware packages like AIMNet which is being used in the current batch of VM jobs to minimize potential microcycle therapeutics. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
This is the first large batch of "rosetta_python_projects" VM jobs but we hope to have more in the future. Very good. I have several machines with lots of memory waiting to do them. BUT: You need to make them selectable from the regular Rosettas. A simple checkbox would do, but even better would be the ability to select the number downloaded (or running) at at time. See LHC, or Cosmology, or WCG for examples. |
Ole Pettersen Send message Joined: 4 Dec 10 Posts: 7 Credit: 11,846,331 RAC: 3,306 |
It is just an informational message, hoping that you will install VBox to crunch more tasks. Thanks. |
Sid Celery Send message Joined: 11 Feb 08 Posts: 2141 Credit: 41,538,222 RAC: 10,691 |
There is a researcher in the group that has submitted a large batch of VM jobs that requires Virtual Box. More information is provided at https://boinc.berkeley.edu/wiki/VirtualBox Virtualization and BOINC Oh! So it's that thing I've been ignoring for several years... #Doh Download BOINC + VirtualBox |
Tomcat雄猫 Send message Joined: 20 Dec 14 Posts: 180 Credit: 5,386,173 RAC: 0 |
aaaf-IDC_pp-FPR_pp-mNHM_pp-NMBEN3_pp_0_1737815_2_1 <core_client_version>7.16.11</core_client_version> <![CDATA[ <message> WU download error: couldn't get input files: <file_xfer_error> <file_name>AIMNet_vm_v2.vdi</file_name> <error_code>-119 (md5 checksum failed for file)</error_code> <error_message>MD5 check failed</error_message> </file_xfer_error> </message> ]]> There goes another one. I have yet to encounter a Python task that doesn't error out. Is it something wrong with my setup, or is there a problem with these tasks? I've tried the version bundled with BOINC and the latest version, still 100% error rate. |
lazyacevw Send message Joined: 18 Mar 20 Posts: 12 Credit: 93,576,463 RAC: 0 |
I feel like I am missing out on some new information or something. Do these new "rosetta_python_projects" run native on Linux? Virtualbox is only needed on Windows and Mac, correct? I was under the impression that all Windows BOINC installations were already containers or VMs on Windows or Macs. |
Falconet Send message Joined: 9 Mar 09 Posts: 354 Credit: 1,276,393 RAC: 1,656 |
I feel like I am missing out on some new information or something. Do these new "rosetta_python_projects" run native on Linux? Virtualbox is only needed on Windows and Mac, correct? I was under the impression that all Windows BOINC installations were already containers or VMs on Windows or Macs. The applications page says: Linux running on an AMD x86_64 or Intel EM64T CPU 1.03 (vbox64) Same as Windows and MacOS. No mention of a native app. |
dcdc Send message Joined: 3 Nov 05 Posts: 1832 Credit: 119,860,059 RAC: 6,147 |
I'm pretty sure it uses VBox regardless of the host OS. I guess that way it can ship with the correct dependancies etc. |
lazyacevw Send message Joined: 18 Mar 20 Posts: 12 Credit: 93,576,463 RAC: 0 |
Interesting. I took a look at the applications page https://boinc.bakerlab.org/rosetta/apps.php and the VirtualBox page https://boinc.berkeley.edu/wiki/VirtualBox but I don't see anything that would indicate a definitive answer. My Linux systems do not have VirtualBox installed. The apps page does indicate they are running on Linux systems but they don't indicate if VirtualBox is or needs to be installed. The same goes for Windows in the apps page. It was my understanding that Rosetta tasks were already written for Linux environments and so it was just easier to containerize Linux on Windows than re-write everything for Windows or Macs. I haven't seen anything in my logs about downloading AIMNet_vm_v2.vdi and when I check out my boinc directory, I see minirosetta and 4.20 executables but I don't see anything about vbox64 or python. I have 671 tasks in progress and I am showing 0 rosetta python tasks. I was just curious if there is anything I need to do to keep my systems up to date so that they can maximize processing capability for Rosetta. |
Falconet Send message Joined: 9 Mar 09 Posts: 354 Credit: 1,276,393 RAC: 1,656 |
If Virtualbox isn't installed, your system won't download any work units or executables related to the vbox app. This "1.03 (vbox64)" indicates the need of the pyrosetta app for virtualbox. |
YAG Send message Joined: 13 Oct 19 Posts: 7 Credit: 13,015,426 RAC: 0 |
Hello, Why could I receive the message "Communication with VM Hipervisor failed"? In the VBox register appears the error «Failed to create the VirtualBox object! NS_ERROR_SOCKET_FAIL (0xC1F30200) IPC Daemon socket error.» I have the Boinc version 7.16.16 and the VirtualBox 6.1.26 r145957 running on Debian GNU/Linux 11 (bullseye) [5.10.0-8-amd64|libc 2.31 (Debian GLIBC 2.31-13)]. The boinc user is added to the vboxusers, vboxguests and libvirt groups. Any clue about what could be happening? I have no experience with VirtualBox, I installed it just as a Rosetta@home requirement. |
Ole Pettersen Send message Joined: 4 Dec 10 Posts: 7 Credit: 11,846,331 RAC: 3,306 |
I got this when I tried to open Oracle VM VirtualBox: |
dcdc Send message Joined: 3 Nov 05 Posts: 1832 Credit: 119,860,059 RAC: 6,147 |
You could try this app: http://leomoon.com/downloads/application/leomoon-cpu-v/ And there are other tests listed here too: https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=4161 |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
Interesting. I took a look at the applications page https://boinc.bakerlab.org/rosetta/apps.php and the VirtualBox page https://boinc.berkeley.edu/wiki/VirtualBox but I don't see anything that would indicate a definitive answer. My Linux systems do not have VirtualBox installed. The apps page does indicate they are running on Linux systems but they don't indicate if VirtualBox is or needs to be installed. The same goes for Windows in the apps page. You need VirtualBox for Linux too. There are different versions of Linux, with different libraries. It can cause problems on any project. Even LHC uses VirtualBox on Linux for Theory and ATLAS, though they also have "native" apps for them. But the native apps need a "container" such as singularity, which is even more complicated to set up than just installing VirtualBox. Though it would be nice to have native versions for Rosetta too if they can manage it, since it runs a little more efficiently, and probably needs less memory. |
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
DEK/ADMIN/dcdc What the heck is going on with your scheduler? Or is there a bug in BOINC? I have now 998 (What the *!&*#?) tasks sitting in my queue. I lost 11 python tasks because something clogged my system with over 600 4.20 tasks I am going to abort 900 tasks because there is no way hell I can plow through that many. That's 332 days of crunching. That just beyond belief!!! Plus that clogs up my system because your project is limited to a certain number of cpu's while I share the rest with several other projects and all your tasks have 8 hour run times. If I drop that to 4 that would be still over 5 months of work. And all these tasks were due on the 24th? Explain this to me..... BOINC bug or your system bug? ---- Sent back 880 or so tasks...geees...i better go find a limiting command |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
What the heck is going on with your scheduler? I have not had that problem since getting rid of the "max concurrent" in the app_config.xml, as we discussed earlier. |
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
What the heck is going on with your scheduler? I have to have max concurrent in order to limit the number of cpu's RAH uses, otherwise my idea of splitting up my system so every project has its own group of cores is out the door and then I run into problem of every project dominating my system and some get all the work for days on end and others don't. If RAH would do like LHC and allow ME to pick how many cores to give it, then I would not have to do max concurrent. I need to find a command to limit the number of files downloaded. OR would project_max_concurrent be any better than max_concurrent? How about ncpus and take out max_concurrent? |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
OR would project_max_concurrent be any better than max_concurrent? Unfortunately, project_max_concurrent won't work either. https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5720&postid=45323#45323 You could try ncpus, but I think that is mainly for multi-threaded apps. What I do is just create separate BOINC instances for each type of work unit when I need to limit them. Then you can just set "use at most % of the processors" to limit it to what you want. It is a bit of a pain, but actually simple enough once you do it the first time. This more or less gives you all you need. https://www.overclock.net/threads/guide-setting-up-multiple-boinc-instances.1628924/ I think that on Windows, they neglect to tell you how to start it up automatically, though you can do it manually easily enough. I used Task Scheduler to start the BOINC client automatically, but don't have the details since I usually use Ubuntu, where it is a bit simpler. I think you have to start up BOINC Manager separately in the "Programs/Startup" folder, but that was on Win7 and I am now on Win10. |
Message boards :
Number crunching :
Problems and Technical Issues with Rosetta@home
©2024 University of Washington
https://www.bakerlab.org