Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 123 · 124 · 125 · 126 · 127 · 128 · 129 . . . 274 · Next

AuthorMessage
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5658
Credit: 5,670,291
RAC: 2,328
Message 102730 - Posted: 19 Sep 2021, 18:00:05 UTC - in response to Message 102720.  

But it is not a issue for Linux machines because your running the native environment of the task.
Us Windows users are emulating Linux via a Virtual Machine and that seems to be where the problem is at.
The Virtual Disk Image file Checksum code that we get does not seem to match the code the server wants and then we get a code -119 MD5 checksum error.

I have to run VirtualBox on my Ubuntu machines also to do the pythons.

But it does not always work quite the same way as on Windows. It is hopefully a small discrepancy, but we all need someone at Rosetta to look into it, and they never bother to even acknowledge problems. Maybe someone can get the attention of the Admin.



Sid Celery is already sending emails to Baker Lab, but they are a M-F office hours only project.
There is no dedicated IT person that monitors the forums, at least not that we have seen.
The only way they know if there is a problem is if the researcher of this specific project sees a lot of failures in his data. Other than that, your SOL.

You were in a conversation with Mikey who showed you how to isolate python tasks so you could limit the cores. The only thing further (which i don't know the commands for) is to "block" the specific task type. Maybe Mikey can help you with that in that other thread.

Your not alone with this MD5 issue, A windows user in Italy is also having the exact same problem.
So far everything we know to try and make this disappear has failed. So its something back in Seattle on their system that is causing this. Now why some windows machines and not others is a mystery, because I am cranking through these just fine.
ID: 102730 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 102733 - Posted: 19 Sep 2021, 20:06:20 UTC - in response to Message 102730.  

You were in a conversation with Mikey who showed you how to isolate python tasks so you could limit the cores. The only thing further (which i don't know the commands for) is to "block" the specific task type. Maybe Mikey can help you with that in that other thread.

Your not alone with this MD5 issue, A windows user in Italy is also having the exact same problem.
So far everything we know to try and make this disappear has failed. So its something back in Seattle on their system that is causing this. Now why some windows machines and not others is a mystery, because I am cranking through these just fine.

I am not sure what you are referring to on MW. If it is "max concurrent" (or "project max concurrent") in an app_config, that causes a problem with excessive downloads. I have posted on it already here.

But that appears to be irrelevant to the present MD5 issue anyway. And I am seeing the same thing on Ubuntu as the Windows users, just a slightly different message.
https://boinc.bakerlab.org/rosetta/results.php?hostid=6143731&offset=0&show_names=0&state=6&appid=

I am trying a new version of BOINC. I was on 7.16.11, the released version from Ubuntu. Now it is 7.16.17 from Locutus-of-Borg.
That solved a problem on QuChemPedIA, but not necessarily here.

Seattle certainly needs to fix it.
ID: 102733 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 102737 - Posted: 19 Sep 2021, 21:52:58 UTC - in response to Message 102733.  

I am trying a new version of BOINC. I was on 7.16.11, the released version from Ubuntu. Now it is 7.16.17 from Locutus-of-Borg.
That solved a problem on QuChemPedIA, but not necessarily here.

The new BOINC 7.16.17 didn't fix it. It is something they have to do.
ID: 102737 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5658
Credit: 5,670,291
RAC: 2,328
Message 102739 - Posted: 19 Sep 2021, 23:35:36 UTC - in response to Message 102733.  

You were in a conversation with Mikey who showed you how to isolate python tasks so you could limit the cores. The only thing further (which i don't know the commands for) is to "block" the specific task type. Maybe Mikey can help you with that in that other thread.

Your not alone with this MD5 issue, A windows user in Italy is also having the exact same problem.
So far everything we know to try and make this disappear has failed. So its something back in Seattle on their system that is causing this. Now why some windows machines and not others is a mystery, because I am cranking through these just fine.

I am not sure what you are referring to on MW. If it is "max concurrent" (or "project max concurrent") in an app_config, that causes a problem with excessive downloads. I have posted on it already here.

But that appears to be irrelevant to the present MD5 issue anyway. And I am seeing the same thing on Ubuntu as the Windows users, just a slightly different message.
https://boinc.bakerlab.org/rosetta/results.php?hostid=6143731&offset=0&show_names=0&state=6&appid=

I am trying a new version of BOINC. I was on 7.16.11, the released version from Ubuntu. Now it is 7.16.17 from Locutus-of-Borg.
That solved a problem on QuChemPedIA, but not necessarily here.

Seattle certainly needs to fix it.


But it is weird, MD5 shows up on your machine and the Italian machine, but not on mine and not that many others are talking about it. But is it all python tasks or just certain specific strings that are affected?
ID: 102739 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Tomcat雄猫

Send message
Joined: 20 Dec 14
Posts: 180
Credit: 5,364,639
RAC: 0
Message 102742 - Posted: 19 Sep 2021, 23:51:09 UTC - in response to Message 102739.  
Last modified: 19 Sep 2021, 23:56:11 UTC

I also have the MD5 issue on Rosetta Python tasks. Presently it's been a 100% fail rate, error while downloading.

aaaf-IDC_pp-FPR_pp-mNHM_pp-NMBEN3_pp_0_1737815_2_1
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
WU download error: couldn't get input files:
<file_xfer_error>
  <file_name>AIMNet_vm_v2.vdi</file_name>
  <error_code>-119 (md5 checksum failed for file)</error_code>
  <error_message>MD5 check failed</error_message>
</file_xfer_error>
</message>
]]>


aaaf-IDC_pp-SAR-AIB_pp-NMBEN3_pp_0_1737353_4_0
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
WU download error: couldn't get input files:
<file_xfer_error>
  <file_name>AIMNet_vm_v2.vdi</file_name>
  <error_code>-119 (md5 checksum failed for file)</error_code>
  <error_message>MD5 check failed</error_message>
</file_xfer_error>
</message>
]]>


aaaf-mAZE_pp-NMALA_pp-mNMVAL_pp-NMBEN3_pp_0_1736941_7_0
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
WU download error: couldn't get input files:
<file_xfer_error>
  <file_name>AIMNet_vm_v2.vdi</file_name>
  <error_code>-119 (md5 checksum failed for file)</error_code>
  <error_message>MD5 check failed</error_message>
</file_xfer_error>
</message>
]]>


I see you have had good luck with these Python tasks, what's your vBox setup?
ID: 102742 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Tomcat雄猫

Send message
Joined: 20 Dec 14
Posts: 180
Credit: 5,364,639
RAC: 0
Message 102743 - Posted: 20 Sep 2021, 0:00:12 UTC - in response to Message 102742.  
Last modified: 20 Sep 2021, 0:18:45 UTC

Good lord, I just checked my Virtual Box and all the failed tasks apear to still be in there.

I'm using 6.1.12, the version supplied on the BOINC website, with the Oracle VM VirtualBox Extension Pack installed on Windows 10. Visualization extensions have been enabled in the BIOS.
Should I use the latest version (6.1.26) or the one supplied by BOINC? What about extensions, should I install any?
ID: 102743 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Pegasus

Send message
Joined: 22 Oct 06
Posts: 5
Credit: 6,509,968
RAC: 7,683
Message 102745 - Posted: 20 Sep 2021, 0:52:37 UTC

I received an odd message today:

Rosetta@home: Notice from server
VirtualBox is not installed
9/19/2021 6:01:50 PM

It is correct that I do not have VirtualBox installed, and I have not used VirtualBox to run BOINC programs.
ID: 102745 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1467
Credit: 14,310,302
RAC: 16,075
Message 102749 - Posted: 20 Sep 2021, 5:09:51 UTC - in response to Message 102728.  

How can I make it stop? I have been a long time supporter of RH, but will disconnect from the project if it keeps wasting my bandwidth with 2GB failed downloads.

Temporarily STOP REQUESTING NEW WORK FOR THE PROJET by settling NO NEW WORK for Rosetta on your PC(S) in BOINC until the project fixes the problem.
That means you will get no work at all, even though there are just under 13 million Rosetta 4.20 Tasks available.
Grant
Darwin NT
ID: 102749 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1467
Credit: 14,310,302
RAC: 16,075
Message 102750 - Posted: 20 Sep 2021, 5:16:20 UTC - in response to Message 102745.  

I received an odd message today:

Rosetta@home: Notice from server
VirtualBox is not installed
9/19/2021 6:01:50 PM

It is correct that I do not have VirtualBox installed, and I have not used VirtualBox to run BOINC programs.
That was sorted out yesterday.
It started on the 18th & when it did, it stopped those without VirtualBox from getting any Rosetta work- even that which doesn't need VirtualBox. Once they fixed it, there were no more notices about a lack of VirtualBox in Event logs, and work started flowing again- as can be seen in the Graph below.
In particular, the big dip on the 18th.


Grant
Darwin NT
ID: 102750 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Admin
Project administrator

Send message
Joined: 1 Jul 05
Posts: 4805
Credit: 0
RAC: 0
Message 102761 - Posted: 20 Sep 2021, 17:58:33 UTC

There is a researcher in the group that has submitted a large batch of VM jobs that requires Virtual Box. More information is provided at https://boinc.berkeley.edu/wiki/VirtualBox.

This is the first large batch of "rosetta_python_projects" VM jobs but we hope to have more in the future. Since this is our first attempt at using BOINC's VM option, we will undoubtedly run into issues as we scale up. Please be patient as we try to resolve these issues. Our standard rosetta application is still being used but the number of available jobs can fluctuate as it has before depending on the computing demand from our researchers.

The VM app feature allows researchers much more flexibility to run scientific software that otherwise would not be possible on R@h. For example, the "rosetta_python_projects" app allows researchers to run deep learning based structure prediction pipelines including RoseTTAFold and AlphaFold2, and other python based sofware packages like AIMNet which is being used in the current batch of VM jobs to minimize potential microcycle therapeutics.
ID: 102761 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 102763 - Posted: 20 Sep 2021, 20:06:43 UTC - in response to Message 102761.  

This is the first large batch of "rosetta_python_projects" VM jobs but we hope to have more in the future.

Very good. I have several machines with lots of memory waiting to do them.

BUT: You need to make them selectable from the regular Rosettas.
A simple checkbox would do, but even better would be the ability to select the number downloaded (or running) at at time.
See LHC, or Cosmology, or WCG for examples.
ID: 102763 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ole Pettersen

Send message
Joined: 4 Dec 10
Posts: 7
Credit: 10,545,261
RAC: 5,681
Message 102765 - Posted: 21 Sep 2021, 0:15:46 UTC - in response to Message 102745.  

It is just an informational message, hoping that you will install VBox to crunch more tasks. Thanks.
ID: 102765 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 1965
Credit: 38,160,504
RAC: 9,210
Message 102768 - Posted: 21 Sep 2021, 1:19:33 UTC - in response to Message 102761.  

There is a researcher in the group that has submitted a large batch of VM jobs that requires Virtual Box. More information is provided at https://boinc.berkeley.edu/wiki/VirtualBox

Virtualization and BOINC
BOINC uses virtualization to allow scientists to develop applications on their preferred operating system (usually Linux) and then run them on volunteered Windows and Mac computers. We call these VM apps.

BOINC's virtualization support uses a system called VirtualBox, which is open-source software maintained and distributed by Oracle. For more information about VirtualBox, please see their web site at https://www.virtualbox.org/

To run VM apps, your computer must have VirtualBox installed. Starting with BOINC version 7.2.28, the recommended BOINC installer for Windows includes VirtualBox as well. You can also install VirtualBox separately.

We recommend the VirtualBox version that is included in the recommended BOINC Windows installer, since newer VirtualBox versions do not work correctly for all projects.

If you are running Windows, it is essential that you install a version of BOINC that matches your version of Windows - use 64-bit BOINC on 64-bit versions of Windows, 32-bit BOINC on 32-bit versions of Windows. Otherwise, BOINC will not be able to detect the presence of VirtualBox.

Oh! So it's that thing I've been ignoring for several years... #Doh

Download BOINC + VirtualBox
ID: 102768 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Tomcat雄猫

Send message
Joined: 20 Dec 14
Posts: 180
Credit: 5,364,639
RAC: 0
Message 102781 - Posted: 21 Sep 2021, 9:54:11 UTC - in response to Message 102768.  
Last modified: 21 Sep 2021, 9:54:22 UTC

aaaf-IDC_pp-FPR_pp-mNHM_pp-NMBEN3_pp_0_1737815_2_1
<core_client_version>7.16.11</core_client_version>
<![CDATA[
<message>
WU download error: couldn't get input files:
<file_xfer_error>
  <file_name>AIMNet_vm_v2.vdi</file_name>
  <error_code>-119 (md5 checksum failed for file)</error_code>
  <error_message>MD5 check failed</error_message>
</file_xfer_error>
</message>
]]>


There goes another one. I have yet to encounter a Python task that doesn't error out. Is it something wrong with my setup, or is there a problem with these tasks? I've tried the version bundled with BOINC and the latest version, still 100% error rate.
ID: 102781 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile lazyacevw

Send message
Joined: 18 Mar 20
Posts: 12
Credit: 93,576,463
RAC: 0
Message 102783 - Posted: 21 Sep 2021, 12:05:08 UTC

I feel like I am missing out on some new information or something. Do these new "rosetta_python_projects" run native on Linux? Virtualbox is only needed on Windows and Mac, correct? I was under the impression that all Windows BOINC installations were already containers or VMs on Windows or Macs.
ID: 102783 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Falconet

Send message
Joined: 9 Mar 09
Posts: 350
Credit: 1,000,634
RAC: 0
Message 102784 - Posted: 21 Sep 2021, 12:28:43 UTC - in response to Message 102783.  

I feel like I am missing out on some new information or something. Do these new "rosetta_python_projects" run native on Linux? Virtualbox is only needed on Windows and Mac, correct? I was under the impression that all Windows BOINC installations were already containers or VMs on Windows or Macs.



The applications page says:

Linux running on an AMD x86_64 or Intel EM64T CPU 1.03 (vbox64)

Same as Windows and MacOS. No mention of a native app.
ID: 102784 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 114,331,182
RAC: 51,367
Message 102786 - Posted: 21 Sep 2021, 12:54:04 UTC

I'm pretty sure it uses VBox regardless of the host OS. I guess that way it can ship with the correct dependancies etc.
ID: 102786 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile lazyacevw

Send message
Joined: 18 Mar 20
Posts: 12
Credit: 93,576,463
RAC: 0
Message 102787 - Posted: 21 Sep 2021, 14:37:28 UTC - in response to Message 102784.  
Last modified: 21 Sep 2021, 14:41:50 UTC

Interesting. I took a look at the applications page https://boinc.bakerlab.org/rosetta/apps.php and the VirtualBox page https://boinc.berkeley.edu/wiki/VirtualBox but I don't see anything that would indicate a definitive answer. My Linux systems do not have VirtualBox installed. The apps page does indicate they are running on Linux systems but they don't indicate if VirtualBox is or needs to be installed. The same goes for Windows in the apps page.

It was my understanding that Rosetta tasks were already written for Linux environments and so it was just easier to containerize Linux on Windows than re-write everything for Windows or Macs.

I haven't seen anything in my logs about downloading AIMNet_vm_v2.vdi and when I check out my boinc directory, I see minirosetta and 4.20 executables but I don't see anything about vbox64 or python.

I have 671 tasks in progress and I am showing 0 rosetta python tasks. I was just curious if there is anything I need to do to keep my systems up to date so that they can maximize processing capability for Rosetta.
ID: 102787 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Falconet

Send message
Joined: 9 Mar 09
Posts: 350
Credit: 1,000,634
RAC: 0
Message 102788 - Posted: 21 Sep 2021, 14:44:07 UTC - in response to Message 102787.  

If Virtualbox isn't installed, your system won't download any work units or executables related to the vbox app.

This "1.03 (vbox64)" indicates the need of the pyrosetta app for virtualbox.
ID: 102788 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
YAG

Send message
Joined: 13 Oct 19
Posts: 7
Credit: 13,015,426
RAC: 200
Message 102795 - Posted: 21 Sep 2021, 20:42:57 UTC

Hello,

Why could I receive the message "Communication with VM Hipervisor failed"? In the VBox register appears the error
«Failed to create the VirtualBox object!
NS_ERROR_SOCKET_FAIL (0xC1F30200) IPC Daemon socket error.»

I have the Boinc version 7.16.16 and the VirtualBox 6.1.26 r145957 running on Debian GNU/Linux 11 (bullseye) [5.10.0-8-amd64|libc 2.31 (Debian GLIBC 2.31-13)].
The boinc user is added to the vboxusers, vboxguests and libvirt groups.

Any clue about what could be happening? I have no experience with VirtualBox, I installed it just as a Rosetta@home requirement.
ID: 102795 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 123 · 124 · 125 · 126 · 127 · 128 · 129 . . . 274 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org