Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 200 · 201 · 202 · 203 · 204 · 205 · 206 . . . 257 · Next

AuthorMessage
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1217
Credit: 13,359,660
RAC: 431
Message 105883 - Posted: 9 Apr 2022, 4:03:44 UTC - in response to Message 105878.  

Q9450 Doesn't support AVX. Virtualbox jobs get stuck if avx is missing.
Not on cosmology or LHC, that's only a Rosetta requirement. I have 5 computers with no AVX and they all run Cosmology and LHC VB tasks fine.

Which Virtualbox program does this?

Virtualbox has the old vbox, which handles 32-bit programs, and rarely gives errors these days.

They also have the newer vbox64, which handles 64-bit programs, and often gives errors for the Python tasks, especially on computers where the CPU does not support AVX.

Also, the Cosmology tasks on my computer do not appear to use any Virtualbox program at all. Most of them use the multithreaded feature, which divides the work among 4 virtual CPU cores, without using a Virtualbox program.

My computer doesn't participate in LHC, so I have no information on what LHC tasks do.
ID: 105883 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1217
Credit: 13,359,660
RAC: 431
Message 105884 - Posted: 9 Apr 2022, 4:10:53 UTC - in response to Message 105882.  

BOINC is giving me error messages about being unable to run Python tasks because the need very large amounts of free memory to even start, such as about 19 GB. Is this a real requirement for them, or just bad calculations?
Sometimes they ask for more than they will actually need, but they do use a lot. For example I have a 6 core i5 with 16GB of RAM. It can get 5 Pythons running, then the memory is 80-90% full and doesn't load the 6th.

19GB sounds unusual, since that would stop any from running on my above machine at all, and I've never seen that.

Make that SOME Python tasks. My computer has 32 GB of memory, and often runs seven Python tasks at once. There's no good reason to believe that every Python task requires the same amount of memory, even for just the task startup. The amount of memory required for task startup is often much more than the task is actually uses after it starts.
ID: 105884 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,169,950
RAC: 9,484
Message 105885 - Posted: 9 Apr 2022, 4:24:57 UTC - in response to Message 105883.  
Last modified: 9 Apr 2022, 4:27:12 UTC

Q9450 Doesn't support AVX. Virtualbox jobs get stuck if avx is missing.
Not on cosmology or LHC, that's only a Rosetta requirement. I have 5 computers with no AVX and they all run Cosmology and LHC VB tasks fine.

Which Virtualbox program does this?
I'm using version 5.2.44, from here: https://www.virtualbox.org/wiki/Download_Old_Builds_5_2 It's 64 bit.

Also, the Cosmology tasks on my computer do not appear to use any Virtualbox program at all. Most of them use the multithreaded feature, which divides the work among 4 virtual CPU cores, without using a Virtualbox program.
That can't be be right. If the tasks are called "Cosmology Legacy", they're not Vbox and use 1 core. The Vbox ones are the only multicore ones. Linux might be different if you use that (but it appears you're on Windows only).
ID: 105885 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,169,950
RAC: 9,484
Message 105886 - Posted: 9 Apr 2022, 4:29:21 UTC - in response to Message 105884.  

BOINC is giving me error messages about being unable to run Python tasks because the need very large amounts of free memory to even start, such as about 19 GB. Is this a real requirement for them, or just bad calculations?
Sometimes they ask for more than they will actually need, but they do use a lot. For example I have a 6 core i5 with 16GB of RAM. It can get 5 Pythons running, then the memory is 80-90% full and doesn't load the 6th.

19GB sounds unusual, since that would stop any from running on my above machine at all, and I've never seen that.

Make that SOME Python tasks. My computer has 32 GB of memory, and often runs seven Python tasks at once. There's no good reason to believe that every Python task requires the same amount of memory, even for just the task startup. The amount of memory required for task startup is often much more than the task is actually uses after it starts.
Time to throw memory at it, Some of my machines will take a lot. I have three that will take 128GB (but have 64, 40, 32), one that will take 64GB (but has 16), and three that will take 8GB (and have 8).
ID: 105886 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1217
Credit: 13,359,660
RAC: 431
Message 105887 - Posted: 9 Apr 2022, 5:36:45 UTC - in response to Message 105885.  
Last modified: 9 Apr 2022, 5:44:00 UTC

Q9450 Doesn't support AVX. Virtualbox jobs get stuck if avx is missing.
Not on cosmology or LHC, that's only a Rosetta requirement. I have 5 computers with no AVX and they all run Cosmology and LHC VB tasks fine.

Which Virtualbox program does this?
I'm using version 5.2.44, from here: https://www.virtualbox.org/wiki/Download_Old_Builds_5_2 It's 64 bit.

Also, the Cosmology tasks on my computer do not appear to use any Virtualbox program at all. Most of them use the multithreaded feature, which divides the work among 4 virtual CPU cores, without using a Virtualbox program.
That can't be be right. If the tasks are called "Cosmology Legacy", they're not Vbox and use 1 core. The Vbox ones are the only multicore ones. Linux might be different if you use that (but it appears you're on Windows only).

I'm on Windows only,

Where's any evidence that the multicore Cosmology tasks use Virtualbox?

I've tried to find memory boards compatible with my computer but with more memory each. I didn't find any.
ID: 105887 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,169,950
RAC: 9,484
Message 105888 - Posted: 9 Apr 2022, 5:47:33 UTC - in response to Message 105887.  
Last modified: 9 Apr 2022, 5:50:51 UTC

I've looked for memory boards compatible with my computer but with more memory each and didn't find any. Crucial gives me a page not found error when I try there.
If you want say a single stick of 32GB DDR4 3200, type this into Ebay:
1x32GB DDR4 3200

Or if you want two 16GB sticks of 2600:
2x16GB DDR4 2600

Look what motherboard and CPU you have and decide what the max size and speed sticks they will take are. Let me know the CPU and MB models if you want me to find out.

Where's any evidence that the multicore Cosmology tasks use Virtualbox?
This is the only types I've ever received:

Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2281906_1648978147.594575_0	00:11:31 (00:55:39)	04-04-2022 12:25 AM	04-04-2022 12:25 AM	6C	80.54	Reported: OK +	Xeon 1			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2282323_1648978163.467520_0	00:09:20 (00:47:28)	04-04-2022 03:40 AM	04-04-2022 03:40 AM	6C	84.76	Reported: OK +	Xeon 1			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2280998_1648978097.037935_0	00:11:03 (00:56:54)	04-04-2022 03:51 AM	04-04-2022 03:53 AM	6C	85.82	Reported: OK +	Xeon 1			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2282271_1648978161.905174_0	00:10:23 (00:52:24)	04-04-2022 03:50 AM	04-04-2022 03:53 AM	6C	84.11	Reported: OK +	Xeon 1			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2281924_1648978148.115440_0	00:10:44 (00:56:44)	04-04-2022 04:01 AM	04-04-2022 04:02 AM	6C	88.10	Reported: OK +	Xeon 1			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2282764_1648978177.782067_0	00:09:22 (00:47:37)	04-04-2022 03:59 AM	04-04-2022 04:01 AM	6C	84.73	Reported: OK +	Xeon 1			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_1534696_1647543101.040648_0	00:16:26 (01:48:42)	17-03-2022 07:33 PM	17-03-2022 08:35 PM	8C	82.68	Reported: OK	Xeon 2			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_1533456_1647543016.270791_0	00:16:44 (01:53:57)	17-03-2022 08:28 PM	17-03-2022 08:35 PM	8C	85.12	Reported: OK	Xeon 2			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_1533220_1647543009.642839_0	00:29:44 (02:44:12)	17-03-2022 08:10 PM	17-03-2022 08:35 PM	8C	69.03	Reported: OK	Xeon 2

and

Cosmology@Home	camb_legacy	wu_031822_080624_0_0_0	11:15:54 (11:10:24)	20-03-2022 09:44 PM	20-03-2022 09:44 PM		99.19	Reported: OK +	Ryzen			
Cosmology@Home	camb_legacy	wu_032922_124412_2_0_0	07:02:49 (07:01:03)	30-03-2022 02:29 AM	30-03-2022 08:17 PM		99.58	Reported: OK	Ryzen			
Cosmology@Home	camb_legacy	wu_032922_124409_0_0_0	07:04:07 (07:02:22)	30-03-2022 02:30 AM	30-03-2022 08:17 PM		99.59	Reported: OK	Ryzen			
Cosmology@Home	camb_legacy	wu_032922_124334_0_0_0	09:43:32 (09:41:25)	30-03-2022 05:09 AM	30-03-2022 08:17 PM		99.64	Reported: OK	Ryzen			
Cosmology@Home	camb_legacy	wu_032922_124419_1_0_0	11:27:54 (11:25:41)	30-03-2022 06:54 AM	30-03-2022 08:17 PM		99.68	Reported: OK	Ryzen			
Cosmology@Home	camb_legacy	wu_032922_124318_0_0_0	08:56:56 (08:55:02)	30-03-2022 04:24 AM	30-03-2022 08:17 PM		99.65	Reported: OK	Ryzen			
Cosmology@Home	camb_legacy	wu_032922_123743_0_0_0	07:12:05 (07:10:26)	30-03-2022 02:39 AM	30-03-2022 08:17 PM		99.62	Reported: OK	Ryzen			
Cosmology@Home	camb_legacy	wu_032922_123737_0_0_0	07:32:27 (07:30:48)	30-03-2022 02:59 AM	30-03-2022 08:17 PM		99.64	Reported: OK	Ryzen			
Cosmology@Home	camb_legacy	wu_031722_143954_1_0_0	07:24:06 (07:23:01)	18-03-2022 12:41 AM	18-03-2022 01:41 AM		99.76	Reported: OK	Xeon 1

6 cores (or whatever you choose) for Vbox and 1 core for legacy.
ID: 105888 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1217
Credit: 13,359,660
RAC: 431
Message 105889 - Posted: 9 Apr 2022, 6:21:14 UTC - in response to Message 105888.  
Last modified: 9 Apr 2022, 6:22:51 UTC

I've looked for memory boards compatible with my computer but with more memory each and didn't find any. Crucial gives me a page not found error when I try there.
If you want say a single stick of 32GB DDR4 3200, type this into Ebay:
1x32GB DDR4 3200

Or if you want two 16GB sticks of 2600:
2x16GB DDR4 2600

Look what motherboard and CPU you have and decide what the max size and speed sticks they will take are. Let me know the CPU and MB models if you want me to find out.

Too late at night here to check now.

Where's any evidence that the multicore Cosmology tasks use Virtualbox?
This is the only types I've ever received:

Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2281906_1648978147.594575_0	00:11:31 (00:55:39)	04-04-2022 12:25 AM	04-04-2022 12:25 AM	6C	80.54	Reported: OK +	Xeon 1			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2282323_1648978163.467520_0	00:09:20 (00:47:28)	04-04-2022 03:40 AM	04-04-2022 03:40 AM	6C	84.76	Reported: OK +	Xeon 1			

The 6C implies 6 cores per task and says nothing about whether Virtualbox is used.

6 cores (or whatever you choose) for Vbox and 1 core for legacy.

We agree on the legacy tasks.
ID: 105889 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,169,950
RAC: 9,484
Message 105890 - Posted: 9 Apr 2022, 6:33:15 UTC - in response to Message 105889.  
Last modified: 9 Apr 2022, 6:34:36 UTC

Look what motherboard and CPU you have and decide what the max size and speed sticks they will take are. Let me know the CPU and MB models if you want me to find out.

Too late at night here to check now.
I go to bed at noon and get up at midnight :-)

This is the only types I've ever received:

Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2281906_1648978147.594575_0	00:11:31 (00:55:39)	04-04-2022 12:25 AM	04-04-2022 12:25 AM	6C	80.54	Reported: OK +	Xeon 1			
Cosmology@Home	camb_boinc2docker	camb_boinc2docker_2282323_1648978163.467520_0	00:09:20 (00:47:28)	04-04-2022 03:40 AM	04-04-2022 03:40 AM	6C	84.76	Reported: OK +	Xeon 1			
The 6C implies 6 cores per task and says nothing about whether Virtualbox is used.
They are all Vbox, it's just not shown on the Boinctasks history. I've never seen anything multicore that doesn't have (vbox64) at the end of the application name.
ID: 105890 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
computezrmle

Send message
Joined: 9 Dec 11
Posts: 62
Credit: 9,680,103
RAC: 0
Message 105892 - Posted: 9 Apr 2022, 7:29:46 UTC

robertmiles wrote:
Virtualbox has the old vbox, which handles 32-bit programs
.
.
.
Where's any evidence that the multicore Cosmology tasks use Virtualbox?

It's mentioned at the apps overview page of any project:
https://boinc.bakerlab.org/rosetta/apps.php
https://www.cosmologyathome.org/apps.php


no "()" in the "Version" col indicates a native/legacy app without special requirements like SSE, AVX ...
(vbox64) in the "Version" col indicates an app that requires VirtualBox 64-bit.
(vbox64_mt) in the "Version" col indicates an app that requires VirtualBox 64-bit and is able to run multicore VMs.
64-bit vbox apps don't run on a computer that has 32-bit vbox installed.
ID: 105892 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
kotenok2000
Avatar

Send message
Joined: 22 Feb 11
Posts: 183
Credit: 103,927
RAC: 3
Message 105894 - Posted: 9 Apr 2022, 10:21:00 UTC - in response to Message 105890.  

I saw multithreaded yafu, primegrid LLR and amicable numbers.
ID: 105894 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,169,950
RAC: 9,484
Message 105895 - Posted: 9 Apr 2022, 10:22:28 UTC - in response to Message 105894.  

I saw multithreaded yafu, primegrid LLR and amicable numbers.
Yes, many projects can multithread without virtualpox. Milkyway is another.
ID: 105895 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 1905
Credit: 35,075,502
RAC: 6,283
Message 105897 - Posted: 10 Apr 2022, 1:28:26 UTC - in response to Message 105624.  
Last modified: 10 Apr 2022, 1:28:42 UTC

Sid, again aagb

Greg: not sure if this is the right msg to refer back to, but returning home after 3 days away, I found 3 tasks that had stalled
One task was aagb as discussed before, but the other two were aaam.
After 3 days of 24hr running, and having 10 tasks run at a time, this is comparatively good and 7 out of 10 of my current tasks are aagb all running fine, indicating problem tasks aren't quite the way we assumed before.
Those three and only one other task (after a reboot) errored out in the last 75 hours

For what that's worth (possibly nothing)
ID: 105897 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile bcavnaugh
Avatar

Send message
Joined: 7 Dec 13
Posts: 7
Credit: 2,389,640
RAC: 0
Message 105900 - Posted: 10 Apr 2022, 16:01:51 UTC
Last modified: 10 Apr 2022, 16:02:40 UTC

For me no more (vbox64) Task from this project.
It causes issues with the BOINC Client Software and it is not well programed to run under the BOINC.Client Software.
ID: 105900 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1669
Credit: 6,549,701
RAC: 2,502
Message 105903 - Posted: 10 Apr 2022, 19:41:51 UTC - in response to Message 105895.  

I saw multithreaded yafu, primegrid LLR and amicable numbers.
Yes, many projects can multithread without virtualpox. Milkyway is another.


Some years ago, a R@H admin said that is almost impossible to have a multithread app.
But i don't if it is possible with the new python app...
ID: 105903 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
kotenok2000
Avatar

Send message
Joined: 22 Feb 11
Posts: 183
Credit: 103,927
RAC: 3
Message 105904 - Posted: 10 Apr 2022, 20:02:16 UTC - in response to Message 105903.  

Fortunately we can run folding@home if we want multithreaded protein simulation.
ID: 105904 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,169,950
RAC: 9,484
Message 105906 - Posted: 11 Apr 2022, 1:35:35 UTC - in response to Message 105904.  
Last modified: 11 Apr 2022, 1:36:06 UTC

Fortunately we can run folding@home if we want multithreaded protein simulation.
Yes, folding at home is doing everything not possible elsewhere, funny that. Not all programmers are created equal. In it's simplest form, of course it's possible, if you want to download a 24 core protein task , the server can send you a batch of 24 things to do, and run them all at the same time. This is in fact how LHC ATLAS works. It you look inside the virtual machine, there are 200 tasks to do. It sets off doing 1 at a time on each core until they've all run out.
ID: 105906 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile vaughan

Send message
Joined: 17 Sep 05
Posts: 4
Credit: 21,277,287
RAC: 53
Message 105914 - Posted: 12 Apr 2022, 5:22:44 UTC
Last modified: 12 Apr 2022, 5:24:55 UTC

Trying to complete a task remaining from the Formula BOINC sprint. It is a aagb-SAR-AIB-GPN task at 3d 3hr 35min at 99.991% completed with 26 seconds to go. However, it doesn't make fast progress.
I am reluctant to abort it as it has tied up my laptop for such a long time. CPU is set to 100 percent. CPU is Intel I7-9750H @ 2.6Ghz, 32GB dual channel RAM. Rosetta reports vbox64 application in use. VB is 6.1.32 r149290
ID: 105914 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,169,950
RAC: 9,484
Message 105915 - Posted: 12 Apr 2022, 6:26:15 UTC - in response to Message 105914.  

Trying to complete a task remaining from the Formula BOINC sprint. It is a aagb-SAR-AIB-GPN task at 3d 3hr 35min at 99.991% completed with 26 seconds to go. However, it doesn't make fast progress.
I am reluctant to abort it as it has tied up my laptop for such a long time. CPU is set to 100 percent. CPU is Intel I7-9750H @ 2.6Ghz, 32GB dual channel RAM. Rosetta reports vbox64 application in use. VB is 6.1.32 r149290
Is it actually using CPU time? Check the task manager. If it isn't, it's fucked.
ID: 105915 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1828
Credit: 107,699,595
RAC: 8,812
Message 105916 - Posted: 12 Apr 2022, 17:36:44 UTC - in response to Message 105914.  

Yeah unfortunately there are lots of bad VirtualBox tasks that waste lots of CPU time. I don't know of any way to deal with them other than to abort them unfortunately.
ID: 105916 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1479
Credit: 7,169,950
RAC: 9,484
Message 105918 - Posted: 13 Apr 2022, 3:39:16 UTC - in response to Message 105916.  

Yeah unfortunately there are lots of bad VirtualBox tasks that waste lots of CPU time. I don't know of any way to deal with them other than to abort them unfortunately.
I don't experience this. I have two computers that do 99% of them ok. I have 5 computers that never do them. Perhaps there's an instruction in CPUs required for a small number, but all my machines are either new enough to have it or old enough to break every one of them.
ID: 105918 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 200 · 201 · 202 · 203 · 204 · 205 · 206 . . . 257 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2022 University of Washington
https://www.bakerlab.org