Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 71 · 72 · 73 · 74 · 75 · 76 · 77 . . . 276 · Next

AuthorMessage
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,720,516
RAC: 8,867
Message 98940 - Posted: 9 Sep 2020, 21:25:00 UTC - in response to Message 98865.  
Last modified: 9 Sep 2020, 21:25:26 UTC

Regarding resource share settings......I have Rosetta at 1 and WCG at 9999, and yet Rosetta still takes control and suspends WCG tasks during this transition between machines. I am glad the BOINC client works for you 100% as intended. Which I am sure you have tested. Meanwhile I'll simply continue to complicate things.


And so will I, as like you Boinc never does what I ask. You join Rosetta at 1 and it panics, thinking it's not done any over the last 10 days (WTF?) but it should do a 1/10000. So it runs it at 100%. Changing what projects you run and what the weighting is should reset the counter. I changed this on mine to make things slightly more sensible, in config.xml: <rec_half_life_days>1.000000</rec_half_life_days> - this means it looks at the last day instead of the last 10 days to figure out what to run.

PS: Click on a person's name and it shows everything they have EVER done.


It shows you with a recent credit on over a dozen. I guess it's an average over quite some time - I think it's a month.
ID: 98940 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1894
Credit: 8,767,285
RAC: 10,641
Message 98941 - Posted: 9 Sep 2020, 22:48:54 UTC - in response to Message 98939.  

Peter Huycker said
It's a pity Boinc doesn't manage multiple computers and we have to use third party programs to do so. I use Boinctasks, and in fact I'd use it for a single machine too, because it's display is 10 times better than Boinc. For a start it colour codes running, queued, etc, and collapses a queue of 50 tasks into one line. The actual Boinc manager is unusable as an interface.


I hope that's an option if it's ever implemented as I much prefer to manage each pc as it's own computer and then choose whether to run the same project as other computers or to run it's own set of projects. Sometimes I have every pc running the same project while other times I prefer to run something different on each pc.

In some cases I just can't blast thru the tasks at a project because I am a 'Team friendly person' meaning if you are on my team and I am behind you I will not pass you as long as you are crunching. It's the old thing 'just because I can doesn't mean I should' for me!!! I have more resources than any otehr cruncher on my team and could be easily #1 on every running project I crunch for but that then ruins the incentive for my teammates to keep on crunching because they are #1 or #2. I am already #1 at enough projects that I can move things around to keep the pressure on but not pass them. ie at PrimeGrid a teammate is almost 30 million credits ahead of me but a new challenge could easily see me doing 5 million credits a week if I really want too. He just doesn't have, nor can he afford, the kind of horsepower needed to keep me behind him, but he keeps crunching the way he is and that's a good thing for Boinc in general!!
ID: 98941 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,720,516
RAC: 8,867
Message 98978 - Posted: 12 Sep 2020, 20:37:24 UTC - in response to Message 98941.  

Peter Huycker said
It's a pity Boinc doesn't manage multiple computers and we have to use third party programs to do so. I use Boinctasks, and in fact I'd use it for a single machine too, because it's display is 10 times better than Boinc. For a start it colour codes running, queued, etc, and collapses a queue of 50 tasks into one line. The actual Boinc manager is unusable as an interface.


I hope that's an option if it's ever implemented as I much prefer to manage each pc as it's own computer and then choose whether to run the same project as other computers or to run it's own set of projects. Sometimes I have every pc running the same project while other times I prefer to run something different on each pc.

In some cases I just can't blast thru the tasks at a project because I am a 'Team friendly person' meaning if you are on my team and I am behind you I will not pass you as long as you are crunching. It's the old thing 'just because I can doesn't mean I should' for me!!! I have more resources than any otehr cruncher on my team and could be easily #1 on every running project I crunch for but that then ruins the incentive for my teammates to keep on crunching because they are #1 or #2. I am already #1 at enough projects that I can move things around to keep the pressure on but not pass them. ie at PrimeGrid a teammate is almost 30 million credits ahead of me but a new challenge could easily see me doing 5 million credits a week if I really want too. He just doesn't have, nor can he afford, the kind of horsepower needed to keep me behind him, but he keeps crunching the way he is and that's a good thing for Boinc in general!!


You've never seen Boinctasks have you? You can do exactly what you want aswell as what I want. I can select one computer, a few, or all of them, and give them an instruction. I like you set different machines doing different things. Some are better at certain projects, some can't do them at all.
ID: 98978 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Aravah

Send message
Joined: 12 Apr 20
Posts: 6
Credit: 1,101,172
RAC: 0
Message 98987 - Posted: 13 Sep 2020, 9:23:17 UTC

I am seeing Rosetta task requesting much much more memory than usual?
Is this expected?
Application Rosetta 4.20
Name kp8RjDVk_fold_and_dock_SAVE_ALL_OUT_1009390_4381
State Waiting for memory
Received Thu 10 Sep 2020 16:00:19 BST
Report deadline Sun 13 Sep 2020 16:00:18 BST
Estimated computation size 80,000 GFLOPs
CPU time 01:09:16
CPU time since checkpoint ---
Elapsed time 01:17:21
Estimated time remaining 07:32:17
Fraction done 5.772%
Virtual memory size 33.02 GB
Working set size 28.44 GB
Directory slots/6
Progress rate 4.320% per hour
Executable rosetta_4.20_x86_64-pc-linux-gnu
ID: 98987 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
MarkJ

Send message
Joined: 28 Mar 20
Posts: 72
Credit: 25,010,478
RAC: 43
Message 98988 - Posted: 13 Sep 2020, 9:40:06 UTC - in response to Message 98987.  

I am seeing Rosetta task requesting much much more memory than usual?
Is this expected?
Application Rosetta 4.20
Name kp8RjDVk_fold_and_dock_SAVE_ALL_OUT_1009390_4381
State Waiting for memory

Most of us had these earlier in the week. I aborted all the fold_and_dock tasks. They seem to have a serious problem with the amount of memory they need.
BOINC blog
ID: 98988 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 98989 - Posted: 13 Sep 2020, 9:47:09 UTC - in response to Message 98987.  

Is this expected?
Several users have reported fold_and_dock tasks trying (and usually eventually failing) to allocate tens of gigabytes of memory. If you have a vast amount of swap space they might be able to complete, but you’re probably as well just aborting them and doing something else instead.
ID: 98989 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1482
Credit: 14,623,003
RAC: 16,029
Message 98990 - Posted: 13 Sep 2020, 10:42:39 UTC - in response to Message 98987.  

I am seeing Rosetta task requesting much much more memory than usual?
Is this expected?
Because of the size of your cache, and Rosetta being your secondary project, it's taken you several days to start processing what was a resend of those problem Tasks.
Grant
Darwin NT
ID: 98990 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,720,516
RAC: 8,867
Message 98994 - Posted: 13 Sep 2020, 18:02:21 UTC - in response to Message 98989.  
Last modified: 13 Sep 2020, 18:03:11 UTC

Is this expected?
Several users have reported fold_and_dock tasks trying (and usually eventually failing) to allocate tens of gigabytes of memory. If you have a vast amount of swap space they might be able to complete, but you’re probably as well just aborting them and doing something else instead.


Don't most people have way more than 10s of GB of swap space? Most people have their OS set to the default of using whatever is required. One of my machines has 2TB free, two have 1TB free, and the other three have about 200GB free. I assume those tasks would have a shot at running on my machines. I don't know if I've had any, and may not have noticed. I only notice if something isn't using the processor very well (the CPU time and wall time are drastically different in Boinctasks), or if it's crashing (the task changes colour and shows "computation error").
ID: 98994 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 98996 - Posted: 13 Sep 2020, 19:21:12 UTC - in response to Message 98994.  
Last modified: 13 Sep 2020, 19:55:46 UTC

What’s not clear is whether those tasks legitimately need all the memory they’re allocating (in which case they should run to completion on a system with sufficient resources) – or whether it’s a bug, and the tasks will simply keep requesting more memory (however much the system makes available) until they fail.

I disable the paging file on every machine I operate. If that causes anything to fail to run, I need more RAM!
ID: 98996 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1224
Credit: 13,848,401
RAC: 2,043
Message 98997 - Posted: 13 Sep 2020, 19:41:11 UTC - in response to Message 98994.  

Is this expected?
Several users have reported fold_and_dock tasks trying (and usually eventually failing) to allocate tens of gigabytes of memory. If you have a vast amount of swap space they might be able to complete, but you’re probably as well just aborting them and doing something else instead.


Don't most people have way more than 10s of GB of swap space? Most people have their OS set to the default of using whatever is required. One of my machines has 2TB free, two have 1TB free, and the other three have about 200GB free. I assume those tasks would have a shot at running on my machines. I don't know if I've had any, and may not have noticed. I only notice if something isn't using the processor very well (the CPU time and wall time are drastically different in Boinctasks), or if it's crashing (the task changes colour and shows "computation error").

You appear to be assuming that the available swap space is all of the unused space on the hard drive. I doubt if this is correct.

However, I just tried checking my Windows 10 computer, and it appears that Windows 10 no longer uses anything called swap space.

I remember that some previous versions of Windows had a reserved area on the hard drive known as swap space,
ID: 98997 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 98998 - Posted: 13 Sep 2020, 19:55:30 UTC - in response to Message 98997.  
Last modified: 13 Sep 2020, 19:58:09 UTC

‘Swap space’ is the name for disk-space-as-memory in UNIX-family operating systems.

Windows calls it a ‘paging file’, and it’s still very much present in Windows 10.
(Control Panel > System > Advanced system settings > Advanced tab > Performance group > Advanced tab > Virtual memory group)
ID: 98998 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,720,516
RAC: 8,867
Message 98999 - Posted: 13 Sep 2020, 19:56:07 UTC - in response to Message 98996.  

What’s not clear is whether those tasks legitimately need all the memory they’re allocating (in which case they should run to completion on a system with sufficient resources) – or whether it’s a bug, and the tasks will simply keep requesting more memory (however much the system makes available) until they fail.

I disable the page file on every machine I operate. If that causes anything to fail to run, I need more RAM!


If you have an SSD, paging isn't as bad as it used to be with those rust spinners, where if it started being used, the interface became so slow you weren't sure if what you clicked was being done slowly or it hadn't noticed, so you click again then it ends up loading 5 of them slowing it down even further, and you spend so much time waiting for the computer you have no time to go to work to earn money to buy more RAM.
ID: 98999 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,720,516
RAC: 8,867
Message 99000 - Posted: 13 Sep 2020, 19:58:08 UTC - in response to Message 98997.  

Is this expected?
Several users have reported fold_and_dock tasks trying (and usually eventually failing) to allocate tens of gigabytes of memory. If you have a vast amount of swap space they might be able to complete, but you’re probably as well just aborting them and doing something else instead.


Don't most people have way more than 10s of GB of swap space? Most people have their OS set to the default of using whatever is required. One of my machines has 2TB free, two have 1TB free, and the other three have about 200GB free. I assume those tasks would have a shot at running on my machines. I don't know if I've had any, and may not have noticed. I only notice if something isn't using the processor very well (the CPU time and wall time are drastically different in Boinctasks), or if it's crashing (the task changes colour and shows "computation error").

You appear to be assuming that the available swap space is all of the unused space on the hard drive. I doubt if this is correct.

However, I just tried checking my Windows 10 computer, and it appears that Windows 10 no longer uses anything called swap space.

I remember that some previous versions of Windows had a reserved area on the hard drive known as swap space,


On this Windows 10 computer (and all of them unless you change it), paging is automatic. It will increase it if it needs it, but doesn't reserve a chunk of disk for it.
ID: 99000 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 1224
Credit: 13,848,401
RAC: 2,043
Message 99001 - Posted: 13 Sep 2020, 22:52:17 UTC - in response to Message 98998.  

‘Swap space’ is the name for disk-space-as-memory in UNIX-family operating systems.

Windows calls it a ‘paging file’, and it’s still very much present in Windows 10.
(Control Panel > System > Advanced system settings > Advanced tab > Performance group > Advanced tab > Virtual memory group)

Thanks.

Windows 10 made it hard to find the right control panel if you haven't used it lately!

Try Windows System / System and Security / System for the first few steps.

About 4,8 GB currently on my computer.
ID: 99001 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,720,516
RAC: 8,867
Message 99002 - Posted: 13 Sep 2020, 23:11:16 UTC - in response to Message 99001.  
Last modified: 13 Sep 2020, 23:12:02 UTC

‘Swap space’ is the name for disk-space-as-memory in UNIX-family operating systems.

Windows calls it a ‘paging file’, and it’s still very much present in Windows 10.
(Control Panel > System > Advanced system settings > Advanced tab > Performance group > Advanced tab > Virtual memory group)

Thanks.

Windows 10 made it hard to find the right control panel if you haven't used it lately!

Try Windows System / System and Security / System for the first few steps.

About 4,8 GB currently on my computer.


Not sure why Microsoft has split all the controls up and put them in stupid places that only they think make sense. I often have to use the search to find the simplest of things. I want the old control panel with a sensible list. It's usually actually quicker to google the problem.

That 4.8GB should automatically increase as required though. At least we never get "out of memory" messages like in very early Windows.
ID: 99002 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Aravah

Send message
Joined: 12 Apr 20
Posts: 6
Credit: 1,101,172
RAC: 0
Message 99006 - Posted: 14 Sep 2020, 22:49:50 UTC

Thanks to all who replied to my previous message. I aborted that task and all has returned to normal :) Except I notice some tasks are awarded very few credits for the same Gflops as other tasks that are give an order of magnitude more.
For example, just 35.12 credits for the first one but 326.58 for the second. It is to do with whether you're reporting the task result first or second?

Name rb_09_10_37482_36690__t000__1_C1_SAVE_ALL_OUT_IGNORE_THE_REST_1010270_821_0
Workunit 1127882972
Created 12 Sep 2020, 0:02:14 UTC
Sent 12 Sep 2020, 0:07:00 UTC
Report deadline 15 Sep 2020, 0:07:00 UTC
Received 14 Sep 2020, 4:25:44 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 4466108
Run time 12 hours 55 min 41 sec
CPU time 11 hours 4 min
Validate state Valid
Credit 35.12
Device peak FLOPS 3.09 GFLOPS
Application version Rosetta v4.20
x86_64-pc-linux-gnu
Peak working set size 1,135.22 MB
Peak swap size 1,276.45 MB
Peak disk usage 29.31 MB

Name pdl1_graft_v1_SAVE_ALL_OUT_IGNORE_THE_REST_8ae9bz6m_1009438_2_0
Workunit 1127762369
Created 11 Sep 2020, 18:09:26 UTC
Sent 11 Sep 2020, 19:34:11 UTC
Report deadline 14 Sep 2020, 19:34:11 UTC
Received 14 Sep 2020, 2:58:16 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 4466108
Run time 11 hours 49 min 45 sec
CPU time 9 hours 59 min 46 sec
Validate state Valid
Credit 326.58
Device peak FLOPS 3.09 GFLOPS
Application version Rosetta v4.20
x86_64-pc-linux-gnu
Peak working set size 1,010.59 MB
Peak swap size 1,143.78 MB
Peak disk usage 30.09 MB
ID: 99006 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 99007 - Posted: 14 Sep 2020, 23:18:55 UTC - in response to Message 99006.  

Rosetta tasks are almost never sent to more than one computer. The low credit one looks like a bug which crops up occasionally: somehow the task ran more than once, and only the last (very short) run received credit, even though it was the first run that did almost all the work.
ID: 99007 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 9,720,516
RAC: 8,867
Message 99009 - Posted: 15 Sep 2020, 10:35:23 UTC - in response to Message 99007.  

Rosetta tasks are almost never sent to more than one computer. The low credit one looks like a bug which crops up occasionally: somehow the task ran more than once, and only the last (very short) run received credit, even though it was the first run that did almost all the work.


Is this just a credit bug? Is the data from the big first run still saved?
ID: 99009 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 99010 - Posted: 15 Sep 2020, 11:53:13 UTC - in response to Message 99009.  

Good question. I don’t know whether the results from the second run are added to, or overwrite, those from the first.
ID: 99010 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Breno

Send message
Joined: 8 Apr 20
Posts: 28
Credit: 11,068,791
RAC: 11,938
Message 99012 - Posted: 15 Sep 2020, 16:46:43 UTC

I don't really know if this is a persistent issue. I noticed today that some finished foldit WUs stopped at 45,78% during results upload transfers, they allow re-attempts to send results, but doing so manually causes a very slow upload even w/o any upload transfer limits. Anyway, just thought team R@h should know.
ID: 99012 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 71 · 72 · 73 · 74 · 75 · 76 · 77 . . . 276 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org