Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 59 · 60 · 61 · 62 · 63 · 64 · 65 . . . 71 · Next

AuthorMessage
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 930
Credit: 10,498,888
RAC: 4,194
Message 97708 - Posted: 27 Jun 2020, 20:31:13 UTC - in response to Message 97698.  

12 CPU WUs hogging up my PC, using only 1 cpu core.
I will for the time being disconnect from this project until the issue is resolved.

That probably indicates that you have told BOINC Manager that it can use only one CPU core.

In Advanced view, click on Options, then Computing preferences. Adjust the fraction of the available CPU cores (show as CPUs here), possibly adding 1% to the fraction you really want to keep roundoff from causing problems.

After adjusting this, click on Save at the bottom of that window.
ID: 97708 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 930
Credit: 10,498,888
RAC: 4,194
Message 97709 - Posted: 27 Jun 2020, 20:35:39 UTC - in response to Message 97701.  
Last modified: 27 Jun 2020, 20:54:53 UTC


I turned that nonsense off. Go into Boinc's properties and change the "switch between applications" to a huge number. I set mine to a year. I do not want stuff changing before it's finished.

Thanks for the tip, Peter. I switched it from 120 minutes to 1,000 for now to see what happens, double what I've observed as the process time for a Rosetta task. And now I'm suspending my second project temporarily to see if Rosetta resumes the task when the screensaver kicks in.
Eric

If you use a huge value here, DON'T add project CPDN or project RNA World without first setting a smaller value instead. Both of these projects have tasks that run for months, so the huge value will keep any other projects from getting turns for CPU use.

Note that if you have enough main memory and have the setting to keep tasks in main memory even when not running, you won't be using the disk checkpoints except except when BOINC restarts. The project applications need to be able to adjust any timeouts when restarting from pauses, unless the timeouts are based on time when running rather than clock time.
ID: 97709 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
EHM-1
Avatar

Send message
Joined: 21 Mar 20
Posts: 23
Credit: 47,222
RAC: 642
Message 97710 - Posted: 27 Jun 2020, 21:19:51 UTC - in response to Message 97704.  
Last modified: 27 Jun 2020, 21:28:47 UTC

@EricM:

It seems that BOINC is having trouble deciding how much work to download for your computer. As I understand it, that decision is based in part on what BOINC has seen the computer complete in the past. Does the machine have an irregular usage pattern? (Powered off frequently/​irregularly? Variable amount of other work being done while BOINC is running?) The missed deadlines and high average turnaround time (2.81 days: only just within the 3-⁠day deadline) may be contributing.

Check all your Computing preferences. Post values/​screenshots here, and we might spot something amiss. Regarding pauses: do you have any restrictions in your Daily schedules settings?

To try to get some tasks, try increasing Store at least N days of work. Do it in steps: add around 0.4 (slightly more than one 8-⁠hour task time), save, wait a couple of minutes for BOINC to contact the server, and see if it downloads some tasks. If not, repeat. As soon as you get some tasks, reduce N again to maybe 0.3 (slightly less than one task) to avoid your machine getting flooded with work it cannot complete until BOINC learns better how long each task will take. Set Store up to an additional to 0, as at this stage the last thing you need is BOINC using poor estimates to opportunistically download even more work.

Thanks to the three of you for your input!

@Peter, your 17-hour comment addresses a question I almost posed. It would be good of the Lords of BOINC to make clear in the settings if the application switching interval setting refers to elapsed time or task run-time.

@Brian, I was planning to implement your suggestions, but first waited to see how BOINC would react to my having suspended World Community Grid / OpenPandemics (my only other active project). As I suspected might occur, Rosetta started processing again at screensaver invocation, and I noted that BOINC had fetched four new Rosetta units. Of course I don't know if that was prompted by the suspension and/or the result of my having changed the application switching interval, or mere coincidence.
Given this, would you say I should still change my work storage settings?

To answer your questions:

  • My computer usage would be irregular seen over a couple decades, but is more regular the past few years. Typically, the computer is on 15 hours a day. Sometimes I'm working with it for much of the day, during which BOINC does not run. Other times, the sceensaver might come on 5-10 times per day and run for an hour each time. I have the screensaver set to come on after ten minutes, then I think the comp sleeps after an hour. But, depending on how far back we go -- I ran SETI@home for 20 years -- there are times when the computer is off for a few days or a few weeks, and there was a ten-year era when it would often be running only on weekends.
  • See my computing preferences screenshot below.


@Robert, I think your comment regarding deadlines and the server determining reliability of my computer could explain at least part of the reason for Rosetta being stalled the past few days. I was away Sun-Tue, so maybe the server gave up on the partially completed work units on my machine? If the cause is something like that, it makes me think that Rosetta is less tolerant that SETI was in this regard.
Eric



system: up-to-date Windows 10, Intel quad-core 3.6 GHz processor, 8 GB RAM
ID: 97710 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 930
Credit: 10,498,888
RAC: 4,194
Message 97715 - Posted: 27 Jun 2020, 21:41:04 UTC - in response to Message 97710.  
Last modified: 27 Jun 2020, 22:13:52 UTC

EricM,

@Robert, I think your comment regarding deadlines and the server determining reliability of my computer could explain at least part of the reason for Rosetta being stalled the past few days. I was away Sun-Tue, so maybe the server gave up on the partially completed work units on my machine? If the cause is something like that, it makes me think that Rosetta is less tolerant that SETI was in this regard.
Eric

It's likely that the server gave up for the partially completed tasks. This may make it think your computer is unreliable at returning results fast enough.

Rosetta@home NEEDS results from previous tasks to generate most of the next round of workunits. Also, it considers COVID-19 work urgent. This means short deadlines are likely.

SETI recently decided to pause their last few years of work to analyze the results, and think about what to try next. In other words, they do not consider their work urgent, so long deadlines are likely.

Also, you might try changing your settings to use 50% (or 51% to avoid roundoff error) of the CPUs, but 100% of the CPU time. This will make it finish each task faster, although sometimes with fewer tasks in progress.

That would increase your chances of returning tasks on time.
ID: 97715 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 598
Credit: 2,846,362
RAC: 23,067
Message 97716 - Posted: 27 Jun 2020, 21:54:50 UTC - in response to Message 97695.  

When the user changes the weighting, it should have immediate effect.
I does, as you yourself noted.
Boinc went utterly mental and ran almost exclusively LHC, presumably doing some weird lookback over the last week and seeing it hadn't done any.


It takes time for changes to settle down as it now has to balance out the new debts & credits between projects to match the new settings. That takes time to do- ie the time necessary to process the work to produce the Credit to match the new Resource share settings.
Grant
Darwin NT
ID: 97716 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Stevie G

Send message
Joined: 15 Dec 18
Posts: 36
Credit: 122,003
RAC: 212
Message 97717 - Posted: 27 Jun 2020, 22:09:21 UTC - in response to Message 97676.  

Actually, today Rosetta disappeared from my project list. When I try to log on, it says it is unable.
This is a drag.

With no response from Rosetta, I added World Community Grid.

Steven Gaber
Oldsmar, FL
ID: 97717 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 598
Credit: 2,846,362
RAC: 23,067
Message 97718 - Posted: 27 Jun 2020, 22:09:53 UTC - in response to Message 97710.  

Typically, the computer is on 15 hours a day. Sometimes I'm working with it for much of the day, during which BOINC does not run.
Why?
Rosetta (like Seti) applications are set to run at Idle priority (the lowest level). Any other running programme of similar priority will get equal CPU resources. If it's priority is higher, then Rosetta applications will slow & even stop to allow the higher priority application to use the CPU resources.
If there is an application that is affected by having BOINC doing work in the background, you can use the Exclusive applications option to stop BOINC when just that particular application is running.

Back in the days of single core or just hyperthreaded systems, yeah you often needed to stop BOINC to allow other programmes to run OK. But with multi core/thread systems, unless the programme you are running is heavily multi threaded it just isn't necessary anymore.
Grant
Darwin NT
ID: 97718 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile robertmiles

Send message
Joined: 16 Jun 08
Posts: 930
Credit: 10,498,888
RAC: 4,194
Message 97719 - Posted: 27 Jun 2020, 22:21:56 UTC - in response to Message 97718.  

Typically, the computer is on 15 hours a day. Sometimes I'm working with it for much of the day, during which BOINC does not run.
Why?
Rosetta (like Seti) applications are set to run at Idle priority (the lowest level). Any other running programme of similar priority will get equal CPU resources. If it's priority is higher, then Rosetta applications will slow & even stop to allow the higher priority application to use the CPU resources.
If there is an application that is affected by having BOINC doing work in the background, you can use the Exclusive applications option to stop BOINC when just that particular application is running.

Back in the days of single core or just hyperthreaded systems, yeah you often needed to stop BOINC to allow other programmes to run OK. But with multi core/thread systems, unless the programme you are running is heavily multi threaded it just isn't necessary anymore.

I've found that this will handle CPU time problems, However, the paused BOINC applications usually do NOT release any memory they have allocated, unless you shut down BOINC. Therefore, your plan does not help if the important contention is for memory instead of for CPU time, unless you use the setting for releasing memory from BOINC applications that are not running.
ID: 97719 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
EHM-1
Avatar

Send message
Joined: 21 Mar 20
Posts: 23
Credit: 47,222
RAC: 642
Message 97720 - Posted: 27 Jun 2020, 22:26:11 UTC - in response to Message 97718.  
Last modified: 27 Jun 2020, 22:29:17 UTC

Thanks, @Robert and @Grant. I've never received such prompt and good input in a forum before.
Grant, I suspect the only program I use that might tax the processor at times would be ArcGIS, which might have moments when it could be considered heavily multi-threaded. I'm out of my depth trying to determine whether I'll regret giving BOINC more resource access.
Robert, I think I'll try your 51/100 settings tip. I wonder if I should alter the memory settings?
Eric

system: up-to-date Windows 10, Intel quad-core 3.6 GHz processor, 8 GB RAM
ID: 97720 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 287
Credit: 3,217,079
RAC: 20,667
Message 97721 - Posted: 27 Jun 2020, 22:34:23 UTC - in response to Message 97707.  

That would interfere with the way it recovers from times when one of the projects has no tasks available to send.

Instead, it looks back over the last few weeks, and tries to get tasks from whichever project would move it toward the new weighting.


I'm not sure I follow you. If I've made a choice that I want different weightings, I obviously want past history forgotten. I want from now on to have the ratio I set. Anything else is just confusing.

Imagine you ran project A at 10 and project B at 1. Then you decide one day that you want the reverse, A=1 and B=10. The way it works now, the history shows it's been doing way too much A, so it will consequently do B exclusively, which is not what you just asked it to do.
ID: 97721 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 287
Credit: 3,217,079
RAC: 20,667
Message 97722 - Posted: 27 Jun 2020, 22:38:33 UTC - in response to Message 97710.  

* My computer usage would be irregular seen over a couple decades, but is more regular the past few years. Typically, the computer is on 15 hours a day. Sometimes I'm working with it for much of the day, during which BOINC does not run. Other times, the sceensaver might come on 5-10 times per day and run for an hour each time. I have the screensaver set to come on after ten minutes, then I think the comp sleeps after an hour. But, depending on how far back we go -- I ran SETI@home for 20 years -- there are times when the computer is off for a few days or a few weeks, and there was a ten-year era when it would often be running only on weekends.
* See my computing preferences screenshot below.


Any particular reason you stop Boinc so much? I leave mine on all the time, with very few programs that it slows down, so I set those as exclusive applications to pause Boinc when they run. 3 games, and VLC media player. Some stop the GPU, some stop the CPU also.

Or if you're always running something that needs some CPU for your work, tell Boinc to use less than all the cores, leaving some for you.
ID: 97722 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 287
Credit: 3,217,079
RAC: 20,667
Message 97723 - Posted: 27 Jun 2020, 22:41:38 UTC - in response to Message 97719.  

I've found that this will handle CPU time problems, However, the paused BOINC applications usually do NOT release any memory they have allocated, unless you shut down BOINC. Therefore, your plan does not help if the important contention is for memory instead of for CPU time, unless you use the setting for releasing memory from BOINC applications that are not running.


Surely the paused Boinc programs would be shoved in the swap file by Windows?
ID: 97723 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
EHM-1
Avatar

Send message
Joined: 21 Mar 20
Posts: 23
Credit: 47,222
RAC: 642
Message 97724 - Posted: 27 Jun 2020, 22:43:15 UTC - in response to Message 97722.  

Any particular reason you stop Boinc so much? I leave mine on all the time, with very few programs that it slows down, so I set those as exclusive applications to pause Boinc when they run. 3 games, and VLC media player. Some stop the GPU, some stop the CPU also.

Or if you're always running something that needs some CPU for your work, tell Boinc to use less than all the cores, leaving some for you.

I guess when I set it up years ago, I just didn't want it to keep the computer running all day. But maybe I'll try giving it more time and resources.
Eric

system: up-to-date Windows 10, Intel quad-core 3.6 GHz processor, 8 GB RAM
ID: 97724 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 287
Credit: 3,217,079
RAC: 20,667
Message 97725 - Posted: 27 Jun 2020, 22:47:49 UTC - in response to Message 97724.  

Any particular reason you stop Boinc so much? I leave mine on all the time, with very few programs that it slows down, so I set those as exclusive applications to pause Boinc when they run. 3 games, and VLC media player. Some stop the GPU, some stop the CPU also.

Or if you're always running something that needs some CPU for your work, tell Boinc to use less than all the cores, leaving some for you.

I guess when I set it up years ago, I just didn't want it to keep the computer running all day. But maybe I'll try giving it more time and resources.
Eric


My 6 run 24/7. Up to 2.5kW (ouch). The plug they all run from is actually warm.
ID: 97725 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 85
Credit: 919,679
RAC: 10,912
Message 97726 - Posted: 27 Jun 2020, 22:52:29 UTC - in response to Message 97710.  

@EricM:

OK: this is becoming much more clear.

The fact that you now have Rosetta tasks suggests that it was indeed the other project that was getting priority. The missed deadlines are due to the other settings.

The project needs your computer to give each task 8 hours of CPU time inside 72 hours of wall time. From what you’ve written, tasks will be taking at best 38 hours to complete (maximum 10 hours per day, at 50% CPU time, and tasks not running at all at other times because either you’re using the computer or it’s asleep). To avoid BOINC downloading more work than it can complete, it’s essential to keep Store at least low. If your computer is frequently/​always connected to the Internet, it doesn’t matter: BOINC can always fetch more work when it needs it. You will not be overloading the project server by requesting work that infrequently. I would still recommend reducing Store up to an additional to zero, though in my experience that setting doesn’t seem to influence the amount of work downloaded in any event.
ID: 97726 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1562
Credit: 5,859,149
RAC: 195
Message 97727 - Posted: 27 Jun 2020, 22:55:40 UTC - in response to Message 97721.  
Last modified: 27 Jun 2020, 22:56:51 UTC

That would interfere with the way it recovers from times when one of the projects has no tasks available to send.

Instead, it looks back over the last few weeks, and tries to get tasks from whichever project would move it toward the new weighting.


Peter Hucker said:
I'm not sure I follow you. If I've made a choice that I want different weightings, I obviously want past history forgotten. I want from now on to have the ratio I set. Anything else is just confusing.

Imagine you ran project A at 10 and project B at 1. Then you decide one day that you want the reverse, A=1 and B=10. The way it works now, the history shows it's been doing way too much A, so it will consequently do B exclusively, which is not what you just asked it to do.


Easy answer to your hypothesis...the Boinc Developers made that formula and they are the only ones that can change it. Join there email group and ask them why it's the way it is, I'm sure Dr A will give you a response at some point.
ID: 97727 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 598
Credit: 2,846,362
RAC: 23,067
Message 97728 - Posted: 27 Jun 2020, 23:02:00 UTC - in response to Message 97720.  
Last modified: 27 Jun 2020, 23:03:08 UTC

Grant, I suspect the only program I use that might tax the processor at times would be ArcGIS, which might have moments when it could be considered heavily multi-threaded.
That's a pretty old FAQ, from the looks of it v10 can make use of extra cores & threads, but not much unless it is one of their server applications.
If you are running a more current version than v10, then i would expect it would make better use of extra cores & threads, possibly even GPU support (if it can make use of GPU support, and you have a supported GPU then i'd suggest taking advantage of that- anything up to a 100 times boost in performance is possible with a high end GPU v a high end CPU if the software can make use of it).


I'm out of my depth trying to determine whether I'll regret giving BOINC more resource access.
Only way to know is to try it and see- Set it so BOINC isn't suspended while the computer is in use, and see if it affects your other programmes, or if it affects system responsiveness over all.
If so, as suggested by Peter you could just reserve 1 or 2 CPU cores/threads for non BOINC use. That way BOINC will never make use of them, even if they aren't being uses by other programmes.
Grant
Darwin NT
ID: 97728 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 85
Credit: 919,679
RAC: 10,912
Message 97729 - Posted: 27 Jun 2020, 23:05:36 UTC - in response to Message 97725.  

Peter Hucker wrote:
24/7. Up to 2.5kW
That’s ≈£4,000 p.a. on electricity? Ouch indeed…
ID: 97729 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 287
Credit: 3,217,079
RAC: 20,667
Message 97731 - Posted: 27 Jun 2020, 23:09:50 UTC - in response to Message 97727.  

Easy answer to your hypothesis...the Boinc Developers made that formula and they are the only ones that can change it. Join there email group and ask them why it's the way it is, I'm sure Dr A will give you a response at some point.


Yeah ok.... they're not the listening type.
ID: 97731 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 598
Credit: 2,846,362
RAC: 23,067
Message 97732 - Posted: 27 Jun 2020, 23:10:24 UTC - in response to Message 97726.  

I would still recommend reducing Store up to an additional to zero, though in my experience that setting doesn’t seem to influence the amount of work downloaded in any event.
It does, just in very odd ways- It is best to have it as low as possible.

Having a large value for Additional days will result the cache running down significantly below the Store at least value, before it then gets more work to build it back up to the Store at least + Additional days value (and the larger each of those values, the greater the run down before it rebuilds the cache again). If people really feel a need for a cache, set it in Store at least value. The Additional days value should be as small as possible.
Weird things are much less likely to occur that way.
Grant
Darwin NT
ID: 97732 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 59 · 60 · 61 · 62 · 63 · 64 · 65 . . . 71 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2020 University of Washington
https://www.bakerlab.org