How does BOINC adjust initial runtime estimate?

Message boards : Number crunching : How does BOINC adjust initial runtime estimate?

To post messages, you must log in.

AuthorMessage
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 95088 - Posted: 22 Apr 2020, 0:04:22 UTC

I've completed and reported a number of tasks with Rosetta (not "mini") v4.15 the new 36hour runtime, but BOINC Manager still shows unstarted WUs with an estimated runtime of 7H27M.

I've been keeping very small cache of work, because experience tells me that if I set things to try to keep a 24 hour cache of work, that I'll get way too much for the three day deadlines.

When should I expect BOINC Manager to start predicting a new WU will take 36hrs?
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 95088 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1481
Credit: 14,602,674
RAC: 15,288
Message 95090 - Posted: 22 Apr 2020, 0:07:49 UTC - in response to Message 95088.  

I've completed and reported a number of tasks with Rosetta (not "mini") v4.15 the new 36hour runtime, but BOINC Manager still shows unstarted WUs with an estimated runtime of 7H27M.
You changed the Target CPU Runtime to 36 hours in your Rosetta project settings?
Grant
Darwin NT
ID: 95090 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 95091 - Posted: 22 Apr 2020, 0:19:11 UTC - in response to Message 95090.  

Yes. And I understand that a few have to crunch that way. But I've done that, and BOINC Manager still shows a new, unstarted WU with a 7.5H runtime expected.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 95091 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1481
Credit: 14,602,674
RAC: 15,288
Message 95093 - Posted: 22 Apr 2020, 0:47:54 UTC - in response to Message 95091.  
Last modified: 22 Apr 2020, 1:20:20 UTC

Yes. And I understand that a few have to crunch that way. But I've done that, and BOINC Manager still shows a new, unstarted WU with a 7.5H runtime expected.

Just had a fiddle myself.
I'm still waiting for new work to be requested, but It looks like Estimated completion times aren't re-calculated each time the Scheduler makes contact. It's looking like it only occurs when new work is requested.


I bumped up my Target CPU Runtime from 8 hours to 12hrs, updated on the Manager, had a couple of Tasks complete and manually reported them. The newly running tasks appear to be set for the new Target time (progress rate % per hour is less than for the other Tasks). Reduced the Target CPU Run time to 4 hours, updated. Still no change to Estimated completion times.
Just waiting now for when the system requests some new work to see if they change then (Estimated completion times are approx 30min less than usual actual processing time, so they should at least adjust to reflect that). It could be a few hours yet before new work is requested.


Edit- i just picked up new work, and the Estimated completion times for non running Tasks were adjusted, by a minute or so.
But since i increased & then decreased the CPU Targe Runtimes, the amount it varied by could very well be result of those changes.






I think the moral of the story is that you are better off with no cache at all. Yeah, if the project has frequent outages & periods of lack of work, then a cache that keeps you busy till the project is back makes sense.
But if you run more than 1 project, there is no need at all for a cache. If one project runs out of work, the other project will take up the slack. When the other project comes back up, your Resource Share settings will sort thing out.
And with no cache you will never get saddled with more work than you can possibly do, even if multiple projects you do work for change processing time & deadline requirements at the same time.
Grant
Darwin NT
ID: 95093 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 95096 - Posted: 22 Apr 2020, 1:07:35 UTC

My system has requested (and received) new work about every 12 hours for the last many days with the 36hr runtime setting.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 95096 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1481
Credit: 14,602,674
RAC: 15,288
Message 95099 - Posted: 22 Apr 2020, 1:26:18 UTC - in response to Message 95096.  

My system has requested (and received) new work about every 12 hours for the last many days with the 36hr runtime setting.
Were those Tasks that were running with the new Target CPU Runtime, or existing ones that were still running to the old Target CPU Runtime?
I'm thinking that they were older ones, so the Estimated time was still reflecting their completion times. But as they all complete, and you're left with nothing but Tasks with the new Target CPU Runtime, then the Estimated Completion times should start heading upward, relatively quickly (the Estimated completion time is actually supposed to dampen rapid changes in value (so that outliers don't affect it), but as the batch of Tasks that errored out in a matter of seconds showed, that isn't necessarily the case).
Grant
Darwin NT
ID: 95099 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 95102 - Posted: 22 Apr 2020, 1:40:47 UTC - in response to Message 95099.  

The only one I have, that has not already started, was requested this morning. Several days after upping to 36hrs.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 95102 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1481
Credit: 14,602,674
RAC: 15,288
Message 95110 - Posted: 22 Apr 2020, 2:45:09 UTC
Last modified: 22 Apr 2020, 2:47:29 UTC

I've recently been thinking of what they could do to improve Estimated completion times, and it all depends on how it is Initially determined, and then modified as work is returned. The occasional extremely long running Task doesn't appear to impact it (otherwise 10hrs would be added on the the Estimated completion time every time a Task went over it's Target CPU time & was ended by the Watchdog timer). But ridiculously short times do, even for Tasks that are errors (any Task that is Invalid or an Error shouldn't be included in any Estimated completion time calculations).

The BOINC system is set up for Tasks that have variable processing times due to different Tasks & different hardware being used.
But Rosetta uses a nominal fixed processing time.

So the method that would make the most sense would be for each time the Manager Requests work (as that's when it updates Estimated Completion times) would be for the Server to send the Target CPU Runtime to yet to start processing Tasks on that system to be the value for the Estimated completion time.
So even when work is initially sent, the Estimated completion time would be correct. When new Tasks or applications come out, the Estimated completion time would be correct. If there are a bunch of longer running than usual of shorter running than usual Tasks, people won't run into deadline issues because the Estimated completion times would be correct. If people change their Target CPU Runtimes, the Estimated completion time for any unstarted Tasks would be correct.

It would reduce significantly the amount of Task wastage that presently occurs due to caches, deadlines & way off target Estimated completion times (although once there are no more Rosetta Mini Tasks in the system the present Estimated completion times should improve considerably, however they are still susceptible to extremely short completion times for some reason).
Grant
Darwin NT
ID: 95110 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 95144 - Posted: 22 Apr 2020, 17:48:05 UTC

Finally now new work units for v4.15 are showing 33.5hr estimates, much better! That took it about a week to figure out!
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 95144 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : How does BOINC adjust initial runtime estimate?



©2024 University of Washington
https://www.bakerlab.org