Shorter WU deadlines

Message boards : Number crunching : Shorter WU deadlines

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Team TMR

Send message
Joined: 2 Nov 05
Posts: 21
Credit: 1,583,679
RAC: 0
Message 9691 - Posted: 24 Jan 2006, 8:38:34 UTC

I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines.

Is this a permanent change?

It's not a big deal for us in the most part, except on a couple of PCs which are only switched on a few hours a day.
ID: 9691 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Hoelder1in
Avatar

Send message
Joined: 30 Sep 05
Posts: 169
Credit: 3,915,947
RAC: 0
Message 9698 - Posted: 24 Jan 2006, 11:09:31 UTC - in response to Message 9691.  

I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines.

Is this a permanent change?

It's not a big deal for us in the most part, except on a couple of PCs which are only switched on a few hours a day.

Great ! I have always wondered why such a fast-paced project permits work to potentially sit in user cashs for a week or more to perhaps be returned after the project has performed the final analysis on a particular numerical experiment and moved on to other things (I keep my cash size at 0.1 days to give them quick
feedback for that reason). I suspect that some of these new WUs will now sit in large user cashs, happily awaiting their expiration. ;-) I guess it would have been a good idea to warn those users who keep large cash sizes, ahead of sending out the new WUs (like in the news section on the home page ?) to give them a chance to reduce their cash sizes... ;-))
ID: 9698 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
STE\/E

Send message
Joined: 17 Sep 05
Posts: 125
Credit: 4,100,301
RAC: 206
Message 9700 - Posted: 24 Jan 2006, 11:51:39 UTC

All the WU's I have Downloaded in the last 24 hours still have the 1 Month Deadline.
ID: 9700 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 9701 - Posted: 24 Jan 2006, 11:59:18 UTC - in response to Message 9698.  

I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines.
...

Great ! I have always wondered why such a fast-paced project permits work to potentially sit in user cashs for a week or more...


er, not great.

The big advantage of a long deadline with a short wu is that it makes an ideal backup project, either to tide over periods with no work on another project (notice how my Rosetta stats slump when LHC has work) or when a machine's availablility is unpredictable for other reasons.

For example, there is a machine that is dual boot, it usually runs windows but is occasionally booted into Linux for maybe a half day or full day once a week. Ive been running Rosetta on it when it is in Linux because the long deadlines mean that the work does not time out while the box is running windows. I will now have to use Einstein for that role.

Also, if a project deadline is less than double the cache size, the box will only run in EDF mode whenenver that project is present. This makes it impossible to run a single CPDN WU on the same 2cpu box as Rosetta - EDF makes ROsetta take over the cpu permanently. With 7 day deadlines this happens at a cache size of 3.5 days - needed sometime locally due to intemittent power issues that affect the network between me and the Internet.

My preference would be for the project to set varying dealines according to how urgently they need the work back. LHC does this and it works very well for crunchers, and I believe for the project. When the scientists want the turnround fast they set a short deadline -- those WU then jump the queue in long caches.

If the EXAMPLE_WU series are needed back by the end of FEb, they'd be sent with a 28 day deadline now, but shortened by 7 days eech week till they were on down to a 7 day deadline, then perhaps shortened to a 2 day deadline to pick up the stragglers really fast.

The LHC scheduler seems to have no problem keeping a mix of long and short deadline work in my longer caches - I think they use the standard scheduler code for this.

River~~
ID: 9701 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Hoelder1in
Avatar

Send message
Joined: 30 Sep 05
Posts: 169
Credit: 3,915,947
RAC: 0
Message 9705 - Posted: 24 Jan 2006, 13:27:19 UTC - in response to Message 9701.  

I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines.
...

Great ! I have always wondered why such a fast-paced project permits work to potentially sit in user cashs for a week or more...


er, not great.

... anyway, thanks for the clarifications. ;-) I had naively (and it seems erroneously) assumed that the cache works on a first-in-first-out basis. As I said, personally short expiration dates are fine with me. So lets see what we hear out of Seattle on this - oh and sorry for misspelling 'cache' in my last post. ;-)
ID: 9705 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile River~~
Avatar

Send message
Joined: 15 Dec 05
Posts: 761
Credit: 285,578
RAC: 0
Message 9729 - Posted: 24 Jan 2006, 20:36:44 UTC - in response to Message 9705.  

I had naively (and it seems erroneously) assumed that the cache works on a first-in-first-out basis. ...


In normal running (when projects are run in Round Robin mode) the cache for each project does indeed run first-in-first-out.

However, as soon as that is predicted to make any WU late for a deadline, the machine switches to EDF - earliest deadline first - mode, and then the WU from all projects are put into deadline order. In this mode the short deadline WU from LHC get crunched before long deadline WU that came in earlier.

In particular, if you have a cache of (say) 3 days work and you get a WU with a 2 day deadline, it instantly puts the box into EDF precisely because the new work would otherwise go to the back of the queue.

On Pirates@home they have sent out WU with 6-hour deadlines - these usually run instantly on download as with almost any mix of work they would otherwise be 'late'.

In addition, the server will not send out more short deadline work that your box can crunch in 90% of the alloted time - so a batch of short deadline work gets spread around many boxes even when each box is asking for a lot of work.

This combination of behaviour built into the client and server means that the scientists can in fact combine 'urgent' work (short deadline) with 'background' work (long deadline) and BOINC gets it right most of the time.

As you say, it is up to UW how to use this, but my hope would be that they mix the deadlines and look into varying them even within the same series of WU (if this is possible)

R~~
ID: 9729 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
David Baker
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 17 Sep 05
Posts: 705
Credit: 559,847
RAC: 0
Message 9753 - Posted: 25 Jan 2006, 2:06:17 UTC - in response to Message 9698.  

I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines.

Is this a permanent change?

It's not a big deal for us in the most part, except on a couple of PCs which are only switched on a few hours a day.

Great ! I have always wondered why such a fast-paced project permits work to potentially sit in user cashs for a week or more to perhaps be returned after the project has performed the final analysis on a particular numerical experiment and moved on to other things (I keep my cash size at 0.1 days to give them quick
feedback for that reason). I suspect that some of these new WUs will now sit in large user cashs, happily awaiting their expiration. ;-) I guess it would have been a good idea to warn those users who keep large cash sizes, ahead of sending out the new WUs (like in the news section on the home page ?) to give them a chance to reduce their cash sizes... ;-))



Sorry for the lack of warning! we switched to one week for just the reason you point out--we want to analyze the results and be able to draw conclusions as soon as possible!

another change is that the maximum work unit length has been increased to eliminate (hopefully!) the time out problem.

ID: 9753 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Moderator9
Volunteer moderator

Send message
Joined: 22 Jan 06
Posts: 1014
Credit: 0
RAC: 0
Message 9760 - Posted: 25 Jan 2006, 3:40:01 UTC
Last modified: 25 Jan 2006, 3:44:40 UTC

The following two posts have been moved to Forum discussion thread

Reason - Off Topic

rbpeake - Posted in reply to David Baker

This is a great example for having an Announcements section ...

Angus - Replied to rbpeake

I respectfully disagree. ...

Both posts moved to this thread.

{MODERATOR NOTE: This move announcement will be deleted}
Moderator9
ROSETTA@home FAQ
Moderator Contact
ID: 9760 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikus

Send message
Joined: 7 Nov 05
Posts: 58
Credit: 700,115
RAC: 0
Message 9979 - Posted: 27 Jan 2006, 3:27:08 UTC - in response to Message 9698.  

I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines.

Is this a permanent change?

It's not a big deal for us in the most part, except on a couple of PCs which are only switched on a few hours a day.

Great ! I have always wondered why such a fast-paced project permits work to potentially sit in user cashs for a week or more to perhaps be returned after the project has performed the final analysis on a particular numerical experiment and moved on to other things (I keep my cash size at 0.1 days to give them quick
feedback for that reason). I suspect that some of these new WUs will now sit in large user cashs, happily awaiting their expiration. ;-) I guess it would have been a good idea to warn those users who keep large cash sizes, ahead of sending out the new WUs (like in the news section on the home page ?) to give them a chance to reduce their cash sizes... ;-))

I guess change is to be expected, even though it is not always welcome.

I run rosetta as the __only__ BOINC project on a machine that is offline (except when I temporarily change it, I've specified in BOINC Manager: "Network activity suspended"); I have to both dial-in (from another machine) and set up a proxy before my rosetta machine can talk to the server. So communication takes place when I set it up, NOT when the machine feels like it.

I was mystified to see (when I connected today) that the current WU the machine was working on got 'preempted'. I did not for a while realize that a just-downloaded "short deadline" WU (several pages further down the work list) had been started in its stead. And I was further confused by seeing more and more "short deadline" WUs being downloaded, even though the client kept telling me that the machine was OVERLOADED !!

For me, the four-week-deadline WUs were ideal - they could chug along without me paying much attention. I could connect at my convenience (based on when the queue of WUs for 'uploading' got long). With the change to one-week-deadline WUs, I feel I am being FORCED to pay more attention and to connect to rosetta at shorter intervals. In fact, had I anticipated that I would come to feel rosetta deadlines "breathing down my neck", I might well have chosen a different project.
ID: 9979 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 9980 - Posted: 27 Jan 2006, 3:32:01 UTC - in response to Message 9760.  
Last modified: 27 Jan 2006, 3:32:25 UTC

The following two posts have been moved to Forum discussion thread

Reason - Off Topic

rbpeake - Posted in reply to David Baker

This is a great example for having an Announcements section ...

Angus - Replied to rbpeake

I respectfully disagree. ...

Both posts moved to this thread.

{MODERATOR NOTE: This move announcement will be deleted}


I'm sorry, but my post is not in the thread you referred to:

I can't remember the exact words (since I can't find it), but if I think it would be meaningless taken out of context in which it was posted.

Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 9980 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 9984 - Posted: 27 Jan 2006, 4:23:18 UTC - in response to Message 9753.  
Last modified: 27 Jan 2006, 4:24:37 UTC

I've just noticed that all WUs we've downloaded in the last 24 hours have 7 day deadlines on them, but when I looked last year, WUs had 4 week (or 1 month) deadlines.

Is this a permanent change?

It's not a big deal for us in the most part, except on a couple of PCs which are only switched on a few hours a day.

Great ! I have always wondered why such a fast-paced project permits work to potentially sit in user cashs for a week or more to perhaps be returned after the project has performed the final analysis on a particular numerical experiment and moved on to other things (I keep my cash size at 0.1 days to give them quick
feedback for that reason). I suspect that some of these new WUs will now sit in large user cashs, happily awaiting their expiration. ;-) I guess it would have been a good idea to warn those users who keep large cash sizes, ahead of sending out the new WUs (like in the news section on the home page ?) to give them a chance to reduce their cash sizes... ;-))



Sorry for the lack of warning! we switched to one week for just the reason you point out--we want to analyze the results and be able to draw conclusions as soon as possible!

another change is that the maximum work unit length has been increased to eliminate (hopefully!) the time out problem.



Dr. Baker -

Could you PLEASE post information about substantial changes in WU size or duration in the NEWS or Technical News section? Many of us who have been around BOINC a long time look first to the News section when something unusual appears.
Something as radical as changing the deadline from a month to a week can play havoc with work queues, and require some adjustment by us crunchers.
Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 9984 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
STE\/E

Send message
Joined: 17 Sep 05
Posts: 125
Credit: 4,100,301
RAC: 206
Message 9989 - Posted: 27 Jan 2006, 4:52:36 UTC

Something as radical as changing the deadline from a month to a week can play havoc with work queues, and require some adjustment by us crunchers.
==========

I have to agree on that Angus, but then thats what the Abort Tab is for I figure, which I have no regrets on using when a Project pulls something like that ... I don't like my other Projects getting put on the back shelf because 1 Project decides to Override the other Projects by cutting their Deadline down so dramatically without warning ...
ID: 9989 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Team TMR

Send message
Joined: 2 Nov 05
Posts: 21
Credit: 1,583,679
RAC: 0
Message 10007 - Posted: 27 Jan 2006, 9:20:49 UTC

For us, we've had to rejig the projects that some PCs work on, taking some PCs off Rosetta completely.

We had Rosetta running on a few slow PCs, which take 1-2 days to complete a WU. When you factor in that those PCs are only on for less than 8 hours a day, it becomes 4-6 days to complete - and then a weekend arives (PCs are off) and it takes over a week to complete.

If the short deadline WUs were smaller so they completed quicker it wouldn't be such a problem.
ID: 10007 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jeff Gilchrist

Send message
Joined: 7 Oct 05
Posts: 33
Credit: 2,398,990
RAC: 0
Message 10061 - Posted: 27 Jan 2006, 21:07:43 UTC - in response to Message 9753.  

Sorry for the lack of warning! we switched to one week for just the reason you point out--we want to analyze the results and be able to draw conclusions as soon as possible!


This is causing me some problems. I have some machines that are not frequently connected to the Internet so I have my cache set to queue 10 days of work and I sometimes can only connect once a week to upload.

I am now not able to fill up my cache because I cannot complete the work in time for the deadline. Could you reconsider this and possibly select 2 weeks instead of 1 so that you still get your work back reasonably fast but you dont penalize some people?
ID: 10061 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B@H] Ray
Avatar

Send message
Joined: 20 Sep 05
Posts: 118
Credit: 100,251
RAC: 0
Message 10091 - Posted: 28 Jan 2006, 6:37:23 UTC

Dave
Any chance of making the deadlinc 2 weeks or even 10 days rather than 7 days? That short deadline sent one of my systems into EDF and it only has 3 Rosetta units.

Summery of other units on that system:

SETI: 6 units with deadlines of Feb. 9, 2006
SETI BATA: 1 unit with a deadline of Mar. 7, 2007 (2007 is correct)
CPDN: 1 Sulpher unit with a deadline of Dec. 18, 2006

Just the 3 Rosetta units with a deadline of Feb. 2 and Feb. 3 sent it into EDF, but of course that is good for Rosetta getting the time when in EDF until someone suspends it to get some other work done.

What surprised me was when the Rosetta units I had were almost finished it called for more and got two more while in EDF. Currently the cache is set for 3 days.

Ray Brown


Pizza@Home Rays Place Rays place Forums
ID: 10091 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B@H] Ray
Avatar

Send message
Joined: 20 Sep 05
Posts: 118
Credit: 100,251
RAC: 0
Message 10094 - Posted: 28 Jan 2006, 7:03:51 UTC

I faially had to suspend Rosetta to get the system back to doing CPDN till I get up in the morning.
Ray
ID: 10094 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Keck_Komputers
Avatar

Send message
Joined: 17 Sep 05
Posts: 211
Credit: 4,246,150
RAC: 0
Message 10103 - Posted: 28 Jan 2006, 11:17:01 UTC - in response to Message 10091.  

What surprised me was when the Rosetta units I had were almost finished it called for more and got two more while in EDF. Currently the cache is set for 3 days.

Ray Brown

NWF and EDF are not mutually exclusive. Although you rarely see EDF without NWF it is possible.

BOINC WIKI

BOINCing since 2002/12/8
ID: 10103 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Stanley A Bourdon

Send message
Joined: 17 Sep 05
Posts: 3
Credit: 112,907
RAC: 4
Message 10124 - Posted: 28 Jan 2006, 18:06:41 UTC - in response to Message 10094.  

I faially had to suspend Rosetta to get the system back to doing CPDN till I get up in the morning.
Ray


If you Waite a little while negative debt will build high enough that no more Rosetta WU will be downloaded until the negative debt is payed off by other projects running. For this to happen the negative debt for a project must be grater than the connect to time. for a 60 minute switch time debt must be more negative than -3600. May take a day or two to work out. I use 1500 minutes so I need to be more negative than 90,000. This can take over a week to re-balance.

this assumes that you are using a client after 4.71 ? or for sure a 5.x.x client.
Stanley


Boinc Wikipedia - the FAQ in active change
ID: 10124 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Joe

Send message
Joined: 26 Sep 05
Posts: 3
Credit: 587,393
RAC: 1,026
Message 10475 - Posted: 5 Feb 2006, 10:25:42 UTC

I loved the one month deadlines. I understand why shorter may be better but unfortunately they dont work for me. I suppose the percentage of folks who stop crunching the project due to the change is negligable.
ID: 10475 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Johnathon

Send message
Joined: 5 Nov 05
Posts: 120
Credit: 138,226
RAC: 0
Message 10484 - Posted: 5 Feb 2006, 20:47:45 UTC - in response to Message 10475.  


ID: 10484 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · Next

Message boards : Number crunching : Shorter WU deadlines



©2024 University of Washington
https://www.bakerlab.org