How's the new credit system shaping up?

Message boards : Number crunching : How's the new credit system shaping up?

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 23996 - Posted: 21 Aug 2006, 2:47:33 UTC
Last modified: 21 Aug 2006, 2:51:00 UTC

I've completed updating my files on the new rosetta wus. I've included them into the "cross project comparison spreadsheet that is being compiled for Eric K (seti beta). Al (aka Pappa) is also doing this. I've added extra rows to mine for this project to show "old ralph, and all 3 stages of the development of the new Rosetta credit system. Stage 1, 2 credits/model. Stage 2, Calculated credit, and stage 3, calculated with correction factor (NOTE: they haven't disclosed the actual progression, so my names for them are made up by me. I don't know what they're called at rosetta/ralph). The following spreadsheet shows what I've actually done for the various projects and what I've claimed vs what I've been granted. The "benchmark calculated claimed credit" is calculated using (dhrystone+whetstone)x 3600 seconds/1728000, and can be seen at the bottom. In an Ideal credit world the claimed and granted credits would be as close as possible to the benchmark calculated credit/hour.

CC is claimed credit
GC is Granted credit
GWC is Granted Work credit (rosetta new system)

All ralph/rosetta numbers are "Brown" for easy location. Calculated benchmark credit/hour is green. NOTE: this data covers my 5 puters.

It appears to me and my findings that Rosetta is about "spot on" with the new system as it compares to BOTH the "benchmark system" as used by standard clients and also with cross project parity.

Good Job Rosetta.

[edit} darn it, the third column over is supposed to read "# of result entries"




ID: 23996 · Rating: -2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 24001 - Posted: 21 Aug 2006, 3:26:50 UTC

And here's the rosetta wus that I've done. I also see a variation in GWC and models, but it comes out pretty decent when averaged.


ID: 24001 · Rating: -3 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ananas

Send message
Joined: 1 Jan 06
Posts: 232
Credit: 752,471
RAC: 0
Message 24015 - Posted: 21 Aug 2006, 5:54:30 UTC

Einstein have been adjusting their credits to the standard, they reduced it in two steps lately.

They grant about 68% of the previous values now, bringing them very close to the average values of other projects.
ID: 24015 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tralala

Send message
Joined: 8 Apr 06
Posts: 376
Credit: 581,806
RAC: 0
Message 24030 - Posted: 21 Aug 2006, 7:58:10 UTC

Hi mmciastro,


Thanks - these are very interesting data. So the average seems to be okay for the moment but there are occasional anomalies which should be reduced. I hope you will be update your tables further I am interested in such a comparison very much.
ID: 24030 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Conrad Poohs
Avatar

Send message
Joined: 22 Feb 06
Posts: 3
Credit: 28,405
RAC: 0
Message 24088 - Posted: 21 Aug 2006, 14:13:32 UTC

My main puzzle is that the WUs seem to have shrunk considerably. All of a sudden they do in four or five hours (maybe 8).



ID: 24088 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 24089 - Posted: 21 Aug 2006, 14:17:43 UTC - in response to Message 24088.  

My main puzzle is that the WUs seem to have shrunk considerably. All of a sudden they do in four or five hours (maybe 8).



What do you mean by shrunk? The WU calculation lenght is based on your preferences. So if you specify 20 hours, each WU will run 20 hours.
ID: 24089 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile carl.h
Avatar

Send message
Joined: 28 Dec 05
Posts: 555
Credit: 183,449
RAC: 0
Message 24091 - Posted: 21 Aug 2006, 14:20:36 UTC

Thierry I think he is referring to points/credits !
Not all Czech`s bounce but I`d like to try with Barbar ;-)

Make no mistake This IS the TEDDIES TEAM.
ID: 24091 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1831
Credit: 119,384,475
RAC: 9,496
Message 24098 - Posted: 21 Aug 2006, 14:36:44 UTC - in response to Message 24088.  
Last modified: 21 Aug 2006, 14:37:02 UTC

My main puzzle is that the WUs seem to have shrunk considerably. All of a sudden they do in four or five hours (maybe 8).


Andy

I've had a look at your results - and there's a big drop in run-time over the last week or two. Have you changed the Target CPU run time in the preferences, or changed the profile your computer is running under (home, work, school etc)?
ID: 24098 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
R.L. Casey

Send message
Joined: 7 Jun 06
Posts: 91
Credit: 2,728,885
RAC: 0
Message 24137 - Posted: 21 Aug 2006, 18:13:08 UTC - in response to Message 24001.  

And here's the rosetta wus that I've done. I also see a variation in GWC and models, but it comes out pretty decent when averaged.

.
.
.
Thanks much for your efforts in aggregating WU data! I very much appreciate your assistance. Please keep up the great work!
ID: 24137 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 24139 - Posted: 21 Aug 2006, 18:17:50 UTC

I found it much easier to compare when using a "per hour" indicator. I wonder if the project could add that to the results page. It would be easier for people to understand and compare. As it is now users have to figure out whether or not thing have gone bonkers with a calculator. It would be nice to see displayed.

thanks for your support as well. Every once in a while it's good to hear nice things, especially lately.
ID: 24139 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
doc :)

Send message
Joined: 4 Oct 05
Posts: 47
Credit: 1,106,102
RAC: 0
Message 24187 - Posted: 21 Aug 2006, 21:51:48 UTC

yes, thanks for your efforts :)
my very limited personal results look similar, most are damn close, with a rare outlier in either direction, seems to average out in the long run for me too though.
ID: 24187 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
AMD_is_logical

Send message
Joined: 20 Dec 05
Posts: 299
Credit: 31,460,681
RAC: 0
Message 24198 - Posted: 21 Aug 2006, 22:57:11 UTC

My results were averaging out nicely, at least until I got some t000__BOINC_SRAMAN_periLABRELAX_SAVE_ALL_OUT__1178 WUs, which seem to be a bit too generous.
https://boinc.bakerlab.org/rosetta/result.php?resultid=33627639
ID: 24198 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 24200 - Posted: 21 Aug 2006, 23:16:59 UTC - in response to Message 24198.  

My results were averaging out nicely, at least until I got some t000__BOINC_SRAMAN_periLABRELAX_SAVE_ALL_OUT__1178 WUs, which seem to be a bit too generous.
https://boinc.bakerlab.org/rosetta/result.php?resultid=33627639

criminy, Me too, I have one from my 2800 that came in last night 33629145 that was supposed to be 20-28 credits but came in at 80+. t000__BOINC_SRAMAN_periLABRELAX_SAVE_ALL_OUT__1178_6987_0

I have ONE on my mobile 3700, 33625267 that also came in overnite. Was supposed to be 40ish but came in at 112ish. t000__BOINC_SRAMAN_periLABRELAX_SAVE_ALL_OUT__1178_3382_0

and ONE on my 3700 33631855 that should have been 43 but came in at 107. also overnite. t000__BOINC_SRAMAN_periLABRELAX_SAVE_ALL_OUT__1178_9512_0

ONE on my P4 1.8 33626258 that came in at 51 and was supposed to be 16. t000__BOINC_SRAMAN_periLABRELAX_SAVE_ALL_OUT__1178_4307_0

Anyone seeing a pattern? I think the t000_BOINC_SRAMANs' need some adjustment from the staff.
ID: 24200 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 24202 - Posted: 21 Aug 2006, 23:22:48 UTC

Ethan, how about a sticky thread for reporting "suspected" incorrect "granted Work credit" values. I think we found one.
ID: 24202 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tralala

Send message
Joined: 8 Apr 06
Posts: 376
Credit: 581,806
RAC: 0
Message 24244 - Posted: 22 Aug 2006, 8:00:09 UTC - in response to Message 24202.  
Last modified: 22 Aug 2006, 8:01:15 UTC

Ethan, how about a sticky thread for reporting "suspected" incorrect "granted Work credit" values. I think we found one.

I support that idea! :-)
edit: Perhaps we wait until the new credit system goes live - David Kim is still working on it.
ID: 24244 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tralala

Send message
Joined: 8 Apr 06
Posts: 376
Credit: 581,806
RAC: 0
Message 24245 - Posted: 22 Aug 2006, 8:00:49 UTC - in response to Message 24244.  
Last modified: 22 Aug 2006, 8:01:35 UTC

edit: double post, sorry!
ID: 24245 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 24257 - Posted: 22 Aug 2006, 9:55:04 UTC

tralala, that was kinda of the reason to have it. It's in testing. much like in ralph, they may need to know where the bugs are so they can squash them. Hopefully without reading every thread to find them. :)
ID: 24257 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 24300 - Posted: 22 Aug 2006, 18:28:29 UTC

You all are on top of it. The SRAMAN batch was submitted using a credits/model value determined from a ralph test run. So far the test run of 100 jobs has produced 240 structures with a credits/model value of 5.3. But surprisingly, the value on R@h is currently 2.2 with over 100,000 models returned (quite a big sample). Because of this discrepancy, we decided to use R@h to determine the credit/model value as results come in. There will obviously be some variation in the value for the first few results but the value will stabilize rather quickly.

Today, I will be working on the final system and after it's activated, I will post some information about it.

The SRAMAN batch will start using the R@h credits/model value with the new system.
ID: 24300 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tralala

Send message
Joined: 8 Apr 06
Posts: 376
Credit: 581,806
RAC: 0
Message 24303 - Posted: 22 Aug 2006, 18:58:56 UTC - in response to Message 24300.  

You all are on top of it. The SRAMAN batch was submitted using a credits/model value determined from a ralph test run. So far the test run of 100 jobs has produced 240 structures with a credits/model value of 5.3. But surprisingly, the value on R@h is currently 2.2 with over 100,000 models returned (quite a big sample). Because of this discrepancy, we decided to use R@h to determine the credit/model value as results come in. There will obviously be some variation in the value for the first few results but the value will stabilize rather quickly.

Today, I will be working on the final system and after it's activated, I will post some information about it.

The SRAMAN batch will start using the R@h credits/model value with the new system.

This might happen in the future as well. On Ralph there is currently a daily quota of 20 WU/day, which means in extreme cases 100 jobs could be sent to only 5 hosts. If 3 or 4 of them are using the optimized client the credits/model value will be greatly distorted. There are ways to avoid this but since David Baker asked not to come forward with suggestions I keep my mouth. I'm sure you will find a solution, but in case you want to hear suggestions, feel free... ;-)
ID: 24303 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 24305 - Posted: 22 Aug 2006, 19:06:17 UTC

But we are not going to use Ralph so the variation will only occur as the first results come in. You can email me your suggestions at dekim at u washington edu.
ID: 24305 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · 4 · Next

Message boards : Number crunching : How's the new credit system shaping up?



©2024 University of Washington
https://www.bakerlab.org