New Crediting system: questions

Message boards : Number crunching : New Crediting system: questions

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 7 · Next

AuthorMessage
BennyRop

Send message
Joined: 17 Dec 05
Posts: 555
Credit: 140,800
RAC: 0
Message 22607 - Posted: 17 Aug 2006, 7:28:27 UTC

While looking for some P4s to see how they did - I noticed Avi's 2.9Ghz Celeron. Claimed credit= 20.7 points. Granted credit= 47.9. (Others were closer to 1.5 times the claimed credit.)

If this holds true for the rest of the P4 line, then those who were discouraged by the disparity of the P4 line versus the AMD cpus may have had their prayers answered now.

But the range of scores on the Celeron bring up the question that's been asked before: Is the credit/model score being based on the same consistent set of systems, (thus various algorithms used in the WUs operate faster than others on the Celeron) - or is it based on an average of whatever machines were available at the time on Ralph? (With higher scores when larger numbers of optimized clients were represented.. and lower scores when larger numbers of P4s on Linux running the standard client were represented?)


ID: 22607 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 22625 - Posted: 17 Aug 2006, 8:56:35 UTC

I just noticed a seriously wrong credit/work value being used. For some reason, my script gave extremely low values for a handful of work units so as a quick fix, I reset the credit/work values to adjust as results come in. Sorry about this mishap. I'll have to do some damage control this morning but now I must sleep.
ID: 22625 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Hoelder1in
Avatar

Send message
Joined: 30 Sep 05
Posts: 169
Credit: 3,915,947
RAC: 0
Message 22662 - Posted: 17 Aug 2006, 12:14:39 UTC - in response to Message 22602.  

We may actually have the credit/model values adjust for all work units as results come in and use ralph tests to serve as a starting point.
I think this is an excellent idea - perhaps the sample sizes on Ralph were really a little bit too small to determine reliable averages...

Team betterhumans.com - discuss and celebrate the future - hoelder1in.org
ID: 22662 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
MikeMarsUK

Send message
Joined: 15 Jan 06
Posts: 121
Credit: 2,637,872
RAC: 0
Message 22669 - Posted: 17 Aug 2006, 13:02:28 UTC - in response to Message 22662.  
Last modified: 17 Aug 2006, 13:09:50 UTC

We may actually have the credit/model values adjust for all work units as results come in and use ralph tests to serve as a starting point.
I think this is an excellent idea - perhaps the sample sizes on Ralph were really a little bit too small to determine reliable averages...


The low score I noticed had a very high number of decoys generated, it's possible that because each decoy was so quick, there were rounding issues in the inital data collection causing the credit to be too low?

But overall, once the glitches are worked out, it's looking good. I think even the people unhappy with the new system at the moment will come to like it given time, since it gives a very clear way to compare the performance of an Ubermachine :-) (Whether highly overclocked or massively parallel).

ID: 22669 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Athlonheizer
Avatar

Send message
Joined: 19 Jun 06
Posts: 2
Credit: 213,458
RAC: 0
Message 22697 - Posted: 17 Aug 2006, 15:40:12 UTC

Why do a lot of Work units with long computation time have fewer Credits than others with short computation time?
Longer computation times mean nevertheless more Credits. Or?
Otherwise the Creditsystem would be garbage.

Athlon
ID: 22697 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Francis

Send message
Joined: 6 Jun 06
Posts: 2
Credit: 25,569
RAC: 0
Message 22722 - Posted: 17 Aug 2006, 16:59:46 UTC - in response to Message 22625.  

I just noticed a seriously wrong credit/work value being used. For some reason, my script gave extremely low values for a handful of work units so as a quick fix, I reset the credit/work values to adjust as results come in. Sorry about this mishap. I'll have to do some damage control this morning but now I must sleep.


Would that explain this:

""Computer ID 275524
Report deadline 22 Aug 2006 19:21:18 UTC
CPU time 85019
stderr out <core_client_version>5.4.9</core_client_version>
<stderr_txt>
# random seed: 2986845
# cpu_run_time_pref: 86400
# DONE :: 1 starting structures built 38 (nstruct) times
# This process generated 38 decoys from 38 attempts


BOINC :: Watchdog shutting down...
BOINC :: BOINC support services shutting down...

</stderr_txt>
Validate state Valid
Granted credit 160.321852290615
Granted work credit 0.9088843809044
application version 5.25 ""

Just wondering ;-)
Francis
ID: 22722 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
MikeMarsUK

Send message
Joined: 15 Jan 06
Posts: 121
Credit: 2,637,872
RAC: 0
Message 22728 - Posted: 17 Aug 2006, 17:11:43 UTC

That doesn't look right at all! I'm sure you'll find that the work credit suddenly jumps upwards once he's had the chance to rerun his scripts.

ID: 22728 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 22730 - Posted: 17 Aug 2006, 17:20:27 UTC

David Kim explained this morning that he made a mistake in his scripts so the Granted work credit is wrong for a few of WUs. This could explain that.
ID: 22730 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 22733 - Posted: 17 Aug 2006, 17:37:48 UTC

Yes, these extreme differences are due to the script error. I will fix them soon and adjust the work credits. Sorry about that. The smaller differences when considering the credit/cpu time are due to the random nature of rosetta. The same work unit with different random number seeds will have different search paths so they are likely to have different credit/cpu time values when comparing small numbers of runs. The values should average out and in my opinion the differences are acceptable.
ID: 22733 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B^S] thierry@home
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 281,902
RAC: 0
Message 22734 - Posted: 17 Aug 2006, 17:38:40 UTC

You had a good (short) night? ;-)
ID: 22734 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Avi

Send message
Joined: 2 Aug 06
Posts: 58
Credit: 95,619
RAC: 0
Message 22736 - Posted: 17 Aug 2006, 17:41:23 UTC - in response to Message 22733.  

Yes, these extreme differences are due to the script error. I will fix them soon and adjust the work credits. Sorry about that. The smaller differences when considering the credit/cpu time are due to the random nature of rosetta. The same work unit with different random number seeds will have different search paths so they are likely to have different credit/cpu time values when comparing small numbers of runs. The values should average out and in my opinion the differences are acceptable.


So setting the work unit length to 24hrs would give more or less credits than the standard 3hrs?
ID: 22736 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Saenger
Avatar

Send message
Joined: 19 Sep 05
Posts: 271
Credit: 824,883
RAC: 0
Message 22744 - Posted: 17 Aug 2006, 17:58:18 UTC - in response to Message 22736.  

So setting the work unit length to 24hrs would give more or less credits than the standard 3hrs?

It should give 8 times more credit per WU, and the same credit per hour in a perfect world.
Grüße vom Sänger
ID: 22744 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 22746 - Posted: 17 Aug 2006, 18:01:20 UTC - in response to Message 22736.  


So setting the work unit length to 24hrs would give more or less credits than the standard 3hrs?


If you are talking about the particular work unit, more credits because you will generate more models, but if you are talking about overall what the best work unit length is for maximum credits, there are a number of factors that can come into play and I don't know the answer.
ID: 22746 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Steve Cressman
Avatar

Send message
Joined: 25 Jul 06
Posts: 23
Credit: 9,432
RAC: 0
Message 22876 - Posted: 18 Aug 2006, 2:08:14 UTC

Granted credit 27.5148233945073
Granted work credit 35.3874794761889

Just wondering which of these two scores I'm receiving.
98SE XP2500+ @ 2.1GHz Boinc v5.8.8

And God said"Let there be light."But then the program crashed because he was trying to access the 'light' property of a NULL universe pointer.
ID: 22876 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 22879 - Posted: 18 Aug 2006, 2:13:19 UTC

both, but the granted credit is what gets reported to the stats sites.
ID: 22879 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Avi

Send message
Joined: 2 Aug 06
Posts: 58
Credit: 95,619
RAC: 0
Message 22882 - Posted: 18 Aug 2006, 2:14:09 UTC - in response to Message 22876.  

Granted credit 27.5148233945073
Granted work credit 35.3874794761889

Just wondering which of these two scores I'm receiving.


This is the way i hear its "supposed" to be currently.

There are currently two ranking systems. One is the traditional, Granted Credit, which is exported to the ranking sites. The latter is the new one being tested. From what I hear, those using stock apps and are linux or P4 might be getting higher. Those using optimized apps should be getting lower. What the future plans are about KEEPING both, Im unsure, but it provides a way to see that the new credit system is "working right." The new RAC on this site should soon be sorted by the new, "Granted work credit" if its not done already.

ID: 22882 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Steve Cressman
Avatar

Send message
Joined: 25 Jul 06
Posts: 23
Credit: 9,432
RAC: 0
Message 22891 - Posted: 18 Aug 2006, 2:51:25 UTC

Thanx for the info , was just curious.
I think I like the new credit system better, hope it gets adopted soon.
:)
98SE XP2500+ @ 2.1GHz Boinc v5.8.8

And God said"Let there be light."But then the program crashed because he was trying to access the 'light' property of a NULL universe pointer.
ID: 22891 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikus

Send message
Joined: 7 Nov 05
Posts: 58
Credit: 700,115
RAC: 0
Message 22899 - Posted: 18 Aug 2006, 3:58:11 UTC
Last modified: 18 Aug 2006, 4:10:43 UTC

I don't know what to think of this new crediting system. I'm running offline with a vanilla Linux client, which in the past kept receiving lower benchmark scores than did Windows clients. So far, I've reported to the new system (all at the same time, when I dialed in this Thursday) the completion of four WUs. The respective numbers, granted credit (old) on left, work credit (new) on right:
66.4 : 055.8
66.6 : 056.0
68.8 : 149.4
68.6 : 159.2

Is it just a coincidence that the two high (new) numbers are for WUs I reported on the day their deadline would have expired, whereas the two low (new) numbers are for WUs whose completion I reported ahead of their deadline date ?

And I am disappointed that there are two WUs on which the new crediting system will give me 16% lower numbers than the old system (which was already giving me lower numbers than for many other participants).
.
ID: 22899 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Steve Cressman
Avatar

Send message
Joined: 25 Jul 06
Posts: 23
Credit: 9,432
RAC: 0
Message 22908 - Posted: 18 Aug 2006, 5:57:27 UTC - in response to Message 22899.  
Last modified: 18 Aug 2006, 6:01:15 UTC

I don't know what to think of this new crediting system. I'm running offline with a vanilla Linux client, which in the past kept receiving lower benchmark scores than did Windows clients. So far, I've reported to the new system (all at the same time, when I dialed in this Thursday) the completion of four WUs. The respective numbers, granted credit (old) on left, work credit (new) on right:
66.4 : 055.8
66.6 : 056.0
68.8 : 149.4
68.6 : 159.2

Is it just a coincidence that the two high (new) numbers are for WUs I reported on the day their deadline would have expired, whereas the two low (new) numbers are for WUs whose completion I reported ahead of their deadline date ?

And I am disappointed that there are two WUs on which the new crediting system will give me 16% lower numbers than the old system (which was already giving me lower numbers than for many other participants).
.

From what I have read in other threads, there was a problem with the new credit system last night but it was caught quickly and corrected. Also read that they would be correcting the ones that are wrong. You just happened to submit a couple results during that time. The new system when fully implemented will be very good for you.
:)

P.S. I found this very interesting link to an informative page.
98SE XP2500+ @ 2.1GHz Boinc v5.8.8

And God said"Let there be light."But then the program crashed because he was trying to access the 'light' property of a NULL universe pointer.
ID: 22908 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Avi

Send message
Joined: 2 Aug 06
Posts: 58
Credit: 95,619
RAC: 0
Message 22914 - Posted: 18 Aug 2006, 6:27:48 UTC - in response to Message 22908.  

P.S. I found this very interesting link to an informative page.


Whose site is that? It looks and sounds official, yet its not the correct url.

ID: 22914 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 . . . 7 · Next

Message boards : Number crunching : New Crediting system: questions



©2024 University of Washington
https://www.bakerlab.org