New credit system now being tested at RALPH@home

Message boards : Number crunching : New credit system now being tested at RALPH@home

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 8 · Next

AuthorMessage
zombie67 [MM]
Avatar

Send message
Joined: 11 Feb 06
Posts: 316
Credit: 6,621,003
RAC: 0
Message 22001 - Posted: 8 Aug 2006, 2:20:58 UTC

FYI

http://ralph.bakerlab.org/


Reno, NV
Team: SETI.USA
ID: 22001 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 22012 - Posted: 8 Aug 2006, 4:14:45 UTC

For anyone that doesn't already know, Ralph is a test project for Rosetta. New programs, and ideas are tested there prior to release to the widest audience on Rosetta. So, if you are an experienced BOINC and/or Rosetta user, and are interested, you may attach to Ralph (with a low resource share) and learn more about what's coming. Otherwise, once any kinks are worked out, the changes will be brought to Rosetta in time.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 22012 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Whl.

Send message
Joined: 29 Dec 05
Posts: 203
Credit: 275,802
RAC: 0
Message 22016 - Posted: 8 Aug 2006, 5:43:10 UTC
Last modified: 8 Aug 2006, 5:58:11 UTC

Does look a bit of a daft way to do things. 2 credits per model ? Or is there more to it than that? I've had this machine produce over 90 models for some jobs (think the most I've seen is 99) and 4 or 5 for others at a 4 hour setting.
ID: 22016 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Hoelder1in
Avatar

Send message
Joined: 30 Sep 05
Posts: 169
Credit: 3,915,947
RAC: 0
Message 22018 - Posted: 8 Aug 2006, 6:15:05 UTC - in response to Message 22016.  
Last modified: 8 Aug 2006, 6:21:14 UTC

See this explanation by David Kim, quoted from the Ralph forum:
The version that will eventually run on Rosetta@home will have work unit specific credit per model values that are determined from test runs on Ralph. It will be a requirement for lab members to not only test new work units on Ralph but to also determine the average credit per model value from their test runs for production runs. The credits should remain somewhat consistent with other projects since the average values will be based on the standard boinc crediting scheme. If things look okay on Ralph, Rosetta@home will use the credit per model crediting method while Ralph will switch back to the standard method.
So the two credits per structure are just for preliminary tests - no need to be concerned...
Team betterhumans.com - discuss and celebrate the future - hoelder1in.org
ID: 22018 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Trog Dog
Avatar

Send message
Joined: 25 Nov 05
Posts: 129
Credit: 57,345
RAC: 0
Message 22024 - Posted: 8 Aug 2006, 13:08:22 UTC - in response to Message 22012.  

For anyone that doesn't already know, Ralph is a test project for Rosetta. New programs, and ideas are tested there prior to release to the widest audience on Rosetta. So, if you are an experienced BOINC and/or Rosetta user, and are interested, you may attach to Ralph (with a low resource share) and learn more about what's coming. Otherwise, once any kinks are worked out, the changes will be brought to Rosetta in time.


OK just put my money where my mouth is and signed up :)
ID: 22024 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 22027 - Posted: 8 Aug 2006, 13:53:03 UTC

Yeah, I think the 2 credits per model was just for credits on RALPH while they test it. Not the plan for Rosetta. Everyone agrees that some models are worth more then others, because they take 10-30 times longer to crunch then simple proteins. And so they will receive appropriate credit for that work once these benchmarks are established.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 22027 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
ShootStraight

Send message
Joined: 5 Mar 06
Posts: 14
Credit: 2,691,550
RAC: 0
Message 22031 - Posted: 8 Aug 2006, 16:37:19 UTC - in response to Message 22018.  
Last modified: 8 Aug 2006, 17:07:57 UTC

The version that will eventually run on Rosetta@home will have work unit specific credit per model values that are determined from test runs on Ralph. It will be a requirement for lab members to not only test new work units on Ralph but to also determine the average credit per model value from their test runs for production runs. The credits should remain somewhat consistent with other projects since the average values will be based on the standard boinc crediting scheme. If things look okay on Ralph, Rosetta@home will use the credit per model crediting method while Ralph will switch back to the standard method.


Forgive me if I'm not understanding, but I take this to mean that we are going to basing our credit system to be comparable to BOINC's dysfunctional one? IMHO thats a horrendous strategic blunder. I'm gonna need someone to explain the wisdom behind this, cause all I see are problems.

BOINC cross-project campatability should be the very last item on the punchlist, if at all. Make something that works. Make something that intrinsically attracts members and powerful hardware. Do this and the rest of BOINC will follow (and be comparable). BOINC doesnt work. BOINC attracts optimizations and 6 y.o. with Cyrix chips (passed off to be quad opterons)and makes martyrs out of everyone else. Why on earth would we want to statiscally compare ourselves with such a system, if it was even possible?

Edit: I'd like to thank whomever gave me a negative rating on this post - I'll wear it with pride since it came from some anonymous cowardly prick that has absolutely nothing constructive to say. edit/

-SS
ID: 22031 · Rating: -2.0000000000002 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 22038 - Posted: 8 Aug 2006, 17:25:58 UTC

ShootStraight, I don't see a negative rating anymore... but edits like that are certainly a good way to earn them.

Could we wait for a definitive description of the new credit system before we dismiss it as a "horrendous strategic blunder"? A number of people have spent considerable time to devise a new, fair, honest, non-horrendous system. They need to run some tests on Ralph to kick the tires and try it out. Once they've reviewed results of the new system, they will be in a better position to explain what they found and how they plan for it work.

If you don't want to connect to Ralph and help (actually, I don't see any WUs there presently anyway), then at least try to give the folks that have been working on this the benefit of the doubt, and some time to study how their ideas are working in practice on Ralph.

I don't see how offering negative comments on a one sentence description of a complex credit system is going to help move this forward or add any additional direction to the work being done.
Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 22038 · Rating: 0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 22039 - Posted: 8 Aug 2006, 17:27:25 UTC

shootstraight,

what problems do you envision?
ID: 22039 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Tom Philippart
Avatar

Send message
Joined: 29 May 06
Posts: 183
Credit: 834,667
RAC: 0
Message 22041 - Posted: 8 Aug 2006, 17:36:03 UTC
Last modified: 8 Aug 2006, 17:36:11 UTC

no new work@ralph :(
ID: 22041 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tralala

Send message
Joined: 8 Apr 06
Posts: 376
Credit: 581,806
RAC: 0
Message 22052 - Posted: 8 Aug 2006, 19:30:03 UTC

shootstraight

The goal is to follow those projects which have abandoned the BOINC credit system and invented a cheating-free one such as climateprediction (and BBC), Seti and Einstein. These projects grant fixed credit per WU of the same length and avoid the broken benchmark approach - that is exactly what Rosetta plans to do. What David Kim meant with BOINC compatibility is that the credit granted for a specific amount of computing should be similar on all BOINC projects. This means one must grant about the same credit per computer second as other projects would do (on the same computer).
ID: 22052 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Jack Shaftoe
Avatar

Send message
Joined: 30 Apr 06
Posts: 115
Credit: 1,307,916
RAC: 0
Message 22053 - Posted: 8 Aug 2006, 20:03:01 UTC - in response to Message 22052.  
Last modified: 8 Aug 2006, 20:34:14 UTC

These projects grant fixed credit per WU of the same length and avoid the broken benchmark approach - that is exactly what Rosetta plans to do.


Curious - anyone know if this new measuring stick will get applied to past credits as well? Will people using the stock clients see credits jump and people using optimized clients see credits fall?

Team Starfire World BOINC
ID: 22053 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Keith Akins

Send message
Joined: 22 Oct 05
Posts: 176
Credit: 71,779
RAC: 0
Message 22054 - Posted: 8 Aug 2006, 20:11:53 UTC

This might affect RAC's as happened on CPDN, but I don't think that total credits will be retroactively affected.

Note:

CPDN's RAC's were really inflated across the board before their new credit system went into affect.
ID: 22054 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Jack Shaftoe
Avatar

Send message
Joined: 30 Apr 06
Posts: 115
Credit: 1,307,916
RAC: 0
Message 22055 - Posted: 8 Aug 2006, 20:33:46 UTC - in response to Message 22054.  

That's really too bad. It's going to take a *long* time for the lopsidedness of the current system to go away then.
ID: 22055 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Keith Akins

Send message
Joined: 22 Oct 05
Posts: 176
Credit: 71,779
RAC: 0
Message 22056 - Posted: 8 Aug 2006, 20:51:02 UTC

I don't think that as far as rosetta is concerned that the new system will be bad.

Rosetta is already in fairly equal parity with other projects and any affect on RAC's should be minimal.
ID: 22056 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Jack Shaftoe
Avatar

Send message
Joined: 30 Apr 06
Posts: 115
Credit: 1,307,916
RAC: 0
Message 22057 - Posted: 8 Aug 2006, 21:02:17 UTC - in response to Message 22056.  
Last modified: 8 Aug 2006, 21:07:15 UTC

I don't think that as far as rosetta is concerned that the new system will be bad.


I'm not saying the new system will be bad, I'm just saying it would be better if this got fixed completely (RAC credit and Total credit) instead of just partially (just RAC). I hope that will be considered.
ID: 22057 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
MikeMarsUK

Send message
Joined: 15 Jan 06
Posts: 121
Credit: 2,637,872
RAC: 0
Message 22058 - Posted: 8 Aug 2006, 21:05:43 UTC

It would be nice if the old credit got fixed, but getting the new credit fixed first is the more important thing! After a month or so the RACs will be nearly 100% reflecting the new system.

ID: 22058 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 22061 - Posted: 8 Aug 2006, 21:59:15 UTC - in response to Message 22058.  

It would be nice if the old credit got fixed, but getting the new credit fixed first is the more important thing! After a month or so the RACs will be nearly 100% reflecting the new system.

Indeed it will, however the credit race by "total credit" will be flawed forever. lol Anyone one ever wonder what the leader board would be like without optimized apps?
ID: 22061 · Rating: -2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Jack Shaftoe
Avatar

Send message
Joined: 30 Apr 06
Posts: 115
Credit: 1,307,916
RAC: 0
Message 22065 - Posted: 8 Aug 2006, 22:32:05 UTC - in response to Message 22061.  

After a month or so the RACs will be nearly 100% reflecting the new system.


And after 7 or 8 years the total credit will be nearly 80% reflecting. Ugh. Hope they have a plan to fix that too.
ID: 22065 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
ShootStraight

Send message
Joined: 5 Mar 06
Posts: 14
Credit: 2,691,550
RAC: 0
Message 22068 - Posted: 8 Aug 2006, 23:18:19 UTC

Feet1st,

I havent dismissed the credit system as such, however I feel the premise of cross-project compatibility as a primary requisite (with projects still using the dysfunctional system) is a fundamentally flawed approach . It with profound and deep respect to the amount of work being done to correct this problem and the memory of those that are no longer with us - due to the problem itself or the acrimony that ensued (and persists)- that I feel compelled to speak, lest time and effort be wasted on another approach that doesnt work. We have an opportunity to make significant changes for the better, and its worthwhile doing right.

What is the right way, I dont know. I like the idea of a fixed credit/unit, but simple logic would dictate; parity with something that has no parity, and holds statistical reliability less than a poll in Glamour magazine is something that should be questioned and scrutinized for reasons which should be obvious, and have been discussed ad-nauseum previously.

Lastly my apologies to anyone whos feathers I might have inadvertantly ruffled with my edit. I guess someone else came along with their own input, and I couldnt edit it out.

Anyhow thats it for me...

-SS


ID: 22068 · Rating: -9.9920072216264E-15 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · 4 . . . 8 · Next

Message boards : Number crunching : New credit system now being tested at RALPH@home



©2024 University of Washington
https://www.bakerlab.org