Cross Project Credit Equality

Message boards : Number crunching : Cross Project Credit Equality

To post messages, you must log in.

AuthorMessage
Profile Biggles
Avatar

Send message
Joined: 22 Sep 05
Posts: 49
Credit: 102,114
RAC: 0
Message 23326 - Posted: 19 Aug 2006, 2:57:45 UTC - in response to Message 23325.  
Last modified: 19 Aug 2006, 2:59:36 UTC

Jose, the sole point of mentioning Leiden Classical was to point out that it is a project that A) has a quorum and B) has never had optimised science apps. With those conditions met, I believe XS wouldn't have as high an RAC on that project as they do on Rosetta even if all members were able to join and get work. It was one project mentioned amongst several.

I'll put my main point again in another way - on BOINC, one hour of CPU time on one project should give around the same credit as that same machine spending an hour on another project. 1 SETI hour ~= 1 Einstein hour ~= 1 Climate Prediction hour ~= 1 Predictor hour ~= 1 Leiden Classical hour ~= 1 Rectilinear Crossing Numbers hour and so on.

Imagine there is a machine that gets around 10 credits per hour on SETI. It should get approximately the same on Einstein. But it isn't perfect, so maybe it gets 11 credits per hour on Einstein. And maybe it only gets 9 credits per hour on Malaria Control. But that same machine on Rosetta running an optimised client would be getting perhaps 30 credits per hour. That's not fair to those who use standard clients, or to those who have processors that don't produce such staggeringly outrageous benchmark scores. Nor is it fair to those who try and compete over BOINC as a whole.

I want to see that past unfairness corrected. Not only that, but I expect to use a new stats system that awards credit based on work done, not on what is claimed by dodgy benchmarks. Yet if Rosetta keeps on exporting stats based on the current claimed and granted credit then it keeps on being inaccurate.

EDIT - Another thing Jose, the DC Vault is managed by Team Ninja. They dictate what goes in it. The DC Vault also covers a lot more than just BOINC. It has no bearing on BOINC and is not what is the matter of discussion.
ID: 23326 · Rating: 0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 23327 - Posted: 19 Aug 2006, 3:09:22 UTC - in response to Message 23326.  
Last modified: 19 Aug 2006, 3:12:30 UTC

I'll put my main point again in another way - on BOINC, one hour of CPU time on one project should give around the same credit as that same machine spending an hour on another project. 1 SETI hour ~= 1 Einstein hour ~= 1 Climate Prediction hour ~= 1 Predictor hour ~= 1 Leiden Classical hour ~= 1 Rectilinear Crossing Numbers hour and so on.


Why is this so darned important? There is NO valid reason to compare results on DC projects under BOINC than it is to say distributed.net needs to have parity with SoB, or D2OL, or DPAD, or F@H, or any other of the myriad of DC projects. Each needs to stand on it's own, and recruit people and CPUs based on the merits of the project and how it's run.

BOINC is simply a piece of software that is only one part of what allows a DC project to do work. It has no relevance to credit parity across projects. This is a myth that was started by David Anderson - something out of a bad dream.


Maybe a few project principles need to stand up to Anderson, and say "Screw you - we're doing our own thing with credits. Too bad." Since BOINC is open-source, what can he do? Nothing. and maybe it will empower others to see the fallacy, and quit wasting precious man-hours on trying to achive the impossible.
Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 23327 · Rating: -2 · rate: Rate + / Rate - Report as offensive    Reply Quote
John McLeod VII
Avatar

Send message
Joined: 17 Sep 05
Posts: 108
Credit: 195,137
RAC: 0
Message 23328 - Posted: 19 Aug 2006, 3:15:36 UTC - in response to Message 23325.  

Leiden Classical another of the great Boinc Projects that have not allowed mew members in months and whose files get corrupted while its manager is enoying the sun at Curacao.


its called been a Alpha project, hence why people are not allowed to flood in, the server is not upto the task there of letting loads of people in at the moment


But many teams have used thee fact that the project is closed to new teams to prop their standing in the DC Vault: an endeavor that is supposed to include only projects that are open for recruitment. You see nothing when the bending of the rules favors your team.

I had no idea that was being done (I hadn't even heard of the DC Vault). If the rules there are that a project needs to be open to recruitment, then those are the rules there, and they should be enforced there. However, those rules do not apply to other places that track BOINC statistics such as BOINC Stats, Munday Web and others.


Leiden Classic BOINC Synergy 5 9,781.420765
Be consistent man: that is a serious rules violation that is proping your teams 5th place in that endeavor . I thought you were against all rule violations. Ah , that rule breaking favors your team thus is not cheating.

Working under the rules allowed by the developers of a project (in the case of Rosetta using optimized clients) is cheating because your team and you do not like it.

Nice double standard you have there John.




OK, I just read through the Leiden thread at DC vault, and the moderator is aware that Leiden is limiting users, and (s)he believes this is a temporary problem, and is not going to remove Leiden for now just to add it back in a few weeks. I am assuming that if Leiden does not open up membership again in a timely manner, it will be removed until it does.

My statement was that if the rules of the DC Vault were being broken, they should be enforced. The problem with Rosetta is that the expectation that all BOINC projects would produce about the same # of credits / hour of crunching on a given host was violated, thus leading to unfairness, allegations of cheating, and a general flame fest.

In both cases I am advocating adhering to the rules. Just in one case a more general rule than the one you are looking at. This is not a double standard.


BOINC WIKI
ID: 23328 · Rating: 9.9920072216264E-15 · rate: Rate + / Rate - Report as offensive    Reply Quote
John McLeod VII
Avatar

Send message
Joined: 17 Sep 05
Posts: 108
Credit: 195,137
RAC: 0
Message 23329 - Posted: 19 Aug 2006, 3:18:01 UTC - in response to Message 23327.  
Last modified: 19 Aug 2006, 3:36:10 UTC

I'll put my main point again in another way - on BOINC, one hour of CPU time on one project should give around the same credit as that same machine spending an hour on another project. 1 SETI hour ~= 1 Einstein hour ~= 1 Climate Prediction hour ~= 1 Predictor hour ~= 1 Leiden Classical hour ~= 1 Rectilinear Crossing Numbers hour and so on.


Why is this so darned important? There is NO valid reason to compare results on DC projects under BOINC than it is to say distributed.net needs to have parity with SoB, or D2OL, or DPAD, or F@H, or any other of the myriad of DC projects. Each needs to stand on it's own, and recruit people and CPUs based on the merits of the project and how it's run.

BOINC is simply a piece of software that is only one part of what allows a DC project to do work. It has no relevance to credit parity across projects. This is a myth that was started by David Anderson - something out of a bad dream.


Maybe a few project principles need to stand up to Anderson, and say "Screw you - we're doing our own thing with credits. Too bad." Since BOINC is open-source, what can he do? Nothing. and maybe it will empower others to see the fallacy, and quit wasting precious man-hours on trying to achive the impossible.

Ok, reworded by request of the moderator.

Cross project credit equality has been part of the design of BOINC since day 1 (before there were any projects on BOINC other than an alpha test of S@H). It is important to the developers of BOINC that this be so. It is important to others that this be so. It has no effect on those that wish to compare a single projects output to itself.


BOINC WIKI
ID: 23329 · Rating: 2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Biggles
Avatar

Send message
Joined: 22 Sep 05
Posts: 49
Credit: 102,114
RAC: 0
Message 23331 - Posted: 19 Aug 2006, 3:27:18 UTC - in response to Message 23327.  

Why is this so darned important? There is NO valid reason to compare results on DC projects under BOINC than it is to say distributed.net needs to have parity with SoB, or D2OL, or DPAD, or F@H, or any other of the myriad of DC projects. Each needs to stand on it's own, and recruit people and CPUs based on the merits of the project and how it's run.

BOINC is simply a piece of software that is only one part of what allows a DC project to do work. It has no relevance to credit parity across projects. This is a myth that was started by David Anderson - something out of a bad dream.


It was a feature of BOINC that was always part of its original design. David Anderson was the guy behind BOINC, that makes it a lot more than a myth.

Also, remember that the combined RAC is supposed to give an indication of the total throughput of the project in terms of TeraFLOPS. That might not matter to the individual users, but it does matter to the admins when the amount of computing power they have at their disposal affects their success in getting research grants or other resources.

Lastly on the topic of cross BOINC crediting, proper parity gives rise to healthy competition. And we all know how powerful a motivator competition can be.

I will concede that cross-project parity is of relatively little importance however. I'd certainly like to see it fixed, but the whole credit system needs fixing here first.

The thing is, even if we isolate Rosetta from BOINC and don't worry about parity with other projects, the credit system still doesn't work even for just this project. The reason for that is that Athlon 64s on Windows (general statement, mostly true) are getting 3 times the points of any other system for the same amount of actual work. I already explained elsewhere that clock for clock a Barton core Athlon XP will do approximately the same amount of work as a Newcastle/Winchester/Venice core Athlon 64, but that it gets far less points.

And that is killing healthy competition. It removes a lot of incentive for many people to crunch for this project if they can't get stats that measure up to the work they are actually doing. There's a helluva lot of P4s out there, and even with optimised clients they get far less credit than an Athlon 64 would, despite doing more work than the stats would indicate. That'll drive people away. The project needs to sort out the credit issue before people leave en mass.
ID: 23331 · Rating: 2 · rate: Rate + / Rate - Report as offensive    Reply Quote
John McLeod VII
Avatar

Send message
Joined: 17 Sep 05
Posts: 108
Credit: 195,137
RAC: 0
Message 23332 - Posted: 19 Aug 2006, 3:33:33 UTC - in response to Message 23330.  

Please split the cross project equality into another topic, it's a big enough topic to warrant one.

John, please reword your post. . it's demeaning to say that someone will never understand something.

I understand that it is demeaning, we have discussed this on other threads in different projects for several months now with the same result. I and others state that it was part of the design, and is important to the developers and others, and he says that he doesn't care that it is part of the design.


BOINC WIKI
ID: 23332 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 23334 - Posted: 19 Aug 2006, 3:40:25 UTC - in response to Message 23332.  

Please split the cross project equality into another topic, it's a big enough topic to warrant one.

John, please reword your post. . it's demeaning to say that someone will never understand something.

I understand that it is demeaning, we have discussed this on other threads in different projects for several months now with the same result. I and others state that it was part of the design, and is important to the developers and others, and he says that he doesn't care that it is part of the design.


I understand fully that it was part of the design.

The myth is that it will ever happen. It's impossible, and is not worth the effort being put into it. It needs to be DROPPED from the BOINC agenda so the science projects can proceed with their real work. There has probably been thousands of wasted programmer hours and way too many CPU cycles wasted talking about it.


I fully agree that **WITHIN** a project that parity is needed for all operating systems on all platforms.

Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 23334 · Rating: -2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ethan
Volunteer moderator

Send message
Joined: 22 Aug 05
Posts: 286
Credit: 9,304,700
RAC: 0
Message 23335 - Posted: 19 Aug 2006, 3:41:52 UTC

Moving some posts here to continue this discussion.
ID: 23335 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
John McLeod VII
Avatar

Send message
Joined: 17 Sep 05
Posts: 108
Credit: 195,137
RAC: 0
Message 23337 - Posted: 19 Aug 2006, 3:42:12 UTC - in response to Message 23333.  

The thing is, even if we isolate Rosetta from BOINC and don't worry about parity with other projects, the credit system still doesn't work even for just this project. The reason for that is that Athlon 64s on Windows (general statement, mostly true) are getting 3 times the points of any other system for the same amount of actual work. I already explained elsewhere that clock for clock a Barton core Athlon XP will do approximately the same amount of work as a Newcastle/Winchester/Venice core Athlon 64, but that it gets far less points.

And that is killing healthy competition. It removes a lot of incentive for many people to crunch for this project if they can't get stats that measure up to the work they are actually doing. There's a helluva lot of P4s out there, and even with optimised clients they get far less credit than an Athlon 64 would, despite doing more work than the stats would indicate. That'll drive people away. The project needs to sort out the credit issue before people leave en mass.


I don't think this will be an issue with the new system.

Work credit takes the boinc-derived credit claims from many computers and determines an average. This average is then applied to each computer that ran that WU.

Since it's a set work for credit constant, the cpu type is irrelevant. If one computes faster than the other, it gets more credits since it's able to process more work in the same time.

Let me know if I read your post wrong.

It will be much less of an issue. The remaining part is that if enough of those hosts are running core clients that inflate credit requests, cross project comparison will not work as well. Yes, I know that cross project comparison is not perfect as it stands, but it should be worked on so that it gets better.


BOINC WIKI
ID: 23337 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 23338 - Posted: 19 Aug 2006, 3:44:55 UTC - in response to Message 23329.  

Cross project credit equality has been part of the design of BOINC since day 1 (before there were any projects on BOINC other than an alpha test of S@H). It is important to the developers of BOINC that this be so. It is important to others that this be so. It has no effect on those that wish to compare a single projects output to itself.


It does have a profound effect on a project in that it has to waste endless time trying to achieve an impossible goal.

Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 23338 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
John McLeod VII
Avatar

Send message
Joined: 17 Sep 05
Posts: 108
Credit: 195,137
RAC: 0
Message 23343 - Posted: 19 Aug 2006, 3:57:18 UTC - in response to Message 23334.  

Please split the cross project equality into another topic, it's a big enough topic to warrant one.

John, please reword your post. . it's demeaning to say that someone will never understand something.

I understand that it is demeaning, we have discussed this on other threads in different projects for several months now with the same result. I and others state that it was part of the design, and is important to the developers and others, and he says that he doesn't care that it is part of the design.


I understand fully that it was part of the design.

The myth is that it will ever happen. It's impossible, and is not worth the effort being put into it. It needs to be DROPPED from the BOINC agenda so the science projects can proceed with their real work. There has probably been thousands of wasted programmer hours and way too many CPU cycles wasted talking about it.


I fully agree that **WITHIN** a project that parity is needed for all operating systems on all platforms.

The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close.


BOINC WIKI
ID: 23343 · Rating: 0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 23345 - Posted: 19 Aug 2006, 4:03:18 UTC - in response to Message 23343.  
Last modified: 19 Aug 2006, 4:04:11 UTC

Please split the cross project equality into another topic, it's a big enough topic to warrant one.

John, please reword your post. . it's demeaning to say that someone will never understand something.

I understand that it is demeaning, we have discussed this on other threads in different projects for several months now with the same result. I and others state that it was part of the design, and is important to the developers and others, and he says that he doesn't care that it is part of the design.


I understand fully that it was part of the design.

The myth is that it will ever happen. It's impossible, and is not worth the effort being put into it. It needs to be DROPPED from the BOINC agenda so the science projects can proceed with their real work. There has probably been thousands of wasted programmer hours and way too many CPU cycles wasted talking about it.


I fully agree that **WITHIN** a project that parity is needed for all operating systems on all platforms.

The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close.


And how do you propose to do that with an open-source core client? It can be made to report itself as any legitimate (non-optimized, although optimizing by itself is perfectly legal) client.

Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 23345 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Biggles
Avatar

Send message
Joined: 22 Sep 05
Posts: 49
Credit: 102,114
RAC: 0
Message 23346 - Posted: 19 Aug 2006, 4:04:47 UTC - in response to Message 23343.  

The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close.


I do agree with you, but with BOINC being open source there is nothing to prevent someone from putting subtle changes into the source. A couple of percent here and there isn't really noticable to begin with, but it adds up.

Really, in using Ralph to calculate averages for the granted work credit, the benchmark and some form of timer need to be built into Rosetta itself so that it can't be easily modified. Even then it would still be possible to hack the binaries, but it would at least be something that was challenging instead of something that any kid with a bit of programming knowledge could change.
ID: 23346 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 23355 - Posted: 19 Aug 2006, 4:32:49 UTC - in response to Message 23346.  
Last modified: 19 Aug 2006, 4:33:16 UTC

The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close.


I do agree with you, but with BOINC being open source there is nothing to prevent someone from putting subtle changes into the source. A couple of percent here and there isn't really noticable to begin with, but it adds up.

Really, in using Ralph to calculate averages for the granted work credit, the benchmark and some form of timer need to be built into Rosetta itself so that it can't be easily modified. Even then it would still be possible to hack the binaries, but it would at least be something that was challenging instead of something that any kid with a bit of programming knowledge could change.


Doesn't that effectively remove RALPH as the beta (alpha?) test site for Rosetta? It sounds like you're talking about one type of Ralph client application to determine the credit/decoy for a given WU type, and then using a different application on Rosetta to actually crunch the majority of the work.
Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 23355 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
John McLeod VII
Avatar

Send message
Joined: 17 Sep 05
Posts: 108
Credit: 195,137
RAC: 0
Message 23359 - Posted: 19 Aug 2006, 4:46:34 UTC - in response to Message 23346.  

The attempt should be made to keep parity across projects. I admit it will be impossible for it to be perfect, but it should be possible to get fairly close. The place to start would be a clear statement banning the optimized BOINC clients from participating in RALPH. If this is done and enforced, then the new method should come fairly close.


I do agree with you, but with BOINC being open source there is nothing to prevent someone from putting subtle changes into the source. A couple of percent here and there isn't really noticable to begin with, but it adds up.

Really, in using Ralph to calculate averages for the granted work credit, the benchmark and some form of timer need to be built into Rosetta itself so that it can't be easily modified. Even then it would still be possible to hack the binaries, but it would at least be something that was challenging instead of something that any kid with a bit of programming knowledge could change.

Build a timer into Rosetta? That sounds a great deal like FLOPS counting. Not a bad idea, but substantially more difficult to program than using fixed credits per task, which works if tasks within a series have the same difficulty.


BOINC WIKI
ID: 23359 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Biggles
Avatar

Send message
Joined: 22 Sep 05
Posts: 49
Credit: 102,114
RAC: 0
Message 23405 - Posted: 19 Aug 2006, 12:14:16 UTC - in response to Message 23355.  

Doesn't that effectively remove RALPH as the beta (alpha?) test site for Rosetta? It sounds like you're talking about one type of Ralph client application to determine the credit/decoy for a given WU type, and then using a different application on Rosetta to actually crunch the majority of the work.


What I mean is the Ralph version of Rosetta should have a timer and a benchmark. Benchmark within the science app itself to give a fair, non-optimised benchmark score. A timer also to prevent the modifying of the reported work unit run time.

For instance, if I was running Ralph, and my goal was to skew the averages of work units so that more credit was received for them, I need to use a bigger BOINC benchmark by using an optimised client, or if the benchmark is accurate, by making the work unit appear longer than it actually is. Hence the need for the actual science app - on Ralph if they use it to generate the average credits for a series of units, or Rosetta if they decide to make it more robust and a better average - to have a benchmark and a timer in it.

John MacLeod VII wrote:
Build a timer into Rosetta? That sounds a great deal like FLOPS counting. Not a bad idea, but substantially more difficult to program than using fixed credits per task, which works if tasks within a series have the same difficulty.


I don't see the difficulty in adding a few system calls like GetProcessTimes() or GetThreadTimes(). It doesn't need to be in the main body of the science code itself, rather after it. That way we have a measure of how much CPU time was actually used, as opposed to just what a BOINC client reports. This nullifies one of the ways of possibly cheating.
ID: 23405 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
John McLeod VII
Avatar

Send message
Joined: 17 Sep 05
Posts: 108
Credit: 195,137
RAC: 0
Message 23571 - Posted: 19 Aug 2006, 20:52:42 UTC - in response to Message 23568.  


John, in the new system it doesn't matter what client users have. . the credit claims get averaged out and everyone gets the same work credit.


OK, this is the wrong thread - it needs to be in the discussion about BOINC credit being compatible across projects. Please move it to the correct thread as I cannot answer a post in a different thread and get the back link.

Credits per hour does matter when you are trying to do cross project compatibility, and I believe that the new scheme uses some hosts (RALPH) as a bed to fix the credits per result for each series. In particular, I was interested in disallowing the "optimized" core clients from being used to determine that score.


BOINC WIKI
ID: 23571 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Cross Project Credit Equality



©2024 University of Washington
https://www.bakerlab.org