RAC cheats, is this a problem

Message boards : Number crunching : RAC cheats, is this a problem

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Moderator9
Volunteer moderator

Send message
Joined: 22 Jan 06
Posts: 1014
Credit: 0
RAC: 0
Message 12313 - Posted: 20 Mar 2006, 4:23:43 UTC - in response to Message 12306.  

...The solution for me, I think, is to just chill out - and don't look at the credits - if I really don't care. So, I won't look any more. I won't! I won't! I won't! Do you think that will work?


LOL:
I am skeptical if you can make that work for you, it doesn't work for a lot of people. But I am pulling for you. ;>)

If it helps any, the project really is looking hard at this issue. It is my belief that rescue is on the way but I am afraid it may take some time before the fix is implemented. Now I am not talking months and months, but I am certain it will be some number of weeks. Right now the project is working on bug fixes, and preparing for an international protein folding event in April. So I suspect that the bugs will be fixed by then, but the credit problem may come after that event. But the project team may have different ideas. I have sent them some messages on this issue, and i am waiting for some answers. When I know you will know, I promise.

Moderator9
ROSETTA@home FAQ
Moderator Contact
ID: 12313 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Nightlord

Send message
Joined: 6 Dec 05
Posts: 5
Credit: 1,635,379
RAC: 0
Message 12355 - Posted: 20 Mar 2006, 22:29:05 UTC - in response to Message 12306.  
Last modified: 20 Mar 2006, 22:37:26 UTC

to Nightlord:

I do use the truXoft client enabled only for SETI - the results for all the other projects will be unmolested using this method. You might ask why I want to adjust my credits (for SETI) - this is accepted in the SETI project since I am also using an optimized SETI app on a P4HT which you may not know receives an unrealistic benchmark because of the HT. I once used the Crunch3r optimized Boinc client and do not use it now for the very reason that it is an unfair credit exploit in the other projects.

The solution for me, I think, is to just chill out - and don't look at the credits - if I really don't care. So, I won't look any more. I won't! I won't! I won't! Do you think that will work?


Fair enough.

Are you aware that turning off the calibration in the truXoft client does not remove the benchmark optomisation which is performed by that client? Your benchmarks and hence claimed credit on Rosetta will be higher than the standard boinc client would return. However, I can now see that is not your intention.

Briefly switch back to the standard client, run the benchmarks and compare to confirm if you wish.

[edit]
Be sure to back-up your boinc directory before switching clients. I accidentally just trashed a cache full of WU in verifying this.
[/edit]


ID: 12355 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bob Guy

Send message
Joined: 7 Oct 05
Posts: 39
Credit: 24,895
RAC: 0
Message 12560 - Posted: 23 Mar 2006, 8:50:22 UTC - in response to Message 12355.  


Are you aware that turning off the calibration in the truXoft client does not remove the benchmark optomisation which is performed by that client? Your benchmarks and hence claimed credit on Rosetta will be higher than the standard boinc client would return.



Actually I think that's not accurate - the benchmark by the truXoft client will result in LOWER claims in projects not using the 'optimize' feature. This seems counterintuitive to me but that is what is documented at the truXoft site. This is evidently because the client code has a modified benchmark that is tailored to the 'optimize' feature - i.e. they work together. The code for 'claimed credit' is not the same code as in the standard client.

I've even considered going back to the standard client because of this but by my measurement there is little difference and I like the credits I'm getting from Seti.

ID: 12560 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
James

Send message
Joined: 27 Mar 06
Posts: 4
Credit: 23,809
RAC: 0
Message 12956 - Posted: 2 Apr 2006, 18:34:08 UTC - in response to Message 12268.  
Last modified: 2 Apr 2006, 18:35:39 UTC

Looking at some of the top computers shows that they may be (are!) exploiting the credit system. My computer with a standard Boinc client charges about 14 credits per hour while many of the computers in the top RAC list are charging 40 to 60 credits per hour. Is this reasonable? I think not!

In addition some of those computers appear to be doing 1 or 2 hours of work and claiming that they have done 4 to 8 hours of work thereby further inflating their credit claims.

Any fool can create a 'compile your own' Boinc client containing any number of credit exploits. All reputable Boinc projects should ONLY allow an official Boinc client - self compiled clients should be strictly prohibited. I realize that some individuals with odd computer hardware would then not be able to run Boinc projects. In such cases a review of their clients by project developers would be necessary or they would simply be forced to use compatible hardware or just not run Boinc projects.

The reason that an official Boinc client is needed is that many who now process the workunits will be discouraged by continuing credit exploitation (cheating) and simply refuse to contribute any more of their computer time. Potential new users, upon hearing about these exploits and finding that the project developers have failed to take any action, will also not contribute their computer time.

It should also be simple for any project to set a maximum credits per hour per workunit value and enforce it. I realize this is a simplistic solution and may not address all of the projects' requirements.

I find this credit exploitation offensive and the failure by the developers to take any action equally offensive. I contribute my computer time because I believe the science being done here is important. Perhaps I am the crazy one.



There really is not too much that a project can do about this. The BOINC code has been publicly released. It is very easy to compile it so it looks identical to the official version, and yet has some benchmark adjustments. Many of these adjusted clients have been made so they fall with in the normal range of credit claims, but at the top end of them.

In some cases the modifications to the BOINC client are simply taking advantage of special features of the computer for which they are compiled and are in fact producing a more accurate and legitimate benchmark for that system. The problem comes in when the project code does not use those features of the system. This causes the benchmark to be high, but the actual processing time to be slower, thus producing higher credit claims. In that case the project might be accused of not provide an application that takes full advantage of the computing power available, but they are hardly doing that intentionally, it is just difficult to have a special version for every possible special system out there.

As to a limit on the credit per Work Unit. This might work on other projects, but it will not on Rosetta. With the time setting it is possible to run a work unit from between 1 hour and days. With that level of variability where would you set the maximum credit? How would you verify it without overloading the servers? In any case you would still have to accommodate the broad range of possible credit claims made by different systems. With the time setting legitimate credit claims can average from 10 to over 2400 credits for a single work unit. Hourly claims range from 10 to around 50, depending on system speed. So this is just not a workable answer.

As far as the developers not taking action. They have deployed the standard BOINC system. While they have chosen not to use redundancy, the ability to make that choice is part of the server software. They have removed some of the more severe violations from the stats as they are identified. The project team is thinking about a variety of possible solutions to the credit issue, but right now they are focused primarily on killing the bugs in the application. Once that is done, they have stated publicly that they will return to the issue of credit claims and awarding of credits owed from a range of processing problems.


Mod, you are twisting this a bit. Regardless of an 'opimized' client a project can calibrate the claimed credits - look no further than einstein@home. They most definately adjust the credits to get rid of the use of inflated benchmarks.

As for the somewhat weak claim that people are merely doing this because boinc doesn't fully utilize their resources (which is the essence of your claim) that is a boinc issue and not a system issue. I know for a fact that AMDs are supported poorly in BOINC compilations in general. That doesn't mean I 'deserve' more credits.

Yes, the source code has been released to the public. Perhaps you should also note that the client doesn't crunch the WUs - the Rosetta application does. It has nothing to do with the project other than to enable Rosetta's app to run and to manage preferences. Rosetta controls the project and has chosen NOT to release their source (unlike SETI where optimization can occur) and has chosen NOT to offer system specific compilations to maximize cpu efficiency.

Which is neither here nor there. RAC cheating is an issue in that it seems to be a vanity issue where people feel the need to be in the 'top computer' section. Given that the run times of these WUs are known it's not exactly like you can't figure out how to adjust credits - Einstein has.
ID: 12956 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Snake Doctor
Avatar

Send message
Joined: 17 Sep 05
Posts: 182
Credit: 6,401,938
RAC: 0
Message 13026 - Posted: 4 Apr 2006, 4:22:45 UTC - in response to Message 12956.  
Last modified: 4 Apr 2006, 4:23:10 UTC

James wrote

Mod, you are twisting this a bit. Regardless of an 'opimized' client a project can calibrate the claimed credits - look no further than einstein@home. They most definately adjust the credits to get rid of the use of inflated benchmarks.

As for the somewhat weak claim that people are merely doing this because boinc doesn't fully utilize their resources (which is the essence of your claim) that is a boinc issue and not a system issue. I know for a fact that AMDs are supported poorly in BOINC compilations in general. That doesn't mean I 'deserve' more credits.

Yes, the source code has been released to the public. Perhaps you should also note that the client doesn't crunch the WUs - the Rosetta application does. It has nothing to do with the project other than to enable Rosetta's app to run and to manage preferences. Rosetta controls the project and has chosen NOT to release their source (unlike SETI where optimization can occur) and has chosen NOT to offer system specific compilations to maximize cpu efficiency.

Which is neither here nor there. RAC cheating is an issue in that it seems to be a vanity issue where people feel the need to be in the 'top computer' section. Given that the run times of these WUs are known it's not exactly like you can't figure out how to adjust credits - Einstein has.


I guess one good twist deserves a solid fast spin.

We Must look for intelligent life on other planets as,
it is becoming increasingly apparent we will not find any on our own.
ID: 13026 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jimi@0wned.org.uk

Send message
Joined: 10 Mar 06
Posts: 29
Credit: 335,252
RAC: 0
Message 13055 - Posted: 4 Apr 2006, 19:36:30 UTC

Look at:

https://boinc.bakerlab.org/rosetta/show_user.php?userid=58151

I know that chew* was running that x2 3800+ at 3.2GHz at least, and that he's using the AMD optimised client. Is there anything wrong in that? I don't think so. It's a very useful reference. By what criteria does einstein@home trim out unlikely benchmarks? Would this disqualify beautifully overclocked machines like this? One has to be careful and realistic in seperating the wheat from the chaff.
ID: 13055 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 13059 - Posted: 4 Apr 2006, 20:56:18 UTC
Last modified: 4 Apr 2006, 20:57:37 UTC

I wrote about this in another thread, but let me copy the relevant part here as well.

James wrote:
Regardless of an 'opimized' client a project can calibrate the claimed credits - look no further than einstein@home. They most definately adjust the credits to get rid of the use of inflated benchmarks.

Rosetta controls the project and has chosen NOT to release their source (unlike SETI where optimization can occur) and has chosen NOT to offer system specific compilations to maximize cpu efficiency.

Which is neither here nor there. RAC cheating is an issue in that it seems to be a vanity issue where people feel the need to be in the 'top computer' section. Given that the run times of these WUs are known it's not exactly like you can't figure out how to adjust credits - Einstein has.


Since you keep mentioning Einstein as a "model" to follow, where did you read that they do this kind of calibration? (web address please). My BOINC massively underclaim credits (as using akosf's app my PCs complete a WU in 1/4th of the time it used to take) for Einstein. From looking at my results, Einstein just uses a quorum of 3 and grants the credit of the middle claim e.g. wu6428418.
My BOINC's claim was for 13.99 credits, someone else's 56 and we all 3 received the middle one of 41 credits.

A project which is using quorum of 3,4 etc is effectively slashing effective CPU speed available to 1/3rd or 1/4th of donated "raw" CPU speed. I see this as an ultimate waste of donated resources and personally have stopped crunching for projects which did this just to appease credit-obsessed people, unless there is a valid science reason to operate with a 3-4x redundancy.

Anyway, afaik the "credit calibration" feature you mentioned is used in SETI-Beta and I hope Rosetta and other projects will use it as soon as it goes mainstream.
Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 13059 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nuadormrac

Send message
Joined: 27 Sep 05
Posts: 37
Credit: 202,469
RAC: 0
Message 13313 - Posted: 9 Apr 2006, 11:27:57 UTC
Last modified: 9 Apr 2006, 11:30:17 UTC

My Einstein credits get very much under-estimated using akosf's algorithm. And what's more D40 which I'm using now, has a major improvement over C37 on my Athlon 64. C37 gave a major improvement, bring the then albert WUs down to about a half hour or so of crunch time (don't exactly remember now). Then longer albert units came out on E@H that took about 1.5 hours. Anyhow, D40, which includes 3d now optimizations in addition to the sse optimizations that akosf included previously is down to about 1 hour (or 1/3 of what optimized science app c37 included). My claimed credits are low there, and remain low...

However, unless someone else is using the optimized app, the quorum of 3 gives me more standard credit. I simply will not use an optimized CC, because most of my crunching is not on SETI (where I use crunch3r's science app) or even SETI and Einstein. True, projects like CPDN will be unaffected, but projects like Rosetta right here, definitely will...

However, BOINC was made open source for a reason, and trying to force only "official clients" is not the answer. It was in part to allow for running on currently unsupported CPU platforms... What's more, some projects (like some of the Japanese cell computing projects), state where they're listed that they require a non-standard client.

The answer is what SETI is doing with enhanced which is now in beta. In all the SETI beta WUs I've received thus far, the claimed credits are virtually identical of each other, regardless of computer or speed. Looking at some of my results on SETI beta, the actual variance from 1 credit claim to another is < 1 credit point (and in many cases is within a 10th of a point). I think the SETI staff has a fine handle on that, and one that will dispense of questions wrt optimized core clients once and for all.
ID: 13313 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Neal Chantrill
Avatar

Send message
Joined: 6 Nov 05
Posts: 52
Credit: 1,199,615
RAC: 0
Message 13324 - Posted: 9 Apr 2006, 14:41:18 UTC

https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=197389

Seems a bit suspect, what do you think??

13,000k higher than the second placed computer in RAC.

CLICK ME TO VISIT THE CLANGERS FORUM
ID: 13324 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 13338 - Posted: 9 Apr 2006, 17:34:22 UTC - in response to Message 13313.  

The answer is what SETI is doing with enhanced which is now in beta. In all the SETI beta WUs I've received thus far, the claimed credits are virtually identical of each other, regardless of computer or speed. Looking at some of my results on SETI beta, the actual variance from 1 credit claim to another is < 1 credit point (and in many cases is within a 10th of a point). I think the SETI staff has a fine handle on that, and one that will dispense of questions wrt optimized core clients once and for all.


I fully agree with you that the new SETI-beta system is best and accurate and FAIR, as one can see completely different computers/OS/compilers e.g. a P4/Win and a Mac claiming the very same amount of credit for the same work done.

I hope the other projects implement it as soon as it goes mainstream.

But note: a lot of people with massively overclocked CPUs now getting high benchmarks (which fit entirely in L2 cache) in BOINC, will get a little disappointed, because the new SETI-beta system will award credits based on TRUE work done, which often depends on memory-CPU bandwidth rather than raw CPU clock speed.

Then you'll see a 2GHz P4 getting almost the same credits with a 3GHz one.
Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 13338 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mo

Send message
Joined: 12 Apr 06
Posts: 1
Credit: 6,163
RAC: 0
Message 13531 - Posted: 12 Apr 2006, 9:19:57 UTC - in response to Message 13338.  

The second reply in this thread points out that this is science, not a game; it's human nature to be competitive, but wondering whether RAC cheating is a problem seems a bit over-the-top. This is all about the work units - as long as the integrity of the work being done is maintained, does it matter if someone will actually go out of their way to fudge their numbers?

I only recently installed BOINC and look at my accruing credit for what it is: http://en.wikipedia.org/wiki/BOINC_Credit_System - not where it places me on the world charts.

That said, the veracity of any claims made about the computing power of the network are compromised by said cheating... any change that will lead to a truer picture of how much computing power that BOINC really harnesses is desireable.
ID: 13531 · Rating: 2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 115,269,606
RAC: 46,871
Message 13556 - Posted: 12 Apr 2006, 17:38:49 UTC - in response to Message 13531.  

The second reply in this thread points out that this is science, not a game; it's human nature to be competitive, but wondering whether RAC cheating is a problem seems a bit over-the-top. This is all about the work units - as long as the integrity of the work being done is maintained, does it matter if someone will actually go out of their way to fudge their numbers?


The reason it's so important is that the competition is a lot more intense if the credit is known to be fair. I saw a thread that said TSCRussia (one of the biggest DC teams) were avoiding BOINC projects until the credit is made fair. I think the project will get a lot more work throughput when a fair credit system is introduced.
ID: 13556 · Rating: 3 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 15664 - Posted: 7 May 2006, 22:33:56 UTC - in response to Message 13556.  

The reason it's so important is that the competition is a lot more intense if the credit is known to be fair. I saw a thread that said TSCRussia (one of the biggest DC teams) were avoiding BOINC projects until the credit is made fair. I think the project will get a lot more work throughput when a fair credit system is introduced.


This is very true, Rosetta TeraFLOPS will greatly improve by implementing a SETI-Enhanced-like credit mechanism, even if most of current crunchers are willing to look another way, because Rosetta's goals are so important.
Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 15664 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
BennyRop

Send message
Joined: 17 Dec 05
Posts: 555
Credit: 140,800
RAC: 0
Message 15670 - Posted: 7 May 2006, 23:27:23 UTC

An important thing to keep in mind that none of us represents 100% of the Rosetta crunching volunteer force.

In my first DC project, I joined a project because I wanted to do something useful with a system that was on 24/7 - and needed to have network traffic to keep the system from getting kicked off. I joined a team from the forum where I ran across mention of the DC project; and after a bit of time, noticed that I was climbing through the ranks of the team. Noticing the rate of climb between myself and others in the team, I figured that by adding another system, I could catch up to.. (fill in blank).. and if I added yet another system, I could catch up to.. (fill in another blank). I got bit by the competitive spirit. :)

I started off innocently.. and got bit by the competitive spirit. So those that start off innocently can find themselves thinking, "I could do more." We've got non competitive people, very competitive people, non-competitive people about to become competitive people, very competitive people that are forced to give up being competitive, and all kinds of other competitive settings represented here.

Fair and equal stats will bring an increase of competition and TeraFlops to the project from those that have a competitive spirit. Be it for person standings or team standings. Having a reliable client that doesn't require babysitting will really help keep these competitive folks here after their guantlets are over or whatever other goal they're reaching for.


ID: 15670 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Aglarond

Send message
Joined: 29 Jan 06
Posts: 26
Credit: 446,212
RAC: 0
Message 16053 - Posted: 12 May 2006, 14:54:58 UTC

Here is my point of view. I believe that Rosetta and Simap are most intersting projects. I didn't care about Einstein until optimized clients came. Now I'm stuck between caring about science and hunting for credits. I can get more than 2000 credits per day crunching for Einstein or less than 700 per day crunching for Rosetta. And it is hard to decide what makes me feel better.
If there is possibility how to get enough points (close to what I can get on Einstein) while crunching for Rosetta, it would satisfy me the most. Rosetta would get the results and I would get the points. However, someone would call it cheating. I don't want to cheat. So I will stay with Einstein and come back later to see if something changed.

Happy crunching.

P.S. one idea: what about kindly asking Akos Fekete, who made optimizations on Einstein, if he could look at Rosetta and try to make optimizations here? Although, it may be necessary to pay him for his effort, as he is rather busy.
ID: 16053 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Aglarond

Send message
Joined: 29 Jan 06
Posts: 26
Credit: 446,212
RAC: 0
Message 16270 - Posted: 14 May 2006, 19:35:58 UTC - in response to Message 16053.  

P.S. one idea: what about kindly asking Akos Fekete, who made optimizations on Einstein, if he could look at Rosetta and try to make optimizations here? Although, it may be necessary to pay him for his effort, as he is rather busy.


Update: David has contacted Akos, but he will stay with Einstein for now.
ID: 16270 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FluffyChicken
Avatar

Send message
Joined: 1 Nov 05
Posts: 1260
Credit: 369,635
RAC: 0
Message 16296 - Posted: 15 May 2006, 9:56:24 UTC

Aglarond,

So why do credit accumulation make you happier than the actual science enhancement of a project ?
Team mauisun.org
ID: 16296 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dag
Avatar

Send message
Joined: 16 Dec 05
Posts: 106
Credit: 1,000,020
RAC: 0
Message 16332 - Posted: 15 May 2006, 18:53:00 UTC - in response to Message 16296.  
Last modified: 15 May 2006, 19:08:51 UTC


So why do credit accumulation make you happier than the actual science enhancement of a project ?

One thing to consider, with a well tuned client one could produce a lot of science for the project. I have 2 and a half cores running on Windoz and 5 cores running on Linux, and all are
dag
--Finding aliens is cool, but understanding the structure of proteins is useful.
ID: 16332 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2

Message boards : Number crunching : RAC cheats, is this a problem



©2024 University of Washington
https://www.bakerlab.org