Is this for real???

Message boards : Number crunching : Is this for real???

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 6 · Next

AuthorMessage
Winkle

Send message
Joined: 22 May 06
Posts: 88
Credit: 1,354,930
RAC: 0
Message 17347 - Posted: 30 May 2006, 10:11:24 UTC

Compare the claimed speed of one of mine... https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=230088
Dell 2800 running Server 2003.
Xeon 3G. 1316.2 million ops/sec (Floating)

With

https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=210800
Xeon 3.06G 26989.43 million ops/sec (Floating)

I can't see the op-sys making *that* much difference.
ID: 17347 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
AMD_is_logical

Send message
Joined: 20 Dec 05
Posts: 299
Credit: 31,460,681
RAC: 0
Message 17365 - Posted: 30 May 2006, 13:44:49 UTC - in response to Message 17345.  

Could one of the project staff explain why in the first case this Xeon with outrateous benchmarks is producing decoys 8.6 times faster than my system, and in the second case, is producing them at the same speed my system is?


It seems some WUs always report 99 decoys no matter how many models were actually made. I don't know why.
ID: 17365 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Rhiju
Volunteer moderator

Send message
Joined: 8 Jan 06
Posts: 223
Credit: 3,546
RAC: 0
Message 17387 - Posted: 30 May 2006, 18:59:10 UTC - in response to Message 17365.  
Last modified: 30 May 2006, 18:59:36 UTC

Hi: there's a small bug in one of the modes (the workunits with "JUMP" in the title) where it always outputs "99 decoys" in the stderr output. Sorry about that; I also just noticed it! The actual number of decoys created is less, and credits are tied to the number of floating point ops not decoys. So this is not a huge problem (that's why we didn't notice it before), and definitely one we can fix!


Could one of the project staff explain why in the first case this Xeon with outrateous benchmarks is producing decoys 8.6 times faster than my system, and in the second case, is producing them at the same speed my system is?


It seems some WUs always report 99 decoys no matter how many models were actually made. I don't know why.


ID: 17387 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
doc :)

Send message
Joined: 4 Oct 05
Posts: 47
Credit: 1,106,102
RAC: 0
Message 17389 - Posted: 30 May 2006, 19:37:32 UTC

the whole boinc benchmark based credit system is a problem in itself :)
ID: 17389 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Robert Everly

Send message
Joined: 8 Oct 05
Posts: 27
Credit: 665,094
RAC: 0
Message 17391 - Posted: 30 May 2006, 20:03:55 UTC

What about those high credit claims?
ID: 17391 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Aglarond

Send message
Joined: 29 Jan 06
Posts: 26
Credit: 446,212
RAC: 0
Message 17401 - Posted: 30 May 2006, 22:47:27 UTC
Last modified: 30 May 2006, 23:36:26 UTC

I was using Boincstudio with optimization on Rosetta for a few days on 6 different computers and there was one common thing. All computers had FLOPS+IOPS between 9500 - 12500. And all of them were claiming around 50-70 credits for 10000s WU. Actually P3 500MHz 160MB RAM was claiming very similar credits to A64 3200+ 1GB RAM.

As I'm looking at credit claims of other people, those who has high claims has often FLOPS and IOPS like 3693 and 6249 (together 9942) and are claiming around 60 credits per WU. It looks pretty familiar. Now I think this is the way how to recognize BoincStudio clients with optimization turned on.

+++ Update: +++
I was looking at top 10 members of several random teams. My small statistic is pretty simple: every second user is cheating with credit claims! (I was not counting users with hidden computers. Together I was counting around 30-40 users. I treated user as cheating, when at least one of his computers has benchmarks more than 2 times higher as what I would expect for that particular processor type.)
ID: 17401 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
kevint

Send message
Joined: 8 Oct 05
Posts: 84
Credit: 2,530,451
RAC: 0
Message 17409 - Posted: 31 May 2006, 4:49:35 UTC - in response to Message 17401.  
Last modified: 31 May 2006, 5:06:12 UTC

I was using Boincstudio with optimization on Rosetta for a few days on 6 different computers and there was one common thing. All computers had FLOPS+IOPS between 9500 - 12500. And all of them were claiming around 50-70 credits for 10000s WU. Actually P3 500MHz 160MB RAM was claiming very similar credits to A64 3200+ 1GB RAM.

As I'm looking at credit claims of other people, those who has high claims has often FLOPS and IOPS like 3693 and 6249 (together 9942) and are claiming around 60 credits per WU. It looks pretty familiar. Now I think this is the way how to recognize BoincStudio clients with optimization turned on.

+++ Update: +++
I was looking at top 10 members of several random teams. My small statistic is pretty simple: every second user is cheating with credit claims! (I was not counting users with hidden computers. Together I was counting around 30-40 users. I treated user as cheating, when at least one of his computers has benchmarks more than 2 times higher as what I would expect for that particular processor type.)


Did you find these users in the top teams or was it random?

I have something like 28 active computers on my account, about 1/2 are dedicated to Rosetta, and I am just breaking 15K a day.


SETI.USA


ID: 17409 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
XS_The_Machine

Send message
Joined: 2 Jan 06
Posts: 47
Credit: 4,612,053
RAC: 0
Message 17433 - Posted: 31 May 2006, 15:13:15 UTC - in response to Message 17401.  

I was using Boincstudio with optimization on Rosetta for a few days on 6 different computers and there was one common thing. All computers had FLOPS+IOPS between 9500 - 12500. And all of them were claiming around 50-70 credits for 10000s WU. Actually P3 500MHz 160MB RAM was claiming very similar credits to A64 3200+ 1GB RAM.

As I'm looking at credit claims of other people, those who has high claims has often FLOPS and IOPS like 3693 and 6249 (together 9942) and are claiming around 60 credits per WU. It looks pretty familiar. Now I think this is the way how to recognize BoincStudio clients with optimization turned on.

+++ Update: +++
I was looking at top 10 members of several random teams. My small statistic is pretty simple: every second user is cheating with credit claims! (I was not counting users with hidden computers. Together I was counting around 30-40 users. I treated user as cheating, when at least one of his computers has benchmarks more than 2 times higher as what I would expect for that particular processor type.)



If you could, Please cite a few examples. Thank You
ID: 17433 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Moderator9
Volunteer moderator

Send message
Joined: 22 Jan 06
Posts: 1014
Credit: 0
RAC: 0
Message 17457 - Posted: 31 May 2006, 20:09:48 UTC - in response to Message 17345.  

https://boinc.bakerlab.org/rosetta/result.php?resultid=21696168
https://boinc.bakerlab.org/rosetta/result.php?resultid=21548945
It's producing the same number of decoys for this WU in 10,000 seconds that my 2Ghz Athlon 64 3000+ machine produced in 86,000. It's getting twice the number of credits that my default client is getting for the same amount of work. Very strange..

https://boinc.bakerlab.org/rosetta/result.php?resultid=21795438
https://boinc.bakerlab.org/rosetta/result.php?resultid=21786691
and here's a set where the speeddemon created 12 in 10,000 seonds, and my machine produced 97 in 86,000.

Could one of the project staff explain why in the first case this Xeon with outrateous benchmarks is producing decoys 8.6 times faster than my system, and in the second case, is producing them at the same speed my system is?


Different WU types will produce a different number of models per unit time irrespective of the system used.
Moderator9
ROSETTA@home FAQ
Moderator Contact
ID: 17457 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FluffyChicken
Avatar

Send message
Joined: 1 Nov 05
Posts: 1260
Credit: 369,635
RAC: 0
Message 17460 - Posted: 31 May 2006, 20:22:48 UTC - in response to Message 17409.  

I was using Boincstudio with optimization on Rosetta for a few days on 6 different computers and there was one common thing. All computers had FLOPS+IOPS between 9500 - 12500. And all of them were claiming around 50-70 credits for 10000s WU. Actually P3 500MHz 160MB RAM was claiming very similar credits to A64 3200+ 1GB RAM.

As I'm looking at credit claims of other people, those who has high claims has often FLOPS and IOPS like 3693 and 6249 (together 9942) and are claiming around 60 credits per WU. It looks pretty familiar. Now I think this is the way how to recognize BoincStudio clients with optimization turned on.

+++ Update: +++
I was looking at top 10 members of several random teams. My small statistic is pretty simple: every second user is cheating with credit claims! (I was not counting users with hidden computers. Together I was counting around 30-40 users. I treated user as cheating, when at least one of his computers has benchmarks more than 2 times higher as what I would expect for that particular processor type.)


Did you find these users in the top teams or was it random?

I have something like 28 active computers on my account, about 1/2 are dedicated to Rosetta, and I am just breaking 15K a day.



Of course with your hidden computer it hard to see what you have and are running ;-)

But are you using the standard BOINC 5.4.9
or with Truxoft, Crunch3r, BoincStudio ?

Team mauisun.org
ID: 17460 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 112,159,649
RAC: 26,366
Message 17463 - Posted: 31 May 2006, 20:31:11 UTC

The computer listed in the first post is rated at over 16x the floating benchmark of my 2.3GHz Athlon XP! (59,996 vs 1,662) - I'd be suprised if it were 1.5x in reality.

I'd like to point out that some of us have a good (although not entirely legitimate!) reason for hiding our computers - I ran BOINC on a friend's work computer for a brief period a while ago and it has the company name as part of the domain name, which is displayed. If I were able to rename the display name of the computers then I'd quite happily unhide them again.
ID: 17463 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 17464 - Posted: 31 May 2006, 20:47:23 UTC - in response to Message 17463.  

...it has the company name as part of the domain name, which is displayed.

Actually...it DOESN'T. You can see your OWN computer name etc... but others cannot. Check the chicken's machines. THAT is all others can see about yours. The owner of the machines gets more detail.

Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 17464 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
BennyRop

Send message
Joined: 17 Dec 05
Posts: 555
Credit: 140,800
RAC: 0
Message 17474 - Posted: 31 May 2006, 21:48:37 UTC - in response to Message 17457.  

https://boinc.bakerlab.org/rosetta/result.php?resultid=21696168
https://boinc.bakerlab.org/rosetta/result.php?resultid=21548945
It's producing the same number of decoys for this WU in 10,000 seconds that my 2Ghz Athlon 64 3000+ machine produced in 86,000. It's getting twice the number of credits that my default client is getting for the same amount of work. Very strange..

https://boinc.bakerlab.org/rosetta/result.php?resultid=21795438
https://boinc.bakerlab.org/rosetta/result.php?resultid=21786691
and here's a set where the speeddemon created 12 in 10,000 seonds, and my machine produced 97 in 86,000.

Could one of the project staff explain why in the first case this Xeon with outrateous benchmarks is producing decoys 8.6 times faster than my system, and in the second case, is producing them at the same speed my system is?


Different WU types will produce a different number of models per unit time irrespective of the system used.

I believe I was comparing WUs that were as close as possible. Rhiu confirmed the first set of links suffered from a buggy result listing 99 decoys regardless of how many were created. (As AMD_is_logical had pointed out.)

The second set were T0283_CONTACTS_MAP_FROM_hom006_535 - which I get the impression that you expect that two identically configured (hardware and software) machines running T0283_CONTACTS_MAP_FROM_hom006_535 WUs for 86,400 seconds will produce wildly different numbers of decoys; while I'd expect two machines like mine to get near 100 decoys and the difference between the two machines would be plus or minus 1 decoy.

If the 500 node blade server that has been shown was used to test out 24 hour WUs, then what was the largest difference in decoys for similar WUs.. such as T0283_CONTACTS_MAP_FROM_hom006_535?
ID: 17474 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 112,159,649
RAC: 26,366
Message 17479 - Posted: 31 May 2006, 22:15:40 UTC - in response to Message 17464.  
Last modified: 31 May 2006, 22:46:41 UTC

...it has the company name as part of the domain name, which is displayed.

Actually...it DOESN'T. You can see your OWN computer name etc... but others cannot. Check the chicken's machines. THAT is all others can see about yours. The owner of the machines gets more detail.


well that clears that up then! they're on display again ;) cheers!

[edit] well i've set them to be on display but it's not filtered through yet. I'll check again tomorrow.
ID: 17479 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Winkle

Send message
Joined: 22 May 06
Posts: 88
Credit: 1,354,930
RAC: 0
Message 17498 - Posted: 1 Jun 2006, 6:16:58 UTC

From what I can gather, the benchmarks are contained in the client_state.xml file.
Perhaps these are being edited to give unrealistis results. I haven't tried it, nor do I really want to screw around with it. Can anyone comment on this ?
ID: 17498 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Feet1st
Avatar

Send message
Joined: 30 Dec 05
Posts: 1755
Credit: 4,690,520
RAC: 0
Message 17508 - Posted: 1 Jun 2006, 13:08:10 UTC - in response to Message 17498.  

From what I can gather, the benchmarks are contained in the client_state.xml file.
Perhaps these are being edited to give unrealistis results. I haven't tried it, nor do I really want to screw around with it. Can anyone comment on this ?

I don't care to mess with it either, but yes, it's no harder than modifying SOME XML file in SOME way after your BOINC benchmark runs. Frankly I'm surprised it took so long for this issue to come up, because I'm very confident that it IS just that simple to falsify your credit claims.

Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/
ID: 17508 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bob Guy

Send message
Joined: 7 Oct 05
Posts: 39
Credit: 24,895
RAC: 0
Message 17522 - Posted: 1 Jun 2006, 21:34:04 UTC - in response to Message 17508.  
Last modified: 1 Jun 2006, 21:34:34 UTC

it's no harder than modifying SOME XML file in SOME way after your BOINC benchmark runs.

This has gone on since the start of Boinc. The DC project developers have allowed this to continue because it encourages users to crunch the WUs. After all, the credits cost the DC project nothing. I believe the ignoring of credit exploitation is a compromise of ethics and is bad behaviour on the part of the DC project developers.

There are several easy ways to exploit the benchmark system. One is to have a 'script' running that examines the XML files at intervals and replaces the values contained therein. I can easily, in less than an hour, write a C++ or Delphi program that runs in the background which examines and modifies the XML files. It is very easy to just recompile the Boinc source and put any value you like in the result - benchmarking would even never need to be run. The standard benchmark has a number of constant multipliers, you would only need to find them in the binary and hack the binary with any common hex editor.

I'm sure I'm not revealing any secrets here - any competent programmer/hacker would already have figured this out long ago. I'm also sure that the Boinc developers knew all this when the system was being developed, and if they didn't then their program design process was poor by my standards. At any rate the Boinc developers had to have realized, at a very early date, that the benchmark system was fundamentally flawed and made a 'deal with the devil' in the interest of getting their WUs crunched.

If I sound like a disgruntled user, you're right.
ID: 17522 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 17523 - Posted: 1 Jun 2006, 22:00:07 UTC

I guess all of us have read in DB's journal that Rosetta is going to implement a fpops credit system (a la SETI-Enhanced, coded by Rom more). Let's watch and see how fpops is going to work.

Obviously even under the new fpops scenario, one could still hack the rosetta executable binary, to inflate credit claims.

I guess another way would be to assign a fixed number of credits per model, depending on type of WU, based on a benchmark machine (like F@H does). But even then, one could send back bogus results data (unless results are not encoded to prevent tampering).

So to summarize, the most bullet-proof method IMO is to assign a fixed # of credits per model (per type of WU) and encode the results. This is how F@H does it AFAIK.

The current BOINC-way "solution" is to operate with a 4-5 times redundancy, effectively slashing computing power to 1/5th of the raw donated CPU power, a huge waste of resources.

Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 17523 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 17525 - Posted: 1 Jun 2006, 23:56:51 UTC - in response to Message 17522.  

[quoteThis has gone on since the start of Boinc. The DC project developers have allowed this to continue because it encourages users to crunch the WUs. After all, the credits cost the DC project nothing. I believe the ignoring of credit exploitation is a compromise of ethics and is bad behaviour on the part of the DC project developers.

There are several easy ways to exploit the benchmark system. One is to have a 'script' running that examines the XML files at intervals and replaces the values contained therein. I can easily, in less than an hour, write a C++ or Delphi program that runs in the background which examines and modifies the XML files. It is very easy to just recompile the Boinc source and put any value you like in the result - benchmarking would even never need to be run. The standard benchmark has a number of constant multipliers, you would only need to find them in the binary and hack the binary with any common hex editor.

I'm sure I'm not revealing any secrets here - any competent programmer/hacker would already have figured this out long ago. I'm also sure that the Boinc developers knew all this when the system was being developed, and if they didn't then their program design process was poor by my standards. At any rate the Boinc developers had to have realized, at a very early date, that the benchmark system was fundamentally flawed and made a 'deal with the devil' in the interest of getting their WUs crunched.

If I sound like a disgruntled user, you're right.[/quote]

Don't forget, one of the primary reasons given for leaving SETI Classic and going to BOINC was to eliminate the credit cheating. HA!

Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 17525 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Aglarond

Send message
Joined: 29 Jan 06
Posts: 26
Credit: 446,212
RAC: 0
Message 17582 - Posted: 3 Jun 2006, 19:33:15 UTC - in response to Message 17522.  
Last modified: 3 Jun 2006, 19:40:09 UTC

it's no harder than modifying SOME XML file in SOME way after your BOINC benchmark runs.

This has gone on since the start of Boinc. The DC project developers have allowed this to continue because it encourages users to crunch the WUs. After all, the credits cost the DC project nothing. I believe the ignoring of credit exploitation is a compromise of ethics and is bad behaviour on the part of the DC project developers.


Yap, it is too easy. But only on Rosetta, in other projects you need to meet another cheater on the same WU. Actually I tried it on one of my computers. I moved decimal point just by one number from 1313.28 to 13132.8 and from 2730.351 to 27303.51 . From that moment on, it got 10 times more credit for every WU. It is too easy to cheat.

Dr. Baker is encouraging teams to compete, to be the team that did most work done during CASP7. But what is this all about, when it is so easy to cheat?

And I believe that there are some people cheating in all major teams. Think about this: BoincStudio comes with Boinc version 5.5.0 . This version can give you more credits, but you have to turn credit correction on. It makes perfect sense to use it on Einstein, Seti or Sztaki, when someone uses optimized applications. However why would someone use it, if he is crunching 24 hours a day for rosetta?

If you want some evidence for cheating, look at members of teams that earns most credits. Look at their computers, and pick some with big RAC. Look at any crunched result and if you see boinc version 5.5.0 he is probably cheating. And if he crunches 24 hours a day, he is very likely to cheat. You can count it this way: Pick one computer, count sum of all results that this computer did yesterday. Divide it by the number of CPUs this computer has. If it is over 60000s, this computer is obviously crunching Rosetta 24 hours a day.

When I was writing this message, this computer was perfect example of what I'm talking about. It had FLOPS=4542 and IOPS=14766. When I looked at some random result, it was using boinc 5.5.0 and sum of CPU times of crunched units on 2.6.2006 was 141278. Computer has 2 CPUs. That makes over 70000s per day and that means this computer is crunching only for Rosetta.
To owner of this computer: Please don't be angry at me, I just had to pick someone as example.
ID: 17582 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 . . . 6 · Next

Message boards : Number crunching : Is this for real???



©2024 University of Washington
https://www.bakerlab.org