Posts by Trog Dog

41) Message boards : Number crunching : How to fake out the new credit system (Message 25092)
Posted 27 Aug 2006 by Profile Trog Dog
Post:
To my knowledge there is presently no screening. Since we DO have a new credit system, I'll not comment on your other question. Perhaps you could rephrase it if there are questions there. Under both credit systems, the "optimized" client's credit claims are accepted as within any such limits.


OK then, with no screening what is to stop someone writing a new optimised client that claims even more than the current batch do, or writing a script that increments a hosts benchmarks?


Are there any hosts that run a standard client and unmodified xml benchmarks that are getting less granted than claimed?
42) Message boards : Number crunching : How to fake out the new credit system (Message 25082)
Posted 27 Aug 2006 by Profile Trog Dog
Post:
To my knowledge there is presently no screening. Since we DO have a new credit system, I'll not comment on your other question. Perhaps you could rephrase it if there are questions there. Under both credit systems, the "optimized" client's credit claims are accepted as within any such limits.


OK then, with no screening what is to stop someone writing a new optimised client that claims even more than the current batch do, or writing a script that increments a hosts benchmarks?
43) Message boards : Number crunching : How to fake out the new credit system (Message 25079)
Posted 27 Aug 2006 by Profile Trog Dog
Post:

Keep in mind that the hypothetical 10x claim is pretty easily screened and omitted from the averages as well.


Is that kind of screening being carried out? If it is, why not just narrow the acceptable range so that it catches the optimised clients and be done with the rest?
44) Message boards : Number crunching : Old/New Credit system comparisons (Message 25077)
Posted 27 Aug 2006 by Profile Trog Dog
Post:

You will need to manually rebench between the switch (H.T. to non- H.T.) to make a valid conclusion.


Is it enough to just set use max cpu to 1 or do I have to completely disable HT in the bios?

Disable in bios.


Cheers
45) Message boards : Number crunching : Old/New Credit system comparisons (Message 25076)
Posted 27 Aug 2006 by Profile Trog Dog
Post:
I realize it isn't simple; but I wanted enough data points to show the effect of the new credit system on P4s.

Graphing with enough data points shows that the credit system seems to be working - as I was showing here: link


That should help, Trog Dog.. Thanks.


OK, I've set all other projects to no new work, its part way crunching a long Einstein and it's got a couple of SIMAP wu's to crunch. I'll post once it's cleared it's cache. BTW here's the box
46) Message boards : Number crunching : Old/New Credit system comparisons (Message 25070)
Posted 27 Aug 2006 by Profile Trog Dog
Post:

You will need to manually rebench between the switch (H.T. to non- H.T.) to make a valid conclusion.


Is it enough to just set use max cpu to 1 or do I have to completely disable HT in the bios?
47) Message boards : Cafe Rosetta : Word link 3 (Message 25065)
Posted 27 Aug 2006 by Profile Trog Dog
Post:
Darwin
48) Message boards : Number crunching : Old/New Credit system comparisons (Message 25064)
Posted 27 Aug 2006 by Profile Trog Dog
Post:
Does someone have a viewable system that they run 24/7 with a 512 Meg P4 running at 3Ghz with HT off? One with HT On? (With 2 or 3 hour WU run times selected).
So we can see the difference between old and new systems with Windows using a P4 with HT off, Windows using a P4 with HT on, Linux using a P4 with HT off, Linux using a P4 with HT on?
All systems preferably dedicated crunchers, not being left running Zoo Tycoon all night until the animals all croak.

What are suggestions for typical Macs? (High end, medium, low end)?




I've got a P4 that meets those specs with HT on. It's a bog standard Dell Dimension 4600 running win XP. I'll give you two days 100% Rosetta.

49) Message boards : Number crunching : How to fake out the new credit system (Message 25061)
Posted 27 Aug 2006 by Profile Trog Dog
Post:
Here's an example of why it's difficult to cheat the system:

After 10,000 models the average credit is 10 per model. Enter Mr C who changed his client to claim 10 times as much as he earned. He submitts 10 models and wants 1,000 instead of the 100 credits.

1,000 + 100,000 / 10,010 = ~10,09

By overclaiming 10 times he increased the average per model from 10 to 10.09 at a very early stage of the run.

The reason the average credit doesn't go down from the change of the system is that to the project it doesn't matter if one user claims 20 credits and the other 60, or both of them claim 40 credits. The total is still 80.


And even though it should be obvious: Faster systems still get more credit. They get the same amount of credit per model, but by virtue of being faster they do more models per hour - hence more credits.




Agreed, but if Mr C get's in at some point before the 10,000 result mark then he has a bigger impact, particularly if joined by his mates Mr D, E & F. Now if these 4 mates also happen to have top of the range boxes, and a couple each then it has an even bigger impact.

One user by themselves won't be able to make a difference, but teams of them acting in concert could.
50) Message boards : Cafe Rosetta : Word link 3 (Message 25053)
Posted 27 Aug 2006 by Profile Trog Dog
Post:
source
51) Message boards : Number crunching : How to fake out the new credit system (Message 25052)
Posted 27 Aug 2006 by Profile Trog Dog
Post:
The current system can still be manipulated. Nobody at a project level has said that the optimised clients can't be used.

Over on the RALPH boards somebody posted a snippet from a post by one of the project scientists where he states the biggest wu run he had was 1.5 million. Now assume there are 150,000 hosts (according to the front page there are 173,441 - but lets keep the numbers round). Assume that everybodys computer is the same and connects at the same frequency - that means you will each have 1000 wu's to process. Thats 1000 chances you get to influence the credit claimed.

Of course not everybodys computer is the same, if you can process twice as many as the next guy then you will have 667 chances to affect the credits against his 333. Yes you will increase his credits too, but as you're completing wu's at a greater rate than he is you will increase the difference between your scores at an exponential rate.

If everytime you report, you increase your benchmarks the effect is even greater. Can't be done? QMC used to have a maximum limit of 1000 credits per wu (it's since been increased to 2000)there's already been one reported incident of a host returning wu's reporting different benchmarks and times taken such that each credit claim was just under the 1000 credit limit.

52) Message boards : Cafe Rosetta : Word link 3 (Message 24987)
Posted 26 Aug 2006 by Profile Trog Dog
Post:
Alcoholism
53) Message boards : Cafe Rosetta : Word link 3 (Message 24911)
Posted 26 Aug 2006 by Profile Trog Dog
Post:
sunrise
54) Message boards : Number crunching : Points/Credits on DC in General (Message 24733)
Posted 24 Aug 2006 by Profile Trog Dog
Post:
I`m at WCG which like most operates a quorum (how many I know not). My points are all over the place, here maybe a reason.
If WCG hasn't changed latetely, they use Boinc only to distribute work. Their credits are calculaded by points their own SW would have given times a fixed factor.

Norbert
PS: Their factor used to be a bit to the high side (a share of 75 for WGC and 100 for other projects gave equal RAC for me) but seems to be adjusted lately.


WCG grant standard BOINC credits, however, since they initially started with the United Devices client they had a different credit system already in place. What they did so as not to disadvantage BOINC users is apply a multiple to BOINC credits so that they roughly equate to what would have been earned if the BOINC user had been using the UD client, but this is not exported - it's strictly an in-house measure of credits. Only BOINC credits are exported to BOINC stats sites.
55) Message boards : Number crunching : Discussion of the new credit system (Message 24705)
Posted 24 Aug 2006 by Profile Trog Dog
Post:



I will not prove you wrong, or try to counter your devious ideas. The suggestion I made just takes one extra reason OUT of overclaiming. It doesn't prevent every single cheat you can come up with.

In the current system, overclaiming gives you extra credit, and MIGHT make a difference on your next WU or your teammate's. But YOU are the main beneficiary.

In my suggestion, the first reason is gone. What remains is you MIGHT get more credits on your next WU or your teammate's.

So, I still think that it could be implemented, and probably very easy :

Cuurent system : Calculate a new rolling average, then assign average points

My idea : assign averige points, then calculate new rolling average.


Thanks for your ideas Sloom. We'll soon see what happens ;) I hope that I am proved wrong.
56) Message boards : Number crunching : Discussion of the new credit system (Message 24701)
Posted 24 Aug 2006 by Profile Trog Dog
Post:



Maybe, maybe not. I would have to do some longterm calculations on this, but point is that you will probably not gain much yourself. By the time you get the next WU from the same batch in, the effect of your previous claim will have diminished. Especially if the WU has had a lot of results reported already. Chances that some of your teammates gain anything are slim, too. And even if you do gain anything, fact is that other members/teams will have a higher chance of benefitting from you, unless you're in a team that is over half the active participants. So, you might be winning in actual points, but other teams will be winning more. So you're actually helping your opponents more than you help yourself.

Having said that, there's a loophole in that when someone reports their work in large batches (like 10 WU's). In that case, 9 out of the 10 would benefit (assuming that no ther WU's are sent by other members inbetween yours). Still, much more than 9 following your contribution would benefit....

Besides, your reasoning goes for the current system as well - and more so, because you gain immediately no matter what the effect on your next or teammate's WU.


G'day again LosAlcoholicos~Sloom

"Maybe, maybe not" - that uncertaintity would be enough for some individuals and/or teams to do what I'm hypothesising. That afterall, has been amongst the reasons given over the last weeks/months as to why optimised clients are used - "everybody else is using them" "team ?? are using them so why shouldn't we" "os ?? users are using them so why shouldn't we" "my cpu is disadvantaged compared to your cpu, so I'm using them".

"by the time you get your next wu from the same batch in" - what about mutiple hosts - same user multiple hosts? Never see the same wu type across multiple hosts?

For every cynical & devious idea I'm thinking up, I bet that there are ten more out there in the wild. Prove me wrong - and I say that without cynicism, because I hope that I am wrong. Only time will tell.
57) Message boards : Number crunching : Discussion of the new credit system (Message 24697)
Posted 24 Aug 2006 by Profile Trog Dog
Post:
http://boinc.bakerlab.org/rosetta/forum_thread.php?id=2197#24664

I'm making a point of this because I think it should be considered as a possibility.

I don't like tilting at windmills, but for the integrity of this project the cynical viewpoint must be considered.
58) Message boards : Number crunching : Discussion of the new credit system (Message 24693)
Posted 24 Aug 2006 by Profile Trog Dog
Post:
Trog Dog, for someone to do anything, there has to be a motivation to do so. If someone hasn't already switched to using an optimized boinc core client, when you actually got whatever you claimed, then I don't think there's much motivation to switch to one now. 90% + of the active attached hosts are using standard boinc clients. Remember only the first few results returned of a new wu could be granted substantially higher claims, even when averaging. I doubt there's much incentive to switch to one now.

Yes, this might be a loop hole, but it's a SMALL one that might be exploited. Should it be addressed/considered? Yes I don't think it's much to get worried over though.

In Ralph, we discussed "cherry picking", however, from my own results I see no way that I could pick and choose which gave more. I think by the time enough results come in that "cherry Picking" might be possible, they'd be issuing a different wu and it wouldn't matter.



G'day mmciastro

There's plenty of psych/legal studies that have been carried out that correlate the propensity to "infringe" to the liklihood of getting caught. Typically increased penalties for speeding won't cause drivers to slow down, but an increased police/highway patrol presence , speed cameras will(ie chances of being caught will). So let's move from the abstract.

Suddenly under this new credit system a user (of an optimised client) can claim "but it wasn't benefiting me" - so less stigma. If everybody is getting more credits, how do you identify the users claiming more - check every wu returned? Only the project can do that.

Something else that just came to mind - intraproject it won't necessarily matter - interproject it will. I want to get my team to number one in BOINC combined statistics - solution have the team attach to Rosetta with bogus benchmarks.

Wait and see - I hope I'm wrong, but I've seen too many arguments over credits and "what MY boxes are ENTITLED to", to view things any differently.
59) Message boards : Number crunching : Discussion of the new credit system (Message 24687)
Posted 24 Aug 2006 by Profile Trog Dog
Post:


Looking at your post, there might be a very easy solution : people get the running average, BEFORE their claim is taken into account. This would make it useless to overclaim, in fact it would NOT give you more credits, but it WOULD give more credits to those that report WU's after yours. How's that for discouraging optimized clients? You would only benefit from overclaiming if you were actually the first to report a certain WU (what are the odds?). In all other cases, overclaiming would give more credits to everyone behind you - but not yourself, thus you would not get higher in the ranking - the opposite would be true.


Example : first guy reports, claims and gets 100 credits. You report, claiming 300, getting 100, setting the running average to 200. Third guy reports, claims 100, and gets 200, running average goes down to 166, etc....



G'day Los Alcohilocos~Sloom

If you overclaim you won't get the benefit on your first result, but you will on every subsequent result from the same type of wu. Sounds like a great incentive to me - not only can I doctor my subsequent credits but also those of all my teammates.

The fact that everybody else potentially benefits is not really a disincentive, if you and your teammates get your results before everyone else.

Potentially this means as a team you all set your runtime to the minimum, and your credit claims to the maximum.
60) Message boards : Number crunching : Discussion of the new credit system (Message 24685)
Posted 24 Aug 2006 by Profile Trog Dog
Post:
Hi TrogDog,

David Kim mentioned in a deleted thread, that he is able to spot grossly overclaiming hosts, flag them, exclude them from the credit calculation and award them a low 2 credits/model ratio. That would at least solve the problem of manually edited XML-files.

For optimized clients which overclaim modest (within 500% of the standard client) your conclusions are right theoretically but the question is how it will affect reality in practice. I think Rosetta is big enough that overclaiming hosts are in the minority and don't really affect the granted credit much, if compared to the whole user base. The incentive to use the optimized client is also minimized, while it will push up the granted credit for all gradually it won't give you an edge over the competition. But your reasoning is yet another argument to wait with establishing the credit/model ratio until many different hosts have returned results.


G'day Tralala

I hope I'm proved to be overly cynical, however, David Kim also said that it was a relatively trivial matter to backdate credits and he was prepared to do it - we all know where that ended up. If we have an official response from the project that credit escalation will not be tolerated and will be actively sought out then it will do much to address this problem.

I also think that the motivation to use an optimised client is increased, not only will it increase your subsequent results (how many times do you only crunch one wu from each type?) but also those of your teammates.

As I said I hope that I'm being overly cynical and pessimistic.


Previous 20 · Next 20



©2024 University of Washington
https://www.bakerlab.org