Hi, This is the thread to discuss the new credit system. If you want to dicuss the communication or the actions of the project staff please post here.

---

I am a forum moderator! Am I?

Let me first state how I read the tech news. If I got something wrong, please correct me.

The first in gets granted whatever s/he askes, no pending credit.
The second gets average, could be more, could be less.
The third gets again something different.
There will be no recalculation of once granted credit accourding to a common average.
No measure will be taken about the incomparable benchmarks of "optimized" and stock clients.

So it's not "Same work, same credits", at least for the early birds.

Crunchers with "optimized" clients and short runtimes will get most, people with long runtimes and stock clients will get least for the same amount of work done.

Why don't you simply wait a few days/hours until granting the credit, so that all has averaged out 'til the granting starts?

I don't know how long it will take for the first 1000 results or 100K decoys or whatever's needed are back and a reasonable sample is possible. But I would prefer this to immediately granting different values.

So it's not "Same work, same credits", at least for the early birds.

That's how I read it too. And Mod.DE's statement about different BOINC clients not making any difference is incorrect, especially as far as early reporters go. Anyone using a modified client to overclaim will increase the average that's used to grant them credit - and the earlier they report, the greater the impact they'll have on the average.

---

So it's not "Same work, same credits", at least for the early birds.

That's how I read it too. And Mod.DE's statement about different BOINC clients not making any difference is incorrect, especially as far as early reporters go. Anyone using a modified client to overclaim will increase the average that's used to grant them credit - and the earlier they report, the greater the impact they'll have on the average.

If thats true, then please rosetta,give me a "Target CPU run time" choice of 15 minutes for my E6600 system LOL

---

So it's not "Same work, same credits", at least for the early birds.

That's how I read it too. And Mod.DE's statement about different BOINC clients not making any difference is incorrect, especially as far as early reporters go. Anyone using a modified client to overclaim will increase the average that's used to grant them credit - and the earlier they report, the greater the impact they'll have on the average.

Let's call it the "early bird"-problem. You are right, it is theoretically a problem, the question is how big is the practical impact? I think not very big, however I fully support the suggestion to wait until the first 10.000 results / 100.000 models are returned to start already with a very good average. I would even go further and stop changing the credit/model-ratio after a proper ratio has been establised. This would lead to the same credit/model-ratio applied to all participants.

So it's not "Same work, same credits", at least for the early birds.

That's how I read it too. And Mod.DE's statement about different BOINC clients not making any difference is incorrect, especially as far as early reporters go. Anyone using a modified client to overclaim will increase the average that's used to grant them credit - and the earlier they report, the greater the impact they'll have on the average.

If thats true, then please rosetta,give me a "Target CPU run time" choice of 15 minutes for my E6600 system LOL

Even that won't work, since you can't determine which kind of WU you'll receive (fresh or old ones). Perhaps it would be better to remove the 1hour target time though.

So it's not "Same work, same credits", at least for the early birds.

That's how I read it too. And Mod.DE's statement about different BOINC clients not making any difference is incorrect, especially as far as early reporters go. Anyone using a modified client to overclaim will increase the average that's used to grant them credit - and the earlier they report, the greater the impact they'll have on the average.

If thats true, then please rosetta,give me a "Target CPU run time" choice of 15 minutes for my E6600 system LOL

Even that won't work, since you can't determine which kind of WU you'll receive (fresh or old ones). Perhaps it would be better to remove the 1hour target time though.

Not really as the 1hr target is esencially a 'do as close to one decoy as possible. It does mean I can set my part timers to 1hr (if they're on a decent connection) and there a good chance it'll get done before the report due time.

Now the prefs are fixed I've switched to 1hr, mainly as I'm so used to 24hrs I'm double checking what I said about bandwidth usage and how it behaves.

---

Team mauisun.org

Those with optimized boinc core clients could do the following:

Set "connect to" very low, like .001 days. This will ensure reporting takes place quickly, roughly every minute 25 seconds if needed.

Set Runtime to lowest possible, One hour.

Then from time to time when a new wu is released they have the opportunity to overclaim successfully for one/two models of the new wu.


The project could run the new WUs on known systems until they get back 1/2 a dozen to alleviate this, or instead of using average, they could use the mean.

...I fully support the suggestion to wait until the first 10.000 results / 100.000 models are returned to start already with a very good average. I would even go further and stop changing the credit/model-ratio after a proper ratio has been establised. This would lead to the same credit/model-ratio applied to all participants.

Hope they have a good look at this suggestion.

The fact that all get the same credit/model sounds good to me.

Anders n

---

FluffyChicken:
Each WU download was around 2.5 megabytes and grows bigger for the WUs with larger proteins. The uploaded data from was smaller; perhaps as large as 500k for running a WU 24 hours. By using 24 hour run times instead of 1 hour run times, you'll save at least 23*2.5 MB, and probably save a bit on the result files as well. (The point of the option for long run times was to save bandwidth.)

---

First impressions of the new credit system.

It does nothing to address the use of optimised clients, the use of optimised clients will in fact be encouraged. The individual user may not get full benefit of their claim, but enough users reporting optimised results will heavily skew the granted credits. This promotes a "safety in numbers" type attitude - meaning it's harder to spot the overclaiming hosts.

By this I mean there is nothing to stop someone modifying their xml files such that they have benchmarks 1000 times greater than would actually be calculated by the standard client. How would that affect their results, if they happened to report first they would be granted their astronomical claimed credit. If they didn't report first, then they wouldn't immediately get any advantage from their claimed credit, however, that claimed credit would increase the total_claimed and have an impact on subsequent reporters by increasing the average. Every subsequent granted credit would be radically greater than it should be (what you may miss out on in your first returned result you would be granted in your second and subsequent results). It would only take a relatively small number of these rogue hosts to seriously skew the granted credits (upwards of course).

Now we get to the really insidious part. It's much harder to track down these rogue hosts, why? because of the obfuscation inherent in the system of averaging the claimed credits - all of a sudden everyone is granted more - but you can't track the culprits (unless Rosetta is prepared to, and nulify the falsified claims).

What does this mean - rampant inflation of credits.

I hope I am wrong, but on first reading of this credit system, it will actually encourage many people to claim whatever they want with impunity. They will not necessarily initially get the benefit of what they claim but they will get the benefit of every previous overclaimer.

If this occurs then the credit system will not actually reflect work done, it will reflect the ever increasing falsification of benchmarks.

Why will this happen - because no one has said that it shouldn't (therefore it must be allowed).

---

Hi TrogDog,

David Kim mentioned in a deleted thread, that he is able to spot grossly overclaiming hosts, flag them, exclude them from the credit calculation and award them a low 2 credits/model ratio. That would at least solve the problem of manually edited XML-files.

For optimized clients which overclaim modest (within 500% of the standard client) your conclusions are right theoretically but the question is how it will affect reality in practice. I think Rosetta is big enough that overclaiming hosts are in the minority and don't really affect the granted credit much, if compared to the whole user base. The incentive to use the optimized client is also minimized, while it will push up the granted credit for all gradually it won't give you an edge over the competition. But your reasoning is yet another argument to wait with establishing the credit/model ratio until many different hosts have returned results.

FluffyChicken:
Each WU download was around 2.5 megabytes and grows bigger for the WUs with larger proteins. The uploaded data from was smaller; perhaps as large as 500k for running a WU 24 hours. By using 24 hour run times instead of 1 hour run times, you'll save at least 23*2.5 MB, and probably save a bit on the result files as well. (The point of the option for long run times was to save bandwidth.)

I know how big they are, I've been here through all the file size problems (As it effected me a lot).

But if you have a look next time you download the smaller times you should find (or at least I have) that it get one large 'query/target' file (the .gz's) , then 4 or 5 smaller 'task' files aimed at that target. I do not know the cut of for when it decides to get another target. I know we suggested this while descussing the issues of bandwidth usage in days gone by. Though I didn't notice it at first (since I've been using 24hrs for ages) until I installed at a few family remote who where on broadband and I set it to short run times.

---

Team mauisun.org

...I fully support the suggestion to wait until the first 10.000 results / 100.000 models are returned to start already with a very good average. I would even go further and stop changing the credit/model-ratio after a proper ratio has been establised. This would lead to the same credit/model-ratio applied to all participants.

Hope they have a good look at this suggestion.

The fact that all get the same credit/model sounds good to me.

Anders n

I just received "Pending Credit" for a WU just reported. Hmmm, perhaps the suggestion already has been implemented to wait for a stable average.

I just looked at the results of my systems at WU's from 23 aug and later.
To me it looks like the credits are granted in random order, that the cpu-time spend on a WU doesn't matter anymore.
I'll give it another day or two, but if it still not make any sense to me, I switch.

(And I really supported the idea of a fair credit system, but so far this isn't)

---

First impressions of the new credit system.

It does nothing to address the use of optimised clients, the use of optimised clients will in fact be encouraged. The individual user may not get full benefit of their claim, but enough users reporting optimised results will heavily skew the granted credits. This promotes a "safety in numbers" type attitude - meaning it's harder to spot the overclaiming hosts.

By this I mean there is nothing to stop someone modifying their xml files such that they have benchmarks 1000 times greater than would actually be calculated by the standard client. How would that affect their results, if they happened to report first they would be granted their astronomical claimed credit. If they didn't report first, then they wouldn't immediately get any advantage from their claimed credit, however, that claimed credit would increase the total_claimed and have an impact on subsequent reporters by increasing the average. Every subsequent granted credit would be radically greater than it should be (what you may miss out on in your first returned result you would be granted in your second and subsequent results). It would only take a relatively small number of these rogue hosts to seriously skew the granted credits (upwards of course).

Now we get to the really insidious part. It's much harder to track down these rogue hosts, why? because of the obfuscation inherent in the system of averaging the claimed credits - all of a sudden everyone is granted more - but you can't track the culprits (unless Rosetta is prepared to, and nulify the falsified claims).

What does this mean - rampant inflation of credits.

I hope I am wrong, but on first reading of this credit system, it will actually encourage many people to claim whatever they want with impunity. They will not necessarily initially get the benefit of what they claim but they will get the benefit of every previous overclaimer.

If this occurs then the credit system will not actually reflect work done, it will reflect the ever increasing falsification of benchmarks.

Why will this happen - because no one has said that it shouldn't (therefore it must be allowed).

Looking at your post, there might be a very easy solution : people get the running average, BEFORE their claim is taken into account. This would make it useless to overclaim, in fact it would NOT give you more credits, but it WOULD give more credits to those that report WU's after yours. How's that for discouraging optimized clients? You would only benefit from overclaiming if you were actually the first to report a certain WU (what are the odds?). In all other cases, overclaiming would give more credits to everyone behind you - but not yourself, thus you would not get higher in the ranking - the opposite would be true.


Example : first guy reports, claims and gets 100 credits. You report, claiming 300, getting 100, setting the running average to 200. Third guy reports, claims 100, and gets 200, running average goes down to 166, etc....

---

I just looked at the results of my systems at WU's from 23 aug and later.
To me it looks like the credits are granted in random order, that the cpu-time spend on a WU doesn't matter anymore.
I'll give it another day or two, but if it still not make any sense to me, I switch.

You have a target CPU time from 1 hour. That will always result in greater variations, since for bigger proteins you are just doing one model per WU. Model runtimes vary and so will your credit for such short WU. If you want smoother credit granting switch to 4 hours or more and give it a week or two instead of a day or two. ;-)

Trog Dog, for someone to do anything, there has to be a motivation to do so. If someone hasn't already switched to using an optimized boinc core client, when you actually got whatever you claimed, then I don't think there's much motivation to switch to one now. 90% + of the active attached hosts are using standard boinc clients. Remember only the first few results returned of a new wu could be granted substantially higher claims, even when averaging. I doubt there's much incentive to switch to one now.

Yes, this might be a loop hole, but it's a SMALL one that might be exploited. Should it be addressed/considered? Yes I don't think it's much to get worried over though.

In Ralph, we discussed "cherry picking", however, from my own results I see no way that I could pick and choose which gave more. I think by the time enough results come in that "cherry Picking" might be possible, they'd be issuing a different wu and it wouldn't matter.

...I fully support the suggestion to wait until the first 10.000 results / 100.000 models are returned to start already with a very good average. I would even go further and stop changing the credit/model-ratio after a proper ratio has been establised. This would lead to the same credit/model-ratio applied to all participants.

Hope they have a good look at this suggestion.

The fact that all get the same credit/model sounds good to me.

Anders n

(R.L. Casey 11:42 UTC):
I just received "Pending Credit" for a WU just reported. Hmmm, perhaps the suggestion already has been implemented to wait for a stable average.

I've checked a number of results, and they show that the switch to granting "Pending Credit" happened today between 11:23 UTC and 11:27 UTC.

Hi TrogDog,

David Kim mentioned in a deleted thread, that he is able to spot grossly overclaiming hosts, flag them, exclude them from the credit calculation and award them a low 2 credits/model ratio. That would at least solve the problem of manually edited XML-files.

For optimized clients which overclaim modest (within 500% of the standard client) your conclusions are right theoretically but the question is how it will affect reality in practice. I think Rosetta is big enough that overclaiming hosts are in the minority and don't really affect the granted credit much, if compared to the whole user base. The incentive to use the optimized client is also minimized, while it will push up the granted credit for all gradually it won't give you an edge over the competition. But your reasoning is yet another argument to wait with establishing the credit/model ratio until many different hosts have returned results.

G'day Tralala

I hope I'm proved to be overly cynical, however, David Kim also said that it was a relatively trivial matter to backdate credits and he was prepared to do it - we all know where that ended up. If we have an official response from the project that credit escalation will not be tolerated and will be actively sought out then it will do much to address this problem.

I also think that the motivation to use an optimised client is increased, not only will it increase your subsequent results (how many times do you only crunch one wu from each type?) but also those of your teammates.

As I said I hope that I'm being overly cynical and pessimistic.

---