Credit system seems unfair, anyone else thinks so?

Message boards : Number crunching : Credit system seems unfair, anyone else thinks so?

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile TheRiceKing

Send message
Joined: 23 Apr 07
Posts: 2
Credit: 619,489
RAC: 0
Message 49272 - Posted: 1 Dec 2007, 15:01:20 UTC

Credit system seems unfair, anyone else thinks so?


ID: 49272 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Daniel Kohn

Send message
Joined: 30 Dec 05
Posts: 18
Credit: 2,899,939
RAC: 0
Message 49273 - Posted: 1 Dec 2007, 15:19:58 UTC - in response to Message 49272.  

Credit system seems unfair, anyone else thinks so?


No, I think it's fair enough. As long as I can get a sense of how much work I've completed, that's enough for me. I just wish I could get my company to join up.
ID: 49273 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mikkie

Send message
Joined: 26 Nov 07
Posts: 2
Credit: 2,770
RAC: 0
Message 49274 - Posted: 1 Dec 2007, 15:20:34 UTC - in response to Message 49272.  

Credit system seems unfair, anyone else thinks so?
Explain why!
ID: 49274 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 116,724,892
RAC: 76,333
Message 49279 - Posted: 1 Dec 2007, 16:31:38 UTC

i think it's almost as fair as it can get. There are fluctuations, but they're generally quite minor.
ID: 49279 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [KWSN]John Galt 007
Avatar

Send message
Joined: 4 Aug 06
Posts: 6
Credit: 1,017,647
RAC: 0
Message 49282 - Posted: 1 Dec 2007, 16:57:17 UTC

I think what the original poster is getting at is what I am experiencing:


123448301 112219447 27 Nov 2007 22:23:45 UTC 1 Dec 2007 15:40:33 UTC Over Success Done 86,143.66 251.36 139.76

The difference between cliamed and granted is about 44%. It seems that all of my results end up with a lower granted than claimed. Is this due to the BOINC CC or the Rosetta app?
ID: 49282 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Luuklag

Send message
Joined: 13 Sep 07
Posts: 262
Credit: 4,171
RAC: 0
Message 49290 - Posted: 1 Dec 2007, 20:19:03 UTC - in response to Message 49282.  

I think what the original poster is getting at is what I am experiencing:


123448301 112219447 27 Nov 2007 22:23:45 UTC 1 Dec 2007 15:40:33 UTC Over Success Done 86,143.66 251.36 139.76

The difference between cliamed and granted is about 44%. It seems that all of my results end up with a lower granted than claimed. Is this due to the BOINC CC or the Rosetta app?


it has to do with the comparrison to other machines that finished sort like WU's. the claimed credit is based on the speed of your machine and the time crunched. the granted credit is based on "things" dont have a word for it at this moment, produced, so data the scientists can work with, that explains the differece, if you work for 4 hours but produce crap, they dont think your'e worth much credit.
ID: 49290 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 49296 - Posted: 1 Dec 2007, 21:18:14 UTC

Credit is issued per completed model. Regardless of the beauty or lack thereof in the model produced. The rate of credit granted per model is dynamically adjusted as results from comparable tasks come in.
Rosetta Moderator: Mod.Sense
ID: 49296 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David Emigh
Avatar

Send message
Joined: 13 Mar 06
Posts: 158
Credit: 417,178
RAC: 0
Message 49300 - Posted: 1 Dec 2007, 22:56:09 UTC
Last modified: 1 Dec 2007, 23:03:29 UTC

I want to make it clear at the outset that I personally don't crunch for credit. I know some do, however, and I respect that also.

In the interest of providing data to satisfy curiosity, I looked through my task history and found this one for which I was granted more credit than claimed.
Rosie, Rosie, she's our gal,
If she can't do it, no one shall!
ID: 49300 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Luuklag

Send message
Joined: 13 Sep 07
Posts: 262
Credit: 4,171
RAC: 0
Message 49301 - Posted: 1 Dec 2007, 23:33:10 UTC - in response to Message 49300.  

I want to make it clear at the outset that I personally don't crunch for credit. I know some do, however, and I respect that also.

In the interest of providing data to satisfy curiosity, I looked through my task history and found this one for which I was granted more credit than claimed.



well thats because you made an large number of decoys.. you should try to devide credit givven by the number of decoys and credit claimed by the cpu time runned and something to do with memory and processor speed etc.. that should give you some data to compare better.
ID: 49301 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1829
Credit: 116,724,892
RAC: 76,333
Message 49302 - Posted: 2 Dec 2007, 0:00:12 UTC - in response to Message 49301.  

I want to make it clear at the outset that I personally don't crunch for credit. I know some do, however, and I respect that also.

In the interest of providing data to satisfy curiosity, I looked through my task history and found this one for which I was granted more credit than claimed.



well thats because you made an large number of decoys.. you should try to devide credit givven by the number of decoys and credit claimed by the cpu time runned and something to do with memory and processor speed etc.. that should give you some data to compare better.


It's fairly straight forward:

claimed credit is your benchmark * time
granted credit is the average claim per decoy (model) from all submissions so far for that WU * number of decoys produced.

ID: 49302 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 49304 - Posted: 2 Dec 2007, 0:17:18 UTC

It's been a while since I looked at the formula for credit. Claimed Credit is based upon the official boinc formula which uses the Benchmarks. It is ((whetstone + dhrystone) X CPU seconds)/1,728,000= claimed credit/task. Rosetta divides this by the number of decoys completed. This number is then "claimed credits/decoy". Each time decoys are returned Rosetta adds this value to the previous returned value and divides this by the total number now returned. This is the (current) average credit granted/decoy. It then looks at the task returned, multiplies the number of completed decoys by the "current" claimed credit/decoy and issues this amount which is called "granted credit/task". Since claimed credit is based upon the benchmark, and different computers take diffferent amounts of time to complete, claimed credit/time varies.

I have received questions about "optimized third party boinc core clients", these generally claim as much as 3 times what the stock one does, and since normal variation is generally less than the 3X provided by these clients. I compiled the following chart using a value of 3 as the "average claimed credit/decoy", then then 9 for that of the third party boinc core clients.

Last I knew many WUs(tasks) contained ~100K decoys (with some variations). So what the chart shows is what happens to "granted credit/decoy" with 3 different scenarios. Two with a 10% and one with a 20% penetration rate for third party boinc CC's. (see legend).



You can see the deviation in Granted Credit/decoy after the return of 100, 1000, and 5,000 decoys (of the 100K) available. One can see that within very few (relative) returns, that variations in claims make little difference.
ID: 49304 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 49305 - Posted: 2 Dec 2007, 0:26:26 UTC
Last modified: 2 Dec 2007, 0:27:46 UTC

My AMD64 2800 has been connected to Rosetta on/off for a while. Below is a chart of the periods data was collected, and the "average" granted credit/hour for the listed period. As you can see, there's not been much variation. Note: the first collection period was for work done prior to Rosetta's new credit method being implemented.

ID: 49305 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5664
Credit: 5,711,666
RAC: 764
Message 49306 - Posted: 2 Dec 2007, 0:34:43 UTC - in response to Message 49305.  
Last modified: 2 Dec 2007, 0:35:45 UTC

My AMD64 2800 has been connected to Rosetta on/off for a while. Below is a chart of the periods data was collected, and the "average" granted credit/hour for the listed period. As you can see, there's not been much variation. Note: the first collection period was for work done prior to Rosetta's new credit method being implemented.



never charted my machine, but i think you and i have similar amd cpus, but your stepping is a bit higher than mine. whatever that means.
ID: 49306 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 49307 - Posted: 2 Dec 2007, 0:55:42 UTC

Hey Astro!

Last you knew, there were 100,000+ "STEPS" in a decoy (or model). But a task generally runs 1-100 models in 1-24hrs.

So, I'm not sure what your chart is meant to show, but with a 100,000 decoys premise, it is most definately flawed.
Rosetta Moderator: Mod.Sense
ID: 49307 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 49308 - Posted: 2 Dec 2007, 1:02:28 UTC - in response to Message 49307.  
Last modified: 2 Dec 2007, 1:33:27 UTC

Hey Astro!

Last you knew, there were 100,000+ "STEPS" in a decoy (or model). But a task generally runs 1-100 models in 1-24hrs.

So, I'm not sure what your chart is meant to show, but with a 100,000 decoys premise, it is most definately flawed.

Correct me oh wise one?? I was under that impression that a task such as "gb3_DC_BOINC_MFR_ABRELAX_PICKED" contained 100K models/decoys (models and decoys the same thing), and that each distributed task merely started searching at different point within the wu. So, if it contained 100K models(varying of course), and I did 10 of them, then I did .01% of the wu(task). Correct me and I'll reword/rework it. For example, for this wu :gb3_DC_BOINC_MFR_ABRELAX_PICKED_2342_11052_0 I did 20 decoys. 20 out of ? many?

Hmm, perhaps I'm confusing "model" with "decoy"?? My data show a granting of 3 credits/ "decoy", then the occasional influx of a 9 from third party clients.

the chart was calculated based upon 3 credits/decoy. So, If I did a wu and completed ONE decoy, I'd claim 3 credits. IF I were the first to report a decoy of that task (or whatever the proper name is), then Rosetta would look at my claim of 3, divide it by the number of decoys reported (since mine was first that number would be 1), so 3/1=granted credit of 3/decoy X 1 decoy = granted credit of 3. Then someone else using a stock CC would return on and return ONE decoy, then Rosetta would take my first claim/decoy of 3, add the second persons claim of 3/decoy and come up with a total of 6, then divide by the number of decoys of that "group???" returned, which is now 2. That would mean the second person would get 3 for his decoy returned.

Carry it out thusly:
Claimed/decoy, Granted/decoy
1) 3=3
2) (3+3)/2=3
3) (3+3+3)/3=3
.... carry out to the 10 return and it coming from opti CC
10) (3+3+3+3+3+3+3+3+3+9)/10= 36/10= 3.6
etc, etc
ID: 49308 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 49310 - Posted: 2 Dec 2007, 1:29:05 UTC
Last modified: 2 Dec 2007, 1:33:41 UTC

Ah! OK, yes, collectively we all might run 100,000 decoys on a given type of task.

"Model" and "decoy" are synonymous.

So, I apologize for missing your meaning, you had said WUs, but I think you meant something more along the lines of a batch of WUs. ...I'm still unclear what your chart is intended to illustrait. Labels on the axis would help.

[edit]
...20 out of how many? I've not seen that published. But you can get a feel for the scale by looking at the "results" link under "return participants" on the homepage. ...at least when that function is working.
Rosetta Moderator: Mod.Sense
ID: 49310 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 49311 - Posted: 2 Dec 2007, 1:40:26 UTC - in response to Message 49310.  
Last modified: 2 Dec 2007, 2:06:31 UTC

Ah! OK, yes, collectively we all might run 100,000 decoys on a given type of task.

"Model" and "decoy" are synonymous.

So, I apologize for missing your meaning, you had said WUs, but I think you meant something more along the lines of a batch of WUs. ...I'm still unclear what your chart is intended to illustrait. Labels on the axis would help.

[edit]
...20 out of how many? I've not seen that published. But you can get a feel for the scale by looking at the "results" link under "return participants" on the homepage. ...at least when that function is working.

Sorry, I editted the last post.

Basically the X axis shows each decoy as returned and the Y axis shows the "averaged" granted credit/decoy issued.

for example, (assuming stock CC's return "roughly" the same claim/decoy of a particular 'group' or task) I just arbitrarily picked a value of three. (I know it varies with machines,etc). Then when some one using an optimized CC returned one decoy requesting 3X or a claim for 9.

Essentially, it shows Rosetta has a method with equalizes credit regardless of which CC one uses (especially after 100 decoys have been returned and is being used in the average). It ensures credit is as fair, if not fairer than the vast majority of Boinc projects available. Also, since it closely follows that of the "Boinc Standard", and deviation is small, then credit comparisons between users is more meaningfull, than that of many boinc projects, since it ensures everyone gets nearly the same for the same amount of work applied.

In other words, a variation of 3X, such as with Opti CC's, is an extreme compared to the smaller variations in standard CC's, so if credit can be "equalized" with the CC's variation, then variations between processors/benchmarks is smaller and should happen more quickly.
ID: 49311 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1018
Credit: 4,334,829
RAC: 0
Message 49428 - Posted: 5 Dec 2007, 18:33:29 UTC

Here's a plot of the actual average credit/model value used for a particular job.

ID: 49428 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 49429 - Posted: 5 Dec 2007, 18:46:56 UTC - in response to Message 49428.  
Last modified: 5 Dec 2007, 18:47:15 UTC

Here's a plot of the actual average credit/model value used for a particular job.


Job,, that's the word which eluded me earlier.. And DK uses Model instead of Decoy (committing to memory). My chart using a fixed guess of 3 and only carrying it to 5K models (used to say decoys) doesn't show the gentle rise over time that the actual chart does. However, both the real and my guess show relatively even credit across the jobs life, and a similar unrest at the begining of a run of a job.

Kewl
ID: 49429 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nothing But Idle Time

Send message
Joined: 28 Sep 05
Posts: 209
Credit: 139,545
RAC: 0
Message 49441 - Posted: 6 Dec 2007, 13:08:01 UTC

John Galt 007 wrote:
I think what the original poster is getting at is what I am experiencing: The difference between claimed and granted is about 44%. It seems that all of my results end up with a lower granted than claimed.


Same here. I went to Boinc Combined Stats site and averaged the ratios for all the projects (columns) in the Rosetta row. The average was 0.817 which I interpret to mean that Rosetta is giving 18.3% less credit than the average of all other projects. I think the credit algorithm is fair but the amount of credit granted per unit of time seems low compared to the amount claimed and the amount granted by other projects. If credit chasing is what drives you I presume you would take your cpus elsewhere? Just my opinion; my resource share is reduced to just 7% now where it used to be 50%.


ID: 49441 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Number crunching : Credit system seems unfair, anyone else thinks so?



©2024 University of Washington
https://www.bakerlab.org