Credit system not fair

Message boards : Number crunching : Credit system not fair

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
[RKN] schatten1411 , Mitglied des Teams und des VEREINS Rechenkraft.net

Send message
Joined: 25 Apr 07
Posts: 12
Credit: 441,995
RAC: 0
Message 46195 - Posted: 14 Sep 2007, 13:35:29 UTC - in response to Message 46159.  

What are you gonna do with the credits when you get them?


isn`t the problem. i know it`s hard to build project like rosetta. but it isn`t the way that u or i become more points for same work because sun longer shining at your location. other wish, an row in stats where i can see how many WU`s are done all over time or flops are invested in a project. but correct results will be enough.

Does it really matter how many you get?


isn`t a matter of principle ?
ID: 46195 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FluffyChicken
Avatar

Send message
Joined: 1 Nov 05
Posts: 1260
Credit: 369,635
RAC: 0
Message 46196 - Posted: 14 Sep 2007, 15:00:02 UTC
Last modified: 14 Sep 2007, 15:01:44 UTC

Being here when the credit system was switch (and on the testing of it in Ralph) and since then there has been no known alterations to it, I do know it quite well.


One of the blantant inaccuracies of the current crediting was the first people to report a result. This would either be high or low (low if a Mac PPC or Linux at the time or you where the first and you nechmark is higher than you should be getting) This would average out for the next people along eventually.
The idea of holding results in Pending for a few days the using that average was brought up. But people like instant credit, though they do not get it at most project + it probably takes more coding at the server.

R@H could provide a graph of the variation from target to target type, they could even create a live 'avarege credit per model' for each target in a scatter graph or somthing. Probably not the best thing to do but at least it would answer a lot of questions

Though with a compiler change in BOINC for windows in the more recent versions and evening up in general of linux there will have been an alteration of where this rolling average tends to in the past months.
But these 20 credit granted if they are included MUST be dragging the average down for that target IF they are included that is.



But the problem is standing. I`ve say: u need x steps to calculate a protein, more steps -> more points. U can receive this. Now credit will granted by "monday + apple * dogs u`ve seen at last week" and all in association to rotation of sun. It will don`t understand. No one can follow this pointsystem.

It is quite simple.

Per target type

Average credit per model = Claimed credit for the 'task' using boinc banchmark / Number of models completed during the 'task'


1st result is return, get claimed credit (an average credit per model is calculated)
2nd result returned, credit based on average credit per model, new average (rolling average) credit per model is calulated form these tow computers
3rd result returned, credit now based on the rolling average credit per model, new rolling average
4th etc...


This of course assumes that the average time model is similar throughout the target type.
Since this is not true, this is where the general variation comes from... but it is hoped that over time everyone will be effected the same way....


I`ve say: u need x steps to calculate a protein, more steps -> more points.
Just afaik they cannot calculate steps (without a hit on calculation performance and a recode) and since different step can vary in time you would still find you're complaining about the same thing.
3 hrs will vary in points granted.


Team mauisun.org
ID: 46196 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nothing But Idle Time

Send message
Joined: 28 Sep 05
Posts: 209
Credit: 139,545
RAC: 0
Message 46219 - Posted: 14 Sep 2007, 18:53:38 UTC - in response to Message 46193.  
Last modified: 14 Sep 2007, 18:54:05 UTC

Mod.Sense said: Idle, the real question about your granted credit is... is it about the same prior to the BOINC change as it is after? If the credit system is working well, then the actual credit granted shouldn't have moved at all due to the changed benchmarks.
Sorry, don't keep close track of my credits/model or even the rac. However, the thrust of my post is to point out what Fluffy Chicken intimated: that one's granted credit depends not only on the number of models produced but also on the moving average at the moment of reporting. Intuitively that means if I run all my tasks for 24-hours while an undefined (perhaps large) number of hosts are running the same tasks for 3 hours then it would seem that my granted credit is more likely to be based on a large sampling in the baseline and therefore more accurate or stable. However, those running 3-hour run times would generally report first more often and be more effected by the smaller baseline and resulting moving average. The question is can one take advantage of this effect by reporting early as opposed to late. I would like to see more people increase their run times, maybe even raise the minimum allowable run time to 6 or 8 hours. Comments on this?
ID: 46219 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5664
Credit: 5,754,939
RAC: 2,104
Message 46229 - Posted: 14 Sep 2007, 21:56:34 UTC - in response to Message 46219.  

Mod.Sense said: Idle, the real question about your granted credit is... is it about the same prior to the BOINC change as it is after? If the credit system is working well, then the actual credit granted shouldn't have moved at all due to the changed benchmarks.
Sorry, don't keep close track of my credits/model or even the rac. However, the thrust of my post is to point out what Fluffy Chicken intimated: that one's granted credit depends not only on the number of models produced but also on the moving average at the moment of reporting. Intuitively that means if I run all my tasks for 24-hours while an undefined (perhaps large) number of hosts are running the same tasks for 3 hours then it would seem that my granted credit is more likely to be based on a large sampling in the baseline and therefore more accurate or stable. However, those running 3-hour run times would generally report first more often and be more effected by the smaller baseline and resulting moving average. The question is can one take advantage of this effect by reporting early as opposed to late. I would like to see more people increase their run times, maybe even raise the minimum allowable run time to 6 or 8 hours. Comments on this?


i thought sometime back when we last hashed this issue it was said that run times did not affect credit. is this true or not? the paragraph above would imply that a longer run time does affect your credit. though i can't really see any difference in my credit as i have not kept track of each wu. however after the SAN crash I set my run time back to 6 hrs and pretty much all my credits are above the claimed. perhaps that just these wu's. but in earlier versions of boinc and the last capri run my granted credits were at or lower than claimed with the same run time. i have also done a month or more at 4hrs and still my credits came up higher than claimed. so i don't know what is certain or not about longer or shorter run times.
ID: 46229 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 46231 - Posted: 14 Sep 2007, 22:51:49 UTC

greg_be
What's being discussed here is how to make your credit a bit more stable. There is no sure fire way to enhance nor degrade your credit. But the line of thought in the quoted comment is that if we report our work after a fair number of other people have, then the rolling average of credit per model should be fairly well established and therefore have less variation from one task to the next. If you are the second person in the whole project to return a task, your credit per model is entirely based on what the first person claimed. No telling if that first report was comparatively high or low relative to your machine.

Target WU runtime does not effect credit. But if one person runs tasks for 3hrs, and another for 24, the second should expect an average of 8 times more credit PER TASK. So, if you look at it one way, yes, the credit expectation between the two is very different. But if you are going to talk about credit issued by a project, you really are talking about credit PER HOUR (or minute) OF COMPUTING. The second person will complete 8 times more models for a given task then the first, and so they will receive 8 times more credit. On the other hand, the first will complete 8 tasks in a day, and the second will only complete one (per CPU).
Rosetta Moderator: Mod.Sense
ID: 46231 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Beezlebub
Avatar

Send message
Joined: 18 Oct 05
Posts: 40
Credit: 260,375
RAC: 0
Message 46269 - Posted: 15 Sep 2007, 14:23:57 UTC
Last modified: 15 Sep 2007, 14:27:52 UTC

I run 8 hrs per WU and normally get between 60-150 credits it seems.
This:
105011759 95303650 12 Sep 2007 0:38:27 UTC 12 Sep 2007 14:50:12 UTC Over Success Done 25,952.16 40.62 20.00
104979356 95273618 11 Sep 2007 21:13:20 UTC 12 Sep 2007 14:46:01 UTC Over Success Done 34,851.23 54.55 20.00

This has just come to my attention as I was perusing my results.
Also these on this page: https://boinc.bakerlab.org/rosetta/results.php?userid=5335&offset=60

All seem to be Capri14 WU's.
e6600 quad @ 2.5ghz
2418 floating point
5227 integer

e6750 dual @ 3.71ghz
3598 floating point
7918 integer


ID: 46269 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Jmarks
Avatar

Send message
Joined: 16 Jul 07
Posts: 132
Credit: 98,025
RAC: 0
Message 46282 - Posted: 15 Sep 2007, 16:36:00 UTC - in response to Message 46269.  

I run 8 hrs per WU and normally get between 60-150 credits it seems.
This:
105011759 95303650 12 Sep 2007 0:38:27 UTC 12 Sep 2007 14:50:12 UTC Over Success Done 25,952.16 40.62 20.00
104979356 95273618 11 Sep 2007 21:13:20 UTC 12 Sep 2007 14:46:01 UTC Over Success Done 34,851.23 54.55 20.00

This has just come to my attention as I was perusing my results.
Also these on this page: https://boinc.bakerlab.org/rosetta/results.php?userid=5335&offset=60

All seem to be Capri14 WU's.


The above link is bad I thinks. Also these are WU's that should be in the Problem with Rossetta 5.8 thread.

Jmarks
ID: 46282 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2

Message boards : Number crunching : Credit system not fair



©2024 University of Washington
https://www.bakerlab.org