New Crediting system: questions

Message boards : Number crunching : New Crediting system: questions

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · Next

AuthorMessage
Alan Roberts

Send message
Joined: 7 Jun 06
Posts: 61
Credit: 6,901,926
RAC: 0
Message 23547 - Posted: 19 Aug 2006, 20:04:33 UTC

David Kim,

While I'm trying to stay far away from the Forever Credit War, I would like to help with the launch of the new credit system where possible. Looking over the modest collection of machines I have crunching Rosetta, I spotted the following results:

32975001
(99 old granted, 330 work granted)

32961368
(151 old granted, 622 work granted)

33015232
(97 old granted, 236 work granted)

These seem so out of proportion that I thought they might represent startup glitches in the new system that need correction. All WUs share the common trait of having reported in the 0500ish UTC time frame on 17-AUG. Hope this helps in some way.

Is there an target for when we'll see new system totals and RACs alongside the old numbers in the participant, machine, and team pages?

Cheers,
Alan

ID: 23547 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Scott14o

Send message
Joined: 7 Apr 06
Posts: 24
Credit: 2,147,598
RAC: 180
Message 23550 - Posted: 19 Aug 2006, 20:16:41 UTC

is the Granted work credit going toward our RAC?

also... Is the TeraFLOPS estimate being affected by optimized machines? For example, optimized machines might be making the TeraFLOPS estimate be much higher than it really is.
ID: 23550 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Saenger
Avatar

Send message
Joined: 19 Sep 05
Posts: 271
Credit: 824,883
RAC: 0
Message 23554 - Posted: 19 Aug 2006, 20:19:07 UTC - in response to Message 23550.  

is the Granted work credit going toward our RAC?

also... Is the TeraFLOPS estimate being affected by optimized machines? For example, optimized machines might be making the TeraFLOPS estimate be much higher than it really is.

If theay use the usual BOINC calculation of TFlop, it will definitely be too high now, considering the claimed amount of "optimized" clients abound. It should again reflect reality after the change (if the new method is done right).
ID: 23554 · Rating: -0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
MikeMarsUK

Send message
Joined: 15 Jan 06
Posts: 121
Credit: 2,637,872
RAC: 0
Message 23555 - Posted: 19 Aug 2006, 20:19:26 UTC
Last modified: 19 Aug 2006, 20:20:18 UTC

RAC is currently based on boinc credit, a second RAC column may appear which is based on work credit.

Teraflops will be modified in both directions by the benchmark results - Linux machines will be underreporting for example. If tereflops were calculated from the work credit it would probably return a similar result since it's basically an average of the boinc credit.

ID: 23555 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ananas

Send message
Joined: 1 Jan 06
Posts: 232
Credit: 752,471
RAC: 0
Message 23592 - Posted: 19 Aug 2006, 22:06:10 UTC

@Alan : The ones you found really look strange but for all 3 hosts they are the very first ones on the new system, just a few hours after it has been introduced. That is most likely a flaw that already has been fixed a few hours later.
ID: 23592 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dave Wilson

Send message
Joined: 8 Jan 06
Posts: 35
Credit: 379,049
RAC: 0
Message 23640 - Posted: 20 Aug 2006, 10:00:41 UTC

I just noticed that some of my computers that claimed 122.00 received only 16.40 credit when they are all set to run 24 hours per work unit. I also see a 30 -65% lower granted on all my computers. It may not be about the credit but it is enough that I will see in two weeks if this works out then I will decide if I should just save electricity and give up on Rosetta as well.
See for yourself my computers are not hidden.

Dave

It is not about the credit but if I can not get the same for the work I do as the others then why should I pay as much or more then them.
ID: 23640 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 23642 - Posted: 20 Aug 2006, 10:23:28 UTC

Dave, your "actual" granted credits should not have changed, they are still testing the system. If you look at "your results", you'll notice the titles of the columns have changed. They used to be "claimed credit" and "granted credit". Now they are "granted credit" and "granted work credit". In the old sytem (still being used) "claimed credit" = "granted credit", so they renamed the "claimed credit" column "granted credit" and changed "granted credit" to "granted work credit" which shows what your credit would be under the new system which is supposed to be replacing the old system in a week or two (who really knows exactly when).

does this help?

tony
ID: 23642 · Rating: -2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile carl.h
Avatar

Send message
Joined: 28 Dec 05
Posts: 555
Credit: 183,449
RAC: 0
Message 23643 - Posted: 20 Aug 2006, 10:23:29 UTC

My humble opinion.

The points on the new system are way too low. If they are in cm change them to mm.

People are paying lot`s of money for excellent equipment and seeing tiny figures this WILL upset them. It`s psychological.
Not all Czech`s bounce but I`d like to try with Barbar ;-)

Make no mistake This IS the TEDDIES TEAM.
ID: 23643 · Rating: -0.99999999999999 · rate: Rate + / Rate - Report as offensive    Reply Quote
tralala

Send message
Joined: 8 Apr 06
Posts: 376
Credit: 581,806
RAC: 0
Message 23644 - Posted: 20 Aug 2006, 10:25:32 UTC - in response to Message 23640.  

I just noticed that some of my computers that claimed 122.00 received only 16.40 credit when they are all set to run 24 hours per work unit. I also see a 30 -65% lower granted on all my computers. It may not be about the credit but it is enough that I will see in two weeks if this works out then I will decide if I should just save electricity and give up on Rosetta as well.
See for yourself my computers are not hidden.

Dave

It is not about the credit but if I can not get the same for the work I do as the others then why should I pay as much or more then them.


Hi Dave,

I checked on your windows host and it seems you're using the optimized 5.12.36 boinc client from crunch3r. Can that be? The granted work credit will be lower than the current granted credit, if you are using the optimized client, but this will be true for all hosts, so that all hosts which use the optimized boinc clients will "suffer" whereas those who use the standard boinc client will probably gain a little bit.

Nevertheless the result will be fair credit granted on actual work done and not on reported benchmarks. As for your Macs I don't know whether you are using any kind of optimized client but it might well be that the old Macs (with G4 and G5 processors) don't run Rosetta with the same speed as corresponding Intel and AMD PCs.

That being said currently the granted work credit fluctuates a lot and is sometimes really out of line. It should average out in the long run but strange results may happen.

To sum it up: The granted work credit should get more regular and 16 credits for 24hour WU shouldn't happen. Nevertheless your RAC may drop by some degree after the new credit system is the primary one as for others who use optimized clients.
ID: 23644 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
tralala

Send message
Joined: 8 Apr 06
Posts: 376
Credit: 581,806
RAC: 0
Message 23645 - Posted: 20 Aug 2006, 10:30:57 UTC - in response to Message 23643.  
Last modified: 20 Aug 2006, 10:31:35 UTC

My humble opinion.

The points on the new system are way too low. If they are in cm change them to mm.

People are paying lot`s of money for excellent equipment and seeing tiny figures this WILL upset them. It`s psychological.


I agree it's psychological and many people will be upset. However as I see it there will be a drop in RAC for users which use optimized boinc clients, since it's not reasonable to grant as many credits as the optimized boinc clients claimed. It has been suggested to not pay attention to project parity regarding the granting of credits but it will cause upset among the BOINC community if Rosetta starts to grant three times the credit than other BOINC projects do. This will harm Rosetta as well. So there is a choice between Scylla and Charybdis and there is no other way than to seek a compromise in granting credits (more than the standard BOINC client claims but less than the optimized BOINC client claims).
ID: 23645 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ananas

Send message
Joined: 1 Jan 06
Posts: 232
Credit: 752,471
RAC: 0
Message 23649 - Posted: 20 Aug 2006, 10:39:05 UTC
Last modified: 20 Aug 2006, 10:42:03 UTC

On Dave's Windows box I see a core client by TruXoft, this client seems to be "lite-optimized" compared to the stock client and thus gives you a higher benchmark result, even with calibration disabled - it is just normal (and correct) that the new credits are a bit lower (like 157.65 vs. 120.34).

I am using the same one, the 5.2.12tx has about the same benchmark results as the 4.19 that I used before but the Berkeley 5.x clients have lower values.


I don't know about the core clients for Apple, BOINC always had trouble to keep different architectures and operating systems compareable.


The new system will hopefully give the same credits for the same work across all platforms.
ID: 23649 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile carl.h
Avatar

Send message
Joined: 28 Dec 05
Posts: 555
Credit: 183,449
RAC: 0
Message 23650 - Posted: 20 Aug 2006, 10:39:13 UTC

Tralala,

Yep I see your point but feel this is somewhat of a half measure to solve the problem. Without wanting to start a big argument again, you`re either with the Boinc program or you`re not.

For Boinc`ers the new measure is too small and their overall stats will be affected.

I can now see why some have said that Rosetta needs to move from Boinc altogether. I feel more problems are in the offing. I see no way out personally !
Not all Czech`s bounce but I`d like to try with Barbar ;-)

Make no mistake This IS the TEDDIES TEAM.
ID: 23650 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dave Wilson

Send message
Joined: 8 Jan 06
Posts: 35
Credit: 379,049
RAC: 0
Message 23653 - Posted: 20 Aug 2006, 10:47:34 UTC - in response to Message 23644.  

I just noticed that some of my computers that claimed 122.00 received only 16.40 credit when they are all set to run 24 hours per work unit. I also see a 30 -65% lower granted on all my computers. It may not be about the credit but it is enough that I will see in two weeks if this works out then I will decide if I should just save electricity and give up on Rosetta as well.
See for yourself my computers are not hidden.

Dave

It is not about the credit but if I can not get the same for the work I do as the others then why should I pay as much or more then them.


Hi Dave,

I checked on your windows host and it seems you're using the optimized 5.12.36 boinc client from crunch3r. Can that be? The granted work credit will be lower than the current granted credit, if you are using the optimized client, but this will be true for all hosts, so that all hosts which use the optimized boinc clients will "suffer" whereas those who use the standard boinc client will probably gain a little bit.

Nevertheless the result will be fair credit granted on actual work done and not on reported benchmarks. As for your Macs I don't know whether you are using any kind of optimized client but it might well be that the old Macs (with G4 and G5 processors) don't run Rosetta with the same speed as corresponding Intel and AMD PCs.

That being said currently the granted work credit fluctuates a lot and is sometimes really out of line. It should average out in the long run but strange results may happen.

To sum it up: The granted work credit should get more regular and 16 credits for 24hour WU shouldn't happen. Nevertheless your RAC may drop by some degree after the new credit system is the primary one as for others who use optimized clients.

As far as the G4 and G5s they actually get more work done than the PCs of equal speed always have no matter what client I used. Each processor in my duel 2.5 G5 does more work than my 3.0 gig P4. As you see that a G4 that will only get 16.44 in 24 hours is totally un-exceptible. As for the Crunch3r on the only PC I have it does not calibrate for Rosetta so there should not be any issue there at all.
I don't know what is getting reported from where to where but my RAC has gone down from 3,000+ to 2,300. Last month I shut my computers off for 8 days and save $110. on my electric bill I guess I will save $400. the following month if I shut them all down.
ID: 23653 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
tralala

Send message
Joined: 8 Apr 06
Posts: 376
Credit: 581,806
RAC: 0
Message 23655 - Posted: 20 Aug 2006, 10:54:04 UTC - in response to Message 23653.  


As far as the G4 and G5s they actually get more work done than the PCs of equal speed always have no matter what client I used. Each processor in my duel 2.5 G5 does more work than my 3.0 gig P4. As you see that a G4 that will only get 16.44 in 24 hours is totally un-exceptible. As for the Crunch3r on the only PC I have it does not calibrate for Rosetta so there should not be any issue there at all.
I don't know what is getting reported from where to where but my RAC has gone down from 3,000+ to 2,300. Last month I shut my computers off for 8 days and save $110. on my electric bill I guess I will save $400. the following month if I shut them all down.


1. Your RAC is still calculated from the old system so your drop has nothing to do with the testing of the new system. Perhaps it is the result of your temporarily shutdown.

2. 16 credits for 24 hours is unacceptable and clearly a bug which hopefully gets corrected.

3. After Rosetta switches to the new credit system nobody can say for sure what will happen to your RAC. It might very well be that it will drop even further. I just say that to avoid disappointment.

4. I wouldn't base my decision whether to turn off my machines or not based on the credits you receive but whether you can afford it or not.

ID: 23655 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Dave Wilson

Send message
Joined: 8 Jan 06
Posts: 35
Credit: 379,049
RAC: 0
Message 23659 - Posted: 20 Aug 2006, 11:06:23 UTC - in response to Message 23655.  


1. Your RAC is still calculated from the old system so your drop has nothing to do with the testing of the new system. Perhaps it is the result of your temporarily shutdown.

2. 16 credits for 24 hours is unacceptable and clearly a bug which hopefully gets corrected.

3. After Rosetta switches to the new credit system nobody can say for sure what will happen to your RAC. It might very well be that it will drop even further. I just say that to avoid disappointment.

4. I wouldn't base my decision whether to turn off my machines or not based on the credits you receive but whether you can afford it or not.

I can accept the drop if it was caused by me. It for me is not a matter of if I can afford it or not, I as do others, want to contribute as much as possible, but if my contribution of the same or greater cost is recognized as less than someone else who spends less time or money gets, just because he uses a different system, then I have difficulty continuing.

Please don't think of this as an argument of any kind. It is just my observation and very probable action if this continues in this direction.
If anything it is more PC vs Mac treatment than anything else.
Don't start on that either it's my opinion and nothing anyone can say will change that.
ID: 23659 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ananas

Send message
Joined: 1 Jan 06
Posts: 232
Credit: 752,471
RAC: 0
Message 23665 - Posted: 20 Aug 2006, 11:18:17 UTC

The new credit system will not even look at the system you are using. It tries to measure the plain work, that has been done, without knowing anything about the host that did this work.
ID: 23665 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
AMD_is_logical

Send message
Joined: 20 Dec 05
Posts: 299
Credit: 31,460,681
RAC: 0
Message 23682 - Posted: 20 Aug 2006, 13:04:06 UTC - in response to Message 23640.  

I just noticed that some of my computers that claimed 122.00 received only 16.40 credit when they are all set to run 24 hours per work unit.


Well, looking at one of your results https://boinc.bakerlab.org/rosetta/result.php?resultid=33393022 it looks like it took nearly a day to do 7 decoys.

I found a couple of results from that WU that I did on an Athlon XP 2600+
https://boinc.bakerlab.org/rosetta/result.php?resultid=33324921
https://boinc.bakerlab.org/rosetta/result.php?resultid=33315348
That did 34 or 35 decoys in about 10 hours. It looks like the credit system is working, but certain older MAC computers just don't crunch Rosetta very fast.

ID: 23682 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Prom

Send message
Joined: 21 Jun 06
Posts: 23
Credit: 931,604
RAC: 0
Message 23969 - Posted: 20 Aug 2006, 23:46:20 UTC - in response to Message 22527.  
Last modified: 20 Aug 2006, 23:55:17 UTC

Ok I know this is a questions thread and not a for or against thread although I can't really see the new system as working the way it is supposed to so I will try to phrase it as a question.

My first results on the new system were a bit less than usual and I'm not running an optimized client. I can live with that however as long as all the machines in the project get the same treatment. Then suddenly some of my results went up to double the amount of usual credit so as to somehow even out the results and now it is more or less within +/-50% of usual. Where my problem comes in is how will these new credits accurately reflect a machines performance if the results will allways have some discrepancies between different workunits. While optimizing my machine I could see immediately after the first completed workunit that the performance was better, but now I wouldn't even be able to see if it was the same machine. With credit being based on the amount of structures built there will even be discrepancies amongst the same workunits. From watching the program a lot I have noticed that some structures take longer to complete than others from the same workunit. I have noticed 3 or even 5 structures being built in 2 hours time and then only 1 in the next hour. If credit is based on mostly results for the shorter structures I would get less credit whenever my structures take longer than usual and the people who were lucky enough to get shorter structures based probably on their random number would get more credit.

I can't see results as evening out either in the same way that gambling also doesn't even out and some people will lose and some will win. Dave, I have just scanned over your results and some of them is 16% of the claimed credit while others are 20%, 25% and even 60%. Those are all macs, but the interesting thing is that the 16%, 20%, 25% and 60% is not mixed across those machines, but each of them come from mainly one machine. The windows pc on the other hand has ranges of between 40% to 90%. AMD, I can't however see such a large discrepancy based only on whether or not the machine is optimized for the particular workunit or not and certainly not for 7 decoys generated over 35 or your machine whose floating-point measurement is only about 40% more.

Possibly Rosetta isn't using most resources most of the time for some reason, but then the usual claimed credit should also be less. Or you were unlucky enough to get a random number that generated less structures which would prove my point that the results will not even out as is hoped. How will these issues be adequately resolved? I'm not doing this for the credit, but if Rosetta grants it then they should do it properly and even a discrepancy of 1% will not give results that are even enough and will eventually result in quarrelling. I hope they keep the old system when switching over to the new one as I don't see it long before they will eventually have to move back to the old one and restore everything based on the old results unless they can resolve ALL these discrepancies effectively to give REALLY ACCURATE results.
The new credit system will credit you for the (massive amount of) work you do/have done. The more work the computer does (i.e. the faster the computer) the more credits you get. If you have more CPUs, you get more credits. If you overclock, you get more credits. If you raise the FSB and reduce a memory bottleneck, you'll get more credits.

Looks like a good, fair system.

Sorry for having to point out the obvious here as nobody else has done so yet, but this is exactly what the old system did and it did it even more accurately. If this is your criteria for a good and fair system then the old one wins hands down imo.
BBLounge - Broadband and Technology forum
ID: 23969 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ethan
Volunteer moderator

Send message
Joined: 22 Aug 05
Posts: 286
Credit: 9,304,700
RAC: 0
Message 23972 - Posted: 20 Aug 2006, 23:55:12 UTC

It's known that a given simulation within a WU will vary. However, this variability will be both within a single WU and all the others of the same type. By definition, the credits will be based on the average of these results. Since it's not possible to know in advance how long each decoy will take, it should average out.

The other side of it is one WU type may grant slightly more or less credit than others. They're trying to keep that to a minimum, but as they've posted, they have tools to find people who take advantage of the system with this method.

Once the average credit/decoy is calculated, it's entirely dependant on how fast a given machine is. If Intel cpus can do more science in an hour due to features of its hardware, it will end up getting more credits (and vice-versa).

ID: 23972 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Prom

Send message
Joined: 21 Jun 06
Posts: 23
Credit: 931,604
RAC: 0
Message 23976 - Posted: 21 Aug 2006, 0:13:09 UTC - in response to Message 23972.  

It's known that a given simulation within a WU will vary. However, this variability will be both within a single WU and all the others of the same type. By definition, the credits will be based on the average of these results. Since it's not possible to know in advance how long each decoy will take, it should average out.

The other side of it is one WU type may grant slightly more or less credit than others. They're trying to keep that to a minimum, but as they've posted, they have tools to find people who take advantage of the system with this method.

Once the average credit/decoy is calculated, it's entirely dependant on how fast a given machine is. If Intel cpus can do more science in an hour due to features of its hardware, it will end up getting more credits (and vice-versa).

I know the results for any decoy will be an average of the decoys for that entire workunit type. But this is exactly what bothers me. Although the results will even out across the workunit they won't even out on the same machine. Some people will generate less decoys than others for the same workunit in the same amount of time possibly based only on their random number. I have seen this on my own machine after doing the same workunit downloaded multiple times. Users who have the bad luck of getting these long decoys will think it is unfair that someone with the same machine gets more credit for the same workunit for the same amount of time. And if I downloaded a whole 6 workunits at once and they end up being the same workunit and it grants a bit less credit than other workunits I would be forced to either be happy with getting less credit for all of them or to dump all of them. These are the two main issues I want resolved and I don't see that happening with the current system.
BBLounge - Broadband and Technology forum
ID: 23976 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · Next

Message boards : Number crunching : New Crediting system: questions



©2024 University of Washington
https://www.bakerlab.org