Message boards : Number crunching : New Crediting system: questions
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · Next
Author | Message |
---|---|
Ethan Volunteer moderator Send message Joined: 22 Aug 05 Posts: 286 Credit: 9,304,700 RAC: 0 |
I know the results for any decoy will be an average of the decoys for that entire workunit type. But this is exactly what bothers me. Although the results will even out across the workunit they won't even out on the same machine. Some people will generate less decoys than others for the same workunit in the same amount of time possibly based only on their random number. I have seen this on my own machine after doing the same workunit downloaded multiple times. Users who have the bad luck of getting these long decoys will think it is unfair that someone with the same machine gets more credit for the same workunit for the same amount of time. And if I downloaded a whole 6 workunits at once and they end up being the same workunit and it grants a bit less credit than other workunits I would be forced to either be happy with getting less credit for all of them or to dump all of them. These are the two main issues I want resolved and I don't see that happening with the current system. Thanks for your input, this is the type of feedback the folks programming the new system appreciate getting. I'm sure they'll do their best to address it as it continues to be developed. |
John McLeod VII Send message Joined: 17 Sep 05 Posts: 108 Credit: 195,137 RAC: 0 |
It's known that a given simulation within a WU will vary. However, this variability will be both within a single WU and all the others of the same type. By definition, the credits will be based on the average of these results. Since it's not possible to know in advance how long each decoy will take, it should average out. This I did not know. Is there any interest in FLOPs counting? BOINC WIKI |
Hoelder1in Send message Joined: 30 Sep 05 Posts: 169 Credit: 3,915,947 RAC: 0 |
These are the two main issues I want resolved ...I just wanted to point out that on your machine the averaging out of the granted work credit seems to have worked quit well. So far you did 19 WUs for which work credit was granted. For the first 9 of these the sum of the granted work credit is 261.61, for the second set of 9 WUs it is 248.07 - with a difference of just 5%. Something else to note: It would require a quite detailed and lengthy analysis to establish that one type of WU grants more work credit than another. You would need at least something like 10 WUs of the same type to average out the algorithm-based randomness on the model level. These WUs should perferrable be from the same host or else you would also have to account for the different CPU speeds. Since the WU names don't get exported by XML this analysis would have to be done by extracting the information from the Web pages. I suggest that anyone capable of doing this anaysis and writing a suitable script should go to the stockmarket and try to identify stocks that are doing better than others. ;-) There is one other complications: If the following statement by David Kim (from a few days ago in this thread) is still valid We may actually have the credit/model values adjust for all work units as results come in and use ralph tests to serve as a starting point.than once you did your statistical analysis, the credit/model for a specific WU may likely have been adjusted by new data coming in while you did the analysis - once you have enough data to establish that one WU grants more or less credit than another, than so does the script that calculates the credit/model on the server and the inequality will be corrected. So bottom line, I very much doubt that we will ever see anyone trying, let alone succeeding, with this trick - in which case there would still be the counter measures that David K mentioned... Team betterhumans.com - discuss and celebrate the future - hoelder1in.org |
BennyRop Send message Joined: 17 Dec 05 Posts: 555 Credit: 140,800 RAC: 0 |
date seconds old score new system Aug 15th: 85,637.94 267.35 267.35 Aug 16th: 84,394.61 263.47 403.61 Aug 17th: 86,390.48 268.61 260.29 Aug 18th: 85,987.86 267.36 276.98 Aug 19th: 85,638.19 266.27 262.05 The 15th is the old system. Starting on the 16th, my first new system report started at 323 points.. and has been adjusted up to 403 now. The next 3 seem in line. Given that there were problems with the script used that first day, the results of the 17th, the 18th, and 19th seem to show that the system can average out over time. (This is on an Athlon 64 3000+ cpu at 2Ghz, 1Gig ram, 24/7 machine that is is only taxed by doing web surfing.) |
tralala Send message Joined: 8 Apr 06 Posts: 376 Credit: 581,806 RAC: 0 |
There are two different issues: a) the "natural" variation of decoys in the same WU type due to the randomness of the algorithm b) different credits/hour-ratios among different WU With a) we have to live with. However this really should average out over time and if you had just bad luck with one WU which gave you a bad decoys/wu ration you will be lucky with the next one. b) is a problem with which the project has to deal with and which should be minimized. If not resolved b) would result in cherry picking no matter how difficult it might sound at first glance (human mind is very clever). David Kim mentioned using the RALPH values as a starting point and applying constant correction while the result on Rosetta come in. If this is practical it would counter any attempt in cherry picking and bring all different types of WU to the same value over time. There are probably many other solutions to that but it will require some finetuning. In the end I hope the variation between different WU will be within 10-20% which I think is acceptable. |
Dave Wilson Send message Joined: 8 Jan 06 Posts: 35 Credit: 379,049 RAC: 0 |
Wow, my Power Mac duel 2.5 G5 was getting 1,100 credits per 24 hours now with the new credit system it is getting 220. That is about an 80% reduction in credits. If this keeps up I will make it 100%. |
dcdc Send message Joined: 3 Nov 05 Posts: 1830 Credit: 119,208,549 RAC: 2,517 |
Wow, my Power Mac duel 2.5 G5 was getting 1,100 credits per 24 hours now with the new credit system it is getting 220. That is about an 80% reduction in credits. If this keeps up I will make it 100%. 1,100 credits per day seems very high, even for a 2CPU machine like that. I think an 80% reduction sounds very high too though. There's quite a large variation in the credit awarded under the new system - your previous credits are more consistent. Hopefully that'll improve with testing. |
Dave Wilson Send message Joined: 8 Jan 06 Posts: 35 Credit: 379,049 RAC: 0 |
Wow, my Power Mac duel 2.5 G5 was getting 1,100 credits per 24 hours now with the new credit system it is getting 220. That is about an 80% reduction in credits. If this keeps up I will make it 100%. Actually that is about a 16 hour turnaround. This machine has 5.5 gig ram and is almost dedicated full time. Since it has been getting this kind of credits from the beginning I don't think that it is at all out of line. The 80% is of course still accurate. |
Ananas Send message Joined: 1 Jan 06 Posts: 232 Credit: 752,471 RAC: 0 |
If your 4.44 is an optimized Client, that would explain the reduction. There has been a 4.43 and a 4.70 but no official 4.44 I think. With a stock 5.2.13, one PowerMac7,3 Dual I have seen had 1105.87 million intops/sec and 3573.43 million flops/sec |
tralala Send message Joined: 8 Apr 06 Posts: 376 Credit: 581,806 RAC: 0 |
Wow, my Power Mac duel 2.5 G5 was getting 1,100 credits per 24 hours now with the new credit system it is getting 220. That is about an 80% reduction in credits. If this keeps up I will make it 100%. Dave I understand your frustration however the new credit system awards fair credits based on actual work done (besides some fluctuation in the results which should be reduced). If it happens to be 220 per day that is the actual amount of computing it does (I have and AMD Athlon 64 @2.4 GHZ which gets around 300 per day if fulltime switched on). The rosetta app does not compile as fast for macs as for windows machine I think. |
Jose Send message Joined: 28 Mar 06 Posts: 820 Credit: 48,297 RAC: 0 |
Wow, my Power Mac duel 2.5 G5 was getting 1,100 credits per 24 hours now with the new credit system it is getting 220. That is about an 80% reduction in credits. If this keeps up I will make it 100%. THat is why there is a gallery of Mac optimizers available and used. |
Ethan Volunteer moderator Send message Joined: 22 Aug 05 Posts: 286 Credit: 9,304,700 RAC: 0 |
I believe Apple has moved to an Intel platform and claimed a big jump in performance. Unfortunately not all hardware is equal, there will be differences based on how fast your processor is and its architecture. |
Ananas Send message Joined: 1 Jan 06 Posts: 232 Credit: 752,471 RAC: 0 |
The PowerPC does not target the same group of users as the Intel architecture. I doubt that we could satisfy 200+ users on our AIX systems with three CPUs on x86 base. Those IBM machines are not very strong when it comes to float (when we need that for batch runs, we add 3 more CPUs on the fly), but their data throughput is immense. |
Whl. Send message Joined: 29 Dec 05 Posts: 203 Credit: 275,802 RAC: 0 |
You would have known if you had read more John. ;-) Other people have made the same good points, but it was all lost in the fuss that broke out, deleted threads, or the threads falling right out of view as nobody is allowed to post in them anymore. |
Dave Wilson Send message Joined: 8 Jan 06 Posts: 35 Credit: 379,049 RAC: 0 |
Wow, my Power Mac duel 2.5 G5 was getting 1,100 credits per 24 hours now with the new credit system it is getting 220. That is about an 80% reduction in credits. If this keeps up I will make it 100%. Without getting into a Mac/PC mud slinging war the comparison you mention only proves my point, a single Athlon @ 2.4 GHZ gets 300 while two 64bit PowerPC@2.5 GHZ chips with Altivec enabled clients only gets 220 combined, is more than twice reduced. The Rosetta app has always run fine before but the method for calculating credits is very flawed and I will not believe that these processors are in anyway one third the speed as the athlon. |
Avi Send message Joined: 2 Aug 06 Posts: 58 Credit: 95,619 RAC: 0 |
Its mentioned elsewhere about the optimization (not the boinc client, the core rosetta client) for the architecture its running on. The post said that there is still some "legacy" code, which I would suppose runs slowly. Perhaps with the new credit system, the software developers can see which machines are STILL getting low credit and see if some of the code needs to be updated? |
Ananas Send message Joined: 1 Jan 06 Posts: 232 Credit: 752,471 RAC: 0 |
The new system is machine independant, it counts the output, not the claimed power of a machine. The PowerPC is very weak in floatmath, if the program uses no specific optimisations (vectorizing). As Rosetta does not have a specific version for the different Mac versions, those big machines are as much screwed as the latest AMD and Intel machines. They all offer more power but the project clients are unable to use it. A dual Athlon MP 2600+ has beaten a 600k$ AIX machine in SETI Classic by 20% on the same workunits - I could directly compare them as one of those is my machine (no need to tell that it's the smaller of those two) and the other one I could use for a few days for load and performance tests :-) Everything but basic floatmath is faster on that AIX box than on the PC, but that is what most DC programs need. A very early demand for BOINC has been to make it report all machine capabilities to the BOINC server, which would allow to return the best client that is available for that system. They are now implementing it I think. When that has been done, it will pay off for long running projects like Rosetta to compile more than 3 or 4 different basic clients for the basic host infrastructures - and it will pay off to crunch with a cutting edge machine. |
Dave Wilson Send message Joined: 8 Jan 06 Posts: 35 Credit: 379,049 RAC: 0 |
It is not the Mac that will be "Screwed" by this I am now switching all my computers to get no new work. When they are dry, It's been fun till you "Screwed me" Guessing my final Rosetta contribution at around 380,000. You don't need or want me anymore. Dave |
Ethan Volunteer moderator Send message Joined: 22 Aug 05 Posts: 286 Credit: 9,304,700 RAC: 0 |
The new system is Computer and OS blind. It doesn't check what type of Proc or OS you're running, it just downloads an executable and processes it. If you computer can process it faster than average, your work credits will be higher than average. |
Ananas Send message Joined: 1 Jan 06 Posts: 232 Credit: 752,471 RAC: 0 |
@Dave : I did not want to offend you, I just tried to explain what happened and why. Maybe my choice of words has not been so very good but English isn't my first language :-/ |
Message boards :
Number crunching :
New Crediting system: questions
©2024 University of Washington
https://www.bakerlab.org