Credit always low

Message boards : Number crunching : Credit always low

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,860,059
RAC: 3,073
Message 65856 - Posted: 27 Apr 2010, 11:59:06 UTC

For clarity, from the start:

Claimed Credit

This is based on the quick BOINC benchmark - 50% whetstone and 50% dhrystone, multiplied by the CPU time taken to process the Work Unit (WU). The benchmark is almost entirely a benchmark of the CPU core and doesn't take into account other factors such as cache size and memory speed etc.

Granted Credit

Each completed WU that your computer completes can contain many decoys (i.e. models). This is done to allow Rosetta to create work units that approximately
run for a desired time - 8 hours by default I believe(?). So one completed 8 hour WU might contain anything from 1 to 50+ decoys.

The granted credit is calculated as the average claimed credit from all previously submitted WUs of the same type, multiplied by the number of decoys.

An example

Take this WU: simpleF2_1f0s_2cx1_ProteinInterfaceDesign_15Apr2010_19616_136

My claimed credit was 47.97.
This was based on a benchmark that calculated I should get 15.98 credits per CPU-hour, and it ran for 10,806s (3.00 hours). 15.98 x 3.00 = 47.97 credits claimed. Therefore, if my benchmark had been twice as high, my claimed credit would have been twice as high.

My granted credit was 51.54.
The submitted WU contained 484 decoys. From previous submissions of this task, the R@H servers knew the claimed credit was, on average, 0.1065 credits per decoy. My computer therefore received 0.1065 x 484 = 51.54 credits.

So, in summary, if you computer gets a high or low benchmark compared to its R@H crunching ability, claimed will be high or low. Granted credit is based on the work completed so if your CPU has a large cache which doesn't help the benchmark but does help R@H crunching, you will probably find that claimed is lower than granted, and vice-versa.

The result is this:

PC 'A' has a small cache but a fast cpu core (e.g. 3GHz Dual-core AMD Athlon)
This computer gets a very high benchmark but can't match that on Rosetta.
On average it claims 50 credits per 8 hour task and is granted 40 credits.

PC 'B' has a large cache but slower FPU performance (e.g. 2.8GHz Dual-core Intel Core2)
This computer gets a lower benchmark but rosetta runs faster because of the additional cache.
On average it claims 30 credits per 8 hour task and is granted 40 credits.

One is claiming more than it receives and one is claiming less, but both are getting equal credit because their performance is equal on R@H.

HTH
Danny
ID: 65856 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,860,059
RAC: 3,073
Message 65858 - Posted: 27 Apr 2010, 12:12:56 UTC - in response to Message 65855.  

I did the above for my 920 and 3 other 920s:

my 920: 17,7409880459578
Computer-Id 1143608: 17,4120945379865
Computer-Id 1141244: 17,0467817199461
Computer-Id 1159340: 18,0163583022697

These numbers are actually credits per hour per core. My 920 and the last 920 seem to oced to the same level, the other two do not seem to be oced (or they are using optmizted BOINC versions).
Looking at these numbers, it does not look like overclocking does make any difference. I still doubt very much, that it would make any difference, if I would rum my computer at stock clocks.
And just do not try to tell me, my computer is not overclocked correctly.

Jochen


I don't have Excel on this computer to work out the average credit per CPU-hour, but if yours is lower could it be because they're using Vista (which I believe is more efficient at scheduling more cores, and you have lots), or because they're using x64? I'm not sure what difference this would make but it might be one of those, or possibly that Rosetta is getting a boost from turboboost on the stock machines which I presume is disabled on yours due to the overclock?
ID: 65858 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65859 - Posted: 27 Apr 2010, 12:49:16 UTC

I wrote a little program in C++ for this. I do not trust Excel... ;) No, actually it took me less time to write a little parser than preparing the data for an Excel sheet.

And of course this is different to the BOINC-RAC, since it does not take into consideration in what period of time these results were provided. With this credits per hour per core you should be able to calculate the RAC quite exact, if your computer runs 24/7.
The only reason for divergences is whether or not the is load from other programs _while_ crunching.

Jochen
ID: 65859 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65860 - Posted: 27 Apr 2010, 13:03:53 UTC

I have now parsed some data from my other computer (Q9650).

Credits per hour per core: 21,0304418341766
21,0304418341766 * 4 (cores) * 24 (hours) = 2018,9224160809536 RAC. This is pretty close to the real RAC of currently 2005: https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=1096431

Jochen

ID: 65860 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2146
Credit: 41,570,180
RAC: 8,210
Message 65868 - Posted: 28 Apr 2010, 3:59:05 UTC - in response to Message 65856.  

For clarity, from the start: ...

Thanks for that. That's the neatest explanation I've seen.
ID: 65868 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,860,059
RAC: 3,073
Message 65869 - Posted: 28 Apr 2010, 8:13:44 UTC - in response to Message 65856.  

Oops - it should say:
The granted credit is calculated as the average claimed credit per decoy from all previously submitted WUs of the same type, multiplied by the number of decoys.


:D
ID: 65869 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65870 - Posted: 28 Apr 2010, 8:26:29 UTC

So what you are trying to tell me, is that I got a fast machine, but Rosetta can not make any use of it?!? In this case, I rather quit crunching Rosetta on this machine...

Jochen
ID: 65870 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,860,059
RAC: 3,073
Message 65871 - Posted: 28 Apr 2010, 9:54:50 UTC - in response to Message 65870.  

So what you are trying to tell me, is that I got a fast machine, but Rosetta can not make any use of it?!? In this case, I rather quit crunching Rosetta on this machine...

Jochen

No, the 920 is about as good as you can get on rosetta, and an overclocked one is even better. What i'm trying to tell you is that a machine that consistently gets less credits than claimed can still be getting more credits per WU than a machine that gets more credits than claimed. A difference between claimed and granted is irelavant because claimed is based on a benchmark that doesn't reflect rosetta very well and so can be artificially high or low.
To put it another way, if we both ran exactly the same WU for 8hrs and I was using an old athlon we'd get the same credit for each decoy processed but yours would complete many more decoys in 8hrs than mine would so you'd get proportionally more credit for that WU. I'll have a look at your numbers this evening and get back to you though.
ID: 65871 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,217,610
RAC: 822
Message 65872 - Posted: 28 Apr 2010, 10:59:01 UTC - in response to Message 65871.  

So what you are trying to tell me, is that I got a fast machine, but Rosetta can not make any use of it?!? In this case, I rather quit crunching Rosetta on this machine...

Jochen

No, the 920 is about as good as you can get on rosetta, and an overclocked one is even better. What i'm trying to tell you is that a machine that consistently gets less credits than claimed can still be getting more credits per WU than a machine that gets more credits than claimed. A difference between claimed and granted is irelavant because claimed is based on a benchmark that doesn't reflect rosetta very well and so can be artificially high or low.
To put it another way, if we both ran exactly the same WU for 8hrs and I was using an old athlon we'd get the same credit for each decoy processed but yours would complete many more decoys in 8hrs than mine would so you'd get proportionally more credit for that WU. I'll have a look at your numbers this evening and get back to you though.


Jochen has a 3 to 4 day cache so you will have to go a few pages to get to some completed units, but he is farily consistently doing units in the 1 to 2 hour range and only doing about 5 to 10 decoys per unit. At least he is returning units in that time frame, 3 to 4 days, and you need to go all the way to 240 workunits to find units that have been returned.
ID: 65872 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65876 - Posted: 28 Apr 2010, 12:07:04 UTC - in response to Message 65872.  

Jochen has a 3 to 4 day cache so you will have to go a few pages to get to some completed units, but he is farily consistently doing units in the 1 to 2 hour range and only doing about 5 to 10 decoys per unit. At least he is returning units in that time frame, 3 to 4 days, and you need to go all the way to 240 workunits to find units that have been returned.


Basicly correct. It is a 3 day cache, but it might be messed up a bit, since my internet was 'broken' for 24 hours until 10 AM today (MESZ = GMT+2). Currently the first page with done results start at 220 (hit the Next-button once on the first result page and change the end of the URL from 'offset=20' to 'offset=220').

From my calculations, my 920 shuold reach a RAC of 3500. I was rather expecting a RAC close to 4000...

Jochen
ID: 65876 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65877 - Posted: 28 Apr 2010, 13:34:21 UTC
Last modified: 28 Apr 2010, 13:35:09 UTC

I have just processed some more data form my 920:

Processd results: 177
Min. credits per hour: 13,7128212108908
Max. credits per hour: 26,3064721433467
Ave. credits per hour: 18,2945678916994
Estimated RAC: 3512,55703520629

How is it possible, that the credits per hour vary bei 50 percent? I was not running anything else but Rosetta on this computer. Is this a hyperthreading issue?

Jochen
ID: 65877 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,694,507
RAC: 0
Message 65881 - Posted: 28 Apr 2010, 16:17:06 UTC - in response to Message 65877.  
Last modified: 28 Apr 2010, 16:17:28 UTC

I have just processed some more data form my 920:

Processd results: 177
Min. credits per hour: 13,7128212108908
Max. credits per hour: 26,3064721433467
Ave. credits per hour: 18,2945678916994
Estimated RAC: 3512,55703520629

How is it possible, that the credits per hour vary bei 50 percent? I was not running anything else but Rosetta on this computer. Is this a hyperthreading issue?

Jochen


I just realized your PC has 3GB of RAM... and is running 8 threads of JUST Rosetta... That's even below the 512/core recommendation.

Not sure whether HT hinders or helps out Rosetta crunching.
ID: 65881 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65882 - Posted: 28 Apr 2010, 16:55:39 UTC - in response to Message 65881.  

I just realized your PC has 3GB of RAM... and is running 8 threads of JUST Rosetta... That's even below the 512/core recommendation.


I am aware of this problem. Actually it has 6 GB of RAM, it is just the limiting 32-bit OS... As well I have been watching the memory consumption since I reinstalled BOINC last Saturday. Average usage is 300 MB per WU, with a peak here and there up to 320 MB. But usually there are still 400 MB of RAM left...

Should not be much of a problem. I did not see WUs 'waiting for memory'.

Jochen
ID: 65882 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,860,059
RAC: 3,073
Message 65884 - Posted: 28 Apr 2010, 18:09:37 UTC - in response to Message 65881.  

Not sure whether HT hinders or helps out Rosetta crunching.

From previous posts it helps on i7 - not so sure on P4.
ID: 65884 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65890 - Posted: 29 Apr 2010, 14:50:09 UTC - in response to Message 65869.  

The granted credit is calculated as the average claimed credit per decoy from all previously submitted WUs of the same type, multiplied by the number of decoys.


That is even worse than I expected. What is it good for granting credits by the average claimed credit for a decoy?!? There are so many different computers out there that this does not seem to be a fair method at all. As well using an optimized BOINC client will result in claiming far to much credits - as long as there is no optimized Rosetta client.

I would run Rosetta even without any crediting system... But since there is one, it should be fair and reasonable.

There sure is a way to count the actual FLOPS for a WU. In my mind this would be a better base for a crediting system.

Jochen
ID: 65890 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,860,059
RAC: 3,073
Message 65892 - Posted: 29 Apr 2010, 15:36:43 UTC - in response to Message 65890.  
Last modified: 29 Apr 2010, 15:41:16 UTC

The granted credit is calculated as the average claimed credit per decoy from all previously submitted WUs of the same type, multiplied by the number of decoys.


That is even worse than I expected. What is it good for granting credits by the average claimed credit for a decoy?!? There are so many different computers out there that this does not seem to be a fair method at all. As well using an optimized BOINC client will result in claiming far to much credits - as long as there is no optimized Rosetta client.

I would run Rosetta even without any crediting system... But since there is one, it should be fair and reasonable.

There sure is a way to count the actual FLOPS for a WU. In my mind this would be a better base for a crediting system.

Jochen


It's fair - I think you misunderstand?

I'll give another example:

If my slow computer finishes a decoy in 10 mins and your computer (which is 5x faster) completes the same decoy in 2 mins then we'll both receive the same amount of credit for that decoy (because they've done the same amount of work). In a given period, your computer will complete 5x as many decoys as my computer and so will get 5x the credit. Likewise, a 30% overclock should result in 30% quicker completion of each decoy, and so 30% more credit in a given period.

The claimed credit has very little effect because the value of each decoy is calculated from the average credit claimed for all previous decoys of that type. Therefore if you have a massively high benchmark (and consequently an excessive claimed credit), it will have a very minor effect on the granted credit - it will only serve to slightly increase the average claim for that type of decoy.

Danny
ID: 65892 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 65893 - Posted: 29 Apr 2010, 15:37:37 UTC

Decoys are the finished product that the project needs. If one machine can produce one in an hour and another machine takes 90 minutes to produce one... they both produced the same amount of work and granting credit per decoy assures they get the same credit for the same result. All without needing to have a second person run each and every task just to confirm credit. Each decoy is different, that's the whole point. So it is not possible to say with certainty ahead of time what the actual computational requirement of a given decoy will be. Averaging the reported credit per decoy is the most appropriate system the team has come up with. It allows for variation both in the machines running the work, and in the work being run.

Point of order, the recommendation is that machines running Rosetta should have a minimum of 512MB of memory. In general, that allows some margin for an operating system and one task to run. So when you extend to more then one core, the expectation would be something less then 512MB for each additional core (because any additional requirement by the operating system will be minimal). No specific per core recommendation is given, but Jochen's approach of reviewing usage of actual running tasks is a good one. Just keep in mind that the requirements do vary by type of task being performed.
Rosetta Moderator: Mod.Sense
ID: 65893 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65894 - Posted: 29 Apr 2010, 20:14:15 UTC

I see, why this approach was chosen.

But looking at this WU, which was a very short running one, with lots of granted credits, it is just all about if you get the right WUs at the right time. This is the big problem I see in this approach.
There are two things one has to do, to gain a certain influence on the granted credits (posting exactly what this is might be considered 'annoying', so I won't ;) )

But anyway, since I can not come up with a better idea, I accept it the way it is.

BTT: As well I noticed, that my 920 now is getting granted approx. 10 more cedits per WU. I did not do any changes anything to the computer, though...

Jochen
ID: 65894 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 65899 - Posted: 30 Apr 2010, 4:37:24 UTC

Yes, the variability is the one major point the credit system doesn't address. It makes small sample sizes difficult to compare, but the system also assures that it all averages out over time. I guess I'm just saying that since the whole thing is based on averages of thousands of reports, your odds of being unlucky are equal to those of being lucky.
Rosetta Moderator: Mod.Sense
ID: 65899 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65954 - Posted: 4 May 2010, 12:41:30 UTC

Now this is a ridicilous result: https://boinc.bakerlab.org/rosetta/result.php?resultid=335073708
28 credits for 7 hours of calculation.

I guess I will start aborting long running models again. This is just a waste of time...

Jochen
ID: 65954 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Credit always low



©2024 University of Washington
https://www.bakerlab.org