I want to start this thread because I (and I guess there are more people with me) have some questions.
Lets use this thread for questions and not for discussion of pros and contras of the crediting system.

On Ralph, one of the project people said:

The new crediting system is pretty simple.

First, we determine how much credit to grant per model for each work unit by running tests on Ralph. We will use the average credit per model from the Ralph tests for production runs on Rosetta@home.
These values will be work unit specific so work units that take longer to generate structures will have higher values.

I have explained this below in a previous post. The only difference is that I decided to keep the old crediting system so users can compare their work with either system. If you take a look at the top participants page you will see two new columns, "Recent average work credit" and "Total work credit". These are the new credit per model based values. I've been using 2 credits per model on Ralph JUST FOR TESTING. Rosetta@home will use work unit specific values determined from the Ralph test runs.

and

Everyone keep in mind that the current standard boinc crediting system will still be used.

Also, minor modifications to the credit/model values will not make that much of a difference in the long run. The important thing to know is that given any credit/model value, users will be on a level playing field. I think we can all agree that this is the major drive/motivation for coming up with a new method. Making sure it closely matches the BOIINC credit values is not as important since we will still use the old system along with the new.

So if I understand well: The 2 crediting systems will be used both together? So you can choose to count on the old or the new system?

---

Member of Dutch Power Cows

I want to start this thread because I (and I guess there are more people with me) have some questions.
Lets use this thread for questions and not for discussion of pros and contras of the crediting system.

On Ralph, one of the project people said:

The new crediting system is pretty simple.

First, we determine how much credit to grant per model for each work unit by running tests on Ralph. We will use the average credit per model from the Ralph tests for production runs on Rosetta@home.
These values will be work unit specific so work units that take longer to generate structures will have higher values.

I have explained this below in a previous post. The only difference is that I decided to keep the old crediting system so users can compare their work with either system. If you take a look at the top participants page you will see two new columns, "Recent average work credit" and "Total work credit". These are the new credit per model based values. I've been using 2 credits per model on Ralph JUST FOR TESTING. Rosetta@home will use work unit specific values determined from the Ralph test runs.

and

Everyone keep in mind that the current standard boinc crediting system will still be used.

Also, minor modifications to the credit/model values will not make that much of a difference in the long run. The important thing to know is that given any credit/model value, users will be on a level playing field. I think we can all agree that this is the major drive/motivation for coming up with a new method. Making sure it closely matches the BOIINC credit values is not as important since we will still use the old system along with the new.

So if I understand well: The 2 crediting systems will be used both together? So you can choose to count on the old or the new system?

Hello Kristof!
Yes, my understanding is that there will at least for a time be the 2 systems shown. Smart move, allows for adjustments and to keep a happy family.
Has Bubbles posted any more pics of his shoes? <BG>
I ought to post that story here, these guys might get a kick out of it.
Thanks for your time,
Movieman

---

since we will still use the old system along with the new.

There were two things that David Kim was talking about that could be described by that quote.

1. They use lab machines (something trusted) on Ralph running the standard boinc client to generate a credit/model for each major WU. The credit/model is thus based on the old credit system. If our Rosetta machines produce models faster than the ones on Ralph, we'll end up with more credits/hour than the machines in Ralph. If our Rosetta machines produce models slower than the machines on Ralph, we'll end up with fewer credits/hour than the machines in Ralph. The new system would have the credit/model based on the old credit system; so both systems would be running at the same time.

2. He mentioned keeping track of the new credit system, plus the claimed score from the old credit system for each of our computers. It should make troubleshooting easier, and comparisons possible.

Mention which of the two cases you're more interested in, and hopefully we can get David (either/or/both) to comment on it..

---

since we will still use the old system along with the new.

There were two things that David Kim was talking about that could be described by that quote.

1. They use lab machines (something trusted) on Ralph running the standard boinc client to generate a credit/model for each major WU. The credit/model is thus based on the old credit system. If our Rosetta machines produce models faster than the ones on Ralph, we'll end up with more credits/hour than the machines in Ralph. If our Rosetta machines produce models slower than the machines on Ralph, we'll end up with fewer credits/hour than the machines in Ralph. The new system would have the credit/model based on the old credit system; so both systems would be running at the same time.

2. He mentioned keeping track of the new credit system, plus the claimed score from the old credit system for each of our computers. It should make troubleshooting easier, and comparisons possible.

Mention which of the two cases you're more interested in, and hopefully we can get David (either/or/both) to comment on it..

I have to toss this in just for a little levity:
We don't run machines as slow as what they use in the labs!<BG>
Ok, I would guess that they have some decent equipment and I think I saw that they have a 500 node Linux server? Correct me on that if I'm wrong.
On a machine by machine basis, I know that XS,FreeDC and the Dutch Power Cows have some pertty decent crunchers. I've got high end dual xeons and now a dual Opty to play with but some of these guys make me look like a kid with an abacus.
There are individuals with way beyond $100,000.00 in equipment in their homes.
One of our newer members has 37 PC's running in his house and he isn't the most "Xtreme" of the group. Another runs way over 100 machines for Rosetta, all his own, all from his pocket.
Thanks for your time,
Movieman

---

I have to toss this in just for a little levity:
We don't run machines as slow as what they use in the labs!<BG>
Ok, I would guess that they have some decent equipment and I think I saw that they have a 500 node Linux server? Correct me on that if I'm wrong.
On a machine by machine basis, I know that XS,FreeDC and the Dutch Power Cows have some pertty decent crunchers. I've got high end dual xeons and now a dual Opty to play with but some of these guys make me look like a kid with an abacus.
There are individuals with way beyond $100,000.00 in equipment in their homes.
One of our newer members has 37 PC's running in his house and he isn't the most "Xtreme" of the group. Another runs way over 100 machines for Rosetta, all his own, all from his pocket.
Thanks for your time,
Movieman

The new credit system will credit you for the (massive amount of) work you do/have done. The more work the computer does (i.e. the faster the computer) the more credits you get. If you have more CPUs, you get more credits. If you overclock, you get more credits. If you raise the FSB and reduce a memory bottleneck, you'll get more credits.

Looks like a good, fair system.

---

The new credit system will credit you for the (massive amount of) work you do/have done. The more work the computer does (i.e. the faster the computer) the more credits you get. If you have more CPUs, you get more credits. If you overclock, you get more credits. If you raise the FSB and reduce a memory bottleneck, you'll get more credits.

Looks like a good, fair system.

While I like the new crediting system, what about guys like me who still do some Rosetta on paper and snail mail the results? :0

---

(Click for detailed stats)

From what I understand, there are currently several differnt work units out. Some are on different length proteins, and some are using different methods. The plan is to test each on RALPH and come to an average credit per decoy, for each work unit type (processing and protein mix). The work units that have larger proteins will get more credit.
The point of this is to avoid relying on the FLOPS that BOINC returns, since they can be tweaked.

---

The new credit system will credit you for the (massive amount of) work you do/have done. The more work the computer does (i.e. the faster the computer) the more credits you get. If you have more CPUs, you get more credits. If you overclock, you get more credits. If you raise the FSB and reduce a memory bottleneck, you'll get more credits.

Looks like a good, fair system.

And it won't be disputable - higher credits relative to someone else or some other machine means that the machine with the higher credits HAS done more work not just claimed that it has :)

---

And it won't be disputable - higher credits relative to someone else or some other machine means that the machine with the higher credits HAS done more work not just claimed that it has :)

That's the goal, however right now they are not quite there. Differen WU will give very differen credits/hour and it is not clear how they gonna fix that.

---

If it's not in an acceptible state, then why move it from RALPH? Seems to me that it needed at least a couple more test runs on RALPH to equal out credits.

---

And it won't be disputable - higher credits relative to someone else or some other machine means that the machine with the higher credits HAS done more work not just claimed that it has :)

That's the goal, however right now they are not quite there. Differen WU will give very differen credits/hour and it is not clear how they gonna fix that.

Different WUs will give different credits per MODEL. The credit per HOUR should be very consistent on a given box, regardless of which WU or Rosetta approach is being used to study a protein.

---

Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/

If it's not in an acceptible state, then why move it from RALPH? Seems to me that it needed at least a couple more test runs on RALPH to equal out credits.

IMHO:

There is a reason why taking the "new credit system" from RALPH and play it here for comments and tweaking makes sense: the system needs a higher range of computers/os/configurations to see how it works under pressure. RALPH as good as a system for tests that is is, doesnt have a good representation of all the machines/OS present: with the new machines and cpu's now available and soon available the posibilities for credit levels will be endless . (Alas that exciting hardware is not fully represented in RALPH)

I hope people have noticed that the old system is being kept and that will allow for comparison (And yes Back up ) . I just think what would have happened if a RALPH-only -tested credit system collapses due to the complexity of the issues and unfroseen problems that ussually sprout in a complex project like this.. So I appreciate the fact that the old system is there , while a "real" life testing is done to the new.

That said: The only way this testing is going to work, if everyone comes with it with "clean hands" no predetermined agendas, openess to others people's ideas and the care to use the chance to makes sugestions and make them with civility.


The developers and scientists have demonstrated that they pay attention to our needs and our opinions. They are given us a chance seldom given in a DC project.

Let's not blow it. ( Deep inside I know we will not)

And it won't be disputable - higher credits relative to someone else or some other machine means that the machine with the higher credits HAS done more work not just claimed that it has :)

That's the goal, however right now they are not quite there. Differen WU will give very differen credits/hour and it is not clear how they gonna fix that.

Different WUs will give different credits per MODEL. The credit per HOUR should be very consistent on a given box, regardless of which WU or Rosetta approach is being used to study a protein.

So the difference lies in the machines we bring to the project. That is fair for me.

Just a message from DK on what their plans currently are:

http://ralph.bakerlab.org/forum_thread.php?id=233#2121

Today's outage will simply add 2 columns to the results section. Everything will continue to run off the old/current credit system. . there will just be additional fields to look at.

---

Well there are no extra columns on my results and the granted credit is way down on the claimed one!

The granted vs. claimed credits is certainly inconsistent with granted being both higher and lower, even on the same PC. I have one that granted 50% more than claimed on one WU, but less than claimed on the other. I'm not even going to try to figure it out; I'll just keep crunching.

---

The granted vs. claimed credits is certainly inconsistent with granted being both higher and lower, even on the same PC. I have one that granted 50% more than claimed on one WU, but less than claimed on the other. I'm not even going to try to figure it out; I'll just keep crunching.

They could be using different algorithms, and your cpu cache is better for one of them.

BTW, could there be someway to see if a CPU performs "abnormally well" using a certain algorithm and prefer sending some to them? i.e. AMD and intel seem to sometime reverse the whet/dhyr results (one is better at one than the other) etc.

My main comptuer returned 3 results after the upgrade and was given almost double credit. I checked from a top team, they got 1/2 to 1/3 of the credit (for 3+ WU's).. prolly optimized clients?

---

The granted vs. claimed credits is certainly inconsistent with granted being both higher and lower, even on the same PC. I have one that granted 50% more than claimed on one WU, but less than claimed on the other. I'm not even going to try to figure it out; I'll just keep crunching.

I just did a quick test on the Windows users of my team (all using standard client). There were 9 WUs returned since Rosetta came up again and the average of the granted credit is very, very close to the claimed credit (to within 5 % or so). There is some scatter between the granted and claimed credit values: for seven of the 9 WUs the granted credit is within +/-20% of claimed credit. Observation for those who followed the recent activities on Ralph: I think David Kim adjusted the "credit/model correction factor" some more when moving the new system to Rosetta: while in the recent Ralph tests the granted credit on average was still about 10-15% higher than the claimed credit (for Windows/standard client users), the match now seems almost perfect - well as far as I can tell from the 9 WUs I looked at. If you want my opinion, I think David K did a great job so far. :-)

---

Team betterhumans.com - discuss and celebrate the future - hoelder1in.org

I think it looks good, the Linux machines are not under-credited anymore and the time/credits ratio on the same machine for compareable WUs seems to be quite constant.

It looks as if my 5.2.12tx client did increase the credits on some machines even without having calibration enabled :-/ I don't mind if they get adjusted (i.e. reduced) to the new values if that is possible.

I'm using the same correction factor but I calculated the credit/model values from R@h results to get things started since tests to determine the values were not carried out on ralph for already existing work units. If I couldn't determine the value because results were not returned yet, the value will get set and then adjust as results come in. We may actually have the credit/model values adjust for all work units as results come in and use ralph tests to serve as a starting point.

Sorry, I didn't have time to add the columns on the php pages. Ralph was unexpectedly (at least to me) shut down for a move to a new location while I was working on the transition, also, it is taking quite a while to extract and add the total_work_credit values to the database.

Please keep in mind that there will be differences when comparing the claimed and granted credits due to the random nature of the predictions. The values should average out though.