A few things about RAC

Message boards : Number crunching : A few things about RAC

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile mitro

Send message
Joined: 18 Sep 05
Posts: 6
Credit: 225,313
RAC: 0
Message 552 - Posted: 26 Sep 2005, 21:56:00 UTC

First, obviously I "have a dog in this fight" but this isn't possible :) :


Secondly, how is the RAC figured, since I have on computer that has a RAC by itself of 700, yet I'm only at 300.

Thanks in advance.
ID: 552 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 556 - Posted: 27 Sep 2005, 0:01:14 UTC - in response to Message 552.  
Last modified: 27 Sep 2005, 0:09:06 UTC

First, obviously I "have a dog in this fight" but this isn't possible :) :


Secondly, how is the RAC figured, since I have on computer that has a RAC by itself of 700, yet I'm only at 300.

Thanks in advance.



This may help explain it. http://boinc-doc.com/boinc-wiki/index.php?title=RAC on Paul Buck's Wiki site goes into some detail in how the RAC is calculated.

I see you have three computers attached and basically the simplest answer is the RAC is an average of the three. Your one PC with the 700 RAC is being brought down by the other 2 computers you have attached to the project with much lower RAC's when it averages out the numbers.

(edit)I was incorrect in some respect in how I put it. The average of the 3 computers is also related to how often they report in work. Paul's site states the easiest way to think of it is as a speedometer of how fast your putting out work and the average of your 3 computers is around 300 (/edit)

ID: 556 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile mitro

Send message
Joined: 18 Sep 05
Posts: 6
Credit: 225,313
RAC: 0
Message 561 - Posted: 27 Sep 2005, 0:19:05 UTC - in response to Message 556.  



I see you have three computers attached and basically the simplest answer is the RAC is an average of the three. Your one PC with the 700 RAC is being brought down by the other 2 computers you have attached to the project with much lower RAC's when it averages out the numbers.

(edit)I was incorrect in some respect in how I put it. The average of the 3 computers is also related to how often they report in work. Paul's site states the easiest way to think of it is as a speedometer of how fast your putting out work and the average of your 3 computers is around 300 (/edit)


Any other BOINC project I've paticipated in was simply the RAC of each computer added together... if I have 3 computers and they all has a RAC of 300 then my RAC is 900. I'm utterly confused.

ID: 561 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
RDC

Send message
Joined: 16 Sep 05
Posts: 43
Credit: 101,644
RAC: 0
Message 563 - Posted: 27 Sep 2005, 0:31:52 UTC

RAC has always driven me nuts. Some projects I have seem to act that way you describe and others seem to operate as an average of the connected computers. Then there's projects I can't figure out as the more work I pump in, the lower my RAC becomes if sent up as completed instead of in bulk once a day.

I believe it's supposed to be the way it is operating here with the average speed of all connected computers to the account as that technically makes more sense and that's the way I interpret the Wiki entry. I could be wrong though, wouldn't be the first time :P
ID: 563 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 565 - Posted: 27 Sep 2005, 0:35:40 UTC - in response to Message 561.  
Last modified: 27 Sep 2005, 0:37:49 UTC



I see you have three computers attached and basically the simplest answer is the RAC is an average of the three. Your one PC with the 700 RAC is being brought down by the other 2 computers you have attached to the project with much lower RAC's when it averages out the numbers.

(edit)I was incorrect in some respect in how I put it. The average of the 3 computers is also related to how often they report in work. Paul's site states the easiest way to think of it is as a speedometer of how fast your putting out work and the average of your 3 computers is around 300 (/edit)


Any other BOINC project I've paticipated in was simply the RAC of each computer added together... if I have 3 computers and they all has a RAC of 300 then my RAC is 900. I'm utterly confused.

It's obviously "NONE OF THE ABOVE".

I have 4 boxes on Rosetta - with RACs of 210,136,127,97

My Participant RAC is 311, which is neither the SUM (570), or AVG (140) of the machine RACs.

Participant RAC also bears no resemblance of how much credit is actually being granted each day, which is somewhere close to 1000 credit per day.

So RAC seems to be quite meaningless.


Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 565 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 621 - Posted: 27 Sep 2005, 15:11:40 UTC

RAC, "properly" done cannot be added in any shape, manner, or form.

You cannot add the RAC of three computers to get the RAC of an account.

RAC has a time component and so, when a result is posted:

1) the RAC for the computer is calculated based on the Credits and time of "posting"
2) The RAC for the account is calculated based on the Credits and time of "posting"

So, the account has RAC updates that are three times (assuming 3 computers in the account) as often as each of the computers. Since the RAC is updated on the account with shorter intervals the Account RAC should be higher than the sum of the RACs for the three computers ...

In general, I ignore RAC as it is pretty useless though in theory it should be a good metric.
ID: 621 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 631 - Posted: 27 Sep 2005, 15:46:43 UTC - in response to Message 621.  
Last modified: 27 Sep 2005, 15:48:22 UTC

RAC, "properly" done cannot be added in any shape, manner, or form.

You cannot add the RAC of three computers to get the RAC of an account.

RAC has a time component and so, when a result is posted:

1) the RAC for the computer is calculated based on the Credits and time of "posting"
2) The RAC for the account is calculated based on the Credits and time of "posting"

So, the account has RAC updates that are three times (assuming 3 computers in the account) as often as each of the computers. Since the RAC is updated on the account with shorter intervals the Account RAC should be higher than the sum of the RACs for the three computers ...

In general, I ignore RAC as it is pretty useless though in theory it should be a good metric.


Can you actually provide examples of these calculations, using REAL numbers, so someone could understand how this works ?


Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 631 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 633 - Posted: 27 Sep 2005, 15:58:07 UTC - in response to Message 631.  
Last modified: 27 Sep 2005, 15:58:53 UTC

Can you actually provide examples of these calculations, using REAL numbers, so someone could understand how this works ?

I could I suppose, but I am not sure that that is a wise investment of my time ... an example like that, based on experience, would likely take 2-3 days of work to make one that makes sense ... so, I am not sure that I will commit to that ...

The basic formulation is:

RAC(new)=RAC(old)*d(t)+ (1-d(t))*credit(new)

Where d(t) is the decay function, and t is the time (in seconds) since the last RAC recalculation.

d(t)=e^(-ln(2*t)/604800)

void update_average(
    double work_start_time,       // when new work was started
                                    // (or zero if no new work)
    double work,                    // amount of new work
    double half_life,
    double& avg,                    // average work per day (in and out)
    double& avg_time                // when average was last computed
) {
    double now = dtime();

    if (avg_time) {
        // If an average R already exists, imagine that the new work was done
        // entirely between avg_time and now.
        // That gives a rate R'.
        // Replace R with a weighted average of R and R',
        // weighted so that we get the right half-life if R' == 0.
        //
        // But this blows up if avg_time == now; you get 0*(1/0)
        // So consider the limit as diff->0,
        // using the first-order Taylor expansion of
        // exp(x)=1+x+O(x^2).
        // So to the lowest order in diff:
        // weight = 1 - diff ln(2) / half_life
        // so one has
        // avg += (1-weight)*(work/diff_days)
        // avg += [diff*ln(2)/half_life] * (work*SECONDS_PER_DAY/diff)
        // notice that diff cancels out, leaving
        // avg += [ln(2)/half_life] * work*SECONDS_PER_DAY

        double diff, diff_days, weight;

        diff = now - avg_time;
        if (diff 1.e-6) {
            avg += (1-weight)*(work/diff_days);
        } else {
            avg += M_LN2*work*SECONDS_PER_DAY/half_life;
        }
    } else if (work) {
        // If first time, average is just work/duration
        //
        double dd = (now - work_start_time)/SECONDS_PER_DAY;
        avg = work/dd;
    }
    avg_time = now;
}

ID: 633 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile mitro

Send message
Joined: 18 Sep 05
Posts: 6
Credit: 225,313
RAC: 0
Message 644 - Posted: 27 Sep 2005, 17:48:24 UTC
Last modified: 27 Sep 2005, 17:50:01 UTC

Well that formula is good to see. But that still doesn't explain how an account's RAC can be lower than 1 computer on that account.
ID: 644 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Ethan
Volunteer moderator

Send message
Joined: 22 Aug 05
Posts: 286
Credit: 9,304,700
RAC: 0
Message 645 - Posted: 27 Sep 2005, 17:54:38 UTC - in response to Message 644.  

Well that formula is good to see. But that still doesn't explain how an account's RAC can be lower than 1 computer on that account.


My quick reply (correct or not) would be that the overall account's RAC is averaged over a shorter period of time than an individual computer's. If your computer is averaged over a month, and your account over a week, the former will raise and lower more slowly.

ID: 645 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile mitro

Send message
Joined: 18 Sep 05
Posts: 6
Credit: 225,313
RAC: 0
Message 648 - Posted: 27 Sep 2005, 18:59:08 UTC - in response to Message 645.  
Last modified: 27 Sep 2005, 19:01:11 UTC

My quick reply (correct or not) would be that the overall account's RAC is averaged over a shorter period of time than an individual computer's. If your computer is averaged over a month, and your account over a week, the former will raise and lower more slowly.

That sounds about like it to me, although it still makes no sense why the two would have a different time period. :)

OH.... and the #1 & #3 Top computers (based on RAC) are still bogus. And #2 is the real deal. :p
ID: 648 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 649 - Posted: 27 Sep 2005, 19:05:25 UTC - in response to Message 644.  
Last modified: 27 Sep 2005, 19:06:23 UTC

Well that formula is good to see. But that still doesn't explain how an account's RAC can be lower than 1 computer on that account.

The "kicker" in RAC is that the number is updated ONLY when there is a result credited. So, if I run one computer, in theory, the account RAC and the computer RAC will track. There will be minor differences because of the small time interval between updates to the database.

However, when we have two computers posting credit, and one "stops" the other computer and its RAC will be updated along with the Account's RAC. BUT, the mechanism to "decay" the RAC on the idle computer is not running ... it is not running because of the impact on the database (it would require continuous full table scans).

In the situation described above, one RAC will stay "frozen" the others will change. THis could lead one RAC to be higher than the others, forever ...

The second case is harder to "see" and that is why an example would be good, I just don't have time to make up examples ... anyway, the TIME component of the RAC calculation is an exponential function, so, you can get situations where because of the speed of the credit grants may be such that it artificially raises the RAC score.

The easier visualization is the person that drives 70 miles in two hours ... did they drive 35 MPH, or did they do 80 MPH and stop for lunch along the way?

Given the data you cannot tell which is the case. Similar thing with RAC... This is why most of the 'old-timers' that have been doing BOINC since the BOINC Beta days don't use RAC for anything ... and tell you to not look at it ... it is a flawed metric. In theory, it makes tremendous sense. In most cases it is useless because the time quantity "poisons" the data points.

Most statistic sites DO decay the RAC score, but, again, if you roll it up, even "properly" calculated, you cannot use the score on the stat site to also look at the project site. The numbers will ALWAYS be different.

So, I stand my my original statements ... RAC is useless other than bragging rights ...
ID: 649 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile mitro

Send message
Joined: 18 Sep 05
Posts: 6
Credit: 225,313
RAC: 0
Message 650 - Posted: 27 Sep 2005, 19:13:18 UTC
Last modified: 27 Sep 2005, 19:14:13 UTC

Well you have one thing right... the RAC here is useless... it should be a representation of what a user is doing "recently"

I think I'll see how long it takes mine to go back to zero.
ID: 650 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Housing and Food Services

Send message
Joined: 1 Jul 05
Posts: 85
Credit: 155,098,531
RAC: 0
Message 652 - Posted: 27 Sep 2005, 19:19:21 UTC - in response to Message 650.  

Well you have one thing right... the RAC here is useless... it should be a representation of what a user is doing "recently"

I think I'll see how long it takes mine to go back to zero.


They are going to be looking at the way RAC is calculated when David Kim returns from vacation next week. He mentioned they might be turning on redundancy (doing each work unit twice) if the RAC scores are way out of whack. The main downside to that is they cut in half the amount of work they can do.
ID: 652 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 655 - Posted: 27 Sep 2005, 20:20:54 UTC - in response to Message 652.  

Well you have one thing right... the RAC here is useless... it should be a representation of what a user is doing "recently"

I think I'll see how long it takes mine to go back to zero.


They are going to be looking at the way RAC is calculated when David Kim returns from vacation next week. He mentioned they might be turning on redundancy (doing each work unit twice) if the RAC scores are way out of whack. The main downside to that is they cut in half the amount of work they can do.


There's at least two issues in there.

One is that RAC is junk, and bears no resemblance to average work being done.

Credit is also being granted incorrectly without the quorum procedure in place.
Reduncancy is more that just credit validation also - it should also compare the work output to make sure folks are no returning junk.

I'm not sure how a 2 user quorum would work - average the two scores?

The 3 or more quorum enables ridiculously high and/or low claimed credit to be thrown out.


Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 655 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrwizer
Avatar

Send message
Joined: 18 Sep 05
Posts: 23
Credit: 507,085
RAC: 0
Message 657 - Posted: 27 Sep 2005, 20:42:23 UTC - in response to Message 655.  



There's at least two issues in there.

One is that RAC is junk, and bears no resemblance to average work being done.

Credit is also being granted incorrectly without the quorum procedure in place.
Reduncancy is more that just credit validation also - it should also compare the work output to make sure folks are no returning junk.

I'm not sure how a 2 user quorum would work - average the two scores?

The 3 or more quorum enables ridiculously high and/or low claimed credit to be thrown out.



You definitely need three, otherwise how can you determine which result is correct?
ID: 657 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Housing and Food Services

Send message
Joined: 1 Jul 05
Posts: 85
Credit: 155,098,531
RAC: 0
Message 659 - Posted: 27 Sep 2005, 20:52:19 UTC - in response to Message 657.  
Last modified: 27 Sep 2005, 20:52:27 UTC

You definitely need three, otherwise how can you determine which result is correct?


In a previous message the scientists stated that they are able to tell if a scientific result is 'bogus'. They are able to tell if it's correct outside of duplicating work units. . So implementing redundancy sounds like it would only be to 'fix' RAC calculations. . . which would be sad that a small number of participants cause the computing power of the grid to be cut in half or a third so their name is at the top of a list.
ID: 659 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Keith E. Laidig
Volunteer moderator
Project developer
Avatar

Send message
Joined: 1 Jul 05
Posts: 154
Credit: 117,189,961
RAC: 0
Message 660 - Posted: 27 Sep 2005, 21:09:21 UTC - in response to Message 659.  
Last modified: 27 Sep 2005, 21:11:14 UTC


With Rosetta and R@H, the bogus work would be immediately apparent. In the case that the actual protein strcutre isn't known, the R@H project would collect thousands and thousands of possible structure and then undertake a secondary analysis to determine those structures that most likely reflect the actual structure.


Since the 'correct' (or at least better) predicted structures would form an ensemble (or collection) of similar structures very 'near' each other and since it is unlikely that random results would provide the correct result - or anything approaching the correct result and hence this project - just any crap submitted would stick out like a sore thumb.



ID: 660 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrwizer
Avatar

Send message
Joined: 18 Sep 05
Posts: 23
Credit: 507,085
RAC: 0
Message 661 - Posted: 27 Sep 2005, 21:13:52 UTC - in response to Message 659.  

You definitely need three, otherwise how can you determine which result is correct?


In a previous message the scientists stated that they are able to tell if a scientific result is 'bogus'. They are able to tell if it's correct outside of duplicating work units. . So implementing redundancy sounds like it would only be to 'fix' RAC calculations. . . which would be sad that a small number of participants cause the computing power of the grid to be cut in half or a third so their name is at the top of a list.


Not correct in the sense of real science, rather correct from the standpoint or credit given. If you have two results each claiming very different credit amounts, which one do you give? The high or low. This is why you would need three to average or knock out the extreme request.
ID: 661 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 693 - Posted: 28 Sep 2005, 14:04:49 UTC

As it states in the Wiki, the default BOINC software awards the low in that case (redundency=2).

This eliminates the "cheat" of submitting an artifically high claim to inflate scores.

Adding redundency for the sake of adding redundency is not a good move (in *PAUL's* opinion), and if they have a way of ensuring that the answers are correct without it ... it should not be used. But, this implies that the validation does detect error before credit is assigned. Or, that if an error is detected, credit is deducted.

Though redundency does appear to reduce the amount of work done by the project, if it gives higher assurance that the end product is valid, then it is not a loss at all.
ID: 693 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Number crunching : A few things about RAC



©2024 University of Washington
https://www.bakerlab.org