Credit always low

Message boards : Number crunching : Credit always low

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 65893 - Posted: 29 Apr 2010, 15:37:37 UTC

Decoys are the finished product that the project needs. If one machine can produce one in an hour and another machine takes 90 minutes to produce one... they both produced the same amount of work and granting credit per decoy assures they get the same credit for the same result. All without needing to have a second person run each and every task just to confirm credit. Each decoy is different, that's the whole point. So it is not possible to say with certainty ahead of time what the actual computational requirement of a given decoy will be. Averaging the reported credit per decoy is the most appropriate system the team has come up with. It allows for variation both in the machines running the work, and in the work being run.

Point of order, the recommendation is that machines running Rosetta should have a minimum of 512MB of memory. In general, that allows some margin for an operating system and one task to run. So when you extend to more then one core, the expectation would be something less then 512MB for each additional core (because any additional requirement by the operating system will be minimal). No specific per core recommendation is given, but Jochen's approach of reviewing usage of actual running tasks is a good one. Just keep in mind that the requirements do vary by type of task being performed.
Rosetta Moderator: Mod.Sense
ID: 65893 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65894 - Posted: 29 Apr 2010, 20:14:15 UTC

I see, why this approach was chosen.

But looking at this WU, which was a very short running one, with lots of granted credits, it is just all about if you get the right WUs at the right time. This is the big problem I see in this approach.
There are two things one has to do, to gain a certain influence on the granted credits (posting exactly what this is might be considered 'annoying', so I won't ;) )

But anyway, since I can not come up with a better idea, I accept it the way it is.

BTT: As well I noticed, that my 920 now is getting granted approx. 10 more cedits per WU. I did not do any changes anything to the computer, though...

Jochen
ID: 65894 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 65899 - Posted: 30 Apr 2010, 4:37:24 UTC

Yes, the variability is the one major point the credit system doesn't address. It makes small sample sizes difficult to compare, but the system also assures that it all averages out over time. I guess I'm just saying that since the whole thing is based on averages of thousands of reports, your odds of being unlucky are equal to those of being lucky.
Rosetta Moderator: Mod.Sense
ID: 65899 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65954 - Posted: 4 May 2010, 12:41:30 UTC

Now this is a ridicilous result: https://boinc.bakerlab.org/rosetta/result.php?resultid=335073708
28 credits for 7 hours of calculation.

I guess I will start aborting long running models again. This is just a waste of time...

Jochen
ID: 65954 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Snags

Send message
Joined: 22 Feb 07
Posts: 198
Credit: 2,729,042
RAC: 7
Message 65986 - Posted: 5 May 2010, 17:59:24 UTC - in response to Message 65954.  

Now this is a ridicilous result: https://boinc.bakerlab.org/rosetta/result.php?resultid=335073708
28 credits for 7 hours of calculation.

I guess I will start aborting long running models again. This is just a waste of time...

Jochen


What's a waste of time? Continuing to run models that go past your target CPU time? I disagree quite strongly. If a workunit is well and truly stuck, eating up cpu time without making any progress (not always a simple matter to determine) or if it continues to run longer than target + 4 hours (CPU time NOT elasped time, for those reading along) without the watchdog kicking in and ending things, then yea, put it out of its misery and move on.
But not the workunit you've linked to. That appears to be one of the triage types Sarel described here back in January. I'll quote his post in full:
Hello,

The credit variation that you're seeing is a characteristic of the *gbn* simulations. In addition to jobs that show low per-WU credit, you will also see high per-WU credit, which, over time, averages out. I've commented on this variability in the ROSETTA 2.05 and in the protein-interface design threads, and briefly the idea is that in order to save time, these simulations try to identify simulations that would not yield good models early and triage those (short runs). On the other hand, those simulations that seem to yield good results will be tried again and again to improve them (long runs). Over time, any particular machine will receive jobs of both kinds and its credit will be appropriately averaged. The bottomline is that though at any given moment it might seem that the credit is unevenly allocated, over a longer period of time it smooths out.

That said, I'm testing now a second type of protocol which is efficient even if simulations are not triaged early (the jobs will be named "tyrsim"). It's a different (complementary) approach to the problem of finding a binder and I'm waiting to see how it behaves on ROSETTA @ HOME. If it allocates credit more evenly over time we will shift our focus to using this protocol.

Thanks, Sarel.


I may have imagined it, as I can't find it at the moment, but I thought there was another post describing similar WUs something like this: The workunit employs a filter to each decoy it begins. It may run through quite a few decoys before it gets one that passes the filter but when it does it continues working with that decoy to another predescribed end even if that takes it past the target run time. I think this is what happened in your workunit. Which means (if my description is accurate) it wasn't a waste of time at all and in fact would be a waste only if you had aborted it.

It's easy to see how looking at this one WU could lead one to think the credit scheme is unfair. As mod.sense said, this will even out over the longer term; you will eventually get every kind: long, short, exciting and dull.

And you will get every other type of workunit. Not every one performs any kind of triage as described above, some run every decoy through the same number of steps in (roughly) the same amount of time.

Which brings up a point I think gets lost here often. Rosetta puts out a large variety of workunits. They don't just study many different proteins, they use a number of different strategies to do so. And they are always looking for new strategies and to improve the current ones. Well, after all, that's their primary remit, at least as I understand it. Anyone seeking a new credit scheme would have to keep this ever changing variety in mind.

(It would be nice to have a web page listing all the projects, proteins and strategies being explored. A catalog of Rosetta as it were. Perhaps a job for a team of eager undergraduates. Whoever compiled such a catalog would certainly learn a lot about the work being done.)

To get back on topic, judging any particular workunit as a waste of time based on credit would suggest that credit has a relationship to the value of the work done. It doesn't.
And if you want to determine if a particular machine is well or badly suited to run rosetta, credit is, at best, only the starting point of a complex investigation. As evidenced by this and many similar threads.

Snags
ID: 65986 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 65988 - Posted: 5 May 2010, 18:34:04 UTC

Just looking at the credits, aborting those WUs is a gain. ;)

Jochen
ID: 65988 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Snags

Send message
Joined: 22 Feb 07
Posts: 198
Credit: 2,729,042
RAC: 7
Message 66001 - Posted: 6 May 2010, 14:04:54 UTC - in response to Message 65988.  

Just looking at the credits, aborting those WUs is a gain. ;)

Jochen

I haven't read a credit thread in a while; is this where I'm supposed to start questioning your character and calling you names? :)

Snags
ID: 66001 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jochen

Send message
Joined: 6 Jun 06
Posts: 133
Credit: 3,847,433
RAC: 0
Message 66008 - Posted: 6 May 2010, 17:33:45 UTC - in response to Message 66001.  

I haven't read a credit thread in a while; is this where I'm supposed to start questioning your character and calling you names? :)

Well this thread is labeled 'Credit always low', so if there's a place... And most likely I was involved in the last credit thread as well.

You can call me names, that's what I probably deserve. But don't start questioning my character, before you know me. :) In most cases, I'm reasonable and liberal. But when it comes to BOINC and credits...

It's not me being a credit-whore by nature, it's BOINC that turned me into a credit-whore... I was happy the way it was at SETI, a long time before BOINC was released. I left SETI, when everybody using Crunch3er's optimized client was called a cheater. It's impossible to build optimized clients for Rosetta. That's why I came here. (Beside the usual reasons why one crunches Rosetta WUs. <- just because you are already about to question my character. ;) )


Jochen
ID: 66008 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sorceress
Avatar

Send message
Joined: 7 Feb 09
Posts: 7
Credit: 20,370
RAC: 0
Message 66449 - Posted: 3 Jun 2010, 23:18:16 UTC
Last modified: 3 Jun 2010, 23:43:16 UTC

(Hint) IT'S NOT MONEY!!

I have followed this thread and I believe that over time, Rosetta does not compensate its user's well for the work that is done.

Overall, it's very stingy.

The 'nickle and dime' mentality used here is laughably 'non-gratis'. 51 credits for 13,000 sec work is downright robbery. Period!

The point the gurus miss is that credits are ALL the rewards we have for the help we provide. Credits don't cost anything, no money out of the pocket and they aren't worth anything, you can't buy a cup of coffee with them, nor are they edible. It is expensive to run a computer 24hrs p/day, so It would be nice to be reasonably compensated for doing so. There is no reason for the credit rewards to be so low. (It's NOT MONEY people!) 13,000 sec of time for any work done should be worth 150 or more credits, no matter the work unit type, the computer MO or any other ridiculous reason. So why is Rosetta, like many of the projects, so adverse to a GENEROUS reward system? A few of the projects seem to understand 'reward system' and have no problem compensating its user well. Let Roseeta be one of them!! Raise the bar!
ID: 66449 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
transient
Avatar

Send message
Joined: 30 Sep 06
Posts: 376
Credit: 10,836,395
RAC: 0
Message 66453 - Posted: 4 Jun 2010, 8:52:11 UTC

Although credits are not money they are apparently perceived as objects of value. Not granting as much credits you might wish can keep the 'value' of a credit from inflation. Another reason to restrict the 'value' is to keep it in line with other BOINC projects.

Although the 'value' of a credit may need adjusting on the Rosetta project, some very low credits in comparison to run-time are not because the project is 'stingy'. In that case it might just be the 'luck' of the draw.
ID: 66453 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1887
Credit: 6,129,451
RAC: 5
Message 66454 - Posted: 4 Jun 2010, 11:10:36 UTC - in response to Message 66449.  

(Hint) IT'S NOT MONEY!!

I have followed this thread and I believe that over time, Rosetta does not compensate its user's well for the work that is done.

Overall, it's very stingy.

The 'nickle and dime' mentality used here is laughably 'non-gratis'. 51 credits for 13,000 sec work is downright robbery. Period!

The point the gurus miss is that credits are ALL the rewards we have for the help we provide. Credits don't cost anything, no money out of the pocket and they aren't worth anything, you can't buy a cup of coffee with them, nor are they edible. It is expensive to run a computer 24hrs p/day, so It would be nice to be reasonably compensated for doing so. There is no reason for the credit rewards to be so low. (It's NOT MONEY people!) 13,000 sec of time for any work done should be worth 150 or more credits, no matter the work unit type, the computer MO or any other ridiculous reason. So why is Rosetta, like many of the projects, so adverse to a GENEROUS reward system? A few of the projects seem to understand 'reward system' and have no problem compensating its user well. Let Roseeta be one of them!! Raise the bar!


Another 'theory' is that if you keep the credits low the 'credit whores' won't come and make the server work extra hard. Just the people that 'believe' in the idea of your project will come and since they are always going to be here the server load stays fairly steady. More credits means more people meaning more work for the project and that can mean more monetary expenditures. Credits ARE low here, but they may plan it that way.
ID: 66454 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 66459 - Posted: 4 Jun 2010, 13:37:04 UTC

I will not be drawn in to a credit value discussion, but I did want to point out that if you believe the credits are truly your only reward and return on investment, then you are missing the point of Rosetta@home. The point being that you have helped conduct research that may save your life some day. And when there is another worldwide pandemic or epidemic, indeed it may some day prevent a virus from killing 30% of the world's population the way the black plague did.

The plague was a bacterium not a virus, so small pox is perhaps a better example. Read some of the information on small pox and ask yourself how prepared mankind is to attack future viruses; especially when the doctors always seem to say "we just have to let it run it's course", "it's a virus, there's no specific treatment for it" when a virus has infected your body. Even the flu viruses that infect a major portion of the population each year are not currently addressed very well by current medicine. We produce vaccines, but not always in sufficient quantities, or in time to help people. And even with the vaccine, you can still get the flu, it's just a different strain then the vaccine was created to address. And H1N1 was a rather telling demonstration of how ill-prepared mankind is to address such things without new techniques.
Rosetta Moderator: Mod.Sense
ID: 66459 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1887
Credit: 6,129,451
RAC: 5
Message 66471 - Posted: 5 Jun 2010, 11:47:21 UTC - in response to Message 66459.  

I will not be drawn in to a credit value discussion, but I did want to point out that if you believe the credits are truly your only reward and return on investment, then you are missing the point of Rosetta@home. The point being that you have helped conduct research that may save your life some day. And when there is another worldwide pandemic or epidemic, indeed it may some day prevent a virus from killing 30% of the world's population the way the black plague did.

The plague was a bacterium not a virus, so small pox is perhaps a better example. Read some of the information on small pox and ask yourself how prepared mankind is to attack future viruses; especially when the doctors always seem to say "we just have to let it run it's course", "it's a virus, there's no specific treatment for it" when a virus has infected your body. Even the flu viruses that infect a major portion of the population each year are not currently addressed very well by current medicine. We produce vaccines, but not always in sufficient quantities, or in time to help people. And even with the vaccine, you can still get the flu, it's just a different strain then the vaccine was created to address. And H1N1 was a rather telling demonstration of how ill-prepared mankind is to address such things without new techniques.


VERY well put!
ID: 66471 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Sorceress
Avatar

Send message
Joined: 7 Feb 09
Posts: 7
Credit: 20,370
RAC: 0
Message 66472 - Posted: 5 Jun 2010, 11:52:54 UTC - in response to Message 66459.  
Last modified: 5 Jun 2010, 12:14:07 UTC

I will not be drawn in to a credit value discussion, but I did want to point out that if you believe the credits are truly your only reward and return on investment, then you are missing the point of Rosetta@home. The point being that you have helped conduct research that may save your life some day. And when there is another worldwide pandemic or epidemic, indeed it may some day prevent a virus from killing 30% of the world's population the way the black plague did.


While I am not a scientist, I am attached to this project BECAUSE of the reasons you pointed out. BUT, if I(we) are going to help you do this 'important' research then you should reward us far better than you do. The credit reward system you now have in place is too low for the work done. Example: After a year and a half of work for this project, all I have to show for it is 14,000 credits. In contrast, I have been in DNET for less that two months and have 40,000 credits, in Collatz less than a year and have 360,000, all using the same computer. Is their 'work' more important than yours?? This is MY point.

Since credits are not 'real' money out of your pocket, it would not cost you much(if any) to appoint a higher credit return for our time and effort. Instead of 50 credits allowed, make it 150. Instead of 100, make it 300 hundred. If I never find a cure for some 'plague', then all I DO have for reward is the 'credits' for the use of my computer time. Your credits do not pay for the electricity this computer uses, but a better credit reward system would make it easier for ME to. Don't be so stingy! Ante up on the credits! Make me smile :)
ID: 66472 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chris Holvenstot
Avatar

Send message
Joined: 2 May 10
Posts: 220
Credit: 9,106,918
RAC: 0
Message 66474 - Posted: 5 Jun 2010, 13:44:02 UTC

Sorceress - I understand what you are saying but I am not sure how it could be implemented in the bigger "World of BOINC" without some world governing body (sayest the the rotund one speaking only slightly tongue in cheek)

What effect would it have if the good people running this project were to decide that since credits don't "cost" them anything, they could just wave their magician's want and triple the amount of credit granted per CPU cycle.

Well, for a short time Rosetta would be more competitive and would attract a few more participants, but how long would it take before SETI or one of the other projects decided "we can do them one better" and raise the amount of credit they award per unit of work.

Things would quickly get out of hand. It would be like the "dot com" bubble all over again.

What would you suggest? A certain fixed amount of credit per hour of CPU tie based on the specific chip implemented might be one way to go, but that would have to be built into the core BOINC routines and no doubt you would hear screams of pain from some of the SETI participants who run exotic cooling systems to allow them to over clock to the "n-th" degree.

I don't think that there is any "completely fair" way to grant credits across projects when the work done by those projects differs so greatly. You will always have one project which seems to grant bit more credit than another for what is perceived to be the same mount of work.

I just look at credits as being a benchmark as to how well I am doing within a project and as a measure of what I am accomplishing for the project. And with the credit structure they have setup here it can also serve as a flag when something is not working right (see the long running work unit discussion)

So I suggest you pick your project according to what is important to you - there are many worthy BOINC projects and I am not going to make a judgment if finding ET is more important than climate prediction or prime numbers - and then just settle in and enjoy being a part of a community working towards a common goal.

Have a great day.

ID: 66474 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 66476 - Posted: 5 Jun 2010, 17:10:27 UTC
Last modified: 5 Jun 2010, 17:12:42 UTC

I am not a scientist either, nor a member of the Project Team. I am a volunteer user with several years of armchair knowledge and review of the information provided by the scientists. My only reward is the belief that I am leaving the world a little bit better place then it was when I arrived, and that I am doing what little I am able to do to help those that have the knowledge to conduct this research. My credits and stats (earned under another user ID) are not significant in the big scheme of life.

I would assert that your credit comparisons are very similar to having a million Yen rather then a million US dollars. Some credits have higher true value then others. Rosetta credits are more valuable then credits from some other projects.

The credit system is rather difficult to describe, but is in fact truly rooted in the BOINC credit system and benchmarks. Some variation between specific models is unavoidable, but all other variations between machine abilities and different types of tasks and different application versions over time are fully reflected and accounted for within the current system, so it is actually extremely fair as Rosetta@home users earn and compare credits between different processors with different cache sizes and memory available.
Rosetta Moderator: Mod.Sense
ID: 66476 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Chilean
Avatar

Send message
Joined: 16 Oct 05
Posts: 711
Credit: 26,669,447
RAC: 1
Message 66478 - Posted: 5 Jun 2010, 19:35:16 UTC - in response to Message 66449.  
Last modified: 5 Jun 2010, 19:40:05 UTC

I would assert that your credit comparisons are very similar to having a million Yen rather then a million US dollars. Some credits have higher true value then others. Rosetta credits are more valuable then credits from some other projects.

The credit system is rather difficult to describe, but is in fact truly rooted in the BOINC credit system and benchmarks. Some variation between specific models is unavoidable, but all other variations between machine abilities and different types of tasks and different application versions over time are fully reflected and accounted for within the current system, so it is actually extremely fair as Rosetta@home users earn and compare credits between different processors with different cache sizes and memory available.


I was just about to day that.
ID: 66478 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 1916
Credit: 35,478,949
RAC: 1,330
Message 66480 - Posted: 5 Jun 2010, 23:12:02 UTC

If credits were awarded for value of contribution, Seti would award negative credits. That seems fair to me.
ID: 66480 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1887
Credit: 6,129,451
RAC: 5
Message 66489 - Posted: 6 Jun 2010, 12:49:59 UTC - in response to Message 66480.  

If credits were awarded for value of contribution, Seti would award negative credits. That seems fair to me.


This thread could get really long if we started bashing Seti. ;-)))
ID: 66489 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
JackOnTheRoxs

Send message
Joined: 28 Jun 09
Posts: 3
Credit: 1,925,358
RAC: 0
Message 66494 - Posted: 6 Jun 2010, 16:02:59 UTC - in response to Message 66489.  

If credits were awarded for value of contribution, Seti would award negative credits. That seems fair to me.


This thread could get really long if we started bashing Seti. ;-)))



In my opinion, the credit received and accumulated here on Rosetta, is only relavent when compared to the other participants accumulations here on Rosetta. It cannot be compared to a Seti accumulation or other BOINC project.

Of course I run all Intel CPU's, with lots of RAM and large L2 cache's, so I almost always get more granted credit than claimed ;-)

My primary workhorse is an Intel Q9650 (Quad 3.0 Ghz - Non OC'd - 12 Meg L2 and 8 Gigs of RAM running Windows 7 64 Bit. It runs Rosetta 100% 24x7x365 and returns a RAC of ~2500 per day.

I'll be building an I7 950 for my daughters graduation gift in a few weeks. It will be interesting to see how it compares, given the slight disparity in clock speed, twice the "cores" due to hyperthreading, but half (actually quarter if you factor hyperthreading) the L2 Cache.

Yes, I know, the I7 and the Core2Quad are different architechtures ;-)

Now if I wanted to be a credit whore, I would crank up the other two Q9650's to full steam and my daughters I7 up for a burn in. I imagine I could easily crack 12,000.

I keep track of my accumulation and RAC, and compare it to the others around me to keep it interesting, but I crunch here because I hope the research makes a difference.
ID: 66494 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Credit always low



©2023 University of Washington
https://www.bakerlab.org