Message boards : Number crunching : Tells us your thoughts on granting credit for large protein, long-running tasks
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · Next
Author | Message |
---|---|
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1675 Credit: 17,739,876 RAC: 22,908 |
This is alot of the Seti mindset of of a single credit system, ie "credit new", for every project and fails to take into account the reason Admins setup and run their projects.It's not a Seti mindset, but BOINC. The whole idea of BOINC, was that using a single manager could allow people to run multiple project just by attaching to them & not having to install a stand alone programme for each & every project, then battle to somehow get them to all share the computer's time with each other. Since (ideally) all projects used the same Credit mechanism then it's possible to compare computation power between projects, and people will do a project based in their work preference, not based on how much it payed out (ie recognised them) for work done. Many projects make a big noise about how many GFLOPs of computation they are doing, but since those values are worked out based on Credit, and so many projects award Credit based on fantasy numbers, the claimed computation performance of many projects is nothing other than pure fantasy with no basis in reality whatsoever. And due to the way Credit New works, even projects that use Credit New without modification, any claims they make of levels of GFLOPs are also invalid (With Seti it was generally estimated that the real computing power done was around 3 times higher than the Credit derived FLOPs count would indicate). Credits are ONLY comparable within a project NOT from project to projectYet next to eliminating cheating, that was one of the main the reasons for having them- comparison between systems & between projects. If giving out more credits than another project encourages cheatingHow does giving out more Credit encourage cheating??? we crunchers give our resources where we choose to for a million different reasons...ie, badges, credits, the science, the humanity of a project, the fact our relatives work for the project, it sounds cool, etc etc ETC.Exactly- people aren't rational and do stupid things, repeatedly. Which is why Credit was meant to be the same across all projects, so the only reason for doing a project was because it interested you (for whatever reason) and not because one project gives more recognition for work done than another (ie Credits). But the fact is people are stupid and irrational, hence Credits are important (just as much so as they really aren't). Grant Darwin NT |
MeeeK Send message Joined: 7 Feb 16 Posts: 31 Credit: 19,737,304 RAC: 0 |
I am running rosetta with 24 threads 24/7/365 (2*Ryzen5 3600). Each Maschine have 32GB of RAM. My idea for solving the discussion abiut 4GB WUs is to work with the data boinc have about our computers. 12 cores and 32 GB means, i can handle 6 4GB WUs and and 6 1GB WUs. In total 32GB used. We all know, that there is alsways enough headroom in each WU. In my use case rosetta could send a 4GB, a 1GB, a 4GB, a 1GB... If computationtime stays the same. Next idea is to make this 4GB kind of priority and send them just in time. Last thought abiut that: Add a multiplier to these 4GB WU-points for compensation. Maybe 1.5 or something lime this. |
CIA Send message Joined: 3 May 07 Posts: 100 Credit: 21,059,812 RAC: 0 |
Why not just create a new project called "RosettaBIG" and be very clear in the project description that it's essentially the same as Rosetta, but intended for machines that can run 24/7 and have lots of RAM. "If you are unsure if your computer is qualified for this project, please join the normal, original Rosetta project instead." You might not get a ton of participants, but the people who do sign up would have big iron machines and are willing to donate lots (24/7) of CPU time. |
MeeeK Send message Joined: 7 Feb 16 Posts: 31 Credit: 19,737,304 RAC: 0 |
Two projects is too much work. Have to run everything twice ( have a look in the server section). And its expensive. I think the idea of making BigWUs as kind of priority when computer ia able to handle should be the easiest. All computers are benchmarked, so it's known what they can handle. In my case, rosetta knows, that i am able to handle 6 BigWUs per computer. They send me 6 and when one is finished, they send a new one with priority. Shouldn't be that complicated, or amni wrong? |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1675 Credit: 17,739,876 RAC: 22,908 |
Shouldn't be that complicated, or amni wrong?Work is allocated based on meeting deadlines & disk space restrictions. Sufficient RAM isn't taken in to account. As things stand, since the original post in this thread the number of large RAM requirement Tasks is an even smaller percentage of the total number of Tasks available than it was before the post was made 3 months ago. Grant Darwin NT |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1675 Credit: 17,739,876 RAC: 22,908 |
Why not just create a new project called "RosettaBIG" and be very clear in the project description that it's essentially the same as Rosetta, but intended for machines that can run 24/7 and have lots of RAM.No need for another project. BOINC supports multiple Applications for a single project, and makes it possible to select which one(s) to run. With Seti, there were 2 different science applications- MultiBeam (MB) listed as Seti@homev8, and AstroPulse (AP) listed as Astro Pulse v7 In the Project preferences were the additional options. Run only the selected applications AstroPulse v7: yes SETI@home v8: yes If no work for selected applications is available, accept work from other applications? yes Selecting Yes for any of the "Run only the selected applications" options means you would get those Tasks whenever they were available. Selecting no for all of them would result in no work. Selecting no for just one of them meant you wouldn't get any work for that application. "If If no work for selected applications is available, accept work from other applications?" is set to Yes, that means you would only get work for the applications that had "Run only the selected applications" set to Yes, unless there was no work of that type presently available, but there was work available for the other application in which case it would then send you some of those. Once work for your "Run only the selected applications" Yes selected application(s) was available again, you would receive it again, and no further work from the other application(s) would be sent to you. Replace AstroPulse v7 and SETI@home v8 with Rosetta, Rosetta mini, Rosetta Max (for any new super huge RAM requirement type Tasks) and you get the general idea. Under Credit New the Credit granted for each application can be the same or different, depending on the configuration values used, with Rosetta's modified Credit mechanism, i don't know. Many years ago the Scheduler logic for work allocation was broken, it may not may not have been fixed at some stage. As it was, i ended up just selecting Yes for all options in order to continue receiving work. Grant Darwin NT |
Remarc Send message Joined: 23 Mar 20 Posts: 14 Credit: 302,773 RAC: 0 |
i reject any other tasks that don't have a label covid I read somewhere what them have few tasks while one pc, all do not reach the hands to add some computers with ryzen and fx |
Falconet Send message Joined: 9 Mar 09 Posts: 353 Credit: 1,227,479 RAC: 2,238 |
They usually have 11k tasks ready to send but recently that has dropped to a couple hundred. Last time this happened there was a competition or something. |
Jackjr Send message Joined: 30 Sep 19 Posts: 1 Credit: 509,189 RAC: 0 |
I am _in favor of Bryn Mawr's suggestion_. I accidentally logged an unfavorable response. |
Stevie G Send message Joined: 15 Dec 18 Posts: 107 Credit: 832,067 RAC: 1,340 |
Normally, credit is granted based on the cumulative reported CPU time per model. And so a fast machine with lots of memory computes more models and gets more credit than an older system. But, in the case of these 4GB WUs, they will not even be sent to machines that do not have at least 4GB of memory (and normally BOINC would only be allowed to use less than 100% of that, so I should say where BOINC is allowed to use at least 4GB). So there will be no struggling Pentium 4s reporting any results to reflect the difficulty in the cumulative average. Comparatively, my computer falls in the slower range, with fewer resources: AuthenticAMD AMD A6-6400K APU with Radeon(tm) HD Graphics [Family 21 Model 19 Stepping 1] (2 processors) AMD AMD Radeon HD 7400/7500/8300/8400 series (Scrapper) (768MB) driver: 1.4.1848 OpenCL: 1.2. and 8 GB RAM. It currently crunches three other projects besides R@H, running 24/7/365. So it takes 8 hours to complete one R@H task. BOINC sent me 16 tasks, all with the same deadline, three days hence on Septmber 17, 2021. There are only 72 hours in those thre days. Right now I have 15 uncompleted tasks, at 8 hours each, totalling 120 hours to complete. Ain't gonna happen. I could do it in maybe 5 days. Also, R@H takes up the greatest portion of my computer's resources by a huge margin, at 1.59GB of the 1TB hard drive. That's 10 times more than all my other projects combined. And it grants less credit than other projects. My fastest R@H task took miraculously only 4.22 hours to complete, for which my account was granted a measly 3.75 credits. By comparison, Milky Way takes 14.33 MB of disk usage and tasks generally take 1.5 to 3 hours to complete . My longest ever Milky Way task took 7.93 hours to complete, with 230.36 credits granted. I know you all think that the work done by R@H is more important than any other BOINC project (although World Community Grid fans would dispute that) and maybe it is. That's not the point. My complaint is not about credits granted, or competition or recognition. That's not why I do this. It's about efficient use of compuer resources and the value of work accomplished. The 3-day deadline is unreasonable. Other projects allow deadlines of a week to ten days and grant more credits. It's not fair expect me to complete 90 hours of work in 72 days, using up the vast majority of my computer's resources, for a pittance in credits. It's almost insulting and does not compare favorably with other projects. Your answer might be that I should get a faster CPU with more cores, a large graphics card. etc. Maybe I will some day. But at the moment, this is what I have. My computer and I provide honest work, slowly, reliably, over the long haul. I'm working with what I've got. But with R@H, it's not eficient use of the effort. S. Gaber |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1675 Credit: 17,739,876 RAC: 22,908 |
So it takes 8 hours to complete one R@H task. BOINC sent me 16 tasks, all with the same deadline, three days hence on Septmber 17, 2021. There are only 72 hours in those thre days. But I now have 15 uncompleted tasks, at 8 hours each, totalling 120 hours to complete. Ain't gonna happen. I could do it in maybe 5 days.Set your cache to 0 (or close to it) That will reduce the likely hood of getting more work than the system can handle. Let the extra Tasks time out- do not abort them. That will allow BOINC to find out there is an issue with getting so much work, and it will adjust itself accordingly. With so few cores, and so many projects, you need to operate with no (or next to no cache ie 0.01 days & 0.01 additional days) to allow BOINC to meet you Resource share settings in a few weeks. Otherwise it will take months to sort itself out. Also, R@H takes up the greatest porotion of my computer's resources by a huge margin, at 1.59GB of the 1TB hard drive. That's 10 times more than all my other projects combined.And? It's not like CPU time which actually costs you money. You've already paid for the storage, it's not costing you anything extra to use it. And you get Credit for the work you do for the project, so what if it needs more storage space space than some other projects? And it grants less credit than other projects. My fastest R@H task took miraculously only 4.22 hours to complete, for which my account was granted 3.75 credits. By comparison, Milky Way takes 14.33 MB of disk usage and tasks generally take 1.5 to 3 hours to complete . My longest ever Milky Way task took 7.93 hours to complete, with 230.36 credits granted.The Cobblestone has a set definition of how much work over what period of time gives 1 Cobblestone of Credit. If projects ignore the definition of the Cobblestone & payout whatever they feel like, there's not much that can be done. And going to battle with higher & higher levels of Credit to attract crunchers would just be stupid in the extreme. My complaint is not about credits granted, or competition or recognition. It's about efficient use of compuer resources and the value of work accomplished??? Credit is what is given in recognition of the work your system does. How else would you determine the value of work accomplished? The 3-day deadline is unreasonable.3 days to do 8 hours of work is not a big ask. And given that the work being done is for medical research, there are probably quite a few people whose lives will depend on the outcome of that research that would consider it to be time sensitive. Yeah, Gravity waves & the like, if it takes a few months to return a result. No big deal. But if a result from Rosetta or a similar project leads to a treatment for cancer or Covid etc, then the sooner the better. Hence the 3 day deadline. Other projects allow deadlines of a week to ten days and grant more credits.Yeah, so what? Deadlines have nothing to do with Credit (although some projects do pay a bonus for getting work back well before the deadline). It's not fair expect me to complete 90 hours of work in 72 days, using up the vast majority of my computer's resources, for a pittance in credits. It's almost insulting and does not compare favorably with other projectsSo since you are more interested in Credit then you would be best served doing work for those projects that pay more Credit. Grant Darwin NT |
Bryn Mawr Send message Joined: 26 Dec 18 Posts: 389 Credit: 12,073,013 RAC: 4,827 |
So it takes 8 hours to complete one R@H task. BOINC sent me 16 tasks, all with the same deadline, three days hence on Septmber 17, 2021. There are only 72 hours in those thre days. Right now I have 15 uncompleted tasks, at 8 hours each, totalling 120 hours to complete. Ain't gonna happen. I could do it in maybe 5 days. Which means that Boinc has supplied exactly the number of tasks that your machine could do in 3 days - 72 hours on each of 2 processors = 144 hours capacity. |
Stevie G Send message Joined: 15 Dec 18 Posts: 107 Credit: 832,067 RAC: 1,340 |
So it takes 8 hours to complete one R@H task. BOINC sent me 16 tasks, all with the same deadline, three days hence on Septmber 17, 2021. There are only 72 hours in those thre days. Right now I have 15 uncompleted tasks, at 8 hours each, totalling 120 hours to complete. Ain't gonna happen. I could do it in maybe 5 days. That would be true if I didn't have three other projects to run. If I do all of those Rosetta tasks on time, my other projects would suffer. |
Bryn Mawr Send message Joined: 26 Dec 18 Posts: 389 Credit: 12,073,013 RAC: 4,827 |
So it takes 8 hours to complete one R@H task. BOINC sent me 16 tasks, all with the same deadline, three days hence on Septmber 17, 2021. There are only 72 hours in those thre days. Right now I have 15 uncompleted tasks, at 8 hours each, totalling 120 hours to complete. Ain't gonna happen. I could do it in maybe 5 days. I agree that Boinc has not taken that into account but I stand by the statement as made. |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1675 Credit: 17,739,876 RAC: 22,908 |
As Rosetta suffers when you do work for the other projects. You have set your Resource share settings, and now BOINC is trying to meet those settings. To do so, it needs to do that much work. Then it will go back to doing your other projects- and none of Rosetta- until Rosetta is owed more processing time again. As the balance of work to be done gets closer to your Resource share settings, then the next time it does some Rosetta work it won't need nearly as many Tasks.So it takes 8 hours to complete one R@H task. BOINC sent me 16 tasks, all with the same deadline, three days hence on Septmber 17, 2021. There are only 72 hours in those thre days. Right now I have 15 uncompleted tasks, at 8 hours each, totalling 120 hours to complete. Ain't gonna happen. I could do it in maybe 5 days. Depending on your Resource share settings, it eventually may get to the point where it only has to do 1 Rosetta task at any given time in order to meet your Resource share requirements. That is the whole point of BOINC- it allows you to do multiple projects- even if your system can only do one at a time it's still possible for you to do work for dozens of projects. It will process the work for each project & over time it will meet your Resource share settings- however the less computing resources you have, the larger your cache & the more projects you do, the longer it will take for things to balance out. And since you only have 2 cores, a relatively large cache setting & 4 active projects it will take BOINC several months for it to do work for each project to eventually meet your Resource share settings. And that's if you don't do any micro-management of things (aborting Tasks, setting & removing No new Tasks. Suspending & re-enabling various projects etc). Grant Darwin NT |
Stevie G Send message Joined: 15 Dec 18 Posts: 107 Credit: 832,067 RAC: 1,340 |
You wrote, "Deadlines have nothing to do with Credit (although some projects do pay a bonus for getting work back well before the deadline)... So since you are more interested in Credit then you would be best served doing work for those projects that pay more Credit[quote] That's not true. As I said, credits, recognition and competition are not my primary motivations for doing BOINC. Citizen participation has made important contributions and advanced the progress of science. I joined S@H almost at its beginning, and ran it 24/7/365 for years on some very slow computers. I'm on my fifth computer now. Formerly, I ran Skynet POGS, till they finished. And Orbits until they folded. I did citizen water quality monitoring in the Chesapeake Bay area 30 years ago. I participate in Globe at Night project. These days, in addition to R@H, I run Milky Way, WCG and Asteroids (whenever they get their act together). My main point was the effect Rosetta was having on those other projects. I aborted any tasks (mostly WCG) that were taking 3 to 5 days to complete. I did as you suggested and changed my cache to 0 and .01, let those projects run out and selected No New Tasks. Right now, my computer is finishing 29 of the 38 tasks Milky Way sent. I will let them run out, cancel No New Tasks and see what happens. S. Gaber |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1675 Credit: 17,739,876 RAC: 22,908 |
That's not true. As I said, credits, recognition and competition are not my primary motivations for doing BOINC.Seriously? This is -exactly- what you typed previously My complaint is not about credits granted, or competition or recognition. It's about efficient use of computer resources and the value of work accomplishedMy response. Credit is what is given in recognition of the work your system does. How else would you determine the value of work accomplished? My main point was the effect Rosetta was having on those other projects.And my point is the effect those other projects have on Rosetta. Rosetta has the same impact on those projects, that those projects have on Rosetta. The fact is that there is no over all impact on other projects by project Whatever. There may be times where it is the only one being worked on, but there will be plenty of times where it isn't being worked on at all (depending of course on what your Resource share settings are). You set your Resource share settings, and then BOINC will do it's best to meet those settings. With your limited number of cores, and multiple projects, and the different deadlines & runtimes between all of the projects the end result is it will take a month or more for Rosetta to be able to meet your Resource share settings. And that is as long as projects don't have issues, and you don't try to micro manage things. You don't just seem to understand the fact that since you only have 2 cores, if you do more than one project then there will be times that the system will be processing one project & one project only in order to meet your Resource share settings between all the projects that you have chosen to do work for. That is simply a basic fact- you have 2 cores, so only 2 Tasks can be processed at any given time. The more projects you have, then the longer it takes to do enough work for each project so that your Resource share settings can be met. And the more you fiddle, the longer it will take. As in months. Multiple months. 3, 4 or more months. Seriously. I aborted any tasks (mostly WCG) that were taking 3 to 5 days to complete.Which as i pointed out before is only going to mess things up even more & take things even longer to be sorted out. The more you fiddle, the longer it takes for your Resource share settings to be met. Don't abort work. Don't set no new Tasks if you do want to do work for that project. If BOINC gets too much work off of one project or another, let them timeout so it then knows that it's an issue. The more you fiddle with things, the longer it will take for them to settle down. I did as you suggested and changed my cache to 0 and .01, let those projects run out and selected No New TasksI did not suggest setting No new tasks. I pointed out that is the wrong thing to do. Setting No new tasks will just make things worse. If you want to do work for a project, then do not set No new Tasks. Let all the projects you want to do work for, get work when BOINC requests it. Reducing the cache will help settle things down sooner, but setting & un-setting No new Tasks will just result in surges of work for one Project or another. Grant Darwin NT |
Stevie G Send message Joined: 15 Dec 18 Posts: 107 Credit: 832,067 RAC: 1,340 |
So why are those projects that you say award excessively high numbers of credits allowed to do so? Doesn't that undermine the legitimacy of the BOINC credit system? |
Stevie G Send message Joined: 15 Dec 18 Posts: 107 Credit: 832,067 RAC: 1,340 |
Seriously? Yes. Seriously. Did you read the history of my participation in citizen science projects that I engaged in without expectation of reward or recognition? |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1675 Credit: 17,739,876 RAC: 22,908 |
So why are those projects that you say award excessively high numbers of credits allowed to do so?Because they don't have to use the official BOINC Credit system, nor stick to the actual definition of what a Cobblestone it. Doesn't that undermine the legitimacy of the BOINC credit system?Yes. Yes i did.Seriously?Yes. Seriously. Did you read the history of my participation in citizen science projects that I engaged in without expectation of reward or recognition? And in the very same paragraph you them complain about your perceived lack of recognition here at Rosetta. And you still don't respond to the question i keep asking you every time you make the statement that recognition isn't important, then complain about the lack of it- if we don't use Credit for recognition of work done, then what do we use? Grant Darwin NT |
Message boards :
Number crunching :
Tells us your thoughts on granting credit for large protein, long-running tasks
©2024 University of Washington
https://www.bakerlab.org