Why does Rosetta do this unusual 8 hour thing?

I suggest you ask someone from the project that.
My personal WAG (Wild Arse Guess)- they feel they can get more useful results done for each Task sent out with their run for time method.

---

Grant
Darwin NT

Peter Hucker wrote

I thought Boinc already did this. If I set up a new machine with Boinc, it downloads a sensible amount. Or is that just a standard amount of tasks set at the project end? For example the huge climate change ones only downloaded one. but projects with small tasks downloaded many.

This seems like a very basic thing that should have been included from the start. I don't understand Boinc programmers at all. And they don't like my attitude, so I won't bother asking.

ALOT of those settings are changeable by the Admins of the Project running thre software, so the basic downloaded software does X but the Admins change it to Y or Z or whatever.

SRBase, for example, seems to send me HUNDREDS AND HUNDREDS of tasks even if I use a 1% resource share on a 6 core pc, it has NOTHING AT ALL to do with my cache settings of 0.05 and 0.05 when I get over 400 hundred tasks that each take over 2 hours to run!!! The only way to stop them is to Suspend the Project but by then I've already got over 200 tasks!!! There are 1 or 2 other Projects that also do the same thing but I haven't run them in a while.

I don't understand why Boinc can't make a good estimate before it's run one. If you look at the properties of a work unit you've downloaded, it tells you how many flops it has to do.

Did you read what i posted?
BOINC's estimates are based on the properties of the Task it downloads. They are supplied by the project. If they aren't close to the actual work to be done then the Estimated time will be off. The value supplied by the project can be updated by BOINC when it contacts the servers- but once again, that is after it has actually done the work. In the case of Rosetta with a target Runtime, they've set the initial Estimate to the default Target CPU runtime.
All the initial Estimates all entirely dependant on the project supplying reasonably accurate FLOPS values for the work to be done.

Boinc also knows how many flops per second your CPU runs at, based on its own benchmarks. So it knows pretty accurately how long those tasks will take.

No it doesn't because benchmarks aren't reality and the efficiency of the application & the types of operations it uses will have a massive impact on the processing capability of the CPU.
This is why as work is done BOINC re-calculates the Estimated completion time (on all other projects).

---

Grant
Darwin NT

Peter Hucker wrote:

Why does Rosetta do this unusual 8 hour thing?

That’s a question for somebody who’s been here a lot longer than I have…

(There’s a reasonable chance the answer is here in these forums, since they go all the way back to the beginning of R@h. Finding it among all the noise might be easier said than done, mind…)

Speculating, I’d guess that it was the easiest way to get pre-existing Rosetta applications to work within the BOINC framework, or that it was a deliberate design choice to ensure predictability of task durations (given the large variety of modelling scenarios that are possible).

Why does Rosetta do this unusual 8 hour thing?

I suggest you ask someone from the project that.
My personal WAG (Wild Arse Guess)- they feel they can get more useful results done for each Task sent out with their run for time method.

Please elaborate on your guess. Surely the amount of work done will be precisely equal either way. If I have a fast computer and they change to the usual method, I'd return them in say 3 hours instead, but so what? The same processing time would be done.

---

Peter Hucker wrote

I thought Boinc already did this. If I set up a new machine with Boinc, it downloads a sensible amount. Or is that just a standard amount of tasks set at the project end? For example the huge climate change ones only downloaded one. but projects with small tasks downloaded many.

This seems like a very basic thing that should have been included from the start. I don't understand Boinc programmers at all. And they don't like my attitude, so I won't bother asking.

ALOT of those settings are changeable by the Admins of the Project running thre software, so the basic downloaded software does X but the Admins change it to Y or Z or whatever.

SRBase, for example, seems to send me HUNDREDS AND HUNDREDS of tasks even if I use a 1% resource share on a 6 core pc, it has NOTHING AT ALL to do with my cache settings of 0.05 and 0.05 when I get over 400 hundred tasks that each take over 2 hours to run!!! The only way to stop them is to Suspend the Project but by then I've already got over 200 tasks!!! There are 1 or 2 other Projects that also do the same thing but I haven't run them in a while.

The resource share setting in Boinc is terrible. If you haven't run SRBase in a while, it will try to run it at 100% for a bit to catch up for some insane reason.

---

I don't understand why Boinc can't make a good estimate before it's run one. If you look at the properties of a work unit you've downloaded, it tells you how many flops it has to do.

Did you read what i posted?
BOINC's estimates are based on the properties of the Task it downloads. They are supplied by the project. If they aren't close to the actual work to be done then the Estimated time will be off. The value supplied by the project can be updated by BOINC when it contacts the servers- but once again, that is after it has actually done the work. In the case of Rosetta with a target Runtime, they've set the initial Estimate to the default Target CPU runtime.
All the initial Estimates all entirely dependant on the project supplying reasonably accurate FLOPS values for the work to be done.

Boinc also knows how many flops per second your CPU runs at, based on its own benchmarks. So it knows pretty accurately how long those tasks will take.

No it doesn't because benchmarks aren't reality and the efficiency of the application & the types of operations it uses will have a massive impact on the processing capability of the CPU.
This is why as work is done BOINC re-calculates the Estimated completion time (on all other projects).

OK so a better idea would be for Boinc to just download one single task per CPU core, run them, then have an estimate to fill the buffer the next time.

---

OK so a better idea would be for Boinc to just download one single task per CPU core, run them, then have an estimate to fill the buffer the next time.

No, the better idea is that projects provide an accurate estimate of the work to be done. If that is not possible, then provide a higher than necessary estimate. That way people will get enough work to keep their system going, even after the first Tasks have completed (if their settings make it possible of course), without them getting more work than they can possibly do, and allow BOINC to determine just how capable their systems are & how much work they can do & need to do to meet project deadlines, cache settings, resource share settings, time the system is able to do work & all the other competing factors that go in to determining what to do and when and how many.


These days, given the capability of modern hardware (even phones!), it would probably be better if the defaults for BOINC & it's projects were 0 cache, do not suspend when system is in use, assume system is on for 4hrs per day, allow it to use up to 75% of RAM & 5GB or 25% of storage (whichever is less).
If people want different settings, they can do so. But for initial defaults these would allow new system to process significant amounts of work, without impacting on the system's performance, allow Resource share to be honoured much sooner than present, and avoid systems initially getting more work than they can handle.

---

Grant
Darwin NT

OK so a better idea would be for Boinc to just download one single task per CPU core, run them, then have an estimate to fill the buffer the next time.

No, the better idea is that projects provide an accurate estimate of the work to be done. If that is not possible, then provide a higher than necessary estimate. That way people will get enough work to keep their system going, even after the first Tasks have completed (if their settings make it possible of course), without them getting more work than they can possibly do, and allow BOINC to determine just how capable their systems are & how much work they can do & need to do to meet project deadlines, cache settings, resource share settings, time the system is able to do work & all the other competing factors that go in to determining what to do and when and how many.


These days, given the capability of modern hardware (even phones!), it would probably be better if the defaults for BOINC & it's projects were 0 cache, do not suspend when system is in use, assume system is on for 4hrs per day, allow it to use up to 75% of RAM & 5GB or 25% of storage (whichever is less).
If people want different settings, they can do so. But for initial defaults these would allow new system to process significant amounts of work, without impacting on the system's performance, allow Resource share to be honoured much sooner than present, and avoid systems initially getting more work than they can handle.

Agreed, the defaults could be improved. I also think on installation, the user should be presented with a nice GUI interface asking if they want to change settings, with nice simple explanations as to what they do. And nothing should require those pesky config files! Stuff that non-advanced users should leave alone, just put it in an "advanced" tab, with warnings not to touch unless you know what you're doing, and also give sensible limits and recommendations.

---

I don't like changes in the fan noise from my computer. This means that whenever a task finishes, a new task should be ready to start.

This means that the client should keep track of average download speed for each BOINC project it is connected to, and the progress of each task, so it can estimate how soon it needs to start downloading another task to replace to one close to finishing. This gets more complicated if more than one task is running and close to finishing.

Some users don't have reliable internet connections 24 hours a day, and some BOINC projects shut down their servers a few hours every day. In these situations, a buffer of tasks not started yet is needed. I would prefer separate buffer settings for each client, for each BOINC project, to handle differences in these problems. BOINC doesn't allow that, though.

I remember seeing one BOINC project where giving a reliable estimate of the floating point operations in advance was impossible. The project could have calculated the maximum possible number (which would often take years to run). However, their application program had a large number of short cuts it could sometimes take and sometimes not. With use of these short cuts, total run time per task was almost always below one day. In other words, calculating the maximum run time was a task wasn't useful. Instead, it was common to have checkpoints for the tasks, so they could be restarted after an interruption. If the checkpoint method is sufficiently well designed, the checkpoint files can even be sent to a different computer and resumed there.

I suspect that both Rosetta@home and Ralph@home (the alpha test site for Rosetta@home) have each task divided into steps, called decoys, where useful checkpoints can be written between each decoy and the next. If it doesn't complete all of the decoys, it sends the checkpoint files back as part of its output. There are usually, but not always, 100 decoys available per task. It used to be that the first decoy was always the same for all tasks, and only intended to check if the computer was working properly and measure its computation speed. The other 99 were either improvement steps from just one starting point, or one step each for 99 different starting points.

This meant that a piece of work to be done could be, if necessary, divided into a series of workunits, with all but the first built from the results of the previous workunit in the series.

However, I've seen a recent workunit reported as getting useful results from the first decoy, even though it did not go on to a second decoy.

Is this still what happens? The Rosetta@home staff hasn't been posting enough information here lately to help me guess.

Note that a task will not always shut down after exactly 8 hours, or whatever the target time is for your computer. Instead, after each decoy, it estimates whether it will be able to complete another decoy within the target time. If it decides it can, it will start another decoy. If it decides it cannot, it will shut down. This avoids wasting CPU time on decoys that don't complete.

Do we need checkpoints? I remember a very long job, with no checkpoints, running on a computer at one of my previous jobs, Every time that computer was shut down for software updates, the very long job would have to be started again, from its beginning. Finally, we were told to delay all software updates on that computer until the very long job finished. It then took three more months for the job to finish. After that, our IT department told the company the program came from that they would not renew the license on the program unless they got a new version with checkpoints. Since the license fee for the program was about $100,000 a year, it didn't take long for the new version with checkpoints to arrive.

I also think on installation, the user should be presented with a nice GUI interface asking if they want to change settings, with nice simple explanations as to what they do.

Why create more issues than there already are?
If they want to change things, then they should do what everyone has done since computers existed. Make use the help files and documentation.

And nothing should require those pesky config files!

There is no way on the plant you could have a menu system that supports all the things that people want to do with their systems.
The Simple view is for those that want to set things up & then let them do their thing. The Advanced view allows people more options with what BOINC can & can't do and when.
If you want even more than that, the configuration files are the best option.

Stuff that non-advanced users should leave alone, just put it in an "advanced" tab, with warnings not to touch unless you know what you're doing, and also give sensible limits and recommendations.

BOINC already has that.
View. Advanced view. Options, Computing options (etc).
Which is OK if you have only the one computer. If you have more than that, then use the web based preferences.

Making something idiot proof just results in a greater level of idiocy, and greater levels of frustration for everyone else.

---

Grant
Darwin NT

I don't like changes in the fan noise from my computer. This means that whenever a task finishes, a new task should be ready to start.

How odd - I quite enjoy the break from the endless noise when it calms down for a bit.

Do we need checkpoints? I remember a very long job, with no checkpoints, running on a computer at one of my previous jobs, Every time that computer was shut down for software updates, the very long job would have to be started again, from its beginning. Finally, we were told to delay all software updates on that computer until the very long job finished. It then took three more months for the job to finish. After that, our IT department told the company the program came from that they would not renew the license on the program unless they got a new version with checkpoints. Since the license fee for the program was about $100,000 a year, it didn't take long for the new version with checkpoints to arrive.

Does Boinc wait for a checkpoint before switching over to a task from a different project, mid WU?

---

I also think on installation, the user should be presented with a nice GUI interface asking if they want to change settings, with nice simple explanations as to what they do.

Why create more issues than there already are?
If they want to change things, then they should do what everyone has done since computers existed. Make use the help files and documentation.

That's not how decent programs work. Boinc is the only program I know of that still uses ini files. Those went out last century.

And nothing should require those pesky config files!

There is no way on the plant you could have a menu system that supports all the things that people want to do with their systems.
The Simple view is for those that want to set things up & then let them do their thing. The Advanced view allows people more options with what BOINC can & can't do and when.
If you want even more than that, the configuration files are the best option.

There is a finite list of things you can do in the config files, those could easily be in a GUI.

Stuff that non-advanced users should leave alone, just put it in an "advanced" tab, with warnings not to touch unless you know what you're doing, and also give sensible limits and recommendations.

BOINC already has that.
View. Advanced view. Options, Computing options (etc).
Which is OK if you have only the one computer. If you have more than that, then use the web based preferences.

Another mess - preferences on the server and preferences on the computer and users forgetting what they set where.

Making something idiot proof just results in a greater level of idiocy, and greater levels of frustration for everyone else.

Well what we've got at the moment is forums full of people asking how to change settings in a system that belongs on a 486 in DOS.

---

Some users don't have reliable internet connections 24 hours a day, and some BOINC projects shut down their servers a few hours every day. In these situations, a buffer of tasks not started yet is needed. I would prefer separate buffer settings for each client, for each BOINC project, to handle differences in these problems. BOINC doesn't allow that, though.

And that is a perfect example of why it just isn't possible to have a GUI to allow people to set every possible combination of what they might want to achieve.
The idea behind BOINC was to allow a single client to manage multiple projects, and allow people to allow projects to be done in accordance with their personal preferences. And BOINC does make that possible, there's no need to cache more than a few hours work as BOINC will get new to run work before it runs out of current work, and it will do so according to Resource share settings- so there isn't any need for different caches for different projects.

But if people want to go that route, much of it can be done with configuration files.

---

Grant
Darwin NT

[snip]

Does Boinc wait for a checkpoint before switching over to a task from a different project, mid WU?

It tries to do so. However, when it cannot, it will just leave the incomplete task in memory until it can switch back, if there is enough memory available to allow this. When this happens, the main thing to has to do for the restart is recalculate all of the time delays that were in effect for the incomplete task.

Some users don't have reliable internet connections 24 hours a day, and some BOINC projects shut down their servers a few hours every day. In these situations, a buffer of tasks not started yet is needed. I would prefer separate buffer settings for each client, for each BOINC project, to handle differences in these problems. BOINC doesn't allow that, though.

And that is a perfect example of why it just isn't possible to have a GUI to allow people to set every possible combination of what they might want to achieve.
The idea behind BOINC was to allow a single client to manage multiple projects, and allow people to allow projects to be done in accordance with their personal preferences. And BOINC does make that possible, there's no need to cache more than a few hours work as BOINC will get new to run work before it runs out of current work, and it will do so according to Resource share settings- so there isn't any need for different caches for different projects.

But if people want to go that route, much of it can be done with configuration files.

Everything currently in the config files could be placed in a GUI.

---

[snip]

Does Boinc wait for a checkpoint before switching over to a task from a different project, mid WU?

It tries to do so. However, when it cannot, it will just leave the incomplete task in memory until it can switch back, if there is enough memory available to allow this. When this happens, the main thing to has to do for the restart is recalculate all of the time delays that were in effect for the incomplete task.

I have "leave application in memory while suspended" unticked. Does that screw up the above?

I set it that way due to a shortage of RAM.

---

I also think on installation, the user should be presented with a nice GUI interface asking if they want to change settings, with nice simple explanations as to what they do.

Why create more issues than there already are?
If they want to change things, then they should do what everyone has done since computers existed. Make use the help files and documentation.

That's not how decent programs work. Boinc is the only program I know of that still uses ini files. Those went out last century.

The best I can tell, many programs still use such files, but don't give them names ending with .ini.

And nothing should require those pesky config files!

There is no way on the plant you could have a menu system that supports all the things that people want to do with their systems.
The Simple view is for those that want to set things up & then let them do their thing. The Advanced view allows people more options with what BOINC can & can't do and when.
If you want even more than that, the configuration files are the best option.

There is a finite list of things you can do in the config files, those could easily be in a GUI.

If you don't mind an EXTREMELY large GUI, with a new section downloaded whenever you connect to another BOINC project, to handle the settings specific to that project.

That's not how decent programs work. Boinc is the only program I know of that still uses ini files. Those went out last century.

They never went out, they just changed names, or location. The only difference is that most of the settings can be done using a GUI, although in many cases advanced settings still require manually editing a file.
Welcome to computers.

There is a finite list of things you can do in the config files, those could easily be in a GUI.

Yeah, just like Microsoft Word (and many other programmes that have been around forever), that have so many features & functions and options that people are still requesting things that Word has supported for years.
They just couldn't find them looking through the now massive menus, and didn't take the time to look at the help file.
It is not a suitable or viable option.

Everything currently in the config files could be placed in a GUI.

Not one that could actually be be used.

Another mess - preferences on the server and preferences on the computer and users forgetting what they set where.

Just because people are forgetfull doesn't make it a mess.
Here you are advocating for things that are orders of magnitude more complicated, yet you consider 2 options- web based & locally based preferences, a mess.
Please make up your mind.

Well what we've got at the moment is forums full of people asking how to change settings in a system that belongs on a 486 in DOS.

Then you obviously can't remember what DOS was like.
There is no comparison.

---

Grant
Darwin NT

I have "leave application in memory while suspended" unticked. Does that screw up the above?

I set it that way due to a shortage of RAM.

Yes, the statement that when it can't complete it keeps it in memory is only true if you have that setting checkmarked.

If a task is kept "in memory", but is not running, it will get paged out to the swap file, so long as swap space is available.

---

Rosetta Moderator: Mod.Sense