Please Add Opt-Out Option for Beta Workunits

Message boards : Number crunching : Please Add Opt-Out Option for Beta Workunits

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Major Tom MIB
Avatar

Send message
Joined: 1 Jul 06
Posts: 7
Credit: 128,300
RAC: 0
Message 46819 - Posted: 23 Sep 2007, 4:04:44 UTC

or how can I opt-out of beta work units?
ID: 46819 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 46820 - Posted: 23 Sep 2007, 4:14:26 UTC
Last modified: 23 Sep 2007, 4:15:26 UTC

The true beta work is done on Ralph. Is there some specific reason you feel an opt-out option is necessary?
Rosetta Moderator: Mod.Sense
ID: 46820 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Major Tom MIB
Avatar

Send message
Joined: 1 Jul 06
Posts: 7
Credit: 128,300
RAC: 0
Message 46821 - Posted: 23 Sep 2007, 4:23:27 UTC

I'm running the latest version of Boinc and I'm attempting to track down where there are problems with workunits staying in memory and a few other weird things, so it would be nice if I could only run 'known stable' workunits. If it 'can't happen', then I'll have to hold off on Rosetta work units for a while--they're dropping like flies :(
ID: 46821 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
zombie67 [MM]
Avatar

Send message
Joined: 11 Feb 06
Posts: 316
Credit: 6,589,590
RAC: 317
Message 46824 - Posted: 23 Sep 2007, 5:22:31 UTC
Last modified: 23 Sep 2007, 5:23:58 UTC

If beta work is done on RALPH, then why do we see beta work here? With RALPH, there should never been the need for "problem with..." threads here.

A side note: 5.80 clearly has problems, so why is it already here?
Reno, NV
Team: SETI.USA
ID: 46824 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 46825 - Posted: 23 Sep 2007, 5:28:19 UTC

While the project needs to run under two versions, one will always need to have a unique name from the other. At this point the word "beta" is used to distinguish the two. All the work performed by Rosetta has previously been tested on Ralph.

The recent failing tasks (with CAPRI in the name) have been removed from the work queue. A few strays may remain, but the project team has posted that you may abort them if they are causing problems on your system. So they should not interfere with your testing.

At this point, there is no way to define a specific Rosetta version you wish to run, or to avoid running.
Rosetta Moderator: Mod.Sense
ID: 46825 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 46826 - Posted: 23 Sep 2007, 5:42:57 UTC

A side note: 5.80 clearly has problems, so why is it already here?


Testing on Ralph will never turn up everything. And the existence of errors on Rosetta is not an indication that such testing has failed or been performed improperly.

Every effort is made to avoid such problems, but with a constantly changing application, it will always be a challenge.

Rosetta Moderator: Mod.Sense
ID: 46826 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Major Tom MIB
Avatar

Send message
Joined: 1 Jul 06
Posts: 7
Credit: 128,300
RAC: 0
Message 46827 - Posted: 23 Sep 2007, 5:54:50 UTC

Fair enough, I don't check stable projects as frequently as beta projects, so I was surprised when I did check it and discovered it wasn't having a 'good day'. I'll try another batch and if I still have problems, I'll look else where for the cause. Thanks.
ID: 46827 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
zombie67 [MM]
Avatar

Send message
Joined: 11 Feb 06
Posts: 316
Credit: 6,589,590
RAC: 317
Message 46828 - Posted: 23 Sep 2007, 6:03:40 UTC - in response to Message 46826.  

A side note: 5.80 clearly has problems, so why is it already here?


Testing on Ralph will never turn up everything. And the existence of errors on Rosetta is not an indication that such testing has failed or been performed improperly.


? This makes no sense to me. I understand that testing will never turn up every bug. However, 5.80 turned up a butt-load of problems. Yet it was released here knowing the bugs. What then, is the point of RALPH?

Reno, NV
Team: SETI.USA
ID: 46828 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
zombie67 [MM]
Avatar

Send message
Joined: 11 Feb 06
Posts: 316
Credit: 6,589,590
RAC: 317
Message 46833 - Posted: 23 Sep 2007, 8:11:58 UTC - in response to Message 46831.  

The only problem I've had with 5.80 is with the CAPRI WU's, and not with every CAPRI WU at that. For me this means there's hardly a 'butt-load of problems'.

The 5.80 are (still) throwing out -161 all over the place on RALPH.
Reno, NV
Team: SETI.USA
ID: 46833 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile anders n

Send message
Joined: 19 Sep 05
Posts: 403
Credit: 537,991
RAC: 0
Message 46839 - Posted: 23 Sep 2007, 10:27:38 UTC - in response to Message 46833.  

The only problem I've had with 5.80 is with the CAPRI WU's, and not with every CAPRI WU at that. For me this means there's hardly a 'butt-load of problems'.

The 5.80 are (still) throwing out -161 all over the place on RALPH.


Are you sure it's 5.80 and not the sort of Wu that does it?
ID: 46839 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 46852 - Posted: 23 Sep 2007, 13:00:25 UTC

None of us really has enough information to make a fair assessment about whether a given version or test has passed or should be released to Rosetta. The Ralph boards might be full of problem reports, and if those represent a failure rate of less then 1%, it might mean it is more stable that Rosetta is running now.

Also, as Anders n points out, the apparent problems (other then the server failure) seem to be the tasks, not the release.

And as for errors on Ralph, they are not presently testing a new release there. 5.80 is already out. They are testing the tasks. And yes, if they see failures in a given new task they are working on, they go back and rework them.

v5.80 was posted on Ralph on Sept 12. So it's been out there more then 10 days. In that time, there are only 13 user posts to that thread. Some of which describe more then one task failure. But still, you're only running 2 or 3 failures per day. On the otherhand, Ralph doesn't send out work every day. See, we just don't have all the information. The Project Team does, and they use it to try and keep things running smoothly on Rosetta.
Rosetta Moderator: Mod.Sense
ID: 46852 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 46873 - Posted: 24 Sep 2007, 0:12:38 UTC - in response to Message 46852.  
Last modified: 24 Sep 2007, 0:15:07 UTC

v5.80 was posted on Ralph on Sept 12. So it's been out there more then 10 days. In that time, there are only 13 user posts to that thread. Some of which describe more then one task failure. But still, you're only running 2 or 3 failures per day. On the otherhand, Ralph doesn't send out work every day. See, we just don't have all the information. The Project Team does, and they use it to try and keep things running smoothly on Rosetta.


According to the apps page on each project, 5.80 was installed on 9/12 on Ralph and installed on 9/13 on Rosetta.

How can that possibly be adequate testing? Less than 24 hours? Get real.

Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 46873 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rbpeake

Send message
Joined: 25 Sep 05
Posts: 168
Credit: 247,828
RAC: 0
Message 46874 - Posted: 24 Sep 2007, 0:24:36 UTC - in response to Message 46873.  

...How can that possibly be adequate testing? Less than 24 hours? Get real.

Sounds like a judgment call that went bad...

Hopefully this will be remembered in the future, but to their credit, I think they were onto the problem pretty quickly with the bad CAPRI units (although admittedly there should not have been a problem that was let loose onto Rosetta in the first place! ;)

Regards,
Bob P.
ID: 46874 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
rsubler

Send message
Joined: 24 Jun 07
Posts: 8
Credit: 172,618
RAC: 0
Message 46878 - Posted: 24 Sep 2007, 1:28:13 UTC
Last modified: 24 Sep 2007, 1:29:34 UTC

I'm just a simple volunteer, offering unused computer time to BOINC distributed processing. I definitely do not want to invest the effort to be a useful beta tester. All I ask is that my electric bill produce potentially useful science results.

My main machine is a mid-level AMD X2 3800+ with 1 gig RAM running under Windows XP SR2 with the latest updates and drivers.

In the last month Einstein has produced 6000 credits with zero problems, SETI has done 3000 credits with no problems (after I setup a 3 day queue) and Rosetta has done 3000 credits (12 valid results) with:
- 1 invalid result (my only simple invalid result in 40,000 credits),
- 2 CAPRI failures, and
- 1 Waiting for Memory failure (after a reboot, when BOINC was the only active application, running one Einstein and one Rosetta process).
This is equal to 20% or 25% failure rate.

From here, it certainly looks like the current application and/or WUs were released before their time, especially to volunteers who do not want to be beta testers.

Hoping for future improvements,
Ron
ID: 46878 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 46883 - Posted: 24 Sep 2007, 2:22:36 UTC - in response to Message 46873.  

How can that possibly be adequate testing? Less than 24 hours? Get real.


Well, again, you do not have enough information. How much was changed in v5.80 as compared to the prior stable release? If there were only a couple lines of code changed, and they pertained to an energy calculation which you confirmed required revision, how much testing would be "adequate"?

...and again, the main problem seems to have been with specific type of work, not with the Rosetta version.

Rosetta Moderator: Mod.Sense
ID: 46883 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
rsubler

Send message
Joined: 24 Jun 07
Posts: 8
Credit: 172,618
RAC: 0
Message 46885 - Posted: 24 Sep 2007, 3:14:12 UTC
Last modified: 24 Sep 2007, 3:17:20 UTC

Well, again, you do not have enough information. How much was changed in v5.80 as compared to the prior stable release? If there were only a couple lines of code changed, and they pertained to an energy calculation which you confirmed required revision, how much testing would be "adequate"?

Enough to produce less than my observed 20%+ failure rate.

...and again, the main problem seems to have been with specific type of work, not with the Rosetta version.

From our perspective, errors in data are the same as software errors -- wasted time.
ID: 46885 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FluffyChicken
Avatar

Send message
Joined: 1 Nov 05
Posts: 1260
Credit: 369,635
RAC: 0
Message 46919 - Posted: 24 Sep 2007, 18:00:26 UTC

d
To me if it doesn't bugger up my computer then it is fine. And that is what Ralph@home testing is really for to tremove any obvious bugs.
Rosetta is also about the designing of the program and evolution of it. Part of the failed units returns usefull information to the team, it says hey something is wrong with what you are doing, you cannot do that. Now improve it.
It is not a program that sits still trying to find that perfect cure (well just yet ;-) You need the failure to find places to improve it. Unfortunatly the BOINC platform doesn't suit this kind of development (wrt credit). We shouldn't actually need to give them much information about failed units as they should be able to run a script and get a list of the failed tasks, the setup of the computer etc.. and ata clance see and major problem.

We're just here to advise in what the problems may be in reallity, unless they are skipping the last step and relying on us.
Team mauisun.org
ID: 46919 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nemesis
Avatar

Send message
Joined: 12 Mar 06
Posts: 149
Credit: 21,395
RAC: 0
Message 46933 - Posted: 24 Sep 2007, 21:08:25 UTC - in response to Message 46883.  

How can that possibly be adequate testing? Less than 24 hours? Get real.


Well, again, you do not have enough information. How much was changed in v5.80 as compared to the prior stable release? If there were only a couple lines of code changed, and they pertained to an energy calculation which you confirmed required revision, how much testing would be "adequate"?

...and again, the main problem seems to have been with specific type of work, not with the Rosetta version.


I thought every new type WU was also supposed to be tested on Ralph before being released to the Rosetta masses.

I doubt that whatever these new WUs are could have been tested sufficiently with the new app in less than 24 hours, with the few folks that run Ralph.
Nemesis n. A righteous infliction of retribution manifested by an appropriate agent.


ID: 46933 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Saenger
Avatar

Send message
Joined: 19 Sep 05
Posts: 271
Credit: 824,883
RAC: 0
Message 47290 - Posted: 1 Oct 2007, 19:25:27 UTC - in response to Message 46883.  

...and again, the main problem seems to have been with specific type of work, not with the Rosetta version.

I don't care what precise snippet of the software package is the buggy one, it's all Rosetta. It's supposed to be tested over @Ralph, to be 99% bugfree here.

Could you please inform us users somewhere once you stopped using Rosetta as a RalphII outfit?
Grüße vom Sänger
ID: 47290 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile jaxom1
Avatar

Send message
Joined: 5 Jun 06
Posts: 180
Credit: 1,586,889
RAC: 0
Message 47291 - Posted: 1 Oct 2007, 19:44:09 UTC

I put a lot of work to QMC because of these.

<shrug>

They will get it all straightend out eventually.
ID: 47291 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Number crunching : Please Add Opt-Out Option for Beta Workunits



©2024 University of Washington
https://www.bakerlab.org