Improvements to Rosetta@home based on user feedback

Message boards : Number crunching : Improvements to Rosetta@home based on user feedback

To post messages, you must log in.

1 · 2 · 3 · 4 · Next

AuthorMessage
David Baker
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 17 Sep 05
Posts: 705
Credit: 559,847
RAC: 0
Message 10758 - Posted: 14 Feb 2006, 19:47:03 UTC

the current server problems are likely due to the fact that the latest abinitio runs were sent out with nstruct 10 rather than nstruct 50 to avoid the max time problem, but this is creating too much traffic for our web server as some of the jobs are very short.

David Kim has a very nice fix for all of the work unit time related problems. The new app will have a default target run time of 8 hours, and this rather than -nstruct will determine how many structures are generated per work unit. You will be able to change this target run time to fit your individual preferences--dial up users may wish to make this somewhat longer to reduce traffic still more.

We are also increasing the work unit turnaround time limit from one week to two weeks on most jobs. really high priority calculations will still get the one week maximum, but these we will try to keep in the minority.
ID: 10758 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Dimitris Hatzopoulos

Send message
Joined: 5 Jan 06
Posts: 336
Credit: 80,939
RAC: 0
Message 10760 - Posted: 14 Feb 2006, 22:39:01 UTC

Great, Great news. Good job.

Now I'll be able to suggest to others to run Rosetta, because I was concerned that they would start and drop from the project soon afterwards, due to the traffic requirements.

Any chance for an optimised app? ;-)


Best UFO Resources
Wikipedia R@h
How-To: Join Distributed Computing projects that benefit humanity
ID: 10760 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Scribe
Avatar

Send message
Joined: 2 Nov 05
Posts: 284
Credit: 157,359
RAC: 0
Message 10767 - Posted: 15 Feb 2006, 7:09:46 UTC

....and what is happening on the WU size front? Is compression not far away to reduce the bandwidth required?
ID: 10767 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dgnuff
Avatar

Send message
Joined: 1 Nov 05
Posts: 350
Credit: 24,773,605
RAC: 0
Message 10768 - Posted: 15 Feb 2006, 7:52:47 UTC - in response to Message 10767.  

....and what is happening on the WU size front? Is compression not far away to reduce the bandwidth required?


(David Baker) You will be able to change this target run time to fit your individual preferences--dial up users may wish to make this somewhat longer to reduce traffic still more.


I believe that if we can increase the run time of each WU, that should do a lot to help this, effectively getting more compute time done for each megabyte downloaded.

Mostly to David, but if this change happens, how will we be able to control this?

Or did I completely miss the point?
ID: 10768 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Carlos_Pfitzner
Avatar

Send message
Joined: 22 Dec 05
Posts: 71
Credit: 138,867
RAC: 0
Message 10772 - Posted: 15 Feb 2006, 13:52:01 UTC - in response to Message 10768.  

....and what is happening on the WU size front? Is compression not far away to reduce the bandwidth required?


(David Baker) You will be able to change this target run time to fit your individual preferences--dial up users may wish to make this somewhat longer to reduce traffic still more.


I believe that if we can increase the run time of each WU, that should do a lot to help this, effectively getting more compute time done for each megabyte downloaded.

Mostly to David, but if this change happens, how will we be able to control this?

Or did I completely miss the point?


I believe that that users may be able to control this editing your rosetta@home preferences

However I did edit my preferences for home and did not find this option yet
*nor on global preferences nor on rosetta preferences

BTW: Is need a option to control how many streams for (up/dl) the client app
opens simultaneus --- actually it opens 2 streams
and that 2 files transfers at the same time, is clogging my 14400 bps dialup
blocking me to surf internet while download -or- upload occurs

*If it transfers 1 file at each time my access to internet would not be blocked
and all transfers will occurs quickly w/o too frequent upload/download errors

be understand correct -> I have 2 files to upload -or- download
actually app try sendind -or- receiveing the two files simultaneously

what is need is: upload 1 file, when this upload done, then upload the 2nd file
*the same for downloads


Click signature for global team stats
ID: 10772 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Andrew

Send message
Joined: 19 Sep 05
Posts: 162
Credit: 105,512
RAC: 0
Message 10776 - Posted: 15 Feb 2006, 15:10:35 UTC - in response to Message 10772.  

BTW: Is need a option to control how many streams for (up/dl) the client app
opens simultaneus --- actually it opens 2 streams
and that 2 files transfers at the same time, is clogging my 14400 bps dialup
blocking me to surf internet while download -or- upload occurs

*If it transfers 1 file at each time my access to internet would not be blocked
and all transfers will occurs quickly w/o too frequent upload/download errors

be understand correct -> I have 2 files to upload -or- download
actually app try sendind -or- receiveing the two files simultaneously

what is need is: upload 1 file, when this upload done, then upload the 2nd file
*the same for downloads


This is a boinc client issue not a rosetta client issue, so you should post on the boinc msg boards.
ID: 10776 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Andrew

Send message
Joined: 19 Sep 05
Posts: 162
Credit: 105,512
RAC: 0
Message 10777 - Posted: 15 Feb 2006, 15:16:16 UTC - in response to Message 10768.  
Last modified: 15 Feb 2006, 15:16:41 UTC

Mostly to David, but if this change happens, how will we be able to control this?


When this was initially discussed, I believe it was said that there would be a section added to the rosetta preferences page.

Similar to say what CPDN has done... CPDN has added a graphics section to control their screensaver.


ID: 10777 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
David Baker
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 17 Sep 05
Posts: 705
Credit: 559,847
RAC: 0
Message 10779 - Posted: 15 Feb 2006, 15:24:36 UTC - in response to Message 10768.  

....and what is happening on the WU size front? Is compression not far away to reduce the bandwidth required?


(David Baker) You will be able to change this target run time to fit your individual preferences--dial up users may wish to make this somewhat longer to reduce traffic still more.


I believe that if we can increase the run time of each WU, that should do a lot to help this, effectively getting more compute time done for each megabyte downloaded.

Mostly to David, but if this change happens, how will we be able to control this?

Or did I completely miss the point?


you will be able to control this, and we will post instructions as soon as the new app is released later this week
ID: 10779 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [B@H] Ray
Avatar

Send message
Joined: 20 Sep 05
Posts: 118
Credit: 100,251
RAC: 0
Message 11151 - Posted: 21 Feb 2006, 20:36:33 UTC

I just got some of thses units with the 8 hr. run time and 2 week deadline. Just started running one, will be able to tell after that if it takes much longer.


Pizza@Home Rays Place Rays place Forums
ID: 11151 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile nasher

Send message
Joined: 5 Nov 05
Posts: 98
Credit: 618,288
RAC: 0
Message 11289 - Posted: 24 Feb 2006, 5:01:49 UTC

sounds great .

I always like to hear that our feedback is listend to and is important

thanks for the updates
ID: 11289 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Plum Ugly

Send message
Joined: 3 Nov 05
Posts: 24
Credit: 2,005,763
RAC: 0
Message 11470 - Posted: 27 Feb 2006, 17:59:28 UTC

IS there any way we can get rosetta to purge old or past due work units with out suspending and aborting them on our machines.I have several that have been running older units that are the past due date time.This needs to be adressed for the people running farms.
ID: 11470 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile nasher

Send message
Joined: 5 Nov 05
Posts: 98
Credit: 618,288
RAC: 0
Message 11513 - Posted: 1 Mar 2006, 9:13:46 UTC

i know that you can manualy delete the old Work Units

I am sure there would have to be an update to the BOINC code to make it posible to remove past due work units though. (i could be wrong though)


ID: 11513 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Steve Shedroff

Send message
Joined: 7 Nov 05
Posts: 11
Credit: 250,657
RAC: 0
Message 11558 - Posted: 2 Mar 2006, 11:45:26 UTC

Since the change to WU matched to processing time to minimize transfer losses, I have seen my daily averages and my overall average WU/day soar. I also noticed the total project Teraflops go up. Good move. I noticed that the number of pending WU is much lower on each machine. I assume this is also good for the project, since there is less work at risk of a lost/delayed machine, you will be able to lower the level of redundant WU's sent out to insure total coverage. Net higher throughput and higher reliability.
ID: 11558 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Scribe
Avatar

Send message
Joined: 2 Nov 05
Posts: 284
Credit: 157,359
RAC: 0
Message 11561 - Posted: 2 Mar 2006, 14:38:18 UTC - in response to Message 11558.  

.... you will be able to lower the level of redundant WU's sent out to insure total coverage. Net higher throughput and higher reliability.


Yuo cannot get lower than one to send out.....the redundancy level is currently one....

ID: 11561 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
David Baker
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 17 Sep 05
Posts: 705
Credit: 559,847
RAC: 0
Message 11563 - Posted: 2 Mar 2006, 16:20:10 UTC - in response to Message 11558.  

Since the change to WU matched to processing time to minimize transfer losses, I have seen my daily averages and my overall average WU/day soar. I also noticed the total project Teraflops go up. .


great! the solution that David Kim is working on now should catch most of the remaining errors
ID: 11563 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
vavega
Avatar

Send message
Joined: 2 Nov 05
Posts: 82
Credit: 519,981
RAC: 0
Message 12140 - Posted: 17 Mar 2006, 8:47:32 UTC

would it be possible for someone to write in a script that would automatically abort a wu if it's stuck at 1% for more than say an hour? for those of us running farms this would be a godsend until the 1% stall problem is solved.


ID: 12140 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Win2Kuser

Send message
Joined: 2 Nov 05
Posts: 7
Credit: 2,372,223
RAC: 0
Message 12201 - Posted: 18 Mar 2006, 19:49:43 UTC

I'd second the automatic abort thing, sounds like a great idea until such time as the bug fix is worked out.

I would think that the script would probably be needed for Boinc manager rather than the Rosetta client as it's the manager that handles individual jobs.

Would this be feasable?

It would certainly ease the presure to get a fix in place for those of us have have farms and are running out of patience...
ID: 12201 · Rating: -1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Runaway1956

Send message
Joined: 5 Nov 05
Posts: 19
Credit: 535,400
RAC: 0
Message 12653 - Posted: 25 Mar 2006, 3:35:45 UTC

When I created this thread, I was only thinking of myself, and other crunchers.

https://boinc.bakerlab.org/rosetta/forum_thread.php?id=1287

But, the subject, better compression of the files you send out, would help to relieve some of the stress on your servers when "crap happens". As well as decreasing your bandwidth during normal times...
ID: 12653 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
David Baker
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 17 Sep 05
Posts: 705
Credit: 559,847
RAC: 0
Message 12693 - Posted: 25 Mar 2006, 16:54:38 UTC - in response to Message 12140.  

would it be possible for someone to write in a script that would automatically abort a wu if it's stuck at 1% for more than say an hour? for those of us running farms this would be a godsend until the 1% stall problem is solved.




I just yesterday emailed David Anderson asking whether such a feature could be incorporated into boinc. haven't heard back yet. if someone can figure out how to do this outside of boinc it would be great.
ID: 12693 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Laurenu2

Send message
Joined: 6 Nov 05
Posts: 57
Credit: 3,818,778
RAC: 0
Message 12750 - Posted: 28 Mar 2006, 7:42:23 UTC - in response to Message 12693.  

would it be possible for someone to write in a script that would automatically abort a wu if it's stuck at 1% for more than say an hour? for those of us running farms this would be a godsend until the 1% stall problem is solved.




I just yesterday emailed David Anderson asking whether such a feature could be incorporated into boinc. haven't heard back yet. if someone can figure out how to do this outside of boinc it would be great.

Any word back yet? There is a lot CPU time being wast evey day by this

If You Want The Best You Must forget The Rest
---------------And Join Free-DC----------------
ID: 12750 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Improvements to Rosetta@home based on user feedback



©2024 University of Washington
https://www.bakerlab.org