No work from project with 89,369 queued

Message boards : Number crunching : No work from project with 89,369 queued

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
Profile [B@H] Ray
Avatar

Send message
Joined: 20 Sep 05
Posts: 118
Credit: 100,251
RAC: 0
Message 2173 - Posted: 3 Nov 2005, 18:03:43 UTC

Got plunty of work last night but not today. Still have some on the system that runs mostely Rosetta, but LHC has work now so I set that to 50% until Rosetta is working correct again. LHC and CPDN will keep that system busy. Only 4 waiting to be done for Rosetta there.

Still have plunty of Rosetta on the other system that usually runs CPDN most of the time but going down, should last another 4 to 5 days on that system.


Pizza@Home Rays Place Rays place Forums
ID: 2173 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 2178 - Posted: 3 Nov 2005, 18:33:14 UTC
Last modified: 3 Nov 2005, 18:33:27 UTC

Here is an update. The database is still being purged. It has gone through over 500,000 WU's and results but there is close to another 500,000 to go. The purging and archiving is also burdening the database server of course. I ask for peoples' patience as it is taking a while to write out the data into archive files. I do not want to just delete the WU records without archiving them so I can still link the structure predictions (output data) with who and what computer did them to give everyone feedback and credit.

We'll be sure to add new hardware in the future if necessary. I'd like to acknowledge our hard working servers with an image :)

ID: 2178 · Rating: 2 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Bok

Send message
Joined: 17 Sep 05
Posts: 54
Credit: 3,514,973
RAC: 0
Message 2179 - Posted: 3 Nov 2005, 18:40:34 UTC

David,

thanks for that. I for one will be patient :)

Seems to be a long time to do the purging though? What is the bottleneck do you know ? I work with large volume DB's, mostly Oracle & DB2 mind, and I know this can't be a fair comparison, but I've run applications which unload millions of transactions in a matter of minutes.

I presume this is mysql or PostGres ? I don't want to serve up the obvious but are the tables optimized, is it IO writing to disk etc etc.

Be interesting (at least to me) to know :)

Thanks again!

Bok


Free-DC

Stats for all projects

Custom Stats
ID: 2179 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 2180 - Posted: 3 Nov 2005, 18:48:19 UTC

Hi Bok,

We use mysql and I haven't optimized the set up yet but I'm glad you mentioned it because that is one of the first things on my list to do. It is also writing a bunch of data to disk to generate archive files. I'm also printing out debug statements since this is the first time using the purge utility and I want to monitor it's progress.
ID: 2180 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Bok

Send message
Joined: 17 Sep 05
Posts: 54
Credit: 3,514,973
RAC: 0
Message 2181 - Posted: 3 Nov 2005, 18:55:17 UTC

It can be quite amazing how much mysql improves with just running

optimize table xxxxxx

frequently. Though it's only if there is regular fragmentation of the data.

I ran into fairly recently on my stats system, adding that prior to every run fixed a lot of issues with speed.

Bok
Free-DC

Stats for all projects

Custom Stats
ID: 2181 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 2184 - Posted: 3 Nov 2005, 19:05:56 UTC

Great!, thanks Bok, I'll look into it.
ID: 2184 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Thomas J Leary DDS

Send message
Joined: 23 Oct 05
Posts: 1
Credit: 17,582
RAC: 0
Message 2201 - Posted: 3 Nov 2005, 22:30:23 UTC

I have been on Seti for years but cannot get BOINC to work. Please help.
ID: 2201 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rbpeake

Send message
Joined: 25 Sep 05
Posts: 168
Credit: 247,828
RAC: 0
Message 2206 - Posted: 3 Nov 2005, 22:46:13 UTC - in response to Message 2201.  

I have been on Seti for years but cannot get BOINC to work. Please help.


The Rosetta server is temporarily down for data purging. Please see this thread

https://boinc.bakerlab.org/rosetta/forum_thread.php?id=272

Regards,
Bob P.
ID: 2206 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 2217 - Posted: 4 Nov 2005, 1:29:00 UTC

Please let me know if anyone is still experiencing problems getting work. The database load is down but I am going to continue purging tonight.
ID: 2217 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Divide Overflow

Send message
Joined: 17 Sep 05
Posts: 82
Credit: 921,382
RAC: 0
Message 2218 - Posted: 4 Nov 2005, 2:04:49 UTC - in response to Message 2217.  
Last modified: 4 Nov 2005, 2:06:51 UTC

Please let me know if anyone is still experiencing problems getting work. The database load is down but I am going to continue purging tonight.

I'm not able to get any work right now, nor am I able to report a handful of my last results.
EDIT: I see that the database is down, so you must be working on that purge at the moment. Thanks for keeping the server status page updated!
ID: 2218 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Webmaster Yoda
Avatar

Send message
Joined: 17 Sep 05
Posts: 161
Credit: 162,253
RAC: 0
Message 2220 - Posted: 4 Nov 2005, 2:16:32 UTC - in response to Message 2217.  
Last modified: 4 Nov 2005, 2:18:49 UTC

Please let me know if anyone is still experiencing problems getting work. The database load is down but I am going to continue purging tonight.


Most of my computers now have some Rosetta work units but I still get intermittent problems, like (times are GMT+8):

4/11/2005 10:12:25 AM No schedulers responded
and
4/11/2005 10:13:53 AM No work from project
and
4/11/2005 10:18:34 AM Project is down


*** Join BOINC@Australia today ***
ID: 2220 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 2222 - Posted: 4 Nov 2005, 2:18:30 UTC

Sorry, I'm still working on the database server. But hope to finish within the next hour.
ID: 2222 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 2226 - Posted: 4 Nov 2005, 4:15:31 UTC

I am purging the database more overnight. The load on the database has reduced though so let me know if you are still experiencing problems getting work.
ID: 2226 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Carlos

Send message
Joined: 29 Oct 05
Posts: 1
Credit: 408,410
RAC: 400
Message 2227 - Posted: 4 Nov 2005, 4:59:52 UTC - in response to Message 2226.  

I am purging the database more overnight. The load on the database has reduced though so let me know if you are still experiencing problems getting work.


Time is Central Time Zone (USA)

11/3/2005 10:55:17 PM|rosetta@home|Fetching master file
11/3/2005 10:55:22 PM|rosetta@home|Master file download succeeded
11/3/2005 10:55:27 PM|rosetta@home|Sending scheduler request to https://boinc.bakerlab.org/rosetta_cgi/cgi
11/3/2005 10:55:27 PM|rosetta@home|Reason: To fetch work
11/3/2005 10:55:27 PM|rosetta@home|Requesting 43200 seconds of new work
11/3/2005 10:55:32 PM|rosetta@home|Scheduler request to https://boinc.bakerlab.org/rosetta_cgi/cgi succeeded
11/3/2005 10:55:32 PM|rosetta@home|No work from project
11/3/2005 10:55:37 PM|rosetta@home|Deferring communication with project for 54 seconds
11/3/2005 10:56:33 PM|rosetta@home|Sending scheduler request to https://boinc.bakerlab.org/rosetta_cgi/cgi
11/3/2005 10:56:33 PM|rosetta@home|Reason: To fetch work
11/3/2005 10:56:33 PM|rosetta@home|Requesting 43200 seconds of new work
11/3/2005 10:56:38 PM|rosetta@home|Scheduler request to https://boinc.bakerlab.org/rosetta_cgi/cgi succeeded
11/3/2005 10:56:38 PM|rosetta@home|No work from project
11/3/2005 10:56:43 PM|rosetta@home|Deferring communication with project for 54 seconds
11/3/2005 10:57:39 PM|rosetta@home|Sending scheduler request to https://boinc.bakerlab.org/rosetta_cgi/cgi
11/3/2005 10:57:39 PM|rosetta@home|Reason: To fetch work
11/3/2005 10:57:39 PM|rosetta@home|Requesting 43200 seconds of new work
11/3/2005 10:57:44 PM|rosetta@home|Scheduler request to https://boinc.bakerlab.org/rosetta_cgi/cgi succeeded
11/3/2005 10:57:44 PM|rosetta@home|No work from project
11/3/2005 10:57:49 PM|rosetta@home|Deferring communication with project for 54 seconds

ID: 2227 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile stephan_t
Avatar

Send message
Joined: 20 Oct 05
Posts: 129
Credit: 35,464
RAC: 0
Message 2232 - Posted: 4 Nov 2005, 6:42:39 UTC

As everyone probably noticed, it seems to be working again now (7am UTC).
Team CFVault.com
http://www.cfvault.com

ID: 2232 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Scribe
Avatar

Send message
Joined: 2 Nov 05
Posts: 284
Credit: 157,359
RAC: 0
Message 2233 - Posted: 4 Nov 2005, 7:06:44 UTC

Yup, WUs coming down!
ID: 2233 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Webmaster Yoda
Avatar

Send message
Joined: 17 Sep 05
Posts: 161
Credit: 162,253
RAC: 0
Message 2244 - Posted: 4 Nov 2005, 13:14:17 UTC - in response to Message 2226.  
Last modified: 4 Nov 2005, 13:14:55 UTC

I am purging the database more overnight. The load on the database has reduced though so let me know if you are still experiencing problems getting work.


Plenty of work now, thanks David.

I think the purge is now so far ahead that it purges results the minute they have come in and validated (and credit has been granted). All I see now on all my hosts is work units not yet completed (even minutes after they have been reported).

Hopefully these results aren't lost to the science?
*** Join BOINC@Australia today ***
ID: 2244 · Rating: 1 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Doug Worrall
Avatar

Send message
Joined: 19 Sep 05
Posts: 60
Credit: 58,445
RAC: 0
Message 2245 - Posted: 4 Nov 2005, 13:18:16 UTC - in response to Message 2244.  

I am purging the database more overnight. The load on the database has reduced though so let me know if you are still experiencing problems getting work.


Plenty of work now, thanks David.

I think the purge is now so far ahead that it purges results the minute they have come in and validated (and credit has been granted). All I see now on all my hosts is work units not yet completed (even minutes after they have been reported).

Hopefully these results aren't lost to the science?



Same here,
Finish w/u gets uploaded given credit "Dissapears" Magic,into cyber space,I hope not?
Thanks
Doug
ID: 2245 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Honza

Send message
Joined: 18 Sep 05
Posts: 48
Credit: 173,517
RAC: 0
Message 2272 - Posted: 4 Nov 2005, 16:34:46 UTC - in response to Message 2226.  

I am purging the database more overnight. The load on the database has REDUCED though so let me know if you are still experiencing problems getting work.


But the homepage says:
Our database is being purged and old workunits and results are being archived. As a result, the load on the server is HIGH and work flow has been reduced.

I think the homepage should state "load on the server is HIGH" other way round -> "As a result, the load on the server is LOWER/REDUCED/etc."

My English is not very good but this put me in confusion on first reading :-)
ID: 2272 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile David E K
Volunteer moderator
Project administrator
Project developer
Project scientist

Send message
Joined: 1 Jul 05
Posts: 1480
Credit: 4,334,829
RAC: 0
Message 2276 - Posted: 4 Nov 2005, 17:50:37 UTC

The database purge has finished. Completed WU's and results should now be kept in the database for 1 week before they get purged and archived so users can view their results. WU's and results (and the credits granted) that have been purged and archived have not been lost. The credits for each user is totaled and kept in the database. The result output (structures and rmsd vs energy) can and will still be linked to users and computers for the results that have been archived. Thanks for everyone's patience!!
ID: 2276 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Number crunching : No work from project with 89,369 queued



©2024 University of Washington
https://www.bakerlab.org