What's up with this?

1/3/2011 7:44:15 PM rosetta@home [error] Error reported by file upload server: [mem_prub_run05_centroid_round03_A_subrun_007643_SAVE_ALL_OUT_IGNORE_THE_REST_22824_46_0_0] locked by file_upload_handler PID=-1

Never a dull moment... :-(

Get a couple dozen of finished jobs with "upload pending" across several hosts as well..

But there are now 9million jobs to be process in queue and all servers show green... :-(

Ralf

The whole site went down for about 8 hours, presumably to fix the problems over the holiday, and everyone's now uploading all their work. It's taking up to an hour to validate my first uploads but that should improve. New work started coming down 2 hours ago.

Hopefully they've fixed the stats export too.

---

I get this now:
1/4/2011 1:42:28 PM|rosetta@home|Message from server: Project is temporarily shut down for maintenance

Also all server parts are listed as 'Disabled'. Looks like it could be being worked on.

---

Well, now all but the web server show "disabled" (in green?), so hopefully this will all be fixed soon, would be 7 days by now, that's quite a time even considering the holidays.
At least some form of acknowledgment on the projects web site would have been nice to show that they are aware of the problems... :?

Ra;f

Ditto. Looks like I spoke a bit too soon. A finished file uploaded but I couldn't report the task back for the moment. Hopefully not long now....

Looks like the stats have now been published.

---

Rosetta Moderator: Mod.Sense

I see the scheduler is back up and now I can upload finished work units, but it's still not sending out work. I have ~50 nodes waiting for work, several of them hyperthreaded. I joined yesterday. Is this much downtime typical? I usually have them fetch enough work for 2 days. Should I up this to 3, or 5, or 10? Since most of them were new to the project they hadn't yet received a full two days of work unites.

Is this much downtime typical?

Not really. At a guess there have only been about 5 outages over 12 hours in the last 6 months. While some of those took a while to get fixed, Rosetta is quite stable the majority of the time.

Between Xmas and New Year I'd say it was typical. In fact it was far worse last year with just about everyone out of Rosetta WUs for days. This time I'd guess most people are still running - just a bit of a hiccup for a few hours.

Todays credits are between 2 and 4 times my highest ever. Nothing lost that I can see.

---

Plasmon Attack said:

Is this much downtime typical?

Absolutely not, Rosetta is one of the most stable BOINC-based projects around. Unfortunately, the level of communications with the community is. Mod.Sense does his best but I think that when we hit one of these speed bumps he has as much trouble getting system status from the admins as we do.

I really think that if the "home page" is up then the system administrators should have a little two or three line post providing a little insight as to what the problem is.

If the project is completely down, then the same status could be posted on their Facebook page.

No one is expecting "real time" updates - keeping the community informed is not a burden, it is a common courtesy which should be extended all members of the team.

I have been experiencing some problems w/ uploading, starting in late December 2010. Even then, it had been taking multiple tries to u/l completed WU. My last successful u/l was on Jan 3 and I have two pending right now.

Has anyone else had problems u/l WU in this time frame?

Thanks.

---

I have been experiencing some problems w/ uploading, starting in late December 2010. Even then, it had been taking multiple tries to u/l completed WU. My last successful u/l was on Jan 3 and I have two pending right now.

Has anyone else had problems u/l WU in this time frame?

Thanks.

yup, sitting on a boatload of finished units.

---

Is there anywhere we can go to get "fficial information" on what's going on? Or is this it (i.e. just a bunch of questions with no official response)?

---

Is there anywhere we can go to get "fficial information" on what's going on? Or is this it (i.e. just a bunch of questions with no official response)?

When the project went down there was a temporary message put up on their webserver stating that the fileserver crashed and that they are working to restore it as soon as possible.

---

I too having been waiting for a while for upload and additional work items - All servers (except the feeder) are working and active according to the status report.

Is someone working on this?

Bill H
----------------------------------------------------------------------------

08/01/2011 07:29:16 rosetta@home update requested by user
08/01/2011 07:29:21 rosetta@home Sending scheduler request: Requested by user.
08/01/2011 07:29:21 rosetta@home Reporting 2 completed tasks, requesting new tasks for CPU and GPU
08/01/2011 07:29:24 rosetta@home Scheduler request completed: got 0 new tasks
08/01/2011 07:29:24 rosetta@home Message from server: Server error: can't attach shared memory

From front page.

Jan 7, 2010
Well, our luck ran out. The SAN controller that has been causing so much trouble in the last few months finally tipped over in a rather distructive fashion, corrupting the binary tree on which the filesystem is based. We're trying to rebuild the thing but the sheer number of files in the filesystem (> 10M files) makes this process very, very slow. We're bringing the project up from a recent backup (12/09/10) but the backup wasn't a perfect replica of the environment, so we're having to scramble to get all the parts working together again. We only need a few more weeks and then our new, next generation SAN will be ready to be put into place... I just thought the old one would last a few more week. I apologize for the hassle and appreciate your patience as we get things online again... KEL 01/07/11

---

Have a crunching good day!!

This time I'd guess most people are still running - just a bit of a hiccup for a few hours.

I really wish I hadn't written that...