Message boards : Number crunching : Problems and Technical Issues with Rosetta@home
Previous · 1 . . . 21 · 22 · 23 · 24 · 25 · 26 · 27 . . . 55 · Next
Author | Message |
---|---|
Link Send message Joined: 4 May 07 Posts: 356 Credit: 382,349 RAC: 0 |
I'm letting them count down to Project Backoff, and if it Fails, too bad. But if it Reverts to Retry again, ad doesn't work, I'm going to Abort Transfer. That's pretty much the most stupid thing you can do. Best thing to do is to let BOINC do it's job without clicking on any buttons, it's designed to take care such things by itself. Once the servers will work as they should, BOINC will upload all those tasks and eventually request new ones. If your cache is getting empty and you want to do something clever, switch on your backup project(s). . |
BarryAZ Send message Joined: 27 Dec 05 Posts: 153 Credit: 30,843,285 RAC: 0 |
Link, actually, it does depend on the project -- going 'passive mode' with a number of projects tends to be suboptimal. SETI is notorious about this -- one has to 'encourage' downloads to complete, otherwise they will time out and simply not happen. For them it is an unresolved I/O and scheduler handling issue which they've had for months. Typically (aside from SETI), my involvement is somewhat responsive to the project situation. For example, with Rosetta right now, rather than waste time 'pushing buttons' -- I simply suspend the project and have other projects get the cycles. (As an aside, it never makes sense to me to have a single CPU or a single GPU project active on a workstation). Once Rosetta comes back online (and clears backlogs) - which I would guess would be by Tuesday, I simply take it off of suspend and let life go on. I use that approach with other projects when they are dealing with relatively extended outages (say 12 hours or more). SETI takes a bit of additional handling due to the combination of I/O and scheduler issues, PLUS its weekly Tuesday 4 hour outage and backlog, plus the very high user volumes involved. So in addition to hand holding new work downloads, often, after I have gotten a batch of new work, I go to No New Work there -- that keeps the scheduler happy for handling my uploads and reports. I guess I'm responding to that 'let BOINC do its thing' position as there were folks over on SETI who (over a year ago) insisted that this was the only 'right thing' to do. As SETI is one of the worst projects for doing that, I found that position shall we say, a bit counter intuitive. I'm letting them count down to Project Backoff, and if it Fails, too bad. But if it Reverts to Retry again, ad doesn't work, I'm going to Abort Transfer. |
BarryAZ Send message Joined: 27 Dec 05 Posts: 153 Credit: 30,843,285 RAC: 0 |
The thing here is Rosetta uses different servers for their web pages -- which is why we can post here. It is also quite possible that the Rosetta Servers are OK, and that the UW IT folks have made one of their periodic IP address changes over the weekend without prior notification to Baker Labs folks. Server page is all green however... I'll bet I've seen that 'server status' page actually show something as 'down' maybe once over a few years. AFAIK it's rarely if ever accurate or updated. Oh well.[/quote] |
GALAXY-VOYAGER Send message Joined: 25 Oct 12 Posts: 15 Credit: 47,437 RAC: 0 |
I'm letting them count down to Project Backoff, and if it Fails, too bad. But if it Reverts to Retry again, ad doesn't work, I'm going to Abort Transfer. Yeah, I've had second thoughts. It's just that I'm fed up with things like this happening (especially with R@H). I've decided to let it run it's course. It seems to be working (albeit ever So Slowly, because it's Still Doing The same thing). When you suggest for me to Turn On my back-Up Projects, I assume you mean any other Projects I Run. Actually, I Run 5 Other Projects besides R@H (excluding Orbit@Home, which has never been Active since I Joined it last year), and I Run them all at once. I've Temporarily Suspended ALL Other Projects for 12-24 Hours to see if that helps (including E@H, but allowed One Single Task in E@H to keep running because it has about 39 Hours Estimated Remaining Time): All other Tasks in each Project are not due until March 3rd or after. So, I'll give R@H The Run of The Mill, and see what happens. |
Link Send message Joined: 4 May 07 Posts: 356 Credit: 382,349 RAC: 0 |
Link, actually, it does depend on the project -- going 'passive mode' with a number of projects tends to be suboptimal. SETI is notorious about this -- one has to 'encourage' downloads to complete, otherwise they will time out and simply not happen. For them it is an unresolved I/O and scheduler handling issue which they've had for months. SETI is rather special case, they get simply much more computing power than they can use with their current internet connection. Sure, for a particular cruncher pushing buttons might help, overall it does not (if we neglect, that the more of us push buttons and try to push SETI's internet connection harder, the more packet loss occur). If I download a WU, somebody else doesn't. From the project's view it doesn't matter which machine sits idle, they send as many WUs out as they can (well, actually they try to send a bit more out than they can and hence the download issues). That's why I moved now two of my 3 CPUs to Rosetta, they can apparently use them, for SETI it does not matter, the WUs I'd crunch will be done by another machine, which eventually would sit idle if I got those WUs. So yes, not pushing any buttons is counter intuitive, but only as long as you look at it from your end. From the project's point of view it either does not matter or makes things worse. . |
mikey Send message Joined: 5 Jan 06 Posts: 1895 Credit: 9,109,273 RAC: 6,439 |
I'm letting them count down to Project Backoff, and if it Fails, too bad. But if it Reverts to Retry again, ad doesn't work, I'm going to Abort Transfer. Rosetta folks take their weekends seriously, THEY DO NOT WORK ON WEEKENDS. That means until they show up for work today, Monday, they have NO CLUE we are having any problems! As long as the fix is not broken or missing hardware it should be back up and running pretty quickly. Obviously broken or missing hardware could take a bit longer. I run Rosie full time on my pc's, no other cpu projects at all. BUT I DO have one project set up at zero percent for instances just like this, when my Rosie work cache ran dry my zero percent project picked right up and I am happily crunching for them. When Rosie is fixed it will go back to standby mode and Rosie will be back to it's 100% mode. |
Link Send message Joined: 4 May 07 Posts: 356 Credit: 382,349 RAC: 0 |
When you suggest for me to Turn On my back-Up Projects, I assume you mean any other Projects I Run. Actually, I Run 5 Other Projects besides R@H (excluding Orbit@Home, which has never been Active since I Joined it last year), and I Run them all at once. Well, I don't know what and how you run, but you sounded in your previous post like those people over at SETI, who run only SETI and insist that SETI should have availablility like google or so because they don't accept that their CPUs or GPUs are idle. If you run 5 projects all at once you should actually not need to do anything, once the Rosetta servers will be back up, BOINC will upload and report completed tasks and request new ones like nothing happend. . |
amgthis Send message Joined: 25 Mar 06 Posts: 81 Credit: 203,879,282 RAC: 0 |
Server page is all green however... I'll bet I've seen that 'server status' page actually show something as 'down' maybe once over a few years. AFAIK it's rarely if ever accurate or updated. Oh well.[/quote] |
amgthis Send message Joined: 25 Mar 06 Posts: 81 Credit: 203,879,282 RAC: 0 |
Server page is all green however... I'll bet I've seen that 'server status' page actually show something as 'down' maybe once over a few years. AFAIK it's rarely if ever accurate or updated. Oh well.[/quote] Looks like the server status pages have been updated to 'disabled'. Someone is paying attention. I'm sure the servers will be back up soon. |
TJ Send message Joined: 29 Mar 09 Posts: 127 Credit: 4,799,890 RAC: 0 |
Rosetta folks take their weekends seriously, THEY DO NOT WORK ON WEEKENDS. That means until they show up for work today, Monday, they have NO CLUE we are having any problems! As long as the fix is not broken or missing hardware it should be back up and running pretty quickly. Obviously broken or missing hardware could take a bit longer. They know; I sent a PM yesterday (roughly 36 hours ago) morning to the mail address provided by Ethan (to use when there are issues). Greetings, TJ. |
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Rosetta folks take their weekends seriously, THEY DO NOT WORK ON WEEKENDS. That means until they show up for work today, Monday, they have NO CLUE we are having any problems! As long as the fix is not broken or missing hardware it should be back up and running pretty quickly. Obviously broken or missing hardware could take a bit longer. And yet here we are with a broken something or another with no one bothering to do anything about it. This is so typical of baker labs. I think I will lower my % on Rosie and stick it into Poem or Einstein. I have been with this project for a long time now and it is always the same story. No communication from anyone. |
TJ Send message Joined: 29 Mar 09 Posts: 127 Credit: 4,799,890 RAC: 0 |
It is working again! The server page shows all "running". Uploading and reporting is working, but with thousands of WU's the network connections will get "problems". Happy crunching. Greetings, TJ. |
Warped Send message Joined: 15 Jan 06 Posts: 48 Credit: 1,788,185 RAC: 0 |
Rosetta folks take their weekends seriously, THEY DO NOT WORK ON WEEKENDS. That means until they show up for work today, Monday, they have NO CLUE we are having any problems! As long as the fix is not broken or missing hardware it should be back up and running pretty quickly. Obviously broken or missing hardware could take a bit longer. That's wishful thinking: 1. Monday is a public holiday in the USA. 2. The e-mail address is likely only looked-at when Ethan is at work. I'm expecting at least a further 24 hours without any life from the project. Edit: I am pleasantly surprised. Looks like the router's power supply was switched back on! |
John_Waters Send message Joined: 23 Jun 11 Posts: 15 Credit: 6,838,499 RAC: 0 |
"Rosetta folks take their wekends seriously" We're well into the third day of outage,and apparently they're not working today either. I run six,sometimes seven,8-core machines 24/7 in support of R@H. Electricity alone runs about $200 a month. That's another way of saying may wife's NOT happy. And she doesn't realize that I'm averaging about as much on hardware,software,cables,etc. It's just mind-boggling to me that they would have such a cavalier attitude towards the contributors that no one would be on standby for just this occurence. I have 108 w/u's ready to turn in. Then I'll think about whether I should continue spending all that money. |
Link Send message Joined: 4 May 07 Posts: 356 Credit: 382,349 RAC: 0 |
It's just mind-boggling to me that they would have such a cavalier attitude towards the contributors that no one would be on standby for just this occurence. I have 108 w/u's ready to turn in. Then I'll think about whether I should continue spending all that money. I always wonder why there are so many people, who don't realize that: 1. BOINC is designed to handle such outages of your favourite project(s) -> see WU cache and the backup project feature. 2. BOINC was made for to use spare cycles of computers that would be running anyway. Nobody ever asked the volunteers to build additional computers just for crunching. 3. BOINC is intended for projects with little money, projects that can't buy the computing power they need. Many of them also can't pay anybody who sits and watches the servers all the time. Considering #1 that would be wasted money anyway, we don't need 24/7 availability. . |
GALAXY-VOYAGER Send message Joined: 25 Oct 12 Posts: 15 Credit: 47,437 RAC: 0 |
I'm letting them count down to Project Backoff, and if it Fails, too bad. But if it Reverts to Retry again, ad doesn't work, I'm going to Abort Transfer. Okay, thanks for the encouragement and information. I didn't realise that it was Totally unattended at any time: I thought that there were Admin/Mods on the job all the time, in the form of a Roster System. Anyway, my latest Idea (mentioned above) has seemed to work. All Completed R@H Tasks have Uploaded. Seems like that could be the Trick: Temporarily Suspend Other Projects if it is practical to do so, and let Rosie Run The Flower store ...LOL :-) Happy Crunching, (ps .. I have now Resumed All Projects) |
Link Send message Joined: 4 May 07 Posts: 356 Credit: 382,349 RAC: 0 |
Are there any sort of cricket graphs available for the Rosetta servers like the ones we have at SETI? That has proven to be very useful when someone wants to know what the servers are doing even if the server status page isn't working. . |
mikey Send message Joined: 5 Jan 06 Posts: 1895 Credit: 9,109,273 RAC: 6,439 |
Server page is all green however... Looks like the server status pages have been updated to 'disabled'. Someone is paying attention. I'm sure the servers will be back up soon. [/quote] One thing to remember is almost ALL webpages are cached and only refreshed on a periodic basis, some by the minute, some by the hour and some by the day or week. There is no way to tell unless you check and notice a change, or they come out and actually tell us. |
mikey Send message Joined: 5 Jan 06 Posts: 1895 Credit: 9,109,273 RAC: 6,439 |
"Rosetta folks take their wekends seriously" We're well into the third day of outage,and apparently they're not working today either. I run six,sometimes seven,8-core machines 24/7 in support of R@H. Electricity alone runs about $200 a month. That's another way of saying may wife's NOT happy. And she doesn't realize that I'm averaging about as much on hardware,software,cables,etc. One thing to remember like in real estate...location, location, location! Rosetta is run out of the University of Washington, as in Washington STATE!! That means West Coast time, when I posted it was 7AM East Coast time and they were probably still snoring logs! They didn't even get up and think about work until I was getting ready for my LUNCH!! When I was active at Seti, MANY years ago which is based in California, the EARLIEST folks wouldn't even come to work before 9am, and MOST of them it was more like NOON!!! I was getting rid to go home and they were just coming to work!! So remember location, location, location, it is VERY important when thinking of WHEN things will be done! |
Link Send message Joined: 4 May 07 Posts: 356 Credit: 382,349 RAC: 0 |
One thing to remember is almost ALL webpages are cached and only refreshed on a periodic basis, some by the minute, some by the hour and some by the day or week. There is no way to tell unless you check and notice a change, or they come out and actually tell us. It's written on the server status when it was updated the last time. . |
Message boards :
Number crunching :
Problems and Technical Issues with Rosetta@home
©2024 University of Washington
https://www.bakerlab.org