Message boards : Number crunching : Problems and Technical Issues with Rosetta@home
Previous · 1 . . . 161 · 162 · 163 · 164 · 165 · 166 · 167 . . . 309 · Next
Author | Message |
---|---|
Falconet Send message Joined: 9 Mar 09 Posts: 354 Credit: 1,276,393 RAC: 2,018 |
Ah ha! Well sorry for the misspelling. Quick typing and no thinking after work. Since these Robetta jobs aren't run at Rosetta@home, it doesn't make sense that they would get queued at Rosetta@home. They seem to be directly sent from Robetta to the Baker Lab cluster and to the HHMI's Janelia Research Campus. My bet is the 2.6 million tasks on the Rosetta@home queue are all Pythons. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 12,116,986 RAC: 9,863 |
Now...if you go here: https://robetta.bakerlab.org/queue.php?id=&target=&username=&seq=&page=2 and look at the active tasks at random, you will see that the majority are queued for RoseTTAFold which is the AI.The first thing I spotted when looking at the "user input" page was the word KILL somewhere in the sequence. Either I'm a homicidal maniac, or it's because I'm playing fallout 4 on a Ryzen 3900XT that should be folding. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 12,116,986 RAC: 9,863 |
So how come there are so many usernames, it certainly looks like Boinc.https://robetta.bakerlab.org/queue.php?id=&target=&username=&seq=&page=2Since these Robetta jobs aren't run at Rosetta@home, it doesn't make sense that they would get queued at Rosetta@home. |
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
So how come there are so many usernames, it certainly looks like Boinc.https://robetta.bakerlab.org/queue.php?id=&target=&username=&seq=&page=2Since these Robetta jobs aren't run at Rosetta@home, it doesn't make sense that they would get queued at Rosetta@home. That's not users related to us. That's the person who submitted the protein. Again...if its Robetta, its not us. That's all there is to it. I'll have to dig around some more. I have always known from other sources that Robetta server supplied Rosetta servers. That Robetta was where everything is stored. That may not be the case, but it's something that will take some digging around. The group is not that transparent on how their setup works. |
Falconet Send message Joined: 9 Mar 09 Posts: 354 Credit: 1,276,393 RAC: 2,018 |
Robetta, as far as I can tell, is separate from Rosetta@home and is used mostly by researchers outside of the Baker Lab/IPD. It's an interface for users who wish to get computing power for their jobs. Jobs that require the use of Rosetta 4.20 that are submitted to Robetta get sent to Rosetta@home but the rest goes to the other servers that they set up when they launched RoseTTAFold. |
aperience Send message Joined: 26 May 21 Posts: 1 Credit: 178,981 RAC: 53 |
So how come there are so many usernames, it certainly looks like Boinc.https://robetta.bakerlab.org/queue.php?id=&target=&username=&seq=&page=2Since these Robetta jobs aren't run at Rosetta@home, it doesn't make sense that they would get queued at Rosetta@home. Hi, regular Boinc user here, first time posting. I don't know why, but I've never had issues with python projects tasks. I haven't done anything special apart from downloading VirtualBox alongside Boinc. I assume many people did, unless there's some controversy around VirtualBox that I'm unaware of. I've seen that I can't have more than 2 python tasks at a time, whatever the reason is, but it's not a limitation for me, as I mostly run tasks for Numberfields@home. I'm just posting to tell you that users like me exist, although we may be rare (I assume I'm not the only one). I'm not saying that there aren't users related to the running team among the ~1000 users running python tasks right now, I can just tell that I'm not one of them and that it's possible to run python tasks through Boinc and VB. I guess I'm just lucky it works on my laptop. I can give you more info if you want but I haven't really followed the discussion and I'm not an expert with computers, so try to give instructions if you want technical details. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
I have posted this many times before. They should make it a sticky if there were any moderator around to do it. If you are running VirtualBox 6.1.x, you will get the "Vm job unmanageable" problem with the pythons. That is true whether you are running Windows or Linux. The difference is that it can be fixed with Windows. You go back to VirtualBox 5.2.44 https://www.virtualbox.org/wiki/Download_Old_Builds_5_2 Unfortunately, that does not work on Linux, at least Ubuntu. Firstly, Ubuntu 20.04.3 works only with VBox 6.1.x. Secondly, even going back to Ubuntu 18.04.6, which allows you to install VBox 5.2.44, still has the problem. They need to fix it at the project end, by compiling a new Vbox wrapper. They did it on LHC, and it works there. (It has to do with the COM interface, in case you are interested.) NB: If you reboot frequently, you may not see the problem. It usually occurs after the pythons have been running 12 hours or so, but I have seen it even after a reboot on Ubuntu. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 12,116,986 RAC: 9,863 |
That's only one of the problems, I can't get python to work on 6 of 7 Windows PCs no matter which version of VB I use. One of them fails validation at the server end, the rest just sit with almost 0 CPU time. Pythons for 12 hours? They average 2 hours here. As for only being able to run a couple at a time, you need a lot of RAM. I can run 5 but it won't do 6 in 16GB. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
Pythons for 12 hours? They average 2 hours here. No, not an individual run for 12 hours. After running a series of them continually. I didn't say anything about running only two. I usually run at least eight, and am presently running twenty on a Ryzen 3900X with 80 GB of memory. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 12,116,986 RAC: 9,863 |
The two was the post before yours. Sorry, I was too lazy to quote who I was talking to in which bit. I'm giving a dodgy machine another go on the old VB, but I think I've tried this. It just doesn't use the CPU at all.Pythons for 12 hours? They average 2 hours here. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
I'm giving a dodgy machine another go on the old VB, but I think I've tried this. It just doesn't use the CPU at all. I wish I had a fix, but you have tried a lot. Good luck. |
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Robetta, as far as I can tell, is separate from Rosetta@home and is used mostly by researchers outside of the Baker Lab/IPD. It's an interface for users who wish to get computing power for their jobs. Ok..so then where do they get the million something tasks in queue? But yet there appears to be only a few thousand released? There has always been a million something in queue, even back when it was just 4.2 alone. So something doesn't add up. And that you can't see what is next in line....but yet you can see Robetta? Plus someone kept quoting Robetta information some time ago as if that was where RAH gets its work. |
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
Pythons for 12 hours? They average 2 hours here. Holy cow! 80 gigs?!?! That's more than my budget can afford! It hurt enough to put in 32 on top of the 16 I put in about 4 years ago. I just can't see investing much more for being a volunteer. At best another 1080 or better, but that's it. |
Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0 |
I have posted this many times before. They should make it a sticky if there were any moderator around to do it. Jim, I went back to 6.1 and I do not have problems. I can run all my projects there. Going back to 5.2 is a good place to start for trouble shooting Python VM problms, but this can affect other projects. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 12,116,986 RAC: 9,863 |
I have a Ryzen with 64GB, but it's my main computer. Less than that is pitiful by today's standards. It will take 128GB.Pythons for 12 hours? They average 2 hours here. I have two Boinc only machines with 36GB in them. I upped them just enough to run LHC. |
Jim1348 Send message Joined: 19 Jan 06 Posts: 881 Credit: 52,257,545 RAC: 0 |
I have a Ryzen with 64GB, but it's my main computer. Less than that is pitiful by today's standards. It will take 128GB. Most projects don't take nearly as much as the pythons or LHC of course. I like memory, but beyond 64 GB you have stability problems, since you have to use all four slots. Sometimes it works, but you often have to juggle memory around. You may have to spend more than you anticipated. Two slots is a lot safer. |
Falconet Send message Joined: 9 Mar 09 Posts: 354 Credit: 1,276,393 RAC: 2,018 |
Robetta, as far as I can tell, is separate from Rosetta@home and is used mostly by researchers outside of the Baker Lab/IPD. It's an interface for users who wish to get computing power for their jobs. The queue we see in the Rosetta@home page represents the jobs that the IPD directly submits to run at Rosetta@home + whatever gets submitted at Rosetta to run on Rosetta@home, the rb_11111_11111 jobs. If I could see the Rosetta@home queue, it would likely be close to 100% Rosetta Python jobs. The Pythons are refilled from the queue up to a max of 5,000 ready to send. I don't know why (server resources constraints?) but it's not like Rosetta@home can do a lot of these at any given time so no point in increasing that value, So yeah, those 2.6 million jobs on the queue are Pythons. Sid Celery posted something a few months ago that he received from Admin or someone like that who said that the Python job that had been submitted by one of the IPD researchers was "huge". I think one of the features on Robetta is to make sure that everything is open - that is, any researcher can see what is being worked on both present and past and maybe avoid duplication of work. I recall during the pandemic that they asked Robetta users to make sure their jobs were visible to others so that everyone could benefit. (I have a suspicion that researchers can hide their jobs - often times, I try to search for the jobs I'm running on my computers using the ID number but Robetta doesn't return any results). I don't know who said that but my impression has always been that work that comes from Robetta is labelled rb and everything else that isn't labelled rb is directly submitted to Rosetta@home. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 12,116,986 RAC: 9,863 |
Most projects don't take nearly as much as the pythons or LHC of course.Everything works better with more memory, if you're not using it you get a massive disk cache. Using all four slots does not cause stability problems. Always test your new memory with memtest before use, even quality stuff has duds. |
Mr P Hucker Send message Joined: 12 Aug 06 Posts: 1600 Credit: 12,116,986 RAC: 9,863 |
Sid Celery posted something a few months ago that he received from Admin or someone like that who said that the Python job that had been submitted by one of the IPD researchers was "huge".It's not that big, it's only a few million tasks. I've seen the queue at 15 million. But maybe that was several projects at once. |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1725 Credit: 18,380,064 RAC: 20,136 |
Ok..so then where do they get the million something tasks in queue? Back when there were Rosetta 4.20 Tasks, all those millions were Rosetta 4.30 Tasks and if you checked the Total queued jobs number it would gradually run down to zero (or jump up again as new work was released). Now most of the work is Python, and that's what that number shows. Extremely occasionally it jumps up again when that extremely rare batch of Rosetta 4.20 work is released., However most of the time it sits around the 2-2.7 million mark, this is because the amount of Python work being done is being done at roughly the same rate as new Python work is released. The Unset value in the Application task list is the amount of work that's ready to go for that particular application (i think the ratio is 6:1 Rosetta 4.20:Python). The Tasks ready to send value under the Computing Status is the Rosetta 4.20 & Python Tasks by application values combined. The Total queued jobs value is both the Rosetta 4.20 work, and the Python work and both the Unset 4.20 & Python work all combined. It is the total of all types of work at all stages of yet to be processed. Grant Darwin NT |
Message boards :
Number crunching :
Problems and Technical Issues with Rosetta@home
©2024 University of Washington
https://www.bakerlab.org