New kind of app on Ralph

Message boards : Number crunching : New kind of app on Ralph

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1860
Credit: 8,154,097
RAC: 8,001
Message 101970 - Posted: 31 May 2021, 10:19:19 UTC - in response to Message 101968.  

I would suggest posting about these issues in the Ralph forums.
These forums don't get much attention from the project, but since they are posting new applications to Ralph, there's a much greater chance of them seeing posts about Ralph issues with them over at Ralph than here at Rosetta.


No sense.
- ralph forum is a desert (i think admins don't read both the project forums)
- different numbers of app (1.02 here and 0.20 on Ralph). Is the same app? We don't know
- 1.02 is in production and not in beta on Ralph. So we post here bugs
ID: 101970 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mmonnin

Send message
Joined: 2 Jun 16
Posts: 54
Credit: 20,058,207
RAC: 3,589
Message 101972 - Posted: 31 May 2021, 12:51:13 UTC - in response to Message 101969.  

Apparently, when i got one and another one is in waiting the CPU crunches one of the pythons ones and nothing else.

Same here. I have a 12 cores AMD Ryzen and even if i stop all other wus, the system crunchs only 1 python.


You're out of memory. 7.6GB required per task.
ID: 101972 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1860
Credit: 8,154,097
RAC: 8,001
Message 101973 - Posted: 31 May 2021, 13:50:51 UTC - in response to Message 101972.  

You're out of memory. 7.6GB required per task.


Uh, i'll need other ram modules!
ID: 101973 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Daedalus

Send message
Joined: 1 Aug 08
Posts: 39
Credit: 9,974,441
RAC: 1,074
Message 101974 - Posted: 31 May 2021, 16:20:04 UTC - in response to Message 101969.  
Last modified: 31 May 2021, 16:24:33 UTC

Apparently, when i got one and another one is in waiting the CPU crunches one of the pythons ones and nothing else.

Same here. I have a 12 cores AMD Ryzen and even if i stop all other wus, the system crunchs only 1 python.


When i have one of these python jobs i can force rosetta to crunch classical WU's by cancelling all the pythons in the queue. Anyway it doesn't finish them all. I get a unmanageable VM message after several hours of my electricity wasted so i killed the tasks.

Edit: the funny thing is i don't get -of course- those python VM jobs on another machine which has no virtualbox installed. So a radical solution to this problem could be to uninstall virtualbox. I won't.I use it.
ID: 101974 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 1981
Credit: 38,436,901
RAC: 13,958
Message 101976 - Posted: 31 May 2021, 17:36:25 UTC - in response to Message 101966.  

I downloaded other 5 python wsus.
Some questions:
- is it possible to crunch more than 1 wu at once?
- will be possible to select this app in personal profile of the project?


I have several going at once on one PC.

Forever a wish here at Rosetta.

2 completed successfully here so far. Ralph had one exceed disk space.
Peak disk usage: 9,559.60 MB on good Rosetta task.

Peak working set size 3,058.41 MB
Peak swap size 4,704.56 MB
Peak disk usage 9,051.43 MB

No complaints about resource demands if they turn out to require this much to run...
ID: 101976 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
biodoc

Send message
Joined: 19 Feb 06
Posts: 14
Credit: 30,298,003
RAC: 7,011
Message 101977 - Posted: 31 May 2021, 18:44:11 UTC - in response to Message 101974.  

Apparently, when i got one and another one is in waiting the CPU crunches one of the pythons ones and nothing else.

Same here. I have a 12 cores AMD Ryzen and even if i stop all other wus, the system crunchs only 1 python.


When i have one of these python jobs i can force rosetta to crunch classical WU's by cancelling all the pythons in the queue. Anyway it doesn't finish them all. I get a unmanageable VM message after several hours of my electricity wasted so i killed the tasks.

Edit: the funny thing is i don't get -of course- those python VM jobs on another machine which has no virtualbox installed. So a radical solution to this problem could be to uninstall virtualbox. I won't.I use it.


An alternative to uninstalling VBOX is to add this line to the "options" section of your cc_config.xml file and then restart the boinc client.

<dont_use_vbox>1</dont_use_vbox>
ID: 101977 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Daedalus

Send message
Joined: 1 Aug 08
Posts: 39
Credit: 9,974,441
RAC: 1,074
Message 101978 - Posted: 31 May 2021, 19:11:19 UTC - in response to Message 101977.  

Thank you a lot ! Will do.
ID: 101978 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 101979 - Posted: 31 May 2021, 22:56:51 UTC - in response to Message 101966.  
Last modified: 31 May 2021, 23:03:58 UTC

Peak disk usage: 9,559.60 MB on good Rosetta task.

Good grief. And we can't select.
I think my 32 core Ryzen 3950X's (two of them) will need to find other things to do.
Their habit of ignoring their crunchers will be reciprocated very shortly.
ID: 101979 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1860
Credit: 8,154,097
RAC: 8,001
Message 101980 - Posted: 1 Jun 2021, 6:39:35 UTC - in response to Message 101974.  

When i have one of these python jobs i can force rosetta to crunch classical WU's by cancelling all the pythons in the queue. Anyway it doesn't finish them all. I get a unmanageable VM message after several hours of my electricity wasted so i killed the tasks.


This is why is a good idea to permit the choose of apps on user's profile.
ID: 101980 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
marmot

Send message
Joined: 10 Nov 15
Posts: 17
Credit: 2,388,247
RAC: 365
Message 101981 - Posted: 1 Jun 2021, 6:40:24 UTC - in response to Message 101979.  
Last modified: 1 Jun 2021, 6:40:53 UTC

And we can't select.
....
Their habit of ignoring their crunchers will be reciprocated very shortly.


I see this complaint often and I share it.

It's not hard to give us the ability to choose app/WU types.
ID: 101981 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile [VENETO] boboviz

Send message
Joined: 1 Dec 05
Posts: 1860
Credit: 8,154,097
RAC: 8,001
Message 101982 - Posted: 1 Jun 2021, 6:41:56 UTC - in response to Message 101979.  

Peak disk usage: 9,559.60 MB on good Rosetta task.

Good grief. And we can't select.
I think my 32 core Ryzen 3950X's (two of them) will need to find other things to do.
Their habit of ignoring their crunchers will be reciprocated very shortly.


A couple of 4Tb SSD and the problem is solved :-P
Despite the joke, this is a new page (maybe important) in Rosetta scientific research, but it's a pity that volunteers are ignored
ID: 101982 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 101986 - Posted: 1 Jun 2021, 12:28:23 UTC - in response to Message 101982.  

I have taken a preemptive strike, and will run half the cores of my Ryzen 3950X's on WCG (mostly MCM). They don't take much memory or disk space. Maybe I can still devote half the cores to Rosetta, but if they don't give us any choice on how many to run, I have to plan for the worst case. It would be nice if they allowed us to set the maximum number of Python's that we could run. That would make it more manageable.

By the way, I am running my first 1.02 Rosetta Python now, and BoincTasks is showing that it is using 7.6 GB of memory.
So disk space is not the only thing we need to worry about.

I agree that it may be a great project, but their customer relations are dismal, or non-existent to be more accurate.
How do they expect them to be crunched, anyway?
ID: 101986 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 101988 - Posted: 1 Jun 2021, 15:13:28 UTC - in response to Message 101986.  

By the way, I am running my first 1.02 Rosetta Python now, and BoincTasks is showing that it is using 7.6 GB of memory.

It completed successfully, but errored out on another machine that had only 8 GB. I think this will be a learning experience.
https://boinc.bakerlab.org/rosetta/result.php?resultid=1388548962

But maybe they will send you only one at a time? Even one will catch some people by surprise.
Some control at our end would be better.
ID: 101988 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Daedalus

Send message
Joined: 1 Aug 08
Posts: 39
Credit: 9,974,441
RAC: 1,074
Message 101989 - Posted: 1 Jun 2021, 15:51:08 UTC

I haven't changed my cc_config.xml yet but i don't receive python tasks anymore.
ID: 101989 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mmonnin

Send message
Joined: 2 Jun 16
Posts: 54
Credit: 20,058,207
RAC: 3,589
Message 101994 - Posted: 1 Jun 2021, 21:47:37 UTC - in response to Message 101988.  

By the way, I am running my first 1.02 Rosetta Python now, and BoincTasks is showing that it is using 7.6 GB of memory.

It completed successfully, but errored out on another machine that had only 8 GB. I think this will be a learning experience.
https://boinc.bakerlab.org/rosetta/result.php?resultid=1388548962

But maybe they will send you only one at a time? Even one will catch some people by surprise.
Some control at our end would be better.


I already mentioned the memory and disk space requirements as well as being able to run more than once at a time. If the PC has memory.
ID: 101994 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 101995 - Posted: 2 Jun 2021, 0:32:07 UTC - in response to Message 101994.  

... as well as being able to run more than once at a time. If the PC has memory.
But have you seen more than one running at a time?
ID: 101995 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 101997 - Posted: 2 Jun 2021, 7:45:04 UTC - in response to Message 101995.  
Last modified: 2 Jun 2021, 7:46:27 UTC

Never mind. Looking through my logs, I see I was running four at a time, which completed on 31 May, and then nothing more until the next day.
They ran from about 4 hours 50 minutes to 5 hours 45 minutes. Maybe more would have downloaded if I had more memory.
(I have 48 GB on that machine).
ID: 101997 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile eiernacken1983

Send message
Joined: 30 May 19
Posts: 5
Credit: 27,711,484
RAC: 83,970
Message 101998 - Posted: 2 Jun 2021, 8:15:29 UTC
Last modified: 2 Jun 2021, 8:50:26 UTC

My Threadripper is running some Rosetta python projects 1.02 (vbox 64):

The work units are now running for 2 days and 6 hours. Progress is 99.8 %, and further progress is advancing extremely slow.

Is there any chance of getting them done succesfully. Deadline is getting closer (7 hours left)...
ID: 101998 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 102001 - Posted: 2 Jun 2021, 12:01:23 UTC - in response to Message 101998.  
Last modified: 2 Jun 2021, 12:03:41 UTC

The work units are now running for 2 days and 6 hours. Progress is 99.8 %, and further progress is advancing extremely slow.

It looks like you have 20 of them downloaded. Depending on how many are trying to run at one time, you may be running into cache limitations,
or even worse swapping out memory to disk.
I would limit the number running to maybe four or so until you work out how many you can run at once.

I am limiting mine to two using an app_config.xml file placed in the project folder.
Create it with Wordpad, and save as an ".xml" file, then restart BOINC to activate:

<app_config>

<app>
   <name>rosetta_python_projects</name>
   <max_concurrent>2</max_concurrent>
</app>

</app_config>

ID: 102001 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 102002 - Posted: 2 Jun 2021, 13:45:21 UTC - in response to Message 102001.  

Create it with Wordpad, and save as an ".xml" file, then restart BOINC to activate:

Or you can use any text editor in Linux. Notepad would probably be simpler in Windows, just use the "save as" function to save it as an ".xml" file.
ID: 102002 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : New kind of app on Ralph



©2024 University of Washington
https://www.bakerlab.org