Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 172 · 173 · 174 · 175 · 176 · 177 · 178 . . . 309 · Next

AuthorMessage
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 104708 - Posted: 7 Feb 2022, 19:59:31 UTC - in response to Message 104703.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

[snip]

Good grief..do I need 4 massive sticks of memory now? I just dropped a 150 on two big sticks. Plus 65 on a new 550GB drive (78% free) I think I might put a project limit of 12 tasks into play unless someone has a way of isolating Python with a code? I've put to much money into this computer now.
Lightweight, I have 64GB in my main machine. 3 of my machines will take 128GB if necessary.



LOL you got money to burn or a good connection on parts.
ID: 104708 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104709 - Posted: 7 Feb 2022, 20:10:05 UTC - in response to Message 104708.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

[snip]

Good grief..do I need 4 massive sticks of memory now? I just dropped a 150 on two big sticks. Plus 65 on a new 550GB drive (78% free) I think I might put a project limit of 12 tasks into play unless someone has a way of isolating Python with a code? I've put to much money into this computer now.
Lightweight, I have 64GB in my main machine. 3 of my machines will take 128GB if necessary.



LOL you got money to burn or a good connection on parts.
RAM costs less than a tank of petrol.
ID: 104709 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104711 - Posted: 7 Feb 2022, 20:38:47 UTC - in response to Message 104700.  
Last modified: 7 Feb 2022, 21:28:52 UTC

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

BOINC complained with this:

LHC@home: Notice from server
CMS Simulation needs 7629.39MB more disk space. You currently have 0.00 MB available and it needs 7629.39 MB.
2/7/2022 7:55:21 AM DESKTOP-LFM92VN
--------------------------------------------------------------------------------
Rosetta@home: Notice from server
rosetta python projects needs 19073.49MB more disk space. You currently have 0.00 MB available and it needs 19073.49 MB.
2/7/2022 7:55:34 AM DESKTOP-LFM92VN

snip ,snip..
does that meen you don't have "Leave non gpu tasks in memory while suspended" set , you will loose work past the last checkpoint but it uses more ram.
see my thread https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14903
All about the dreded "messing you about from server"
The trick iz Give it loadza disk space , everything the disk has got.
and my memory settings are "when in use 99%" when not in use 100%"
The story continues . . . .
ID: 104711 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1725
Credit: 18,382,444
RAC: 19,446
Message 104712 - Posted: 7 Feb 2022, 21:17:10 UTC - in response to Message 104700.  

Good grief..do I need 4 massive sticks of memory now?
No, you just need to read the messages that you are getting.
Every single one of the messages you posted refers to disk space, not RAM. And it's not just Pythion- it is all of your projects complaining about a lack of disk space
That is disk space, not RAM.

All of your projects need more disk space, but there is 0 MB available.


LHC@home: Notice from server
CMS Simulation needs 7629.39MB more disk space.
LHC@home: Notice from server
ATLAS Simulation needs 9536.74MB more disk space.
QuChemPedIA@home: Notice from server
NWChem needs 4096.03MB more disk space.
Rosetta@home: Notice from server
rosetta python projects needs 19073.49MB more disk space.
SiDock@home: Notice from server
CurieMarieDock on BOINC + zipped input, checkpoints and progress bar needs 128.00MB more disk space.

And the common denominator in all of those messages.
You currently have 0.00 MB available
All of your projects need more disk space.
Grant
Darwin NT
ID: 104712 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104713 - Posted: 7 Feb 2022, 21:25:17 UTC - in response to Message 104712.  
Last modified: 7 Feb 2022, 21:25:53 UTC

Good grief..do I need 4 massive sticks of memory now?
No, you just need to read the messages that you are getting.
Every single one of the messages you posted refers to disk space, not RAM. And it's not just Pythion- it is all of your projects complaining about a lack of disk space
That is disk space, not RAM.

All of your projects need more disk space, but there is 0 MB available.


LHC@home: Notice from server
CMS Simulation needs 7629.39MB more disk space.
LHC@home: Notice from server
ATLAS Simulation needs 9536.74MB more disk space.
QuChemPedIA@home: Notice from server
NWChem needs 4096.03MB more disk space.
Rosetta@home: Notice from server
rosetta python projects needs 19073.49MB more disk space.
SiDock@home: Notice from server
CurieMarieDock on BOINC + zipped input, checkpoints and progress bar needs 128.00MB more disk space.

And the common denominator in all of those messages.
You currently have 0.00 MB available
All of your projects need more disk space.
No, Boinc has a severe bug. I very much doubt he has precisely 0.00MB available. I think it's certain sizes of disk, it wraps around (as though they've used a small integer instead of a large one). I get around it by adjusting the amount of disk Boinc is allowed to use by a small amount. Last time I changed it from 95% to 100%. The disk was only half full, but crappy Boinc can't add up.
ID: 104713 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104714 - Posted: 7 Feb 2022, 21:27:19 UTC - in response to Message 104711.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

BOINC complained with this:

LHC@home: Notice from server
CMS Simulation needs 7629.39MB more disk space. You currently have 0.00 MB available and it needs 7629.39 MB.
2/7/2022 7:55:21 AM DESKTOP-LFM92VN
--------------------------------------------------------------------------------
Rosetta@home: Notice from server
rosetta python projects needs 19073.49MB more disk space. You currently have 0.00 MB available and it needs 19073.49 MB.
2/7/2022 7:55:34 AM DESKTOP-LFM92VN

snip ,snip..
does that meen you don't have "Leave non gpu tasks in memory while suspended" set , you will loose work past the last checkpoint but it uses more ram.
see my thread https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14903
All about the dreded "messing you about from server"
The trick iz Give it loadza disk space , and my memory settings are "when in use 99%" when not in use 100%"
The story continues . . . .
I put 100% everything, since it never uses what it says it will.
ID: 104714 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104715 - Posted: 7 Feb 2022, 21:36:38 UTC - in response to Message 104714.  

I put 100% everything, since it never uses what it says it will.

Put a silly big number in the "use no more than" box , like 1000GB , untick the other two , boinc works it out and can use all possible.
at least that's the theory I am working with at the moment.
ID: 104715 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104716 - Posted: 7 Feb 2022, 21:43:58 UTC - in response to Message 104715.  
Last modified: 7 Feb 2022, 21:46:04 UTC

I put 100% everything, since it never uses what it says it will.
Put a silly big number in the "use no more than" box , like 1000GB , untick the other two , boinc works it out and can use all possible.
at least that's the theory I am working with at the moment.
I think you can put 0 in a lot of boxes to mean infinity. But I seem to be using:

Use at most 1000GB of disk space
Leave at least 0.1GB of disk space
Use at most 99% of total disk space
Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)
Use at most 100% of page file (I really don't understand this option. The page file is managed by windows, Boinc doesn't know about it. Boinc sees it all as RAM.)
Use at most 100% memory while computer is in use
Use at most 100% memory while computer is idle
tick Leave application in memory while suspended.

These options may differ slightly from what you see as I don't use the standard Boinc Manager. It's a horrid outdated simple black and white interface which doesn't even group a queue of 300 tasks into one line, it's totally unusable. I use Boinctasks.
ID: 104716 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 104717 - Posted: 7 Feb 2022, 23:01:48 UTC - in response to Message 104709.  

I have 49 gigs of memory and Python x 13 used up close to 65% of that memory and nothing else could run.
Then BOINC got a brain and sidelined a bunch and put SiDock x 5 into play. and I am down to 56% usage.

Such pigs of memory these Pythons.

[snip]

Good grief..do I need 4 massive sticks of memory now? I just dropped a 150 on two big sticks. Plus 65 on a new 550GB drive (78% free) I think I might put a project limit of 12 tasks into play unless someone has a way of isolating Python with a code? I've put to much money into this computer now.
Lightweight, I have 64GB in my main machine. 3 of my machines will take 128GB if necessary.



LOL you got money to burn or a good connection on parts.
RAM costs less than a tank of petrol.



Not for me...32 gigs at my usual store from Corsair is 148 euro.
Tank of fuel 98 unleaded 104 for at 1.71/L
ID: 104717 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104718 - Posted: 8 Feb 2022, 0:04:24 UTC - in response to Message 104717.  
Last modified: 8 Feb 2022, 0:05:35 UTC

RAM costs less than a tank of petrol.
Not for me...32 gigs at my usual store from Corsair is 148 euro.
Tank of fuel 98 unleaded 104 for at 1.71/L
113 Euros here for 1 x 32GB DDR4 3200MHz. Tank of 50 litres standard unleaded is 83 Euros. Similar. Ok, it's less than a seatbelt or speeding fine.
ID: 104718 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104719 - Posted: 8 Feb 2022, 1:21:10 UTC - in response to Message 104717.  

Greg, technical questions are being asked, you can respond here, or tell me what to say if you don't want to create an account.
https://github.com/BOINC/boinc/issues/4629#issuecomment-1032102228
ID: 104719 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
.clair.

Send message
Joined: 2 Jan 07
Posts: 274
Credit: 26,399,595
RAC: 0
Message 104720 - Posted: 8 Feb 2022, 1:27:33 UTC
Last modified: 8 Feb 2022, 1:51:49 UTC

Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)

If those kind of settings don't work , nothing will
Boinc does do many small writes to the disk all the time , I am sad enough to have sat and watched `Resource Monitor` trying work what it was doing.
The wording in the boinc manager settings is for a full check point
"Request tasks to checkpoint at most every 60 seconds"
Having had a read this thread Rosetta Python floods disks with snapshots
I changed my checkpoint interval to 3600 seconds to reduce the number of checkpoints it does to save on disk wrights , a few seconds of time
and for the sake of generaly messing about with the system.
and reading the output files of valid work units it now only does one full checkpoint per hour instead of six.
The default for python in the output file is 600 the BM setting does change it
I only restart them when I have to , not a setting recommended for regular shutdown and restarted Pc`s
ID: 104720 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104721 - Posted: 8 Feb 2022, 2:08:25 UTC - in response to Message 104720.  

Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)

If those kind of settings don't work , nothing will
Boinc does do many small writes to the disk all the time , I am sad enough to have sat and watched `Resource Monitor` trying work what it was doing.
The wording in the boinc manager settings is for a full check point
"Request tasks to checkpoint at most every 60 seconds"
Having had a read this thread Rosetta Python floods disks with snapshots
I changed my checkpoint interval to 3600 seconds to reduce the number of checkpoints it does to save on disk wrights , a few seconds of time
and for the sake of generaly messing about with the system.
and reading the output files of valid work units it now only does one full checkpoint per hour instead of six.
The default for python in the output file is 600 the BM setting does change it
I only restart them when I have to , not a setting recommended for regular shutdown and restarted Pc`s
Well mine says 10 minutes, but the writing is continuous. I just ran some rare covid GPU units from WCG. The disk was doing 20MB/sec write non-stop. Another Boinc screwup.
ID: 104721 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1725
Credit: 18,382,444
RAC: 19,446
Message 104723 - Posted: 8 Feb 2022, 2:49:38 UTC - in response to Message 104721.  

Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)
Well mine says 10 minutes, but the writing is continuous. I just ran some rare covid GPU units from WCG. The disk was doing 20MB/sec write non-stop. Another Boinc screwup.
What write to disk setting?
I've looked and can't any such setting in the Rosetta project settings or in the Computing preferences settings. How can something that doesn't exist be a screwup?
Grant
Darwin NT
ID: 104723 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104724 - Posted: 8 Feb 2022, 2:56:03 UTC - in response to Message 104723.  

Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)
Well mine says 10 minutes, but the writing is continuous. I just ran some rare covid GPU units from WCG. The disk was doing 20MB/sec write non-stop. Another Boinc screwup.
What write to disk setting?
I've looked and can't any such setting in the Rosetta project settings or in the Computing preferences settings. How can something that doesn't exist be a screwup?
If you're using the horrid Boinc Manager, I think it's under "Request tasks to checkpoint" under computing tab in computing preferences.
ID: 104724 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1725
Credit: 18,382,444
RAC: 19,446
Message 104725 - Posted: 8 Feb 2022, 3:10:01 UTC - in response to Message 104724.  
Last modified: 8 Feb 2022, 3:10:37 UTC

Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)
Well mine says 10 minutes, but the writing is continuous. I just ran some rare covid GPU units from WCG. The disk was doing 20MB/sec write non-stop. Another Boinc screwup.
What write to disk setting?
I've looked and can't any such setting in the Rosetta project settings or in the Computing preferences settings. How can something that doesn't exist be a screwup?
If you're using the horrid Boinc Manager, I think it's under "Request tasks to checkpoint" under computing tab in computing preferences.
Account page, Computing preferences, Computing, Other,
Request tasks to checkpoint at most every xx seconds
is as it says- It is a request, not a hard limit or requirement.

It is up to each application that is running as to when it actually checkpoints- if the project programmer chooses not to take input from the user settings, then that's somthing you need to takae up with that project. It is not a BOINC failure.
The most frequent complaint has been about applications not checkpointing often enough, and people losing work.
Grant
Darwin NT
ID: 104725 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 104727 - Posted: 8 Feb 2022, 7:29:12 UTC - in response to Message 104719.  

Greg, technical questions are being asked, you can respond here, or tell me what to say if you don't want to create an account.
https://github.com/BOINC/boinc/issues/4629#issuecomment-1032102228



I'm there adding my remarks, thanks
ID: 104727 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104731 - Posted: 8 Feb 2022, 18:47:36 UTC - in response to Message 104725.  

Write to disk at most every 600 seconds (this is a pointless setting which is blatantly ignored)
Well mine says 10 minutes, but the writing is continuous. I just ran some rare covid GPU units from WCG. The disk was doing 20MB/sec write non-stop. Another Boinc screwup.
What write to disk setting?
I've looked and can't any such setting in the Rosetta project settings or in the Computing preferences settings. How can something that doesn't exist be a screwup?
If you're using the horrid Boinc Manager, I think it's under "Request tasks to checkpoint" under computing tab in computing preferences.
Account page, Computing preferences, Computing, Other,
Request tasks to checkpoint at most every xx seconds
is as it says- It is a request, not a hard limit or requirement.

It is up to each application that is running as to when it actually checkpoints- if the project programmer chooses not to take input from the user settings, then that's somthing you need to takae up with that project. It is not a BOINC failure.
The most frequent complaint has been about applications not checkpointing often enough, and people losing work.
Since virtually no project obeys this, it might aswell not be there. And on Boinctasks it's called "write to disk at most x seconds".
ID: 104731 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 399
Credit: 12,294,748
RAC: 6,222
Message 104732 - Posted: 8 Feb 2022, 18:52:22 UTC - in response to Message 104731.  

Since virtually no project obeys this, it might aswell not be there. And on Boinctasks it's called "write to disk at most x seconds".


That, surely, is BoincTasks and Efmer’s problem and nothing to do with Boinc.
ID: 104732 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 9,863
Message 104733 - Posted: 8 Feb 2022, 19:05:02 UTC - in response to Message 104732.  
Last modified: 8 Feb 2022, 19:06:52 UTC

Since virtually no project obeys this, it might aswell not be there. And on Boinctasks it's called "write to disk at most x seconds".
That, surely, is BoincTasks and Efmer’s problem and nothing to do with Boinc.
Presumably it was the Boinctasks author assuming "Request tasks to checkpoint at most every xx seconds" actually got obeyed once in a blue moon. Yes Boinctasks can have bugs, but it's 50 times easier to use than Boinc Manager. They really need to work on the interface. For example, I download 600 MW tasks at once to run on my GPU (just 2 hours work). Boinc Manager displays each of these on its own seperate line, no colour coding, nothing, I feel like I'm not in a GUI at all.
ID: 104733 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 172 · 173 · 174 · 175 · 176 · 177 · 178 . . . 309 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org