Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 84 · 85 · 86 · 87 · 88 · 89 · 90 . . . 311 · Next

AuthorMessage
mippi

Send message
Joined: 26 Mar 20
Posts: 5
Credit: 53,228,166
RAC: 0
Message 100884 - Posted: 30 Mar 2021, 9:19:36 UTC

I have the same problem: "Message from server: Rosetta needs 6675.72MB RAM but only 3432.76MB is available for use."
I use a Linux workstation with 12GB memory and 12 cores. I have never seen such message before.
ID: 100884 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Vester
Avatar

Send message
Joined: 2 Nov 05
Posts: 258
Credit: 3,651,260
RAC: 29
Message 100885 - Posted: 30 Mar 2021, 9:46:14 UTC

My PC had a shortage of memory message and only ran one of four WUs although it was only using 3.7GB of 8GB RAM. In Windows 10, I set Virtual Memory (page files) to 10000MB, upper and lower limits. The PC is now running four WUs instead of just one.
ID: 100885 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1734
Credit: 18,532,940
RAC: 17,945
Message 100886 - Posted: 30 Mar 2021, 9:59:11 UTC - in response to Message 100884.  

I use a Linux workstation with 12GB memory and 12 cores. I have never seen such message before.
With 12 cores you need more than 16GB of RAM- a Single Task can require over 4GB of RAM, although 1 to 1.3GB is more common.
For a while now the memory requirements for Tasks have been very low, but we did get a batch of work that required over 1GB of RAM shortly before we ran out of work for a while there.
Grant
Darwin NT
ID: 100886 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Grant (SSSF)

Send message
Joined: 28 Mar 20
Posts: 1734
Credit: 18,532,940
RAC: 17,945
Message 100887 - Posted: 30 Mar 2021, 10:01:29 UTC - in response to Message 100883.  

sudden appearance of additional memory demands
Not just memory: I’m now seeing
Rosetta needs 5472.67MB more disk space
(on an admittedly very small partition set aside for the BOINC data directory, but it’s been fine for the last year) – so it looks like the new batch is unusually resource-hungry…
With my 12 cores/threads & 0.2 day cache, the most disk space i've seen needed for Rosetta has been around 2.7GB. Usually it's around 2.2GB or less.
Grant
Darwin NT
ID: 100887 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile trevG

Send message
Joined: 5 Nov 13
Posts: 9
Credit: 687,475
RAC: 0
Message 100888 - Posted: 30 Mar 2021, 10:36:30 UTC

I've been doing Rosetta well enough for some time but now it seems greater than the 4gb installed RAM is needed for current projects.
Is this correct and the large swap file not any good?
From Boinc Tasks log:
4551 30-03-2021 11:20 (to change preferences, visit a project web site or select Preferences in the Manager)
4550 30-03-2021 11:20 max CPUs used: 3
4549 30-03-2021 11:20 max disk usage: 7.00 GB
4548 30-03-2021 11:20 max memory usage when idle: 3972.95 MB
4547 30-03-2021 11:20 max memory usage when active: 3445.92 MB
4546 30-03-2021 11:20 Preferences:
4545 30-03-2021 11:20 Reading preferences override file
4544 Rosetta@home 30-03-2021 11:20 General prefs: no separate prefs for home; using your defaults
4543 Rosetta@home 30-03-2021 11:20 Computer location: home
4542 Rosetta@home 30-03-2021 11:20 General prefs: from Rosetta@home (last modified 30-Mar-2021 11:20:31)
4541 Rosetta@home 30-03-2021 11:20 Project requested delay of 31 seconds
4540 Rosetta@home 30-03-2021 11:20 Message from server: Rosetta needs 6675.72 MB RAM but only 3972.95 MB is available for use.
4539 Rosetta@home 30-03-2021 11:20 No tasks sent
4538 Rosetta@home 30-03-2021 11:20 Scheduler request completed: got 0 new tasks
4537 Rosetta@home 30-03-2021 11:20 Requesting new tasks for CPU
4536 Rosetta@home 30-03-2021 11:20 Sending scheduler request: Requested by user.
4535 Rosetta@home 30-03-2021 11:20 update requested by user
ID: 100888 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2146
Credit: 41,570,180
RAC: 8,210
Message 100889 - Posted: 30 Mar 2021, 11:19:31 UTC - in response to Message 100879.  

Homepage says 65,000 tasks are available and I just got lots of tasks on my AMD Ryzen 5 2500U laptop.
None were Robetta tasks. Some resends but also some new ones.

Grabbed one lot of 11 then.
Within the last half hour I grabbed another 3 lots of 11, 2 of which were Robetta, but no more left to get now.
ID: 100889 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Garry Heather

Send message
Joined: 23 Nov 20
Posts: 10
Credit: 362,743
RAC: 0
Message 100890 - Posted: 30 Mar 2021, 11:31:18 UTC

My Pi 4 rig that was in difficulty earlier seems to have sorted itself out although it did experience a number of download errors beforehand. I've now got a cache of 2 days work so unless a number of those abort I should be OK while everything stabilises again.
ID: 100890 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 12 Aug 06
Posts: 1600
Credit: 12,116,986
RAC: 4,044
Message 100894 - Posted: 30 Mar 2021, 17:50:55 UTC - in response to Message 100873.  

Doesn't work if you use a wristwatch
What’s a wristwatch, Grandad? ;-⁠)
What do you do, take your phone out of your pocket? That's a backwards step.

will "bogeys at your 2 o'clock position" ever fade out due to the use of digital clocks? Or will we always have analogue ones?
Maybe more due to the removal of humans from air combat. Having simpler mechanisms, analogue clocks might make a comeback after the apocalypse…
I play Fallout 4. Apocalypses are fun! Although there are a few benefits in the game that break the laws of physics so would be difficult to implement in real life.
ID: 100894 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2146
Credit: 41,570,180
RAC: 8,210
Message 100895 - Posted: 30 Mar 2021, 18:16:52 UTC - in response to Message 100889.  

Homepage says 65,000 tasks are available and I just got lots of tasks on my AMD Ryzen 5 2500U laptop.
None were Robetta tasks. Some resends but also some new ones.

Grabbed one lot of 11 then.
Within the last half hour I grabbed another 3 lots of 11, 2 of which were Robetta, but no more left to get now.

Wasn't really paying attention but 4-5hrs later (~4hrs ago) more came on stream and 1m+ queued more recently
ID: 100895 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FurryGuy

Send message
Joined: 16 May 11
Posts: 2
Credit: 3,684,958
RAC: 0
Message 100897 - Posted: 30 Mar 2021, 18:49:20 UTC

I'm not getting out of memory or disk space messages, the tasks are simply "waiting to start."

I I suspended all other tasks from other projects, received a number of new Rosetta tasks. They all refused to do anything other than just sit there.

An idle CPU is not a good thing when it should be maxed out running tasks.
ID: 100897 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
strombergFs

Send message
Joined: 18 Mar 21
Posts: 11
Credit: 150,490
RAC: 0
Message 100898 - Posted: 30 Mar 2021, 18:56:10 UTC

My odroidC4 does not get anymore tasks.
i got a note: Rosetta required 6.x GB suddenly. Around 3.7 available.
I dont understand why this is, all the other days i could run 4 tasks without any problem.
Now i do not get even one tasks.
Also i have 5GB swap but Rosetta does not use any space of it, but this was also before i think.

My other two computers have again new tasks, all okay there.
Some ideas ?
Thanks a lot.
ID: 100898 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 100902 - Posted: 30 Mar 2021, 19:49:23 UTC - in response to Message 100897.  

the tasks are simply "waiting to start."
At the risk of sounding incredibly pernickety: is it ‘Ready to start’ or ‘Waiting to run’? (They mean subtly different things, and ‘waiting to start’ isn’t a state a task can be in…)

If the former, this sounds a lot like a problem that was reported recently: tasks sit ‘Ready to start’ even though nothing else is running and there’s no obvious reason why they can’t run. I’m not sure we ever understood the root cause of the issue at the time, but it seemed that the tasks wanted more memory than was available (even if in that case we’d expect them to say ‘Waiting for memory’). Fiddling with the memory settings in Computing preferences seemed to get them going. If that doesn’t work, try selecting the Event Log options cpu_sched_debug and mem_usage_debug and looking in the Event Log for clues.
ID: 100902 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 100903 - Posted: 30 Mar 2021, 20:02:11 UTC - in response to Message 100898.  

My odroidC4 does not get anymore tasks.
i got a note: Rosetta required 6.x GB suddenly. Around 3.7 available.
I dont understand why this is, all the other days i could run 4 tasks without any problem.
Now i do not get even one tasks.
Also i have 5GB swap but Rosetta does not use any space of it, but this was also before i think.

My other two computers have again new tasks, all okay there.
There’s a new batch of work units today that are marked as needing 6.6 GB of memory, so the server won’t even send them to hosts it knows have less than that. I’m not sure swap space is taken into account in that decision. You might want to consider a different project until Rosetta comes back with tasks suited to smaller machines.

The i7 has only picked up resends of older tasks (with much smaller memory requirements).

The MacBook has returned a lot of errors lately. Might be a platform bug; who knows.
ID: 100903 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
strombergFs

Send message
Joined: 18 Mar 21
Posts: 11
Credit: 150,490
RAC: 0
Message 100905 - Posted: 30 Mar 2021, 20:18:39 UTC - in response to Message 100903.  

Thank you very much for the information.
Is that high memory size requirement normal ?
The last week since the C4 operates i had no problems.

I read before about Raspi4 and other arm boards that it is possible to run Rosetta on them with linux..
Now i have 15!x C4s waiting to operate for Rosetta.
A bit sad these unsafe information.

Is it very likely that "normally" tasks of around 1GB are available ?
Or was i just lucky these last days ?

Thanks.

Does somebody have more information regarding the use of swap with Rosetta?
Is there a reason the swap is not used ( at least not on the C4 until now) ?
Thanks
ID: 100905 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
strombergFs

Send message
Joined: 18 Mar 21
Posts: 11
Credit: 150,490
RAC: 0
Message 100906 - Posted: 30 Mar 2021, 20:19:49 UTC - in response to Message 100903.  

the MacBook errors are okay. I had one freeze, now all is okay again.
ID: 100906 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Brian Nixon

Send message
Joined: 12 Apr 20
Posts: 293
Credit: 8,432,366
RAC: 0
Message 100907 - Posted: 30 Mar 2021, 21:10:57 UTC - in response to Message 100905.  

Is that high memory size requirement normal ?
It wasn’t normal before today. Whether it becomes the new normal, we’ll have to wait and see. It’s also not yet clear whether these tasks do actually need all the memory and disk space they say they do, or whether it’s a misconfiguration.
ID: 100907 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sid Celery

Send message
Joined: 11 Feb 08
Posts: 2146
Credit: 41,570,180
RAC: 8,210
Message 100909 - Posted: 30 Mar 2021, 23:29:46 UTC - in response to Message 100903.  
Last modified: 30 Mar 2021, 23:34:44 UTC

My odroidC4 does not get anymore tasks.
i got a note: Rosetta required 6.x GB suddenly. Around 3.7 available.
I dont understand why this is, all the other days i could run 4 tasks without any problem.
Now i do not get even one tasks.
Also i have 5GB swap but Rosetta does not use any space of it, but this was also before i think.

My other two computers have again new tasks, all okay there.
There’s a new batch of work units today that are marked as needing 6.6 GB of memory, so the server won’t even send them to hosts it knows have less than that. I’m not sure swap space is taken into account in that decision. You might want to consider a different project until Rosetta comes back with tasks suited to smaller machines.

The i7 has only picked up resends of older tasks (with much smaller memory requirements).

The MacBook has returned a lot of errors lately. Might be a platform bug; who knows.

Interesting/weird...

Checking my main PC, which isn't reporting any disk or RAM errors, it has 18Gb RAM free (60+%), is running 16 tasks concurrently, but using less than 6Gb RAM total on Rosetta tasks and, for a while, 70% CPU total (though I'm having some weird problems on the whole PC that may be affecting that CPU usage figure, so don't pay attention to that)

Edit: the available RAM and disk space is definitely related to the settings in the Disk & Memory tab and doesn't account for swap space at all
ID: 100909 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100911 - Posted: 31 Mar 2021, 0:26:16 UTC - in response to Message 100883.  
Last modified: 31 Mar 2021, 0:26:54 UTC

Not just memory: I’m now seeing
Rosetta needs 5472.67MB more disk space
(on an admittedly very small partition set aside for the BOINC data directory, but it’s been fine for the last year) – so it looks like the new batch is unusually resource-hungry…


Rosetta needs 7437.13MB more disk space.

Isn't that adorable? Say hello to two less hosts after they finish their current tasks, @Rosetta. I don't know if I have the time that's required to provide the space that is needed.

"Makes use of unused COP cycles" Sounds so easy, doesn't it? I know that I have been here but a short time, but @Rosetta is higher maintenance than any turbulent girlfriend that I've ever had.
ID: 100911 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mippi

Send message
Joined: 26 Mar 20
Posts: 5
Credit: 53,228,166
RAC: 0
Message 100912 - Posted: 31 Mar 2021, 0:36:46 UTC - in response to Message 100886.  

I use a Linux workstation with 12GB memory and 12 cores. I have never seen such message before.
With 12 cores you need more than 16GB of RAM- a Single Task can require over 4GB of RAM, although 1 to 1.3GB is more common.
For a while now the memory requirements for Tasks have been very low, but we did get a batch of work that required over 1GB of RAM shortly before we ran out of work for a while there.


I have a few workstations with similar parameters and they work perfectly fine for many years, so I don't think I need 16GB memory. Moreover, Rosetta should be a project which is run in the background. So, I should not equip my computer to meet Rosetta requirements, but Rosetta should try to use my resources.
ID: 100912 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mrhastyrib

Send message
Joined: 18 Feb 21
Posts: 90
Credit: 2,541,890
RAC: 0
Message 100913 - Posted: 31 Mar 2021, 1:19:44 UTC - in response to Message 100912.  

Moreover, Rosetta should be a project which is run in the background. So, I should not equip my computer to meet Rosetta requirements, but Rosetta should try to use my resources.


I hear you, but all programs specify the minimum hardware requirements. My complaint would be that those requirements for @Rosetta seem to change without notice.

On the other hand, it seems like they don't have enough work to consistently utilize the hardware available to them. If that's the case, then it makes no sense to spend time fine-tuning the program so that even more capacity is available.
ID: 100913 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 84 · 85 · 86 · 87 · 88 · 89 · 90 . . . 311 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2024 University of Washington
https://www.bakerlab.org