Posts by Klimax

1) Message boards : Number crunching : Waiting for memory, have 60 GB RAM and 30 cores available to Boinc. (Message 109943)
Posted 30 Oct 2024 by Klimax
Post:
One of the major flaws of Windows, is if an application requests XXX amount of RAM to use, Windows reserves that XXX amount of RAM, even if the application never actually uses that XXX amount of RAM, but only X amount. And you don't have any control over what the application requests for RAM reservation requested. That comes from the task generator template values the admin/scientist sets up for the tasks to use for work generation.

OTOH, Linux does not do this and only allocates on the fly how much memory to reserve only at the request of the application when it actually runs. This issue is a big problem for tasks that need a huge amount of RAM reserved. The Windows users over at GPUGrid run into this issue all the time. The solution for Windows users is to set up a VERY large pagefile on the order of 50-64GB in size so that so that the when the application tells Windows to reserve 30GB of memory, it can actually do as requested by utilizing the pagefile.

It should be noted that programmers can just reserve memory without actual allocation by using VirtualAlloc function with specific flags (https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualalloc). Unfortunately, it means custom allocators.

BTW: I thought Linux stopped by default overcommitting too some time ago.
2) Message boards : Number crunching : Problems and Technical Issues with Rosetta@home (Message 109887)
Posted 17 Oct 2024 by Klimax
Post:
BTW: My cache is configured for 1+1. As long as computer is running 24h there soul be a day of reserve.
Setting it that way may not give you what you might expect it to.
If you want 2 days worth, then set it to 2+ 0.01.

Those additional days are just that- additional days. They will only be added on when the cache gets low enough to reach the "Store at least value" and it needs to be topped up. Then it will also top up the additional day, which will then run down again until the "Store at least value" is reached again.


With it set to 1+1 you will get one day's worth, plus another day's worth, but then the cache will run down to just under 1 day's worth, then it will refill the 1 day & then re-fill the second additional day.
With it set to 2+ 0.01 as it returns a Task, it will download another to keep the cache at the 2 days level.

After quick verification on another project... damn you are correct. I was misunderstanding that option for past 17 years or so.
3) Message boards : Number crunching : Problems and Technical Issues with Rosetta@home (Message 109859)
Posted 13 Oct 2024 by Klimax
Post:
Also only three days of deadline?
Which has been the case for years now, and which is why you don't return a large amount of the work you download- you miss the deadline almost 50% of the time.

Your initial Estimated completion times are set by the project at 8 hours, but your actual Target CPU time appears to be 12 hours. So even with a small cache, you would miss deadlines. You need no cache, or at the very most a very small cache to avoid missing deadlines (1 day or less).
Running more than one project there is no need for a cache at all. If one project doesn't have work, the other will make up that gap, till the first project has work again. No missed deadlines, no not getting work from one project because the other has filled the cache with work.

Ideally
           Store at least 0.02 days of work
Store up to an additional 0.01 days of work

But if you really feel the need for a cache
           Store at least 0.35 days of work
Store up to an additional 0.01 days of work
would be plenty.


If all of your projects have server issues/lots of periods with no work, then "Store at least xx days of work could be set to 1 day (or 1.5 days if you go with the default Target CPU time of 8 hours).

While on surface or first pass you'd be correct, things are bit more complex (Side note: My question about deadlines shows how long ago I fully paid attention to project).

First, variable memory consumption. Often several tasks are waiting for others to finish or to drop currently allocated memory (Seems that nlike others, I got tasks that keep all 2GB allocated even way later).

Second, BOINC or Rosetta has very weird accounting of remaining time to completion. (Lots of task are now around 50% mark yet estimated time to completion is still 12 hours, while waiting tasks have 8 hours. (And yes, my target runtime is 12 hours)

All those canceled were in any case from beginning of month were computer was configured for all 20 cores to be used and ran out of memory very fast making lost of tasks waiting for memory and thus lots of cancellations. I have since then changed configuration to use only 10 cores, so same situation shouldn't occur.

BTW: My cache is configured for 1+1. As long as computer is running 24h there soul be a day of reserve.

ETA: Looks like some tasks finally dropped in memory usage to 1GB and one to 700MB.
4) Message boards : Number crunching : Problems and Technical Issues with Rosetta@home (Message 109850)
Posted 12 Oct 2024 by Klimax
Post:
All of my tasks are using after several hours about 2GBs. Also only three days of deadline? Will be tight.
5) Message boards : Number crunching : Problems and Technical Issues with Rosetta@home (Message 109801)
Posted 2 Oct 2024 by Klimax
Post:
So that explains why WUs are failing on one of my computers. 20 threads and only 16GB of RAM and fairly small paging file. They could have warned us...
It's been the general rule of thumb since i've been here (a bit over 4 years)- 1.5GB of RAM per core/thread is needed in order to do Rosetta work.
It's only been recently with the Beta application that Tasks have used less (there were batches of Rosetta 4.20 work that have Tasks that used 2- 4GB each).


Edit-
Interestingly- on one system all running tasks are using up to 1.6GB of RAM, on the other only 2 are using more than 1GB of RAM, the rest 400-700MB.

Argh. I just ( after writing a reply) realized what happened. NumberFields uses OpenCL for multiprecision arithmetic and OCL compiler will during compilation use up lots of RAM (sharp increase to few GBs, after it it will return back to fairly small footprint). So I have enough of virtual memory, when it's not being exhausted by another project...
6) Message boards : Number crunching : Problems and Technical Issues with Rosetta@home (Message 109798)
Posted 2 Oct 2024 by Klimax
Post:
Glitch fixed, and lo and behold- new work!


And an interesting batch it is- It's Beta 6.06 work, but they're using 1.4 to 1.7GB of RAM each, and it looks like their target Runtime is 8 hours (unlike the usual 200-400MB of RAM and 3 hrs runtime of previous Beta Tasks.

So that explains why WUs are failing on one of my computers. 20 threads and only 16GB of RAM and fairly small paging file. They could have warned us...
7) Message boards : Rosetta@home Science : Unable to get any work (Message 108750)
Posted 7 Dec 2023 by Klimax
Post:
Hi there. Apologies if this is posted in the wrong area, but I am unable to get work.
I bought this new computer about a month ago. it has an Intel(R) Arc(TM) A770 Graphics card and a 13th Gen Intel(R) Core(TM) i7-13700F processor.
I hope you can help.
I've tried attaching to several boinc projects, and I'm not getting any work.
Is this just bad luck or is something else wrong?


Try PrimeGrid and Einstein, Asteroids also has cpu and gpu tasks but I don't know if they can use your Intel Arc gpu or not

Also NumberFields support Intel GPUs.
8) Message boards : Number crunching : สล็อตเว็บตรง (Message 108538)
Posted 28 Aug 2023 by Klimax
Post:
There might be a bug in report feature. (There's PHP warning message at the top of page on successful report)
9) Message boards : News : Rosetta's role in fighting coronavirus (Message 108533)
Posted 27 Aug 2023 by Klimax
Post:
Is Rosetta@home still running? I've been a member of this project for years, and I'm not getting any tasks. I looked at my Projects page and it says "Won't get new tasks." What did I do? How do I get new tasks?

Pretty sure "Won't get new tasks" tells BOINC not to request any new work from Project.

Recently there were some new batches of work.
10) Message boards : Cafe Rosetta : DreamLab now available for Android and iOS (Message 93954)
Posted 9 Apr 2020 by Klimax
Post:
Hm, mobile only and it is not clear if it is even available outside of UK.
11) Message boards : Number crunching : Rosetta@home using AVX / AVX2 ? (Message 93952)
Posted 9 Apr 2020 by Klimax
Post:
It's time for SSEx/Avx support??

Are there even any x86-64 CPUs without at least SSE2 support?


SSE2 from Wiki:
Introduced by Intel with the initial version of the Pentium 4 in 2000... AMD added support for SSE2 in 2003

It should be noted, that x86-64 mandates SSE2 support and as such any 64-bit CPU supports it.
12) Message boards : Number crunching : How might memory effect R@h processing? (Message 93463)
Posted 5 Apr 2020 by Klimax
Post:
[snip]

Yes, smt/hyper threading is off. The point is though, on my Intel laptops, the speed has remained the same between 4.07 to 4.12. On my tr it's dropped over 60% regardless of what I try, I can't put it much simpler than that.

Note that all of your Intel laptops have a much lower number of cores, and will therefore have much less of a problem with too many cores trying to share the limited speed path to main memory. I can't put it much simpler than that, either.



4.07 is the reference yes. 4.12 is 60% slower than that, I don't know what is so difficult to understand, it's has nothing to do with memory or anything else. You dont have a 64/128 chip to only run it on 5 cores to keep the same productivity because of a software change. Is there a mod/developer that can possibly comment on this issue?

Resource contention, nothing less, nothing more. Nothing exotic or unknowable. ZEN architecture is strong on memory bandwidth but bad on memory latency. Even if you have relatively larger cache there will be still lots of memory access that has to be fulfilled from main memory. Because of that too many core doing access will overload memory bus and their performance will drop of sharply. (Especially when latency of memory access is already high)

I suggest you experiment with number of concurrent task for RAH to find equilibrium. Alternately, you could try to see if you can get memory to higher frequency without worsening its timing. Also ensure that there is no swapping. (IIRC each 4.12 task needs about 2GB of RAM which amounts to about 128GB of RAM in use beside all other processes already running)
13) Message boards : Number crunching : Stalled downloads (Message 93194)
Posted 3 Apr 2020 by Klimax
Post:
So maybe allow repeated attempts to download for a maximum 24hrs, else abort? But could be a loss of connectivity at the user end, or at the project end?


Yes, some means for BOINC to give up is required. Perhaps it should wait even longer if the file is "particularly large" (intentionally left for others to define). The other issue is that in the case here at R@h the problems seemed to be with small files, but the small file was required to run a given WU. That WU might have had many GBs of other required files that came down ok. So the actual size of the particular file having problems is not really the only consideration. If you abort the WU, all of the WU files are being lost (unless other WUs your machine has on-board use them as well).

I should think the BOINC Manager can see the difference between a lack of connectivity being the cause, and a dropped frame. So that info. could be used. If there was a loss of connectivity, then perhaps you double the maximum timeout.

...all good discussion for BOINC message boards.


Connectivity detection is also partially broken in BOINC clients:
If BOINC client can not download file or contact project server it contacts "reference site" which is google.com for last few years.
If it get "HTTP 200 - OK" response from google - all work as expected: BOINC understands there are problem with particular project but Internet connection is OK and proceeds with other projects.

But if google return any other response(not "200 OK") - BOINC stupidly interprets it as connectivity issues (no internet connection) and temporarily ceases all internet activity and increase "back-off" timer each time.
And Google often returns other codes - because a lot of such similar automated requests from thousands of computers around the world running BOINC often trigger the anti-bot filters of Google (it blocks such requests and offers to solve CAPTCHA to continue work). BOINC of course can not understand it and react to this by declaring "connectivity issues".

Although the fact that it was able to get an error message from Google (no matter which and for what reason) means the exactly opposite - that the Internet connection is working OK.

I think it is work around for various captive portals and security appliances/gateways that might block connection with custom error page.
14) Message boards : Number crunching : Rosetta 4.1+ and 4.2+ (Message 92859)
Posted 1 Apr 2020 by Klimax
Post:
Would it be possible to support Windows on ARM (WOA) too?
15) Message boards : News : Rosetta's role in fighting coronavirus (Message 92718)
Posted 31 Mar 2020 by Klimax
Post:
I was surprised when I saw a comment from someone who had queue for a couple of days. I joined the project about 10 days ago and I usually got 2-3 extra tasks besides the 12 running tasks. My tasks take 7-8 hours to complete so I never got a large queue. Wonder why there's such a big difference.

Also since yesterday I didn't receive any new tasks and I'm now running the last 5 tasks.

BOINC settings. For example I have 1 day of work + extra 1 day as a buffer with Rosetta runtime set at 12 hours.
16) Message boards : Number crunching : Problems with Minirosetta v1.54 (Message 60540)
Posted 7 Apr 2009 by Klimax
Post:
Klimax, why don't you go ahead and take a dump and EMail it to me, along with details on what you observered with it as it ran. I will forward it to the Project Team.

Ups,didn't know :-(
Last time I reported it,I was told to let it finish and upload.(IIRC)
Mail is being prepared.
17) Message boards : Number crunching : Problems with Minirosetta v1.54 (Message 60526)
Posted 7 Apr 2009 by Klimax
Post:
Again another task is now not crunching due to "Accepted Energy:1.#QNAN" and "Accpeted RMSD:1.#QQ".
It is 39.50% Complete ; Model:11 Step 7788. I have now suspended task.

I can create dump file.Should I?

Or is it already fixed in next version?
18) Message boards : Cafe Rosetta : S'tel klat drawkcab! (Message 59708)
Posted 21 Feb 2009 by Klimax
Post:
Siht si ssendab...
19) Message boards : Number crunching : Problems with Minirosetta v1.54 (Message 59465)
Posted 8 Feb 2009 by Klimax
Post:
Hello.
Following task (http://boinc.bakerlab.org/rosetta/result.php?resultid=225859224) is suspended as it has produced "accepted energy": QNAN(Not a Number?) and RMSD: QO.Model number 25 step 9518. Running time: 20h 2min 21sec.
Set runtime 24h.
For now suspended.No crash before.
OS:Windows 7 beta.I can create dump file using task manager.

Should I let it try to finish?

Thanks


I'd suggest allowing it to run normally. Was it still using CPU time? If you want to kind of cut it off, but get it to report in, let it run, then exit (not close) BOINC and restart it, let it run about 2 minutes, then exit again and restart, until you've done that 5 times and the task should be ended and report in with "too many restarts".


OK,set runtime at 8hours,so watchdog would cut it at 24hours.It has now uploaded and reported it.I have dump files as well,if somebody in team is interested.(Captured at reported time and step)
And I see I was not alone... :-(
20) Message boards : Number crunching : Problems with Minirosetta v1.54 (Message 59418)
Posted 7 Feb 2009 by Klimax
Post:
Hello.
Following task (http://boinc.bakerlab.org/rosetta/result.php?resultid=225859224) is suspended as it has produced "accepted energy": QNAN(Not a Number?) and RMSD: QO.Model number 25 step 9518. Running time: 20h 2min 21sec.
Set runtime 24h.
For now suspended.No crash before.
OS:Windows 7 beta.I can create dump file using task manager.

Should I let it try to finish?

Thanks


Next 20



©2026 University of Washington
https://www.bakerlab.org