Swap files are for poor people without enough RAM :-)

I waz tiepin while you waz postin :-),

The caching for science projects works differently than if you are copying a video file, which would all have to be transferred. But in a scientific algorithm, you usually read from a location, do a calculation, a then store the value back, either into the original location or a related one. Therefore, by storing the information in DRAM memory, most of the writes are done to the memory. You transfer to the SSD only the residual writes remaining at the end of the cache latency period.

In fact, if you made the cache latency (write-delay) long enough, you would never have to transfer any of the writes to the SSD.
That is effectively what a ramdisk does, but it requires a lot more memory. You would have to store the entire BOINC data folder.

Modern SSDs take 3000 write cycles, pythons write about 2MB/s per task on a fast CPU, so if you have a 1TB SSD, that would last for 5 years even running 10 at once, by which time you'd want to buy a bigger one anyway. However hard disks hate moving heads back and forth and fall apart with that much random access.

Yes , some of the pythons need a kick in the compilers
Amazingly I have 31 python and 7 R4,2 tasks running ATM and I have been through them to clear out two 0 cpu dud work units , it is a pain having to that at least once a day
Rosetta is using 235GB of disk space though the most I have seen was 266GB
Ram use right now is 59GB total system use 71GB on `standby` and only 40MB `free` of 128GB fitted in 8 slots [crashes ?? wot crashes !! . . . . tic tic tic . . . BOOM :),
SSD write bombardment by pythons , following an idea by [Greg I think] I have put in a 500GB SSD Samsung 870 evo [£58 on ebay new still sealed]
I will see how long it lasts , though I haven't installed the additional "Samsung Magician" apps yet to keep an eye on the write rate , trim, garbage clean up etc
installed only boinc on it , to speed up python work unit loading times . it looked like the fastest kid on the block in benchmarks at low price , there is faster stuff out there at a high cost
I did look at M2 NVME drives but getting them to work in win7 looks like a pain of magical incantations on the command line to load the drivers , win8.1 onwards has them in already [I checked MS forum]
OK time to post this drivel on the forum and see what happens :)

Standby memory? Is that disk cache? In windows, all my RAM is always in use, but the disk cache just takes whatever is left. You can ignore that.
HWInfo (or many other free utilities) will show you the disk SMART data so you can see how much life is left. The drive reports % life left.
Why are you still on Windows 7? 10 was free. NVME is about 8 times faster. If your MB does have a slot for it, you can get cards to take them.

Standby memory? Is that disk cache? In windows, all my RAM is always in use, but the disk cache just takes whatever is left. You can ignore that.
HWInfo (or many other free utilities) will show you the disk SMART data so you can see how much life is left. The drive reports % life left.
Why are you still on Windows 7? 10 was free. NVME is about 8 times faster. If your MB does have a slot for it, you can get cards to take them.

It probably is `Disk Cache`
In winders `Resource Monitor` in the `Memory` tab , an MS app built in to W7 etc if you can find it, I keep it `pinned` to the taskbar , it also does Disk , Network , CPU
It is using Win 7 Ultimate, I don't know if that version [ that the licence limit will work with twin CPUs sockets] was free , I know from win 8 onwards the max memory that is ok is now something like 192GB
unlike win 7 home that is limited to 16GB.
No M2 slot onboard , though have seen PCIe-M2 adapters are on Ebay

Modern SSDs take 3000 write cycles, pythons write about 2MB/s per task on a fast CPU, so if you have a 1TB SSD, that would last for 5 years even running 10 at once, by which time you'd want to buy a bigger one anyway. However hard disks hate moving heads back and forth and fall apart with that much random access.

On my Ryzen 3900X, I was seeing the OS write 4 TB/day (for 20 work units), or 46 MB/sec if I have my numbers right.
So you can do it if you limit the number of tasks to only a few at a time. And I think that Linux writes a bit less than Windows from what I can see, though it has other problems.

You can make it work if you are careful.

---

It is using Win 7 Ultimate, I don't know if that version [ that the licence limit will work with twin CPUs sockets] was free , I know from win 8 onwards the max memory that is ok is now something like 192GB
unlike win 7 home that is limited to 16GB.
No M2 slot onboard , though have seen PCIe-M2 adapters are on Ebay

I had my machines some on Win 7 home and some win 7 ultimate. They all got an upgrade to Win 10 home or win 10 pro for free. But I don't think they still do it, unless you fiddle with the settings and say you're disabled!

EDIT: The only thing I see is this.
https://www.windowscentral.com/how-manage-disk-write-caching-external-storage-windows-10
That is just disk write caching, as I previously discussed. It uses only a small amount of memory, not the GB that you need to protect the SSDs from the pythons.

It's still system RAM, just a very small amount. Hence the need for 3rd party software if you want to make more use of your RAM for system caching.

---

Grant
Darwin NT

Modern SSDs take 3000 write cycles, pythons write about 2MB/s per task on a fast CPU, so if you have a 1TB SSD, that would last for 5 years even running 10 at once, by which time you'd want to buy a bigger one anyway. However hard disks hate moving heads back and forth and fall apart with that much random access.

On my Ryzen 3900X, I was seeing the OS write 4 TB/day (for 20 work units), or 46 MB/sec if I have my numbers right.

That sounds right, if you multiply my numbers up I come to 3.5TB a day for 20 work units. I was just guessing an average from watching the task manager. It'll differ depending on CPU speed, I was watching an i5 8600K.

So you can do it if you limit the number of tasks to only a few at a time. And I think that Linux writes a bit less than Windows from what I can see, though it has other problems.

You can make it work if you are careful.

I see an SSD as a consumable (like GPUs that wear out running 24/7). I get them dirt cheap second hand and expect to change them when they're too small or wear out. Most of my Boinc machines are on hard disks because I had loads kicking about not big enough for other uses. As they break I get SSDs. My main computer that gets thrashed all the time has 50% life left on the SSD, but it's an ancient model with old technology that didn't last so long, and it's getting too small anyway.

EDIT: The only thing I see is this.
https://www.windowscentral.com/how-manage-disk-write-caching-external-storage-windows-10
That is just disk write caching, as I previously discussed. It uses only a small amount of memory, not the GB that you need to protect the SSDs from the pythons.

It's still system RAM, just a very small amount. Hence the need for 3rd party software if you want to make more use of your RAM for system caching.

My main computer currently reads:
Memory in use: 22.1GB
Cache memory: 38.4GB
That's a big cache. Ok, reading up on it, you get 10% RAM disk write cache. So 6.4GB for me. Surely that's enough. Windows server will use 50% RAM.

EDIT: The only thing I see is this.
https://www.windowscentral.com/how-manage-disk-write-caching-external-storage-windows-10
That is just disk write caching, as I previously discussed. It uses only a small amount of memory, not the GB that you need to protect the SSDs from the pythons.

It's still system RAM, just a very small amount. Hence the need for 3rd party software if you want to make more use of your RAM for system caching.

Right. That is the point. You need a lot more. But I think .clair. mentioned Samsung Magician. I have used it when I was on Windows, and it includes around a GB, or maybe less, but could be enough to save an SSD if you did not run too many work units.

---

But I think they really need to develop the pythons a bit and call back when they are ready.

To be honest, i would classify the present Python work as being at Alpha test level of development- they are still not even good enough for Beta testing. They are no where near being ready for actual deployment IMHO.
Excessive system requirements. Errors that result in systems being black listed from getting work- but not even advising those systems of what has happened, let along informing them of what they need to do to get work again. And worst of all- so many Tasks that just don't process & sit there taking up disk & RAM, blocking possibly OK Tasks from being downloaded & worked on requiring manual intervention to remove them. Not to mention the manual intervention often needed to clean up the VirtualBox VM environments.

Yep, Alpha software- not yet remotely ready for live deployment.

---

Grant
Darwin NT

I think you have accurately portrayed it.

---

EDIT: The only thing I see is this.
https://www.windowscentral.com/how-manage-disk-write-caching-external-storage-windows-10
That is just disk write caching, as I previously discussed. It uses only a small amount of memory, not the GB that you need to protect the SSDs from the pythons.

It's still system RAM, just a very small amount. Hence the need for 3rd party software if you want to make more use of your RAM for system caching.

Right. That is the point. You need a lot more. But I think .clair. mentioned Samsung Magician. I have used it when I was on Windows, and it includes around a GB, or maybe less, but could be enough to save an SSD if you did not run too many work units.

My Windows 10 is using 10% of my RAM = 6.5GB for a write cache. And if you tick the box to turn off write cache buffer flushing, it helps even more. Right click the drive, properties, hardware, properties, change settings, policies, tick "turn off buffer flushing".

Right. That is the point. You need a lot more. But I think .clair. mentioned Samsung Magician. I have used it when I was on Windows, and it includes around a GB, or maybe less, but could be enough to save an SSD if you did not run too many work units.

I'd forgotten about Samsung Magician's Rapid Mode. Had a quick look and couldn't find any info on if the cache is adjustable at all, or if the more RAM you have the larger it is by default (as it would have to be smaller in a system with limited RAM).
The installation manual only shows it as using around 1.3GB of RAM when checking to see if it is installed and running, but it doesn't give any idea of how much RAM was on the system it was installed on.

---

Grant
Darwin NT

To be honest, i would classify the present Python work as being at Alpha test level of development- they are still not even good enough for Beta testing. They are no where near being ready for actual deployment IMHO.

I thought this was what Ralph@home was for?

Excessive system requirements.

Not really, try running LHC.

To be honest, i would classify the present Python work as being at Alpha test level of development- they are still not even good enough for Beta testing. They are no where near being ready for actual deployment IMHO.

I thought this was what Ralph@home was for?

That's meant to be for Beta testing.
These Python tasks haven't even reached that level of development IMHO.

Excessive system requirements.

Not really, try running LHC.

Excessive in the fact that what it requires to run is well in excess of what is ever actually used.

---

Grant
Darwin NT

I thought this was what Ralph@home was for?

That's meant to be for Beta testing.
These Python tasks haven't even reached that level of development IMHO.

Actually the "alph" in Ralph means alpha doesn't it? Anyway, testing is testing, it's a bit pedantic to say if it's alpha or beta. The point is on Ralph we don't mind stuff going wrong, so that's where it should be. It probably does work on their own machines, but fails when sent out to 1000s of different combinations of hardware, so it needs to go to Ralph.

Not really, try running LHC.

Excessive in the fact that what it requires to run is well in excess of what is ever actually used.

How do you know it's not using it?

I thought this was what Ralph@home was for?

That's meant to be for Beta testing.
These Python tasks haven't even reached that level of development IMHO.

Actually the "alph" in Ralph means alpha doesn't it?

Which shows how bad things are.
Even for public Alpha testing, it should at least be close to Beta test quality. Not just initial release Alpha testing.

Anyway, testing is testing, it's a bit pedantic to say if it's alpha or beta.

There is a huge difference between Alpha & Beta testing. Hence the different names. Getting things right isn't pedantic, it avoids messes like we presently have here with the Python work.

Not really, try running LHC.

Excessive in the fact that what it requires to run is well in excess of what is ever actually used.

How do you know it's not using it?

Because when we initially had hundreds of posts about people not being able to get work due to not enough RAM or DIsk space, others that did have work were posting what was actually being used. That got some of requirements reduced, but they are still excessive relative to what is actually being used.

---

Grant
Darwin NT

I had my machines some on Win 7 home and some win 7 ultimate. They all got an upgrade to Win 10 home or win 10 pro for free. But I don't think they still do it, unless you fiddle with the settings and say you're disabled!

I just had a quick look around the net and an upgrade to win10pro may still be possible
Maybe if I tell MS that I am insane and I boinc 24/7
You don`t have to be mad to work here , but it helps.

Actually the "alph" in Ralph means alpha doesn't it?

Which shows how bad things are.
Even for public Alpha testing, it should at least be close to Beta test quality. Not just initial release Alpha testing.

I would agree with you if it was say a freeware program to edit photos that you were letting the public use. They expect it to pretty much work. But we know we're testing stuff in Ralph, we don't care if hardly any tasks work, we know we're helping them test it. Why should they waste time and money testing it in house when we can do it? That's the point of Boinc, to help the researchers.

Anyway, testing is testing, it's a bit pedantic to say if it's alpha or beta.

There is a huge difference between Alpha & Beta testing. Hence the different names. Getting things right isn't pedantic, it avoids messes like we presently have here with the Python work.

That's like having two categories, fast cars and slow cars. Where do you draw the line?

How do you know it's not using it?

Because when we initially had hundreds of posts about people not being able to get work due to not enough RAM or disk space, others that did have work were posting what was actually being used. That got some of requirements reduced, but they are still excessive relative to what is actually being used.

Are you referring to how much RAM is required for the task to consider starting? Maybe that's hard to predict.