Posts by lakotamm

1) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 95409)
Posted 26 Apr 2020 by lakotamm
Post:

My pi 4 4gb is running at about 4 watts per my apc ups (I don’t have a more accurate way to measure). It only has HDMI turned off at the moment - on my next pi 4 I’ll be playing a little more with power settings.

The currently running pi 4 can be found here:
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4215281

I also went ahead and setup a pi 3B+. Trying to run 4 processes, I ended up running out of memory and thrashing, even with zram - 3 processes seems to be working much better.

Let's do a performance comparison in few days. I will keep mine running and then we can compare gained points. I had to change zram compression back to deflate and increase the size to 1,9GB. Afterwards, all seems fine.
2) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 95408)
Posted 26 Apr 2020 by lakotamm
Post:
Newer versions of zram also an option for mem_limit, have a look here:
https://www.kernel.org/doc/html/latest/admin-guide/blockdev/zram.html
You could set it to 70-80% of RAM for example while also keeping the max size at 200%. This would enable utilizing the swap as well (although a kernel with zswap would be more ideal for this).

After you measure the overhead that deflate causes (is it kswapd?), you may try some other compression algorithm as well.

I'll soon have a look at whether I can patch out malloc via LD_PRELOAD to enable KSM. That could help a lot.

You may want to see whether setting /proc/sys/vm/page-cluster to 0 could help with the zram overhead.
https://www.kernel.org/doc/html/latest/admin-guide/sysctl/vm.html#page-cluster


Thanks, I have 4 days to deliver my Bachelor thesis, so I will look into these things afterwards. Right now it is running fine without swap with max 520MB RAM taken while using 1,35GB ZRAM. I would like to try lz4hc compression as well, however it seems like it is not a part of the Raspbian 5.4 kernel.

The kswapd0 is definitely using a noticeable amount of CPU time. However the consumption is in "peaks". Once in a while it uses 100 percent and it sits idle most of the time.
According to top:
System has been up for 49h 20min = 2960min and that translates to 11760min of CPU time.
kswapd0 has been running for 238min and 22s.
boinc has been running for 92min and 38s
systemd has been running for 23min and 25s
wpa_cupplicant has been running for 3min 39s

According to this, the kswapd0 is taking 2,02% of the CPU time. Not too bad I would say.

i did that and committed 2 GB in there. it is somewhat a 'waste' of space on the SD card but i found it is actually useful.
there are quite a lot of tmpfs being mounted and perhaps some daemons running in the background at low priority.
i found that while boinc is running, the swap space is apparently used. i'd imagine that some of the tmpfs partitions are saved into swap instead.
that would actually free up some precious ram memory and allow r@h tasks to run

in terms of zram, i'm a little concerned that it may take more cpu 'effort' than simply swap

It could be a good idea to measure speed of writing to zram and flash. LZO/LZO-RLE/LZ4 compression should be fairly fast and I doubt that SD card is faster. When it comes to deflate, SSD drive will likely be faster. However I do not have one here, and I would really like to avoid writing to the SD card more than necessary.
3) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 95396)
Posted 26 Apr 2020 by lakotamm
Post:
- I am running 4 concurrent jobs on RPI 3B+, which is the same as on RPI 4
- do we have any consumption and performance numbers from RPI4?
4) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 95366)
Posted 25 Apr 2020 by lakotamm
Post:
An update from running 3B+ crunching few days.

- My RPI has so far reached avg credit 585.58.
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=3710842

- I personally doubt that RPI4 can be more efficient than well set up 3B+. The consumption of 3B+ should be only around 300mA with USB hub and ethernet disabled.

- I upgraded to kernel 5.4 which is currently being tested by the Raspbian team. There is almost no difference in benchmarks.

- rb_04 tasks seem have a variable rate of compressibility and this caused some troubles... Using LZO-RLE compression it happened to me that my RAM was completely taken by my ZRAM data, compressed with a factor of only 1.5. There was a lot of "virtual" space left on the ZRAM - only 1,45GB was taken, but there was no space to store more compressed data. SWAP was not getting used either, and the CPU utilisation was 50%, from that 30% was used for compressing and constant writing to and from the SD card. The system was well... almost totally irresponsive. Nasty.

After this experience, I switched back to deflate compression, I set the virtual ZRAM size to 1,88GB - simply the double of my RAM. I can tell that reading from and writing to ZRAM is quite slower. However, it seems to be working pretty fine.

zramctl:
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram3 deflate 481.5M 269.6M 113.7M 121.4M 4 [SWAP]
/dev/zram2 deflate 481.5M 269.1M 114M 121.4M 4 [SWAP]
/dev/zram1 deflate 481.5M 280.4M 119.3M 127.1M 4 [SWAP]
/dev/zram0 deflate 481.5M 280.4M 118.7M 126.6M 4 [SWAP]

I will update the guide for using deflate instead of lzo.
https://www.reddit.com/r/BOINC/comments/g0r0wa/running_rosetta_covid19_workunits_on_raspberry_pi/
5) Message boards : Number crunching : Rosetta running on ARM platforms (Message 95318)
Posted 24 Apr 2020 by lakotamm
Post:
Its interesting. I have one RPi3B+ with 1Gigs of RAM and it crunched a lot of WUs but meanwhile R@H increased minimum RAM requirements from 850MB to ~1700MB and now it hasn't got WU-s.

My RPI 3B+ is still happily crunching.
I often get the message that 1,7GB is required, but in the end I always get WUs. But it already happened that new WUs did not come for 24h, so I have a 1 day buffer.

I have also an OdroidC2 with 2Gis of RAM and it crunching fine with 2 WUs in parallel around 50°C.

Is this due to RAM limitations?

BTW my RPI4B+ (4Gig of RAM) crunching only one WU at a time because of stock heatshink and it runs around 55-60°C.

Could it make more sense to downclock it and run more WUs in parallel? CPU cores usually become more efficient at lower frequencies.
6) Message boards : Number crunching : The most efficient cruncher rig possible (Message 95276)
Posted 24 Apr 2020 by lakotamm
Post:
https://www.cnx-software.com/2020/04/23/50-odroid-c4-raspberry-pi-4-competitor-combines-amlogic-s905x3-soc-with-4gb-ram/amp

Could this be a candidate for beating the efficiency of ryzen?
7) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 95038)
Posted 21 Apr 2020 by lakotamm
Post:
Why does Ethernet go down when turning off USB? Can’t they be disabled independently?

Unfortunately, in this case, the ethernet is connected to the USB hub, so I shut down the USB hub together with the ethernet. I do not know how big the consumption is if you shut down only the ethernet module.
Ethernet module on 3B+ and 4 is 1Gb if I remember correctly, which is I guess it simply is not that well power-optimized.

This might sound like over doing it, but every watt counts in your rig efficiency and your temperatures.

Getting the carbon footprint (from both electricity consumption and hardware production) as low as possible should definitely be a priority.
8) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 95005)
Posted 20 Apr 2020 by lakotamm
Post:
I thought this would be a good place to ask. I want to set up a personal Raspberry Pi4 for myself and possibly my kids. Can anyone suggest a good setup to purchase that is reasonably priced but is well setup for running R@h?


I would suggest Raspberry pi 4 with at least 2GB of RAM. Preferably 4GB of you can afford it. I would not buy 3B, or 3B+ unless you can get it 2nd hand for a cheap price.

1GB RAM - usable, with limitations for Rosetta. It is still fun, but I do not expect from this version to be well supported for a very long time. It is very easy to use out 1GB RAM while doing any kind of work. Ubuntu server uses 25% RAM just by being idle.
2GB RAM - enough to do calculations atm, you might still want some extra zram/swap though. If you want to do some more serious work - connect it as a PC, browse the internet etc, you will probably hit the limits.
4GB RAM - more than enough for the current workloads. You might not be able to use the full potential just yet, but I would expect that in the future it might prolong the useful life of the RPI.
9) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 95002)
Posted 20 Apr 2020 by lakotamm
Post:
Per this article, the pi’s wireless uses 10x the power of Ethernet, just being idle (unfortunately, it doesn’t provide WiFi load power usage).


This is not true for 3B+. Disabling Ethernet together with USB hub provides the biggest consumption reduction possible - almost 200mA. Wifi consumes around 55mA. For me, disabling Ethernet means a significant reduction in temperature.
https://www.raspberrypi.org/forums/viewtopic.php?p=1286716
10) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94944)
Posted 19 Apr 2020 by lakotamm
Post:
I think that I finalized a guide for getting Rosetta to run on Raspberry Pi 3B+ (and also 2B v1.2, 3B, 4 - 1GB). It is available here:
https://www.reddit.com/r/BOINC/comments/g0r0wa/running_rosetta_covid19_workunits_on_raspberry_pi/

Unfortunately, I am not able to update the post here in this thread.

The speed of calculations seems like a wrong estimate, so do not base anything on it. I am sorry for misleading.
While test running the WUs, I noticed that the average speed of calculations with kernel 4.19. and the Deflate algorithm is around 28000sec. Ubuntu server with kernel 5.3 with LZO-RLE seemed to need around 26500 sec to complete a work unit, which would indicate an improvement around 5,5 percent, However, later I found that the Kernel 4.19 with LZO needs around 27000sec, which is very close. It is impossible to objectively compare the algorithms with only a few results.

I also discovered that my RPI has a significantly higher benchmark on Ubuntu server with kernel 5.3 than on Raspbian with kernel 4.19. Maybe this could explain the difference?
Raspbian:
Measured floating point speed 1333.92 million ops/sec
Measured integer speed 29853.32 million ops/sec
Ubuntu:
Measured floating point speed 1631.85 million ops/sec
Measured integer speed 36629.84 million ops/sec

What I managed to measure fairly accurately is the compression ratio of Rosetta tasks. Zramctl showed that even with overhead, LZO was having a compression ratio around 2.55-2.65, which is pretty nice. Here is LZO-RLE which I am running on Ubuntu right now:
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram3 lzo-rle 598.8M 188.4M 69.8M 73.2M 4 [SWAP]
/dev/zram2 lzo-rle 598.8M 187.8M 69.4M 72.9M 4 [SWAP]
/dev/zram1 lzo-rle 598.8M 187.8M 70.2M 73.6M 4 [SWAP]
/dev/zram0 lzo-rle 598.8M 189.1M 70.5M 73.9M 4 [SWAP]

I can definitely tell that the compression of LZO is worse than deflate. With deflate, 1GB ZRAM was enough for 4 tasks. With LZO and the same amount of ZRAM, I stalled the system and got it to a point when it was not responding, meanwhile it was stuck writing to SD card. Unfortunately, I did not test the compression ratio of deflate before switching to LZO.

I would like to try other compression algorithms like LZ4 later. But for now, I feel like I am at a point where I am satisfied with the performance of both Raspbian and Ubuntu. Now it will be important to get as many as possible onboard and get their RPIs to crunch numbers effectively. Let's just hope that the new changes to Rosetta won't mess up too much with RPis.

Thanks for all the help and advice! Even though I did not implement all the suggestions, it was great to get feedback and guidance on what to concentrate on.
11) Message boards : Number crunching : Large proteins (Message 94850)
Posted 19 Apr 2020 by lakotamm
Post:
It would be great if there were still units for ARM64 devices with 0,95GB RAM. It seems like the community has found ways to run multiple WUs on these devices in combination with RAM compression. The word is still just getting spread though and I expect the computing power coming from these devices to only increase.

That being said, my laptop with 24GB RAM is ready to take new 4GB WUs with ease.
12) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94832)
Posted 19 Apr 2020 by lakotamm
Post:
I spent last few evenings working on this. I think that I managed to learn quite a lot about zram, compression, BOINC memory management, and I started learning bash to be able to read your script.

Here is the result:
I am able to run 4 WUs at the same time on my 3B+ 24/7 automatically starting, without having any swap use, using zram.

Solutions which I applied:
- Instead of using custom script I switched to a tool called zram-tools which is in the repository. This is using standard LZO algorithm which is significantly faster than Deflate (used by the script). I found that Rosetta's tasks can be compressed with a ratio of around 2,6 using LZO, so I set my zram partition size to 250% of my ram size (2,4GB). This seems to increase the speed of calculations by around 3-5%. I never reach the limit in size ( it very rarely stores more than 600MB data - ca 240MB after compression), but just to be sure, I set up a 1GB swap partition.

- I increased the allowed memory use of BOINC to 300%. This allows 4 WUs to start even if the RAM is not empty to start with. The kernel simply moves data from the ram to the zram partitions. I have not experimented with the value itself.

- I increased the buffer of WUs to around 1 day. I will have to experiment with this a bit more, but it seems like this could be enough.

I tested Ubuntu server and I found out that it uses almost 200MB ram idle. Raspbian lite uses only around 55MB. Right now I am testing these settings on Raspbian, however, I would definitely like to try them also on Ubuntu and see whether there is any performance difference.
13) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94568)
Posted 15 Apr 2020 by lakotamm
Post:
Could you perhaps try a development version of BOINC to see if it has been fixed already? I see that the website binaries for PC have already been updated, but ARM seems to be lagging behind (maybe compile from source?). I could think of a quick workaround script that detects when a job is finishing and temporarily suspends all others while it is saving its files, but that would be a major hack. Or perhaps trying a faster SD card or faster file system might help (maybe not).


I am considering upgrading to Ubuntu. And I could also use a newer version of BOINC.

Are you positive that the task wouldn't start by itself even if you waited for a few minutes? Maybe the things are related: it is trying to start the next job while the previous is preparing its results? (Just guessing) Could you find anything in the logs (boinccmd --get_messages)? Boinc usually explains why it is or is not doing things.


I noticed that when 2 or more WUs are writing or reading from Flash (e.g. if I resume 2 tasks at once), the system becomes slow to respond. Unfortunately, I already restarted RPI after the error. Since then there is nothing special in the logs. sometimes it is quite difficult to get tasks - often they are not allowed to be sent. And few minutes later its not a problem.


131: 15-Apr-2020 20:06:36 (low) [Rosetta@home] Sending scheduler request: To fetch work.                                                                                                              
132: 15-Apr-2020 20:06:36 (low) [Rosetta@home] Requesting new tasks for CPU                                                                                                                           
133: 15-Apr-2020 20:06:38 (low) [Rosetta@home] Scheduler request completed: got 0 new tasks                                                                                                           
134: 15-Apr-2020 20:06:38 (low) [Rosetta@home] No tasks sent                                                                                                                                          
135: 15-Apr-2020 20:06:38 (user notification) [Rosetta@home] Message from server: Rosetta needs 1907.35 MB RAM but only 966.93 MB is available for use.                                               
136: 15-Apr-2020 20:11:44 (low) [Rosetta@home] project suspended by user                                                                                                                             
137: 15-Apr-2020 20:12:14 (low) [Rosetta@home] update requested by user                                                                                                                               
138: 15-Apr-2020 20:12:18 (low) [Rosetta@home] Sending scheduler request: Requested by user.                                                                                                          
139: 15-Apr-2020 20:12:18 (low) [Rosetta@home] Not requesting tasks: suspended via Manager                                                                                                            
140: 15-Apr-2020 20:12:20 (low) [Rosetta@home] Scheduler request completed                                                                                                                            
141: 15-Apr-2020 20:12:33 (low) [Rosetta@home] task rb_04_12_20719_20959_ab_t000__h001_robetta_IGNORE_THE_REST_07_07_911363_14_1 suspended by user                                                    
142: 15-Apr-2020 20:12:38 (low) [Rosetta@home] task rb_04_12_20719_20959_ab_t000__h001_robetta_IGNORE_THE_REST_07_07_911363_14_1 suspended by user                                                    
143: 15-Apr-2020 20:12:41 (low) [Rosetta@home] task rb_04_12_20719_20959_ab_t000__h001_robetta_IGNORE_THE_REST_04_05_911363_16_1 suspended by user                                                    
144: 15-Apr-2020 20:12:42 (low) [Rosetta@home] task rb_04_12_20719_20959_ab_t000__h001_robetta_IGNORE_THE_REST_08_05_911363_12_1 suspended by user                                                    
145: 15-Apr-2020 20:12:55 (low) [Rosetta@home] project resumed by user                                                                                                                                
146: 15-Apr-2020 20:13:04 (low) [Rosetta@home] update requested by user                                                                                                                               
147: 15-Apr-2020 20:13:07 (low) [Rosetta@home] Sending scheduler request: Requested by user.                                                                                                          
148: 15-Apr-2020 20:13:07 (low) [Rosetta@home] Not requesting tasks: some task is suspended via Manager                                                                                               
149: 15-Apr-2020 20:13:09 (low) [Rosetta@home] Scheduler request completed                                                                                                                            
150: 15-Apr-2020 20:14:03 (low) [Rosetta@home] task rb_04_12_20719_20959_ab_t000__h001_robetta_IGNORE_THE_REST_07_07_911363_14_1 resumed by user                                                      
151: 15-Apr-2020 20:15:04 (low) [Rosetta@home] task rb_04_12_20719_20959_ab_t000__h001_robetta_IGNORE_THE_REST_04_05_911363_16_1 resumed by user                                                      
152: 15-Apr-2020 20:15:29 (low) [Rosetta@home] task rb_04_12_20719_20959_ab_t000__h001_robetta_IGNORE_THE_REST_08_05_911363_12_1 resumed by user                                                      
153: 15-Apr-2020 20:16:31 (low) [Rosetta@home] Sending scheduler request: To fetch work.                                                                                                              
154: 15-Apr-2020 20:16:31 (low) [Rosetta@home] Requesting new tasks for CPU                                                                                                                           
155: 15-Apr-2020 20:16:34 (low) [Rosetta@home] Scheduler request completed: got 2 new tasks                                                                                                           
156: 15-Apr-2020 20:16:37 (low) [Rosetta@home] Started download of rb_04_15_21668_21257_ab_t000__robetta_FLAGS                                                                                        
157: 15-Apr-2020 20:16:37 (low) [Rosetta@home] Started download of rb_04_15_21668_21257_ab_t000__robetta.zip                                                                                          
158: 15-Apr-2020 20:16:39 (low) [Rosetta@home] Finished download of rb_04_15_21668_21257_ab_t000__robetta_FLAGS                                                                                       
159: 15-Apr-2020 20:16:39 (low) [Rosetta@home] Finished download of rb_04_15_21668_21257_ab_t000__robetta.zip                                                                                         
160: 15-Apr-2020 20:16:39 (low) [Rosetta@home] Started download of rb_04_15_21668_21257_ab_t000__robetta.200.3mers.index.gz                                                                           
161: 15-Apr-2020 20:16:39 (low) [Rosetta@home] Started download of rb_04_15_21668_21257_ab_t000__robetta.200.11mers.index.gz                                                                          
162: 15-Apr-2020 20:16:41 (low) [Rosetta@home] Finished download of rb_04_15_21668_21257_ab_t000__robetta.200.3mers.index.gz                                                                          
163: 15-Apr-2020 20:16:41 (low) [Rosetta@home] Finished download of rb_04_15_21668_21257_ab_t000__robetta.200.11mers.index.gz                                                                         
164: 15-Apr-2020 20:17:10 (low) [Rosetta@home] Starting task rb_04_15_21668_21257_ab_t000__robetta_IGNORE_THE_REST_03_11_916702_5_0                       


It is still a mystery to me when a new task is allowed to start. Sometimes it starts by itself, sometimes only 1 task is left to run. There is nothing about that in the logs.

Also, could you perhaps post some stats while the Pi is fully loaded and happily crunching with BOINC? I would be interested in memory usage and compression ratio: `ps -e v` (or the relevant boinc lines of `top`), `free`, `cat /proc/swaps` and `zramctl` (or the appropriate folder in sysfs)


Running 4 Rosetta tasks (note, this is only the beginning though):
top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1550 boinc 39 19 324092 164704 62372 R 94.1 16.6 7:54.03 rosetta_4.15_aa
1615 boinc 39 19 365552 238068 94284 R 94.1 24.0 5:52.67 rosetta_4.15_aa
1618 boinc 39 19 334516 210708 97852 R 94.1 21.3 5:24.04 rosetta_4.15_aa
1535 boinc 39 19 332424 123680 54592 R 88.2 12.5 7:25.72 rosetta_4.15_aa

free:
total used free shared buff/cache available
Mem: 990132 534196 154188 752 301748 389820
Swap: 1320160 57856 1262304

Filename Type Size Used Priority
/dev/zram0 partition 330040 14252 5
/dev/zram1 partition 330040 15048 5
/dev/zram2 partition 330040 14592 5
/dev/zram3 partition 330040 13196 5

NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram3 deflate 322.3M 12.7M 3.9M 4.9M 4 [SWAP]
/dev/zram2 deflate 322.3M 14.2M 4.5M 5.4M 4 [SWAP]
/dev/zram1 deflate 322.3M 14.5M 4.6M 5.5M 4 [SWAP]
/dev/zram0 deflate 322.3M 13.7M 4.3M 5M 4 [SWAP]

I have already seem more than 200MB data in each partition earlier.
14) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94524)
Posted 15 Apr 2020 by lakotamm
Post:
Seeing 8 boinc processes in your listing worries me. Maybe some of them got stuck? Anyway, could you perhaps share the results after completion (time, credits, decoy count, WS_MAX)?

It seems like it is just htop showing it in a weird way - top shows it correctly.
https://prnt.sc/rzn8ih

You can see the stats here:
https://boinc.bakerlab.org/rosetta/results.php?hostid=3710842
It seems like had I had 1 error so far. Let's see whether more appear.

The biggest issue seems to be right now the start-up of a WU. If I have 4 WUs running and one gets finished, another one won't start by itself. Instead, it stays waiting for memory. I need to manually suspend one of the running WUs - which allows the new WU to start. Afterwards I can again resume the suspended WU again.

I might make a script for this and possibly use an external drive for storing the tasks. Or use only 2-3 cores for Rosetta. However, this would mean loosing 25 percent CPU time unused. So far I have not found a way of selecting only 2-3 cores for a project an using the rest for another project.
15) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94478)
Posted 14 Apr 2020 by lakotamm
Post:
Not sure how compressing RAM/Swap would work out for you, however it's pretty easy to set up the swap partition on an external drive, so you don't have to worry about SD card wear as much.

I have disabled swap (set the size to 0). Everything seems to be running fine.

However, not all Boinc projects use a lot of RAM.
You could run 2 threads running Rosetta, and 2 running other project(s).

Thanks for the idea, I added Universe@Home and I might try to join also LHC@Home. There are definitely moments when Rosetta is not using all available RAM - just because it is so big. The question is - how to set it up so that it runs each project on 2 cores? I changed the resource share of Universe@home to 1. And I am testing it...

I recommend keeping at least 1 core free on the Pi. If you’ve got a 1GB Pi only run one at a time. If using zram you might be able to run two at a time. Why? zram needs some CPU time to do its job so keeping cores free means they are available without having to swap the task out.

I will try to test this over the next few days, and try to see how long the WU takes with no cores free and a single-core free.

When the tasks start they unzip over 1.5GB of data and copy files to the slot directory. That is a killer for the SD card. I believe the unzip part is multi-threaded so it will use all available cores but it still takes a few minutes to write to the SD card.

I noticed that if I suspend Rosetta and switch, it takes over 2 minutes to return from Universe to Rosetta. I guess that demonstrates the load...

For anyone interested, here are current measurements of RPI 3B+:
https://www.raspberrypi.org/forums/viewtopic.php?p=1286716

According to these results, 4 core load + WIFI results in ca. 390mA. I did not test this myself.
16) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94427)
Posted 14 Apr 2020 by lakotamm
Post:
This is my current state:
CLI only, 4MB VRAM
ZRAM algorithm from sangaku
4 WUs running again. Maybe it just needs time? Certain order?...

https://prnt.sc/ryz2ua
https://prnt.sc/ryz560

pi@raspberrypi:~ $ pi@raspberrypi:~ $ cat /proc/swaps
Filename Type Size Used Priority
/var/swap file 2024444 0 -2
/dev/zram0 partition 330040 21776 5
/dev/zram1 partition 330040 21796 5
/dev/zram2 partition 330040 21552 5
/dev/zram3 partition 330040 21660 5

Edit: 30 minutes later, all 4 tasks still running, but now the usage of ZRAM partitions has significantly increased to 200 000 each. Swap use is still 0
17) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94425)
Posted 14 Apr 2020 by lakotamm
Post:
Thank you guys for advices. In the morning, after waking up, only a single task was running. During night, RPI downloaded Rosetta for Portable devices. Maybe it is more RAM demanding? I managed to initialize one more task by stopping and running again the already running task, but not more.

Thanks for sharing! I like your setup. Are you connecting over wifi? Could you perhaps measure the power draw? You may use either a kill-a-watt or the built-in power gauge inside your laptop if you connect to a high power charger port.
I'm interested in how low can we get in power? Can you underclock the CPU/GPU? Can you switch off the LEDs? Can we switch wifi on periodically only for transfers (from cron for example)?

I will try to measure consumption in the evening. I have not tried any fancy turning on-off wifi yet.

3 Rosetta tasks in 1GB sounds like pushing the limit a bit, but we'll see how it performs. You may get some more space if you added a little bit more zram and if you select a better algorithm. Basically I would adjust the end of the script like this:
mem=$(( ($totalmem * 4 / 3 / $cores)* 1024 ))
modprobe deflate
modprobe zlib
modprobe lz4hc_compress
core=0
while [ $core -lt $cores ]; do
  echo deflate > /sys/block/zram$core/comp_algorithm ||
   echo zlib > /sys/block/zram$core/comp_algorithm ||
   echo lz4hc > /sys/block/zram$core/comp_algorithm ||
   echo lz4 > /sys/block/zram$core/comp_algorithm
# not sure which one this kernel has
  echo $mem > /sys/block/zram$core/disksize
  mkswap /dev/zram$core
  swapon --discard -p 5 /dev/zram$core # reclaim memory better
  let core=core+1
done

I used your code, but it seems like the use of zram partitions has even lowered. I am not sure why. I do not quite understand how to properly configure zram.

You may set the GPU memory as low as you like, because you can run a vncserver in a virtual framebuffer without a real X11. You may need to double check ~/.vnc/xstartup to execute an lxsession or at least a terminal. But why do you need X, if you can use boinccmd, boinctui or even remote RPC?

Thanks, I did not know about boinctui. I have not tried using remote RCP yet, but I definitely will have to learn how to do so. It seems like vncserver does not start with anything less than 32MB.
Even though I am fine using CLI in most cases, it still feels more comfortable for me to use GUI.
18) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94397)
Posted 13 Apr 2020 by lakotamm
Post:
That is interesting, am I overengineering this? I could try to disable ZRAM to see how it works.

What is so far your average time per WU? Are all your cores running?

My times so far:
7:48
7:53
19) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 94393)
Posted 13 Apr 2020 by lakotamm
Post:
Updates:
32MB RAM seems to be enough for the GPU to run VNC at 720p.


So far I have seen some issues with WUs initializing and disabling repeatedly - I guess that the RAM space is simply quite on the limit. Let's see how it works in the long run.
20) Message boards : Number crunching : Raspberry Pi compatibility (Message 94380)
Posted 13 Apr 2020 by lakotamm
Post:
I created a guide for RPI 3B+ - I seem to be able to run 3, sometimes 4 WUs. Not bad.
https://boinc.bakerlab.org/rosetta/forum_thread.php?id=13795&postid=94371#94371


Next 20



©2024 University of Washington
https://www.bakerlab.org