Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to?
Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next
Author | Message |
---|---|
sgaboinc Send message Joined: 2 Apr 14 Posts: 282 Credit: 208,966 RAC: 0 |
there is something else i kind of had a brain wave, if pi4 is after all running stable, the checkpoint interval can probably be extended out like 5 minutes that probably have an effect on the overall power consumption, and does less writes to the sd card as well |
Endgame124 Send message Joined: 19 Mar 20 Posts: 63 Credit: 20,367,622 RAC: 2,215 |
there is something else i kind of had a brain wave, if pi4 is after all running stable, the checkpoint interval can probably be extended out like 5 minutes This is a interesting point. Perhaps the power fluctuations I see are checkpoints writing to flash, not the ups trying to pick a specific wattage to display. If I get some time tonight, I’ll try interesting and decreasing the checkpoint time |
bkil Send message Joined: 11 Jan 20 Posts: 97 Credit: 4,433,288 RAC: 0 |
I expect it to only have minor effects. A checkpoint file takes about 20kB over here, although it may need a little computation to produce. At the same time, if you increased the target cpu time, it would need to copy & decompress the gigantic zip file less often. But again, the amortized cost of this should be low. I wouldn't be surprised if you saved more energy with mount options like `noatime,lazytime,commit=60,max_batch_time=50000` or tweaking the block device, and/or with lzo-compressing btrfs. Do share your benchmark results with us if you find anything. |
sgaboinc Send message Joined: 2 Apr 14 Posts: 282 Credit: 208,966 RAC: 0 |
thanks good hint, i think i'd add that no atime in the mounts (in fstab) it would significantly reduce writes to sd card oh ok it seemed noatime is default in Raspbian distribution from www.raspberrypi.org i'm also getting 4.20 WU, in 4.20 one of the large database files is now made shared in the project directory, that may have an improvement on its own. it is more likely that disk reads will simply become cache hits in memory |
mobluse Send message Joined: 5 Apr 20 Posts: 6 Credit: 72,232 RAC: 8 |
There is this system to run 64-bit Debian Buster inside Raspbian Buster 32-bit: https://github.com/sakaki-/raspbian-nspawn-64 $ sudo apt update $ sudo apt upgrade $ sudo apt install -y raspbian-nspawn-64 $ sudo reboot ... ... $ ds64-shell pi@debian-buster-64:~ $ sudo apt update pi@debian-buster-64:~ $ sudo apt upgrade pi@debian-buster-64:~ $ sudo apt install boinc boinc-client boinc-manager boinctui pi@debian-buster-64:~ $ sudo service boinc-client start pi@debian-buster-64:~ $ boinctui In boinctui you can add projects. You can also start GUI boincmgr from a Terminal 32-bit using: $ ds64-runner boincmgr I couldn't start boincmgr from the start menu. I got Error: Failed to change to directory "/var/lib/boinc-client" (No such file or directory) This is an improvement of: https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=271456&hilit=covid&start=25#p1649357 This works to get tasks from Rosetta@home on Pi 4 B 4 GB, but for Pi 3 one might need to do tricks to get more memory: https://www.reddit.com/r/BOINC/comments/g0r0wa/running_rosetta_covid19_workunits_on_raspberry_pi/ I've not tried BOINC on Raspberry Pi 3. |
bkil Send message Joined: 11 Jan 20 Posts: 97 Credit: 4,433,288 RAC: 0 |
Thanks for the nspawn trick. We've developed the Raspberry Pi 3B+ trick right here on this forum: |
sgaboinc Send message Joined: 2 Apr 14 Posts: 282 Credit: 208,966 RAC: 0 |
reposting here just in case it helps for Pi4 and the recent Raspbian Buster images, nspawn isn't needed really to run rosatta@home, i ran 'native' it is probably more efficient and saves sd space i got my clues from here http://marksrpicluster.blogspot.com/2020/04/do-something-useful-with-your-pi4.html basically
sudo reboot and normally things should be started. check with systemctl status boinc-client.service of course install either boinc-manager graphical gui apt-get install boinc-manager or boinctui apt-get install boinctui to manage the runs. one can also use boinccmd from the command line which doesn't need additional manager gui ideally take additional steps to get sshd up and running. i did in addition to get avahi (zeroconf, MDNS) up and running so that i can 'find' it on the lan when it is booted up one may want to add in cc_config.xml <options> <allow_remote_gui_rpc>1</allow_remote_gui_rpc> </options> set the password in /var/lib/boinc/gui_rpc_auth.cfg install rpi-monitor https://rpi-experiences.blogspot.com/ apt-get install rpimonitor and one can run pretty much run 'headless' connect remotely to manage the boinc-client instance and monitor cpu loads and temperature on the web at port 8888 (rpi-monitor). running headless is really conveinent, there is only one cable usb C for power that's all. you can place the pi in a good location with wifi access and well ventilated (it needs a good heat sink or sink fan as well) to dissipate the heat |
PorkyPies Send message Joined: 6 Apr 20 Posts: 45 Credit: 1,650,779 RAC: 0 |
I don’t know about the rest of you but I control my Pi’s using BOINCtasks where I can see all of the cluster at once. BOINCtasks (BT) is a Windows app, but will run under wine. One of the things BT can show, which BOINC Manager doesn’t easily show, is how much memory each task is using. I have some instructions I posted on my blog in 2013, but it might be worthwhile updating them, although not much has changed. MarksRpiCluster |
Endgame124 Send message Joined: 19 Mar 20 Posts: 63 Credit: 20,367,622 RAC: 2,215 |
reposting here just in case it helps This is basically my setup as well. However, I also reduce the video memory and set a 2GB Zram swap space and set Boinc to use the extra memory as even the 4GB Pi occasionally runs short of memory with 4 Rosetta processes: #Reduce memory assigned to GPU: sudo raspi-config - Advanced Options - Memory split - 16MB Setup zram: sudo apt-get install zram-tools sudo vi /etc/default/zramswap PERCENTAGE=50 (275 for Pi 3B+) sudo zramswap stop sudo zramswap start #increase swap as a last resort so we don't run out of memory sudo dphys-swapfile swapoff sudo vi /etc/dphys-swapfile CONF_SWAPSIZE=1024 sudo dphys-swapfile swapon #Edit the Boinc global preferences to use more than 100% of system memory: sudo vi /etc/boinc-client/global_prefs_override.xml update xml to: <global_preferences> <max_ncpus_pct>100.000000</max_ncpus_pct> <vm_max_used_pct>90.000000</vm_max_used_pct> <ram_max_used_busy_pct>175.000000</ram_max_used_busy_pct> <ram_max_used_idle_pct>175.000000</ram_max_used_idle_pct> <cpu_usage_limit>100.000000</cpu_usage_limit> </global_preferences> boinccmd --read_global_prefs_override
I haven't been been using rpi monitor. How much CPU does it have? I'm currently doing the following to save power on my pi: Disable HDMI: sudo tvservice --off Disable Wifi echo "dtoverlay=disable-wifi" | sudo tee -a /boot/config.txt Disable Bluetooth and bluetooth services sudo systemctl disable hciuart echo "dtoverlay=disable-bt" | sudo tee -a /boot/config.txt Turn off Power LED: echo 0 | sudo tee /sys/class/leds/led1/brightness Have you done any power testing with Ethernet vs Wifi? Also, which Heatsink are you using? I've been running passive with the Flirc Aluminum case - it works quite well, but I've found that its VERY easy to give the case a shock and lock up the Pi. |
Endgame124 Send message Joined: 19 Mar 20 Posts: 63 Credit: 20,367,622 RAC: 2,215 |
I don’t know about the rest of you but I control my Pi’s using BOINCtasks where I can see all of the cluster at once. BOINCtasks (BT) is a Windows app, but will run under wine. One of the things BT can show, which BOINC Manager doesn’t easily show, is how much memory each task is using. I've been managing all of my computers individually. I would be very interested to read about some management tools to handle everything from one place. |
hnapel Send message Joined: 8 Apr 20 Posts: 8 Credit: 835,346 RAC: 0 |
This is a general problem with the PI4 if you put any load on it, you need to install active cooling. B.t.w. the device will not get killed so easily, when the temperature gets over 80 degrees Celsius, a throttling mechanism comes into play, but running the thing at 80 degrees all the time may adversely affect its longevity, so try to shave off some degrees with a cooler or there's also massive cooling blocks that cover the whole little board. |
sgaboinc Send message Joined: 2 Apr 14 Posts: 282 Credit: 208,966 RAC: 0 |
reposting here just in case it helps thanks for the hint just did that, every byte matters Setup zram: i think i'd try zram at a different attempt, i've that nagging feeling that zram would after all 'steal cycles' from the main threads running the wus. i did have a pretty large swap that is *unused* (very little of it tends to be used), i actually set boinc memory use to 100% both in use and busy. this over commits some ram into swap. i'm wary of exceeding 100% as the spec as it may imply the actual wu memory blocks being swapped out. the notion about swap is *unused memory*, i'd imagine that there are various tmpfs mounts and various daemons. if those blocks aren't in use, i'm happy for them to sit in swap. and i found running 3 threads in 4GB memory leaves some needed margins, large wu runs pretty much at full speeds at 1.75ghz and i tend to get better credits per task returns for similar wu vs 4 tasks concurrently. i reasoned that to be related to cache contention, for 4 threads there may be more cache misses resulting in the threads hitting memory and hence doing wait states for dram access, that slows things down overall. and in addition, i observed, temperature spikes occasionally and cpu loads run high more so than do 3 threads at once. i'd think it is partly happening when transiting between task uploads and setting up and running new tasks.
Rpi-monitor https://github.com/XavierBerger/RPi-Monitor basically provides a nice web interface to monitor the system statistics, core temperature, cpu load, memory used and it draws nice temperature graphs. the temperature graphs has helped me decide on the preferred operating points, e.g. i'd prefer to run at a lower temperature and to get a max credits per task per run period. temperature together with memory and credit considerations is partly how i decided to run just 3 threads instead of 4. i tend to see more credit per task per run period (and more models completed too) running 3 tasks than 4 and there are less temperature spikes running 3. bigger (memory) tasks tend to simply run quietly without adverse incidences e.g. waiting for memory etc
oh wow, i did trying tvservice -off, it didn't seem to make much difference (on the temperature graph). Hence, i left that alone. i'm running on a single cable i.e. power (from usb C), it simply works over wifi, no ethernet. i've not made power measurements but i'd guess this setup is probably ok for the lower power draw. i read some forum posts that ethernet can draw quite some currents (like 100ma) if the cable is connected and live. Also, which Heatsink are you using? I've been running passive with the Flirc Aluminum case - it works quite well, but I've found that its VERY easy to give the case a shock and lock up the Pi. well here is my setup: https://boinc.bakerlab.org/rosetta/forum_thread.php?id=13732&postid=95585 that temperature graph there is from rpi-monitor |
Boulybud Send message Joined: 3 Apr 20 Posts: 11 Credit: 351,250 RAC: 0 |
I have a question for you, according to you, with a Raspberry Pi 4 4GB that works with 3 tasks, what would be the best settings for spot storage? Raspberry Pi no OC which runs 24/7 Here are the settings I put : Other Store at least 0.50 days of work Store up to an additional 0.10 days of work |
sgaboinc Send message Joined: 2 Apr 14 Posts: 282 Credit: 208,966 RAC: 0 |
i did 0.1 / 0 i.e. don't cache tasks, not so much for storage but so that it won't download a large cache of wu. This helps in case i'd want to shutdown, so that i won't need to cancel lots of tasks. in fact even with near zero cache, i tend to see at least 1 task cached ahead, that's ok at least it isn't a large number of them. |
Endgame124 Send message Joined: 19 Mar 20 Posts: 63 Credit: 20,367,622 RAC: 2,215 |
A few more pieces of data. I used uhubctl to power off the usb on my pi 4 and wired networking continued to work. The Ethernet is not connected to usb in the pi 4 - see this thread: https://www.raspberrypi.org/forums/viewtopic.php?t=244673 Continuing network power testing, I created a 10 GB file and copied it across the network to my freenas. Wired: copy took 10 minutes 17 seconds. The network copy took <1watt (my ups did not read a power difference while running the copy. It also does not read different power values with the cable unplugged) Wireless: when Placed appx 19 inches away from my router, the copy took 17 minutes 53 seconds via my TP Link Archer ac router. Power used was <1watt (my ups did not read different power usage during the copy) When placed in my office (across my house on a different floor... maybe 50 feet through a wall in addition to the floor), copy used 2w power (measured from a different ups, power measurement is not exact and not completely comparable). Copy took >20 minutes... I forgot to use “time” with the copy command. I can rerun if anyone cares about the time taken for copy. |
Endgame124 Send message Joined: 19 Mar 20 Posts: 63 Credit: 20,367,622 RAC: 2,215 |
I now have my newest Pi 4 4GB online. This one seems to be stable at the maximum overclock with these settings: arm_frequency=2147 over_volt=6 With USB, HDMI, Bluetooth, Wifi, and Power LED turned off, the system is consuming 6W as read from my UPS. This Pi 4 is in a Flirc case, and is the top of a stack of 3 Pi 4s. A single fan is blowing on the stack. Maximum temp seen so far is 54 C Host details: https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4453627 |
sgaboinc Send message Joined: 2 Apr 14 Posts: 282 Credit: 208,966 RAC: 0 |
hi, Pi4 has got a new 8GB version, and more importantly 64 bit OS ;) https://www.raspberrypi.org/blog/8gb-raspberry-pi-4-on-sale-now-at-75/ https://www.raspberrypi.org/forums/viewtopic.php?f=117&t=275370 https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=275372 it seem to be something to try out, it is Beta though, this is really on the bleeding edge lol |
Endgame124 Send message Joined: 19 Mar 20 Posts: 63 Credit: 20,367,622 RAC: 2,215 |
hi, Pi4 has got a new 8GB version, and more importantly 64 bit OS ;) I'll be ordering one today! I don't expect dramatically different performance from the 4GB model, but every once in a while I need more than 4GB of ram for Rosetta. Does anyone know if there is planned long term support for ARM in Rosetta? I would hate to buy 3 8GB Pis and have support dropped in a few months. |
sgaboinc Send message Joined: 2 Apr 14 Posts: 282 Credit: 208,966 RAC: 0 |
it seemed to be an instant sold out on launch, Pi4s are getting too hot lol https://shop.pimoroni.com/products/raspberry-pi-4?variant=29157087412307 |
PorkyPies Send message Joined: 6 Apr 20 Posts: 45 Credit: 1,650,779 RAC: 0 |
I'm currently doing the following to save power on my pi: I usually blacklist the WiFi and Bluetooth. Not to save power. I have 20 Pi's and I would rather they not use WiFi at all, there wouldn't be much bandwidth left for the other devices. Most of them are Pi3's so are running Einstein work. cd /etc/modprobe.d sudo nano raspi-blacklist.conf and add the following # wifi blacklist brcmfmac blacklist brcmutil # bluetooth blacklist btbcm Not sure if they are still correct, I got this from forum posts on raspberrypi.org some years ago. I also remove the pi-bluetooth package. MarksRpiCluster |
Message boards :
Number crunching :
Running on a 4GB Raspberry Pi 4 - How to?
©2024 University of Washington
https://www.bakerlab.org