Posts by Endgame124

21) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 96702)
Posted 21 May 2020 by Endgame124
Post:
A few more pieces of data.

I used uhubctl to power off the usb on my pi 4 and wired networking continued to work. The Ethernet is not connected to usb in the pi 4 - see this thread:
https://www.raspberrypi.org/forums/viewtopic.php?t=244673

Continuing network power testing, I created a 10 GB file and copied it across the network to my freenas.

Wired: copy took 10 minutes 17 seconds. The network copy took <1watt (my ups did not read a power difference while running the copy. It also does not read different power values with the cable unplugged)

Wireless: when Placed appx 19 inches away from my router, the copy took 17 minutes 53 seconds via my TP Link Archer ac router. Power used was <1watt (my ups did not read different power usage during the copy)

When placed in my office (across my house on a different floor... maybe 50 feet through a wall in addition to the floor), copy used 2w power (measured from a different ups, power measurement is not exact and not completely comparable). Copy took >20 minutes... I forgot to use “time” with the copy command. I can rerun if anyone cares about the time taken for copy.
22) Message boards : Number crunching : The most efficient cruncher rig possible (Message 96609)
Posted 18 May 2020 by Endgame124
Post:
In regards to my pi3, it is no longer consistently receiving Rosetta work units. I’m wondering if boinc / Rosetta is getting wise to adjusting the amount of memory in the config file. At this point, I’m likely to pull the pi 3 off Rosetta and assign it fully to other projects.
23) Message boards : Number crunching : The most efficient cruncher rig possible (Message 96608)
Posted 18 May 2020 by Endgame124
Post:
On my overclocked pi 4, I had misconfigured swap sizes, and it was occasionally failing when it picked up more than 5gb of Rosetta processes. I also Bumped the over voltage to 5 just to be 100% certain I wasn’t hitting voltage problems.

After correcting the issue, it has run perfectly for around 3 days. It will take some time for the RAC to stabilize though. It’s still going up every time I check it.
24) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 96310)
Posted 9 May 2020 by Endgame124
Post:
My Pi3B+ seems to have topped out at about 600 credit with 3 Rosetta processes. I’ve dropped down to 2 processes, and switched zram to lz4 to see if it affects the average credit or not.


Dropping to 2 processes did not improve performance (dropped to 530 average credit and still seems to be trending downward) so I think we can say with confidence that in pretty much all cases, as long as you aren’t thrashing swap to a sdcard, more processes is better.

I may try 3 processes again with lz4, but perhaps it would be better to switch to ZSTD? Better compression with similar overhead to lzo.
25) Message boards : Number crunching : Running Rosetta on Raspberry Pi 3B+ (how to guide) (Message 96238)
Posted 7 May 2020 by Endgame124
Post:
My Pi3B+ seems to have topped out at about 600 credit with 3 Rosetta processes. I’ve dropped down to 2 processes, and switched zram to lz4 to see if it affects the average credit or not.
26) Message boards : Number crunching : How I overclocked my raspberry pi 4 (Message 96025)
Posted 4 May 2020 by Endgame124
Post:
I've been curious, how are the pi's doing now that the database is only stored once for the project rather than once in each active WU slot? Does the memory footprint of the active WUs get reduced and allow you to activate another core?

You can easily get 4 Rosetta processes running on a Pi 4 4GB with a 2GB zram config (2GB seems to be overkill, but it also doesn't seem to harm the system either).

For a Pi 3B+, 3 Rosetta processes really pushes the host with using swap. I think it would actually slow down the processing to go with 4 processes, and likely 2 processes would be a better fit.

I have 1 stock Pi 4 4GB for a baseline comparison, one Pi 4 4GB overclocked, and 1 Pi 3B using a large ZRAM configuration (250% of system ram). All are using Flirc cases and stay well under 60C.

If you want to track my Pis, here are the direct links to the systems:

Stock Pi 4 4GB:
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4215281

Pi 4 4GB @ over_volt=4 and arm_freq=2015 mhz:
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4269102

Stock Pi 3B+ with Zram and 3 Rosetta Processes
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4244063
27) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 96013)
Posted 4 May 2020 by Endgame124
Post:
I don’t know about the rest of you but I control my Pi’s using BOINCtasks where I can see all of the cluster at once. BOINCtasks (BT) is a Windows app, but will run under wine. One of the things BT can show, which BOINC Manager doesn’t easily show, is how much memory each task is using.

I have some instructions I posted on my blog in 2013, but it might be worthwhile updating them, although not much has changed.

I've been managing all of my computers individually. I would be very interested to read about some management tools to handle everything from one place.
28) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 96012)
Posted 4 May 2020 by Endgame124
Post:
reposting here just in case it helps
basically

[list]
* install boinc
sudo apt-get install boinc-client


* set 64 bits kernel
edit /boot/config.txt at end add
arm_64bit=1


*snip*

This is basically my setup as well. However, I also reduce the video memory and set a 2GB Zram swap space and set Boinc to use the extra memory as even the 4GB Pi occasionally runs short of memory with 4 Rosetta processes:

#Reduce memory assigned to GPU:
sudo raspi-config - Advanced Options - Memory split - 16MB

Setup zram:
sudo apt-get install zram-tools
sudo vi /etc/default/zramswap
PERCENTAGE=50 (275 for Pi 3B+)
sudo zramswap stop
sudo zramswap start

#increase swap as a last resort so we don't run out of memory
sudo dphys-swapfile swapoff
sudo vi /etc/dphys-swapfile
CONF_SWAPSIZE=1024
sudo dphys-swapfile swapon

#Edit the Boinc global preferences to use more than 100% of system memory:
sudo vi /etc/boinc-client/global_prefs_override.xml

update xml to:
<global_preferences>
<max_ncpus_pct>100.000000</max_ncpus_pct>
<vm_max_used_pct>90.000000</vm_max_used_pct>
<ram_max_used_busy_pct>175.000000</ram_max_used_busy_pct>
<ram_max_used_idle_pct>175.000000</ram_max_used_idle_pct>
<cpu_usage_limit>100.000000</cpu_usage_limit>
</global_preferences>

boinccmd --read_global_prefs_override



https://rpi-experiences.blogspot.com/
apt-get install rpimonitor

and one can run pretty much run 'headless' connect remotely to manage the boinc-client instance and monitor cpu loads and temperature on the web at port 8888 (rpi-monitor). running headless is really conveinent, there is only one cable usb C for power that's all. you can place the pi in a good location with wifi access and well ventilated (it needs a good heat sink or sink fan as well) to dissipate the heat

I haven't been been using rpi monitor. How much CPU does it have?

I'm currently doing the following to save power on my pi:

Disable HDMI:
sudo tvservice --off

Disable Wifi
echo "dtoverlay=disable-wifi" | sudo tee -a /boot/config.txt

Disable Bluetooth and bluetooth services
sudo systemctl disable hciuart
echo "dtoverlay=disable-bt" | sudo tee -a /boot/config.txt

Turn off Power LED:
echo 0 | sudo tee /sys/class/leds/led1/brightness

Have you done any power testing with Ethernet vs Wifi?

Also, which Heatsink are you using? I've been running passive with the Flirc Aluminum case - it works quite well, but I've found that its VERY easy to give the case a shock and lock up the Pi.
29) Message boards : Number crunching : The most efficient cruncher rig possible (Message 95698)
Posted 1 May 2020 by Endgame124
Post:
Yeah, I set it to 3 threads from the start. Happy to bump it up to 4 to see the impact though.

If you keep yours at 3, and I leave my stock one at 4, we should get some good comparison points!
30) Message boards : Number crunching : The most efficient cruncher rig possible (Message 95676)
Posted 1 May 2020 by Endgame124
Post:
I've got a 4Gb Pi4 running 3 threads on Raspian to add to the data pool too. Everything stock, and passive heatsink/case:

https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4021327

Yours has been running longer than mine. Has it had 3 Rosetta processes since you joined it to the project? If so, we may need to re-evaluate the points a 4 process pi 4 can do.
31) Message boards : Number crunching : The most efficient cruncher rig possible (Message 95665)
Posted 1 May 2020 by Endgame124
Post:
If you want to track my Pis, here are the direct links to the systems:

Stock Pi 4 4GB:
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4215281

Pi 4 4GB @2015 mhz:
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4269102

Stock Pi 3B+ with Zram and 3 Rosetta Processes
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=4244063
32) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 95629)
Posted 30 Apr 2020 by Endgame124
Post:
there is something else i kind of had a brain wave, if pi4 is after all running stable, the checkpoint interval can probably be extended out like 5 minutes
that probably have an effect on the overall power consumption, and does less writes to the sd card as well

This is a interesting point. Perhaps the power fluctuations I see are checkpoints writing to flash, not the ups trying to pick a specific wattage to display. If I get some time tonight, I’ll try interesting and decreasing the checkpoint time
33) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 95620)
Posted 30 Apr 2020 by Endgame124
Post:
I still wouldn't declare a winner right now. There are lots of power consumption tweaks that you could apply to your Raspberry Pi 4 that nobody had bothered to measure up until now.

Also you are comparing the system AC power consumption of an rPi with a theoretical TDP (thermal design power) of a CPU itself (that is itself usually less than the maximal power used). If you add the motherboard, memory, fans, storage and then multiply this all with 80-85% AC conversion efficiency of a power supply, the difference wouldn't be that great as you think.

Also, you may optimize your budget if you ditched the SD card and booted over PXE and ran multiple Pi's from a single power supply (an ATX one one should supply a few dozen maybe).

And while we are mostly mentioning Raspberry Pi 4, there exists lots of other modern single board computers (SBC) on the market with 1-4GB RAM that can run 64-bit Linux, some cost less than a Pi, you may want to look these up. You should also not forget that you may be eligible for a discount when purchasing 100 of anything. I myself got into contact with an rPi employee and got a discount of about 40% for a volume purchase.

When you were comparing the overclocked Pi's you've omitted some important factors:
    - Overclocked systems need more expensive cooling
    - Overclocked systems carry a higher maintenance cost (lockups and potentially file system corruption every once in a while as you already see here, memory bit errors, etc)
    - Overclocked systems _may_ see a higher annual failure rate, increasing costs some more



Despite all of the above advantages, I still agree that managing a hundreds hosts requires great skills and is not for everyone, so purchasing a big box is definitely the easier route for most.



Quick post from a phone, but why would you use a 80% efficient power supply with a $4000 processor? You can pickup a 96% efficient, 600 watt, fanless seasonic titanium power supply for less than $200. With a 12 year warranty, you could likely use it over 4 system builds as well.
34) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 95619)
Posted 30 Apr 2020 by Endgame124
Post:
as for myself, Pi4 vs desktop is more a matter of different 'use cases', as the Pi4 runs pretty much like a server, no monitor and in fact only the USB C that is supplying power.
i can let it run without shutting down frequently. the power efficiency is a delightful finding.
my current desktops can produce more points in less hours, in fact in 4 hours, my desktop haswell does as much points as a non-overclocked Pi4 running for 24 hours.
so that would look like 6 times more efficient in terms of points per hour.
but in terms of power efficiency assuming 4 watts at 1.75 ghz on Pi4, Pi4 produces more points (1785) in 24 hours for about 100 watts hours
while my desktop haswell produces 1400 points running 8 concurrent threads in 4 hours, but it consume 100 watts x 4 hours ~ 400 watts hours
so by the points per watt benchmark Pi4 wins out (probably significantly) based on points per watt if these are indeed the numbers

You could also park a threadripper 3990x in a corner, headless, and let it run like a server as well.

Now, if there goal is just to see the max points per watt that can be obtained on a pi 4, down volting it and reducing MHz is the way to go - maybe you could hit threadripper efficiency with low enough voltage? You would need a lot of pis to get an appreciable amount of credit though.
35) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 95616)
Posted 30 Apr 2020 by Endgame124
Post:

When you bump up the core count, the power increase should be less than linear, because there are various shared components within the CPU. On the other hand, when you are overclocking a CPU, consumption should increase at least linearly with clock and additionally with the square of voltage!

So it should be rarely worth it from a performance/watt perspective, but do share your math if you have _exact_ numbers. We must be extra cautious with our measurements, because dividing such small numbers can introduce big errors, especially if you don't do enough sampling and averaging.


We need to define "worth it" from a performance / watt perspective, because buying dozens of Pi 4s is fairly costly, but it also requires you to manage dozens of systems. The Competition, so to speak, is pretty much a threadripper 3990X (https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=3832214), which is generating about 119,000 average credit. At 280W of power (https://www.anandtech.com/show/15483/amd-threadripper-3990x-review/2) , that is 425 Credit / watt without adding in extra system overhead.

If a "stock" pi 4 4gb gets about 850 Average Credit based on results earlier in the thread, and uses about 3.5 Watts, that is only 243 Credit / watt - not only would we need 140 Pi 4s to match the threadripper in average credit, and we would also use more power.

At over_volt 4 and 2015 Mhz, I'm seeing around 4.5W - 5W total consumption (my UPS doesn't measure fractional Watts, so it's very difficult to give an exact measurement here). If we assume a 35% increase in Average Credit (linear based on clockspeed), we would expect 1148 Credit for the Pi 4 (230-255 credit / watt). We still would be less efficient than the threadripper, but we would "only" need 103 Pis to match the threadripper in average credit.

In comparison to a more consumer and budget friendly CPU, the Ryzen 7 3900X (https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=3896047) which is getting about 27,500 average credit at likely 140W (https://www.anandtech.com/show/14605/the-and-ryzen-3700x-3900x-review-raising-the-bar/19), or ~197 Credit / Watt, the Pi 4 4GB still comes out ahead in credit per watt even overclocked.

In Summary, if you want the absolutely best performance per Watt, you're better off going for the top end Threadripper. Against most other processors, an Overclocked Pi 4 will be more efficient, and overlocking the Pi 4 allows you spend less money and allows you to manage fewer physical systems.
36) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 95575)
Posted 30 Apr 2020 by Endgame124
Post:
This article is pretty good regarding over clocking and power usage ( see table 2/3 through article)

https://qengineering.eu/overclocking-the-raspberry-pi-4.html
37) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 95568)
Posted 29 Apr 2020 by Endgame124
Post:
That sounds nice, especially the default consumption of 3.5W. Could you perhaps plot a diagram with each attempted frequency and the lowest over_voltage at the given frequency? It can also be set to a negative number. I'm interested in how much undervolting counts in temperatures.


Would a table like this work?

Freq Over_Volt Result on 4 Units
2125 6 Fail
2100 6 Fail
1900 4 Pass

Of note, the rebooting caused from the testing at 2125 and 2100 caused my work units to fail. The testing I did at over_volt 4 and 2000 freq seemed to be affected by this - the work units were canceled by Boinc itself at some point, and the 4 completed at 1900 were different units. I'm now trying 2015 freq on over_volt 4 and it seems to be working fine (2025 caused the random hangs - the OS wouldn't crash, but it would get "stuck" for 20 seconds, then recover, then get stuck again).
38) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 95542)
Posted 29 Apr 2020 by Endgame124
Post:
My pi 4 4GB was NOT stable at over_voltage 6 and clockspeed of 2125 Mhz with 4 Rosetta threads, though 3 threads worked fine. With 4 threads, it would run for 3-4 minutes, then spontaneously reboot as soon as the 4th rosetta process finished loading. I also tried over_voltage 4 and 2000Mhz with the same behavior, so I'm now trying at over_voltage 4 and clockspeed of 1900, and it seems stable over the last 30 minutes. I'll try pushing 2000Mhz at over_voltage 4 later
39) Message boards : Number crunching : The most efficient cruncher rig possible (Message 95522)
Posted 29 Apr 2020 by Endgame124
Post:
"Endgame124" wrote:

Note, power reduced pi 4 is using around 3.5 watts. Maybe I didn’t follow something, but a pi 4 running for 24 hours should use around 40 to 48 total watt hours, not 960

Thanks Endgame124

hi all,

please note this, the power consumption of Pi4 in my earlier post https://boinc.bakerlab.org/rosetta/forum_thread.php?id=13791&postid=95478 may be grossly overstated

if this is true, Pi4 consume 4 watts that would only be 4 watts . 24 hours ~ 96 watt . hours in 24 hours and it gets about 1000+ points (overclocked 1.75ghz) in that time frame. it would actually make a Pi4 very efficient in terms of points per watt it becomes 4 times more efficient than say my haswell desktop takes 400 watt . hours to get the same amount of points


I see where the confusion came from. I listed the total power draw on my ups, and left it to the reader to calculate the power draw of the pi. If you don’t read closely, it looks like the pi 4’s power draw is much higher. I can’t edit the post, so I can’t fix it to make it more clear. I’ll try to collect everything into a summary post later.
40) Message boards : Number crunching : Running on a 4GB Raspberry Pi 4 - How to? (Message 95521)
Posted 29 Apr 2020 by Endgame124
Post:
Additional temp data

11) Passive w/ top on, hdmi off, wifi off, bluetooth off, led off. 58c

Looks like turning off bluetooth / wifi / led makes no temp difference.

I have now acquired another PI 4. This one I'm overclocking with these settings:

over_voltage=6
arm_freq=2125

UPS reads a solid 5 watts for this configuration, so a 41.7% increase in clock speed requires a 42.9% increase in power.

2147 is the maximum frequency, so I have a little room if I want to add more clock. I'm not sure over_voltage 6 is required for this clock speed, I'll probably try backing voltage down if it looks like its stable at 2125.

Apparently the GPU clock is attached to things like the bus speed, so there may be additional gains to be had increasing the GPU speed, even though I have hdmi off. First things first, though, and the first is just verifying stability.


Previous 20 · Next 20



©2022 University of Washington
https://www.bakerlab.org