So I am running my 16TB node off of my rpi 4 8GB (USB 3.0). The OS, DB, and identities are on an SSD (USB 3.0). I have 2 USB 2.0 ports left. Will the pi 4 8GB be powerful enough to handle another 16TB node once this one is full? In terms of powerful enough, I mean the CPU, USB bandwidth, and USB 2.0 speeds.
If not, I have 2x rpi 3 model b+ that I can use (They have 1GB RAM only).
Why don’t you try it out and let us know? Now is a good time, you’ll start getting a lot of traffic from Salt Lake as soon as you spin it up (the vetting requirement has temporarily been removed on that satellite whilst network performance testing is ongoing)
Sounds like a bit of a Franken-node to me, though! 
(1GB RAM on a Pi3 sounds a tad unlikely to have any decent performance, not sure it’s worth the bother unless you’re bored)
I was thinking about migrating my node to a pi5 with 8GB RAM to reduce electricity costs. Now that the code is more optimized that might be possible.
I have one concern. How do you all speed up the file walker? Currently I just finish any filewalker process in minutes thanks to ZFS. The Pi5 doesn’t has enough memory for ZFS. So it needs to be ext4 I guess. How do you make it fast?
I reduced retain.concurrency to 1 and enabled lazy filewalker.
Seems to be coping just fine. Gets a bit hammered during garbage collection, but most of that is IOWait from the USB spinning rust. The nodes I have running on Pi5s with SSDs are doing perfectly fine (will never turn a profit but I have more money than sense…)
I would also suggest a large-ish, high endurance SD card. The logging is quite extensive when there is a lot of traffic. Or just boot from a NVMe drive using a NVMe HAT (I will gradually install NVMEs for boot drives on all my “potatoes”)
I’m running 3 nodes (1.7TB, 1.7TB, and 6TB) on a RPi3, the 4 and 5 should be more than capable of a second node. This is also just running each node on individual HDDs and everything else on the Pi’s MicroSD. The 1GB of memory on the RPi3 is probably the main bottleneck, followed by the SATA to USB dock I’ve got for the drives. I forget the dock’s chipset but I know it has some I/O wait issues. The filewalker operations give me grief but I haven’t DQ’d yet. I’ve got a gifted RPi4 (4GB I think?) that I’m going to swap in at some point and also set up (at minimum) the DBs on an external M.2 drive attached to USB. Probably going to go a long way for my rig.
Edit to add that I’ve been running this for… 3 years now I think? Drives are basically full.
I have 2x 10 TB running on a Pi4B with 4 GB and struggling with file walker. That must be due to RAM and USB limitations. It’s a mess, but I’m not convinced to invest in new hardware.
So you think a pi 4 8GB would be able to keep up with 2x 16tb drives?
Compare USB specs - if they are similar : NO.
On my Syno 1GB, these impruvements work:
What do you mean?
For USB on the pi 4 I have the following:
2x USB 3.0
2x USB 2.0
OS and DB drive (SSD): plugged into USB 3.0
16TB Node: plugged into USB 3.0
2x USB 2.0 ports remain.
The Pi is limited to 10 MB/s transfer rate, on USB3.
I do not have it at hand, but that’s an issue.
That doesn’t sound right…
Is there no other way? How about a caching SSD. Would that help?
Did a bit of digging, it seems one problem I’m having is that the external dock doesn’t support UAS with the current firmware. It is a JMicron JMS567 chipset and I know there are many discussions around about this not properly providing UAS on Linux although it does work in Windows I think? Looks like if you’re going to go with something with a JMicron bridge chipset you’d want to find a JMS568.
You could use something like LVM cache or bcache.
I wonder how well ZFS special metadata device works when under low RAM too.
But if you haven’t purchased the Pi 5, consider buying an N100 mini PC, you could put in a 32GB SODIMM.
Don’t know, but should help. If the RAM can’t cache the entire filesystem, which would be ideal, than a SSD cache would be next best thing.