Raspberry Pi Node Operators?

Coming up on 8TB of data on my Synology NAS and was looking to expand with another node using Raspberry Pi… Anyone have recommendations or bad experiences or any tips for hosting with RPi? Share your RPi setup :smiley:

Take a look at :point_down:

Many had problems with USB connections.
Take a look at this, also:

https://forum.storj.io/t/new-storagenode-build-intel-n100/24861

what was your problem with the USB? What does the n100 have to do with Raspberry Pi?

USB drivers very often have problems with overheat of the controller (they rarely designed for a long work), also the cable maybe not good and disk may disappear from time to time.
The external USB disk also should have an external power supply (the USB itself is not enough to give it needed power).
And almost all 2.5" disks are SMR.

I gave you an example of something else than Synology, because I am too in searching for new setups, and I want to avoid paying premium for low performance hardware.
I didn’t use USB drives for Storj, but the forum is full of posts with sno’s having problems with them, especialy on big nodes.
And with raspi, as I understand, the SD card needs to be replaced from time to time.
But do what you want, it’s your money and time.

mine WD elements 12 tb near full: works fine with dbs on usb.

this may be related to writes for "orders db " or/and Log’s / swapfile
if you redirect them to a durable drive/ssd there should be no big writes, (loglevel fatal) so near zero sd card wear. (not sure but what else?)

Don’t…

Probably avoid in 2024 - the cost of a Rpi and it’s TDP is no longer class leading, the N100 and AMD 5/7 U processors are far superior, with the drawback of an initially higher purchase cost. If you are buying something, I would buy hardware that can do more than just host a storagenode, with the Rpi you will be very limited on what else you can run on it and will be facing many years to turn any form of profit to cover initial purchase costs given the Storj current SNO pricing model.

If you have a spare Rpi4 kicking around, and a spare old HDD then it’s ok for running a storagenode - you would benefit from adding an SSD for LVMcache or DB and log redirects, but that can be done once the node has grown and been vetted.

For the HD connection, avoid 3.5" disks supplied in enclosures, the Rpi4 is picky on USB controllers used, stick to self powered external 3.5" hot docks or enclosures - needs to support UAS mode and USB 3.1 ideally with Linux support, you can usually tell if SMART works - controller might be ok.

Don’t enable Quirks mode to disable UAS - if that’s needed, replace the USB HDD enclosure else you will get high I/O waits, and unresponsive Storagenode.

Stick to Raspberry Pi OS 64bit lite - has the best support for USB controller as of Jan 2024…

CP

2 Likes

Someone posted a while ago the exact opposite. I don’t know who has first hand experience with UAS on or off…

Thanks for all the replies, I literally have a bag full of raspberry pi 4Bs I got for like $15 so I’m going to give it a shot with one of these kits from geekworm. I’ll let you know how it turns out. I figure if a Synology system can run a StorJ node, then a Raspberrypi 4 shouldn’t have that much more issue. I’d also be curious to see how the RPi 5 holds up too. I’ll probably be testing that as well.

1 Like

Lot of good stuff in here, thanks CP~

Very cool board you linked, thank you. I run 2x 3b v1.2 1gb with 8TB 5400rpm disks, they work well and generally have low cpu load.

1 Like

I use RPi 3’s and 4’s and a number of other single board computers with USB drives. There are no issues with them. They all work fine. I tend to run one node per device, other people run more. Seems to work fine for them as well.

3 Likes

UAS on raspberry pi 4 shall be definitely avoided. On others — it depends.

Finding an enclosure that supports it properly and does not exceed raspberry pi’s power budget is a whole separate, mostly unsuccessful ordeal.

Using usb mass storage mode is slightly more compute intensive but has a benefit of actually being stable. UAS, on the other hand, is supposed to provide higher max performance (irrelevant for storagenode usecase) but in reality causes hangs, stalls, and disconnects. Between the two — slightly more power use vs data loss — it’s not really a choice.

Care to do as ;

lsusb

Would be interesting to see which hardware the controllers are, then can see if they are in the auto quirks ban list - would then be a great post where we could list the USB to SATA controllers that work well with Storj, and the ones where people experience issues.

Lots of hate on hear, don’t get me wrong I love the Rpi, I’ve developed on it for many years but some of the things being posted are misleading, and will catch newbies out - Is it really the Storj official line that starting a new node on Rpi4 in 2024 using a 8TB disk will in anyway turn a profit in under 2 years ? Great for learning, but as I posted in response to the OP better out there… The Hat board indeed is cool, but has a picky controller USB to SATA bridge, you will see issues with Samsung EVO SSD’s and it seems to dislike Toshiba Enterprise 10TB+ hdd, but I like the vendor and the boards they create really push the Pi limits.

On the UAS front, it is a completely different technology - it’s parallel disk queues and is very relevant to storage node performance - It allows for Pre-Emptive deletes, and non-blocking IO - You won’t see much mention about it, as it really is not easy to make work, but you can get it working and the performance is very good with the right USB to SATA controller.

I.e Many are available, but this supports full UAS mode with tweaks below;

ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge

It needs some boot parameters being applied, specifically
coherent_pool=1M
to address a out of memory condition under heavy read I/O on Mass Storage Driver on Rpi4, and CMA defaults… it’s a mess, and is different on different distros so no wonder people jump for the Quirks mode enabled…

You also need to manually find the sweet spot on adjusting, it’s different for different disk geometry, but a good place to start is 50, then increment by 10, you will probably find around the 140 mark you will start getting bus resets, then turn it back abit.

/sys/class/block/sdb/queue/max_sectors_kb (obviously replace SDB :stuck_out_tongue: )

This is caused by USB to SATA controllers hard coding this parameter to 512 or worse, that causes the disk to send data to the Rpi quicker than the Pi can process, the buffers fill up, then you usually get a USB bus reset, before the kernel ejects the disk and a reboot is required.

This is all much better on the Rpi5 - we have new USB controller to work with, and brand new drivers, so hopefully all the pain of USB storage will be a legacy of the Rpi4 and older…

Anyhow, I’ve forgotten what I’m replying for now :yawning_face: I completely fail at multi-tasking…

CP :heart:
#Opinions are my own

2 Likes

Unfortunately it doesn’t help much. I was forced to reflash it (not replace though) every 1,5 years, because OS got updates and sometime it’s corrupted, because SD card is not reliable device.

In my case, lsusb doesn’t show any SATA bridge controller device. Primarily just hubs, keyboard mouse, etc.

For instance…

Seagate RSS LLC Backup Plus Hub (Mass Storage)
Seagate RSS LLC Backup Plus Hub

Those are the names of the two devices for the drive that is connected to this RPi3.

An RPi4 shows…

BUFFALO INC. (formerly MelCo., Inc.)

No hub on that device.

So, not sure this is what you are looking for. These are all combinations of external drives to USB. I have several more but my guess is the results will be similar.

I would love to see someone trying out Rpi 5. Making flash drive as your boot drive was the right decision in my case. No more nightmares about SDcard

1 Like

he propably end up with this :joy:

To bad, i have no time…would love to do some trial and error with raspi’s

Im using Raspberry Pi’s
2x Raspberry Pi 3b+
3x Raspberry Pi 2b

Raspberry Pi 3 → 1HDD 8TB Node
Raspberry Pi 3 → 1HDD 6TB Node

Raspberry Pi 2 → 2HDD 1TB Node & 1TB Node
Raspberry Pi 2 → 1HDD 1TB Node
Raspberry Pi 2 → 2HDD 500GB Node & 500GB Node

The Pi is on the Back of the HDD with a 3D-Printed Mount


4 Likes