Raspberry 3/4 Node Owner - Did you do any optimizations?

@anon27637763 I will tend to disagree to some extent.

I’m trying to understand where 20-40ms latency difference comes from.
On my RPi4 with the OCZ Vertex 2 SSD connected through USB3-SATA adapter I can get >300 MBps sustained read-write rate and a few thousands random reads per second. So the RPI’s USB bus must be fast enough to not likely cause the latency bump comparable to tens of ms.

Something did change, though I’m not sure it aligns exactly with that release.

@Vadim It doesn’t encrypt the piece’s raw data for the client however I’d expect the whole communication channel between the client and the node should be encrypted otherwise it’d be susceptible for MitM attacks.

On Raspberry Pi USB ports

All quoted text is from:

Raspberry Pi USB Hardware Documentation


Please note, that I have all models of the RPi including the Pi Zero and Zero W but not the 4… and have done extensive experimentation over the last 6 or 7 years on each of the boards I have acquired. This does not make me an expert in any way, and I’m not sure I respect those who claim to be experts anyway.

Pi Hardware

In all models prior to the Pi 4, the USB ports connect to a combo hub/Ethernet chip, which is itself a USB device connected to the single upstream USB port on BCM2835. On the Pi 4, the USB hub chip is connected to the SoC using a PCIe bus.

Thus, all RPi models other than the Pi 4 will very likely experience significant performance issues when used as a Storj Node. The shared bus bandwidth and processing resources will necessarily slow down data transfers to/from BOTH the USB and Ethernet ports. This will most likely result in any given RPi node (other than a model 4) processing network requests far slower than any other platform/node running in the same WAN subnet.

For the Pi 4, a fully-featured host controller drives the downstream USB ports. Downstream USB is provided by a Via Labs VL805 chip

this is connected to the BCM2711 SoC using a PCIe link, which is extremely fast.

All connected USB 2.0 devices are connected via an internal hub which connects to the upstream PCIe link via a single USB 2.0 bus, giving a maximum combined bandwith for all USB 2.0 devices of 480Mbits/s.

USB 3.0 ports (blue) are ALSO connected to the USB 3.0 bus via the USB 3.0 specific pins in the socket. USB 3.0 devices are constrained only by the total bandwidth available over the PCIe link.

Therefore, in order to have any improvement in performance using an RPi 4, one must use a USB 3.0 External HDD. All USB 2.0 External HDD are going to suffer bandwidth constraints similiar to all prior RPi USB connected devices. However… the precise speed of the PCIe link is not ennumerated in any documentation I could find… nor is the bus width indicated.

Unfortunately, the RPi documentation is very poor. Broadcom is quite overprotective of their chip designs even so much as to deny the Open Source community the ability to boot their chips without their proprietary binary blob tainting the Linux kernel.

My guess is that the RPi 4 PCIe bus width is simulated through internal multiplexing and the bus speed is constrained by both the complexity of the internal circuitry as well as the parasitic operational noise on the board itself. All-in-all the RPi boards are excellent hobbyist tools for experimentation, but are unsuitable for any production environment application.

However… an old netbook running some USB connected drives may provide similar low power requirements but much better data piece catching success. Don’t know, haven’t tested it…

3 Likes

Mhh i think there is no netbook whit a Power consument whit less than 3Watt.

Soon i will set up my second node whit a raspberry pi 4, WD Red and later the PI Hat.
Would nice to see if there are any other results than whit the raspberry 3.
But at the moment i think, we need a special green Raspberry Satelite ^^

1 Like

I’d really like you to update this thread when you try out the Pi Hat.

My guess is that it won’t improve anything, because the GPIO interface isn’t high speed.

However, it’s worth a shot and an interesting experiment even if it doesn’t improve the success rate.

Btw, if somebody is comfortable overclocking their RPi4 (e.g. 1.5->2GHz with cpu cooler) it’d be interesting to see benefits it can give.

I also can check later, maybe in 3-4 hours.

Its not a normal Sata Hat and not work over the GPIO.
The GPIO is only to power the Raspberry PI.
The Hat use two JMS561 Chips for fast transfer over USB 3.
In Raid 0 its possible to get 400MB/s reading/writing performance.
I will make a review whit the new Hat when i get it. Maybe after this, if some are interested, we can order some together whit all and get a better price.

Im still in contact whit the Devs and i will get the new Version of the Hat. But have to wait because the Virus in China.

@xopok would nice to heare some results. My “Gameserver Pi 4” run at 2150 Mhz and its stable.

2 Likes

RPI 4 hat is on USB3 interface, so it wil be the same as USB hdd

1 Like

running a raspberry does not actually have to be green in all cases, imagine SNOs running a Pi + 2 inefficient USB drive enclosures…but I get your point and kinda like it

@MadeInGermany I like it :slight_smile: If pricing for green would also be lower I’m sure some people will easily slum it with slightly higher latency :slight_smile:

1 Like

Yeah, that what im thinking. So i say 75% of the normal price. So this nodes would get only 75% too. But thats okay. But dont know if a Satelite can ask a node what CPU are using and if it only can connect whit nodes whit ARM Cpus. Dont know. Just an idea. Maybe someone if the Storj will read this ^^

If not, lets have a look how we can do our Pi nodes better :slight_smile:

And how are you going to enforce the rule that only RPi nodes are allowed there?
I’m pretty sure I would want to join the satellite with my server. Traffic is traffic, even if it pays less (well, unless I max out my internet connection).

1 Like

Nothing greener than running your node on hardware that would have been up and running anyway. I think the idea is that the current satellites are already a much greener option than large data centers. No need for a separate one.

1 Like

I write i dont know if it possible that a Satelite ask the Node befor a connections starts, which CPU they had. If yes, its simple to check, if the node have a ARM CPU. You can ask this simple whit a Browser. But if a client can ask this? I dont know. And that was just a funny idea :smiley: Not something that i want know for all Raspberry user or i shout down everything ^^

Well, I guess I should get an ARM based server then, it would be faster than a Pi.
However, since the node software is open source, I could probably mod it to return whatever I want as the CPU :slight_smile:
I don’t think this idea would be possible to implement properly.

2 Likes

I think it doesn’t make much sense to limit who can join satellites.
What if instead we can allow storagenodes and clients to set bid/ask prices themselves?

For example, any storage node can define the price per stored TB*month and per downloaded TB (e.g. in the ‘run’ command, right next to the available storage and bandwidth.

The client at the upload time can also set the price they are ok paying for these.

The satellite then can then determine a set of storagenodes that together can provide the necessary services for the price while also trying to guarantee with a high probability that there is the sufficient number of other nodes in the network the repair traffic can be directed to.

RPi nodes can get really low on these prices and it’d be economically inefficient for dedicated server owners to compete.

I don’t agree, as it could cause that small single node operators wouldn’t stand a chance against larger SNOs that wouldn’t mind set the price really low with a performance way over the PIs.

2 Likes

A small RPi-backed SNO can set a price that is lower than the electricity cost of a large SNO and still be profitable. What’s the point for the large one to set such a low price and effectively lose money?

If SNO A has a server running with 24 TB of drive space and only 18 TB of that drive space is currently contracted out for “Project X” … then the SNO might incur nearly zero extra overhead by running an SN and allocating 5 TB to a storagenode.

While the storagenode isn’t going to pay for the entire server’s electricity cost, the SNO can recover some portion of the sunk costs of running the server.

1 Like