I’ve been reading through the Storj documentation recently. First of all, I’m really impressed with the technology. I’m not a cybersecurity specialist but I very much appreciate the decentralized structure and the access management where authorization and encryption are encoded in the access keys themselves.
There’s a few questions and clarifications I’d like to ask. I suspect that some of these are probably explained somewhere in the documentation but they’re still not entirely clear to me. If there’s other threads that cover one or more of the points, please let me know.
Do I understand correctly that any files uploaded via Uplink are user-side encrypted and require 2.68x the upload bandwidth? Does this multiplier also apply to downloads? As far as I can tell, it doesn’t affect the actual file size in the bucket.
Do I understand correctly that all that needs to be done to make an access grant “S3-compatible” is to register it via the corresponding CLI command? What exactly happens there in the background? Is there a direct interaction with a server involved or is this more of a registration in the sense of “embedding additional information” in the key locally?
In Uplink CLI, what exactly is the difference between “access create”/”access restrict” and “share”? It sounds as if they serve a similar purpose (providing access keys to different objects, handling permissions etc.). When should one use the “access” vs the “share” commands?
Is there still a significant speed advantage of Rclone vs Uplink for downloads and uploads (as described here: Hotrodding Decentralized Storage) despite Uplink now also supporting parallel transfers and multi-segment uploads? If so, I guess it’s because of the 2.68x larger transfer size due to encryption?
Is there any difference in structure or behavior between objects uploaded via Uplink (user-side encrypted) and objects uploaded via S3-compatible applications (server-side encrypted)? Or are they the same once uploaded and the different upload/download modes only affect behavior during transfer?
No. With uplink the client initiates 35 download connections, of which 29 have to finish. So there is a bit of overhead, but only at most 6/29 ≈ 20% in case all 6 nodes are still so fast they manage to send data back.
That’s correct. Storj may change the redundancy parameters at any time, but it is an implementation detail, so they show (and charge for) numbers before applying redundancy.
No difference. Any file uploaded through an S3 gateway is downloadable through uplink and vice versa (assuming the same cryptographic keys are used).
That’s correct. They are just different modes of transport.
I don’t know the answers to other questions, sorry!
Yes, or you may use a satellite UI to do it for you, however it will do the same under the hood.
If you register your access grant on Gateway-MT, then It’s described there: Understanding Server-Side Encryption - Storj Docs
You may also host your own M[ulti]T[enant] Gateway or Self-hosted S3-compatible Gateway (aka Gateway-ST - S[ingle] T[enant] Gateway). For the Gateway-ST the access grant will be in the config file of that Gateway service, so it must be hosted securely in the trusted environment.
It depends on many factors. If you Configure Rclone Natively - Storj Docs, then perhaps they would be on par with the uplink, because both will produce 2.68x upload traffic due to encryption, erasure codes and an overhead for the long tail cancellation (your libuplink in rclone/uplink will request 110 nodes for each segment of the file (=< 64MiB) and will start upload pieces of that segment in parallel, when the first 80 are finished, all other got canceled - thus you always uploads to the fastest nodes for your location), but rclone also have a retry feature, which is currently missing in uplink.
But if you configure rclone with S3 protocol, then the type of the transfer will change - it will not encrypt and erasure code your files, it will securely transfer it to the closest GatewayMT instance to your location, and encryption, erasure coding and uploading to thousands nodes will happen there, not on your device. So in this case you will upload with an expansion factor of 1.0x and resources of your device will be spend only on preparing/combining/buffering chunks for multipart uploads/downloads.
In general, if your upstream bandwidth is less or equal 1Gbps, the S3 protocol would be likely faster than native, but depends - you need to test it, it’s highly depends on your location and setup.
Not a big difference. The share command is a combine: it can generate a new access grant, register it on GatewayMT and generate a shared link in the one command, see examples.
With the access create command you may just generate a new access grant and also import it as a new access to uplink, see examples.
With the access register you may only generate an S3 credentials.