Not just the compute but crucially the gateway will absorb the 2.7x upload bandwidth requirement which may be off-putting businesses running tight on bandwidth.
We are working on it right now
2 Likes
That’s not possible, at least not one that is on your network. Maybe for a hosted gateway, as you could send a 1:1 copy to the hosted gateway and then the hosted gateway splits it into pieces, but that’s something Storj/Tardigrade staff or maintainers would need to directly comment on if they’re going to allow that since it’s no longer encrypted before it enters their custody.
Throwing a wild idea out (I do not know if this is a good one): What about downloading multiplayer virtual reality worlds? Would this make sense from a performance and usability standpoint?
“Virtual reality and virtual currency may be a very obvious marriage. VR will be to crypto what the iPhone was to the internet” - Balaji Srinivasan
Ok podcast on this starting at 9:45 https://hwcdn.libsyn.com/p/d/1/e/d1eb8e8b5de909ab/WBD146.mp3?c_id=51569405&cs_id=51569405&expiration=1607588449&hwt=cad0bfa02821d808777360751ae1ac82
Why? You can encrypt the data and then give it to the gateway to encode and split up the encrypted data.
Isn’t this being done right now? The uplink encrypts the data and only then splits it up, so that the satellite can repair the file without having to decrypt it.
Sorry, I was indeed referring to a hosted gateway.
My bad.
Yes, libuplink does it all before it leaves your system- we’re not talking about that paradigm though. The suggestion was the gateway taking on some of those functions, some of which are compute or network heavier, so that the uploadee did not need to care, or have as much supporting resources for, uploading of objects to Storj. Currently, yes, things are encrypted in libuplink before they’re pieced out to SNO’s that the SAT listed out… the thought suggestion being, with a hosted gateway, you’d transfer a 1:1 byte copy to the hosted gateway through a SSL encrypted tunnel, to of which then does the encrypt (compute heavy) and avoids the 2.7x byte size multiplier due to the EC profile Storj uses for block resilience. The issue I saw with this is, yes, the originator of the data would have to, especially to meet compliance, encrypt the data on their side with GPG before starting the upload- this would defeat the desire to shift compute costs to the hosted gateway.
As for the 2.7x size increase due to the EC profile, this would’ve helped get closer to other CDN like performance where you are only downloading the 1:1 byte copy of the original file. An end user downloading at 270Mbps but only getting 100Mbps effective data out of it could cause some annoyance.
However, if the client has to encrypt the data (with PGP or whatever) anyway, then we can split the two functions of libuplink.
Encrypt the data on the client machine, then give it to the gateway, which encodes it and distributes to the nodes. This would save some CPU cycles and a lot of bandwidth for the client.
If the client wants to store unencrypted data (maybe it’s something that will be publicly accessible on a website or something), he should be able to that as well, just skip the encryption step on libuplink or something.
Downloads are 1:1, overhead of 2.7 is only for uploads.
Actually there is a slight overhead in downloads as well; libuplink downloads more stripes than it needs in case some nodes are slower. I don’t know the exact factor though.
yes but it is only a bit of overhead and the slowest transfers are cancelled before they can finish. But it’s less than 1.5