I think that there is some mis interpretation about how big customers are. In the Netherlands we have something we call MKB (middle/small business) and accounts up to 250 employees. I’m guessing those don’t have the hardware ore connection to handle that kind of traffic. But I’m also sure that most of them don’t have that much data to upload.
I’m talking about much bigger company’s (like 5000 employees) or organizations that also handle a lot more data. Those company’s definitely have the capacity to handle that kind of traffic. Doesn’t mean they handle 30TB a day of internet traffic 
You know the one good thing about a local S3 gateway? It saves Storj money. That’s it. There’s nothing good about it from a customer point of view, including encryption. Customers have been encrypting their S3 data before handing it to S3 since S3 was launched 20 years ago. Or, they just don’t care about encrypting their data.
It’s like talking about all the advantages of uplink. For a customer, there aren’t any. Again, it just saves Storj money and lets them offload overhead onto customers.
A customer shouldn’t have any knowledge about uplink or what it does. They should only care about a S3 Gateway if this somehow benefit them. And I think that (in most instances) only benefit them if it is cheaper to host it themselves instead of using the hosted one. Its for Storj to decide if they can financially & technically make it work, I just don’t know what the current costs are to host it in the current form. If they can run the hosted gateway + needed infra for a few pennys, then its going to be verry difficult to make a pricing plan that undercuts the competitors.
For a customer there are plenty. All those reasons they choose storj are invalidated by slapping centralized s3 on top. Including massive performance and distributed instant access.
S3 gateway is a crutch. That they may choose to host themselves to alleviate some of the issues.
- The Storj satellite is centralized, same as the S3 gateway.
- Show me a benchmark where uplink is faster when compared to S3 running the same parallelism.
- You get the same global distribution using the S3 gateway as with uplink, only the distribution is relative to the S3 gateway’s location instead of the uploading location.
You do get these fun disadvantages with uplink:
- it takes twice as much customer bandwidth for uploads
- it takes way more CPU power to erasure code the data, especially with parallel transfers
- with a parallelism of 16, you get to manage 1200 simultaneous network connections (16 x 75) instead of 16. For one file upload.
- you have to find software that supports uplink or do it yourself
Irrelevant. You transfer data to and from nodes. Not satellite. Also, satellite is distributed as well.
Rent Amazon vps and test it yourself. Takes 20 minutes and twenty bucks. Also look for the Edinburgh University paper who characterized a very specific usecase.
Performance is vastly different.
Not accurate.
Not twice. But yes, more. That’s the cost.
Same objection. This work has to happen somewhere.
Irrelevant. Why would you upload a tiny file with 1200 connections?
Rclone supprorts it and bridges to literally anything including SFTP. Not really a problem.
The only benefit of a gateway external to the network is weak uplink. Customer with weak uplink can legitimately prefer external gateway, be that s3 or sftp or what have you. This is solving a different problem.
PS. You keep posting ignorant misinformation that is a Google search away from being corrected. Given the amount of attention to detail required to write backup software, that casts a pretty strong shadow on your product.
I already explained in some other thread, that S3 protocol is not designed for the distributed storage, it also doesn’t support encryption at rest.
Basically every node will need to run a gateway-mt and contact all other nodes the same way, as this is done by the current distributed gateway-mt, when the customer come to upload or download. It’s not guaranteed that the upload will end on the same node, which now works as gateway-mt.
There are several security issues:
- Running gateway-mt requires trust, because it will contact an auth service, which stores encrypted access grants and provides them by request. Then the gateway will use this decrypted access grant and act as an uplink on the customer’s behalf.
- The SNO can get a decrypted access grant of the customer, if they intercept it from the memory, because they physically have access to their own device. They can also modify the binary to log all decrypted access grants.
- The SNO can collect used S3 credentials too.
The second problem - in the S3 software you need to provide a gateway URL. Theoretically to have a distributed load balancing we can register under the one domain name all IPs of SNOs’ gateway-mt instances, so the selection would a DNS round-robin. Working with BGP on an automatic basis with high IPs updates sounds risky.
If not going with a gateway-mt approach, then the unencrypted whole object will be stored on the node. It sounds like IPFS, i.e. no encryption at rest, no redundancy, no parallelism.
S3 doesn’t use segments under the hood. They have a concept of chunks for multipart upload/download, but they must go to the one gateway, otherwise you will not be able to reassemble them to the object.
If there would be a load balancer hosted by Storj, then it’s already implemented as gateway-mt, no need to reimplement the native protocol, but now use S3 instead of DRPC.
It makes sense only if the load balancer will not be hosted by Storj too and will not be required to be hosted by a customer. Which again means that the each node should be a gateway-mt instance alongside with storagenode.
Small thing nobody said: Wasabi and B2 can give egress away because a download costs them almost nothing, it’s their own servers in their own racks. On Storj every TB out is money that is paid to nodes. So “free egress” can’t exist here (somebody should pay for it), and that one of key points of the whole pricing discussion.
It can be compensated by increasing the storage price twice, which we did for a tiered pricing. However, not so much of interest, now we are back to a simplified pricing: no free 1x egress for 2x storage price, no segments fee, all simple - minimum object size is 50kB, 30 days retention, $7/TB-mo storage and $7/TB egress.
As a consumer, I’m not paying my ISP any extra because I have node related traffic. I think that’s for most SNO’s. Off course when every household is a SNO, the ISP probably change somethings. Until then, every penny that a SNO gets (including me) for egress is a win. So maybe egress payout could be lowered even further?