I would like to use STORJ for multiple applications which only support S3 for object storage. I checked out the managed S3 gateway, but I don’t like to give away my encryption keys and deploying a separate S3 gateway-st for each application doesn’t make sense to me. Finally, I found some documentation (1) about how to setup the S3 gateway-mt and tested it. It seems to work fine, but as it required a not insignificant amount of debugging, I’m wondering if anyone else is using it this way (besides development).
Is self-hosting the S3 gateway-mt even intended and suitable for a (semi-)production environment?
Initially it’s not intended to be a self-hosted unless you want to launch an own Storj network, because you need to host several services to make it work (linksharing service, Auth service and Gateway-MT) and it’s a multi-tenant gateway (it can work not only with your API Keys, but anyone’s else, so if you publish it, I can register on it as well with my access grant).
Regarding resource consumption it would be easier to host only one service - S3-gateway (Gateway-ST, single-tenant S3-compatible Gateway) inside your secured perimeter (it doesn’t support https out-of-the-box) and use it in all your applications.
If you run only Gateway-MT without all dependent services, you already provide your access grant in encrypted form to our services (Auth service specifically), so why to bother with your own Gateway-MT at all in this case? You may use ours as well.
I’m aware of these dependencies and of course I also run an internal auth service beside the S3 gateway-mt.
As far as I understand, using one S3 gateway-st for all my applications would mean, that all applications have the same permissions on the buckets as they’re using the same access grant. My goal is to have an access grant per application so they only can access their own bucket.
Is this a use-case you may want to support in the future?
If so, I could work on a Kubernetes Helm Chart for deploying the S3 gateway-mt including dependencies.
I do not think, it is in our roadmap. For the per-application access grant you have either use a separated Gateway-ST for each of them, or Gateway-MT with all dependences, or do not use S3 protocol at all.