Storj DCS S3 gateway and HTTP endpoint

Product Discussions
1

Hi!

I was looking into using Storj DCS as an S3 replacement for a service I plan to host publicly (on fly.io) but I cannot see any way of using it as the services requires the objects stored remotely to be accessed via a URL.

The only option I can see is using the uplink to generate a HTTP endpoint and put it behind a reverse proxy for SSL.

I there anything already hosted by Storj DCS for this? Like the S3 gateway (which I love) but for HTTPS serving of content.

Thanks in advance!

(Btw, I didn’t knew where to put this topic because there is no Help section for DCS questions :sweat_smile: )

2

I may be misunderstanding (and don’t know fly.io) but the link share service could be what you’re after. Files can be linked to (and downloaded) directly with SSL support, just not using your own domain name - Linksharing Service - Storj DCS Docs

Presigned URLs could also be what you’re looking for - Using presigned URLs - Storj DCS Docs

2 Likes
3

This is exactly what I was looking for :slight_smile: Being able to store an image and return a valid URL for the frontend to embed it, for example.

Thanks!

4

As an extra point and future improvement suggestion, it would be nice to have a way to generate a passphrase that can be used to be able to download files through the gateway without signing them.

That way we can serve static content while adding a CDN in front of it for custom domain name + cache.

That is a very common use case of S3 and its competitors and not all of them have this option!

5

Isn’t this the linksharing service I linked to above? Like this - How to bulk get the browser links of a folder? - #2 by Stob

1 Like
6

Woops! It looks like. I misread about it yesterday and I thought it was only for manually share indivual objects. It may work then!

I will try this afternoon putting it behind Cloudflare and lets see :slight_smile:

1 Like
7

I think it doesnt’ work as I expected :confused:

When I generate a sharing link I can see that:

  • If it is a link share for the bucket it shows the whole bucket, which is confussing as anyone can see all files and directories - not good when you are storing avatars for example, it is easy to be able to download all of them.
  • If I share (or build from the bucket URL) an object URL, it shows me a full web page with information about Storj and a button to actually download the file.

It is nice for casual file sharing, but my use case is different, like when you expose an S3 bucket over HTTP or a DigitalOcean Space over CDN:

You can build a URL and the file downloads directly, but you cannot traverse directories (at least by default), and no “intermediate” page is shown, so the links can be used to put in HTML elements attributes like <img>.

Edit: loking a bit deeper in the docs I found this: Host a Static Website with the Uplink CLI and Linksharing Service - Storj DCS Docs . Will investigate.

8

I’ll be honest I hadn’t checked those pages in a while and the recent updates have removed a lot of the helpful detail. You need this section on this page - share - Storj Docs

“To download content directly, use /raw/ in Linkshare URL”

You can use a raw link directly in an IMG tag!

1 Like
9

You can also host your content as a static web site: Host a Static Website with the Uplink CLI and Linksharing Service - Storj DCS Docs (but https is not supported yet, so if you need to have https, you need to use either your own hosted proxy or self-hosted S3 Gateway in web-site mode and proxy with https support or CDN services like Fastly).