S3 supports public buckets and I see from here that Storj supports public links.
One difference: With S3 either the bucket owner (or download requester) pays for the downloads.
How is Storj handling this right now? Are there plans to put limits / payment on downloads, and if so will ‘requester pays’ be implemented at the same time?
Hi @davetapley ,
Welcome to the Forum!
At this time we do not do limits on download count or change the payer
Requester Pays is a fascinating feature that we don’t hear too many people ask for. For the time being, there are some alternative workflows. Most obviously, you can simply share URLs with only people you trust. Another option is to restrict a set of credentials to expire, and generate the URL with those time-bound credentials, to limit potential abuse. Some of our customers pair expiring credentials with their own payment systems.
There are problems in implementing limits that “count” “downloads”, overuse of parenthesis intended. Some downloading clients/tools deliberately open multiple connections to download different parts of a file in parallel. Also, remember that objects on the Storj network are distributed to many Storage Node Operators. Its conceivable that a network glitch causes part of a download to be lost. Coordinating re-requests of byte ranges while only charging for a fixed number of bytes would be problematic. Finally, we can’t typically assume that “public” downloaders have Storj network credentials.
I’m curious to hear more about your use case. I’d love to be able to reference a strong advocate for Requester Pays.
I’m curious to hear more about your use case.
I have a bunch of time lapse images from web cams (presently on S3).
I’d love to share them for people to download and play with, but I’m nervous about getting slammed with S3 fees.
That led me to S3s requester pays, which would work, but I wanted to see what else was available, and that led me to Storj
I can’t imagine it’d be crazy traffic, but I want to be mindful that it doesn’t get out of hand if someone decides they want to write a script to download all the images. It’s currently around 265 GB, and growing at 3.4 GB / week.
At $7/TB, if someone goes crazy and downloads them all, it’ll cost you ~$1.86. We also have bandwidth limits that are set somewhat low for new users. For instance the Free Tier has a 150 GB Egress Bandwidth limit per Project. You’ll have to reach out to bump these limits. These tools are in place to help us with capacity planning and to help new customers avoid surprises. See Usage Limits - Storj DCS.
Another advantage of Storj is that all charges can be pre-paid. The details are here [Payment Methods - Storj DCS]. There’s a learning curve to using Storj token, but it’s the “nuclear option” to avoid overspending.
I think you should sign up for the Free Tier, and try it out.