I just realised how expensive object storage is also Tardigrade

Developer Discussions DCS
62

@super3 firstly I think it’s great you even asked the community for our thoughts!

I suppose we should start with “where are Storj now?” It’s still early days, how are promotions and new customers going? Have we on boarded any big fish?

If things are looking good and it’s just a case of waiting for the traffic organically then that’s fine, but if new customers aren’t where you expected them to be by now, I agree with some of the other comments that perhaps a smaller piece of the pie is better than no pie at all. I’d personally be happy to accept a smaller payout for egress (assuming Storj does the same!) on the assumption it would increase egress usage.

Ofcourse it’s easy for me to say that because I’m offering shed loads of storage and bandwidth. The smaller SNOs may disagree and find running a small node even less profitable.

That said, I think most of us (including me who gets relatively large payouts) are disappointed with the payouts mainly because our nodes are being significantly under-utilised.

1 Like
63

Using Tardigrade with something like Nextcloud or Owncloud has a lot of pros, but also a lot of cons in terms of costs. Here is my use case:

  • a Single Nextcloud instance hosting 2TB of files
  • these files are mainly photos, documents and some mid-sized files (300Mb each)
  • they represent 200K objects
  • there are 40 users in this instance
  • there are approximately 8TB of egress each month, mainly by desktop syncing.

Just do the math (cots of: storage + egress + objects).

The problem here is mainly the cost is unpredictable in this case. We just can’t prevent users post a link to a 3GB public file on Twitter and get it downloaded hundreds of times. Or, even worst, tell the users to not upload small files as the cost of each small object grows the bill and to ZIP them before upload…

We would prefer pay each GB a higher price (erasure, encryption, distribution and metadata justifies it), but at least, one can easily estimate his bill by the end of the month.

Otherwise, only big companies can deal with it :frowning:

2 Likes
64

For your information, after using Tardigrade for a half year, I would like to revisit my post above.

Basically I have replaced Nextcloud with Syncthing.

Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers in real time, safely protected from prying eyes. Your data is your data alone and you deserve to choose where it is stored, whether it is shared with some third party, and how it’s transmitted over the internet.

I noticed there was no need to run a VPS server to sync data among PC, laptop, and smartphone. You just have to install Syncthing to them, sync data among these peers, directly.

On Syncthing it is also possible to sync data with your friends by sharing a folder. Think of it as Dropbox / Nextcloud without a central server.

In order to back up data to Tardigrade, you can set tasks on Duplicati. Duplicati has a native implementation of Tardigrade. For more information, see: https://documentation.tardigrade.io/how-tos/backup-with-duplicati

Duplicati is a backup tool. It can group, dedupe, and compress small files into bigger blocks. It is a great tool for reducing the costs of cold storage. It also supports versioning.

You can set up Duplicati to run backup tasks hourly, daily, weekly, and so on. You may also run tasks minutely, but I am not quite sure if it supports to watch for changes to run tasks automatically.

On Duplicati, data is downloaded from Tardigrade to verify backup (and also when you restore a file, of course). Since data is synced among peers first, you can keep egress consumption minimized, and it should be much cheaper than syncing among Nextcloud clients via Tardigrade.

Also, this way every data is distributed. There is no central point of hacking. This is nice too, right?

Cheers,

3 Likes
65

great to hear your update @Andisers - im very happy that you set up Duplicati and reported back on it!

2 Likes
66

maybe charging only for used storage size while engress bandwidth to be removed from the equation … for actual test case… i mean … charging egress is basically to pay the SNO for operation the storage nodes. where SNOs love it … but not consumers…

a little change in the business model may help tho … lets think about it… maybe turning the egress to ingress may not be a bad idea too… as long the files are uploaded … you already paid both storage + upload fees … i dont restrict you from further downloading of the files… that could be a better way maybe…

67

I myself use Backblaze’s B2 for hot object storage. They have a partnership with Cloudflare which provides users with free egress if setup properly and only charges $0.005/GB/MO for storage. There are transaction fees but those are very minimal compared to the free egress you get for piping your egress through cloudflare. I think Storj needs to be a little bit more competitive to capture more customers.

3 Likes