Can Storj be deployed privately?

Can Storj be deployed privately?

1 Like

Hello @yuanqidan,
Welcome to the forum!

Yes, but it will be isolated from Storj DCS:

So, if you can share what do you want to do, we could help better.
If you do not require to pay to your storagenodes and have a trusted environment, it perhaps better to run specialized storage cluster instead.

Ooohhh great, I had the same question on my mind.

My scenario would be to power a private distributed cloud storge (no payment) that “consumers” interface with via S3 API.
all participating machines would all be trusted.

Given my compute strategy heavily relies on a multi-cloud + on-premise mix, the requirement for high-bandwidth, low-latency connections disqualifies most “specialized storage cluster” softwares.

I was wondering if StorJ’s concepts might fill the problem space better?

I’m wondering if/ how hard it is to disable the payment logic for a private setup?

“Low latency” may be where you hit a snag.
Whilst bandwidth is almost limitless with Storj, latency and TTFB tend to be its Achilles heel.

I’m finding time to last byte to be a major issue on the Storj service. I get great bandwidth for a while, but invariably the last bit of each file takes seconds or longer.

It’s probably not the last bytes of the file, just the last bit to download.

Or when pulling a ton of small files with heavy parallelism most come in very quickly but a few just hang or barely trickle in.

Not a big deal for some use cases, but a dealbreaker for anything even vaguely real-time, at least for me.

for honestly, it’s hard to disable the accounting, but you may never invoke the payment itself.
For the private usage I still believe that something like Ceph + MinIO could be better, however you may also test both solutions as well.
This:

The latency itself can be low with higher parallelism, but TTFB is not improved yet, however, we are working on it in multiple directions, for example

yes, it’s likely the slowest node in the selection.

Can storj test networks be deployed to production environments for storage? @Alexey

Why does the image always fail to pull when deploying with docker-compose?

What’s error?
It’s working for me, just tested.

it’s possible, but you need to reconfigure it to use public addresses instead of localhost.
Test networks are not designed to be a public, so they useful for tests, not for production.

It may be a network issue.
Can the image be uploaded to Docker Hub or could you please provide a tar file?

I would suggest to fix the network issue before continue, and unblock img.dev.storj.io.
I tried several options, but you must have an access to this repo to be able to use storj-up.
Otherwise your other option is only storj-sim: Test network · storj/storj Wiki · GitHub

Thanks for the info. I do have a good experience w/ StorJ + restic mounts w/ a local file cache on write-heavy loads.

For most of my usages the data access pattern behaves like a sliding window over time, older data is less frequently access and TTFB less relevant.
Although I havent tested, a restic based read/ write cache might provide a low enough latency :thinking:

Good point :+1: