Announcing Community Satellite Pilot Program

Product Discussions
21

A couple more comments from me!

To your point about trust and reputation, one of the design goals from the beginning of Storj v3 is that trust and reputation is largely a human thing.

There’s two sides to trust. On the customer side, Satellite operators are going to need to advertise their trustworthiness to their customers. This could be full on advertising (like ads, though without running afoul of misusing our trademark and branding), or as simple as “hey, I’m your family member, want to use this Storj OSP Satellite I set up for file storage?”. Some entities might have a head start on trust. I imagine people might consider signing up for an Internet Archive Satellite or a Mozilla Foundation Satellite before they consider signing up for Bob49183’s Satellite.

On the storage node side, community Satellites are not going to just automatically be trusted by storage nodes by default. The only Satellites a fresh storage node installation will trust are in https://www.storj.io/dcs-satellites. Satellite operators will have to persuade storage node operators to add their Satellites to their trust list, and of course, storage node operators will be eager to delete Satellites that don’t hold up their end of the deal.

This second part sounds really complicated and time consuming, so we made the concept of trusted Satellite lists (the new one at https://www.storj.io/unpaid-community-satellites). Now, instead of managing a full list of Satellites you trust as a storage node operator directly yourself, you can delegate that trust to a list operator. Anyone can run their own list of trusted Satellites. Perhaps @deathlessdd you want to start your own list of curated community Satellites, and if you do a good job at maintaining a list of healthy Satellites, storage node operators will trust you and add your trusted list. Then, Satellite operators will only have to persuade you to get added to your list, instead of every storage node operator. You might have a certain set of requirements, just as I’ve alluded to how https://www.storj.io/dcs-satellites will have really strong requirements, but https://www.storj.io/unpaid-community-satellites doesn’t currently. I expect we will start more trust lists as we get more familiar with community Satellites together. An obvious one to start would be one for paid-community-satellites, :slight_smile:

In summary: here are all the different actors in the Storj community:

  • Storage node operators - we all love them!
  • Satellite operators - new! so far, Storj Labs has been the only Satellite operator, which does call into question the network’s decentralization.
  • Edge service operators - we’ll add more assistance during a later phase, but anyone can set these up too.
  • Trust list operators - new! so far, we’ve been the only one, but anyone who can serve a text file over HTTPS can become one. This is not the same as a Satellite operator! A trust list operator is largely going to be operating with human language more than code and protocols, but it’s still just as important of a job!

Yep, we’re early. We don’t have documentation for this yet! We’re looking for some development-minded early adopters to help flesh this out actually. Here is what we have so far:

First, we have our development instructions of how to run a test network: docs/Test-network.md at a330d6526c0857363a903bebbb04cf7344f9648b · storj/docs · GitHub Of course, this includes the Satellite, so this is definitely a start.

Second, we started development on a wizard for Satellite setup here: https://review.dev.storj.io/c/storj/storj/+/9523.

Volunteers wanted!

I don’t know if my comments about trust covered this sufficiently for you, but the fact that not every Satellite is automatically trusted by storage nodes, and storage node operators must explicitly add a Satellite to their trust list (perhaps indirectly through a curated list) is a huge barrier to entry. I actually feel pretty good about this.

One thing we’ve discussed is having a foundation (that includes non-Storj employees) run one of the community Satellite trust lists I mentioned earlier. This could be the group that “approves” Satellites for inclusion on a specific community Satellite trust list.

We’ve discussed having Satellite operators put collateral to a trust list operator in escrow of some kind to cover this. Certainly different trust list operators could handle this differently.

I mentioned limits in the launch announcement, and you’re right, I probably should have said something about this. Storage node operators should be extremely cautious about adding brand new Satellites with no history to their trust list. Trust must be earned, and hopefully that is something you get from a well known trusted list. One of the code changes we want to make is that Storage Nodes will limit their exposure to new Satellites, kind of like how Satellites limit their exposure to new storage nodes during the vetting process.

We’ve thought very hard about this too. At this point we believe the benefits outweigh the risks, which is a similar decision that many other open source companies have concluded, too.

This is why it’s really important the uplink is open source! If you get an uplink from a well known source that uses end to end encryption, even a malicious Satellite operator can’t get your data.

Agreed, right now all we have is the configuration file of the trusted sources. I do think the trusted sources list is actually a key innovation to making this work, but I also agree that a storage node operator should be able to click a button and kick a poorly behaving Satellite to the curb directly. Kicking poorly behaving Satellites out of trust lists is probably even better, but we do want to empower SNOs directly.

You’re right, this is the start of the next phase of our experiment! Community Satellites are kind of like where we were with v3 storage nodes back in January 2019. It’s a bit of the wild west, but I think it’s best to involve the community in this step.

One final note to point out to everyone is that we actually covered some of this in our whitepaper 5 years ago! Make sure to check out section 4.21 in https://www.storj.io/storjv3.pdf. The thing that has changed is that “Tardigrade” is now called “Storj DCS” and “Storj” is now called “Storj OSP”.

Did any of this help?

8 Likes
22

Yeah, that helps. And trust lists would take some of the burden off node operators. But it’s all the more important that the node does some cleanup automatically if satellites are removed from trust lists, because they won’t be aware if and when this happens. Perhaps after a 7 day delay or something, as I can understand you want to protect against accidental removal.

There is also some risk to satellite operators with this approach, as they can become completely reliant of these trust lists if there is a single or a few popular ones. Being removed from the most popular trust lists could instantly kill a satellite as they lose almost all their nodes.

Given that you’ve confirmed the new list is indeed going to be a bit of a wild west, I urge you to alter this request.

At this point, without the proper tools available, it really doesn’t seem wise to use your production node to join this test. Instead the request should be to spin up a separate node for this test to limit the impact on good functioning nodes. If aim going to join in on this, that’s what I’ll be doing for sure. Otherwise node operators might end up with data from defunct satellites taking up space, many trust errors in the logs and no easy way to clean that mess up. This isn’t production ready in this phase, which is fine, but it then shouldn’t be run on production nodes.

This actually mirrors advise for testnet nodes mentioned here: Stefan Testnet Satellite

2 Likes
23

Do you feel this way even with the below caveat?

We don’t intend to add resource-consuming Satellites to this unpaid Community Satellite list until storage nodes have support for these management features.

1 Like
24

What is the Edge service operator is?
some computing power operations? data repair? audit?

25

gateway.storjshare.io (our globally hosted S3 compatible gateway), link.storjshare.io (for content and DNS hosting), and auth.storjshare.io (a service that powers the two). Running these well are currently taking us more energy than running a Satellite, believe it or not, which is why we’re focusing on community Satellites first.

1 Like
26

Yes, at the moment nodes already suffer from error messages in the logs because of the removed stefan-benten satellite. And blobs will never get cleaned up if a satellite is being removed from the list for some reason. If you don’t separate these things, you’re going to encourage node operators to do their own manual clean up in the blobs folders and risk them making mistakes and effecting production satellites. There may also be unintended side effects. For example, I don’t think the dashboard is currently able to deal with satellites with different payout values, so payout info may display incorrect totals etc. Mixing these things with production nodes before that is settled just seems unwise.

4 Likes
27

Okay fair enough, I’ll make a change.

4 Likes
28

Will it be also decentrolized? for example with help of load balanser to lot of several gateway nodes? or link sharing nodes?

29

Thanks for the change.
Is there a reason you’d want to keep the original trust list on these experimental nodes? Otherwise it might be better to leave it off to keep full separation.

30

You should offer some extended options for comunity sats operators; for example:

  • if a sat operator wants to promote a high quality high availability storage network, he should have the option to filter out or filter in storagenodes based on performance, availability etc. He can than charge more from customers and pay more to SNOs. Like a premium tire. This is an ideea from other threads, and I think it’s a good one.
    Storj DCS remains a base service accesible for anyone, but comunity sats can focus in developing netorks with some specifics… like premium networks, low cost networks, localised to an geographic area etc
2 Likes
31

It’s open source code. There is nothing stopping satellite operators to make such adjustments to the code.

2 Likes
32

I wonder how can a sat build reputation if beeing new, no SNOs accept it to offer space, and no customers use it?

2 Likes
33

They pay node owners to take their data. Not now of course. But down the road the economics will matter. People had to setup nodes and trust Storj Labs to pay them at one time when they were new. Incentives helped in that regard.

1 Like
34

As a node operator I would like to be able to have different payout addresses for different satellite operators.

I also think simple satellite trust list won’t suffice. I would like to see and compare hard metric facts like number of nodes, geographically distribution, data flow, lost data, uptime, payment record etc.
I like what Sia is publishing on their nodes: siascan
They run this public service and publish such metrics and I think something like that would offer valuable information for both customers and SNOs.

Which is also a risk for the customers and their data. A single entity controlling access to the SNOs also does not sound much like decentralization. It is more another single point of failure.

In the long run maybe the entire software development could or even should be handed over by such an independent foundation and not be run by the company with its own interests?

That part would be much easier if folder names on the storage node would have clear names and would be grouped by satellite or operator. I am not so much for full automatic or time based removal but for removal initiated by the SNO (via dashboard or something).

2 Likes
35

To be fair, there could be many curated lists and satellites could also advertise nodes to list them separately to prevent relying on a lists. But yeah, it’s something to consider. And it might be something the satellite operator needs to have stats on.

But then the node operator at least needs to be notified if a satellite has been removed of the lists they use, so they can initiate clean up.

36

Yes for sure but it will be hard if there is one big popular list to compete with. Like everywhere else, where you have one big player to dominate the space.

Also for sure. I think that would be the best way to trigger an email notification or display on dashboard and then the node operator can initiate a clean up.

1 Like
37

Perhaps there should be a feature like satellite pinning. So node operators won’t untrust the satellite automatically if a list drops it. The satellite operator could promote that. Or maybe if a list drops a satellite it will just warn nodes that it is dropped, but not immediately make the satellite untrusted. But yeah, it seems like something that needs to be solved.

1 Like
38

Satallite statistics are very important, I suggest building an statistics website. For that to happen we need Storj Labs approving to include a little public API endpoint on satellites.

1 Like
39

I feel some more thought should be put into the storagenode software itself on where it could fail when handling multiple independently operated satellites. Payment/dashboard was already mentioned. For example this:

Multiple satellites running multiple filewalker/deletes/hammering audits at any time sounds not appealing. I certainly would not want to fail audits on an important satellite while another satellite operator had for example halted GC and runs massive delete requests after that.

While I like the idea of community satellites I have to admit that it’s currently beyond my imagination why anyone would seriously run his own satellite and even more as customer why to trust a 3rd party satellite operator with my data. If serious enterprises don’t trust Storj enough to put their data on Storj DCS, why should they trust somebody else or who should they trust?

Edit: Oh and what came into my mind: What will the storagenode update process look like with independently operated satellites?

40

Yeah. While Storj is supposedly decentralized, it is actually centralized around the satellite. Satellite goes away, so does the data and there are multiple reasons why a satellite may go away. I think that this can be compared to torrents (and private trackers) quite well - while I may seed files from multiple trackers, if a tracker goes away, then that file is no longer accessible to others. Torrents are decentralized in a sense that the same file could have been uploaded to multiple trackers, so Storj is more centralized than torrents (unless the customer wants to pay multiple satellites and upload multiple copies of his data).

Now, if there was a way to move files to a different satellite (without downloading and uploading) or something, it would make the system more decentralized and it would also make the community satellites make some sense.

2 Likes