Is it time for a Storj blockchain?

On their recent build conference, Microsoft presented ‘Azure Confidential Ledger’:

I must admit, that I understand only half of it, but the overall concept seems to create a storage that makes sure data is tamper-proof, immutable and transactions are verifiable and traceable.
For that Microsoft uses a blockchain as a ledger.

Microsoft has sketched out some use cases:

Confidential Ledger appeals to use cases where critical metadata records must not be modified, including in perpetuity for regulatory compliance and archival purposes. Here are a few examples of things you can store on your Ledger:

  • Records relating to your business transactions (for example, money transfers or confidential document edits).
  • Updates to trusted assets (for example, core applications or contracts).
  • Administrative and control changes (for example, granting access permissions).
  • Operational IT and security events (for example, Azure Security Center alerts).

I am wondering, if such an offering would be technically possible and would be interesting and attract potential Storj|DCS customers (public sector, health, security and defense but also media)?
Storj Labs has already many many nodes worldwide which could run the customer blockchains on top of storage in a secure fashion and an ever increasing blockchain would result in additional storage requirements and maybe some egress traffic.

The blockchain is a bad idea to store data:

It also doesn’t solve the scale problem:

The blockchain is useful only to store something what must have a transactions by design and small enough like a financial transaction.
For anything else it’s useless, expensive and bloat usage of resources.

3 Likes

Essentially what you’re asking is: “Should Storj Labs build a second completely different product?”

At some point they may be interested in that, but the use cases of something like a storage blockchain are very specific and usually involve small amounts of data with low performance requirements. This is not something you can just tack on to what Storj already is and at this point I think Storj DCS still should get 100% of the attention from the devs, before they start to add on completely separate networks/projects. Maybe when they are big enough and have thousands of devs they can take on more.

5 Likes

I’m thinking about other possibility - to store the metadata in Storj DCS and allow an access to it for owner via own hosted satellite. It could be slower but it will resolve the problem with emergency access to your data.

I see several problems there - how to add this satellite to nodes’ whitelist, otherwise your requests would be rejected by nodes. Where to store the bootstrap nodes list? How to got an access? If it would be published via linksharing - then you also need part of the Storj DCS infrastructure to have an access to it.
Of course, you can host the linksharing too, but it will have the same authorization problem.

I think, the one of the solutions - is sign the identity of the own satellite as a usual node, then it would have a trusted certificate with limited possibilities (to allow an access only to the owner’s metadata and its data for example), it even can help in repair of that data, but it seems would require some coordination with the Storj DCS satellites for some operations like audit and repair.

I generally agree, however from the Microsoft website I am not sure if they store complete data in that blockchain or just meta data for verifying data integrity and transaction or protocol logs for immutability hence rather small data pieces.

My first question would be if a product like Microsoft is offering with the Azure Confidential Ledger does make sense at all from a technical perspective. Like from the perspective what @Alexy has said about in what areas a blockchain is useful and where not.

But secondly I would acknowledge that the big cloud providers are desperately trying to make their clouds more secure given all those data breaches. As per the Microsoft offering this includes now encryption, distribution and even zero knowledge. Selling points where Storj|DCS already shines. But then it is no longer an USP.
That means we have to acknowledge the competition will get harder, this seems inevitable.

If there is demand for that Microsoft solution, it’s worth to have a look at it. And if such a solution could potentially open a new segment of potential customers for Storj|DCS even more. As Storj|DCS offers already encryption, distribution and zero knowledge, there might be ways to offer immutability additionally. Maybe this would not require a blockchain but a ‘simple’ switch that makes a bucket immutable and undeletable when it gets created or to kick off some integrity logs that get stored alongside the files (maybe in some bucket that is read-only forever). Maybe a blockchain is required but could be done in a way that the data remains on Storj|DCS and only meta data for integrity etc. get stored in an immutable blockchain.

I don’t know if that would require a completely different product but whatever the result is, it is worth to note where the other cloud providers are heading to.

Ahh, well I stand corrected. That could definitely be an interesting addition to the project. It would still require lots of development effort, but might eventually make the entire thing even more resilient and less reliant on Storj Labs.

@jammerdan it’s a different kind of product. Those blockchains can be useful for immutable record keeping. It’s basically not usable for any kind of large data storage. The metadata suggestion @Alexey made could work as it would be limited to the relatively small metadata of segments only. Though even that may grow pretty large over time. And not being able to remove old metadata from the chain is actually a burden there.

1 Like

Is meta data for integrity and logs for record keeping that large to not fit for a blockchain?

I don’t think SNOs would mind…

This wouldn’t be stored on nodes, but would be an unpaid storage burden on anyone trying to run a full node for this blockchain.

It makes sense if you want to have a distributed transfers for small chunks of data like financial transactions from your clients. It’s something like XRP or XLM, they are not a truly distributed blockchains and there is no mining, they used only for secure transfers which is guarantee to do not have double spends and faster than any wire transfer. However both sides should agree to take it as a payment method. Using blockchain as a service can guarantee trustless for end parties.

However it’s still not usable to store any big amount of data even with encryption - the overhead makes it useless. They still lean on coordination which is counterproductive regarding speed and latency, if you even do not mention the full replication to every node.
But it could be useful for personal information if you place nodes in the specific geographic regions.

Who would run a full node without incentive?

For those azure implementations? Parties who both rely on the functioning of it.

For trustless decentralized implementations. No one. So that introduced another complication. The blockchain would need a reward system, preferably proof of stake based, with anyone staking needing a full node. Staking pools would be required to let the little guy stake. It’s be a whole thing.

1 Like

Ok, lets imagine a SNO would run a blockchain node alongside his storage node.
Would it be possible to design the incentive in a way that it depends on the storage node he is running? Like he would receive +10% on his storagenode reward? Of course paid in Storj.

In theory lots of things are possible. You could have nodes register their signed orders on the blockchain and do the entire payout cycle in smart contracts. Since orders settling isn’t part of the data operations flow, it can be done asynchronously, so the performance might not be as big a bottleneck there. But this is not a small thing. It would require massive rework, new tokens/coins, meaning new support from exchanges needed. It’s all the things Storj Labs has made pretty clear not to be interested in right now. So… maybe some day?

1 Like

Wasabi seems to follow this path of tamper proof storage:

You can achieve the same thing with specific access grant permissions. Just don’t grant it the delete permission.

2 Likes

It seems that these news are about S3 Object lock which specifically means:

Wasabi Object Lock is available in two retention modes:

  • Compliance mode
  • Governance mode

With compliance mode, a protected file or object can’t be overwritten by any user or Wasabi engineer. When an object is locked in compliance mode, its retention date can’t be shortened. Immutable objects in Compliance mode will remain immutable until the end of their retention period.

With governance mode, only users with special permission, such as the root user in the account can reduce the retention settings. This allows you to grant special permission to some users if necessary.

I understand it that - depending on the selected mode - no one as the ability to delete or alter data. This would include the account holder.

At least it seems that there is much more behind it than simply turning off delete permissions:

Locks can NOT be shortened in Compliance mode. Using Object Lock with Compliance mode prohibits everyone from deleting objects until the retention period has expired. Be careful how you use this great power – it comes with great responsibility.

It seems that also versioning must be enabled:

Buckets using Object Lock must also have Versioning enabled. This is the same process required when using AWS S3.

From what I know, Storj DCS does not have a versioning feature. I don’t know exactly how that is implemented in AWS or Wasabi, but it sounds really interesting if customer have to store multiple versions of their data instead of changing existing data.

Right so with access rights you can replicate the functionality of governance mode, but not compliance mode. Versioning is impossible to do server side due to the encryption. This has to be done client side so that different versions or increments are stored in different objects.

2 Likes