Oh you mean like the unpaid test net satellites introduced here and here.
Yeah, I wasn’t counting those on the Storj end either. All other satellites are paid as well. That still doesn’t provide evidence that the data stored on either paid satellites or paid filecoin storage providers is actual “real” data and not random noise. It just means someone is paying for it. In both cases.
I appreciate the links btw. But it seems the vast majority of data usage is slingshot
I’m a little surprised to see this add up to 37PiB, when elsewhere the total usage was reported as 33PiB (though 37PB, maybe it’s a unit confusion). However, I don’t want to nitpick, but it seems that without that one use case there isn’t much else left.
I don’t really know why this is still a point of discussion. All you have to do is read the Filecoin FAQ.
After I made a deal with a storage provider and sent my data to them, how exactly is the data supposed to be recoverable and healable if that storage provider goes down?
Automatic repair of faulted data is a feature we’ve pushed off until after the mainnet launch. For now, the way to ensure resiliency is to store your data with multiple storage providers, to gain some level of redundancy. If you want to learn more about how we are thinking about repair in the future, here are some notes (opens new window).
How do I know that my storage storage provider will not charge prohibitively high costs for data retrieval?
To avoid extortion, always ensure you store your data with a fairly decentralized set of storage providers (and note: it’s pretty difficult for a storage provider to be sure they are the only person storing a particular piece of data, especially if you encrypt the data).
Miners currently provide a ‘dumb box’ interface and will serve anyone any data they have. Maybe in the future, storage providers will offer ACLs and logins and such, but that requires that you trust the storage provider. The recommended (and safest) approach here is to encrypt data you don’t want others to see yourself before storing it.
They clearly acknowledge that file loss and even extortion are a valid concern and their response is that you can mitigate that by using multiple storage providers, basically punting the responsibility back to you as the client. If you trust a single entity there is a risk of loss, if you trust 2, there is a smaller risk of loss, but it isn’t just gone. And with no repair integrated (yet), availability can only get worse and not better.
It strikes me that there is really nothing decentralized about the data storage itself unless you make it decentralized for yourself. Though that really feels to me like “I stored my data with copies on AWS, GCP and Azure, so it’s decentralized!”. What is decentralized is the deal making processes, payment and data auditing. But data storage really isn’t unless you as a user ensure a distributed set of storage operators. And how many replications should you upload then? If you create 3 then you already have a larger expansion factor than the maximum on Storj (about 2.7) and you’re still nowhere near the 11 9’s of durability that Storj offers.
Lets just link some reference here: https://oceanstore.cs.berkeley.edu/publications/papers/pdf/erasure_iptps.pdf
And if you’re ok with first party info (section 3.4): https://www.storj.io/storjv3.pdf
I think it’s possible to build services on top of filecoin to implement erasure coding and repair. At which point that can probably be done with less stringent requirements on expansion factor, because of the higher requirements for individual nodes. But you would be reliant on some sort of orchestrator… say a satellite perhaps? At which point the remarks on Storj not being as decentralized would immediately apply to filecoin as well. I personally don’t think that is an issue, especially if there are several such services to choose from. Which is also why I hope to see more development on support for storj community satellites soon.
But getting back to the remark that there is no wide spread evidence of file loss. You (probably unintentionally) actually linked to pages yourself that refer to data loss for sling shot, when you linked to their pages about repair functionality.
Slingshot Repair is an effort dedicated to restoring the copies of Slingshot data that were lost as part of recent events on Dec 13, 2021. Unlike the Restore program, this program is targeted at the data that still has at least 1 replica in an active deal in the Filecoin network (there are ~54TiBs of such unique data).
So ~54TiBs of data that still has a replica… but what is that ominous Restore program they are refering to, for which this is apparently not the case? Let’s look at the linked doc.
Objective
The Slingshot Restore Program is an initiative to restore ~12.5PiB of Slingshot data on Filecoin as quickly as possible.
Ooooh, only 12.5PiB. Or roughly 1/3rd of all data on filecoin. No big deal.
Perhaps I’m misreading this… but that sounds highly problematic.
All of those links basically only talk about block rewards and going by your own words customers are barely paying anything into the system compared to these block rewards. Could you respond to the question of what happens when those dry up? Because these links don’t provide an answer from what I can tell.
So that’s a link to a press release about an investment fund that doesn’t mention any certifications or SLA’s at all. I did some Googling to see if I could find that anywhere else… but no. Do you have actual evidence of this? Because it kind of feels like you just linked something random and unrelated.
You didn’t post an example of clients with hundreds PiB use cases. You posted a link that shows there is one client accounting for almost 80% of all stored at 30PiB. The rest are 6PiB or lower. Where are the multiple clients with intentions of hundreds of PiB?
It is, you count all paid data on Filecoin, but add an additional requirement for Storj. All data on the trusted satellites is paid for. There is 9.6PiB of paid data on Storj. That’s what you should be comparing to by your own definition.