Network Growth Update

[Alexey]

@MarviBiene

The request to join link ends on a 404 page

The link in the forum post has been updated. Also asked to make a redirect (maybe not needed).

[nerdatwork]

@Alexey

link in the forum post has been updated

I am curious to know the thought process that goes behind changing links. What drives developers to change links ? A list of reasons would suffice. I want to learn how the logic works.

[Knowledge]

We changed where our documentarion was hosted once or twice. There was a migration process but some links.didnt carry. There can also be times that documents are updated to new versions and previous versions are removed if out of date.

[nerdatwork]

I understand that and I have noticed the new change from docs.storj.io to storj.dev. In my experience when I migrated static websites to CMS it kept the links intact. Would migration always change links ? Could there be any other reasons?

[EasyRhino]

@littleskunk

In the last few days we basically jump back and forth between picking RS numbers that would give us high durability but it also gets more expensive. Or we reduce durability (should still be better than other cloud providers) and make it cheap.

Ooh what if the RS settings were selectable, so the user can pick between “durable” or “cheap”.

… actually that’s a terrible idea no one would pay for cloud storage with the idea of “well you are more likely to lose your data with this one but it’s cheaper”. Maybe options of fast vs cheap would work better.

[Alexey]

It’s now possible too, but not by the customer.

[snorkel]

It’s better to not give this option to be handled by the customer. Many don’t understand how things work, even though they think they do; they can use wrong settings and realise it too late. As long as these are hanlded by the Storj team, it also provides support and advises, and help the client make the best choise tailored to his needs.

[jammerdan]

@Bryanm

To help us expedite further growth with these larger customers, we have also made the decision to obtain a SOC2 certification for Storj as a company. Compliance is a critical requirement for many customers and a SOC2 certification would certainly help streamline adoption of our services. We are hoping to obtain a certification that covers as many of the Storj products as possible, including the public network (though we recognize that this does represent a challenging argument based on the controls required for SOC2.) Throughout this process we are committed to partner with our node operator community to successfully complete this work. If that is achieved it will dramatically increase the demand for the public network.

I can’t spare you this: For years the community has emphasized the importance of compliance and certification for customers and that it can even bring new customers. There have been numerous threads on this. For example:

@jammerdan

Technical certification or certification for legal or other specific compliance can also justify higher prices and bring more customers. Why is there no ISO, DIN, TÜV, GDPR, HIPAA, CJIS, FERPA, MPA certification until today?

Tardigrade independent 3rd party certification / audits?

Has Storjlabs got plans to perform an independent certification process or audits of Tardigrade to prove the claimed security of data so that it potentially can improve attraction to corporate customers? Data security in clouds is a big thing in the EU and especially here in Germany and if you can present a certificate from TÜV or something that everything is safe and secured even though stored on private HDDs all over the world, this could help a lot.

Has Storj considered SOC 1, SOC 2, etc?

Enterprise clients increasingly expect some degree of audit/risk management. AWS, for example, has a laundry list of reports attesting to the controls that they have in place. Has Storj considered working with an auditor to determine what can and cannot be done?

@jammerdan

Storj DCS compliance with GDPR and others have been asked many times in the past. Just recently again and has been answered as follows:

@john

What is the status on compliance/audits/certification for Storj DCS (e.g. like Wasabi: GDPR, HIPAA, CJIS, FERPA, MPA, etc. 2)

• We have an active project underway and will have an update to share in the near future

@jammerdan

Has Storj ever consulted with a professional SOC2 auditor and have checked if the public Storj DCS network in its current state cannot be considered compliant for sure?

There are even more threads in which you have been asked again and again on the status on this issue and you have been made aware of new opportunities in the data storage business that can come along with proper compliance and certification.
And I think one of your last answers or the last answer on this was:

@Bryanm

Q. Where are with compliance / certifications

• We are working on our SOC2 certification and are more mature in terms of the less complex frameworks such as HIPAA and GDPR

Now we have to read:

@Bryanm

To help us expedite further growth with these larger customers, we have also made the decision to obtain a SOC2 certification for Storj as a company. Compliance is a critical requirement for many customers and a SOC2 certification would certainly help streamline adoption of our services. We are hoping to obtain a certification that covers as many of the Storj products as possible, including the public network (though we recognize that this does represent a challenging argument based on the controls required for SOC2.)

While it sounds great that such a decision has been made, it appears that not much has been achieved until now in regards to the public network.

I strongly suggest not to focus on SOC2 only but also on the other certifications and compliance requirements that do exist. Wasabi has a nice overview of what they cover and that should give an idea what Storj also should seek for: What third-party compliances have been attained by Wasabi?. This seems to be sort of a minimum requirements for a successful cloud storage business.

Without the proper compliance and certification the big business will move their data to compliant competitors like this one that I have made Storj aware of over and over again.

New decentralized storage competitor from Germany? Impossible Cloud

https://www.impossiblecloud.com/ Can anyone figure out what tech they are using? Is it based on Storj open source code?

As it seems it was just pure luck that this 10PB-customer fits into the Select network, Storj management needs to take this as a wake-up call for the public network and make 2025 the year of compliance and certification. To attract large enterprise customers and government customers, certifications of compliance are ultimately required, probably even code reviews, code audits, audits of coding practices, review and auditing of company and management processes, and more. This can take long, should have been started years ago and it is crucial to start this process now.

Finding an auditor with experience in decentralized infrastructure could be beneficial and time saving. Seal Storage Technology’s successful SOC2 audit by Audit Peak is a good example:
I found this:

Seal Sets New Industry Standard with SOC2 Compliant Decentralized Storage

Seal Storage Technology, a leading provider of blockchain-powered cloud storage, is proud to announce the successful completion of a System and Organization Controls (SOC) 2 audit conducted by the highly respected audit firm, Audit Peak. This milestone underscores Seal’s commitment to maintaining the highest standards of data security and integrity, giving clients the confidence that their valuable data is stored with the utmost vigilance, fostering trust through transparent and reliable business practices.

As the first compliant blockchain-powered cloud storage provider, Seal recognizes the critical importance of maintaining robust data security measures. Globally, enterprises are increasingly recognizing the importance of process controls and compliance, which is why Seal made SOC2 certification a core priority. This commitment sets Seal apart as the only decentralized cloud storage provider currently certified to the stringent SOC2 standards. Seal’s dedication to upholding these rigorous data security and integrity measures underscores its pioneering role in the decentralized cloud storage field, especially in a digital landscape where data protection is paramount.

So maybe talking to Audit Peak is a starting point:

Home

[tylkomat]

Certifications cost a lot of time, effort and most importantly: money. If you don’t have a customer at your doorstep that requires it, it may not make much sense to go down that road. Better focus on the product first.

[jammerdan]

@tylkomat

If you don’t have a customer at your doorstep that requires it, it may not make much sense to go down that road. Better focus on the product first.

Storj’s significant investment in their product for this 10PB customer ultimately proved futile due to the lack of certification, which turned out to be a non-negotiable requirement. This experience highlights the importance of prioritizing certification alongside product development.
In reality, you can’t afford to wait until a customer expresses interest in your storage solution to start the certification process. Certification is a lengthy and ongoing process that requires proactive effort. Moreover, potential customers with strict regulatory requirements will not even consider your offering, let alone contact you, if your storage solution doesn’t meet their necessary standards.

[littleskunk]

@EasyRhino

actually that’s a terrible idea no one would pay for cloud storage with the idea of “well you are more likely to lose your data with this one but it’s cheaper”.

Other cloud providers are doing this all day. The cheapest tier usually is just a single datacenter and you have to pay more if you want higher durability.

[littleskunk]

@jammerdan

Storj’s significant investment in their product for this 10PB customer ultimately proved futile due to the lack of certification, which turned out to be a non-negotiable requirement. This experience highlights the importance of prioritizing certification alongside product development.

Oh yea we are clearly underworked and have time to implement everything you can think of alongside with what we are currently developing…

[ACarneiro]

No need to get defensive. Nobody says you’re underworked and nobody is having a go at developers.
But clearly someone dropped a ball during the negotiating process for this customer by not considering certification requirements (the customer seems to have forgotten about it too!).
It’s fine, no harm done. It’s income for Storj and ultimately that is A Good Thing, but there is clearly something to be learned by this.

[littleskunk]

@ACarneiro

No need to get defensive. Nobody says you’re underworked and nobody is having a go at developers.

We are getting back into “you should have done this years ago” which is pointless. Everytime that comes up I have to explain again and again that we have limited resources and can’t do everything at the same time. We have to prioritize. It doesn’t look like performance improvements was a bad priority. Sure we could be soc2 compliant by now but at what cost? Think about all the other things we wouldn’t have time for in the meantime instead of pretending there would be a magic unicorn that can get us a soc2 certificate with no additional work.

[ACarneiro]

Fair comment and I understand your frustration. I’m sure you understand ours too, though.

I do not know the makeup of the Storj team, but certification compliance doesn’t seem like something that should be taking up developer time. It’s more “legalese”, therefore it is plausible that it could have been happening at the same time as the code improvements.
But surely you must concede that there is no point in fine tuning the product if customers cannot take it because of the certification requirements?

Again, these are strategic decisions that are to be made by the people “in the know”, but in this particular case at least it seems to have partially backfired.

[littleskunk]

@ACarneiro

But surely you must concede that there is no point in fine tuning the product if customers cannot take it because of the certification requirements?

So far storj select was enough to make these customers happy. This 10 PB deal was a close call but even that one is fine with storj select and we just agreed on a timeline to work on pentests and other stuff. From that point of view it has worked out.

And what you call fine tuning are big features like object lock and to make this possible we had to implement object versioning first. We are working on these 2 features for months now. The implementation is almost finished and hopefully soon this will get us more customers. Customers that we wouldn’t get if we would postpone features like these just to get a soc2 certificate. Again there is no shortcut that would allow us to do both.

[Toyoo]

@EasyRhino

… actually that’s a terrible idea no one would pay for cloud storage with the idea of “well you are more likely to lose your data with this one but it’s cheaper”. Maybe options of fast vs cheap would work better.

A potential use case for low R/S numbers is a caching CDN: it is not actually expected to durably store content on its own, because the content can be re-fetched at any time from the primary source. I can imagine this being offered by Storj, and the only net-new thing to implement to the basic operation of the storage network would be repair workers getting an ability to download data from the primary source, as opposed to just other storage nodes.

With no GET_REPAIR costs and low R/S numbers a CDN might be quite cost-efficient for Storj.

[jammerdan]

@luco_mtb

storj promises something it doesn’t deliver.

What did they promise?

@luco_mtb

again.

How?

[Roxor]

We still seem to be getting our old regular growth rate (around 10GB per-day per-/24?). We’ll fill: it will just take longer.

[alpharabbit]

Honestly, I like that outcome. Some people here have been way to , so a little setback might be healthy.