Updates on Test Data

Haha :joy:

But I think you got my motivations wrong. I just think of it as a fun little benchmark for my underused and unused RAIDZ2. I am still miles away of going the extra mile and insert my unused disks in two other fiber places I have. And to spend actual $ to buy new hardware, I am still lightyears away :smile:

So you see, my skepticism has not changed at all (for now) :smile:

3 Likes

Pretty sure @IsThisOn has “been onboard” for quite a while now :slight_smile:

1 Like

Nope. Once for the ZFS test, and now since June 9th.

1 Like

I stand corrected. Welcome! :hugs:

3 Likes

Thanks :partying_face:
You know what seems wild to me?
That the regulars here on the forum seem to be in some kind of party mode because of this test data.

Let’s see if I can get this straight.
There is/was:

  • No real customer growth for years
  • very little ICO money left
  • hope that storj has tons of real $ left
  • hope that storj will actually spend that $ reserves and not just call it a day
  • maybe finally found the first “real customer”
  • that customer only stores data for 30 days
  • that customer has not signed anything
  • now that there for the first time is actual load, things are breaking left and right and we fix stuff (sync) that should have been fixed years ago or should never be a problem to begin with

and you guys are partying and dreaming about buying new X20 exos to expand you node?
I envy your optimism :heart_eyes:

5 Likes

It’s OK. We can have all the optimism because we know you’re always there to piss on our bonfire and bring it back down… :wink:

EDIT: but seriously, it’s good to have dissenting voices, so thanks for that.
This is just fun for me and I am in a very enviable position where buying a drive here and there doesn’t really matter to me. I get to learn a lot of Linux and there are some lovely (and less lovely but no less interesting) and very clever people here who have taught me a lot.

So even if it folds tomorrow I’ll still bag it as a win :smiley:

9 Likes

This is how they get you, little by little, step by step… today a drive, tommorow one more drive, and at the end of the year you ask yourself: “How the hell I got 1PB of storage?” :man_facepalming:t2:
:rofl::rofl::rofl::rofl::rofl:

4 Likes

And potentially next year “What the hell do I do with 1PB of storage?” :joy:

8 Likes

I believe this gif could explain it.

1 Like

It might look like its breaks left and right, but i bet its small fraction of a percent of all nodes, it just looks BIG on forum, but if network works, that is the true indicator.
I like this site https://status.storjstats.info/
Edit: i mean this site: https://status.storj.io/
nice there are more, everything in stats, You got even more storjs team?;D
the more is the better to show off :smile:

No worry: https://www.youtube.com/watch?v=gkLCE1LI_9M
Sorry, i had to.
Lets make only good value on the topic from now on :smile:

1 Like

well, I think we have to differentiate here what I mean by things breaking left and right.
I think there is an uptime component and a node component to this.

Uptime: We had a 3 hour impact on June 11th. That is already pretty close to the promised 99.95% 4h and 20min. But that is the customers problem, not mine. I don’t care.

Nodes: Trash not being deleted, used storage report, bloom filter… the list of horror stories is long. I use the word horror stories since I don’t know how much of them is cabbage :leafy_green: nodes and how much is storjs fault. I do know that poor performance (and potential fragmentation on ZFS) due to using sync is 100% storj fault that should never have happened to begin. Heck, if some noob like me can figure out that sync makes no sense (and thous disabled it way before storj) a company with a development team should not make that rookie mistake.

I think you probably need to see it a bit more from their perspective. Also, assume they’re not all dumb.
Reliability and integrity of data was more important than performance to begin with, especially in the first iterations of the network which was not much more than a proof of concept. Sync may have come from that concern. Also, it wasn’t a problem for the levels of traffic then.
Gradually, as the network grew and the technology proves itself and is tweaked more and it’s more reliable you can start (cautiously, I think) moving toward performance optimisation.
Perhaps it is not happening as fast as you think it should, but it doesn’t seem like an unreasonable approach to me.

4 Likes

You and the team keep repeating this, but it makes no sense.
That is what the expansion factor is for.
So that is no excuse. It was a poor decision or an oversight.

I just want to reiterate that I am in no way affiliated with Storj and I am very much not a techie.

The expansion factor is for that, indeed, assuming everything works well and all nodes do what they should.
From a risk management approach it makes sense to reduce the risk of data corruption on the nodes when you’re still not entirely sure whether everything works as it should.
You are looking at this in hindsight so I think it may be biasing your judgment somewhat.

2 Likes

With that argument, you could justify anything. But in reality, security or anything else in life is always a risk-benefit analysis.

No, since I made the change before storj did.

Precisely. They went down the risk-avoidance root.

You still made the change years after storj turned on sync so you were benefiting from hindsight already.
Unless I completely failed to understand your post :slight_smile:

2 Likes

Not really. They put a security net (sync) over something that already had a security net (expansion). That made no sense.

They turned off sync and I turned it off before them.

It does if you don’t know that the expansion works as planned.

Yes, because:
1 - You already had 3 years of knowledge that sync is not necessary (they didn’t have it when they turned it on at the start)
2 - You don’t really care if data integrity is compromised. As an SNO that is not our direct problem. Therefore you have no incentive to be cautious. They do. That makes for different risk-benefit conclusions between you and them.

2 Likes

If it does not work, you have other problems than sync :smile:

No. I looked at the workload and the architectural structure. I had no knowledge advantage at all, if anything I was in a disadvantage over storj.

While totally off topic, that is what expansion factor combined with checksums is for, not sync! That is the whole point of my argument, you never needed to add sync into the mix to get integrity! This was just waste.

My understanding of synchronous vs asynchronous writes suggests that async may be marginally riskier is dodgier setups.
But as I said earlier on I am not a techie so will have to bow down to your likely superior knowledge.

1 Like