Please tell me it’s more than 10 petabytes.
Sure no problem. It is more than 10 petabytes.
7 Likes
I am wondering if you are aware that there might be other reasons for SNOs not adding capacity? I tried to make that clear in my post:
From what I see on Github I would have to wait at least until my nodes are on version 1.106 and then maybe get a picture what is currently really used and what’s not. And all the old deleted stuff finally cleaned out.
The unreliability of the numbers everywhere but also of the storagenode software and the satellites together with slow rollout of fixes because it is “low priority” is currently my main blocker to add anything.
It is also what @BrightSilence has said:
There is a lot on the forum about issues with the databases, space discrepancies, problems with filewalkers, garbage not deleting, nodes restarting and I see all of that on my nodes too. The way this all works with so many issues at the moment is beyond my comprehension.
So even if you would make such a forum post today detailing a signed deal I would not add and go for additional capacity or nodes.
My suggestion on adding more customers with large space requirements I made here:
Edit: Just to give you an idea what I am talking about: If I look at the average used space over all nodes this gives me only 40% of the space that all my nodes claim they are using. And the trash over all nodes is telling me its size is 1/3 of the reported average used.
And this does not add up. Running du give me complete different space than the nodes do. Some must have tons of still uncollected garbage on it while some have not correct updated used space databases as filewalkers do not finish.
So basically even if I wanted to I could not tell how much space is used altogether, how much trash is there, how much garbage has notbeen collected and on.
So it is really impossible to make a decision to add even more terabytes to this obscure situation.
Edit2: The resume feature for used space filewalker is also one of the features I need to get the numbers correct.
1 Like
Only online not full nodes are selected also using the choice of 2
1 Like
They are basically crappy USB-Sticks in a “SSD” Case 
At least his USB-Stick is from Samsung…
I think there is a significant amount of SNO who are willing to upgrade as soon as their disk are filling. But the speed of the expanding depends… mostly the subnet limitation will lead to the “one by one” approach which i already mentioned here.
So the first question from SNO side is: Do you need just contunously one node with space available at one ip or do you need multiple nodes behind the same ip for distribution? If second, how much nodes makes sense for you?
Because from the SNO perspective there is no difference in incoming traffic but only in cost of disks.
Currently i see about 1.5 to 2.5TB/day ingress per ip on my nodes, that will fill 18TB disks (16TB usable) at a rate on 8 days, maybe more if some data will be deleted…
For me i can keep up the pace and add one more node per week (currently up to 104).
The second question is: Do you provide information about the duration of the signed contract?
If the contract term was signed for lets say 12 months, the SNO run their math and could decide if it is worth it to spin up even more nodes.
Third and final question: How long in advance are we informed before the customer data arrives?
Samsung says on the cover, but my Syno identifies it as Silicon Motion Taiwan. 
I don’t know who makes the chips and the controller if the digital ID says one thing and the cover says other, if it’s Samsung or just a rebranding.
Maybe you misread Samsung for Smasnug, and it’s just a knock-off too 
2 Likes
Ideea for trash optimisation:
- the trash is just a safety backup, right?
- why do we need to backup all the pieces?
- we just need the smallest number of pieces to recreat a segment or a file or whatever.
- so let’s store only the smallest number of pieces in trash, by cleaverly choosing the nodes that store those pieces evenly, not to overload some and free up too much on others.
This should be part of the bloom filter generation process.
On 8 sticks bought in sealed retail packages from reputable store, with all the serial numbers, logos, materials etc.?
Just buy one and test it. 
It’s likely has Samsung nand chips (barely passing the QA, bottom of the barrel yield, instead of sending them to landfill they make cheap consumer thumb drives because nobody expects any reliability from them) and silicon motion controllers – because they are cheap. Again, cost drives lack of bothering with custom firmware to report “Samsung” for the final product… it’ garbage all around.
Generally, Samsung is great at manufacturing parts – nand and DDR chips mostly, but that’s it. Every single end user product made by Samsung is incoherent trash. These are no exceptions.
Yes… but some operators are deleting the trash manually or disabled it in their config, so the remaining pieces could be not enough to re-create missing pieces.
Your node should store only one piece of the segment. So 1 of the current required amount. You cannot store “some” pieces in the trash, protecting only “some” segments. It doesn’t make sense.
You can certainly do a whole lot better than that. I was careful in my message to say subnets with free space. That wouldn’t be close to 20k. And yes, I realize that the choiceofn node selection makes it more difficult, but you could calculate and average and mention that the fastest nodes would get n times more than the average. I’m betting if you do those things, your node will suddenly fall neatly within that range.
I’m happy to do my own calculation if at the time you can share the number of subnets with free space, the total amount of data expected to be uploaded and the n you settled on. But it would be better to just provide the results as some guidance. Because many will likely not know how to do that calculation or will simply not bother.
I’m not surprised. This testing patterns is significantly different from how the network used to behave and this is what testing is for, right? So rather than airing my frustration and bashing the current situation, I just stick to calmly informing Storj that this will and does impact my behavior in regards to upgrading. That’s all they really need to know to know they should fix these things. I’m sure they will, but I’m also not surprised that issues popped up under such conditions.
This is not possible with how bloom filters work. They just know your node shouldn’t have this piece but aren’t aware of which other nodes have pieces for the same segment. You can put a random file that was never even uploaded through store in the folder and the bloom filter will clean it up too. It’s not a list of files to be deleted, but it’s a cleaver filter that would match all files to keep.
3 Likes
I thought so at the beginning but it turns out any math I can come up with just gets slapped by reality. My grow rate is simply off the scale and I haven’t found a math that would explain that. Instead measuring my grow rate for a day and estimate my size after 30 days seems to be very accurate.
Sure try your luck. 6K at the moment, 20, 6-8. So just by math this gives us a range of 0.8TB for slow nodes and up to 50TB for fast nodes. And now? If you would be in my situation what kind of numbers (=promises) will you put out there? 0.8TB? That is a bit low isn’t it? But 50TB would also be unfair. But please let me know if you can come up with some numbers that will work better than mine.
So best correct answer is Everyone should calculate it by themselves.
take 5or 10 day average added space(not ingress) then you can estimate how much you will get a day and then x30 thats all.
3 Likes
You sound skeptical, but this is actually useful information for me. I think my nodes are tending towards the upper range of performance and now I can decide a little bit what I should do. Probably wouldn’t be worth it to buy more than 2x20TB per IP and I might get rid of half of my IP’s as the test has shown there are diminishing returns with the new node selection. So maybe I’ll hold on to 3 of them and buy 6 20TB HDD’s. Or start with 3 and see how fast they fill. I know buying more is not going to be worth it now.
Also, I think the lower range is technically 0TB as it is theoretically possible that you are almost never selected.
That said, I wasn’t aware the n number would be this high, so I was indeed expecting smaller ranges. Still, this information is still more valuable than “big deal signed”.
4 Likes
Bryanm, why don’t You guys just include in the contract a time for safe kick off.
some start-up clause, like “at the first 90 days of a cooperation, if the offered space turns out to be not enough at some point, we are obligated to provide needed storage in not longer than 3 months” or something. Take stress out of Your selfs.
This is a tank with hungry fishes. Seems we can hoover everything thrown at us. But the meat have to be in the tank first!
Or, if the meat is x2, x3 like good ol’ surge times? man that will be gone in no time!
1 Like
If we have over 22000 nodes, but only 6000 with free-space… then we’re not that hungry. It means most SNOs that filled a node… didn’t bring any new space online. And now we’re being paid for an additional 20PB of reserved-capacity and still waffling on expansion?
Storj needs those burst nodes, yesterday.
2 Likes
It took me a while to find this answer. I also tested several IP addresses and there latency, and the latency is pretty high at my location (Netherlands).
I tested 2 completely different type of internet connections/providers and just for fun, on my mobile phone. This probably explains why I don’t get more throughput even after I created more nodes and assigned more resources to the older nodes. The last few days it was almost stable at the 100mbit. The days before I had traffic for several hours per day that went up to 160mbit.
For all I know, everything runs fine on my side. No crashing/rebooting docker containers, no hiccups in internet connectivity (I work from home with VPN and remote sessions and had several MSTeams meetings without any problems). Even with running filewalker or GC, other nodes picked up the slack. So I’m quiet happy how stable everything is. I also expanded my old nodes to be ready for new customers, and created new one for when I need them (also handy for testing).
The only “issue” I see right now is the latency. So hopefully there are also some new big customers in the pipeline that are in Europe (even nicer, the Netherlands) that can bring my nodes on theire knees
For reference, I had 4 nodes and created 8 extra in the last 2 weeks. Right now, 6 of the 12 nodes have available capacity. The others 6 are sized as small as possible and already filled up, but are ready to put into action after I expand those.
My ping results
3 Likes
I’m hungry, and I have the room for more capacity. Unfortunately, I don’t have fiber (hopefully later this year). But with the current latency, its probably not gone matter that much.
But I hope they reconsider the current held back amount. Maybe it doesn’t matter much in the end. But somehow it does not feel right when you spin up extra disks, make new nodes, etc while in the first year a large portion of the payout will be held back. Meanwhile my power bill needs to be paid each month. I know the logic behing the held back payouts, but someone can dream… 