Some are just stiff englishmen, that don’t react/undestand a joke, and need explaining everything. 
2 Likes
I have no idea. I also have 1 WD Ultrastar and 1 Toshiba drive in the mix. We will see which one dies first. Problem with the Seagates is that they have a high error rate in the first months. Not a big deal. Send them in and get a replacement. Might be acceptable for a storage node. I haven’t thought about these details. For now I just keep them running as long as they make it.
2 Likes
4 Likes
@littleskunk is it a test pattern now looks like this or I need to think why it during my day go so much down?
Yes that is by design. We have to fine tune it a bit but over all we expect peak load for a few hours and less load for the rest of the day.
4 Likes
During the test, I see an increase in reported data by the satellites and the really used data on the disks.
I also see many disks, that have to handle so many uploads that done filewalkers apparently starve because they take to long to finish. Also, the storagenodes are being restarted automatically every 1-2 days (I don’t know why…).
Other people also getting this problem?
- How can I prevent the (unwanted) auto restarts of the storagenodes? They don’t seem have to do with updates or overt errors.
- Shouldn’t it be wiser to disable lazy filewalkers?
I’m willing to bet it’s an OOM (out of memory) kill by the OS.
1 Like
If so, it would be in dmesg. Which isn’t the case. Besides there’s plenty memory (and swap) left on the device.
Now do your calculations with a TTL of 1 day (or less) and that’s how I started the conversation. I just wanted to know if STORJ is aware of this “attack vector”. Then your 20TB will be written every week if you have a 1 Gbit/s connection.
2 Likes
The drive will still have other data on it. If a drive is 70% full, then only 30% of it can be re-written (assuming that 70% is long-term data).
You are describing a theoretical attack scenario where an attacker has an infinite (there is no such thing) upload capacity, is willing (and has the capital) to pay however long he/she/it needs to keep the network full (which it will not be, but let’s go with the flow) because even storing the data for 1ns still takes money, but it will actually live on the drive for as long as the expired-cleanup chore is going through the loop (so further decreasing the damage to the hardware), and all of this is happening without anyone (either storj or SNOs) noticing anything about it.
In reality (which is often far far away from theory) the tiny ToS letters to the client will come in effect: Do not abuse the network. Storj bans the attacker, we have a massive bonfire, dancing around it and chanting “so long spammer!” and the world moves on.
As I said, the sun will shine tomorrow and there is absolutely nothing anyone can do about it.
2 Likes
Technically, if an owner controls a c2/botnet, this could be possible. The black hat effectively would have no cost for pushing data in the network - cost for storj data would be active. But significant enough to warrant deterrence?
If MTBF is significantly greater above a 300TB workload per year, this likely is a concern that may be valid to protect the distributed nature of the network. Most are likely using non enterprise drive. The network will repair, but if farmers only earned enough to buy the next drive, they might quit.
I’m diamond hold’ing - so I believe in the project, but just want to highlight that to bring awareness.
That doesn’t solve your problem. Garbage collection would still be very expensive.
Better solution is to look into improving your file system. I am using ZFS with enough memory to cache the metadata in it. I also have an SSD so that this metadata cache survives a restart. With that garbage collection takes just minutes. Similar performance should be possible with ext4 + inode cache. So pick one and try it out while you can.
Is that a persistent L2ARC… or a special metadata device? I like the idea of a metadata device that’s also able to store all your small files… but I also like the idea of a L2ARC that can fail and nothing terrible happens. Decisions decisions…
Well since I’m on the defending side of that (full disclosure: I’m a Cloudflare partner, helped with the early stages of their loadbalancers, before they were offered to the public), when a significant attack happens, a lot of alarm bells start going off. The “victim” starts notifying upstream providers of the attackers. Since that amount of traffic (ie the infamous 2.5Tbps attack targetting Google) doesn’t go unnoticed long, the ISPs have a vested interest in stopping it as soon as possible. Let’s take the SEA-ME-WE3 cable for example (full disclosure again: worked for one of the 92 owners of that cable). That cable, which is the longest underwater cable in the world as far as I know, has an online capacity of 2.3Tbps. It’s very likely half of the world (Australia to UK) would notice it if it suddenly became saturated.
The attacker’s upstream see the notices on their abuse@ email (they are technically forced to act upon them, except some “attacker friendly” ISPs) and they put the IPs on monitoring. As soon as they are convinced that there is indeed an attack, they cut them off. The victim can null-route IPs on its end to keep the servers online in the meantime.
Back to storj: the incoming data per second isn’t the problem. The packets per second are. If we are talking about that much traffic, it’s more likely routers would start crashing left and right, long before any drive reliability issues come into play. This is a valid concern. Worrying about drives failing because they store TTL data isn’t.
The Storj Drives (see previous reply) have a 550TBW/year and 2.5M hours of MTBF. Taking everything to face value, that means if you write 550TB per year, that drive should last for 285 years. As you can see, those are statistical “guidelines”. It doesn’t mean that if you write 551TB in a year, the drive will fail immediately.
Besides that, everyone (on the “OMG TTL IS DEATH!” camp) seems to think that as soon as the new client signs, all existing data will be immediately converted to TTL data and no other data types will ever be uploaded. There will be normal data (ie data uploaded and deleted at a random future interval by the client), forever data (never to be deleted as long as the account is paid for), low TTL data (I’m guessing nothing less than a week), medium TTL data (30 days), and long-term TTL data (3 months). Do I have any proof of this? No, but I can confidently say that others will use the network as they have been using it so far.
You shouldn’t. You should sell your payout the moment you receive it. This isn’t an investment.
5 Likes
You are laughing, but few years ago nobody thought that it HDDs have any kind of non-trivial wear on mechanical operations except for statistical puff of smoke at the end of the bathtub curve. Now we start seeing HDD producers trying to actually quantify this wear.
Will customers negotiate between themselves whose turn it is to generate the peak load?
1 Like
I’m not seeing that, I’m seeing everything “good” have a 550TBW/year rating, from HDDs to SSDs. The rest (ie leftovers on the production line) ~300TBW/year.
They might decide to test out peak load at the same time. But we have that situation already just on a scale that storage nodes don’t care. Every now and than there was an increase in uploads for a few days. All the nodes are happy and the moment the peak load is over they start a new thread complaining about the reduced traffic. Nothing changes on that side.
4 Likes
I don’t want to entice for selling your STORJ as fast as possible. But is there some service out there which takes all the tokens getting sent to a for example specified ERC20 address, and converting them immediately to a desired token like USDT or ETH without further actions?