Can a SMR drive handle these requirements?

Right now, no, not really, as @BrightSilence pointed out. In a future where Storj is a major player—neither so as well. Instead, your node will need to compete with nodes set up by professionals. Again, I was writing about that before: the moment Storj as a network becomes big, Backblaze (or other companies like them) may have interest in setting up Storj nodes.

Even right now there are providers with data centres, decent peering and a large number of IP addresses, selling raw storage for quite well below 1.5 USD/TB. Not including labour costs, these providers would profit from setting up Storj nodes even now if there was enough data to bother. These guys would also be in position to specialise hardware, OS and software for Storj as well, something small SNOs aren’t prepared for.

That’s whom you should fear, not the existing centralised services.

hmmmm don’t we have something in the satellite that geographically spread out the file parts?.. so they don’t end up congregated in the same region?.. regional power failure will just take the files offline if what you described happens :face_with_monocle:

It’s limited. The only thing in place is preventing pieces for the same segment from ending up on the same /24 subnet. While this limits what data centers can do, they can obviously still get lots of /24 subnets if they’re big enough.

2 Likes

Instead, your node will need to compete with nodes set up by professionals.

I don’t think that is the case.

Nodes set up by professionals (what do you mean by that? I assume someone who invested money in hardware?) would have a hard time competing with free riders like me. Because we spend 0$ on hardware, like Storj recommends. Sure, there are some opportunity cost, because I could have sold the gifted HDD on ebay and I need some amount of additional electricity, but the costs are almost 0$.

That way my node has has a TCO of 13$ (5w added power consumption of one disk) per year for a total capacity of 8TB or a yearly TCO of 1,6$/TB. That is pretty hard to compete with.

Or you take the other road and spend actual money on hardware. But than I would argue that you can’t compete with the real pros. Open Source Storage Server: 60 Hard Drives 480TB Storage
That is a one time buying cost of 36$/TB, which I could never reach at a 480TB scale.

But because of my small scale, I actually have 0$ one time costs, which gets me a head start.
On the other hand, this brings up the question if there are enough of us to keep up with demand?

That is why I am personally have such a high interest in this project, I really wonder how it evolves. That fact that Storj in its infancy makes it so exiting :grinning:

Of course it is not black and white and there are is more nuance to that. Cloud storage is still an lucrative business and margins are good so that operators like @BrightSilence can spend money on hardware and still make a profit.

Please allow me to quote myself from a different discussion, I’ve cut the quote to the parts relevant here:

I think this reasoning is still valid under the assumption of Storj as a network becomes big, i.e. storing at least exabytes, not petty petabytes. If anything, they now offer even cheaper storage commercially, making the profit even bigger. These would be the professionals I have in mind.

1 Like

I agree with the current assessment, but I think you’ll find that when Storj gets big enough for these players to notice, market effects will start to have a big impact on the number of node operators as well. And the point @IsThisOn made then still holds. If there are plenty of operators that operate at a much lower cost than data centers, they will eventually win out. Keep in mind that Storj still heavily subsidizes node operators at the moment. This will have to be gone by the time the network is that big as well and it will all come down to supply and demand markets to set the node payouts. Or rather the amount of upload per node that determines the node payouts.
This is exactly why I don’t spend money I haven’t earned with Storj. I think we’re in the good days for node operators right now… and I expect payouts to become worse as the network grows more popular. So I spend now, so I don’t have to spend anymore later. And collect as much data as possible while the going is good.

7 Likes

For me, there is this “worth it” level. Basically, let’s say I can set up a node for free (free hardware and electricity), but I am not going to get more than $5/month for it. $5 is still profit, but it would not be “worth it” in time and effort for setting it up and looking after it. Running multiple small nodes (with the same IP) adds to the effort and time requirement.

That’s why I use a big server with RAID and a lot of space (I was out of space only for a couple of weeks or so, when I got a bad batch of new drives and had to wait for replacements and then test those replacements). I can then use the same hardware for something else - I would not use the single drives connected to separate raspberries for anything other than Storj.

If, at some point in the future, Storj reduces the payment for TB stored or uploaded, but load increases to compensate that, I think bigger nodes would become more profitable compared to small ones, well, because I could fit more drives in my server and have 50 or more TB (~35TB now, storj uses about 22), but it may be a bit difficult to set up and manage a whole farm of raspberries with single drives.

3 Likes

We have had inquiries in the past, and have been told by some data center operators that they are using Storj on their hot standby drives. They figure since they are there anyway, might as well make a little on them being nodes rather than doing nothing.

Beyond that, I don’t know details like how many IP blocks they may have or how much storage is being utilized.

I would assume though that there are node operators inside businesses as much as in homes. If you can make money off the corporate network, someone will surely do it.

2 Likes

Thanks @Toyoo for the clarification. So you mean real pros and not prosumers like us.

We have had inquiries in the past, and have been told by some data center operators that they are using Storj on their hot standby drives. They figure since they are there anyway, might as well make a little on them being nodes rather than doing nothing.

Damn that is another thought competitor for us :slight_smile:

@Pentium100 Single drives are more work, but don’t trust Storj to not fargment the hell out of my ZFS pool :grin: And I am still on my first HDD.

1 Like

Fragmentation of my pool is 29%, but this is just the free space. Storj databases are probably all over the pool, but they are not that big and can be cached. Storj files should not be fragmented (since the files are never appended or modified, only created and deleted) and while files are written to the pool all over the place, there is no way to predict which files are going to be accessed more frequently (to put them close to each other), if my node starts getting more egress I’ll probably have to add more RAM or an SSD for caching, depending on whether this is because of a few very popular files or not.

My node does not store the Storj files directly to the pool though, Storj node runs inside a VM that has a virtual disk with ext4, that virtual disk is on a zvol. I have enabled “discard=unmap” so that deleted files from the node actually free up space in the pool.

1 Like

I have always wondered the same and would wish a better geographically spread out of files, so we don’t see distributions like this

We are ready…

1 Like

It looks like chia setup, it not sute for storj, it will work very slow.

This is a combination of (at least) three factors. One is the distribution of nodes. There’s just not that many nodes outside of North America and Europe. Second is that the current approach to spread pieces by /24 blocks prefers countries with many /24 blocks, i.e. countries which operated Internet infrastructure earlier. Third is that nodes further from you will more likely lose races during uploads.

The first factor could be improved by promoting becoming a node operator more, especially as revenue should be more enticing due to usually lower costs of life in areas outside of North America and Europe.

The other two factors could be helped by changing the way nodes are spread. One proposition is here. Though, any such approach will be quite a bit more complex to implement and maintain than the current one, which has the advantage of being extremely simple and fast, putting low load on satellites.

Another idea would be for the uploader to voluntarily agree to slower uploads by reducing the number of nodes participating in a race. Then nodes further away will have more chances of winning the race.

1 Like

You just do RAID0 over all these drives and it will be fast :wink:

USB wont make it fast, lot of small files will kill all speed.

Yes, these are not storagenodes