I answered you’re survey where you are confusing bandwidth and ISP caps.
You are missing the point home user have a contract with the ISP to use the net for personal use as much as they want, like watching 10 4K Netflix stream simultaneously is fine, they get pay by Netflix for the bandwidth usage but if you read the fine prints in the contract you signed with you ISP, you can’t resell this bandwidth. In the current state you have a few thousands node worldwide with very small bandwidth usage so you’re still under the radar, if your system growth up you’ll become noticeable and the ISP’s will require Storj to pay for the bandwidth usage or the users to switch to a commercial license.
1 Like
@lex-1 you mentioned you are in Europe, so you might be interested to read Regulation (EU) 2015/2120: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32015R2120
I haven’t read it thoroughly yet (I will do later), but a quick glance shows that ISPs cannot discriminate how users use their bandwidth.
2 Likes
This is intresting thread and i will continue reading it.
I havnt read anything from my isp but having used the same ISP for over 15 years with different usage of my bandwidth and have never had any problems or anything regarding my usage
very easy to block the ports or block your account.
I’d like to point out that one could use secure tunneling from a home run node through to an Internet facing domain… This would shield the ports used as well as the traffic itself from ISP analysis. The only thing the ISP would see is a secure connection between your home network and an Internet host. They would have no ability to block your traffic based on ports… and for all they know, you are legitimately using the tunnel for something like a personal VPN.
I’d like to point that… the difference between a personal copy of A/V copyrighted material and a commercial copy is whether or not the copy is being used for a Public Performance … there is wide disagreement about the legal meaning of this term. It’s possible that making a copy and adding to your home network’s streaming service that runs locally on your own private LAN is actually a “Public” Performance… and therefore in violation of copyright law.
My concern is about the users cost not the nodes income (if the nodes are loosing money it just mean we have too many of them) for the customers the monthly storage fee is justify although if the nodes are paid $1.5/TB with a small 1.5 redundancy scheme and a small gross margin to Storj to recoup at least its development cost the customer has to be charged at $3/TB/month or $36/TB/year we are already more expensive than a spare HDD if you add a $20/TB download fee, this solution is not economical for a residential customer wanting to backup its HDD off-site and is still very expensive for file sharing when you can found plenty of provider offering the service for free.
No problem and thanks you for your detail answer. The distributed data storage is a great idea, we need to make it practical and economically viable.
I seriously doubt that even a cash starve startup would be willing for saving a few bucks to risk loosing $thousands of development cost, using an unproven supplier.
As far as outerfoming the big guys on speed, I have a doubt, when some of the nodes are potentially running at <1Mbs on ADSL links and raspberry PI servers.
This is not the target market for Storj node use. The target market is someone setting up storage for a web domain. Internet accessible storage is much more expensive than locally connected devices. Most web domains and services rent storage space via either IaaS servers or Resource services like Amazon EC2.
I dont think you understand fully how it works even if theres some people on <1Mbs they would only be storing a 2MB file for a customer how hard is it to upload a single 2MB file, Its also a team effort for all the nodes running not just a single node by itself.
I think I understand how it works. Its based on reed-salmon like in raid 5, it require from the node a 99,7% server reliability, like a data center with redundant processors UPS, generators and dual fibers the data centers achieve a 99.99% up-time with redundant data center and speed with Tbs fibers and you think that residential nodes with no redundancy and with a million time slower link are going to compete? Sure its a collective effort but Storj is accepting ADSL nodes it doesn’t even understand the difference between bandwidth and usage caps, if you’re a user and your data has end-up on ADSL nodes it will take weeks to recover.
Be reasonable 99.7% up-time is unrealistic for a residential node, for a few months why not, until a CPU/disk crash power/ISP failure, thunderstorm, hurricane, tornado, earth quake take your link down for a few days. Storj doesn’t check nodes speed and location, storing redundant data within 20-50 miles is useless in case of hurricane which are pretty common on the east coasts Atlantic or Pacific. They still have a lot of work to do to make it better than the big guys.
Still Storj has an enormous potential market, they are completely ignoring, the video steamers. They need a local presence at the ISP level to reduce ISP distribution fees, they don’t care if data are lost as they have multiples backup of the videos but with a $20+/TB upload fee forget about them, who care they are only 50%+ of the internet traffic.
Well apparently you don’t fully understand how STORJ works. They already put as much effort as possible into distributing data geographically with an IP-filter and will continue to do so, exactly because of these reasons.
if you’re a user and your data has end-up on ADSL nodes it will take weeks to recover.
Since you perfectly explained how data is divided, you still fail to understand what it means. If all my data ends up on ADSL nodes, I would still recover it very quickly. If 80 ADSL nodes store my data and I download it and each node has an upload of only 80kB/s, I would still get 6.4MB/s download. This is of course not lightning fast and would be divided with other users that may download pieces from the same node. However it still doesn’t make downloading my data take weeks.
I mostly agree on the rest though, uptime requirements are insane but they know it and I guess that is part of why offline disqualification got suspended and why there are new concept drafts.
If their system fails, they will recognize it and change it. After all, they want to make some money, not see their project fail, so I wouldn’t worry about it too much.
I am curious about the price myself. Having a cheap backup solution for home owners would be nice but if it is too expensive, most will likely just use other solutions like sia.
I might not fully understand everything about STORJ policy but I still have some notions of propagation, further away you are, worst is the latency and bandwidth.
I don’t understand your 80 nodes dispersion, each node can accept 20 Mbs but are limited to <1Mbs upload so why would the data be distributed on 80 nodes? Still in your scheme it would still take 43 hours to recover a TB I think AWE can do a little bet better.
I’m glad we agree on the remaining.
Cheap backup solution goes from an HDD stored in your office drawer to an HDD online at a friend location with reciprocal capability.
What do thing about STORJ been some kind of a streamer network proxy?
If you read the whitepaper, you see that data is split into multiple pieces (~80, idk exactly anymore) which get uploaded to different nodes. Therefore download speed for your data file is the sum of the upload speed of each node (assuming that the node only serves your data at the moment).
Therefore even on slow ADSL nodes you get ~ 80 * 80kB/s = 6.4MB/s.
This is more like a worst case and of course AWS can do better. This is also just for one data file. Your next data file could be stored on completely different nodes so if you try to get that one at the same time, your download speed is double.
But I doubt that most SNOs are on slow ADSL so it shouldn’t be that bad.
According to a blog post by STORJ before going into beta, STORJ is already at par with AWS in terms of speed: https://storj.io/blog/2019/08/the-role-of-qualification-gates-in-getting-to-beta-and-beyond/
The inflation factor of each piece is initially over 2.7x but could drop close to 1.2x at which point repair would be triggered. However, that has so far not been needed since the last wipe. I think on average the expansion factor right now isin the 2-2.5x range.
This would be a very competitive price for consumers. Just go and have a look. Most will charge $10/TB, some of the cheapest ones at perhaps $5/TB. And they make you pay for all of it even if you don’t use it all, that makes a big difference. Even though storj is not aimed at consumers, one could easily build a consumer solution on top of it and charge the customer the same value they pay and just make money on the unused space. You’d be undercutting everyone. You can’t compare a spare harddisk at home with absolutely no redundancy with a cloud service, obviously that’s cheaper, but it also is much less reliable and useful.
The uptime requirement is 99.3% for nodes right now. However the network can deal with much worse uptimes, which is being shown right now as nodes are currently not paused for breaking the uptime requirement, yet all qualification gates for beta (including reliability) have been met. There is some stretch in those numbers and this is still being tuned.
What makes you say they are ignoring them? You can already stream video from the storj network. I’ve tried it with 4K streams even. There is even a part in the whitepaper where it outlines how the storj network could be used as a CDN by creating many more redundant pieces all across the world and having local viewers access local nodes to stream from.
I’ve skipped over a lot of the posts about speed or reliability because I can summarize my response to all of them. You obviously see problems with this bold projects because you haven’t read up on how those problems were solved. For speed, downloads for individual pieces are started from 35 nodes at the same time, but only 29 pieces are needed to reconstruct the piece, so only the fastest 29 have to finish. This filters out slower nodes from bogging down the downloads. If you have to download large amounts, there is no reason you can’t be downloading multiple pieces at the same time, allowing you to download from thousands of nodes simultaneously. You will have no problem at all saturating your own connection if you go about it this way.
As for reliability, not a single piece was ever lost so far. There are extensive calculations that predict high file retention rates and there is extensive testing to confirm it. Of course you always take a risk when you store data on something new, but it’s no different from any other startup you choose to rely on.
Most of the issues you point out have simply been solved. I understand that reading the white paper may be a large time investment, but I assure you it’s a good read. But before continuing this discussion any further, please at least read this blog post about qualification gates that @kevink posted as well.
It will show you that nearly all your concerns have been dealt with.
Is it a certainty that this will work out perfectly? Of course not. New ideas and startups always have challenges, but we should be focusing on the ones that are actually still a problem. I love to discuss concerns around possible issues and I’ve raised some myself in the past, but if concerns are based on the lack of knowledge about the network, the discussion becomes a waste of time.
2 Likes
I’d also like to point out that each uploaded piece is split into 130 pieces, then the 80 of those to upload the fastest are kept (the remaining 50 uploads are canceled) and only 29 of those are required to reassemble the original piece. So yes, some of the nodes storing your data are going to be slow ADSL links, however:
- If they can’t download a piece fast enough during the upload process they will be one of the 50 aborted uploads and will lose out to the 80 other nodes who handled the upload faster.
- When calculating the speed of downloading a file from the network, since only 29 pieces are required, we’re not talking about the average speed of all 80 nodes that hold part of the file, we’re talking about the slowest speed of the 29 nodes that can upload their share the fastest. ADSL nodes may not win here, either, causing the piece download from them to be aborted.
Storj accepts ADSL nodes because an ADSL node geographically close to you may be substantially faster to pull a piece from than a 10Gb redundant fiber data center on the other side of the planet.
4 Likes
Great, everything is written in your white paper and you don’t need any users inputs. So I won’t post anymore stupids questions, I’ll just keep using the system while it pay the electric bill.
I’ll be you, I would revise the Storage Node Earnings Estimator https://storj.io/storage-node-estimator/ before someone start a class action suit for false advertisement, if I follow it, I should be making $67 the first month while I’m making $4 after 4 months. With parameter like 200Mb/s max download speed it look like a Nigerian scam operation, we are in 2019 1 Gbps is standard 10 Gbps is optional.
Still I like your idea of a distributed storage system, I’m not here to make a profit. Comparing yourself to highly secure systems like AWS or Google is ridiculous, you might compare maybe with Hubic 10€/month for 10TB in secure data centers with unlimited upload/download and in a few years when you have proven that your data are as safe as the ones stored in reputable data centers you could try to compete with the big guys.
@BrightSilence is just another SNO like you so you should point all your frustration towards Storj.
I really really admire the confidence you have in claiming this. Here’s some insight.
I would like you to post some proof of your claim. Storj’s code is on github feel free to read it and show the loopholes.