Production readyness?

I just tested the uplink with the new version and I have to say, I am a bit confused.
The upload speed shown in the cli barely gets to 5MB/s (which is my max) and often drops to below 1MB/s, sometimes even below 100kB/s although my router and the host itself show a constant stream of 5MB/s.

However the bad part about my experience: The first upload already failed after a few minutes because a piece could not be uploaded to more than 80 nodes… This can NOT happen! How am I supposed to take tardigrade seriously if uploading my data fails because either the satellite or the uplink itself are incapabale of distributing the pieces correctly?

64.03 MiB / 3.39 GiB [—>_________________________________________________________________________________________________________________________________________________________________] 1.85% 1.44 MiB p/s
2020-01-24T21:50:49.932+0100 FATAL Unrecoverable error {“error”: “segment error: ecclient error: successful puts (75) less than success threshold (80); segment error: ecclient error: successful puts (75) less than success threshold (80)”, “errorVerbose”: “group:\n— segment error: ecclient error: successful puts (75) less than success threshold (80)\n\tstorj.io/uplink/ecclient.(*ecClient).Put:177\n\tstorj.io/uplink/storage/segments.(*segmentStore).Put:67\n\tstorj.io/uplink/storage/streams.(*streamStore).upload:236\n\tstorj.io/uplink/storage/streams.(*streamStore).Put:92\n\tstorj.io/uplink/storage/streams.(*shimStore).Put:49\n\tstorj.io/uplink/stream.NewUpload.func1:53\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n— segment error: ecclient error: successful puts (75) less than success threshold (80)\n\tstorj.io/uplink/ecclient.(*ecClient).Put:177\n\tstorj.io/uplink/storage/segments.(*segmentStore).Put:67\n\tstorj.io/uplink/storage/streams.(*streamStore).upload:236\n\tstorj.io/uplink/storage/streams.(*streamStore).Put:92\n\tstorj.io/uplink/storage/streams.(*shimStore).Put:49\n\tstorj.io/uplink/stream.NewUpload.func1:53\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}

3 Likes

I now also tested the download speed. I don’t remember what the goal was but personally I was a bit sad because I only got 25MB/s tops but could download with 100MB/s.
However CPU load was quite high, over 100% in htop on a Ryzen 2200 with over 3GHz per core.

2 Likes

I may be wrong but i think the CPU % in htop is confusing because each core has 100% so if you have 8 cores i think that gives you a max of 800%

(anyone please correct me if im wrong!)

What @kevink wrote about is what I meant that Storj V3 would turn into Storj V2.2
But I myself can’t even verify it, try it. I and thousands SNO need to wait for invites or production. This is ridiculous.
Dear StorjLabs, you should not release a product for all people when you are not sure that it satisfies at least a hundreds of people who test it. You can, but the result is known. We all already saw it and I don’t like it.

Мy advice for program “minimum”

  1. Delay production
  2. Send invites for all SNO
  3. Give 1TB store and 1TB/m download for all period up to production for all SNO
  4. Collect feedback, fix and polish troubles.

If it is cost expensive, make a unpaid satellite for this. I am sure that there will be many farmers for testing even if it will be unpaid.

More over i suggest create an persistent unpaid Tardigrade satellite for storage space donations for new physical clients. I am sure that all of the “industrial” SNOs (50-100TB+) will allocate several free TB for promos. And indeed, instead of garbage test data, we can store promo traffic.

8 Likes

Yes 100% means one core is fully used. Since I have 8 cores 800% would be full cpu load.

hi @Krey
Thanks for your thoughts. I’m forwarding them to my coworkers. The waitlist will not be a problem for too much longer. I know its been anticipated by people for a long time.

Question for the other posters here: I see that we have a bunch of people who clicked the “heart” icon on OP. Im guessing this means you are interested in the topic, and have some similar values.

I have been speaking with one of our engineers about the getting more direct feedback on testing… and his answer was enthusuastic: "Please let him test it very throughout and post all problems "

Would some of you also be interested in gaining access and helping to test &/or give feedback? Please let me know. YOu would have to agree to also tell us about any problems you spot.

@krey I am going to DM you in a few minutes

  • Jocelyn
1 Like

Hello Jocelyn,
I can test this for you if you have some guidance on what to test exactly and what kind of feedback you like.
If do we post the issues on github?
In earlier stage I did some stuff with Atikh for a diary on running a storagenode, but I guess this will be more intensive testing…

I also see some discussion in this thread about if testing is done the right way or even at all, I think it’s needless to see testing is probably the most underestimated part of all projects. And then again it might be the most important part of your project, although it takes time to do this (at least 50-60%).
At another project I’m working on we also have much more intensive testing with all kind of scenario’s worked out. We are that far in this, that if a new release isn’t signed of by a group of testers, the release cannot pass through!

Please let me know

Also wold like testing it.

I am working on .net platforms.
Agree give feedback as moch as i get.
First i need to over my storagenode log analizer, similar with success_rate written by @Alexey
but want to read som error finding features.

hi everybody

This topic is important to us. therefore I have created an impromptu group that any member can request to join. In this group you can request an invite - you just have to promise to test the platform and give us your honest and constructive feedback . After you join the group, you;ll see a new, hidden category underneath “Ideas and Suggestions” area of the forum. It will also add a badge to your forum profile Ive added a few folks who clicked like on the original post already.

Sound like a fair deal? Want in?

Go to https://forum.storj.io/g/constcrit to get started.

ETA: im marking this post as the ‘solution’. but that is only to nail it to the top of the page I want to ensure that everyone see the link for this feedback group invite

1 Like

A few production input from me, specifically for tardigrade:

  • It does not mention a “free trial” anywhere in it. It mentions “your credit card will not be charged until production” somewhere before adding a credit card, but this phrasing is ambiguous and nowhere as powerful as “Thank you for joining tardigrade’s beta. This decentralized storage service is completely free of charge until production release, please use us well”. It is also best to not ask for credit card at all during free trial, just let them try it, no strings attached. Plastering every page with “I want your CC right now” is not a good way to get people’s trust and business.

  • Tardigrade landing page is not marketable. This one needs serious overhaul before production release. Font is hard to look at, text size is miniscule, and call to action is way too weak to be inviting. Please take a look at S3 competitor like wasabi.

3 Likes

I think if I remember correctly the card is required for a technical reason … We did have discussions about the best way to convey that we arent charging immediately, but the propblem with popups is sometimes users dont read them. I know the current display might not be ideal to each person, but it seemed like a tradeoff, in order to make sure people have important information

I think if I remember correctly the card is required for a technical reason

Food for thoughts:

  • S3 & Google cloud provide free tier and free $300 credits to attract customers.
  • Microsoft Azure provide 12 months of trial
  • Wasabi provides a 1 month free trial that does not require a credit card.

Perhaps some STORJ can be reserved for free trial purposes to help onboarding new customers, as well as eliminates the needs of “CC during trial” at least for the first few years until we are getting a market traction?

1 Like

Glad to see this being discussed as most SNOs prob have these kind of worries in back of their mind. It comes down to how much faith you have in Storj team, that they will launch production at right time. I also think it might be exaggerated that if storj will have a very bad bug in early production, the project will fail and not be able to recover to profitability.

1 Like

@JB273 Id love to have you in the group! @stefanbenten is it possible to get an invite over? @littleskunk you have a volunteeer recruit!

If you guys are looking for SNOs to contribute storage to a trial/sample/test/sandbox infrastructure/satellite for further R&D and testing work, drop me a line. Definitely don’t mind putting a couple of unpaid TB in each location up for some middle ground between your test infrastructure and production SNOs to work through issues.

I would much rather see a few TB go to ensuring the platform works for the remainder of my own and everyone else’s infrastructure than having problems and as a result having reduced overall utilization.

1 Like

:rofl: :joy: @jocelyn, good one…!!
I’ll guess need to wait for some invitation from someone?
Do I need to discuss way of testing and needs for this with @stefanbenten and @littleskunk?
I run mainly VPS nodes for this and my other projects… (but sure we can figure something out for this…)

Did you try above link ?

Just did right now, but was replying to the message from @jocelyn where she was stating an invitation needs to be send out…

1 Like

You have mentioned Ryzen 2200 as CPU earlier. It has 4 cores without HT/SMT. So 400% would be max possible cpu load.

Yes I was wrong, I have a Ryzen 2400 which has 4 cores with SMT.