New us2.tardigrade.io Satellite!

Hey SNO friends!

I just wanted to let you know about a couple of exciting upcoming developments that will affect you!

As you know, we have two load-testing Satellites, saltlake.tardigrade.io and europe-north-1.tardigrade.io. We’re adding a new feature testing Satellite! This new feature testing Satellite will be another invite-only Satellite (not open to general signups, like saltlake and europe-north-1) where we will be trying new features coming to the network. It won’t be very load heavy, but we are going to be adding it to the trusted satellites list. This satellite will be us2.tardigrade.io.

We’re excited about the features we’re currently putting in test:

  1. We’re finally adding native support for multipart uploads. We’ll be testing that on us2.tardigrade.io.
  2. We’re making this Satellite the first multi-region Satellite. Up until this point, Satellites exist within a single geographic region, but this Satellite (and soon all other Satellites) will survive even if an entire region goes down. This is why the naming scheme is somewhat new (us2.tardigrade.io).

We’ll most certainly test additional features on it in the future.

One other note - we know there’s currently a bug with loading tardigrade.io/trusted-satellites on Arm64 storage nodes, that should be fixed in this next rollout.

Happy New Year, and thanks again for all you do!

28 Likes

As soon as the new satellite is added you might see it on the dashboard with a 0% score. Don’t panic. This is expected.

image

After a few minutes it will jump up to 100%

9 Likes

Hey new satellite operator, welcome to the network. In the first month or so you won’t be audit yet but if you operate correctly and have good uptime then you can expect to have 75% of your data kept in a hidden folder. For months 4-6 50%, 7-9 25%. Don’t worry, after 15 months half your data will be revealed!
:grin:

10 Likes

:rofl:
That’s not bad…

In all seriousness, I like a new satellite but I hate to start the 75% held back amount phase again… My node is over 20 months old but now it’s like a baby to that satellite. I should vet the satellite instead :smiley:

Anyways, good luck testing! Looking forward to more features in tardigrade.

14 Likes

How do we know if we were invited to this satellite?

1 Like

Congratulation for the newcomer. Take care about him and Happy New Year


baby tardigrade (ref. Live Science)

5 Likes

Tardigrade customers might get invited. All SNOs will be part of its network automatically.

2 Likes

So, when will this new satelitte be added?

Oh…Every New Year… we have “the same story about a new satellite” :slightly_smiling_face:
It will be a new Storj tradition for cost optimization and reduce payouts for SNO :smiley:

How about saltlake.tardigrade.io and europe-north-1.tardigrade.io, it will be decommissioned like a Stefan satellite?

3 Likes

I haven’t got that satellite.

It’s not on the trusted satellites list yet. This was just an announcement that it would happen.

1 Like

I want you to panic first so that by the time the satellite gets added everyone is prepared :slight_smile:

7 Likes

We have no plans to decommision these!

For what it’s worth, we expect low volumes of data on this new feature testing Satellite (us2.tardigrade.io) in the near term. We’ll give everyone a chance to build reputation with it.

7 Likes

Thanks for the information @jtolio !
Can I ask you about: Could you please share more information (on blog or forum) how the new “multi-region” architecture is working?

1 Like

Yeah, we’ll be sharing much more here soon, but the basics are we are backing the us2.tardigrade.io Satellite via a multiregion https://github.com/cockroachdb/cockroach deployment. This way, Cockroach keeps track of all of our Satellite application state, and the app servers are basically stateless on top of it. So by using Cockroach we get it for free. We just haven’t been exclusively using Cockroach in a multi-region configuration for all of our Satellites before this (some Satellites still rely on Postgres).

5 Likes

What is “multipart uploads” ?
Because apart from this I don’t see any exciting news. testing…testing…testing. We need one more GIF with S. Balmer.

Do you mind sharing under the hood details on how Cockroach is being deployed multi-regionally? On top of Kubernetes? (if this details are open-sourced feel free to share ref)

“Multipart uploads” is a feature that S3 supports that we haven’t supported natively before. You can read more from Amazon here: https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html, but we’ll be adding our own docs soon.

Sure! Currently we just pay Cockroach Labs to host it for us. They are doing a great job running it and helping us master Cockroach. https://www.cockroachlabs.com/product/cockroachcloud/. I think I heard that Cockroach runs their own clusters in Kubernetes.

We’ve started configuring our own global Cockroach Clusters for smaller side services and intend to get better at running them ourselves, but it’s early days there.

3 Likes

it looks like this mean, that shold be esier to upload big files.

as i understand today big problem, that if some file part upload faild then it wont make retry and whole uplad it failed, is it resolved somehow in multipart?

4 Likes

It highly depends on the implementation.

For instance, AWS S3 multipart upload allows you to upload parts in a non ordered way and also retry if one of the parts fail, as the upload is only committed when you explicitly completes it the file will be “hanging” until then.

2 Likes