Hello, I am very new to the project and just have a few questions that I am struggling to find.
I don’t understand what running the linkshare service achieves? It is possible to get a url for public downloading without it?
s3 compatibility, you have to go through a gateway? Firing up a small vm while not terrible on the budget feels like an extra network hop. Hard for me to sell to my colleges that you get faster/same response times then using straight amazon s3. Are there any plans to integrate compatibility on the satellites to remove such a hop? Most of our applications use s3 to cache data to ease load off of a very busy database system. We cache large objects like so… local ram cache → redis → s3 → database. I’d love to go half the cost and switch to tardigrade.
I understand the files are split up and sent to many nodes. On retrieval though, are we regionally bound? Meaning if I hit us-central the data is pulled from many nodes in that region to reduce latency?
I will try to answer you to the best of my knowledge:
This sounds like you are asking about the uplink service, not linkshare.
Uplink acts as a gateway to the Tardigrade storage service which you can then use to manage access grants, buckets, etc. - Using the Uplink CLI - Storj Docs
LinkShare is the public facing service which can provide access to objects if you don’t run your own Uplink service.
If you do not want to run your own Uplink service then the new Gateway-MT may be suitable… see below.
Due to the end to end encryption on the Tardigrade network running your own Uplink gateway means that StorJ Labs would never be able to read your buckets or files. If true end to end encryption can be a compromise then the Multi-Tenant Gateway is in Beta - S3 Compatible Gateway Hosted by Storj - Storj Docs.
Due to the decentralised nature of the network you should not think in regions, and even the satellite names are now misnomers as they are multi-region. At a file request the quickest nodes will respond, whether they are geographically local doesn’t really play a part, speed of download/upload is the priority.
TY @Stob , That makes a lot of sense. Running a gateway locally in our environment would give us easier product integration but keeps everything encrypted to or from our datacenter. Its the end to end encryption that is the gotcha on s3 integration.
I would add a little bit.
The uplink is a command line tool, it’s not a service.
When you generate an URL with it, it will use a hosted Tardigrade linkshare service to host this link by default.
You can run your own linkshare service to be independent of our hosted version.
The linksharing service allows you even host your own static site: [UPDATE] We've launched Linkshare short URLs and static site hosting!
You also can use many tools with a native connectors:
To connect s3-compatible tools to Tardigrade you need either hosted Gateway-MT or self-hosted Tardigrade S3 Gateway. The choice is depends on your needs.
The Gateway-MT uses server-side encryption, the self-hosted Tardigrade S3 Gateway uses client-side encryption (the last is more secure).
In case of Gateway-MT you need only create an access grant and register it on Gateway-MT, it can be done via Browser in the satellite UI or via uplink CLI on your own computer (the last is more secure) and you won’t need to host or run anything yourself.
In case of Tardigrade S3 Gateway you will got more secure interface to Tardigrade, because files are encrypted before the leaving your network, unlike Gateway-MT, but with two downsides: it uses many connections to independent nodes as a native connector, i.e. uses more bandwidth (the current expansion factor is 2.7) and you need to host it yourself inside your infrastructure.
In both cases as result you will have endpoint, Access Key and Secret Key. This information is enough to connect almost any s3-compatible tool to the Tardigrade.
Thanks guys! I totally get it now. I did an rclone and the data was ingested very fast. On to trying out the nodejs and .net client libraries. Tardigrade is very cool.
Side Question, what do people commonly use as a solution for sharing files with friends and family that leverages tardigrade as the storage? I mean for non-tech friendly people so they can just drag and drop to upload files etc. Thinking more web ui then me sitting with non-tech people setting up filezilla for them all
The usual way is to use uplink share --url "sj://mybucket/Documents/My Wedding wishes list.odt", but you can take a look on experimental web-interface: [Testers Needed] Experimental Web Interface + Duplicati
Or you can configure rclone to mount a bucket to the folder on your PC: rclone mount, this will give you OneDrive-like experience. However, there is not so much GUI options to share a link.
The FileZilla will give you easy drag’n’drop option, and maybe they implement a native share in their interface sometime in the future.
I’m probably not answering any of your questions but here is a link that maybe helpful for just get an idea of the potential speed: sintel | Tardigrade
hey @cyth just checking back in. Hows the product workin g out on your side? Any issues with response time, any issues or success stories you can share ? thanks!
