March 4, 2020, 6:31am
You follow the documentation until the point of installing nextcloud:
So you have your gateway running.
Then in nextcloud you configure your external storage like this:
But be aware that currently there is a bug preventing you from succesfully uploading files >64MB to nextcloud. We’ll have to wait until that one is fixed. Smaller files work fine and all other operations too.
But because of rate limiting, doing many uploads/downloads/deletes will only make parts of them succeed… Also have to wait for a proper rate limiting implementation here too.
So my advice currently is to only use it for testing and “proof-of-concept” but nothing else until those bugs are fixed.
Thanks for you reply.
I managed to set up the external storage on my existing Nextcloud instance.
I then tried to upload a small file (“Hello world.txt”) from a browser, and it failed soon due to the error below.
An exception occurred while completing a multipart upload: Error executing “CompleteMultipartUpload” on “
http://localhost:7777/sample/Hello%20world.txt?uploadId=Upload2”; AWS HTTP error: Server error:
POST http://localhost:7777/sample/Hello%20world.txt?uploadId=Upload2 resulted in a
500 Internal Server Error response: <?xml version="1.0" encoding="UTF-8"?>
InternalErrorWe encountered an internal error, pleas (truncated…) InternalError (server): We encountered an internal error, please try again. - <?xml version="1.0" encoding="UTF-8"?>
InternalErrorWe encountered an internal error, please try again./sample/Hello world.txt3L1373L137
Would you please let me know how to solve the issue?
Thanks in advance,
Please don’t. Storjlabs seems to be basing their decisions on this rate limit on who requests an increase and encourages you to request that. So pleas go to
support.tardigrade.io and click the new request button to ask for your rate limit to be increased if you run into this. They will need your feedback in order to determine the correct setting for everyone. If you wait for them to fix it, they also don’t get the signal that the current limit is too low. Don’t hesitate to ask for this increase on your account.
Hi, I think
that issue should be discussed on another discussion.
March 4, 2020, 8:56am
Seems your gateway is down or inaccessible. Please, verify that either by a minio browser
http://localhost:7777/ , or
aws s3 CLI.
I think your gateway could be running on other PC rather than the PC with Nextcloud
Based on the post indicated above (
https://documentation.tardigrade.io/how-tos/set-up-tardigrade-with-nextcloud-primary-storage) I set up gateway via SSH on the same machine (VPS) as Nextcloud is hosted.
Next I tried to add a file manually on the folder on Windows 10, and then on SSH the error below is returned:
March 4, 2020, 11:00am
Ah yes I can reproduce this if you try to upload multiple files. Never tried that before. Guess I can add that to my already open github issue about Nextcloud.
Mind sharing the URL of the issue on GitHub?
March 4, 2020, 11:21am
Only weird thing is that I was trying to reproduce your first error and was only able to do that once…
@Alexey, would you please check my post above about the error I still get?
Thanks in advance,
March 20, 2020, 6:42am
The issues with nextcloud aren’t completely fixed yet. Just wait patiently. Alexey won’t be able to help either.
We’ve flagged this to the engineering team, and will post updates when we hear back - thank you!
So basically the manual on
https://documentation.tardigrade.io/how-tos/set-up-tardigrade-with-nextcloud-primary-storage has been completely rewritten and the steps look quite different than before-
Are steps explained there applicable for adding Tardigrade to Nextcloud as an external storage? Any help appreciated. Thanks,
April 1, 2020, 3:48pm
Yeah this new connector and how-to is quite a mystery to me too…
So the “old way” is: set up the gateway on the machine where your nextcloud is running. Then in nextcloud set up an external aws3 storage, the config should look like the picture I posted a month ago:
How to add Storj to Nextcloud "as external storages"?
Note that uploading files >64MB is still not working but everything else seems to be fine.
yet since a couple of the recent versions
gateway_linux_amd64.zip (which was available on
0.34.6, for example) is gone.
I managed to get it worked with an existing Nextcloud installation
If you are interested in getting tardigrade as an external storage, just follow these steps:
1. Follow the official documentation until the step “Run the S3 gateway”, included.
nb: there is an error in the command to run the gateway. It shouldn’t be “./uplink_linux_amd64 setup” but rather “./gateway_linux_amd64 setup”
2. Go to “Settings > Administration > External Storage” page
3. Add an S3 storage
April 1, 2020, 7:16pm
The gateway moved to a different repository:
Ah it looks like I got it working. Here are steps I did:
1 - Downloading files (uplink and gateway):
curl -L https://github.com/storj/gateway/releases/latest/download/gateway_linux_amd64.zip -o gateway_linux_amd64.zip && unzip gateway_linux_amd64.zip
curl -L https://github.com/storj/storj/releases/latest/download/uplink_linux_amd64.zip -o uplink_linux_amd64.zip && unzip uplink_linux_amd64.zip
chmod 755 gateway
chmod 755 uplink
2 - Setting up (uplink):
Create a bucket by running
./uplink mb sj://example (See
/dcs/api-reference/uplink-cli/ for other commands like deleting/removing a bucket, listing files inside a bucket, etc.)
3 - Setting up (gateway):
screen (to let gateway run on another screen)
./gateway setup (See
/dcs/api-reference/s3-gateway/ for more detail)
Make sure to copy
Access key, and
Secret key to somewhere safe (like a password manager)
4 - Setting up (Nextcloud):
Open External storages setting page
Input a folder name
Amazon S3 for
Access key for
Bucket (See above: 2.2)
Hostname (See above: 3.4)
Port (See above: 3.4)
Enable Path Style
Access key (See above: 3.4)
Secret key (See above: 3.4)
Click three-dot-icon for advanced settings
Click check mark to complete the configuration
And the external folder should be available!
If you want to sync the folder to your local machine, open the Nextcloud client and follow the instruction displayed there.
Please make sure that syncing the external storage with other machines multiplies the bandwidth. Also storing a bunch of tiny files can be costly (See: Nextcloud on Storj? - #17 by kevink).
On Tardigrade there are three ways of calculating fees (Object Storage fee, Per Object Fee, Bandwidth Fee). I would recommend you to read
https://documentation.tardigrade.io/pricing/billing-and-payment carefurelly before you move your files to the external folder on your Nextcloud instance.
While Storj have arrived at 1.0 the procedures written above are subject to change after future release. Please follow
https://documentation.tardigrade.io/ periodically for available changes.
Also according to
I’m looking into linking my nextcloud server with storj dcs. Are the above steps still valid and/or is there an up-to-date howto to do so?
Thanks in advance!