Like this post says, this forum does not seem to cater very much for Storj clients, most discussions are for node operators. (Maybe open a section for clients?) But surely, if people are using Storj as client, what tools do they use?
I know that it is technically possible to use HBS3, Duplicati and RClone for backing up a NAS to Storj (via the S3 interface). And I have seen the list of tools that are supposed to work (be it in “preview” or not).
But I have found close to no feedback on how well that works, or any comparisons (speed, downsides/upsides).
Is there anyone that has tested various backup solutions NAS → Storj and is happy with one or more of them?
My needs:
Back up a huge amount of small files, and a respectable amount of 2-4GB files, from NAS (if possible, QNAP).
I have a delta of about 50GB to upload every week, on a total of about 2TB source. With a respectable amount of files that got changed and/or deleted.
This is for disaster recovery, so no hard need for history on the target backup system (storj). I give absolute priority to ease of restoring.
It should be reliable, well maintained and reasonably fast. And not too expensive.
Am on B2 now, via RClone. But since I operate a couple of Storj nodes, why not give some love back?
As for performance and usage, there are quite a lot of end users who have used the tools you have mentioned. Performance issues usually stemmed from having to adjust the parallelism outside of the default numbers. There are videos and guides out there, for instance here is one…
My personal recommendation is duplicacy (sic, not duplicati) with chunk size cranked up to 64 to cater to the segment size. You might want to set it to 32, I depending on the new data composition. The default is 4, and is quite small for storj.
I’ve a very good experience with Truenas backup as well.
Speed depends mostly on host preformance (CPU cores and memory).
Also bandwidth could be a bottle neck.
In my case i’ve plently from all of them so i set simultaneously Transfers to 8. But as already mentioned here you may want to set those for your needs.
This “backup” is using rclone copy under the hood. It’s not a backup in a conventional sense of this word, this is sync. (i.e. if you source data is corrupted (as a result of hardware or user error), the corruption will propagate to your target on the next sync).
In fact, the narrator in the video is using words “synchronize your data”, “cloud sync”, etc. Not “backup”, in spite of what the video title says.
I would consider switching to something else. Hyperbackup eventually corrupts the target datastore (happened to me more than once for it to be a fluke) and its very design is fragile: try to cancel in-progress backup, it takes over 10 min to do so. What is it doing all this time? The problem is it keeps the datastore in the inconsistent state during backup and does not tolerate interruptions. Network glitches and power outages won’t be waiting 15 minutes for it to gracefully disconnect.
It’s only a matter of time until you lose data. I recommend not using any of synology software for any purpose. There are plenty of high quality alternatives.
None. I got rid of Synology diskstations years ago, my life has significantly improved as a result, I now spend time I used to waste triaging Synology issues and filing bug reports that never get fixed on something else, and I’m not planning on ever going back.
Or if this is just poorly worded question and you are actually asking for specific recommendation for you to use — read third post in this thread.
At least they did not modify OpenSSH authentication code by inserting calls into their opaque layer, and having “secret default password” in the past. Or did they? I don’t trust anything anymore
There were a few more, in oder DSM 4xx, but I can’t find them anymore on their web site.
But because they use open source components, they had to release their modifications. They did it in the best TiVo traditions – technically their contributions are open source, but practically absolutely useless for anyone else including those who try to understand what they have actually done: the modifications are calls into their opaque layer. Bastards.
Yes I use this too.
As you said, due to Storj not supporting versioning, should you copy over corrupted data you’re out of luck.
Do you know if versioning is coming to Stroj? Otherwise I should write a script to achieve this.
