If Storj now owns software that exposes object storage as a regular filesystem… will I be able to use it more like a drive-letter in Windows? If it just became a regular directory for my normal backup apps that would be a lot easier.
I’ve played with mounting S3-compatible services before: but it was always clunky and needed apps from other companies to work.
My understanding is that it works just on Linux at the moment and there is a Mac and Windows App on the way. I might be mistake here since I am just a single day ahead of you and haven’t done my own research yet.
Do you have a link? In the official documentation it still tells me to use linux subsystem. I wouldn’t call that a native client. And on the release page I also see nothing that looks like a windows executable. So I didn’t find any trace of a native client yet. (Again we are taking about just a few minutes of research on my end)
For a little more context on the announcement, Petagene is a dynamic company with a talented team of file storage experts based in Cambridge, UK. PetaGene is the creator of the cunoFS distributed file storage mount client.
On the heels of our acquisition of Valdi, announced in July, we’re announcing adding another critical capability to the platform. We’re very excited about the acquisition of PetaGene for several reasons - you should check out Ben’s blog. I want to share a similar message tuned for how this impacts node operators and long-time Storj community members.
CunoFS provides key features that are extremely relevant in our top use cases. We’ve seen a lot of interest over the years in a dropbox-like experience with Storj and cunoFS is the best POSIX-compatible client for S3 object storage software we’ve seen.
This acquisition provides additional differentiation in a number of key use cases in media and entertainment, AI/HPC, as well as scientific data. Though cunoFS is storage system agnostic, we expect the acquisition will drive additional sales and growth of Storj’s object storage service and especially storage in conjunction with GPU sales from Valdi.
In terms of growth, cunoFS gives us a fast path to migrate workloads from other providers onto Storj and also an opportunity to easily be a secondary backup to other cloud providers.
We’re extremely excited about this opportunity, how it drives further differentiation and opens up new ways to grow the network. If you want to learn more about cunFS, check out their documentation and try it out!
It sounds like cunoFS is designed to make the most of multiple connections… and Storj is designed to make the most of multiple nodes. If the bottom-end of cunoFS software can be tailored to more directly align with Storj’s view-of-the-world (while retaining S3 compatibility for everyone else)… it should be able to absolutely spank Storj nodes: and squeeze out every drop of speed!
Potato SNOs beware! A Dropbox competitor is coming to crush your setups! (and pay you for the privilege)
They use a parallelism which is one of the key features of Storj, thus speed will be much faster than with any other S3-compatible storage across the globe and independently of the time periods during the day.
Judging from the buzzword salad on the main page, it’s not parallelism—at least not directly. To employ parallelism, you need to have the consumer application request different data ranges. The only chance to employ parallelism if the consumer application doesn’t support it directly would be downloads based on statistical predictions… possible, but difficult to get 100% correct.
To me this tool looks more like a lot of preemptive caching of metadata and short-circuiting syscalls by injecting itself into the consumer processes and taking over them at libc call sites. The former is clear given the tool stores metadata in separate database-like files. The latter is not unlike what for example wine does, and in general is tricky, but can result in really good performance indeed if the consumer application allows it.
The latter was also the general approach of providing file system-like API to unaware applications on Unix-likes before FUSE was popularised. Some prior work: avfs, plasticfs, there’s actually quite a lot of similar projects.
In any case, pulling off a composite approach like cuneFS is no small feat.
Also, having both cuneFS and storage itself under one roof will probably allow making some optimizations in future. For example, making it possible to update the metadata database concurrently from many nodes while merging changes satellite-side would be a big boost to performance.
I’ll second Toyoo’s analysis. cunoFS on Linux eliminates the overhead of user space functions calling file system calls into kernel space only to be satisfied by FUSE plugin running in user space. There is predictive data loading, and a lot of workarounds to make object storage fit in a local filesystem environment. Making it all work well is of course the hard part.
The above is true for the classic version of cunoFS that is highly optimized for high performance Linux applications. There’s a lot of work to map cunoFS’s value to Windows and MacOS environments. You’re more likely to want LAN file locking, write caches, etc., and the final versions may run on top of a local file-sharing protocol such as SMB or NFS.
Circling back to the original question, I expect to see short-term benefit for storage node operators, rather than Windows or MacOS end users.