Average is highly misleading and not useful. Even median does not work here. You need to build a histogram of sizes. There are several peaks.
3 Likes
I have an other bright idea or better to say question
Why does storj node not offer an block device management feature?
If an operator has one dreive to offer it could start one node with
docker run --device=/dev/sdc ... and let do storj its magic
Storj keeps telling us to use extra space on drives that we have online for other reasons anyways. Adding a flag to give a node an entire device (so there can’t be “other reasons”)… doesn’t help with that message 
1 Like
@Roxor I had the same opinion, that storj node supports to run on spare space, but then discovered that in practice only full spare drives can be used.
I had already issues after 2y with storj and bitcoind on the same drive.
The random access of storj will simply degrade any other application to “timeout”
I found this to be very much the case some time ago, but node performance has gotten much better over the years. What hardware are you running your nodes on?
Some old HDD Seagate Ent. or Constellation v5
I simply used a second drive for the bitcoin node and all was fine
bitcoin really uses that much disk access?
wait how big is the full bitcoin blockchain now anyway?
blocks 677G (can be an HDD)
chain and fullindex 68G (should be on flash)
To reinvent a filesystem? We do not have so much resources to implement it in a reasonable time. The existing filesystems likely works better, because they implemented by much more great Community or companies.
However, you may enable a dedicated disk feature:
1 Like
Think of hashstore without a file system…
how? It still relies on the file system and its ability to use RAM for caching.
If we would go this way, we would need to replicate a whole layer to support DIY filesystem only to manage hundreds files.
Too much waste of resources and time for zero difference, or likely make it worse.
Not to create an own fs, but to manage a filesystem,
better to say to format the drive with a fs and parameters of storj taste.
I do not think it’s a good idea to do so, sorry.
You, as an Operator can tune parameters, our software is designed to work with default parameters.
If you can improve by tuning - it’s good for you 
2 Likes
That’s like saying operators can also change node’s source code to tune it. Possible, but unlikely in vast majority of cases.
I didn’t know you can expose a block device to a docker container. I can see this working. A setup script could format block storage to an already pre-tuned file system, which will likely be better than many, many defaults operators run with now. That said, hashstore likely obsoletes any filesystem tuning at this point.
2 Likes
You could copy and implement bluestore, as used in ceph. Probably not in a weekend of course, but it is probably doable.
Why? Usually Ceph is replaced with Storj. But downgrade and slow down? I don’t think so.
No no, nothing with ceph at all. I’m just saying if you need an efficient and well tested storage backend for a node a full filesystem is overkill. Instead of making your own, you could use bluestore directly on a block device. I think I saw a go implementation somewhere.
This is also mean that we need a separate team to reinvent a wheel filesystem, I do not think that it would be so easy and quick. The main pain would be to implement a correct caching.
1 Like