Has it ever been considered to use in-memory databases?

[Alexey]

It would require a separate chore to move data from the operational database to the historical database, it also will require to redo how this information is handled in the storagenode API (it must combine data from two sources (three if we account both operational databases - in-memory and on the disk, also historic DB) for each metric).
it’s as complicated as it sounds. As result - it likely slow down the node and anything what would request these metrics.

I do have more hope into this research:

@elek

So far I have heard about multiple attempt to improve the piece store. They are usually save the metadata of pieces in a database like system and save the raw data in custom files.

(technically it would be possible to store both metadata and data in database, but it’s quite slow. I tried it, and measured it )

Storing metadata and data in separated locations can be done in multiple different ways:

1. choose a database which already supports it, and stores the data outside of the main indexed data.

Badger looks like sg. like this. It stores the data outside of the SST files, and most of the LSM tree operation doesn’t need to touch the binary files itself.

Jacob has proof of concept of this approach (https://review.dev.storj.io/c/storj/storj/+/12066 ) and I also had an earlier experiment (GitHub - elek/storj-badger-storage).