Perhaps the most efficient storage was based on LevelDB in Storj v2 (Introducing KFS: a local file store inspired by Kademlia), but it has had a hard limit of max allocated size per node, and since it was a custom solution, the repair tools for LevelDB didn’t work well. It also used more memory to operate a node (exactly like you suggests) and we have had many complains about high memory usage.