I am not a developer so I cannot make suggestions how to this can be achieved. But the flaws and conceptual problems of the current implementation are getting obvious more and more every day.
We have seen at least ideas that try to reduce or optimize the pressure on the storage.
One of the best solutions seems to not use the databases at all. It was suggested many times to make them optional or at least as many of them as possible as an unknown number of SNOs have no use for the data they hold.
I believe that is a suggestion.
Other suggestion is to move as much of it into RAM. Running the databases not on the slowest storage but on the fastest available, seems logical to me.
Other suggestion is to separate actual data from history to keep the working databases and data loss in case of corruption small.
And if Storj is running out of ideas, they should talk to the developers of Sqlite:
Maybe they have additional ideas or can implement something that helps the Storj use case.
Better solutions for the filewalkers have been made to not to run them on every restart and make it configurable when to run them. Also saving state has been suggested and of course helps when it does not restart from the scratch every time.
Some of this is already getting implemented but for some reason not rolled out as fast as it should be.