What is the worst that would happen if I delete all of my SQLite databases and restart storagenode?

[cdhowie]

Sorry, @Alexey, I wrote this post while you were splitting the topic. Please move this post too.

---

@SGC

ECC enables you to perform regular scrubbing of your memory for little performance loss, which is required to keep a stable integrity of the data that isn’t often used, RAM isn’t super stable over long periods of time with little usage.
sure it’s not relevant for short periods, but if you expect the node to run for like months without a reboot, you should expect such things to make life alittle less troublesome… its not for fun the server standart is ECC.

Yes, I agree. But again, that discussion is totally irrelevant to whether the databases are stored in a ramdisk. Whatever RAM is free is used for the I/O cache so if you are picking between corrupting a DB in RAM or corrupting the I/O cache, you’re going to have a bad time either way.

@SGC

instead of having the db in ram being worked on and saving changes to an ssd for recovery if shit hits the fan, then you now will have a db in ram, saving it in ram, so that you can loose the whole database

@SGC

I cannot remember if RAM is more or less safe than on disk, but i can tell you what happens if the power goes out… then its safer on disk

If the databases are not critical for node operation and can rebuilt, then it doesn’t actually matter. That’s why I want to hear from @Alexey on this topic.

I don’t care about any metrics I’d lose in the database if they don’t impact node operation. I have a separate monitoring server that collects and records historical data, so the metrics in the node database is redundant to me.

@SGC

so in the end i would almost argue that the performance benefit you would see would be very low if any

Seeing 1-2 orders of magnitude improvement on DB writes that are already contending for disk IOPS isn’t an improvement? Okay…

Keep in mind that sync() on RAM is ~free; the data is already there. On an SSD it means pushing a write buffer to NAND cells and there is still some latency on that. Not as much as an HDD but still significantly more than on RAM.

@SGC

and you just end up taking up more memory

This is the only point that I will concede – you are going to shrink the available RAM for the I/O cache. But keep in mind that, at least on Linux, the contents of a ramdisk can be swapped to disk. This has the net effect of ejecting DB pages not used to swap. If your swap is on SSD then you only get the addition performance hit of writing the database to disk when the system decides it is advantageous to free up RAM. So you’ll see significantly better DB performance in the average case, and in the worst case you’ll see the same performance as using an SSD as storage (which would be rare as the swapping algorithm is pretty good about deciding what to swap out when there is a lot of free RAM).

@SGC

get ZFS and it will basically put your DB in RAM by default… atleast it’s most frequently accessed parts, to make everything run better… and it would give you so many more advantages on a server with the RAM to spare from stuff like RAM drives for storagenode databases.

For reads, sure. For writes you still need to have writethrough to some kind of persistent storage, even if it’s an SSD cache that does writeback to an HDD, and writes are where you see the contention. During writes, you cannot even read from the database so writes must be fast.

Uncontended reads on the database should generally be of no consequence with or without ZFS.