Lost databases during node migration

Hello all,
pretty “interesting” situation happened with me today.
due to situation in Ukraine i was need to move 2 nodes to new location, without any mind i picked them (bare HDDs), and moved them to a new location.
And here “interesting” part coming, after config.yaml check i found that i forgot that DBs were on separate SSD!!! and it is already formatted and DBs are not accessible anymore…

So the question is, what to do in this case?
i have copy of that dbs before i moved them to ssd, but they pretty old (i think ~6 months). Should i use them or should i just launch node without them at all? or… node already lost for me?

p.s. im talking about this dbs
image

Just to assuage your immediate worries - no, the nodes will survive losing their DB. What will be gone is all the history from those DBs. If you have turned on the nodes without their DBs and they ran, AFAIK, you shouldn’t place the old DBs back as it will actually cause data discrepancy and get you DQed.

For now, just hang tight while more knowledgeable people respond. I just wanted to let you know you can relax, and a bit more downtime on your nodes is probably better - than rushing things right now

As @thelastspark said the nodes can survive without the DBs, you just lose the bandwidth and traffic history for each node.

If you have all the blob data, identities and configs setup then just start the first node. The DBs should be recreated from scratch. Check the log file for any issues. If no problems show up then start the second node and check it also runs fine without errors.

3 Likes

This is true only if they would restore old blobs, fortunately databases just would have an old Stat and history, so, only part of the history will be lost. But if you already started without DBs - they were recreated and you may leave it as is.

So for my own understanding, assuming blobs were not touched/copied/recovered, even if you turned the node on without old DB files and it ran recreating them (thus loosing history), you can later overwrite said DB files with previous ones to restore most of the historical data (except the newly created one) without issue??

Lauched node without databases as you advised earlier.
as i can see Used space is wrong, it was more than 4TB used. Will it be fixed with future audits or?

Yes. Once the file walker has finished, the pie chart will update.

2 Likes

Yes. The outdated occupied space should be recovered by filewalker during restart, all other will have an outdated information.

1 Like