Doesn’t look like it will self heal and indeed “E:\Storage\blobs\pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa\tr” doesn’t exist in directory, but probably is stored somewhere in the database?!
Audits seem fine but the fact is that used space filewalker for Sattelite ID 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE fail to complete.
this is a dumb idea but you could manually create a directory called ‘tr’ in that location from the error.
I had a vaguely similar error where I had a manually created garbage test file in a blobs folder and it caused the whole node to fail because it wasn’t named appropriately like a satellite ID. I just deleted the test file.
This is interesting. Normally that filewalker just traverses a directory tree, rather than getting paths from anywhere, so it wouldn’t look for pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa\tr unless there was actually a tr directory under pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa. Unless your node is old enough (circa 2019) that it has V0 pieces?
Yeah, maybe older than that even. I have excluded the test satellites and run the known procedure for it…
I just know used space file walker fails because of this issue, resulting in a 2TB disparity between Dashboard and actually used spaced in Windows.
Hmm. Since the error is coming from readdir(), that suggests that there is an entry called tr under the pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa directory, but that it can’t be read. If that is the case, it ought to be fixed with a filesystem check (is chkdsk still a thing in Windows?).
It’s also possible, I suppose, that this mystery path is coming from the V0 pieceinfo database. I just don’t see any mechanism where that error message would show up with “readdir” in it.
Creating a tr directory as @EasyRhino suggested is probably safe? And it might solve the problem without a whole filesystem check.
The only case I’m worried about is if there were a lot of files in the tr directory and it was corrupted in a way that the filesystem check might be able to recover them all. But I think that’s not very likely.
That’s the thing, there isn’t a tr directory on that windows path.
Exactly, where’s this info coming from, where it expects to find a tr directory path?
CHKDSK /F was the first thing I did…no errors found. @Alexey, your thoughts?
If the fs check passes, creating the tr directory manually should fix the problem.
If you wouldn’t mind investigating further, we could try looking at the contents of the v0 pieceinfo database to see if that’s where the problem is. Do you have the sqlite3 command-line tool installed, or some other tool that can read sqlite3 databases?
And I would suggest to check the database with @thepaul
I would assume that the database in question is pieceinfo.db. Do you have any records here? I would suggest to check on it’s copy:
sqlite3 /path/to/pieceinfo.db "select count(*) from pieceinfo_"
That’s bewildering. I’ve been combing through the source code and from what I can see, the only place that the tr could have come from is the directory contents. A directory entry had to have been there for tr, but then when we tried to read tr we got the file not found error.
Could it have been corrupted into a symbolic link pointing at a nonexistent file? That would show up in a directory entry but would give a file not found error when it was opened. The problem with that theory is you probably would have gotten an error when you created the tr directory yourself.
Or perhaps chkdsk just doesn’t find all forms of corruption? I don’t know how thorough it is on modern ntfs.
I’m really stretching here to find any kind of explanation.
No idea @thepaul.
There was there indeed a file “tr” without extension…just “tr”; not a folder though.
I deleted it and created the folder, no issues.
I’d assume if I had lost a whole directory, I’d be penalized (perhaps disqualified) for missing pieces no?
Which doesn’t seems to be the case as my Audits seem ok.
I added “tr” folder and I’ll monitor it if now it finishes all 4 file walker processes successfully.
Bingo! This is a result of the filesystem corruption and fixing it with chkdsk… It sometimes happens on NTFS after the filesystem corruption, the directory may be recovered as a file.
If you do list without doing a restriction to show only folders, the file with the expected name would be listed as well.
