As it starts to be more and more usable tool, I would suggest to make it part of storj github files with compiled file versions, like storagenode. not everyone have knolage how to compile software.
Perhaps it wouldn’t be needed soon.
I’ve deleted the zero sized hashtable and started over, but I still get the error unable to read header: EOF.
All three satellites successfully recreated the hashtable from scratch, only one failed.
Summary
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\1b\log-000000000000001b-00000000…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\24\log-0000000000000024-00000000…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\2a\log-000000000000002a-00000000…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\2d\log-000000000000002d-00000000…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\2f\log-000000000000002f-00004fd0…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\31\log-0000000000000031-00000000…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\32\log-0000000000000032-00004fd1…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\33\log-0000000000000033-00004fd2…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\34\log-0000000000000034-00000000…
Counting X:\Storj\hashstore\1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE\s0\35\log-0000000000000035-00004fd4…
Record count=2480
Using logSlots=13
hashstore: logSlots too small: logSlots=13
storj.io/storj/storagenode/hashstore.CreateHashTbl:74
storj.io/storj/storagenode/hashstore.CreateTable:137
main.(*cmdRoot).Execute:101
github.com/zeebo/clingy.(*Environment).dispatchDesc:129
github.com/zeebo/clingy.Environment.Run:41
main.main:29
runtime.main:285
Run again with ‘write-hashtbl -f -s 14’ (without quotes)
If 14 doesn’t work, use 15,16,… until it does.
Using that command worked,
thanks @RecklessD @Alexey
Are you working on increasing the fsck-speed at the beginning of the opening? Sometimes it is like 2-3 minutes just for one satellite, which seems way too long.
fsck should be fast, but the table scan requires to read them from the disk, so I do not think it can be speed up only by the storagenode software.
After recreating the hashtbl, previously my used disk was 1.4 TB and now it is 433 GB, trash only 11,78 MB, is that normal?
Did you regenerate all tables (for s0 and s1) for all satellites?
Did you remove the prefix database?
Did you have any errors?
- I’ve generated everything (s0 and s1) all satellites
- There are no errors
- What is a database prefix and what should I do with it? (I didn’t touch the database prefix)
There is a database, named as used_space_per_prefix.db, which was implemented to do not do a full rescan for a piecestore, however, it’s often a root cause of used/free space discrepancy (because it’s updated not realtime).
You may try to stop the node (and remove the container, if you use docker to run the node), delete this DB and start the node (create a new container with the same parameters for the docker method), this DB will be recreated and also will start a filewalker (for the fully migrated to the hashstore backend node it doesn’t makes much of sense, but sometimes it may fix a discrepancy).
When I recreated the hashtbl, I copied it to the s0 and s1 folders, removed the database prefix, and then ran node.
- Why did the copied hashtbl disappear and become the hashtbl-000000001 file?
- Is that a rebuild process ?
- If so, how long does the rebuild process take ?
- Does the rebuild process affect used space ?
Please enlighten me @Alexey
I hope that you recreated these hashtables for s0 and s1 respectively and didn’t copy the same hashtable.
Perhaps it was compacted, it’s not the same as rebuilding with a write-hashtbl tool.
You may check your logs for compaction messages.
Yes, after compaction some logs would be deleted and used space should be updated too.
I agree. Good idea.
But it’s a bigger change as we need to modify build scripts.
Until that I compiled the current binary for SNOs who are less familiar with go compilation:
BTW, I think it should be very rare when this tool is required (= in case of disk errors).
If your hashtbl disappears or zero sized without disk failures (for example because power failure), please let us know.
If hashtbl is broken, I would suggest to check it first what’s wrong with it, instead of automatic re-generation.
I try to recover metadata, but after counting it gives me this error
Record count=449670
Using logSlots=20
The file exists.
main.(*cmdRoot).Execute:96
github.com/zeebo/clingy.(*Environment).dispatchDesc:129
github.com/zeebo/clingy.Environment.Run:41
main.main:29
runtime.main:285
where is this file exist? i cant find it
This could be related to a bug, when Windows doesn’t allow to delete a source log after compaction, if there are opened handlers (e.g. someone downloads pieces in the same time), so this duplicated source log will remain.
However, if I’m right, I do not know how to proceed. In your excerpt there is no a log name, so I do not know what should be removed.
But if I’m wrong and you deleted this log, your node could be disqualified due to missing pieces.
this is from write-hashtbl after it end counting.
I see, so no way to figure out which one is excess.
I think it tells me that result is already exist, but i cant find it