I used both commands for 3 nodes and all migrated and are working fine. I didn’t even bother to understand what’s the difference between them. 
I activated boath, like on my picture, and working looks like only passive
I realy dont understand why it not working, i checked everything
2025-03-13T22:13:10+02:00 INFO piecemigrate:chore all enqueued for migration; will sleep before next pooling {“active”: {“12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”: false, “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”: false, “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”: false, “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”: false}, “interval”: “10m0s”}
looks like it not working on windows gui
storj/storagenode/piecemigrate/chore.go at main · storj/storj · GitHub
looks like this place is responsible for Active migration.
But i Do not understand where is this data coming into this function
func NewChore(log *zap.Logger, config Config, store *satstore.SatelliteStore, old Backend, new piecestore.PieceBackend) *Chore {
chore := &Chore{
log: log,
Loop: sync2.NewCycle(config.Interval),
config: config,
old: old,
new: new,
reportingBatchSize: 10000,
migrated: make(map[storj.NodeID]bool),
migrationQueue: make(chan migrationItem, config.BufferSize),
baselineDataRate: mon.FloatVal("migration_chore"),
}
_ = store.Range(func(sat storj.NodeID, data []byte) error {
b, _ := strconv.ParseBool(string(bytes.TrimSpace(data)))
chore.SetMigrate(sat, true, b)
return nil
})
return chore
}
Perhaps you need to change CRLF ending to LF and change the encoding to UTF8 without BOM.
after this change migration do not execute at all.
Did you restart the node after that change?
Also, did you check the size of blobs and hashstore?
Yes I made a restart, and after restart when it ON UTF-8 this row do not apear any more
2025-03-14T09:19:16+02:00 INFO piecemigrate:chore all enqueued for migration; will sleep before next pooling {“active”: {“12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”: false, “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”: false, “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”: false, “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”: false}, “interval”: “10m0s”}
as I understand then migration do not start at all.
I think the CRLF change is not working. Could you please return CRLF back. UTF8 is ok. The main point that it shouldn’t have BOM (see unicode - What's the difference between UTF-8 and UTF-8 with BOM? - Stack Overflow).
Nodepad++ makes it UTF-8 without BOM. it can make it with BOM but it separate. I didnt found how to change CRLF
Just remove the ending at all (you can enable to show endings).
The change to UTF8 could break things only if there are invisible characters (special codes).
Yes I cheked there is no ending, so UTF-8 not working, By default id UTF-16LE BOM
This is because you replaced these files from PowerShell.
My Windows node shows UTF8 and CRLF:
However, you may return the previous encoding, if it works for you.
This file I also have in UTF-8 and CRLF, it was made by node I just changed it, but
migrate_chore file i made in power shell
My recently migrated node of 1.5TB keeps compacting and compacting non-stop in the last 4 days. I see these entries in log from 11.03 untill now:
compact once started - 389 times
compaction computed details - 389 times
hashtbl rewritten - 389 times
compact once finished - 389 times
finished compaction - 75 times
Is this normal? The harddrive is working way harder than the piecestore + badger cache.
I believe this hashstore implemantation is doom to fail.
And if this is the normal way, how about in case of someone uploads and deletes millions of pieces daily, because he needs it to or to sabotage the network? It will keep compaction working endlessly, degrading the network’s performance.
I don’t know what those finishes are and how are they different:
INFO hashstore compact once finished {"Process": "storagenode", "satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "store": "s1", "duration": "34.195636216s", "completed": true}
INFO hashstore finished compaction {"Process": "storagenode", "satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "store": "s1", "duration": "24m43.492837941s", "stats": {"NumLogs":262,"LenLogs":"253.8 GiB","NumLogsTTL":20,"LenLogsTTL":"12.0 GiB","SetPercent":0.9600693517174765,"TrashPercent":0.3651869184526864,"Compacting":false,"Compactions":3,"TableFull":0,"Today":20160,"LastCompact":20160,"LogsRewritten":60,"DataRewritten":"32.2 GiB","Table":{"NumSet":1081937,"LenSet":"243.7 GiB","AvgSet":241815.72485643803,"NumTrash":304302,"LenTrash":"92.7 GiB","AvgTrash":327035.057962156,"NumSlots":4194304,"TableSize":"256.0 MiB","Load":0.2579538822174072,"Created":20160},"Compaction":{"Elapsed":0,"Remaining":0,"TotalRecords":0,"ProcessedRecords":0}}}
there is a chance, that active converting is just not reporing anything, I tried once more as you suggested, make it UTF-8 and LF end of line.
No logs, but disk activity is 2x or even 3x
Maybe your logs are still not on info
It should be improved in the rolling version
1 Like
it is info, other I woldnt see anything about migration chore at all.
also i see all downloads and uploads
This problem not exists in v1.122
What storjnode version you are using?
1.123.4 so latest version.