Storagenode have free space, during log show node full, and no ingres for 3 days

[Vadim]

Logsat some point shown that node full

020-07-11T16:34:09.035+0300	INFO	piecestore	upload started	{“Piece ID”: “AQT22IGTC73TUKOGHDTHNH53J22EUWOMDBPALMFBZFXT5MIZTCBA”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “PUT”, “Available Space”: -1907390620928}
2020-07-11T16:34:09.041+0300	ERROR	piecestore	upload failed	{“Piece ID”: “AQT22IGTC73TUKOGHDTHNH53J22EUWOMDBPALMFBZFXT5MIZTCBA”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “PUT”, “error”: “out of space”, “errorVerbose”: “out of space\n\tstorj.io/common/rpc/rpcstatus.Error:82\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:360\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:996\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:56\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}

at same time on dashboard it shot 900+ GB free space, and no ingress for 3 days since this log input.

Any coment from storj team?

this free space not got free during deletion of data, some got free from GE on stefen benten, but it was before i got some data.

[Vadim]

Now after restart it looks like made some recalculations and started geting data. So i think during some Node not calculated durig GE deleted data i think.

[BrightSilence]

There seem to be a few scenarios in which used space isn’t calculated correctly. This sounds very similar to what happens during GC, where it’s counted twice. A restart usually helps, but shouldn’t be needed.

[Alexey]

Yes, it’s just need sometimes a few hours before update the information.

[shoofar]

the best informatic solution.

Turn off,
turn on

Works most of the time

[Alexey]

The binary logic, so binary solutions