Dashboard and log fails

Node Operators troubleshooting
,
1

Hello, I noticed that the Dashboard has a problem which started yesterday. Almost no Ingress/Egress compared the day before. Today after 16 hours and some downloads in the logs i am still at 0 Bytes Ingress which does not make any sense.

In the Log I see the following errors I never saw before:

2020-05-14T13:36:10.065Z WARN piecestore:monitor Used more space than allocated. Allocating space {“bytes”: 8000000000000}
2020-05-14T13:40:33.517Z ERROR nodestats:cache Get held amount query failed {“error”: “heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"; heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"; heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"; heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"; heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"; heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"”, “errorVerbose”: “group:\n— heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"\n\tstorj.io/drpc/drpcwire.UnmarshalError:26\n\tstorj.io/drpc/drpcstream.(*Stream).HandlePacket:156\n\tstorj.io/drpc/drpcmanager.(*Manager).manageStreamPackets:313\n— heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"\n\tstorj.io/drpc/drpcwire.UnmarshalError:26\n\tstorj.io/drpc/drpcstream.(*Stream).HandlePacket:156\n\tstorj.io/drpc/drpcmanager.(*Manager).manageStreamPackets:313\n— heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"\n\tstorj.io/drpc/drpcwire.UnmarshalError:26\n\tstorj.io/drpc/drpcstream.(*Stream).HandlePacket:156\n\tstorj.io/drpc/drpcmanager.(*Manager).manageStreamPackets:313\n— heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"\n\tstorj.io/drpc/drpcwire.UnmarshalError:26\n\tstorj.io/drpc/drpcstream.(*Stream).HandlePacket:156\n\tstorj.io/drpc/drpcmanager.(*Manager).manageStreamPackets:313\n— heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"\n\tstorj.io/drpc/drpcwire.UnmarshalError:26\n\tstorj.io/drpc/drpcstream.(*Stream).HandlePacket:156\n\tstorj.io/drpc/drpcmanager.(*Manager).manageStreamPackets:313\n— heldamount service error: protocol error: unknown rpc: "/heldamount.HeldAmount/GetPayment"\n\tstorj.io/drpc/drpcwire.UnmarshalError:26\n\tstorj.io/drpc/drpcstream.(*Stream).HandlePacket:156\n\tstorj.io/drpc/drpcmanager.(*Manager).manageStreamPackets:313”}

My Dashboard shows I have 195GB left.
So is this a calculation error and my node is simply full?
“docker exec -it storagenode-v3 /app/dashboard.sh” shows also 195GB space left (8TB allocated). My free space besides the node is about 3TB.

image
image909×694 40.9 KB

2

Your trash folder might have more than 195GB in it, so your node won’t accept new data until this is cleared up.

1 Like
3

Thanks for the reply. Can I clean it by my self? which folder is this exactly?

Can I just delete the files in the folder …/storage/trash?

4

No. Do not delete any files manually. If a satellite mistakenly deletes a piece it can be restored from trash but if you delete it and that piece gets audit, you will fail audit.

1 Like