Pieces error: marshaled piece header too big!

I’m seeing new error that I couldn’t find within the forum. What is marshaled piece header too big! ?
on v1.18.1

2021-01-07T22:40:03.966Z        ERROR   piecestore      upload failed   {"Piece ID": "D5OQHG3FDOPKT2TGEJ4GDIZS3UNGFDXOQK6NOF52UO3IJQB6KYUQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "PUT", "error": "pieces error: marshaled piece header too big!", "errorVerbose": "pieces error: marshaled piece header too big!\n\tstorj.io/storj/storagenode/pieces.(*Writer).Commit:166\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:384\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:996\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:29\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51"}

readwrite.go shows this comment and sounds like it did happen for me.

	if len(headerBytes) > (V1PieceHeaderReservedArea - v1PieceHeaderFramingSize) {
		// This should never happen under normal circumstances, and it might deserve a panic(),
		// but I'm not *entirely* sure this case can't be triggered by a malicious uplink. Are
		// google.protobuf.Timestamp fields variable-width?
		mon.Meter("storagenode_pieces_pieceheader_overflow").Mark(len(headerBytes))
		return Error.New("marshaled piece header too big!")
	}

Thanks for bringing this issue to our attention.
The problem appears to be that piece headers store complete order limits, and some changes were made to order limits recently which has increased their size, causing them to sometimes go over the piece header size limit.
We have people working on this right now.
Until then, using a bucket with a shorter name may possibly help.

6 Likes

We had a point release today.
I believe the issue should be resolved.
Thanks again!

4 Likes

Thanks, That is a respectable fast response!
I also take it as no action needed for SNO. :slight_smile:

1 Like