Hello!
I started some days ago , it was working fine and now I get the msg 
Your node has been suspended on 118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW 1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE
it is permanent ? 2 satellites from 6 on the list
You got suspended, not disqualified. The suspension is temporary if you manage to sort out the problem.
Find GET_AUDIT messages in the log and try to figure out if there is some error.
I get Some error about Download failed and “userserialsdb error: disk I/o error”
it is a new usb 3 disk
Is it Windows or Linux node ?
Download failed is not a problem, disk I/O error is critical. It means that the operating system cannot read from the HDD, probably because of hardware failure.
If using a USB HDD, you should use a split data/power Y cable to give more current to your HDD.
If this is your problem it will be fixed in v1.6.3 that is rolling out right now. Your node will recover automatically. This database will be moved to RAM to solve the locking issue.
GET_AUDIT lines with this error will also be shown as download failed. So it likely is the reason for suspension.
Yes, I meant download failed for normal GET.
Try to look for failed GET_AUDIT specifically. But others have reported the same issue on audits, so it’s likely what caused it. I’m still fairly certain v1.6.3 should fix this issue for you.
thanks for the answers guys!
it is linux
now it is showing 1 satellite suspended not 2
This problem raises concerns from me. Can you check your disk for errors?
Yep , Going to run test on the disk
—edit–
Do this cause the node get suspended ?
ERROR piecestore download failed {“Piece ID”: “IQU3OSJZIZCVG6FSMVX3MZNRNFNUBSE6RWLEIH2NORV6R2BEUMCA”, “Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “Action”: “GET_AUDIT”, “error”: “usedserialsdb error: disk I/O error”, “errorVerbose”: “usedserialsdb error: disk I/O error\n\tstorj.io/storj/storagenode/storagenodedb.(*usedSerialsDB).Add:35\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:76\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:459\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:56\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
2020-06-11T06:48:44.385Z INFO piecestore download started {“Piece ID”: “E4RKXCGZGLVOAQPUXSRG3Z35LMXLZ6OBICAAQNCSHOQQM46Z47YA”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “GET_AUDIT”}
2020-06-11T06:48:44.393Z ERROR piecestore download failed {“Piece ID”: “E4RKXCGZGLVOAQPUXSRG3Z35LMXLZ6OBICAAQNCSHOQQM46Z47YA”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “GET_AUDIT”, “error”: “usedserialsdb error: disk I/O error”, “errorVerbose”: “usedserialsdb error: disk I/O error\n\tstorj.io/storj/storagenode/storagenodedb.(*usedSerialsDB).Add:35\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:76\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:459\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:56\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
2020-06-11T13:47:05.513Z INFO piecestore download started {“Piece ID”: “BIWFRKIYPA2QPLPH7SZDY6LXITIRALU4ZCOZX7YOLFYF6BJYBYFQ”, “Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “Action”: “GET_AUDIT”}
2020-06-11T13:47:05.524Z ERROR piecestore download failed {“Piece ID”: “BIWFRKIYPA2QPLPH7SZDY6LXITIRALU4ZCOZX7YOLFYF6BJYBYFQ”, “Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “Action”: “GET_AUDIT”, “error”: “usedserialsdb error: disk I/O error: input/output error”, “errorVerbose”: “usedserialsdb error: disk I/O error: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*usedSerialsDB).Add:35\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:76\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:459\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:56\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
2020-06-11T15:21:41.662Z INFO piecestore download started {“Piece ID”: “QM5E6G5R6FX5QJGNUZUPKD5FROEFYMBKTGVOVQRWEZYYXWC37NTA”, “Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “Action”: “GET_AUDIT”}
2020-06-11T15:21:41.674Z ERROR piecestore download failed {“Piece ID”: “QM5E6G5R6FX5QJGNUZUPKD5FROEFYMBKTGVOVQRWEZYYXWC37NTA”, “Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “Action”: “GET_AUDIT”, “error”: “usedserialsdb error: disk I/O error: input/output error”, “errorVerbose”: “usedserialsdb error: disk I/O error: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*usedSerialsDB).Add:35\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:76\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:459\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:56\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
2020-06-11T20:31:49.479Z INFO piecestore download started {“Piece ID”: “BTD7MKCPTKZCBARPDMWZG6YXPJZ577LIQVBQAMILLJM5B6ZVGGJQ”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “GET_AUDIT”}
2020-06-11T20:31:49.491Z ERROR piecestore download failed {“Piece ID”: “BTD7MKCPTKZCBARPDMWZG6YXPJZ577LIQVBQAMILLJM5B6ZVGGJQ”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “GET_AUDIT”, “error”: “usedserialsdb error: disk I/O error: input/output error”, “errorVerbose”: “usedserialsdb error: disk I/O error: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*usedSerialsDB).Add:35\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:76\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:459\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:56\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
2020-06-11T22:01:04.505Z INFO piecestore download started {“Piece ID”: “TGXWEFUKQMZCBWGNSLG6ZD34OWENEIJI6YN2UNBB5UEGPPD7JLKQ”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET_AUDIT”}
2020-06-11T22:01:04.517Z ERROR piecestore download failed {“Piece ID”: “TGXWEFUKQMZCBWGNSLG6ZD34OWENEIJI6YN2UNBB5UEGPPD7JLKQ”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET_AUDIT”, “error”: “usedserialsdb error: disk I/O error: input/output error”, “errorVerbose”: “usedserialsdb error: disk I/O error: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*usedSerialsDB).Add:35\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:76\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:459\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:56\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
2020-06-12T03:55:14.382Z INFO piecestore download started {“Piece ID”: “KKQRQVIOK3LKVVFQ55KZGOQUDT5XJJLU3XUSI65LT4ASJNHGZPYA”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “GET_AUDIT”}
2020-06-12T03:55:14.391Z ERROR piecestore download failed {“Piece ID”: “KKQRQVIOK3LKVVFQ55KZGOQUDT5XJJLU3XUSI65LT4ASJNHGZPYA”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “GET_AUDIT”, “error”: “usedserialsdb error: disk I/O error: input/output error”, “errorVerbose”: “usedserialsdb error: disk I/O error: input/output error\n\tstorj.io/storj/storagenode/storagenodedb.(*usedSerialsDB).Add:35\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:76\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:459\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:1004\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:107\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:56\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:111\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:62\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:99\n\tstorj.io/drpc/drpcctx.(*Tracker).track:51”}
Do I have to do/fix something or just wait ?
This is the reason your node is suspended. You need to fix it before your node is disqualified. What did the test return for disk error ?
I think I move it accidentally when I was fixing/testing the setup and the USB cable got disconnected
didn’t found errors on the hard drive 
Apparently I no longer have the node suspended
it is normal on the node stat that the “Last contact” its negative ? shouldn’t it be positive?
Thanks
Yes it should be positive. Try restarting your setup once.
I reboot and like 5 minutes before it start to show
LAST CONTACT
-419m ago
Is it showing ONLINE now ?
always show it as online, even if it say LAST CONTACT : -420
Could you show the last 20 lines of the log?
2020-06-13T17:30:20.505Z INFO piecestore upload canceled {“Piece ID”: “VZ32O6Y55ZJVDDQYBMMNWQVIRPCJZRR4IXBRR5R3VYEMJLS2UXSA”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “PUT_REPAIR”}
2020-06-13T17:31:08.092Z INFO piecestore upload started {“Piece ID”: “RHEHENPIH7PXAGLMSCY722M76A3LQLS25WEBL2HC36SBJJBGRS5Q”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Action”: “PUT”, “Available Space”: 920250585344}
2020-06-13T17:31:08.609Z INFO piecestore upload canceled {“Piece ID”: “RHEHENPIH7PXAGLMSCY722M76A3LQLS25WEBL2HC36SBJJBGRS5Q”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Action”: “PUT”}
2020-06-13T17:31:17.244Z INFO piecestore upload started {“Piece ID”: “AL2E56VURJ345BU4WMTFELN2AZPZJSXARXAEEIJ7HCK7USRUWD3A”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “PUT”, “Available Space”: 920250578176}
2020-06-13T17:31:17.371Z INFO piecestore upload canceled {“Piece ID”: “AL2E56VURJ345BU4WMTFELN2AZPZJSXARXAEEIJ7HCK7USRUWD3A”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “PUT”}
2020-06-13T17:31:20.121Z INFO piecedeleter deleted {“Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Piece ID”: “AL2E56VURJ345BU4WMTFELN2AZPZJSXARXAEEIJ7HCK7USRUWD3A”}
2020-06-13T17:31:25.885Z INFO piecestore upload started {“Piece ID”: “IL2BDDUWC4SY237DEDHAMXEC22NX5CFQJTJCL44TTRVRE7VUWAHQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Available Space”: 920250578176}
2020-06-13T17:31:26.014Z INFO piecestore upload canceled {“Piece ID”: “IL2BDDUWC4SY237DEDHAMXEC22NX5CFQJTJCL44TTRVRE7VUWAHQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”}
2020-06-13T17:31:59.050Z INFO piecestore upload started {“Piece ID”: “EAYCLQY4E3OB7GTUFGFPBPKUQJMGE3N37ROPGUZDHPV3DXNX7I7A”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Action”: “PUT”, “Available Space”: 920250575616}
2020-06-13T17:32:06.540Z INFO piecestore upload canceled {“Piece ID”: “EAYCLQY4E3OB7GTUFGFPBPKUQJMGE3N37ROPGUZDHPV3DXNX7I7A”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Action”: “PUT”}
2020-06-13T17:32:26.900Z INFO piecestore upload started {“Piece ID”: “PKOT65UPIX3QG77LIZFPTF4IAPXET2K3VOBRIIPYIO4MWQ57D2WA”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “PUT_REPAIR”, “Available Space”: 920248255744}
2020-06-13T17:32:40.142Z INFO piecestore upload canceled {“Piece ID”: “PKOT65UPIX3QG77LIZFPTF4IAPXET2K3VOBRIIPYIO4MWQ57D2WA”, “Satellite ID”: “1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE”, “Action”: “PUT_REPAIR”}
2020-06-13T17:33:19.648Z INFO piecestore upload started {“Piece ID”: “6D4XKC4GUK2YMU6QZTXXLY4BWIZB54UTEXT7KK474JOAUU2PYAQQ”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “PUT”, “Available Space”: 920245935872}
2020-06-13T17:33:19.751Z INFO piecestore upload canceled {“Piece ID”: “6D4XKC4GUK2YMU6QZTXXLY4BWIZB54UTEXT7KK474JOAUU2PYAQQ”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “PUT”}
2020-06-13T17:33:56.135Z INFO piecestore upload started {“Piece ID”: “PUQVZNS2RT6ASH2UCCEY4XWVUZCICLG6UUJFUD24YMN475C7GRFA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Available Space”: 920245934080}
2020-06-13T17:34:02.186Z INFO piecestore upload canceled {“Piece ID”: “PUQVZNS2RT6ASH2UCCEY4XWVUZCICLG6UUJFUD24YMN475C7GRFA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”}
2020-06-13T17:34:12.750Z INFO piecestore upload started {“Piece ID”: “723KVIA5KAU6WM2HSAUKXQFPVJ77IDJ4CV2L3IGJH2L32367MZKA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”, “Available Space”: 920243614208}
2020-06-13T17:34:14.590Z INFO piecestore upload canceled {“Piece ID”: “723KVIA5KAU6WM2HSAUKXQFPVJ77IDJ4CV2L3IGJH2L32367MZKA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “PUT”}
2020-06-13T17:34:29.412Z INFO piecestore upload started {“Piece ID”: “A2DFJMAB62X4QZJDZ7ZS4SAEZKDQ6E5DKZ3SIPHLSAOVTNPG7B4A”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Action”: “PUT”, “Available Space”: 920242889472}
2020-06-13T17:34:29.527Z INFO piecestore upload canceled {“Piece ID”: “A2DFJMAB62X4QZJDZ7ZS4SAEZKDQ6E5DKZ3SIPHLSAOVTNPG7B4A”, “Satellite ID”: “12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs”, “Action”: “PUT”}