Hello, same problem here. All of a sudden two satellites suspended my node… I have been checking last time maybe two hours ago and everything was fine.
I ran the successrate script and here is the result (which doesn’t look that bad…):
/home/storj/STORJ/bin/successrate.sh storagenode.log
LOG: storagenode.log
========== AUDIT ==============
Critically failed: 0
Critical Fail Rate: 0.000%
Recoverable failed: 0
Recoverable Fail Rate: 0.000%
Successful: 56
Success Rate: 100.000%
========== DOWNLOAD ===========
dl_canratio: 10.978%
Failed: 510
Fail Rate: 2.873%
Canceled: 1949
Cancel Rate: 10.978%
Successful: 15295
Success Rate: 86.150%
========== UPLOAD =============
Rejected: 0
Acceptance Rate: 100.000%
---------- accepted -----------
Failed: 6
Fail Rate: 0.011%
Canceled: 56
Cancel Rate: 0.104%
Successful: 53931
Success Rate: 99.885%
========== REPAIR DOWNLOAD ====
Failed: 41
Fail Rate: 1.310%
Canceled: 0
Cancel Rate: 0.000%
Successful: 3089
Success Rate: 98.690%
========== REPAIR UPLOAD ======
Failed: 0
Fail Rate: 0.000%
Canceled: 2
Cancel Rate: 0.036%
Successful: 5538
Success Rate: 99.964%
========== DELETE =============
Failed: 0
Fail Rate: 0.000%
Successful: 14812
Success Rate: 100.000%
This is confirmed by looking into storagenode.log:
grep -E "GET_AUDIT" storagenode.log|grep 12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs
2022-06-13T01:50:08.605+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "L4IWHSLBSMK6NWCP7MD6EFDAJ5UTA42M433FX7GQYKSFK2V4JV7Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T01:50:08.707+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "L4IWHSLBSMK6NWCP7MD6EFDAJ5UTA42M433FX7GQYKSFK2V4JV7Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T02:00:15.063+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "KLJZXUNBXO7L5JFGSH2A2W2ESJLGRVPQ6PQSOXZORNNYOSHAMBQA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T02:00:15.179+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "KLJZXUNBXO7L5JFGSH2A2W2ESJLGRVPQ6PQSOXZORNNYOSHAMBQA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T07:37:09.589+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "NZJW2CEB7HHBDWBORD5OLGXQNN4LEDSXJD5HGYZVPZEGAED7JKNA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T07:37:09.662+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "NZJW2CEB7HHBDWBORD5OLGXQNN4LEDSXJD5HGYZVPZEGAED7JKNA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T09:50:41.330+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "ACJ4K7KOTV7BDBOHJN3GFRZHHUWRI3F5CE3B2AW5QIXC24IFQ4VQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T09:50:41.587+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "ACJ4K7KOTV7BDBOHJN3GFRZHHUWRI3F5CE3B2AW5QIXC24IFQ4VQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T11:22:16.236+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "RHAOJUL4U3EYFQZ5DXOSBQROYRQUZUD6PYXTXR23BDWJUSTIOIPQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T11:22:16.482+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "RHAOJUL4U3EYFQZ5DXOSBQROYRQUZUD6PYXTXR23BDWJUSTIOIPQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T11:40:47.459+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "Q3CFTAORTA5FYUIBMRY2Z3T35CAS45FWDEAZR5TE6JEDNMFX2YWQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T11:40:48.291+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "Q3CFTAORTA5FYUIBMRY2Z3T35CAS45FWDEAZR5TE6JEDNMFX2YWQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T14:16:40.094+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "IXF3VO6VXKO6MSDGJ4EUFD6V54V64MNPKTRREEP2IGZ26X6W5EYA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T14:16:40.252+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "IXF3VO6VXKO6MSDGJ4EUFD6V54V64MNPKTRREEP2IGZ26X6W5EYA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T14:46:41.411+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "YCNLXBN5HHST5DYZMRADMFAMAYNWTDGOAS3RNRGIFPGVUPATWWQQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T14:46:41.652+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "YCNLXBN5HHST5DYZMRADMFAMAYNWTDGOAS3RNRGIFPGVUPATWWQQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T17:25:13.190+0200 INFO piecestore download started {"Process": "storagenode", "Piece ID": "SDVCAGWZQEL6PBPXIWFLA5WDTINVS4EECE7ITZMN4IIGCULRV46A", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2022-06-13T17:25:13.323+0200 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "SDVCAGWZQEL6PBPXIWFLA5WDTINVS4EECE7ITZMN4IIGCULRV46A", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}