Node Suspension

Kinda jealous not gonna lie, feel a bit left out here.

You prolly left Littleskunk confused asking for email on other thread while he was schooling that kid.

1 Like

LOL I know but seriously wheres mine I actually did something right… So much aggression today need to lighting up theres worse things in life…

:arrow_down:

:nerd_face:

1 Like

First of all log rotate is an option for windows as well

Second, log rotate is able to deal with locked files. It can still rotate them.

1 Like

To add to that on another node which didnt receive an email (probably because those are turned off now) seems to be suspended as well due to the database locked bug, as the dbs are fine when running

"PRAGMA integrity_check;"

but unlike the first node this one doesnt receive any traffic anymore

Yes for system logs. but all the verbose ingress / egress exchange is useless past 24/48h imo.

But ok i get it it’s my responsability to deal with it

Here is my storj node logrotate config file:

/opt/storj/node.log
{
   rotate 12
   monthly
   missingok
   notifempty
   compress
   copytruncate
}

It keeps 12 months of logs in compressed format.

1 Like

@brandon thanks for the update. I was banging my head to find out how to resolve the issue. If you were to send these emails to someone in the future, please do let the node operator know how to resolve their issue. I am running on node version 1.1.1 and the gui did not show any alerts about suspensions.

1 Like

Okay. i have been following this thread for 16 hours as I received my suspended emails 20 hours ago. I am trying to be patient but I am not receiving data anymore because of this. I have grepped 2 days of logs and i have not failed any audits. My node ID is: 18tRN3muj4H1GizFNpaVtftWuWwrZ1xXDG2r9FQJKqyvhTUjLW Can someone please check and mark it for resuming normal activity?

Logs are posted below and show no error messages. From what I read, it should resume normal activity after a few successful audits…but i have not seen that and it’s been almost 20 hours.

Thank you!

nlitend1

2020-04-23T20:46:08.677Z INFO Configuration loaded from: /app/config/config.yaml

2020-04-23T20:46:08.718Z INFO Operator email: XXX@gmail.com

2020-04-23T20:46:08.719Z INFO operator wallet: XXX

2020-04-23T20:46:10.353Z INFO version running on version v1.1.1

2020-04-23T20:46:14.037Z INFO db.migration Database Version {“version”: 33}

2020-04-23T20:46:18.368Z INFO preflight:localtime start checking local system clock with trusted satellites’ system clock.

2020-04-23T20:46:19.169Z INFO preflight:localtime local system clock is in sync with trusted satellites’ system clock.

2020-04-23T20:46:19.170Z INFO bandwidth Performing bandwidth usage rollups

2020-04-23T20:46:19.170Z INFO Node 18tRN3muj4H1GizFNpaVtftWuWwrZ1xXDG2r9FQJKqyvhTUjLW started

2020-04-23T20:46:19.170Z INFO Public server started on [::]:28967

2020-04-23T20:46:19.170Z INFO Private server started on 127.0.0.1:7778

2020-04-23T20:46:19.170Z INFO trust Scheduling next refresh {“after”: “7h54m8.087281627s”}

2020-04-23T20:46:19.332Z INFO version running on version v1.1.1

2020-04-23T20:46:23.094Z INFO piecestore download started {“Piece ID”: “YREVMBQOCFIU7L7SJGH6LCKSAGV5QH4XSJMKM7CJCZZBOMN4TWWA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”}

2020-04-23T20:47:15.163Z INFO piecestore download started {“Piece ID”: “OZ4M2S4DLB4DDTAE6XSNWOLSPLX73EGIYMHGIJUZWVUDYBUNNN4Q”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET_AUDIT”}

2020-04-23T20:47:28.751Z INFO piecestore download started {“Piece ID”: “ZVIORB6V5ZRM2ZWGNV7DALNDDOXL32OOO2NTELVPXBAFVO65I4YA”, “Satellite ID”: “121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6”, “Action”: “GET”}

2020-04-23T20:47:28.920Z INFO piecestore download started {“Piece ID”: “UPRFVRK4IKHFC2BWMSROQGZQ7FQP2JBXSLQ7M4QFCZO5KJMNUW6Q”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET”}

2020-04-23T20:48:01.408Z INFO piecestore download started {“Piece ID”: “UPRFVRK4IKHFC2BWMSROQGZQ7FQP2JBXSLQ7M4QFCZO5KJMNUW6Q”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET”}

2020-04-23T20:48:15.405Z INFO piecestore download started {“Piece ID”: “B7JLVGUOVGFRPC5NNSPRDK7FH47BQPSRRCOOP6LGLAC5FFJNIZFA”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET”}

2020-04-23T20:48:18.030Z INFO piecestore download started {“Piece ID”: “YV5IUUOCLI7EBW726JZNPS4Z747GTK3NMBPXFWC344YUBDM2QUYA”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET”}

2020-04-23T20:48:24.321Z INFO piecestore download started {“Piece ID”: “FXJHOWEYO5ZAQOEHQHDNKDGPT2JBBGQKX75AY4WZECXULFDCI7RA”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET”}

2020-04-23T20:48:26.287Z INFO piecestore download started {“Piece ID”: “AHICSGE5TZE4EREZNKFVQ34NTTI23MWUGBZSGCHQLSVDVD2N5QBQ”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”}

2020-04-23T20:48:33.255Z INFO piecestore download started {“Piece ID”: “UPRFVRK4IKHFC2BWMSROQGZQ7FQP2JBXSLQ7M4QFCZO5KJMNUW6Q”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET”}

2020-04-23T20:48:45.948Z INFO piecestore download started {“Piece ID”: “B7JLVGUOVGFRPC5NNSPRDK7FH47BQPSRRCOOP6LGLAC5FFJNIZFA”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET”}

2020-04-23T20:48:50.715Z INFO piecestore download started {“Piece ID”: “YV5IUUOCLI7EBW726JZNPS4Z747GTK3NMBPXFWC344YUBDM2QUYA”, “Satellite ID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “GET”}

Is your node full? Is that why your not getting data?

Nope. Over 750GB free. I was getting close to 250GB a day and now 0. :frowning:

But the good thing, you have paid downloads

Even if i’m not getting “downloaded” messages in the logs? I’ve noticed all “download started” but none of the “downloaded”.

This looks like a network connected drive issue.
How is your HDD connected to the PC with storagenode?

My node has been suspended on asia-east and us-central but I do not understand why.
It seems to be related to an audit regarding to the messages I have received but the node has not been down for more than 200 hours.

How can I know what’s happened ?

Thank you

Follow here:

Hi,

Node : 1mB1fgDmophpSY2BEutEPVR9S4pZ6LC7P4Q1gGpEWbpE3uGhmg

Yesterday, i received 2 emails saying my node is suspended on 2 satellites, checked my node. Did not found any issue. 100% audit rate. Restarted the whole thing and updated the OS in one shot, then hours later got 2 others emails.

So far it is suspendend on:

europe-west-1
us-central-1
saltlake
asia-east-1

Any clue please ?

Welcome to the forum @buchette!

Try this thread

1 Like

Did not expect this. As far as I know the storage is fine because I use it daily. Machine runs 24/7. I know for the last few days there has been some internet connection issues.

Your Storage Node on the asia-east-1 Satellite was suspended. You were suspended on 2020-04-22 at 21:37 UTC.

You won’t receive any new data on this Satellite until you resolve the issue causing audit failures on your node.

EDIT: Upon looking at the log, it looks like I am seeing a ton of database locked errors.
So I just signed into my modem to check on my port forwards and looks like now the ISP took that over now.

UPDATE: Just got this email now, another satellite suspends me. Your Storage Node on the europe-west-1 Satellite was suspended. You were suspended on 2020-04-23 at 08:29 UTC.
Your Storage Node on the us-central-1 Satellite was suspended. You were suspended on 2020-04-23 at 08:33 UTC.

Guess this figures out why. I disable there advance security features, unsure if it is still blocking the request regardless on the setting I put.