Troubleshooting with some nodes

Node Operators troubleshooting
1

Hey,

i run several nodes on a Mini-PC via USB on several external harddiscs.
This works fine for around 1,5 years.

The biggest one of the external harddrives have 6 nodes on it.
Configured with the Windows Tool to manage several nodes.

The biggest node on this disc run absolutely fine.
The others are in trouble in the last few days.

Can you read something inside of this example log ?
And can help me to figure out what is wrong, to get them back working fine?
I am just c&p the last few lines for you.

here was originally code of the log file. but this never help anyone. So i deleted it

I found no Programmer Code function,
hopefully my c&p part will be shown pretty :wink:

Can you give me any info, what is wrong on my nodes after more than 1 year?
I touched nothing on this.

Thank you

2

Are they out of space? For the record it is not recommended to run multiple nodes off of a single drive.

2 Likes
3

Thank you for your answer!
“Not recommended bla bla … yawn”" It run for over 1,5 years. So what?
Something seams to change, but what?

Answer:
No, there is enough free space on the disc.
Any other commends of someone who can read and interpret the logs?

4

This is very unprofessional and rude. I highly doubt anyone would be willing to help when you act like this. Be civil and you will get help else search the forum since you can “read”.

5 Likes
5

FYI this is not just some recommendation, but something that node operators have to abide by, as noted in the Node Operator Terms and Conditions, section 4. Node Operator Responsibilities:

  • 4.1.4. You will provide and maintain the Storage Node so that, at all times, it will meet the following minimum requirements (“Minimum Storage Node Requirements”):

  • 4.1.4.1. Have a minimum of one (1) hard drive and one (1) processor core dedicated to each Storage Node;

So please abstain from making dismissive statements about these terms and be respectful to fellow community members and leaders, as required in our Rules of Conduct - These folks are volunteering their time to help others with their issues, please treat them as you would like to be treated.

4 Likes
6

You are right! But please be shure, i dont mean it rude.

It is just a point for me that my system works for so a long time without any problems until now.

7

Ok, also thank you for your information!

I will shut down all my nodes immediately! Maybe i find fast enough the button for a gracefull exit, otherside i shut it down immediately to be conform every rule after 1,5 years.

In absolut respect: Many thanks all of you.
Have a good day and a perfect future

Request is solved (my nodes are down as requested) and thread can be closed.
Thank you for your good tips.

solved

1 Like
8

Did you lost databases recently? Or how much free space do you have on this drive?

9

Hey Alexey,
thank you for your questions.

I do not know how to check for lost databases. I am working on Windows 11.
But the revocations.db file is still in installation folder existing.

The free space might be a point as i figured out some strange and unlogical differences.
For all nodes combined plus 10% over useing free space
there must be total additional 2TB free space.
But there was only 200GB shown free in File-Explorer.

That is absolutely too much difference in total.

Until last night i stopped all of my nodes and run checkdsk -f -r on all hard drives.
Crossing my fingers that this helps.

10

There is an instruction:

Please check them before continue.
The next step would be to check your logs for errors related to any of filewalkers:

11

Hey Alexey,
many, many thanks for your help here.

My given chkdsk will run until tomorrow … That never should be cancelled during work.
So i will wait :wink:

My *.log Files are deleted until now / there is only one example copied.
And other old *.exe files around are cleaned up.

Thank you for your link, i will read and respect em tomorrow after discs are back.

After all, i think it will be a good idea to start one of the nodes only and read the new log …
Just upcoming tomorrow.

Thank you

12

Are you running multiple nodes on the one disk? if so, it’s against Node Operator Terms & Conditions and they are inevitable affecting each other, not much sense in doing so if they are behind the same /24 subnet of public IPs - they would just split the same traffic and get not more than a one node in total. Or do you also break /24 filtering and uses VPN for them? In this case the one drive failure will shutdown them all at once and we can lose the segment, because they now dependent and may have more than one piece for the same segment. Lost segment = lost data = lost customers = lost network = lost payout. Sounds like shooting yourself in the foot.

13

Yes and its not working out so well either.

1 Like
14

And it will be horrible in doing 6 filewalkers almost in the same time, it will takes ages to finish, so likely they never finished and believes that they have more free space than they actually have.
I would say that this is expected outcome and nodes now crashes.

2 Likes
15

Hey Alexey and all the others here,

thank all of you for your commends and infos.
I thing all of you think really absolute to negative.
Your horrible pictures … ^^

All of my nodes are back online.
In the end there only was one node that extremly encrease the trash folder.
The meaning of 10% over-use would be 5GB and not 3TB^^

So this one node was now separated on another hard drive and should be checked the next days what happened.

All around works now fine as in the past.
So thank you expecially Alexey for your really good instructions to cover a problem.

Just for info:
This is just a hard drive for rearing new nodes.
It seems to be time to separate them in the next days to there own drive.

You can think what you want, but the idea so rearing new nodes is absolute ok for me.

All nodes are back online!
I will separate them the next days to there own drives.
Thank all of you for your commends and suggestions.

2 Likes