Fatal message on SN

It will be very useful if you could enable in you configuration file what @Storgeez has commented to you and when that createFile error happens, report here your log output.

As commented above, I probably found the root cause of the fatal error that @Storgeez reported but there is this createFile one that so far we don’t have a clue what’s going on.

Thanks for your collaboration.

1 Like

Is there any ETA for this to continue release (of the next version that fixes the issues, I suppose), or any place we can see the status? I could not find any relevant PR/issue on GitHub :sweat_smile:

There is not, the only information that I have is that we are planning to tackle it this week.
My commit is a small fix, but I will ping today to my mates to see if they can have a gander and hopefully we could merge it before Thursday (it turns that I’m off tomorrow).

Regarding Github, most of us we use Gerrit, my commit is https://review.dev.storj.io/c/storj/storj/+/2129

I suppose it is not easy to give a date, I just wanted to know if you think it would be a matter of days or weeks - I have a Windows node waiting for update to be on the same page as linux to migrate it :smiley:

It’s difficult to estimate because, so far, we are not sure what’s going with the createFile error.
:crossed_fingers: to find the issue during this week and next week we could start to rollout a new version which fixes this problem.

Storj core devs are doing an amazing task building this platform - my intention is not to make pressure on this at all <3

That’s very much appreciated it and we like to hear that :wink:

5 Likes

Thanks for the reply, very much appreciated :slight_smile:

1 Like

I let you know that we have started the rollout of the v1.6.4
The release includes my commit commented in my previous post and a very few more with minor improvements related to this problem (https://github.com/storj/storj/releases/tag/v1.6.4)

So far, we have started the release, which means that the version will be rolled out slowly as we usually do with any release.

Please, keep on eye your nodes and when they get updated, if your node was crashing because of this problem, let us know if the new release has remediated the problem or not.

Last, but not least, I remind you that we are still on the hunt for the createFile error, so,if you can, please configure the options that @Alexey has commented on Fatal message on SN and report back here if the error happens again.

We really appreciate your collaboration.

4 Likes

after i Vacumned databases, this error not acure any more in my nodes, for 3-5 days.
so it can be that some threads just was not synced and sometimes it checked faster than path was made.

Thanks a lot for the update.

The rollout phases are going to be the usual ones? up to 3 days in Windows and 7 days on Docker

The lstat version of the bug that I had occured several times in one day and then not for a week, so it seems pretty random.

Yes, it’s going to be at usual pace which is: Every approximated 24 hours window the cursor increases following this sequence: 5%, 10%, 20%, 40%, 80% and 100%. At the time that the cursor is updated to 100%, right after we release the docker image of the new version.

5 Likes

We have already published the roll out of v1.6.4 to 80% of the Windows nodes.
I’m nudging you here to make sure if with this version your Nodes haven’t crashed with the errors reported in this thread, otherwise, please let us know.

Thanks for your collaboration.

3 Likes

Sofar all fine - but just got the update 2h ago…

1 Like

Short update - second node gut the new version - so one running 34h on it the other 4h. Both ~10TB data.
No issues so far - but traffic load is also very low currently.