Wrong answer, sorry. If file size was the most important concern, we would all disable logging, because why bother.
And that is why there are levels of verbosity so as to limit file size for both search time and drive space. It is also wasting I/O and processing time if the additional info isn’t being used for anything.
Disabling logs completely would hinder your ability to examine common errors and node status issues after the issue happened.
I think you know all this though and seem to be wanting to argue about it for some reason… I’m not really looking to do that.
1 Like
I use WARN level, it shows errors only. If something happen I see much better whats the problem, and logs are much smaller, no need to clear it for long time.
Think you could take a look at @Mircoxi PR and maybe give him some pointers? Thanks for your feedback on the issue!
I use “fatal” for less writes on my system ssd.
Amount of writes is so minuscule, that it does not amount to anything if you worry about SSD wear. And if you use windows — there are so much more unnecessary writes going on with their diag, telemetry, and un-disable-able tracing that storj logs are drownin in all that noise.
I store info level logs for everything. Reason — cheap insurance. Think about this this way: say, hypothetically, tomorrow your node is disqualified for losing data. You lose your held amount.
But if you had logs you could, for example, find out that the pieces you “lost” were explicitly deleted at satellite’s request weeks before. So, it’s a bug on satellite. And now instead of losing held amount you report the bug and get a big thanks from the team.
Anyway. Throwing away data to “save” SSD is misguided. Did not you buy it to store data;). It’s like buying a car and not driving it to avoid wear…
3 Likes
But you’re close to the right answer now:
Disabling logs completely would hinder your ability to examine common errors and node status issues after the issue happened.
There’s just one little step from this statement to actually answer the question correctly. Maybe you do want to try? I don’t bite.
Im with you. using fatal atm. if errors pop up, i go back to error, no node with info for me. 24gb log within 2Y are insane. minecraft server logs by day and make every day an zip file, only the aktual an latest.log. history easy to delete, files are not locked, could delete everything older than 6 month automaticaly. why not something like this, i know logrotate, but im not in the mood to do the reading and set it up.
.
Then… don’t ?! Why are you complaining then?
The principle is: one job — one app.
Don’t want logs in one clump — there is logrotate. Do you want storj to re-implement logrotate in storagenode? Do you really think this would be a productive use of their time?
And by the way, 24Gb of uncompressed text in two years is not much at all.
Ultimately, it does not matter what log verbosity is. Just save everything. And rotate into archived segments. With logrotate. Don’t reinvent the wheels.
Not everyone wants to run info.
The fact that the log verbosity is so horrendous, AND storj removed log rotate makes it hilarious.
Obviously some people have a hard time managing their logs, but ultimately managing your logs should be convenient, whether that’s info level or error level. People in info level shouldnt be given only errors, just like people in error shouldn’t be getting info logs(context canceled).
it was not implemented. And I still think that should not be done by the application. You have a logrotate or you may use scripts. Setup once and it’s done.
1 Like
Storagenode needs to handle SIGHUP. Today it quits on it (!?). Instead, it shall reopen logs.
As a workaround I have to send logs to stdout and run storagenode under daemon utility to suppprt log redirection and SIGHUP. It’s an extra unnecessary running process on the system.
If storj official recommendation is to use logrotate, then the SIGHUP shall be properly handled.
Here is more information: signals - Why is kill -HUP used in logrotate in RHEL? Is it necessary in all cases? - Unix & Linux Stack Exchange
1 Like
I would suggest to open an issue on our GitHub: Issues · storj/storj · GitHub
However, as far as I know it’s not common behavior to handle HUP signal for closing log file, some applications handle it, some just restart or ignore or quit.
You may also configure logrotate to use copy/truncate instead. So, it will copy the log, and then truncate the existing one. This is a usual behavior for most of programs and likely will work independently of used OS.
1 Like
But the is additional, avoidable IO…
I’ll file an issue on GitHub.
Seems there’s a common issue with all new commits right now that might be blocking it - I’ve identified them as starting with a commit a few days ago that changed a database table, which makes sense since the test failures are all to do with that table. I left a comment in the PR discussion letting them know, but I don’t know if any of the team have gotten around to taking a look yet.
But, on the bright side - I finally figured out how to read the logs from the test suite, and I’m happy to confirm that the context canceled stuff are definitely only showing as Info level in them! As soon as it’s merged, our SSDs will all live a little longer 
1 Like
Using copytruncate on a busy node will likely cause the loss of some log lines, specially if the storage is slow due to the heavy activity. (I have noticed loss up to 30 seconds).
There is a warning right there in the man page
copytruncate
Truncate the original log file in place after creating a copy,
instead of moving the old log file and optionally creating a new
one, It can be used when some program can not be told to close
its logfile and thus might continue writing (appending) to the
previous log file forever. Note that there is a very small time
slice between copying the file and truncating it, so some log-
ging data might be lost. When this option is used, the create
option will have no effect, as the old log file stays in place.
Maybe that’s OK for some people, but you need to be aware. The right solution is if the daemon handle SIGHUP
2 Likes
Then please submit an issue to let developers know about this problem, or submit a PR to fix it - we are glad to accept the Community contribution!
Here is the issue: Storagenode to handle SIGHUP for interoperation with logrotate. · Issue #6171 · storj/storj · GitHub.
3 Likes