Native logs rotation in Windows with a simple PowerShell script

Hello,

Unfortunately, in the current windows version of the Storage Node there is no way to rotate or limit the size of the generated logs files. You cannot delete them from the GUI because they are locked by the services processes. Using Clear-Content in PowerShell to reset the files works but there is no easy solution if you want to keep a few days / weeks of logs.

I have written a small PowerShell script that can rotate the Storjs Logs without the need of Stop>Start the services to release the files.

Simply save the code below in a .ps1 file and change the variables to you needs. Then every time you start the script (can be scheduled easily in a task) it will:

  • Look for any *.log files in $PathLogLive

  • Copy all those logs to $PathLogArchive in a new file with a datetime suffix (you can change the format with $DateTimeFormat)

  • Clear the existing logs

  • Delete any *.log file in the to $PathLogArchive that is older that $RententionInDays

It’s been running on my node on a scheduled task for a few days without issues. Hope this helps!

PowerShell source code:

#-----------------------------------------------
#-------------------- INFOS --------------------
#Script:   StorjLogsRotation
#Version:  1.0.1.2
#Date:     27/04/2020
#Author:   JDA
#------------------- Variables ------------------
$PathLogLive      = 'S:\Storj\Storage Node\'
$PathLogArchive   = 'S:\Storj\Storage Node\Logs\'
$DateTimeFormat   = "yyyy-MM-dd HH'h'mm"
$RententionInDays = 8
#-----------------------------------------------


Function StorjLogsRotation{
 ForEach($L in (Get-ChildItem -LiteralPath $PathLogLive -Filter '*.log' -Recurse:$False)){
  RotateLog -Source $L.FullName -DestinationPath $PathLogArchive -RententionInDays $RententionInDays
 }
}


Function RotateLog([String]$Source, [String]$DestinationPath,[Int]$RententionInDays){
 $Format = '{0}_[{1}]{2}'

 If(Test-Path -LiteralPath $Source){
  $S = Get-Item -LiteralPath $Source
  $Destination = Join-Path -Path $DestinationPath -ChildPath ($Format -F $S.BaseName, ((Get-Date).ToString($DateTimeFormat)), $S.Extension)
 
  If(!(Test-Path -LiteralPath $DestinationPath)){$Null = New-Item -Path $DestinationPath -Type Directory -Force}
  Copy-Item -Path $Source -Destination $Destination -Force
  Clear-Content -LiteralPath $Source -Force

  Get-ChildItem -LiteralPath $DestinationPath -File -Filter ($Format -F $S.BaseName, '*',$S.Extension) | ? LastWriteTime -le ((Get-Date).AddDays(-$RententionInDays)) | Remove-Item -ErrorAction SilentlyContinue
 }
}

StorjLogsRotation

PS: Depending on your disk speed, every time the logs are rotated there is a small change you loose a few lines of logs.

If your copy takes too long, wouldn’t the copy contain older values than the current log file. Stopping makes sure no entries are lost.

Yes, it is not impossible, but stopping / restarting the node daily have disadvantages also. It hurt downtime and the node restart is heavy on disk if you store lots of data (probably some kind of health check going on). Logrotate does something similar, so the risk is the same, but unless your disks are really slow the risk is minimal.

For me node uptime and stability are far superior to a few lines of logs that could be missing, I’m just providing an alternative to something that should be implemented in the program. Log rotation is a basic function that any service should implement by himself (and that way it guaranties you don’t lose anything.)

PS: I have updated the post with a warning about it

doing that has zero effect on the log file

You cannot delete the logs without stopping the Services. You can zero them with a command line, but no from the GUI. I have updated the post to make it clearer about the why you could need this script.

Furthermore if a significant number of nodes restarts scheduled at midnight, that could hurt file availability. Please never schedule a restart at set moments.

1 Like

Amen. :slight_smile:

I am doing this by hand, time to time, with my Toolbox.

done by 2 click, and it not delete it only archive

1 Like

i take it that this is related to the windows storagenode software that doesn’t use docker?

Yes only tested on the native service version of Storage Node.
I was usind docker on windows at first but not real reason to continue now that you have a service.

most likely faster that way anyways…

Faster i dont know but it sure take less ressources and it’s way easier to install and maintain. :slight_smile:

Yeah it is only usable with GUI version. I not see any advantage to use docker in windows at all, it like emulate windows on windows.