Iβm going to look into it after the weekend. I have to figure something out, to troubleshoot it on my end on a Windows setup. My storj setup is currently running on Linux.
Thanks man - let me know if you get stuff working on windows ![]()
I did a big overhaul of the entire script and changed the way how it parses the log files, as well as the earnings output.
Firstly, I think I fixed @st99ab issue. Windows uses another charmap by default compared to Unix, which was not able to parse the special characters used by the earnings calculator. I forced the output to be decoded to UTF-8, which should fix the bug.
I did not test this myself in a Windows environment, so please keep me updated, if it works now.
Secondly, I think I also fixed @HGPlays issue. Similarly to the first mentioned bug, Windows is not able to decode my way of coloring the strings. I changed it to work regardless of the OS.
But most importantly: You need to adjust your storj_dashboard.json files! The script is now able to read all the logs in given directory. This adds support for multiple rotated log files. Change your log file specification so that it shows the path to the log files instead. (see github, if you are unsure). Unfortunately, this does not work with compressed files. I have planned to add support for compressed files in the future.
Please remember that the script is still being actively worked on and may therefore contain errors or simply not work for you. Continue to report your issues and I will try to fix them.
The only real question remaining so far for me is who will eventually win in the battle of the author vs this code:
The code is winning with the score 3:0 so far.
Yeah, your config still has the path to the log file itself in it. You need to change it to the directory of the logs instead.
In your case you need to change
Node 01: [M:/storj1/logs/storagenode.log", "M/storj1/database"]
to
Node 01: [M:/storj1/logs/", "M/storj1/database"]
I also suspect, that the earnings calculator is still not parsed correctly. I added some additional error logging in the most recent version. Please update and send new screenshots.
Why all your route deviders are / windows use \ in path
in json I mean.
I Will try that ![]()
More Words
i dont belive this was the cause - if i change to \ i get errors in the script.
Then you likely need to use \\ insteadβ¦
Thanks - will try ![]()
more words
It doesnt appear to recognize the filewalker anymore. I have a node that its definitely running, but the dashboard shows 0 running. Any info I can provide to help? GC and TC are recognized.
lrrrac@bulldops ~> std 02
02
[INFO] Processing Node 02
[ OK ] /storj2/2/logs/storj.log read
[ OK ] Running earnings calculator for Node 02βββ Node 02 - Detailed information
ββββββββββββββββββββ NODE MAIN STATS βββββββββββββββββββββββββββββββββββββ FILEWALKER βββββββββββββββββ
β ββ β
β Current Total: 0.77 $ Uptime: 0d 3h ββ GARBAGE TRASH USED SPACE β
β Estimated Total: 2.08 $ ββ COLLECTOR CLEANUP FILEWALKER β
β ββ β
β Disk Used: 1.56 TB ββ SL unknown unknown unknown β
β Unpaid Data: 310.53 GB ββ AP1 unknown unknown unknown β
β ββ EU1 unknown unknown unknown β
β ββ US1 unknown unknown unknown β
β ββ β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ All Nodes - Summary βββββββββββββββββββββββββββββββββββββββββ
ββββββββββββββββββββ NODE MAIN STATS βββββββββββββββββββββββββββββββββββββ FILEWALKER βββββββββββββββββ
β ββ β
β Current total: 0.77 $ ββ GARBAGE TRASH USED SPACE β
β Estimated total: 2.08 $ ββ COLLECTOR CLEANUP FILEWALKER β
β ββ β
β Disk used: 1.56 TB ββ SL 0 running 0 running 0 running β
β Unpaid Data: 310.53 GB ββ AP1 0 running 0 running 0 running β
β ββ EU1 0 running 0 running 0 running β
β ββ US1 0 running 0 running 0 running β
β ββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
lrrrac@bulldops ~> cat /storj2/2/logs/storj.log | grep used-space-filewalker
2024-07-12T09:56:27-04:00 INFO pieces used-space-filewalker started {βProcessβ: βstoragenodeβ, βSatellite IDβ: β12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3Sβ}
lrrrac@bulldops ~> cat /storj2/2/logs/storj.log | grep Config
2024-07-10T10:54:59-04:00 INFO Configuration loaded {βProcessβ: βstoragenodeβ, βLocationβ: β/app/config/config.yamlβ}
2024-07-11T09:25:08-04:00 INFO Configuration loaded {βProcessβ: βstoragenodeβ, βLocationβ: β/app/config/config.yamlβ}
2024-07-11T09:25:27-04:00 INFO Configuration loaded {βProcessβ: βstoragenodeβ, βLocationβ: β/app/config/config.yamlβ}
2024-07-12T09:56:24-04:00 INFO Configuration loaded {βProcessβ: βstoragenodeβ, βLocationβ: β/app/config/config.yamlβ}
Do you have altered your log format in any way in your storj config? Or have you disabled the lazy filewalkers? Because my script seaches for the string βsubprocess startedβ in conjunction with the various file-walkers (βused-space-filewalkerβ). Since your log only states βused-space-filewalker startedβ (βsubprocessβ missing), my script doesnβt work.
Alexey, can you confirm, if not using lazy filewalkers alters the log output in any way? Or something else causing the log to omit the substring βsubprocessβ to be missing?
I have disabled lazy. Is there a way to make it not dependent on lazy being true or false?
Damn, I never tested my script with lazy filewalkers being disabled. My script there does not support non-lazy-filewalker yet. I will support this eventually, but for now you cannot use my dashboard, unfortunately.
Thanks for finding this bug.
Is it necessary to search for βsubprocessβ?
(Works fine with lazy enabled)
Just to verify; any default-ish configuration will not work with your script?
For example; I have fairly standard docker config with no explicit flag set to enable lazy-filewalker. I assumed that by default its disabled.
Lazy-filewalker is enabled by default. My script should work, if you havenβt touched your config.





