Storj Terminal Dashboard

I’m going to look into it after the weekend. I have to figure something out, to troubleshoot it on my end on a Windows setup. My storj setup is currently running on Linux.

2 Likes

Thanks man - let me know if you get stuff working on windows :slight_smile:

I did a big overhaul of the entire script and changed the way how it parses the log files, as well as the earnings output.

Firstly, I think I fixed @st99ab issue. Windows uses another charmap by default compared to Unix, which was not able to parse the special characters used by the earnings calculator. I forced the output to be decoded to UTF-8, which should fix the bug.
I did not test this myself in a Windows environment, so please keep me updated, if it works now.

Secondly, I think I also fixed @HGPlays issue. Similarly to the first mentioned bug, Windows is not able to decode my way of coloring the strings. I changed it to work regardless of the OS.

But most importantly: You need to adjust your storj_dashboard.json files! The script is now able to read all the logs in given directory. This adds support for multiple rotated log files. Change your log file specification so that it shows the path to the log files instead. (see github, if you are unsure). Unfortunately, this does not work with compressed files. I have planned to add support for compressed files in the future.

Please remember that the script is still being actively worked on and may therefore contain errors or simply not work for you. Continue to report your issues and I will try to fix them.

2 Likes

i gave it another go:

still seems there is issues or maybe im doing something wrong.

The only real question remaining so far for me is who will eventually win in the battle of the author vs this code:

The code is winning with the score 3:0 so far.

1 Like

Yeah, your config still has the path to the log file itself in it. You need to change it to the directory of the logs instead.
In your case you need to change
Node 01: [M:/storj1/logs/storagenode.log", "M/storj1/database"]
to
Node 01: [M:/storj1/logs/", "M/storj1/database"]

I also suspect, that the earnings calculator is still not parsed correctly. I added some additional error logging in the most recent version. Please update and send new screenshots.

new try:

cmd:

Why all your route deviders are / windows use \ in path
in json I mean.

I Will try that :slight_smile:

More Words

i dont belive this was the cause - if i change to \ i get errors in the script.

1 Like

Then you likely need to use \\ instead…

Thanks - will try :smiley:
more words

its getting better but still not working fully.

It doesnt appear to recognize the filewalker anymore. I have a node that its definitely running, but the dashboard shows 0 running. Any info I can provide to help? GC and TC are recognized.

lrrrac@bulldops ~> std 02
02
[INFO] Processing Node 02
[ OK ] /storj2/2/logs/storj.log read
[ OK ] Running earnings calculator for Node 02

═══ Node 02 - Detailed information

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ NODE MAIN STATS β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ FILEWALKER ────────────────┐
β”‚ β”‚β”‚ β”‚
β”‚ Current Total: 0.77 $ Uptime: 0d 3h β”‚β”‚ GARBAGE TRASH USED SPACE β”‚
β”‚ Estimated Total: 2.08 $ β”‚β”‚ COLLECTOR CLEANUP FILEWALKER β”‚
β”‚ β”‚β”‚ β”‚
β”‚ Disk Used: 1.56 TB β”‚β”‚ SL unknown unknown unknown β”‚
β”‚ Unpaid Data: 310.53 GB β”‚β”‚ AP1 unknown unknown unknown β”‚
β”‚ β”‚β”‚ EU1 unknown unknown unknown β”‚
β”‚ β”‚β”‚ US1 unknown unknown unknown β”‚
β”‚ β”‚β”‚ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

═════════════════════════════════════════ All Nodes - Summary ═════════════════════════════════════════

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ NODE MAIN STATS β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ FILEWALKER ────────────────┐
β”‚ β”‚β”‚ β”‚
β”‚ Current total: 0.77 $ β”‚β”‚ GARBAGE TRASH USED SPACE β”‚
β”‚ Estimated total: 2.08 $ β”‚β”‚ COLLECTOR CLEANUP FILEWALKER β”‚
β”‚ β”‚β”‚ β”‚
β”‚ Disk used: 1.56 TB β”‚β”‚ SL 0 running 0 running 0 running β”‚
β”‚ Unpaid Data: 310.53 GB β”‚β”‚ AP1 0 running 0 running 0 running β”‚
β”‚ β”‚β”‚ EU1 0 running 0 running 0 running β”‚
β”‚ β”‚β”‚ US1 0 running 0 running 0 running β”‚
β”‚ β”‚β”‚ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
lrrrac@bulldops ~> cat /storj2/2/logs/storj.log | grep used-space-filewalker
2024-07-12T09:56:27-04:00 INFO pieces used-space-filewalker started {β€œProcess”: β€œstoragenode”, β€œSatellite ID”: β€œ12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”}
lrrrac@bulldops ~> cat /storj2/2/logs/storj.log | grep Config
2024-07-10T10:54:59-04:00 INFO Configuration loaded {β€œProcess”: β€œstoragenode”, β€œLocation”: β€œ/app/config/config.yaml”}
2024-07-11T09:25:08-04:00 INFO Configuration loaded {β€œProcess”: β€œstoragenode”, β€œLocation”: β€œ/app/config/config.yaml”}
2024-07-11T09:25:27-04:00 INFO Configuration loaded {β€œProcess”: β€œstoragenode”, β€œLocation”: β€œ/app/config/config.yaml”}
2024-07-12T09:56:24-04:00 INFO Configuration loaded {β€œProcess”: β€œstoragenode”, β€œLocation”: β€œ/app/config/config.yaml”}

Do you have altered your log format in any way in your storj config? Or have you disabled the lazy filewalkers? Because my script seaches for the string β€œsubprocess started” in conjunction with the various file-walkers (β€œused-space-filewalker”). Since your log only states β€œused-space-filewalker started” (β€œsubprocess” missing), my script doesn’t work.

Alexey, can you confirm, if not using lazy filewalkers alters the log output in any way? Or something else causing the log to omit the substring β€œsubprocess” to be missing?

2 Likes

I have disabled lazy. Is there a way to make it not dependent on lazy being true or false?

Damn, I never tested my script with lazy filewalkers being disabled. My script there does not support non-lazy-filewalker yet. I will support this eventually, but for now you cannot use my dashboard, unfortunately.

Thanks for finding this bug.

1 Like

Is it necessary to search for β€œsubprocess”?
(Works fine with lazy enabled)

Just to verify; any default-ish configuration will not work with your script?
For example; I have fairly standard docker config with no explicit flag set to enable lazy-filewalker. I assumed that by default its disabled.

Lazy-filewalker is enabled by default. My script should work, if you haven’t touched your config.

1 Like