Visual Dashboard - Grafana Mon: 24hr Docker log > Telegraf > InfluxDB

With some labor and contribution from @Alexey grep statements, @Kiwwiaq and @BrightSilence 's amazing expansion on my basic raw draft of SuccessRates.sh, I managed to get the data in grafana.

I’ll be uploading the dashboard files, and some instructions on how to spin it up here:

15 Likes

Nice work. I’ve done a PR of what I had to do to make it work with my stack. Could you commit your Grafana dashboard or publish it to Grafana.com?
Thanks!

1 Like

Thanks for the PRs! I will review and accept. I should have the dashboard up when nodes start receiving data again.

I uploaded the dashboard to the github link if you want to check it out.

Also added alerts, you can push it to discord, telegram, SMS or email.

It’s possible for an tuto on sinology please ?
who he can help me please

Yeah, it wasn’t easy, but awesome adventure.
Seems promising
Anyway, it can be also written for windows hosts someday. (with windows’ host environment on a docker or just as a service on the host written in python/powershell/etc.)

2 Likes

Glad you got it going. It’s def not the easiest right now. Depending on the level of detail from the SNOboard I may contribute more to make it easier for all. It’s nice to see another node on it. Thanks for sharing.

1 Like

Another way would be to build a docker image with the configured stack: InfluxDB, Telegraf and Grafana. It would just require a volume to the SNO folder and an Etherscan API token
I might give an eye to that idea by mid-August.

1 Like

That’s what I imagined too. Simple docker compose config and a single command would boot it up in an instant.

Another one: https://github.com/gsxryan/storj_telegraf_mon

1 Like

Why do i dont have any pie diagram on the left side and on the right only a straight line?

If you started it just by now(15 mins) and didn’t change anything in config about intervals and timeout, so just wait about 30 minutes. I had to wait too

It is running more than 24 hours now.

Pie charts don’t appear until you get data. You should see put and get ratios first, your output to influx may not be working from successrates.sh

You can try ‘telegraf --config /path/to/storjtelegraf.conf --test’

Then just see the logs of telegraf and influxdb.
Does the telegraf show any errors executing script?
Does the influxdb show any error getting requests from the telegraf?

Or maybe, influxdb is not configured, and telegraf sends requests to unexisting database(for anybody else, even has such problem)

This are all outputs from telegraf -test with StorJHealth outputs:

StorJHealth,NodeId=1CMJNQ,host=storj-debian Deleted=553,FailedCrit=0,FailedWarn=0,Ratio=100,Success=1223 1564333469000000000
StorJHealth,NodeId=1CMJNQ,host=storj-debian DLFailed=44,DLRatio=99.409,DLStarted=8665,DLSuccess=7398,PUTAcceptRatio=100,PUTFailed=1192,PUTLimit=0,PUTRatio=97.631,PUTStarted=50314,PUTSuccess=49120 1564333469000000000
StorJHealth,NodeId=1CMJNQ,host=storj-debian GETRepairFail=0,GETRepairRatio=100,GETRepairSuccess=0,PUTRepairFailed=0,PUTRepairRatio=100,PUTRepairSuccess=0 1564333469000000000
StorJHealth,NodeId=1CMJNQ,host=storj-debian InfoDBcheck=0,Reboots=0,VoucherCheck=0 1564333469000000000

So i cant find any errors

The Unix epoch timestamp is not correct. That would be Thursday, March 6, 6927 3:28:20 AM. You can try without the $(date) in the script, and allow Telegraf to stamp the time.

See last few post. https://github.com/gsxryan/storj_telegraf_mon/issues/12

So i did it but whitout any luck.

Here my export with “telegraf -input-filter exec -test”:

2019-07-29T06:38:15Z I! Using config file: /etc/telegraf/telegraf.conf
> StorJHealth,NodeId=1CMJNQ,dc=eu-central-1,host=storj-debian Deleted=407,FailedCrit=0,FailedWarn=0,Ratio=100,Success=1244 1564382310000000000
> StorJHealth,NodeId=1CMJNQ,dc=eu-central-1,host=storj-debian DLFailed=52,DLRatio=99.339,DLStarted=9108,DLSuccess=7811,PUTAcceptRatio=100,PUTFailed=413,PUTLimit=0,PUTRatio=99.016,PUTStarted=41961,PUTSuccess=41548 1564382310000000000
> StorJHealth,NodeId=1CMJNQ,dc=eu-central-1,host=storj-debian GETRepairFail=0,GETRepairRatio=100,GETRepairSuccess=0,PUTRepairFailed=0,PUTRepairRatio=100,PUTRepairSuccess=0 1564382310000000000
> StorJHealth,NodeId=1CMJNQ,dc=eu-central-1,host=storj-debian InfoDBcheck=0,Reboots=0,VoucherCheck=0 1564382310000000000
> StorJToken,WalletAddress="0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF",dc=eu-central-1,host=storj-debian,stat=tokens BalanceEUR=0,BalanceSTORJ=0,BalanceUSD=0 1564382310000000000
> StorJToken,dc=eu-central-1,host=storj-debian,stat=prices STORJPriceEUR=0.1554,STORJPriceUSD=0.1728 1564382310000000000
> StorJHealth,dc=eu-central-1,host=storj-debian,path="/DriveShare/StorjNode" directory_size_kilobytes=1118379516 1564382310000000000

And here some latest content in influx:

> select * from StorJHealth
name: StorJHealth
time                DLFailed DLRatio DLStarted DLSuccess Deleted FailedCrit FailedWarn GETRepairFail GETRepairRatio GETRepairSuccess InfoDBcheck NodeId  PUTAcceptRatio PUTFailed PUTLimit PUTRatio PUTRepairFailed PUTRepairRatio PUTRepairSuccess PUTStarted PUTSuccess Ratio Reboots Success VoucherCheck dc           directory_size_kilobytes host         path
----                -------- ------- --------- --------- ------- ---------- ---------- ------------- -------------- ---------------- ----------- ------  -------------- --------- -------- -------- --------------- -------------- ---------------- ---------- ---------- ----- ------- ------- ------------ --           ------------------------ ----         ----
1564326000000000000                                                                                                                                                                                                                                                                                                       1066821092               storj-debian "/DriveShare/StorjNode"
1564326000000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564327800000000000                                                                                                                                                                                                                                                                                                       1069359936               storj-debian "/DriveShare/StorjNode"
1564327800000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564329600000000000                                                                                                                                                                                                                                                                                                       1071764952               storj-debian "/DriveShare/StorjNode"
1564329600000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564331400000000000                                                                                                                                                                                                                                                                                                       1074227496               storj-debian "/DriveShare/StorjNode"
1564331400000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564333200000000000                                                                                                                                                                                                                                                                                                       1076625504               storj-debian "/DriveShare/StorjNode"
1564333200000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564335000000000000                                                                                                                                                                                                                                                                                                       1079044744               storj-debian "/DriveShare/StorjNode"
1564335000000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564336800000000000                                                                                                                                                                                                                                                                                                       1081416844               storj-debian "/DriveShare/StorjNode"
1564336800000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564338600000000000                                                                                                                                                                                                                                                                                                       1083928408               storj-debian "/DriveShare/StorjNode"
1564338600000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564340400000000000                                                                                                                                                                                                                                                                                                       1086222400               storj-debian "/DriveShare/StorjNode"
1564340400000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564342200000000000                                                                                                                                                                                                                                                                                                       1088561724               storj-debian "/DriveShare/StorjNode"
1564342200000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0                                                  storj-debian 
1564344000000000000                                                                                                                                                                                                                                                                                          eu-central-1 1090895940               storj-debian "/DriveShare/StorjNode"
1564344000000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564345800000000000                                                                                                                                                                                                                                                                                          eu-central-1 1093412408               storj-debian "/DriveShare/StorjNode"
1564345800000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564347600000000000                                                                                                                                                                                                                                                                                          eu-central-1 1095756724               storj-debian "/DriveShare/StorjNode"
1564347600000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564349400000000000                                                                                                                                                                                                                                                                                          eu-central-1 1098140588               storj-debian "/DriveShare/StorjNode"
1564349401000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564351200000000000                                                                                                                                                                                                                                                                                          eu-central-1 1100420904               storj-debian "/DriveShare/StorjNode"
1564351201000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564353000000000000                                                                                                                                                                                                                                                                                          eu-central-1 1102755100               storj-debian "/DriveShare/StorjNode"
1564353001000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564354800000000000                                                                                                                                                                                                                                                                                          eu-central-1 1104953460               storj-debian "/DriveShare/StorjNode"
1564354801000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564356600000000000                                                                                                                                                                                                                                                                                          eu-central-1 1107282168               storj-debian "/DriveShare/StorjNode"
1564356601000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564358400000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564358400000000000                                                                                                                                                                                                                                                                                          eu-central-1 1109644416               storj-debian "/DriveShare/StorjNode"
1564360200000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564360200000000000                                                                                                                                                                                                                                                                                          eu-central-1 1112037016               storj-debian "/DriveShare/StorjNode"
1564362000000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564362000000000000                                                                                                                                                                                                                                                                                          eu-central-1 1112573368               storj-debian "/DriveShare/StorjNode"
1564363800000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564363800000000000                                                                                                                                                                                                                                                                                          eu-central-1 1113097896               storj-debian "/DriveShare/StorjNode"
1564365600000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564365600000000000                                                                                                                                                                                                                                                                                          eu-central-1 1113572072               storj-debian "/DriveShare/StorjNode"
1564367400000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564367400000000000                                                                                                                                                                                                                                                                                          eu-central-1 1114067276               storj-debian "/DriveShare/StorjNode"
1564369200000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564369200000000000                                                                                                                                                                                                                                                                                          eu-central-1 1114594640               storj-debian "/DriveShare/StorjNode"
1564371000000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564371000000000000                                                                                                                                                                                                                                                                                          eu-central-1 1115144528               storj-debian "/DriveShare/StorjNode"
1564372800000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564372800000000000                                                                                                                                                                                                                                                                                          eu-central-1 1115667996               storj-debian "/DriveShare/StorjNode"
1564374600000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564374600000000000                                                                                                                                                                                                                                                                                          eu-central-1 1116208844               storj-debian "/DriveShare/StorjNode"
1564376400000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564376400000000000                                                                                                                                                                                                                                                                                          eu-central-1 1116732400               storj-debian "/DriveShare/StorjNode"
1564378200000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564378200000000000                                                                                                                                                                                                                                                                                          eu-central-1 1117258416               storj-debian "/DriveShare/StorjNode"
1564380000000000000 0        100     0         0         0       0          0          0             100            0                0           Default 100            0         0        100      0               100            0                0          0          100   0       0       0            eu-central-1                          storj-debian 
1564380000000000000

When this in “folder_size.sh” works:

echo -n StorJHealth,host=$HOSTNAME,path=\""$@"\" directory_size_kilobytes=$SIZE "$(date +'%s%N')"

why does it not in successrate.sh

??