Looks like the promised download of stefan-benten satellite data has begun

Pentium100 · March 25, 2020, 2:52pm

Or it’s just some other test. Anyway, nice to see some egress.

The data does not seem to be deleted after download though, it will probably be deleted later:

BrightSilence · March 25, 2020, 5:12pm

Yep, every month I think this is the month that payout will be low. Then something like this happens.

Pac · March 25, 2020, 7:14pm

@BrightSilence me too ^^

Do you think it’s StorjLabs’ way to make SNOs do net get too low payments?

BrightSilence · March 25, 2020, 7:17pm

I don’t think it’s entirely a coincidence, no.

jeremyfritzen · March 25, 2020, 8:15pm

Which software did you use to get this kind of diagram?

Pentium100 · March 25, 2020, 9:08pm

The first graph is from node API, the second is using this script to get the data.

github.com

Pentium100MHz/storjv3-tools/blob/a6f453a335e6c5da410ac691223805c7132505e4/actions.php

<?php

$resultfile="/opt/mon/actions2.json";
$lastrunfile="/opt/mon/actions2.lastrun.txt";

$lastrun=trim(file_get_contents($lastrunfile));
exec("/usr/bin/docker logs storagenode --since ".$lastrun." 2>&1",$op);
file_put_contents($lastrunfile,time());

$totalsread=file_get_contents($resultfile);
$totals=json_decode($totalsread,1);
$totals_old=$totals;

foreach ($op as $line) {
	$parts=explode("\t",$line);
	$origin=$parts[2];
	$severity = preg_replace('/\e[[][A-Za-z0-9];?[0-9]*m?/', '', $parts[1]); //remove terminal color codes
	if (($severity=="INFO") && ($origin == "piecestore")) {
		$result=str_replace(" ","_",$parts[3]);
		$json=json_decode($parts[4],1);

This file has been truncated. show original

In both cases, the data is then used by Zabbix to create the graphs.

The script is run by cron every minute and counts the new log entries (so, even when the log is erased, say, during an update, the count remains). The results are put in a json file, then Zabix just queries that file. I tested it with php7, but it should run on other versions too.

cdhowie · March 26, 2020, 1:54am

FWIW, you can do all of this without an external script; Zabbix can analyze logs directly using keys like:

log.count[{$STORJ_LOG_PATH},\tpiecestore\tdeleted\t,,,skip]
log.count[{$STORJ_LOG_PATH},\tpiecestore\tdownloaded\t,,,skip]
and so on

Pentium100 · March 26, 2020, 5:15am

That should work. The reason I wrote my script (this is the second version) was for performance. Running “docker logs …” (or, if I redirected the logs to a file, reading that file) takes a long time (when the node has been up for a few days) and if it was run for every data point my node VM would probably have unix load of 10.

My scripts runs once a minute and counts everything, then Zabbix or cacti can just read the values from the json file.

cdhowie · March 26, 2020, 2:00pm

Note that log.count behaves this way as well. The agent remembers where it was in the file and only the count of matched lines is sent to the server.

Pentium100 · March 26, 2020, 3:33pm

That’s good to know, I don’t think I have used “log.count” before.

cdhowie · March 26, 2020, 3:44pm

Note that it requires at least version 3.2 of the Zabbix agent – this was a bit annoying for me since the distro I’m running on has 3.0 packaged. I had to build the 4.0 agent from source.

Dinip · April 4, 2020, 12:07am

how do you graph it in zabbix? I’m new to zabbix, I got the .json to show on zabbix but now how do I make a graph from it?
Thanks!

Pentium100 · April 4, 2020, 2:15am

Sorry, I didn’t mention that you need another script to read from that file (it looked obvious to me).

#!/bin/bash
sat=$1
query=$2
res=`cat /opt/mon/actions2.json | jq ".\"$sat\".$query"`
nul=`echo "$res" | grep null | wc -l`
if [ $nul -eq 1 ]; then
        echo 0
else
        echo "$res"
fi

Save that as /opt/mon/actions2-zabbix.sh and put this line in the config of your zabbix agent:

UserParameter=storjactions2[*], /opt/mon/actions2-zabbix.sh $1 $2

Now you can create items like this: storjactions2[total, none.deleted], storjactions2[118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW, GET.download_canceled] and so on.