Deletes from stefan-benten satellite over?

kevink · March 16, 2020, 2:20pm

Since before the last update I noticed that the amounts of deletes dropped severely and I thought it may be because of the update period. But now the 72 hours are over and I’m wondering if the transition from stean-benten is complete or if there is another reason that the deletes stopped.
Espeically my 3rd node had over 800GB and I reduced its allocation to 500GB but it still has 680GB, so I don’t think the transition is complete yet.

Pentium100 · March 16, 2020, 2:41pm

At least, on my node, the deletes were coming not from Stefan’s satellite:
Total requests:

Requests from stefan-benten:

Requests from europe-west:

So, pretty much all delete requests came from europe-west. At least on my node.

kevink · March 16, 2020, 2:43pm

oh that is interesting. Thanks for sharing your detailed stats! I should probably get such a stats system running too What are you using?

ok that means there is still a lot of deleting to be done by stefan-benten… good, my nodes are all full xD

Pentium100 · March 16, 2020, 2:48pm

I use zabbix (for the graph) and my script for collecting stats:

github.com

Pentium100MHz/storjv3-tools/blob/a6f453a335e6c5da410ac691223805c7132505e4/actions.php

<?php

$resultfile="/opt/mon/actions2.json";
$lastrunfile="/opt/mon/actions2.lastrun.txt";

$lastrun=trim(file_get_contents($lastrunfile));
exec("/usr/bin/docker logs storagenode --since ".$lastrun." 2>&1",$op);
file_put_contents($lastrunfile,time());

$totalsread=file_get_contents($resultfile);
$totals=json_decode($totalsread,1);
$totals_old=$totals;

foreach ($op as $line) {
	$parts=explode("\t",$line);
	$origin=$parts[2];
	$severity = preg_replace('/\e[[][A-Za-z0-9];?[0-9]*m?/', '', $parts[1]); //remove terminal color codes
	if (($severity=="INFO") && ($origin == "piecestore")) {
		$result=str_replace(" ","_",$parts[3]);
		$json=json_decode($parts[4],1);

This file has been truncated. show original

This works on php 7, but it should work with other versions as well, I just did not test it. You might want to change the file locations in lines 3, 5 and 7.

What it does - you run it using cron or whatever, this gets logs from the time since last run, parses them and creates a json file with the statistics. Which means that after you update the node and the logs disappear, the stats here remain.

Then it’s just a matter of getting the values from the json file to whatever monitoring system you are using :).

kevink · March 16, 2020, 2:51pm

Thanks a lot. php is not my favorite language xD I’ll check it out.

Pentium100 · March 16, 2020, 2:53pm

I use bash, C or php, whichever seems easiest to do a particular task.

seanr22a · March 23, 2020, 4:00pm

I hope all old test data get deleted soon. I’ve had a couple of TB of test data for many months and it don’t seem to decrease. This is a simple du -sh from the blob directory, I don’t know how to translate these ‘names’ to a satellite.

2.0T abforhuxbzyd35blusvrifvdwmfx4hmocsva4vmpp3rgqaaaaaaa
198G pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa
232G qstuylguhrn2ozjv4h2c6xpxykd622gtgurhql2k7k75wqaaaaaa
223G ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa
985G v4weeab67sbgvnbwd5z7tweqsqqun7qox2agpbxy44mqqaaaaaaa

Mark · April 23, 2020, 1:05am

Any news on this? My node fills fast with Saltlake test data but the Stefan test data does not get deleted. Have the plans for the Stefan satellite been changed, or is it just going to take a while for the Stefan satellite to delete the test data?

Alexey · April 23, 2020, 1:17am

Please, take a look