Been a while @champmine18 
Did you update to the latest version of the script? Like the title of this topic mentions as well as multiple of my posts just a few above yours.
There was a breaking update of the node databases with v1.104. But I had a fix for that ready before it even started rolling out to production nodes.
yes, I updated to the latest version of the script.
A few days ago everything was fine, but after the node was autoupdated to v1.104.5 I get the above error. I guess it has something to do with reorganization of the DB such that things you rely on in the script are no longer there. Did not debug yet but will do when I can find time.
Thanks for your continuing efforts on this wonderful tool 
It definitely works on that version and there were no DB changes between 1.104.1 and 1.104.5. The error you get also doesnāt make much sense as itās a syntax error. Kind of suggests your copy of the script has been altered. Please redownload the latest version of the script. It should work.
Thanks @BrightSilence
but sorry to say, still not working. Something seems to be wrong in the query string syntax (so it is not the call syntax that is causing the error but the DB operations done by the query string). I still donāt know exactly which part it is, will keep looking into it.
If you freshly download from github-raw, does it work for you?
This topic would be littered with complaints if that didnāt work. The strange thing is that syntax error suggests itās an invalid query. That shouldnāt even depend on the databases.
Yes, it works for me and for many others too.
Maybe you can make it print the query string, see if there is anything obvious that is wrong with the query as its generated on your system?
SELECT x.satellite_name satellite
,COALESCE(a.put_total,0) put_total
,COALESCE(a.get_total,0) get_total
,COALESCE(a.get_audit_total,0) get_audit_total
,COALESCE(a.get_repair_total,0) get_repair_total
,COALESCE(a.put_repair_total,0) put_repair_total
,COALESCE(c.bh_total,0) bh_total
,COALESCE(b.total,0) disk_total
,COALESCE(p.bh_payout,0) bh_payout
,COALESCE(p.get_payout,0) get_payout
,COALESCE(p.get_repair_payout,0) get_repair_payout
,COALESCE(p.get_audit_payout,0) get_audit_payout
,COALESCE(d.rep_status,'') rep_status
,COALESCE(d.vet_count,0) vet_count
,COALESCE(d.uptime_score,0) uptime_score
,COALESCE(d.audit_score,0) audit_score
,COALESCE(d.audit_suspension_score,0) audit_suspension_score
,COALESCE(d.joined_at, '') sat_start_dt
,COALESCE(f.surge_percent, 100) surge_percent
,COALESCE(g.held_so_far,0) held_so_far
,COALESCE(g.disp_so_far,0) disp_so_far
,COALESCE(g.postponed_so_far, 0) postponed_so_far
,COALESCE(f.disposed,0) disposed
,COALESCE(f.payout,0) payout
,COALESCE(f.paid_out,0) paid_out
,CASE WHEN f.payout > f.paid_out THEN f.payout - f.paid_out ELSE 0 END postponed
,CASE WHEN f.paid_out > f.payout THEN f.paid_out - f.payout ELSE 0 END paid_prev_month
,COALESCE(h.receipt, '') receipt
,COALESCE(h.receipt_amount,0) receipt_amount
,COALESCE(1+strftime('%m', date('2024-05-01')) - strftime('%m', date(d.joined_at)) +
(strftime('%Y', date('2024-05-01')) - strftime('%Y', date(d.joined_at))) * 12, 0) AS month_nr
,COALESCE(SUBSTR(f.pay_stat, 1, LENGTH(f.pay_stat)-2), '') AS pay_status
,COALESCE(c.seconds_bh_included,2592000) seconds_bh_included --Assume full month if NULL. This basically only happens when no storage has been reported by the satellite yet.
,COALESCE(c.disk_last_report,0) disk_last_report
FROM (
SELECT DISTINCT active_sat.satellite_id,
CASE
WHEN sat.address IS NOT NULL THEN sat.address
WHEN hex(satellite_id) = '84A74C2CD43C5BA76535E1F42F5DF7C287ED68D33522782F4AFABFDB40000000' THEN 'ap1.storj.io:7777*'
WHEN hex(satellite_id) = 'AF2C42003EFC826AB4361F73F9D890942146FE0EBE806786F8E7190800000000' THEN 'eu1.storj.io:7777*'
WHEN hex(satellite_id) = 'F474535A19DB00DB4F8071A1BE6C2551F4DED6A6E38F0818C68C68D000000000' THEN 'europe-north-1.tardigrade.io:7777*'
WHEN hex(satellite_id) = '7B2DE9D72C2E935F1918C058CAAF8ED00F0581639008707317FF1BD000000000' THEN 'saltlake.tardigrade.io:7777*'
WHEN hex(satellite_id) = 'A28B4F04E10BAE85D67F4C6CB82BF8D4C0F0F47A8EA72627524DEB6EC0000000' THEN 'us1.storj.io:7777*'
WHEN hex(satellite_id) = '04489F5245DED48D2A8AC8FB5F5CD1C6A638F7C6E75EFD800EF2D72000000000' THEN 'us2.storj.io:7777*'
WHEN hex(satellite_id) = '004AE89E970E703DF42BA4AB1416A3B30B7E1D8E14AA0E558F7EE26800000000' THEN 'satellite.stefan-benten.de:7777* (shut down)'
ELSE '-UNKNOWN-'
END satellite_name,
sat.added_at AS satellite_added_at
FROM (
SELECT satellite_id, interval_start FROM bandwidth_usage WHERE interval_start >= '2024-05-01' AND interval_start < '2024-06-01'
UNION
SELECT satellite_id, timestamp interval_start FROM su.storage_usage WHERE interval_start >= '2024-05-01' AND interval_start < '2024-06-01'
) active_sat
LEFT JOIN satellites sat
ON active_sat.satellite_id = sat.node_id
) x
LEFT JOIN
psu.piece_space_used b
ON x.satellite_id = b.satellite_id
LEFT JOIN (
SELECT
satellite_id
,SUM(put_total) put_total
,SUM(get_total) get_total
,SUM(get_audit_total) get_audit_total
,SUM(get_repair_total) get_repair_total
,SUM(put_repair_total) put_repair_total
FROM bw.bandwidth_usage a
WHERE interval_start >= '2024-05-01' AND interval_start < '2024-06-01'
GROUP BY satellite_id
) a
ON x.satellite_id = a.satellite_id
LEFT JOIN (
SELECT
satellite_id
,SUM(at_rest_total) bh_total
,strftime('%s', max(edt)) - strftime('%s', min(sdt)) seconds_bh_included
,max(last_window_bh)*3600 / (strftime('%s', max(edt)) - strftime('%s', max(sdt))) disk_last_report
FROM (SELECT timestamp interval_start, satellite_id, at_rest_total, interval_end_time edt,
(SELECT interval_end_time
FROM su.storage_usage su2
WHERE su1.satellite_id = su2.satellite_id
AND su2.timestamp < su1.timestamp
AND su2.interval_end_time <> '0001-01-01 00:00:00+00:00' /* ignore incomplete records */
ORDER BY timestamp DESC
LIMIT 1) sdt,
LAST_VALUE(at_rest_total) OVER
(PARTITION BY satellite_id ORDER BY interval_end_time
RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) last_window_bh
FROM su.storage_usage su1
WHERE su1.interval_end_time <> '0001-01-01 00:00:00+00:00' /* ignore incomplete records */)
WHERE interval_start >= '2024-05-01' AND interval_start < '2024-06-01'
GROUP BY satellite_id
) c
ON x.satellite_id = c.satellite_id
LEFT JOIN (
SELECT
satellite_id
,CASE WHEN disqualified_at IS NOT NULL THEN 'Disqualified @ ' || datetime(disqualified_at)
WHEN suspended_at IS NOT NULL THEN 'Suspended Audits @ ' || datetime(suspended_at)
WHEN offline_suspended_at IS NOT NULL THEN 'Suspended Downt. @ ' || datetime(offline_suspended_at)
WHEN offline_under_review_at IS NOT NULL THEN 'Downtime review @ ' || datetime(offline_under_review_at)
WHEN audit_success_count < 100 THEN 'Vetting '
WHEN audit_reputation_score < 0.998 OR audit_unknown_reputation_score < 0.998 THEN 'WARN: Audits failing'
WHEN online_score < 0.98 THEN 'WARN: Downtime high'
ELSE 'OK' END AS rep_status
,date(joined_at) AS joined_at
,MIN(audit_success_count, 100) AS vet_count
,100.0*online_score AS uptime_score
,((1-audit_reputation_score)*100.0)/(1.0-0.96) AS audit_score
,((1-audit_unknown_reputation_score)*100.0)/(1.0-0.6) AS audit_suspension_score
FROM r.reputation
) d
ON x.satellite_id = d.satellite_id
LEFT JOIN (
SELECT
satellite_id
, CASE WHEN INSTR(codes, 'D') > 0 THEN 'Disqualified, ' ELSE '' END
||CASE WHEN INSTR(codes, 'X') > 0 THEN 'Graceful Exit, ' ELSE '' END
||CASE WHEN INSTR(codes, 'S') > 0 THEN 'Sanctioned Country, ' ELSE '' END
||CASE WHEN INSTR(codes, 'T') > 0 THEN 'Tax form 1099 missing, ' ELSE '' END AS pay_stat
,CASE WHEN surge_percent = 0 THEN 100 ELSE surge_percent END AS surge_percent
,disposed/1000000.0 disposed
,paid/1000000.0 payout
,distributed/1000000.0 paid_out
FROM h.paystubs
WHERE period = '2024-05'
) f
ON x.satellite_id = f.satellite_id
LEFT JOIN (
SELECT
satellite_id
,COUNT(period) n_months_prec
,MAX(period) last_period
,SUM(held)/1000000.0 held_so_far
,SUM(disposed)/1000000.0 disp_so_far
,(SUM(paid)-SUM(distributed))/1000000.0 postponed_so_far
FROM h.paystubs
WHERE period < '2024-05'
GROUP BY satellite_id
) g
ON x.satellite_id = g.satellite_id
LEFT JOIN (
SELECT
satellite_id
, CASE
WHEN SUBSTR(receipt, 1, 4) = 'eth:' THEN 'https://etherscan.io/tx/'||SUBSTR(receipt, 5)
WHEN SUBSTR(receipt, 1, 7) = 'zksync:' THEN 'https://zkscan.io/explorer/transactions/'||SUBSTR(receipt, 8)
END receipt
, amount/1000000.0 AS receipt_amount
FROM h.payments
WHERE period = '2024-05'
) h
ON x.satellite_id = h.satellite_id
LEFT JOIN (
SELECT
satellite_id
,disk_space_price/100.0 bh_payout
,egress_bandwidth_price/100.0 get_payout
,repair_bandwidth_price/100.0 get_repair_payout
,audit_bandwidth_price/100.0 get_audit_payout
FROM p.pricing
) p
ON x.satellite_id = p.satellite_id
ORDER BY CAST(COALESCE(x.satellite_added_at, '9999-12-31') AS date), x.satellite_name;
Can you put the code in a code block so the forum doesnāt mess up the format?
```
Code here
```
thanks @BrightSilence, I put the block as code. Maybe you can give it a look to find where the error occurs
It is still there on several of my nodes with different versions of ubuntu under the hood.
It doesnāt have any syntax errors from what I can tell. Iām not sure where else to look. If you want you can zip up satellites.db, bandwidth.db, storage_usage.db, piece_spaced_used.db, reputation.db, heldamount.db and pricing.db and send them to me. I suggest using Storj and PM me the link.
Iād be happy to have a look. In the mean time, which python version are you using?
python --version or python3 --version
@BrightSilence thank you.
Python 3.6.9
Thanks for the offer to work on my .db files, I will try a few things myself first and if needed I will send you those. Thanks again.
@champmine18 Iām facing the same issue. Did you solve that sintaxe error?
Iām working on a new update to include an indicator whether GC is running on your node as well as the last processed folder as an indication of progress. This requires a little more testing on my end still, but hereās a sneak peek.
The date is the date the bloom filter was created.
Ps. Unlike other scripts I saw posted, this is not dependent on logs and therefor doesnāt run into issues when logs are removed or rotated. It uses the progress stored in garbage_collection_filewalker_progress.db, so it should be more reliable.
Can this be done for startup piece scan as well?
Unfortunately only GC progress is stored in a db. Iād be happy to add more if Storj ever stores info regarding those other processes in the dbās. But they did this specifically to implement the resume function, which exists for GC only, I believe.
This feature is now live. Please report any issues you encounter with it.
So this is basically saying my bloomfilter is out of date, this is the most recent one, make sure I get a new one soon for this satelitte?
(GC @ Folder:is Date:2024-08-22)
They are always a few days old. It takes some time to make a db snapshot and create the bloom filters. This is normal.
I just looked at one node I restarted and found it processing a bloom filter from the SLC satellite, and as soon as it finished, processing another one generated by SLCā¦ I guess I have a backlog to work through.
You may see that backlog in the storage/retain folder (docker) or "%ProgramFiles%\Storj\Storage Node\retain\" (Windows service).
my retain dir is in my config dir (docker), but itās empty, so that means there are none pending?
