You mean @Roxor’s suggestion?
I don’t know what I meant. Your answer was perfectly clear, I don’t really understand how I could misunderstand your answer.
I see your answer in another comment, with your pool Pool of multiple 4 disk wide RAID-z1 vDevs, with a mirrored special meta device in front. And man the Intel P3600 SSDs were nice for their time. Still are, but I drooled so hard over those, back in the day.
What would happen if you exhausted the space on your special meta device vDev?
This is probably why our views diverge. I’m oversimplifying a bit, but the primary load is part of a collection of proxy services. So while it’s best for the data on each disk to remain stable (because refilling it takes time, and performance is lower when it’s empty): any disk can “just die” and be replaced. That’s what I meant by saying it rewards capacity: there’s no reason to give up space to parity/mirroring. As long as any particular HDD always has about 500GB free… Storj can use the rest. It’s also why ZFS was initially chosen: for the online scrubs: you can ensure each proxy is still “clean”. 
I’m still hoping hashstore does something clever. But… I’m not switching until it ships as the default. I get things now, when there’s millions of .sj1 files: and a understandable and observable filesystem beneath them. No funny business…
Cool. My node barely sees any increase still. I guess deleting marathon is not yet over for everyone
If there is no space left on special device new metadata will be written to data vdevs, same way it would without special device.
If you expect you are going to use up entire SSD it would make sense to over provision it slightly to reduce wear (with vendor tools or by partitioning manually). Or better yet, replace with larger ones before it’s full.
Makes super sense, thank you. I can’t imagine choosing a meta data array that’s too small to begin with, but I can totally see a future, where the array that it’s accelerating that has grown enough to me the former inadequate.
Would you in that case remove one from the mirror, replace with larger, rebuild and then ditto with the other, to then expand the “new” mirror?
ZFS allows multiple special device vdevs. So you can just add another one.
Yes, you can increase size of special virtual device just like any other device — replacing drives one by one and resilvering.
Unlike many other filesystems zfs replace allows the disk being replaced to remain in the pool until resilver is completed so that fault tolerance is not diminished for the duration of replacement.
Or what @alpharabbit suggested above.
There are pros and cons to each approach
As an anecdotal reference, on my arrays, the special device usage varies between 0.3 and 0.7% of used storage. For example, on one of the systems with 70TB used space I see 547 GB allocated on the special device. So its pretty achievable 
So noted. I seem to have spoke too soon, deletes continue. 
2 cents,
Julio
Yeah, yeah - that’s the ticket. Like it’s that Videodrome movie from the 80s…heh
2 cents,
Julio
hi just letting You know, that all that over-average traffic this month, all month, did nothing to my payment. im getting less and less for the same nodes. Months ago it was steady, around 100$ now its 15-20% less. Im not happy, just letting You know.
(Because my out to fiat, is from 100$.)
Edit:
@Julio Well, my nodes are very old, like from 2019/2020 soo probably a lot of deletion of that “free tier” data purged, prooobably, more than new data.
This is a clear, simple, and quick summary.
Yes, now that you mention it, I see payment has arrived this afternoon. Checking price, Storj is already down 4%.
Was almost same $ amount as last month here.
Groovy.
2 cents,
Julio
Storj needs to make its global network more attractive to potential customers. Rather than hiding its Select network to potential customers, the company should focus on getting the global network audited and certified. This is crucial if the interesting large, high-value potential customers are currently unable to use the network due to lack of compliance with general required standards. Until this issue is addressed, the network’s potential for growth and success will be very limited in that field.
The payment received today was the highest so far so I can’t complain about the earnings.
This is for this year.
Received May 6: $180
Received April 4: $149
Received March 4: $151
Received February 5: $157
That’s great to hear! I bet a lot of SNOs just want to see at least a small bump every month: that’s enough to keep them happy.
Looking back I had a bump every month. March/April payments this year went in the wrong direction but I think that is because during Feb/March I converted to hashstore on all my nodes making them very slow so I lost a lot of races.
From $30 to $31 lol.
I think the large ingress bandwidth is partially or mostly canceled out by faster deletion. It’s unfortunately the worst kind of traffic from a profit perspective. It uses bandwidth and IOPS but generates little storage revenue. I mean, the impact is, modest, though.
I see the same consistenly upward trend (ignoring the test data last year).
Payout across all nodes from 2022-06 to 2025-04...
Pay period 10.0.17.120 10.0.17.161 10.0.17.134 10.0.17.118 192.168.0.15 192.168.0.219
Month 2022-06: $0.00 $0.00 $0.00 $0.00 $0.00 $0.00
Month 2022-07: $0.03 $0.00 $0.00 $0.00 $0.00 $0.00
Month 2022-08: $0.43 $0.00 $0.00 $0.00 $0.00 $0.00
Month 2022-09: $1.33 $0.00 $0.00 $0.00 $0.00 $0.00
Month 2022-10: $4.22 $0.00 $0.00 $0.00 $0.00 $0.00
Month 2022-11: $4.31 $0.00 $0.00 $0.00 $0.00 $0.00
Month 2022-12: $6.38 $0.43 $0.00 $0.00 $0.00 $0.00
Month 2023-01: $10.30 $0.58 $0.00 $0.00 $0.00 $0.00
Month 2023-02: $9.87 $1.21 $0.00 $0.00 $0.00 $0.00
Month 2023-03: $13.88 $5.14 $0.00 $0.00 $0.00 $0.00
Month 2023-04: $17.68 $4.32 $0.00 $0.00 $0.00 $0.00
Month 2023-05: $16.18 $5.23 $0.00 $0.00 $0.00 $0.00
Month 2023-06: $16.95 $8.91 $0.00 $0.00 $0.00 $0.00
Month 2023-07: $13.78 $6.54 $0.27 $0.00 $0.00 $0.00
Month 2023-08: $15.37 $7.84 $0.78 $0.00 $0.00 $0.00
Month 2023-09: $13.58 $9.29 $0.61 $0.00 $0.00 $0.00
Month 2023-10: $30.82 $10.80 $1.65 $0.00 $0.00 $0.00
Month 2023-11: $17.04 $13.04 $2.40 $0.00 $0.00 $0.00
Month 2023-12: $16.75 $12.85 $3.11 $0.00 $0.00 $0.00
Month 2024-01: $19.59 $15.68 $6.99 $0.00 $0.00 $0.00
Month 2024-02: $19.31 $16.00 $7.90 $0.00 $0.00 $0.00
Month 2024-03: $21.44 $32.77 $9.35 $0.00 $0.13 $0.00
Month 2024-04: $18.76 $17.41 $12.97 $0.01 $0.42 $0.00
Month 2024-05: $19.38 $18.15 $14.59 $0.09 $1.66 $0.00
Month 2024-06: $29.92 $19.79 $17.35 $0.71 $4.23 $0.00
Month 2024-07: $38.15 $21.77 $15.89 $2.45 $9.10 $0.31
Month 2024-08: $35.04 $24.36 $20.17 $3.75 $13.48 $1.43
Month 2024-09: $20.79 $16.46 $13.24 $1.86 $4.74 $0.37
Month 2024-10: $20.04 $17.57 $24.78 $3.02 $5.79 $1.42
Month 2024-11: $19.68 $16.96 $14.68 $3.17 $8.78 $1.56
Month 2024-12: $20.49 $18.16 $15.64 $3.43 $9.60 $1.69
Month 2025-01: $20.89 $18.66 $16.18 $5.03 $10.44 $2.76
Month 2025-02: $19.19 $17.31 $15.30 $4.56 $10.86 $2.93
Month 2025-03: $21.73 $19.60 $17.32 $5.48 $13.33 $3.72
Month 2025-04: $21.70 $19.67 $17.50 $5.66 $13.65 $5.27
#!/bin/zsh
# CONFIG
start_date_str="2022-06"
end_date_str="2025-04"
ip_addresses=(
"10.0.17.120"
"10.0.17.161"
"10.0.17.134"
"10.0.17.118"
"192.168.0.15"
"192.168.0.219"
)
# END CONFIG
current_year=${start_date_str%-*}
current_month_num=$((10#${start_date_str#*-})) # ensure base-10, even if starts from 0
end_year=${end_date_str%-*}
end_month_num=$((10#${end_date_str#*-}))
echo "Payout across all nodes from $start_date_str to $end_date_str..."
printf '%-13s' "Pay period"
for ip in "${ip_addresses[@]}"; do
printf '%15s ' "${ip} "
done
echo
while (( current_year < end_year || (current_year == end_year && current_month_num <= end_month_num) )); do
current_month_formatted=$(printf "%02d" $current_month_num)
year_month="${current_year}-${current_month_formatted}"
printf '%s' "Month ${year_month}: "
for ip in "${ip_addresses[@]}"; do
data=$(curl -s "http://${ip}:14002/api/heldamount/paystubs/${year_month}/${year_month}")
if [ "$data" = 'null' ]; then
printf '$%-14.2f ' "0.000000"
else
total_paid_usd="$(echo "${data}" | jq '(map(.paid) | add ) / 1000000')"
printf '$%-14.2f ' "${total_paid_usd}"
fi
done
echo
(( current_month_num++ ))
if (( current_month_num > 12 )); then
current_month_num=1
(( current_year++ ))
fi
done
My ingress dropped 2x last several days, but official stats do not show the drop of ingress.
Is it only my ingress dropped? OR it is statistic error?
I have the same drop