As an experiment, I am organizing an Office Hours event at 2026-08-05 13:00-13:30 UTC
Anybody can join who has question about:
operating/configuring storagenodes
hashstore/piecestore / migration
self-hosted satellite configuration
any similar technical topic
NOTE: It’s an opportunity to talk about technical questions. No business question will be discussed. I deny to discuss any question related to Chapter 11 / marketing / financial situation / token strategies. I am not the right person to comment any of those question. I am the right person to talk about the code.
If nobody is calling in, I will do my daily job. But I am ready to answer (or at least try answering) technical questions or helping with technical problems.
I would Ask about compaction problem, I understand that broken log is hard to recover, but compaction cold just ignore it and compact next file. Because this compaction problem I have already 30tb not compacted files.
hashstorwrite also not help, as it also crash on broken log.
I’m sure this is in the FAQ somewhere, but…
Do SNOs need to take any actions manually for * hashstore/piecestore / migration, or can they wait? How can they quickly tell if it’s complete/successful?
I’m curious if there are still three and a half people who run freebsd binaries, does the number increase or decrease?
I already had to migrate from 13.3 to 13.4 when golang decided to drop old EOL freebsd versions. Now I’m planning migrating away from TrueNAS into vanilla FreeBSD 15, so I don’t have to face compatibility issues going forward. I ask AI to inventory mine and iXSystems modifications of FreeBSD behaviour in TrueNAS and to configure vanilla FreeBSD to parity for my requirements. The testing in VM went well, soon I’ll pull the trigger.
The point being, if freebsd is dropped – so will my nodes. I"m not migrating to retched linux.
Yeah. This is why I keep self-hosting everything despite the absurd cost in time, electricity, and mental energy: I’m tired of rebuilding my life around vendors’ constantly shifting priorities.
Maybe that tradeoff makes no sense for most people. For me, amid the general enshittification, decay, sellouts, and technology increasingly becoming a popularity contest, I don’t see another viable path. Ugh.
I created a build that skips unreadable or broken parts and have been testing it since yesterday. I haven’t seen it encounter an unreadable part yet, but nothing seems to be broken either. You can test it too, if you want.
Furthermore, if I understood correctly, if the first round of compaction fails, the BF for the given store will not be processed. This fix should resolve that issue as well.
Ok thank you, may be you can push it to Storj official, that they review it and add to their repo. Also if i remember truly, without storj signed it should not work with sattelite?
Or we would have now lot of malicious nodes here.
@elek @littleskunk
Sorry to disturb you, but how is upper posts are possible?
As i remember we cant run own binaries with Storj satellites?
This mean i can change node that it can hold a audit slices and get rewords for storing lot of data on small HDD.
It is like a security question.
I dont mean that Aleksman4o did something wrong, but it just shouldn’t be possible. Or I am wrong somewhere?
No, you can’t. Audit checks will come down on you very quickly.
Technically, a protocol-compatible custom binary can communicate with the satellites. There is no remote attestation of the exact executable, and the security model must assume that storage node software may be malicious.
However, this does not allow a node to keep only predefined “audit slices”. The satellite selects both the segment and a random stripe to audit, and the requested offset is not known to the node beforehand. The returned stripe is verified against shares downloaded from other nodes using erasure coding. If a node stores only a fraction of each piece, it has approximately the same fractional chance of passing each audit, and repeated audits make continued cheating extremely unlikely.
Storage payouts are also not calculated from disk usage reported by the node. The satellite calculates at-rest usage from its own metadata describing which pieces were assigned to each node. A malicious node could accept and then delete data and may be counted temporarily, but it will eventually fail audits and be contained or disqualified.
So exact-binary enforcement is not the security boundary; unpredictable audits, satellite-side accounting, reputation and disqualification are.
P.S.: You can look at my four commits that differentiate the code from the official release and build the binary yourself.
I would be interested to hear about any scheduling operations though. As I run Storj on my (True)NAS, it shares CPU & IO with my work, a lot of which I schedule so as to avoid conflicts - however this doesn’t extend to Storj, when it seems to wake up randomly to do it’s thing (filewalkers, trash etc). From my point of view, it’s a shame when many run together as then they just trip each other up for the duration of the overlap.If I could kick off an intensive Storj process myself then I could improve both the runtime of that process, and my own local requirements
It cant take slice from other node and compare with my, as every piece are unic. I dont think that storj take slice today from slice from byte 30 to 50, tomorrow from 21 to 35. this slices should be like hashes or something, because storj holds only hash of files itself. I am not an expert and can be wrong. That’s why i wanted to hear a word from Storj about it. because I remember that it have ben told long time ago, that we cant make own binaries.
May be just to put specific configurable timing for compaction, but as i remember it was made specially depending from start time, so everyone run it in random time not at once.
The pieces are unique, but they are not mathematically independent. They are different erasure-coded shares of the same segment.
The satellite does not compare the bytes returned by one node directly with identical bytes from another node. It requests the same randomly selected stripe index from multiple different pieces and uses FEC correction to reconstruct the valid codeword. It can then identify which returned shares differ from their corrected values.
Therefore, when the same segment is audited again, a different stripe can be selected.
The satellite does not need to store the file or its hash for this. Storing hashes on the node would not help either, because the node must return the actual requested bytes.
