Trash comparison thread

afbeelding

It’s the result of over-provisioned segment uploads. 110 pieces are started, only 80 are finished. The remaining 30 are long tail cancelations. So that’s mostly what the success rates are about as well.

This is different from the zombie segments mentioned earlier. Those appear when a file has multiple segments (>64mb) and earlier segments have finished, but the upload is interrupted before the file is completed. The segment being uploaded at that moment will be canceled on all nodes and won’t end up on them, but the previously finished segments for that same file stay behind and get caught by the zombie reaper on the satellite (apparently twice a week) and then garbage collection on nodes end.

Btw, didn’t mean to put you down, it was just not yet the moment to say told you so. :wink: You may well end up being right, we just don’t really have the proof yet to pin it down exactly. It may be true that long tail cancellation doesn’t normally end up in trash… but perhaps there are scenarios in which they can end up there. If you look at the trash sizes per satellite, they do seem to coincide with the satellites that have the most uploads recently (excluding repair). So it does seem to be something related to uploads. Of course zombie segments are also related to uploads.

takes a lot to put me down… xD i don’t really attribute a lot of meaning to individual words because well in most communication over 10% is really understood between people… so yeah… when one starts to realize that is true… it sort of becomes a crabshot to try and successfully happen through the point for those that actually has the chance of understanding what one really meant…

i’m not saying the words doesn’t make sense… only that the true intent is more often than not misunderstood rather than clearly understood… we are simply biased and see all kinds of patterns and things we want, don’t want or have different understandings of words… because how do we describe words… with more sequences of words… which again is explained by more sequences of words…

becomes a pretty confusing and at times very specialized language between all branches of people… hell you cannot have 5 people working together for a week before the group start developing their own sort of language… so as in jokes and slang

alas i digress… back on point…

so in relation to the zombie hordes, i’m wide open fields of internet and you likewise if memory serves… so more speed equals less time exposed and thus less zombies… could be as simple as that…

but yeah i duno… you explained it very elegantly tho … good job
can be very difficult to relay complex topics

@stefanbenten
sorry about me not understanding the context completely.

if bright explained it correctly, then the zombie segments are essentially the same as cancelled uploads and thus wouldn’t the answer simply be to delete them as well upon … disruption of the connection…

or is there a reason to keep them, in case the connection is resumed or what not…
and in that case wouldn’t it just be sensible to reduce the time before deletion of these particular zombie segments… i mean it sucks when a download crashes… but it also seems kinda crazy if i can resume it in a full week… (and if one could do that, can’t it be exploited…)

currently some of the 1tb nodes have like 300-400gb trash… going from a week to a daily schedule would reduce this by a factor of 7 making it a much more reasonable number…

ofc just postulating…

1 Like