Is it possible to “jumpstart” a new node by directly copying data from an existing on to it? If not, I think that may be a feature worth considering. It would allow new nodes to contribute much more quickly and economically, because no network traffic would need to be paid. If the network then decides that that extra redundancy on those copied chunks aren’t needed, it can replace it over time while still benefiting from the extra availability of the data.
If that doesn’t work, there are still some other ways data should be transferable to new node, for example, by splitting the data into two, leaving half on the old node and half on the new one for increased availability without the networking overhead. As I understand, there are a lot of SNOs using multiple nodes and the ability to set one up from another one would probably benefit all involved parties.
I was suggesting having a procedure to split/merge nodes here: Join or split nodes. Though I no longer think it’s a good idea without further refinements.
Jumpstarting a new node by copying data (not moving) might result in breaking guarantees of uncorrelated failures that are the basis of the durability guarantees of Tardigrade, and hence will be undesirable for the customers.
Moreover, the data is tied hard to the NodeID. If you would copy the NodeID (identity) too and start both clones, this identity will be disqualified in a few minutes, because each of the clone will not have a data from the brother.
The identity must be unique in the network.
If you copy just data, the new identity will not recognize this data and it will be there forever. Even Garbage collector will not touch it.
I understand it doesn’t work at the moment, but I guess the network would benefit from allowing it, no?
Please, explain how?
Each erasure encoded piece is unique, they are not duplicates. There is no replication at all.
Please, read these blogs for details:
Your copy will be just waste of your space.
If the network would need an additional speed, there is planned to change the erasure coding parameters: