I found that it is good practice to always use trailing slashes for the source and destination. If /new already exists then your command would copy /old into /new/old.
3 Likes
i just recently move my 4tb to 8tb… copy takes nearly a week. i left my node running during that time. so nothing was lost during the copy. i did the last round of robo copy with everything stopped, that toke nearly 1 day to compare and copy. all done using robocopy.
i doubt rsync will do any better. we are talking about nearly a million files here… thats how much hdd can do.
now i come to a stage that i feels i should just leave it and run another node on the same machine on another disk, without having to go through this process.
2 Likes
It sounds like it would be ideal for a SNO to have two IPs: one to actively grow a node… and the second to hold all the nodes that have been filled? So you fill a node… then park it behind the shared IP with other filled nodes (since they only need enough shared ingress to replace deleted trash)? Then buy a new HDD to start a fresh node again on the unique IP?
I guess the first step is always the hardest: fill your first node before trying anything fancy 
i guess that might work as well. but your node would probably start all over again to gain traction. it takes time to get the node from new to 4-5years old node. i would probably put in extra disk when i build up another hdd to run node. but if im running behind a same IP, does it make any diff? @Alexey ?
I believe I did read somewhere on Github the last year there was a plan to counter this by moving the data away by repair from such IP migrated nodes, I believe around the time they were implementing the placement rules I guess to be able to spinup the Select service.
This was to make sure there won’t be too many segments of the same file residing on one IP, where the segments were distributed across multiple IPs before the node IP move.
I don’t believe using 2 IPs will make any difference for the new node. The ones that are filled will not get ingress anymore, just a few MB a day, and the rest goes to unfilled one, and is not split anymore. I hope I remember corectly, because this was answeared some time ago. Alexey will enlighten us.
1 Like
This will be a harmful for the network - your nodes may contain pieces of the same segment. So if your ISP decided to do a maintenance for example and all these pieces will become unavailable, increasing risk to lost data.
Lost data = lost customer = lost reputation = no network = no payout. Sounds like trying to shoot yourself in the foot.
@ZhiYuan Regarding receiving data, if the nodes behind the same /24 subnet of public IPs, they will split the ingress, so you will have two barely filled nodes, and if you start them in the same time, the vetting process could take more time than if there is only one node, because each of the node will have less data for the same time interval and they could receive audits less often. To be vetted on one satellite the node should pass 100 audits from it.
1 Like
But if one is full, does the other still receives half of ingress?
No, the remaining will split the ingress traffic almost equally, because the full node will not be selected in a first place.
2 Likes
in short, ingress is split between all non-full nodes.
4 Likes
There is another way to move a large node, but it’s a little icky. I’ve used it several times though and it does work.
While the node is running just do the following, using whatever native copy is fastest for your OS.
- Lower the shared space to far below what’s already stored on your node.
- Restart the node and wait for at least 10 minutes (ideally you could wait in this state for a week or more to get rid of some trash before the copy, but if you don’t have time, wait at least 10 minutes to make sure satellite cache for node selection has been updated and you no longer receive new data.)
- Copy the blobs folder only to the new location with the node online.
- Stop the node
- Copy the rest of the data over. (This is where trash could still take a while, unless you’ve waited for it to be smaller by lowering node capacity longer before doing this)
- Start the node using the new location.
The downside of this method is that your blobs folder will have data that was deleted while copying the node. So you end up with some bloat. However, that should easily get caught and cleaned up again by garbage collection. And since the node is only down for step 5, you can usually keep the downtime below 4 hours and prevent repair from kicking in on your data.
You have to be really careful though and not make any mistakes, because you can easily mess up your node doing this wrong. But I’ve done it about 6 or 7 times now and it works just fine.
2 Likes
wouldn’t stop the piece scan on startup help here? (or letting the filewalk finish?)
Not sure what your question is, but that was referring to the node selection cache on the satellite. That is updated once every few minutes and until the cache is updated, your node might still be in there and might still receive new data. It is independent of the filewalker.
Question is:
Does filewalk slow down copy action?
Yes. 20 characters limit