Здравствуйте. У меня подобная проблема. Ни с того ни с сего начала перезагружаться одна нода. До этого год с лишним работала нормально.
Пишет ошибку
2023-05-23T08:33:30.786+0500 ERROR piecestore download failed {“Process”: “storagenode”, “Piece ID”: “J2BDPTOAM2ZYION6MZL42AJSNO7HMZMXYPHLQG2HFK5LSTMVAGEA”, “Satellite ID”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “Action”: “GET”, “Remote Address”: “172.18.0.1:48722”, “error”: “untrusted: unable to get signee: trust: rpc: tcp connector failed: rpc: dial tcp: lookup us1.storj.io: operation was canceled”, “errorVerbose”: “untrusted: unable to get signee: trust: rpc: tcp connector failed: rpc: dial tcp: lookup us1.storj.io: operation was canceled\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:140\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:62\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:599\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:251\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:61\n\tstorj.io/common/experiment.(*Handler).HandleRPC:42\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:124\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:66\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:114\n\tstorj.io/drpc/drpcctx.(*Tracker).track:35”}
Нода работает под Docker на Synology.
Здравствуйте @shaltay!
Очень похоже на проблемы с сетью
видимо есть подобные ошибки и для других сателлитов.
Попробуйте сконфигурировать другой DNS сервер в DHCP секции вашего роутера, например, 8.8.8.8. Правда это сработает только после перезагрузки Synology или отключения/включения ethernet кабеля в Synology.
Другой вариант, попробовать сконфигурировать другой DNS прямо на Synology.
Или самый простой - добавьте опцию --dns 8.8.8.8
в вашу команду docker run
до имени образа (потому что это опция docker, а не узла).
Другие ноды работают. Но на всякий прописал принудительно гугловский и ip4 и ip6 dns адреса.
Логи с проблемной:
Ошибки ratelimit
следствие предыдущих неудачных попыток зарегистрироваться на сателлите, либо потому что внешний адрес и порт недоступны, либо была какая-то ошибка при регистрации. В приведённом скриншоте предыдущих ошибок не видно.
- Проверьте, пожалуйста, identity для этого узла: Step 5. Create an Identity - Storj Node Operator Docs
- также убедитесь, что в опции
ADDRESS
указан актуальный адрес и если это DDNS hostname, что он ресолвится в публичный IP, совпадающий с WAN IP на вашем роутере:
nslookup your.ddns.hostname 8.8.8.8
должен вернуть такой же IP как на Open Port Check Tool - Test Port Forwarding on Your Router и совпадающий с WAN IP на роутере.
P.S. Фото текста не так полезно, как сам текст , пожалуйста, в следующий раз скопируйте текст и вставьте его тут между двумя новыми строчками с тремя backticks, вот так:
```
текст логов
```
приватную информацию можно замаскировать или удалить.
ash-4.4# nslookup ***.ru 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: ***.ru
Address: ..170.140
ash-4.4# nslookup ***.synology.me 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: ***.synology.me
Address: ..170.140
Name: ***.synology.me
Address: ::902b:ca7:211:32ff:feea:af63
Файлы находятся в папке ноды, нуть прописан правильно:
ash-4.4# grep -c BEGIN ca.cert
2
ash-4.4# grep -c BEGIN identity.cert
3
2023-05-23 11:46:38,554 INFO spawned: 'storagenode-updater' with pid 64
2023-05-23T11:46:38.565+0500 INFO Configuration loaded {"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
2023-05-23T11:46:38.565+0500 INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet"}
2023-05-23T11:46:38.565+0500 INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.email"}
2023-05-23T11:46:38.565+0500 INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.address"}
2023-05-23T11:46:38.565+0500 INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.private-address"}
2023-05-23T11:46:38.565+0500 INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet-features"}
2023-05-23T11:46:38.565+0500 INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-bandwidth"}
2023-05-23T11:46:38.565+0500 INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "contact.external-address"}
2023-05-23T11:46:38.565+0500 INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-disk-space"}
2023-05-23T11:46:38.566+0500 INFO Anonymized tracing enabled {"Process": "storagenode-updater"}
2023-05-23T11:46:38.566+0500 INFO Running on version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.78.3"}
2023-05-23T11:46:38.566+0500 INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
2023-05-23T11:46:38.606+0500 INFO Configuration loaded {"Process": "storagenode", "Location": "/app/config/config.yaml"}
2023-05-23T11:46:38.606+0500 INFO Anonymized tracing enabled {"Process": "storagenode"}
2023-05-23T11:46:38.607+0500 INFO Operator email {"Process": "storagenode", "Address": "**********************"}
2023-05-23T11:46:38.607+0500 INFO Operator wallet {"Process": "storagenode", "Address": "0x****************************"}
2023-05-23T11:46:39.086+0500 INFO server kernel support for tcp fast open unknown {"Process": "storagenode"}
2023-05-23T11:46:39.173+0500 INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.78.3"}
2023-05-23T11:46:39.173+0500 INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode"}
2023-05-23T11:46:39.180+0500 INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.78.3"}
2023-05-23T11:46:39.180+0500 INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode-updater"}
2023-05-23T11:46:39.840+0500 INFO Telemetry enabled {"Process": "storagenode", "instance ID": "1MBwuz3R4AAXfhvfHFFLjtPyUFetYJnHcc94PZEYQN6kEJqTW4"}
2023-05-23T11:46:39.840+0500 INFO Event collection enabled {"Process": "storagenode", "instance ID": "1MBwuz3R4AAXfhvfHFFLjtPyUFetYJnHcc94PZEYQN6kEJqTW4"}
2023-05-23 11:46:39,840 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-05-23 11:46:39,840 INFO success: storagenode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-05-23 11:46:39,840 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-05-23T11:46:39.863+0500 INFO db.migration Database Version {"Process": "storagenode", "version": 54}
2023-05-23T11:46:40.726+0500 INFO preflight:localtime start checking local system clock with trusted satellites' system clock. {"Process": "storagenode"}
2023-05-23T11:46:41.740+0500 INFO preflight:localtime local system clock is in sync with trusted satellites' system clock. {"Process": "storagenode"}
2023-05-23T11:46:41.741+0500 INFO Node 1MBwuz3R4AAXfhvfHFFLjtPyUFetYJnHcc94PZEYQN6kEJqTW4 started {"Process": "storagenode"}
2023-05-23T11:46:41.741+0500 INFO Public server started on [::]:28967 {"Process": "storagenode"}
2023-05-23T11:46:41.741+0500 INFO trust Scheduling next refresh {"Process": "storagenode", "after": "4h57m7.284417984s"}
2023-05-23T11:46:41.741+0500 INFO Private server started on 127.0.0.1:7778 {"Process": "storagenode"}
2023-05-23T11:46:41.741+0500 INFO bandwidth Performing bandwidth usage rollups {"Process": "storagenode"}
2023-05-23T11:46:41.741+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
2023-05-23T11:46:41.742+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "TLTAUVI33MDRWCRE2RLIGWAR4OLNUHLTGLR5OHRAJICPQV34YT7Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 0, "Size": 97536, "Remote Address": "172.18.0.1:54562"}
2023-05-23T11:46:41.743+0500 ERROR piecestore download failed {"Process": "storagenode", "Piece ID": "TLTAUVI33MDRWCRE2RLIGWAR4OLNUHLTGLR5OHRAJICPQV34YT7Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Remote Address": "172.18.0.1:54562", "error": "untrusted: unable to get signee: trust: rpc: tcp connector failed: rpc: dial tcp: lookup us1.storj.io: operation was canceled", "errorVerbose": "untrusted: unable to get signee: trust: rpc: tcp connector failed: rpc: dial tcp: lookup us1.storj.io: operation was canceled\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:140\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:62\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:599\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:251\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:61\n\tstorj.io/common/experiment.(*Handler).HandleRPC:42\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:124\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:66\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:114\n\tstorj.io/drpc/drpcctx.(*Tracker).track:35"}
2023-05-23T11:46:41.797+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"}
2023-05-23T11:46:41.817+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2023-05-23T11:46:41.905+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:42.438+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:42.758+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:42.941+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2023-05-23T11:46:43.013+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2023-05-23T11:46:43.070+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:43.198+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2023-05-23T11:46:44.133+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:44.761+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:45.241+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 3, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:46.813+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 3, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:47.765+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 3, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:49.400+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 4, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:51.502+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 4, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:52.777+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 4, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:46:53.250+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "2YEVN7KA4GYVK66ADCWKGMFXQ7ERT5IKY36HKOUULA5G22RLTY5A", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 2100992, "Size": 2560, "Remote Address": "172.18.0.1:54688"}
2023-05-23T11:46:53.329+0500 ERROR piecestore download failed {"Process": "storagenode", "Piece ID": "2YEVN7KA4GYVK66ADCWKGMFXQ7ERT5IKY36HKOUULA5G22RLTY5A", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Remote Address": "172.18.0.1:54688", "error": "trust: rpc: tcp connector failed: rpc: dial tcp 34.150.199.48:7777: operation was canceled", "errorVerbose": "trust: rpc: tcp connector failed: rpc: dial tcp 34.150.199.48:7777: operation was canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
2023-05-23T11:46:57.561+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 5, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:47:00.192+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 5, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:47:00.292+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "OHTKBB7U2XM6JX2SKSI2JXTKEHA5OU6M7N6M55WVS255NMYSMBRQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 2216448, "Size": 2304, "Remote Address": "172.18.0.1:54722"}
2023-05-23T11:47:00.322+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "5R7JLT4XNHIOWEJZCUXS3XCCT3ZURBA5GEWJXGAFIUA3VSQTORMQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 1837056, "Size": 2560, "Remote Address": "172.18.0.1:54724"}
2023-05-23T11:47:00.412+0500 ERROR piecestore download failed {"Process": "storagenode", "Piece ID": "OHTKBB7U2XM6JX2SKSI2JXTKEHA5OU6M7N6M55WVS255NMYSMBRQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Remote Address": "172.18.0.1:54722", "error": "trust: rpc: tcp connector failed: rpc: context canceled", "errorVerbose": "trust: rpc: tcp connector failed: rpc: context canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
2023-05-23T11:47:00.451+0500 ERROR piecestore download failed {"Process": "storagenode", "Piece ID": "5R7JLT4XNHIOWEJZCUXS3XCCT3ZURBA5GEWJXGAFIUA3VSQTORMQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Remote Address": "172.18.0.1:54724", "error": "untrusted: unable to get signee: trust: rpc: tcp connector failed: rpc: dial tcp: lookup eu1.storj.io: operation was canceled", "errorVerbose": "untrusted: unable to get signee: trust: rpc: tcp connector failed: rpc: dial tcp: lookup eu1.storj.io: operation was canceled\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).VerifyOrderLimitSignature:140\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).verifyOrderLimit:62\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Download:599\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func2:251\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:61\n\tstorj.io/common/experiment.(*Handler).HandleRPC:42\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:124\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:66\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:114\n\tstorj.io/drpc/drpcctx.(*Tracker).track:35"}
2023-05-23T11:47:01.198+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "WY7WOZES2L3WAJULSLBO6WR4NWPROCPXNVBE25INOVMJ2V7NHXMA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 651008, "Size": 2560, "Remote Address": "172.18.0.1:54724"}
2023-05-23T11:47:01.349+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "BCBZODLRKFM46INHVUNNPS6IUK52HOEJJ75FQR7YNRXTU7XIKD7Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 0, "Size": 6144, "Remote Address": "172.18.0.1:54734"}
2023-05-23T11:47:01.364+0500 ERROR piecestore download failed {"Process": "storagenode", "Piece ID": "WY7WOZES2L3WAJULSLBO6WR4NWPROCPXNVBE25INOVMJ2V7NHXMA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Remote Address": "172.18.0.1:54724", "error": "trust: rpc: tcp connector failed: rpc: context canceled", "errorVerbose": "trust: rpc: tcp connector failed: rpc: context canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
2023-05-23T11:47:01.448+0500 ERROR piecestore download failed {"Process": "storagenode", "Piece ID": "BCBZODLRKFM46INHVUNNPS6IUK52HOEJJ75FQR7YNRXTU7XIKD7Q", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Remote Address": "172.18.0.1:54734", "error": "trust: rpc: tcp connector failed: rpc: dial tcp 34.172.100.72:7777: operation was canceled", "errorVerbose": "trust: rpc: tcp connector failed: rpc: dial tcp 34.172.100.72:7777: operation was canceled\n\tstorj.io/common/rpc.HybridConnector.DialContext.func1:190"}
2023-05-23T11:47:05.424+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "7YSGGXYVMAEGVKGEAYYCYM54XFYIUHQRCOZB5CUE4S7Y55CG3UOA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET_REPAIR", "Offset": 0, "Size": 1536, "Remote Address": "172.18.0.1:54748"}
2023-05-23T11:47:05.974+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "7YSGGXYVMAEGVKGEAYYCYM54XFYIUHQRCOZB5CUE4S7Y55CG3UOA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET_REPAIR", "Offset": 0, "Size": 1536, "Remote Address": "172.18.0.1:54748"}
2023-05-23T11:47:13.720+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 6, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:47:16.256+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "MG4RKY6OUEOJNWDTAWJTJ4ZFIHD3DHIYICREPQ3UU3TZUUORRTJA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_REPAIR", "Offset": 0, "Size": 1536, "Remote Address": "172.18.0.1:54802"}
2023-05-23T11:47:16.603+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "MG4RKY6OUEOJNWDTAWJTJ4ZFIHD3DHIYICREPQ3UU3TZUUORRTJA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_REPAIR", "Offset": 0, "Size": 1536, "Remote Address": "172.18.0.1:54802"}
2023-05-23T11:47:16.875+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 6, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"}
2023-05-23T11:47:23.736+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "FU5KXMN5K7PNSUP26P3QBMGO6U5BTR4E3TUYN6VF3FGQNYR56NAA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET_AUDIT", "Offset": 398592, "Size": 256, "Remote Address": "172.18.0.1:54830"}
2023-05-23T11:47:24.506+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "FU5KXMN5K7PNSUP26P3QBMGO6U5BTR4E3TUYN6VF3FGQNYR56NAA", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Action": "GET_AUDIT", "Offset": 398592, "Size": 256, "Remote Address": "172.18.0.1:54830"}
И так по кругу перегружаясь.
2023-05-23T11:48:38.017+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "attempts": 5, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:48:38 2023-05-23T11:48:38.170+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 5, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:48:40 2023-05-23T11:48:40.448+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 5, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:48:44 2023-05-23T11:48:44.808+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "IBIHGJT3MV5XS6F5MEKPJHW4BYQCA27NBEP6VACFPRJFG773NTIQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET_REPAIR", "Offset": 0, "Size": 28160, "Remote Address": "172.18.0.1:55392"} stdout
11:48:45 2023-05-23T11:48:45.260+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "IBIHGJT3MV5XS6F5MEKPJHW4BYQCA27NBEP6VACFPRJFG773NTIQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET_REPAIR", "Offset": 0, "Size": 28160, "Remote Address": "172.18.0.1:55392"} stdout
11:48:47 2023-05-23T11:48:47.591+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "2UGFC4NRRXDNRFNGLTHRXNYN7CJ2WX5BLHQT3AUWX4YSHFOKFVZA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 851968, "Size": 2560, "Remote Address": "172.18.0.1:55408"} stdout
11:48:47 2023-05-23T11:48:47.802+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "2UGFC4NRRXDNRFNGLTHRXNYN7CJ2WX5BLHQT3AUWX4YSHFOKFVZA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 851968, "Size": 2560, "Remote Address": "172.18.0.1:55408"} stdout
11:48:54 2023-05-23T11:48:54.547+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "attempts": 6, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:48:54 2023-05-23T11:48:54.737+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 6, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:48:55 2023-05-23T11:48:55.904+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "QDN6A6LHWX5STP44NQW4L4ESSKAJU7IIDLF677RYCPBQTWGAWUEA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 1303552, "Size": 2560, "Remote Address": "172.18.0.1:55438"} stdout
11:48:56 2023-05-23T11:48:56.097+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "QDN6A6LHWX5STP44NQW4L4ESSKAJU7IIDLF677RYCPBQTWGAWUEA", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET", "Offset": 1303552, "Size": 2560, "Remote Address": "172.18.0.1:55438"} stdout
11:48:57 2023-05-23T11:48:57.452+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 6, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:48:59 2023-05-23T11:48:59.092+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "MML6QXY7V5JBGH5Y6YP6XCDMFV4VPCJTF6IXXVVDQYNFJMGYY2MQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET_REPAIR", "Offset": 0, "Size": 4352, "Remote Address": "172.18.0.1:55448"} stdout
11:48:59 2023-05-23T11:48:59.394+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "MML6QXY7V5JBGH5Y6YP6XCDMFV4VPCJTF6IXXVVDQYNFJMGYY2MQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "GET_REPAIR", "Offset": 0, "Size": 4352, "Remote Address": "172.18.0.1:55448"} stdout
11:49:00 2023-05-23T11:49:00.445+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "CZOVMT5G73KB6WWLVSMWJJVRZOS5O2WSK6XB5VYLDEFLFJOXANCA", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "Action": "GET_AUDIT", "Offset": 307200, "Size": 256, "Remote Address": "172.18.0.1:55456"} stdout
11:49:01 2023-05-23T11:49:01.086+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "CZOVMT5G73KB6WWLVSMWJJVRZOS5O2WSK6XB5VYLDEFLFJOXANCA", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "Action": "GET_AUDIT", "Offset": 307200, "Size": 256, "Remote Address": "172.18.0.1:55456"} stdout
11:49:05 2023-05-23T11:49:05.189+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "L4VIWTPZPENX5SQ64KGT6HHIJ43K7TEFQPO5VU4CEKB2376ZVMIA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_REPAIR", "Offset": 0, "Size": 5120, "Remote Address": "172.18.0.1:55480"} stdout
11:49:05 2023-05-23T11:49:05.327+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "L4VIWTPZPENX5SQ64KGT6HHIJ43K7TEFQPO5VU4CEKB2376ZVMIA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_REPAIR", "Offset": 0, "Size": 5120, "Remote Address": "172.18.0.1:55480"} stdout
11:49:05 2023-05-23T11:49:05.846+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "IF5V3LQBHYCKSMOYQF7TC3FZPANGCIWONIHI2TC6IMVPBZDTZ2QQ", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Action": "GET_REPAIR", "Offset": 0, "Size": 76288, "Remote Address": "172.18.0.1:55486"} stdout
11:49:06 2023-05-23T11:49:06.872+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "IF5V3LQBHYCKSMOYQF7TC3FZPANGCIWONIHI2TC6IMVPBZDTZ2QQ", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Action": "GET_REPAIR", "Offset": 0, "Size": 76288, "Remote Address": "172.18.0.1:55486"} stdout
11:49:07 2023-05-23T11:49:07.134+0500 ERROR piecestore:cache error getting current used space: {"Process": "storagenode", "error": "unrecoverable error accessing data on the storage file system (path=config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1; error=lstat config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1: input/output error). This is most likely due to disk bad sectors or a corrupted file system. Check your disk for bad sectors and integrity"} stdout
11:49:07 2023-05-23T11:49:07.134+0500 ERROR services unexpected shutdown of a runner {"Process": "storagenode", "name": "piecestore:cache", "error": "unrecoverable error accessing data on the storage file system (path=config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1; error=lstat config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1: input/output error). This is most likely due to disk bad sectors or a corrupted file system. Check your disk for bad sectors and integrity"} stdout
11:49:07 2023-05-23T11:49:07.134+0500 INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"} stdout
11:49:07 2023-05-23T11:49:07.134+0500 INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"} stdout
11:49:07 2023-05-23T11:49:07.134+0500 INFO contact:service context cancelled {"Process": "storagenode", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"} stdout
11:49:08 Error: unrecoverable error accessing data on the storage file system (path=config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1; error=lstat config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1: input/output error). This is most likely due to disk bad sectors or a corrupted file system. Check your disk for bad sectors and integrity stdout
11:49:08 2023-05-23 11:49:08,201 INFO exited: storagenode (exit status 1; not expected) stdout
11:49:09 2023-05-23 11:49:09,204 INFO spawned: 'storagenode' with pid 54 stdout
11:49:09 2023-05-23 11:49:09,204 WARN received SIGQUIT indicating exit request stdout
11:49:09 2023-05-23 11:49:09,205 INFO waiting for storagenode, processes-exit-eventlistener, storagenode-updater to die stdout
11:49:09 2023-05-23T11:49:09.205+0500 INFO Got a signal from the OS: "terminated" {"Process": "storagenode-updater"} stdout
11:49:09 2023-05-23 11:49:09,206 INFO stopped: storagenode-updater (exit status 0) stdout
11:49:09 2023-05-23T11:49:09.258+0500 INFO Configuration loaded {"Process": "storagenode", "Location": "/app/config/config.yaml"} stdout
11:49:09 2023-05-23T11:49:09.258+0500 INFO Anonymized tracing enabled {"Process": "storagenode"} stdout
11:49:09 2023-05-23T11:49:09.259+0500 INFO Operator email {"Process": "storagenode", "Address": "asperfilev@gmail.com"} stdout
11:49:09 2023-05-23T11:49:09.259+0500 INFO Operator wallet {"Process": "storagenode", "Address": "0x**************************************"} stdout
11:49:10 2023-05-23T11:49:10.057+0500 INFO server kernel support for tcp fast open unknown {"Process": "storagenode"} stdout
11:49:10 2023-05-23T11:49:10.815+0500 INFO Telemetry enabled {"Process": "storagenode", "instance ID": "1MBwuz3R4AAXfhvfHFFLjtPyUFetYJnHcc94PZEYQN6kEJqTW4"} stdout
11:49:11 2023-05-23T11:49:10.815+0500 INFO Event collection enabled {"Process": "storagenode", "instance ID": "1MBwuz3R4AAXfhvfHFFLjtPyUFetYJnHcc94PZEYQN6kEJqTW4"} stdout
11:49:11 2023-05-23T11:49:10.839+0500 INFO db.migration Database Version {"Process": "storagenode", "version": 54} stdout
11:49:11 2023-05-23T11:49:11.987+0500 INFO preflight:localtime start checking local system clock with trusted satellites' system clock. {"Process": "storagenode"} stdout
11:49:12 2023-05-23 11:49:12,989 INFO waiting for storagenode, processes-exit-eventlistener to die stdout
11:49:13 2023-05-23T11:49:12.990+0500 INFO preflight:localtime local system clock is in sync with trusted satellites' system clock. {"Process": "storagenode"} stdout
11:49:13 2023-05-23T11:49:12.990+0500 INFO trust Scheduling next refresh {"Process": "storagenode", "after": "6h11m31.570351252s"} stdout
11:49:13 2023-05-23T11:49:12.990+0500 INFO Node 1MBwuz3R4AAXfhvfHFFLjtPyUFetYJnHcc94PZEYQN6kEJqTW4 started {"Process": "storagenode"} stdout
11:49:13 2023-05-23T11:49:12.990+0500 INFO Public server started on [::]:28967 {"Process": "storagenode"} stdout
11:49:13 2023-05-23T11:49:12.990+0500 INFO bandwidth Performing bandwidth usage rollups {"Process": "storagenode"} stdout
11:49:13 2023-05-23T11:49:12.990+0500 INFO Private server started on 127.0.0.1:7778 {"Process": "storagenode"} stdout
11:49:13 2023-05-23T11:49:12.990+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"} stdout
11:49:13 2023-05-23T11:49:13.019+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"} stdout
11:49:13 2023-05-23T11:49:13.047+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo"} stdout
11:49:13 2023-05-23T11:49:13.064+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"} stdout
11:49:13 2023-05-23T11:49:13.166+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:13 2023-05-23T11:49:13.244+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"} stdout
11:49:13 2023-05-23T11:49:13.301+0500 INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"} stdout
11:49:13 2023-05-23T11:49:13.518+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:13 2023-05-23T11:49:13.681+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:13 2023-05-23T11:49:13.998+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 1, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:15 2023-05-23T11:49:15.037+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:15 2023-05-23T11:49:15.363+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:16 2023-05-23T11:49:16.002+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 2, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:16 2023-05-23 11:49:16,002 INFO waiting for storagenode, processes-exit-eventlistener to die stdout
11:49:17 2023-05-23T11:49:17.091+0500 INFO piecestore download started {"Process": "storagenode", "Piece ID": "FGZC55POSGD2YC2AKP323SIG6YNPPO3VOE3QL57LECFKJ5KXMT5A", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "Action": "GET_AUDIT", "Offset": 1552384, "Size": 256, "Remote Address": "172.18.0.1:55600"} stdout
11:49:18 2023-05-23T11:49:17.344+0500 INFO piecestore downloaded {"Process": "storagenode", "Piece ID": "FGZC55POSGD2YC2AKP323SIG6YNPPO3VOE3QL57LECFKJ5KXMT5A", "Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "Action": "GET_AUDIT", "Offset": 1552384, "Size": 256, "Remote Address": "172.18.0.1:55600"} stdout
11:49:18 2023-05-23T11:49:17.564+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo", "attempts": 3, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:19 2023-05-23T11:49:19.018+0500 ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 3, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:143\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:99\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75"} stdout
11:49:19 2023-05-23 11:49:19,018 INFO waiting for storagenode, processes-exit-eventlistener to die stdout
11:49:20 2023-05-23 11:49:20,020 WARN killing 'storagenode' (54) with SIGKILL stdout
11:49:20 2023-05-23 11:49:20,022 INFO stopped: storagenode (terminated by SIGKILL) stdout
11:49:20 2023-05-23 11:49:20,023 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM) stdout
11:49:45 2023-05-23 11:49:45,494 INFO Set uid to user 0 succeeded stdout
11:49:45 2023-05-23 11:49:45,517 INFO RPC interface 'supervisor' initialized stdout
11:49:45 2023-05-23 11:49:45,518 INFO supervisord started with pid 1 stdout
11:49:46 2023-05-23 11:49:46,520 INFO spawned: 'processes-exit-eventlistener' with pid 11 stdout
11:49:46 2023-05-23 11:49:46,522 INFO spawned: 'storagenode' with pid 12 stdout
11:49:46 2023-05-23 11:49:46,523 INFO spawned: 'storagenode-updater' with pid 13 stdout
11:49:46 2023-05-23T11:49:46.577+0500 INFO Configuration loaded {"Process": "storagenode", "Location": "/app/config/config.yaml"} stdout
11:49:46 2023-05-23T11:49:46.577+0500 INFO Anonymized tracing enabled {"Process": "storagenode"} stdout
11:49:46 2023-05-23T11:49:46.578+0500 INFO Operator email {"Process": "storagenode", "Address": "asperfilev@gmail.com"} stdout
11:49:46 2023-05-23T11:49:46.578+0500 INFO Operator wallet {"Process": "storagenode", "Address": "0x**********************"} stdout
11:49:46 2023-05-23T11:49:46.755+0500 INFO Configuration loaded {"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
Отключил IP6 посмотрим.
Появилась ещё интересная строчка в логах.
Error: unrecoverable error accessing data on the storage file system (path=config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1; error=lstat config/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1: input/output error). This is most likely due to disk bad sectors or a corrupted file system. Check your disk for bad sectors and integrity
Вам нужно проверить диск на ошибки и исправить их. Возможно identity тоже повреждена.
Если после исправления ошибок на диске проблема сохранится, тогда продолжим:
Есть хоть одна ошибка ping satellite failed
но без ratelimit
? Потому что ratelimit
это последствие частых попыток регистрации с ошибкой (неправильный внешний адрес например), а не причина.
Я бы порекомендовал попробовать использовать просто публичный IP. Возможно сателлит не может надёжно ресолвить предоставленный hostname в IP и использует предыдущий из кэша, а он сменился. Или пробует использовать IPv6, а он у вас не настроен на этом хосте или не разрешены порты узла для IPv6 UDP/TCP.
У вас сохранился бэкап этой identity? Если да, сохраните текущую и восстановите из бэкапа, только убедитесь, что она та же самая, например так:
Если она была не подписана - подпишите новым токеном авторизации и используйте эту версию.
Если бэкапа нет, попробуйте сделать так:
- Остановите и удалите контейнер storagenode
- Сделайте резервную копию папки identity
- удалите из текущей папки с identity файлы
identity.cert
иca.cert
- Скопируйте файл вида
identity.1560971067.cert
какidentity.cert
, а файл видаca.1560971067.cert
какca.cert
(цифры у вас будут свои - это дата в Unix формате, когда вы подписывали identity в первый раз). - Подпишите эту identity новым ключом авторизации
- Проверьте, что NodeID не изменился, таким запросом: RaspberryPi crash - #5 by Alexey
- Если NodeID тот же - стартуйте узел и понаблюдайте за логами.
Стал проверять по мануалу: Алгоритм починки BTRFS — NixTux
И проверка идёт уже 3 день, нумаю что ноде кабзда. Надо было синхронизировать на отдельный диск , а после проверять уже проблемный диск.
unresolved ref dir 707568 index 0 namelen 54 name x3ol5nwdpjmtccmq5vb2tcylv2umdebxeqmo6ehjbt7zeszysa.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589695
root 257 inode 3589695 errors 2001, no inode item, link count wrong
unresolved ref dir 52472 index 0 namelen 54 name jzo77nuduenrx7corf725yf37ktrlbg4zzo6qzpgsmuoveio7a.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589740
root 257 inode 3589740 errors 2001, no inode item, link count wrong
unresolved ref dir 432523 index 0 namelen 54 name uuorhcacji7opgohxbqd7654e6rj55h2hbqeuhj2trspwhan4a.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589742
root 257 inode 3589742 errors 2001, no inode item, link count wrong
unresolved ref dir 432266 index 0 namelen 54 name pm5pc6rvdmne2znxpwq5iwe2ckahaz4byzfrp5p2qqtxnsmtaq.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589748
root 257 inode 3589748 errors 2001, no inode item, link count wrong
unresolved ref dir 1248306 index 0 namelen 54 name jdlkda5bq2lilfefuinrdly5eacssxykuoupc3wgbhk7pc5fxq.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589751
root 257 inode 3589751 errors 2001, no inode item, link count wrong
unresolved ref dir 432976 index 0 namelen 54 name o57hguuggf6dkbyrvk5k66dsebfmjwuxpstgvlriujb3jslb7q.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589753
root 257 inode 3589753 errors 2001, no inode item, link count wrong
unresolved ref dir 707923 index 0 namelen 54 name z5rbimjjx3rhj5kkss4bwqdwwwt22mzmn6j2l47rb36qxqjhza.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589772
root 257 inode 3589772 errors 2001, no inode item, link count wrong
unresolved ref dir 707701 index 0 namelen 54 name ol7v3cio5bk2oy5t33srntee45fsqr4kaifs4ae624hejmhioa.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589807
root 257 inode 3589807 errors 2001, no inode item, link count wrong
unresolved ref dir 707670 index 0 namelen 54 name tasotf337zfexhdbfwj4mnhmvre6vidpx3l7c6onwufuq3olda.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589816
root 257 inode 3589816 errors 2001, no inode item, link count wrong
unresolved ref dir 707727 index 0 namelen 54 name hkjetik7ceesza4ja6yw7zz6hxzbyd7k7mbfbgfgjeqqqawmna.sj1 filetype 1 errors 6, no dir index, no inode ref
Trying to rebuild inode:3589825
Ох. BTRFS, понятно. Оно не готово к нормальному использованию, к сожалению. Лучше тогда zfs, если хочется попробовать что-то новое. Для узлов лучше всего ext4, LVM полезно иметь тоже. Не сильно много гибкости, но бывает полезно, особенно когда хочется переехать на другой диск без downtime.
Подставил рядом другой диск, слил с проблемного инфу по мануалу. How do I migrate my node to a new device? - Storj Node Operator Docs
ash-4.4# rsync -aP --delete /volume2/storj/data /volumeUSB1/usbshare/node1
sending incremental file list
deleting data/revocations.db
data/
data/trust-cache.json
1,374 100% 0.00kB/s 0:00:00 (total: 0%) (xfr#1, ir-chk=1309/1312)
deleting data/storage/used_serial.db
deleting data/storage/storage_usage.db
deleting data/storage/secret.db
deleting data/storage/satellites.db
deleting data/storage/reputation.db
deleting data/storage/pricing.db
deleting data/storage/pieceinfo.db
deleting data/storage/piece_spaced_used.db
deleting data/storage/piece_expiration.db
deleting data/storage/orders.db
deleting data/storage/notifications.db
deleting data/storage/info.db
deleting data/storage/heldamount.db
deleting data/storage/bandwidth.db
data/orders/unsent/unsent-orders-121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6-1685700000000000000.v1
3,354 100% 105.66kB/s 0:00:00 (total: 0%) (xfr#2, ir-chk=1045/1312)
data/orders/unsent/unsent-orders-12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo-1685700000000000000.v1
7,157 100% 174.73kB/s 0:00:00 (total: 0%) (xfr#3, ir-chk=1037/1312)
data/orders/unsent/unsent-orders-1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE-1685700000000000000.v1
4,310 100% 102.66kB/s 0:00:00 (total: 0%) (xfr#4, ir-chk=1035/1312)
data/storage/
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2a2i6cribujsomsow5c45wr4p5rqdtjoti7dyciultzapzjqia.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2r6kdpqrxswwblvtyj5agy5lzrgj7q2wrhew4vwuzcb6rrgmxq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/34eo7q6oouoghnxgdnnr3pgca6d2ewwpwmyfgphe7nfh7fksla.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/3kpz4hawrqg2torskefmkggnjjv6bryz3j62cx3gquog75chba.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/4hfrprvi6cr2ktlpgbwel6olhhzduogqo6ubtv77ah6y77yz6a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/5frjuo32txrsugohkp45x2g7boq2gjuh6caxq7enlcrpuqodza.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/632bljgzcq5nhlihniget4eyutw554lblv4usiaq2mmatpuhfa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/657goqli7moa26byzyxke4ufb56q3kmhyeiemmq2wvcp37z27a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/acyfdw63ft55kv4x5brqc5dtoegm57njrapiv6w2bib3vnxmxq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/bhl4blnqpvnindp3cmzypwzpuvcjetla2nuk4a4s6idtq727rq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/d44uyjgg5ffzbasqc72scan7zukc6w5o4ifo33qkk5f5i4x5vq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/dbbawmppokaajppecbaxprlmrcwnwi4ksktfgsxnzrvek7mypa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/fp4gvmwww6oxrpzjc76gokdv7uihyub4zoy5avyfmjiu32riha.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/fxx4terxs6yaahmj4272lkhsufytga5xhu7fwvfi3naiiudahq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/gs76u6ao22s5diza64vaxbxncp3agikki5ttf3wgi5u4tfa6vq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ikoswl5s36mubiayp435b344xwfuzxn2puygkzpy6sigjkvnla.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/il5qdksjc4gcrna4hlf3frf627z577qjyw2yfrsgkea3hysf2a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/jk6eccyzetwr7ev64st7izvzmohhd7vfc6w4wvt26vxlisa3dq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/jsmtb76wexfsx5aizmvhux6gtl57n4mv4s2b3efg5slmv6cupq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/krs6drargyq4bzx7ouspidbioes3l7aobkdshto7qoyyahktxa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/kt3emfwo77tzomwojkbsddc7uhd3zdpzxt367k2eqjfesrmjhq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/luecauggtypbsh77ep2mdfnpfk2zodtgi3iemw7klpsk4ll2ra.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/mgtidzwmoahvumcl6fdifmdxcxwoyunzkanidulhwtf42jw7tq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/n6mnrk4ndbhovvnjdo5glopfc55xllzhlay2bqoepmbqwluqeq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/nbhtstaffuhcz5scopqegro422jp44vcuxmt5md7bsazfbgpgq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/o7ur6a5lnhxfbeknwmfvopqgllgwpjzw4k56fmaknqa5pyab2a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ojvnjnww2mtyg3klbclznkg6jwbh4mmfh6yq4hsaxon7ufyiua.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/qottjrn4jzhwgcpyqdwuu2q6wvc355rm27dsqbhusjczp2ofwa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/qscqkbdj5gvd6lbwqbybxs4ulubwtjbow3vxgslplsfrt7yusa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/qtythjaid6ivg6gdd3bwrywxxpgct2j2v3yfzs7aydshxczd3q.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/rgy3ykxpdgmpwfvr7fihpqtvdsqu43nquoeu5n7xqazp64sy5a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/rujf74p7gjp4gygizlkurctravxkwii2cvdz2rtd5hijyzlfpq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/se57odxg5dzpl2dlvitfrymigamy7ortxne7no7q4xdiogct4q.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/sgfuos74one5nefnwhixylpcznk4gngcr55toqsa7gqyar77wa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/tyyhwim67imka3ctichc3z2v6u45ybjfwtfzxqbdww2li7u23q.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/uczeabm6w3jzlnxvqhuckhalzeie5nvjwdh6bcaausr45oxtba.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/uxpg5rhbcfpkklz5mg5j2fr2wcjmdne2zkt7agqv4iwbdhzbaq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/vi4hygtfsvvrp2nbguymjgyw645jizujiozx2ldgy6kkwk5drq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ygcfca2zin346qozmzsz4xiat34t73vvmclbuotvoklljllxca.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/zkghj6fnrdgw4roreugwbqnjpfsxeag35csftdx7honfdc365a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/qoeyy2bonwu7m7xevcgvsvakdjugpnm5n4ieaipnuy6q6vixuq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/oygwfvrgqubzziy6ovxhuqpx4ygluea3bs7xfewzn6673rmsfa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/xhx6nmh3b6ciit2lycu2j5mo5ewcg2rxcysg44enxapyn3rlia.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/d7eyrnpvfxcmmwbpabqb42nkkmxzrb6cnzxqhwa6nenwryiusq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/fohpxhruzi7pwqxkxrasbegykxevhibqffgtnx7tsuhxeswjea.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/mgyns55uq4iir3rkd4xxmerbbkp4xcxe7okuljkt6uztizp3rq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/7mr6lxyfjqy4swnukd3gsmr3xtt66w2qgx5wqyc3jb6zp3pdyq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/fae2cagrqfte5iv4niiywveqavbtkg55kcdnplmppaygwm7bkq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/yx5upwh4zwdhrrhk4nuiaqruresjz7numoekxussuryeko6beq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/4idsbggwcl47k3ulxscqbmcvfm7l5xql63ms73rcoie7nvuwva.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ssf7v5rew3zqs4qzkxnbizy7lauplq6a7ga7mrnbgm3vq2klga.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/gldmerwop366cvbkowzm5kav4tcddlsnt7akwrok46ndowdyka.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/cp3iynla3ftlibkrn2o7jjdhof3pknf7sp7stt75vxt4jcif3a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/jgjn6escjszhrndvfpo3yghqdzu3m2ihquarisgditldu6cfeq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/45qbtlcesqoy2zsqxft2sjs7jw6uqvwhdyzytkkvfxkuzldl2a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/b4k4ewknsshsgb7xesb5logqhof3eajwjxqok352gja3ckgynq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/mfaykss2ksge3mgr6kwnnwehc7do424xdac7timctxulhxmnua.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/otykbyz6c7w6655m7rkh56fvfxdsle7tbbpnr5kwpvy5obtf7q.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/qndsvjwe6c5v2k654szguwcbmhsvxvabvsnmhxmqpgz4ogf2ua.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2vudlkj2so24rbyytk6bz7naeikzudi5r3hmnf4u6c2lnaf2cq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ryndk3wb7ja6iwtwarl2fe55aqa64h2ll5qnzmr7o5skg5ebna.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/zgwzn2cmcwuw37khcsaj3u7vrbmqsuv22zmqyv2jzxncjtzncq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/wyesxsdq6x6s6lo2bbjx4onxfhmtm4fzryma75yivkxqslp6aq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/j6qiihpfnhmmsyeneremdthjpvofebtb6x2okr56uciopgtpmq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/kqgxxr7apf3bv6zyus6h6eywkygalwdzxbsgmceqxxxjd5tlzq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/bl27m7dflu6q42f36exi5x7kzrc2m4soyzt54l6bx4zokmpcmq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/mj22bdyrq4ee3negxjxur6xvtfikdisxewtoucixdx5evto7wa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/36pag5dmsqbydxa6bwxej2l3gvkgcpapl3cm7vnbr7yctrqela.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/aycdmkk66ebe3pts7cksgb7jtarkxbouefyaiggosd4q6iqpsa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ahz7bk7zvliabix6wlwzdtbnzfbrq2rthkbgyy32rbbjopmw4a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/qsoi3ty3roxgtdvd5f7iwkvtl7nmrhf5qooeoj5wkkjhyx726a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/d5jqqsjlbdnxqrvbpp4gnca6fyptkl7xjb5zutuvqzv6ab52ha.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/aqifsiedgfsaav4b3yw3byjufelkdgzz2ay4v2oz2qx4hy27tq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/to22chfesvqkmzqu54dmiqa6oj6rxakym6ipje4beorut4y6ka.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/5elsxfphs3uier4dxdtnwsk7fzozihxuzetpddjqvgpxmoicoa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/tjsuctjn6gao5rjfzqucv2yx4csk7arkaqy573saah3naz46yq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/tygergm5lbfz54zjqvczc2msbbzimuyuu5jbo6vauowy7unzma.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/leekj5aoax27ae4gcbf53rkfz5bpmv6h6gkjborxb556kvtooq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/euo2bwwxsgbe2pylvbonggcwnkdbkte7fb5ehbeww442vkpckq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/u4swyfhgekl25kf4kq4wlp2vf2zyff77x2hw47fiyqhw5xeiwa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/zj6x7lmhwe2zvvcyielrbbn3lg4vctjmppqrmyi2y67dwuwfqq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/76dtlabtx66ypehiw3cpgq2wil6bqxlrhu7t5jz3w3szqzo5yq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ncst6qv6hv2quem7i7vq5bticy7t4ybwql744fqkwgy3f7kjra.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/tbiap4z35o6actffmup6ckjgufexcf2d5sxq6ls6lhbc55w4ka.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ghbnrpyuaegr55e6py5j3l4pa6c4ydp6sonmmom3qcqftrmyeq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/xrdvq6tz25xct5kczk57niyfxxqpmkhq6qnf5pqru53innmjsa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/4vzoe7w5t6dbu4dvwvruoxmlihnj47omuv5dxemawpsppq2zxq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/v5c3ojxywsielp3jvgblo3q6uzro3ftirckbj4hyamhn2p5d5a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/246vwped3mxwngygmhgw225q5xeim4aari6ibmks7cj2455mya.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/7kgkpexm6sa6twtsgz5qugibalxz6z7j5o7yjejox7en2f6aeq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/nkcnpm36e2p7pm6i3yvzkwobtyvk64h3eaiegrvze3nssadpvq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/nmwainpx4tzuztmwv3e7nnbnqoja4oyyru5xm6g2aj7xnb6fjq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/o3hcyipcp2ctcwlxwbyijrptwbawgfxm3fcsmzwlzze6vuoncq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/wigffuulbznjsqzwcuaatqduvruk2t5goo2xi2s7vgzj47taqa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/nqmiz2ae2qdlpysldyacsjctkfyposkynuyw5vdhyhb4xxsjda.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/w75gukqrbinbgfumlz3eyztiedvvjk4aaarcm6vc6ajsxejgaq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/zhmmvs7ke5uvwr2kaxnytpcslsxxiufivvkz33f2mzyc7oufha.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/et53akntyim5cvjzf7sysgsxd3likynr2coynhjqgqkmswfuba.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/bab4eooynpml36g6kxnfngrov3xktr23pwphbmpqzf4nvpgdjq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/vhzghicvmj4oblkpydzrp2meimuq4rbwg53gudmvmvsgnkidna.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2kiwogy4mjjrgmglnaocfswqwsroshaqfmpxxmj5g3uuy5htwa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/inond4fzd56mbqxdzxxi7hjudgn22hwi2uoxtk6hu6hkpggtrq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/mv73cfzzfwteitn6t4m3gqxbivf4viru7e5ig34uyaczwggtpa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ubmedp42lqvtvkujmu622grw5eosmml4eihvleiyy6rz76jdza.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/eymphmuq7oijt7x3tlqnqlzo5nokpvzhqza3cs3rmwwnbxrzmq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/x2l73sllvwgljzseaj4fjj22v2xbodemh2wiaxifzhhkobthza.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/2b23a6z74beaqwz7olbkvxgfx6l2ukfn7ain747aohgeajeana.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/xzzo6uumwunp7aparbwo6wpx6vaazix4j3zc5cqymdpxchmkrq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/3ddpmdr3kd4qzliqdzecnjd6mpfsq5ncb6id6gsqajn2iprcxa.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/trlsnsjzssaci6hrpirkfxqexcpdfdq3i2a5puc4oyvcnakihq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/ojw5yz4pjsdsro6gudu7e5v4vpzs5iv43dpgr4dplxqukfuleq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/zo5qmgqb6ajhd3jmyef6e63cgzxnnrurrdpuea5hjl64mrechq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/7qojgibm4yupj26l3ymliqk3zkhoy7q3gvyqeyoh6f25jnsknq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/5jzn4d5kdqxyurqa5kxllvj5agvbltl4kv7gddenrctyz6ebtq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/xkksbrh3xp3z376vfi3yw3nlxaou4jh7sqktq4xkjoyh36sm6a.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/4j33z4gzd2ple76igc4d2xuu4vihuoysf7vvr36uwquu6kiatq.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/kh7ltku6qokygdae3rzwi4ahi5v6vztt3dieigz2vhr32pgz7q.sj1") failed: Input/output error (5)
rsync: readlink_stat("/volume2/storj/data/storage/blobs/ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa/3o/sdxloiifspfhgyehgdorugfhw33tze7bkm3thendduezsl5h3a.sj1") failed: Input/output error (5)
IO error encountered -- skipping file deletion
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1462) [sender=3.1.2]
Запустил на новом диске ноду. Нода запустиласть, но 2 TB данных не стало. (
Пока работает.
Вобщем дисквалифицировали ноду по всем сателитам.
Вопрос, если я сделаю identity с новым ключом авторизации, я могу запустить ноду со старыми данными и новым ключом?
5% потерянных данных неминуемо дисквалифицирует узел.
да, к сожалению, потеря данных ведёт к необратимой дисквалификации.
это будет та же самая дисквалифицированная identity.
Если вы хотите запустить новый узел, тогда надо сгенерировать новую identity (а не скопировать старую), подписать её новым токеном авторизации и запустить узел с чистым хранилищем (старые данные не будут признаны новой identity и будут медленно удаляться в течение нескольких недель с помощью Garbage Collector, нет смысла растягивать “удовольствие”, они не будут оплачиваться).
Понял. Спасибо. Следующие ноды будут на EXT4.
К сожалению, BTRFS пока что production-unready, test purposes only.
Пока что неплохо себя зарекомендовали “родные” файловые системы (ext4 in Linux, NTFS in Windows), после настройки zfs тоже становится более-менее (скорее менее в плане производительности) пригодной.