Hello, today I decided to setup a new node and I noticed that in the logs it keeps running into a fatal error, in previous versions every time I saw “Configuration loaded from: /app/config/config.yaml” I would take it as a restart, is this still the case? If so why is my node restarting?
2019-08-29T23:17:35.739Z e[34mINFOe[0m Configuration loaded from: /app/config/config.yaml
2019-08-29T23:17:35.749Z e[34mINFOe[0m Operator email: user@example.com
2019-08-29T23:17:35.790Z e[34mINFOe[0m operator wallet:
2019-08-29T23:17:36.100Z e[34mINFOe[0m version running on version v0.19.0
2019-08-29T23:17:36.106Z e[34mINFOe[0m db.migration Latest Version {“version”: 18}
2019-08-29T23:17:36.110Z e[34mINFOe[0m Node 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F started
2019-08-29T23:17:36.112Z e[34mINFOe[0m Public server started on [::]:28967
2019-08-29T23:17:36.112Z e[34mINFOe[0m Private server started on 127.0.0.1:7778
2019-08-29T23:17:36.111Z e[34mINFOe[0m bandwidth Performing bandwidth usage rollups
2019-08-29T23:17:36.113Z e[34mINFOe[0m piecestore:monitor Remaining Bandwidth {“bytes”: 25000000000000}
2019-08-29T23:17:36.136Z e[34mINFOe[0m version running on version v0.19.0
2019-08-29T23:18:21.408Z e[34mINFOe[0m Configuration loaded from: /app/config/config.yaml
2019-08-29T23:18:21.425Z e[34mINFOe[0m Node ID: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F
2019-08-29T23:18:29.107Z e[34mINFOe[0m Got a signal from the OS: “interrupt”
2019-08-29T23:18:30.015Z e[31mERRORe[0m nodestats:cache Get stats query failed {“error”: “node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F; node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F; node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F; node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F”, “errorVerbose”: “group:\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:75\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:106\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:105\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:77\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:75\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:106\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:105\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:77\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:75\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:106\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:105\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:77\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:75\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:106\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:105\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:77\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}
2019-08-29T23:18:30.562Z e[31mFATALe[0m Unrecoverable error {“error”: “rpc error: code = Canceled desc = context canceled”}
2019-08-29T23:18:36.943Z e[34mINFOe[0m Configuration loaded from: /app/config/config.yaml
2019-08-29T23:18:37.002Z e[34mINFOe[0m Node ID: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F
2019-08-29T23:18:43.469Z e[34mINFOe[0m Got a signal from the OS: “interrupt”
2019-08-29T23:18:46.171Z e[31mFATALe[0m Unrecoverable error {“error”: “rpc error: code = Canceled desc = context canceled”}
2019-08-29T23:20:24.809Z e[31mERRORe[0m nodestats:cache Get disk space usage query failed {“error”: “node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F; node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F; node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F”, “errorVerbose”: “group:\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:118\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:128\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:127\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:89\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:118\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:128\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:127\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:89\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:118\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:128\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:127\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:89\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}
2019-08-29T23:22:53.645Z e[34mINFOe[0m Configuration loaded from: /app/config/config.yaml
2019-08-29T23:22:53.701Z e[34mINFOe[0m Node ID: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F
2019-08-29T23:23:12.217Z e[34mINFOe[0m Got a signal from the OS: “interrupt”
2019-08-29T23:23:12.218Z e[31mFATALe[0m Unrecoverable error {“error”: “rpc error: code = Canceled desc = context canceled”}
2019-08-29T23:26:07.198Z e[34mINFOe[0m Got a signal from the OS: “terminated”
2019-08-29T23:27:05.048Z e[34mINFOe[0m Configuration loaded from: /app/config/config.yaml
2019-08-29T23:27:05.097Z e[34mINFOe[0m Operator email: user@example.com
2019-08-29T23:27:05.099Z e[34mINFOe[0m operator wallet:
2019-08-29T23:27:05.433Z e[34mINFOe[0m version running on version v0.19.0
2019-08-29T23:27:05.438Z e[34mINFOe[0m db.migration Latest Version {“version”: 18}
2019-08-29T23:27:05.440Z e[34mINFOe[0m piecestore:monitor Remaining Bandwidth {“bytes”: 25000000000000}
2019-08-29T23:27:05.441Z e[34mINFOe[0m bandwidth Performing bandwidth usage rollups
2019-08-29T23:27:05.442Z e[34mINFOe[0m Node 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F started
2019-08-29T23:27:05.442Z e[34mINFOe[0m Public server started on [::]:28967
2019-08-29T23:27:05.442Z e[34mINFOe[0m Private server started on 127.0.0.1:7778
2019-08-29T23:27:05.469Z e[34mINFOe[0m version running on version v0.19.0
2019-08-29T23:30:16.026Z e[31mERRORe[0m nodestats:cache Get disk space usage query failed {“error”: “node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F; node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F”, “errorVerbose”: “group:\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:118\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:128\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:127\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:89\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetDailyStorageUsage:118\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage.func1:128\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheSpaceUsage:127\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func2:89\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}
2019-08-29T23:31:46.230Z e[31mERRORe[0m nodestats:cache Get stats query failed {“error”: “node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F; node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F”, “errorVerbose”: “group:\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:75\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:106\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:105\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:77\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n— node stats service error: rpc error: code = PermissionDenied desc = node not found: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:75\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:106\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:168\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:105\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:77\n\tstorj.io/storj/internal/sync2.(*Cycle).Run:87\n\tstorj.io/storj/internal/sync2.(*Cycle).Start.func1:68\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}
2019-08-29T23:37:08.344Z e[31mERRORe[0m server rpc error: code = PermissionDenied desc = untrusted peer 1QzDKGHDeyuRxbvZhcwHU3syxTYtU1jHy5duAKuPxja3XC8ttk
2019-08-29T23:42:05.709Z e[34mINFOe[0m version running on version v0.19.0
2019-08-29T23:42:13.127Z e[34mINFOe[0m piecestore upload started {“Piece ID”: “EUOHKKDI3HJPTCXBYV4DKWQTCEIKJVFPQKV3MJMXYAEKRHYDYKEA”, “SatelliteID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “PUT”}
2019-08-29T23:42:20.372Z e[34mINFOe[0m piecestore upload failed {“Piece ID”: “EUOHKKDI3HJPTCXBYV4DKWQTCEIKJVFPQKV3MJMXYAEKRHYDYKEA”, “SatelliteID”: “118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW”, “Action”: “PUT”, “error”: “piecestore protocol: rpc error: code = Internal desc = transport: transport: the stream is done or WriteHeader was already called”, “errorVerbose”: “piecestore protocol: rpc error: code = Internal desc = transport: transport: the stream is done or WriteHeader was already called\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:324\n\tstorj.io/storj/pkg/pb._Piecestore_Upload_Handler:851\n\tstorj.io/storj/pkg/server.(*Server).logOnErrorStreamInterceptor:23\n\tgoogle.golang.org/grpc.(*Server).processStreamingRPC:1127\n\tgoogle.golang.org/grpc.(*Server).handleStream:1178\n\tgoogle.golang.org/grpc.(*Server).serveStreams.func1.1:696”}
2019-08-29T23:42:20.372Z e[31mERRORe[0m server piecestore protocol: rpc error: code = Internal desc = transport: transport: the stream is done or WriteHeader was already called
storj.io/storj/storagenode/piecestore.(*Endpoint).Upload:324
storj.io/storj/pkg/pb._Piecestore_Upload_Handler:851
storj.io/storj/pkg/server.(*Server).logOnErrorStreamInterceptor:23
google.golang.org/grpc.(*Server).processStreamingRPC:1127
google.golang.org/grpc.(*Server).handleStream:1178
google.golang.org/grpc.(*Server).serveStreams.func1.1:696
2019-08-29T23:42:21.046Z e[34mINFOe[0m Configuration loaded from: /app/config/config.yaml
2019-08-29T23:42:21.060Z e[34mINFOe[0m Node ID: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F
2019-08-29T23:42:27.631Z e[31mERRORe[0m server rpc error: code = PermissionDenied desc = untrusted peer 1QzDKGHDeyuRxbvZhcwHU3syxTYtU1jHy5duAKuPxja3XC8ttk
2019-08-29T23:42:27.885Z e[34mINFOe[0m Got a signal from the OS: “interrupt”
2019-08-29T23:42:30.569Z e[31mFATALe[0m Unrecoverable error {“error”: “rpc error: code = Canceled desc = context canceled”}
2019-08-29T23:47:38.365Z e[31mERRORe[0m server rpc error: code = PermissionDenied desc = untrusted peer 1QzDKGHDeyuRxbvZhcwHU3syxTYtU1jHy5duAKuPxja3XC8ttk
2019-08-29T23:52:50.794Z e[31mERRORe[0m server rpc error: code = PermissionDenied desc = untrusted peer 1QzDKGHDeyuRxbvZhcwHU3syxTYtU1jHy5duAKuPxja3XC8ttk
2019-08-29T23:55:20.738Z e[34mINFOe[0m Configuration loaded from: /app/config/config.yaml
2019-08-29T23:55:20.751Z e[34mINFOe[0m Node ID: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F
2019-08-29T23:55:27.674Z e[34mINFOe[0m Got a signal from the OS: “interrupt”
2019-08-29T23:55:30.082Z e[31mFATALe[0m Unrecoverable error {“error”: “rpc error: code = Canceled desc = context canceled”}
2019-08-30T00:02:49.815Z e[31mERRORe[0m server rpc error: code = PermissionDenied desc = untrusted peer 1QzDKGHDeyuRxbvZhcwHU3syxTYtU1jHy5duAKuPxja3XC8ttk
2019-08-30T00:03:25.336Z e[34mINFOe[0m Configuration loaded from: /app/config/config.yaml
2019-08-30T00:03:25.356Z e[34mINFOe[0m Node ID: 1Bnd7fDXsGt4ofcrkjW4TK73mjeJLFLYJz251sxZDzzBvrsP7F
2019-08-30T00:03:30.376Z e[34mINFOe[0m Got a signal from the OS: “interrupt”
2019-08-30T00:03:31.884Z e[31mFATALe[0m Unrecoverable error {“error”: “rpc error: code = Canceled desc = context canceled”}