The error: ERROR contact:service ping satellite failed started to appear on one of my nodes, while nothing has been touched on it. DNS and port forwarding all seem to work just fine on the server, but the error is there, the node shows as offline on the page and QUIC as misconfigured.
Logs:
2025-12-26 13:05:15,556 INFO stopped: storagenode (exit status 0)
2025-12-26 13:05:15,557 WARN stopped: processes-exit-eventlistener (terminated by SIGTERM)
2025-12-26 13:05:56,917 INFO Set uid to user 0 succeeded
2025-12-26 13:05:56,932 INFO RPC interface 'supervisor' initialized
2025-12-26 13:05:56,933 INFO supervisord started with pid 1
2025-12-26 13:05:57,936 INFO spawned: 'processes-exit-eventlistener' with pid 40
2025-12-26 13:05:57,940 INFO spawned: 'storagenode' with pid 41
2025-12-26 13:05:57,947 INFO spawned: 'storagenode-updater' with pid 42
2025-12-26T13:05:58Z INFO Configuration loaded {"Process": "storagenode-updater", "Location": "/app/config/config.yaml"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "console.address"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet-features"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.private-address"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "contact.external-address"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "healthcheck.enabled"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "healthcheck.details"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.wallet"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "operator.email"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "storage.allocated-disk-space"}
2025-12-26T13:05:58Z INFO Invalid configuration file key {"Process": "storagenode-updater", "Key": "server.address"}
2025-12-26T13:05:58Z INFO Anonymized tracing enabled {"Process": "storagenode-updater"}
2025-12-26T13:05:58Z INFO Running on version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.142.7"}
2025-12-26T13:05:58Z INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
2025-12-26T13:05:58Z INFO Configuration loaded {"Process": "storagenode", "Location": "/app/config/config.yaml"}
2025-12-26T13:05:58Z INFO Anonymized tracing enabled {"Process": "storagenode"}
2025-12-26T13:05:58Z INFO Operator email {"Process": "storagenode", "Address": "storjwallet@gmail.com"}
2025-12-26T13:05:58Z INFO Operator wallet {"Process": "storagenode", "Address": "0x8D5E5264D5C2DBc527611cBB855a9c9C7027B9d1"}
2025-12-26T13:05:58Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.142.7"}
2025-12-26T13:05:58Z INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode"}
2025-12-26T13:05:58Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.142.7"}
2025-12-26T13:05:58Z INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode-updater"}
2025-12-26 13:05:59,877 INFO success: processes-exit-eventlistener entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-12-26 13:05:59,877 INFO success: storagenode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-12-26 13:05:59,878 INFO success: storagenode-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2025-12-26T13:06:00Z INFO server kernel support for server-side tcp fast open remains disabled. {"Process": "storagenode"}
2025-12-26T13:06:00Z INFO server enable with: sysctl -w net.ipv4.tcp_fastopen=3 {"Process": "storagenode"}
2025-12-26T13:06:03Z INFO hashstore hashstore opened successfully {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "open_time": "2.412731251s"}
2025-12-26T13:06:59Z INFO hashstore hashstore opened successfully {"Process": "storagenode", "satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "open_time": "56.50116524s"}
2025-12-26T13:07:32Z INFO hashstore hashstore opened successfully {"Process": "storagenode", "satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "open_time": "32.872753239s"}
2025-12-26T13:07:34Z INFO Telemetry enabled {"Process": "storagenode", "instance ID": "12e8b3gMjXGQzhbAfVDswDbyT5Y9CFViq9HBq8Kb3eHTAUVKJNf"}
2025-12-26T13:07:34Z INFO Event collection enabled {"Process": "storagenode", "instance ID": "12e8b3gMjXGQzhbAfVDswDbyT5Y9CFViq9HBq8Kb3eHTAUVKJNf"}
2025-12-26T13:07:34Z INFO db.migration Database Version {"Process": "storagenode", "version": 62}
2025-12-26T13:07:40Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock. {"Process": "storagenode"}
2025-12-26T13:07:41Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock. {"Process": "storagenode"}
2025-12-26T13:07:41Z INFO collector expired pieces collection started {"Process": "storagenode"}
2025-12-26T13:07:41Z INFO Node 12e8b3gMjXGQzhbAfVDswDbyT5Y9CFViq9HBq8Kb3eHTAUVKJNf started {"Process": "storagenode"}
2025-12-26T13:07:41Z INFO Public server started on [::]:28967 {"Process": "storagenode"}
2025-12-26T13:07:41Z INFO Private server started on 127.0.0.1:7778 {"Process": "storagenode"}
2025-12-26T13:07:41Z INFO failed to sufficiently increase send buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details. {"Process": "storagenode"}
2025-12-26T13:07:41Z INFO trust Scheduling next refresh {"Process": "storagenode", "after": "6h15m53.07983113s"}
2025-12-26T13:07:41Z INFO bandwidth Persisting bandwidth usage cache to db {"Process": "storagenode"}
2025-12-26T13:07:41Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-12-26T13:07:41Z INFO lazyfilewalker.trash-cleanup-filewalker starting subprocess {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-12-26T13:07:41Z INFO lazyfilewalker.trash-cleanup-filewalker subprocess started {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-12-26T13:07:41Z INFO pieces used-space-filewalker started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-12-26T13:07:41Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-12-26T13:07:41Z INFO piecemigrate:chore all enqueued for migration; will sleep before next pooling {"Process": "storagenode", "active": {"121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6": false, "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S": false, "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs": false, "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE": false}, "interval": "10m0s"}
2025-12-26T13:07:41Z INFO lazyfilewalker.used-space-filewalker subprocess started {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-12-26T13:07:41Z INFO collector expired pieces collection completed {"Process": "storagenode", "count": 0}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess trash-filewalker started {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode", "dateBefore": "2025-12-19T13:07:41Z"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess Database started {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess trash-filewalker completed {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode", "bytesDeleted": 0, "numKeysDeleted": 0}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker subprocess finished successfully {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-12-26T13:07:42Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "elapsed": "755.529523ms"}
2025-12-26T13:07:42Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker starting subprocess {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2025-12-26T13:07:42Z INFO lazyfilewalker.used-space-filewalker.subprocess Database started {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker subprocess started {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess trash-filewalker started {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode", "dateBefore": "2025-12-19T13:07:42Z"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess Database started {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess trash-filewalker completed {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode", "bytesDeleted": 0, "numKeysDeleted": 0}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker subprocess finished successfully {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2025-12-26T13:07:42Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "elapsed": "291.490081ms"}
2025-12-26T13:07:42Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker starting subprocess {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker subprocess started {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess trash-filewalker started {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Process": "storagenode", "dateBefore": "2025-12-19T13:07:42Z"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess Database started {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Process": "storagenode"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess trash-filewalker completed {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "numKeysDeleted": 0, "Process": "storagenode", "bytesDeleted": 0}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker subprocess finished successfully {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-12-26T13:07:42Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "elapsed": "280.95992ms"}
2025-12-26T13:07:42Z INFO pieces:trash emptying trash started {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker starting subprocess {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-12-26T13:07:42Z INFO lazyfilewalker.trash-cleanup-filewalker subprocess started {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-12-26T13:07:43Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess trash-filewalker started {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode", "dateBefore": "2025-12-19T13:07:42Z"}
2025-12-26T13:07:43Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess Database started {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode"}
2025-12-26T13:07:43Z INFO lazyfilewalker.trash-cleanup-filewalker.subprocess trash-filewalker completed {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Process": "storagenode", "bytesDeleted": 0, "numKeysDeleted": 0}
2025-12-26T13:07:43Z INFO lazyfilewalker.trash-cleanup-filewalker subprocess finished successfully {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-12-26T13:07:43Z INFO pieces:trash emptying trash finished {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "elapsed": "386.267547ms"}
2025-12-26T13:07:43Z INFO lazyfilewalker.used-space-filewalker subprocess finished successfully {"Process": "storagenode", "satelliteID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2025-12-26T13:07:43Z INFO pieces used-space-filewalker completed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Lazy File Walker": true, "Total Pieces Size": 151645787648, "Total Pieces Content Size": 151375973888, "Total Pieces Count": 526980, "Duration": "1.866250663s"}
2025-12-26T13:07:43Z INFO pieces used-space-filewalker started {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2025-12-26T13:07:43Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2025-12-26T13:07:43Z INFO lazyfilewalker.used-space-filewalker subprocess started {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2025-12-26T13:07:43Z INFO lazyfilewalker.used-space-filewalker.subprocess Database started {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Process": "storagenode"}
2025-12-26T13:07:44Z INFO lazyfilewalker.used-space-filewalker subprocess finished successfully {"Process": "storagenode", "satelliteID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE"}
2025-12-26T13:07:44Z INFO pieces used-space-filewalker completed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Lazy File Walker": true, "Total Pieces Size": 18010456064, "Total Pieces Content Size": 18006419456, "Total Pieces Count": 7884, "Duration": "601.296427ms"}
2025-12-26T13:07:44Z INFO pieces used-space-filewalker started {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-12-26T13:07:44Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-12-26T13:07:44Z INFO lazyfilewalker.used-space-filewalker subprocess started {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-12-26T13:07:44Z INFO lazyfilewalker.used-space-filewalker.subprocess Database started {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Process": "storagenode"}
2025-12-26T13:07:44Z INFO lazyfilewalker.used-space-filewalker subprocess finished successfully {"Process": "storagenode", "satelliteID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2025-12-26T13:07:44Z INFO pieces used-space-filewalker completed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Lazy File Walker": true, "Total Pieces Size": 34542267904, "Total Pieces Content Size": 34502336000, "Total Pieces Count": 77992, "Duration": "543.071149ms"}
2025-12-26T13:07:44Z INFO pieces used-space-filewalker started {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-12-26T13:07:44Z INFO lazyfilewalker.used-space-filewalker starting subprocess {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-12-26T13:07:44Z INFO lazyfilewalker.used-space-filewalker subprocess started {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-12-26T13:07:44Z INFO lazyfilewalker.used-space-filewalker.subprocess Database started {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Process": "storagenode"}
2025-12-26T13:07:45Z INFO lazyfilewalker.used-space-filewalker subprocess finished successfully {"Process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2025-12-26T13:07:45Z INFO pieces used-space-filewalker completed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Lazy File Walker": true, "Total Pieces Size": 19264088832, "Total Pieces Content Size": 19237966080, "Total Pieces Count": 51021, "Duration": "589.134932ms"}
2025-12-26T13:08:03Z INFO hashstore beginning compaction {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "store": "s0", "stats": {"NumLogs":0,"LenLogs":"0 B","NumLogsTTL":0,"LenLogsTTL":"0 B","SetPercent":0,"TrashPercent":0,"TTLPercent":0,"Compacting":false,"Compactions":0,"Today":20448,"LastCompact":0,"LogsRewritten":0,"DataRewritten":"0 B","DataReclaimed":"0 B","DataReclaimable":"0 B","Table":{"NumSet":0,"LenSet":"0 B","AvgSet":0,"NumTrash":0,"LenTrash":"0 B","AvgTrash":0,"NumTTL":0,"LenTTL":"0 B","AvgTTL":0,"NumSlots":16384,"TableSize":"1.0 MiB","Load":0,"Created":20382,"Kind":0},"Compaction":{"Elapsed":0,"Remaining":0,"TotalRecords":0,"ProcessedRecords":0}}}
2025-12-26T13:08:03Z INFO hashstore compact once started {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "store": "s0", "today": 20448}
2025-12-26T13:08:03Z INFO hashstore compaction computed details {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "store": "s0", "nset": 0, "nexist": 0, "modifications": false, "curr logSlots": 14, "curr logSlots size": "1.0 MiB", "next logSlots": 14, "next logSlots size": "1.0 MiB", "candidates": [], "rewrite": [], "duration": "1.278107ms"}
2025-12-26T13:08:04Z INFO hashstore compact once finished {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "store": "s0", "duration": "608.089221ms", "completed": true}
2025-12-26T13:08:04Z INFO hashstore finished compaction {"Process": "storagenode", "satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "store": "s0", "duration": "609.848223ms", "stats": {"NumLogs":0,"LenLogs":"0 B","NumLogsTTL":0,"LenLogsTTL":"0 B","SetPercent":0,"TrashPercent":0,"TTLPercent":0,"Compacting":false,"Compactions":0,"Today":20448,"LastCompact":20448,"LogsRewritten":0,"DataRewritten":"0 B","DataReclaimed":"0 B","DataReclaimable":"0 B","Table":{"NumSet":0,"LenSet":"0 B","AvgSet":0,"NumTrash":0,"LenTrash":"0 B","AvgTrash":0,"NumTTL":0,"LenTTL":"0 B","AvgTTL":0,"NumSlots":16384,"TableSize":"1.0 MiB","Load":0,"Created":20382,"Kind":0},"Compaction":{"Elapsed":0,"Remaining":0,"TotalRecords":0,"ProcessedRecords":0}}}
2025-12-26T13:09:56Z INFO reputation:service node scores updated {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "Total Audits": 4827, "Successful Audits": 3146, "Audit Score": 1, "Online Score": 0.7718846567736145, "Suspension Score": 1, "Audit Score Delta": 0, "Online Score Delta": 0, "Suspension Score Delta": 0}
2025-12-26T13:09:56Z INFO reputation:service node scores updated {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "Total Audits": 3809, "Successful Audits": 2628, "Audit Score": 1, "Online Score": 0.7638004667116509, "Suspension Score": 1, "Audit Score Delta": 0, "Online Score Delta": 0, "Suspension Score Delta": 0}
2025-12-26T13:09:57Z INFO reputation:service node scores updated {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Total Audits": 11379, "Successful Audits": 8588, "Audit Score": 0.9999975532801907, "Online Score": 0.737480551335028, "Suspension Score": 1, "Audit Score Delta": 0, "Online Score Delta": 0, "Suspension Score Delta": 0}
2025-12-26T13:09:57Z INFO reputation:service node scores updated {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Total Audits": 4975, "Successful Audits": 4002, "Audit Score": 1, "Online Score": 0.7450938518139459, "Suspension Score": 1, "Audit Score Delta": 0, "Online Score Delta": 0, "Suspension Score Delta": 0}
2025-12-26T13:17:41Z INFO piecemigrate:chore all enqueued for migration; will sleep before next pooling {"Process": "storagenode", "active": {"121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6": false, "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S": false, "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs": false, "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE": false}, "interval": "10m0s"}
2025-12-26T13:17:41Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 1, "error": "ping satellite: context deadline exceeded", "errorVerbose": "ping satellite: context deadline exceeded\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:227\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:169\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:89\n\tstorj.io/common/sync2.(*Cycle).Run:102\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93"}
2025-12-26T13:17:41Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 1, "error": "ping satellite: context deadline exceeded", "errorVerbose": "ping satellite: context deadline exceeded\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:227\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:169\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:89\n\tstorj.io/common/sync2.(*Cycle).Run:102\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93"}
2025-12-26T13:17:41Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 1, "error": "ping satellite: context deadline exceeded", "errorVerbose": "ping satellite: context deadline exceeded\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:227\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:169\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:89\n\tstorj.io/common/sync2.(*Cycle).Run:102\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93"}
2025-12-26T13:17:41Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 1, "error": "ping satellite: context deadline exceeded", "errorVerbose": "ping satellite: context deadline exceeded\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:227\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:169\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:89\n\tstorj.io/common/sync2.(*Cycle).Run:102\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93"}
2025-12-26T13:20:58Z INFO Downloading versions. {"Process": "storagenode-updater", "Server Address": "https://version.storj.io"}
2025-12-26T13:20:58Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode", "Version": "v1.142.7"}
2025-12-26T13:20:58Z INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode"}
2025-12-26T13:20:58Z INFO Current binary version {"Process": "storagenode-updater", "Service": "storagenode-updater", "Version": "v1.142.7"}
2025-12-26T13:20:58Z INFO Version is up to date {"Process": "storagenode-updater", "Service": "storagenode-updater"}
2025-12-26T13:27:41Z INFO piecemigrate:chore all enqueued for migration; will sleep before next pooling {"Process": "storagenode", "active": {"121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6": false, "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S": false, "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs": false, "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE": false}, "interval": "10m0s"}
2025-12-26T13:27:42Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE", "attempts": 2, "error": "ping satellite: context deadline exceeded", "errorVerbose": "ping satellite: context deadline exceeded\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:227\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:169\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:89\n\tstorj.io/common/sync2.(*Cycle).Run:102\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93"}
2025-12-26T13:27:42Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "attempts": 2, "error": "ping satellite: context deadline exceeded", "errorVerbose": "ping satellite: context deadline exceeded\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:227\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:169\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:89\n\tstorj.io/common/sync2.(*Cycle).Run:102\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93"}
2025-12-26T13:27:42Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 2, "error": "ping satellite: context deadline exceeded", "errorVerbose": "ping satellite: context deadline exceeded\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:227\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:169\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:89\n\tstorj.io/common/sync2.(*Cycle).Run:102\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93"}
2025-12-26T13:27:42Z ERROR contact:service ping satellite failed {"Process": "storagenode", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "attempts": 2, "error": "ping satellite: context deadline exceeded", "errorVerbose": "ping satellite: context deadline exceeded\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:227\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:169\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:89\n\tstorj.io/common/sync2.(*Cycle).Run:102\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:93"}