Unexpected fault address

Node Operators troubleshooting
1

Dear all,

today my computer performed some updates: operative system is Ubuntu 18.04.3 LTS. After having restarted the computer I restarted the storagenode, but when I try to start the dashboard, the following error shows up:

Blockquote
fmas@delta:~$ docker exec -it storagenode /app/dashboard.sh
2019-09-27T19:45:51.981Z INFO Configuration loaded from: /app/config/config.yaml
2019-09-27T19:45:52.006Z INFO Node ID: 1VyLHATWP4fNTrCdFX3GKqkHRAyz7Y77RSXQaT3ysrTvLx8fq4
2019-09-27T19:45:52.036Z FATAL Unrecoverable error {“error”: “transport error: connection error: desc = "transport: error while dialing: dial tcp 127.0.0.1:7778: connect: connection refused"”, “errorVerbose”: “transport error: connection error: desc = "transport: error while dialing: dial tcp 127.0.0.1:7778: connect: connection refused"\n\tstorj.io/storj/pkg/transport.DialAddressInsecure:31\n\tmain.dialDashboardClient:37\n\tmain.cmdDashboard:66\n\tstorj.io/storj/pkg/process.cleanup.func1.2:264\n\tstorj.io/storj/pkg/process.cleanup.func1:282\n\tgithub.com/spf13/cobra.(*Command).execute:762\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:852\n\tgithub.com/spf13/cobra.(*Command).Execute:800\n\tstorj.io/storj/pkg/process.Exec:73\n\tmain.main:296\n\truntime.main:203”}

Is there someone available to help?

Thank you very much for your help.

best,

andrea

2

You can see if node is running via:

docker ps --no-trunc

1 Like
3

Hello beast,

thanks for your suggestion. I have tried what you mentioned, see the result:

Blockquotefmas@delta:~$ docker ps --no-trunc
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cb5a86018f7d66be717bec3fb376fe17e3f1335d80197b8bb2ade44424901978 storjlabs/storagenode:alpha “/entrypoint” 23 hours ago Restarting (2) 27 seconds ago storagenode
49d59ff00ec67380789c3ac78c5ad3704327bc0bc8881aeb9e76d62c0d65a758 storjlabs/watchtower:latest “/watchtower storagenode watchtower --stop-timeout 300s” 2 weeks ago Up 2 hours watchtower
fmas@delta:~$

4

Please, show the output of the command:

docker logs --tail 20 storagenode
5

Blockquote
fmas@delta:~$ docker logs --tail 20 storagenode
internal/poll.runtime_pollWait(0x7f63b95e1ea8, 0x72, 0x0)
/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc000119398, 0x72, 0x0, 0x0, 0xef9734)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(…)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc000119380, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:384 +0x1f8
net.(*netFD).accept(0xc000119380, 0xc000060cc0, 0xc000056700, 0x7f63bb9416d0)
/usr/local/go/src/net/fd_unix.go:238 +0x42
net.(*TCPListener).accept(0xc00029d6a0, 0xc000060cf0, 0x4110a8, 0x30)
/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc00029d6a0, 0xe60d20, 0xc00018b9e0, 0xdb22e0, 0x187c0a0)
/usr/local/go/src/net/tcpsock.go:261 +0x47
net/http.(*Server).Serve(0xc000348000, 0x1044fa0, 0xc00029d6a0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:2896 +0x286
storj.io/storj/pkg/process.initDebug.func2(0xc0002f8780, 0x1044fa0, 0xc00029d6a0, 0xc00019b0c0)
/go/src/storj.io/storj/pkg/process/debug.go:52 +0x15d
created by storj.io/storj/pkg/process.initDebug
/go/src/storj.io/storj/pkg/process/debug.go:50 +0x38f
fmas@delta:~$

6

From the ps command output:

STATUS                            PORTS         NAMES
Restarting (2) 27 seconds ago                   storagenode

It seems that perhaps your node didn’t shutdown or restart properly.

I would suggest shutting down, then checking to make sure it’s not running… then restart the node.

If the node still indicates that it is continuously restarting, you probably will need to remove the node and recreate it.

Of course… check the logs as well as @Alexey indicates.

7

Ok, add more lines

docker logs --tail 40 storagenode
8

fmas@delta:~$ docker logs --tail 40 storagenode
storj.io/storj/pkg/process.Ctx.func1(0xc0002b75c0, 0xc000314f60)
/go/src/storj.io/storj/pkg/process/exec_conf.go:89 +0x41
created by storj.io/storj/pkg/process.Ctx
/go/src/storj.io/storj/pkg/process/exec_conf.go:88 +0x1b6

goroutine 10 [IO wait]:
internal/poll.runtime_pollWait(0x7f0e59e4dea8, 0x72, 0x0)
/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc000117498, 0x72, 0x0, 0x0, 0xef9734)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(…)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc000117480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:384 +0x1f8
net.(*netFD).accept(0xc000117480, 0xc000060cc0, 0xc000184000, 0x7f0e5c18a6d0)
/usr/local/go/src/net/fd_unix.go:238 +0x42
net.(*TCPListener).accept(0xc00028f7e0, 0xc000060cf0, 0x4110a8, 0x30)
/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc00028f7e0, 0xe60d20, 0xc00017e1e0, 0xdb22e0, 0x187c0a0)
/usr/local/go/src/net/tcpsock.go:261 +0x47
net/http.(*Server).Serve(0xc00032e000, 0x1044fa0, 0xc00028f7e0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:2896 +0x286
storj.io/storj/pkg/process.initDebug.func2(0xc0002b7380, 0x1044fa0, 0xc00028f7e0, 0xc000191400)
/go/src/storj.io/storj/pkg/process/debug.go:52 +0x15d
created by storj.io/storj/pkg/process.initDebug
/go/src/storj.io/storj/pkg/process/debug.go:50 +0x38f

goroutine 21 [syscall]:
os/signal.signal_recv(0x0)
/usr/local/go/src/runtime/sigqueue.go:147 +0x9c
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
/usr/local/go/src/os/signal/signal_unix.go:29 +0x41

goroutine 14 [select]:
database/sql.(*DB).connectionResetter(0xc0001b03c0, 0x104a820, 0xc000191800)
/usr/local/go/src/database/sql/sql.go:1065 +0xfb
created by database/sql.OpenDB
/usr/local/go/src/database/sql/sql.go:723 +0x193
fmas@delta:~$

9

Ok. The stacktrace is too long.

docker logs --tail 100 storagenode
10

fmas@delta:~$ docker logs --tail 100 storagenode

goroutine 14 [select]:
database/sql.(*DB).connectionResetter(0xc0001c43c0, 0x104a820, 0xc00007bf00)
/usr/local/go/src/database/sql/sql.go:1065 +0xfb
created by database/sql.OpenDB
/usr/local/go/src/database/sql/sql.go:723 +0x193
2019-09-27T20:37:43.708Z INFO Configuration loaded from: /app/config/config.yaml
2019-09-27T20:37:43.749Z INFO Operator email:
2019-09-27T20:37:43.749Z INFO operator wallet:
unexpected fault address 0x7f0e59d7f000
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x7f0e59d7f000 pc=0xb174f1]

goroutine 1 [running]:
runtime.throw(0xef76d1, 0x5)
/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc00016c998 sp=0xc00016c968 pc=0x4335e2
runtime.sigpanic()
/usr/local/go/src/runtime/signal_unix.go:391 +0x455 fp=0xc00016c9c8 sp=0xc00016c998 pc=0x448e35
github.com/boltdb/bolt.(*DB).page(…)
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:796
github.com/boltdb/bolt.(*DB).mmap(0xc00000c960, 0x0, 0x0, 0x0)
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:282 +0x251 fp=0xc00016ca88 sp=0xc00016c9c8 pc=0xb174f1
github.com/boltdb/bolt.Open(0xc000038727, 0x15, 0x180, 0xc00016cb98, 0xc0001b03c0, 0xc000190c80, 0xc00016cbb0)
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:230 +0x2ae fp=0xc00016cb50 sp=0xc00016ca88 pc=0xb16ebe
storj.io/storj/storage/boltdb.New(0xc000038727, 0x15, 0xefd33b, 0xb, 0x2, 0x2, 0xc0003313a0)
/go/src/storj.io/storj/storage/boltdb/client.go:41 +0x7f fp=0xc00016cc30 sp=0xc00016cb50 pc=0xb2732f
storj.io/storj/pkg/revocation.newDBBolt(0xc000038727, 0x15, 0xc000038720, 0x4, 0xc000038727)
/go/src/storj.io/storj/pkg/revocation/common.go:52 +0x4e fp=0xc00016cc80 sp=0xc00016cc30 pc=0xb73dbe
storj.io/storj/pkg/revocation.NewDB(0xc000038720, 0x1c, 0xe, 0xc0002e18c0, 0x1c)
/go/src/storj.io/storj/pkg/revocation/common.go:34 +0x1bf fp=0xc00016cce0 sp=0xc00016cc80 pc=0xb73caf
storj.io/storj/pkg/revocation.NewDBFromCfg(...)
/go/src/storj.io/storj/pkg/revocation/common.go:21
main.cmdRun(0x1888ae0, 0xc0001d24e0, 0x0, 0xd, 0x0, 0x0)
/go/src/storj.io/storj/cmd/storagenode/main.go:143 +0x521 fp=0xc00016d280 sp=0xc00016cce0 pc=0xc29711
storj.io/storj/pkg/process.cleanup.func1.2(0x104aae0, 0xc0001cbcc0)
/go/src/storj.io/storj/pkg/process/exec_conf.go:264 +0x13b fp=0xc00016d318 sp=0xc00016d280 pc=0xaec53b
storj.io/storj/pkg/process.cleanup.func1(0x1888ae0, 0xc0001d24e0, 0x0, 0xd, 0x0, 0x0)
/go/src/storj.io/storj/pkg/process/exec_conf.go:282 +0x17df fp=0xc00016dd50 sp=0xc00016d318 pc=0xaeddcf
github.com/spf13/cobra.(*Command).execute(0x1888ae0, 0xc0001d2340, 0xd, 0xd, 0x1888ae0, 0xc0001d2340)
/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:762 +0x460 fp=0xc00016de28 sp=0xc00016dd50 pc=0x62cbb0
github.com/spf13/cobra.(*Command).ExecuteC(0x1888880, 0xc00007c180, 0x1, 0x1)
/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ea fp=0xc00016def8 sp=0xc00016de28 pc=0x62d5ea
github.com/spf13/cobra.(*Command).Execute(…)
/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
storj.io/storj/pkg/process.Exec(0x1888880)
/go/src/storj.io/storj/pkg/process/exec_conf.go:73 +0x17f fp=0xc00016df48 sp=0xc00016def8 pc=0xae8f0f
main.main()
/go/src/storj.io/storj/cmd/storagenode/main.go:296 +0x2d fp=0xc00016df60 sp=0xc00016df48 pc=0xc2af8d
runtime.main()
/usr/local/go/src/runtime/proc.go:203 +0x21e fp=0xc00016dfe0 sp=0xc00016df60 pc=0x434f7e
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc00016dfe8 sp=0xc00016dfe0 pc=0x463541

goroutine 13 [select]:
database/sql.(*DB).connectionOpener(0xc0001b03c0, 0x104a820, 0xc000191800)
/usr/local/go/src/database/sql/sql.go:1052 +0xe8
created by database/sql.OpenDB
/usr/local/go/src/database/sql/sql.go:722 +0x15d

goroutine 12 [chan receive]:
storj.io/storj/pkg/process.Ctx.func1(0xc0002b75c0, 0xc000314f60)
/go/src/storj.io/storj/pkg/process/exec_conf.go:89 +0x41
created by storj.io/storj/pkg/process.Ctx
/go/src/storj.io/storj/pkg/process/exec_conf.go:88 +0x1b6

goroutine 10 [IO wait]:
internal/poll.runtime_pollWait(0x7f0e59e4dea8, 0x72, 0x0)
/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc000117498, 0x72, 0x0, 0x0, 0xef9734)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(…)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc000117480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:384 +0x1f8
net.(*netFD).accept(0xc000117480, 0xc000060cc0, 0xc000184000, 0x7f0e5c18a6d0)
/usr/local/go/src/net/fd_unix.go:238 +0x42
net.(*TCPListener).accept(0xc00028f7e0, 0xc000060cf0, 0x4110a8, 0x30)
/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc00028f7e0, 0xe60d20, 0xc00017e1e0, 0xdb22e0, 0x187c0a0)
/usr/local/go/src/net/tcpsock.go:261 +0x47
net/http.(*Server).Serve(0xc00032e000, 0x1044fa0, 0xc00028f7e0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:2896 +0x286
storj.io/storj/pkg/process.initDebug.func2(0xc0002b7380, 0x1044fa0, 0xc00028f7e0, 0xc000191400)
/go/src/storj.io/storj/pkg/process/debug.go:52 +0x15d
created by storj.io/storj/pkg/process.initDebug
/go/src/storj.io/storj/pkg/process/debug.go:50 +0x38f

goroutine 21 [syscall]:
os/signal.signal_recv(0x0)
/usr/local/go/src/runtime/sigqueue.go:147 +0x9c
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
/usr/local/go/src/os/signal/signal_unix.go:29 +0x41

goroutine 14 [select]:
database/sql.(*DB).connectionResetter(0xc0001b03c0, 0x104a820, 0xc000191800)
/usr/local/go/src/database/sql/sql.go:1065 +0xfb
created by database/sql.OpenDB
/usr/local/go/src/database/sql/sql.go:723 +0x193
fmas@delta:~$

11

You should remove your wallet address from your post…

12

Never understood if this is an issue or not.

13

How is your disk connected?

14

It is an internal drive, SATA.

15

Try to stop the container and rename the kademlia, after that start the container back.

16

I have already used the command:

docker stop -t 300 storagenode

Therefore I believe the node is stopped.

How to rename kademlia?

Thank you.

17

Find a file with name kademlia and rename it to kademlia.bak

19

I have done as you suggested. It did not work.

fmas@delta:~ docker exec -it storagenode /app/dashboard.sh 2019-09-27T21:00:32.219Z INFO Configuration loaded from: /app/config/config.yaml 2019-09-27T21:00:32.250Z INFO Node ID: 1VyLHATWP4fNTrCdFX3GKqkHRAyz7Y77RSXQaT3ysrTvLx8fq4 2019-09-27T21:00:32.252Z FATAL Unrecoverable error {"error": "transport error: connection error: desc = \"transport: error while dialing: dial tcp 127.0.0.1:7778: connect: connection refused\"", "errorVerbose": "transport error: connection error: desc = \"transport: error while dialing: dial tcp 127.0.0.1:7778: connect: connection refused\"\n\tstorj.io/storj/pkg/transport.DialAddressInsecure:31\n\tmain.dialDashboardClient:37\n\tmain.cmdDashboard:66\n\tstorj.io/storj/pkg/process.cleanup.func1.2:264\n\tstorj.io/storj/pkg/process.cleanup.func1:282\n\tgithub.com/spf13/cobra.(*Command).execute:762\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:852\n\tgithub.com/spf13/cobra.(*Command).Execute:800\n\tstorj.io/storj/pkg/process.Exec:73\n\tmain.main:296\n\truntime.main:203"} fmas@delta:~

20

Please, show logs

docker logs --tail 100 storagenode
21

fmas@delta:~$ docker logs --tail 100 storagenode

goroutine 25 [select]:
database/sql.(*DB).connectionResetter(0xc0001ac3c0, 0x104a820, 0xc000125c40)
/usr/local/go/src/database/sql/sql.go:1065 +0xfb
created by database/sql.OpenDB
/usr/local/go/src/database/sql/sql.go:723 +0x193
2019-09-27T21:15:26.360Z INFO Configuration loaded from: /app/config/config.yaml
2019-09-27T21:15:26.385Z INFO Operator email:
2019-09-27T21:15:26.385Z INFO operator wallet:
unexpected fault address 0x7f797d0d0000
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x7f797d0d0000 pc=0xb174f1]

goroutine 1 [running]:
runtime.throw(0xef76d1, 0x5)
/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc00014c998 sp=0xc00014c968 pc=0x4335e2
runtime.sigpanic()
/usr/local/go/src/runtime/signal_unix.go:391 +0x455 fp=0xc00014c9c8 sp=0xc00014c998 pc=0x448e35
github.com/boltdb/bolt.(*DB).page(…)
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:796
github.com/boltdb/bolt.(*DB).mmap(0xc0003661e0, 0x0, 0x0, 0x0)
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:282 +0x251 fp=0xc00014ca88 sp=0xc00014c9c8 pc=0xb174f1
github.com/boltdb/bolt.Open(0xc00017a2e7, 0x15, 0x180, 0xc00014cb98, 0xc0001aa3c0, 0xc00015ef80, 0xc00014cbb0)
/go/pkg/mod/github.com/boltdb/bolt@v1.3.1/db.go:230 +0x2ae fp=0xc00014cb50 sp=0xc00014ca88 pc=0xb16ebe
storj.io/storj/storage/boltdb.New(0xc00017a2e7, 0x15, 0xefd33b, 0xb, 0x2, 0x2, 0xc00036c080)
/go/src/storj.io/storj/storage/boltdb/client.go:41 +0x7f fp=0xc00014cc30 sp=0xc00014cb50 pc=0xb2732f
storj.io/storj/pkg/revocation.newDBBolt(0xc00017a2e7, 0x15, 0xc00017a2e0, 0x4, 0xc00017a2e7)
/go/src/storj.io/storj/pkg/revocation/common.go:52 +0x4e fp=0xc00014cc80 sp=0xc00014cc30 pc=0xb73dbe
storj.io/storj/pkg/revocation.NewDB(0xc00017a2e0, 0x1c, 0xe, 0xc0002ef480, 0x1c)
/go/src/storj.io/storj/pkg/revocation/common.go:34 +0x1bf fp=0xc00014cce0 sp=0xc00014cc80 pc=0xb73caf
storj.io/storj/pkg/revocation.NewDBFromCfg(...)
/go/src/storj.io/storj/pkg/revocation/common.go:21
main.cmdRun(0x1888ae0, 0xc00019b450, 0x0, 0xd, 0x0, 0x0)
/go/src/storj.io/storj/cmd/storagenode/main.go:143 +0x521 fp=0xc00014d280 sp=0xc00014cce0 pc=0xc29711
storj.io/storj/pkg/process.cleanup.func1.2(0x104aae0, 0xc0001c3b80)
/go/src/storj.io/storj/pkg/process/exec_conf.go:264 +0x13b fp=0xc00014d318 sp=0xc00014d280 pc=0xaec53b
storj.io/storj/pkg/process.cleanup.func1(0x1888ae0, 0xc00019b450, 0x0, 0xd, 0x0, 0x0)
/go/src/storj.io/storj/pkg/process/exec_conf.go:282 +0x17df fp=0xc00014dd50 sp=0xc00014d318 pc=0xaeddcf
github.com/spf13/cobra.(*Command).execute(0x1888ae0, 0xc00019b2b0, 0xd, 0xd, 0x1888ae0, 0xc00019b2b0)
/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:762 +0x460 fp=0xc00014de28 sp=0xc00014dd50 pc=0x62cbb0
github.com/spf13/cobra.(*Command).ExecuteC(0x1888880, 0xc00007c180, 0x1, 0x1)
/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ea fp=0xc00014def8 sp=0xc00014de28 pc=0x62d5ea
github.com/spf13/cobra.(*Command).Execute(…)
/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
storj.io/storj/pkg/process.Exec(0x1888880)
/go/src/storj.io/storj/pkg/process/exec_conf.go:73 +0x17f fp=0xc00014df48 sp=0xc00014def8 pc=0xae8f0f
main.main()
/go/src/storj.io/storj/cmd/storagenode/main.go:296 +0x2d fp=0xc00014df60 sp=0xc00014df48 pc=0xc2af8d
runtime.main()
/usr/local/go/src/runtime/proc.go:203 +0x21e fp=0xc00014dfe0 sp=0xc00014df60 pc=0x434f7e
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc00014dfe8 sp=0xc00014dfe0 pc=0x463541

goroutine 22 [chan receive]:
storj.io/storj/pkg/process.Ctx.func1(0xc0002f4d80, 0xc00032aa20)
/go/src/storj.io/storj/pkg/process/exec_conf.go:89 +0x41
created by storj.io/storj/pkg/process.Ctx
/go/src/storj.io/storj/pkg/process/exec_conf.go:88 +0x1b6

goroutine 23 [select]:
database/sql.(*DB).connectionOpener(0xc0001aa3c0, 0x104a820, 0xc00015fb00)
/usr/local/go/src/database/sql/sql.go:1052 +0xe8
created by database/sql.OpenDB
/usr/local/go/src/database/sql/sql.go:722 +0x15d

goroutine 20 [IO wait]:
internal/poll.runtime_pollWait(0x7f797d2caf30, 0x72, 0x0)
/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc000161418, 0x72, 0x0, 0x0, 0xef9734)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(…)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc000161400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:384 +0x1f8
net.(*netFD).accept(0xc000161400, 0xc000065cc0, 0xc0000fae00, 0x7f797f5c3008)
/usr/local/go/src/net/fd_unix.go:238 +0x42
net.(*TCPListener).accept(0xc0003184c0, 0xc000065cf0, 0x4110a8, 0x30)
/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc0003184c0, 0xe60d20, 0xc000139cb0, 0xdb22e0, 0x187c0a0)
/usr/local/go/src/net/tcpsock.go:261 +0x47
net/http.(*Server).Serve(0xc0000f22a0, 0x1044fa0, 0xc0003184c0, 0x0, 0x0)
/usr/local/go/src/net/http/server.go:2896 +0x286
storj.io/storj/pkg/process.initDebug.func2(0xc0002f4b40, 0x1044fa0, 0xc0003184c0, 0xc00015f700)
/go/src/storj.io/storj/pkg/process/debug.go:52 +0x15d
created by storj.io/storj/pkg/process.initDebug
/go/src/storj.io/storj/pkg/process/debug.go:50 +0x38f

goroutine 10 [syscall]:
os/signal.signal_recv(0x0)
/usr/local/go/src/runtime/sigqueue.go:147 +0x9c
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
/usr/local/go/src/os/signal/signal_unix.go:29 +0x41

goroutine 24 [select]:
database/sql.(*DB).connectionResetter(0xc0001aa3c0, 0x104a820, 0xc00015fb00)
/usr/local/go/src/database/sql/sql.go:1065 +0xfb
created by database/sql.OpenDB
/usr/local/go/src/database/sql/sql.go:723 +0x193
fmas@delta:~$