PLEASE HELP - "Error 1067: The process terminated unexpectedly"

[NS1]

Hi everyone,
Around 8-12 month ago, a fan from the front panel of my cpu casing was making noise. I was afraid that the noise will create vibration and might cause issues for the HDD so I disconnected the fan but I left it in the casing.

Today I turn off my pc (node) and removed the fan (I was going to use/attached the fan to an old gpu in another pc). But when I turn back on my pc (node), it was showing offline status.

I notice the Storj service was not working in the background, so I tried to turn it on manually but I keep getting the error message: “Error 1067: The process terminated unexpectedly”

I search google to find a solution for the “Error 1067” message, an article suggested to use the command “sfc /scan now”, I tried it and restarted my pc but this time the dashboard was not loading, only an error message was loading “This site can’t be reached, localhost refused to connect.”

My node is an old gaming pc with core i3, 16 gb ram and 3tb HDD. My node has been running since January 2020 without any major issues! I was planning to upgrade my node to a 20gb hdd but I was not able to do it because of covid.

I was earning around 9-10 usd per month, but the last 2-3 month, my earning was reduced to 6-7 usd and I have around 32 usd payout pending!

Right now I very depressed because of my financial issues, I’m thinking about giving up and just turn off the node without doing any graceful exit. I got into STORJ because I saw it as a good investment to earn some passive income.

Please help me fix the issue, I’m using the windown gui version, I attached all the screenshot below, Thank you.

[Stob]

Hi @NS1,
Please check the storagenode log file for the latest entries (How do I check my logs? - Node Operator) and then post them between three backticks

```
log text
```

[NS1]

Hi @Stob , please check the log files below:

PS C:\WINDOWS\system32> Get-Content "$env:ProgramFiles/Storj/Storage Node/storagenode.log" -Tail 20 -Wait
>>
2021-12-30T08:13:12.473+0400    INFO    Configuration loaded    {"Location": "C:\\Program Files\\Storj\\Storage Node\\config.yaml"}
2021-12-30T08:13:12.473+0400    INFO    Operator email  {"Address": "waitingareply@live.com"}
2021-12-30T08:13:12.473+0400    INFO    Operator wallet {"Address": "0x75508Cae2247d308589B67aeD4911D895231f3B5"}
2021-12-30T08:13:12.548+0400    FATAL   Unrecoverable error     {"error": "trust: malformed cache: invalid character '\\x00' looking for beginning of value", "errorVerbose": "trust: malformed cache: invalid character '\\x00' looking for beginning of value\n\tstorj.io/storj/storagenode/trust.LoadCacheData:96\n\tstorj.io/storj/storagenode/trust.LoadCache:36\n\tstorj.io/storj/storagenode/trust.NewPool:89\n\tstorj.io/storj/storagenode.New:378\n\tmain.cmdRun:177\n\tstorj.io/private/process.cleanup.func1.4:363\n\tstorj.io/private/process.cleanup.func1:381\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:61\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2021-12-30T08:13:31.149+0400    INFO    Configuration loaded    {"Location": "C:\\Program Files\\Storj\\Storage Node\\config.yaml"}
2021-12-30T08:13:31.149+0400    INFO    Operator email  {"Address": "waitingareply@live.com"}
2021-12-30T08:13:31.149+0400    INFO    Operator wallet {"Address": "0x75508Cae2247d308589B67aeD4911D895231f3B5"}
2021-12-30T08:13:31.395+0400    FATAL   Unrecoverable error     {"error": "trust: malformed cache: invalid character '\\x00' looking for beginning of value", "errorVerbose": "trust: malformed cache: invalid character '\\x00' looking for beginning of value\n\tstorj.io/storj/storagenode/trust.LoadCacheData:96\n\tstorj.io/storj/storagenode/trust.LoadCache:36\n\tstorj.io/storj/storagenode/trust.NewPool:89\n\tstorj.io/storj/storagenode.New:378\n\tmain.cmdRun:177\n\tstorj.io/private/process.cleanup.func1.4:363\n\tstorj.io/private/process.cleanup.func1:381\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:61\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2021-12-30T08:43:44.020+0400    INFO    Configuration loaded    {"Location": "C:\\Program Files\\Storj\\Storage Node\\config.yaml"}
2021-12-30T08:43:44.272+0400    INFO    Operator email  {"Address": "waitingareply@live.com"}
2021-12-30T08:43:44.272+0400    INFO    Operator wallet {"Address": "0x75508Cae2247d308589B67aeD4911D895231f3B5"}
2021-12-30T08:43:49.664+0400    FATAL   Unrecoverable error     {"error": "trust: malformed cache: invalid character '\\x00' looking for beginning of value", "errorVerbose": "trust: malformed cache: invalid character '\\x00' looking for beginning of value\n\tstorj.io/storj/storagenode/trust.LoadCacheData:96\n\tstorj.io/storj/storagenode/trust.LoadCache:36\n\tstorj.io/storj/storagenode/trust.NewPool:89\n\tstorj.io/storj/storagenode.New:378\n\tmain.cmdRun:177\n\tstorj.io/private/process.cleanup.func1.4:363\n\tstorj.io/private/process.cleanup.func1:381\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:61\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2021-12-30T08:50:34.840+0400    INFO    Configuration loaded    {"Location": "C:\\Program Files\\Storj\\Storage Node\\config.yaml"}
2021-12-30T08:50:34.842+0400    INFO    Operator email  {"Address": "waitingareply@live.com"}
2021-12-30T08:50:34.843+0400    INFO    Operator wallet {"Address": "0x75508Cae2247d308589B67aeD4911D895231f3B5"}
2021-12-30T08:50:35.093+0400    FATAL   Unrecoverable error     {"error": "trust: malformed cache: invalid character '\\x00' looking for beginning of value", "errorVerbose": "trust: malformed cache: invalid character '\\x00' looking for beginning of value\n\tstorj.io/storj/storagenode/trust.LoadCacheData:96\n\tstorj.io/storj/storagenode/trust.LoadCache:36\n\tstorj.io/storj/storagenode/trust.NewPool:89\n\tstorj.io/storj/storagenode.New:378\n\tmain.cmdRun:177\n\tstorj.io/private/process.cleanup.func1.4:363\n\tstorj.io/private/process.cleanup.func1:381\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:61\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2021-12-30T08:52:30.452+0400    INFO    Configuration loaded    {"Location": "C:\\Program Files\\Storj\\Storage Node\\config.yaml"}
2021-12-30T08:52:30.454+0400    INFO    Operator email  {"Address": "waitingareply@live.com"}
2021-12-30T08:52:30.454+0400    INFO    Operator wallet {"Address": "0x75508Cae2247d308589B67aeD4911D895231f3B5"}
2021-12-30T08:52:30.701+0400    FATAL   Unrecoverable error     {"error": "trust: malformed cache: invalid character '\\x00' looking for beginning of value", "errorVerbose": "trust: malformed cache: invalid character '\\x00' looking for beginning of value\n\tstorj.io/storj/storagenode/trust.LoadCacheData:96\n\tstorj.io/storj/storagenode/trust.LoadCache:36\n\tstorj.io/storj/storagenode/trust.NewPool:89\n\tstorj.io/storj/storagenode.New:378\n\tmain.cmdRun:177\n\tstorj.io/private/process.cleanup.func1.4:363\n\tstorj.io/private/process.cleanup.func1:381\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.Exec:65\n\tmain.(*service).Execute.func1:61\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}

[Stob]

@NS1

trust: malformed cache: invalid character

The error is the same as here - FATAL Unrecoverable error {"error": "trust: malformed cache: invalid character '\\x00' looking for beginning of value" - #14 by peem

@peem

Remove the trust-cache.json from the data location

If removing the file doesn’t bring the node back online then you should run the disk check.

[NS1]

Its fixed now, I just removed the “trust-cache.json” file and restarted my node!

I was very frustrated since this morning thinking about: “how can removing a fan from the casing was able to cause this issue?” I’m very happy, my node is back online.

Thank you so much @Stob, Merry Christmas and Happy New Year

[aseegy]

Hello everyone.

Tonight my node stopped working.

I am running Windows 10, when in services I try to restart the node I get the following error:

Windows could not start the Storj V3 Storage Node service on Local Computer.

Error 1067: The process terminated unexpectedly.

This the print screen:

The difference from user @NS1 is the fact that in the log I do not any entry with FATAL unrecoverable error.

Below a few lines of the log:

2024-03-30T02:23:42+01:00	INFO	piecestore	download started	{"Piece ID": "6ATSPBUCYFL34DPWT5YX75HASQNFOXOXIB77NS4DA6SNAAXVZOMQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 391936, "Size": 62720, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:23:42+01:00	INFO	piecestore	downloaded	{"Piece ID": "6ATSPBUCYFL34DPWT5YX75HASQNFOXOXIB77NS4DA6SNAAXVZOMQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 391936, "Size": 62720, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:06+01:00	INFO	piecestore	download started	{"Piece ID": "TNMJ6X5LC7C26MCVE27E7EXHDFRTGOXORXFM62672E3LKVANCKKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 233472, "Size": 45568, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:06+01:00	INFO	piecestore	downloaded	{"Piece ID": "TNMJ6X5LC7C26MCVE27E7EXHDFRTGOXORXFM62672E3LKVANCKKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 233472, "Size": 45568, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:07+01:00	INFO	piecestore	download started	{"Piece ID": "TNMJ6X5LC7C26MCVE27E7EXHDFRTGOXORXFM62672E3LKVANCKKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 329984, "Size": 60160, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:07+01:00	INFO	piecestore	downloaded	{"Piece ID": "TNMJ6X5LC7C26MCVE27E7EXHDFRTGOXORXFM62672E3LKVANCKKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 329984, "Size": 60160, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:07+01:00	INFO	piecestore	download started	{"Piece ID": "TNMJ6X5LC7C26MCVE27E7EXHDFRTGOXORXFM62672E3LKVANCKKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 411904, "Size": 24064, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:07+01:00	INFO	piecestore	downloaded	{"Piece ID": "TNMJ6X5LC7C26MCVE27E7EXHDFRTGOXORXFM62672E3LKVANCKKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 411904, "Size": 24064, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:07+01:00	INFO	piecestore	download started	{"Piece ID": "TNMJ6X5LC7C26MCVE27E7EXHDFRTGOXORXFM62672E3LKVANCKKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 435712, "Size": 29440, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:07+01:00	INFO	piecestore	downloaded	{"Piece ID": "TNMJ6X5LC7C26MCVE27E7EXHDFRTGOXORXFM62672E3LKVANCKKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET", "Offset": 435712, "Size": 29440, "Remote Address": "79.127.226.101:35474"}
2024-03-30T02:24:21+01:00	INFO	Stop/Shutdown request received.

I am now running HDD checks with DLGDIAG but I do not know it has any repairing capability.

I thought I could share Windows Event Register as well. A printscreen around the time the service stopped:

I am adding the log of the storagenode updater as well:

2024-03-30T02:24:21+01:00	INFO	Download started.	{"From": "https://github.com/storj/storj/releases/download/v1.97.3/storagenode-updater_windows_amd64.zip", "To": "C:\\WINDOWS\\TEMP\\storagenode-updater_windows_amd64.773291649.zip"}
2024-03-30T02:24:22+01:00	INFO	Download finished.	{"From": "https://github.com/storj/storj/releases/download/v1.97.3/storagenode-updater_windows_amd64.zip", "To": "C:\\WINDOWS\\TEMP\\storagenode-updater_windows_amd64.773291649.zip"}
2024-03-30T02:24:23+01:00	INFO	Restarting service.	{"Service": "storagenode-updater"}
2024-03-30T02:24:23+01:00	INFO	Stop/Shutdown request received.
2024-03-30T02:28:23+01:00	INFO	Configuration loaded	{"Location": "C:\\Program Files\\Storj\\Storage Node\\config.yaml"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "server.private-address"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "server.address"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "storage.allocated-bandwidth"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "operator.email"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "storage.allocated-disk-space"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "healthcheck.enabled"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "operator.wallet-features.0"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "operator.wallet"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "contact.external-address"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "healthcheck.details"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file key	{"Key": "storage.path"}
2024-03-30T02:28:23+01:00	INFO	Invalid configuration file value for key	{"Key": "log.level"}
2024-03-30T02:28:23+01:00	INFO	Anonymized tracing enabled

Looks like something has happened here.
Any help is appreciated, thanks.

[aseegy]

Apparently also the extended test passed ok. @Alexey have you got any ideas?

Thanks!

[Alexey]

Yes. You likely have a duplicated keys in your config.yaml like a wallet feature option.
You may check this (PowerShell):

& "$env:ProgramFiles\Storj\Storage Node\storagenode.exe" run --config-dir "$env:ProgramFiles\Storj\Storage Node\\" --log.output=stderr

[aseegy]

Nothing happens apparently.

[Alexey]

I missed a quote in the command. Fixed. You need to close this window or press Ctrl-C. Please try again.

[aseegy]

It worked now and it highlights an issue with line 334.

operator.wallet-features: ["zksync"]

This line is there since one year or even more, since I adopted zksync. Should I remove it?

[Alexey]

You just need to make it unique.
So, if you have somewhere the same key, you just need to modify this one and remove the added one.

[aseegy]

It worked.

Thanks a lot!