QNAP TS-332X - Node is failing to start

arturgontijo · July 12, 2021, 2:07pm

Hey everyone, it’s been a while! My first node was too slow and error prone that I gave up to run it last year.

But now I’ve got a QNAP TS-332X and it’s great!

First of all:

Firmware 4.5.4.1723
Container Station (2.3.5.1706 and also tried with 2.1.3.1360)

My first attempt was to use the QNAP-App but both “latest” versions (1.1.0 and 1.1.2) have the same issue for me:

I inspected its logs and got this:

Mon Jul 12 10:04:59 BRT 2021 ----------- Installation REMOVED ------ 
Mon Jul 12 10:05:27 BRT 2021 ======= NEW INSTALL (ver 1.1.2) ======
Mon Jul 12 10:05:27 BRT 2021 RUNNING pkg_init (ver 1.1.2) 
Mon Jul 12 10:05:27 BRT 2021  pre install completed (ver 1.1.2) 
Mon Jul 12 10:05:28 BRT 2021 INSTALL run completed (ver 1.1.2) 
Mon Jul 12 10:05:28 BRT 2021  RUNNING post install @ / 
check/put entry in cron file: 5 0 * * * /STORJ/scripts/storagenodeupdate.sh /share/CACHEDEV1_DATA/Public/storagenode.conf/config.json  
Mon Jul 12 10:05:30 BRT 2021  post install COMPLETED (ver 1.1.2) 
Mon Jul 12 10:05:30 BRT 2021  STORJ : Request to start (start) 
Mon Jul 12 10:05:31 BRT 2021  STORJ : Request to start (start) completed

So no idea why it’s not starting (nor even creating a Docker container).

Then my second approach was to use the CLI, what I’ve done so far:

Connected to my QNAP using ssh
Tried to run the following command:

docker run --rm \
	-e SETUP=true \
	--mount type=bind,source=/root/credentials,destination=/app/identity \
	--mount type=bind,source=/root/storj,destination=/app/config \
	--name storagenode storjlabs/storagenode:latest
>>>
Unable to find image 'storjlabs/storagenode:latest' locally
latest: Pulling from storjlabs/storagenode
58ab47519297: Pull complete 
e69b284058c2: Pull complete 
4aecfb8c0f6e: Pull complete 
343bdb3df068: Pull complete 
a3781acb482d: Pull complete 
297ca68e0454: Pull complete 
Digest: sha256:3e4d1809f328918b7f99a0a0889c6d4701ddbb0806ec73add73edbf5e7e9686f
Status: Downloaded newer image for storjlabs/storagenode:latest
./storagenode: line 1: syntax error: unexpected word (expecting ")")

I noticed through some other topics that the ARMv8 builds are not being tested anymore (or something like that). So I also tried with some hard coded tags like storjlabs/storagenode:arm and :<<HASH>>-go1.15.7-arm64v8 but still got the same ./storagenode: line 1: syntax error: unexpected word (expecting ")") error =/

For me it’s obviously that the ./storagenode is not being recognized as an executable rather than a shell script, but I have no idea how to force that. Maybe tweaking my $PATH?

Is there a third way to start a node in an ARM NAS? Maybe building it from source, is that a valid approach or am I doing something wrong here?

Thanks!

Roberto · July 12, 2021, 2:51pm

Hi, you don’t have to use the qnap app. You have to use commands via ssh. There is the guide both on the site and here.
I have TS 230

arturgontijo · July 12, 2021, 3:00pm

Hi @Roberto thanks for your time!

I already tried to “run commands via ssh”, as you can see in my second approach. Basically I’m facing the annoying ./storagenode: line 1: syntax error: unexpected word (expecting ")") error there =/

Could you please paste here your QNAP’s Firmware and Container Station version?

Roberto · July 12, 2021, 4:09pm

Qnap firmaware 4.5.4.1723
Conyainer station v2.3.5.1708
The problem is not in the qnap. Maybe you are using a text editor that changes the quotes, some text programs change them. Make sure you copy the commands from the explanation correctly. I am an inexperienced person and I have succeeded

docker run --rm -e SETUP=“true” --mount type=bind,source=“/share/Public/storagenode.conf/Identity/storagenode”,destination=/app/identity --mount type=bind,source=“/share/Public/Storage”,destination=/app/config --name storagenode storjlabs/storagenode:latest

Alexey · July 12, 2021, 9:04pm

I would say that problem is in QNAP. This error only appears on certain QNAP models.
See Search results for 'syntax error: unexpected word ' - Storj Community Forum (official)

Please try

docker run -it --rm --entrypoint sh storjlabs/storagenode

When you see /app # prompt, try:

arch

Then

./storagenode version

To exit:

exit

Also please post result of the command

docker info

arturgontijo · July 12, 2021, 10:04pm

Hey @Alexey thanks for your time!

Yeah, the issue is definitely with my QNAP =/
I was doing exactly what you mentioned, so, here are the outputs:

[/share/CACHEDEV1_DATA] # docker run -it --rm --entrypoint sh storjlabs/storagenode
/app # arch
aarch64
/app # ./storagenode version
./storagenode: line 1: syntax error: unexpected word (expecting ")")

One thing that I know that’s working is:

/app # mv storagenode storagenode_original
/app # wget https://github.com/storj/storj/releases/download/v1.33.1/storagenode_linux_arm64.zip
/app # unzip storagenode_linux_arm64.zip
...
     4 -rwxr-xr-x    1 root     root          91 Jun 27 21:01 dashboard.sh
 31464 -rwxr-xr-x    1 root     root       30.7M Jul 12 22:01 storagenode
 15492 -rw-r--r--    1 root     root       15.1M Jul 12 22:00 storagenode_linux_arm64.zip
 28860 -rwxr-xr-x    1 root     root       28.2M Jun 27 21:12 storagenode_original
/app # ./storagenode version
Release build
Version: v1.33.1
Build timestamp: 27 Jun 21 21:12 UTC
Git commit: 1dc61476405150a94695397f01d77b67d15e3f9c

After that I commit the container, creating a new image, and that’s it…I was able to run the node…BUT that’s not THAT maintainable =/
Any idea?

Alexey · July 13, 2021, 6:39am

Thank you for testing this!
I filed a bug report to the team based on your info.

I cannot reproduce that in the emulator (I do not have any aarch64 device unfortunately), but you give us a useful information, thanks!

paragpadsumbiya · July 27, 2021, 12:15pm

Hello @arturgontijo

For storj app through UI as I can see its showing service unvaliable so can you please check is your apache or web server is running in your QNAP ?

arturgontijo · July 29, 2021, 11:30am

Hey, thanks for your time.
I’ve decided to go with the fully “command line” approach.
My node is up and running with the latest release, but like I said before, I had to tweak the binary to make it work. Will wait for the next release to see if it will work “out of the box”, if not I’ll analyze why the releases are not covering aarch64 and try to fix it in a PR there =).

artur · July 30, 2021, 3:16pm

Hi everyone. Thanks to everyone looking into this. I looked into this issue today, and the problem was that we mistakenly included aarch32/arm (instead of aarch64/arm64) executables in arm64-tagged storage node images. In most instances, it would run just fine, but some CPUs might have problems with this. The issue was fixed with this change, and new images will be available in our Docker Hub repository soon (it’s currently being built). After the soonest release, it will also update automatically for people who have automatic updates set up.

Thank you so much for reporting this, and I hope you will continue to be a great addition to the Storj network!

arturgontijo · August 3, 2021, 10:55pm

Hi @artur (nice name, btw ), so I just pulled this new arm64v8 image but it is still not working with my QNAP:

[~] # docker run -it --rm --entrypoint sh storjlabs/storagenode:87ceea65e-v1.35.3-go1.15.7-arm64v8                  
/app # ./storagenode version
./storagenode: line 1: syntax error: unexpected word (expecting ")")
/app # arch
aarch64

Must I wait for the “official” latest release?

Alexey · August 3, 2021, 11:38pm

The fix is merged, but it’s not released. So, either workaround:

arturgontijo:

One thing that I know that’s working is:

/app # mv storagenode storagenode_original
/app # wget https://github.com/storj/storj/releases/download/v1.33.1/storagenode_linux_arm64.zip
/app # unzip storagenode_linux_arm64.zip
...
     4 -rwxr-xr-x    1 root     root          91 Jun 27 21:01 dashboard.sh
 31464 -rwxr-xr-x    1 root     root       30.7M Jul 12 22:01 storagenode
 15492 -rw-r--r--    1 root     root       15.1M Jul 12 22:00 storagenode_linux_arm64.zip
 28860 -rwxr-xr-x    1 root     root       28.2M Jun 27 21:12 storagenode_original
/app # ./storagenode version
Release build
Version: v1.33.1
Build timestamp: 27 Jun 21 21:12 UTC
Git commit: 1dc61476405150a94695397f01d77b67d15e3f9c

or wait for the next release.

stefanbenten · August 4, 2021, 7:57pm

Would anyone of you be able to test this image?

stefanbenten/storagenode:87ceea65e-v1.35.3-go1.15.7-arm64v8

This should have the correct binary in it. If thats proven, will push it to the right repo

arturgontijo · August 4, 2021, 8:11pm

@stefanbenten yeah, it works!

[~] # docker run -it --rm --entrypoint sh stefanbenten/storagenode:87ceea65e-v1.35.3-go1.15.7-arm64v8
Unable to find image 'stefanbenten/storagenode:87ceea65e-v1.35.3-go1.15.7-arm64v8' locally
87ceea65e-v1.35.3-go1.15.7-arm64v8: Pulling from stefanbenten/storagenode
58ab47519297: Already exists 
00146f04c1dc: Pull complete 
7c03d953a33e: Pull complete 
8d6e307b834e: Pull complete 
e1849e7907c7: Pull complete 
5b53ab7d50df: Pull complete 
Digest: sha256:58d91883447b68fa034437eefe447647e5bb3fb65a818a4aad27bd26070c43f1
Status: Downloaded newer image for stefanbenten/storagenode:87ceea65e-v1.35.3-go1.15.7-arm64v8
/app # ./storagenode version
Release build
Version: v1.35.3
Build timestamp: 04 Aug 21 16:45 UTC
Git commit: 87ceea65ea4ebf2fbd27fff239611d5eae3613df
/app #

stefanbenten · August 4, 2021, 8:46pm

Thank you! Rollout completed!

Alexey · August 6, 2021, 6:38am

Should be on Docker Hub already.
Is it working with a normal :latest?

arturgontijo · August 6, 2021, 9:24am

Hey @Alexey @stefanbenten @artur , thanks for the effort.
The latest image is working fine.

My next goal is to use the watchtower image, as I’m getting:

# docker run -d --restart=always --name watchtower -v /var/run/docker.sock:/var/run/docker.sock storjlabs/watchtower storagenode watchtower --stop-timeout 300s
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
31f712e2203f964cd039f642013a3c7e232214068333dfcfb53335dac5458b06

The container is running but still not sure if it’s working there.
But I think this is out of STORJ scope, so, will fix this one by myself.

Thanks, guys!

Alexey · August 6, 2021, 4:39pm

It should work though. Just keep it running.