Dashboard uses CPU and RAM when it always running

Node Operators FAQ
1

Sorry, unfortunately I can’t create new topic, but I would like pay you attention on SNO dashboard performance (docker exec -it storagenode /app/dashboard.sh)

image
image1573×786 388 KB

test enviroment is:
4 CPU of Xeon E3-1275v6
16GB of RAM

Here another example:

image
image1556×794 412 KB

test enviroment is:
4 CPU of Celeron J3455
4GB of RAM

Information collected using netdata, statistics from storage node container.
image

As you can see, when it running, it consume a lot of CPU resources. I think it especially important on RPi and other low power systems.

1 Like
2

Here is detailed graph when dashboard is not running

image
image1542×771 339 KB

test enviroment is:
4 CPU of Xeon E3-1275v6
16GB of RAM

image
image1540×775 328 KB

test enviroment is:
4 CPU of Celeron J3455
4GB of RAM

3

@Odmin Interesting, but correct, I can confirm it too.

Sn%C3%ADmka%20obrazovky%20z%202019-07-14%2014-38-46
Sn%C3%ADmka%20obrazovky%20z%202019-07-14%2014-38-46817×298 41 KB

4

That the dashboard takes performance and CPU is to be expected. It keeps track of your usage in real time. There is no need to have it running 24/7.

1 Like
5

Yes, I agree, I just pay attention for it because our topic is “Storage node performance improvements ideas”, and it not a issue.
I believe new SNO board that will release soon will mach better.

6

how are you doing to have this display with these graphics please?

7

I use netdata, it simple and powerful monitoring and troubleshooting tool.
You can read more about here

Instalation is very simple:
bash <(curl -Ss https://my-netdata.io/kickstart.sh)

After installation is complete, you will have everything out of box, no need additional configuration.

Only one parameter I changed on /etc/netdata/netdata.conf
history = 86400
For keep one day history, it will cost about 522 MB of memory. If it on RPi or other system with 1GB of RAM please do not change this parameter and after investigation please stop netdata service for keep more RAM for storagenode.

8
9

I’m seeing the same impact. In light of these learnings, I would love to have an option to only update the dashboard once a minute or so. --update-interval 60 Something like that?

1 Like
10

Thanks for reporting it and provides all that useful information and graphs.
I have opened a ticket in our issue tracker https://storjlabs.atlassian.net/browse/V3-2169

11

You are welcome! I always glad to help :slightly_smiling_face:
Thank you too for pay attention to this idea!

12

Thanks for taking action @ifraixedes , just to make sure you’re aware. That link is not publicly available.

13

Thanks for the heads up; I know that our issue tracker ins’t open however having the link here helps to easily identify that this problem reported in this thread is already tracked.

2 Likes
14

This is good stuff, thank you!

1 Like
15

I’ve noticed my nodes seemed to run OK for a while but after a few hours or days would notice memory consumption would grow and swap usage and disk wait times would increase (seemingly due to increased swap usage). Originally I put this down to overloaded system but I’ve now got a strong suspicion this is due to the dashboard script and that when exiting this it continues to run and consume resources. I noticed this as I started to spot multiple dashboard processes running in the docker instances even though I had terminated these and was no longer running the dashboard. Perhaps it is because I am Ctrl-C’ing out of the dashboard to quit but not aware of any other way to quit. Everything seems to run much better when I do not run the dashboard (or at least I have to restart the node after running the dashboard). Any thoughts?

16

I’ve noticed my nodes seemed to run OK for a while but after a few hours or days would notice memory consumption would grow and swap usage and disk wait times would increase (seemingly due to increased swap usage). Originally I put this down to overloaded system but I’ve now got a strong suspicion this is due to the dashboard script and that when exiting this it continues to run and consume resources. I noticed this as I started to spot multiple dashboard processes running in the docker instances even though I had terminated these and was no longer running the dashboard. Perhaps it is because I am Ctrl-C’ing out of the dashboard to quit but not aware of any other way to quit. Everything seems to run much better when I do not run the dashboard (or at least I have to restart the node after running the dashboard). Any thoughts?

17

Hello @julez,
Welcome to the forum!
Yes, the dashboard is uses RAM and CPU when always running, as you can read in the current topic

18

Thanks for getting back to me @Alexey. The problem I am having is that the dashboard continues to run when I control-C out of it and continues to consume resources. When I want to look at the dashboard again it start a new instance and consumes even more resource. How do I exit it properly so it quits and stops consuming resource?

19

This is depends on how you run the dashboard.
If you run it like this:

docker exec -it storagenode ./dashboard.sh

and then decided to exit by Ctrl-C, it will be stopped and removed from the running processes.

But if you run it in a different way - it could become a zombie process.
To free up resources you always can restart the container:

docker restart -t 300 storagenode
1 Like
20

Thanks @Alexey. I was missing the -it part. Maybe I should’ve RTFM or should’ve known better. Thanks again

1 Like