On my Node’s dashboard the bandwidth, and disk utilization graphs not populating. When I reboot the server they repopulate with old data, but upon first refresh they go away.
Can you run developer tools in the browser when you click the refresh button? It could be that the reply is malformed.
I assume you’ve cleared the cache, cookies, etc.
Thanks for the reply. I don’t see any errors in Developers tools, and yes cleared cache, tried multiple computers, multiple browsers.
Also, it’s not only when I refresh. It’s whenever I go to the dashboard. Oddly enough the only time I see the data is the first time after I reboot the node.
Oddly enough it does appear the Disk Space Used Circle is populating correctly. I’m not sure if that’s helpful or not.
Also the CLI dashboard does not load. There’s a line about my location, and my node ID, but that’s it. I let it sit for hours, and nothing populates
Are there any error messages that show up in the logs immediately after the node is started?
It is possible there is a problem with one (or more) of the databases. You should check them for errors:
Baker,
Thanks for the suggestion. I repaired the databases. They all came back OK, but I"m not sure if that means they were repaired or if the process ran successfully. Unfortunately, it didn’t appear to fix the issue. If i remove the DB’s It appears I’ll lose some stats which isn’t all that huge of a deal. Do you know if they’re recreate? Also that wouldn’t effect my status on an satellites would it?
So the first steps from that article just check the integrity of the databases. There is no automatic repair, so if they all check as “OK”, then there is no problem. The node can run with empty databases, but yes you would loose historical stats in the dashboard. This would not effect your status on the satellites, as the local database is only for local operations.
That being said, if the integrity check came back as okay, then I doubt the databases are to blame. So I wouldn’t start them over yet. Can you watch the log in real time via the terminal and see if any errors are thrown as you access the dashboard?
I found a warning in the logs. I never noticed it before, but that log is a quick one. It won’t let me paste it into the text because it detects some of it as links, so please forgive me for the screenshot
Hello @spilegi,
Welcome to the forum!
I saw that you already checked your databases, however, the only reason for zeros on dashboard is broken or empty databases.
So, make sure that you specified a correct path to the databases, when you check them. Otherwise the sqlite would just create an empty ones and of course they would be “ok”.
Also, please, use a Chrome or Edge browsers, all other could not display the dashboard properly.
Thanks @Alexey and @baker for the responses. I think I’ll just save off the databases and start fresh. I’m starting to cause too much downtime, trying to track my downtime lol Is it safe to delete all the databases in the storage directory? I just want to confirm so I don’t interfere with the running of my node. It’s been up for 8 months and I don’t want to lose the 4TB of data I’m storing. Ironically this all started because I moved the server to a new PC with more storage
You do not need to start fresh! You will lose your held amount with that.
You can start with a clean databases of course, they only break your stat on the dashboard, but the data will be in safe.
You can delete (or better to rename) databases, they should be re-created on start.
Sorry yes that’s what I meant sorry lol. I plan to wipe the sqlite DB’s and start fresh on the dashboard.
Just to follow up, I wiped the dB’s and still had the same issue. But I misread the bottom of the page. My Storj files are stored on NFS storage. The problem started just as I switched to NFS. Looks like nfs and sqlite don’t play well together. I’m thinking I’ll switch the backed storage to iscsi to resolve the issue. If it was just the stats, I’d be ok with it not working, but it appears to be causing a memory leak. Every few days I have to restart my node as it’s been building up to 8 GB of memory used
Sounds like that’s your problem. As far as I know iSCSI is the only supported network protocol.
I plan to migrate to iscsi. I agree I believe this will resolve the issue. Once completed if successful I’ll post the results in case anybody else runs into the same issue.
Yes, the NFS and SMB are not compatible with sqlite and thus - with storagenode
You can take a look on https://forum.storj.io/tag/nfs and https://forum.storj.io/tag/smb
