Multiple issues with node

Node Operators troubleshooting
,
1

When writing this post, the node has already used 28GB of memory, and that number is still rising at the speed of about 0.08GB/s. CPU is at ~2%, the web dashboard is not working. docker ps shows that it has been up for 14 minutes. No new logs are being logged. The latest logs:
2026-06-25T00:54:15Z WARN hashstore mismatched log tail {“process”: “storagenode”, “satellite”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “store”: “s1”, “id”: 7180, “path”: “config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s1/0c/log-0000000000001c0c-000050d7”, “table”: [{“Key”:“CECGNE3YUYSZUP2SBYPSFDI2QT3FVHK2PVGRAVPE6B6SIE6L2ISA”,“Offset”:3067840,“Log”:7180,“Length”:746496,“Created”:20604,“Expires”:41390},{“Key”:“IIPYGAR7OGPBSD2YJWXH4KPMZEKJATWWKVH6SANLPBA4CBKRIIMA”,“Offset”:2465408,“Log”:7180,“Length”:602368,“Created”:20604,“Expires”:41390},{“Key”:“VYT65VK3ATELXUJCKPIR4UXZIKRWMBMAUZXVWBVJQ3NA4RSZFUKA”,“Offset”:2460992,“Log”:7180,“Length”:4352,“Created”:20604,“Expires”:41390},{“Key”:“ZLCFJBBJM3SVWJIUEHFX6QNG5RRPYCSXBKGKKNSAXGHFHCQHANUA”,“Offset”:2456576,“Log”:7180,“Length”:4352,“Created”:20604,“Expires”:41390},{“Key”:“HG2VBL2LWANCJE55QJWL3XRN5VYISCRDXMEZOAR6ZFEDTSEROFGQ”,“Offset”:1866944,“Log”:7180,“Length”:589568,“Created”:20604,“Expires”:41390}], “log”: [{“Key”:“R3ROIAQJIXUCMBEMG42RYBQ3NLR6JMLNSNFPEA7JUJRMCZQ5YDYA”,“Offset”:3814400,“Log”:7180,“Length”:137472,“Created”:20604,“Expires”:41390},{“Key”:“CECGNE3YUYSZUP2SBYPSFDI2QT3FVHK2PVGRAVPE6B6SIE6L2ISA”,“Offset”:3067840,“Log”:7180,“Length”:746496,“Created”:20604,“Expires”:41390},{“Key”:“IIPYGAR7OGPBSD2YJWXH4KPMZEKJATWWKVH6SANLPBA4CBKRIIMA”,“Offset”:2465408,“Log”:7180,“Length”:602368,“Created”:20604,“Expires”:41390},{“Key”:“VYT65VK3ATELXUJCKPIR4UXZIKRWMBMAUZXVWBVJQ3NA4RSZFUKA”,“Offset”:2460992,“Log”:7180,“Length”:4352,“Created”:20604,“Expires”:41390},{“Key”:“ZLCFJBBJM3SVWJIUEHFX6QNG5RRPYCSXBKGKKNSAXGHFHCQHANUA”,“Offset”:2456576,“Log”:7180,“Length”:4352,“Created”:20604,“Expires”:41390}]}

2026-06-25T00:54:16Z WARN hashstore mismatched log tail {“process”: “storagenode”, “satellite”: “12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S”, “store”: “s1”, “id”: 7436, “path”: “config/storage/hashstore/12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S/s1/0c/log-0000000000001d0c-00000000”, “table”: [{“Key”:“XNBOJJP52KDOCSUOLNCABG7R25Y7VDIXNPJ5I66UJVEFEICTHLUQ”,“Offset”:1072969344,“Log”:7436,“Length”:1305344,“Created”:20592,“Expires”:0},{“Key”:“IHI2EERKHX4PR6BX4HHSDH7VQAG5FHDCS5SAO4HPQZOA5AUAGURA”,“Offset”:1071663936,“Log”:7436,“Length”:1305344,“Created”:20592,“Expires”:0},{“Key”:“B37GSOCXTIM7L2WDNQLE5BXH3B23K75QXW4N7TCITZ7GFSACMTPA”,“Offset”:1071661312,“Log”:7436,“Length”:2560,“Created”:20592,“Expires”:0},{“Key”:“MOHZXSVIVOOYKIQNFJ6XVP776KDT74DZ44KL6W2T7D6VT4SFUIHA”,“Offset”:1071479232,“Log”:7436,“Length”:182016,“Created”:20592,“Expires”:0},{“Key”:“I4JK5VU4IPAJWJR53XDG5MZEI24OAV5YM3Q3CX5L2K7GYKFA7HIA”,“Offset”:1070821696,“Log”:7436,“Length”:1280,“Created”:20592,“Expires”:0}], “log”: [{“Key”:“XNBOJJP52KDOCSUOLNCABG7R25Y7VDIXNPJ5I66UJVEFEICTHLUQ”,“Offset”:1072969344,“Log”:7436,“Length”:1305344,“Created”:20592,“Expires”:0},{“Key”:“IHI2EERKHX4PR6BX4HHSDH7VQAG5FHDCS5SAO4HPQZOA5AUAGURA”,“Offset”:1071663936,“Log”:7436,“Length”:1305344,“Created”:20592,“Expires”:0},{“Key”:“B37GSOCXTIM7L2WDNQLE5BXH3B23K75QXW4N7TCITZ7GFSACMTPA”,“Offset”:1071661312,“Log”:7436,“Length”:2560,“Created”:20592,“Expires”:0},{“Key”:“MOHZXSVIVOOYKIQNFJ6XVP776KDT74DZ44KL6W2T7D6VT4SFUIHA”,“Offset”:1071479232,“Log”:7436,“Length”:182016,“Created”:20592,“Expires”:0},{“Key”:“4PXZYYH3HRJQWNJPILHW6N2S6MATGCNXSY7IYOTMNGBZY6ZSKUIQ”,“Offset”:1070823040,“Log”:7436,“Length”:656128,“Created”:20592,“Expires”:0}]}

2

Node eating memory like this almost always meant so far that your storage is too slow. We have not seen similar reports recently, but in the past that was usually direct cause for memory usage (and general node slowness, like too slow to load the dashboard).

3

Do you use a network attached storage, like smb sshfs nfs or similar? This always results in a higher memory usage, if even works.
The network storage is not supported, please use a local storage or at least iscsi.
However, it’s always better to run the node directly on your fileserver.

See also Search results for 'hashstore mismatched log tail order:latest' - Storj Community Forum (official)