# Out of Memory issue (memory leak?)

**URL:** <https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848>\
**Category:** troubleshooting\
**Tags:** nfs\
**Created:** [April 12, 2020, 5:54pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848 "2020-04-12T17:54:21Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![dariop84](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/dariop84/32/1136_2.png) [@dariop84](https://forum.storj.io/u/dariop84)\
**Post date:** [April 12, 2020, 5:54pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/1 "2020-04-12T17:54:21Z")

</div>

Hi All,

I have a brand new node (4 Core / 16GB RAM) where the storagenode container is eating up all the memory until it reaches the limit and it gets killed for OOM…

I’m using the latest version (v1.1.1) of storjlabs/storagenode:beta (is it right to use the beta?)

The machine is fresh installed and super clean, just Centos 7.7.1908 (Core) + ntpd + docker, nothing else.

I saw that several people had the same issues in June 2019, but then it has been fixed, so any idea why is still happening?

---

<div class="post-metadata">

**Author:** ![deathlessdd](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/deathlessdd/32/6067_2.png) [@deathlessdd](https://forum.storj.io/u/deathlessdd)\
**Post date:** [April 12, 2020, 6:45pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/2 "2020-04-12T18:45:05Z")

</div>

Hi and welcome  
Without knowing how your setup is running your node there’s no telling what the issue is without some logs, Is your system caching all data in ram before putting in on the hard drive because your hard drive can’t keep up? Run

> docker stats

To get an Idea of what is going on with your node.  
The issues in june were more related to rpi with low ram, A system with 16gigs of ram should no where becoming close to running out of memory ever.

---

<div class="post-metadata">

**Author:** ![dariop84](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/dariop84/32/1136_2.png) [@dariop84](https://forum.storj.io/u/dariop84)\
**Post date:** [April 12, 2020, 7:07pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/3 "2020-04-12T19:07:27Z")

</div>

Hi, thanks!

Well, there is not much more to say about the setup, it’s a VM in a vSphere 6.7 cluster and the only “special” things is that I’m using a NFS mount to store data, but performance of the NFS is much much higher that the amount of work that my node is doing so it’s should not be an issue at all

this is docker stats

CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS  
b89860a76b94 watchtower 0.00% 6.504MiB / 15.51GiB 0.04% 1.35kB / 0B 0B / 0B 9  
a9ef8a5b9877 storagenode 1.40% 7.394GiB / 15.51GiB 47.67% 11.1GB / 320MB 0B / 0B 67

this is the ram usage

[root@storj01 ~]# free -m  
total used free shared buff/cache available  
Mem: 15885 12509 2781 8 593 3073  
Swap: 6555 0 6555

I don’t see any error in the storagenode container logs, just a ton of:

INFO piecestore upload canceled… “Action”: “PUT”, “error”: “context canceled”, “errorVerbose”:

---

<div class="post-metadata">

**Author:** ![deathlessdd](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/deathlessdd/32/6067_2.png) [@deathlessdd](https://forum.storj.io/u/deathlessdd)\
**Post date:** [April 12, 2020, 7:14pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/4 "2020-04-12T19:14:42Z")

</div>

Looks like its more of an issue running with vsphere but I dont have much experience with vsphere tho but Im willing to bet its using ram as a cache, also NFS comes to mind where it wasnt recommended to use ever, with a node.

---

<div class="post-metadata">

**Author:** ![nerdatwork](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/nerdatwork/32/1135_2.png) [@nerdatwork](https://forum.storj.io/u/nerdatwork)\
**Post date:** [April 12, 2020, 7:29pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/5 "2020-04-12T19:29:52Z")

</div>

> [@dariop84](#):
>
> I’m using a NFS mount to store data

> [@Possible memory leak, processes not clearing from RAM](https://forum.storj.io/t/possible-memory-leak-processes-not-clearing-from-ram/2682):
>
> Been running a node for a few months now, I’ve seen some strange behaviour on 0.25.1. I see huge spikes in memory with one or two processes usually not being killed. Not sure if this is normal behaviour? My normal RAM usage sits at around 4GB, with storj running for more than 24 hours or so that increases to 12GB. I can’t find anything out of the ordinary in the logs. When I restart the container usage returns to normal. Not sure if anyone else is seeing this? Edit: Not sure how visib…

---

<div class="post-metadata">

**Author:** ![iops](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/iops/32/70_2.png) [@iops](https://forum.storj.io/u/iops)\
**Post date:** [April 12, 2020, 7:55pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/6 "2020-04-12T19:55:08Z")

</div>

> [@dariop84](#):
>
> I’m using a NFS mount to store data

My issues went away when I started storing the data on a HDD. This is a SQLite issue as opposed to a storj one. My performance is far better, and therefor payments are better, after ditching NFS shares.

---

<div class="post-metadata">

**Author:** ![dariop84](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/dariop84/32/1136_2.png) [@dariop84](https://forum.storj.io/u/dariop84)\
**Post date:** [April 12, 2020, 9:00pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/7 "2020-04-12T21:00:05Z")

</div>

Hi All,

OK I have moved the data to an iSCSI (over 10G) datastore, let’s see if things are better!

---

<div class="post-metadata">

**Author:** ![Odmin](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/odmin/32/92_2.png) [@Odmin](https://forum.storj.io/u/Odmin)\
**Post date:** [April 12, 2020, 9:22pm UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/8 "2020-04-12T21:22:53Z")

</div>

Please move to iSCSI ASAP, NFS have big issues with sqlite and consume much more resources then iSCSI. Also, please reserve all RAM on your VM.

---

<div class="post-metadata">

**Author:** ![dariop84](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/dariop84/32/1136_2.png) [@dariop84](https://forum.storj.io/u/dariop84)\
**Post date:** [April 13, 2020, 8:29am UTC](https://forum.storj.io/t/out-of-memory-issue-memory-leak/5848/9 "2020-04-13T08:29:31Z")

</div>

Looks like iSCSI solved the issue 😉  
Thanks!
