Hello,
I just got a docker node up, and after a couple of hours, it stopped complaining there is no more space. I still have plenty of space available on the disk, but for some reason, the docker image is somehow limited. It has 10G.
Debian 11.8, all latest updates done.
Any thoughts?
Regards,
did you mount the volume correctly? Maybe the node was storing the data directly inside the container?
1 Like
i have a dedicated user “storj” and the StarageLocation is in the home dir of the user. The container files is in the standard /var/lib… location.
what I could get until now from web search is that there is a limitation at the mounting/mapping inside the container at 10G, default from docker. But still don’t know how to change it.
Hello @cnicules,
Welcome to the forum!
Please show result of the command
df --si -T
and your docker run command (you may mask the private info).
Filesystem Type Size Used Avail Use% Mounted on
udev devtmpfs 2.1G 0 2.1G 0% /dev
tmpfs tmpfs 410M 721k 409M 1% /run
/dev/sda2 ext4 2.0T 111G 1.8T 6% /
tmpfs tmpfs 2.1G 0 2.1G 0% /dev/shm
tmpfs tmpfs 5.3M 0 5.3M 0% /run/lock
tmpfs tmpfs 410M 0 410M 0% /run/user/1003
and
docker run -d --restart unless-stopped --stop-timeout 300
-p 28967:28967/tcp
-p 28967:28967/udp
-p 5.xx:14002:14002
-e WALLET=“0xxxx83”
-e EMAIL=“cipxxxxx”
-e ADDRESS=“5.xx:28967”
-e STORAGE=“1600GB”
–user $(id -u):$(id -g)
–mount type=bind,source=“/home/storj/.local/share/storj/identity/storagenode/”,destination=/app/identity
–mount type=bind,source=“/home/storj/StorjStorage”,destination=/app/config
–name storagenode storjlabs/storagenode:latest --operator.wallet-features=zksync-era
So, you used a system drive to also store the node’s data…
Not a good choice, but - well…
From the error it looks like you (perhaps) do not have enough permissions to operate a data…
Could you please change the owner to $(id -u):$(id -g)?
sudo chown $(id -u):$(id -g) -R /home/storj/StorjStorage
and try to invoke your full docker run command?
P.S. please note - you need to change permissions from a regular shell, not under root…
i don’t get it, what you are getting at
I run the docker with the user that has full ownership as it’s in his home dir, and he created it.
and yes I did permit that user to run docker images and he can
all worked well until it hit 10G of space
du -sh StorjStorage/
10G StorjStorage/
then inside container, I started to get
sed: couldn’t flush /etc/supervisor/sedeqKe8J: Disk quota exceeded
sed: couldn’t flush /etc/supervisor/sedeZhLre: Disk quota exceeded
sed: couldn’t flush /etc/supervisor/sedhfi3V8: Disk quota exceeded
sed: couldn’t flush /etc/supervisor/sediWjUVt: Disk quota exceeded
at various googling I get something like this docker and image size limit - Stack Overflow
Please give me result of the command
docker exec -it storagenode df --si
docker exec -it storagenode df --si
Filesystem Size Used Avail Use% Mounted on
overlay 2.0T 108G 1.8T 6% /
tmpfs 68M 0 68M 0% /dev
shm 68M 0 68M 0% /dev/shm
/dev/sda2 2.0T 108G 1.8T 6% /app/config
tmpfs 2.1G 0 2.1G 0% /proc/acpi
tmpfs 2.1G 0 2.1G 0% /sys/firmware
Interesting. Your container should not have any issues with a free space at /etc/, it’s not mounted differently.
tmpfs is low, but not where is it used…
Is it a snap docker?
it’s a normal cli debian docker, installed with apt
Did you check quota for this user and group?
See also debian - Disk quota exceeded problem - Unix & Linux Stack Exchange
hmm, per user was no limit, but a default limit at 10G was set. i removed it also. Restarted and will monitor.
1 Like
I do not know, why Debian doing so.
I switched everything to Ubuntu several years ago - no weird issues so far
yeah got to 22G, yeeee.
Thanks
1 Like