JWvdV
June 13, 2024, 7:47pm
1
As a spin-off of topic in test data, the problem now is that I have so much primary IO for uploading and downloading that nice processes like the filewalkers don’t seem to finish and finally are being killed.
So, I was wondering wheter there are options for this.
Especially notice to @elek , with thanks to @littleskunk in the topi:
Mitsos
June 13, 2024, 7:54pm
2
The easiest option is to just add more RAM if you can. Linux automatically caches into RAM if it is just sitting there empty, and reclaims it back if needed by other programs.
2 Likes
JWvdV
June 13, 2024, 7:59pm
3
I already have 16G RAM with zram caching (+20G unused SSD swap), on a total of 32GB of data.
Mitsos
June 13, 2024, 8:14pm
4
You shouldn’t use swap on an SSD, it’s only a way to kill it faster.
If you can, add more sticks of RAM. If not, someone else may have a better idea (ie LVM cache, which I don’t use so can’t comment).
1 Like
Alexey
June 14, 2024, 4:35am
5
I would recommend to take a look on this thread:
Hi, I maxed out my system RAM and is not enough for my nodes. Less than 1gb per TB.
I have ext4 in a Ubuntu 22 server. How can I add a 4tb SSD specialized for metadata caching? First cache in RAM then SSD. My system is slowing down with this massive gargabe deleting
elek
June 18, 2024, 5:26pm
6
I am not fully sure that the Linux kernel will use swap for inode caching. I think it’s different story, but can be wrong.
Also: 16GB seems to be small. I have seen 66GB inode cache on my machine…
JWvdV
June 18, 2024, 5:51pm
7
For sure it won’t, but using swap the real RAM use is lower, so there remains more space for the virtual file system.
How can i check this number?
JWvdV
June 18, 2024, 9:31pm
9
That number, you actually can’t. Because it’s in the past and Elek’s system
But if you want to check for now on your own system:
cat /proc/meminfo
Then you find it in slab.
1 Like
elek
June 19, 2024, 4:13pm
10
sudo slabtop -o
And check ext4_inode_cache
…
(-o
is just for executing once instead of referesh it multiple times. Easier to grep)
3 Likes