# Lsof: (stat: No such file or directory)

**URL:** <https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009>\
**Category:** troubleshooting\
**Tags:** badger, zfs, corrupted-filesystem\
**Created:** [September 22, 2024, 9:24am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009 "2024-09-22T09:24:14Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\
**Post date:** [September 22, 2024, 9:24am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/1 "2024-09-22T09:24:14Z")

</div>

I’m full on all my nodes. should I be worried?

storageno 3218927 3219213 storageno root mem REG 252,0 4594899 /app/storagenode (stat: No such file or directory)  
storageno 3218927 3219871 storageno root mem-W REG 0,37 26 /app/config/revocations.db (stat: No such file or directory)  
storageno 3218927 3219871 storageno root mem REG 0,37 9455810 /app/config/storage/filestatcache/003883.sst (stat: No such file or directory)  
storageno 3218927 3219871 storageno root mem REG 0,37 9463320 /app/config/storage/filestatcache/003953.sst (stat: No such file or directory)  
storageno 3218927 3219871 storageno root mem REG 0,37 9463215 /app/config/storage/filestatcache/003952.sst (stat: No such file or directory)  
storageno 3218927 3219871 storageno root mem REG 0,37 9461957 /app/config/storage/filestatcache/003987.sst (stat: No such file or directory)  
storageno 3218927 3219871 storageno root mem REG 0,37 9456456 /app/config/storage/filestatcache/00001.mem (stat: No such file or directory)  
storageno 3218927 3219871 storageno root mem REG 0,37 39039703 /app/config/storage/filestatcache/000063.vlog (stat: No such file or directory)  
storageno 3218928 root mem REG 252,0 4594898 /app/storagenode-updater (stat: No such file or directory)  
storageno 3218928 3218934 storageno root mem REG 252,0 4594898 /app/storagenode-updater (stat: No such file or directory)  
storageno 3218928 3218935 storageno root mem REG 252,0 4594898 /app/storagenode-updater (stat: No such file or directory)

ps -p 3218928 -o pid,cmd  
PID CMD  
3218928 /app/storagenode-updater run --binary-location /app/storagenode --config-dir config --identity-dir identity --version.server-address=[https://version.storj.io](https://version.storj.io)  
root@zfstest:~# ps -p 3218927 -o pid,cmd  
PID CMD  
3218927 /app/storagenode run --config-dir config --identity-dir identity --version.server-address=[https://version.storj.io](https://version.storj.io) --storage.allocated-disk-space=16TB

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [September 23, 2024, 2:06am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/2 "2024-09-23T02:06:32Z")

</div>

This is usually happen if the filesystem is corrupted or something is really missing.  
Is it a binary setup or docker?  
I would suggest to stop the node and check and fix the filesystem, then check the mount for `/app/config`.

---

<div class="post-metadata">

**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\
**Post date:** [September 23, 2024, 6:23am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/3 "2024-09-23T06:23:56Z")

</div>

Docker install. All nodes are up and running. No audit fails.  
Fscked

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [September 23, 2024, 6:50am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/4 "2024-09-23T06:50:45Z")

</div>

Seems these errors from the system journals, not from the node, right?

Could it be a permissions issue? Do you run containers with `sudo`? Do you use `--user` option?  
Please post the output inside the container:

```auto
docker exec -it storagenode ls -l /app/config

```

You need to figure out, which one node though.

---

<div class="post-metadata">

**Author:** ![donald.m.motsinger](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/donald.m.motsinger/32/71_2.png) [@donald.m.motsinger](https://forum.storj.io/u/donald.m.motsinger)\
**Post date:** [September 23, 2024, 11:32am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/5 "2024-09-23T11:32:25Z")

</div>

I assume @agente didn’t post the whole output of `lsof` (Please put the loglines between 3 backticks otherwise they’re hard to read.)

I always have these “stat: No such file or directory” “errors”. They only appear with the file descriptor “mem”. Further down there are the same files with the real file descriptor without errors. I just ignore these “errors”

ChatGPT says this

> The error message stat: No such file or directory related to the mem file descriptor in lsof typically occurs when the process has a memory-mapped file (such as a shared library, a dynamically loaded module, or a memory-mapped file from disk) that has since been removed from the file system.

---

<div class="post-metadata">

**Author:** ![arrogantrabbit](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/arrogantrabbit/32/11200_2.png) [@arrogantrabbit](https://forum.storj.io/u/arrogantrabbit)\
**Post date:** [September 23, 2024, 2:31pm UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/6 "2024-09-23T14:31:23Z")

</div>

> [@donald.m.motsinger](#):
>
> ChatGPT says this

LLM is not an authoritative source of any, let alone technical, information. Quoting LLM is like quoting a random word generator. You have to verify the output. And if you do — quote that instead.

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [September 24, 2024, 6:11am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/7 "2024-09-24T06:11:08Z")

</div>

Interesting. My docker nodes doesn’t seems have them.  
Could it be a ZFS-specific somehow?

---

<div class="post-metadata">

**Author:** ![donald.m.motsinger](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/donald.m.motsinger/32/71_2.png) [@donald.m.motsinger](https://forum.storj.io/u/donald.m.motsinger)\
**Post date:** [September 24, 2024, 10:02am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/8 "2024-09-24T10:02:22Z")

</div>

I’m using ext4 for my nodes.

---

<div class="post-metadata">

**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\
**Post date:** [September 24, 2024, 11:04am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/9 "2024-09-24T11:04:26Z")

</div>

I have this error on both. zfs and ext4. btw nodes are working without issue.  
I will post more “well formatted” logs

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [September 25, 2024, 6:28am UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/10 "2024-09-25T06:28:48Z")

</div>

We asked this question, maybe it is not an issue, but I do not trust LLM too 🙂

---

<div class="post-metadata">

**Author:** ![thepaul](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/thepaul/32/65_2.png) [@thepaul](https://forum.storj.io/u/thepaul)\
**Post date:** [September 25, 2024, 5:37pm UTC](https://forum.storj.io/t/lsof-stat-no-such-file-or-directory/28009/11 "2024-09-25T17:37:03Z")

</div>

In this case at least, the LLM is partially correct. That can happen when a file has been opened and has been unlinked from the filesystem since. The more likely situation here, though, is that the file is in a different mount namespace (e.g., a docker container) from the lsof instance and can’t be looked up by its path alone. There are probably other situations where this can happen as well.

Whether it shows the full “No such file or directory” error in each situation appears to depend on what version of lsof you have. The authors would prefer it not to happen, so they play whack-a-mole with these situations and try to eliminate the error. For example, see [https://github.com/lsof-org/lsof/pull/239](https://github.com/lsof-org/lsof/pull/239) .

In short, you don’t need to worry about this.
