[casaroli]

Summary

rmdir() of a tmpfs directory that is not empty fails with EBUSY. POSIX specifies ENOTEMPTY or EEXIST. tmpfs_rmdir() now returns -ENOTEMPTY for a directory with entries. An empty directory that is open still gives EBUSY.

Impact

• All build modes and architectures: the change is in common tmpfs code.
• A program that removes a non-empty tmpfs directory now gets ENOTEMPTY, not EBUSY. This is a change of an error code that flat-build programs can see. remove() gives the same error.
• FAT, SmartFS, littlefs, mnemofs, xipfs and the pseudo file system return ENOTEMPTY. The rename path of tmpfs already returns ENOTEMPTY first and then EBUSY, in the same order.
• Nothing in nuttx or nuttx-apps tests for EBUSY from rmdir(). examples/mount expects ENOTEMPTY. The caller in fs_rename.c does not test the error code.
• No documentation change: Documentation/ does not list the error codes of rmdir().

Testing

Test program rmdirbusy.c, 6 checks on a new tmpfs mount:

check	master	this PR
rmdir(), the directory has a directory	FAIL EBUSY	PASS ENOTEMPTY
remove(), the directory has a directory	FAIL EBUSY	PASS ENOTEMPTY
rmdir(), the directory has a file	FAIL EBUSY	PASS ENOTEMPTY
rmdir(), the directory has a file and is open	FAIL EBUSY	PASS ENOTEMPTY
rmdir(), the directory is empty and open	PASS EBUSY	PASS EBUSY
rmdir(), the directory is empty and closed	PASS	PASS

The results are the same on qemu-armv8a:knsh (kernel, arm64, QEMU with HVF) and qemu-armv8a:nsh (flat, arm64, HVF): master 2 PASS, 4 FAIL; this PR 6 PASS.

ostest exits with status 0 on master and with this PR: qemu-armv8a:knsh, qemu-armv8a:nsh, and the flat qemu-armv8a:nsh build with the test. All have CONFIG_FS_TMPFS.

No test in CI checks this error code: the cmocka test rmdir02 asks for ENOTEMPTY in its table, but it checks only that rmdir() fails.

Builds with EXTRAFLAGS="-Wno-cpp -Werror", each OK: qemu-armv8a:knsh and qemu-armv8a:nsh (with CONFIG_FS_TMPFS), sim:nsh (macOS host; the only warnings come from the host linker, -ld_classic).

The tests ran on master 39a1f27; the branch is rebased on 3d57048 (no change in the touched files).

tools/checkpatch.sh -c -u -m -g passes.

[github-actions]

MemBrowse Memory Report · Explore what changed →

qemu-armv8a

• Code: .text.tmpfs_rmdir +8 B (+0.0%, 360,200 B)

No memory changes detected for:

• arduino-mega2560
• esp32-devkitc
• hifive1-revb
• mirtoo
• qemu-intel64
• rx65n-rsk2mb
• s698pm-dkit
• stm32-nucleo-f103rb

---

_{Catch memory regressions in CI with MemBrowse. Free for open source.}