[HELP] Improve NuttX bloated size #20414

Activity

1. acassis

   added

   Community: QuestionFurther information is requestedFurther information is requested

   on Sep 30, 2026
2. arnavsharma990 commented on Oct 1, 2026

   

   arnavsharma990

   on Oct 1, 2026

   Contributor

   More actions

   I did a linked-image size investigation on stm32f103-minimum:nsh
   using arm-none-eabi-gcc 16.2.0, arm-none-eabi-size, and
   nm --print-size. Baseline re-verified after each test, tree clean,
   all source stubs reverted:

   51608 .text / 480 .data / 2952 .bss / 52088 B flash

   Isolated flash deltas:

   • CONFIG_DISABLE_MOUNTPOINT=y: -2812 B (includes removed
     mount/NSH commands, not just the dir layer)
   • CONFIG_ARCH_STACKDUMP=n: -1320 B
   • CONFIG_FDCLONE_STDIO=y: -24 B
   • Temporarily stubbing fdlist_extend() to -EMFILE: -196 B
     (upper bound only, reverted)
   • Single-iovec read/write fastpath with compat kept: +36 B
     (negative result, reverted)

   The interesting discrepancy is that current linked-image results don't match
   several historical figures. E.g. the growable FD table is reported as
   ~1.9 KB, while removing fdlist_extend() from the current linked image
   changes it by only ~196 B.

   Could you share the baseline commit/configuration and size command used
   for the historical measurements? I'd like to reproduce the original
   deltas under the same conditions and trace which changes account for the
   reported growth.

   ❤️React with ❤️1Reacted by Alan C. Assis
3. acassis commented on Oct 2, 2026

   

   acassis

   on Oct 2, 2026

   ContributorAuthor

   More actions

   @arnavsharma990 I think there are others missing pieces on this puzzle.

   A way to spot these difference is using the nm to list all functions from a 2016 build and the current build, then you can spot the size changes and the new added functions.

   I used it initially to spot the biggest functions on Nutt, then I was able to create the minnsh that was really small, but it was just a PoC because it removed even the VFS that remove support to device files, so we cannot have application using drivers in the Unix way.

   https://acassis.wordpress.com/2015/04/27/how-to-verify-which-functions-are-eating-space-in-an-elf-file/

   👍React with 👍1Reacted by Arnav Sharma
4. arnavsharma990 commented on Oct 2, 2026

   

   arnavsharma990

   on Oct 2, 2026

   Contributor

   More actions

   2016 → current forensic comparison

   As suggested by @acassis, I completed the 2016 → current forensic comparison using nm plus isolated parent→child builds (same GCC 16.2.0 for both sides, so the delta is source/config rather than toolchain; era toolchains would shift the absolutes slightly).

   Note

   For stm32f103-minimum:nsh, the linked image grew from 38,872 B in 2016 to 52,088 B currently (+13,216 B / ~34%).

   Beyond the features originally listed, the comparison attributes most of the rest to NSH evolution, serial/termios functionality, libc time/getopt, configuration enablement (e.g. FS commands unlinked in 2016, linked now), and smaller VFS/heap/workqueue changes.

   Reproduced with parent→child builds rather than inferred from symbols

   Change	Measured linked-image delta
   getopt_long() refactor	+248 B at introduction (2021), ~+380 B net after later hardening
   ls -l timestamp support	+1,068 B linked, including the gmtime_r/strftime chain
   Heap	+548 B net — not a major driver

   Important

   Net of overlaps, the ~13.2 KB is accounted for down to ~100–150 B of scatter plus alignment/exidx effects.

   One side observation: the historical per-feature estimates don't map 1:1 onto current linked deltas (e.g. FD table ~1.9 KB claimed vs ~200 B upper bound measured today), so the historical comparison seems more informative than summing today's symbol sizes.

   I have the nm lists and pair-build logs saved — happy to share them or dig into whichever subsystem you'd find most useful next.
5. jingfei195887 commented on Oct 9, 2026

   

   jingfei195887

   on Oct 9, 2026

   Contributor

   More actions

   I got the same result on ARM stm32f103-minimum:nsh with a 34% system image increase. Thank you @acassis for your analysis breakdown. I share similar conclusions and have investigated the "directories opened through the file layer (~2.1 KB)" item.

   About the "directories opened through the file layer (~2.1 KB)" item

   here is the mechanism behind the growth, and a concrete proposal to let size-constrained configs opt back out. Measured on the final linked image, stm32f103-minimum:nsh, both sides built with the same arm-none-eabi-gcc 10.3.1.

   The architectural change

   2016 (7.19) — directories were a self-contained leaf subsystem under fs/dirent/. opendir() was roughly:

   dir   = kumm_zalloc(sizeof(struct fs_dirent_s));
   inode = inode_search(&path, ...);
   /* populate dir from inode, return */

   No file descriptor, no struct file, no VFS open/read/close path. readdir/closedir/seekdir/rewinddir were standalone functions operating directly on fs_dirent_s.

   Current — a directory is a first-class file descriptor. opendir() now lives in libc:

   dir     = lib_malloc(sizeof(*dir));   /* DIR is basically { int fd } */
   fd      = open(path, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
   dir->fd = fd;

   readdir() → read(fd) → file_readv → dir_read(); closedir() → close(fd). A new fs/vfs/fs_dir.c bolts directories onto the file layer through a synthetic g_dir_inode + g_dir_fileops, with dir_open/close/read/seek/ioctl and dir_allocate.

   Where the ~2 KB actually goes (the non-obvious part)

   The visible directory symbols actually shrank: opendir+readdir+closedir went 530 B → 170 B. So the growth is not in the dir calls themselves. It is in the machinery the dir path now forces to be linked:

   • fs_dir.c glue itself: ~790 B .text + 40 B RAM (g_dir_inode)
   • the generic open() / fd-allocation path, the struct file lifecycle, the vectored file_readv read path, and fd_path construction (inode_getpath + asprintf + a PATH_MAX scratch buffer)

   In a configuration that also opens regular files, most of this machinery is shared and effectively free. In a directory-centric minimal image it is paid for by directories — which is why this shows up as a standalone item.

   Why it is deliberate, not a regression

   Making directories real descriptors is exactly what provides dirfd(), fdopendir(), O_DIRECTORY, fchdir(), and /proc/self/fd consistency. A wholesale revert to the 2016 leaf implementation would drop those POSIX behaviours, so the bulk of this item is the cost of a conformance feature, not fat to trim.

   Proposal: an opt-out CONFIG for a lightweight DIR implementation

   Since the bulk of the ~2.1 KB is the descriptor machinery rather than the directory logic, the natural lever is to let size-constrained builds opt back out of it with a Kconfig — e.g. CONFIG_FS_DIRENT_LITE (name TBD):

   • When disabled (default): current behaviour — directories are first-class fds, full POSIX conformance.
   • When enabled: opendir() resolves the inode and allocates a fs_dirent_s directly (the 2016 leaf model), and readdir/closedir/seekdir/rewinddir operate on that struct instead of going through open()/read(fd)/close(fd). No synthetic g_dir_inode, no forced file_readv/fd-lifecycle pull-in, no fd_path construction.

   What this reclaims and what it costs:

   • Reclaims: most of the fs_dir.c glue plus the dir-only portion of the forced file-layer path — i.e. the bulk of this item, for configs that only need opendir/readdir/closedir.
   • Costs: the descriptor-based POSIX behaviours that motivated the current design — dirfd(), fdopendir(), O_DIRECTORY returning a usable dir fd, fchdir() on a dir fd, and dir fds appearing in /proc/self/fd. These would compile out under the lite path.

   Mountpoint filesystems are unaffected either way: both the legacy and current paths dispatch directory I/O through inode->u.i_mops->{opendir,readdir,...}, so no per-filesystem changes are needed — the opt-out is confined to the VFS dir layer.

   The open question for maintainers is whether carrying two dir implementations behind a CONFIG is acceptable maintenance surface for the size win. If it is, I can prototype CONFIG_FS_DIRENT_LITE, measure the exact linked-image delta on stm32f103-minimum:nsh, and run opendir/readdir/dup(dirfd) on sim:nsh to confirm the conformance boundary behaves as documented.

   Bottom line: the ~2.1 KB is overwhelmingly the price of directories-as-descriptors (a POSIX feature), not waste. It is not meaningfully reclaimable by local cleanups; the only lever that recovers most of it is an explicit opt-out CONFIG that trades the dir-fd POSIX behaviours for size — which is why it belongs behind a Kconfig rather than as an unconditional change.
6. acassis commented on Oct 9, 2026

   

   acassis

   on Oct 9, 2026

   ContributorAuthor

   More actions

   Hi @jingfei195887 thank you for you comments, I think a CONFIG_FS_DIRENT_LITE should be a good idea!
   Maybe we could add too a CONFIG_VFS_LITE to create a low footprint solution for boards with small amount of flash memory.
7. jingfei195887 commented on Oct 10, 2026

   

   jingfei195887

   on Oct 10, 2026

   Contributor

   More actions

   Code size analysis for stm32f103-minimum:nsh by modules

   Good suggestion @acassis . I'm looking into simplifying VFS. Before that, I think we'd better to sync up code size growth. so I broke down the problem from a module perspective based on the stm32f103-minimum:nsh growth from NuttX 7.19 (2016) to current master, down to the object level, with the historically quoted per-feature numbers reconciled against what the final linked image actually shows.

   Both sides built with the same arm-none-eabi-gcc 10.3.1: 7.19 = 02146758ef + apps 5291d3e9bc; current = cc2edca964 + apps 1f8cac8b87.

   Method

   Derived from the linker .map, not from summing nm symbols: attribute each placed section to its object file (not the section-name suffix, else -ffunction-sections .text.<symbol> fragments are dropped); count only addr != 0 sections (skip --gc-sections discards that linger at addr=0, size>0); cross-check against size (text+data) as a correctness gate.

   Validation: current side placed-vs-ELF error 0.29%, 7.19 side 0.76% (fixed unattributed overhead - padding, vector table, .ARM.exidx - is a larger fraction of a 39 KB image). The 7.19 map placed total is 38,872 B, byte-for-byte the historical 2016 figure in this issue. Per-module deltas also agree line-for-line with an independent nm-based pass.

   Totals

   Metric	7.19	current	delta
   ELF Flash (text+data)	39,170	52,427	+13,257
   map placed total	38,872	52,273	+13,401

   The per-library table below sums to the map-placed +13,401; the ~150 B gap to the ELF +13,257 is the unattributed overhead noted above. Per-module figures are map-placed, the headline Flash figure is ELF-measured.

   Library (subsystem)	7.19	current	delta
   libsched.a (sched / sem / mqueue / wqueue / timers)	9,422	13,736	+4,314
   libfs.a (VFS / filesystem)	3,094	6,611	+3,517
   libapps.a (NSH + apps)	10,429	12,974	+2,545
   libc.a (C library)	5,473	7,870	+2,397
   libdrivers.a (drivers)	2,727	4,118	+1,391
   libmm.a (memory manager)	1,451	2,107	+656
   libarch.a (arch startup/exception)	4,540	3,861	-679

   Three cross-cutting corrections (object-level de-duplication)

   Object diffs have the same "a new symbol is really a renamed/relocated/merged old one" trap as symbol sums:

   1. assert.o +690 in libsched is a relocation, not new code - 7.19's assert was in arch/.../up_assert.c (libarch); it moved to sched/misc/assert.c. This almost exactly explains why libarch shrank 679.
   2. libapps per-object diffs are a naming artifact - 7.19 nsh_command.o vs current nsh_command.c.<path>_1.o, zero shared keys. Matched by logical name, nsh_command grew only +127, not the +1782 a raw diff shows.
   3. The signal subsystem net is -517 (it shrank) - the nx* refactor made signals smaller, directly contradicting the "signal atomic fast/slow ~2.8 KB growth" line in the original list.

   libsched.a +4,314

   Largest mover - and not because of a few named features. Every object is partitioned into exactly one group below, so the column sums to the full +4,314. The headline is that the scheduler got broadly heavier: core/task/pthread/clock/group/env together add ~2.75 KB, more than all the named new features combined.

   Group	7.19 -> cur	net	nature
   scheduler core + startup (sched_*, nx_start/os_start)	2123 -> 2970	+847	broad per-object growth
   semaphore (incl. new sem_rw rwlock)	534 -> 1348	+814	atomic paths + rwlock
   assert/panic	0 -> 754	+754	relocated from arch (not new; assert.o is +690)
   task mgmt (task_*)	1597 -> 2139	+542	broad per-object growth
   POSIX timer (timer_initialize/release)	0 -> 524	+524	feature: preallocated timers
   workqueue (kwork_* merged into kwork_thread)	327 -> 796	+469	thread model
   pthread (pthread_*)	48 -> 462	+414	join/exit/cancel paths
   clock (clock_*)	136 -> 464	+328	timekeeping
   group (group_*)	387 -> 709	+322	task-group lifecycle
   env (env_*)	538 -> 840	+302	environment handling
   misc (tls/errno/instrument)	36 -> 174	+138
   exit	28 -> 60	+32
   irq	160 -> 170	+10
   mqueue (mq_*)	452 -> 254	-198	shrink
   watchdog (wd_*)	711 -> 244	-467	shrink
   signal (sig_*/nxsig)	2345 -> 1828	-517	nx* refactor shrank it

   So libsched's +4,314 is dominated not by the named new features (semaphore +814, timer +524, workqueue +469, which net against signal -517 / watchdog -467 / mqueue -198) but by broad, small per-object growth across the scheduler core, task management, pthread, clock, group and env (~2.75 KB combined), plus the assert relocation from arch (+754, offset by libarch -679).

   libfs.a +3,517

   One theme: directories, read/write and ioctl all routed through the file layer / vectored path. Full partition, sums to +3,517.

   Subsystem	net	reason
   growable fd table (fs_files)	+743	dynamic fd list
   inode search (fs_inodesearch)	+628	inode_search_setup rework (incl. FS-tree rwlock)
   generic ioctl (fs_ioctl)	+474	split-out ioctl layer (FIOC_FILEPATH)
   vectored read/write (fs_read+fs_write)	+404	scalar -> struct uio
   open (fs_open)	+282	O_DIRECTORY + dir_allocate dispatch
   directory layer (fs_dir + old dirent)	+300	directories-as-fds
   mkdir/rmdir (split out)	+266	standalone objects
   umount (fs_umount2)	+222
   uio (fs_uio)	+184	vectored-I/O helper
   other objects (net)	+14
   TOTAL	+3,517

   fs_dir.o (+832) absorbs old fs_opendir+fs_readdir+fs_closedir; the visible dir symbols actually shrank (530->170 B). The growth is the file-layer machinery the dir path now forces to link (opendir -> open(O_DIRECTORY) -> fd -> file_readv -> dir_read), which buys dirfd()/fdopendir()/fchdir()//proc/self/fd. Nearly all of libfs is implementation bloat from interface unification - the subsystem with the most reduction leverage.

   libapps.a +2,545

   Almost entirely NSH feature growth. Matched by logical name (object naming scheme changed between versions). Full partition, sums to +2,545.

   delta	object (logical)	reason
   +1,498	nsh_fscmds	more filesystem commands
   +796	nsh_parse	heavier parser
   +528	nsh_mntcmds	new mount/umount commands
   +400	nsh_syscmds	new system commands (uname)
   +364	readline (fd+common)	new fd-based readline
   +211	nsh_envcmds	environment commands
   +127	nsh_command	dispatch table
   -1,379	other objects (net)	many 7.19 commands shrank/merged (ddcmd, dbgcmds, etc.)
   +2,545	TOTAL

   The growing objects are features users asked for (config-gated via CONFIG_NSH_DISABLE_*); the large -1,379 "other" shows a lot of the 2016 command set was simultaneously slimmed, so the net is much smaller than the gross additions.

   libc.a +2,397

   ls -l time formatting, getopt_long, termios - not printf. Full partition, sums to +2,397.

   group	net	driver
   ls -l time (strftime+gmtime_r)	+910	timestamp display
   getopt_long (getopt_common, net of old getopt)	+380	long-option parsing
   realpath	+346	NSH canonical paths
   termios (cfspeed)	+320	serial line config
   printf engine (libvsprintf+vasprintf)	+180
   uname (utsname)	+142
   other objects (net)	+119	strtoul, itoa, streams, mutex, getcwd, ...
   TOTAL	+2,397

   The core printf engine grew only +92 (1131->1223) - not a major contributor, contrary to the usual assumption. The real growth is the ls -l time chain (+910) and getopt_long (+380, net of the removed old getopt), both individually config-selectable.

   libdrivers.a +1,391

   Essentially just serial vectoring + termios. Full partition, sums to +1,391.

   group	net	reason
   serial (serial.o +1,413, serial_io +250)	+1,663	uart_read/write -> uart_readv/writev(struct uio*); adds baud table, tcdrain/tcflush/cfsetspeed
   syslog family (net)	-180	reorganized into fewer, larger objects (syslog_device/putc/stream removed -> syslog_write+syslog_channel) - net shrink, not ~0
   other objects (net)	-92	dev_null etc. removed
   TOTAL	+1,391

   The driver-side growth is entirely serial, mirroring the VFS readv/writev move; vectoring and termios can't be cleanly split because both landed in serial.o. The syslog refactor and other objects actually net down 272 B.

   libmm.a +656

   Allocator core heavier, plus locking and mmap. Full partition, sums to +656.

   group	net	driver
   allocator core (malloc/free/realloc/shrink/init)	+778	delayed-free + heavier paths
   mm_lock	+88	concurrency (replaces umm_sem)
   mm_map	+76	mmap support
   umm_* user-heap wrappers	+61
   other objects (net)	-347	mm_addfreechunk/mm_size2ndx/umm_sem merged away
   TOTAL	+656

   The historical "delayed-free ~1,032 B" over-counts: measured net for delayed-free + lock is ~458 B. The allocator core growth is partly offset by small objects merged away (-347).

   Historical estimate vs measured net

   Features spanning multiple .a are netted across the whole image:

   Historical change (year)	est. B	measured net B	note
   Semaphores atomic fast/slow (2024)	2,839	+1,054	incl. new sem_rw rwlock
   posix_spawn file actions (2023)	2,474	+678	task_* setup chain
   Directories via file layer (opendir)	2,148	+736	directories-as-fds
   Full _assert / panic path	1,973	-59	assert.o relocated from arch (+690); stackdump is in arch (CONFIG_ARCH_STACKDUMP=n -> -1,320, per @arnavsharma990)
   All I/O via readv/writev (2024)	1,942	+588 (VFS)	serial vectoring bundled with termios, counted separately (+1,983)
   Growable fd table (2021)	1,911	+743	above the ~196 B stub upper bound @arnavsharma990 measured, It needs to be verified again.
   Heap delayed-free list (2023)	1,032	+458	incl. new mm_lock
   FS-tree read/write lock (2024)	749	+628	inside the inode_search rework
   Memory-mapping tables (2022)	279	+76	mm_map
   Sum of historical items	15,347		above the measured total growth of 13,401 B

   Takeaways:

   1. The historical items cannot be summed - their total (15,347 B) exceeds the entire measured growth (13,401 B); they overlap and were each measured under different scenarios.
   2. Each item measures smaller than its estimate - semaphores 2,839->1,054, spawn 2,474->678, opendir 2,148->736, FD table 1,911->743. Only the FS-tree rwlock (749->628) is close.
   3. _assert/panic barely grows in the final image (-59)

   libsched (+4.3 K) and libfs (+3.5 K) are the two largest movers and where a size effort has the most leverage; libapps/libc are mostly feature growth; libarch shrank. Reproduction data (maps, configs, member-level JSON) available - happy to drill further into any single library.

   👍React with 👍2Reacted by Xiang Xiao and Arnav Sharma
8. Swatantra-66 commented on Oct 13, 2026

   

   Swatantra-66

   on Oct 13, 2026

   Contributor

   More actions

   @acassis @jingfei195887 @arnavsharma990

   Regarding the forensic breakdown across subsystems—particularly libfs.a (+3,517 B) and libsched.a (+4,314 B)—and building on the DEFAULT_SMALL precedent in draft PR #20628:

   The directory layer in fs_dir.c accounts for ~2.1 KB of the growth. Routing directories through the file layer (open(O_DIRECTORY) -> file_readv -> dir_read) was introduced for POSIX descriptor conformance (dirfd, fdopendir, fchdir, /proc/self/fd). However, on memory-constrained targets like stm32f103-minimum:nsh and 16-bit AVR architectures, pulling in the entire descriptor apparatus just to support directory traversal imposes an unnecessary footprint penalty.

   Introducing CONFIG_FS_DIRENT_LITE (configured as default !DEFAULT_SMALL) provides the appropriate opt-out mechanism:

   1. Type Definition (<dirent.h>):
      Under CONFIG_FS_DIRENT_LITE, DIR wraps an internal handle/pointer to struct fs_dirent_s directly, rather than storing an integer file descriptor fd.
   2. File Layer Decoupling:
      opendir(), readdir(), and closedir() resolve the target inode directly and invoke the filesystem operations (inode->u.i_mops->readdir) or pseudo-root traversals. This completely bypasses:
      • The file descriptor table (fs_files.c)
      • The synthetic directory inode (g_dir_inode) and operations table (g_dir_fileops)
      • The struct uio vectored I/O path (file_readv)
      • Path reconstruction (fd_path / inode_getpath)
   3. Conformance Boundary:
      APIs that strictly depend on directory descriptors (dirfd(), fdopendir(), fchdir() on directory fds) are conditionally compiled out under this option.

   I can assist with the following:

   • Prototyping the conditional decoupling between libs/libc/dirent/ and fs/vfs/fs_dir.c.
   • Measuring the exact linked image delta (.text, .data, .bss) on stm32f103-minimum:nsh.
   • Running regression verification on sim:nsh across basic directory commands (ls, cd, nested path traversals).

   👍React with 👍1Reacted by Jing Fei
9. Swatantra-66

   added 3 commits that reference this issue on Oct 19, 2026

   fs/vfs: Add CONFIG_FS_DIRENT_LITE lightweight directory API

   ...

   04340c3

   fs/vfs: Add CONFIG_FS_DIRENT_LITE and CONFIG_DISABLE_DIRECTORY_OPERAT…

   ...

   e402d70

   fs/vfs: Add CONFIG_FS_DIRENT_LITE lightweight directory API

   ...

   2e5a36d
10. xiaoxiang781216

    linked a pull request that will close this issuelibs/libc: enable div64_const on 32bit targets #20996on Oct 26, 2026