Multidirectional Resonant Limiting (MRL): In-place <=5% path-gauge GGUF merging of Qwen2.5-1.5B & Qwen2.5-Coder-1.5B in Termux (0.0000% super-block loss)

[slinkzyy]

Quick thank-you and small gift back to the open-source community:

• Repo & Scripts: https://github.com/slinkzyy/qwen2.5-mrl-path-hybrid

Credits & Thanks

• Qwen Team (QwenLM/Qwen2.5 & QwenLM/Qwen2.5-Coder) for the base and coder models.
• Termux (termux/termux-app) for the Android Linux environment where the analysis, mmap weight surgery, and benchmarks were run.
• Aarstad/agy-termux-musl and wallentx/antigravity-cli-termux for the VA39/faccessat2 patches and ~80 MB musl loader that made running google-antigravity/antigravity-cli (Gemini 4 Argon) natively on Android possible.
• ggml-org/llama.cpp and Ollama for the Q4_K/Q6_K GGUF super-block format and local inference runtime.

Technical Summary

When merging qwen2.5:1.5b and qwen2.5-coder:1.5b (separated by 5.5T code pretraining tokens), naive linear/row-level merges either trigger repetition loops or blur multi-step arithmetic (gcd(1071, 462) = 21).

1. End-to-End Path Gauge (0.3% -> 90.05%): Comparing isolated matrices shows only 0.0%–0.3% agreement within 5% because weight decay during Coder pretraining shifted output projections (ffn_down, attn_output) to ~0.73x and RMSNorm weights to ~1.25x. Tracing the composite path (norm -> gate/up -> down and norm -> V -> O) and normalizing by the layer median gauge reveals that 90.05% of the 1,536 residual directions and 49.17% of the 8,960 SwiGLU neuron pathways still agree within <= 5%.
2. Multidirectional Resonant Limiting (MRL): Rather than rescaling Frobenius norms (which over-amplifies the principal singular value sigma_max by +8%–10% per matrix / +30% per layer and causes repetition loops), MRL only steps individual coordinates (i, d) that sit on <= 5% compatible paths, agree in sign, and match within <= 5% (+41.4% SNR gain on invariant paths, leaving > 5% specialist weights untouched).
3. Zero-Loss In-Place GGUF Super-Block Surgery (0.0000% error on untouched weights): Re-quantizing Q4_K -> FP32 -> Q4_K from scratch via min-max introduces a 4.38% L2 error across 85.2% of blocks. Preserving llama.cpp's original Q4_K (144B) and Q6_K (210B) block scale headers (d, dmin, scales) via mmap and updating only matched 4-bit/6-bit bins keeps all untouched coordinates and super-blocks 100% bit-exact (0.0000% loss), scoring 15/16 (93.8%, +15.4% combined improvement over the parent models, 10.3 tok/s on ARM64).

All Python scripts (FastGGUF zero-copy parser, MRL merger, streaming benchmark suite) and layer-by-layer JSON metrics are MIT-licensed in the repo above.