vram-optimization

Cross-vendor 3D Gaussian Splatting trainer - video to splat to mesh, Vulkan or CUDA.

Nodes to run Hunyuan Image 3 locally with BF16 and NF4 quantized options in Comfyui

MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert caching, LRU hot cache, OpenAI-compatible proxy, GGUF multi-shard.

Stateless LLM runtime that dynamically routes, loads, executes, and unloads models per request with bounded VRAM caching and intelligent model selection.

ComfyUI custom node that controls the order of node execution with linear routing of any data type through infinite I/O slots + option to free VRAM & RAM at any point in a workflow with device-agnostic memory management utilities managed by ComfyUI that safely unload all models, while preserving all connected data & models through to the next node.

The Unified Latent-State Memory Fabric (UL-SMF) is a hardware-software co-designed memory compression fabric that solves the memory bottleneck in long-context Transformer inference. By combining FSQ with dynamic 16-dimensional latent mapping, UL-SMF compresses Key-Value (KV) cache tensors by up to 384x while maintaining >94% semantic retention.

MSVC+CUDA llama.cpp fork for Windows: KDA/GDN, long-context KV placement, TurboQuant/TCQ when measured, speculative draft. Lab defaults — TriAttention opt-in only, not recommended.

KeSSie HUGE Context Semantic recall for Large Language Models

BoneMemory: Universal Async Core for AI-Toolkit. The hardware-agnostic VRAM manager for every model: Image, Video, and Audio. Whether you’re training a Rank 16 LoRA on an entry-level GPU or pushing Rank 1024 on an RTX 4090, BoneMemory eliminates memory bottlenecks. Architecture that makes any hardware punch above its weight. Zero OOM.

Constant-memory sequence modeling engine combining selective holographic-compression (ASH-C) with a coordinate pointer network (HEP-DNA). Bypasses the linear KV Cache bottleneck on consumer GPUs.

Lightweight LOWVRAM optimised KREA2 webui

Predictive VRAM Virtualization Engine

Sticky-block topology lottery scheduler for transformer fine-tuning.

LEMA (Layer-wise Efficient Memory Abstraction): A hardware-aware framework for fine-tuning LLMs in VRAM-constrained environments using asynchronous binary pre-fetching and triple-tier memory orchestration.

Private, autonomous AI lab — turn your PC into a local-first intelligence platform without melting your GPU. Python + FastAPI + local LLMs.

Perkunas AI Training Platform is a memory-aware model training and serving system for serious language model experimentation under tight hardware limits. It combines streaming training, rich telemetry, guarded recovery, checkpoint export, and OpenAI-compatible serving.

MiniMax H3 (Hailuo 3.0) front-end for ComfyUI — text-to-video, image-to-video, first-and-last-frame and reference-to-video with native audio. A local LLM writes H3's structured prompts and self-corrects them. Character consistency from a subject photo, live preview, VRAM handoff.

Ultra-Low Bit KV-Cache Compression optimization layer built on top of llama.cpp for LLM inference. Reduces VRAM overhead by ~75-80% using custom CUDA kernels.

Reducing reserved memory on NVIDIA GPUs

NMOS (Neural Memory OS) is a predictive partial execution engine enabling 70B-level reasoning on 4GB VRAM. It uses the “Zero-Lag” hypothesis, leveraging typing latency as a compute window to mask memory limits via async layer prefetching and speculative decoding.