Memory Magician: Run Huge MoE Models on Your Laptop

Poster discovered that sequentially offloading Mixture-of-Experts (MoE) experts to RAM/SSD is inefficient because expert activations are non-uniform; they propose tracking hot experts and dynamically moving them to VRAM to improve speed and enable larger models on limited hardware, and seek help implementing this in llama.cpp due to limited C/C++ skills.