The 35B-A3B model utilizes a clever architecture where the majority of experts reside on SSD storage, loading into active memory only when required. By activating roughly 3 billion parameters per token, the system maintains high-level performance across STEM problem solving and chart interpretation without the heavy footprint typically required for such intelligence. A smaller 1B version is also available, specifically optimized for entry-level hardware.
Performance gains are driven by Lookahead Expert Scheduling, which predicts necessary experts for upcoming layers while the current layer is still processing. This technique, paired with layer-wise quantization, enables speeds exceeding 17 tokens per second on an iPad Pro M4. Furthermore, developers can leverage visual token compression up to 16x to balance speed and detail, with the model retaining a score of 90.9 on MMBench. To ensure sustained reliability, ModelBest introduced a Self-Replay Fine-Tuning method, integrating high-quality model-generated responses into the training cycle to prevent skill degradation.




Comments (0)
No comments yet. Be the first!