LATEST TECH ARTICLES

Core Technologies of Mobile AI Quantization and NPU Optimization In Part 2, we discussed our selection of Gemma-2B as the ideal Small Language Model (SLM) for our project and shared our experiences benchmarking CPU and GPU performance in a constrained smartphone environment. However, the initial tests revealed significant challenges: noticeable latency delays and out-of-memory errors. To run LLMs in real-time on a mobile device held in the palm of your hand—not on a data ce