Why This Test Matters One SoC generation changed inference speed by 29%. We ran Gemma3 270M INT8 on four Galaxy A-series devices to find out where on-device LLM becomes practically usable. We tested gemma-3-270m-it-int8 via MediaPipe CPU backend on the Galaxy A16, A26, A36, and A56, measuring latency, token throughput, memory, and accuracy across 25 prompts. We also compared parallel (all 4 devices simultaneously) vs serial (each device independently, 2 runs) execution to ver

Core Technologies of Mobile AI Quantization and NPU Optimization In Part 2, we discussed our selection of Gemma-2B as the ideal Small Language Model (SLM) for our project and shared our experiences benchmarking CPU and GPU performance in a constrained smartphone environment. However, the initial tests revealed significant challenges: noticeable latency delays and out-of-memory errors. To run LLMs in real-time on a mobile device held in the palm of your hand—not on a data ce