Perplexity open-sources Lily, a custom local inference engine for Apple Silicon

Perplexity open-sourced Lily, a local inference engine built for its Mac Hybrid Compute feature. Lily uses a Rust runtime with custom Metal kernels, deeply optimized exclusively for the Qwen3.6-35B-A3B model on Apple Silicon, without depending on PyTorch or MLX. On M5 Max, prefill throughput averages 1.23x MLX-LM and decode 1.35x. Optimization strategies include GPU path matching to inference phases, minimizing data movement by decompressing weights on-chip and keeping routing on GPU, and selecting kernel configurations based on measured workload shapes. The engine achieves 97.9% and 90.3% of maximum sustained weight read bandwidth for MoE GEMM and GEMV respectively. Speculative decoding proved 18% slower on this hardware.