XNews

Open-source model advances: DeepSeek, Qwen and Mistral

Open-source model advances: DeepSeek, Qwen and Mistral

Open-source model projects announced major releases and efficiency advances. - DeepSeek-V3.2 introduced sparse attention (DSA), expert distillation and reinforcement learning to match or exceed closed-source reasoning performance while reducing compute costs. - DeepSeek claims top-tier math and programming performance, parity with Gemini 3 Pro on some benchmarks, and IMO/IOI gold-level results. - Alibaba released Qwen3 Next variants (80B FP8) with transformer compatibility and broad platform availability; Qwen3-VL-235B leads OpenRouter image processing with 48% share. - Mistral released Devstral Small 2 at 24B parameters, runnable on local notebooks while maintaining competitive coding performance and strong privacy properties. - Performance trends show medium-sized models (15B–70B) are favored for cost/latency balance; specialized sparse attention and distilled experts enable smaller models to approach larger-model capabilities.

Key insights

DeepSeek-V3.2 claims architecture and training advances to rival closed models

DeepSeek-V3.2 introduced a DSA sparse attention mechanism and a training regimen combining specialized expert models with knowledge distillation and large-scale RL to transfer domain expertise into a general model. The paper reports parity or superiority on several reasoning and agent benchmarks, IMO/IOI-level math performance, and reduced inference cost through sparse attention. The approach emphasizes focusing compute on salient tokens for long-context tasks and using synthetic expert data to teach deep reasoning.

Qwen releases FP8 80B models and dominates image processing on OpenRouter

Qwen announced Qwen3-Next 80B variants with FP8 inference for faster performance and compatibility with Transformers, vLLM, and SGLang. Qwen3-VL-235B reached a reported 48% market share on OpenRouter for image processing workloads, signalling strong adoption in multimodal tasks and community support.

Mistral targets local privacy with a capable small model

Mistral published Devstral Small 2, a 24B-parameter model designed to run locally on notebooks while offering competitive code-generation performance relative to much larger models. The release signals a push toward privacy-friendly deployments and reduced reliance on large remote inference stacks.

Open-source trend: medium-sized models optimize cost-performance trade-offs

Multiple reports indicate a market preference for 15B–70B parameter models that balance latency, cost, and capability—small models are being deprioritized for being underpowered, and extremely large models face cost/latency constraints. Architecture innovations like sparse attention and distilled expert training help medium models approach larger-model capabilities.

Source posts