XNews

Jev AI agent framework with structured state

Jev AI agent framework with structured state

TypeSafe AI released Jev, a new category of 'System One' AI model designed for fast, structured decisions inside software. Jev abandons text generation entirely, outputting type-safe results with calibrated probabilities. It responds in 70-500ms, costs $0.042 per million input tokens with free output, and claims 20-200x faster and 40-400x cheaper than comparable LLMs.

Key insights

No text generation means no hallucinations

Jev's architecture prevents hallucination by never generating free text. Output structures and values are predefined, making type errors mathematically impossible. Each output includes an epistemically honest probability. This addresses the RLHF-induced problem where models learn to discard uncertain patterns and fake confidence.

RLCD replaces RLHF for calibrated decisions

Founder Diogo Almeida, a ChatGPT and GPT-4 paper co-author, spent two years building Jev using a new training method called Reinforcement Learning for Calibrated Decisions (RLCD). The approach optimizes decision calibration rather than human-pleasing behavior, enabling models to state when they are uncertain.

Parallel sampling achieves radical speed improvement

Jev employs parallel sampling where all outputs are computed at once, unlike LLMs which generate tokens sequentially. This mirrors how Transformers replaced RNNs with parallelism. Response times range 70-500ms versus 3-329 seconds for mainstream frontier models. Workflow evals using GPT-6 Astra and Fable 5.1 as reference answers show 20-200x faster performance.

Smart if statements: target use cases

Jev targets intelligent branching in code: classification, routing, scoring, extraction, and guardrails. Use cases include customer service ticket routing, massive data processing (42 dollars per billion tokens with free output), real-time applications needing sub-100ms decisions, and as a rapid safety check layer for LLM outputs. Demo showed DOOM AI control at 10 calls/second (~$7/hour).

Jevons Paradox: name reflects economics

Jev is named after William Stanley Jevons, whose paradox holds that efficiency gains increase total consumption. The company bets that making AI decision costs nearly free will explode usage volume. They price output tokens at zero, claim profitability, and plan continued price reductions.

Open source course: learn SGLang from scratch

Datawhale and RadixArk launched a collaborative open-source course to build a mini-SGLang from zero. Part III (Attention backends, quantization, layered caching, PD separation) and Part IV (profiling, PR workflow) are authored by official SGLang members. The course targets converting learners into SGLang contributors, covering concepts from KV Cache to Continuous Batching.

SGLang course emphasizes coding ethics

The course includes a distinctive coding ethics chapter opposing AI slop contributions: contributors must understand why code works, profiling must come first, and performance PRs require before/after comparisons with reproducible commands. The project explicitly states it doesn't welcome open source resume-padding.

Source posts