Structured Output Formats Like JSON Reduce LLM Response Diversity

A study tested 44 language models and found that requesting structured output formats like JSON or XML significantly narrows response diversity. - When asked to 'say a random word,' the most common response was 'serendipity,' occurring in 41% of normal interactions. - Forcing JSON output increased the dominance of 'serendipity' to 64%. - The effect is attributed to post-training on structured output data, which biases models toward a 'standard answer' and reduces variability.
Key insights
Structured Output Formats Act as a Diversity Suppressor in LLMs
The research across 44 models demonstrates that JSON and XML format requirements steer models toward a narrow set of high-probability tokens. In the random-word task, the top answer ‘serendipity’ jumped from 41% to 64% share when JSON was used. This suggests that fine-tuning for structured outputs introduces a convergence bias that undermines generative diversity, a critical consideration for creative or brainstorming applications.