State of the Art Models Exhibit Regression in Schema Enforcement for Tool Calls
Developer Armin Ronacher reported a performance regression in tool-calling reliability when testing advanced Anthropic models, specifically Claude Opus 4.8 and Sonnet 5, within the Pi application. While older Anthropic models successfully adhered to nested tool schemas, these newer models frequently generated non-existent arguments inside nested edit arrays, violating strict schema constraints. This behavior highlights ongoing challenges in how frontier models serialize structured inputs and adhere to developer-defined constraints, pointing to a potential need for sampler-level grammar-aware decoding rather than relying purely on model reasoning. (source: https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/)