Do GPTs Dream of Routing Tests?

 A “routing test” in ChatGPT is an experiment that sends the same query through multiple models, prompts, or tool paths, measures quality, latency, and cost, and learns the optimal route. However, it often becomes opaque; accountability, reproducibility, and consent-by-design are the key concerns.

For chat services, GPT functions like a CPU: a core computational unit that processes natural language and reasoning. Performance improvements matter, but on their own they do not create a full user experience. The real value lies in the surrounding “OS-like layers,” which include routing, safety guardrails, UI/UX design, and integration with external APIs or search. These correspond to schedulers, memory management, interfaces, and drivers in traditional computing, and they form the key points of differentiation among services. Since GPT-3.5, competition has often been framed as if the model itself were the essence, but in reality overall system design defines the quality of the experience. OpenAI’s routing mechanism, for example, dynamically switches models depending on conversation context to ensure safety and appropriateness—a prime example of OS-level design beyond CPU performance. The competition is shifting from pure model benchmarks toward holistic design with transparency and reproducibility, much like how Apple and Google competed for dominance with smartphone operating systems. In AI as well, the axis is moving from CPU-style performance battles to OS-style ecosystem competition, making the user’s choice of platform increasingly decisive.