As organizations move from generative AI prototypes to production-grade systems, AI agent scalability is increasingly constrained by reliability and maintainability rather than raw model capability. Large language models are inherently probabilistic, forcing developers to embed retries, fallbacks, and branching logic directly into agent code. Over time, this entanglement of business logic and inference strategy creates brittle systems that are hard to test, optimize, and govern.
New research from Asari AI, MIT CSAIL, and California Institute of Technology proposes a different architectural approach. The team introduces Probabilistic Angelic Nondeterminism (PAN) and a Python framework called ENCOMPASS, which separates what an AI agent does from how uncertainty is handled at inference time.
Instead, developers write a clean “happy path” and mark only uncertain points with a branchpoint() primitive. At runtime, ENCOMPASS explores these points using search strategies without altering the workflow code. This design keeps agents “program-in-control,” a model preferred in enterprise environments for predictability, auditability, and governance.
The research demonstrates clear benefits in complex tasks such as legacy code migration. For example, in a Java-to-Python task, ENCOMPASS’s beam search outperformed simpler sampling while remaining maintainable. Importantly, smarter search scaled better with inference cost than adding more refinement loops.
From a cost and governance perspective, this separation matters. Inference strategy becomes a tunable runtime concern rather than an application rewrite, allowing teams to balance accuracy, latency, and compute spend per use case. It also simplifies oversight: problematic behaviors can correct by adjusting search policies globally rather than modifying each agent individually.
Overall, the research signals a shift toward more modular, software-engineering–driven approaches to AI agent design. As agentic systems become core to enterprise operations, separating logic from search may prove essential for scaling reliably, efficiently, and sustainably.
Source:
