The premise of first-generation legal AI was straightforward: take a strong general model, tune it on legal text, and put it behind everything. Given what was available in 2022 that was the right call. Held on to in 2026, it is a liability — and not mainly for the reason vendors debate.

Legal work is not one task

Extracting defined terms from an eight-hundred-page bundle, reasoning about a jurisdiction argument across three legal systems, drafting to a house style, and classifying documents for privilege are four different problems. They reward different things: recall and structure in the first, chain-of-reasoning depth in the second, register and consistency in the third, precision and a low false-negative rate in the fourth.

A single model chosen for the hardest of those is being overpaid for the easiest, and a single model chosen for cost will fail the hardest quietly. Neither failure announces itself.

What routing actually decides

Capability is only the first of four things a routing decision settles, and in practice it is rarely the one that matters most.

  • Capability: which architecture is actually good at this class of task
  • Cost: what the task is worth, since inference is now an operating line rather than a rounding error
  • Residency: whether this task may leave the perimeter at all, for this client and this jurisdiction
  • Policy: whether the firm has approved this model for this practice area and this sensitivity

The last two are not optimisations. They are the difference between a system a firm can put in front of a client's security team and one it cannot.

Model selection is not a deployment decision you make once. It is a runtime decision the platform should make on every task.

Governance is the real argument

Once routing exists as a layer, it becomes the place where firm policy is expressed. A practice group can be restricted to local inference. A client mandate prohibiting a named provider can be enforced rather than remembered. A matter marked sensitive can be barred from any external call regardless of what a user asks for.

And because the decision is made in one place, it can be recorded in one place. Every inference writes which model version produced which output, on which matter — the provenance a supervising partner needs under Rules 5.1 and 5.3, and the record a regulator will ask for when the question of who decided what finally gets asked.

What it demands from infrastructure

Orchestration of this kind is not a wrapper around an API key. It needs multiple endpoints under management, task classification good enough to route on, context that survives crossing a model boundary, a policy engine that binds before dispatch, and isolation guarantees that hold identically whichever model ends up serving the request.

That is a real system, and it is why the capability tends to appear where the firm controls the whole stack. A vendor whose commercial model is one model behind one endpoint has no reason to build it and every reason to argue it is unnecessary.

The lock-in question

There is a second-order benefit worth stating plainly. A firm that routes has already done the work of not depending on any single provider's price, availability or terms. A firm that standardised on one endpoint has to redo its integration to change its mind — which is why, when the terms change, it usually does not.