Legal AI procurement has become an IT decision as much as a practice decision, and the two functions tend to evaluate different things. A practice group evaluates whether the output is good. An IT director has to evaluate whether the arrangement is survivable — and the questions that settle that are almost never covered in a demonstration.

What follows is the list worth sending before the second meeting. The value is not only in the answers; it is in which questions produce a pause.

Data and infrastructure

  • Where is client data processed — which countries, which providers, which sub-processors?
  • Is client data used for training, fine-tuning or evaluation, and is the exclusion contractual or merely policy?
  • What is the retention window, measured from what event, and who is able to extend it?
  • How is matter isolation implemented — in application code, or beneath it?
  • Is retrieval scoped per matter at index time, or filtered after the fact?
  • What happens to our data, our indexes and our logs on termination, and within what period?

The fifth question is the one that separates architectures. Filtering after retrieval means the boundary lives in the same code path as the feature work — which is where a change made under deadline can move it without anyone noticing.

Security and compliance

  • Which certifications does the vendor hold, and will they provide the full report rather than the badge?
  • Does the scope of that report cover the AI infrastructure, or only the surrounding SaaS platform?
  • What is the incident response process, and what is the contractual notification window?
  • Has there been third-party penetration testing of the inference path specifically?
  • What is logged about a model call — the full context supplied, or only the user action?
  • Could information from our matters surface in a response served to another customer, by any pathway?

The second is asked least and matters most: a report scoped to the web application tells you very little about the system that actually handles the documents.

Commercial and contractual

  • What is the limitation of liability, and how does it compare to the value of a single affected matter?
  • Are audit rights a right of inspection, or a right to receive a report the vendor commissioned?
  • What notice applies to a material change in data handling, and may we exit without penalty?
  • What does the vendor actually represent about privilege — in the contract, not the brochure?
  • What happens to our data and our terms on a change of control?
  • How is inference priced, what happens if usage doubles, and what is fixed for how long?

On the fourth: most vendors represent nothing at all about privilege, because no vendor is in a position to. That is a defensible answer. An answer that implies otherwise is the one to worry about.

How to run it

Send the list in writing and ask for written answers. Two things follow. You get a record you can put in front of a client's security team, and you find out which questions the vendor has never had to answer — which is itself the most useful output of the exercise.

Expect a competent cloud vendor to answer most of these well. The ones that consistently produce hedging are the isolation questions, the audit-rights question and the liability cap. Those are not oversights; they are the structural limits of the model. Knowing exactly where they sit is what lets a firm decide which work can go there and which cannot.