Most defence AI vendor questions asked in a first meeting are about capability, and capability is the one thing a first meeting cannot establish. The questions below are about evidence, exposure and cost, which are things a supplier either can document in the room or cannot. Each is given with the reason it matters and the shape of a weak answer.
They draw on published procurement and risk guidance rather than on any proprietary method: the US Government Accountability Office's technology readiness assessment guide, the NIST AI Risk Management Framework, US Department of Defense Directive 3000.09 and the UK Ministry of Defence's published approach to AI-enabled capability. Where the question is about a supplier's public record, the method is set out in the verification guide, and the companies with documented operational use show what a strong answer looks like in practice.
Evidence of deployment
1. Which named public source reports a military using this system? This is the only question that separates a claim from a record. A weak answer names no publisher, no date and no outlet, and moves straight to confidentiality before any published source has been offered.
2. Who is the customer on your most recent announced contract? Announcements are verifiable as announcements; they are frequently not verifiable as procurements. A weak answer is "a European NATO member" with no accompanying detail on scope, quantity or configuration.
3. What was the last exercise or evaluation this system took part in, and who ran it? A named organiser and a dated event can be checked independently. A weak answer is "multiple international trials", which is unfalsifiable and therefore useless as evidence.
4. May we speak to a reference user in the same role as us? Reference users test whether the supplier's account survives contact with someone who lives with the product. A weak answer supplies a reseller, an investor or a partner presented as a user.
5. Where does the delivered configuration differ from the one you have demonstrated? Demonstration builds routinely carry sensors, compute or connectivity that the fielded article will not. A weak answer denies that any difference exists rather than itemising it.
The system itself
6. What data was the model trained on, and who owns it? Training data determines both behaviour and legal exposure, and ownership determines whether it can be reused. A weak answer is "proprietary datasets" with no description of provenance, licensing or volume.
7. How does performance degrade outside the training distribution, and how would we detect that in the field? Real deployments encounter conditions absent from training data. The NIST AI Risk Management Framework treats measurement and monitoring as core functions. A weak answer is a single headline accuracy figure.
8. Which function is automated: detection, classification, target selection, or the application of force? These are four different systems with four different review requirements, and the loop vocabulary conceals the difference, as the guide to in the loop, on the loop and out of the loop sets out. A weak answer restates a tier label.
9. Can the system be operated with no person selecting the specific object engaged, and can that mode be disabled and audited? Mode, not airframe, determines where a system sits against DoD Directive 3000.09's categories. A weak answer asserts that the mode does not exist without showing how it is prevented.
10. What is the intervention window in seconds, and what is the behaviour on loss of link? Supervision that cannot be exercised in time is supervision in name only. A weak answer gives no number, or gives one measured under laboratory latency.
11. What does the system log, who can read it, and can we export it? Logs are the only durable record of what was decided and by whom. A weak answer offers a vendor-hosted dashboard with no export and no retention commitment.
Security, export and legal exposure
12. Under which national export licensing regime does this fall, and has a licence been issued for our country? An unlicensable product is not a product. A weak answer treats licensing as an administrative formality to be resolved after signature.
13. Does any element approach a Missile Technology Control Regime Category I threshold or contain controlled foreign-origin content? Published MTCR guidance sets Category I at complete systems able to deliver 500 kg to 300 km, and Category I transfers carry a strong presumption of denial. A weak answer has never considered the question.
14. What legal or weapons review has the system undergone, and may we see the summary? Reviews are conducted against a stated concept of use, so the summary tells a buyer what use was assumed. A weak answer conflates a legal review with a compliance certificate.
15. What is your supply chain of record for compute, sensors and firmware, and where is each manufactured? Component origin drives both security assessment and export treatment. A weak answer stops at the first-tier supplier.
Commercial terms and sustainment
16. Who holds the intellectual property in models trained on our data? This is frequently the largest unpriced term in a defence AI contract. A weak answer defers it to a schedule that does not yet exist.
17. What happens to our data, models and source code if you are acquired or wound up? Escrow arrangements are cheap to agree at signature and impossible to obtain later. A weak answer cites the company's funding as a reason the question is unnecessary.
18. What is the software update path in a disconnected environment, and who signs the updates? A model that can only be updated over a commercial network is not deployable everywhere it is being sold. A weak answer assumes connectivity.
19. What is the five-year sustainment cost, itemised? Content authoring, model retraining, sensor calibration and licence renewal accumulate well past the headline price. A weak answer quotes a percentage of capital cost with no line items.
20. What acceptance test are you prepared to be measured against, and who writes it? A supplier confident in its product will help write the test that could fail it. A weak answer proposes that the vendor supplies both the test and the result.
What the answers are worth
None of these questions establishes capability, and a supplier that answers all twenty well may still deliver poorly. What they establish is whether a claim is documented, whether exposure has been thought about, and whether the commercial terms have been priced. That is the same standard the index applies when it assigns one of four evidence grades, and the reason a readiness level is not a substitute is set out in the readiness-level guide. A longer written checklist for autonomy suppliers specifically is in the due-diligence piece.
Limits of this reading
This list is generic. It is not tailored to any jurisdiction's procurement rules, and a buyer operating under a specific national framework should treat its own regulations as controlling. The guidance documents cited are US and UK; other frameworks exist and are not surveyed here.
A weak answer is not evidence of a weak product. Small suppliers frequently answer badly on commercial and export questions simply because they have not been asked before, and thin public evidence often reflects customer confidentiality rather than the absence of a customer. The questions are diagnostic prompts, not a scoring system, and nothing here constitutes an assessment of any named company.
Frequently asked questions
What should a buyer ask a defence AI supplier first?
Ask which named public source reports a military using the system. It is the only question that distinguishes a documented record from a claim, and the answer can be checked immediately without relying on anything the supplier says next.
How can a buyer test an AI vendor's autonomy claims?
Ask which function is automated: detection, classification, target selection or the application of force. Then ask whether a mode exists in which no person selects the specific object engaged, whether it can be disabled, and whether that setting is auditable.
What is the most commonly missed term in a defence AI contract?
Intellectual property in models trained on the customer's data, followed by escrow arrangements covering acquisition or insolvency. Both are inexpensive to settle before signature and effectively unobtainable afterwards.
Sources
- US Government Accountability Office, Technology Readiness Assessment Guide (GAO-20-48G), January 2020
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0)
- US Department of Defense, Directive 3000.09, Autonomy in Weapon Systems
- UK Government, Ambitious, safe, responsible: our approach to the delivery of AI-enabled capability in Defence
- US Department of State, Missile Technology Control Regime (MTCR) Frequently Asked Questions
- idf.ai, The Israeli Defense-AI Deployment Index v1.0, 24 August 2026
Independent publication of idf.ai. Not affiliated with, endorsed by, or connected to the Israel Defense Forces, the Israeli Ministry of Defense, or any government body. Compiled entirely from publicly published sources. No classified, restricted or non-public information. Listed companies may dispute any entry: send the published source that contradicts it and the entry will be amended or removed.