AI vendor discussions are full of confident claims (“secure,” “enterprise-ready,” “no training on your data”). Boards should treat AI procurement like any other critical dependency: verify what matters, contract it, and monitor it.
1) Data Handling (The Non-Negotiables)
Ask the vendor:
- What data is processed (inputs, outputs, logs, files, tool results)?
- Where is it processed and stored (regions, subprocessors)?
- What is the retention policy by plan/tier?
- Is customer content used for training by default, opt-in, or never?
- Can you enforce data minimization and redaction controls?
If your AI workflows touch personal data, your organization still bears GDPR accountability—even if the model is a vendor service (GDPR).
2) Security Controls (Not “Trust Me,” Show Me)
Ask for evidence of:
- access controls, MFA, admin separation
- vulnerability management and disclosure process
- audit logs (what happened, who did it, when)
- tenant isolation (for multi-tenant offerings)
- incident notification timelines
For LLM products, include LLM-specific risks like prompt injection, system prompt leakage, and excessive agency (OWASP LLM Top 10 2025).
3) Governance and Accountability
Boards should ask management to ensure each vendor relationship has:
- a named internal accountable owner
- an approved scope of use (“allowed use”)
- periodic risk review and re-approval process
- a documented exit plan (data export, deletion, replacement)
NIST AI RMF gives a practical way to structure this governance across the lifecycle (NIST AI RMF).
4) Transparency: What Will You Be Able to Prove?
Ask:
- Can we get logs and metrics suitable for audits?
- Can we reconstruct an incident (inputs, outputs, actions, sources) without guesswork?
- What changes without notice (models, safety policies, routing, retention)?
- How do you communicate changes and breaking behavior?
5) Cost and “Runaway Spend” Controls
For LLM and agentic systems, cost is a risk dimension:
- rate limits and quotas
- budget caps per team/system
- alerts on spikes and anomalous usage
- safeguards against “unbounded consumption” (an OWASP risk category)
6) Regulatory Alignment (Practical, Not Performative)
Ask:
- Do you support AI governance standards (e.g., ISO/IEC 42001 alignment)?
- If operating in the EU, how do you support EU AI Act obligations relevant to our role?
- Do you support customer AI literacy programs with role-based guidance (Article 4 expectations)?
Red Flags in AI Marketing
Boards should treat these as “slow down and verify” signals:
- “We’re compliant” with no scope definition or evidence
- “No training on your data” while retention/logging is unclear
- vague answers on subprocessors, regions, or incident response
- “It’s just a chatbot” while it has tool access or can trigger actions
- refusal to define accountability or provide auditability
I consult with boards on technology vendor risk and deliver board-level courses on cyber, AI, and regulations—including practical vendor evaluation patterns and templates. Contact me.
Relevant Sources
- AI Risk Management Framework (overview) — NIST — https://www.nist.gov/itl/ai-risk-management-framework
- AI RMF 1.0 (NIST AI 100-1) — NIST — https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10
- OWASP Top 10 for LLM Applications 2025 — OWASP — https://genai.owasp.org/resource/owasp-top-10-for-llm-applications-2025/
- ISO/IEC 42001 explained (AI management systems) — ISO — https://www.iso.org/cms/%20render/live/en/sites/isoorg/home/insights-news/resources/iso-42001-explained-what-it-is.html
- GDPR (Regulation (EU) 2016/679) — EUR-Lex — https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng
- EU AI Act Article 4: AI literacy — EU AI Act Service Desk — https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-4