Security questionnaire template for AI coding tool vendors

Send this to an AI coding tool vendor, or fill it in for your own product. Each question has a note on what a good answer looks like: delete the notes before sending. Replace [brackets] and delete this note.

• Vendor and product: [name]
• Product version or plan assessed: [plan]
• Completed by: [name, role, email]
• Date: [YYYY-MM-DD]

1. The product

• 1. What does the product do with code? [Answer] Good answer: names each capability, such as completion, chat, review, or agents that edit files, run commands or open pull requests.
• 2. Can it act without a person approving each step? Which actions? [Answer] Good answer: a list of actions, and which of them the customer can switch off or put behind an approval.
• 3. Where does it run? [Answer] Good answer: says which parts run on the user's machine, in the vendor's cloud, or in the customer's own infrastructure.

2. Data flow

• 4. What customer data leaves our environment, and where does it go? [Answer] Good answer: a data-flow description listing code, prompts, file contents, logs and metadata, and every destination.
• 5. Which model providers and other subprocessors receive our data? In which regions? [Answer] Good answer: a named list with regions, and how customers are told about changes.
• 6. Is our data used to train or improve any model, yours or a provider's? [Answer] Good answer: a clear no that covers every provider, backed by contract terms or a setting enforced for the whole organisation.
• 7. How long are prompts, code and outputs kept, by you and by each provider? [Answer] Good answer: a retention period per data type, including logs and backups, and whether zero retention is available.
• 8. Can we export and delete our data? How quickly is deletion completed? [Answer]

3. Access and identity

• 9. Which sign-in methods are supported: SSO, SAML, OIDC, MFA? [Answer]
• 10. Which roles exist, and can we limit who approves or deploys AI changes? [Answer]
• 11. What access does the product need to our repositories? Can it be limited to some repositories and to read-only? [Answer] Good answer: per-repository installation, the exact permission scopes, and no standing write access to protected branches.
• 12. How are tokens and secrets we give the product stored? [Answer] Good answer: encrypted at rest, never shown again after saving, never sent to a model.

4. Agent safety

• 13. Where do agents run commands, and what can that environment reach? [Answer] Good answer: an isolated sandbox per task or project, no access to other customers, and network limits.
• 14. Can an agent read secrets from our environment or CI? [Answer]
• 15. How does the product defend against prompt injection from issues, comments, files and web pages? [Answer] Good answer: says what an injected instruction could achieve at worst, and which controls limit it. "Our model is trained to ignore it" is not enough on its own.
• 16. Can an agent change CI, infrastructure or authentication code? Can we stop that? [Answer]
• 17. Can an agent merge or deploy without a human approval? [Answer] Good answer: no, or only where the customer has explicitly allowed it.

5. Quality and review

• 18. How are tests run on AI changes, and does a test result apply to the exact version that merges? [Answer]
• 19. What happens when a check cannot run, for example when tests fail to install? [Answer] Good answer: the change is held, not passed.
• 20. Can the product produce an SBOM and check licenses? [Answer]

6. Audit and evidence

• 21. What is logged: tasks, models used, approvals, commands, merges? [Answer]
• 22. How long are audit logs kept, and can we export them? [Answer]
• 23. Can you show, for one change, who asked for it, which model wrote it, who approved it and what was tested? [Answer]

7. Security programme

• 24. Which independent audits or certifications do you hold? Attach the reports or letters. [Answer] Good answer: the report itself or a bridge letter, with its period. A logo on a website is not evidence.
• 25. How do you test your own security: penetration tests, bug bounty, vulnerability disclosure? [Answer]
• 26. How and how fast will you tell us about an incident that affects our data? [Answer] Good answer: a time limit in the contract, a named contact, and what the notice will contain.
• 27. Where is your status page and your security contact? [Answer]

8. Legal

• 28. Do you offer a data processing agreement? [Answer]
• 29. Who owns the code the product writes for us? [Answer]
• 30. Do you indemnify customers against intellectual property claims over generated code? On which plans? [Answer]

Assessment

Decision: [approved / approved with conditions / rejected], by [name], on [YYYY-MM-DD].