The Data Path Ledger: A Startup Method for Answering AI Security Questionnaires

A 15-person startup closes its first enterprise pilot. Two days later procurement sends a spreadsheet with 140 rows. Row 61 asks whether customer data is used to train or fine-tune any model. The account executive pastes the question into Slack. The CTO replies from memory: no, we call an API. The row gets filled in, the pilot moves forward, and nobody thinks about it again.

Four months later the customer’s security team sends a follow-up. Do your application logs contain customer prompts? Who at your tracing vendor can read them? How long are they kept? The honest answers are yes, nobody has checked, and whatever the vendor shipped as the default. The original reply was not exactly false. It was incomplete, and it now sits in a signed document.

This article is for founders, CTOs, and whoever ends up owning sales engineering at a small AI company. It explains why questionnaire answers drift away from reality, then describes a lightweight method for keeping them accurate without hiring a compliance team. The scenario above is a composite, not a specific company.

Why the answers drift

Questionnaire answers are written by whoever is closest to the deal, about a system engineering keeps changing. Two things make this worse for AI features than for ordinary SaaS.

First, an

AI feature moves customer content through more places than the product diagram shows.

The prompt is assembled from user input and retrieved documents. It goes to a model provider. The response is stored. Traces and logs capture both sides for debugging. Embeddings land in a vector index. A sample of real conversations ends up in an evaluation set. Someone exports a failing session into a support ticket. Each of these hops has its own retention, its own readers, and sometimes its own vendor.

Second,

The questions have changed.

Buyers now ask whether the vendor trains on customer data, which LLM sub-processors sit in the call chain, whether zero-data-retention agreements exist with those providers, and whether customers can opt out of AI features. Older questionnaires covered encryption, access control, and breach notification. The AI questions probe exactly the places a small team has not looked.

What a wrong answer costs

There are three costs, in the order they tend to appear.

Delay.

An answer written from memory produces follow-up questions, and every follow-up pauses the deal. Security review is often the longest stretch between a verbal yes and a signature. A team that cannot answer cleanly spends it in Slack threads.

Inconsistency.

Different deals get different answers to the same question because different people wrote them. Larger customers have their own legal and security teams, and an answer that changed between two contracts is harder to explain than an uncomfortable answer that stayed the same.

Liability.

Questionnaire responses are often attached to the agreement or referenced by it. A present-tense statement that turns out to be wrong can become a misrepresentation problem. I am not a lawyer, and the exact contract language matters, so have counsel review how your responses are incorporated. The practical rule holds either way: write only what you can show.

The data path ledger

The method has one artifact: a short file kept next to the code, with one entry for every place customer content can go. Count hops, not systems. A single observability vendor might appear twice if it receives both traces and error reports.

• hop: tracing operator: content: full prompts and completions retention_days: 30 readers: two named engineers, vendor support on request trained_on: false deletion: tenant-level purge via API evidence: retention setting in config repo, DPA section 4 verified_on: 2026-10-01

The last two fields carry the discipline. Evidence means something you could hand over: a settings page, a clause in a signed agreement, a retention value in version control. The verified date means someone looked recently, not that someone believed it once.

A first pass usually covers these hops: client input, your API layer, prompt assembly, model provider, stored responses, logs and traces, vector index, evaluation and debug datasets, support and analytics tools, and backups. Most teams find one or two entries they did not know existed. That finding is the main return on an afternoon of work.

Tier every claim before it goes in the spreadsheet

The ledger holds facts. A questionnaire needs sentences. Before a sentence goes in, sort it into one of three tiers.

Proven:

You can show it today, through a configuration, a log, or a report.

Contracted:

A signed term with a vendor or customer covers it, and you can point to the clause.

Intended:

It is policy or plan, but nothing enforces it yet.

Proven and Contracted claims can be stated in the present tense. Intended claims get a date or stay out of the document.

Here is how the first sentence in the Row 61 story looks after tiering.

Original: “We do not train on customer data.”

Rewritten: “We do not use customer content to train or fine-tune models. Our model provider’s API terms (version dated [date], clause [reference]) exclude API inputs from training. Prompts and completions are retained in our tracing tool for 30 days and are not used for training.”

A tempting extra sentence is “Evaluation sets are built only from content customers have opted in to share.” That one fails the test unless an opt-in mechanism exists and the ledger entry for evaluation data points to it. If it does not, the sentence is Intended, and it either gets a date or gets cut.

The same logic covers the SOC 2 question that appears in most enterprise questionnaires. SOC 2 is an attestation issued by an independent auditor, not a certification, and a Type 2 report covers an observation period typically running three to twelve months. If you do not have a report, say so plainly. If an audit window has genuinely started, state the start date. Then offer the ledger and a call with the engineer who maintains it. For a small company, a precise description of the data path often settles the reviewer’s concern faster than a badge would.

Keeping answers true after the first deal

Store approved answers in an answer library, each one linked to the ledger entries it depends on, with a verified date. Two triggers should send an answer back for re-verification. One is the calendar, quarterly is enough. The other is change: a new vendor, a new log destination, a new model provider, a new data store.

The second trigger is the one worth wiring into engineering. Add a single line to the pull request template: does this change where customer content goes? If yes, the ledger file changes in the same pull request. The cost is a few seconds per change, and drift gets caught at its source rather than during a customer’s audit.

The ledger also makes deletion questions answerable. When a customer asks how their data is removed if they leave, the ledger shows the deletion path hop by hop. The usual gap is that the vector index and the evaluation sets have no purge by tenant, because nobody designed one. Finding that while building the ledger is far cheaper than finding it in a customer’s review. The same file also answers most of the data-handling items an investor’s technical reviewer sends, which I covered in Technical Due Diligence When an AI Assistant Wrote Part of Your Codebase.

When the honest answer is unwelcome

Sometimes the ledger shows something you would prefer not to write down. Write it down anyway, with three parts: the gap, the interim control, and the date.

“Prompts are retained in our tracing tool for 30 days. We are moving to metadata-only tracing by [date]. Until then, access is limited to two named engineers.”

A specific gap with a control and a date gives a reviewer something to approve. A vague assurance gives them something to question. In my experience of how founders handle these reviews, the second path produces longer threads and more distrust than the first.

What the questionnaire is really asking

A security questionnaire is often the first time a startup describes itself in writing to someone with the right to audit that description. Recall produces answers that were true on the day someone remembered them. A ledger produces answers that are tied to evidence and a date. For a small team selling into larger companies, that difference tends to matter more than the length of the compliance list behind it.