July 7, 2026

How we keep AI honest in qualitative coding: grounding, provenance, and the suggested/confirmed state machine

A practical look at how OpenVerbatim separates AI suggestions from reviewed evidence through grounding checks, provenance, and explicit states.

AI can make qualitative coding feel dramatically faster. A long transcript that once required an uninterrupted afternoon can produce candidate codes in minutes. A team can see possible patterns earlier, compare interpretations sooner, and spend less time staring at the first blank codebook. That speed is useful. It is also dangerous if the tool treats fluent output as if it were already analysis.

OpenVerbatim is designed around a simple rule: AI output is suggested until a reviewer action or explicit project policy changes its state. That rule sounds small, but it reshapes the product. It affects how suggestions are created, how quotes are checked, how reviewers make decisions, how themes are built, and how later answers cite evidence. The goal is not to make AI timid. The goal is to make AI useful without letting it smuggle unreviewed claims into the research record.

This post explains the three ideas behind that design: grounding, provenance, and the suggested/confirmed state machine.

Grounding: every suggestion needs evidence

In qualitative research, a code is not only a label. It is a claim that a piece of source material expresses something analytically meaningful. If the tool proposes a code without a supporting quote, the reviewer has to hunt through the transcript to decide whether the suggestion is real. That slows the workflow and encourages superficial acceptance.

Grounding means a coding suggestion should arrive with the evidence that supports it. The reviewer should see the relevant quote, the source location, and a rationale for why the code applies. This does not make the suggestion automatically correct. A quote can be too broad, too narrow, or interpreted in the wrong direction. But grounding gives the reviewer something concrete to inspect.

Grounding also helps reject fabricated or poorly localized suggestions. If a system cannot point to the material it claims to analyze, the product should not treat that output as ready for review. A grounded workflow makes the assistant accountable to the transcript. The assistant may propose, but the source material remains the reference point.

Provenance: the path matters

Provenance is the record of how a piece of analytic work came to be. Who or what created the suggestion? What quote did it reference? Was it accepted, edited, or rejected? If it was edited, what became the final wording? If a theme includes that code, can the team still trace the path back to the original source?

Without provenance, qualitative analysis becomes hard to audit. A final report may contain accurate findings, but the team cannot easily explain how those findings moved from raw material to interpretation. That is a problem for peer review, client review, internal research governance, and simple team memory. Research projects often last longer than the active attention of any single person. The tool should preserve the trail.

OpenVerbatim pairs current working state with append-only provenance events. In plain language, the interface can show the latest status of a suggestion while the system also records the state changes that led there. That combination keeps the product usable day to day without discarding the history needed for later review.

The suggested/confirmed boundary

The most important boundary is between suggested and confirmed. Suggested means the system has produced a candidate analytic action. Confirmed means the project has accepted it under the team’s rules. The distinction is important because AI suggestions can be useful before they are trustworthy. A suggestion can reveal a pattern, save a reviewer time, or prompt a better code name. It still should not be treated as final evidence until it crosses a clear boundary.

The boundary also protects downstream features. Suppose a tool lets an assistant produce codes and then immediately asks questions over those codes. The answers may sound polished, but they may be built on unreviewed material. OpenVerbatim’s design encourages later work to depend on confirmed evidence where appropriate. Theme clustering and ask-your-data workflows become more defensible when their inputs have passed through review.

This does not require every project to move slowly. Teams can define autonomy levels. A conservative study may require manual confirmation for every suggestion. A faster exploratory pass may allow some suggestions to move under explicit policy while routing uncertain or new material to a reviewer. What matters is that the state transition is visible and recorded.

Accept, edit, reject is not a minor UI detail

The A/E/R review loop matters because correction is the normal work of qualitative analysis. Accept means the suggestion is good enough to become part of the reviewed dataset. Edit means the system found something useful, but the researcher needs to refine the label, span, rationale, or wording. Reject means the suggestion should not be used as evidence for that interpretation.

Those actions should be fast. If reviewing a suggestion takes too many clicks, researchers will either avoid AI assistance or accept weak work because correction is tedious. The interface should make careful review efficient. OpenVerbatim’s three-pane workspace is designed around this principle: sources, transcript context, and the review/codebook pane stay close enough that a reviewer can make decisions without losing orientation.

The review loop also creates training value for the team. Rejections and edits reveal where the assistant overreaches, where the codebook is unclear, and where the study needs sharper definitions. A good AI workflow does not hide disagreement. It makes disagreement actionable.

Honest AI is constrained AI

There is a temptation to measure AI coding by how much it can do automatically. More automation can be valuable, but only if the constraints are explicit. In research, an unconstrained assistant is not honest simply because it sounds careful. It is honest when the product limits what the assistant is allowed to finalize, exposes its evidence, and records how decisions change state.

That is why OpenVerbatim treats the assistant as a proposer, not an unbounded authority. The system can generate suggestions and help organize work, but confirmation is a state change governed by review or policy. This is the practical meaning of keeping AI honest: the product should preserve the difference between “the assistant said this” and “the research project accepts this as evidence.”

The same principle applies to answers. Ask-your-data features are compelling because they let researchers interrogate a dataset conversationally. But an answer should be anchored to reviewed evidence and provide citations back to source material. Otherwise, it becomes a persuasive paragraph without a research trail.

What teams should test

When evaluating an AI coding tool, test the correction path, not just the first output. Use a transcript excerpt where a participant changes their mind, uses a metaphor, or describes a process with conflicting emotions. Ask the tool to propose codes. Then inspect whether each suggestion includes a quote, whether the quote actually supports the code, and whether editing the suggestion preserves provenance.

Next, test downstream behavior. After accepting some suggestions and rejecting others, ask a question that should depend only on reviewed material. Does the tool cite the accepted evidence? Does it avoid rejected material? Can you jump from the answer to the source? These are the moments where a product reveals whether honesty is built into the workflow or merely promised in marketing copy.

AI will not remove the interpretive responsibility of qualitative research. It can, however, make the early analytic pass faster and more systematic. The condition is that the product keeps evidence, state, and provenance visible. OpenVerbatim’s design starts from that condition because research teams deserve speed that can still be checked.

Try the evidence loop

Review the workflow before you commit your own data.

OpenVerbatim's public sandbox runs in the browser with generated demo material, so researchers can inspect the review loop without creating an account.