Analytics & Research · THE NO-PANIC PLAN
Audit evidence provenance before publishing a report
Use this workflow for a business report, research summary, public information page, or other evidence-based publication. It helps readers trace how a claim was supported and how derived results were produced; it does not decide whether a claim is true or replace specialist review. Higher-risk health, legal, financial, safety, and regulated claims need the evidence standard and reviewer required for that domain.
MISSION Create a traceable record connecting each material report claim to its source, the person or organization responsible for it, the extraction or calculation performed, and the review decision before publication.
Review the claim-to-source trail before releasing the reportTHE REAL-WORLD BIT
What happens outside this browser tab?
Set the publication scope and evidence owner; break the draft into individually checkable claims; record source identity, version, dates and responsible agents; test each source's directness, method, scope and limitations; map evidence to claims and record transformations or calculations; resolve contradictions and mark unsupported claims; then run an independent reproducibility and citation check, preserve the reviewed version, and schedule refreshes for time-sensitive evidence.
YOUR CHECKLIST, WITH FEWER DRAMATIC SIGHES
One step at a time.
Follow the order below. If a step names a Nirmion tool, its link is right there with it.
- 01
Define the publication, decision, and evidence owner
Record the report or page version, intended audience, decision it may influence, publication date, domains covered, and the person who owns the evidence file. Name the reviewer who can challenge source quality and the subject-matter owner for any regulated or technical claim. Set a review threshold before collecting links: which claims require a primary source, an independent confirmation, a calculation recheck, or specialist approval? A source register can make work traceable, but the publication owner remains responsible for the meaning and limits of each claim.
- 02
Break the draft into specific, checkable claims
List each factual statement that could affect a reader's understanding or decision, one claim per row. Include the exact draft wording, a stable claim ID, date or time period, population or product scope, unit, and whether the statement is quoted, measured, inferred, or calculated. Split compound sentences so one source cannot appear to support several unrelated facts. Mark opinions, recommendations, assumptions, forecasts, and definitions separately from observations, and flag claims whose wording implies more certainty, generality, or causal evidence than the draft intends.
- 03
Capture where every evidence item came from
For each source, record a unique source ID, publisher or author, title, canonical URL or repository identifier, publication or dataset version, date published, date accessed, and the exact page, table, figure, or record used. Record who collected or supplied the item and what activity produced any export, transcription, cleaning, or transformation. Retain a permitted copy, dataset snapshot, stable identifier, or retrieval note when a live page can change; do not treat a screenshot or a hash alone as proof that a source is authentic. The W3C PROV model's entity, activity, and agent concepts provide a useful structure for this record. Use Audit Evidence Index to organize the source trail.
- 04
Check whether each source actually supports its claim
Open the original source where possible and compare the cited passage or data with the exact claim. Record whether the source is primary or secondary, its methods and population, the period it covers, known limitations, and whether it is independent of the claimant. For marketing claims, FTC guidance says the needed substantiation depends on the claim and context; do not generalize that standard to every subject. For reporting, SPJ advises testing information from sources and using original sources when possible. If a source does not support the wording, narrow or remove the claim rather than adding a loosely related citation.
- 05
Link derived results to their inputs and transformations
For each table, percentage, summary, or calculated claim, record the input source IDs, selection rules, exclusions, units, formula or code version, and the steps that transform source material into the reported value. Preserve the relevant input snapshot and output where permission allows, and have another reviewer rerun a high-impact calculation from the same inputs. The National Academies distinguishes reproducibility using the same data, methods, and code from replicability using new data; describe which one the review actually attempted. Do not imply a reanalysis was independent if it used the same data and procedure.
- 06
Resolve gaps, conflicts, and changing sources explicitly
Compare all sources attached to a claim and record agreements, disagreements, missing evidence, and any differences in date, definition, method, or population. Seek clarification from the source owner or locate a more direct source; do not average incompatible figures or hide a conflict in a footnote. If evidence remains insufficient, mark the claim unverified, qualify its scope, remove it, or route it to the named specialist. For a live policy, price, product feature, or dataset, set a recheck date and state what change would trigger another review. Keep the original and revised wording so the review decision is visible.
- 07
Run an independent sign-off and preserve the reviewed version
Have a reviewer who did not assemble the evidence sample material claims and follow each one from final wording to source and any calculation. Check that citations identify the correct source and version, links open, quotations retain context, units and dates match, and every displayed result can be regenerated or explained. Citation Formatter can format a web reference, but it cannot verify the source or claim. Use Audit Evidence Index to record the decision, reviewer, unresolved limitations, and next review date; preserve the exact approved draft and its evidence register so later edits can be compared and rechecked.
THE HELPER CREW
Tools for the fiddly bits.
These are the currently published Nirmion tools matched to this guide. Open a tool page for its accepted inputs and limits.
RECEIPTS, PLEASE
Sources & review notes
Each source is linked to the steps it supports. Open it to check its scope and current guidance.
Source checked 2026-10-10