What verification means

This page states exactly what a PASS from refereekit’s checks promises. It is narrower than you will assume.

The checkers

Each kind of evidence has one checker, and each checker makes one promise. A PASS means exactly that promise and no more. The table below states each promise word for word as the code does. The page kind never passes; its row states what its FLAG still promises (see A FLAG carries a guarantee).

Evidence kind	What the model supplies	What it establishes
`equation`	`label`: the label printed beside a displayed equation, without its parentheses, for example 7, 2.1, A.3 or 26a	the exact label is printed beside a displayed equation, at a place recorded in the PDF; what the equation states is unchecked
`figure`	`label`: the figure’s label as its caption prints it, for example 3 for ‘Figure 3.’	a figure caption carries exactly this label; the figure’s content is unchecked
`page`	`page`: the physical PDF page, counting the first page of the file as 1, not the number printed on the page	the page exists; its wording is unchecked
`quotation`	`page`: the physical PDF page, counting the first page of the file as 1, not the number printed on the page; `text`: the manuscript’s words, copied exactly	the quoted words occur on the named page

Why a check stops short of a PASS is under Floors, Extraction limits and Every verdict names its reason.

What each verdict promises

Verdict	What it promises	What it does not
PASS	On a quotation, the words you quoted are on the page you named. On an equation or a figure, the label is printed as the table above says. That is all.	It does not say the sentence around them is true, or fair to the paper. What an equation states and what a figure shows are unchecked.
FLAG	The page exists. Fewer than four words were quoted, so the wording was not checked.	It does not say the words are on that page, yet the page still enters what your drafts are written from. Read it yourself.
FAIL	The words are not on that page, or the page does not exist. For an equation, its label was not found beside a displayed equation. A FAIL never enters what your drafts are written from.	It checks the text the PDF holds, so words the PDF does not hold as text can FAIL. It never says the paper lacks the words or the equation, or that anyone invented them.
UNANCHORED	Nothing. The quotation names no page, so it was not checked.	It does not say the words are in the paper, or that they are not.
A clean finish	Nothing is missing and nothing FAILs.	It is not approval. Only citations in a form the check reads (`p. N`, `page N`, `Eq. (N)`) were checked. Every FLAG, every UNANCHORED quotation and every claim not in quotation marks is still yours to judge.

Quotation-scoped

A quotation is checked when an answer links it, again when a draft cites it, and in your own text when you ask. The examples use the tutorial’s public paper.

The check ignores case, spacing and line breaks. Ligatures, curly quotes and dash widths do not count; any other difference, a digit included, fails. So 58% never matches a paper’s 5-8%. Only words inside quotation marks are compared, so a paraphrase is never checked.

PASS: the quoted words occur on the named page
FAIL: not found on page 3

A `FLAG` carries a guarantee

Naming a page with nothing quoted, such as “page 2 sets up the model”, gives a FLAG: the page exists. Naming a page the paper lacks is a FAIL, however little you quote:

FAIL: page 99 does not exist

A FLAG enters the claim pool, the checked evidence your drafts are written from, as a bare pointer. A FAIL never does. Read the page yourself before you rely on a FLAG.

Floors

A quotation of fewer than four words is never a PASS. It is checked as a FLAG.
In prose, a quoted span under 12 characters is dropped, because so short a phrase can match by accident. The citation is read as its page alone.
A quoted span over 400 characters is dropped too, as a pasted passage.

Equation and figure checks read no quoted words, so they have no floor.

Every verdict names its reason

The triage list at the end of a review groups verdicts by their reason:

found: the words are on the page.
wrong-page: the words are on another page, which the evidence names.
form-differs: the words are on the page, but a symbol differs. That symbol may matter.
absent: the words are not on the page. The evidence may show the closest line.
no-such-page: the page does not exist.
no-quotation: too few words to check, so a FLAG.
The three equation reasons are under Extraction limits.

The reason never changes the verdict.

Extraction limits

Figures

A caption is read when its line starts with “Fig.” or “Figure”, a number and a period. A caption written “Figure 3:” is not read, so a citation of that figure can FAIL.

Equations

An equation passes only when its exact label is printed beside a displayed equation that refereekit recorded in the PDF; what the equation says is never checked. The rule errs toward refusing, but it can err either way: a table row can be recorded as a label, and some real labels are missed. The tutorial’s paper numbers no displayed equation:

FAIL: equation (18) is unresolved: extraction found no candidate with this label, and no equation label in this document is supported; this is no evidence that the paper lacks it

Such a FAIL never says the paper lacks the equation. Its reason is one of three:

equation-unresolved: no such label was recorded. The evidence says why.
equation-evidence-unavailable: the session predates equation records. Start a new session on the PDF.
unsupported-equation-label: a form the check does not know, such as the range 1-3.

To see what an equation’s PASS rests on, press e in pi’s /refx-flags, which crops the place from the PDF.

The two draft flags

Before a draft’s prompt, refereekit checks each saved equation claim again. One that fails is kept out and shown as EXCLUDED. An empty pool is drafted from all the same, and the private view shows a line that begins EMPTY POOL.

After the draft, it reads each citation written in one of six forms, in any case and with or without the period:

p. N or page N;
Eq. (label) or equation (label);
Fig. (label) or figure (label).

No other form is checked, such as pp. 4-5 or Eq. 7. A citation that does not hold gets one of two flags, and the prose is left as written.

not in verified pool: the draft cites a page, equation or figure the prompt never offered. Ask about it, or cut the sentence.

failed re-verification: the anchor is in the pool, but the draft’s words for it are not on that page of the original PDF. The model may have altered a quotation. Only a quotation of four words or more and 12 to 400 characters is flagged. The flag adds the verdict’s reason.

Checking your own text

Ask the agent to check a draft after you edit it. No model runs. The check reads the page and equation forms above, and quotations of 12 to 400 characters. A quotation counts only for a page named in its own sentence. With this in the tutorial’s ours/report.txt:

The model is defined on p. 1 as "a finite set of nodes". Page 2 sets up the
sampler. On p. 4 the authors write "To test the method, we generate random"
sequences, and Eq. (3) is the key identity. On p. 3 they call it "a finite
set of edges". The authors write "a finite set of nodes" in Fig. 1, as on
pp. 3-4.

the drafts view shows:

checked against the original bound to this session (pdf sha256 9d0fc72513a2)
  PASS  p. 1 "a finite set of nodes": found on page 1 [found; pool: matching-pool-quotation]
  PASS  p. 4 "To test the method, we generate random": found on page 4 [found; pool: absent-anchor]
  FAIL  p. 3 "a finite set of edges": not found on page 3 [absent; pool: absent-anchor]
  FLAG  p. 2: page 2 exists; no quotation to verify: 0 words, need 4 [no-quotation; pool: anchor-only] -- wording unchecked
  FAIL  Eq. (3): equation (3) is unresolved: the label was seen on pages 1, 3 and 9 but extraction did not accept it as a displayed equation's label (graphics-beside-label, no-column, no-display, no-math, not-at-column-edge, not-centred, page-margin, tabular, text-after-label); this is no evidence that the paper lacks it [equation-unresolved; pool: absent-anchor]
  UNANCHORED "a finite set of nodes": quoted, no page named; not checked
5 citation(s): 2 PASS, 1 FLAG, 2 FAIL; 1 unanchored
not verified: citation forms other than 'p. N' and 'Eq. (N)', quotations under the floors, and every unquoted claim
  unsupported forms: Fig. 1, pp. 3-4
exit 1: at least one citation FAILs against the document

An anchor is the page, equation or figure a claim points to. The pool: label compares the claim with the claim pool:

matching-pool-quotation: the pool holds these words for this anchor.
anchor-only: the pool holds the anchor, but not these words.
absent-anchor: the pool holds neither.

The label never says the words are right.

The bound original

The first PDF a session reads is its original, and it is never replaced. A different PDF is refused. Ask the agent to store it as a revision, or start a fresh session.

Comparing a revision

Ask the agent to compare the session with a revised PDF, stored beside the original. Each claim in the pool gets one outcome:

Claim	Outcomes
Quotation of four words or more	unchanged, moved, or not found (which does not say the authors removed it)
Page pointer, or shorter quotation	page exists, or page gone
Equation or figure	anchor exists, or anchor unverified
Anything else	uncheckable

If a page of the revision gave no text, a quotation found nowhere is uncheckable: it may be on that page. The comparison cannot tell you whether the revision answers your report.

What verification cannot do

A PASS never says a claim is true. On an equation it confirms only the printed label, never what the equation says. Checking a derivation stays your own work.