Reviewing in your coding agent
You run refereekit by chatting with an agent: pi, Claude Code, Codex, Claude Desktop or the ChatGPT desktop app. You read the paper’s words, and type yours, in a private view. refereekit never gives what it shows there to the agent’s model.
Status: not yet qualified. Every protection below is built and tested offline. What is not yet checked by hand is how each agent version behaves; What is not yet confirmed lists the open points. pi is supported first. The ChatGPT desktop app is a preview.
What you need
Section titled “What you need”- A supported computer: Ubuntu 24.04 on x86-64, or macOS 14 or newer on Apple silicon.
- uv, and refereekit installed by the setup prompt (see Install).
- Your agent, at or above the version refereekit supports: Claude Code 2.1.295, Codex 0.159.2, pi 0.99.1. The desktop apps have no minimum version yet.
The agent’s model is not trusted with the paper
Section titled “The agent’s model is not trusted with the paper”Two language models are involved. The agent’s model is the one you chat with. The model backend is a second one that reads the paper for refereekit. You choose it, and you declare that its account keeps nothing (see Confidentiality).
refereekit never gives the paper to the agent’s model. So never paste a paper, an answer or a draft into the chat, and never type a question, a verdict or a key there. Those go in the private view.
What you give the agent
Section titled “What you give the agent”What you type in the chat reaches the agent’s model, and stays in the
agent’s history and logs, and so do the PDF’s file name and the venue.
Keep the title and the authors out of both. Rename a PDF named after its
title, to paper.pdf say, before you name it to the agent.
The first start
Section titled “The first start”Each agent needs one step from you after setup.
- Claude Code asks whether to approve the project’s MCP server, the
part that gives it refereekit’s tools. Approve
refereekit, or the tools are missing. - Codex asks whether to trust the reviews folder. Trust it, or Codex
has no refereekit tools. Codex also runs refereekit’s hook guard only
after you approve it with
/hooks. Approve it again after each update. - pi loads refereekit when you start it in the reviews folder.
- Claude Desktop and the ChatGPT desktop app load refereekit when you quit them and open them again.
pi’s commands
Section titled “pi’s commands”In pi you can open each view yourself:
| Command | Opens |
|---|---|
/refx-review |
the questions, your verdict and both drafts |
/refx-drafts |
a draft, to edit and check |
/refx-flags |
the triage list |
/refx-note |
venue notes |
/refx-model |
model access |
/refx-tutorial |
the practice review |
Each takes a session reference, such as /refx-review s-1a2b3c4d, or asks
you to choose one. /refx-model takes none.
The guides
Section titled “The guides”Claude Code and Codex get short guides, one per workflow, that tell the agent which tools to call. A first review needs the core ones. Ask the agent for the full set to add six more. How you invoke each, per agent:
| Workflow | Profile | Claude Code | Codex | pi |
|---|---|---|---|---|
| setup | core | /refx-setup |
$refx-setup |
|
| tutorial | core | /refx-tutorial |
$refx-tutorial |
/refx-tutorial |
| start | core | /refx-start |
$refx-start |
|
| status | core | /refx-status |
$refx-status |
|
| draft | core | /refx-draft |
$refx-draft |
|
| check | core | /refx-check |
$refx-check |
|
| finish | core | /refx-finish |
$refx-finish |
|
| delete-session | core | /refx-delete-session |
$refx-delete-session |
|
| help | full | /refx-help |
$refx-help |
|
| questions | full | /refx-questions |
$refx-questions |
|
| voice | full | /refx-voice |
$refx-voice |
|
| venue | full | /refx-venue |
$refx-venue |
|
| recheck | full | /refx-recheck |
$refx-recheck |
|
| rebuttal | full | /refx-rebuttal |
$refx-rebuttal |
|
| drafts | core | /refx-drafts |
||
| flags | core | /refx-flags |
||
| model | core | /refx-model |
||
| note | core | /refx-note |
||
| review | core | /refx-review |
Where a cell is empty, ask in plain words, such as “open model access”. Do the same in the desktop apps, which have no guides. Most guides run only when you invoke them, so the agent does not choose one on its own. It is not a lock.
Where the paper’s words appear
Section titled “Where the paper’s words appear”Every step that shows the paper’s words, or takes yours, happens in a private view. Its form, and how well it keeps the text to itself, depends on the agent.
pi: the extension’s own view
Section titled “pi: the extension’s own view”In pi the private view is drawn inside pi, as dialogs, panels and an editor. The model does not read them, and pi does not save them. It saves each tool call, whose result is codes and counts.
Its limits:
- pi’s dialogs have no password field, so a key shows on your screen as you type it.
- pi shows dialog titles to every extension it loads. Load no extension you do not trust in a reviews folder.
- If you edit a draft in your own editor rather than pi’s, that editor may keep backup or history files where refereekit cannot see them.
The other agents: the review window
Section titled “The other agents: the review window”In the other agents the private view is a review window in your browser. It is a page on your own computer, behind a key that changes each time it opens. It loads nothing from elsewhere and runs no script.
The window is weaker than pi’s view:
- Your browser keeps the window’s address; its key expires when the window closes.
- A browser that restores tabs after a crash may restore what you typed.
- A client that opens the window itself sees its address, and may log it.
- Any program running as you can reach the page.
- A browser or screen tool the agent controls can read the window as you do. Turn such tools off, in pi too.
- Your editor’s backup files are the same risk as in pi.
“Open it in your editor” needs an editor that opens its own window, such as VS Code. A terminal editor such as vim cannot run there.
No screen, no review
Section titled “No screen, no review”A review step needs a private view. Where there is none, it refuses with
no-private-view before it reads a file or calls a model. That happens
in pi’s print mode, and on a computer with no display, such as over SSH.
Starting a session, its status and the checks still work.
What protects the manuscript, and where it stops
Section titled “What protects the manuscript, and where it stops”Three layers, each with a limit.
- Tool results carry counts and codes only. Never a quotation, a draft or a message. Limit: code that imports refereekit’s Python modules directly is not refused.
- The hook guard blocks the agent’s ordinary routes to private
files, such as the sessions,
.envand the PDFs. Limit: it sees the agent’s built-in tools only, not a script the agent writes and runs. - The guides tell the agent not to try. The server sends the same rules as its instructions. Limit: a model can ignore an instruction.
This prevents accidental leaks; it is not a sandbox.
A hook that is not running protects nothing. Claude Code blocks a call whose hook fails to run. A hook missing from the settings, or in a folder Claude Code does not trust, protects nothing. Codex runs the hook only after you approve it. pi blocks on any hook error, once refereekit has loaded. File or shell tools from another MCP server or pi extension are not guarded.
Desktop apps: no hooks
Section titled “Desktop apps: no hooks”Claude Desktop and the ChatGPT desktop app run no hooks, so layer 2 is missing there. What still holds lives in refereekit’s server. Tool results are counts and codes. Your words and keys are refused as tool arguments.
Whatever the app’s model might do with its other tools, such as reading
.env, rests on the instructions alone. Connect no file or shell tool to
a desktop app you review in.
Set model access in the private view. A setting you export in a shell never reaches a desktop app.
What is not yet confirmed
Section titled “What is not yet confirmed”- Whether Claude Desktop passes the server’s instructions to its model. If it does not, the model still reads the verdicts’ meanings in the tool descriptions, and the server still refuses your words and keys.
- Whether a live Codex enforces its read limit on every call.
- Whether Codex shows refereekit’s confirmation forms without a setting of your own.
- Whether Codex and the desktop apps open the review window through their client.
What doctor reports
Section titled “What doctor reports”Doctor, the check of your machine the agent runs, never reports a hook
as active, since a file on disk does not prove the next call runs
through it. For the desktop apps it reports unguarded.
Deleting a session, and other yeses
Section titled “Deleting a session, and other yeses”Deleting a session needs your yes on your own screen, and so does removing refereekit. You see what the step would destroy, in pi’s dialog, the client’s own form or the review window. A reply in the chat is not that yes.
Update and remove
Section titled “Update and remove”To update, ask the agent. It refreshes refereekit’s files for the installed version. To remove refereekit from the folder, ask the agent, and say yes in the private view.
Both change only files refereekit recorded as its own, and leave a file
you edited alone. Your sessions, STYLE.md, questions, policy and notes
are never touched. To upgrade refereekit itself, ask the agent too.