北京大学校徽SEKE研究组
登录

← 返回论文

论文 Blog

[arXiv 2026] CONTRA: Discovering and Qualifying Behavior-Changing Questions for Selective Clarification in LLM Code Generation

The Script Worked. The Last Backup Was Gone.

2026年10月8日 · 房正

The Script Worked. The Last Backup Was Gone. - 1The Script Worked. The Last Backup Was Gone. - 2The Script Worked. The Last Backup Was Gone. - 3The Script Worked. The Last Backup Was Gone. - 4

CONTRA helps coding agents ask before an unstated choice becomes costly.

Imagine asking a coding agent: “Prune stale backups to free space. A backup is stale after 30 days.” The newest successful backup is 40 days old. A straightforward implementation deletes it alongside the others. The cleanup succeeds. The last recovery point disappears. No syntax error. One unasked question: “Should the newest successful backup survive, even when it is stale?” The request never settled that exception. The agent did. Evidence before interruption Asking about every detail is not the answer. CONTRA uses three stages: Discover. Search broadly for missing decisions, then merge overlapping questions. Qualify. Keep questions about required behavior that the requirement has not resolved. For each, compare two plausible answers through independently generated programs and shared valid inputs. Select. Use the conversation to choose a qualified question or stop. An answer that settles another candidate removes the need to ask it again. One solution deletes every stale backup; another protects the newest successful copy. Both clean up old files, but only one leaves a recovery point. CONTRA’s plugin compares the current implementation with a plausible, runnable alternative. When a distinguishing test passes on the current version but fails on the alternative, it exposes a concrete difference in behavior. That identifies a choice for the developer to review, not which outcome they want . A test that passes on both versions does not prove equivalence. The user chooses the policy The user answers: “Keep the newest successful backup, even if it is old.” Now the retention rule is explicit. The selector rechecks the remaining queue instead of treating clarification as a questionnaire. The paper’s main method clarifies before final code generation. Its Claude Code plugin works after an edit, surfacing unstated decisions and testing alternative implementations in the background. The decisions “done” leaves out In a recorded Brotli run, the plugin exposed a dependency gap and an implicit encoding choice, then linked a verified distinguishing test. The accompanying screenshot shows those entries. Complete decision entries: 2.9% → 46.3% In the paper’s 40-task repository study, the plugin increased the share of entries stating what the request left open, what was implemented, and a plausible alternative. Test the consequences. Let the user choose. Read the paper: https://arxiv.org/abs/2610.01769 · Code components and plugin: https://github.com/fangz-cs/Contra