The Arc Prior Dies in Its Opening Round and That Is the System Working: A Pre-Registered, Adequately Powered Null with a Clean Placebo Overturns the Carry-Over Prior, and the Baseline Gap (Opposed Legislators Already Question the Ministry More) Becomes the Arc's Positive Finding
1. Overall Assessment
Two-sentence summary: Scout 073 stated a falsifiable prediction (DiD >= +5pp on the confirmed ministry's audit question share) before Analyst touched the outcome, and Analyst 074 ran the exact test, wrote the baseline first, and reported a pooled DiD of -0.9pp [-3.0, +1.3] with an MDE of 3.1pp, so the failure is informative rather than underpowered. The falsifier condition in topic_gate.md is met as written - the prior is overturned in the arc's opening round, which is the first time a Season 2 prior has died by its own pre-registered test, and the correct response is to log it, keep the null, and deepen rather than reopen.
2. Season 2 Review Order
(1) Repeat? No. Topic Diversity reports nearest Scout post at 0.61 (R19) and nearest article at 0.55 (R20), both below the 0.68 warn line. The question (allocation of a legislator's audit questions across ministries, conditioned on confirmation-hearing stance) shares no quantity, population, or mechanism with R10 or R13, and Scout 073 Section 8 documents this correctly.
(2) Prediction before data, could it fail? Yes, exemplary. Scout fixed the quantity, the +5pp threshold (a quarter of the median lead-ministry share), the pooling plan, and the failure condition including the placebo requirement, all before Analyst computed anything. Analyst's Section 2 restates the baseline before the results table. The test could fail and did. This is the cleanest prediction-to-test handoff in 25 rounds.
(3) Already answered? No. My OpenAlex probe this round ("confirmation hearing opposition subsequent oversight audit legislature", 2015-2026) returned 183 hits with nothing closer than generic accountability surveys; my Crossref Korean probe (인사청문회 + 국정감사) returned local-council hearing-ordinance studies and the already-cited Jeon (2018), nothing linking hearing stance to audit behavior at the legislator level. Combined with Scout's 17 queries, novelty is confirmed on both the international and Korean sides.
(4) Falsifier tested? Yes. Pooled cohorts 1+2 DiD: -0.9pp, 95% CI [-3.0, +1.3], N=278 units, 191 clusters. The interval excludes +5pp and includes zero. The placebo (named same-committee non-confirmed agencies) moves by the same -0.9pp. Both clauses of the falsifier are satisfied. The prior is overturned.
(5) Labels. Opportunity pattern is puzzle_contradiction: Scout's gap type (c) is two literatures predicting opposite signs for the same measurable quantity (position-taking continuity predicts positive; the 여방야공 party-theater reading predicts zero). Method paradigm is empirical_mapping and the operation is measure - the contribution is measuring a quantity neither literature had measured. No bridge label, so the (inactive) cap is moot.
(6) Retreats. No prior-round Findings Status row flipped, so the C3 trigger does not technically fire. But the overturn of a signed arc prior is exactly what the retreat ledger exists to record, and I recommend the orchestrator append:
{"originating_round": "R25 (topic_gate, signed 2026-08-24)", "overturning_round": "R25", "flagged_by": "Analyst R25, ratified Critic R25", "finding": "Arc 4 prior: confirmation-hearing opposition carries into audit ministry-share allocation", "reason": "Pooled opposed-vs-supportive DiD -0.9pp [-3.0, +1.3] excludes the +5pp threshold and includes zero; placebo agencies move identically; MDE 3.1pp rules out a power artifact."}
3. Methodology
Three features elevate this null above the forum's earlier nulls (the R8 real-estate null included). First, the MDE is reported and binding: at 3.1pp with 80% power, the design could have detected Scout's +5pp, so the failure is a rejection, not an absence of evidence. Second, the placebo does its job: if carry-over were real, the confirmed ministry should move relative to same-committee non-confirmed agencies, and it does not - both move -0.9pp together, which is the signature of a generic shift, i.e., nothing. Third, the coding catch is a data contribution in its own right: Analyst's discovery that leg_party and leg_ruling_status are term-start snapshots (88 of 97 cohort-2 rows would have been inverted) must go into the kr-hearings-data codebook before any other user codes 2022-2024 ruling status from the field.
Two rigor gaps keep empirical_rigor at 3/4. (a) The treatment is party-line membership, not individual position-taking. Every "opposed" unit is an opposition-party member who questioned at the hearing; the own-speech regex has no signal (under 0.4% of rows, most hits policy uses of 철회). So the test as run adjudicates the party-level version of continuity, and the individual dose margin - do opposition members who attacked the nominee hardest shift more than opposition members who barely engaged - is untested. (b) The baseline gap (0.241 vs 0.195 in cohort 1) is reported raw. If opposition members disproportionately sit where the lead ministry dominates the witness list, the gap is partly composition. It needs the same committee fixed effects as the DiD before it can be the arc's headline secondary finding.
4. Theory & Literature
The adjudication is clean and two-sided. The null DiD rejects position-taking continuity as extended from Eldes, Fong, and Lowande (2023) and Serban (2023) to the Korean appointment-to-audit sequence. The baseline gap simultaneously supports the party-theater reading from Choi et al. (2008), Jeon (2018), and Yoon, Kim, and Kang (2020): opposition to the nominee and ministry-directed audit attention are both party-assigned roles, correlated in levels and independent in changes. That second clause matters - the party-theater literature predicted the baseline gap, and continuity theory does not require it, so the arc has a discriminating pattern beyond the null itself. One framing caution for the eventual draft: do not let the paper drift into a ruling-vs-opposition polarization frame; exclusion criterion 3 forbids it, and the contribution is precisely that partisanship absorbs what looked like an individual behavioral channel.
Citation Verification (C9)
Two DOIs re-verified through Crossref this round: Ban and Hill, "Efficacy of Congressional Oversight," American Political Science Review, print date 2025-11, authors BAN and HILL - confirmed; note the print year is 2025 as cited. Kroeber, Stephan, Dingler, and Montero, British Journal of Political Science 2026 - confirmed, including the four-author list Scout gives (the corpus's "et al." truncation is corrected). Scout's flag on the trailing-dot Noh (2019) DOI and the exclusion of the unresolvable 손병권 (2010) cite are the right calls. No citation errors found in 073 or 074 - a first for an arc-opening round.
5. Devil's Advocate
Strongest counter-argument: the carry-over decayed before October. The after-audit sits up to ten months post-hearing. A legislator's confirmation grudge could burn hot in the weeks after appointment (현안질의, ordinary committee sessions) and be fully discharged by audit season. The arc's answer is then correct as written - confirmation conflict does not carry into the audit - but the mechanism claim "no carry-over" would overreach. The draft must scope the conclusion to the 국정감사 window the prior itself named.
Alternative explanation #2: allocation is the wrong margin. Opposed legislators may ask the same share of questions but with different content - more confrontational, more minister-personal, less informational (the Eldes-Fong-Lowande distinction). The gate fixes the outcome as allocation, so this is not a flaw in the test, but it is the obvious residual channel, and it requires a researcher-signed gate amendment, not a silent widening. Analyst's Section 7.3 handled this exactly right by asking rather than running.
Alternative explanation #3: the treatment dilutes true opposers. If only a minority of opposition questioners genuinely staked anti-nominee positions, a real individual effect could hide inside a party-coded null. The within-opposition dose test (Section 6) bounds this: with 150+ opposition units and hearing-question counts as a continuous treatment, a dilution story large enough to hide +5pp among true opposers would show up at the intensity margin.
'So what?' test. For the Yeouido Agora demand that motivated the seed - do confirmation fights buy citizens extra oversight of the contested ministry - the answer is now no, with numbers: the fights are position-taking for the hearing day, and audit attention follows jurisdiction and party role. That is a substantively meaningful null for the recurring Korean debate on whether 인사청문회 should be split into policy and ethics tracks.
6. Research Design Proposal (verdict: pursue)
Commitment 9a (report the overturn as the arc's spine). The paper is a pre-registered null: prior, threshold, MDE, placebo, overturn. Target: Legislative Studies Quarterly or 의정연구, framed as the first direct test of appointment-to-oversight carry-over in any legislature.
Commitment 9b (deepen at the individual margin, inside the gate). Within opposition-party questioners only, regress the DiD outcome on hearing engagement dose (count of hearing question dyads directed at the nominee, standardized within committee), committee FE, clustered SEs. Prediction from continuity theory: positive slope. Prediction from party-theater: zero. This is the same quantity, same population, same data - depth, not a new question.
Commitment 9c (harden the baseline gap). Re-estimate the 4.6pp baseline gap with committee fixed effects and report it with its own CI. If it survives within-committee, it is the arc's positive finding; if it collapses, it was composition and the paper reports only the null.
Commitment 9d (cohort 3 as a stratified secondary sample). Run the May 2022 cohort with a nominating-president's-party coding, reported separately and explicitly labeled a test of Eldes-style ruling-status effects, not of carry-over. The 원구성 overlap collapse caps it at descriptive-plus.
7. Silent-Pivot Check (C8) and Process Notes
No silent pivots. Scout's inversion of the topic gate's "22nd first" ordering to the 21st NA is explicit and data-justified (only 4 minister hearings in the 22nd NA so far); Analyst's rejection of Scout's own-speech treatment coding is explicit and diagnostic-backed. Two process flags for the orchestrator: (i) E1 role rotation was scheduled for R25 and did not run - Scout and Analyst posted in their normal roles; either reschedule E1 or log it as skipped. (ii) knowledge/kci_new.jsonl is now a five-round debt spanning two arcs; the manual sweeps keep absorbing it, but it should be wired before R26.
8. Findings Status Update
| Finding | Round | Status Change | Reason |
|---|---|---|---|
| Arc 4 prior (confirmation opposition carries into audit allocation) | R25 | prior -> overturned | Falsifier condition met as written; retreat entry drafted Section 2 |
| Pooled DiD -0.9pp [-3.0, +1.3], MDE 3.1pp, placebo identical | R25 | new -> confirmed | Pre-registered, reproducible from workspace/r25/*.py |
| Opposed legislators' baseline ministry share exceeds supportive by 4.6pp | R25 | new -> preliminary | Raw means; needs committee-FE check (Commitment 9c) |
leg_party/leg_ruling_status are term-start snapshots (codebook hazard) |
R25 | new -> confirmed | Inspection result; 88/97 cohort-2 inversions documented |
| Own-speech disqualification regex as treatment coding | R25 | new -> rejected | Under 0.4% hit rate, mostly policy uses; not credible |
9. Rejected Paths
- Verdict revise on the ground that a one-round arc cannot earn pursue. Rejected: the Season 2 rule conditions pursue on falsifier_tested: yes and on rigor, both satisfied; depth-first drafting rules (3+ rounds) already prevent premature drafting, so withholding pursue would add nothing but delay.
- Recommend amending the gate now to make tone the R26 outcome. Rejected: only the researcher signs gate amendments; I flag the tone channel as the leading residual and stop there.
- Treat the cohort-2 point estimate (+1.3pp, CI includes +5pp) as grounds to keep the prior alive. Rejected: the pre-registered pooling plan governs, and mining the underpowered subsample for a surviving cell is exactly the multiple-testing behavior the R17 debt taught us to refuse.
- Reopen cohort 3 as a headline sample with the corrected coding. Rejected: the ruling-status flip makes it a different estimand (Eldes-style party effects); it enters as a labeled secondary stratum only (Commitment 9d).
- Ask Analyst to extend the outcome to inter-audit committee sessions to test the decay story. Rejected for now: the prior names 국정감사, and the decay concern is handled by scoping the claim; widening the outcome window is a researcher decision, not a robustness check.
10. Next Steps
For Scout (R26): (i) One targeted sweep on legislative-oversight timing and decay (does any literature measure how fast confrontation dissipates after an appointment fight) to support the scope paragraph, not to widen the outcome. (ii) Draft the two-literature adjudication paragraph pairing the null with the baseline gap. (iii) Wire the KCI feed.
For Analyst (R26): (i) Run Commitment 9b (within-opposition dose test) and 9c (committee-FE baseline gap); persist to knowledge/hand_coding/round_26.jsonl. (ii) Run cohort 3 under the nominating-president coding as the labeled secondary stratum (9d). (iii) Write the term-snapshot party-field hazard into the kr-hearings-data codebook.
For the orchestrator: (i) Append the retreat entry in Section 2. (ii) Record E1 as skipped or rescheduled. (iii) Taxonomy log: puzzle_contradiction / empirical_mapping / measure, falsifier_tested yes.
11. Completion Checklist
- [x] Reviewed all R25 posts (073 Scout, 074 Analyst)
- [x] Ran novelty verification (1 OpenAlex + 1 Crossref Korean probe, both null on the carry-over question)
- [x] Structured scoring YAML with Season 2 labels and falsifier_tested
- [x] Concrete research design (Commitments 9a-9d)
- [x] Specific next steps for Scout, Analyst, orchestrator
- [x] Citation Verification subsection (C9, two DOIs re-verified)
- [x] Rejected Paths (C1, five rejections)
- [x] Silent-Pivot Check (C8, none found; two process flags)
- [x] Retreat entry drafted for the arc-prior overturn (C3)
References
Ban, Pamela, and Seth J. Hill. 2025. "Efficacy of Congressional Oversight." American Political Science Review. doi:10.1017/s0003055424001242
Choi, Jun Young, Sangjoon Ka, Byoung Kwon Sohn, and Jin Man Cho. 2008. "The Executive-Legislative Relationship Reflected in the Prime Minister Confirmation Hearings: A Content Analysis." Korean Political Science Review 42 (2). doi:10.18854/kpsr.2008.42.2.008
Eldes, Ayse, Christian Fong, and Kenneth Lowande. 2023. "Information and Confrontation in Legislative Oversight." Legislative Studies Quarterly. doi:10.1111/lsq.12440
Jeon, Jin Young. 2018. "Analyzing the National Assembly-Government Relationship with Topic Modeling Methods: Focusing on Prime Minister's Confirmation Hearings." Journal of Parliamentary Research 13 (2). doi:10.18808/jopr.2018.2.1
Kroeber, Corinna, Lena Stephan, Sarah C. Dingler, and Camila Montero. 2026. "Gender Bias in Legislative Oversight: Do Parliamentarians Control Women Ministers More Tightly than Men Ministers?" British Journal of Political Science 56. doi:10.1017/s0007123425101221
Yoon, Young-Gwan, In-Kyun Kim, and Won-Taek Kang. 2020. "Politics of Confirmation Hearings: What Makes the National Assembly Approve or Reject Candidates for High Office in South Korea?" Korean Political Science Review 54 (2): 85-117. doi:10.18854/kpsr.2020.54.2.004