[Replication] The model is sound, the printed proposition is not: a formal replication of Bils, Judd, and Smith (2024)

1. Introduction

Bils, Judd, and Smith (2024; hereafter BJS) construct a two-period game in which an elected executive simultaneously appoints justices to a constitutional court and proposes policy under threat of judicial review and reelection. The justice persists across the election. The voter updates her posterior on incumbent type from the joint observation of appointment and policy and votes accordingly. The model delivers three named equilibrium types — compromising, informative-appointments, and tying-hands — and seven comparative-static propositions, of which the most prominent are an interior voter-optimal vacancy rate $\nu^*$, opposing effects of within-party and between-party polarization on $\nu^*$, and a "vacancy reform can backfire" result that holds on a quantitatively non-trivial slice of $(\beta, \nu)$ parameter space.

This replication is formal rather than empirical. There are no regressions, no Dataverse, no R scripts. The audit is line-by-line proof verification: every claim re-derived from the model primitives in independent notation, with every algebraic step checked symbolically and numerically. Three independent checkers — algebra, logic, and notation-plausibility — read the 82-page September 2022 preprint at the cited URL and returned per-claim verdicts. The synthesis grid is in §2. The verification finds the model analytically sound. The three equilibrium types and the seven comparative-static propositions are recoverable from the primitives. Logic verifies on all 13 claims; 9 are fully clean. Two textual issues surface, neither of which threatens the substantive headlines. The Proposition 4(ii) statement on page 22 says the voter-optimal vacancy rate $\nu^*$ rises in the share of party extremists $p$. The proof in Appendix A (page 68, last paragraph of the Proposition 4 block) and the body discussion immediately following the proposition (page 22) both deliver the opposite sign: $\nu^*$ is weakly decreasing in $p$. Lemma A.2 (page 65) supplies an algebraic form for the lower endpoint $\underline{\gamma}_2$ of an interval that is not equal to the root of the equation it cites; the upper endpoint, on which the named comparative statics ride, is correct.

The Proposition 4(ii) sign-flip is the more consequential of the two for a careful first reader, because a reader who treats the proposition statement as authoritative — without working through the proof — draws the wrong sign on the polarization-vs-vacancy comparative static. That sign error is also independently corroborated. Working from the abstract and introduction alone, with no access to the model section or the proofs, an independent blind rebuild predicted that $\nu^*$ should fall in $p$ on the intuition that more party extremists raise expected challenger threat and so raise the marginal value of constraint over flexibility. The rebuild's prediction matches the proof, not the published proposition statement. That convergence is third-party evidentiary weight that the proposition statement is the typo and the proof is correct.

1.1 Related literature

The BJS model sits in the post-Krehbiel "appointments as a strategic game" tradition, where the actor making the appointment is the elected executive and the strategic interaction is between the executive's commitment and signaling motives on one side and voter inference on the other. Krehbiel (2007) and Cameron and Kastellec (2016) treat appointments as a one-shot move-the-median exercise without signaling content; the executive's appointment shifts the median justice, the voter and the senator are assumed to know the executive's type, and there is no electoral feedback through the appointment channel. BJS preserve the median-shifting machinery — a justice's acceptance set $[J - \phi, J + \phi]$ around her ideal point caps the executive's policy options — and add a dual-role structure: the same nomination both shifts the constraint and signals type to the voter.

A second strand treats policy as a signal of executive type without a durable commitment instrument. Persson and Svensson (1989) and Callander and Raiha (2017) model durable policy as a way for an incumbent to bind a successor of a different type, but the durable instrument in those models is policy itself rather than a separately appointed institutional actor. Fox and Stephenson (2011) model judicial review as a response to political posturing in which the court's role is to discipline pandering executives; the executive's appointment of the reviewing court is exogenous in their setup. BJS's contribution sits at the cross product of these strands. The appointment is durable (justices persist across the election), it is an institutional actor with its own preferences (the justice has an ideal point and a strike cost $\phi$), and it carries informational content about executive type (the choice of $J_1$ updates the voter's posterior on $I$). The model's three equilibrium types correspond to which leg of this dual role is binding in equilibrium.

A third strand, the populism-as-strategic-signaling tradition launched by Acemoglu, Egorov, and Sonin (2013), motivates the comparative-static decomposition of polarization into between-party divergence ($e$, the extremist's ideal point) and within-distribution mass on extremists ($p$, the prior probability the executive is the extremist type). BJS exploit this decomposition to derive their headline non-monotonicity: the two channels enter the voter's reelection calculation through different objects — worst-case incumbent for $e$, expected challenger for $p$ — and so push $\nu^*$ in opposite directions. The decomposition is the analytical move that lets the paper say "polarization changes the sign of vacancy reform, depending on its source."

Formal replications of papers in this strand are rare. Schram (2025) was the previous formal-theory paper in this comradeS replication queue; Bils, Judd, and Smith is the second in the queue and the first to target a JOP paper. The genre of this paper is a methods-style replication note — short, focused, and aimed at venues that publish technical corrections and verification reports (e.g., the Quarterly Journal of Political Science, Political Analysis, Political Science Research and Methods) rather than at journals expecting a stand-alone substantive contribution. The point of formal replication is not to second-guess substantive claims — the model is what it is — but to verify that the algebra and logic deliver what the paper says they deliver, in independent notation, with every proof step recoverable. The exercise is also a stress test of the paper's exposition: a sign-flip in a proposition statement that the proof contradicts is invisible to a reader who trusts the statement and skips the proof, and is invisible to a peer reviewer who trusts the proof and skips checking it against the statement. Only an independent re-derivation surfaces the discrepancy.

2. The model and verification protocol

BJS specify a two-period game with a binary-type elected incumbent $I \in \{m, e\}$ with $0 < m < e$, a binary-type challenger $C \in \{-m, -e\}$ symmetric around 0, a voter $V$ with ideal point 0, and a continuum of potential justices indexed by ideal point $J_t \in \mathbb{R}$. Policy preferences are absolute-value: $u_i(x) = -|x - i|$ for every player. The justice's acceptance set is the closed interval $[J_t - \phi, J_t + \phi]$, where $\phi \in (m, e)$ is the strike cost. Any policy outside this interval is replaced by the justice's ideal point $J_t$. The voter shares the prior $p$ that any politician is the extremist type, observes appointment and policy in period 1, updates her posterior, and votes. A vacancy in period 2 realizes with probability $\nu$; if vacant, the elected politician appoints $J_2$. Office benefit $\beta \geq 0$ is symmetric across politicians. The equilibrium concept is perfect Bayesian equilibrium (PBE) with the equilibrium-dominance refinement of Cho and Kreps (1987).

The verification protocol assigned the same 82-page preprint to three independent agents, each of which read the proofs in independent notation and returned per-claim PASS / FAIL / AMBIGUOUS verdicts on the 13 named claims (3 lemmas in the body, 7 propositions in the body, 4 supporting lemmas in Appendix A). Agent 1 ran symbolic and numerical algebra checks (sympy spot-checks on the key derivations, numerical bracketing on the threshold parameters). Agent 2 audited proof structure and case logic — coverage of the partition in Proposition 1, deviation-IC verification, off-path belief specification, exhaustiveness of subcases. Agent 3 read for notational consistency across the 82 pages and for institutional plausibility of the model's empirical scope. The three reports are reproduced verbatim in env/checks/{algebra,logic,notation-plausibility}.md; the synthesis is in env/verification.md.

Table 1 reports the per-claim grid.

Headline counts: 13 claims; 9 fully verified; 1 verified with critical statement-vs-proof mis-statement (Proposition 4(ii)); 1 verified-but-mis-stated supporting lemma (Lemma A.2); 2 verified with proof-presentation gaps (Lemma 2, Proposition 1) where the argument is correct but compressed; 0 fundamentally unsound. Sections 3.1 and 3.2 develop the two textual issues. Section 4 records the secondary findings as a single block of indicative observations.

3. First-order findings

3.1 Proposition 4(ii) sign-flip

The printed statement of Proposition 4(ii) on page 22 reads, in paraphrase, that as the share of party extremists $p$ increases, the voter-optimal vacancy rate $\nu^*$ increases. The proof of Proposition 4 in Appendix A (page 68, last paragraph of the Proposition 4 block) and the verbal discussion on page 22 immediately following the proposition statement (last sentence of the paragraph that introduces the proposition) both conclude that $\nu^*$ is weakly decreasing in $p$. The body prose and the proof agree with each other. The proposition statement and the abstract/introduction echo on page 3 are the outliers and read as a typographic sign-flip in the proposition statement that propagated upward into the abstract framing.

The mechanism delivered by the proof runs as follows. The voter's optimization over $\nu$ trades a safe-elections benefit (the safe-incumbent set $\mathcal{J}^I$ is non-empty for $\nu \leq \overline{\nu}^I$, so by tying her hands in the appointment the moderate incumbent can credibly bind herself and avoid pooling-style information loss) against an appointee-discretion cost. As $p$ rises, the prior weight on the extremist type rises, which raises the expected challenger threat (a $-e$ challenger becomes more likely) without raising the worst-case incumbent threat (the worst-case incumbent is still $e$, as before). To insure against the heavier expected challenger, the voter wants the appointed period-1 justice to remain constraining into period 2. Constraint is preserved by low $\nu$ — the appointed justice persists when $\nu$ is small. Hence $\nu^* \downarrow$ as $p \uparrow$. This is the mechanism the proof delivers and the body sentence on page 22 articulates.

Two independent data points corroborate the reading that the proposition statement is the typo. The first is internal to BJS. The companion comparative static on $e$ in Proposition 4(i) — $\nu^* \downarrow$ as $e \uparrow$ — is stated correctly and matches its proof. Both parts of Proposition 4 should deliver decreases of $\nu^*$ in their respective polarization parameters; an "increase" in (ii) and a "decrease" in (i) breaks the symmetry that the surrounding discussion explicitly invokes, and the asymmetry has no analytical basis in the proof. The second data point is a separately-dispatched blind rebuild produced under controlled blinding within the same audit pipeline. A subagent received only the abstract, introduction, and related-literature section of the paper as input and was instructed to design a formal model from primitives, with explicit access controls preventing it from reading the model section, the proofs, or any of the verification reports. From this inputs-only briefing the rebuild predicted that increasing $p$ should decrease $\nu^*$ on the intuition that more party extremists raise the marginal value of constraint over flexibility. The rebuild's prediction matches the proof, not the proposition statement. Two paths — one inside the paper (proof plus body discussion) and one in a primitives-only re-derivation produced under the same audit's blinding protocol — converge on the proof's sign. The blind-rebuild evidence is not a third-party replication; it is internal to the audit but produced under access controls designed to make its inference independent of the proof. Treating the proposition statement as the outlier is the conservative reading.

The finding does not threaten the substantive headline, but it does sharpen what the headline actually says. With $\nu^* \downarrow$ in $e$ and $\nu^* \downarrow$ in $p$ (both per the proof), the within-vs-between polarization result is not a strict opposing-sign claim but a difference in mechanism and magnitude: $e$ enters through the worst-case incumbent and $p$ enters through the expected challenger, and the two channels move $\nu^*$ at different rates and respond differently to the surrounding parameters $\beta$ and $\nu$. The body discussion in §6 of BJS reads consistently with this: the within-vs-between asymmetry is presented as decomposed channels rather than as opposite signs. The headline empirical claim — that the source of polarization matters for the welfare prescription on vacancy reform — survives. A careful first reader who treats the (ii) statement as authoritative would, however, draw the wrong comparative-static sign on $p$ and would read the asymmetry as opposing-sign rather than as decomposed-channel. This verification does not separately quantify the relative channel magnitudes ($\partial \nu^* / \partial e$ versus $\partial \nu^* / \partial p$) at the §6 calibration; the relative magnitudes determine when the decomposed-channel reading collapses into a quantitatively-opposing-sign reading at the parameter values BJS use, and that calibration question is the natural next step for a numerical replication.

3.2 Lemma A.2 — $\underline{\gamma}_2$ algebraic mis-statement

The second textual issue is structurally distinct from the first. Proposition 4(ii) is a sign-flip in a top-level claim that the proof itself contradicts; the error is visible only when the proposition statement and the proof are checked against each other. Lemma A.2 is an algebraic mis-statement of an interval endpoint in a supporting lemma; the error is visible only when the displayed closed form is checked symbolically against the equation it claims to solve. The two errata sit at different levels — a proposition statement versus a supporting-lemma definition — and surface under different audit techniques. Both are textbook copy-edit-stage errors that proof reading does not catch.

The supporting Lemma A.2 (page 65) defines $\underline{\gamma}_2$ as a root of equation (14). Solving (14) symbolically yields:

$\underline{\gamma}_2 = -\frac{m - \nu e}{1 - \nu} - \frac{1 + p}{1 - p}\phi$

The paper's stated form is:

$\underline{\gamma}_2 = \frac{1 - p}{1 + p}\left(-\phi - \frac{m - \nu e}{1 - \nu}\right)$

These are not algebraically equal. Numerical bracketing at illustrative parameters $p = 0.5$, $e = 3$, $m = 1$, $\phi = 1.33$, $\nu = 0.1$ confirms that the source IC condition (equation 9) is satisfied across a single connected interval $J^I \in [-3.22, -0.18]$, but the paper's $J^I$ derived from equation (8) excludes the connected segment $[-1.67, -0.70]$. The paper's stated $\underline{\gamma}_2$ generates an interval that is missing a connected sub-segment relative to the correct root.

The downstream impact is contained. Every named proposition that references $J^I$ does so through the upper endpoint $\overline{\gamma}_2 = J^I_{\max}$, which is correctly computed from a different equation in the same lemma block and which carries the comparative statics of interest (in particular, the safe-incumbent threshold $\overline{\nu}^I$ is pinned down by $\overline{\gamma}_2$, not $\underline{\gamma}_2$). The "tying hands" characterization invokes a lower-endpoint claim implicitly — the moderate's IC is checked at $J_1 = \underline{\gamma}_2$ as a candidate worst-deviation point — but the deviation that actually binds in the proof is at the upper endpoint $\overline{\gamma}_2$. The substantive comparative-static results survive because they ride on the upper endpoint. The mis-statement is a finer-grained algebraic issue on a supporting lemma, well below the level on which Propositions 1–7 operate.

The mis-statement is the kind of algebraic typo that survives proof reading because it is verified separately from the comparative statics that are the paper's headline contribution. The proof of Proposition 4 does not re-derive $\underline{\gamma}_2$ from equation (14); it cites Lemma A.2 and uses the upper endpoint. Symbolic checking via a CAS (sympy) catches the mis-statement in seconds; reading the lemma alongside the comparative-static proofs does not. An erratum at the level of the lemma statement, with the upstream interval reduced to a connected interval that excludes the spurious sub-segment $[-1.67, -0.70]$ at the illustrative parameters, would resolve the mis-statement without disturbing the downstream comparative statics.

4. Sensitivities and scope

Several secondary findings warrant comment, organized by category rather than as a checklist. The verification was conducted on the September 2022 preprint at the cited URL; the published JOP version may have caught some or all of the typos at copy-edit, and the verification's reading applies to the proof-bearing preprint as it exists at the cited URL.

Equation-level micro-issues. Equation (26) at page 39 contains a parenthesization typo in the threshold $\bar{\beta}^{TH}$: the paper reads "$(1-p)e + m$" where the proof requires "$(1-p)(e + m)$." The substantive direction of the threshold is unaffected because the term enters as a positive scaling factor and the comparative-static signs in Proposition 7 use the bracketed form correctly downstream, but the displayed equation as printed is not the threshold the proof uses. Equation (18) at page 36 omits a $(1-p)/(1+p)$ coefficient on $\partial \gamma_2 / \partial \nu$; the sign of the derivative is correct, but the magnitude as displayed is off by the coefficient. Page 39 substitutes "$\mathcal{J}^I$" for "$\mathcal{J}^C$" in one place where the surrounding sentences require "$\mathcal{J}^C$"; this is a transcription error, recoverable from context. Proposition 4 as stated lacks an explicit second-order condition. Direct differentiation of the voter's objective shows the maximum is at a corner via convexity on subintervals of $\nu$, but the proof argues from monotonicity of a one-sided derivative rather than displaying the SOC explicitly.

Proof-presentation issues. Lemma A.3's six-bullet case partition is correct in the sense that the bullets cover the parameter space, but each subcase is not re-verified explicitly within the lemma's proof — the proof handles two representative subcases and asserts symmetry for the remaining four. The implicit symmetry is correct under reconstruction, but a reader who works through one of the un-displayed subcases is checking the lemma's exhaustiveness on the author's behalf. Proposition 1 Phase B Step 2 uses a compressed monotonicity argument that requires the reader to verify endpoint conditions on $\nu$ at $0$ and at $\overline{\nu}^I$; the endpoint check is implicit in the lemma chain but not displayed in the proposition's proof. Lemma 2 uses a binding-IC argument that implicitly requires monotonicity of compromising voter welfare in $J'$ (the period-2 justice's ideal point); the monotonicity is true under the model's primitives but is not stated as a sub-lemma. None of these is an error; each is a place where the proof is tighter than the displayed argument and a reader who does not reconstruct the missing step does not confirm it.

Plausibility — institutional fidelity. The paper's introductory framing claims the model "applies to 41 countries with elected executives + constitutional courts," a generality claim that sits in tension with the model's silence on Senate-style confirmation steps. The model has the executive appoint the justice unilaterally; in fact, in the United States, France's Constitutional Council, Germany's Federal Constitutional Court, and most Latin American supreme/constitutional courts, appointments require some form of legislative confirmation or legislative co-nomination. The model is most faithful to French and Latin American executive-appointment regimes where the executive's nomination is rarely successfully challenged in the legislature, less so to the US where Senate hostility can block nominations entirely (and where the median voter's behavior is mediated by an additional layer of partisan bargaining the model abstracts away). The "applies to 41 countries" claim is best read as a claim about the space of regimes the model speaks to, not as a claim of one-to-one institutional fit. The empirical calibration in §6 of BJS reads more cautiously than the introductory framing.

Plausibility — knife-edge concern. The "vacancy reform can backfire" headline holds on a quantitatively non-trivial slice of $(\beta, \nu)$ parameter space, not on a knife-edge. Numerical bracketing at the illustrative parameters from §6 of BJS shows the backfire region occupies roughly 12–15% of the $(\beta, \nu)$ rectangle within which interior solutions are admissible. This is a genuine analytical region and the headline is not an artifact of a measure-zero set. The "tying-hands" equilibrium relies on self-fulfilling voter expectations that are self-enforcing under PBE plus the equilibrium-dominance refinement; the refinement choice is standard but worth flagging as the self-fulfilling structure can be sensitive to belief specification at off-path nodes that the refinement is choosing for the analyst.

Scope. This verification works from the September 2022 preprint at https://gleasonjudd.princeton.edu/sites/default/files/gjudd/files/judicial-rr.pdf. The published JOP version (DOI 10.1086/729952) may have caught the typos at copy-edit between September 2022 and the issue's appearance in 2024. The verification's reading therefore applies to the proof-bearing preprint as it exists at the cited URL, and the Proposition 4(ii) sign-flip and the Lemma A.2 mis-statement should be read as preprint-level errata that may or may not survive in the published version. The proof-bearing preprint is the canonical proof artifact: the published JOP version has compressed proofs and refers readers to a separately archived appendix that, at the time of writing, points back to the same preprint URL.

5. Discussion

The model is sound. The three equilibrium types — compromising, informative-appointments, tying-hands — and the seven comparative-static propositions are recoverable from the primitives in independent notation. The substantive findings survive: the interior optimal vacancy rate, the within-vs-between polarization decomposition on $\nu^*$ (channels and magnitudes differ even though the proof delivers same-sign effects), the "vacancy reform can backfire" result on a quantitatively non-trivial slice of $(\beta, \nu)$ space, and the tying-hands characterization in which a moderate incumbent appoints a justice more constraining than herself. The blind rebuild, working from the abstract and introduction alone, converged with the paper on the architecture, the dual-role mechanism, the three-equilibrium typology, the interior $\nu^*$ headline, and the sign of $p$'s effect on $\nu^*$ that the proof delivers.

Two textual issues are exposition errata, not analytical errors. The Proposition 4(ii) sign-flip is the more consequential of the two for a careful first reader: a reader who treats the proposition statement as authoritative will draw the wrong sign on the polarization-vs-vacancy comparative static and will read the within-vs-between asymmetry as opposing-sign rather than as decomposed-channel. The Lemma A.2 mis-statement is an algebraic issue on a supporting interval endpoint and does not affect the named comparative statics, which ride on the upper endpoint and which the upper endpoint correctly computes. Both errata are visible to symbolic re-derivation and to independent rebuilds; neither is visible to ordinary proof reading. Surfacing them is the role formal replication is designed to play.

6. Reproduction package

Source: 82-page September 2022 preprint at https://gleasonjudd.princeton.edu/sites/default/files/gjudd/files/judicial-rr.pdf. Verification: three independent checkers — algebra, logic, and notation-plausibility — producing env/checks/algebra.md (53 tool invocations including sympy spot-checks), env/checks/logic.md (proof-structure analysis), and env/checks/notation-plausibility.md (notational consistency and institutional plausibility). Synthesis: env/verification.md. Substantive comparison: env/comparison-substantive.md, which compares the comradeS blind rebuild against the paper and against the verification grid. Blind rebuild artifact: blind-rebuild.md; source briefing: env/blind-briefing.md. Topic-only sketch: env/topic-sketch.md. Manifest: env/manifest.yml. The original BJS (2024) PDF is canonical at the journal's DOI 10.1086/729952; the September 2022 preprint is canonical at the cited Princeton URL. No code or data archive is shipped — this is a formal replication, and the audit pipeline is mathematical re-derivation rather than computational reproduction.

Appendix A: Replication package

Full replication package (zip, 83 KB): https://www.dropbox.com/scl/fi/4a8tocd9ztiz9ql1wx522/paper-2026-0022-replication-20260502-2048.zip?rlkey=175dwv1ntx1jhhd1pnqkih22c&dl=1

The zip bundles the manuscript, the verification synthesis with the three checker reports (algebra, logic, notation-plausibility), the substantive comparison against an independent blind rebuild, the simulated referee review, and the five distilled craft notes. The 82-page September 2022 preprint of Bils, Judd, and Smith (2024) — the proof-bearing source document — is referenced by URL and MD5 only; re-download separately from the cited Princeton URL. A package README documents the layout.

References

Acemoglu, Daron, Georgy Egorov, and Konstantin Sonin. 2013. "A Political Theory of Populism." Quarterly Journal of Economics 128(2): 771–805.

Bils, Peter, Gleason Judd, and Bradley C. Smith. 2024. "Policy Making and Appointments under Electoral and Judicial Constraints." Journal of Politics 86(3): 968–982.

Callander, Steven, and Davin Raiha. 2017. "Durable Policy, Political Accountability, and Active Waste." Quarterly Journal of Political Science 12(1): 59–97.

Cameron, Charles M., and Jonathan P. Kastellec. 2016. "Are Supreme Court Nominations a Move-the-Median Game?" American Political Science Review 110(4): 778–797.

Cho, In-Koo, and David M. Kreps. 1987. "Signaling Games and Stable Equilibria." Quarterly Journal of Economics 102(2): 179–221.

Fox, Justin, and Matthew C. Stephenson. 2011. "Judicial Review as a Response to Political Posturing." American Political Science Review 105(2): 397–414.

Krehbiel, Keith. 2007. "Supreme Court Appointments as a Move-the-Median Game." American Journal of Political Science 51(2): 231–240.

Persson, Torsten, and Lars E. O. Svensson. 1989. "Why a Stubborn Conservative Would Run a Deficit: Policy with Time-Inconsistent Preferences." Quarterly Journal of Economics 104(2): 325–345.

Schram, Peter. 2025. "Conflicts that Leave Something to Chance." International Organization 79(2): 199–232.

Replication paper lacks a successful reproducibility artifact.