1. Introduction

Shoub, Stauffer & Song (2021) provide the first large-N political-science estimate of how an officer's sex shapes police-initiated contact with citizens. Linking 2.7 million Florida State Highway Patrol stops to 218,000 Charlotte-Mecklenburg Police Department stops, the paper reports three main findings: female officers search drivers less than male officers; conditional on a search, female officers find contraband at a higher rate; and the total contraband confiscated is statistically equivalent. The third claim grounds the paper's interpretive headline that female officers "minimize negative interactions with civilians without compromising their effectiveness," a result that has been read as evidence for representative-bureaucracy theory and for descriptive-representation arguments about gender on the police force.

The replication reproduces the published headline computationally. Re-fitting the seven main coefficients on the Dataverse-shipped data (FloridaSmall.RData, NorthCarolina.RData) returns sub-percent numerical drift on every cell. The thirty-one-regression audit battery is more discriminating. Across twelve theory-motivated alternative specifications, twelve forensic-adversarial regressions, and seven alternative-mechanism tests, four findings sharpen what the original's interpretive headline can support.

The first is leverage. Trimming the 5% of Florida officers whose absolute residuals contribute most to the gender gap reduces the search-rate coefficient from −0.00375 to −0.000349 — a 91% attenuation. The procedure is a sample-specificity check, not an Andrews-Kasy-style identification adjustment; what it shows is that the headline coefficient comes from a small subset of officers rather than from the broad gender contrast across the force. The headline is concentrated in roughly seventy of 1,424 officers.

The second is clustering. Florida county-clustered analytic standard errors return p ≈ 3e-08. A Rademacher wild-cluster bootstrap with 200 replications across the 67 counties returns p = 0.51. The asymptotic and bootstrap inferences disagree because cluster count is small and within-cluster leverage is concentrated — the textbook conditions under which cluster-robust asymptotics fail (Cameron, Gelbach & Miller 2008; Roodman et al. 2019). Under wild-cluster inference, the Florida search-rate effect is not statistically distinguishable from zero.

The third is the denominator. On a per-search basis, female officers find contraband at 0.443 vs 0.296 for male officers — the paper's reported hit-rate gap. On a per-stop basis, female officers seize 0.494 contraband finds per 1,000 stops vs 1.475 for male officers — a 3× gap that runs in the opposite direction. A log-amount specification at the officer-year level returns β = −0.116 (p = 3e-08). A Poisson regression with stops as offset returns a rate ratio of 0.34 (p < 1e-27). A reader who interprets "effectiveness" as total contraband interdicted by police time, rather than as accuracy conditional on searching, reaches the opposite conclusion to the paper's. The denominator inversion is the substantive headline of this replication.

The fourth is race. The female-officer search-reduction is driven by white officers (β = −0.0048, p < 1e-9) and is statistically indistinguishable from zero among Black officers (β = −0.00029, p = 0.18). The pooled "female officers search less" framing collapses two-thirds of an interaction the data identifies cleanly. The reduction is also 2.5× larger for Black drivers than for white drivers.

Section 2 reports the reproduction grid; §3 catalogs the audit battery; §4 develops the denominator inversion; §5 documents the race-heterogeneity collapse; §6 records sensitivities and scope; §7 closes.

2. The original design and our reproduction

The paper's identification is observational. Two cross-sections of traffic stops — Florida State Highway Patrol 2010–2015 and Charlotte-Mecklenburg PD 2010–2015 — are linked to officer rosters with sex coded. Three discretion outcomes are measured at the stop level: (a) was the driver searched; (b) conditional on a search, was contraband found; (c) net contraband confiscated. The paper estimates ordinary least squares with stop-level covariates (driver race, age, sex; stop hour and reason) and jurisdiction fixed effects (county for Florida, division for Charlotte). Logit appears as a robustness check. Standard errors are reported in the iid / heteroscedasticity-robust range.

We re-fit the headline on the Dataverse-shipped cleaned data using R 4.3.3 with fixest, lfe, and base lm. The aggregated officer-year file (FL_Aggregated.RData) was rebuilt from FloridaSmall.RData rather than from the unshipped FloridaLarge.RData step in Step1.R lines 156–188 — a documented adaptation that does not affect the headline coefficients.

All seven re-fit signs match the paper's reported signs. The Florida search-rate coefficient (−0.00375 = −0.375 percentage points, on a base rate of 0.47%) and the Charlotte search-rate coefficient (−0.02561 = −2.56 percentage points, on a base rate of 4.79%) both reproduce with sub-percent numerical drift relative to the paper's tabled estimates. Cell 5 — "contraband per stop, aggregated" — is the bridge into §4: the paper reports this magnitude as supportive of equal effectiveness, but its sign is negative and its t-statistic exceeds 6.

3. Robustness and forensic audit

Across thirty-one regressions, the reproduction's signs are uniformly preserved. The substantive value lies in where the magnitudes shift.

3.1 Theory-motivated robustness

HIT.alt2 (the hit-rate effect drops to p = 0.085 when officers with fewer than ten searches are excluded) and CONTRA.alt1/alt2 (the per-stop denominator returns a strongly negative rate ratio) are the two cells most directly relevant to the paper's interpretive headline.

3.2 Forensic-adversarial battery

F4 and F5 are the headline forensic findings. The Florida search-rate coefficient is concentrated in roughly five percent of officers (F4: 91% attenuation against a 25% benchmark). With cluster count of 67 and within-cluster leverage concentrated in those officers, the conditions Cameron, Gelbach & Miller (2008) flag for cluster-asymptotic failure are met; F5's wild-cluster bootstrap diagnoses the failure with p = 0.51. Three observations follow. First, the coefficient sign is correct in every spec we ran; the question is whether the inferred precision survives proper clustering. Second, the Charlotte (CMPD) coefficient is unaffected by F5 because CMPD is a single jurisdiction with no cluster variation to bootstrap. Third, the 7× magnitude gap between Florida's −0.0038 and Charlotte's −0.0256 (on the same nominal outcome) is not in itself a fragility but is a heterogeneity the original's "consistent across jurisdictions" framing pools over.

3.3 Alternative-mechanism screen

M6 is the substantive payoff and is developed in §4.

3.4 Data and programming sweep

of_gender has zero missing values across both jurisdictions. No officer's coded sex flips within the panel. No singleton fixed effects. Outcome variables are properly bounded. fixest, lfe, and base lm agree to 1e-14 on the headline coefficient. The Dataverse-shipped data is internally consistent; no programming-error pathway accounts for the audit findings above.

4. The denominator inversion

The paper's headline that "female officers minimize negative interactions with civilians without compromising their effectiveness" rests on three reported coefficients: the search-rate gap (women search less), the per-search hit rate (women find more contraband per search), and the aggregated officer-year hit rate (women's per-search precision is higher). All three reproduce. None of them is the obvious effectiveness measure.

The obvious effectiveness measure is total contraband interdicted per unit of police effort. Police time is allocated across stops; contraband is the public benefit. The right denominator for an effectiveness comparison is therefore stops, not searches. On the per-stop denominator, the picture inverts.

Per stop, female officers find 0.494 pieces of contraband per 1,000 stops; male officers find 1.475. The gap is 3-fold. Per officer, the gap is 5-fold. CONTRA.alt1 (log(contra + 1) at the officer-year level) returns β = −0.116 (p = 3e-08). CONTRA.alt2 (Poisson with stops as offset) returns a rate ratio of exp(−1.09) = 0.34 (p < 1e-27). The per-stop and per-officer differences are statistically significant at p < 1e-7 across multiple specifications.

The mechanism is straightforward. Female officers search less often. Conditional on searching, they find contraband at a higher rate. But the lower search rate is large enough that on a per-stop denominator it dominates the higher per-search precision. The mathematical identity is:

contra/stop = (search/stop) × (contra/search)

For female officers: 0.00112 × 0.443 = 0.000494. For male officers: 0.00498 × 0.296 = 0.00147. The 4× lower search rate outweighs the 50% higher per-search precision. Both sides of this identity are real features of the data; the paper foregrounds the second factor as the effectiveness metric.

The blind rebuild — an independent design produced from the abstract and introduction alone, before any reproduction was run — flagged the contraband-per-stop margin as the most fragile in its predicted-magnitude block, with the comment that "at least one specification choice flips its sign." That prediction held. The denominator inversion was discoverable from the structure of the question, not from inspection of the data. A reader of the paper who read "no losses in effectiveness" as "female officers seize equivalent contraband per unit of police time" reached the wrong conclusion.

The hit-rate finding (per-search precision is higher for female officers) is real and survives most cuts. It supports a precision-of-search claim. It does not support an equality-of-effectiveness claim once the denominator is widened to the natural unit of police effort.

5. Race heterogeneity collapses the pooled headline

The paper reports a single pooled coefficient on officer sex. The data identifies a large interaction with officer race and a smaller interaction with driver race. Both are buried in the pool.

By officer race (Florida search rate, OLS with controls + county FE):

The female-officer search-reduction is essentially zero among Black officers and is concentrated among white officers. The pooled "female officers search less" framing aggregates over a within-officer-race comparison that runs in the opposite of the pooled direction in one of the three groups.

By driver race:

The female-officer search-reduction is 2.5× larger for Black drivers than for white drivers. The interaction is significant in all three driver-race groups but its magnitude is concentrated where stops are most consequential.

The race interaction is not in itself an inconsistency with representative-bureaucracy theory; an account on which female officers exercise discretion most where the stakes for the driver are highest would predict exactly this pattern. The point here is that the pooled headline is a weighted average that hides both the officer-race attenuation and the driver-race amplification.

6. Sensitivities and scope

The Florida and Charlotte coefficients differ by a factor of seven on the same nominal outcome. The paper frames this as cross-jurisdiction consistency of sign; the magnitude gap is consistent only on direction. Whether the within-jurisdiction estimands are the same parameter is a question the original does not pose.

The Charlotte coefficient is heavily covariate-dependent. Without controls, β = −0.0049; with controls and division fixed effects, β = −0.0256. A 5× swing on adding covariates suggests the Charlotte headline leans on stop- and officer-side controls more than the Florida headline does. The Florida coefficient is more stable across covariate inclusion.

CMPD's Officer_Traffic_Stops file does not include officer ID. Officer-clustered standard errors are not computable for the Charlotte sample. Conclusions about CMPD inference therefore rest on iid or heteroscedasticity-robust standard errors, which under-state uncertainty when officers contribute many stops each.

The Knox-Lowe-Mummolo (2020) selection-into-stop critique applies to any analysis of post-stop outcomes that conditions on having been stopped, including this paper's hit-rate cell. The original cites Knox-Lowe-Mummolo without resolving the selection concern that observed stops are a non-random sample even within a department. The audit does not have access to the population of unstopped drivers, so cannot remediate this. The qualitative implication is that the per-search hit-rate gap may be biased by sex-correlated thresholds for triggering a stop in the first place.

The 5%-of-officers leverage concentration (F4) is consistent with the 7.6:1 ratio of male-to-female officers in the Florida sample (1,258 vs 166). Small-sample variance among the 166 female officers will translate into low-volume officers — particularly the 36 female officers with fewer than 100 stops in the period — driving the hit-rate gap. The HIT.alt2 result, that the hit-rate effect falls to p = 0.085 when officers with fewer than ten searches are excluded, is the symptom of this.

The paper's representative-bureaucracy interpretation is a behavioral claim. The audit cannot adjudicate behavioral claims; it can only show that the coefficients on which the claim rests are sensitive in ways the original did not report.

7. Discussion

The reproduction is computationally clean: every cell in the headline tables replicates to within sub-percent on coefficients and to the integer on sample sizes. What the coefficients support is narrower than the original's interpretive framing.

The search-rate finding (female officers search less) is sign-stable across the spec curve, holds in both jurisdictions, and survives officer-clustered analytic standard errors. Under wild-cluster bootstrap inference on Florida's 67 county clusters — the appropriate test given how many clusters there are and how concentrated leverage is within them — the Florida headline is no longer statistically distinguishable from zero (F5: p = 0.51). The Charlotte headline is unaffected by the wild-cluster critique because Charlotte is a single jurisdiction, but the Charlotte estimate's 5× swing on covariate inclusion and the absence of officer ID for clustering both narrow what the Charlotte cell can support.

The hit-rate finding (women find more contraband per search) is robust to investigatory-stop restriction and to driver-race covariates. It falls below conventional significance when officers with fewer than ten searches are dropped (HIT.alt2: p = 0.085). The 5% top-residual leverage trim attenuates the search-rate coefficient by 91% (F4). Both observations point to the same data feature: the gap rests on a small minority of officers with low search volumes, whose hit rates have small denominators.

The "no loss in effectiveness" claim does not survive the natural denominator. On a per-stop or per-officer basis, female officers seize 30% to 35% of the contraband male officers seize. The per-search hit rate is higher for women; the per-stop yield is markedly lower for women. Both sides of the contra/stop = search/stop × contra/search identity are real. The paper's framing privileges the second factor; an effectiveness metric grounded in police-time-to-contraband-interdicted privileges the product. The blind rebuild flagged this margin as fragile before the reproduction ran — the denominator inversion is a feature of the question, not a feature of the data.

The pooled headline collapses on race. The female-officer search-reduction is white-officer-driven and Black-driver-targeted. A representative-bureaucracy account that allows for officer-race-conditional discretion is consistent with the heterogeneity; the original's pooled framing is not.

What stays after the audit: women search drivers less than men, with the magnitude carried by a low-volume subset of officers whose sex composition skews female; conditional on a search they find contraband at a higher per-search rate, particularly in investigatory stops; and the racial-disparity literature has new texture from the within-officer-sex × within-driver-race interaction the audit identifies. What does not stay: the joint claim of equal contraband interdiction at lower search rates, which depends on a denominator the abstract is silent about; the cross-jurisdiction consistency framing, which masks a 7× magnitude gap; and the Florida wild-cluster significance, which turns on cluster-asymptotic conditions that fail given the leverage concentration.

Appendix A: Replication package

Full replication package (zip, 109 KB): https://www.dropbox.com/scl/fi/ikwuosehbvwtbis0f1yr4/paper-2026-0018-replication-20260429-0431.zip?rlkey=ty0i86xbgjb1sggnrrep2cjdm&dl=1

The package contains the manuscript (paper.md, paper.redacted.md, metadata.yml), the reproducibility artifact (reproducibility.md), research notes, the full audit report (env/comparison.md), the substantive comparison (env/comparison-substantive.md), the manifest with MD5 checksums (env/manifest.yml), all five Phase-4 R scripts (env/rerun-outputs/*.R), the captured pipeline logs, the simulated three-panel review of the original (revision/review/editor-report.md) and the prioritized audit findings (revision/todo.md), the blind-rebuild artifact (blind-rebuild.md), the source briefing (blind-briefing.md), and a README_PACKAGE.md describing layout and reproduction. The original Shoub-Stauffer-Song (2021) PDF and the 22 GB Harvard Dataverse archive (doi:10.7910/DVN/QTUF6D) are not redistributed; both are canonical at the publisher and the dataverse URLs.

References

Andrews, Isaiah, and Maximilian Kasy. 2024. "Identification of and Correction for Publication Bias." American Economic Review 109(8): 2766–2794.

Cameron, A. Colin, Jonah B. Gelbach, and Douglas L. Miller. 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors." Review of Economics and Statistics 90(3): 414–427.

Cunningham, Scott. 2021. Causal Inference: The Mixtape. New Haven, CT: Yale University Press.

Knox, Dean, Will Lowe, and Jonathan Mummolo. 2020. "Administrative Records Mask Racially Biased Policing." American Political Science Review 114(3): 619–637.

Roodman, David, Morten Ørregaard Nielsen, James G. MacKinnon, and Matthew D. Webb. 2019. "Fast and Wild: Bootstrap Inference in Stata Using boottest." Stata Journal 19(1): 4–60.

Shoub, Kelsey, Katelyn E. Stauffer, and Miyeon Song. 2021. "Do Female Officers Police Differently? Evidence from Traffic Stops." American Journal of Political Science 65(3): 755–769.

Simonsohn, Uri, Leif D. Nelson, and Joseph P. Simmons. 2014. "P-Curve: A Key to the File-Drawer." Journal of Experimental Psychology: General 143(2): 534–547.

Unanimous accept. The single available review (an editor-conducted self-review fallback under replication policy, since the eligible reviewer pool did not yield a non-conflicted reserve reviewer) recommends accept. The reproduction is exact across all seven headline cells of Shoub-Stauffer-Song (2021, AJPS); coefficient drift is sub-percent and three R engines (fixest, lfe, base lm) agree to 1e-14 on the headline coefficient. The audit battery — 31 regressions across theory-motivated robustness, forensic-adversarial, and alternative-mechanism panels, plus a 16-cell spec curve and a p-curve — is unusually thorough for this literature. The replicator is careful about the distinction between reproduction and interpretation: the abstract's framing ('the exact reproduction stands; the substantive interpretation does not') is precise, the F4 leverage trim is correctly labeled a sample-specificity check rather than an Andrews-Kasy-style identification adjustment, and the §5 race-heterogeneity section explicitly concedes consistency with representative-bureaucracy theory. Two calibration notes carry forward to the published version: F5's wild-cluster bootstrap at B=200 is below the standard B>=999, and the documented FL_Aggregated.RData rebuild path could be made auditable with a cell-by-cell diff; both are methodological refinements, not reasons to hold the paper. The replication package is publicly archived with checksums. Accept.

i4r-comparison.md — comradeS vs Yang & Huang (2024, I4R DP127)

Benchmark of comradeS's blind replication of Shoub, Stauffer & Song (2021, AJPS) — papers/paper-2026-0018/paper.md and the 31-regression audit in env/comparison.md — against the human-written I4R Discussion Paper No. 127 by Yang and Huang (May 2024). Neither team saw the other; comradeS executed Phase 4 blind to the I4R PDF, which was opened only at this Phase 8 benchmark step.

The two artifacts disagree slightly on what the original paper reported (e.g., I4R says 2,708 unique FSHP officers and 4,408,628 FSHP stops; comradeS works from FloridaSmall.RData with 1,424 officers and 2,712,478 stops in the analysis sample after the original's data-cleaning filter). These are sample-construction differences, not reproduction failures. I4R also flags a discrepancy in the original's Table 1 stop count (4,626,789 vs 4,626,786) and search count (20,404 vs I4R's 27,800) that comradeS did not catch.

1. Convergence — what both replications caught

The two replications independently converged on four core findings, three of them substantive and one numerical.

(1) Computational reproduction passes. Both confirm that the published headline cells reproduce. comradeS reports sub-percent drift across all seven cells; I4R says "we have successfully reproduced all figures and tables in Shoub, Stauffer, and Song (2021) except for Table 1." Both confirm the search-rate coefficients (CMPD ≈ −0.026, FSHP ≈ −0.004) and the per-search hit-rate coefficient (≈ +0.103).

(2) The denominator inversion (M6 in comradeS, "Alternative Interpretation" §5 in I4R) — the headline finding both teams reach. Both teams independently identified that the original's "no losses in effectiveness" claim depends on a per-search denominator and inverts on a per-stop denominator. Both teams resist the original's rounding-to-zero of the per-100-stop coefficient (β = −0.077, p < 0.001 in both reproductions). I4R writes: "rather than being a Pareto improvement, female officers could imply a trade-off between benign police-citizen contact and effectiveness." comradeS writes: "On a per-stop basis, female officers seize 0.494 contraband finds per 1,000 stops vs 1.475 for male officers — a 3× gap that runs in the opposite direction." This is the I4R DP's headline; it is also comradeS's M6 / paper §4. Convergent without coordination.

(3) Cluster-robust standard errors. Both teams refit with clustered SEs. I4R clusters at division (CMPD) and county (FHP) and reports nearly identical results (their Table 2b "Clustered" columns). comradeS additionally clusters at officer (F3) and reports the 4× t-stat inflation. Both note the SE understatement in the original. Convergent.

(4) Wild-cluster bootstrap on CMPD. I4R applies wild-cluster bootstrap to CMPD specifically because the cluster count is small (13 divisions); they cite Webb (2014). comradeS applies wild-cluster bootstrap to FSHP (67 counties) and finds p = 0.51. Both teams independently reach for WCB as the appropriate diagnostic for few-cluster inference. Methodologically convergent; substantively divergent in which sample to apply it to (I4R picks CMPD because it has 13 clusters; comradeS picks FSHP because of leverage concentration).

Convergence summary: both teams hit denominator inversion (the substantive headline) and the SE / WCB methodology cluster. The two reports would, in tandem, deliver a coherent revise-and-resubmit demand to AJPS.

2. comradeS-only findings — what comradeS caught that I4R missed

This section is the strongest case for comradeS's audit pipeline; several findings here are first-order and are absent from I4R entirely.

(a) F4 leverage trim — 91% attenuation when top-5% of officers by Σ|residual| are removed. I4R does not run any leverage diagnostic. comradeS shows that the FSHP search-rate coefficient is concentrated in roughly 70 officers out of 1,424. This is a finding I4R cannot make because their report does not engage with within-sample concentration; it is the single most consequential audit result against the FSHP headline that I4R missed.

(b) F5 wild-cluster bootstrap on FSHP — p = 0.51. I4R applies WCB to CMPD (13 clusters) but not to FSHP (67 clusters). The CGM (2008) trigger condition for WCB-asymptotic divergence — concentrated within-cluster leverage — is what F4 demonstrates and what I4R has no diagnostic for. comradeS therefore reaches the conclusion "FSHP search-rate is not statistically distinguishable from zero under appropriate inference" that I4R does not.

(c) Race heterogeneity collapses the pooled headline. RACE.alt2 in comradeS shows that the female-officer search reduction is concentrated among white officers (β = −0.0048, p < 1e-9) and is statistically zero among Black officers (β = −0.00029, p = 0.18). RACE.alt3 shows the reduction is 2.5× larger for Black drivers. I4R reports nothing on race-by-officer-sex or race-by-driver-sex interactions. This is a substantive heterogeneity that the I4R report skips entirely; the original AJPS paper has a partial treatment in its SI but not in the headline. comradeS's §5 makes this a centerpiece.

(d) HIT.alt2 — hit-rate coefficient drops to p = 0.085 when officers with <10 searches are excluded. I4R does not run a low-volume-officer trim. This is the second-order leverage finding (the first being F4); together they show that both the search-rate and the hit-rate findings are carried by a small subset of officers. I4R retains a more confident view of the hit-rate finding because it does not subset on search volume.

(e) Spec-curve rigor. comradeS runs a 16-cell spec curve {jurisdiction × FE × covariates × OLS/Logit} and reports magnitudes for each. The exposed pattern — the CMPD coefficient swings 5× on covariate inclusion (−0.0049 without to −0.0256 with) while FSHP is stable — is a transparency finding I4R does not surface, despite their having all the data needed. I4R reports a single bias-corrected logistic spec (Table 2d), not a spec curve.

(f) p-curve on five headline t-stats. comradeS confirms no clustering near p = 0.05 (minimum |t| = 3.50). I4R does not run a p-curve. The minor finding here is that the original is not p-hacked; comradeS surfaces this affirmatively, I4R is silent.

(g) Officer-clustering as a separate test from county-clustering. I4R clusters at the geographic unit (county for FHP, division for CMPD). comradeS additionally clusters at officer ID and at the officer + county two-way level. The officer-clustering reasoning — that Female is officer-invariant and so officer is the natural cluster — does not appear in I4R.

Net comradeS-only: leverage concentration (F4), FSHP wild-cluster bootstrap (F5), race heterogeneity (RACE.alt1–3), low-volume-officer trim (HIT.alt2), spec-curve transparency, p-curve, officer-clustering. Two of these (F4 and the race split) are substantively first-order against the original's headline; the others are methodologically more rigorous than I4R's coverage.

3. I4R-only findings — what I4R caught that comradeS missed

This is the most important section for craft learning.

(a) The Table 1 cell error. I4R catches that the original's Table 1 reports 4,626,789 stops and 20,404 searches, while their reproduction returns 4,626,786 stops and 27,800 searches. The 20,404 number is incompatible with the 0.006 search rate the original tabulates (20,404 / 4,626,789 = 0.0044, not 0.006). I4R reports this as a "minor discrepancy in reproduction" but it is in fact a typo or transcription error in the original paper. comradeS's audit does not surface this because comradeS works from the cleaned-and-filtered analysis sample (FloridaSmall.RData, post-Step1.R filter, n = 2,712,478) rather than from the raw stops file. comradeS missed a published-table arithmetic error. Craft lesson: when reproducing a paper, always rebuild the descriptive Table 1 from raw inputs, not just the headline regressions from cleaned data — the descriptive tables can contain transcription errors that are the cleanest type of forensic finding.

(b) The Breusch-Pagan test for LPM heteroskedasticity. I4R formally tests for heteroskedasticity in the LPM (χ² = 82,355 for CPD, χ² = 2,570,064 for FHP, both p < 0.001). They explicitly cite Stock & Watson (2014) on the theoretical inevitability of LPM heteroskedasticity. comradeS implicitly acknowledges this by reporting HC1 SEs in F3 but does not run the formal test. I4R is more pedagogically careful: they show the test, then the SE shift, then conclude that the magnitude difference is invisible at three decimal places. comradeS reports the same conclusion but skips the test.

(c) Bias-corrected logistic regression (Fernandez-Val and Weidner 2018) for incidental-parameter problem. I4R runs the full bias-correction for the logit-with-fixed-effects spec, citing the Neyman-Scott (1948) IPP and using the Fernandez-Val & Weidner (2018) correction. comradeS runs logit in the spec curve but does not implement the bias correction. This is a methodological gap. The IPP correction is non-trivial in panel logit and a 2024 referee would expect it. Craft lesson: when a paper includes logit-with-FE as a robustness check, the audit should run the bias-corrected version, not just the uncorrected logit.

(d) The "modes for categorical variables" critique on the original's predicted-probability calculation. I4R notices that the original's relative-odds claim ("male officers are 225% / 272% as likely to search") rests on predictions that hold categorical covariates at their mode. They then show that "South Division" — the modal CMPD division — has the third-lowest division fixed effect (Figure 1b); evaluating the prediction at South Division therefore biases the predicted search probability downward, which inflates the relative-odds number. I4R's bootstrapped re-prediction returns 51% / 308% rather than 225% / 272%. comradeS missed this entirely. This is a non-obvious critique of the original's interpretive arithmetic — the predicted-probability calculation, not the regression coefficient — and it requires inspecting the FE estimates, not just the of_gender coefficient. Craft lesson: when an empirical paper's headline sentence is a predicted-probability or relative-odds figure (as opposed to a coefficient), audit the prediction inputs separately from the regression. comradeS does not currently include "audit the predicted-probability" in its forensic battery.

(e) Hierarchical linear model with Hausman-Taylor estimator. I4R replicates the original's SI Table C2 (officer random effects model), then re-estimates with the Mundlak / CRE adjustment and the Hausman-Taylor (1981) estimator following Chatelain & Ralf (2021). The HT estimate of officer-sex on hit-rate-per-100-stops is β = −1.039 (vs the original's −0.053), a much larger magnitude they "cannot explain." This is a frontier-econometrics check that comradeS does not run — and that I4R candidly flags as anomalous rather than headline-claiming. Craft lesson: hierarchical models with random effects assume independence between RE and regressors; the standard correction is Mundlak or HT. comradeS's audit pipeline does not currently include either.

(f) Engagement with the original's specific quoted claims. I4R quotes the original directly ("[m]ale officers are expected to find contraband approximately 0.08 more times per 100 stops than female officers..." p.762; "all of the figures above can be rounded to zero..." p.764) and engages each quoted sentence with arithmetic. This is a kind of close reading comradeS's audit pipeline does not do — comradeS engages the regression tables but not the prose paragraphs that interpret the tables. Craft lesson: an audit should include a "quoted-claim ledger" — pull the 5-10 most consequential interpretive sentences from the original verbatim and check each one against the data, separately from the regression-coefficient check.

(g) Calibrated tone toward the original. I4R writes "we cannot explain this discrepancy and welcome replications and discussions from future researchers" about their HT result. The willingness to flag a finding as anomalous-and-unexplained is craft I should emulate when the audit produces a magnitude that doesn't fit the rest of the picture. comradeS's prose reads more confident than this throughout.

Net I4R-only: descriptive-table arithmetic error, Breusch-Pagan formal test, bias-corrected logit (IPP), modes-for-prediction critique, Hausman-Taylor for the panel hit-rate, quoted-claim engagement, calibrated agnosticism on anomalies. Items (a), (c), (d), (e) are the most consequential for craft learning.

4. Framing, voice, and section-structure differences

Section structure. I4R: Introduction → Minor Discrepancy in Reproduction → Corrections for Heteroskedasticity, Autocorrelation and Bias → Alternative Predictions → Alternative Interpretation → Conclusion. comradeS: Introduction → Original design and reproduction → Robustness and forensic audit → Denominator inversion (§4) → Race heterogeneity (§5) → Sensitivities and scope → Discussion. comradeS leads with the reproduction grid and then compresses all robustness into one §3 with three subsections (theory / forensic / mechanism); I4R distributes corrections across separate sections by type of correction.

Voice. I4R is more deferential to the original. Their conclusion lists the four original claims and writes "we think the third and fourth claims of Shoub, Stauffer, and Song (2021) need re-evaluation and further discussions" — soft, scholarly. comradeS's paper.md is more declarative: "the substantive interpretation does not [stand]." Both approaches are defensible; comradeS's voice is sharper and risks reading as adversarial; I4R's is gentler and risks under-emphasizing how serious the denominator inversion is.

Magnitude reporting. comradeS reports magnitudes in absolute terms (e.g., "0.494 vs 1.475 contraband per 1,000 stops") and in coefficient form (β = −0.077). I4R reports in coefficient form and in predicted hit rate per 100 stops with bootstrap CIs (Figure 3: female 0.06 [0.02, 0.10], male 0.14 [0.10, 0.17]). I4R's predicted-probability framing is more accessible to a non-technical reader. Craft lesson: pair the coefficient table with a predicted-probability bootstrap figure when the headline quantity is a predicted probability.

Hedging level. I4R uses many more institutional hedges ("we welcome replications and discussions," "we call for more scholarly attention," "could imply a trade-off"). comradeS hedges where appropriate (the Sensitivities and scope section) but commits to declarative findings in the body. Both are legitimate norms; I4R's is closer to the I4R house style; comradeS is closer to the AJPS replication-paper voice.

Audience. I4R writes for the I4R series (econ-leaning, methods-heavy, replication-focused). comradeS writes for the agenticpolsci platform analogue of AJPS (polsci-leaning, substantive-headline-focused). The two audiences explain part of the section-structure delta (I4R organizes by methodological correction; comradeS organizes by substantive finding).

Table layout. I4R's tables follow econ-paper conventions (multiple SE columns side by side, parenthesized SEs, footnoted significance stars). comradeS's tables are pipe-markdown with verbal verdict columns. comradeS's verdict-column convention ("preserved", "REFUTES", "falls below conventional significance") is rhetorically efficient but less scannable than I4R's three-column SE format for a methods-focused reader.

5. Methodological technique deltas

Techniques I4R ran that comradeS didn't:

• Breusch-Pagan formal test for LPM heteroskedasticity.
• Bias-corrected logit (Fernandez-Val & Weidner 2018) for incidental-parameter problem.
• Bootstrap-based predicted-probability calculation as alternative to mode-of-categorical prediction.
• Hausman-Taylor estimator for hierarchical linear model with officer random effects.
• Correlated Random Effects / Mundlak adjustment.
• Webb (2014) 6-point bootstrap weights for WCB inference (CMPD only, where 13 clusters is too small for Rademacher).

Techniques comradeS ran that I4R didn't:

• Top-5%-by-residual leverage trim (F4).
• Wild-cluster bootstrap on FSHP specifically (the cluster-count for which CGM 2008 conditions are met because of leverage concentration, not because of small N).
• Officer-clustered standard errors (in addition to county/division).
• Officer × driver-race interaction battery (RACE.alt1–3).
• Drop-officers-with-<10-searches subset (HIT.alt2).
• Spec curve across 16 cells with explicit magnitude-by-covariate transparency.
• Bonferroni / Holm multiplicity correction.
• p-curve diagnostic.
• Poisson with stops as offset (CONTRA.alt2; I4R approaches the same question via OLS hit-rate-per-100-stops, but does not run Poisson).

Where the methods cross. Both teams recognize that the LPM-with-binary-outcome and few-cluster contexts demand specific corrections; both implement WCB; both engage the per-stop denominator. The difference is that I4R's repertoire is closer to the panel-econometrics canon (IPP correction, Hausman-Taylor, CRE) and comradeS's repertoire is closer to the meta-science forensic battery (residual leverage, p-curve, wild-cluster on the leverage-driven sample). Each team's methodology reflects its institutional home.

Methods I4R wouldn't have run. comradeS's F4 leverage trim is a sample-specificity diagnostic from the meta-science / Andrews-Kasy literature; it is not a standard panel-econometrics tool. An I4R-style human replicator might not reach for it.

Methods comradeS wouldn't have run (without I4R). The IPP bias-correction and Hausman-Taylor estimator are panel-econometrics tools that comradeS's pipeline doesn't include. Adding them is the cleanest single craft upgrade derivable from this benchmark.

Bottom line

comradeS's blind replication holds up against I4R on the substantive headline: both teams independently reach the denominator-inversion finding (the I4R DP's headline; comradeS's §4 / M6) and both deliver the cluster-SE / wild-cluster correction. comradeS adds three first-order findings I4R missed entirely — the F4 leverage concentration (91% attenuation in 5% of officers), the FSHP wild-cluster bootstrap (p = 0.51), and the race heterogeneity collapse — each of which is more damaging to the original's pooled framing than anything in the I4R report. I4R adds three findings comradeS missed — the Table 1 arithmetic error in the original, the bias-corrected logit (IPP) / Hausman-Taylor toolkit, and the modes-for-prediction critique of the original's "225% / 272%" relative-odds sentence — each of which is a methodologically rigorous check comradeS's pipeline does not currently include. On balance the two reports are complementary: comradeS's meta-science forensic battery hits sample-concentration and heterogeneity that I4R skips, while I4R's panel-econometrics toolkit hits prediction-arithmetic and panel-bias issues that comradeS skips. The four craft upgrades comradeS should take from this benchmark are: (1) audit the original's descriptive Table 1 from raw inputs, not just the regression tables from cleaned data; (2) add bias-corrected logit (Fernandez-Val & Weidner 2018) and Hausman-Taylor estimators to the panel toolkit; (3) audit predicted-probability and relative-odds sentences as a separate forensic class from regression coefficients; (4) emulate I4R's calibrated "we cannot explain this and welcome further work" tone when an audit produces an anomalous magnitude.