ecopol-robustness

Robustness Strategy (ecopol-robustness)

When to trigger

• The headline policy magnitude moves under plausible alternative specifications
• A discussant could attribute the result to one sample, period, or country
• Inference rests on few clusters / few treated units (common in policy evaluations)
• The result is statistically significant but the magnitude — the part a policymaker uses — is imprecise
• You are pre-empting the academic discussant's "have you tried…" before the conference

Robustness is discussant-proofing, not a checklist dump

At EP, robustness has a specific purpose: the paper will be debated live by two invited discussants who act as the referees, and the academic one will arrive with a list of fragility tests. So organize robustness by the threat each test neutralizes, not as a mechanical appendix of "alternative specifications." A reader (and the policy discussant) should see, for each test, which way of being wrong it rules out. Because EP papers carry a recommendation, the central object to defend is the magnitude and its confidence interval, not just the sign.

The threat-to-test map (build the robustness section from this)

Craft moves

• Put a robustness summary in the main text, not buried 40 pages deep: a compact figure or sentence showing the headline number is stable across the threats that matter. The policy discussant will not read the appendix.
• Distinguish robustness from heterogeneity. Stable-everywhere = robust; differs-by-group = a finding to report, not a failure to hide.
• Report the range, not just a star. "The effect lies between X and Y across all specifications" is the EP-useful statement.
• Pre-register the discussant's likely objections (use ecopol-referee-strategy) and answer the top three in the main text proactively — you cannot reply live as easily as in a referee letter, so anticipate.
• Honesty about fragility. If one test moves the number, say what it implies for the policy conclusion rather than hiding it — discussants will find it.

Execution bridge (StatsPAI / Stata MCP)

Run the battery, don't just enumerate it. Full map: execution-with-mcp. Economic Policy is policy-facing applied economics; foreground a credible design and a policy-relevant magnitude.

• Many outcomes / specifications: romano_wolf (step-down FWER) or benjamini_hochberg.
• OVB sensitivity: oster_delta / sensemakr.
• Inference: wild_cluster_bootstrap (few clusters), twoway_cluster / conley.
• Re-fit off one handle: audit_result(result_id) lists missing checks + the exact suggest_function for each.
• Exhibits: etable / did_summary_to_latex from the handle — no retyped numbers.

Decisive checks in the body, exhaustive battery in the appendix. JF execution walkthrough.

Checklist

• Robustness section organized by threat, each test labeled with the threat it neutralizes
• A main-text robustness summary (figure or sentence) — not appendix-only
• Headline magnitude + CI shown to be stable (or its instability honestly bounded)
• Few-cluster / few-treated inference addressed (wild bootstrap / randomization inference)
• Pre-trends / parallel-trends sensitivity reported where the design needs it
• Coefficient-stability or placebo evidence against unobserved confounders
• Heterogeneity reported as a finding, not disguised as robustness
• Top three likely discussant objections answered proactively in the main text

Anti-patterns

• A 20-table appendix of "alternative specifications" with no map to the threats they address
• Defending only significance while the policy-relevant magnitude swings across specifications
• Standard clustered SEs with five treated jurisdictions and no few-cluster correction
• Burying every robustness result in the appendix where the policy discussant never sees it
• Quietly dropping a specification that breaks the result instead of reporting and interpreting it

Output format

【Journal】Economic Policy (EP)
【Skill】ecopol-robustness
【Headline magnitude】X (CI: [.,.]) — stable across threats? Y/N
【Threats neutralized】[sample / functional form / pre-trends / confounders / inference / measurement / external]
【Main-text summary】figure/sentence present? Y/N
【Few-cluster inference】method used
【Honest fragility note】what moves the number and the policy implication
【Next skill】ecopol-tables-figures