search-lit

Literature Search Skill

You are assisting a medical researcher with literature searches and citation management for medical research papers. Every reference you produce must be verified against a live database -- never generate citations from memory alone.

Communication Rules

• Communicate with the user in their preferred language.
• All citation content (titles, abstracts, BibTeX) in English.
• Medical terminology is always in English.

Key Directories

• BibTeX output: User-specified directory (default: current working directory)
• Manuscript workspace: determined by the user or the calling skill

Search Tools: MCP (Primary) + E-utilities (Fallback)

Primary: MCP Tools (Claude.ai Remote)

Fallback: NCBI E-utilities (Direct API via Bash)

When PubMed MCP is unavailable (session timeout, "MCP session has been terminated" error, or "No such tool available" error), fall back to NCBI E-utilities via bundled scripts.

Detection: If any mcp__claude_ai_PubMed__* call returns an error containing "terminated", "not found", "not available", or "not connected", switch ALL subsequent PubMed calls in this session to E-utilities. Do not retry MCP after a disconnect — it will not recover within the same conversation.

Scripts (in ${CLAUDE_SKILL_DIR}/references/):

• pubmed_eutils.sh — Bash wrapper for NCBI E-utilities API
• parse_pubmed.py — Python parser for E-utilities responses

Usage patterns:

EUTILS="${CLAUDE_SKILL_DIR}/references/pubmed_eutils.sh"
PARSER="${CLAUDE_SKILL_DIR}/references/parse_pubmed.py"

# Search PubMed (returns PMIDs)
bash "$EUTILS" search "diagnostic test accuracy meta-analysis radiology" 20 \
  | python3 "$PARSER" esearch

# Get article summaries as markdown table
bash "$EUTILS" fetch_json "16168343,16085191,31462531" \
  | python3 "$PARSER" esummary

# Get detailed metadata
bash "$EUTILS" fetch "16168343" \
  | python3 "$PARSER" efetch

# Generate BibTeX entries
bash "$EUTILS" fetch "16168343,16085191" \
  | python3 "$PARSER" bibtex

# Verify a citation by exact title
bash "$EUTILS" cite_lookup "Bivariate analysis of sensitivity and specificity" \
  | python3 "$PARSER" esearch

# Find related articles for a PMID
bash "$EUTILS" related "16168343" 10 \
  | python3 "$PARSER" esummary

Rate limiting: 3 requests/second without API key, 10/sec with NCBI_API_KEY. The script auto-sleeps 350ms between calls. For batch operations, keep calls sequential.

E-utilities → MCP equivalence:

---

Workflow

Phase 1: Search Strategy

1. Understand the need: Get the research topic, specific question, or manuscript section that needs references.
2. Generate search terms:
   • Identify key concepts (Population, Intervention/Exposure, Comparison, Outcome).
   • Generate MeSH terms for PubMed queries.
   • Build Boolean queries: (concept1 OR synonym1) AND (concept2 OR synonym2).
3. Define scope:
   • Date range (default: last 10 years unless user specifies).
   • Article types (original research, review, meta-analysis, etc.).
   • Language filter (default: English).
4. Present the search plan to the user before executing. Include the Boolean query, databases to search, and filters.

Gate: Wait for user approval before running searches.

Phase 2: Execute Search

1. Search PubMed using search_articles with the Boolean query.
2. Search Semantic Scholar using semanticSearch with natural language query.
3. Search bioRxiv/medRxiv using search_preprints if preprints are relevant.
4. Deduplicate results across databases (match by DOI or title similarity).
5. Present results in a structured table:

| # | Title | Authors (first + last) | Year | Journal | PMID/DOI | Relevance |
|---|-------|----------------------|------|---------|----------|-----------|
| 1 | ...   | Kim J, ... Lee S     | 2024 | Radiology | 12345678 | High      |

6. Ask the user to select which papers to include.

Phase 2.5: Citation Searching (Snowballing)

Optional but recommended for systematic reviews and thorough background work (PRISMA item 7, "records identified through citation searching"). Expands a seed set along the citation graph instead of relying on Boolean recall alone.

Use the deterministic helper references/snowball.py (Semantic Scholar Graph API; nothing generated from memory):

# Expand seed DOIs/PMIDs in all directions, dedup against the existing pool,
# append verified candidates to references/library.bib
python3 references/snowball.py \
  --seed DOI:10.1148/radiol.2024123,PMID:38000001 \
  --direction all \
  --pool references/library.bib \
  --out references/library.bib

• Directions: backward (references the seeds cite), forward (papers citing the seeds), similar (S2 recommendations), or all (default).
• Dedup: against the current references/library.bib by DOI and normalized title, and within the harvested set.
• Trust flag: snowball candidates are written verified=false + verified_by=semantic_scholar. They are candidates, not confirmed citations — run /verify-refs (or Phase 4 verification) to confirm each against PubMed/CrossRef before citing.
• Output contract: appends to references/library.bib only. NEVER writes manuscript/_src/refs.bib (the script hard-refuses that path).
• PRISMA line: the script prints, e.g., Records identified through citation searching (snowballing): N raw (backward=…, forward=…, similar=…); after dedup against existing pool: M new candidates. — record M in the PRISMA flow's citation-searching box.

A deterministic, network-free challenge card (recorded fixtures + expected output + verify.sh) lives in references/snowball_challenge/.

Phase 3: Deep Read

For each selected paper:

1. Retrieve full metadata using get_article_metadata (PubMed) or get_preprint (bioRxiv).
2. Extract key information:
   • Study design
   • Sample size / dataset
   • Key methods
   • Primary findings (with specific numbers)
   • Limitations noted by authors
3. Build a literature matrix if multiple papers selected:

| Paper | Design | N | Key Finding | Limitation | Relevance to Our Study |
|-------|--------|---|-------------|------------|----------------------|

4. Present the matrix to the user for review.

Phase 4: Citation Management

Anti-Hallucination Protocol

This is the most critical part of the skill. Follow these rules without exception:

1. NEVER generate a reference from memory alone. Every reference must come from an API search result.
2. NEVER fabricate DOIs or PMIDs. If you cannot find a DOI/PMID, mark the reference as [UNVERIFIED - NEEDS MANUAL CHECK].
3. Cross-check every reference against the API result:
   • Author names (at least first author and last author)
   • Publication year
   • Journal name
   • Article title (exact match, not paraphrased)
   • Volume and pages (if available)
4. If any field does not match, flag the specific mismatch.
5. For DOI verification, use WebFetch with https://api.crossref.org/works/{DOI} to confirm the DOI resolves correctly.

BibTeX Generation

For each reference (verified or not), generate a BibTeX entry with an explicit verified flag so downstream skills (/lit-sync, /verify-refs, /write-paper) can reason about trust without re-running verification:

@article{FirstAuthorLastName_Year_ShortKey,
  author    = {Last1, First1 and Last2, First2 and Last3, First3},
  title     = {Full Title As Retrieved From Database},
  journal   = {Journal Name},
  year      = {2024},
  volume    = {310},
  number    = {2},
  pages     = {e234567},
  doi       = {10.1001/jama.2024.12345},
  pmid      = {12345678},
  verified  = {true},
  verified_by = {pubmed+crossref},
  verified_on = {2026-04-24},
}

verified flag values (required on every entry):

verified_by lists the data sources that confirmed the entry (e.g., pubmed, crossref, semantic_scholar, or a combination). verified_on is the ISO date of the most recent successful verification.

BibTeX key convention: FirstAuthorLastName_Year_OneWord (e.g., Kim_2024_Validation).

Output

1. Save BibTeX entries to the specified .bib file (append, do not overwrite). Target: references/library.bib (candidate pool for /lit-sync to import into Zotero). NEVER write to manuscript/_src/refs.bib — that is /lit-sync's sole-writer path per docs/artifact_contract.md.
2. Print a summary of all references with verification status:

Verified:    12 references (verified=true)
Unverified:   1 reference  (verified=false) [NEEDS MANUAL CHECK]
Total:       13 references

Phase 4b: Zotero Library Integration

If a Zotero MCP server is available, integrate search results with the user's library:

1. Check for duplicates first: Use zotero_search_items (by DOI) to skip papers already in the library — this search-first step is what dedupes; zotero_add_by_doi does not dedupe on its own.
2. Add papers to Zotero: Use zotero_add_by_doi for DOI-based import (its attach_mode argument governs the OA PDF attach attempt at add time).
3. Organize into collections: Use zotero_manage_collections to file into the relevant project collection.
4. Leverage annotations: Use zotero_get_annotations to reference the user's prior reading notes.
5. Write sync audit: Record collection key, added/skipped/failed counts, and unsynced entries in references/zotero_collection.json so Zotero status is auditable rather than a hidden optional side effect.

Requires Zotero Desktop running with MCP server. Skip this phase if unavailable. If skipped, still write references/zotero_collection.json with status: "skipped" and the reason.

Phase 5: Full-Text Retrieval

Full-text PDF retrieval is delegated to /fulltext-retrieval — the single authored home of the open-access cascade (arXiv → Unpaywall → PMC → OpenAlex → Crossref → landing page, each validated with a %PDF- header + ≥10 KB size). Do not re-implement OA fetching here.

Pass the verified candidate DOIs from references/library.bib:

ENGINE="${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/fulltext-retrieval/fetch_oa.py"
# extract DOIs from references/library.bib → dois.txt (one per line)
python3 "$ENGINE" dois.txt -o pdfs/ -e <contact-email> --report pdfs/retrieval_report.json

For Zotero-resident PDFs and higher-yield, proxy-aware retrieval, use /lit-sync Phase 2.7, which also invokes /fulltext-retrieval and triggers Zotero's native "Find Available PDF".

Alternative sources (legitimate only)

For DOIs that open access cannot reach (listed in pdfs/manual_needed.txt):

• Institutional access / proxy / VPN — through your library's own subscriptions.
• Interlibrary loan (ILL) — request via library services.
• Author contact — email the corresponding author for a copy or preprint.

Never bypass paywalls or publisher access controls, and do not configure unauthorized PDF mirrors. Rate limits and PDF validation are handled inside /fulltext-retrieval.

Phase 6: Gap Analysis

When called during manuscript writing (especially by /write-paper Phase 7):

1. Read the manuscript to extract all inline citations.
2. Compare cited references against the search results.
3. Identify gaps:
   • Key papers in the field that are not cited.
   • Outdated references when newer versions exist.
   • Missing methodological references (e.g., statistical methods, reporting guidelines).
4. Report findings to the user with specific suggestions.

---

Specialized Search Modes

Mode: Manuscript Paper Reference Pool

For supplying a manuscript's reference pool — typically invoked by /write-paper Step 7.3c (or /self-review Phase 2.5c-2) when the reference adequacy gate finds the draft under target or a named method uncited, but usable directly when building out an original-research bibliography.

This mode is deliberately broad: for an original-research article, return 25–40 verified candidates, not the ~10 a quick search settles on. Do not stop early unless the field is genuinely sparse — and if it is, say so explicitly rather than returning a thin list silently. Respect a narrower journal reference cap or user scope when one is given.

Structure the pool across six candidate categories so the gaps the adequacy gate cares about are all covered:

1. Background / disease burden / clinical context — establishes why the question matters.
2. Gap-defining prior studies — the work the manuscript extends or contradicts.
3. Comparator / comparable-design cohorts — studies the Results will be measured against.
4. Methods / statistical canonical sources — the originating reference for every named method, model, score, equation, or diagnostic criterion (e.g. competing-risk model, multiple imputation, E-value, eGFR equation, concordance statistic). This is the category that clears Methods named-method gaps.
5. Reporting-guideline sources — STROBE, TRIPOD(+AI), CONSORT, PRISMA(-DTA), STARD, etc.
6. Interpretation / mechanism / limitation support — grounds Discussion claims.

For each candidate, report: PMID/DOI, verification status, candidate category, the target manuscript section it belongs in, and a one-line why it is needed.

Boundary (unchanged): every entry is API-verified before inclusion, and BibTeX is appended only to references/library.bib — the candidate pool for /lit-sync to import into Zotero. Never write to manuscript/_src/refs.bib; that SSOT belongs to /lit-sync. This mode produces candidates; it does not decide inclusion (the user does) and it does not insert references into the manuscript bib.

Mode: Systematic Search

For systematic reviews or comprehensive literature sections:

1. Document the full search strategy (PRISMA-compliant).
2. Record: database, date of search, query string, number of results.
3. Track inclusion/exclusion at each screening step.
4. Output a PRISMA flow diagram data summary.

Mode: Quick Cite

For quickly finding a single reference the user describes:

1. User says something like "that 2023 paper by Smith about AI in chest X-ray."
2. Search PubMed and Semantic Scholar with the described details.
3. Present top 3 candidates.
4. User confirms which one.
5. Generate BibTeX entry.

Mode: Related Papers

For expanding from a known paper:

1. User provides a PMID or DOI.
2. Use find_related_articles to get related papers.
3. Use Semantic Scholar for citation-based recommendations.
4. Present results ranked by relevance.

For a structured, dedup-aware, PRISMA-countable expansion (backward + forward + similar) prefer Phase 2.5: Citation Searching with references/snowball.py, which appends verified candidates to references/library.bib and reports a citation-searching count.

Mode: Embase Browser Automation

Embase has no public API. Use Chrome browser automation (MCP) to search and export:

1. Navigate to embase.com — institutional SSO authenticates automatically. If cookie error (login?error#), clear Elsevier/Embase cookies and retry.
2. Go to Advanced Search tab.
3. Enter Embase-syntax query (Emtree /exp + :ab,ti field tags). Uncheck "Map to preferred term in Emtree" when using explicit /exp terms.
4. After results appear, use "Select number of items" dropdown → select total count.
5. Click Export (in Results section) → choose CSV format → check fields: Title, Author names, Source, Publication year, Publication type, DOI, Abstract, Language of article, Medline PMID.
6. Click Export → Download tab opens → click Download.
7. CSV is in row format (records separated by blank rows) — parse with:

   # Each record = consecutive rows until blank row
   # Row format: [FIELD_NAME, value1, value2, ...]
   # AUTHOR NAMES row has multiple values (one per author)
   

PubMed → Embase query translation:

• MeSH [Mesh] → Emtree /exp
• [tiab] → :ab,ti
• [Title/Abstract] → :ab,ti
• Boolean operators stay the same (AND, OR)
• Phrase search: use single quotes in Embase ('artificial ascites')

---

Error Handling

• If a search returns 0 results, broaden the query (remove one concept or use broader MeSH terms) and retry.
• CrossRef HTTP errors (token-saving rules):
  • 403 (rate-limited): Do NOT retry. Skip CrossRef silently → verify via PubMed title search instead.
  • 303 (redirect): Follow the redirect if possible. If not, skip CrossRef → PubMed fallback.
  • Any repeated failure: After the first CrossRef 403/303 in a session, assume CrossRef is rate-limiting and skip CrossRef for ALL remaining references. Go directly to PubMed title verification. This avoids N×retry token waste.
  • Never print raw error messages like "Request failed with status code 403." Collect failures silently and report a single summary line at the end: CrossRef unavailable for {N} references (rate-limited). Verified via PubMed instead.
• If a DOI does not resolve via CrossRef (after applying the rules above), try searching PubMed by title to confirm the reference exists.
• If the user provides a reference that cannot be verified by any method, clearly state: "This reference could not be verified. Please check manually before submission."
• Never silently include an unverified reference.

What This Skill Does NOT Do

• Does not download from paywalled journals without user-provided credentials or institutional access.
• Does not assess the quality of evidence (use /analyze-stats or /check-reporting for that).
• Does not write the literature review text (use /write-paper for that).
• Does not fabricate any part of a citation.