Evidence-first reference checks for Python and the terminal.
I started Citeprobe because a failed search is not proof that a citation was made up. Metadata can be partial, APIs can fail, and preprints can differ from published records. Citeprobe keeps those cases separate. It records what each source returned, checks every explicit cited author, and explains why a policy reached its verdict.
Citeprobe does not use an LLM. Reference checks need Python 3.10 or newer,
ordinary CPU hardware, and an internet connection unless cached evidence or a
local index is selected. Born-digital PDF extraction uses pypdf or GROBID.
Scanned PDF extraction uses Poppler and Tesseract. LaTeX source diff needs
latexdiff; PDF compilation can use trusted local TeX or an isolated Docker
container.
Install the current development version:
python -m pip install "citeprobe @ git+https://github.com/Imanm02/citeprobe.git"Then check a DOI, a pasted reference, or a file:
citeprobe "doi:10.1038/nphys1170"
citeprobe "PMID: 24197398" --sources europe_pmc,openalex
citeprobe "Vaswani et al. (2017). Attention Is All You Need. NeurIPS. arXiv:1706.03762"
citeprobe references.bib --policy strict
citeprobe references.bib --checks duplicates
citeprobe references.ris
citeprobe references.json
citeprobe endnote-library.xml
citeprobe batch reference-library.zip
citeprobe paper.pdf
citeprobe extract paper.pdf --backend local --format json
citeprobe benchmark pdf paper.pdf --backend local --runs 3 --expected-references 40
citeprobe paper.pdf --pdf-backend local
citeprobe paper.pdf --pdf-backend auto
citeprobe references.txt --jsonl
citeprobe references.txt --input-format numbered
citeprobe references.bib --suggest-missing
citeprobe references.bib --format html --output report.html
citeprobe references.bib --corrected-bibtex corrected.bib
citeprobe manuscript main.tex --checks undefined,uncited,duplicates
citeprobe latex bib old.bib new.bib --bib-match identifier
citeprobe latex citations old-project new-project
citeprobe latex visual old-project new-project --output diff --mode source-only
citeprobe references.bib --evidence evidence.json --save-run
citeprobe replay evidence.json --format html --output replay.html
citeprobe lifecycle evidence.json --output work-families.jsonPDF input uses a GROBID service at http://127.0.0.1:8070 by default. Run
GROBID locally with Docker, or set CITEPROBE_GROBID_URL to a trusted service
root. Citation consolidation is off by default, so extraction does not hide a
second metadata lookup inside GROBID. Remote services must use HTTPS unless
plain HTTP is explicitly enabled.
For born-digital PDFs, install the local backend and use it without a service:
python -m pip install "citeprobe[pdf] @ git+https://github.com/Imanm02/citeprobe.git"
citeprobe paper.pdf --pdf-backend localUse citeprobe extract when you want parsed references and extraction details
without any metadata-provider requests:
citeprobe extract paper.pdf --backend local
citeprobe extract paper.pdf --backend auto --format json
citeprobe extract paper.pdf --backend grobid --format jsonl --output references.jsonl--pdf-backend auto tries local text extraction first. If usable text is
present but cannot be segmented, it uploads the PDF to the configured GROBID
service. It does not upload after encryption, dependency, size, or parser
failures. Use the OCR backend for a scanned paper:
citeprobe extract scan.pdf --backend ocr --ocr-languages eng --ocr-dpi 300OCR needs pdftoppm and tesseract on PATH. It runs page conversion and
recognition locally, returns word confidence and geometry, and does not need a
GPU.
For the local interface, install the UI extra and start it:
python -m pip install "citeprobe[ui] @ git+https://github.com/Imanm02/citeprobe.git"
citeprobe uiThe interface opens on 127.0.0.1 and uses the same checks, policies, sources,
and report renderers as the Python and terminal interfaces. It has separate
pages for reference audits, archive batches, PDF extraction, manuscript
inspection, LaTeX diff, evidence and local indexes, saved runs, benchmarks,
diagnostics, and examples.
Evidence snapshots can be replayed or turned into work-family reports. Local
indexes can be created, built from verified snapshots, and inspected. The
benchmark page validates and runs pinned manifests.
The LaTeX page can inspect a bounded project ZIP, compare semantic changes, or build a marked revision. Archive and project input limits are selectable. Generated TeX and PDF artifacts have an explicit download selector. The PDF page exposes its reference cap, retries, backoff, and JSON Lines download. Saved runs are paged and can be pruned to a confirmed newest-run count. Review choices can stay on one run, become an exact-match annotation, or remove a prior annotation. Cross-run matching requires a cited identifier that a resolved source record corroborates. Finding position and stable subject keep repeated author findings separate. Revision preconditions can reject stale browser, CLI, and Python writes. Diagnostics reports local dependencies, cache state, source access, checks, and issue guidance without returning environment values.
PDF extraction can use local text, OCR, GROBID, or automatic routing. The GROBID URL is selected when the server starts:
citeprobe ui --grobid-url http://127.0.0.1:8070The browser can choose a PDF backend and GROBID consolidation mode, but it
cannot replace the server's GROBID URL or enable plain HTTP for a remote
service. Consolidation is available only with the GROBID backend.
GROBID text mode calls processFulltextDocument; local text mode uses pypdf,
and OCR text mode uses Tesseract. The public GrobidClient.extract_text and
extract_text_async methods return bounded plain text with backend
provenance.
Its Examples page has 32 copyable terminal and Python workflows with visible
input and output shapes. Text examples can load into the audit form.
The default command queries Crossref, DataCite, Europe PMC, OpenAlex, Semantic Scholar, DBLP, OpenReview, and arXiv concurrently. Responses are cached on disk so repeated work is faster and places less load on public services. Duplicate references inside one batch share each in-flight provider request.
Nine opt-in sources extend that set: ACL Anthology, IACR, PubMed, DOI
resolution, Open Library, CourtListener, GovInfo, official IETF standards, and
a local SQLite metadata index. Opt-in sources are selected with --sources.
Transient timeouts, rate limits, and selected server errors are retried twice
by default. Retry-After is respected within a bounded delay, and provider
responses are capped at 5 MB.
- DOI, PMID, and PMCID identifiers are normalized and compared exactly.
- RFC and versioned Internet-Draft identifiers can be checked against official RFC Editor and IETF Datatracker records.
- arXiv version suffixes are retained in output but ignored for work identity.
- Titles are compared after case, accent, punctuation, whitespace, character, and token normalization.
- Publication years are checked with a policy-specific tolerance.
- Every explicit name in the cited author list is assigned to a distinct name in each source list.
- Silent author-list truncation and author-order changes are separate checks.
- Venue, volume, issue, pages, publisher, and edition can be checked.
- OpenReview acceptance, rejection, withdrawal, and pending state are separate.
- Retractions, removals, concern notices, and corrections stay separate from reference existence.
- Cited HTTP and HTTPS URLs can be checked without downloading response bodies.
- An optional Wayback fallback records a bounded archived snapshot for the URL check without treating that snapshot as bibliographic identity evidence.
- A missing author is treated as absent only when the source declares its author list complete.
- Close candidate ties are reported instead of hidden.
- Conflicting source records are reported instead of silently choosing one.
- Exact and likely duplicate bibliography entries are reported across a batch.
- Preprint, publication, update, correction, and retraction relations can be written as a separate work-family report. Crossref and DataCite relation claims persist through caches, evidence snapshots, and local indexes.
- Metadata fixes are proposed with source and evidence-group provenance.
- Immutable evidence snapshots can be replayed without provider requests.
- Reviewer decisions are stored apart from source evidence in local SQLite.
- Exact-ID review annotations can be shown on later saved runs without changing raw findings, verdicts, reports, or exit codes.
- Review annotations retain an immutable ID, source-result digest, reference position, source-run instance, and availability state.
- Timeouts, rate limits, invalid responses, and missing records remain visible in the result.
et al. is a sentinel, not an author name. Citeprobe checks the explicit names
before it and does not pretend that the omitted names were supplied.
verified means the selected policy received enough evidence and every
applicable selected check passed.
problematic means at least one error-level contradiction was found.
inconclusive means the available evidence was not enough. A network failure
or empty search does not become a fabrication claim.
not_applicable means the input has a URL but no bibliographic metadata to
judge. URL reachability still receives its own check result.
A warning does not turn a verdict into problematic. A warning tied to a
contradicted check can make the verdict inconclusive. --fail-on warning
can still return a nonzero status for automated checks.
| Source | Direct IDs | Search | Author list | Access |
|---|---|---|---|---|
| Crossref | DOI | Title | May be partial | Public; CITEPROBE_MAILTO is recommended |
| DataCite | DOI | Title | Complete when fully typed | Public; CITEPROBE_MAILTO identifies scripted requests |
| Europe PMC | PMID, PMCID, DOI | Title | Complete when structured and fully parsed | Public |
| OpenAlex | DOI, PMID | Title | Complete below its 100-author cap when fully parsed | Public; OPENALEX_API_KEY raises the free daily budget |
| Semantic Scholar | DOI, arXiv, PMID, S2 | Title | Complete when supplied | Public; SEMANTIC_SCHOLAR_API_KEY is optional |
| DBLP | None | Title plus first cited author | Treated as partial | Public |
| OpenReview | None | Paginated v1 and v2 title search | Complete for fully parsed native notes | Public |
| arXiv | arXiv | Title | Complete when supplied | Public |
| ACL Anthology | Anthology ID | Title | Complete when parsed | Public |
| IACR | IACR record | Title | Complete when parsed | Public |
| PubMed | PMID, PMCID, DOI | Title | Complete when structured | Public; NCBI_API_KEY is optional |
| DOI resolver | DOI | None | Partial | Public |
| Open Library | ISBN | Title | Treated as partial | Public |
| CourtListener | Cluster ID | Title | Partial | COURTLISTENER_API_TOKEN |
| GovInfo | Package ID | Title | Partial | GOVINFO_API_KEY |
| Local index | Stored IDs | Title | Preserved from evidence | Local SQLite |
Crossref documents its public and polite access pools. DataCite documents its public REST API. Europe PMC documents its REST search service. OpenAlex documents keyless and free-key budgets. The other adapters use the official Semantic Scholar Academic Graph API and arXiv API. DBLP documents its publication search API. OpenReview publishes its API v2 definition.
Run citeprobe doctor to see configuration status without printing secret
values. Run citeprobe sources to inspect the active source capabilities.
from citeprobe import verify
result = verify("doi:10.1038/nphys1170")
print(result.verdict)
for finding in result.findings:
print(finding.code, finding.severity, finding.message)The result is an immutable typed object. result.to_dict() returns the stable
JSON-ready schema. Async programs can call verify_async() without starting a
second event loop.
Reports can be written as text, JSON, JSON Lines, CSV, Markdown, HTML, or SARIF. HTML reports are self-contained and escape imported reference data.
Corrected BibTeX is a separate artifact, not a report. It applies only
decisions marked suggest, records every applied decision, and leaves
review decisions as comments. The source file is never changed:
from citeprobe import render_corrected_bibtex
exported = render_corrected_bibtex((result,))
print(exported.applied_count, exported.review_count)- How evidence and author checks work
- Command line reference
- Python API examples
- Source access and data limits
- PDF extraction design
- Manuscript inspection
- LaTeX revision analysis
- Local workbench
- Prior-work feature coverage
- Accuracy and performance gates
- Issue code reference
- Current limits and planned work
python -m venv .venv
.venv\Scripts\python -m pip install -e ".[dev]"
.venv\Scripts\python scripts\install_hooks.py
.venv\Scripts\python scripts\quality_gate.pyOn macOS or Linux, replace .venv\Scripts\python with
.venv/bin/python. The tracked pre-push hook runs the same local gate before
every push. It uses local compute and does not start a GitHub Actions job. See
CONTRIBUTING.md before opening a change.
Citeprobe is available under the MIT License.