GrepBoard/Manuals
51 manuals ·

Verify an AI answer's citations with Crossref and PubMed

·Claude Code 2.1.302 · claude-sonnet-5-5 · curl · Python 3 (2026-10-19)·6 steps

Check the DOIs in an AI answer on Crossref for existence, title, author and year, then read the PubMed abstract to confirm the paper supports the claim.

Before you cite the references in an AI answer, check three things: the paper exists, its bibliographic data is right, and it actually supports the claim.

[1] Ask a question that demands sources with DOIs, then extract the DOIs.

console
$ claude -p "Does intermittent fasting improve weight loss compared with continuous calorie restriction? Answer in 4 sentences, and cite 5 peer-reviewed sources with authors, year, title, journal and DOI." > answer.txt$ grep -o 'DOI: [^ ]*' answer.txt | sed 's/DOI: //' > dois.txt$ cat dois.txt10.1001/jamainternmed.2017.093610.1001/jamainternmed.2020.415310.1056/NEJMoa211483310.1186/s12967-018-1748-410.1001/jamanetworkopen.2021.39558

[2] Query Crossref for each DOI. A DOI that does not exist returns HTTP 404 and Resource not found.; 200 means it is registered.

console
$ curl -s -w ' [HTTP %{http_code}]\n' -o body.json "https://api.crossref.org/works/10.1056/NEJMoa2114833" [HTTP 200]

[3] Pull only the DOI, title, first author, year and journal from each response, and compare them with the citation in the answer. q.py:

q.py
import sys,jsond=json.load(sys.stdin)["message"]print(d["DOI"]," | ",d["title"][0][:90]," | ",d["author"][0]["family"]," | ",d["issued"]["date-parts"][0][0]," | ",d["container-title"][0])
console
$ for d in $(cat dois.txt); do curl -s "https://api.crossref.org/works/$d" | python3 -I q.py; done10.1001/jamainternmed.2017.0936  |  Effect of Alternate-Day Fasting on Weight Loss, Weight Maintenance, and Cardioprotection A  |  Trepanowski  |  2017  |  JAMA Internal Medicine10.1001/jamainternmed.2020.4153  |  Effects of Time-Restricted Eating on Weight Loss and Other Metabolic Parameters in Women a  |  Lowe  |  2020  |  JAMA Internal Medicine10.1056/nejmoa2114833  |  Calorie Restriction with or without Time-Restricted Eating in Weight Loss  |  Liu  |  2022  |  New England Journal of Medicine10.1186/s12967-018-1748-4  |  Intermittent versus continuous energy restriction on weight loss and cardiometabolic outco  |  Cioffi  |  2018  |  Journal of Translational Medicine10.1001/jamanetworkopen.2021.39558  |  Intermittent Fasting and Obesity-Related Health Outcomes  |  Patikorn  |  2021  |  JAMA Network Open

All five DOIs existed, and title, first author, year and journal matched the answer. That only means no fakes turned up this time.

[4] For a suspect DOI, also open https://doi.org/<doi>. The second DOI below is altered on purpose (last digit changed for this test; the AI did not produce it).

console
$ curl -s -o /dev/null -w "%{http_code}\n" https://doi.org/10.1056/NEJMoa2114833302$ curl -s -w ' [HTTP %{http_code}]\n' https://api.crossref.org/works/10.1056/NEJMoa2114834Resource not found. [HTTP 404]$ curl -s -o /dev/null -w "%{http_code}\n" https://doi.org/10.1056/NEJMoa2114834404

[5] Finally, compare the abstract with the claim. Crossref rarely returns abstracts (the two records checked had no abstract field), so find the PMID from the DOI in PubMed and fetch the abstract.

console
$ curl -s "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=10.1056/NEJMoa2114833%5Baid%5D&retmode=json" | python3 -I -c 'import sys,json;print(json.load(sys.stdin)["esearchresult"]["idlist"])'['35443107']$ curl -s "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=35443107&rettype=abstract&retmode=text" | grep -A4 '^RESULTS'RESULTS: Of the total 139 participants who underwent randomization, 118 (84.9%) completed the 12-month follow-up visit. The mean weight loss from baseline at 12 months was -8.0 kg (95% confidence interval [CI], -9.6 to -6.4) in the time-restriction group and -6.3 kg (95% CI, -7.8 to -4.7) in the

The abstract reports a 12-month net difference of −1.8 kg (95% CI −4.0 to 0.4; P = 0.11), not significant. That points the same way as the AI's statement that the gap is usually within 1–2 kg and rarely clinically meaningful.

[6] Record one line per citation: DOI · exists (200/404) · bibliographic match · claim match (abstract checked / not checked). If any citation is unverified, leave it out of your text or label it "unverified".

References: Crossref REST API · DOI Handbook · NCBI E-utilities

#research#citations#doi#verification