Management Agreement and Guideline-Citation Metadata Errors in Large Language Models for Small Renal Mass Management: A Prompt-Sensitivity Evaluation
Analysis scripts accompanying the study "Management Agreement and Guideline-Citation Metadata Errors in Large Language Models for Small Renal Mass Management: A Prompt-Sensitivity Evaluation." The scripts reproduce the statistical analyses reported in the manuscript, including inter-rater reliability (Cohen's kappa), model comparisons (Cochran's Q and exact McNemar tests with Bonferroni-adjusted pairwise p values), a scenario-level Monte Carlo permutation test, and an exploratory, hypothesis-generating cluster-robust logistic regression. The study evaluated ChatGPT-5.5, Gemini 3.6 Flash, and Claude Sonnet 5 across 30 standardized small renal
Resource information
- Format
- py
- Original repository
- 23185535
- DOI
- 10.5281/zenodo.23185535
- Credit basis
- Creators in Zenodo record metadata
- Access
- Open files available from the original Zenodo deposit
- Last checked
- Nov 5, 2026
- License
- cc-by-4.0
- Last record update
- Nov 5, 2026
About this resource
Analysis scripts accompanying the study "Management Agreement and Guideline-Citation Metadata Errors in Large Language Models for Small Renal Mass Management: A Prompt-Sensitivity Evaluation." The scripts reproduce the statistical analyses reported in the manuscript, including inter-rater reliability (Cohen's kappa), model comparisons (Cochran's Q and exact McNemar tests with Bonferroni-adjusted pairwise p values), a scenario-level Monte Carlo permutation test, and an exploratory, hypothesis-generating cluster-robust logistic regression. The study evaluated ChatGPT-5.5, Gemini 3.6 Flash, and Claude Sonnet 5 across 30 standardized small renal mass scenarios (90 model responses), plus a sensitivity experiment with a modified citation prompt.