stanford-crfm/helm: v0.5.16

Breaking Changes meta/llama-3.1-405b-instruct-turbo will no longer be used as a LLM-as-judge in Omni-MATH, WildBench, HarmBench, XSTest, and Anthropic Red Team scenarios due to deprecation by Together AI (#4233) Models Deprecate meta/llama-3.1-405b-instruct-turbo (#4233) Scenarios Release Arabic Legal scenario (#4226, #4236, #4237) Release Arabic Content Generation scenario (#4224, #4229, #4236, #4237) Migrate Unitxt integration to use new Unitxt API (#4234) Remove meta/llama-3.1-405b-instruct-turbo from annotators (#4233) Fix AutobencherSafetyScenario parsing the file path string as JSON (#4206) Contributors Thank you to the following contri

By Tony Lee, Yifan Mai, Percy Liang, Dilara Soylu, Rishi Bommasani, Dimitris Tsipras, Yian Zhang, Deepak Narayanan, Josselin Somerville Roberts, Ryan Chi, Eric Zelikman, Farzaan Kaiyom, lxuechen, Frieda Rong, Brian W. Goldman, Drew Arad Hudson, Ananya Kumar, MiguelAFH, Yuhui Zhang, Ben Newman, Nathan Kim, Keshav Santhanam, Haoqin Tu, fladhak, Huaxiu Yao, Tianyi, Qian Huang, Michi YasunagaToolReference information

Resource information

Format: zip
Original repository: 19907377
DOI: 10.5281/zenodo.19907377
Credit basis: Creators in Zenodo record metadata
Access: Open files available from the original Zenodo deposit
Last checked: Nov 5, 2026
License: apache2.0
Last record update: Nov 5, 2026

Resource information

About this resource