5,427,056 AI resources indexedWorking toward the world’s largest AI resource index.

PDF

Research papers

Publications about AI methods and applications.

How to use the library

Resources

Search results
Search stays within research papers.
Research paper

MEDITRON-70B: Scaling Medical Pretraining for Large Language Models

Large language models (LLMs) can potentially democratize access to medical knowledge. While many efforts have been made to harness and improve LLMs' medical knowledge and reasoning capacities, the resulting models are either closed-source (e.g., PaLM, GPT-4) or limited in scale (<= 13B parameters), which restricts their abilities. In this work, we improve access to large-scale medical LLMs by releasing MEDITRON: a suite of open-source LLMs with 7B and 70B parameters adapted to the medical domain. MEDITRON builds on Llama-2 (through our adaptation of Nvidia's Megatron-LM distributed trainer), and extends pretraining on a comprehensively curate

Chen, Zeming, Cano, Alejandro Hernández, Romanou, Angelika, Bonnet, Antoine, Matoba, Kyle, Salvi, Francesco, Pagliardini, Matteo, Fan, Simin, Köpf, Andreas, Mohtashami, Amirkeivan, Sallinen, Alexandre, Sakhaeirad, Alireza, Swamy, Vinitra, Krawczuk, Igor, Bayazit, Deniz, Marmet, Axel, Montariol, Syrielle, Hartley, Mary-Anne, Jaggi, Martin, Bosselut, Antoine
Creative Commons Attribution 4.0 InternationalView resource

Platform labels describe intended use. Read each resource's author credits, requirements and available access methods.