5,399,281 AI resources indexedWorking toward the world’s largest AI resource index.

Browse resources 13 sections
PDF

Typhoon: Thai Large Language Models

Typhoon is a series of Thai large language models (LLMs) developed specifically for the Thai language. This technical report presents challenges and insights in developing Thai LLMs, including data preparation, pretraining, instruction-tuning, and evaluation. As one of the challenges of low-resource languages is the amount of pretraining data, we apply continual training to transfer existing world knowledge from a strong LLM. To evaluate the Thai knowledge encapsulated in each model from the pretraining stage, we develop ThaiExam, a benchmark based on examinations for high-school students and investment professionals in Thailand. In addition,

By Pipatanakul, Kunat, Jirabovonvisut, Phatrasek, Manakul, Potsawee, Sripaisarnmongkol, Sittipong, Patomwong, Ruangsak, Chokchainant, Pathomporn, Tharnpipitchai, KasimaResearch paper

Resource information

Authors
Pipatanakul, Kunat, Jirabovonvisut, Phatrasek, Manakul, Potsawee, Sripaisarnmongkol, Sittipong, Patomwong, Ruangsak, Chokchainant, Pathomporn, Tharnpipitchai, Kasima
Research area
Artificial intelligence
Research topics
Computation and Language (cs.CL), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences
Publication date
2023
DOI
10.48550/arxiv.2312.13951
Upstream last modified
Dec 21, 2023
License
Creative Commons Attribution 4.0 International
Last record update
Nov 5, 2026

About this resource

Typhoon is a series of Thai large language models (LLMs) developed specifically for the Thai language. This technical report presents challenges and insights in developing Thai LLMs, including data preparation, pretraining, instruction-tuning, and evaluation. As one of the challenges of low-resource languages is the amount of pretraining data, we apply continual training to transfer existing world knowledge from a strong LLM. To evaluate the Thai knowledge encapsulated in each model from the pretraining stage, we develop ThaiExam, a benchmark based on examinations for high-school students and investment professionals in Thailand. In addition, we fine-tune Typhoon to follow Thai instructions, and we evaluate instruction-tuned models on Thai instruction datasets as well as translation, summarization, and question-answering tasks. Experimental results on a suite of Thai benchmarks show that Typhoon outperforms all open-source Thai language models, and its performance is on par with GPT-3.5 in Thai while having only 7 billion parameters and being 2.62 times more efficient in tokenizing Thai text.