A Comparative Benchmark of a Moroccan Darija Toxicity Detection Model (Typica.ai) and Major LLM-Based Moderation APIs (OpenAI, Mistral, Anthropic)
Research paper by Hicham Assoudi. arXiv record dated 2025-05-05. Research area: Language technology.
Resource information
- Authors
- Hicham Assoudi
- Research area
- Language technology
- Application domain
- General AI research
- AI methods
- Large language models
- Research topics
- cs.CL
- arXiv record date
- 2025-05-05
- Publication
- arXiv
- Upstream last modified
- May 9, 2025
- License
- CC BY 4.0
- Last record update
- Nov 4, 2026
Abstract
This paper presents a comparative benchmark evaluating the performance of Typica.ai's custom Moroccan Darija toxicity detection model against major LLM-based moderation APIs: OpenAI (omni-moderation-latest), Mistral (mistral-moderation-latest), and Anthropic Claude (claude-3-haiku-20240307). We focus on culturally grounded toxic content, including implicit insults, sarcasm, and culturally specific aggression often overlooked by general-purpose systems. Using a balanced test set derived from the OMCD_Typica.ai_Mix dataset, we report precision, recall, F1-score, and accuracy, offering insights into challenges and opportunities for moderation in underrepresented languages. Our results highlight Typica.ai's superior performance, underlining the importance of culturally adapted models for reliable content moderation.