MMBench
MMBench是由OpenCompass团队推出的多模态基准测试，旨在通过约3000道单项选择题，覆盖20项细粒度能力，全面评估视觉语言模型的综合表现。
In today’s rapidly developing AI era, comprehensively and objectively evaluating the performance of Chinese general large models has become a focus for developers and researchers. SuperCLUE emerges as a comprehensive benchmark for Chinese general large models, providing authoritative references for model capability assessment.
Website Introduction
SuperCLUE was launched by the CLUE academic community in May 2023, aiming to comprehensively evaluate the performance of Chinese large models through a multi-dimensional evaluation system, helping developers and researchers understand the strengths and weaknesses of models.
Key Features
- Basic Abilities: Including 10 capabilities such as semantic understanding, dialogue, logical reasoning, role-playing, code generation, and creation.
- Professional Skills: Involving middle school, university, and professional exams, covering more than 50 capabilities in mathematics, physics, geography, social sciences, etc.
- Chinese-Specific Features: Targeting Chinese-specific tasks, including 10 capabilities such as idioms, poetry, literature, and character forms.
Related Projects
- SuperCLUE-OPEN: Multi-turn open-ended question evaluation, assessing model performance in open dialogues.
- SuperCLUE-OPT: Objective question closed-book test, based on over 3,700 multiple-choice questions, evaluating the objective performance of models in basic, Chinese, and professional abilities.
- Langya List Anonymous Battle: Through an anonymous battle mechanism, dynamically tracking the performance differences of mainstream models domestically and internationally.
Advantages
SuperCLUE’s multi-dimensional evaluation system enables developers to comprehensively understand model performance and optimize accordingly. Additionally, the platform updates rankings monthly, keeping up with model development dynamics and providing users with the latest evaluation results.
Pricing
SuperCLUE is currently open for free, and users can access relevant information and resources through the official website or GitHub project address.
Summary
The CLUE academic community was established in 2023, located in China, dedicated to providing a comprehensive evaluation benchmark for Chinese large models. Through SuperCLUE’s innovative features, users can obtain comprehensive and objective model evaluation results, assisting in model development and optimization.
Relevant Navigation
HELM
HELM（Holistic Evaluation of Language Models）是斯坦福大学推出的开源评估框架，旨在全面、透明地评估基础模型，包括大型语言模型和多模态模型。
Chatbot Arena
Chatbot Arena is an open platform that utilizes anonymous battles and crowdsourced evaluations to compare and rank the performance of large language models (LLMs) in real-time, assisting users in selecting the AI chatbot that best fits their needs.
Stable Chat
StableChat is a free conversational AI assistant launched by Stability AI, based on the Stable Beluga large language model, designed as a research platform for researchers and AI enthusiasts to evaluate model capabilities and safety.
Stable Chat
Stable Chat是Stability AI最新推出的对话式AI助手，基于Stable Beluga大语言模型，旨在为研究人员和AI爱好者提供评估模型功能和安全性的研究平台。
OpenCompass
OpenCompass是由上海人工智能实验室推出的开源大模型评测体系，提供全面、高效的评测框架，支持大语言模型和多模态模型的一站式评测，并定期公布评测结果榜单。
MMLU
MMLU（Massive Multitask Language Understanding）是由加州大学伯克利分校于2020年9月推出的基准测试，旨在评估大型语言模型在多领域的理解和推理能力。
C-Eval
C-Eval is a Chinese foundational model evaluation suite jointly developed by Shanghai Jiao Tong University, Tsinghua University, and the University of Edinburgh. It comprises 13,948 multiple-choice questions across 52 disciplines and four difficulty levels, aiming to comprehensively assess large language models' Chinese comprehension and reasoning abilities.