SuperCLUE
SuperCLUE是由CLUE学术社区推出的中文通用大模型综合性评测基准，旨在从基础能力、专业能力和中文特性能力三个维度全面评估模型表现。
In today’s rapidly developing AI era, objectively and fairly evaluating various large models has become a focal point in the industry. AGI-Eval, jointly launched by Shanghai Jiao Tong University, Tongji University, East China Normal University, and DataWhale, is a large model evaluation community established to meet this demand.
- Public Academic: Industry public academic evaluation sets, available for user download.
- Official Evaluation Sets: Officially built evaluation sets covering multi-domain model evaluations.
- User-Built Evaluation Sets: The platform supports users uploading personal evaluation sets to co-build an open-source community, perfectly combining automated and manual evaluations; it also offers private data set hosting for top academics.
- High User Activity: A platform with over 30,000 crowd-sourced users, achieving more high-quality real data collection.
- Diverse Data Types: Possesses multi-dimensional, multi-domain professional data.
- Diverse Data Collection: Methods such as single data points, expanded data, Arena data, etc., meet different evaluation needs.
- Complete Review Mechanism: Multiple review mechanisms combining machine and human reviews ensure data quality.
Relevant Navigation
MMLU
MMLU (Massive Multitask Language Understanding) is a benchmark test launched by the University of California, Berkeley in September 2020, aiming to comprehensively evaluate large language models' multitask understanding across 57 different domains.
CMMLU
CMMLU是一个专为中文语境设计的综合性评估基准，涵盖67个主题，旨在全面测试语言模型的知识储备和推理能力。
Stable Chat
Stable Chat是Stability AI最新推出的对话式AI助手，基于Stable Beluga大语言模型，旨在为研究人员和AI爱好者提供评估模型功能和安全性的研究平台。
HuggingFace
Hugging Face is a company focused on artificial intelligence and machine learning, offering a wealth of open-source tools and platforms to assist developers in building and deploying AI applications. Its core products include the Transformers library, Hugging Face Hub, and Gradio, supporting various deep learning frameworks, and committed to promoting the popularization and innovation of AI technology.
OpenCompass
OpenCompass是由上海人工智能实验室推出的开源大模型评测体系，提供全面、高效的评测框架，支持大语言模型和多模态模型的一站式评测，并定期公布评测结果榜单。
HELM
HELM (Holistic Evaluation of Language Models) is a comprehensive evaluation system for language models introduced by Stanford University, aiming to assess the performance and characteristics of language models through standardized datasets, unified model interfaces, and multidimensional evaluation metrics.
Devin
Devin, developed by Cognition, is the world's first fully autonomous AI software engineer, capable of self-learning, end-to-end application development and deployment, and autonomously identifying and fixing code bugs. It has demonstrated outstanding performance in the SWE-bench benchmark, surpassing other AI models.