MMLU
MMLU（Massive Multitask Language Understanding）是由加州大学伯克利分校于2020年9月推出的基准测试，旨在评估大型语言模型在多领域的理解和推理能力。
In today’s rapidly developing AI era, objectively and fairly evaluating various large models has become a focal point in the industry. AGI-Eval, jointly launched by Shanghai Jiao Tong University, Tongji University, East China Normal University, and DataWhale, is a large model evaluation community established to meet this demand.
- Public Academic: Industry public academic evaluation sets, available for user download.
- Official Evaluation Sets: Officially built evaluation sets covering multi-domain model evaluations.
- User-Built Evaluation Sets: The platform supports users uploading personal evaluation sets to co-build an open-source community, perfectly combining automated and manual evaluations; it also offers private data set hosting for top academics.
- High User Activity: A platform with over 30,000 crowd-sourced users, achieving more high-quality real data collection.
- Diverse Data Types: Possesses multi-dimensional, multi-domain professional data.
- Diverse Data Collection: Methods such as single data points, expanded data, Arena data, etc., meet different evaluation needs.
- Complete Review Mechanism: Multiple review mechanisms combining machine and human reviews ensure data quality.
Relevant Navigation
HELM
HELM（Holistic Evaluation of Language Models）是斯坦福大学推出的开源评估框架，旨在全面、透明地评估基础模型，包括大型语言模型和多模态模型。
AGI-Eval
AGI-Eval是由多所知名高校和机构联合打造的AI大模型评测社区，提供公正、可信、科学、全面的评测生态，旨在评估基础模型在人类认知和问题解决任务中的通用能力。
H2O EvalGPT
H2O EvalGPT is an open-source tool developed by H2O.ai, designed for evaluating and comparing large language models (LLMs). It offers a transparent and efficient platform to help users understand model performance across various tasks and benchmarks, aiding in selecting the most suitable model for specific needs.
SuperCLUE
SuperCLUE是由CLUE学术社区推出的中文通用大模型综合性评测基准，旨在从基础能力、专业能力和中文特性能力三个维度全面评估模型表现。
Open LLM Leaderboard
Open LLM Leaderboard是由Hugging Face推出的开源大语言模型（LLM）评估平台，提供模型排名、性能评估和社区协作功能，助力开发者和研究者了解和比较不同LLM的表现。
Chatbot Arena
Chatbot Arena is an open platform that utilizes anonymous battles and crowdsourced evaluations to compare and rank the performance of large language models (LLMs) in real-time, assisting users in selecting the AI chatbot that best fits their needs.
Devin
Devin, developed by Cognition, is the world's first fully autonomous AI software engineer, capable of self-learning, end-to-end application development and deployment, and autonomously identifying and fixing code bugs. It has demonstrated outstanding performance in the SWE-bench benchmark, surpassing other AI models.