LMExamQA is a benchmark leaderboard for foundation models, whose core feature is the adoption of the "Language-Model-as-an-Examiner" evaluation approach, that is, letting a language model act as an examiner to evaluate the model under test.
Purpose: To provide a reference for comparison and ranking of foundation model capabilities.
Main capability: Benchmark evaluation and leaderboard display with a language model as the examiner.
Usage: It can be accessed through its website https://lmexam.com/ . The specific operation process is not yet provided.
Applicable scenarios: Scenarios where there is a need to understand or compare the evaluation performance of foundation models.
Limitations: Apart from the above positioning, the evaluation dimensions, question types, scoring criteria, range of covered models, data update frequency, and fee information are all not yet provided.
