This page is lecture material by Princeton University professors Arvind Narayanan and Sayash Kapoor on the difficulties of evaluating large language models (LLMs), and they are also the authors of the book and newsletter AI Snake Oil. The lecture is titled "Evaluating LLMs Is a Minefield."
Purpose: The lecture aims to expose the various pitfalls and challenges encountered when evaluating large language models, helping researchers and practitioners understand the limitations of current evaluation methods.
Main capabilities or topics:
- Prompt sensitivity: Does it measure the model's intrinsic capability or an illusion caused by the prompt?
- Construct validity: Does the evaluation truly measure the concept it intends to measure?
- Data contamination: Has the model merely memorized the answers?
The lecture unfolds through specific cases:
- Has ChatGPT gotten worse over time? It cites the arXiv 2023 paper by Chen, Zaharia, Zou and the article by Narayanan & Kapoor on AI Snake Oil. The conclusion is that there is no evidence of capability degradation, but behavior changes with prompts, requiring slightly different prompts to elicit capabilities.
- Does ChatGPT have a liberal bias? It cites the Public Choice 2023 paper by Motoki et al. and the response by Narayanan & Kapoor. It points out that the original paper used multiple-choice questions and the technique of forcing the model to take a stance, arguing that the political bias of chatbots may not be a construct that exists independently of the user group. It proposes the necessity of natural observation, such as companies publishing transparency reports and researchers creating corpora of real usage.
- Did GPT-4 pass the bar exam and the United States Medical Licensing Examination? It cites the article by Narayanan & Kapoor. It questions whether it merely memorized the answers and provides contamination evidence: perfect on coding benchmarks before September 5, 2021, and zero after; but for legal and medical benchmarks it cannot be determined. Construct validity problem: exams designed for humans measure capabilities that can generalize to the real world; when used for LLMs, they can hardly say anything.
How to use: The page is a lecture slide/content presentation, with no specific installation commands or code.
Applicable scenarios: Suitable for AI researchers, evaluators, policymakers, and readers interested in the methodology of LLM evaluation.
Limitations: The lecture points out that evaluating LLMs is itself fraught with difficulties, with problems such as prompt sensitivity, construct validity, and contamination; closed-source LLMs also bring reproducibility obstacles; and there is policy controversy over requiring AI licensing.Proposal. The lecture also mentions the reproducibility crisis in ML science, with systematic reviews finding that a large number of ML-based studies are flawed. In addition, it discusses that LLMs should not be used to evaluate grant applications, because the models focus on text style rather than scientific content.
The page also introduces an open AI path: the Princeton Language and Intelligence (PLI) initiative, dedicated to keeping AI expertise in the public domain, and continues to discuss this topic in the AI Snake Oil newsletter.
Note: The specific lecture video, slide files, or further details are not yet provided.
