AI Humanizer Benchmark is a public benchmark for evaluating the performance of AI humanizer tools in bypassing AI detectors.
Purpose The benchmark aims to provide independent, verifiable monthly rankings of AI humanizer tools. It uses a unified testing method to compare how different tools rewrite AI-generated text into a human writing style while also evaluating rewrite quality.
Main Capabilities Each month, the benchmark generates new test texts covering 7 writing categories (such as essays, emails, blog posts, marketing copy, etc.). Each participating tool uses default settings and rewrites the same 33 texts via API calls. Then, 7 commercial AI detectors (GPTZero, Originality.ai, Copyleaks, Winston AI, ZeroGPT, QuillBot, and Grammarly) score each rewritten text. In addition, semantic similarity (meaning preservation) and readability as judged by a language model are measured.
Scoring combines four dimensions: bypass rate against detectors (42%), meaning preservation (32%), readability (16%), and consistency across writing types (10%). At the same time, deductions are made for quality issues (such as deviation from the original meaning, significant expansion or reduction in output length, refusal to process, etc.). This results in a final total score (out of 100, higher is better).
The benchmark also provides multiple breakdown ranking views, such as: best quality, best meaning preservation, best readability, best bypass rate, best for students, best for essays, rankings for specific detectors (such as GPTZero, Originality.ai), and more.
How to Use Users can view the current rankings, detailed scores for each tool, and individual rankings for each detector on the website. All raw data (inputs, outputs, detector judgments, and scoring code) is publicly available on GitHub, and anyone can inspect and recalculate the results. The website also provides a FAQ and accepts score dispute submissions.
Use Cases The benchmark is suitable for users who need to understand the performance of AI humanizer tools, such as students, content creators, marketers, academic writers, and others. It helps users choose tools that fit specific needs, such as for a particular detector or writing type.
Limitations The benchmark is run by the team that develops UndetectedGPT, and UndetectedGPT also participates in the rankings, creating a potential conflict of interest. To mitigate this, all data is publicly auditable, and UndetectedGPT uses the same pipeline and scoring criteria as other tools. Testing is based only on default settings and does not reflect performance after custom adjustments. Turnitin cannot be tested automatically, so Originality.ai is used as a substitute. Rankings are updated monthly and may change.
