What is LOKI
LOKI is a synthetic data detection benchmark jointly proposed by Sun Yat-sen University and Shanghai AI Lab, aimed at comprehensively evaluating the ability of Large Multimodal Models (LMMs) to identify synthetic data across multiple modalities such as video, image, 3D, text, and audio. It contains over 18,000 questions, covers 26 subcategories, adopts multi-level annotations, and supports fine-grained anomaly annotations. LOKI tests the perception and reasoning abilities of models, and enhances model interpretability through natural language explanations. By evaluating 22 open-source and 6 closed-source LMMs, LOKI reveals the potential and limitations of these models in synthetic data detection tasks.
Main Features of LOKI
- Multimodal Data Detection: Evaluates the ability of LMMs to identify synthetic video, image, 3D model, text, and audio data.
- Fine-Grained Anomaly Annotation: Provides detailed anomaly annotations, supporting in-depth analysis and understanding of synthetic data.
- Multi-Level Annotation: Includes basic 'synthetic or real' labels, suitable for basic question settings, as well as more complex anomaly detail selection and explanation tasks.
- Comprehensive Evaluation Framework: Supports input in multiple data formats, such as video, image, text, audio, and point cloud, and unifies the APIs of over 25 mainstream LMMs.
- Performance Comparison: Supports comparison among different LMMs, including open-source and closed-source models, and expert synthetic detection models.
- Interpretability Testing: Tests the interpretability of LMMs in synthetic data detection tasks by requiring models to provide natural language explanations.
- Data Diversity: Collects various types of synthetic data, including data from professional domains, such as satellite images and medical images, and audio data such as ambient sounds and music.
- Question Difficulty Grading: Grades question difficulty according to human evaluation metrics, testing the performance of LMMs at different difficulty levels.
- Model Bias Analysis: Analyzes the bias and tendencies of models in synthetic data detection tasks by calculating model bias indices.
-
Promoting AI Development: Advances the development of more powerful and more interpretable synthetic data detection methods to address the challenges brought by AI synthesis technologies.
Technical Principles of LOKI
- Data Collection and Synthesis: LOKI collects data of multiple modalities, including video, image, 3D model, text, and audio. Some of this data comes from public datasets, and some is generated by the latest synthesis models.
- Multimodal Evaluation Framework: LOKI proposes a comprehensive multimodal evaluation framework that supports input in multiple data formats, unifies the APIs of multiple mainstream LMMs, and evaluates the performance of different models under a unified standard.
- Model Evaluation and Comparison: The LOKI benchmark includes evaluations of multiple open-source and closed-source LMMs. By comparing the performance of these models on synthetic data detection tasks, their performance and limitations can be analyzed.
- Natural Language Explanation: LOKI requires models to provide natural language explanations, enhancing model interpretability. It tests the detection ability of models and evaluates the reasons models give to explain their judgments.
LOKI Project Address
- Project Official Website：opendatalab.github.io/LOKI
- Github Repository：https://github.com/opendatalab/LOKI
- arXiv Technical Paper：https://arxiv.org/pdf/2410.09732
Application Scenarios of LOKI
- AI Safety Assessment: LOKI can be used to assess and improve the safety and robustness of AI systems when processing synthetic data, ensuring that AI systems can accurately identify and respond correctly when facing potential synthetic data attacks.
- Content Moderation: On social media, news websites, and other content platforms, LOKI can help detect and filter out AI-generated fake news, deepfake videos, or audio, protecting users from being misled.
- Dataset Validation: During machine learning model training, LOKI can be used to validate the quality and authenticity of datasets, ensuring that the training data does not contain excessive synthetic data and improving the model's generalization ability.
- Law and Compliance: In the legal field, LOKI can help identify and address copyright, privacy, and compliance issues related to synthetic data, for example, detecting and preventing unauthorized content generation and distribution.
- Media and Entertainment: In film, game, and virtual reality production, LOKI can be used to assess and improve the quality of synthetic media content, ensuring that the generated content is both realistic and consistent with the creator's intent.