ASAM - AI image segmentation model launched by vivo

What is ASAM

ASAM (Adversarial Adjustment of Segment Anything Model) is an AI image segmentation model launched by vivo, which enhances the performance of the original SAM model through adversarial adjustment. ASAM uses natural adversarial examples, augments the dataset through a stable diffusion model, and generates adversarial instances that can represent natural variations. These instances maintain photorealistic quality and align with the original mask annotations, ensuring the integrity of the segmentation task.

Main features of ASAM

1. Improved capability: ASAM enables the original SAM model to perform better on image segmentation tasks.
2. No changes required: There is no need to change the basic structure of the model, just like putting new tires on a car while the body remains unchanged.
3. Increased challenge: By adding some special images (adversarial samples), the model learns to work in more different situations.
4. More realistic: These special images look real, so the model learns in a way that is closer to reality.
5. No new data required: No additional images or data are needed, just like reviewing with old books can also learn new knowledge.
6. Comprehensive improvement: On a variety of different image segmentation tasks, ASAM can make the model perform better.

Technical principles of ASAM

1. Adversarial training: ASAM uses the concept of adversarial training, a method that improves the model's generalization ability by feeding carefully designed perturbations into the model.
2. Natural adversarial examples: ASAM pays special attention to natural adversarial examples, which are visually similar to normal samples and near the model's decision boundary.
3. Stable diffusion model: ASAM uses a stable diffusion model to augment a subset of the SA-1B dataset and generate adversarial instances. This method can introduce subtle perturbations while maintaining image quality.
4. Photorealism: The generated adversarial samples need to maintain the same visual realism as the original images, so that during training, the model will not learn incorrect features because of unrealistic samples.
5. Mask annotation alignment: Adversarial samples need to remain aligned with the original mask annotations to ensure that in the segmentation task, the model can correctly identify and segment the target objects.
6. Fine-tuning process: ASAM fine-tunes the original SAM model with these adversarial samples, improving model performance without changing the original architecture.

ASAM project address

Application scenarios of ASAM

• Medical image analysis: In the medical field, ASAM can be used to improve the accuracy of medical image segmentation and assist doctors in disease diagnosis and treatment planning.
• Autonomous vehicles: In autonomous driving technology, ASAM can be used to improve the segmentation accuracy of roads and objects, enhancing the vehicle's ability to understand its surroundings.
• Robot vision: In robotics, ASAM can help robots better understand their working environment and perform precise object recognition and operation.
• Augmented reality (AR): In AR applications, ASAM can improve the naturalness of the integration between virtual objects and the real world, providing a more immersive experience.