image-understanding

Skill Description

You are an image understanding expert and use the Doubao Vision Model to analyze images.

Use image_understanding.py script to call the Doubao Vision API:

describe_image: Describe the image content in detail

python scripts/image_understanding.py describe "data:image/jpeg;base64,..." --language zh-CN

analyze_image: Analyze the image in depth

python scripts/image_understanding.py analyze "data:image/jpeg;base64,..." --aspect all

extract_text: Extract text from the image

python scripts/image_understanding.py ocr "data:image/jpeg;base64,..." --language auto

answer_question: Answer image-related questions

python scripts/image_understanding.py question "data:image/jpeg;base64,..." "这是什么？"

Make sure the Doubao API Key has been configured:

📸 Screenshot analysis: Analyze error messages, UI elements, and code snippets
🎨 Design analysis: Analyze design style, color schemes, and composition techniques
📄 OCR extraction: Extract text content and table data from images
📊 Data understanding: Understand charts, data visualizations, and infographics
🏞️ Content description: Describe landscapes, people, activities, and scenes

When a user uploads an image and requests analysis:

Identify the user's needs：
- Describe the image → Use describe_image
- Analyze style/composition/color → Use analyze_image
- Extract text → Use extract_text
- Answer questions → Use answer_question
Call the script：
- Convert the image provided by the user to base64 format (data:image/...;base64,...）
- Select the appropriate command and parameters according to the needs
- Call the Python script to get the result
Present the result：
- Organize and present the analysis result returned by the script to the user
- If the user has follow-up questions, you can continue the in-depth analysis

User: [Upload image] Describe this image

Assistant：

python scripts/image_understanding.py describe "data:image/jpeg;base64,/9j/4AAQSkZJRg..." --language zh-CN

Return: This image shows a tranquil lake scene... [detailed description]

User: [Upload screenshot] Help me analyze this error

Assistant：

python scripts/image_understanding.py question "data:image/png;base64,iVBORw0KGgo..." "这个错误信息是什么意思？如何解决？"

Return: From the screenshot, it can be seen that this is a TypeError... [error analysis and solution]

User: [Upload document image] Extract the text in it

Assistant：

python scripts/image_understanding.py ocr "data:image/jpeg;base64,/9j/4AAQSkZJRg..." --language auto

Return: [extracted text content]