python-performance-optimization

Source Notice: The content of this skill is derived from wshobson/agents (MIT License, Copyright (c) 2024 Seth Hobson; the full license text is in the same directory LICENSE.txt). Use, modification, and redistribution must comply with the terms of this MIT License and retain the copyright notice.

Chinese Description

This skill provides best practices and pattern guidance related to full-stack development.

---

name: python-performance-optimization description: Profile and optimize Python code using cProfile, memory profilers, and performance best practices. Use when debugging slow Python code, optimizing bottlenecks, or improving application performance.

Python Performance Optimization

Comprehensive guide to profiling, analyzing, and optimizing Python code for better performance, including CPU profiling, memory optimization, and implementation best practices.

When to Use This Skill

• Identifying performance bottlenecks in Python applications
• Reducing application latency and response times
• Optimizing CPU-intensive operations
• Reducing memory consumption and memory leaks
• Improving database query performance
• Optimizing I/O operations
• Speeding up data processing pipelines
• Implementing high-performance algorithms
• Profiling production applications

Core Concepts

1. Profiling Types

• CPU Profiling: Identify time-consuming functions
• Memory Profiling: Track memory allocation and leaks
• Line Profiling: Profile at line-by-line granularity
• Call Graph: Visualize function call relationships

2. Performance Metrics

• Execution Time: How long operations take
• Memory Usage: Peak and average memory consumption
• CPU Utilization: Processor usage patterns
• I/O Wait: Time spent on I/O operations

3. Optimization Strategies

• Algorithmic: Better algorithms and data structures
• Implementation: More efficient code patterns
• Parallelization: Multi-threading/processing
• Caching: Avoid redundant computation
• Native Extensions: C/Rust for critical paths

Quick Start

Basic Timing

import time

def measure_time():
    """Simple timing measurement."""
    start = time.time()

    # Your code here
    result = sum(range(1000000))

    elapsed = time.time() - start
    print(f"Execution time: {elapsed:.4f} seconds")
    return result

# Better: use timeit for accurate measurements
import timeit

execution_time = timeit.timeit(
    "sum(range(1000000))",
    number=100
)
print(f"Average time: {execution_time/100:.6f} seconds")

Detailed patterns and worked examples

Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.

Best Practices

1. Profile before optimizing - Measure to find real bottlenecks
2. Focus on hot paths - Optimize code that runs most frequently
3. Use appropriate data structures - Dict for lookups, set for membership
4. Avoid premature optimization - Clarity first, then optimize
5. Use built-in functions - They're implemented in C
6. Cache expensive computations - Use lru_cache
7. Batch I/O operations - Reduce system calls
8. Use generators for large datasets
9. Consider NumPy for numerical operations
10. Profile production code - Use py-spy for live systems

Common Pitfalls

• Optimizing without profiling
• Using global variables unnecessarily
• Not using appropriate data structures
• Creating unnecessary copies of data
• Not using connection pooling for databases
• Ignoring algorithmic complexity
• Over-optimizing rare code paths
• Not considering memory usage

Capability Boundaries

✅ Good at handling

• Best practices related to full-stack development
• Code architecture design guidance
• Performance optimization suggestions

⚠️ Requires materials

• Specific project context
• Technology stack information

❌ Out of scope

• Business decisions in non-technical domains
• Specific code implementation (guidance only)

Workflow

Step 1: Requirements Analysis

Understand project requirements and technical constraints

Step 2: Architecture Design

Design system architecture and component division

Step 3: Implementation Guidance

Provide specific implementation suggestions and code patterns

Step 4: Code Review

Review code quality and adherence to best practices

Step 5: Optimization Suggestions

Provide performance optimization and improvement suggestions

Gotchas

1. Over-engineering

Avoid using complex design patterns in simple scenarios

2. Ignoring Performance

Do not ignore performance optimization, especially in production environments

3. Security Considerations

Always consider security and data protection

4. Test Coverage

Ensure sufficient test coverage, especially for edge cases

5. Documentation Maintenance

Keep documentation updated in sync with the code