3d-camera-interaction

3D Camera Interaction: Dragging and Boundary Detection

Symptoms

• After zooming, when dragging the model, the mouse moves 100px but the screen distance the model moves is not 100px
• After enlarging the model, only part of the leg/body can be seen, and it cannot be panned normally
• The model position "jumps" when dragging starts

Root Causes

Cause 1: A fixed panSpeed causes movement to be out of sync

Problem: Using a fixed panSpeed = 0.01 for pan calculations

// ❌ 错误方式
const panSpeed = 0.01;
newPosition.add(right.multiplyScalar(deltaX * panSpeed));

Why it happens: When the camera distance changes, the same world-space distance appears as different numbers of pixels on screen. When the distance is close, there are more pixels; when the distance is far, there are fewer pixels.

Solution: Dynamically calculate the pixel-to-world-space mapping based on the camera distance and FOV

// ✅ 正确方式：动态计算
const cameraDistance = camera.position.distanceTo(modelCenter);
const fov = camera.fov * (Math.PI / 180);
const screenHeight = renderer.domElement.clientHeight;
const screenWidth = renderer.domElement.clientWidth;

// 在相机距离处，视口的世界空间高度
const worldHeight = 2 * Math.tan(fov / 2) * cameraDistance;
const worldWidth = worldHeight * (screenWidth / screenHeight);

// 每像素对应的世界空间距离
const pixelToWorldX = worldWidth / screenWidth;
const pixelToWorldY = worldHeight / screenHeight;

// 应用：鼠标移动的像素 × 每像素对应的世界空间距离
newPosition.add(right.multiplyScalar(deltaX * pixelToWorldX));
newPosition.add(up.multiplyScalar(-deltaY * pixelToWorldY));

Cause 2: Boundary constraints based on the center point

Problem: Using the NDC coordinates of the model's center point to determine whether it is out of bounds

// ❌ 错误方式：限制中心点位置
const ndc = position.clone().project(camera);
if (ndc.y > 0.2) clampedY = 0.2; // 限制顶部

Why it happens: After the model is enlarged, the center point is at the center of the screen, but most of the body is already outside the screen. Constraining the center point = constraining it so that only the middle part of the body can be seen.

Solution: Calculate the model's visible area on screen (in pixels), and only correct it when the visible area is too small

// ✅ 正确方式：基于可见像素
const MIN_VISIBLE_PIXELS = 50;

// 1. 计算模型包围盒并投影到屏幕
const box = new THREE.Box3().setFromObject(vrm.scene);
const corners = [/* 8个顶点 */];

let modelMinX = Infinity, modelMaxX = -Infinity;
let modelMinY = Infinity, modelMaxY = -Infinity;

corners.forEach(corner => {
    const projected = corner.clone().project(camera);
    const screenX = (projected.x * 0.5 + 0.5) * screenWidth;
    const screenY = (-projected.y * 0.5 + 0.5) * screenHeight;
    // 更新边界...
});

// 2. 计算可见区域
const visibleWidth = Math.max(0, Math.min(screenWidth, modelMaxX) - Math.max(0, modelMinX));
const visibleHeight = Math.max(0, Math.min(screenHeight, modelMaxY) - Math.max(0, modelMinY));
const visiblePixels = visibleWidth * visibleHeight;

// 3. 只在可见区域太小时校正
if (visiblePixels < MIN_VISIBLE_PIXELS) {
    // 将模型拉回可见区域
}

Key Formulas

Pixel-to-world-space conversion

worldHeight = 2 × tan(fov/2) × cameraDistance
pixelToWorld = worldHeight / screenHeight

World coordinates to screen coordinates

const ndc = worldPos.clone().project(camera);
const screenX = (ndc.x * 0.5 + 0.5) * screenWidth;
const screenY = (-ndc.y * 0.5 + 0.5) * screenHeight; // Y 轴反向

Key Lessons

• 📐 Camera distance affects everything: All pixel↔world-space conversions need to take camera distance into account
• 🔲 Use bounding boxes instead of center points: Boundary detection should be based on the screen area actually occupied by the model
• 🔄 Keep consistent with 2D: Different types of models such as Live2D/VRM should use the same interaction logic thresholds