e2e

LoongCollector E2E Testing Guide

See detailed step templates at reference.md | See reusable scripts at scripts/

Overview
Designing Test Cases
Writing Test Cases
Running Locally (docker-compose)
Debugging
Known Pitfalls

1 Overview

Based on the BDD Godog framework, scenarios are described through .feature files, and the engine regex-matches step functions and passes parameters.

test/e2e/
  test_cases/<case_name>/
    case.feature              # 场景描述
    docker-compose.yaml       # 可选，外部依赖服务
  engine/
    steps.go                  # 所有可用步骤（权威来源）
    setup/ control/ trigger/ verify/ cleanup/

Environment tag：@host、@k8s、@docker-compose(choose one of three, add @e2e）

2 Designing Test Cases

Before writing a feature file, first determine the test matrix. Evaluate item by item whether coverage is needed according to the following dimensions:

2.1 Scenario Dimension Checklist

Dimension	Typical Scenario	When Needed
Basic functionality	Single configuration, single data type end to end	Required
Multiple data types	logs / metrics / traces verified separately	When the plugin supports multiple types
Multiple configurations coexisting	Loading multiple pipeline configurations at the same time	When port/resource contention is involved
Configuration hot reload	Adding/deleting/modifying configurations while running	Continuously running input plugins
Configuration type change	Switching from type A to type B	When the plugin supports multiple protocols/formats
Backpressure and recovery	Downstream unreachable → no data loss after recovery	flusher plugins
External dependency failure	Dependency service restart/unreachable	When there are external dependencies
Large data volume	No OOM/no data loss under high-throughput pressure	Performance-sensitive paths

2.2 Design Output

After determining the scenarios to cover, specify for each Scenario:

Input: what data, what format, how many records
Flow path: the specific input → processor → flusher plugins
Expected output: where to verify, what to verify
External dependencies: what auxiliary services are needed (OTel Collector, Kafka, etc.)

3 Writing Test Cases

3.1 Directory Structure

test/e2e/test_cases/my_feature/
├── case.feature
├── docker-compose.yaml       # 外部依赖
└── otel-collector-config.yaml  # 如果用 OTel Collector

3.2 Feature File Template

@flusher
Feature: my feature name
  Brief description

  @e2e @docker-compose
  Scenario: TestMyFeatureLogs
    Given {docker-compose} environment
    Given {my-config} local config as below
    """
    enable: true
    inputs:
      - Type: input_forward
        Protocol: OTLP
        Endpoint: "0.0.0.0:4320"
    flushers:
      - Type: flusher_otlp_native
        Endpoint: "otel-collector:4317"
    """
    When start docker-compose {my_feature}
    Then wait {10} seconds
    When generate {1} OTLP {logs} via otelgen to endpoint {loongcollectorC:4320}, protocol {grpc}
    Then wait {5} seconds
    Then otlp collector received at least {1} logs from file {/tmp/otel-export/logs.json}

3.3 Mandatory Rules

The configuration must include enable: true
Only use test/engine/steps.go steps already registered in
wait {N} seconds is of type Then , not When
Naming format:Test${功能名}${场景描述}
Do notuse global.ExcutionTimeoutin configurations of continuously running plugins (see §6.1)

3.4 Extending Steps

If new steps are needed, refer to the development and registration process in reference.md §Extending Steps .

4 Running Locally

4.1 Prerequisites

docker --version && docker compose version

If the C++ code has been modified, it needs to be recompiled and the image updated. Two methods:

Method 1: Full build(slow, but guarantees consistency)

make e2e_image   # 从源码构建完整 Docker 镜像 aliyun/loongcollector:0.0.1

Method 2: Incremental update(fast, suitable for iterative debugging)

cd build && make -sj$(nproc) && cd ..
# 替换镜像中的二进制
docker create --name tmp-lc aliyun/loongcollector:0.0.1
docker cp build/loongcollector tmp-lc:/usr/local/loongcollector/loongcollector
docker commit tmp-lc aliyun/loongcollector:0.0.1
docker rm tmp-lc

4.2 Run

cd test/e2e

# 运行整个测试用例（所有 Scenario）
TEST_CASE=flusher_otlp_native go test -v -run "TestE2EOnDockerCompose$" \
  -timeout 600s -count=1 ./...

# 只运行指定 Scenario
TEST_CASE=flusher_otlp_native go test -v \
  -run "TestE2EOnDockerCompose/TestFlusherOTLPNativeLogs$" \
  -timeout 600s -count=1 ./...

4.3 Cleanup (mandatory after a test failure)

You can run the script directly bash .cursor/skills/e2e/scripts/e2e-cleanup.sh, or execute manually:

docker rm -f $(docker ps -aq) 2>/dev/null
docker network prune -f
rm -rf test/e2e/config test/e2e/onetime_pipeline_config
sudo rm -rf test/e2e/report
rm -f test/e2e/test_cases/<case>/testcase-compose.yaml

5 Debugging

# 1. 查看容器日志
docker ps | grep loongcollectorC
docker exec <id> cat /usr/local/loongcollector/log/loongcollector.LOG

# 2. 检查配置是否加载
docker exec <id> ls /usr/local/loongcollector/conf/continuous_pipeline_config/local/

# 3. 检查端口是否监听
docker exec <id> ss -tlnp | grep <port>

# 4. 手动复现 compose 环境
cd test/e2e/test_cases/<case>
docker compose -f testcase-compose.yaml up -d
docker compose -f testcase-compose.yaml logs -f loongcollectorC

6 Known pitfalls

6.1 ExcutionTimeout makes the configuration one-time

Neveruse input_forward、input_file in the configuration of continuous plugins such as global.ExcutionTimeout。

It causes IsOnetime() to return true, causing IsValidNativeInputPlugin(name, true) to look up in the onetime registry, while most inputs only register continuous, resulting in the error unsupported input plugin。

See .cursor/rules/project-knowledge/config-pitfalls.mdc。

6.2 FlusherFile must be a file

The e2e template bind-mounts report/<case>default_flusher.json into the container. If the host path does not exist, Docker will create it as adirectory. This has been automatically handled in BootController.Start() .

6.3 Residue between tests

Multiple Scenarios share the process, andClean() deletes config/report. After an abnormal exit, clean up manually (§4.3).

About this skill

Table of Contents