seatunnel-test-suite

SeaTunnel Test Suite

Self-contained operating manual for writing and reviewing SeaTunnel tests: the rule tables, citable IDs, and the patterns needed to act. Everything required to apply or cite a rule is in this file.

When to use

• Writing a new connector E2E test (*IT.java) or unit test (*Test.java).
• Reviewing a test diff for flakiness, leaks, or layout violations before a PR.
• Stabilizing a flaky or leaking test, or bringing an older test up to these conventions.

Framework contract (E2E)

Every E2E test must follow this structural contract — violating it means the test silently does nothing.

Minimal skeleton — runs on every engine (Zeta, Flink, Spark), which is the default and correct case for most batch source/sink tests:

public class MyConnectorIT extends TestSuiteBase {

    @TestTemplate
    public void testSourceToSink(TestContainer container) throws Exception {
        Container.ExecResult result = container.executeJob("/my_connector_to_assert.conf");
        Assertions.assertEquals(0, result.getExitCode());
    }
}

When to add @DisabledOnContainer

Do not exclude an engine by default. Add the annotation only when a scenario provably cannot run on that engine, and word disabledReason for the specific scenario — not a blanket "engine X is unsupported". The exclusion can go on the class (whole suite) or a single @TestTemplate method (just that case). Real reasons from the codebase:

// Only this CDC case can't run on Spark — exclude at the method, not the class:
@TestTemplate
@DisabledOnContainer(
        value = {},
        type = {EngineType.SPARK},
        disabledReason = "Spark does not support the CDC streaming job")
public void testCdcStreaming(TestContainer container) throws Exception {
    // ...
}

The rules

Two namespaces: E1–E7 govern E2E (*IT) tests (cite as [E3]); U1–U6 govern unit (*Test) tests (cite as [U5]). E6 is Zeta-engine-only. Pick the namespace that matches the file under review — never apply an E-rule to a unit test or vice versa.

E7 — network setup

The SeaTunnel engine runs inside its own Docker container. External service containers must share the same Docker network so the engine can reach them (this is also why E1's .conf files use the alias, not the mapped host port). Setup pattern:

// In your IT class @BeforeAll (NETWORK is inherited from TestSuiteBase):
container = new GenericContainer<>(DockerImageName.parse("my-service:1.2.3"))
        .withNetwork(NETWORK)                            // inherited field from TestSuiteBase
        .withNetworkAliases("my-service-host")           // reachable by this name from engine
        .withExposedPorts(3306)
        .withLogConsumer(new Slf4jLogConsumer(DockerLoggerFactory.getLogger("my-service:1.2.3")));
Startables.deepStart(Stream.of(container)).join();       // parallel startup

In your .conf file (runs inside the engine container):

host = "my-service-host"
port = 3306

In your Java test setup (runs on the host):

String jdbcUrl = String.format("jdbc:mysql://%s:%d/test",
        container.getHost(), container.getMappedPort(3306));

E2 — timeout reference

Chain .ignoreExceptions() onto the Awaitility.await() (the E2 fix) whenever the client throws until the service is up — otherwise the first poll's exception fails the wait instead of retrying. Never pick a bare round number without a scenario to justify it.

E5b — image version pinning

Never use :latest — upstream images can change behavior without warning, breaking CI with no code diff to bisect. Pin to a specific minor or patch version:

// Bad — breaks unpredictably
private static final String IMAGE = "postgres:latest";

// Good — reproducible
private static final String IMAGE = "postgres:14.5";

E6 — thread whitelist, in one paragraph

After the last job finishes, SeaTunnelContainer snapshots the server JVM and fails if a non-system thread survives 120s. Daemon threads from third-party clients (JDBC drivers, HTTP pools) are sometimes unrecyclable. For those only, add a specific name prefix to isIssueWeAlreadyKnow(String) in SeaTunnelContainer, with a comment naming the library. A thread you can close belongs in tearDown(), not the whitelist. System threads (hz.main, pool-N-thread-N, …) are already handled by isSystemThread(...) — don't duplicate them.

UT rules

Use this table for *Test classes (Surefire, no engine container):

Mock setup shape — each test method is Arrange-Act-Assert:

1. Arrange — in @BeforeEach, mock(...) every external boundary, wire the chain with when(...).thenReturn(...), then construct the SUT. Per test, stub only the values that select the branch under test.
2. Act — call the one method under test.
3. Assert — check observable output (U1), not call chains.

Pitfalls that make a mocked test pass for the wrong reason:

• When the SUT reaches a collaborator through a chain (a.getB().getC(key)), stub the whole chain, and stub each lookup with the exact key constant the production code uses — a mismatched key returns null and silently changes the branch taken.
• To exercise a downstream branch, stub every upstream guard non-null first; a null upstream value short-circuits early, so stubbing only the downstream value tests nothing.
• For negative paths, assert the exception message fragment, not just the type (U5) — a wrong-cause throw of the same type would otherwise pass.

When NOT to mock:

• Simple value objects, POJOs, DTOs — use real instances.
• The class under test itself — never mock the SUT.
• When an in-memory fake (e.g. HashMap instead of an interface) is simpler and equally isolated.

Layout (where things go)

One Maven module per connector under seatunnel-e2e/seatunnel-connector-v2-e2e/connector-<name>-e2e/. Files load from the test classpath root (src/test/resources/):

• Test class → src/test/java/org/apache/seatunnel/e2e/connector/<name>/<Name>IT.java (CDC connectors use the ...connectors.seatunnel.cdc.<name> package). *IT = integration (Failsafe), *Test = unit (Surefire).
• Job configs → src/test/resources/<name>_source_to_sink.conf, referenced with a leading slash (executeJob("/<name>_source_to_sink.conf")). License header required (# comments).
• DDL → src/test/resources/ddl/<table>.sql (CDC UniqueDatabase resolves ddl/<template>.sql).
• Container mounts → src/test/resources/docker/..., referenced via the container API (no leading slash).
• Register the module in the parent pom.xml; for a new connector also update CI label and plugin-mapping.properties.

Review workflow

1. Grep for the obvious smells (a clean result is a hint, not a guarantee — 127\.0\.0\.1 and localhost are also matched without a port to catch host references where the port is templated separately):

   grep -nE "Thread\.sleep|127\.0\.0\.1|localhost|:latest" <test-file>
   

2. [E1, E7] External containers joined to NETWORK with aliases? .conf uses alias + internal port?
3. Framework contract: extends TestSuiteBase? @TestTemplate + TestContainer param (not @Test)?
4. @DisabledOnContainer — present only where a scenario truly can't run on an engine (no blanket exclusions), and scoped to the affected method when only one case is incompatible?
5. [E3] Each opened resource closed in tearDown() (or try-with-resources)? Container stopped explicitly?
6. [E4] Streaming/CDC jobs submitted via CompletableFuture.supplyAsync and gated on RUNNING?
7. [E2] Timeouts justified by the scenario table — no bare round numbers?
8. [E5, E5b, E5c] One container shared per class? Image versions pinned (no :latest)? Full type set?
9. Resources in the conventional paths; new module registered in the parent pom?
10. [E6] For Zeta E2E, run the full job set; if the thread-leak check fires, close the client first.
11. For unit tests, walk the [U1–U6] table; for the negative path confirm [U5] asserts the message.
12. Run the targeted test class multiple times to confirm stability:

    ./mvnw -pl seatunnel-e2e/seatunnel-connector-v2-e2e/connector-<name>-e2e \
        -DskipUT -DskipIT=false -Dit.test=<ITClassName> verify
    

Quick-fix reference

Each row cites the rule it enforces, so a quick-fix finding maps directly to the output format below.

Output format

When reviewing, report each finding as [<RuleId>] <file>:<line> — <issue> with a one-line fix, where <RuleId> is an E-rule (E1–E7) for *IT tests or a U-rule (U1–U6) for *Test units — e.g. [E3] FooIT.java:88 — Kafka consumer never closed. Show a before/after snippet only when the fix isn't obvious from the rule. When writing a new class, satisfy all applicable rules and add the Apache License header to every new file (including .conf and .sql).