hive-test

Skill: hive-test

Run Ethereum Hive integration tests against a local Erigon build. Works from a clean environment -- no pre-existing hive installation required.

Metadata

• user-invocable: true
• description: Run Hive integration tests (engine, rpc, eest) against local Erigon
• allowed-tools: Bash, Read, Write, Edit, Glob, Grep

Overview

This skill sets up an ephemeral Hive test environment, builds a local Erigon Docker image, runs the requested test suites, reports results, and cleans up containers.

Arguments

The user may specify one or more test suites in any combination:

Individual suites

Groups

Examples

• /hive-test api - Run just the engine API suite
• /hive-test withdrawals api - Run withdrawals and API suites
• /hive-test engine - Run all engine suites
• /hive-test engine rpc-compat - Run all engine suites plus rpc-compat
• /hive-test eest-devnet - Run devnet EEST tests (BAL/glamsterdam)
• /hive-test eest-rlp - Run EEST RLP block-import tests
• /hive-test all - Run everything

Options

• erigon-path - Path to local erigon source (default: current working directory)
• branch=BRANCH - Clone erigon from a remote branch instead of using the local working directory. The branch is cloned from https://github.com/erigontech/erigon.git into the hive client directory. Example: /hive-test api branch=fix/my-feature
• eest-version=VERSION - Pin EEST fixtures version (e.g. v5.3.0). Default: auto-discover latest.
• bal-version=VERSION - Pin BAL fixtures version (e.g. bal@v5.1.0). Default: auto-discover latest.

Expected Failures (CI thresholds)

Sources of truth: .github/workflows/test-hive.yml (max-allowed-failures per matrix entry) for engine + rpc-compat suites, .github/workflows/test-hive-eest.yml (max-failures per matrix entry, currently 0 everywhere) for eest shards.

Note: Failure counts are version-dependent and may change with newer fixtures. The CI glamsterdam-devnet shard runs BAL EIPs (7708|7778|7843|7928|7954|7976|7981|8024|8037) against the URL and hive branch pinned under the eest_devnet entry in test-fixtures.json (currently tests-bal@v7.3.2 / devnets/bal/7), with --experimental.bal enabled on the erigon side. Reproduce locally by aligning the invocation with those values — make eest-devnet reads them from the manifest via jq and applies them automatically.

Procedure

Phase 0: Discover Versions

Before setup, discover the latest EEST fixture versions from GitHub. Skip this phase if the user provided explicit version overrides (eest-version=, bal-version=).

# Latest standard EEST fixtures (tag matching v*.*.*)
EEST_VERSION=$(curl -s https://api.github.com/repos/ethereum/execution-spec-tests/releases \
  | jq -r '[.[] | select(.tag_name | test("^v[0-9]+\\.[0-9]+\\.[0-9]+
quot;))][0].tag_name')

# Latest BAL fixtures (tag matching bal@v*.*.*)
BAL_TAG=$(curl -s https://api.github.com/repos/ethereum/execution-spec-tests/releases \
  | jq -r '[.[] | select(.tag_name | startswith("bal@"))][0].tag_name')
BAL_BRANCH="tests-${BAL_TAG}"
BAL_TAG_URLENC=$(echo "$BAL_TAG" | sed 's/@/%40/')
BAL_FIXTURES_URL="https://github.com/ethereum/execution-spec-tests/releases/download/${BAL_TAG_URLENC}/fixtures_bal.tar.gz"

Then check whether the EEST mapper at the discovered tag already has the exception entries Erigon needs. If not, the disable_strict_exception_matching workaround is required:

MAPPER_URL="https://raw.githubusercontent.com/ethereum/execution-specs/refs/tags/${BAL_BRANCH}/packages/testing/src/execution_testing/client_clis/clis/erigon.py"
if curl -sf "$MAPPER_URL" | grep -q "GAS_USED_OVERFLOW"; then
    DISABLE_STRICT=""
    echo "Mapper has BAL exception entries — strict matching enabled"
else
    DISABLE_STRICT='--sim.buildarg disable_strict_exception_matching=erigon'
    echo "Mapper missing BAL exception entries — strict matching disabled for erigon"
fi

Log the discovered versions:

EEST: $EEST_VERSION | BAL: $BAL_TAG (branch: $BAL_BRANCH) | Strict matching: enabled/disabled

Phase 1: Setup

1. Determine erigon source path. Default is the current git working directory. Verify it contains a Makefile and go.mod with erigontech/erigon.

2. Choose a work directory. Use mktemp -d /tmp/hive-test-XXXXXX for isolation.

3. Clone hive:

   WORKDIR=$(mktemp -d /tmp/hive-test-XXXXXX)
   cd "$WORKDIR"
   git clone --depth 1 https://github.com/ethereum/hive.git
   cd hive
   

4. Copy or clone the erigon source into hive:

   If branch=BRANCH was specified, clone that branch:

   git clone --depth 1 --branch "$BRANCH" \
     https://github.com/erigontech/erigon.git clients/erigon/erigon
   

   Otherwise, copy the local source:

   # Use rsync to copy, excluding build artifacts and .git
   rsync -a --exclude='.git' --exclude='build/' --exclude='temp/' \
     "$ERIGON_PATH/" clients/erigon/erigon/
   

5. Install Dockerfile.local for local builds: Ensure clients/erigon/Dockerfile.local exists with the correct content. Key requirements:

   • Base image: golang:1.25.7-trixie (Debian, not Alpine)
   • Build command: make erigon
   • Runtime: debian:13-slim with bash curl jq libstdc++6 libgcc-s1

   If clients/erigon/Dockerfile.local doesn't already exist, write the correct version:

   FROM golang:1.25.7-trixie AS builder
   ARG local_path=erigon
   COPY $local_path erigon
   RUN apt-get update && apt-get install -y bash build-essential ca-certificates git \
       && cd erigon && make erigon \
       && cp build/bin/erigon /usr/local/bin/erigon
   
   FROM debian:13-slim
   COPY --from=builder /usr/local/bin/erigon /usr/local/bin/
   RUN apt-get update && apt-get install -y bash curl jq libstdc++6 libgcc-s1 && rm -rf /var/lib/apt/lists/*
   RUN erigon --version | sed -e 's/erigon version \(.*\)/\1/' > /version.txt
   COPY genesis.json /genesis.json
   COPY mapper.jq /mapper.jq
   COPY erigon.sh /erigon.sh
   COPY enode.sh /hive-bin/enode.sh
   RUN chmod +x /erigon.sh /hive-bin/enode.sh
   EXPOSE 8545 8546 8551 30303 30303/udp
   ENTRYPOINT ["/erigon.sh"]
   

6. P2P protocol configuration: Do NOT add --p2p.protocol flags to erigon.sh. Let erigon use its default protocol negotiation.

6b. Parallel execution for Amsterdam (BAL) blocks: Verify that erigon.sh enables parallel execution when Amsterdam is configured. BAL validation only runs in the parallel execution path. If the following block is missing from erigon.sh, add it after the --sync.parallel-state-flushing line:

# Enable parallel execution for Amsterdam (BAL) blocks.
# BAL validation only runs in the parallel execution path.
if [ "$HIVE_AMSTERDAM_TIMESTAMP" != "" ]; then
    export ERIGON_EXEC3_PARALLEL=true
fi

Without this, Amsterdam blocks run through serial execution which has no BAL validation, causing test_bal_invalid tests to incorrectly return VALID.

7. Create client config file:

   cat > erigon-local.yaml <<'EOF'
   - client: erigon
     dockerfile: local
   EOF
   

8. Build hive binary:

   go build .
   

Phase 2: Run Tests

Always use --sim.parallelism 12 for all suites to maximize throughput.

When running multiple suites, launch separate hive sessions in parallel (as background shell commands) whenever the suites use different simulators. This gives runs × parallelism total concurrency. Suites using the same simulator can be combined with --sim.limit "suite1|suite2|...".

Engine suites (sim: ethereum/engine) — combine all with |:

./hive --client-file erigon-local.yaml --sim ethereum/engine \
  --sim.limit "exchange-capabilities|withdrawals|cancun|api|auth" \
  --sim.parallelism 12 --sim.timelimit 30m

RPC compat (sim: ethereum/rpc, limit: compat):

./hive --client-file erigon-local.yaml --sim ethereum/rpc \
  --sim.limit "compat" --sim.parallelism 12 --sim.timelimit 30m

EEST (sim: ethereum/eels/consume-engine):

# $EEST_VERSION discovered in Phase 0 (e.g. v5.4.0), or user-provided via eest-version=
./hive --client-file erigon-local.yaml \
  --sim ethereum/eels/consume-engine \
  --sim.parallelism=12 --docker.nocache=true \
  --sim.buildarg fixtures=https://github.com/ethereum/execution-spec-tests/releases/download/${EEST_VERSION}/fixtures_develop.tar.gz \
  --sim.timelimit 60m

EEST BAL (sim: ethereum/eels/consume-engine, amsterdam filter):

# $BAL_BRANCH, $BAL_FIXTURES_URL, $DISABLE_STRICT discovered in Phase 0
# (e.g. BAL_BRANCH=tests-bal@v5.2.0), or user-provided via bal-version=
./hive --client-file erigon-local.yaml \
  --sim ethereum/eels/consume-engine \
  --sim.limit=".*amsterdam.*" \
  --sim.parallelism=12 --docker.nocache=true \
  --sim.buildarg branch=${BAL_BRANCH} \
  --sim.buildarg fixtures=${BAL_FIXTURES_URL} \
  ${DISABLE_STRICT} \
  --sim.timelimit 60m

EEST RLP (sim: ethereum/eels/consume-rlp):

Tests block import via RLP-encoded blocks loaded at client startup (the historical sync code path). Uses the BlockchainTest fixture format and covers all forks including pre-merge, complementary to consume-engine which only covers Paris+. See https://eest.ethereum.org/main/running_tests/running/#engine-vs-rlp-simulator.

The full RLP test set is too large to run end-to-end in CI — always pass a --sim.limit regex narrowing the scope. CI mirrors this by running only .*eip2930_access_list.*. For local debugging, target a single EIP / opcode group similarly. Fixtures come from the same fixtures_develop.tar.gz archive as consume-engine.

# $EEST_VERSION discovered in Phase 0 (e.g. v5.4.0), or user-provided via eest-version=
# Replace the sim.limit regex to scope to the area under test.
./hive --client-file erigon-local.yaml \
  --sim ethereum/eels/consume-rlp \
  --sim.limit=".*eip2930_access_list.*" \
  --sim.parallelism=12 --docker.nocache=true \
  --sim.buildarg fixtures=https://github.com/ethereum/execution-spec-tests/releases/download/${EEST_VERSION}/fixtures_develop.tar.gz \
  --sim.timelimit 60m

Note: The disable_strict_exception_matching flag is only added when Phase 0 detects that the EEST ErigonExceptionMapper at the discovered tag is missing required entries (e.g. BlockException.GAS_USED_OVERFLOW, BAL exception types). Once upstream updates their tagged releases with the mapper fixes (already on forks/amsterdam HEAD), this flag will no longer be needed automatically.

Phase 3: Parse Results

After each suite, parse the output to extract results:

status_line=$(tail -2 output.log | head -1 | sed -r "s/\x1B\[[0-9;]*[a-zA-Z]//g")
suites=$(echo "$status_line" | sed -n 's/.*suites=\([0-9]*\).*/\1/p')
tests=$(echo "$status_line" | sed -n 's/.*tests=\([0-9]*\).*/\1/p')
failed=$(echo "$status_line" | sed -n 's/.*failed=\([0-9]*\).*/\1/p')

Also check the JSON result files in workspace/logs/*.json for detailed per-test results. Report pass/fail counts and list any failing test names.

Phase 4: Cleanup

Always run cleanup, even if tests fail:

# Clean up hive containers
./hive --cleanup

# Optionally remove the work directory
rm -rf "$WORKDIR"

# Prune dangling docker images from the test run
docker image prune -f

Troubleshooting

Timeout failures

Run suites separately instead of combining them. Increase --sim.timelimit if needed.

Leftover containers

Run ./hive --cleanup or docker rm -f $(docker ps -aq) to remove stale containers. The hive binary has built-in cleanup: ./hive --cleanup --cleanup.older-than 1h.