OpenParable
Skip to content
← Back to skills

Gradle Build

Development
gradle-build

Gradle Build is an Agent Skill for development published in the apache/beam repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Guides understanding and using the Gradle build system in Apache Beam. Use when building projects, understanding dependencies, or troubleshooting build issues.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/apache/beam/blob/HEAD/.agent/skills/gradle-build/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/gradle-build, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Gradle Build System in Apache Beam

Overview

Apache Beam is a mono-repo using Gradle as its build system. The entire project (Java, Python, Go, website) is managed as a single Gradle project.

Key Files

  • build.gradle.kts - Root build configuration
  • settings.gradle.kts - Project structure and module definitions
  • gradle.properties - Global properties and versions
  • buildSrc/ - Custom Gradle plugins including BeamModulePlugin

BeamModulePlugin

Located at buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy

Purpose

  • Manages Java dependencies centrally
  • Configures project types (Java, Python, Go, Proto, Docker, etc.)
  • Defines common custom tasks

Java Project Configuration

apply plugin: 'org.apache.beam.module'
applyJavaNature(
    automaticModuleName: 'org.apache.beam.sdk.io.kafka'
)

Common Commands

Build

# Build entire project
./gradlew build

# Build specific project
./gradlew :sdks:java:core:build
./gradlew -p sdks/java/core build

# Compile only (no tests)
./gradlew :sdks:java:core:compileJava

Test

# Run tests
./gradlew :sdks:java:core:test

# Run specific test
./gradlew :sdks:java:core:test --tests *MyTest

# Skip tests
./gradlew build -x test

Clean

# Clean specific project
./gradlew :sdks:java:core:clean

# Clean everything
./gradlew clean

Formatting

# Java formatting (Spotless)
./gradlew spotlessApply

# Check formatting
./gradlew spotlessCheck

# Format CHANGES.md
./gradlew formatChanges

Publishing

# Publish to Maven Local
./gradlew -Ppublishing :sdks:java:core:publishToMavenLocal

# Publish all Java artifacts
./gradlew -Ppublishing publishToMavenLocal

Pre-commit Tasks

Java

./gradlew javaPreCommit

Python

./gradlew pythonPreCommit

Combined

./gradlew :checkSetup  # Validates Go, Java, Python environments

Useful Flags

FlagDescription
-p <path>Run task in specific project directory
-x <task>Exclude task
--tests <pattern>Filter tests
-PpublishingEnable publishing tasks
-PdisableSpotlessCheck=trueDisable formatting check
-PdisableCheckStyle=trueDisable checkstyle
-PskipCheckerFrameworkSkip Checker Framework
--continueContinue after failures
--infoVerbose output
--debugDebug output
--scanGenerate build scan
--parallelParallel execution

GCP-related Properties

-PgcpProject=my-project
-PgcpRegion=us-central1
-PgcpTempRoot=gs://bucket/temp
-PgcsTempRoot=gs://bucket/temp

Docker Tasks

# Build Java SDK container
./gradlew :sdks:java:container:java11:docker

# Build Python SDK container
./gradlew :sdks:python:container:py312:docker

# Build and push a container into a custom repository
./gradlew :sdks:java:container:java11:docker \
  -Pdocker-repository-root=gcr.io/project \
  -Pdocker-tag=custom \
  -Ppush-containers

If a :docker task produces logs that contain the following:

WARNING: No output specified with docker-container driver.
Build result will only remain in the build cache.

then you must use -PuseDockerBuildx when running :docker tasks in this environment. For example:

# Build and push a go container into a custom repository
./gradlew :sdks:go:container:docker \
  -Pdocker-repository-root=gcr.io/project \
  -Pdocker-tag=custom \
  -Ppush-containers \
  -PuseDockerBuildx

Dependency Management

View Dependencies

./gradlew :sdks:java:core:dependencies
./gradlew :sdks:java:core:dependencies --configuration runtimeClasspath

Force Dependency Version

In build.gradle:

configurations.all {
    resolutionStrategy.force 'com.google.guava:guava:32.0.0-jre'
}

Troubleshooting

Clean Gradle Cache

rm -rf ~/.gradle/caches
rm -rf .gradle
rm -rf build

Common Errors

NoClassDefFoundError

  • Run ./gradlew clean
  • Delete gradle cache

Proto-related Errors

  • Regenerate protos: ./gradlew generateProtos

Dependency Conflicts

  • Check dependencies: ./gradlew dependencies
  • Use --scan for detailed analysis

Useful Tasks

# List all tasks
./gradlew tasks

# List tasks for a project
./gradlew :sdks:java:core:tasks

# Show project structure
./gradlew projects

IDE Integration

IntelliJ

  1. Open repository root as Gradle project
  2. Wait for indexing
  3. Gradle tool window shows all tasks

VS Code

Install Gradle extension for task discovery