OpenParable
Skip to content
← Back to skills

Synapseml Local Setup

Development
synapseml-local-setup

Synapseml Local Setup is an Agent Skill for development published in the microsoft/SynapseML repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Set up and validate SynapseML locally in WSL or Linux. Use when an agent needs SynapseML working locally, runs sbt compile/test, sees Java 21, Scala 2.12 compiler-bridge, bad constant pool index, Spark, or local validation failures.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/microsoft/SynapseML/blob/HEAD/.github/skills/synapseml-local-setup/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/synapseml-local-setup, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

SynapseML Local Setup

Use this skill before any local SynapseML build, compile, or test validation.

Important

  • Always use an explicit SynapseML repo path.
  • Do not run SynapseML SBT with the machine default Java 21. Use JDK 11: /usr/lib/jvm/java-11-openjdk-amd64
  • Java 21 can fail before project code compiles with bad constant pool index: 0 while building Scala 2.12 compiler-bridge_2.12.
  • Compile commands are safe. Some cognitive service tests create, write, list, or delete real Azure resources. Inspect before running those tests and ask for approval if live resources are involved.

Workflow

1. Diagnose the repo and toolchain

Run scripts/synapseml-doctor.sh:

scripts/synapseml-doctor.sh --repo <synapseml-repo>

Capture:

  • Git branch and dirty state.
  • Default Java version.
  • JDK 11 availability.
  • sbt version and SynapseML Scala/Spark versions.

2. Compile with JDK 11

Run scripts/synapseml-sbt.sh:

scripts/synapseml-sbt.sh --repo <synapseml-repo> -- cognitive/Test/compile

Expected result:

  • sbt welcome line says Java 11.
  • core and cognitive main/test classes compile.
  • Command exits with [success].

3. Run a safe local smoke test

Run scripts/synapseml-smoke-test.sh:

scripts/synapseml-smoke-test.sh --repo <synapseml-repo>

Expected result:

  • One local Spark test runs.
  • Output includes All tests passed.

4. Inspect PR-specific tests before running them

Before running service tests, run scripts/check-live-service-tests.sh:

scripts/check-live-service-tests.sh --path <test-file-or-directory>

If it reports live-service hooks, ask the user before running that suite. Do not create or delete Azure Search indexes just to test a PR.

5. Run targeted tests only after safety review

Use the JDK 11 wrapper for any targeted SBT command:

scripts/synapseml-sbt.sh --repo <synapseml-repo> -- '<module>/testOnly <SuiteName> -- -z "<test filter>"'

If tests fail before compiling project code, load references/troubleshooting.md.

Known-good baseline

On 2026-05-05, this setup was validated with:

  • JDK: /usr/lib/jvm/java-11-openjdk-amd64
  • Java: openjdk version "11.0.30"
  • SynapseML: Scala 2.12.17, Spark 3.5.0, sbt 1.10.11
  • Compile: sbt 'cognitive/Test/compile' succeeded.
  • Smoke test: UDFTransformerSuite filtered test succeeded.