OpenParable
Skip to content
← Back to skills

Vstest Build Test

Testing & Quality
vstest-build-test

Vstest Build Test is an Agent Skill for testing & quality published in the dotnet/dotnet repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Build, test, and validate changes in the vstest repository. Use when building vstest projects, running unit tests, smoke tests, or acceptance tests, or when deploying locally built vstest.console for manual testing.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/dotnet/dotnet/blob/HEAD/src/vstest/.github/skills/vstest-build-test/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/vstest-build-test, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Building and Testing vstest

Pre-Build: Environment Setup

Before building, verify the .dotnet toolchain matches the current OS. The repo bootstraps its own .NET SDK into .dotnet/.

Detect OS vs .dotnet Mismatch

Run this check before every first build in a session:

# Determine current OS
OS=$(uname -s)   # "Linux", "Darwin" (macOS), or contains "MINGW"/"MSYS" (Windows/Git Bash)

if [ -d ".dotnet" ]; then
  if [ "$OS" = "Linux" ] || [ "$OS" = "Darwin" ]; then
    # On Linux/macOS the dotnet binary must be an ELF/Mach-O executable, not .exe
    if [ -f ".dotnet/dotnet.exe" ] && [ ! -f ".dotnet/dotnet" ]; then
      echo "MISMATCH: .dotnet contains Windows binaries but OS is $OS"
      rm -rf .dotnet .packages artifacts
      echo "Cleaned .dotnet, .packages, and artifacts for fresh bootstrap"
    fi
  else
    # On Windows the dotnet binary should be dotnet.exe
    if [ -f ".dotnet/dotnet" ] && [ ! -f ".dotnet/dotnet.exe" ]; then
      echo "MISMATCH: .dotnet contains Linux/macOS binaries but OS is Windows"
      rm -rf .dotnet .packages artifacts
      echo "Cleaned .dotnet, .packages, and artifacts for fresh bootstrap"
    fi
  fi
fi

After cleanup (or if .dotnet doesn't exist), the build script automatically downloads the correct SDK version from global.json.

Build

Platform Commands

ActionWindowsLinux / macOS
Restore + Build./build.cmd./build.sh
Restore only./restore.cmd./restore.sh
Build + Pack./build.cmd -pack./build.sh --pack
Release config./build.cmd -c Release -pack./build.sh -c Release --pack
Single project./build.cmd -project <csproj>./build.sh --projects <csproj>

Full Build (Recommended)

For projects with many cross-project dependencies (e.g., HtmlLogger, TrxLogger, vstest.console):

# Linux / macOS
./build.sh --pack

# Windows
./build.cmd -pack

This produces NuGet packages under artifacts/packages/Debug/Shipping/.

Single Project Build

For isolated projects with few dependencies:

# Linux / macOS
./build.sh --projects <path-to-csproj>

# Windows
./build.cmd -project <path-to-csproj>

Warning: This does NOT work for projects like HtmlLogger that have many transitive dependencies. Use --pack / -pack instead.

Test

Unit Tests (Default)

# Linux / macOS
./test.sh

# Windows
./test.cmd

Specific Project(s)

-projects / --projects takes a resolvable path or glob — it is passed through Resolve-Path, so a bare project nickname or category (e.g. smoke, htmllogger) fails with Cannot find path. Point it at the csproj(s):

# Windows
./test.cmd -projects "test\**\*HtmlLogger*\*.csproj"

# Linux / macOS
./test.sh --projects "test/**/*HtmlLogger*/*.csproj"

For a single project you can also build+test its csproj directly with the bootstrapped SDK:

# Windows
./.dotnet/dotnet.exe test test/Microsoft.TestPlatform.Extensions.HtmlLogger.UnitTests/*.csproj -c Debug

# Linux / macOS
./.dotnet/dotnet test test/Microsoft.TestPlatform.Extensions.HtmlLogger.UnitTests/*.csproj -c Debug

Test Categories (smoke / integration / performance / compatibility)

These are switches handled by eng/build.ps1 — NOT -projects values:

# Windows
./test.cmd -smokeTest           # TestCategory=Smoke (a subset of integration tests)
./test.cmd -integrationTest     # full acceptance / integration suite
./test.cmd -performanceTest
./test.cmd -compatibilityTest

# Linux / macOS use the same switch names
./test.sh -smokeTest

-smokeTest and -integrationTest are mutually exclusive (smoke is a subset); passing both throws.

Filter by Test Name

Use the -filter parameter. Do not pass --filter inside TestRunnerAdditionalArguments — eng/build.ps1 explicitly throws if you do.

# Windows
./test.cmd -integrationTest -filter "FullyQualifiedName~MyScenario"

# Linux / macOS
./test.sh -integrationTest -filter "FullyQualifiedName~MyScenario"

Running integration / smoke tests locally (DOTNET_ROOT gotcha)

Integration and smoke tests self-host: they launch test-asset apphosts built against the repo's preview TFM (e.g. net11.0). An apphost resolves its shared runtime from DOTNET_ROOT, falling back to the machine-wide install (C:\Program Files\dotnet), which usually lacks the preview runtime — so it fails instantly with "You must install or update .NET to run this application."

  • test.sh (Linux/macOS) sets DOTNET_ROOT to the repo .dotnet automatically.
  • test.cmd (Windows) does not — set it yourself before running:
$env:DOTNET_ROOT = "$PWD\.dotnet"
${env:DOTNET_ROOT(x86)} = "$PWD\.dotnet\dotnet-sdk-x86"   # only if x86 test hosts run
$env:DOTNET_MULTILEVEL_LOOKUP = "0"
./test.cmd -smokeTest

Manual Validation with vstest.console

After building with --pack / -pack, validate vstest.console changes by unzipping the built package:

  1. Locate the package: artifacts/packages/Debug/Shipping/Microsoft.TestPlatform.<version>-dev.nupkg
  2. Unzip it (.nupkg files are ZIP archives)
  3. Run the local vstest.console against a test project

Alternative: Direct Artifact Paths

  • xplat (netcoreapp): artifacts/<Configuration>/netcoreapp1.0/vstest.console.dll
  • Windows desktop: artifacts/<Configuration>/net46/win7-x64/vstest.console.exe

Test Categories

CategorySpeedWhat it testsHow to run
Unit testsFastIndividual units./test.cmd / ./test.sh (default)
Smoke testsSlowP0 end-to-end scenarios-smokeTest switch
Acceptance / integrationSlowestExtensive coverage-integrationTest switch

Troubleshooting

  • OS mismatch errors: If you see SDK load failures, run the mismatch detection script above to clean and re-bootstrap.
  • Integration/smoke tests fail instantly on Windows with "You must install or update .NET to run this application": test.cmd does not set DOTNET_ROOT, so the self-hosted preview-TFM apphosts look in C:\Program Files\dotnet (which lacks the preview runtime). Set $env:DOTNET_ROOT = "$PWD\.dotnet" before running — see "Running integration / smoke tests locally".
  • If build fails asking for .NET 4.6 targeting pack, install it from Microsoft Downloads
  • Enable verbose diagnostics: see docs/diagnose.md
  • For debugging, add Debugger.Launch at process entry points (testhost.exe, vstest.console.exe)