SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsvstest-build-test

vstest-build-test

Testing & quality

View repository

Updated 10-16 00:32

About this skill

Building and Testing vstest

Pre-Build: Environment Setup

Before building, verify the .dotnet toolchain matches the current OS. The repo bootstraps its own .NET SDK into .dotnet/.

Detect OS vs .dotnet Mismatch

Run this check before every first build in a session:

# Determine current OS
OS=$(uname -s)   # "Linux", "Darwin" (macOS), or contains "MINGW"/"MSYS" (Windows/Git Bash)

if [ -d ".dotnet" ]; then if [ "$OS" = "Linux" ] || [ "$OS" = "Darwin" ]; then # On Linux/macOS the dotnet binary must be an ELF/Mach-O executable, not .exe if [ -f ".dotnet/dotnet.exe" ] && [ ! -f ".dotnet/dotnet" ]; then echo "MISMATCH: .dotnet contains Windows binaries but OS is $OS" rm -rf .dotnet .packages artifacts echo "Cleaned .dotnet, .packages, and artifacts for fresh bootstrap" fi else # On Windows the dotnet binary should be dotnet.exe if [ -f ".dotnet/dotnet" ] && [ ! -f ".dotnet/dotnet.exe" ]; then echo "MISMATCH: .dotnet contains Linux/macOS binaries but OS is Windows" rm -rf .dotnet .packages artifacts echo "Cleaned .dotnet, .packages, and artifacts for fresh bootstrap" fi fi fi


After cleanup (or if `.dotnet` doesn't exist), the build script automatically downloads the correct SDK version from `global.json`.

## Build

### Platform Commands

| Action | Windows | Linux / macOS |
|---|---|---|
| Restore + Build | `./build.cmd` | `./build.sh` |
| Restore only | `./restore.cmd` | `./restore.sh` |
| Build + Pack | `./build.cmd -pack` | `./build.sh --pack` |
| Release config | `./build.cmd -c Release -pack` | `./build.sh -c Release --pack` |
| Single project | `./build.cmd -project <csproj>` | `./build.sh --projects <csproj>` |

### Full Build (Recommended)

For projects with many cross-project dependencies (e.g., HtmlLogger, TrxLogger, vstest.console):

```bash
# Linux / macOS
./build.sh --pack

# Windows
./build.cmd -pack

This produces NuGet packages under artifacts/packages/Debug/Shipping/.

Single Project Build

For isolated projects with few dependencies:

# Linux / macOS
./build.sh --projects <path-to-csproj>

# Windows
./build.cmd -project <path-to-csproj>

Warning: This does NOT work for projects like HtmlLogger that have many transitive dependencies. Use --pack / -pack instead.

Test

Unit Tests (Default)

# Linux / macOS
./test.sh

# Windows
./test.cmd

Specific Project(s)

-projects / --projects takes a resolvable path or glob — it is passed through Resolve-Path, so a bare project nickname or category (e.g. smoke, htmllogger) fails with Cannot find path. Point it at the csproj(s):

# Windows
./test.cmd -projects "test\**\*HtmlLogger*\*.csproj"

# Linux / macOS
./test.sh --projects "test/**/*HtmlLogger*/*.csproj"

For a single project you can also build+test its csproj directly with the bootstrapped SDK:

# Windows
./.dotnet/dotnet.exe test test/Microsoft.TestPlatform.Extensions.HtmlLogger.UnitTests/*.csproj -c Debug

# Linux / macOS
./.dotnet/dotnet test test/Microsoft.TestPlatform.Extensions.HtmlLogger.UnitTests/*.csproj -c Debug

Test Categories (smoke / integration / performance / compatibility)

These are switches handled by eng/build.ps1 — NOT -projects values:

# Windows
./test.cmd -smokeTest           # TestCategory=Smoke (a subset of integration tests)
./test.cmd -integrationTest     # full acceptance / integration suite
./test.cmd -performanceTest
./test.cmd -compatibilityTest

# Linux / macOS: only --integrationTest and --performanceTest exist, because test.sh
# calls arcade's eng/common/build.sh directly and never goes through eng/build.ps1.
./test.sh --integrationTest     # includes smoke; there is no -smokeTest switch here

-smokeTest and -integrationTest are mutually exclusive (smoke is a subset); passing both throws.

Filter by Test Name

On Windows use the -filter parameter. Do not pass --filter inside TestRunnerAdditionalArguments — eng/build.ps1 explicitly throws if you do.

On Linux/macOS test.sh has no -filter parameter, so the filter goes in as an MSBuild property. The single quotes are required, otherwise bash interprets &, |, and the inner quotes.

# Windows
./test.cmd -integrationTest -filter "FullyQualifiedName~MyScenario"

# Linux / macOS
./test.sh --integrationTest --property:'TestRunnerAdditionalArguments=--filter "FullyQualifiedName~MyScenario"'

See CONTRIBUTING.md for more filtering examples.

Running tests locally (DOTNET_ROOT gotcha)

Test executables are built against the repo's preview TFM (e.g. net11.0), and integration and smoke tests additionally launch test-asset apphosts built the same way. An apphost resolves its shared runtime from DOTNET_ROOT, falling back to the machine-wide install (C:\Program Files\dotnet), which usually lacks the preview runtime — so it fails instantly with "You must install or update .NET to run this application." This hits plain unit test runs too, not only integration and smoke tests.

  • test.sh (Linux/macOS) sets DOTNET_ROOT to the repo .dotnet automatically.
  • test.cmd (Windows) does not — set it yourself before running:
$env:DOTNET_ROOT = "$PWD\.dotnet"
${env:DOTNET_ROOT(x86)} = "$PWD\.dotnet\dotnet-sdk-x86"   # only if x86 test hosts run
$env:DOTNET_MULTILEVEL_LOOKUP = "0"
./test.cmd -smokeTest

Manual Validation with vstest.console

After building with --pack / -pack, validate vstest.console changes by unzipping the built package:

  1. Locate the package: artifacts/packages/Debug/Shipping/Microsoft.TestPlatform.<version>-dev.nupkg
  2. Unzip it (.nupkg files are ZIP archives)
  3. Run the local vstest.console against a test project

Alternative: Direct Artifact Paths

  • xplat (netcoreapp): artifacts/<Configuration>/netcoreapp1.0/vstest.console.dll
  • Windows desktop: artifacts/<Configuration>/net46/win7-x64/vstest.console.exe

Test Categories

Category Speed What it tests How to run
Unit tests Fast Individual units ./test.cmd / ./test.sh (default)
Smoke tests Slow P0 end-to-end scenarios -smokeTest switch
Acceptance / integration Slowest Extensive coverage -integrationTest switch

Troubleshooting

  • OS mismatch errors: If you see SDK load failures, run the mismatch detection script above to clean and re-bootstrap.
  • Toolset version <version> has not been restored.: test.cmd / test.sh do not restore. Build once (./build.cmd -c Release) first, or pass -restore -build to the test command.
  • Tests fail instantly on Windows with "You must install or update .NET to run this application": test.cmd does not set DOTNET_ROOT, so the preview-TFM apphosts look in C:\Program Files\dotnet (which lacks the preview runtime). Set $env:DOTNET_ROOT = "$PWD\.dotnet" before running — see "Running tests locally".
  • If build fails asking for .NET 4.6 targeting pack, install it from Microsoft Downloads
  • Enable verbose diagnostics: see docs/diagnose.md
  • For debugging, add Debugger.Launch at process entry points (testhost.exe, vstest.console.exe)