Detect Unity performance regressions in CI

To catch Unity performance regressions before they merge, tag each build’s telemetry with a build_id, run a profiling scenario in CI, and gate the pipeline with framedash perf-diff. When the candidate build regresses beyond your threshold against a known-good baseline, the command exits non-zero and fails the job. This guide wires that up with the Framedash Unity SDK (Unity 2022.3 or later).

For the concepts behind build tagging and the regression gate, see CI-integrated profiling.

1. Tag the build with your CI build ID

Pass the CI commit or build identifier to the SDK as the build_id, so every event from that run is attributed to one build:

TelemetrySDK.Initialize(
    apiKey: System.Environment.GetEnvironmentVariable("FRAMEDASH_API_KEY"),
    buildId: System.Environment.GetEnvironmentVariable("FRAMEDASH_BUILD_ID"));

In a CI or desktop build, Initialize can also read FRAMEDASH_API_KEY from the environment when you leave the key argument empty. See the Unity SDK guide for the full resolution order.

2. Tag the automated session

Call the automated-session API once in your test entry point. BeginAutomatedSessionFromEnvironment() reads the FRAMEDASH_BUILD_ID, FRAMEDASH_GIT_BRANCH, FRAMEDASH_GIT_COMMIT, and FRAMEDASH_TEST_SCENARIO variables that framedash run-profile-test exports, so every event carries the build and its branch, commit, and scenario with no per-event tagging code:

TelemetrySDK.Instance.BeginAutomatedSessionFromEnvironment();
// ... run the profiling scenario ...
TelemetrySDK.Instance.EndAutomatedSession();

3. Emit telemetry from a PlayMode test

The Unity SDK transmits on the player loop, which does not tick in Edit mode. A plain -batchmode -executeMethod run buffers events but never sends them. To emit telemetry from CI, drive the player loop with a PlayMode test (Unity Test Framework), then run the tests headless:

Unity.exe -batchmode -nographics -projectPath <path> -runTests -testPlatform PlayMode -testResults <path>\results.xml -logFile -

The Unity SDK guide has a complete smoke-test example, including the assembly definition the Test Framework needs and the HTTP 202 log line that confirms delivery.

4. Run the gate

framedash run-profile-test exports the FRAMEDASH_* session variables, launches your test command, waits for the telemetry to ingest, then runs the perf-diff gate against a baseline:

framedash run-profile-test \
  --command "Unity.exe -batchmode -nographics -projectPath . -runTests -testPlatform PlayMode -logFile -" \
  --scenario nightly --api-key-file ci-read.key \
  --baseline "$BASE_SHA" --threshold 5 --fail-on-regression

The gate reads telemetry with an analytics:read key, while the launched game sends telemetry with a separate events:write ingest key. Pass the gate key with --api-key-file so FRAMEDASH_API_KEY stays available as the game’s ingest key.

You can also drive the steps by hand:

framedash builds --days 30

framedash perf-diff --baseline "$BASE_SHA" --candidate "$GITHUB_SHA" \
  --threshold 5 --fail-on-regression

perf-diff compares frame time, memory, GPU time, map load time (load_time_ms), and disk I/O (io.read_bytes, io.read_time_ms, io.read_ops) as lower-is-better metrics. Map-load and io.* samples require Unity SDK 0.1.3 or later. Restrict the gate to one metric with --metric, or to one map or platform with --map and --platform.

REST alternative

The same comparison is available over REST for pipelines without the CLI. GET /v1/projects/{id}/builds lists build IDs, and GET /v1/projects/{id}/builds/compare compares two builds; it requires baseline and candidate and accepts days, mapId, platform, and fresh=1. See the API overview.

Next steps

• CI-integrated profiling: the full concept and setup
• CLI reference: every flag for perf-diff and run-profile-test
• Unity SDK guide: installation, headless runs, and automatic metrics