Detect Godot performance regressions in CI

To catch Godot performance regressions before they merge, tag each build’s telemetry with a build_id, run a profiling scenario in CI, and gate the pipeline with framedash perf-diff. When the candidate build regresses beyond your threshold against a known-good baseline, the command exits non-zero and fails the job. This guide wires that up with the Framedash Godot SDK, a C# addon for Godot 4.3 or later on the .NET build.

For the concepts behind build tagging and the regression gate, see CI-integrated profiling.

1. Tag the build with your CI build ID

Pass the CI commit or build identifier to the SDK as the build_id, so every event from that run is attributed to one build. Read FRAMEDASH_BUILD_ID (the variable framedash run-profile-test exports in step 4) so the telemetry’s build_id matches the value the gate waits on:

TelemetrySDK.Initialize(
    apiKey: System.Environment.GetEnvironmentVariable("FRAMEDASH_API_KEY"),
    buildId: System.Environment.GetEnvironmentVariable("FRAMEDASH_BUILD_ID"));

If you tag the build from a different variable (for example CI_COMMIT_SHA), pass that same value to run-profile-test with --build-id so both sides agree; otherwise the gate waits for a build_id that never ingests and times out. The Godot SDK shares the same C# telemetry API as the Unity SDK. See the Godot SDK guide for installation and the .NET setup.

2. Tag the automated session

Call the automated-session API once in your test entry point. BeginAutomatedSessionFromEnvironment() reads the FRAMEDASH_BUILD_ID, FRAMEDASH_GIT_BRANCH, FRAMEDASH_GIT_COMMIT, and FRAMEDASH_TEST_SCENARIO variables that framedash run-profile-test exports, so every event carries the build and its branch, commit, and scenario with no per-event tagging code:

TelemetrySDK.Instance.BeginAutomatedSessionFromEnvironment();
// ... run the profiling scenario ...
TelemetrySDK.Instance.EndAutomatedSession();

3. Build the C# solution for CI

Godot’s editor Build generates the .csproj and .sln for a C# project. godot --headless --build-solutions --quit only compiles an existing solution. On a project that has never been opened in the editor it can exit 0 yet write no project files, so a CI-only project has nothing to build. The CI-safe route is to hand-author the .csproj yourself, then build with the .NET SDK:

<!-- <project>.csproj at the project root; match the Godot.NET.Sdk version to your engine -->
<Project Sdk="Godot.NET.Sdk/4.6.3">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <EnableDynamicLoading>true</EnableDynamicLoading>
  </PropertyGroup>
</Project>

dotnet build

See the never-opened-project caveat on the Godot SDK page for why the headless --build-solutions step is not enough, plus the full hand-authored .csproj. If you can open the project in the editor once, click Build there and commit the generated .csproj / .sln instead.

Godot has no offline queue, so a short-lived headless run can exit before delivery finishes. In your profiling scene, call Flush() and keep the process alive until the verbose log prints the HTTP 202 line. The Godot SDK guide covers the solution-generation caveats and a flush-and-wait example.

4. Run the gate

framedash run-profile-test exports the FRAMEDASH_* session variables, launches your profiling command, waits for the telemetry to ingest, then runs the perf-diff gate against a baseline:

framedash run-profile-test \
  --command "godot --headless --path ." \
  --scenario nightly --api-key-file ci-read.key \
  --baseline "$BASE_SHA" --threshold 5 --fail-on-regression

The gate reads telemetry with an analytics:read key, while the launched game sends telemetry with a separate events:write ingest key. Pass the gate key with --api-key-file so FRAMEDASH_API_KEY stays available as the game’s ingest key.

You can also drive the steps by hand. framedash run-profile-test sets FRAMEDASH_BUILD_ID only for the command it launches, so when you drive the run yourself, export it with a build id of your choice (the step-1 initializer reads it) and reuse the same value for --candidate:

export FRAMEDASH_BUILD_ID="$GITHUB_SHA"
# ... launch your profiling run so the SDK tags telemetry with this build id ...

framedash builds --days 30

framedash perf-diff --baseline "$BASE_SHA" --candidate "$FRAMEDASH_BUILD_ID" \
  --threshold 5 --fail-on-regression

perf-diff compares frame time, memory, GPU time, map load time (load_time_ms), and disk I/O (io.read_bytes, io.read_time_ms, io.read_ops) as lower-is-better metrics. Map-load and io.* samples require Godot SDK 0.1.4 or later. Restrict the gate to one metric with --metric, or to one map or platform with --map and --platform.

GitHub Actions example

A minimal workflow that runs the gate on every pull request. fetch-depth: 0 is required so the baseline commit is present for perf-diff, and the analytics:read key is written to ci-read.key from a secret:

name: perf-regression
on: pull_request
jobs:
  perf-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # perf-diff needs the baseline commit
      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: "8.x"
      - name: Install the Framedash CLI
        run: npm install -g @framedash/cli
      - name: Build the C# solution
        run: dotnet build
      - name: Run the perf-diff gate
        env:
          FRAMEDASH_API_KEY: ${{ secrets.FRAMEDASH_INGEST_KEY }} # events:write, used by the game
          FRAMEDASH_READ_KEY: ${{ secrets.FRAMEDASH_READ_KEY }}   # analytics:read, used by the gate
          BASE_SHA: ${{ github.event.pull_request.base.sha }}
        run: |
          printf '%s' "$FRAMEDASH_READ_KEY" > ci-read.key
          framedash run-profile-test \
            --command "godot --headless --path ." \
            --scenario nightly --api-key-file ci-read.key \
            --baseline "$BASE_SHA" --threshold 5 --fail-on-regression

You still need a Godot .NET editor on the runner to launch the game (install it in an earlier step or use a container image that bundles it). The two keys are distinct: the game ingests with the events:write key in FRAMEDASH_API_KEY, and the gate reads with the analytics:read key in ci-read.key.

GitHub withholds secrets from workflows triggered by a forked PR, so this pull_request gate only works for same-repo (trusted) PRs. On a public repo that accepts fork PRs, guard the job (for example skip it when github.event.pull_request.head.repo.fork is true) or run the gate from a separate trusted workflow, rather than exposing the keys to untrusted PR code.

REST alternative

The same comparison is available over REST for pipelines without the CLI. GET /v1/projects/{id}/builds lists build IDs, and GET /v1/projects/{id}/builds/compare compares two builds; it requires baseline and candidate and accepts days, mapId, platform, and fresh=1. See the API overview.

Next steps

• CI-integrated profiling: the full concept and setup
• CLI reference: every flag for perf-diff and run-profile-test
• Godot SDK guide: installation, headless runs, and automatic metrics