CI-Integrated Profiling

CI-integrated profiling means tagging telemetry from automated builds with a build_id, then reviewing build performance in Framedash. Today, Framedash ships build filters, the Regression page, framedash builds, CI-run framedash perf-diff gates, the framedash run-profile-test runner, threshold profiles, and static threshold alerts. It does not yet run a Framedash-hosted profiling job or hosted previous-build notifications.

How It Works

1. CI Build: Create a game build
2. Automated Testing: Run predefined scenarios automatically
3. Telemetry Collection: The SDK sends performance data with the CI build_id
4. Dashboard Review: Compare builds with the Regression page or build filters
5. CI Gate: Run framedash run-profile-test (or framedash builds + framedash perf-diff) to fail the build on a build-over-build regression
6. Optional Alerts: Use threshold profiles and alert rules for static FPS, frame-time, or memory limits

Setup

1. SDK Configuration in CI

Set the build_id from CI environment variables during SDK initialization.

Unity / Godot

TelemetrySDK.Initialize(
    apiKey: System.Environment.GetEnvironmentVariable("FRAMEDASH_API_KEY"),
    buildId: System.Environment.GetEnvironmentVariable("FRAMEDASH_BUILD_ID"));

UE5

if (auto* Subsystem = GetGameInstance()->GetSubsystem<UFramedashSubsystem>())
{
    FString ApiKey = FPlatformMisc::GetEnvironmentVariable(TEXT("FRAMEDASH_API_KEY"));
    FString BuildId = FPlatformMisc::GetEnvironmentVariable(TEXT("FRAMEDASH_BUILD_ID"));
    // Pass an empty string for EndpointUrl to use the default value.
    Subsystem->InitializeTelemetry(ApiKey, TEXT(""), BuildId);
}

2. Tag the Automated Session

Call the automated-session API once in your automated-test entry point so every event carries the CI build and its branch/commit/scenario. BeginAutomatedSessionFromEnvironment() reads the FRAMEDASH_BUILD_ID / FRAMEDASH_GIT_BRANCH / FRAMEDASH_GIT_COMMIT / FRAMEDASH_TEST_SCENARIO variables that framedash run-profile-test exports:

Unity / Godot

TelemetrySDK.Instance.BeginAutomatedSessionFromEnvironment();
// ... run the profiling scenarios ...
TelemetrySDK.Instance.EndAutomatedSession();

UE5

if (auto* Subsystem = GetGameInstance()->GetSubsystem<UFramedashSubsystem>())
{
    Subsystem->BeginAutomatedSessionFromEnvironment();
    // ... run the profiling scenarios ...
    Subsystem->EndAutomatedSession();
}

The build_id is recorded as a top-level field; branch, commit, and scenario ride the ci.branch / ci.commit / ci.scenario attributes.

3. Define Test Scenarios

Create test scenarios that traverse each map/area to collect consistent data.

4. Run the Profiling Gate

The turnkey way is framedash run-profile-test: it exports the FRAMEDASH_* session variables, launches your profiling build, waits for its telemetry to ingest, then fails the job on a build-over-build regression.

framedash run-profile-test \
  --command "./Build/Game.exe -nullrhi -ExecCmds='Automation RunTest Perf'" \
  --scenario nightly --api-key-file ci-read.key \
  --baseline "$BASE_SHA" --threshold 5 --fail-on-regression

Or drive the steps manually: list comparable build IDs, then fail the pipeline when the candidate regressed beyond your threshold:

framedash builds --days 30

framedash perf-diff --baseline "$BASE_SHA" --candidate "$GITHUB_SHA" \
  --threshold 5 --fail-on-regression

perf-diff compares frame time, memory, GPU time, map load time (load_time_ms), disk I/O (io.read_bytes / io.read_time_ms / io.read_ops), and GPU VRAM (mem.vram) as lower-is-better metrics. Map-load and io.* samples require the Unity 0.1.3 / UE5 0.1.6 / Godot 0.1.4 SDKs or later, and mem.vram requires the UE5 SDK 0.1.7 or later (opt-in, non-headless), Unity SDK 0.1.4 or later, or Godot SDK 0.1.5 or later. A build with no mem.vram data is treated as not comparable for that metric. This is a CI-owned command gate, not a hosted previous-build alert.

Note

The gate reads telemetry with an analytics:read key, while the launched game sends telemetry with a separate events:write ingest key. To avoid a name clash, pass the gate key with --api-key-file rather than the FRAMEDASH_API_KEY environment variable, so FRAMEDASH_API_KEY stays available as the game’s ingest key.

You can also set static thresholds in the dashboard:

• FPS drops below an acceptable floor
• Frame time exceeds 33ms
• Memory usage exceeds your budget

Use Cases

• Pull Request Performance Checks: Attach telemetry to a PR build and review it before merging
• Nightly Build Monitoring: Keep daily performance snapshots by build ID
• Pre-Release Quality Gates: Use framedash perf-diff or CI-owned thresholds before release

Next Steps

• CLI Reference: Using the CLI in CI pipelines
• Heatmaps: Visualizing performance