APEX / Evaluation Control Plane

AI evals should be release gates.

The current APEX build validates generated execution plans before release and detects material quality drift over time. Evaluation is treated as operational evidence — not a dashboard after the fact.

CURRENT BUILD: 9/9 GOLDEN REGRESSION PASS · EXECUTION-PARAMETER VALIDATION · PRINCIPAL PROPAGATION · DRIFT DETECTION
Run Drift Check See the Defect APEX Caught Back to APEX
Golden regression9 / 9 PASS
Query shapes covered9
Execution-contract validationACTIVE
Demonstrated drift severityHIGH
Failed gate actionBLOCK
9/9
Generated query shapes passing regression
94.75%
Known-good quality baseline
84.00%
Current score in degraded window
-10.75 pts
Material quality movement detected
Evaluation Architecture

Validate more than intent accuracy.

APEX freezes known-good cases and validates the governed execution plan all the way to the stored procedure, bound parameters, date interpretation, and calling principal.

Golden Regression

Known-good cases validate generated data-domain behavior before release.

  • Intent
  • Execution shape
  • Stored procedure

Execution Contract

Deterministic controls inspect what will actually touch enterprise systems.

  • Metric + dimension
  • Dates + granularity
  • Top-N + lookup filters
  • Principal propagation

Drift + Release Control

Quality windows are compared to a known-good baseline and failures become release decisions.

  • Material delta detection
  • Severity assignment
  • PASS / BLOCK
Domain FactoryGenerate governed domain assets
Golden CasesFreeze expected behavior
Execution PlanBind proc, params, identity
EvaluationScore behavior
DriftCompare quality windows
Release GatePromote or BLOCK
Regression Finding

The gate already caught a real defect.

Expanding the APEX golden suite exposed a routing-precedence defect in the generated data-domain orchestration path.

Observed classifier result KPI_TIMESERIES_BY_DIMENSION Incorrect runtime route KPI_TIMESERIES_BY_DIMENSION → GROUPBY Expected route KPI_TIMESERIES_BY_DIMENSION → TIMESERIES

A generic grouping-entity heuristic was overriding a strong explicit time-series intent. The regression suite returned BLOCK. Routing precedence was corrected, the original group-by path was revalidated, and the regression remains in the suite to prevent recurrence.

Interactive Drift Check

Try the drift detector.

This public tool runs entirely in the browser. It mirrors the current APEX drift logic by combining a minimum material delta with a z-score significance test. It contains no credentials, enterprise data, or external model calls.

Statistical threshold: |z| ≥ 2.5

Drift Result

Baseline mean94.75%
Current mean84.00%
Delta-10.75 pts
Z-score-25.44
Statistical significanceSIGNIFICANT
SeverityHIGH
RELEASE DECISION: BLOCK
Next layer under active development: model-based semantic scoring for groundedness, correctness, and completeness on version-aware traces, followed by persisted score history and automatic rolling-window drift analysis.