Skip to main content
GET
Read a counterfactual

Authorizations

Authorization
string
header
required

nia_sk_...

Path Parameters

run_id
string<uuid>
required

Response

The report.

Whether the element changes what the tool returns, beyond the tool's own noise. It never says whether the result got better, nor how the model reacts: limits says so, and what else holds the answer back.

above_noise
integer
required
below_noise
integer
required

Completed cases whose overlap is below their own noise.

cases
integer
required
completed
integer
required
created_at
string<date-time>
required
effect
number | null
required

noise_floor - overlap: how much of the first positions the element moves beyond noise.

element
enum<string>
required
Available options:
constraints,
hard,
size,
exclude
engaged
CounterfactualEngagement · object
required

Where the items the person engaged with went without the element, over the completed cases.

limits
enum<string>[]
required
Available options:
not_quality,
model_reaction_not_measured,
trivial_for_hard,
few_cases,
noisy_tool,
cases_skipped,
dry_run
noise_floor
number | null
required

Mean overlap@k of the base with itself.

overlap
number | null
required

Mean overlap@k of the base and the variant.

p_value
number | null
required

The two-sided sign test of below_noise against above_noise: how likely a split this uneven is when the element changes nothing.

run_id
string
required
Required string length: 1 - 512
ties
integer
required
tool
string
required
label
string | null
skipped
Skipped · object

The cases not completed.