> ## Documentation Index
> Fetch the complete documentation index at: https://docs.niadra.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Read a counterfactual

> One report: overlap, noise floor, effect, p-value and the limits it declares itself.



## OpenAPI

````yaml openapi/en/cell.json GET /v1/measure/counterfactual-runs/{run_id}
openapi: 3.1.0
info:
  title: Niadra data API
  version: '1'
  description: >-
    Writing, context, history, objects, identity, privacy and governance of one
    space. Every space has a stable address, with the space and the region in
    its name.
servers:
  - url: https://{space}.{region}.api.niadra.com
    variables:
      space:
        default: acme-prod
        description: The space, which comes in the source key.
      region:
        default: us-east-2
        description: The region of the space, which also comes in the key.
security: []
paths:
  /v1/measure/counterfactual-runs/{run_id}:
    get:
      tags:
        - signals
      summary: Read a counterfactual
      description: >-
        One report: the overlap, the noise floor, the effect, the sign test,
        where the engaged items went, the cases skipped and the limits the
        report declares itself.


        **Authentication.** Source key with the `replay` scope, or a Console
        person token with the `analysis` role (or admin). It answers only in a
        space with the `measurement` feature on (the `features` document); in a
        space without it, 404.
      operationId: counterfactual_run_read_v1_measure_counterfactual_runs__run_id__get
      parameters:
        - in: path
          name: run_id
          required: true
          schema:
            format: uuid
            title: Run Id
            type: string
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/CounterfactualRun'
          description: The report.
        '422':
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'
          description: The request does not match the contract.
      security:
        - sourceKey: []
        - personToken: []
components:
  schemas:
    CounterfactualRun:
      additionalProperties: false
      description: >-
        Whether the element changes what the tool returns, beyond the tool's own
        noise. It never says whether

        the result got better, nor how the model reacts: `limits` says so, and
        what else holds the answer

        back.
      properties:
        above_noise:
          title: Above Noise
          type: integer
        below_noise:
          description: Completed cases whose overlap is below their own noise.
          title: Below Noise
          type: integer
        cases:
          title: Cases
          type: integer
        completed:
          title: Completed
          type: integer
        created_at:
          format: date-time
          title: Created At
          type: string
        effect:
          anyOf:
            - type: number
            - type: 'null'
          description: >-
            `noise_floor - overlap`: how much of the first positions the element
            moves beyond noise.
          title: Effect
        element:
          enum:
            - constraints
            - hard
            - size
            - exclude
          title: Element
          type: string
        engaged:
          $ref: '#/components/schemas/CounterfactualEngagement'
        label:
          anyOf:
            - type: string
            - type: 'null'
          title: Label
        limits:
          items:
            enum:
              - not_quality
              - model_reaction_not_measured
              - trivial_for_hard
              - few_cases
              - noisy_tool
              - cases_skipped
              - dry_run
            type: string
          title: Limits
          type: array
        noise_floor:
          anyOf:
            - type: number
            - type: 'null'
          description: Mean overlap@k of the base with itself.
          title: Noise Floor
        overlap:
          anyOf:
            - type: number
            - type: 'null'
          description: Mean overlap@k of the base and the variant.
          title: Overlap
        p_value:
          anyOf:
            - type: number
            - type: 'null'
          description: >-
            The two-sided sign test of `below_noise` against `above_noise`: how
            likely a split this uneven is when the element changes nothing.
          title: P Value
        run_id:
          maxLength: 512
          minLength: 1
          title: Run Id
          type: string
        skipped:
          additionalProperties:
            type: integer
          description: The cases not completed.
          propertyNames:
            enum:
              - completed
              - no_dry_run
              - tool_error
              - infrastructure_error
          title: Skipped
          type: object
        ties:
          title: Ties
          type: integer
        tool:
          title: Tool
          type: string
      required:
        - run_id
        - tool
        - element
        - created_at
        - cases
        - completed
        - overlap
        - noise_floor
        - effect
        - below_noise
        - above_noise
        - ties
        - p_value
        - engaged
        - limits
      title: CounterfactualRun
      type: object
    Problem:
      additionalProperties: false
      description: RFC 9457 problem details; `code` comes from the versioned error catalog.
      properties:
        code:
          title: Code
          type: string
        detail:
          anyOf:
            - type: string
            - type: 'null'
          default: null
          title: Detail
        request_id:
          anyOf:
            - type: string
            - type: 'null'
          default: null
          title: Request Id
        status:
          title: Status
          type: integer
        title:
          title: Title
          type: string
        type:
          default: about:blank
          title: Type
          type: string
      required:
        - title
        - status
        - code
      title: Problem
      type: object
    CounterfactualEngagement:
      additionalProperties: false
      description: >-
        Where the items the person engaged with went without the element, over
        the completed cases.
      properties:
        gained:
          description: Within the variant's first `k` and not the base's.
          title: Gained
          type: integer
        items:
          title: Items
          type: integer
        kept:
          description: Within the first `k` of both lists.
          title: Kept
          type: integer
        lost:
          description: Within the base's first `k` and not the variant's.
          title: Lost
          type: integer
        mean_shift:
          anyOf:
            - type: number
            - type: 'null'
          description: >-
            The variant's position minus the base's, averaged over the items
            both lists hold; positive when they fell.
          title: Mean Shift
        shown:
          description: Within the first `k` of the base's list.
          title: Shown
          type: integer
      required:
        - items
        - shown
        - kept
        - lost
        - gained
      title: CounterfactualEngagement
      type: object
  securitySchemes:
    sourceKey:
      type: http
      scheme: bearer
      description: nia_sk_...
    personToken:
      type: http
      scheme: bearer
      bearerFormat: JWT

````