ppad-censor-0.6.0: Anytime-valid sequential constant-time testing.
Copyright(c) 2026 Jared Tobin
LicenseMIT
MaintainerJared Tobin <jared@ppad.tech>
Safe HaskellNone
LanguageHaskell2010

Censor.Runner.Report

Description

Uniform report rendering and suite scaffolding for censor executables. Three concentric layers:

  • The lowest layer is a header/case/footer data model streamed through a Renderer. Three renderers are provided — pretty (the ppad terminal aesthetic: bold brand line, dim key column, middle-dot separators, ANSI-colored verdicts on a TTY), json (one censor/report-v3 object per run, deterministic key order, per-case trace arrays when present), and plain (ASCII-only, grep-safe).
  • withReport streams cases through the chosen renderer and returns the accumulated list so callers can drive exit codes.
  • At the top, TraceSuite packages up the entire censor test-suite shape (arg parse, meter selection, header assembly, per-case calibration and attribution, trace record, JSON artefact write) into one call, so a suite's Main.hs contains only the hypotheses and case list.
Synopsis

Report config

data ReportCfg Source #

Runtime knobs for the renderer.

rcColor is set once at startup based on whether stdout is a terminal. All ANSI helpers short-circuit to identity when it is False.

Constructors

ReportCfg 

Fields

Instances

Instances details
Show ReportCfg Source # 
Instance details

Defined in Censor.Runner.Report

Eq ReportCfg Source # 
Instance details

Defined in Censor.Runner.Report

initReportCfg :: IO ReportCfg Source #

Initialise a ReportCfg by probing whether stdout is a TTY. Call once at the start of main and thread the result through the renderer callbacks.

Data model

data ReportHeader Source #

Static header emitted once, before any case row. Populate this at main entry with the run's own metadata; the brand line (ppad censor v<VER>) is fixed and does not appear here.

Constructors

ReportHeader 

Fields

Instances

Instances details
Show ReportHeader Source # 
Instance details

Defined in Censor.Runner.Report

data CaseReport Source #

A single case row.

Constructors

CaseReport 

Fields

Instances

Instances details
Show CaseReport Source # 
Instance details

Defined in Censor.Runner.Report

data ChannelReport Source #

A channel reported beside a case's verdict.

Constructors

ChannelReport 

Fields

Instances

Instances details
Show ChannelReport Source # 
Instance details

Defined in Censor.Runner.Report

data BatteryTail Source #

The tail passed to a renderer's rndClose. Carries the accumulated case list and, optionally, the path to a companion artefact (a trace JSON, a log) that the tool wrote alongside.

Constructors

BatteryTail 

Instances

Instances details
Show BatteryTail Source # 
Instance details

Defined in Censor.Runner.Report

Renderers

data Renderer Source #

A rendering strategy: one callback per lifecycle event.

The lifecycle is rndOpen (once), then rndCase per case (in order), then rndClose (once). Streaming renderers (pretty, plain) emit at each callback; buffered renderers (json) use the case callback as a no-op and emit the full object at close.

Constructors

Renderer 

pretty :: Renderer Source #

The ppad terminal aesthetic: bold brand line, a dim key column, middle-dot (·) separators, ANSI-colored verdicts on a TTY.

json :: Renderer Source #

Emit one censor/report-v3 JSON object per battery, deterministic key order. Header and per-case callbacks are no-ops; the whole object materialises at close, exactly as reportJSON would render it.

plain :: Renderer Source #

ASCII-only, no ANSI, no unicode. Suitable for CI logs and grep pipelines that dislike middle dots.

Driver

withReport :: ReportCfg -> Renderer -> ReportHeader -> ((CaseReport -> IO ()) -> IO (Maybe FilePath)) -> IO [CaseReport] Source #

Stream a battery through a renderer.

The body callback is handed an emit action that records one case row; call it after each hypothesis completes. The body may return the path of a companion artefact (e.g. a trace JSON) to surface in the footer. The accumulated case list is returned so callers can drive an exit code off the verdicts.

Format flags

data Format Source #

Selectable output format.

Constructors

FmtPretty 
FmtJson 
FmtPlain 

Instances

Instances details
Show Format Source # 
Instance details

Defined in Censor.Runner.Report

Eq Format Source # 
Instance details

Defined in Censor.Runner.Report

Methods

(==) :: Format -> Format -> Bool #

(/=) :: Format -> Format -> Bool #

parseFormat :: String -> Maybe Format Source #

Parse a --format argument: pretty, json, or plain.

takeFormatArg :: [String] -> (Maybe String, [String]) Source #

Position-insensitively pluck a --format FMT pair out of an argv list. Returns the format token if present and the argv with that pair removed.

resolveFormat :: Maybe String -> IO Renderer Source #

Resolve an optional format token to a Renderer, dying with a clear message on unknown values. Defaults to pretty when the token is absent.

Standard suite arg surface

data SuiteArgs Source #

The parsed suite argv: [--format FMT] [--batch N] [--margin M] [METER] [EXTRA...]. runTraceSuite takes the first extra as an output path.

Constructors

SuiteArgs 

Fields

parseSuiteArgs :: IO SuiteArgs Source #

Parse the standard suite argv from getArgs. Dies with a clear message if --format or --batch is malformed. The extras list is handed to the runner for further interpretation.

Host environment (re-exports)

data Env Source #

A host-environment snapshot, taken once at run start.

The Linux knob fields (envGovernor, envBoost, envSmt, envParanoid) hold the raw sysfs/procfs file contents; they are Nothing wherever the file does not exist (in particular, everywhere but Linux).

Constructors

Env 

Fields

Instances

Instances details
Show Env Source # 
Instance details

Defined in Censor.Runner.Env

Methods

showsPrec :: Int -> Env -> ShowS #

show :: Env -> String #

showList :: [Env] -> ShowS #

Eq Env Source # 
Instance details

Defined in Censor.Runner.Env

Methods

(==) :: Env -> Env -> Bool #

(/=) :: Env -> Env -> Bool #

captureEnv :: IO Env Source #

Snapshot the host environment. Cheap (a few syscalls and file reads); call once at run start.

Suite scaffolds

data BatchPolicy Source #

How a case's cfgBatch is chosen. The distinction is a repeatability contract the case author asserts about the target, and it cannot be inferred: batching only multiplies real work when each repetition genuinely recomputes.

Constructors

SingleShot

One execution per timed region. The correct policy for a pure Haskell target such as \a -> () <$ evaluate (f a): the f a thunk is shared across the batch and forced once, so repetitions beyond the first are no-ops on an already-evaluated value and a calibrated batch of N would describe one call plus N-1 no-ops. If the target is sub-quantum under the meter, widen the work rather than batching.

RepeatableAuto

Each repetition genuinely recomputes, so calibrate the batch with calibrateBatchReport. Appropriate for foreign (FFI) calls and targets driving mutable state — anything that cannot be memoised by a thunk.

FixedBatch !Int

Pin the batch outright, bypassing calibration. Use when batches must match across meters for comparability.

Instances

Instances details
Show BatchPolicy Source # 
Instance details

Defined in Censor.Runner.Report

Eq BatchPolicy Source # 
Instance details

Defined in Censor.Runner.Report

data TraceCase Source #

One case in a TraceSuite: a display name, its BatchPolicy, and an IO action producing the hypothesis. Existentially quantified over the hypothesis input type so a suite can hold cases with different payload shapes in one list.

TraceCase "encrypt" SingleShot     posAeadEncrypt
TraceCase "mul"     RepeatableAuto posFfiMul

The action is re-executed for the calibration probe (under RepeatableAuto), the main run, and (on Reject) the negative controls, so the builder should be inexpensive and side-effect free.

Constructors

TraceCase !String !BatchPolicy !(IO (Hypothesis a)) 

data TraceSuite Source #

A trace-recording suite: iterate a list of hypotheses under a chosen meter, record each run's per-pair frame trajectory, and write the whole battery as a single censor/report-v3 JSON artefact alongside the human-facing report.

The tool's Main.hs builds one of these and calls runTraceSuite; everything else — arg parse, meter selection, per-case calibrateBatch, record, auto-attribution on Reject, header assembly, withReport driver, JSON write — is done inside.

Constructors

TraceSuite 

Fields

runTraceSuite :: TraceSuite -> IO () Source #

Drive a TraceSuite end to end. The JSON artefact is written to the first positional argument after the meter, defaulting to <progName>-<meter>.json; it carries the same content the json renderer emits to stdout under --format json.

A host-environment snapshot (captureEnv) is taken at run start and rides on the header. Per case: calibrateBatchReport picks the batch and records the meter's noise floor at it (both surface on the case row and in the JSON) unless --batch N pins the batch (no probe, no noise floor — the honest setting for pure Haskell targets is --batch 1), baselineReading records the target's baseline cost on fresh class-A draws at the resolved batch (every policy — fresh draws are genuine work even for pure targets), record streams the run and keeps its frames, and on Reject attribute runs the A/A + B/B diagnostics automatically (two extra runs).

The meter cycles+wall (splitMeterName) runs each case under the split meter (runSplitWith): the case's result is the cycles channel, and the period and wall channels of the same pairs are reported beside it (crChannels). Calibration and the baseline read cycles, and the controls run when any channel rejects.

JSON serialisation

reportJSON :: ReportHeader -> [CaseReport] -> String Source #

Serialise a header + case list as the canonical censor/report-v3 JSON object. Deterministic key order; the per-case block carries crBatch and crTrace as optional fields, present exactly when the runner recorded them.

The two seeds are named apart: samplerSeed at the top level is the harness-wide seed behind the input samplers, and orderSeed inside each case result is the resolved per-run A/B order seed that replays that case.

A split run's case carries a channels array: the period and wall channels read on its own pairs, each with its name, unit, result, and controls on a rejection. The case's result is the cycles channel, which carries the verdict.

The json renderer and runTraceSuite both call this so stdout output and the on-disk artefact are byte-identical.

resultToJSON :: Result -> String Source #

Serialise a Result as a stable JSON object. Includes verdict, pairs, mixture peak log e-value, anytime-valid p-value, effect interval, clip counts at each of the three magnitude bounds, warmup bound, the resolved order seed, the CDF channel as [cut point, peak] pairs, per-component peaks, and the diagnose advisories.

The seed here is the order seed (resSeed), the one that replays the A/B order sequence. The sampler seed is a property of the harness, not of a run, and appears in the report header.

The object is a single line; pipe multiple runs into a line-oriented store and diff or filter with jq.