ppad-eproc-0.5.5: Anytime-valid sequential testing via e-processes.
Copyright(c) 2026 Jared Tobin
LicenseMIT
MaintainerJared Tobin <jared@ppad.tech>
Safe HaskellSafe-Inferred
LanguageHaskell2010

Numeric.Eproc.TwoSample

Description

Anytime-valid two-sample test of equality in law, by a hedge of e-processes on an antisymmetric statistic.

Draw pairs (a_t, b_t), one from each of two laws, independently. Under the null that the laws are equal the pair is exchangeable, so any antisymmetric function of it, f(a, b) = -f(b, a), is symmetric about zero, whatever the law. The hedge bets on several at once, each a component e-process for the null:

  • the sign of the difference d = a - b (ties skipped), against one law running above the other (Numeric.Eproc.Bernoulli.TwoSided);
  • d clipped at each of some bounds, against a shift in mean, without letting rare large differences dominate (Numeric.Eproc.Bounded);
  • at each of some cut points q, 1[a <= q] - 1[b <= q], against the laws' distribution functions differing at q (a change in spread, say, that leaves the mean alone).

The hedge's e-value is the components' mean, mixed and latched by Numeric.Eproc.Mixture: an e-process for the null; the test rejects when its supremum so far reaches 1 / alpha, with probability at most alpha under the null, at any stopping time. The hedge's price is the log of its number of components, against power on each.

Nothing ties the statistic to a pair. update takes any statistic symmetric about zero under the null, and one per cut point bounded by a given bound: the energy of a quartet (two draws from each of two laws), say, which swapping two of its draws negates. update_pair makes the statistics of a pair.

Under an interval null (configInterval), that the clipped difference's mean lies within a margin of zero, the clipped components bet against the interval and alone make the hedge's e-value; the sign and indicator components test the sharp null, and are stepped beside it but not mixed.

The clip bounds and cut points must not depend on the data the test sees: fix them from a warmup the test then discards (clip_bound, cut_points).

Example

>>> let Right cfg = config 1.0e-3 [1.0, 4.0] 1 [0.5]
>>> let ps = take 2000 (cycle [(1, 0), (1, 0), (0, 0), (1, 1)])
>>> decide cfg (foldl' (update_pair cfg) (initial cfg) ps)
Reject
Synopsis

Configuration and state

data Config Source #

Hedge configuration.

data State Source #

The hedge after some observations.

data Verdict Source #

Test outcome at the current sample count.

Reject means the wealth process has ever crossed the rejection threshold, so H_0 is rejected at level alpha. Once a state has rejected it stays rejected, even if subsequent observations drive the current wealth back below threshold; this is the supremum-style guarantee that Ville's inequality actually delivers. Continue means there is not yet enough evidence; collect more samples (or stop and report no rejection -- the type-I error guarantee holds for any stopping rule).

Constructors

Reject 
Continue 

Instances

Instances details
Show Verdict Source # 
Instance details

Defined in Numeric.Eproc.Common

Eq Verdict Source # 
Instance details

Defined in Numeric.Eproc.Common

Methods

(==) :: Verdict -> Verdict -> Bool #

(/=) :: Verdict -> Verdict -> Bool #

data ConfigError Source #

Reasons that a test-configuration smart constructor can reject its inputs. Returned by config, configInterval, config, config, config, and config.

Constructors

InvalidAlpha !Double

significance level outside (0, 1)

InvalidBounds !Double !Double

sample bounds violate lo < hi

InvalidNullMean !Double !Double !Double

null mean outside (lo, hi) (strict, to avoid div-by-zero in the safe-bet ceilings)

InvalidNullInterval !Double !Double !Double !Double

null-interval endpoints violate lo < m_lo <= m_hi < hi (strict at the sample bounds, to avoid div-by-zero in the safe-bet ceilings)

InvalidBaselineRate !Double

baseline rate outside (0, 1)

InvalidComponentCount !Int

component count not positive

InvalidGridSize !Int

grid size below 1

Instances

Instances details
Show ConfigError Source # 
Instance details

Defined in Numeric.Eproc.Common

Eq ConfigError Source # 
Instance details

Defined in Numeric.Eproc.Common

Construction

config Source #

Arguments

:: Double

significance level alpha

-> [Double]

clip bounds

-> Double

indicator statistics' bound

-> [Double]

cut points

-> Either ConfigError Config 

Build a Config for the sharp null, at a level in (0, 1), from positive clip bounds, the positive bound of the indicator statistics (one for a pair's), and the cut points.

>>> either (const False) (const True) (config 1.0e-3 [1, 4] 1 [0])
True

configInterval Source #

Arguments

:: Double

margin

-> Double

significance level alpha

-> [Double]

clip bounds

-> Double

indicator statistics' bound

-> [Double]

cut points

-> Either ConfigError Config 

Build a Config for the interval null that the clipped difference's mean lies within a positive margin of zero; each clip bound must exceed the margin.

initial :: Config -> State Source #

The hedge before any observation, its e-value one.

Calibration

clip_bound :: Double -> [Double] -> Double Source #

A clip bound from a warmup's statistics: twice the nearest-rank p99 of their magnitudes (the ceiling (0.99 m)-th smallest of m), and at least a floor.

>>> clip_bound 1 [0.1, 0.2, -0.3]
1.0
>>> clip_bound 0 (replicate 99 1 ++ [1.0e9])
2.0

cut_points :: [Double] -> [Double] Source #

Cut points from a warmup's pooled values: their nearest-rank quantiles at 0.10, 0.25, 0.50, 0.75 and 0.90, ascending, alike ones once. None from no values.

>>> cut_points [1, 2, 3, 4]
[1.0,2.0,3.0,4.0]

Streaming

update :: Config -> State -> Double -> [Double] -> State Source #

The hedge after one more observation: a statistic symmetric about zero under the null, and one indicator statistic per cut point, within the indicators' bound (clipped to it otherwise).

Precondition: one indicator statistic per cut point, in their order; any beyond them are ignored.

update_pair :: Config -> State -> (Double, Double) -> State Source #

The hedge after one more pair: its difference, and at each cut point whether its first value lies at or below it, less whether its second does (the indicators' bound being one).

decide :: Config -> State -> Verdict Source #

Reject iff the hedge's log e-value has reached log(1 / alpha) at some observation so far.

Inspection

log_evalue :: Config -> State -> Double Source #

The hedge's current log e-value: the log of its components' mean, of the clipped components' alone under an interval null.

log_evalue_sup :: State -> Double Source #

The supremum so far of the hedge's log e-value; zero before any observation.

p_value :: State -> Double Source #

The anytime-valid p-value: min 1 (exp (- log_evalue_sup)).

log_evalue_sign :: State -> Double Source #

The sign component's current log e-value.

log_evalue_magnitude :: State -> [Double] Source #

Each clipped component's current log e-value, in the order of the clip bounds.

log_evalue_cdf :: State -> [Double] Source #

Each indicator component's current log e-value, in the order of the cut points.

log_evalue_sup_sign :: State -> Double Source #

The supremum so far of the sign component's log e-value: which component's evidence peaked is the shape of a difference found.

log_evalue_sup_magnitude :: State -> [Double] Source #

The supremum so far of each clipped component's log e-value, in the order of the clip bounds.

log_evalue_sup_cdf :: State -> [Double] Source #

The supremum so far of each indicator component's log e-value, in the order of the cut points.

samples :: State -> Int Source #

How many observations the hedge has seen.