Survival statistics

Whether a signal's Sharpe survives its own autocorrelation and the fact that it was the best of many — an autocorrelation-robust t-statistic, a false-discovery correction, a block bootstrap and a reality-check p-value.

The metrics catalog answers "what was the Sharpe". crates/vike-analytics/src/stats.rs answers the question that catalog cannot: would this Sharpe survive its own autocorrelation, and the fact that it was selected as the best of many? Four pure functions over &[f64], written for signal research — where hundreds of variants are swept and the survivors need to be more than lucky.

The t-statistic

newey_west_tstat(x, lag) is the Bartlett-kernel Newey–West t-statistic for the mean of an autocorrelated series: V = γ₀ + 2 Σ (1 − k/(L+1)) γₖ over lags 1..L, then t = mean / √(V/n), with L = min(lag, n − 1). A per-bar PnL series is autocorrelated by construction — a position held for many bars earns correlated returns — so the ordinary t-statistic overstates significance. This is the correction, and lag should be at least the holding horizon. NaNs are dropped, not propagated. A degenerate input (fewer than two observations, or a non-positive long-run variance) returns 0.0 rather than a NaN or an infinity: callers compare against a threshold, and zero fails every threshold, which is the safe direction. The test pins the arithmetic: [1, 2, 3, 4] at lag 1 gives exactly 4.0.

two_sided_p(tstat) converts it: 2 · (1 − Φ(|t|)), using the normal rather than Student's t because these series are thousands of bars long, where the two agree to well past any threshold anyone gates on. A NaN statistic maps to 1.0; 1.96 lands on the familiar 0.05.

The false-discovery correction

Sweep 300 variants at a 0.05 p-value and you keep about fifteen from noise alone

bh_fdr(pvalues, q) is the Benjamini–Hochberg step-up procedure. It controls the expected false-discovery rate — the share of rejections that are wrong — at q, and returns a rejection mask in the caller's order. Step-up is the part worth understanding: it finds the largest rank k whose sorted p-value is at or below q·k/n, then rejects every hypothesis at rank k and below. A middling p-value can therefore be rejected on the strength of the ones beneath it. The doc says so directly: that is the intended behaviour and not a bug to "fix". The test pins it — [0.001, 0.08, 0.09] at q = 0.10 rejects all three, where rank two fails its own threshold and rank three passes.

The block bootstrap

block_bootstrap_sharpe(pnl, block_len, n_resamples, periods_per_year, seed) returns the 5th and 95th percentiles of the annualized Sharpe over resamples built from fixed-length contiguous blocks — Kunsch's moving block bootstrap. Resampling individual bars would destroy the autocorrelation that makes a signal's Sharpe uncertain in the first place and would report a falsely tight interval; contiguous blocks of block_len bars preserve the local dependence structure. The test makes the point concrete: an alternating ±1bp series resampled in blocks of two has every block summing to exactly zero, so every resample's Sharpe is 0.0, which a per-bar bootstrap could not do 200 times. A series shorter than one block plus a bar returns (0.0, 0.0): there is nothing to resample, and inventing an interval would be worse than declaring none.

Two honesty notes from the doc itself. The stationary bootstrap is a different procedure — geometric block lengths, wrapping at the series end — and is not what this function does; the name was corrected rather than reused. And the Sharpe inside the resample uses the population standard deviation, while sharpe_from_bar_pnl uses the sample form: not an inconsistency to fix, because one is a point compared against stored oracle numbers and the other is an interval that was never required to match anything.

The reality check

hansens_spa(pnl_matrix, n_resamples, block_len, seed) answers what a per-signal t-statistic cannot: given that the best of k candidates was selected as the best, is its performance still surprising? pnl_matrix is column-major, one inner vector per candidate, rows aligned by bar; shorter columns are truncated to the shortest. Each column is centred at its own mean, making the null "every candidate has zero expected PnL"; the maximum Sharpe across candidates is redrawn under that null by block bootstrap; and the p-value is the share of draws whose maximum matches or beats the observed one. An empty panel or a series shorter than one block returns 1.0 — reject nothing.

One index sequence, shared by every column

Candidates are resampled in lockstep. Independent per-column draws would destroy the cross-sectional dependence between candidates, make the maximum over columns stochastically larger, inflate the null distribution and cost the test power. The test pins it by duplicating a column: the p-value does not move.

What this computes is the recentred bootstrap max-statistic test — White's Reality Check on the maximum Sharpe — not the studentized variant, which standardizes each candidate by its own bootstrap variance before taking the maximum. The doc keeps the name because it is the module's public API, not because it promises the studentized form.

Reproducibility

Seeded streams here reproduce run to run within this implementation, and are deliberately not required to match any other library's generator: reproducing an exact stream buys nothing a confidence interval consumes. Consumers use the interval, not its last digits. Cross-platform determinism is a separate matter and was a real defect: crates/vike-analytics/tests/libm_platform_probe.rs records that the variance folds in block_bootstrap_sharpe and hansens_spa were written as powi(2), which one toolchain lowers to a platform pow() call in a dev build, so the same input hashed differently on two boxes. They are explicit multiplications now, and the two functions carry rows in that probe's PLATFORM_INVARIANT table.

The deflated Sharpe is the closed-form answer to the same selection question, for equity curves rather than signal panels. Monte Carlo resamples trade P&L rather than bar blocks. The per-bar PnL these functions consume comes from the signal backtest.