Monte Carlo

Shuffle and bootstrap resampling of trade P&L into many equity paths, the percentile bands and ruin statistics over them, and the seed rule that keeps every run reproducible.

One backtest gives one equity path — the order the trades happened to land in. crates/ vike-analytics/src/montecarlo.rs characterises how much of the result was that ordering: its header describes the module as building many synthetic equity paths from the observed trade list to characterise outcome dispersion. The Monte Carlo and overfitting tutorial runs it end to end over a finished run.

Two resampling methods

ResampleMethod has two variants, each documented by what it preserves:

Shuffle reorders the trade list, preserving the exact multiset of P&Ls per path. Every path therefore ends at the same terminal equity — start equity plus the sum of the original P&Ls — and what varies is the path: the sequence, and so the drawdown.
Bootstrap samples with replacement, so a path may draw one trade several times and another never. Terminal equity varies as well as the path.

Running it

mc_resample(trade_pnls, start_equity, n_sims, seed, method) runs n_sims paths. Each path is built by adding the resampled P&Ls to start_equity cumulatively — the curve has one more point than there are trades. The McResample it returns carries:

terminal — the final equity of each path
max_drawdowns — the max-drawdown fraction of each path, computed by metrics::max_drawdown (the module's own copy was byte-identical and was removed in its favour)
curves_sample — a representative sample of full curves, up to 20 paths spread evenly across the simulations, for plotting

confidence_bands(curves, qs) then turns equal-length curves into per-step percentile bands: for each requested quantile, a series of the value at that quantile across all paths at each step, returned as (quantile, values) pairs in the order of qs. The percentile is metrics::percentile — the workspace's one percentile, linear interpolation, named numpy_linear on the metrics page.

risk_of_ruin(terminal_equities, ruin_level) is the fraction of paths that end at or below the level; 0.0 for an empty input.

The summary

mc_summary(trade_pnls, start_equity, n_sims, seed, ruin_pct) runs the shuffle method and reduces the result to a McSummary:

field	meaning
`terminal_p5`, `terminal_p50`, `terminal_p95`	the 5th, 50th and 95th percentiles of terminal equity
`max_dd_p50`, `max_dd_p95`	the median and 95th-percentile max drawdown across paths
`prob_loss`	the fraction of paths ending at or below `start_equity`
`risk_of_ruin`	the fraction ending at or below `ruin_pct · start_equity` — ruin is defined by that product

Because the summary uses the shuffle method, every path's terminal equity is the same and the terminal percentiles collapse to it; the drawdown percentiles are the informative rows, and they are exactly the question a shuffle asks — how bad could the sequence have been.

The seed, and why this module is the worked example

The header states the reproducibility contract: seeded, so the same seed gives the same output run to run within this implementation. The generator is rand's StdRng seeded from the u64 you pass, and the manifest notes the dependency exists for seeded resampling — statistical, not bit, parity.

A seed that is a parameter is an input

The workspace's determinism gate, in crates/vike-ops/tests/clock_pin.rs, refuses any RNG dependency inside the crates that must replay deterministically — and its doc names vike_analytics::montecarlo as the worked example of the acceptable shape: a seeded generator whose seed is a caller-supplied parameter is an input, not hidden state, and the code lives outside the replay-critical scope. What the gate refuses is the dependency arriving inside the fold at all, because the difference between StdRng::seed_from_u64(seed) and a thread-local generator is one line in a diff nobody is looking at. The replay page is the other side of that rule.

What is not pinned, and why

There is no golden fixture for this module

The header is explicit. Porting the retired implementation's exact random generator — its Mersenne Twister plus its specific shuffle and range-draw routines — was ruled out as not worth it for this one file, so no fixture was ever frozen and no test replays one. What survives is the provenance of that decision: when the two implementations were run side by side, the same seed produced statistically equivalent, not bit-identical, resampled paths. The tests here pin the invariants instead — a shuffle path's terminal equals start plus the sum of P&Ls exactly, the same seed is deterministic, different seeds differ, a bootstrap path's steps are all drawn from the original P&Ls, and the summary's percentiles are ordered with its probabilities bounded in [0, 1].

Read with

The overfitting page is the other half of asking whether a result is real: resampling tells you how much the ordering mattered, the anti-overfitting statistics tell you how much the selection did.