Objectives

Every ranked search — the grid, successive halving, TPE — ends by sorting rows on one number. crates/vike-backtest/src/objective.rs defines that number: an Objective is a boxed Fn(&BacktestReport) -> f64, higher is better, and run_sweep_with ranks a sweep's rows by any of them, stamping each successful row's score. The module is feature-free by design, like the report it scores: nothing in it names the gated profile parser or the DataFusion tree, so leaf crates reuse the same objectives and its tests run in the default lane.

The built-in objectives

The four classic --rank-by metrics are exposed as objective constructors through RankMetric::objective in crates/vike-backtest/src/harness/sweep.rs, with direction folded in — MaxDrawdown becomes −max_drawdown, the other three pass through — so ranking by Sharpe.objective() through run_sweep_with orders rows exactly as the classic comparator does, NaN-last included; metric_objectives_rank_identically_to_cmp_reports gates it. The classic run_sweep keeps its own comparator untouched, so the default output stays byte-identical and a SweepReport's rank_by serialises as the same bare metric string; an objective-ranked report carries its label instead, e.g. "multi".

Two laws

Both are property-tested, because a sweep's first row feeds downstream selection and "degenerate points cannot be crowned" has to be provable rather than intended.

NaN is unrankable. A NaN score is legal and means the row cannot be ranked; cmp_scores_desc — the one best-first comparator every score-ranked sort in and out of the crate goes through — puts it after every finite score, and sort_scored_rows puts rows with no score at all (failures) last, stably. Every multi_metric input that is NaN poisons the score rather than being substituted, and a non-finite score serialises as null.

The sign law. Every losing run scores strictly below every profitable one, whatever its other statistics say. Only multi_metric can violate it, so only it is bound by it: its shaping terms are factors that mostly sit under 1, and applied multiplicatively to a negative base they would shrink a loss toward zero — reward it — and rank the most catastrophic point first on an all-losing grid, the common case when tuning a bad strategy.

The multi-metric score

multi_metric(params) builds the composite objective; --rank-by multi uses the default MultiMetricParams. The formula, as the type's doc states it:

base    = total_return - max_drawdown
shape   = ln(1 + profit_factor)            // profit_factor clamped to profit_factor_cap
        * winrate_coeff                    // winrate_floor + (1 - winrate_floor) * win_rate
        * trade_count_penalty(n_trades)
        -> clamped into [0, shape_max],  shape_max = ln(1 + profit_factor_cap)
quality = shape / shape_max

score   = if base > 0 { base * shape }
          else        { base * (1 + loss_quality_weight * (1 - quality)) }

So the zero point means "did nothing": above it a quality-shaped profit, below it a quality-amplified loss, and exactly zero a breakeven, no-trade or no-winner run there is nothing to rank. The defaults: target_trades 50 (the statistical-significance knee, at and above which trade_count_penalty is exactly 1.0 and below which it falls linearly), penalty_floor 0.1 (so a thin profitable point stays visible), winrate_floor 0.0, profit_factor_cap 100.0 and loss_quality_weight 1.0 — the worst-quality loss counts double, which keeps loss size the dominant term; only losses within 2× of each other can be reordered by quality, 0.0 disables the shaping, and a negative weight is treated as zero because it could flip a loss positive. The cap exists because metrics::profit_factor returns the house f64::INFINITY sentinel for a run with no losing trades — unclamped, every all-win point would score infinite and rank first — and ln(101) ≈ 4.6 leaves any realistic profit factor untouched.

The clamp on shape is what makes the sign law hold by construction rather than by trusting the report's provenance: a hand-built report or an exotic floor could otherwise make shape negative and flip a profitable run below a losing one, and f64::clamp passes NaN through unchanged so a NaN metric still poisons the score instead of being laundered into a number.

Why the logarithm is not f64::ln

Every ln in the module is libm::log. IEEE 754 requires + − × ÷ and sqrt correctly rounded and requires nothing of ln, so the method spelling resolves to whatever libm the platform ships, and two platforms differ in the last bit. Here that decides a ranking: shape_max is the normaliser of the losing regime's quality term and the upper clamp of shape, so two boxes could order two near-tied rows differently and crown a different winning config off the same data. libm is one pure-Rust implementation compiled into the binary, so both boxes run the same code. What the module deliberately does not claim: this is not a live order path and not even the sweep's default ranking — multi is opt-in, and with the flag absent the binary takes RankMetric::default(), which is Sharpe, and nothing in the module is called. The exposure is a research one, and worth closing for exactly that reason rather than by inflating it into a trading hazard.

The metrics themselves are on metrics and the report; overfitting is what a crowned row is tested against next.