vike
Data providers

Log volume

Why a long backfill or an open desktop window can write gigabytes of JSON log, and the knobs — VIKE_LOG_FILE_LEVEL, the file-level preference, and retention — that bound it.

Every Vike Trader binary logs through vike-log: a console layer on stderr and a non-blocking JSON daily-rolling file layer. The two are filtered independently, and that independence is the whole hazard: RUST_LOG (aliased VIKE_LOG) filters the console only. The file layer defaults to trace and captures the log→tracing bridge, so a batch tool driving a chatty dependency — DataFusion, arrow, hyper — writes an enormous JSON file that no console setting can turn down.

Measured, not hypothetical

crates/vike-log/src/lib.rs's file_level_directive records the cost: one hour of pmxt_backfill wrote 6.4 GB, and a long range once wrote 341 GB and nearly filled the disk hosting a live trading node. The operator's half does not follow from the knob — set it, and on a long run still watch the log directory grow with du -sh $VIKE_LOG_DIR.

VIKE_LOG_FILE_LEVEL

The file layer's directive is VIKE_LOG_FILE_LEVEL first, then the configured file_level. warn keeps the file useful without the per-row firehose; off silences it entirely. An empty or whitespace value falls through to the configuration rather than widening the filter. The registry row in vike_ops::settings::SETTINGS lists the default as trace; the variable is one of the environment reads catalogued on environment variables.

Three places set the configured half:

  • The backfill bins. crates/vike-backfill/src/cli.rs's log_config builds every batch bin's LogConfig with file_level at warn instead of vike-log's trace, because these bins run long and unattended. VIKE_LOG_FILE_LEVEL still wins, so a one-off debug run can ask for trace without a code change.
  • The daemon and the desktop app. preferences.log_file_level in preferences.toml (see preferences) is the file-level knob for vike-tradehub and vike-desktop: vike_config::CONSUMPTION cites crates/vike-tradehub/src/tradehub_cli.rs as its consumer and notes vike-desktop's main sets it identically. crates/vike-config/src/preferences.rs's DEFAULT_LOG_FILE_LEVEL keeps it at trace so the crate changes nothing by itself; the field exists so a deployment can turn it down in a file instead of an environment variable. preferences.log_level is the console twin.
  • Everything else inherits LogConfig::default(): console info, file trace.

vike-tradehub can apply a changed level without a restart: vike_log::LogReloadHandles recomposes the directive through the same pure functions init used, so a reload can never produce a filter a restart would not. reload_file_level returns an error when no file layer is installed, because a level applied to a layer that does not exist has been applied to nothing.

Retention is the disk bound; the level is the write-rate bound

Turning the level down bounds how much is written; nothing bounds how much is kept unless something deletes files. LogConfig::file_max_files — default vike_log::DEFAULT_MAX_LOG_FILES, which is 3 — prunes to the newest N daily files at init, so the count is a retention in days. Three is chosen against the failure it is for: a fault is diagnosed from the run that failed and the one before it. None restores keep-everything. It is deliberately a LogConfig field and not an environment variable: a retention only an operator can arm is a retention nobody has armed, which is exactly what VIKE_LOG_FILE_LEVEL had been.

Pruning matches filename.starts_with(prefix), which is why LogConfig::file_prefix now defaults to empty, meaning "derive from this executable" (effective_file_prefix). The old shared default "vike" starts both vike-desktop.<date> and vike-tradehub.<date> in the one shared log directory, so a one-shot backfill would have deleted the GUI's and the live daemon's logs.

The desktop client is its own firehose

A render loop logs at frame rate. HIGH_VOLUME_TARGETS pins wgpu_core, wgpu_hal, naga and the eframe::native subtree to info in the file, and its doc carries the measurement: a freshly installed Windows desktop client with no VIKE_LOG_FILE_LEVEL set wrote 5.8 GB in 4h10m — about 1.4 GB an hour, roughly 11 GB overnight — of which Vike's own lines numbered about twenty-five; the rest was wgpu_core and eframe::native, tens of lines per frame. Retention cannot help there, because one day is the 5.8 GB. The pins keep wgpu's one-time adapter and device lines, which a GPU bug report needs, and drop the per-frame lifecycle. An explicit mention wins — VIKE_LOG_FILE_LEVEL=wgpu_core=trace restores the firehose when the renderer is what you are debugging — and a pin may only ever narrow: it is never appended to a directive already at off, error or warn.

Two targets are pinned for a different reason

CREDENTIAL_BEARING_TARGETS pins ureq and tungstenite to info even under trace, and this is not about volume. ureq prints a request's path and query only at trace — where a Telegram bot token sits in the URL path and other providers' API keys sit in query parameters — and tungstenite logs the full WebSocket URI at debug, including Binance's listenKey. The pins survive an operator-set trace; VIKE_LOG_FILE_LEVEL=ureq=trace is the typed-out opt-in. It is a denylist, so a newly added dependency that logs URLs would leak again.

Where the file lands

vike_log::resolve_log_dir has four layers, highest first: $VIKE_LOG_DIR; then a config file's log_dir or a --log-dir flag; then the project's own <project>/settings/state/logs, which every shipped binary passes; then <exe_dir>/logs as the last resort for a binary with no project above its working directory — target/debug/logs/… in a checkout, which cargo clean deletes. The file layer is best-effort: a destination that cannot be opened drops the layer with one line on stderr naming the directory and VIKE_LOG_DIR, and the console carries on. It used to panic the process with exit 101 for the sole reason that it could not open a log file; crates/vike-log/tests/unwritable_log_dir.rs is the gate.

The rule to carry: set VIKE_LOG_FILE_LEVEL for any long backfill or replay run, check the retention is armed, and treat a Vike-owned line count of twenty-five in a multi-gigabyte file as the signature of a missing pin rather than a busy system.

On this page