vike
Concepts

The live data seam

DataClient and LiveDataSink — the live mirror of the history store that every venue feed implements, where subscribing returns nothing and every datum arrives through a sink chosen at construction.

crates/vike-data/src/live.rs is the live twin of the history store: DataClient is what a venue feed implements to start and stop streams, and LiveDataSink is where everything a stream produces lands. The contract carried over from an earlier prototype is fire-and-forget. subscribe_bars, subscribe_quotes, subscribe_trades, subscribe_book and subscribe_depth start a stream and return a SubscriptionId immediately; they never carry data. All data returns through the one sink handed to the client at construction — the trait methods take no sink parameter, and there is no subscribe-with-a-callback shape. An Err from a subscribe call means the stream could not be started; a disconnect, a reconnect or a resubscribe after a drop surfaces on the feed's own status path, never through this trait. Reconnect is the feed's concern, not the seam's.

Capabilities, not obligations

A client that cannot serve a verb returns LiveDataError::Unsupported rather than pretending to subscribe. require_live_verb drives that refusal off the venue's declared VenueCaps.live_data row, so a feed that calls it first in every subscribe_* verb cannot drift from the capability matrix, and an unknown venue string refuses every verb. Most bridge feeds do call it; a few still hand-roll their own decision — hyperliquid's snapshot-only subscribe_book and polymarket's subscribe_bars are the ones left — and routing those through the seam is the noted follow-up. subscribe_depth — the DOM's conflating snapshot lane, distinct from the lossless tick-lane book — is Unsupported by default and only venues with a live depth feed override it.

Two lanes, and which price verb a feed must use

Sink methods are called from feed worker threads on every closed bar, intrabar update and tick, so they must be cheap and effectively non-blocking — a slow sink stalls the feed. Downstream is a lossless bar lane, seed_bars and close_bar, where a missed close is a series hole, plus a latest-wins conflating lane, forming_bar, bar_close_tick and mark_tick, where drops are fine and only the newest snapshot matters.

The price verbs are not interchangeable, because each fills a different rung of the price board's resolver chain, and that chain is strictly ordered: mark above side-aware bid/ask above last_trade above bar_close. A feed choosing the wrong verb does not mislabel a number; it re-ranks it against every other price for the symbol. The mapping is fixed:

verbwhat may call itrung
mark_tickonly a venue-published mark or index stream — binance-family @markPrice@1s, bybit tickers.markPrice, okx mark-price, hyperliquid activeAssetCtx.markPxmark
quotea BBO or two-sided book topbid / ask
tradea real printlast trade
bar_close_ticka kline feed's candle close, from every venue that serves klinesbar close — the lowest rung, up to one full interval stale

Mark streams are default on for perps; aster's is default off, its grammar unverified, and opts in with VIKE_MARK_STREAMS_ASTER=1; VIKE_MARK_STREAMS=0 is the master kill. The classes overlap freely: a venue with no mark stream still fills the quote and trade rungs whenever those subscriptions are open, so such a symbol is valued at its live quote or last trade rather than its candle close — the intended law.

The L2 verbs split the same way. book carries folded state behind an Arc — a pump keeping one standing book hands out clones and mutates through Arc::make_mut, so an applied delta no longer deep-clones two whole maps per emission — while book_update is the recordable twin carrying the raw delta, snapshot anchor or status marker that only recording sinks opt into; today the polymarket pump in book mode is the one feed that emits it. l2_snapshot carries the instrument's tick_size so a book-consuming sink quantizes on the grid the feed used rather than re-inferring it from a truncated level set.

Stream health

stream_status is a default no-op that a feed's idle watchdog and reconnect path fire. GapStart says the stream can no longer be trusted from a timestamp: for tick lanes the data in the gap is missing and unrecoverable, for the forming-bar lane it is stale, and the lossless bar lane is repaired by the re-seed before Live fires. Stale is the case a socket-liveness watchdog cannot see — the transport is alive, keepalives arrive, and no fresh data has come inside the subscription's freshness threshold. Both can be true at once, and only the consumer can decide what staleness means: flatten, re-arm, widen.

Teardown costs one timeout, not the sum of them

A feed thread learns it should stop on its next socket read timeout, 2 s on every on-driver venue. DataClient::begin_shutdown raises every stop flag and joins nothing, so a caller holding several clients runs that phase across all of them before any join; shutdown then joins. vike-recorder's teardown used to stop one stream at a time, outside its bounded region, and its total stop time was on a collision course with the unit's TimeoutStopSec= as a profile grew — a binance subscription is two threads per symbol. FeedRegistry is the shared per-subscription registry behind that: one stop flag and one JoinHandle per stream, nothing ever detached, with an optional hook that runs on each spawned thread — the seam a venue threads its core-affinity pin through without this crate depending on vike-exec. Bybit, binance, okx, hyperliquid and polymarket use it, and so do deribit, ig and oanda.

TeeSink fans one feed's sink slot out to several destinations in order — the core ingest adapter plus a tick-recording sink is the shipped composition — and must forward every verb explicitly: inheriting a trait default for bar_close_tick, l2_snapshot or book_update would silently drop candle-close ticks, depth snapshots or recordable book events. On the consuming side, the desktop app's CoreSinkAdapter bridges the sink onto the core's bar and market ingest lanes, and the RecorderSink writer actor behind the hist-datafusion feature buffers live quotes, trades and book events into the store — the path recording a feed rides.

On this page