nkaz001/hftbacktest

HftBacktest is a high-frequency trading framework that provides a Rust-based engine for tick-by-tick backtesting and live deployment of market-making strategies. It supports accurate simulation of L2 and L3 order books with configurable latency, queue position, and fee models, while enabling live trading via IPC connectors for exchanges like Binance and Bybit. The system also includes Python bindings and data utilities to facilitate strategy development, performance analysis, and historical data preparation.

Introduces a new Bybit connector that enables live trading and market data ingestion for the Bybit exchange. The implementation includes WebSocket-based public streams for order book depth (1, 50, and 200 levels) and trades, as well as private streams for order, position, and execution updates. It features a REST client for account management (canceling orders, fetching positions) and an order manager that handles order lifecycle, symbol-to-order-ID mapping, and batch feed processing to ensure consistent state snapshots.

connector/src/bybit · high confidence

The library now includes a new \AlignedArray\ type in the \utils\ module that provides cache-line-aligned memory allocation (64-byte alignment) for performance-critical data structures, along with a \get\_precision\ function to calculate decimal precision from tick size values.

hftbacktest/src/utils · high confidence

Adds a new connector for Binance USD-margined Futures, introducing the \BinanceFutures\ module with REST and WebSocket integration. The implementation includes \MarketDataStream\ for subscribing to depth and trade updates, \UserDataStream\ for handling order and position events via listen keys, and an \OrderManager\ to synchronize order states across REST and WebSocket channels. Message parsing is handled in \msg/\ with deserialization for order responses, stream events (including \TRADE\_LITE\ and \ACCOUNT\_UPDATE\), and REST payloads, ensuring symbols are normalized to lowercase.

connector/src/binancefutures · high confidence

The \hftbacktest.stats\ package is now available, providing a \Stats\ class to compute and visualize performance metrics from backtest records. Users can generate summaries and plots (via Matplotlib or HoloViews) for metrics including Return, Sharpe Ratio, Sortino Ratio, Maximum Drawdown, and various volume/trade counts. The module supports both linear and inverse asset records, allows customization of parameters like book size and trading days per year, and handles data resampling and splitting for detailed analysis.

py-hftbacktest/hftbacktest/stats · high confidence

The \hftbacktest-derive\ crate now provides the \NpyDTyped\ derive macro, which automatically generates implementations for serializing struct fields into NumPy-compatible byte descriptions (handling endianess and type mapping for primitives like f64, i32, bool, etc.), and the \build\_asset\ procedural macro, which generates code to construct asset configurations by matching combinations of asset types, latency models, queue models, exchange models, and fee models.

hftbacktest-derive · high confidence

The connector module now provides a standalone Rust implementation for live trading, establishing communication with bots via Iceoryx IPC. It defines core traits (Connector, ConnectorBuilder, GetOrders) and a PublishEvent enum to handle order submission, cancellation, and market data feeds. The main entry point manages a receive task that translates bot requests into connector actions and a publish task that batches initial state snapshots (orders, positions, depth) and streams live events back to bots. Utility functions support HMAC-SHA256 and Ed25519 signing, robust numeric parsing with fallbacks, and exponential backoff strategies for connection resilience.

connector/src · high confidence

This change introduces the \LiveBot\ component, enabling users to run trading strategies in live market conditions with an interface mirroring the existing backtester. The \LiveBotBuilder\ allows registering instruments and configuring error handling or order hooks before connecting via an IPC channel. Additionally, a \LoggingRecorder\ is provided to track and log state values and market depth updates for live assets.

hftbacktest/src/live · high confidence

This change introduces the core Rust library (\hftbacktest/src\) for the HftBacktest framework, enabling high-frequency trading and market-making strategy development. The library provides tick-by-tick backtesting that accounts for feed and order latency as well as order queue position, supporting both Level-2 (Market-By-Price) and Level-3 (Market-By-Order) full order book reconstruction. It also includes a live trading bot module that allows deploying the same algorithmic code used in backtests to live markets. The implementation defines core types for events (feed, order, position, error), supports serialization via bincode for IPC communication, and exposes feature flags to enable backtesting, live trading, and S3 data access.

hftbacktest/src · high confidence

This change introduces a new connector for Binance Spot trading, adding the \connector/src/binancespot\ module with components for market data streams, user data streams, REST API client interactions, and order management. The implementation supports real-time depth and trade updates via WebSocket, order execution and cancellation via REST, and user account updates. It uses ed25519 for request signing and serde\_qs for deserializing authentication requests, ensuring compatibility with Binance's current API requirements.

connector/src/binancespot · high confidence

The live trading subsystem now uses a new IPC layer built on Iceoryx for high-performance inter-process communication. This change introduces a \Channel\ trait and an \Iceoryx\ implementation that handles sending \LiveRequest\ messages and receiving \LiveEvent\ updates between bots and connectors. Communication is configured via an optional TOML file specified by the \HFTBACKTEST\_CHANNEL\_CONFIG\ environment variable, allowing users to tune buffer sizes and bot limits, with sensible defaults applied if no config is found. Serialization is handled by bincode, and the implementation supports both publisher and subscriber roles for bidirectional data flow.

hftbacktest/src/live/ipc · high confidence

The examples directory now includes a complete, multi-step Python workflow for high-frequency backtesting, featuring scripts to fetch Binance ticker data, download historical L2 depth and trade data from Tardis, convert that data into the native .npz format, generate artificial order latency models, and execute parallel backtests or grid searches using a pre-compiled Rust binary. Additionally, new Rust examples demonstrate a standalone grid trading algorithm and a custom event handling approach using a local processor wrapper.

hftbacktest/examples · high confidence

The examples directory has been expanded with several new Jupyter notebooks, including 'Accelerated Backtesting' (a faster, simplified backtesting mode that excludes queue position modeling), 'Data Preparation' (for normalizing raw exchange feeds), 'Fusing Depth Data' (combining multiple depth streams for higher granularity), and 'GLFT Market Making Model and Grid Trading' (implementing the Guéant–Lehalle–Fernandez-Tapia model). Existing tutorials such as 'Getting Started' and 'High-Frequency Grid Trading' have also been updated to reflect current API usage and latency handling.

examples · high confidence

The collector now supports data collection from Binance (Spot, Coin-M Futures, and CM Futures), Bybit, and Hyperliquid. It connects to exchange WebSocket streams to receive real-time market data (such as trades, order book updates, and best bid/offer) and writes the collected data to gzipped files, rotating daily per symbol. To ensure reliability and compliance with exchange limits, the collector includes automatic reconnection with exponential backoff on errors and a rate-limiting throttler for fetching depth snapshots when stream updates are missed.

collector · high confidence

Added new data conversion utilities in \hftbacktest/data/utils\ for Binance Futures, Binance Historical Market Data, Bybit, Bybit Historical Market Data, DataBento, and Hyperliquid. These utilities convert raw exchange feed files (JSON, CSV, DBN) into the HftBacktest-compatible event format, handling specific requirements such as fused market depth for Bybit, L3 Market-By-Order conversion for DataBento, and synthetic order latency generation based on feed latency.

py-hftbacktest/hftbacktest/data/utils · high confidence

The \hftbacktest.data\ module now exposes new functions for ensuring data integrity: \correct\_local\_timestamp\ adjusts for negative feed latency by offsetting local timestamps, \correct\_event\_order\ splits and reorders events to handle reversed exchange timestamps, and \validate\_event\_order\ raises errors if events are out of sequence. These utilities help users clean and validate market data before backtesting.

py-hftbacktest/hftbacktest/data · high confidence

The backtesting engine now includes configurable models for transaction fees, order latency, and queue position estimation. Users can apply fee structures based on trading value or quantity, with support for maker/taker rates and directional fees (e.g., stamp duties). Latency is modeled via constant values or historical data interpolation, including support for cross-exchange latency adjustments. Queue position is estimated using conservative (RiskAdverse) or probability-based (ProbQueue) models to determine fill likelihood within the order book.

hftbacktest/src/backtest/models · high confidence

The backtest data module now supports parallel pre-loading of market data and adjusts for latency when the data collection site differs from the strategy execution site. This enables more accurate cross-exchange backtesting by accounting for network delays, while parallel loading improves performance by reducing the time required to prepare data for the backtest engine.

hftbacktest/src/backtest/data · high confidence

The Numpy-based data source has been expanded to allow loading backtest data directly from Amazon S3 buckets when the 's3' feature is enabled, removing the requirement for local file storage. Additionally, the module now includes strict validation for Numpy file headers, checking for field type consistency and proper alignment to prevent runtime errors caused by malformed or mismatched data files.

hftbacktest/src/backtest/data/npy · high confidence

The \py-hftbacktest\ Python bindings now expose the core Rust engine, enabling users to run backtests and live trading strategies directly from Python. This release adds FFI bindings for the \MultiAssetMultiExchangeBacktest\ and \LiveBot\ (using Iceoryx IPC), allowing order submission, modification, and cancellation. It also introduces Python-accessible market depth utilities, including \ROIVectorMarketDepth\ with range-of-interest queries, a market depth fusion module (\FuseMarketDepth\), and snapshot support for both backtesting and live modes.

py-hftbacktest/src · high confidence

The backtesting module has been rebuilt to support more accurate simulation of trading environments. A new \AssetType\ trait and implementations for \LinearAsset\ and \InverseAsset\ allow correct calculation of position values and equity for different contract structures. Event processing is now managed by a dedicated \EventSet\ that efficiently schedules and retrieves the next event based on timestamps. Order handling has been refactored to use an \OrderBus\ system that integrates with a \LatencyModel\, allowing strategies to simulate realistic order entry and response delays, including technical rejections. Additionally, the \BacktestRecorder\ has been updated to save performance metrics to CSV and NPZ formats, and the internal state management now correctly applies fills and fees using the new asset and fee models.

hftbacktest/src/backtest · high confidence

The Python bindings for HftBacktest have been updated to version 2.4.4. This release introduces a restructured module layout with dedicated files for bindings, order handling, recording, and state management. Key additions include support for live trading features (conditionally enabled), order modification capabilities, and improved market depth access via \best\_bid\_qty\ and \best\_ask\_qty\ properties. The \Recorder\ class now supports saving records to NPZ files and retrieving per-asset data. Additionally, the \BacktestAsset\ class now accepts NumPy arrays directly for data input and initial snapshots, and supports a \latency\_offset\ argument for order latency modeling to adjust for cross-exchange backtesting scenarios.

py-hftbacktest/hftbacktest · high confidence

The \hftbacktest/src/depth\ module has been restructured to introduce three new market depth backends: \BTreeMarketDepth\, \HashMapMarketDepth\, and \ROIVectorMarketDepth\. The new \HashMapMarketDepth\ is designed to be more robust than the previous B-Tree-based approach for L2 feeds, specifically handling missing depth feeds and preventing incorrect best bid/ask prices when levels are deleted. The \ROIVectorMarketDepth\ offers a high-performance variant that tracks only a specific range of interest (ROI) around the mid-price using vectors, which is beneficial for strategies computing values based on the immediate order book. Additionally, a \FusedHashMapMarketDepth\ is provided to handle fused market data events, including timestamp-based validation to ignore outdated events. These changes are accompanied by a unified \MarketDepth\ trait exposing \best\_bid\_qty\ and \best\_ask\_qty\ for easy access to BBO quantities, and improved snapshot and clear-depth logic to ensure state integrity.

hftbacktest/src/depth · high confidence

The L3 (Level 3, Market-By-Order) backtesting engine has been refactored to use new \L3Local\ and \L3NoPartialFillExchange\ processor implementations. This change introduces a unified \Processor\ trait and a new \LocalProcessor\ trait to standardize order interaction (submit, modify, cancel) and event processing across L2 and L3 models. The new L3 local processor now explicitly handles order modification requests by updating the local order state and sending replacement requests to the exchange model, addressing previous issues where order modifications were not correctly propagated or where \leaves\_qty\ failed to update. The exchange processor models enforce strict full-execution conditions for L3 orders, ensuring that liquidity-taking orders are fully filled at the best price and that limit orders only fill when they are at the front of the queue and meet price criteria. This refactor also includes fixes for order rejection handling and ensures that modified orders do not revert to their original state upon reinsertion.

hftbacktest/src/backtest/proc · high confidence

The project now provides a Rust-based framework for high-frequency trading backtesting and live bot deployment, featuring tick-by-tick simulation, order queue position modeling, and latency accounting. This release introduces a new data format for Rust and includes examples such as a grid trading strategy. Concurrently, the previous Python implementation (including \\_\init\\_.py\, \backtest.py\, and \latencies.py\) has been removed from this location.

hftbacktest · high confidence

Added a new test file (\test\_hftbacktest.py\) that validates the core backtesting functionality. The tests verify the ability to load market data from NumPy files, configure backtest assets with specific parameters (such as tick size, lot size, and queue models), and execute a simulation loop using the \ROIVectorMarketDepthBacktest\ engine. The test suite checks order submission, market depth retrieval, and trade clearing operations.

py-hftbacktest/tests · high confidence

This change introduces the Cargo.toml and pyproject.toml manifests for the HFTBacktest workspace, defining the project structure and dependencies for the core backtesting library, Python bindings, collector, and connector modules. It establishes Rust edition 2024 as the standard, sets the minimum Rust version to 1.91.1, and configures key dependencies including tokio 1.48.0, serde 1.0.228, bincode 2.0.1, and pyo3 0.27.2 for the Python interface. The Python package is configured to require Python 3.11+ and depend on numpy \>=2.0, numba \~= 0.61, and other visualization libraries, while the Rust side integrates features for S3 storage, Iceoryx2 IPC, and specific exchange connectors.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 62 → 63 (+1.9)
• Rubric changed (rubric-2026.09.11 → rubric-2026.09.18) — scores are not directly comparable.

• Code Health 76 → 75 (-0.5)
• Architecture 92 → 93 (+1.5)
• Maturity 68 → 68 (-0.1)
• Readiness 48 → 49 (+1.4)
• Security 68 → 79 (+10.5)
• Event Sourcing 100 → 100 (+0.0)
• Performance 100 (new)

• Boundary-crossing change coupling: connector.rs ↔ mod.rs (connector/src/connector.rs)
• Hotspot: hftbacktest/src/backtest/proc/nopartialfillexchange.rs (hftbacktest/src/backtest/proc/nopartialfillexchange.rs)
• Hotspot: hftbacktest/src/backtest/proc/partialfillexchange.rs (hftbacktest/src/backtest/proc/partialfillexchange.rs)

• Ambiguous naming for time progression. elapse and elapse_bt (likely 'backtest') suggest similar functionality (advancing time), but the distinction is unclear from the names alone. It is not immediately obvious if elapse_bt is a specialized version or if one is deprecated.
• Conflicting accessors for the same property. descr is exposed as both a property returning a typed DType and a method returning a String. This is inconsistent with Rust conventions (properties are fields/getters, methods are actions) and creates ambiguity on how to access the description.
• Dependency hygiene PARTLY measured — Cargo dependencies read, no committed lock to grade for currency
• Documentation: no architecture or design documentation (docs/debugging_backtesting_and_live_discrepancies.rst)
• Documentation: no architecture or design documentation (docs/reference/data_validation.rst)
• Documentation: no installation or build instructions (connector/README.md)
• Documentation: no usage examples (connector/README.md)
• Duplicated block (10 lines × 3) (examples/example_bybit.py)
• Duplicated block (10 lines × 4) (py-hftbacktest/hftbacktest/data/utils/tardis.py)
• Duplicated block (11 lines × 2) (py-hftbacktest/hftbacktest/data/utils/tardis.py)
• Duplicated block (14 lines × 2) (hftbacktest/examples/4_latency.py)
• Duplicated block (15–17 lines × 2) (py-hftbacktest/hftbacktest/data/utils/bybit.py)
• Duplicated block (15–17 lines × 8) (py-hftbacktest/hftbacktest/data/utils/binancefutures.py)
• Duplicated block (17 lines × 2) (py-hftbacktest/hftbacktest/data/utils/bybit.py)
• Duplicated block (29 lines × 2) (py-hftbacktest/hftbacktest/data/utils/tardis.py)
• Duplicated block (30–34 lines × 4) (py-hftbacktest/hftbacktest/data/utils/binancehistmktdata.py)
• Duplicated block (34 lines × 2) (py-hftbacktest/hftbacktest/data/utils/tardis.py)
• Duplicated block (40–44 lines × 3) (py-hftbacktest/hftbacktest/data/utils/binancehistmktdata.py)
• Duplicated block (49 lines × 2) (py-hftbacktest/hftbacktest/data/utils/binancefutures.py)
• Duplicated block (7–8 lines × 5) (py-hftbacktest/hftbacktest/data/utils/tardis.py)
• …and 9 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

nkaz001/hftbacktest was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.