Why Your Algorithm's Live Performance Differs: A Guide to Slice Processing in QuantConnect

Based on the QuantConnect LEAN documentation, the discrepancy between backtesting and live trading regarding missing data is caused by how the engine handles processing latency relative to the data resolution.

The Core Difference: Synchronous vs. Real-Time

1. Backtesting (Synchronous Execution)

In backtesting, the LEAN engine controls the flow of time. It operates synchronously.

• Behavior: If your OnData method takes a long time to process a specific data slice, the engine simply "pauses" the simulation clock.
• Result: The next data slice waits in a queue. Once your algorithm finishes the current calculation, the engine delivers the very next chronological slice.
• Outcome: You never miss data in backtesting, regardless of how inefficient or slow your code is.

2. Live Trading (Real-Time Execution)

In live trading, the engine cannot pause the market. Time moves forward regardless of your algorithm's state.

• Behavior: If your algorithm takes longer to process a slice than the time period that the slice spans (the resolution), the data arriving during that processing time is effectively "dropped" or skipped.
• Result: When your OnData method finally finishes execution, the engine delivers the most recent slice available at that exact moment, skipping the intermediate slices that occurred while the algorithm was busy.
• Outcome: This mechanism ensures that your algorithm always acts on fresh, relevant market data rather than executing trades based on stale prices from seconds or minutes ago.

Example Scenario

The documentation provides a specific example regarding high-frequency data:

• Scenario: You are subscribing to Second Resolution Crypto data.
• Performance: Your OnData logic takes 3.5 seconds to compute.
• Backtest: You receive every single second of data (T1, T2, T3, T4...). The simulation just takes longer to run.
• Live:
  1. You receive the slice at 12:00:00.
  2. Your code processes for 3.5 seconds.
  3. The engine skips data at 12:00:01, 12:00:02, and 12:00:03.
  4. The next event triggers at roughly 12:00:04.5, delivering the slice for the 12:00:04 timestamp.

Solutions

To resolve this in live trading, you must ensure your OnData execution time is significantly lower than your data resolution.

1. Optimize Code: Remove heavy computations (like training machine learning models or complex loops) from OnData.
2. Use Scheduled Events: Move heavy analysis to Train methods or Scheduled Events that run less frequently.
3. Lower Resolution: If your strategy cannot process data fast enough, switch from Resolution.Second to Resolution.Minute.

Q&A

Q: Does LEAN queue data in live trading if my algorithm is slow?
A: No. To prevent trading on stale data, LEAN skips intermediate slices and delivers only the most recent data once the algorithm becomes available again.

Q: How can I perform heavy computations without missing market data?
A: You should perform intensive computations (like model training) in the Train method or a separate Scheduled Event. These run in a separate thread (in live mode) or separate logic flow, preventing the blocking of the main OnData loop.

Q: Will OnData fire if there is no new data?
A: Yes, if "fill-forward" is enabled (which is the default behavior). The engine will generate data points based on the last known price to ensure the algorithm continues to tick, though this does not solve the issue of missing real data due to latency.