Skip to main content

System Design

The FirstBreath Vision System solves a critical scalability problem: Python threads cannot efficiently handle concurrent heavy AI inference loops due to the GIL (Global Interpreter Lock).

To bypass this, we implemented a Producer-Consumer pipeline using Redis as a high-speed broker.

High-Level Data Flow​

Design Principles​

1. Separation of Concerns​

  • I/O Bound work (reading cameras) is separated from Compute Bound work (Inference).
  • This allows us to scale the number of cameras independently of the number of GPUs.

2. Micro-Batching​

  • Instead of processing frames one-by-one, we aggregate them into batches.
  • Why? GPUs are massively parallel. Processing 8 frames at once is significantly more efficient than processing 8 frames sequentially.

3. Ephemeral State​

  • No video is stored on disk (unless recording is explicitly triggered).
  • Redis acts as a short-lived buffer (~500ms max history) to smooth out load spikes.