Data Flow and Background Processing

From store to client

1. A connector fetches a merchant page or partner feed and emits a versioned RawOfferCaptured record.
2. Integration Service validates source schema, maps identifiers, and normalizes money and availability.
3. Catalog resolves the product to a canonical game. Unmatched products enter a reviewed queue.
4. Pricing appends a price observation, updates the current projection, and emits OfferChanged.
5. Deals Engine recalculates qualifying deals; Notification and Realtime consume resulting events.

sequenceDiagram
  participant X as Connector
  participant I as Integration
  participant P as Pricing
  participant D as Deals Engine
  participant R as Realtime
  X->>I: RawOfferCaptured
  I->>P: NormalizedOfferUpserted
  P->>P: append observation + projection
  P->>D: OfferChanged
  D->>R: DealPublished / DealUpdated
  R-->>Client: deal.updated

Workers use at-least-once delivery. Every consumer deduplicates by event id; projections are updated with optimistic versions. Failed events retry with exponential backoff, then move to a dead-letter topic after 10 attempts.