Real-time Analytics — Detailed

flowchart TB
  subgraph Source
    WEB([Web pixels])
    SDK([Mobile SDK])
    SVC[Server events]
  end

  subgraph Ingest
    LB
    COL([Collector / Edge ingester])
    KAFKA[[Kafka]]
    SCHEMA[Schema registry]
  end

  subgraph Sess[Sessionization & enrich]
    SES[Session windowing]
    JOIN[[Stream-stream / stream-table joins]]
    ENRICH([Geo / device / UTM])
  end

  subgraph Stream
    FLINK[[Flink / Kinesis Analytics]]
    HLL[HLL unique counts]
    TUMBLE[Tumbling windows]
    SLIDE[Sliding windows]
  end

  subgraph Storage
    HOT[(KV / Redis hot counters)]
    OLAP[(Druid / ClickHouse / Pinot)]
    LAKE[(S3 / Iceberg)]
  end

  subgraph Serve
    API[Query API]
    DASH[Dashboards]
    ALERT[Real-time alerts]
    EXPORT[Export to warehouse]
  end

  Source --> LB --> COL --> KAFKA --> SCHEMA
  KAFKA --> Sess --> Stream --> Storage --> Serve

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class WEB,SDK,ENRICH client;
    class SVC,SCHEMA,SES,HLL,TUMBLE,SLIDE,API service;
    class OLAP,EXPORT datastore;
    class HOT cache;
    class KAFKA,JOIN,FLINK queue;
    class COL compute;
    class LAKE storage;
    class DASH,ALERT obs;

Glossary & fundamentals

Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.

Tag	Concept	What it is	Page
HLD	Pub/Sub & message brokers	topics, consumer groups, delivery semantics	pub-sub-pattern
HLD	Probabilistic data structures	Bloom, HLL, Count-Min, MinHash, t-digest	probabilistic-data-structures
HLD	Batch & stream processing	Lambda vs Kappa, watermarks, windows	batch-stream-processing