Ad Click Aggregation / Ad Server — Detailed

flowchart TB
  subgraph Edge[Ad request flow]
    APP[Publisher app / page]
    ADX[Ad Exchange]
    BID[Bidders / DSPs]
    CACHE[Ad creative cache]
    CDN
  end

  subgraph Serve
    SEL[Ad selection]
    PACE[Pacing engine]
    CAP[Frequency cap]
    BUDGET[Budget control]
  end

  subgraph Events[Event pipeline]
    IMP[Impression event]
    CLK[Click event]
    CONV[Conversion event]
    KAFKA[[Kafka]]
    DEDUP[Dedup window]
  end

  subgraph Stream[Stream agg - real-time]
    FLINK([Flink / Spark Streaming])
    HLL[HyperLogLog uniques]
    CMS[Count-Min hot keys]
    WIN[Sliding windows]
  end

  subgraph Batch[Batch - exact]
    SPARK([Spark / Beam jobs])
    LAKE[(Data lake)]
    DAILY([Daily reconciled aggregates])
  end

  subgraph Store
    HOT[(Hot KV: per-ad/per-campaign counters)]
    OLAP[(OLAP: ClickHouse / BigQuery / Druid)]
  end

  subgraph Consumer
    REP[Reporting dashboards]
    BIL[Billing]
    ALERT[Alerting]
  end

  APP --> ADX --> BID --> SEL --> CDN
  APP --> IMP
  APP --> CLK --> KAFKA
  IMP --> KAFKA
  CONV --> KAFKA
  KAFKA --> Stream --> HOT --> REP
  KAFKA --> Batch --> OLAP --> BIL

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class APP,ADX,BID,CACHE,SEL,PACE,CAP,BUDGET,IMP,CLK,CONV,DEDUP,HLL,CMS,WIN,BIL service;
    class LAKE,HOT,OLAP datastore;
    class KAFKA queue;
    class FLINK,SPARK,DAILY compute;
    class REP,ALERT obs;

Lambda / Kappa architecture

• Stream layer: fast, approximate counts (HLL, CMS).
• Batch layer: exact, idempotent, reconciles within minutes/hours.
• Serving merges both: real-time first, batch overwrites for accuracy.

Anti-fraud (clicks)

• Bot / non-human traffic filters.
• Per-ip / per-cookie velocity, device fingerprint, conversion rate sanity.

Glossary & fundamentals

Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.

Tag	Concept	What it is	Page
HLD	CDN	edge caching for static assets	cdn
HLD	Pub/Sub & message brokers	topics, consumer groups, delivery semantics	pub-sub-pattern
HLD	CAP / PACELC	C vs A under partition; L vs C otherwise	cap-pacelc
HLD	Probabilistic data structures	Bloom, HLL, Count-Min, MinHash, t-digest	probabilistic-data-structures
HLD	Batch & stream processing	Lambda vs Kappa, watermarks, windows	batch-stream-processing