URL Filter / Web Safe Browsing — Detailed#

flowchart TB
  subgraph Client
    BR[Browser / mail / DNS resolver]
    LCACHE[Local hash prefix DB]
  end

  subgraph Lookup[Lookup]
    HASH[Hash URL + canonicalize]
    PREFIX[4-byte prefix check]
    FULL[Full hash exchange]
  end

  subgraph Server
    SVC[Safe Browsing Service]
    DB[(Threat lists<br/>phishing, malware, social-eng, unwanted)]
    UPD[Update channel]
    HEUR[ML heuristics]
    REPORT[Reports + telemetry]
  end

  subgraph Sources
    HONEY[Honeypots]
    CRAWL([Crawlers])
    PART([Industry partners])
    USER([User reports])
  end

  Client --> Lookup --> Server
  Sources --> Server

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class PART,USER client;
    class BR edge;
    class HASH,PREFIX,FULL,SVC,UPD,HEUR,HONEY service;
    class LCACHE,DB datastore;
    class CRAWL compute;
    class REPORT obs;

Privacy preserving lookup (Google Safe Browsing v4)#

Client downloads only 4-byte hash prefixes of bad URLs.
On match (1 in millions), client asks server for full hash; server replies with full hashes for that prefix bucket.
Server cannot derive the user's exact URL with high confidence.

Glossary & fundamentals#

Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.

Tag	Concept	What it is	Page
`LLD`	Async models	futures / async-await / coroutines / actors	async-models