Pub/Sub Pattern — Notes

Why use it

• Decouple producers from consumers.
• Buffer bursts; consumers process at their own pace.
• Fan-out: one event drives many side-effects (search index, cache invalidation, analytics).

Broker comparison (rough)

	Kafka	RabbitMQ	SQS	Pub/Sub (GCP)	NATS JetStream
Model	log, partitioned	exchange + queue	queue	log-like	log
Order	per-partition	per-queue	best effort (FIFO opt)	per-key	per-subject
Retention	days–months	until consumed	4 days max	7 days	configurable
Throughput	very high	medium	high	very high	very high
Use case	event streaming	task queue	task queue	event streaming	edge messaging

Patterns

• Outbox pattern: write DB row + outbox event in same tx; relay to broker.
• Saga: long-running tx via events + compensations.
• Event sourcing: state = replay of events.

Pitfalls

• Schema evolution: use Avro/Protobuf + schema registry, enforce compatibility.
• Hot partitions: bad key choice → uneven load.
• Consumer lag: alert on it.

Refs

• Kafka design doc (LinkedIn), Confluent EOS post, RabbitMQ in Depth (book), AWS SQS/SNS architecture, NATS JetStream docs.