Resque) — Detailed#

flowchart TB
  subgraph Producers
    APP[App]
    CRON[Periodic]
    API([API trigger])
  end

  subgraph Broker
    Q[(Redis / RabbitMQ / SQS)]
    PRIO[[Priority queues]]
    DELAY([Delayed / scheduled set])
    DLQ[[(DLQ)]]
  end

  subgraph Workers
    POOL([Worker pool / threadpool])
    AUTO[[Auto-scale by queue depth]]
    HEART[Heartbeats]
    GRACEFUL[Graceful shutdown]
  end

  subgraph Job[Job lifecycle]
    SUB[[Enqueue]]
    DEQ[[Dequeue + ack]]
    EXEC[Execute idempotently]
    RETRY[Retry policy]
    SUCC[Success]
    FAIL[Failure]
  end

  subgraph Ops
    UI([Web UI / Sidekiq Dashboard])
    METR[Metrics]
    ALERT[[Queue depth alert]]
  end

  Producers --> Broker --> Workers
  Workers --> Job
  Job --> RES[(Result store)]
  Job --> DLQ
  Ops --- Broker

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class APP,CRON,HEART,GRACEFUL,EXEC,RETRY,SUCC,FAIL service;
    class DLQ,RES datastore;
    class Q cache;
    class PRIO,AUTO,SUB,DEQ,ALERT queue;
    class API,DELAY,POOL compute;
    class UI,METR obs;

Glossary & fundamentals#

Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.

Tag	Concept	What it is	Page
`HLD`	Pub/Sub & message brokers	topics, consumer groups, delivery semantics	pub-sub-pattern
`HLD`	Idempotency & retries	safe re-execution, backoff + jitter	idempotency-retries
`HLD`	Observability	metrics, logs, traces, SLOs	observability