Skip to content

Job / Task Scheduler — Notes#

Functional#

  • Schedule jobs by cron or event.
  • Dependencies / DAGs.
  • Retries, SLAs, alerts.
  • Backfill / rerun.
  • Operator UI + API.

Non-functional#

  • Sub-minute scheduling precision.
  • HA scheduler with leader election.
  • Tens of thousands of concurrent task runs.

Trade-offs#

  • Airflow = batch + DAG-first; Temporal/Cadence = code-first long workflows; Argo = K8s-native.
  • Pull workers vs push from scheduler: pull preferred for scale.
  • At-least-once execution = task author MUST design idempotency.

Refs#

  • Apache Airflow docs; Temporal, Cadence; Argo Workflows.
  • Google "Borg" paper (cluster scheduling cousin).
  • Netflix Conductor docs.