pg_timetable: Advanced scheduling for PostgreSQL

pg_timetable is an advanced standalone job scheduler for PostgreSQL, offering many advantages over traditional schedulers such as cron and others. It is completely database driven and provides a couple of advanced concepts. It allows you to schedule PostgreSQL commands, system programs and built-in operations:

-- Run public.my_func() at 00:05 every day in August:
SELECT timetable.add_job('execute-func', '5 0 * 8 *', 'SELECT public.my_func()');

-- Run VACUUM at minute 23 past every 2nd hour from 0 through 20 every day:
SELECT timetable.add_job('run-vacuum', '23 0-20/2 * * *', 'VACUUM');

-- Refresh materialized view every 2 hours:
SELECT timetable.add_job('refresh-matview', '@every 2 hours', 
  'REFRESH MATERIALIZED VIEW public.mat_view');

-- Clear log table after pg_timetable restart:
SELECT timetable.add_job('clear-log', '@reboot', 'TRUNCATE public.log');

-- Reindex at midnight on Sundays with reindexdb utility:

--  using default database under default user (no command line arguments)
SELECT timetable.add_job('reindex-job', '0 0 * * 7', 'reindexdb', job_kind := 'PROGRAM');

--  specifying target database and tables, and be verbose
SELECT timetable.add_job('reindex-job', '0 0 * * 7', 'reindexdb',
          '["--table=foo", "--dbname=postgres", "--verbose"]'::jsonb, 'PROGRAM');

--  passing password using environment variable through bash shell
SELECT timetable.add_job('reindex-job', '0 0 * * 7', 'bash',
    '["-c", "PGPASSWORD=5m3R7K4754p4m reindexdb -U postgres -h 192.168.0.221 -v"]'::jsonb,
    'PROGRAM');    

Documentation

https://cybertec-postgresql.github.io/pg_timetable/

Main features

• Tasks can be arranged in chains
• Each task executes SQL, built-in or executable command
• Parameters can be passed to tasks
• Missed chains (possibly due to downtime) can be retried automatically
• Support for configurable repetitions
• Builtin tasks such as sending emails, downloading, importing files, etc.
• Fully database driven configuration
• Full support for database driven logging
• Enhanced cron-style scheduling
• Optional concurrency protection
• NEW: YAML-based chain definitions for easy configuration

YAML Configuration

You can now define chains using YAML files instead of SQL inserts, making configuration more readable and maintainable:

chains:
  - name: "Daily ETL Pipeline"
    schedule: "0 2 * * *"  # 2 AM daily
    live: true
    max_instances: 1
    timeout: 3600000  # 1 hour

    tasks:
      - name: "Extract data"
        command: "SELECT extract_sales_data($1)"
        parameters: ["yesterday"]

      - name: "Transform data"
        command: "CALL transform_sales_data()"
        autonomous: true

      - name: "Load to warehouse"
        command: "CALL load_to_warehouse()"

Load YAML chains with:

pg_timetable --file chains.yaml postgresql://user:pass@host/db

See samples/yaml/ for more examples and YAML Schema for complete format specification.

Installation

Complete installation guide can be found in the documentation.

Possible choices are:

• official release packages;
• Docker images;
• build from sources.

Quick Start

Complete usage guide can be found in the documentation.

1. Download pg_timetable executable

2. Make sure your PostgreSQL server is up and running and has a role with CREATE privilege for a target database, e.g.

   sql my_database=> CREATE ROLE scheduler PASSWORD 'somestrong'; my_database=> GRANT CREATE ON DATABASE my_database TO scheduler;

3. Create a new job, e.g. run VACUUM each night at 00:30

   ```sql my_database=> SELECT timetable.add_job('frequent-vacuum', '30 0 * * *', 'VACUUM'); add_job

   ---

   3
   

   (1 row) ```

4. Run the pg_timetable

   terminal pg_timetable postgresql://scheduler:somestrong@localhost/my_database --clientname=vacuumer

5. PROFIT!

Supported Environments

* - there are no official release binaries made for these OSes. One would need to build them from sources.

** - previous PostgreSQL versions may and should work smoothly. Only officially supported PostgreSQL versions are listed in this table.

Contributing

If you want to contribute to pg_timetable and help make it better, feel free to open an issue or even consider submitting a pull request.

Support

For professional support, please contact Cybertec.

Authors

• Implementation: Pavlo Golub
• Initial idea and draft design: Hans-Jürgen Schönig

Star History