neoq/backends/postgres
Adriano Caloiaro bc8df98894 feat: Hold connections and transactions for less time
Previously pg backend would hold connections/transactions during the
execution of job handlers. So, e.g. if a job ran for 30 seconds, a
transaction and its connection would be held for the entirety of those
30 seconds.

The reason it was originally implemented this way is because originally
the vision was to pass every job a `tx` that could be used throughout
the job. If the job failed, its `tx` would be rolled back in neoq
so that user would not have to handle rollbacks and connection handling
themselves, were they to perform database operations in their jobs.
But ultimately, that is a lot of hand-holding at the cost of a lot of
resources, for a use case that is both unlikely and does not work as
soon as the user's application and neoq tables are in different
databases.

This commit reverses that poor decision and opts to improve performance
over giving users an ergonomic transaction handling API.
2023-09-02 14:59:32 +02:00
..
migrations feat: Improve postgres duplicate job detection performance 2023-09-02 14:59:32 +02:00
postgres_backend.go feat: Hold connections and transactions for less time 2023-09-02 14:59:32 +02:00
postgres_backend_test.go feat: Improve postgres duplicate job detection performance 2023-09-02 14:59:32 +02:00
README.md Support postgres backend schema migrations 2023-08-28 08:59:40 +02:00

Migrations

Migrations are implemented with https://github.com/golang-migrate/migrate

Install the CLI

To add new migrations, install the CLI. The version of the CLI is not particularly important.

go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@v4.16.2

Adding migrations

Adding new migrations is done with the migrate create command

migrate create -dir backends/postgres/migrations -ext sql <descriptive_migration_name>

Running migrations

Migrations are run every time neoq initializes the postgres backend. There is no need to run migrations explicitly.