Commit graph

160 commits

Author SHA1 Message Date
Adriano Caloiaro
c16dd3d580 feat: improve PG database initialization errors 2024-02-22 09:38:46 -07:00
Adriano Caloiaro
94d7d68539
Remove TODO for seconds cron syntax in asynq 2024-01-28 07:26:50 -07:00
Phil Constantinou
95ff9152a4 Fix support for sslmode too 2024-01-28 07:13:45 -07:00
Phil Constantinou
b2aa3892c3 URL encode password 2024-01-28 07:13:45 -07:00
Adriano Caloiaro
8e9836e7fe
Switch to MIT license 2024-01-27 17:21:36 -07:00
Adriano Caloiaro
cce17b8e08 README updates 2024-01-26 09:21:05 -07:00
Adriano Caloiaro
99f07276b6 Bump go version for ci 2024-01-26 09:21:05 -07:00
Adriano Caloiaro
7798b35827 Fix lints 2024-01-26 09:21:05 -07:00
Adriano Caloiaro
f21eb9c282 Improve dev environment 2024-01-26 09:21:05 -07:00
Adriano Caloiaro
51452ed7f9
Update env.sample with better examples 2024-01-19 13:30:04 -07:00
Adriano Caloiaro
1472507f7d
Update env.sample with better examples 2024-01-19 13:27:18 -07:00
Adriano Caloiaro
1cae205942
Update goreleaser to ignore README.md and env.sample 2024-01-19 11:48:18 -07:00
dependabot[bot]
a87cc3dda5 chore(deps): bump golang.org/x/crypto from 0.7.0 to 0.17.0
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.7.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.7.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-19 10:04:58 -07:00
Adriano Caloiaro
3b21d026e2
Make matrix chat badge clickable 2024-01-19 09:58:15 -07:00
Adriano Caloiaro
02179794cd
Really fix matrix badge? 2024-01-19 09:49:01 -07:00
Adriano Caloiaro
b2fcace173
Fix matrix chat badge 2024-01-19 09:35:08 -07:00
Adriano Caloiaro
20edd84290
Update neoq Matrix chat badge 2024-01-19 09:28:58 -07:00
Adriano Caloiaro
86f7869440 feat: Multiplex the listener connection 2023-12-12 15:44:28 -07:00
Justin Clift
6eef1d5d5b Trivial typo fix 2023-12-03 08:11:59 -07:00
Adriano Caloiaro
1598184f74 fix: Add deadline to pool connection acquisition 2023-11-11 17:18:35 -07:00
Elliot Courant
e6ece3c568
chore(log): Tweaking logging again, making everything consistent (#101) 2023-11-06 15:07:46 -06:00
Adriano Caloiaro
e95311e76d fix: #98 retries can be picked up by wrong handler
Fixes a bug that can allow retries to end up on
the wrong queue in settings where there are
multiple handlers.
2023-10-21 07:31:54 -06:00
Elliot Courant
8b21247f9b fix(pg): Fixing notify handling name cases oddly.
This makes it so that channel names for NOTIFY/LISTEN/UNLISTEN are
consistent. That is that they are double quoted in all instances except
for when they are referenced by `pg_notify`. This way the channel names
are no longer case-insensitive.

This happens because of how PostgreSQL parses the SQL statements for
NOTIFY/LISTEN/UNLISTEN and treats the arguments as an identifier. Where
as it treats the arguments to `pg_notify` as only a string. This causes
arguments that are not double quoted to be adjusted to lower case when
passed to the SQL statement.

By making all usages consistent, we no longer have to worry about
channel names being odd.
2023-10-15 14:51:13 -06:00
Elliot Courant
3a10351049 bug(pg): Adding test to demonstrate bug with uppercase queue names
When the queue name is uppercase, the listener will never receive the
notification when the job is enqueued. This is done specifically with
the enqueuer and consumer being separate neoq instances (server A kicks
off a job, server B is listening to perform the work).
2023-10-15 14:51:13 -06:00
Adriano Caloiaro
84caae3438 fix: #98 segvault when moving jobs to dead queue 2023-10-13 10:48:29 -06:00
Elliot Courant
fb1fec159f chore: Minor tweak 2023-10-08 08:55:44 -06:00
Elliot Courant
659a654e1c chore(log): Make logging consistent and make sure queue is logged.
I want to be able to know what job a log message is in regards to, so
this adds the queue to all of the log messages where its available. It
also makes some of the `error` logs more consistent with each other.
2023-10-08 08:55:44 -06:00
Elliot Courant
26c69211a9 fix(pg): Fixed data race conditions in unit tests
Since these values are being written in a separate go routine, it is
possible for a datarace to happen since we are reading them
unatomically. This just makes it so that both the reads ands writes are
done atomically.

Resolves #76
2023-10-06 12:50:20 -06:00
Elliot Courant
42da4d8110
fix(pg): Fixed error logs for multi node cron jobs. (#94)
Resolves #76
2023-10-06 12:07:29 -05:00
Adriano Caloiaro
0db1180714 fix: neoq_jobs.id is an integer and should be bigint
Fixes #89
2023-10-05 10:40:08 -06:00
Adriano Caloiaro
845f96afda fix: memory backend does not respect MaxRetries
Fixes #87
2023-10-05 10:40:08 -06:00
Adriano Caloiaro
12a6667bd5 fix: Redis backend does not respect MaxRetries 2023-10-05 10:40:08 -06:00
Adriano Caloiaro
c04ef692a7 Update github action config 2023-10-04 10:05:04 -06:00
Adriano Caloiaro
afd1f4c1aa fix: MaxRetries not persisting in PG backend 2023-10-04 10:05:04 -06:00
Adriano Caloiaro
de2e101736 feat: Default to disable synchronous postgres commit 2023-10-03 08:41:59 -06:00
Adriano Caloiaro
35085be05b feat: Announce jobs using PG trigger
Previously, new, non-future jobs were announced by executing `NOTIFY` in
Go code. Triggers are much better suited for this, and reduces neoq
complexity by allowing PG to perform notification work for most jobs.

"Future" jobs continue to use the `announceJob` method on a timer.
2023-10-03 08:41:59 -06:00
Elliot Courant
fab0736b2b fix(ci): Fixed github actions tests on PRs 2023-09-27 18:12:34 -06:00
Elliot Courant
52e118ebcc chore(test): Adding a test for multiple consumers with the pg backend.
This test is to provide a minimal proof that jobs can be consumed by
multiple workers and in a way can only be consumed once. If the
execCount does not match the expected count then this test will fail
because either too many jobs were executed (like one executing twice) or
a job was dropped when it should not have been.
2023-09-27 18:12:34 -06:00
Adriano Caloiaro
a17ea73f6e fix: Bare 'make' fails due to go vet
`go vet` was missing the `testing` tag necessary to build some of the tests.
2023-09-26 08:37:45 -06:00
43c2bf0761
Update README.md 2023-09-24 13:56:53 -06:00
Adriano Caloiaro
86955cb40c examples: Cleanup 2023-09-24 13:29:33 -06:00
Adriano Caloiaro
6a368e05fc Make multiple queue test less finicky 2023-09-22 07:40:48 -06:00
Adriano Caloiaro
040c0de7db fix: Revert "feat: Hold connections and transactions for less time"
This reverts commit bc8df98894.

This commit was totally wrong about the motivations for holding transactions throught the duration of jobs.

Ths is an embarrassing mistake because it is the crux of the entire Postgres backend. Transactions must be held while jobs are being processed because `FOR UPDATE` requires the transaction that issued the `SELECT * FOR UPDATE ...` query to remain open to hold the lock (https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-ROWS)

The reverted commit allowed jobs to be picked up by multiple workers.
2023-09-22 07:40:48 -06:00
dbda88aee6
Update README.md 2023-09-18 17:35:02 -06:00
30c9da5374
Update README.md 2023-09-18 17:34:24 -06:00
Adriano Caloiaro
84c2aaac0c maint: Update README to point at Getting Started wiki 2023-09-17 00:15:19 +02:00
Adriano Caloiaro
5f01b4e8f1 fix: Fix naked return in memory backend StartCron 2023-09-17 00:15:19 +02:00
Adriano Caloiaro
f2ccb00961 maint: Remove dead code from pg backend 2023-09-17 00:15:19 +02:00
Adriano Caloiaro
bf2e07cf57 fix: Accumulation of cancel() funcs enqueueing on pg backend
`cancelFuncs` in the pg backend was intended for gracefully shutting down the backend with Shutdown(). Adding a
new cancel function every time a job is added to a queue isn't of any use, because there is nothing to "cancel" by
cancelling the context unless a job is in progress.

This was effectively a memory leak.
2023-09-17 00:15:19 +02:00
Adriano Caloiaro
d66905fadd feat: ensure memory backend exits gracefully during Shutdown() 2023-09-17 00:15:19 +02:00