me/neoq
1
0
Fork 0
mirror of https://github.com/acaloiaro/neoq synced 2026-07-21 18:29:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
178 commits 67 branches 81 tags 844 KiB
Commit graph

178 commits

This branch
This branch
All branches
Author SHA1 Message Date
Adriano Caloiaro
17c86fa00e fix: execessive connection usage when monitoring pending jobs 
The previous release attempted to fix an issue where, when LISTEN connections
drop an reconnect, some job announcements might be missed.

This fix reduces the number of connections required to do so.
 2025-01-25 16:48:44 -08:00
Adriano Caloiaro
3b3a7fe388 fix(postgres): jobs may never get processed if no workers hear the initial announcement 
When jobs are inserted into the 'neoq_jobs', the `announce_job` trigger announces the job to
listeners of the job's queue. However, if no workers are available to receive the announcement,
or the announcement is somehow lost, it meant those jobs were marooned on the queue.

This fix adds a periodic check for post-due pending jobs to ensure that a neoq restart is
not necessary to catch marooned, overdue jobs.
 2025-01-25 13:01:03 -08:00
dependabot[bot]
245711b3d4 chore(deps): bump golang.org/x/crypto from 0.17.0 to 0.31.0 
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.17.0 to 0.31.0.
- [Commits](https://github.com/golang/crypto/compare/v0.17.0...v0.31.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-12-22 08:42:09 -07:00
Adriano Caloiaro
a002d3f95f
chore: fix svu install in github action 2024-12-22 08:28:09 -07:00
Adriano Caloiaro
0ef3446f83 chore: use svu for versioning 2024-12-22 08:13:18 -07:00
Adriano Caloiaro
ef93e6856b fix: don't force serial job execution on single-CPU systems 2024-12-22 08:13:18 -07:00
Adriano Caloiaro
dbe8589bbd
feat: add stale bot 2024-08-15 10:05:43 -07:00
dependabot[bot]
54f33643f5 chore(deps): bump google.golang.org/protobuf from 1.28.1 to 1.33.0 
Bumps google.golang.org/protobuf from 1.28.1 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-07-25 08:44:28 -07:00
Adriano Caloiaro
9e3e803ec2 feat: log job_id regardless of error 2024-06-13 08:54:10 -07:00
Adriano Caloiaro
a6f7fbcd36 feat: Add public jobs.WithJobContext() 2024-06-07 07:22:07 -07:00
Adriano Caloiaro
37b6732ad0 feat: Automatically reconnect when the PG listener connection fails 2024-04-23 08:40:45 -06:00
Adriano Caloiaro
0105403be9 feat: add support for recovery callbacks 2024-04-16 13:52:54 -06:00
dependabot[bot]
f6ef118806 chore(deps): bump github.com/jackc/pgx/v5 from 5.3.1 to 5.5.4 
Bumps [github.com/jackc/pgx/v5](https://github.com/jackc/pgx) from 5.3.1 to 5.5.4.
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v5.3.1...v5.5.4)

---
updated-dependencies:
- dependency-name: github.com/jackc/pgx/v5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-04-04 08:01:05 -06:00
Adriano Caloiaro
b401809190 feat: fix infinite scheduling loop when job gets scheduled after deadline 2024-03-27 12:11:14 -06:00
Adriano Caloiaro
441a39271b fix: An incorrect error was thrown when jobs exceeding their deadline were scheduled 
Co-authored-by: Philip Constantinou <1383834+pconstantinou@users.noreply.github.com>
 2024-03-25 11:58:08 -06:00
Fernando Barillas
672e101e6b feat: Update migrations postgres URI parsing and add tests 2024-03-04 19:57:51 -07:00
Adriano Caloiaro
0c32209612 feat: Make fingerprint unique to each queue 2024-03-02 10:56:29 -07:00
Adriano Caloiaro
9a555f1407 chore: Add test coverage over postgres future jobs 2024-03-02 10:16:14 -07:00
Adriano Caloiaro
c16dd3d580 feat: improve PG database initialization errors 2024-02-22 09:38:46 -07:00
Adriano Caloiaro
94d7d68539
Remove TODO for seconds cron syntax in asynq 2024-01-28 07:26:50 -07:00
Phil Constantinou
95ff9152a4 Fix support for sslmode too 2024-01-28 07:13:45 -07:00
Phil Constantinou
b2aa3892c3 URL encode password 2024-01-28 07:13:45 -07:00
Adriano Caloiaro
8e9836e7fe
Switch to MIT license 2024-01-27 17:21:36 -07:00
Adriano Caloiaro
cce17b8e08 README updates 2024-01-26 09:21:05 -07:00
Adriano Caloiaro
99f07276b6 Bump go version for ci 2024-01-26 09:21:05 -07:00
Adriano Caloiaro
7798b35827 Fix lints 2024-01-26 09:21:05 -07:00
Adriano Caloiaro
f21eb9c282 Improve dev environment 2024-01-26 09:21:05 -07:00
Adriano Caloiaro
51452ed7f9
Update env.sample with better examples 2024-01-19 13:30:04 -07:00
Adriano Caloiaro
1472507f7d
Update env.sample with better examples 2024-01-19 13:27:18 -07:00
Adriano Caloiaro
1cae205942
Update goreleaser to ignore README.md and env.sample 2024-01-19 11:48:18 -07:00
dependabot[bot]
a87cc3dda5 chore(deps): bump golang.org/x/crypto from 0.7.0 to 0.17.0 
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.7.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.7.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-19 10:04:58 -07:00
Adriano Caloiaro
3b21d026e2
Make matrix chat badge clickable 2024-01-19 09:58:15 -07:00
Adriano Caloiaro
02179794cd
Really fix matrix badge? 2024-01-19 09:49:01 -07:00
Adriano Caloiaro
b2fcace173
Fix matrix chat badge 2024-01-19 09:35:08 -07:00
Adriano Caloiaro
20edd84290
Update neoq Matrix chat badge 2024-01-19 09:28:58 -07:00
Adriano Caloiaro
86f7869440 feat: Multiplex the listener connection 2023-12-12 15:44:28 -07:00
Justin Clift
6eef1d5d5b Trivial typo fix 2023-12-03 08:11:59 -07:00
Adriano Caloiaro
1598184f74 fix: Add deadline to pool connection acquisition 2023-11-11 17:18:35 -07:00
Elliot Courant
e6ece3c568
chore(log): Tweaking logging again, making everything consistent (#101) 2023-11-06 15:07:46 -06:00
Adriano Caloiaro
e95311e76d fix: #98 retries can be picked up by wrong handler 
Fixes a bug that can allow retries to end up on
the wrong queue in settings where there are
multiple handlers.
 2023-10-21 07:31:54 -06:00
Elliot Courant
8b21247f9b fix(pg): Fixing notify handling name cases oddly. 
This makes it so that channel names for NOTIFY/LISTEN/UNLISTEN are
consistent. That is that they are double quoted in all instances except
for when they are referenced by `pg_notify`. This way the channel names
are no longer case-insensitive.

This happens because of how PostgreSQL parses the SQL statements for
NOTIFY/LISTEN/UNLISTEN and treats the arguments as an identifier. Where
as it treats the arguments to `pg_notify` as only a string. This causes
arguments that are not double quoted to be adjusted to lower case when
passed to the SQL statement.

By making all usages consistent, we no longer have to worry about
channel names being odd.
 2023-10-15 14:51:13 -06:00
Elliot Courant
3a10351049 bug(pg): Adding test to demonstrate bug with uppercase queue names 
When the queue name is uppercase, the listener will never receive the
notification when the job is enqueued. This is done specifically with
the enqueuer and consumer being separate neoq instances (server A kicks
off a job, server B is listening to perform the work).
 2023-10-15 14:51:13 -06:00
Adriano Caloiaro
84caae3438 fix: #98 segvault when moving jobs to dead queue 2023-10-13 10:48:29 -06:00
Elliot Courant
fb1fec159f chore: Minor tweak 2023-10-08 08:55:44 -06:00
Elliot Courant
659a654e1c chore(log): Make logging consistent and make sure queue is logged. 
I want to be able to know what job a log message is in regards to, so
this adds the queue to all of the log messages where its available. It
also makes some of the `error` logs more consistent with each other.
 2023-10-08 08:55:44 -06:00
Elliot Courant
26c69211a9 fix(pg): Fixed data race conditions in unit tests 
Since these values are being written in a separate go routine, it is
possible for a datarace to happen since we are reading them
unatomically. This just makes it so that both the reads ands writes are
done atomically.

Resolves #76
 2023-10-06 12:50:20 -06:00
Elliot Courant
42da4d8110
fix(pg): Fixed error logs for multi node cron jobs. (#94) 
Resolves #76
 2023-10-06 12:07:29 -05:00
Adriano Caloiaro
0db1180714 fix: neoq_jobs.id is an integer and should be bigint 
Fixes #89
 2023-10-05 10:40:08 -06:00
Adriano Caloiaro
845f96afda fix: memory backend does not respect MaxRetries 
Fixes #87
 2023-10-05 10:40:08 -06:00
Adriano Caloiaro
12a6667bd5 fix: Redis backend does not respect MaxRetries 2023-10-05 10:40:08 -06:00