Building a Reliable PostgreSQL Queue: Concurrency, Crashes, Retries, and Scale

a month ago by codeinabox to c/postgresql

We get into building a background task processor using PostgreSQL. It seems easy at first, but there are lots of pitfalls as a system like this scales.
Gonzako 2 points a month ago

Saving it for a read later on!

path: 0 25802854, hotness: undefined, score: 2, children: 0
anzo 1 point a month ago

Would this be a replacement for Valkey in small scale?

path: 0 25806414, hotness: undefined, score: 1, children: 1
francisco_1844 2 points a month ago

Short answer, maybe. Until you specify your needs or use case it is very difficult to say.

In general, most if not all, benchmarks I have seen comparing Postgres to RAM key value stores has always had postgres coming significantly slower. If speed is the most important factor then likely no, you still need a key value store.

Also, you wrote "small scale", but even that can be ambiguous. Is it small scale because it is small, but you still need fast lookups? or is it "small scale" because it is a small setup and speed is not an issue?

It ultimately comes down to your needs and your appetite for having a second technology if you already have postgres. Sometimes, even if a second technology is better at something, the overhead of having the second technology may not be worth it.

path: 0 25806414 25821516, hotness: undefined, score: 2, children: 0
postgresql
postgresql

@programming.dev

login for more options
2251
203
21

The world's most advanced open source relational database

Project
Events
Podcasts
Related Fediverse communities

go to feed...