Commit Graph
1503 Commits
Author SHA1 Message Date
Taimoor ZaeemandSteve Chavez 40a1fea916 refactor: move role insertion in claims to PreQuery.hs
Also add a comment on `AuthResult` type to explain that
we can also access the claims in the db using GUCs.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-03-16 10:24:24 -05:00
Andrei DziahelandSteve Chavez 1f54e2accc add: add "Vary" header to responses
Co-authored-by: Steve Chavez <stevechavezast@gmail.com>
2026-03-16 09:55:20 -05:00
Michał KłeczekandSteve Chavez a4aa58b4de refactor: move socket creation and management to App module
Right now listening sockets initialization, management and usage is split between App, AppState and Admin modules: they are created in AppState.init and remembered in AppState but used only in App and Admin.

It has several negative consequences:
- sockets are initialized even if not needed (eg. command line invocations like dump-config or dump-schema)
- it is impossible to start listening on a socket after initial schema cache load because it requires AppState

This change decouples listen socket management from AppState. Sockets are created only when needed (ie. not in command line tools invocation) and passed to admin application and to Warp by the App module.
2026-03-12 10:03:47 -05:00
Michał KłeczekandSteve Chavez e741c1bca7 refactor: provide AppState infrastructure to wait for schema cache load
This commit replaces ioRef based implementation of schema cache status tracking to MVar based, so that it is possible to wait for schema cache loading.

Waiting for schema cache loading is necessary to implement zero-downtime upgrades  with SO_REUSEPORT, where listening on a socket must wait for schema cache loading.
2026-03-12 09:04:56 -05:00
Taimoor ZaeemandSteve Chavez 93db1a5a47 refactor: remove redundant parameters from actionResponse
Towards #4608.

We already have access to these parameters via `ApiRequest` type,
hence no need to pass them separately.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-03-10 09:55:42 -05:00
Taimoor ZaeemandSteve Chavez 5abacba0d8 fix(error): leaking table and function names when calculating hint
Increase similarity score to 0.75 from 0.33 for table and functions
error hint.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-03-06 16:03:21 -05:00
Michał KłeczekandSteve Chavez 8e1f0a0471 refactor: standardize query logging to follow code logging other observations
This commit addresses the following TODO in observationMessage function:
"TODO pending refactor: The logic for printing the query cannot be done here. Join the observationMessages function into observationLogger to avoid this mempty."

Changing observationMessage to observationMessages (that returns [Text] instead of Text) made it possible to address issue documented as TODO and harmonize query logging with other observations.
2026-03-02 15:46:17 -05:00
Michał KłeczekandSteve Chavez 2408cd332d refactor: get rid of redundant SchemaCacheSummaryObs
There is unnecessary coupling between observation messages and emited log entries. This causes schema loading logic to emit redundant events: SchemaCacheSummaryObs and SchemaCacheLoadedObs.

Logically - we want to emit a single event containing both summary and timing information. How it is logged is a different matter and should be decoupled.

This commit
* changes observationMessage function returning Text to observationMessages returning [Text] so that it is possible to return multiple (or zero) messages to log based on an observation event
* Removes SchemaCacheSummaryObs constructor from Observation type and adds summary text to SchemaCacheLoadedObs
2026-03-02 14:45:31 -05:00
Michał KłeczekandSteve Chavez 58a973e664 refactor: make Observation Logger and Listener modules more cohesive
Observation module exports observationMessage and isDbListenerBug functions. The first one is used only in Logger module and the second one - only in Listener.

This change moves observationMessage function to Logger module and isDbListenerBug function to Listener module, making all three modules more cohesive and lessening dependencies.
It also gives the compiler more opportunities for intra-module inlining and optimization.
2026-03-02 12:26:22 -05:00
Michał KłeczekandSteve Chavez 78f231c87a refactor: failure in schema cache load should not set listener delay
Listener has its own backoff logic independent of schema cache loading. It also uses listener delay value to identify its retries and to trigger schema cache reloads upon failures.
If schema cache reloading sets listener delay it might lead to some unexpected reloading loops.
That's especially important in set ups where listener connection points to master and the pool to replica(s).
2026-02-25 17:34:38 -05:00
Taimoor ZaeemandSteve Chavez 83dc082acf add: config client-error-verbosity to set error verbosity
Set error verbosity using this config. The verbosity can
be set to `verbose` or `minimal` for client error responses.

This only affects client side HTTP responses, server side logs
are not affected by this config.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-25 15:24:27 -05:00
Taimoor ZaeemandSteve Chavez 2edc44c352 refactor: remove ToJSON instance definition on error types
Towards #4088.

- Some of these instances are not used. Reduces number of lines
  significantly.

- Removing this gives us more flexibility for cases like conditional
  encoding based on some outside parameter, without needing to
  add the conditional at type level.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-23 12:10:51 -05:00
Taimoor ZaeemandSteve Chavez 02feaf087e refactor: remove commented out code from Auth/Jwt.hs
Some commented out code got left in the codebase in
5196823365.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-20 11:07:30 -05:00
Taimoor ZaeemandSteve Chavez c52fd3fd2b refactor: rename ApiRequestError type constructor to ApiRequestErr
Rename to keep the naming convention consistent with other
type constructor names.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-19 11:29:51 -05:00
Taimoor ZaeemandSteve Chavez 248b777117 refactor: move error data types to Error/Types.hs
The `Error.hs` module is getting bloated. This moves types
to another module for better separation of concerns.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-19 11:29:51 -05:00
Taimoor ZaeemandSteve Chavez 055efe7abd refactor: move Proxy-Status header to error type class
Headers that apply to all errors should all be in one place.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-16 13:09:01 -05:00
Michał KłeczekandSteve Chavez ca96328142 refactor: Remove unnecessary lazy initialization of stateLogDebouncePoolTimeout
stateLogDebouncePoolTimeout is an MVar initialized on the first logging of PoolAcqTimeoutObs. The code in logWithDebounce has race condition that could lead to creation of multiple debouncers.

This change simplifies logic by getting rid of lazy initialization of debouncer.
2026-02-15 13:14:10 -05:00
Michał KłeczekandSteve Chavez d6816d8d2a refactor: Remove unnecessary SQL.UsageError parameter from PoolAcqTimeoutObs
PoolAcqTimeoutObs is always created with SQL.AcquisitionTimeoutUsageError so the parameter does not provide any useful information.
On the other hand, it complicates debouncing logic in Logger as logWithDebounce has to lazily create the debouncer (which is error prone and hence the logic has a race condition).

This change removes the parameter from PoolAcqTimeoutObs. It is a prerequisite to simplifying logWithDebounce.
2026-02-15 13:14:10 -05:00
steve-chavez e95e815483 fix: don't hide async exceptions in logs
Fixes #4646. Using the repro on #4646, this now produces the log:

```
11/Feb/2026:09:40:08 -0500: Warp server error: stack overflow
```

When:
```
$ curl localhost:3000/
curl: (52) Empty reply from server
```
2026-02-12 19:43:50 -05:00
Laurence Isla d10c779fc6 perf: optimize count=exact when there's no limits, offsets or db-max-rows 2026-01-28 18:43:44 -05:00
Laurence Isla d031bb2df5 refactor: use a single function to get the page_total count 2026-01-28 18:43:44 -05:00
Michal KleczekandGitHub 73a465501e fix: listener running with exception masked after first failure 2026-01-28 17:13:59 -05:00
Laurence Isla 1682677297 fix: filtering the returned representation whenn using or/and filters on mutations 2026-01-27 21:24:19 +00:00
Michał KłeczekandSteve Chavez 21b8c3458a add: Log pg version details of listener connection
Follow-up to #4617 adding more information to log entry produced upon successful listener connection establishement.
2026-01-27 15:44:05 -05:00
Michał KłeczekandSteve Chavez 5356f4e973 add: Log actual host and port of listener connection
Diagnosing problems with listener channel notifications not being handled properly by PostgREST connected to read replicas is difficult. Issues might be related to lost connections and listener not being connected to the right host after failover or database server restarts.
This patch adds logging of actual host:port used by libpq connection opened by the listener. It should make it easier to find out if PostgREST is connected to the right host.
2026-01-27 11:00:30 -05:00
Michal KleczekandGitHub 00c7cb1a22 fix: ensure Listener connections are released
retryingListen function potentially leaks database connections. This patch ensures the connections are released in case of listen/notify errors.
2026-01-26 16:26:24 -05:00
Taimoor ZaeemandGitHub fe0386e9c4 add: string slicing operator for jwt-role-claim-key (#4603) 2026-01-24 05:40:21 -05:00
Laurence IslaandGitHub c15e908266 add(logs): Include a HINT when the LISTEN channel breaks due to a Postgres bug
The HINT shows a SQL command that solves the issue.
2026-01-18 18:47:53 +07:00
Taimoor ZaeemandSteve Chavez c18727ff43 refactor: move walkJSPath to Config/JSPath.hs module
The logic to evaluate `JSPath` belongs to `JSPath.hs` module.
Hence, moving this logic from `Auth/Jwt.hs` to here.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-01-14 22:34:08 +07:00
Kian-Meng AngandSteve Chavez f5c7cde9b7 docs: fix typos
Found via `codespell -S test -L notin,nam,plantext,cryto,corelation,inout,ot`
2026-01-06 05:24:51 +08:00
Michał KłeczekandSteve Chavez e592d568c6 fix: Performance and high memory usage of relation hint calculation
* Calculation of hint message when requested relation is not present in schema cache requires creation of a FuzzySet (to use fuzzy search to find candidate tables). For schemas with many tables it is costly.
This patch introduces dbTablesFuzzyIndex in SchemaCache to memoize the FuzzySet creation.

* Additionally, because of FuzzySet large memory requirements, this patch introduces a limit of 500 relations per schema, above which FuzzySet is not created and hint calculation disabled.
2026-01-03 07:56:12 +08:00
Taimoor ZaeemandSteve Chavez 9ec5b030ce refactor: move content negotiation logic to Negotiate.hs
Moves the `negotiateContent` function to `Plan/Negotiate.hs`
module. This also adds comments to describe the current approach
we have for negotiation.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-01-02 06:32:17 +08:00
388930336d refactor: fix definition of Ord instance for Routine type (#4577)
The `Ord` instance definition for type `Routine` had a logical
error when comparing two routines. The error did not affect any
end users. However, for correctness and completeness reasons, this
commit fixes the error.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Co-authored-by: Joel Jacobson <joel@compiler.org>
2025-12-22 08:34:29 +00:00
steve-chavez 013f078bc4 fix: inaccurate Server-Timing durations
The transaction duration was notably off, doing:

```
curl localhost:3000/rpc/sleep?seconds=5 -i
```

Shows `46.1` for the `transaction;dur`, with this fix we obtain
`5007.3`.

Fixes https://github.com/PostgREST/postgrest/issues/4522

This also fixes inaccurate "schema cache queried" logs,
see https://github.com/PostgREST/postgrest/issues/4551.
2025-12-18 17:35:18 -05:00
Joel JakobssonandGitHub fd6a3bdccf fix: hasSingleUnnamedParam incorrectly matching named parameters
The hasSingleUnnamedParam function was only checking the parameter type
but not whether the parameter actually had no name. This caused functions
with a single NAMED parameter (e.g., `foo(data json)`) to incorrectly
match the single-param fallback mode.

The result was a confusing PostgreSQL error 42883 "function does not exist"
instead of a clean PGRST202 error explaining that no matching function
was found.

Added ppName == mempty check so functions with named parameters don't
incorrectly match the single-param fallback.
2025-12-15 15:34:48 -05:00
Taimoor ZaeemandSteve Chavez 29c2aa015f refactor: create function to handle response preferences
Centralizes handling of preferences that are used to
create `Preference-Applied` header which is returned on
responses.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-12-11 17:55:45 -05:00
Taimoor ZaeemandSteve Chavez abad47ff1b add: log error when db-schemas config contain restricted schemas
We don't allow accessing schemas "pg_catalog" and "information_schema".
Instead of silently failing requests on these schemas, this now adds
logging an error on startup when these schemas are found in `db-schemas`
config.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-12-11 14:28:08 -05:00
Laurence Isla ab9b00ce88 refactor: use only Lazy.ByteString to calculate the response body length 2025-12-10 14:18:19 +00:00
Laurence Isla 9200a98087 fix: add missing Content-Length to empty HTTP 201 responses
For when a preference other than return=representation is requested.
2025-12-10 14:18:19 +00:00
Laurence Isla 3b27b2220d fix: regression that truncates error message when offset is out of bounds
- Happens when offset > the number of rows and when "Prefer: count=exact" header is sent
- Regression introduced in commit 57ef998
2025-12-10 14:18:19 +00:00
Taimoor ZaeemandSteve Chavez 0fa2f569a7 fix: misleading logs on unsupported postgresql versions
Postgrest fails on unsupported pg versions. However before killing
the thread, it continues to print a few more log messages which
were misleading. This commit fixes this by making sure that the
no log message should be printed after the unsupported pg version
observation and kill the thread immediately.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-12-09 13:46:34 -05:00
Taimoor ZaeemandSteve Chavez db2be093b5 refactor: move dumpQi to Config.hs and clarify quoteQi
The `dumpQi` function is only used in the `Config.hs` module, so
it is moved there. This also adds a doctest for `quoteQi` and
clarifies its usage.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-10-30 12:26:29 -05:00
Taimoor ZaeemandSteve Chavez a688878236 fix: db-pre-config function failing with pg reserved words
When db-pre-config is accidentally set to a pg reserved word
like "true", it fails with a confusing error. The function
names should be properly quoted to avoid such errors. This commit
resolves this by quoting the pre-config function name.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-10-29 16:54:24 -05:00
Taimoor ZaeemandSteve Chavez 66f84c5903 refactor: move escapeIdent function to Identifiers.hs
Moves the functions `escapeIdent` and `trimNullChars` to
SchemaCache/Identifiers.hs module.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-10-28 09:47:37 -05:00
Taimoor ZaeemandSteve Chavez 5d9b169380 refactor: sort exports of Identifiers.hs and SqlFragments.hs
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-10-28 09:47:37 -05:00
Michał KłeczekandLaurence Isla 66161104bf refactor: Remove redundant VANull constructor in Auth.JWT module 2025-10-22 00:40:30 +00:00
Michał KłeczekandLaurence Isla 5a4e2e4dec refactor: Encapsulate aud config
This change is an initial step to change JWT aud configuration to regular expression.
Exporting function
audMatchesCfg :: AppConfig -> Text -> Bool
from Config module allows changing the way how JWT aud is configured to be isolated and not affect code in Auth.JWT
2025-10-22 00:40:30 +00:00
Michal KleczekandGitHub be9a0ab467 perf: Index various lists in SchemaCache to change complexity from O(n*n) to O(n) (#4396) 2025-10-20 21:17:50 -05:00
Laurence Isla ea5db2a09a fix: reloading the Schema Cache unnecessarily on a PGRST002 error
When 503 errors happen if the Schema Cache is empty,
it should not retrigger the connection worker since
there's no Schema Cache loaded yet.
2025-10-20 18:42:56 +00:00
Laurence Isla c88ddfe18d fix: not logging the JSON message on a PGRST002 error
It happens right after the configuration is loaded and before
the schema cache is queried.
2025-10-20 18:42:56 +00:00