This adds a new log line that shows each schema cache query time individually, only on
`log-level=debug`. Like so:
```
$ PGRST_LOG_LEVEL=debug postgrest-with-pg-17 -f test/spec/fixtures/load.sql postgrest-run
....
10/Apr/2026:21:48:45 -0500: Schema cache queried in 192.2 milliseconds
10/Apr/2026:21:48:45 -0500: tables: 72.027 ms, keydeps: 20.118 ms, rels: 6.189 ms, funcs: 35.010 ms, comprels: 4.319 ms, dreps: 1.614 ms, mhandlers: 7.419 ms, tzones: 43.025 ms
```
This helps debug specific schema cache queries being slow like on
https://github.com/PostgREST/postgrest/issues/4613#issuecomment-4210191065 and
https://github.com/PostgREST/postgrest/issues/3046#issuecomment-3469059948.
It also closes https://github.com/PostgREST/postgrest/issues/3215,
which main motivation was to find out which query is slow.
Implementation details
---------------------
To time each query inside a transaction in pure SQL, we do:
```sql
-- start timer
select set_config('pgrst.tmp_x', clock_timestamp()::text, false);
-- run the query
select <query>
-- end timer
select set_config('pgrst.tmp_x', (clock_timestamp() - current_setting('pgrst.tmp_x', false)::timestamptz)::text, false);
-- .... repeated for every query
-- at the end we capture all the timings with
select extract('milliseconds' from current_setting('pgrst.tmp_x', false)::interval), extract(..;
```
Considerations
--------------
Only added this on `log-level=debug` because while the queries are fast
and the data is valuable, it triples the amount of queries we run during schema cache refresh,
which could be troublesome on slow networks. It's possible to reduce the
amount of queries by starting and stopping timers in one statement, but
this would still double the amount of queries and makes the code messy,
doesn't seem worth it.
Also it would pollute pg_stat_statements, it's only required to debug certain
extreme cases anyway.
retryingSchemaCacheLoad flushes the pool upon every retry before it starts reloading the schema. This is too early as schema reloading might take some time during which new connections might be acquired. The consequence is that:
* upon successful schema cache reload we might have some connections created with the old schema cache
* we close connections upon each retry and under load we will keep closing and re-opening connections until schema cache load succeeds
This change is to make sure we flush the pool only after successful schema cache querying but before loading (so that connections acquired during loading wait for it and do not interfere with timing the loading process).
DISCLAIMER:
This commit was authored entirely by a human without the assistance of LLMs.
Using debouncer to trigger schema cache loading makes it difficult to understand when exactly it is triggered.
Fixed integer type mapping in OpenAPI 2.0: replaced the invalid integer format with int32/int64 and added the toSwaggerFormat function to map PostgreSQL types to valid OpenAPI 2.0 formats:
smallint -> int32
integer -> int32
bigint -> int64
To avoid repeated querying of `pg_timezone_names` every time schema
cache is reset, `Prefer: timezone` can be disabled by setting
`db-timezone-enabled = false`.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Some helpers are provided for introspecting metrics already (used in JWT cache tests). This change provides facilities to additionally validate emited Observation events.
A new Spec module is also implemented, adding basic tests of schema cache reloading - their main goal is to excercise the new infrastructure.
Also add a comment on `AuthResult` type to explain that
we can also access the claims in the db using GUCs.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Right now listening sockets initialization, management and usage is split between App, AppState and Admin modules: they are created in AppState.init and remembered in AppState but used only in App and Admin.
It has several negative consequences:
- sockets are initialized even if not needed (eg. command line invocations like dump-config or dump-schema)
- it is impossible to start listening on a socket after initial schema cache load because it requires AppState
This change decouples listen socket management from AppState. Sockets are created only when needed (ie. not in command line tools invocation) and passed to admin application and to Warp by the App module.
This commit replaces ioRef based implementation of schema cache status tracking to MVar based, so that it is possible to wait for schema cache loading.
Waiting for schema cache loading is necessary to implement zero-downtime upgrades with SO_REUSEPORT, where listening on a socket must wait for schema cache loading.
Towards #4608.
We already have access to these parameters via `ApiRequest` type,
hence no need to pass them separately.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
This commit addresses the following TODO in observationMessage function:
"TODO pending refactor: The logic for printing the query cannot be done here. Join the observationMessages function into observationLogger to avoid this mempty."
Changing observationMessage to observationMessages (that returns [Text] instead of Text) made it possible to address issue documented as TODO and harmonize query logging with other observations.
There is unnecessary coupling between observation messages and emited log entries. This causes schema loading logic to emit redundant events: SchemaCacheSummaryObs and SchemaCacheLoadedObs.
Logically - we want to emit a single event containing both summary and timing information. How it is logged is a different matter and should be decoupled.
This commit
* changes observationMessage function returning Text to observationMessages returning [Text] so that it is possible to return multiple (or zero) messages to log based on an observation event
* Removes SchemaCacheSummaryObs constructor from Observation type and adds summary text to SchemaCacheLoadedObs
Observation module exports observationMessage and isDbListenerBug functions. The first one is used only in Logger module and the second one - only in Listener.
This change moves observationMessage function to Logger module and isDbListenerBug function to Listener module, making all three modules more cohesive and lessening dependencies.
It also gives the compiler more opportunities for intra-module inlining and optimization.
Listener has its own backoff logic independent of schema cache loading. It also uses listener delay value to identify its retries and to trigger schema cache reloads upon failures.
If schema cache reloading sets listener delay it might lead to some unexpected reloading loops.
That's especially important in set ups where listener connection points to master and the pool to replica(s).
Set error verbosity using this config. The verbosity can
be set to `verbose` or `minimal` for client error responses.
This only affects client side HTTP responses, server side logs
are not affected by this config.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Towards #4088.
- Some of these instances are not used. Reduces number of lines
significantly.
- Removing this gives us more flexibility for cases like conditional
encoding based on some outside parameter, without needing to
add the conditional at type level.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
The `Error.hs` module is getting bloated. This moves types
to another module for better separation of concerns.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
stateLogDebouncePoolTimeout is an MVar initialized on the first logging of PoolAcqTimeoutObs. The code in logWithDebounce has race condition that could lead to creation of multiple debouncers.
This change simplifies logic by getting rid of lazy initialization of debouncer.
PoolAcqTimeoutObs is always created with SQL.AcquisitionTimeoutUsageError so the parameter does not provide any useful information.
On the other hand, it complicates debouncing logic in Logger as logWithDebounce has to lazily create the debouncer (which is error prone and hence the logic has a race condition).
This change removes the parameter from PoolAcqTimeoutObs. It is a prerequisite to simplifying logWithDebounce.
Fixes#4646. Using the repro on #4646, this now produces the log:
```
11/Feb/2026:09:40:08 -0500: Warp server error: stack overflow
```
When:
```
$ curl localhost:3000/
curl: (52) Empty reply from server
```
Diagnosing problems with listener channel notifications not being handled properly by PostgREST connected to read replicas is difficult. Issues might be related to lost connections and listener not being connected to the right host after failover or database server restarts.
This patch adds logging of actual host:port used by libpq connection opened by the listener. It should make it easier to find out if PostgREST is connected to the right host.
The logic to evaluate `JSPath` belongs to `JSPath.hs` module.
Hence, moving this logic from `Auth/Jwt.hs` to here.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
* Calculation of hint message when requested relation is not present in schema cache requires creation of a FuzzySet (to use fuzzy search to find candidate tables). For schemas with many tables it is costly.
This patch introduces dbTablesFuzzyIndex in SchemaCache to memoize the FuzzySet creation.
* Additionally, because of FuzzySet large memory requirements, this patch introduces a limit of 500 relations per schema, above which FuzzySet is not created and hint calculation disabled.
Moves the `negotiateContent` function to `Plan/Negotiate.hs`
module. This also adds comments to describe the current approach
we have for negotiation.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
The `Ord` instance definition for type `Routine` had a logical
error when comparing two routines. The error did not affect any
end users. However, for correctness and completeness reasons, this
commit fixes the error.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Co-authored-by: Joel Jacobson <joel@compiler.org>
Wolfgang Walther