retryingSchemaCacheLoad flushes the pool upon every retry before it starts reloading the schema. This is too early as schema reloading might take some time during which new connections might be acquired. The consequence is that:
* upon successful schema cache reload we might have some connections created with the old schema cache
* we close connections upon each retry and under load we will keep closing and re-opening connections until schema cache load succeeds
This change is to make sure we flush the pool only after successful schema cache querying but before loading (so that connections acquired during loading wait for it and do not interfere with timing the loading process).
This change introduces a match_log function, that implements lenient matching of log output lines. Tests of log output are retrofited to use the function.
Diagnosing problems with listener channel notifications not being handled properly by PostgREST connected to read replicas is difficult. Issues might be related to lost connections and listener not being connected to the right host after failover or database server restarts.
This patch adds logging of actual host:port used by libpq connection opened by the listener. It should make it easier to find out if PostgREST is connected to the right host.
Fixed integer type mapping in OpenAPI 2.0: replaced the invalid integer format with int32/int64 and added the toSwaggerFormat function to map PostgreSQL types to valid OpenAPI 2.0 formats:
smallint -> int32
integer -> int32
bigint -> int64
Some helpers are provided for introspecting metrics already (used in JWT cache tests). This change provides facilities to additionally validate emited Observation events.
A new Spec module is also implemented, adding basic tests of schema cache reloading - their main goal is to excercise the new infrastructure.
Right now listening sockets initialization, management and usage is split between App, AppState and Admin modules: they are created in AppState.init and remembered in AppState but used only in App and Admin.
It has several negative consequences:
- sockets are initialized even if not needed (eg. command line invocations like dump-config or dump-schema)
- it is impossible to start listening on a socket after initial schema cache load because it requires AppState
This change decouples listen socket management from AppState. Sockets are created only when needed (ie. not in command line tools invocation) and passed to admin application and to Warp by the App module.
- Create separate test suite for observability tests
- Create wrapper script `postgrest-test-observability`
- Add to CI and `postgrest-check`
- Move JWT cache tests under observability tests
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Refactoring: State validation helpers used in JwtCacheSpec moved to SpecHelper
to make them available in other Spec modules.
(cherry picked from commit 85a313a8cc)
Fixes#4646. Using the repro on #4646, this now produces the log:
```
11/Feb/2026:09:40:08 -0500: Warp server error: stack overflow
```
When:
```
$ curl localhost:3000/
curl: (52) Empty reply from server
```
(cherry picked from commit e95e815483)
* Calculation of hint message when requested relation is not present in schema cache requires creation of a FuzzySet (to use fuzzy search to find candidate tables). For schemas with many tables it is costly.
This patch introduces dbTablesFuzzyIndex in SchemaCache to memoize the FuzzySet creation.
* Additionally, because of FuzzySet large memory requirements, this patch introduces a limit of 500 relations per schema, above which FuzzySet is not created and hint calculation disabled.
(cherry picked from commit e592d568c6)
The hasSingleUnnamedParam function was only checking the parameter type
but not whether the parameter actually had no name. This caused functions
with a single NAMED parameter (e.g., `foo(data json)`) to incorrectly
match the single-param fallback mode.
The result was a confusing PostgreSQL error 42883 "function does not exist"
instead of a clean PGRST202 error explaining that no matching function
was found.
Added ppName == mempty check so functions with named parameters don't
incorrectly match the single-param fallback.
(cherry picked from commit fd6a3bdccf)
The `test_io.py` module is too bloated (2100+ lines). To
logically group related tests, as a first step, this commit
separates authentication related IO tests into `test_auth.py`
module.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit c561a3749c)
Removes a test related to jwt cache which is stale
since #4084.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 379eaec8e0)
Renames these scripts to `postgrest-with-pg-xx`. The renaming
helps reduce the length of temporary filenames. This is needed
to ensure that socket file names remain under the maximum
allowed length of 107 chars.
Closes#4461.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 462798dc2d)
- Adds fixtures to `test/io/fixtures.sql` to test resource
embedding related queries.
- Moves the resource embedding related tests that no longer
require big schema from `test_big_schema.py` to `test_io.py`.
Closes#4417.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 91abcd49e1)
Updates streaming-commons to version 0.2.3.1. This resolves#3202.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 58efc2680e)
When db-pre-config is accidentally set to a pg reserved word
like "true", it fails with a confusing error. The function
names should be properly quoted to avoid such errors. This commit
resolves this by quoting the pre-config function name.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit a688878236)
To make schema cache loading wait tests robust it is necessary to provide three separate internal config variables:
* "internal-schema-cache-query-sleep" - introduces delay in schema queries execution
* "internal-schema-cache-load-sleep" - introduces delay between schema queries execution and processing their results
* "internal-schema-cache-relationship-load-sleep" - introduces delay in processing relationship query results
Thanks to these changes it is now possible to test various schema loading scenarios with the right granularity robustly (eg. make sure requests wait for schema loading but not for relationship loading).
There are a few benefits for this:
- All fixtures in one module, so single source of truth.
- The fixtures are automatically imported and injected by pytest
so no explicit imports needed for these.
- Linters won't complain about redefinition of outer scope objects.
Co-authored-by: Jens Troeger <jens.troeger@light-speed.de>
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
The `--ready` flag is a wrapper around the admin server
`/ready` request. This is done through using an http client
library in postgrest.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Moves the `set_statement_timeout` and `reset_statement_timeout`
function to postgrest.py. This cleans up test_io.py so it only
contains tests. Also makes these functions reusable in other
other tests like test_cli.py.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Sets hpctixfile to be unique for every postgrest process that
is run.
Previously, this was based on the test name, but issues arise
when two postgrest processes are run under the same test, which
generates two files where one gets overwritten by the other.
Consequently, coverage data used to get lost, which is now fixed
with this commit.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Closes https://github.com/PostgREST/postgrest/issues/4226.
This requires moving query generation to the top App.hs module.
At this point is also simple to log the transaction variables + the
pre-request function call but this is not done here to reduce scope.
Towards fixing #4316. The config error tests were not
correctly tested. This commit allows capturing stderr
output and then checks the error messages properly.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
