Commit Graph
134 Commits
Author SHA1 Message Date
steve-chavez 4d35eb02e7 fix: inaccurate Server-Timing durations
The transaction duration was notably off, doing:

```
curl localhost:3000/rpc/sleep?seconds=5 -i
```

Shows `46.1` for the `transaction;dur`, with this fix we obtain
`5007.3`.

Fixes https://github.com/PostgREST/postgrest/issues/4522

This also fixes inaccurate "schema cache queried" logs,
see https://github.com/PostgREST/postgrest/issues/4551.

(cherry picked from commit 013f078bc4)
2025-12-18 18:18:09 -05:00
Taimoor ZaeemandSteve Chavez 88538c1357 test(io): move authentication related tests to test_auth.py
The `test_io.py` module is too bloated (2100+ lines). To
logically group related tests, as a first step, this commit
separates authentication related IO tests into `test_auth.py`
module.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit c561a3749c)
2025-11-22 12:45:09 -05:00
Taimoor ZaeemandSteve Chavez 1cbe6b7c5f test(io): remove stale jwt cache test
Removes a test related to jwt cache which is stale
since #4084.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 379eaec8e0)
2025-11-19 16:48:25 -05:00
Taimoor ZaeemandSteve Chavez e4e1b626a6 test(io): move resource embedding tests to test_io.py
- Adds fixtures to `test/io/fixtures.sql` to test resource
  embedding related queries.

- Moves the resource embedding related tests that no longer
  require big schema from `test_big_schema.py` to `test_io.py`.

Closes #4417.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 91abcd49e1)
2025-11-17 13:24:49 -05:00
Taimoor ZaeemandSteve Chavez 4aa712b9d8 fix: db-pre-config function failing with pg reserved words
When db-pre-config is accidentally set to a pg reserved word
like "true", it fails with a confusing error. The function
names should be properly quoted to avoid such errors. This commit
resolves this by quoting the pre-config function name.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit a688878236)
2025-10-30 14:32:37 -05:00
Laurence Isla ea5db2a09a fix: reloading the Schema Cache unnecessarily on a PGRST002 error
When 503 errors happen if the Schema Cache is empty,
it should not retrigger the connection worker since
there's no Schema Cache loaded yet.
2025-10-20 18:42:56 +00:00
Laurence Isla c88ddfe18d fix: not logging the JSON message on a PGRST002 error
It happens right after the configuration is loaded and before
the schema cache is queried.
2025-10-20 18:42:56 +00:00
Michał KłeczekandSteve Chavez c08b87749b test: Separated query and loading internal sleep configs
To make schema cache loading wait tests robust it is necessary to provide three separate internal config variables:
* "internal-schema-cache-query-sleep" - introduces delay in schema queries execution
* "internal-schema-cache-load-sleep" - introduces delay between schema queries execution and processing their results
* "internal-schema-cache-relationship-load-sleep" - introduces delay in processing relationship query results

Thanks to these changes it is now possible to test various schema loading scenarios with the right granularity robustly (eg. make sure requests wait for schema loading but not for relationship loading).
2025-10-17 13:42:35 -05:00
Jens TroegerandSteve Chavez 041d4f8ed2 test(pytest): remove unused variables and clean other minor lint to please ruff 2025-10-10 13:23:05 -05:00
Jens TroegerandWolfgang Walther 4ce859bf46 test(pytest): remove star imports to please ruff linter 2025-10-09 10:10:40 +00:00
steve-chavez 09b088d8ff change: change log-query string value to bool
BREAKING CHANGE

As discussed on https://github.com/PostgREST/postgrest/issues/3934#issuecomment-3362806777,
this changes log-query to use a bool value, this:

- Simplifies config for users.
- Reduces effort in testing the different combinations.

Closes #3934
2025-10-03 12:17:12 -05:00
Taimoor ZaeemandSteve Chavez d064faebb9 test(io): remove unused module imports
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-09-22 12:38:47 -05:00
Steve ChavezandGitHub 9f056ac756 fix: not logging tx variables and pre-request (#4341)
Addresses the first item on https://github.com/PostgREST/postgrest/issues/3934
2025-09-19 12:16:10 -05:00
Taimoor ZaeemandWolfgang Walther 98de226dfd test(io): move reusable functions to postgrest.py
Moves the `set_statement_timeout` and `reset_statement_timeout`
function to postgrest.py. This cleans up test_io.py so it only
contains tests. Also makes these functions reusable in other
other tests like test_cli.py.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-09-18 16:51:00 +00:00
steve-chavez 4a81f6b1a6 test: refactor test_log_query
It's easier to maintain without using explicit array indexes for
matching the logs
2025-09-17 19:08:10 -05:00
steve-chavez 15e04903b1 fix: not logging explain query for estimated count
Fixes https://github.com/PostgREST/postgrest/issues/4319
2025-09-15 17:07:12 -05:00
Taimoor ZaeemandSteve Chavez 59ca9e7f0d test(io): move jwt secret length test to test_cli.py
Closes #4316.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-09-11 12:01:21 -05:00
steve-chavez cddfb6cf5e fix: not logging OpenAPI queries
Closes https://github.com/PostgREST/postgrest/issues/4226.

This requires moving query generation to the top App.hs module.

At this point is also simple to log the transaction variables + the
pre-request function call but this is not done here to reduce scope.
2025-09-10 09:04:12 -05:00
Taimoor ZaeemandSteve Chavez 41b6ebea58 refactor: deduplicate host resolution logic and observations
Replaces the "API server listening on unix socket" with simpler
"API server listening on " observation. This allows refactoring
redundant code.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-09-01 15:00:20 -05:00
Taimoor ZaeemandSteve Chavez ef9ac61d9e test(log): postgrest listening on unix socket observation
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-08-30 13:43:18 -05:00
Taimoor ZaeemandSteve Chavez 7a2840fba6 fix: format of IPv6 address logged at PostgREST startup
The IPv6 address logged at the startup like `::1:80` was
wrong because the port isn't clearly separated. This commit
corrects it, now logging as `[::1]:80`.

This is done in accordance to RFC 3986. In short, we did this
have a clear separation between the port and host because
the components of an IPv6 are separated with the ':' character.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-08-29 13:02:20 -05:00
Laurence Isla bf90f95443 fix: logging the Haskell type instead of the listener error message directly
Previously: Just "connection error..."
Now: connection error...
2025-08-22 19:27:38 +00:00
Taimoor ZaeemandSteve Chavez dfe5ba3863 fix(admin): metrics endpoint not responding with Content-Type header
The prometheus metrics text format requires `Content-Type` header
for correct scraping which fails otherwise. Closes #4271.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-08-20 16:13:55 -05:00
steve-chavez 1dce4a9321 Revert "fix: geojson invalid query on unavailable PostGIS"
This reverts commit 0f1ca8faac.

Reverting for now as it adds one more query to the schema cache and
there's no clear way forward on how to integrate the fix with the
current schema cache queries.

See discussion on
https://github.com/PostgREST/postgrest/pull/4246#pullrequestreview-3093174224.
2025-08-13 14:54:53 -05:00
steve-chavez 0f1ca8faac fix: geojson invalid query on unavailable PostGIS
Closes https://github.com/PostgREST/postgrest/issues/4245.

It adds a query on schema cache construction, but doesn't add a new
attribute to the SchemaCache type. We only need the information to be
passed to the `initialMediaHandlers` function to build the builtin media
handlers.
2025-08-06 10:20:01 -05:00
Michal KleczekandGitHub 77ff11de95 feat: JWT cache implementation based on sieve algorithm (#4084)
Changes:

1. Refactoring and some cleanup of JWT handling code:
* Instead of caching AuthResult cache decoded claims (which signature was verified). Validating claims and determining role is done after cache lookup
* Cleaned up API so that usage of it is simplified: lookupJwtCache cache key >>= parseClaims configJwtAud time
* Handling of JwtCacheState initialization and updates of configuration is encapsulated in Auth.JwtCache module

2. Generic high performance (hopefully) scalable, dynamically resizeable cache implementation based on stm, stm-hamt and sieve algorithm. It also integrates with PostgREST measurements infrastructure providing usage stats (ie. hit ratio, evictions count)
2025-07-29 18:51:41 -05:00
Taimoor ZaeemandWolfgang Walther 96bbbe35e1 test: adjust jwt claims error test to avoid failure
The JWT claims IO test fails too often. This breaks our
workflows. This commit adjusts the edge cases in test to
prevent flakiness.
2025-07-19 12:16:29 +00:00
Taimoor ZaeemandSteve Chavez c5ccf742d6 feat: improve error details of PGRST301 error 2025-05-31 13:30:32 -05:00
Taimoor ZaeemandGitHub ea9d4f49b2 fix: log db-schemas and db-extra-search-path in schema cache load error (#4108) 2025-05-30 10:13:19 -05:00
Taimoor ZaeemandSteve Chavez 000cb87ec1 fix: allow db-extra-search-path to accept empty value 2025-05-29 14:25:28 -05:00
Taimoor ZaeemandGitHub 8390df0fa5 fix: jwt error return status 400 for invalid role (#4081) 2025-05-13 15:09:25 -05:00
Taimoor ZaeemandLaurence Isla 1609e32c3a test: add test when using .aud in jwt-role-claim-key 2025-05-10 16:19:22 +00:00
steve-chavez 98ca7c15d5 drop: Admin server config endpoint
BREAKING CHANGE

The endpoint was at risk of being left unprotected when exposing it.

The accompanying `admin-server-config-enabled` config was also dropped.
2025-05-04 13:53:50 -05:00
Laurence IslaandSteve Chavez 57ef9988a5 feat: add Content-Length response header 2025-04-24 12:49:18 -05:00
Taimoor ZaeemandGitHub bc5ec43300 fix: invalid JWTs after jwt-secret is changed in a config reload (#4015) 2025-04-16 09:40:36 -05:00
Taimoor ZaeemandGitHub f7f87b42ca feat: add Proxy-Status header for better error response 2025-04-05 13:43:39 -05:00
Taimoor ZaeemandSteve Chavez dd29e74150 test: fix config value in io test 2025-03-13 10:19:29 +01:00
Taimoor ZaeemandSteve Chavez e3041fc4a0 test: correct jwt parse time test 2025-03-13 10:19:29 +01:00
Taimoor ZaeemandSteve Chavez 36b6a2c86b fix: improve jwt errors 2025-03-13 00:20:54 +01:00
Taimoor ZaeemandGitHub c9a625ced6 feat: Log PoolRequest and PoolRequestFullfilled observations (#3925) 2025-02-28 12:01:10 -05:00
Taimoor ZaeemandGitHub 390ba19932 fix: handle queries on non-existing table gracefully 2025-02-21 13:49:54 -05:00
Laurence IslaandGitHub 9c880c082a feat: allow logging the SQL query to stderr
- Logs the main SQL query when `log-query=main-query`.
- Only logs at the current `log-level`.
2025-02-18 19:17:26 -05:00
M. Taimoor ZaeemandSteve Chavez c96dc3ee90 fix: log 503 client error to stderr 2025-02-08 21:08:29 -05:00
Taimoor ZaeemandGitHub 71a147392a fix: jwt cache is not purged (#3801) 2025-01-29 14:53:10 -05:00
Wolfgang Walther 07d6d75abe chore: remove deprecation warning in IO tests 2024-12-22 18:49:20 +01:00
steve-chavez 87dddd66d2 fix: clarify "listening" logs
It's not immediately clear on which port the API server is listening.
Also it's not clear that the "pgrst" channel is for database
notifications.

Goes from:

<timestamp>: Admin server listening on 0.0.0.0:3001
<timestamp>: Listening on 0.0.0.0:3000
<timestamp>: Listening for notifications on the "pgrst" channel

To:

<timestamp>: Admin server listening on 0.0.0.0:3001
<timestamp>: API server listening on 0.0.0.0:3000
<timestamp>: Listening for database notifications on the "pgrst" channel
2024-10-02 22:47:22 -05:00
Andrei DziahelandGitHub 46537879ae feat: Add resolved host to "Listening on ..." messages (#3560)
This adds resolved host's IP to "Listening on ..." messages emitted when
app and admin servers start.
2024-08-01 11:31:35 -05:00
7c74f6cf0a fix: schema cache loading before the in-db config (#3670)
Fixes #3660. Load the config after getting the pg version but before loading the schema.

The regression happened on f09655b.

Also remove schema cache load wrapper and separate db queries in different functions.

Co-authored-by: Laurence Isla <lau.isla.c@gmail.com>
2024-08-01 10:37:48 -05:00
steve-chavez 6be59066df fix: schema cache retrying without backoff
Fixes https://github.com/PostgREST/postgrest/issues/3523.

Now if there's a failure when obtaining the pg version OR schema cache,
we do the same retrying process. This way we don't add two retries.

Refactors and renames the "connectionWorker" to "schemaCacheLoader".
This makes more sense since what we really want is the schema cache,
the version is the pre-requisite for ensuring our
schema cache queries work.

Additionally, we no longer log ` Attempting to connect to the database...`
at startup unnecessarily. This is only logged whenever there's a retry attempt.
2024-07-10 21:14:24 -05:00
Laurence Isla e0baf7e78e feat: log error message when JWT secret is less than 32 characters long
breaking change: PostgREST now fails to start or reload the config when the JWT secret is less than 32 characters long.
2024-07-04 12:06:24 -05:00