Commit Graph
100 Commits
Author SHA1 Message Date
steve-chavez 03c67f3410 test: move psql_as_superuser as util and reuse it 2026-07-24 13:44:58 -05:00
steve-chavez 205135853d amend: update cabal index so cabal build succeeds 2026-07-11 13:24:09 -05:00
steve-chavez a71ab36096 amend: use warp 3.4.14 for 3522917
Use a release instead of a git commit
2026-07-11 13:12:03 -05:00
steve-chavezandLaurence Isla 490d1dc5d3 add: config to emit warning for legacy target names
Adds the `url_use_legacy_target_names` config.

Enabled (default):
* It allows using the resource name in filters,
  orders or limits when it has an alias, e.g.
  `table?select=alias:target(*)&target.id=eq.1`
* Logs a WARNING with a hint to use the alias
* Returns a Warning header in the response

Disabled:
* It returns an error, only the alias is allowed
* No warnings returned

This feature is deprecated
2026-07-11 02:15:07 +00:00
steve-chavez 2fa8de4e52 chore: add changelog entry for 5077 2026-07-10 19:54:11 -05:00
steve-chavez 3522917021 fix: admin server dies silently by upgrading warp
Uses warp main branch https://github.com/yesodweb/wai/commit/ad041216b643f69a2a9c87cbf4c2988aa4633dd5
since there's no release yet.
2026-07-10 17:56:15 -05:00
steve-chavez 6191243c64 ci: add git to freebsd
Otherwise when using a git commit on stack the build fails because it
lacks the git dependency.
2026-07-06 21:50:26 -05:00
steve-chavez 412c6a1d26 chore: mentions docs on CONTRIBUTING 2026-06-30 10:06:45 -05:00
steve-chavez ded9be6774 nix(loadtest): add comment on loadtest-against 2026-06-22 12:55:56 -05:00
steve-chavez 1ebf480258 amend: required membership for postgrest_test_anonymous
Previous test correction required GRANTing membership for
postgrest_test_anonymous.
2026-06-11 12:00:02 -05:00
steve-chavez e7c8d70333 test: correct test_admin_ready_includes_schema_cache_state
There was an oversight on d556cea, `timeout_authenticator` doesn't have
any privileges on the tables so it shouldn't be making requests to the
API server or it will end up with 401 responses.
2026-06-11 11:51:14 -05:00
steve-chavez 4df0654998 docs: add how-to for Query Identifier 2026-06-06 06:47:33 -05:00
steve-chavez ad790baefb test: explain verbose reveals query identifier
This also requires enabling shared_preload_libraries=pg_stat_statements.
Really we only need compute_query_id=on, but pg_stat_statements is
useful for other tests and it already enables compute_query_id (since
it's `auto` by default).
2026-06-06 06:47:33 -05:00
steve-chavez 6220ab3f31 test: adjust test_positive_pool_metric
Adjusted so it waits for liveness instead of requiring sleeping
2026-05-18 10:46:02 -05:00
steve-chavez 1eba9855c1 test: change wait_for_readiness to an enum
This so we can wait for both readiness and liveness
2026-05-18 10:45:36 -05:00
steve-chavez 8868d13214 test: adjust sleep in test_positive_pool_metric
Otherwise it's not enough time for the pool metric to
reach a negative value.
2026-05-17 16:39:11 -05:00
steve-chavez 70327cf869 test: negative pgrst_db_pool_available in metrics
Proves the failure on https://github.com/PostgREST/postgrest/issues/4622.

This doesn't require additional test infra, only nginx. Taking advantage
of the `stream {}` context which is also compatible with unix socket
besides TCP.
2026-05-17 12:58:58 -05:00
steve-chavez 02a1de2afe test: OPTIONS produce logs but not CORS preflight 2026-05-06 17:21:31 -05:00
steve-chavez 6b3ec5d9e0 test: empty db-anon-role lacks role in logs 2026-05-05 19:29:59 -05:00
steve-chavez 6804ce3cbe test: empty schema cache error has role in logs 2026-05-05 19:29:59 -05:00
steve-chavez 42e53e84e1 test: invalid rpc call contains role in logs 2026-05-05 19:29:59 -05:00
steve-chavez 8105375b64 test: expired JWT shows empty role in logs 2026-05-05 19:29:59 -05:00
steve-chavez e39740cca3 nix: allow empty PGRST_DB_ANON_ROLE on postgrest-run
So we can do `PGRST_DB_ANON_ROLE="" postgrest-run` since empty string has
meaning. argbash conflates both unset and empty string so this requires
a workaround.
2026-05-05 17:32:50 -05:00
steve-chavezandWolfgang Walther b6c6bb27a8 ci: reinstate macos x86 releases
Closes https://github.com/PostgREST/postgrest/issues/4755
2026-04-19 11:26:06 +00:00
steve-chavez bcc8998e5e add: log schema cache queries' timings
This adds a new log line that shows each schema cache query time individually, only on
`log-level=debug`. Like so:

```
$ PGRST_LOG_LEVEL=debug postgrest-with-pg-17 -f test/spec/fixtures/load.sql postgrest-run

....
10/Apr/2026:21:48:45 -0500: Schema cache queried in 192.2 milliseconds
10/Apr/2026:21:48:45 -0500: tables: 72.027 ms, keydeps: 20.118 ms, rels: 6.189 ms, funcs: 35.010 ms, comprels: 4.319 ms, dreps: 1.614 ms, mhandlers: 7.419 ms, tzones: 43.025 ms
```

This helps debug specific schema cache queries being slow like on
https://github.com/PostgREST/postgrest/issues/4613#issuecomment-4210191065 and
https://github.com/PostgREST/postgrest/issues/3046#issuecomment-3469059948.
It also closes https://github.com/PostgREST/postgrest/issues/3215,
which main motivation was to find out which query is slow.

Implementation details
---------------------

To time each query inside a transaction in pure SQL, we do:

```sql
-- start timer
select set_config('pgrst.tmp_x', clock_timestamp()::text, false);
-- run the query
select <query>
-- end timer
select set_config('pgrst.tmp_x', (clock_timestamp() - current_setting('pgrst.tmp_x', false)::timestamptz)::text, false);

-- .... repeated for every query

-- at the end we capture all the timings with
select extract('milliseconds' from current_setting('pgrst.tmp_x', false)::interval), extract(..;
```

Considerations
--------------

Only added this on `log-level=debug` because while the queries are fast
and the data is valuable, it triples the amount of queries we run during schema cache refresh,
which could be troublesome on slow networks. It's possible to reduce the
amount of queries by starting and stopping timers in one statement, but
this would still double the amount of queries and makes the code messy,
doesn't seem worth it.

Also it would pollute pg_stat_statements, it's only required to debug certain
extreme cases anyway.
2026-04-16 13:10:24 -05:00
steve-chavez 6af77360d3 docs: add note about schema cache queries 2026-04-16 13:10:24 -05:00
steve-chavez c9adaed79d docs: target_session_attrs=read-only in libpq>=14 2026-04-10 15:14:41 -05:00
steve-chavez 2fb2e86473 fix: remove red herring warp logs on default log-level
The logs added on e95e815483 are red
herrings under normal operation.

This moves them to `log-level=debug` and removes "error" from the
message prefix.

Fixes https://github.com/PostgREST/postgrest/issues/4799
2026-04-09 16:44:25 -05:00
Steve Chavez 3d98f8d65b refactor: clarify debouncer function 2026-04-07 13:44:03 -05:00
steve-chavez b610b1ea51 docs: clarify set operators need views/functions
Closes https://github.com/PostgREST/postgrest/issues/4780.
2026-04-03 17:02:21 -05:00
steve-chavez effcc080e5 chore: put ai policy first in CONTRIBUTING
Otherwise it's not that visible.
2026-03-23 10:10:58 -05:00
steve-chavez 622c6d3f19 fix: not logging termination unix signals
Under container environments like ECS, it's hard to know when PostgREST
is being terminated.
2026-03-20 13:09:10 -05:00
steve-chavezandWolfgang Walther af4c4157f5 nix: remove uneeded slocat
slocat was introduced to test pipeline mode (see https://github.com/PostgREST/postgrest/pull/2682),
but pipeline mode was not merged https://github.com/PostgREST/postgrest/pull/2707.

So it's really not needed on the loadtests.
2026-03-13 18:56:31 +00:00
steve-chavezandWolfgang Walther 0b7f0197fc chore: update license 2026-03-10 16:41:42 +00:00
steve-chavez 4e4ae3c7a2 chore: add AI policy in CONTRIBUTING
To preserve our scarce review capacity, we reserve the right to reject
seemingly AI generated low-value contributions.

---

Also delete the first paragraph since now it can be interpreted as
low-effort contributions.
2026-03-03 19:56:08 -05:00
steve-chavez 1c6d0816a5 chore: mention tests are required in CONTRIBUTING
Also remove the Haskell conventions subtitle since these are general
recommendations.
2026-03-02 17:57:01 -05:00
steve-chavez 66fda76644 docs: missing link to prefer:tx in config page 2026-02-19 10:23:22 -05:00
steve-chavez abe414ad88 nix: print how to tail the server logs
For easier debugging during loadtests.

Also prefix the echo traces for the postgrest-with-pgrst command.
2026-02-12 20:18:08 -05:00
steve-chavez e95e815483 fix: don't hide async exceptions in logs
Fixes #4646. Using the repro on #4646, this now produces the log:

```
11/Feb/2026:09:40:08 -0500: Warp server error: stack overflow
```

When:
```
$ curl localhost:3000/
curl: (52) Empty reply from server
```
2026-02-12 19:43:50 -05:00
steve-chavez 55cd5a23ff chore: move fixed entry to feature in changelog 2026-01-30 16:23:30 -05:00
steve-chavez ffa393850a chore: update LICENSE 2026-01-11 11:59:31 +08:00
steve-chavez 76e0e1fbff nix: fix sleep default on postgrest-with-pgrst
The sleep was always applying since it had a default.
2026-01-02 06:11:07 +08:00
steve-chavez b8ca1bb04c nix: add new loadtest for errors
Also included it in CI.
2025-12-23 15:00:24 -05:00
steve-chavez 57c5ff5db3 nix: show wait time for fixture loading 2025-12-23 15:00:24 -05:00
steve-chavez 2b83dcc794 nix: sleep param for withPgrst 2025-12-23 15:00:24 -05:00
steve-chavez 7c3bccf4ba nix: show wait time on withPgrst 2025-12-23 15:00:24 -05:00
steve-chavez d2aa6c1b8a nix: timeout param for withPgrst 2025-12-23 15:00:24 -05:00
steve-chavez 013f078bc4 fix: inaccurate Server-Timing durations
The transaction duration was notably off, doing:

```
curl localhost:3000/rpc/sleep?seconds=5 -i
```

Shows `46.1` for the `transaction;dur`, with this fix we obtain
`5007.3`.

Fixes https://github.com/PostgREST/postgrest/issues/4522

This also fixes inaccurate "schema cache queried" logs,
see https://github.com/PostgREST/postgrest/issues/4551.
2025-12-18 17:35:18 -05:00
steve-chavez 8f34afd66e nix: jwt loadtest chain commands and split rsa gen
Chaining the target generation command after the build is done ensures
that if a build takes too long, the dynamic target generation for vegeta
won't cause 401 Unauthorized errors due to already expired JWTs.

Also split the rsa materials writing to another python program for
easier maintenance.
2025-12-16 21:15:28 -05:00
steve-chavez 4cdc4c4861 nix: show build time on postgrest-with-pgrst 2025-12-16 21:15:28 -05:00
steve-chavez 050554e2bb nix: fail jwt loadtest in case of 401
Also print last unauthorized body
2025-12-16 21:15:28 -05:00
steve-chavez 99b8246d6e nix: add --method for postgrest-loadtest
Currently the `postgrest-loadtest -k jwt-*` loadtests use the OPTIONS method,
which is good for isolating the perf changes for JWT auth.

But using GET is more accurate for actual perf improvements,
since it actually hits the database.

This adds a method argument for postgrest-loadtest, so GET can be used like:

```
postgrest-loadtest -k jwt-hs --method GET
```

CI is not changed, this is mostly useful for manually checking variance.
2025-12-12 14:46:46 -05:00
steve-chavez 90bf1762ae nix: rename genTargetsHs to genTargets
Previous name wasn't accurate as RSA is used as well.
2025-12-12 14:46:46 -05:00
steve-chavez 018803473c bump version to 15 2025-10-24 17:39:04 -05:00
steve-chavez 23750e644f bump version to 14.0 2025-10-24 17:39:04 -05:00
steve-chavez 4dff3550b3 docs: new two-part versioning in releases section
Also add entry to the changelog
2025-10-24 17:23:57 -05:00
steve-chavez 3085cc81ec docs: link external auth on architecture page
Partly addresses https://github.com/PostgREST/postgrest/issues/4110
2025-10-24 14:56:24 -05:00
steve-chavez 4ea983163a chore: add changelog for v13.0.8 2025-10-24 14:12:02 -05:00
steve-chavez 1d2350188c chore(changelog): put additions at the top 2025-10-24 13:19:21 -05:00
steve-chavez b20d369176 docs: elaborate on Docker tag
Also link to installation from README.md

Closes https://github.com/PostgREST/postgrest/issues/4385.
2025-10-23 18:57:41 -05:00
steve-chavez 7968b918ec nix: fix postgrest-loadtest command
Uses python for sending SIGNIT cross-platform
2025-10-23 16:05:15 -05:00
steve-chavez 0f7ac1bc39 test: schema cache load max duration
Test to ensure no regression happens after https://github.com/PostgREST/postgrest/pull/4396
2025-10-21 14:10:14 -05:00
steve-chavezandWolfgang Walther 322216c810 chore: remove split-sections
It's a problem for macos on x86_64 and it's no longer needed as mentioned on
https://github.com/PostgREST/postgrest/discussions/4384#discussioncomment-14614110
2025-10-08 06:29:12 +00:00
steve-chavez 09b088d8ff change: change log-query string value to bool
BREAKING CHANGE

As discussed on https://github.com/PostgREST/postgrest/issues/3934#issuecomment-3362806777,
this changes log-query to use a bool value, this:

- Simplifies config for users.
- Reduces effort in testing the different combinations.

Closes #3934
2025-10-03 12:17:12 -05:00
steve-chavez e4458abd33 chore(changelog): adjust jwt sieve change 2025-10-03 12:17:12 -05:00
Steve ChavezandGitHub bcf3457c28 docs: note index on automatic tsvector (#4344) 2025-09-19 13:49:30 -05:00
Steve ChavezandGitHub 9f056ac756 fix: not logging tx variables and pre-request (#4341)
Addresses the first item on https://github.com/PostgREST/postgrest/issues/3934
2025-09-19 12:16:10 -05:00
steve-chavez 4a81f6b1a6 test: refactor test_log_query
It's easier to maintain without using explicit array indexes for
matching the logs
2025-09-17 19:08:10 -05:00
steve-chavez baea47bc85 refactor: make Query module pure
This splits off the transaction concerns into a MainTx module

It's just moving some functions inside the Query module to the MainTx module.
Some types names were also changed so they don't have the "Query" prefix, to avoid confusion.
2025-09-15 23:43:26 -05:00
steve-chavez 15e04903b1 fix: not logging explain query for estimated count
Fixes https://github.com/PostgREST/postgrest/issues/4319
2025-09-15 17:07:12 -05:00
steve-chavez a75ec75fff refactor: put CallReadPlan into CrudPlan 2025-09-15 17:07:12 -05:00
steve-chavez 0acf5a30e6 refactor: remove unnecessary RSPlan constructor
This also removes some unreachable paths in Query module logic.
2025-09-11 20:51:51 -05:00
steve-chavez 4650e7299f chore(changelog): add breaking change for JWT kid 2025-09-11 14:50:21 -05:00
steve-chavez 0bb127057f docs: clarify wording on kid validation 2025-09-11 14:50:21 -05:00
steve-chavez cddfb6cf5e fix: not logging OpenAPI queries
Closes https://github.com/PostgREST/postgrest/issues/4226.

This requires moving query generation to the top App.hs module.

At this point is also simple to log the transaction variables + the
pre-request function call but this is not done here to reduce scope.
2025-09-10 09:04:12 -05:00
steve-chavez 1d2a3e8501 refactor: pass AuthResult to txVarQuery
Saves one parameter to the function.
2025-09-10 09:04:12 -05:00
steve-chavez 966a611d7f refactor: inline txVarQuery and preReqQuery
This way it's easier to see which queries are being executed.

Also use whenJust instead of traverse_.
2025-09-10 09:04:12 -05:00
steve-chavez 1dca23e7af refactor: move accessibleFuncs to SqlFragment 2025-09-04 09:35:33 -05:00
steve-chavez 9c07070263 refactor: move accessibleTables to SqlFragment 2025-09-04 09:35:33 -05:00
steve-chavez 1ace298a1a refactor: move schemaCacheDescription to SqlFragment
This doesn't belong in the schema cache module, as
it's unused there. It's only used on Query.hs.
2025-09-04 09:35:33 -05:00
steve-chavez b2ce322d56 refactor: move snippet logic inside Statements.hs
Some snippet logic was unnecessarily at the Query.hs level.

This is better as we can see what inputs we need to form the main
queries.
2025-09-02 16:26:15 -05:00
steve-chavez 0844743c30 refactor: clarify Plan module and comment 2025-09-02 16:26:15 -05:00
steve-chavez 3e776213b7 refactor: move decoding to Query.hs
Towards solving https://github.com/PostgREST/postgrest/issues/3934.

This makes the Statements internal module pure now it only returns
Hasql.Snippet.
2025-09-02 16:26:15 -05:00
steve-chavez f01d5bfeb3 docs: reorder computed relationships for clarity
Ensure the function inlining is mentioned first
2025-08-19 12:24:47 -05:00
steve-chavez 1dce4a9321 Revert "fix: geojson invalid query on unavailable PostGIS"
This reverts commit 0f1ca8faac.

Reverting for now as it adds one more query to the schema cache and
there's no clear way forward on how to integrate the fix with the
current schema cache queries.

See discussion on
https://github.com/PostgREST/postgrest/pull/4246#pullrequestreview-3093174224.
2025-08-13 14:54:53 -05:00
steve-chavez 0f1ca8faac fix: geojson invalid query on unavailable PostGIS
Closes https://github.com/PostgREST/postgrest/issues/4245.

It adds a query on schema cache construction, but doesn't add a new
attribute to the SchemaCache type. We only need the information to be
passed to the `initialMediaHandlers` function to build the builtin media
handlers.
2025-08-06 10:20:01 -05:00
steve-chavez fa09e4af0d docs: add redirects for broken hashes on auth.rst 2025-07-30 05:28:00 -05:00
steve-chavez fd127f449c docs: correct order of jwt claims sections 2025-07-30 05:14:43 -05:00
steve-chavez ad25479185 chore: put bounded jwt cache as addition 2025-07-30 04:53:22 -05:00
steve-chavez ac155a9391 docs: clarify auth page
Reorganizes the auth page and clarifies some links
2025-07-28 15:43:38 -05:00
steve-chavez aa58e37f03 refactor: add PreQuery module
Small step towards solving #3934.

Adds a module that builds SQL snippets for the transaction variables
and the pre-request function.
2025-07-23 10:06:32 +02:00
steve-chavez 161f37f1f1 test: remove jwt decoding from mixed loadtest
We now have dedicated JWT loadtests, so it's no longer necessary
to conflate JWT here.
2025-07-18 07:25:53 -05:00
steve-chavez 2d4967c4b4 nix: reduce loadtest result verbosity 2025-07-16 17:38:33 -05:00
steve-chavez a183dea9e4 docs: clarify health checks empty response body 2025-07-08 12:45:15 -05:00
steve-chavez e3f8a95b72 nix: complete jwt loadtests
Completes the jwt loadtests, now we have non-cached, cached + worst case
for the cache.

- jwt-hs (edited): now has the cache disabled
- jwt-hs-cache: cache enabled for jwt-hs
- jwt-hs-cache-worst: worst case of the jwt-hs cache
- jwt-rsa (edited): now has the cache disabled
- jwt-rsa-cache: cache enabled for jwt-rsa
- jwt-rsa-cache-worst: worst case of the jwt-rsa cache

Also deletes `nix/tools/generate_targets_rsa.py` and uses a single
python script.

Should prove what's mentioned on
https://github.com/PostgREST/postgrest/pull/4084#issuecomment-2998170423
2025-07-06 19:14:17 -05:00
steve-chavez a87e31b767 nix: shorten loadtest script 2025-07-06 19:14:17 -05:00
steve-chavez 67bd3526c2 nix: use json file for generated jwk on loadtest
Having the JWK as .http file was confusing.

Also split genTargets into two to avoid confusion.
2025-06-20 11:27:44 -05:00
steve-chavez c8adfebb1f nix: fix postgrest-loadtest-to-markdown on NAs
The python script failed when presenting NAs with:

TypeError: boolean value of NA is ambiguous
2025-06-19 16:48:44 -05:00
steve-chavez 49de3c1ab8 nix: adjacent process monitoring report columns 2025-06-19 16:36:25 -05:00
steve-chavez e1c71c92d6 nix: add traces to loadtest commands 2025-06-19 15:04:16 -05:00