Commit Graph
236 Commits
Author SHA1 Message Date
steve-chavez bcc8998e5e add: log schema cache queries' timings
This adds a new log line that shows each schema cache query time individually, only on
`log-level=debug`. Like so:

```
$ PGRST_LOG_LEVEL=debug postgrest-with-pg-17 -f test/spec/fixtures/load.sql postgrest-run

....
10/Apr/2026:21:48:45 -0500: Schema cache queried in 192.2 milliseconds
10/Apr/2026:21:48:45 -0500: tables: 72.027 ms, keydeps: 20.118 ms, rels: 6.189 ms, funcs: 35.010 ms, comprels: 4.319 ms, dreps: 1.614 ms, mhandlers: 7.419 ms, tzones: 43.025 ms
```

This helps debug specific schema cache queries being slow like on
https://github.com/PostgREST/postgrest/issues/4613#issuecomment-4210191065 and
https://github.com/PostgREST/postgrest/issues/3046#issuecomment-3469059948.
It also closes https://github.com/PostgREST/postgrest/issues/3215,
which main motivation was to find out which query is slow.

Implementation details
---------------------

To time each query inside a transaction in pure SQL, we do:

```sql
-- start timer
select set_config('pgrst.tmp_x', clock_timestamp()::text, false);
-- run the query
select <query>
-- end timer
select set_config('pgrst.tmp_x', (clock_timestamp() - current_setting('pgrst.tmp_x', false)::timestamptz)::text, false);

-- .... repeated for every query

-- at the end we capture all the timings with
select extract('milliseconds' from current_setting('pgrst.tmp_x', false)::interval), extract(..;
```

Considerations
--------------

Only added this on `log-level=debug` because while the queries are fast
and the data is valuable, it triples the amount of queries we run during schema cache refresh,
which could be troublesome on slow networks. It's possible to reduce the
amount of queries by starting and stopping timers in one statement, but
this would still double the amount of queries and makes the code messy,
doesn't seem worth it.

Also it would pollute pg_stat_statements, it's only required to debug certain
extreme cases anyway.
2026-04-16 13:10:24 -05:00
steve-chavez 6af77360d3 docs: add note about schema cache queries 2026-04-16 13:10:24 -05:00
steve-chavez c9adaed79d docs: target_session_attrs=read-only in libpq>=14 2026-04-10 15:14:41 -05:00
steve-chavez b610b1ea51 docs: clarify set operators need views/functions
Closes https://github.com/PostgREST/postgrest/issues/4780.
2026-04-03 17:02:21 -05:00
Taimoor ZaeemandSteve Chavez d6c338d588 add: config db-timezone-enabled for optional querying of timezones
To avoid repeated querying of `pg_timezone_names` every time schema
cache is reset, `Prefer: timezone` can be disabled by setting
`db-timezone-enabled = false`.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-04-01 12:48:38 -05:00
arturbent0andWolfgang Walther 5e504a77fa docs: clarify debounce time window in schema cache reloads 2026-03-23 12:04:01 +00:00
Artur Bento de CarvalhoandWolfgang Walther 941c2f2d17 docs: explain schema cache reload behavior with NOTIFY debouncing
Add a "debouncing" section to explain how PostgREST handles multiple NOTIFY events.
2026-03-23 12:04:01 +00:00
Andrei DziahelandSteve Chavez 1f54e2accc add: add "Vary" header to responses
Co-authored-by: Steve Chavez <stevechavezast@gmail.com>
2026-03-16 09:55:20 -05:00
Wolfgang Walther ca858f7544 docs: fix prometheus text format link
Reported by linkcheck.
2026-03-11 08:48:49 +01:00
Taimoor ZaeemandSteve Chavez a36963026e docs: add client error verbosity section to errors page
Add a new section "Client Error Verbosity". Move the details from
the config page to this section and link it.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-03-02 12:07:55 -05:00
Taimoor ZaeemandSteve Chavez 83dc082acf add: config client-error-verbosity to set error verbosity
Set error verbosity using this config. The verbosity can
be set to `verbose` or `minimal` for client error responses.

This only affects client side HTTP responses, server side logs
are not affected by this config.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-25 15:24:27 -05:00
steve-chavez 66fda76644 docs: missing link to prefer:tx in config page 2026-02-19 10:23:22 -05:00
Taimoor ZaeemandSteve Chavez d75b1a7e96 docs: using default parameters on RPC with GET request
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-01-26 08:06:48 -05:00
Taimoor ZaeemandGitHub fe0386e9c4 add: string slicing operator for jwt-role-claim-key (#4603) 2026-01-24 05:40:21 -05:00
Kian-Meng AngandSteve Chavez f5c7cde9b7 docs: fix typos
Found via `codespell -S test -L notin,nam,plantext,cryto,corelation,inout,ot`
2026-01-06 05:24:51 +08:00
Taimoor ZaeemandSteve Chavez abad47ff1b add: log error when db-schemas config contain restricted schemas
We don't allow accessing schemas "pg_catalog" and "information_schema".
Instead of silently failing requests on these schemas, this now adds
logging an error on startup when these schemas are found in `db-schemas`
config.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-12-11 14:28:08 -05:00
Taimoor ZaeemandSteve Chavez d905592115 docs: add asterisk character to reserved characters list
The `*` character is a reserved character in the postgrest
url grammar but we didn't mention it in the docs. Now added.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-11-20 16:20:28 -05:00
Taimoor ZaeemandSteve Chavez 4b8dec8901 docs: clarify postgrest reserved characters in url grammar
Fixes a typo in the documentation where it said that `()`
is a reserved character, when we actually meant to say that
`(` and `)` are reserved characters.

Related discussion in #4254.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-11-20 16:20:28 -05:00
steve-chavez 09b088d8ff change: change log-query string value to bool
BREAKING CHANGE

As discussed on https://github.com/PostgREST/postgrest/issues/3934#issuecomment-3362806777,
this changes log-query to use a bool value, this:

- Simplifies config for users.
- Reduces effort in testing the different combinations.

Closes #3934
2025-10-03 12:17:12 -05:00
Taimoor ZaeemandSteve Chavez a9a1763328 add: --ready flag for postgrest healthcheck
The `--ready` flag is a wrapper around the admin server
`/ready` request. This is done through using an http client
library in postgrest.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-09-28 18:01:10 -05:00
Taimoor ZaeemandSteve Chavez ab2cd766c2 docs: document config file parameter on CLI page
The CLI page had the config file parameter missing. This commit
adds that. Also adds the CLI usage output which shows all
available options. Closes #4300.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-09-26 19:26:47 -05:00
Steve ChavezandGitHub bcf3457c28 docs: note index on automatic tsvector (#4344) 2025-09-19 13:49:30 -05:00
steve-chavez 0bb127057f docs: clarify wording on kid validation 2025-09-11 14:50:21 -05:00
Taimoor ZaeemandSteve Chavez dfe5ba3863 fix(admin): metrics endpoint not responding with Content-Type header
The prometheus metrics text format requires `Content-Type` header
for correct scraping which fails otherwise. Closes #4271.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-08-20 16:13:55 -05:00
steve-chavez f01d5bfeb3 docs: reorder computed relationships for clarity
Ensure the function inlining is mentioned first
2025-08-19 12:24:47 -05:00
steve-chavez fa09e4af0d docs: add redirects for broken hashes on auth.rst 2025-07-30 05:28:00 -05:00
steve-chavez fd127f449c docs: correct order of jwt claims sections 2025-07-30 05:14:43 -05:00
Michal KleczekandGitHub 77ff11de95 feat: JWT cache implementation based on sieve algorithm (#4084)
Changes:

1. Refactoring and some cleanup of JWT handling code:
* Instead of caching AuthResult cache decoded claims (which signature was verified). Validating claims and determining role is done after cache lookup
* Cleaned up API so that usage of it is simplified: lookupJwtCache cache key >>= parseClaims configJwtAud time
* Handling of JwtCacheState initialization and updates of configuration is encapsulated in Auth.JwtCache module

2. Generic high performance (hopefully) scalable, dynamically resizeable cache implementation based on stm, stm-hamt and sieve algorithm. It also integrates with PostgREST measurements infrastructure providing usage stats (ie. hit ratio, evictions count)
2025-07-29 18:51:41 -05:00
steve-chavez ac155a9391 docs: clarify auth page
Reorganizes the auth page and clarifies some links
2025-07-28 15:43:38 -05:00
Wolfgang Walther d013a1fd77 docs: adjust some redirects
Those were reported in the weekly linkcheck.
2025-07-16 11:03:50 +02:00
Wolfgang Walther 55eafdf352 chore(deps): update nixpkgs to 25.05 2025-07-11 15:53:06 +00:00
steve-chavez a183dea9e4 docs: clarify health checks empty response body 2025-07-08 12:45:15 -05:00
Taimoor ZaeemandSteve Chavez 4daa4533d3 docs: horizontal filtering on table-valued functions 2025-07-02 14:28:49 -05:00
Taimoor ZaeemandSteve Chavez dc46aea15e fix: max-affected preference not failing for rpc with strict handling 2025-06-03 14:31:17 -05:00
steve-chavez 72f6036432 docs: external authentication page in explanations
- Move page from External JWT generation
2025-05-30 17:59:51 -05:00
Laurence Isla 9c40a9c6df docs: JWK kid parameter validation 2025-05-29 21:01:39 +00:00
Taimoor ZaeemandSteve Chavez 000cb87ec1 fix: allow db-extra-search-path to accept empty value 2025-05-29 14:25:28 -05:00
Wolfgang Walther 07d9747c40 docs: fix functions link in api/preferences
External link syntax was used instead of internal reference.
2025-05-28 09:09:13 +02:00
Wolfgang Walther 2d5d5ed704 docs: apply redirects
Those redirect, so we might as well hardcode the them.
2025-05-28 09:09:13 +02:00
Taimoor ZaeemandSteve Chavez 94d6ac2f2e docs: explain use of max-affected preference with rpc 2025-05-22 12:23:52 -05:00
Taimoor ZaeemandSteve Chavez 1258ea663c docs: add note in JWT Role Extraction section
Add a note describing that the used JSPath DSL does not
strictly follow the JSONPath as described in RFC 9535
2025-05-18 13:36:43 -05:00
steve-chavez 520631913f docs: add warning for duplicate keys in spread 2025-05-15 17:42:13 -05:00
steve-chavez a659b0005a docs: clarify spread join table 2025-05-15 15:50:54 -05:00
steve-chavez 23f11e3abb docs: clarify fts 2025-05-15 11:57:08 -05:00
steve-chavez 211e27a4d7 docs: clarify spread feature 2025-05-15 11:14:06 -05:00
Laurence Isla c5cba11ecb docs: fix link to SQL Query Logs 2025-05-09 20:59:51 -05:00
steve-chavez 257a7d1488 docs: jwt role extraction link to config
The feature section was missing a link to its config.

Also shorten the section name.
2025-05-09 20:45:37 -05:00
Laurence Isla e0c5b3a314 correct: handle array values in JWT aud claim correctly 2025-05-07 21:54:32 +00:00
steve-chavez 98ca7c15d5 drop: Admin server config endpoint
BREAKING CHANGE

The endpoint was at risk of being left unprotected when exposing it.

The accompanying `admin-server-config-enabled` config was also dropped.
2025-05-04 13:53:50 -05:00
Laurence IslaandSteve Chavez 57ef9988a5 feat: add Content-Length response header 2025-04-24 12:49:18 -05:00