Compare commits

..
99 Commits
Author SHA1 Message Date
steve-chavez 16e2c73a95 bump version to 14.9 2026-04-10 18:25:04 -05:00
steve-chavez fb31654277 fix: remove red herring warp logs on default log-level
The logs added on e95e815483 are red
herrings under normal operation.

This moves them to `log-level=debug` and removes "error" from the
message prefix.

Fixes https://github.com/PostgREST/postgrest/issues/4799
2026-04-10 18:16:19 -05:00
steve-chavez 9230f3f49a Revert "refactor: Simplify schema cache loading triggering logic"
This reverts commit 810023a47c.
2026-04-10 18:16:19 -05:00
steve-chavez 96bbc86756 Revert "refactor: clarify debouncer function"
This reverts commit 577ed4dd4c.
2026-04-10 18:16:19 -05:00
renovate[bot]andWolfgang Walther dd394d6f81 chore(deps): update haskell-actions/setup action to v2.10.4 2026-04-09 08:48:45 +00:00
Steve ChavezandTaimoor Zaeem 577ed4dd4c refactor: clarify debouncer function
(cherry picked from commit 3d98f8d65b)
2026-04-08 10:46:38 +05:00
Michał KłeczekandTaimoor Zaeem 810023a47c refactor: Simplify schema cache loading triggering logic
DISCLAIMER:
This commit was authored entirely by a human without the assistance of LLMs.

Using debouncer to trigger schema cache loading makes it difficult to understand when exactly it is triggered.

(cherry picked from commit a4c1d945ee)
2026-04-08 10:46:38 +05:00
renovate[bot]andWolfgang Walther 687ebf0850 chore(deps): update ubuntu:noble docker digest to 84e77de 2026-04-07 08:10:14 +00:00
Michał KłeczekandSteve Chavez b9c8562641 add: Log pg version details of listener connection
Follow-up to #4617 adding more information to log entry produced upon successful listener connection establishement.
2026-04-06 11:11:16 -05:00
Michał KłeczekandSteve Chavez 34a767a5cc add: Log actual host and port of listener connection
Diagnosing problems with listener channel notifications not being handled properly by PostgREST connected to read replicas is difficult. Issues might be related to lost connections and listener not being connected to the right host after failover or database server restarts.
This patch adds logging of actual host:port used by libpq connection opened by the listener. It should make it easier to find out if PostgREST is connected to the right host.
2026-04-06 11:11:16 -05:00
steve-chavezandTaimoor Zaeem 5eac8bd203 docs: clarify set operators need views/functions
Closes https://github.com/PostgREST/postgrest/issues/4780.
2026-04-06 11:18:42 +05:00
steve-chavez 9f722e0799 bump version to 14.8 2026-04-03 16:51:55 -05:00
Artur Bento de CarvalhoandSteve Chavez 0d4d1dca51 fix: use int32/int64 formats for integer types
Fixed integer type mapping in OpenAPI 2.0: replaced the invalid integer format with int32/int64 and added the toSwaggerFormat function to map PostgreSQL types to valid OpenAPI 2.0 formats:

smallint -> int32
integer -> int32
bigint -> int64
2026-04-03 16:36:55 -05:00
Laurence IslaandSteve Chavez 250747aadc add(logs): Include a HINT when the LISTEN channel breaks due to a Postgres bug
The HINT shows a SQL command that solves the issue.
2026-04-03 16:36:55 -05:00
Michał KłeczekandTaimoor Zaeem aae929a718 test: Schema cache load debouncing
test: adjust replicateM to 100
(cherry picked from commit 328598eaed)
2026-04-03 08:00:57 +05:00
renovate[bot]andWolfgang Walther 77dde73057 chore(deps): update docker/login-action action to v4.1.0 2026-04-02 20:06:52 +00:00
Michal KleczekandSteve Chavez afb95a5268 refactor(test): provide means to validate metrics and observations
Some helpers are provided for introspecting metrics already (used in JWT cache tests). This change provides facilities to additionally validate emited Observation events.
A new Spec module is also implemented, adding basic tests of schema cache reloading - their main goal is to excercise the new infrastructure.
2026-04-02 13:34:38 -05:00
Michał KłeczekandTaimoor Zaeem 8262faa235 refactor: move socket creation and management to App module
Right now listening sockets initialization, management and usage is split between App, AppState and Admin modules: they are created in AppState.init and remembered in AppState but used only in App and Admin.

It has several negative consequences:
- sockets are initialized even if not needed (eg. command line invocations like dump-config or dump-schema)
- it is impossible to start listening on a socket after initial schema cache load because it requires AppState

This change decouples listen socket management from AppState. Sockets are created only when needed (ie. not in command line tools invocation) and passed to admin application and to Warp by the App module.
2026-04-02 12:27:31 +05:00
Michał KłeczekandTaimoor Zaeem 1d40fe5d93 test: Fix flakiness of test_second_request_for_non_existent_table_should_be_quick
Changed divider in assertion (response.elapsed.total_seconds() < first_duration / divider) to 2 (from 10).

(cherry picked from commit 886df84e87)
2026-04-01 15:14:45 +05:00
renovate[bot]andWolfgang Walther 3a4dc5eff3 chore(deps): update codecov/codecov-action action to v6 2026-03-27 11:02:38 +00:00
renovate[bot]andWolfgang Walther 55912515c0 chore(deps): update all dependencies 2026-03-27 08:45:13 +00:00
postgrest-ci[bot]andGitHub 863d1c9e9b docs: explain schema cache reload behavior with NOTIFY debouncing 2026-03-26 11:00:40 +05:00
steve-chavez 806579e659 bump version to 14.7 2026-03-20 14:22:31 -05:00
steve-chavez 1856434a74 fix: not logging termination unix signals
Under container environments like ECS, it's hard to know when PostgREST
is being terminated.
2026-03-20 13:38:40 -05:00
Taimoor ZaeemandSteve Chavez dade15acc4 nix(test): add test suite for observability tests
- Create separate test suite for observability tests

- Create wrapper script `postgrest-test-observability`

- Add to CI and `postgrest-check`

- Move JWT cache tests under observability tests

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-03-20 09:13:47 -05:00
Michał KłeczekandSteve Chavez 79531a612d test(spec): Move metrics state helpers from JwtCacheSpec to SpecHelpers
Refactoring: State validation helpers used in JwtCacheSpec moved to SpecHelper
to make them available in other Spec modules.

(cherry picked from commit 85a313a8cc)
2026-03-19 14:30:18 -05:00
renovate[bot]andWolfgang Walther 9d0e13d961 chore(deps): update ubuntu:noble docker digest to 186072b 2026-03-19 09:15:59 +00:00
renovate[bot]andWolfgang Walther 3e61dc024e chore(deps): update cachix/cachix-action action to v17 2026-03-18 21:24:50 +00:00
renovate[bot]andWolfgang Walther 1bdf773417 chore(deps): update all dependencies 2026-03-18 20:01:41 +00:00
renovate[bot]andWolfgang Walther 55868351f3 chore(deps): update actions/cache action to v5.0.4 2026-03-18 19:09:06 +00:00
renovate[bot]andWolfgang Walther 8c3cb76fbd chore(deps): update ubuntu:noble docker digest to 0d39fcc 2026-03-17 10:22:17 +00:00
renovate[bot]andWolfgang Walther 3bdb69bae7 chore(deps): update actions/create-github-app-token action to v3 2026-03-14 18:06:45 +00:00
renovate[bot]andWolfgang Walther 1ed3c2c197 chore(deps): update all dependencies 2026-03-13 17:23:12 +00:00
renovate[bot]andWolfgang Walther dde145b685 chore(deps): update all dependencies 2026-03-13 17:19:11 +00:00
Wolfgang Walther 720705e32b docs: fix prometheus text format link
Reported by linkcheck.
2026-03-11 09:45:21 +01:00
steve-chavez 6813ceaf31 bump version to 14.6 2026-03-06 16:30:04 -05:00
Taimoor Zaeemandsteve-chavez 73c8ae0bbc fix(error): leaking table and function names when calculating hint
Increase similarity score to 0.75 from 0.33 for table and functions
error hint.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-03-06 16:16:05 -05:00
Laurence Isla 6f356e32b6 chore: update sponsors
* The list of sponsors is now ordered by total contribution value (highest first)
* Removed former backers from sponsor pages
2026-03-03 23:11:22 -05:00
Laurence Isla c4772184e4 chore: update sponsor 2026-03-03 23:11:22 -05:00
dshukertjrandWolfgang Walther 0602080acb docs: Update the Supabase logo to a correct one 2026-02-19 10:54:29 +01:00
steve-chavez dc67e4d3f5 chore: fix changelog entries 2026-02-13 13:40:38 -05:00
steve-chavez d89826b982 bump version to 14.5 2026-02-12 20:12:52 -05:00
steve-chavez 94350fdde6 fix: don't hide async exceptions in logs
Fixes #4646. Using the repro on #4646, this now produces the log:

```
11/Feb/2026:09:40:08 -0500: Warp server error: stack overflow
```

When:
```
$ curl localhost:3000/
curl: (52) Empty reply from server
```

(cherry picked from commit e95e815483)
2026-02-12 20:11:44 -05:00
Taimoor ZaeemandWolfgang Walther 230eb3630c docs(install): update postgresql minimum supported version
PostgREST dropped support for PostgreSQL version 12 however,
it was not reflected in the docs.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-12 10:23:29 +01:00
steve-chavez 5485b8ca9a bump version to 14.4 2026-01-29 14:13:49 -05:00
steve-chavez 895e9c536c chore: remove wrong entry in CHANGELOG 2026-01-29 13:24:20 -05:00
Michal Kleczekandsteve-chavez 16c767134c fix: listener running with exception masked after first failure 2026-01-29 13:22:46 -05:00
Laurence IslaandSteve Chavez 0a8b836435 fix: filtering the returned representation whenn using or/and filters on mutations
(cherry picked from commit 1682677297)
2026-01-29 09:16:08 -05:00
Michal KleczekandSteve Chavez 5796f86100 fix: ensure Listener connections are released
retryingListen function potentially leaks database connections. This patch ensures the connections are released in case of listen/notify errors.

(cherry picked from commit 00c7cb1a22)
2026-01-28 18:26:15 -05:00
Wolfgang Walther 101eac1cce docs: fix links
datrium.com doesn't exist anymore, while euronodes.com seems to only
fail SSL in CI.
2026-01-28 09:57:23 +01:00
renovate[bot]andWolfgang Walther 1ae14afdf2 chore(deps): update haskell-actions/setup action to v2.10.2 2026-01-11 17:36:28 +00:00
Wolfgang Walther 69090bd224 ci: pin backport action to version instead of default branch 2026-01-11 18:34:43 +01:00
renovate[bot]andWolfgang Walther 5d5160fbd7 chore(deps): update haskell-actions/setup action to v2.10.1 2026-01-05 19:01:32 +00:00
steve-chavez 545f45d9de bump version to 14.3 2026-01-03 16:45:56 +08:00
Taimoor Zaeemandsteve-chavez eb55e73645 chore: move changelog entry to unreleased section
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-01-03 16:44:10 +08:00
Michał KłeczekandSteve Chavez e252a4900c fix: Performance and high memory usage of relation hint calculation
* Calculation of hint message when requested relation is not present in schema cache requires creation of a FuzzySet (to use fuzzy search to find candidate tables). For schemas with many tables it is costly.
This patch introduces dbTablesFuzzyIndex in SchemaCache to memoize the FuzzySet creation.

* Additionally, because of FuzzySet large memory requirements, this patch introduces a limit of 500 relations per schema, above which FuzzySet is not created and hint calculation disabled.

(cherry picked from commit e592d568c6)
2026-01-03 15:18:37 +08:00
Taimoor ZaeemandSteve Chavez 01bdb05c89 nix: add config file for hlint
Adds a config file for hlint containing arguments and
custom warnings.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-12-23 11:51:05 -05:00
122ed4d02e refactor: fix definition of Ord instance for Routine type (#4577)
The `Ord` instance definition for type `Routine` had a logical
error when comparing two routines. The error did not affect any
end users. However, for correctness and completeness reasons, this
commit fixes the error.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Co-authored-by: Joel Jacobson <joel@compiler.org>
2025-12-23 11:51:05 -05:00
renovate[bot]andWolfgang Walther 7ff6755af7 chore(deps): update docker/setup-buildx-action action to v3.12.0 2025-12-20 20:24:23 +00:00
steve-chavez 29d6d35020 bump version to 14.2 2025-12-18 21:30:38 -05:00
steve-chavez 4d35eb02e7 fix: inaccurate Server-Timing durations
The transaction duration was notably off, doing:

```
curl localhost:3000/rpc/sleep?seconds=5 -i
```

Shows `46.1` for the `transaction;dur`, with this fix we obtain
`5007.3`.

Fixes https://github.com/PostgREST/postgrest/issues/4522

This also fixes inaccurate "schema cache queried" logs,
see https://github.com/PostgREST/postgrest/issues/4551.

(cherry picked from commit 013f078bc4)
2025-12-18 18:18:09 -05:00
renovate[bot]andWolfgang Walther e126956c1b chore(deps): update haskell-actions/setup action to v2.9.1 2025-12-17 18:01:25 +00:00
Joel JakobssonandSteve Chavez ce7871c047 fix: hasSingleUnnamedParam incorrectly matching named parameters
The hasSingleUnnamedParam function was only checking the parameter type
but not whether the parameter actually had no name. This caused functions
with a single NAMED parameter (e.g., `foo(data json)`) to incorrectly
match the single-param fallback mode.

The result was a confusing PostgreSQL error 42883 "function does not exist"
instead of a clean PGRST202 error explaining that no matching function
was found.

Added ppName == mempty check so functions with named parameters don't
incorrectly match the single-param fallback.

(cherry picked from commit fd6a3bdccf)
2025-12-16 17:24:20 -05:00
Taimoor ZaeemandSteve Chavez a59e6d97c5 refactor: create function to handle response preferences
Centralizes handling of preferences that are used to
create `Preference-Applied` header which is returned on
responses.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 29c2aa015f)
2025-12-16 11:20:33 -05:00
renovate[bot]andWolfgang Walther 40fec696a6 chore(deps): update korthout/backport-action action to v4.0.1 2025-12-15 20:17:46 +00:00
renovate[bot]andWolfgang Walther 9eb36f1244 chore(deps): update all dependencies 2025-12-13 11:31:10 +00:00
Laurence IslaandSteve Chavez 720eb8e528 chore(changelog): move incorrectly placed fixes 2025-12-12 14:47:26 -05:00
Laurence IslaandSteve Chavez aec95f7944 refactor: use only Lazy.ByteString to calculate the response body length 2025-12-12 14:47:26 -05:00
Laurence IslaandSteve Chavez 1c33d2dd38 fix: add missing Content-Length to empty HTTP 201 responses
For when a preference other than return=representation is requested.
2025-12-12 14:47:26 -05:00
Laurence IslaandSteve Chavez 7be638f0f2 fix: regression that truncates error message when offset is out of bounds
- Happens when offset > the number of rows and when "Prefer: count=exact" header is sent
- Regression introduced in commit 57ef998
2025-12-12 14:47:26 -05:00
Taimoor ZaeemandSteve Chavez d4b8109522 fix: misleading logs on unsupported postgresql versions
Postgrest fails on unsupported pg versions. However before killing
the thread, it continues to print a few more log messages which
were misleading. This commit fixes this by making sure that the
no log message should be printed after the unsupported pg version
observation and kill the thread immediately.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 0fa2f569a7)
2025-12-10 20:24:14 -05:00
renovate[bot]andWolfgang Walther f2b96b1d41 chore(deps): update codecov/codecov-action action to v5.5.2 2025-12-10 12:24:00 +00:00
renovate[bot]andWolfgang Walther cc43d356c1 chore(deps): update actions/create-github-app-token action to v2.2.1 2025-12-06 11:00:16 +00:00
renovate[bot]andWolfgang Walther 8bcbbd1bfd chore(deps): update all dependencies 2025-12-03 13:52:35 +00:00
renovate[bot]andWolfgang Walther ec4d2e2af3 chore(deps): update actions/checkout action to v6 2025-12-03 13:52:23 +00:00
Taimoor ZaeemandSteve Chavez e8cb0e33eb test(io): fix freeport function to prevent failures
Sometimes, a healthcheck related test fails as occurred in
https://github.com/PostgREST/postgrest/actions/runs/19771357953/job/56655949002.
This happens due to freeport function accidently picking up a used port.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 50eec773bf)
2025-11-30 13:38:55 -05:00
Taimoor ZaeemandSteve Chavez 05074f41c2 test(io): move fixtures to fixtures/ directory
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 35de13eebd)
2025-11-28 13:23:20 -05:00
Taimoor ZaeemandSteve Chavez c9c617cce1 test(io): rename fixtures.sql to load.sql
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 3baa6b0063)
2025-11-28 13:23:20 -05:00
Taimoor ZaeemandWolfgang Walther e429974f31 nix(shell): remove postgrest/ directory prefix when running pg (#4502)
When running postgres from nix-shell, nix creates a directory
structure like `postgrest/postgrest-with-pg-17-XXX` in the `/tmp`
directory. This commit removes the extra `postgrest/` prefix to
shorten length of absolute path length of filenames.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 802cce9a28)
2025-11-26 08:33:23 +00:00
Taimoor ZaeemandSteve Chavez 88538c1357 test(io): move authentication related tests to test_auth.py
The `test_io.py` module is too bloated (2100+ lines). To
logically group related tests, as a first step, this commit
separates authentication related IO tests into `test_auth.py`
module.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit c561a3749c)
2025-11-22 12:45:09 -05:00
Taimoor ZaeemandSteve Chavez 1cbe6b7c5f test(io): remove stale jwt cache test
Removes a test related to jwt cache which is stale
since #4084.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 379eaec8e0)
2025-11-19 16:48:25 -05:00
Taimoor ZaeemandWolfgang Walther f97200948f nix: shorten postgrest-with-postgresql-xx scripts
Renames these scripts to `postgrest-with-pg-xx`. The renaming
helps reduce the length of temporary filenames. This is needed
to ensure that socket file names remain under the maximum
allowed length of 107 chars.

Closes #4461.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 462798dc2d)
2025-11-18 18:27:17 +00:00
Taimoor ZaeemandWolfgang Walther 0bacce6909 docs: update server-host config in docker-compose example
The docker-compose example did not work with multiple containers
when PGRST_SERVER_HOST is set to `localhost`. This updates the
value to `0.0.0.0` allowing other containers to connect.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 08c6deba12)
2025-11-18 18:13:26 +00:00
renovate[bot]andWolfgang Walther 4c3ac2f087 chore(deps): update actions/checkout action to v5.0.1 2025-11-18 10:06:11 +00:00
Wolfgang Walther 1c6f215bd7 ci/test: remove macos x86 flake check
See previous commit.
2025-11-18 11:05:43 +01:00
Wolfgang Walther 94fb0a489a ci: remove macos x86 builds
The x86 GitHub runner will not be available anymore, soon.

We might be able to re-introduce this, once we can build a static
executable via Nix on darwin, too.
2025-11-18 11:01:41 +01:00
Taimoor ZaeemandSteve Chavez e4e1b626a6 test(io): move resource embedding tests to test_io.py
- Adds fixtures to `test/io/fixtures.sql` to test resource
  embedding related queries.

- Moves the resource embedding related tests that no longer
  require big schema from `test_big_schema.py` to `test_io.py`.

Closes #4417.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 91abcd49e1)
2025-11-17 13:24:49 -05:00
renovate[bot]andWolfgang Walther 810341f635 chore(deps): update ubuntu:noble docker digest to c35e29c 2025-11-15 13:52:10 +00:00
Taimoor ZaeemandWolfgang Walther 4a6f0b4a18 chore(cabal): remove unused haskell dependencies
This should reduce setup time for build process.

- cache: introduced in #2928, defunct since #4084
- clock: introduced in #2928, defunct since #4084
- heredoc: introduced in #714, defunct since #4390
- iproute: introduced in #3560, defunct since #4288

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 99217433b1)
2025-11-10 14:24:48 +00:00
steve-chavez b3b4e5ff35 bump version to 14.1 2025-11-05 09:09:22 -05:00
Taimoor ZaeemandWolfgang Walther 84f437b6c9 chore(changelog): update versioning scheme description
The changelog description mentioned that we follow semantic
versioning but from now on we don't. Hence updated the description
to reflect new versioning policy.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit eb908c696f)
2025-11-04 07:25:35 +00:00
renovate[bot]andWolfgang Walther a3c8065e58 chore(deps): update actions/checkout digest to 71cf226 2025-11-03 20:47:35 +00:00
Taimoor ZaeemandWolfgang Walther c797c09e22 fix: server-host !6 incorrectly binds to IPv4 address
Updates streaming-commons to version 0.2.3.1. This resolves #3202.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 58efc2680e)
2025-11-03 11:40:09 +00:00
Taimoor ZaeemandWolfgang Walther fc6fbe9748 chore(changelog): fix typo in changelog entry
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 8180905890)
2025-11-01 09:18:20 +00:00
Taimoor ZaeemandSteve Chavez 4aa712b9d8 fix: db-pre-config function failing with pg reserved words
When db-pre-config is accidentally set to a pg reserved word
like "true", it fails with a confusing error. The function
names should be properly quoted to avoid such errors. This commit
resolves this by quoting the pre-config function name.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit a688878236)
2025-10-30 14:32:37 -05:00
Taimoor ZaeemandSteve Chavez 939061baff refactor: move escapeIdent function to Identifiers.hs
Moves the functions `escapeIdent` and `trimNullChars` to
SchemaCache/Identifiers.hs module.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 66f84c5903)
2025-10-30 12:24:49 -05:00
Taimoor ZaeemandSteve Chavez 6150d53592 refactor: sort exports of Identifiers.hs and SqlFragments.hs
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 5d9b169380)
2025-10-30 12:24:49 -05:00
Wolfgang Walther d245e07df5 ci: fix tag job with new release workflow
A single component version is the development version, everything with
more components is not. Thus, we only need to check for a single dot.
2025-10-25 10:21:51 +02:00
renovate[bot]andWolfgang Walther e913efb8cb chore(deps): update all dependencies 2025-10-25 08:12:30 +00:00
287 changed files with 8829 additions and 9322 deletions
+42
View File
@@ -0,0 +1,42 @@
freebsd_instance:
image_family: freebsd-14-3
build_task:
# Don't change this name without adjusting .github/workflows/build.yaml
name: Build FreeBSD (Stack)
install_script: pkg install -y postgresql16-client hs-stack git
only_if: |
$CIRRUS_TAG != '' || $CIRRUS_BRANCH == 'main' || $CIRRUS_BRANCH =~ 'v*' ||
changesInclude(
'.github/workflows/build.yaml',
'.github/actions/artifact-from-cirrus/**',
'.cirrus.yml',
'postgrest.cabal',
'stack.yaml*',
'**.hs'
)
stack_cache:
folders: /.stack
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
stack_work_cache:
folders: .stack-work
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
- find main src -type f -iname '*.hs' -exec md5sum "{}" +
build_script: |
stack build -j 1 --local-bin-path . --copy-bins
strip postgrest
bin_artifacts:
path: postgrest
+5
View File
@@ -0,0 +1,5 @@
# TODO: Remove this once a new actionlint release has been cut
# and made its way to us through nixpkgs.
self-hosted-runner:
labels:
- ubuntu-24.04-arm
@@ -0,0 +1,119 @@
name: Artifact from Cirrus
description: Waits for a specific Cirrus CI run to complete, then downloads the artifact and uploads it to the current workflow. This will silently succeed if Cirrus CI did not schedule a task within 2 minutes.
inputs:
download:
description: Name of Artifact to download from Cirrus CI
required: true
task:
description: Name of Cirrus Task
required: true
token:
description: GitHub Token
required: true
upload:
description: Name of Artifact to upload on GitHub Actions
required: true
runs:
using: composite
steps:
- shell: bash
run: echo "GH_TOKEN=${{ inputs.token }}" >> "$GITHUB_ENV"
- name: Wait for Check Suite to be created
id: check-suite
env:
# GITHUB_SHA does weird things for pull request, so we roll our own:
COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
shell: bash
run: |
get_check_runs_url() {
gh api "repos/{owner}/{repo}/commits/${COMMIT}/check-suites" \
| jq -r '.check_suites[] | select(.app.slug == "cirrus-ci") | .check_runs_url'
}
for _ in $(seq 1 12); do
check_runs_url="$(get_check_runs_url)"
if [ -z "$check_runs_url" ]; then
echo "Cirrus CI task has not started, yet. Waiting..."
sleep 10
else
echo "check_runs_url=$check_runs_url" >> "$GITHUB_OUTPUT"
exit 0
fi
done
>&2 echo "Cirrus CI check suite not found. Is Cirrus CI enabled for this repo?"
- name: Find task by name
id: find-task
if: steps.check-suite.outputs.check_runs_url
shell: bash
run: |
get_number_of_tasks() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -r '.check_runs | map(select(.name == "${{ inputs.task }}")) | length'
}
tasks="$(get_number_of_tasks)"
case "$tasks" in
0)
echo "Task not found, assuming it's skipped intentionally..."
exit 0
;;
1)
echo "task_found=1" >> "$GITHUB_OUTPUT"
exit 0
;;
*)
>&2 echo "More than 1 task with the same name found. Don't know what to do..."
exit 1
;;
esac
- name: Wait for Cirrus CI to complete task
if: steps.find-task.outputs.task_found
shell: bash
run: |
get_conclusion() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -r '.check_runs[] | select(.name == "${{ inputs.task }}" and .status == "completed") | .conclusion'
}
while true; do
conclusion="$(get_conclusion)"
if [ -z "$conclusion" ]; then
echo "Cirrus CI task has not completed, yet. Waiting..."
sleep 30
else
if [ "$conclusion" == "success" ]; then
break
else
exit 1
fi
fi
done
- name: Download artifact from Cirrus CI
if: steps.find-task.outputs.task_found
id: download
shell: bash
run: |
get_external_id() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -er '.check_runs[] | select(.name == "${{ inputs.task }}") | .external_id'
}
archive="$(mktemp)"
artifacts="$(mktemp -d)"
until curl --no-progress-meter --fail -o "${archive}" \
"https://api.cirrus-ci.com/v1/artifact/task/$(get_external_id)/${{ inputs.download }}.zip"
do
# This happens when a tag is pushed on the same commit. In this case the
# job is immediately marked as "completed" for us, so we end up here after a few
# seconds - but the actual Cirrus CI task is still running and didn't produce its artifact, yet.
echo "Artifact not found on Cirrus CI, yet. Waiting..."
sleep 30
done
unzip "${archive}" -d "${artifacts}"
echo "artifacts=${artifacts}" >> "$GITHUB_OUTPUT"
- name: Save artifact to GitHub Actions
if: steps.find-task.outputs.task_found
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: ${{ inputs.upload }}
path: ${{ steps.download.outputs.artifacts }}
if-no-files-found: error
+7 -6
View File
@@ -8,6 +8,7 @@ inputs:
required: true
save-prs:
description: Whether to additionally store the cache in a pull request, too. Should only be used for very small caches.
type: boolean
prefix:
description: Cache key prefix to be used in both primary key and restore-keys.
required: true
@@ -18,17 +19,17 @@ inputs:
runs:
using: composite
steps:
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
- uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with:
path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ runner.arch }}-${{ inputs.prefix }}-${{ inputs.suffix }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: |
${{ runner.os }}-${{ runner.arch }}-${{ inputs.prefix }}-
- uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
${{ runner.os }}-${{ inputs.prefix }}-
- uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with:
path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ runner.arch }}-${{ inputs.prefix }}-${{ inputs.suffix }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: |
${{ runner.os }}-${{ runner.arch }}-${{ inputs.prefix }}-
${{ runner.os }}-${{ inputs.prefix }}-
-35
View File
@@ -1,35 +0,0 @@
name: Run anywhere
description: Runs the same code either in a VM or on the bare machine
inputs:
vm:
description: Which VM to run on.
envs:
description: List of relevant environment variables, which might need to be copied into the VM.
prepare:
description: Code to run in a prepare step, e.g. installing dependencies.
run:
description: Code to run as the main action.
required: true
runs:
using: composite
steps:
- if: ${{ inputs.vm == 'freebsd' }}
uses: vmactions/freebsd-vm@83b151f58c6047089f4c80eb5ba2039d158ce093 # v1.5.3
with:
envs: ${{ inputs.envs }}
prepare: ${{ inputs.prepare }}
# Work around https://github.com/vmactions/freebsd-vm/issues/59
run: |
pw user add -n action -m
su action -c '${{ inputs.run }}'
- if: ${{ inputs.vm == '' }}
name: Prepare
shell: ${{ runner.os == 'Windows' && 'pwsh' || 'bash' }}
run: ${{ inputs.prepare }}
- if: ${{ inputs.vm == '' }}
name: Run
shell: ${{ runner.os == 'Windows' && 'pwsh' || 'bash' }}
run: ${{ inputs.run }}
+2 -2
View File
@@ -11,12 +11,12 @@ inputs:
runs:
using: composite
steps:
- uses: nixbuild/nix-quick-install-action@9f63be77f412a248c9d9a65a4c82cf066cdf8f0c # v35
- uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
with:
nix_conf: |-
always-allow-substitutes = true
max-jobs = auto
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
- uses: cachix/cachix-action@1eb2ef646ac0255473d23a5907ad7b04ce94065c # v17
with:
name: postgrest
authToken: ${{ inputs.authToken }}
-3
View File
@@ -4,9 +4,6 @@ codecov:
comment: false
github_checks:
annotations: true
coverage:
status:
project:
+41
View File
@@ -13,6 +13,9 @@
},
"packageRules": [
{
"matchBaseBranches": [
"/^v[0-9]+/"
],
"matchManagers": [
"haskell-cabal"
],
@@ -23,6 +26,44 @@
"/^v[0-9]+/"
],
"groupName": "all dependencies"
},
{
"matchManagers": [
"haskell-cabal"
],
"matchPackageNames": [
"base",
"bytestring",
"containers",
"directory",
"mtl",
"parsec",
"process",
"text"
],
"groupName": "GHC dependencies"
},
{
"matchManagers": [
"haskell-cabal"
],
"matchPackageNames": [
"hasql",
"hasql-dynamic-statements",
"hasql-notifications",
"hasql-transaction",
"hasql-pool"
],
"groupName": "hasql"
},
{
"matchManagers": [
"haskell-cabal"
],
"matchPackageNames": [
"fuzzyset"
],
"allowedVersions": "<0.3"
}
]
}
+5 -6
View File
@@ -9,7 +9,7 @@ on:
jobs:
backport:
name: Backport
runs-on: ubuntu-slim
runs-on: ubuntu-24.04
# It triggers only when PR is already merged on either:
#
# - The merge event itself (action != labeled) or
@@ -28,9 +28,9 @@ jobs:
# This actions creates the github token using the postgrest app secrets
- name: Create Github App Token
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0
with:
client-id: ${{ vars.POSTGREST_CI_APP_ID }}
app-id: ${{ vars.POSTGREST_CI_APP_ID }}
private-key: ${{ secrets.POSTGREST_CI_PRIVATE_KEY }}
permission-contents: write
permission-pull-requests: write
@@ -38,15 +38,14 @@ jobs:
# This is required for backport action to cherry-pick the PR
- name: Fetch PR ref
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
allow-unsafe-pr-checkout: true
ref: ${{ github.event.pull_request.head.sha }}
token: ${{ steps.app-token.outputs.token }}
# Backport action that creates the PR with given settings
- name: Create backport PR
uses: korthout/backport-action@2e830a1d0b8269505846ddd407a70876913ad1f8 # v4.6
uses: korthout/backport-action@3c06f323a58619da1e8522229ebc8d5de2633e46 # v4.3.0
with:
github_token: ${{ steps.app-token.outputs.token }}
pull_description: 'Backport for #${pull_number}.'
+69 -67
View File
@@ -16,7 +16,6 @@ on:
- .github/*
- '*.nix'
- nix/**
- flake.lock
- .cirrus.yml
- cabal.project*
- postgrest.cabal
@@ -31,20 +30,10 @@ concurrency:
jobs:
static:
strategy:
fail-fast: false
matrix:
include:
- name: Linux aarch64
runs-on: ubuntu-24.04-arm
artifact: aarch64
- name: Linux x86-64
runs-on: ubuntu-24.04
artifact: x86-64
name: Nix - ${{ matrix.name }} static
runs-on: ${{ matrix.runs-on }}
name: Nix - Linux x86-64 static
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -53,44 +42,44 @@ jobs:
- name: Build static executable
run: nix-build -A postgrestStatic -A postgrestStatic.tests
- name: Save built executable as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: postgrest-linux-static-${{ matrix.artifact }}
name: postgrest-linux-static-x86-64
path: result/bin/postgrest
if-no-files-found: error
- name: Build Docker image
run: nix-build -A docker.image --out-link postgrest-docker-${{ matrix.artifact }}.tar.gz
run: nix-build -A docker.image --out-link postgrest-docker.tar.gz
- name: Save built Docker image as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: postgrest-docker-${{ matrix.artifact }}
path: postgrest-docker-${{ matrix.artifact }}.tar.gz
name: postgrest-docker-x86-64
path: postgrest-docker.tar.gz
if-no-files-found: error
- name: Test static executable with NixOS' VM test
# GHA's ARM runner does not support KVM
if: runner.arch == 'X64'
run: nix-build -A nixpkgs-nixos-test
macos:
name: Nix - MacOS
runs-on: macos-26
runs-on: macos-15
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Install nix-build-uncached
run: nix-env -f default.nix -iA nix-build-uncached
- name: Install gnu sed
run: brew install gnu-sed
- name: Build everything (default.nix)
run: nix-build-uncached
- name: Build everything (shell.nix)
run: nix-build-uncached shell.nix
- name: Build everything
run: |
# The --dry-run will give us a list of derivations to download from cachix and
# derivations to build. We only take those that would have to be built and then build
# those explicitly. This has the advantage that pure verification will not include
# a download anymore, making it much faster. If something needs to be built, only
# the dependencies required to do so will be downloaded, but not everything.
nix-build --dry-run 2>&1 \
| gsed -e '1,/derivations will be built:$/d' -e '/paths will be fetched/Q' \
| xargs nix-build
stack:
@@ -98,66 +87,66 @@ jobs:
fail-fast: false
matrix:
include:
- name: FreeBSD x86-64
runs-on: ubuntu-24.04
vm: freebsd
artifact: postgrest-freebsd-x86-64
deps: pkg install -y git postgresql16-client hs-stack
- name: Linux aarch64
runs-on: ubuntu-24.04-arm
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-ubuntu-aarch64
deps: sudo apt-get update && sudo apt-get install libpq-dev
- name: MacOS aarch64
runs-on: macos-14
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-macos-aarch64
deps: brew link --force libpq
- name: MacOS x86-64
runs-on: macos-15-intel
artifact: postgrest-macos-x86-64
deps: brew link --force libpq
- name: Windows
runs-on: windows-2022
cache: |
C:\sr\pantry
C:\sr\snapshots
C:\sr\stack.sqlite3
deps: Add-Content $env:GITHUB_PATH $env:PGBIN
artifact: postgrest-windows-x86-64
name: Stack - ${{ matrix.name }}
runs-on: ${{ matrix.runs-on }}
env:
# Putting .stack in the working directory helps with moving this in and out of the FreeBSD VM.
STACK_ROOT: ${{ github.workspace }}/.stack
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- if: ${{ !matrix.vm }}
uses: haskell-actions/setup@6037f33647c3f17758a2356c80fc4a53d7e0685d # v2.12.0
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: haskell-actions/setup@de26526e12bc780fb9d384c1fb61c0bf02e3a40d # v2.10.4
with:
# This must match the version in stack.yaml's resolver
ghc-version: 9.10.3
ghc-version: 9.6.7
enable-stack: true
stack-no-global: true
stack-setup-ghc: true
- name: Cache .stack
- name: Cache ~/.stack
uses: ./.github/actions/cache-on-main
with:
path: .stack
prefix: ${{ matrix.vm }}${{ matrix.vm && '-' }}stack
path: ${{ matrix.cache }}
prefix: stack
suffix: ${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
- name: Cache .stack-work
uses: ./.github/actions/cache-on-main
with:
path: .stack-work
save-prs: true
prefix: ${{ matrix.vm }}${{ matrix.vm && '-' }}stack-work-${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
prefix: stack-work-${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
suffix: ${{ hashFiles('main/**/*.hs', 'src/**/*.hs') }}
- name: Install dependencies
if: matrix.deps
run: ${{ matrix.deps }}
- name: Build with Stack
uses: ./.github/actions/run-anywhere
with:
vm: ${{ matrix.vm }}
envs: STACK_ROOT
prepare: ${{ matrix.deps }}
run: |
stack build --lock-file error-on-write --local-bin-path result --copy-bins
strip result/postgrest*
run: stack build --lock-file error-on-write --local-bin-path result --copy-bins
- name: Strip Executable
run: strip result/postgrest*
- name: Save built executable as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: ${{ matrix.artifact }}
path: |
@@ -166,16 +155,29 @@ jobs:
if-no-files-found: error
freebsd:
name: Stack - FreeBSD from CirrusCI
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: ./.github/actions/artifact-from-cirrus
with:
token: ${{ github.token }}
task: Build FreeBSD (Stack)
download: bin
upload: postgrest-freebsd-x86-64
cabal:
strategy:
matrix:
ghc: ['9.10.3', '9.12.3']
ghc: ['9.6.7', '9.8.4']
fail-fast: false
name: Cabal - Linux x86-64 - GHC ${{ matrix.ghc }}
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: haskell-actions/setup@6037f33647c3f17758a2356c80fc4a53d7e0685d # v2.12.0
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: haskell-actions/setup@de26526e12bc780fb9d384c1fb61c0bf02e3a40d # v2.10.4
with:
ghc-version: ${{ matrix.ghc }}
- name: Cache .cabal
+2 -2
View File
@@ -20,7 +20,7 @@ jobs:
name: Lint & Style
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -36,7 +36,7 @@ jobs:
name: Commit
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 100 # fetch history (last 100 commits) instead of default shallow clone history, this is deemed enough for a PR history
- name: Setup Nix Environment
+4 -3
View File
@@ -41,15 +41,16 @@ jobs:
concurrency:
# Never tag outdated commits on the main branch by skipping superseded commits
group: ci-tag-${{ (github.ref == 'refs/heads/main' && github.ref) || github.run_id }}
cancel-in-progress: true
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
if: vars.RELEASE_ENABLED
runs-on: ubuntu-slim
runs-on: ubuntu-24.04
needs:
- docs
- test
- build
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ssh-key: ${{ secrets.POSTGREST_SSH_KEY }}
- name: Tag latest commit
+2 -3
View File
@@ -14,7 +14,6 @@ on:
- .github/actions/setup-nix/**
- default.nix
- nix/**
- flake.lock
- docs/**
- '!**.md'
@@ -28,7 +27,7 @@ jobs:
name: Build
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -42,7 +41,7 @@ jobs:
name: Spellcheck
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
+2 -27
View File
@@ -7,37 +7,12 @@ on:
jobs:
linkcheck:
name: Linkcheck
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.linkcheck.bin
- name: Run Linkcheck
id: linkcheck
run: postgrest-docs-linkcheck
# This actions creates the github token using the postgrest app secrets
- name: Create Github App Token (Runs only on linkcheck failure)
id: app-token
if: ${{ failure() && steps.linkcheck.outcome == 'failure' }} # only create the token on linkcheck failure
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ vars.POSTGREST_CI_APP_ID }}
private-key: ${{ secrets.POSTGREST_CI_PRIVATE_KEY }}
permission-issues: write # required for commenting on issues
- name: Notify on linkcheck failure by commenting on GH Issue 4106
if: ${{ failure() && steps.linkcheck.outcome == 'failure' }}
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
with:
token: ${{ steps.app-token.outputs.token }}
issue-number: 4106
body: |
**Linkcheck Job Failed!**
A broken link was detected in the docs. Please check the [failed run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details.
- run: postgrest-docs-linkcheck
+68 -92
View File
@@ -9,7 +9,8 @@ on:
concurrency:
# Terminate all previous runs of the same workflow for the same tag.
group: release-${{ github.ref }}
cancel-in-progress: true
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
jobs:
build:
@@ -19,15 +20,13 @@ jobs:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
github:
name: GitHub
permissions:
contents: write
runs-on: ubuntu-slim
prepare:
name: Prepare
runs-on: ubuntu-24.04
needs:
- build
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
@@ -49,7 +48,23 @@ jobs:
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Save CHANGES.md as artifact
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: release-changes
path: CHANGES.md
if-no-files-found: error
github:
name: GitHub
permissions:
contents: write
runs-on: ubuntu-24.04
needs:
- prepare
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Download all artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
@@ -60,26 +75,23 @@ jobs:
mkdir -p release-bundle
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-aarch64.tar.xz" \
-C artifacts/postgrest-linux-static-aarch64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-x86-64.tar.xz" \
-C artifacts/postgrest-linux-static-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-aarch64.tar.xz" \
-C artifacts/postgrest-macos-aarch64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-x86-64.tar.xz" \
-C artifacts/postgrest-macos-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-freebsd-x86-64.tar.xz" \
-C artifacts/postgrest-freebsd-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-ubuntu-aarch64.tar.xz" \
-C artifacts/postgrest-ubuntu-aarch64 postgrest
zip --junk-paths "release-bundle/postgrest-${GITHUB_REF_NAME}-windows-x86-64.zip" \
artifacts/postgrest-windows-x86-64/postgrest.exe
- name: Save release bundle
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: release-bundle
path: release-bundle
@@ -101,14 +113,14 @@ jobs:
gh release edit devel \
-t devel \
--verify-tag \
-F CHANGES.md \
-F artifacts/release-changes/CHANGES.md \
--prerelease
gh release upload --clobber devel release-bundle/*
else
gh release create "${GITHUB_REF_NAME}" \
-t "${GITHUB_REF_NAME}" \
--verify-tag \
-F CHANGES.md \
-F artifacts/release-changes/CHANGES.md \
release-bundle/*
fi
@@ -117,55 +129,70 @@ jobs:
name: Docker Hub
runs-on: ubuntu-24.04-arm
needs:
- github
- prepare
if: |
vars.DOCKER_REPO && vars.DOCKER_USER
env:
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Download aarch64 Docker image
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-docker-aarch64
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Download x86-64 Docker image
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-docker-x86-64
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
- name: Download aarch64 binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-ubuntu-aarch64
- uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
- name: Build aarch64 Docker image
run: |
# This only pushes the image via digest, not a tag. This will not appear
# in the image list on Docker Hub, yet. It will be later added to the main
# tag's manifest.
docker buildx build \
-t "$DOCKER_REPO/postgrest" \
--platform linux/arm64 \
--output push-by-digest=true,type=image,push=true \
--metadata-file metadata.json \
.
echo "SHA256_ARM=$(jq -r '."containerimage.digest"' metadata.json)" >> "$GITHUB_ENV"
- name: Publish images on Docker Hub
run: |
docker load -i postgrest-docker-aarch64.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-arm64"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-arm64"
docker load -i postgrest-docker.tar.gz
docker load -i postgrest-docker-x86-64.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-amd64"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-amd64"
docker manifest create "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}" \
"$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-arm64" \
"$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-amd64"
docker manifest push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker buildx imagetools create --append \
-t "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}" \
"$DOCKER_REPO/postgrest@$SHA256_ARM"
# Only tag 'latest' for full releases
if [ "${GITHUB_REF_NAME}" != "devel" ]; then
echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..."
docker manifest create "$DOCKER_REPO/postgrest:latest" \
"$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-arm64" \
"$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-amd64"
docker manifest push "$DOCKER_REPO/postgrest:latest"
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
docker push "$DOCKER_REPO"/postgrest:latest
docker buildx imagetools create --append \
-t "$DOCKER_REPO/postgrest:latest" \
"$DOCKER_REPO/postgrest@$SHA256_ARM"
else
echo "Skipping push to 'latest' tag for pre-release..."
fi
docker-description:
name: Docker Hub Description
runs-on: ubuntu-24.04
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
github.ref == 'refs/tags/devel'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: peter-evans/dockerhub-description@1b9a80c056b620d92cedb9d9b5a223409c68ddfa # v5.0.0
if: github.ref == 'refs/tags/devel'
name: Docker Hub Description
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
@@ -173,54 +200,3 @@ jobs:
short-description: ${{ github.event.repository.description }}
readme-filepath: ./docker-hub-readme.md
ghcr:
name: GitHub Container Registry
runs-on: ubuntu-24.04-arm
needs:
- github
permissions:
packages: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Download aarch64 Docker image
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-docker-aarch64
- name: Download x86-64 Docker image
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-docker-x86-64
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Publish images on Docker Hub
run: |
docker load -i postgrest-docker-aarch64.tar.gz
docker tag postgrest:latest "ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}-linux-arm64"
docker push "ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}-linux-arm64"
docker load -i postgrest-docker-x86-64.tar.gz
docker tag postgrest:latest "ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}-linux-amd64"
docker push "ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}-linux-amd64"
docker manifest create "ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}" \
"ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}-linux-arm64" \
"ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}-linux-amd64"
docker manifest push "ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}"
# Only tag 'latest' for full releases
if [ "${GITHUB_REF_NAME}" != "devel" ]; then
echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..."
docker manifest create "ghcr.io/${GITHUB_REPOSITORY,,}:latest" \
"ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}-linux-arm64" \
"ghcr.io/${GITHUB_REPOSITORY,,}:${GITHUB_REF_NAME}-linux-amd64"
docker manifest push "ghcr.io/${GITHUB_REPOSITORY,,}:latest"
else
echo "Skipping push to 'latest' tag for pre-release..."
fi
+18 -53
View File
@@ -17,7 +17,6 @@ on:
- .github/actions/setup-nix/**
- default.nix
- nix/**
- flake.lock
- .stylish-haskell.yaml
- cabal.project
- postgrest.cabal
@@ -25,10 +24,6 @@ on:
- test/**
- '!**.md'
defaults:
run:
shell: bash
concurrency:
# Terminate all previous runs of the same workflow for pull requests
group: test-${{ github.head_ref || github.run_id }}
@@ -44,7 +39,7 @@ jobs:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -53,17 +48,17 @@ jobs:
- run: postgrest-cabal-update
- name: Run coverage (IO tests and Spec tests against latest supported PostgreSQL)
- name: Run coverage (IO tests and Spec tests against PostgreSQL 15)
run: postgrest-coverage
- name: Upload coverage to codecov
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
with:
files: ./coverage/codecov.json
token: ${{ secrets.CODECOV_TOKEN }}
- name: Run doctests
if: always()
run: nix-shell --run postgrest-test-doctests
run: postgrest-test-doctests
- name: Check the spec tests for idempotence
if: always()
@@ -74,9 +69,8 @@ jobs:
strategy:
fail-fast: false
matrix:
# Latest version is tested via `coverage` above.
pgVersion: [pg-14, pg-15, pg-16, pg-17, oriole-18, pg-18]
name: ${{ matrix.pgVersion }}
pgVersion: [13, 14, 15, 16, 17]
name: PG ${{ matrix.pgVersion }}
runs-on: ubuntu-24.04
defaults:
run:
@@ -84,37 +78,37 @@ jobs:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: tests.testSpec.bin tests.testObservability.bin tests.testIO.bin tests.testBigSchema.bin withTools.${{ matrix.pgVersion }}.bin cabalTools.update.bin
tools: tests.testSpec.bin tests.testObservability.bin tests.testIO.bin tests.testBigSchema.bin withTools.pg-${{ matrix.pgVersion }}.bin cabalTools.update.bin
- run: postgrest-cabal-update
- name: Run spec tests
if: always()
run: postgrest-with-${{ matrix.pgVersion }} postgrest-test-spec
run: postgrest-with-pg-${{ matrix.pgVersion }} postgrest-test-spec
- name: Run observability tests
if: always()
run: postgrest-with-${{ matrix.pgVersion }} postgrest-test-observability
run: postgrest-with-pg-${{ matrix.pgVersion }} postgrest-test-observability
- name: Run IO tests
if: always()
run: postgrest-with-${{ matrix.pgVersion }} postgrest-test-io -vv
run: postgrest-with-pg-${{ matrix.pgVersion }} postgrest-test-io -vv
- name: Run IO tests on a big schema
if: always()
run: postgrest-with-${{ matrix.pgVersion }} postgrest-test-big-schema -vv
run: postgrest-with-pg-${{ matrix.pgVersion }} postgrest-test-big-schema -vv
memory:
name: Memory
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -129,20 +123,19 @@ jobs:
loadtest:
strategy:
fail-fast: false
matrix:
kind: ['mixed', 'jwt-cache']
kind: ['mixed', 'jwt-hs', 'jwt-hs-cache', 'jwt-hs-cache-worst', 'jwt-rsa', 'jwt-rsa-cache', 'jwt-rsa-cache-worst']
name: Loadtest
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: loadtest.loadtestAgainst.bin loadtest.report.bin loadtest.report-load.bin cabalTools.update.bin
tools: loadtest.loadtestAgainst.bin loadtest.report.bin cabalTools.update.bin
- run: postgrest-cabal-update
@@ -156,35 +149,7 @@ jobs:
latest_tag=$(git tag --merged HEAD --sort=-creatordate "v*" | head -n1)
fi
postgrest-loadtest-against -k ${{ matrix.kind }} "$TARGET_BRANCH" "$latest_tag"
- name: Report P50
# This step checks whether any red cross indicators (:x:) are present in the step summary.
# The loadtest reporter writes them when any of individual steps fails the performance
# regression threshold.
run: |
! (postgrest-loadtest-report -g ${{ matrix.kind }} -p 50 \
| tee "$GITHUB_STEP_SUMMARY" \
| grep ':x:')
- name: Report P0
if: always()
run: |
postgrest-loadtest-report -g ${{ matrix.kind }} -p 0 >> "$GITHUB_STEP_SUMMARY"
- name: Report P90
if: always()
run: |
postgrest-loadtest-report -g ${{ matrix.kind }} -p 90 >> "$GITHUB_STEP_SUMMARY"
- name: Report P95
if: always()
run: |
postgrest-loadtest-report -g ${{ matrix.kind }} -p 95 >> "$GITHUB_STEP_SUMMARY"
- name: Report CPU/MEM
if: always()
run: |
postgrest-loadtest-report-load -g ${{ matrix.kind }} >> "$GITHUB_STEP_SUMMARY"
postgrest-loadtest-report -g ${{ matrix.kind }} >> "$GITHUB_STEP_SUMMARY"
flake:
strategy:
@@ -197,7 +162,7 @@ jobs:
name: Flake Check
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- name: Setup Nix Environment
+1 -6
View File
@@ -25,9 +25,4 @@ loadtest
.history
.docs-build
gen_targets.http
gen_jwks.json
gen_private.json
.pytest_cache
.ruff_cache
postgrest-module-graph.png
.ghc.environment.*
gen_jwk.json
+1 -1
View File
@@ -7,4 +7,4 @@ python:
build:
os: ubuntu-24.04
tools:
python: "3.12"
python: "3.11"
+1 -1
View File
@@ -200,7 +200,7 @@ steps:
# A common setting is the number of columns (parts of) code will be wrapped
# to. Different steps take this into account. Default: 80.
columns: 80
columns: 70
# By default, line endings are converted according to the OS. You can override
# preferred format here.
+3 -156
View File
@@ -4,159 +4,6 @@ All notable changes to this project will be documented in this file. From versio
## Unreleased
## [16.1] - 2026-08-10
### Fixed
- JWT validation uses wrong current time due to a bug in auto-update by @mkleczek in #5159
## [16.0] - 2026-08-07
### Changes
#### HTTP Server
- [Graceful shutdown](https://docs.postgrest.org/en/v16/references/http_server.html#graceful-shutdown) by @mkleczek, @Vlix in #4702
- [server-reuseport](https://docs.postgrest.org/en/v16/references/configuration.html#server-reuseport) allows starting multiple PostgREST instances using the same port on supported platforms by @mkleczek in #4703, #4694
#### Performance
- Optimize schema cache domain type resolution by using [pg_basetype](https://www.postgresql.org/docs/current/functions-info.html#FUNCTIONS-INFO-CATALOG) on PostgreSQL 17+ by @joelonsql in #4567
- [Prefer: count=exact](https://docs.postgrest.org/en/v16/references/api/pagination_count.html#exact-count) no longer does a double count on requests that do not use ranges or `db-max-rows` by @laurenceisla in #3957
- [Prefer: timezone](https://docs.postgrest.org/en/v16/references/api/preferences.html#prefer-timezone) no longer requires the schema cache by @steve-chavez in #5100
+ Previously this required caching [pg_timezone_names](https://www.postgresql.org/docs/current/view-pg-timezone-names.html) which was slow in some systems
#### Integrations
- PostgREST is now tested to work with [OrioleDB](https://github.com/orioledb/orioledb/) in #4845 by @wolfgangwalther
+ See [our guide for running OrioleDB on NixOS](https://docs.postgrest.org/en/v16/integrations/nixos.html)
#### JWT
- [JWT Role Extraction](https://docs.postgrest.org/en/v16/references/auth.html#jwt-role-extract) is now more flexible, supporting the standard JSON Path defined in RFC 9535 by @taimoorzaeem in #4984
#### API
- [Prefer: timezone](https://docs.postgrest.org/en/v16/references/api/preferences.html#timezone) now supports numeric offsets like `05:00` or `-4` by @steve-chavez in #5100
- Fix unexpected results when embedding and filtering the same table more than once by @laurenceisla in #4075
+ You need to set [url-use-legacy-target-names](https://docs.postgrest.org/en/v16/references/configuration.html#url-use-legacy-target-names) to `false`.
- Deprecate filters, orders and limits with the name of an embedded table when it has an alias by @steve-chavez, @laurenceisla in #4075
+ e.g. `?select=alias:table(*)&table.id=eq.1` will not be possible anymore, use `?select=alias:table(*)&alias.id=eq.1` instead.
+ You will see a warning in the logs and a `Warning` header on the client response when this happens.
+ You can disable this behavior now by setting `url-use-legacy-target-names = false`.
- Add `Vary` header to responses by @develop7 in #4609
- Fix automatic transaction retries on `40001 (serialization_failure)` errors to prevent replication lag by @laurenceisla in #3673
#### Observability
- [GHC runtime metrics](https://docs.postgrest.org/en/v16/references/observability.html#ghc-runtime-metrics) by @mkleczek in #4862
- [client-error-verbosity](https://docs.postgrest.org/en/v16/references/configuration.html#client-error-verbosity) to customize responses error verbosity by @taimoorzaeem in #4088, #3980, #3824
- [log-level](https://docs.postgrest.org/en/v16/references/configuration.html#log-level) config is now reloadable by @taimoorzaeem in #5113
- Log error when `db-schemas` config contains schema `pg_catalog` or `information_schema` by @taimoorzaeem in #4359
- Log schema cache queries timings on `log-level=debug` by @steve-chavez in #4805
#### Admin Server
- [admin-server-unix-socket](https://docs.postgrest.org/en/v16/references/configuration.html#admin-server-unix-socket)/[admin-server-unix-socket-mode](https://docs.postgrest.org/en/v16/references/configuration.html#admin-server-unix-socket-mode) to run the admin server on a unix socket by @wolfgangwalther in #5003
- Fix responding with `Something went wrong` on Admin server when under EMFILE by @mkleczek in #5077
#### Deployment
- Make executable for aarch64-linux static instead of Ubuntu-based by @wolfgangwalther in #4193
- Docker image for aarch64-linux is now built from scratch instead of being Ubuntu-based by @wolfgangwalther in #4193
- Besides Docker Hub, docker images are now published to Github Container Registry by @wolfgangwalther in #2836
#### Schema Cache
- Fix requests failing when the schema cache fails to reload, when this happens PostgREST will continue serving requests in "best effort" by @mkleczek in #4873 #4869
- Fix reporting 503s errors unnecessarily while the schema cache is loading at startup by @mkleczek in #4880
- Fix schema cache dump missing RPC transaction isolation level by @taimoorzaeem in #5079
#### Listener
- Fix config `db-channel-enabled` not reloading by @taimoorzaeem in #4894
### Migration to v16
- Drop support for PostgreSQL EOL version 13 by @wolfgangwalther in #4193
+ PostgreSQL 13 end of life was on 2025 ([ref](https://www.postgresql.org/support/versioning/))
+ Upgrade your PostgreSQL version to at least 14 to use this new PostgREST version.
- Fail at startup when `db-schemas` contains schema `pg_catalog` or `information_schema` by @taimoorzaeem in #4359
+ Previously it failed at runtime with `PGRST205` on requests related to these schemas.
+ Remove `pg_catalog` and `information_schema` from `db-schemas`.
- `Prefer: timezone` no longer complies with `handling=lenient` and instead always fails by @steve-chavez in #5128
+ Supporting this required caching `pg_timezone_names`, which was expensive.
+ Ensure your requests always have a valid timezone.
- `jwt-role-claim-key` no longer uses the JSPath DSL and instead uses JSON Path by @taimoorzaeem in #4984
+ Now all config values must start with `$` character.
Example: `.roles.read` -> `$.roles.read`
+ Keys with special characters, with the exception of `_` char must be quoted.
Example: `.roles.write-role` -> `$.roles["write-role"]`
+ String comparison operators (`^==`, `==^` and `*==`) are replaced with regular expression search.
Example: `.roles[?(@ ^== "postgrest_test_")]` -> `$.roles[?search(@, "^postgrest_test_")]`
+ Update the `jwt-role-claim-key` value accoring to the above rules. Also see the syntax reference: [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535.html#name-jsonpath-syntax-and-semanti).
## [14.16] - 2026-07-27
### Fixed
- Fix admin server crashing without a way to recover by @taimoorzaeem in #5096
## [14.15] - 2026-07-13
### Fixed
- Fix admin server dying silently by @Vlix, @mkleczek, @steve-chavez in #5012
## [14.14] - 2026-06-29
### Fixed
- Fix admin server not logging cause of failure by @taimoorzaeem in #5012
## [14.13] - 2026-06-04
### Fixed
- Fix connection retrying message in `PGRST000` error by @netqo in #4980
+ Remove redundant "Retrying the connection." from message because it is logged separately
- Fix request failures when `work_mem` is set on a role by @laurenceisla in #4955
## [14.12] - 2026-05-20
### Fixed
- Fix race condition in pool_available metric causing negative values during network instability by @mkleczek in #4622
## [14.11] - 2026-05-04
### Fixed
- Fix login with uppercase and mixed case role names by @taimoorzaeem in #4678
- Restore Listener query shape so it can be found in `pg_stat_activity` by @mkleczek in #4857 #4859
- The LISTEN channel now automatically recovers when it stops working due to a PostgreSQL bug @laurenceisla in #3147
- Fix misleading "Functions" name on schema cache summary in startup logs by @taimoorzaeem in #4821
## [14.10] - 2026-04-16
### Added
- Log when the pool is released during schema cache reload on `log-level=debug` by @mkleczek in #4668
### Fixed
- Fix unnecessary connection pool flushes during schema cache reloading by @mkleczek in #4645
## [14.9] - 2026-04-10
### Added
@@ -201,7 +48,7 @@ All notable changes to this project will be documented in this file. From versio
- Ensure Listener connections are released by @mkleczek in #4614
- Fix incorrectly filtering the returned representation for PATCH requests when using `or/and` filters by @laurenceisla in #3707
- Fix listener running with exception masked after first failure by @mkleczek in #4615
- Fix listener running with exception masked after first failure by @mkleczek #4615
## [14.3] - 2026-01-03
@@ -805,7 +652,7 @@ All notable changes to this project will be documented in this file. From versio
### Added
- #1933, #2109, Add a minimal health check endpoint - @steve-chavez
+ For enabling this, the `admin-server-port` config must be set explicitly
+ For enabling this, the `admin-server-port` config must be set explictly
+ A `<host>:<admin_server_port>/live` endpoint is available for checking if postgrest is running on its port/socket. 200 OK = alive, 503 = dead.
+ A `<host>:<admin_server_port>/ready` endpoint is available for checking a correct internal state(the database connection plus the schema cache). 200 OK = ready, 503 = not ready.
- #1988, Add the current user to the request log on stdout - @DavidLindbom, @wolfgangwalther
@@ -1288,7 +1135,7 @@ All notable changes to this project will be documented in this file. From versio
- Customize content negotiation per route - @begriffs
- Allow using nulls order without explicit order direction - @steve-chavez
- Fatal error on postgres unsupported version, format supported version in error message - @steve-chavez
- Prevent database memory consumption by prepared statements caches - @ruslantalpa
- Prevent database memory cosumption by prepared statements caches - @ruslantalpa
- Use specific columns in the RETURNING section - @ruslantalpa
- Fix columns alias for RETURNING - @steve-chavez
+14 -23
View File
@@ -1,12 +1,17 @@
# Contributing to PostgREST
## AI Policy
**First:** if you're unsure or afraid of _anything_, just ask or
submit the issue or pull request anyways. You won't be yelled at
for giving your best effort. The worst that can happen is that
you'll be politely asked to change something. We appreciate any
sort of contributions, and don't want a wall of rules to get in the
way of that.
We adhere to [Gentoo's AI policy](https://wiki.gentoo.org/wiki/Project:Council/AI_policy):
> It is expressly forbidden to contribute [...] any content that has been created with the assistance of Natural Language Processing artificial intelligence tools. This motion can be revisited, should a case been made over such a tool that does not pose copyright, ethical and quality concerns.
You can find more about its rationale [here](https://wiki.gentoo.org/wiki/Project:Council/AI_policy#Rationale).
However, for those individuals who want a bit more guidance on the
best way to contribute to the project, read on. This document will
cover what we're looking for. By addressing all the points we're
looking for, it raises the chances we can quickly merge or address
your contributions.
## Issues
@@ -35,14 +40,12 @@ For questions on how to use PostgREST, please use
We have a fully nix-based development environment with many tools for a smooth development workflow available.
Check the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md) on how to set it up and use it.
### Haskell Conventions
* All contributions must pass the tests before being merged. When
you create a pull request your code will automatically be tested.
* All fixes or features must have a test proving the improvement.
* All features must document the new behavior. Critical fixes that introduce new behavior must be documented too.
* All code must also pass a [linter](http://community.haskell.org/~ndm/hlint/) and [styler](https://github.com/jaspervdj/stylish-haskell)
* All code must also pass [hlint](http://community.haskell.org/~ndm/hlint/) and [stylish-haskell](https://github.com/jaspervdj/stylish-haskell)
with no warnings. This helps enforce a uniform style for all committers. Continuous integration will check this as well on every
pull request. There are useful tools in the nix-shell that help with checking this locally. You can run `postgrest-check` to do this manually but
we recommend adding it to `.git/hooks/pre-commit` as `nix-shell --run postgrest-check` to automatically check this before doing a commit.
@@ -50,15 +53,3 @@ Check the [development docs](https://github.com/PostgREST/postgrest/blob/main/ni
### Running Tests
For instructions on running tests, see the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md#testing).
### Structuring commits in pull requests
To simplify reviews, make it easy to split pull requests if deemed necessary, and to maintain clean and meaningful history of changes, you will be asked to update your PR if it does not follow the below rules:
* It must be possible to merge the PR branch into target using `git merge --ff-only`, ie. the source branch must be rebased on top of target.
* No merge commits in the source branch.
* All commits in the source branch must be self contained, meaning: it should be possible to treat each commit as a separate PR.
* Commits in the source branch must contain only related changes (related means the changes target a single problem/goal). For example, any refactorings should be isolated from the actual change implementation into separate commits.
* Tests, documentation, and changelog updates should be contained in the same commits as the actual code changes they relate to. An exception to this rule is when test or documentation changes are made in separate PR.
* Commit messages must be prefixed with one of the prefixes defined in [the list used by commit verification scripts](https://github.com/PostgREST/postgrest/blob/main/nix/tools/gitTools.nix#L11).
* Commit messages should contain a longer description of the purpose of the changes contained in the commit and, for non-trivial changes, a description of the changes themselves.
+21
View File
@@ -0,0 +1,21 @@
# PostgREST Docker Hub image for aarch64.
# The x86-64 is a single-static-binary image built via Nix, see:
# nix/tools/docker/README.md
FROM ubuntu:noble@sha256:84e77dee7d1bc93fb029a45e3c6cb9d8aa4831ccfcc7103d36e876938d28895b AS postgrest
RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY postgrest /usr/bin/postgrest
RUN chmod +x /usr/bin/postgrest
EXPOSE 3000
USER 1000
# Use the array form to avoid running the command using bash, which does not handle `SIGTERM` properly.
# See https://docs.docker.com/compose/faq/#why-do-my-services-take-10-seconds-to-recreate-or-stop
CMD ["postgrest"]
+2 -1
View File
@@ -1,4 +1,5 @@
Copyright (c) 2014-2026 The PostgREST contributors
Copyright (c) 2014 Joe Nelson
Copyright (c) 2019 Steve Chavez
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
+1 -1
View File
@@ -125,7 +125,7 @@ and limited with - range headers. More about
## Data Integrity
Rather than relying on an Object Relational Mapper and custom
imperative coding, this system requires you to put declarative constraints
imperative coding, this system requires you put declarative constraints
directly into your database. Hence no application can corrupt your
data (including your API server).
-5
View File
@@ -1,7 +1,2 @@
packages: postgrest.cabal
tests: true
allow-newer:
hasql:postgresql-libpq
-- https://github.com/martijnbastiaan/doctest-parallel/blob/main/example/README.md#cabalproject
write-ghc-environment-files: always
+1 -1
View File
@@ -1 +1 @@
index-state: hackage.haskell.org 2026-08-10T16:58:32Z
index-state: hackage.haskell.org 2025-10-29T04:02:18Z
+4 -28
View File
@@ -1,6 +1,6 @@
{ system ? builtins.currentSystem
, compiler ? "ghc9123"
, compiler ? "ghc948"
, # Commit of the Nixpkgs repository that we want to use.
# It defaults to reading the inputs from flake.lock, which serves
@@ -44,6 +44,7 @@ let
allOverlays.checked-shell-script
allOverlays.gitignore
(allOverlays.haskell-packages { inherit compiler; })
allOverlays.slocat
];
# Evaluated expression of the Nixpkgs repository.
@@ -52,20 +53,11 @@ let
postgresqlVersions =
[
{ name = "pg-19"; postgresql = pkgs.postgresql_19.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-18"; postgresql = pkgs.postgresql_18.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-17"; postgresql = pkgs.postgresql_17.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-16"; postgresql = pkgs.postgresql_16.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{
name = "oriole-18";
postgresql = pkgs.orioledb.withPackages (p: [ p.postgis p.pg_safeupdate ]);
config = "
default_table_access_method = 'orioledb'
shared_preload_libraries = 'orioledb, pg_stat_statements'
";
}
{ name = "pg-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
];
haskellPackages = pkgs.haskell.packages."${compiler}";
@@ -86,16 +78,6 @@ let
"-f dev --test-show-detail=direct";
inherit (pkgs.haskell) lib;
nixos-lib = import (pkgs.path + "/nixos/lib") { };
runTest = postgrest: test: (nixos-lib.runTest {
hostPkgs = pkgs;
# Replace the top-level `pkgs.postgrest` attribute with our current version on this branch.
defaults.nixpkgs.overlays = [ (_: _: { inherit postgrest; }) ];
# Speeds up evaluation a little bit; documentation is really not required for tests.
defaults.documentation.enable = pkgs.lib.mkDefault false;
imports = [ test ];
}).config.result;
in
rec {
inherit nixpkgs pkgs;
@@ -126,9 +108,6 @@ rec {
inherit (pkgs.haskell.packages."${compiler}") ghcWithPackages;
};
# Used by CI on MacOS
inherit (pkgs) nix-build-uncached;
### Tools
cabalTools =
@@ -139,7 +118,7 @@ rec {
# Development tools.
devTools =
pkgs.callPackage nix/tools/devTools.nix { inherit tests style devCabalOptions hsie; };
pkgs.callPackage nix/tools/devTools.nix { inherit tests style devCabalOptions hsie withTools; };
# Documentation tools.
docs =
@@ -181,7 +160,4 @@ rec {
# Docker images and loading script.
docker =
pkgs.callPackage nix/tools/docker { postgrest = postgrestStatic; };
# NixOS VM tests
nixpkgs-nixos-test = runTest postgrestStatic (pkgs.path + "/nixos/tests/postgrest.nix");
}
+2
View File
@@ -61,3 +61,5 @@ The image is built from scratch using
no commands are listed in the image history. See the [PostgREST
repository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for
details on the build process and how to inspect the image.
This does not apply to the arm64 variant, which is based on Ubuntu.
+2 -2
View File
@@ -87,8 +87,8 @@ url of Authorization is [[../explanations/db_authz.html]]
url of CLI is [[../references/cli.html#cli]]
url of "Connection Pool" is [[../references/connection_pool.html]]
url of Config is [[../references/configuration.html#configuration]]
url of HTTPADMIN is [[../references/http_server.html]]
url of HTTPAPI is [[../references/http_server.html]]
url of HTTPADMIN is [[../explanations/architecture.html#http]]
url of HTTPAPI is [[../explanations/architecture.html#http]]
url of Listener is [[../references/listener.html#listener]]
url of Proxy is [[../explanations/nginx.html]]
url of "Schema Cache" is [[../references/schema_cache.html#schema-cache]]
+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 28 KiB

+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 29 KiB

+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 5.6 KiB

After

Width:  |  Height:  |  Size: 4.8 KiB

+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 5.8 KiB

After

Width:  |  Height:  |  Size: 5.1 KiB

+3 -4
View File
@@ -48,14 +48,14 @@ source_suffix = ".rst"
# The master toctree document.
master_doc = "index"
# This is overridden by readthedocs with the version tag anyway
version = "devel"
# This is overriden by readthedocs with the version tag anyway
version = "14"
# To avoid repetition in <title> we set this to an empty string.
release = ""
# General information about the project.
project = "PostgREST " + version
author = "The PostgREST contributors"
author = "Joe Nelson, Steve Chavez"
copyright = "2017, " + author
# The language for content autogenerated by Sphinx. Refer to documentation
@@ -300,7 +300,6 @@ linkcheck_ignore = [
r"https://www.patreon.com/postgrest",
r"https://blog.frankel.ch/poor-man-api",
r"https://www.cybertec-postgresql.com/.*",
r"https://stackoverflow.com/.*",
# Odd SSL error
r"https://www.dripdepot.com",
r"https://www.euronodes.com",
+1 -3
View File
@@ -6,7 +6,7 @@ Community Tutorials
* `Building a Contacts List with PostgREST and Vue.js <https://www.youtube.com/watch?v=iHtsALtD5-U>`_ -
In this video series, DigitalOcean shows how to build and deploy an Nginx + PostgREST(using a managed PostgreSQL database) + Vue.js webapp in an Ubuntu server droplet.
* `PostgREST + Auth0: Create REST API in minutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest-1-introduction/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service.
* `PostgREST + Auth0: Create REST API in mintutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service.
* `"CodeLess" backend using postgres, postgrest and oauth2 authentication with keycloak <https://www.mathieupassenaud.fr/codeless_backend/>`_ -
A step-by-step tutorial for using PostgREST with KeyCloak(hosted on a managed service).
@@ -37,7 +37,6 @@ Example Apps
* `archtika <https://github.com/thiloho/archtika>`_ - self-hosted CMS
* `delibrium-postgrest <https://gitlab.com/delibrium/delibrium-postgrest/>`_ - example school API and front-end in Vue.js
* `ETH-transactions-storage <https://github.com/Adamant-im/ETH-transactions-storage>`_ - indexer for Ethereum to get transaction list by ETH address
* `fullstack template <https://github.com/jenstroeger/fullstack-webapp-template>`_ - a complete fullstack webapp template using PG as db and message queue, Python and Dramatiq to implement async jobs, db migrations, test runners, and more.
* `general <https://github.com/PierreRochard/general>`_ - example auth back-end
* `guild-operators <https://github.com/cardano-community/koios-artifacts/tree/main/files/grest>`_ - example queries and functions that the Cardano Community uses for their Guild Operators' Repository
* `PostGUI <https://github.com/priyank-purohit/PostGUI>`_ - React Material UI admin panel
@@ -83,7 +82,6 @@ Extensions
Client-Side Libraries
---------------------
* `efcore-postgrest <https://github.com/pedro-gilmora/EF.PostgREST.Provider>`_ - C#
* `postgrest-csharp <https://github.com/supabase-community/postgrest-csharp>`_ - C#
* `postgrest-dart <https://github.com/supabase/postgrest-dart>`_ - Dart
* `postgrest-ex <https://github.com/supabase-community/postgrest-ex>`_ - Elixir
+16 -11
View File
@@ -31,60 +31,65 @@ This section talks briefly about various important modules.
Main
----
The starting point of the program is `Main.hs <https://github.com/PostgREST/postgrest/blob/main/src/executable/Main.hs>`_.
The starting point of the program is `Main.hs <https://github.com/PostgREST/postgrest/blob/main/main/Main.hs>`_.
CLI
---
Main then calls `CLI.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/CLI.hs>`_, which is in charge of :ref:`cli`.
Main then calls `CLI.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/CLI.hs>`_, which is in charge of :ref:`cli`.
App
---
`App.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/App.hs>`_ is then in charge of composing the different modules.
`App.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/App.hs>`_ is then in charge of composing the different modules.
Auth
----
`Auth.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/Auth.hs>`_ is in charge of :ref:`authn`.
`Auth.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Auth.hs>`_ is in charge of :ref:`authn`.
Api Request
-----------
`ApiRequest.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/ApiRequest.hs>`_ is in charge of parsing the URL query string (following PostgREST syntax), the request headers, and the request body.
`ApiRequest.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/ApiRequest.hs>`_ is in charge of parsing the URL query string (following PostgREST syntax), the request headers, and the request body.
A request might be rejected at this level if it's invalid. For example when providing an unknown media type to PostgREST or using an unknown HTTP method.
Plan
----
Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/Plan.hs>`_ generates an internal AST, filling out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request.
Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Plan.hs>`_ generates an internal AST, filling out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request.
A request might be rejected at this level if it's invalid. For example when doing resource embedding on a nonexistent resource.
Query
-----
`Query.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/Query.hs>`_ generates the SQL queries (parametrized and prepared) required to satisfy the user request.
`Query.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Query.hs>`_ generates the SQL queries (parametrized and prepared) required to satisfy the user request.
Only at this stage a connection from the pool might be used.
Schema Cache
------------
`SchemaCache.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/SchemaCache.hs>`_ is in charge of :ref:`schema_cache`.
`SchemaCache.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/SchemaCache.hs>`_ is in charge of :ref:`schema_cache`.
Config
------
`Config.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/Config.hs>`_ is in charge of :ref:`configuration`.
`Config.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Config.hs>`_ is in charge of :ref:`configuration`.
Admin
-----
`Admin.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/Admin.hs>`_ is in charge of the :ref:`admin_server`.
`Admin.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Admin.hs>`_ is in charge of the :ref:`admin_server`.
HTTP
----
The HTTP server is provided by `Warp <https://aosabook.org/en/posa/warp.html>`_.
Listener
--------
`Reload.hs <https://github.com/PostgREST/postgrest/blob/main/src/library/PostgREST/AppState/Reload.hs>`_ is in charge of the :ref:`listener`.
`Listener.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Listener.hs>`_ is in charge of the :ref:`listener`.
+1 -1
View File
@@ -163,7 +163,7 @@ Another option is to define the function with the :code:`SECURITY DEFINER` optio
.. code-block:: postgres
-- login as a user which has privileges on the private schemas
-- login as a user wich has privileges on the private schemas
-- create a sample function
create or replace function login(email text, pass text, out token text) as $$
+1 -18
View File
@@ -16,7 +16,7 @@ Supported PostgreSQL versions
=============================
=============== =================================
**Supported** PostgreSQL >= 14
**Supported** PostgreSQL >= 13
=============== =================================
PostgREST works with all PostgreSQL versions still `officially supported <https://www.postgresql.org/support/versioning/>`_.
@@ -181,23 +181,6 @@ If you want to have a visual overview of your API in your browser you can add sw
With this you can see the swagger-ui in your browser on port 8080.
.. _docker_cpu_contraint:
Docker Resource Constraints
---------------------------
PostgREST does not support ``--cpus`` `constraint option <https://docs.docker.com/engine/containers/resource_constraints/#configure-the-default-cfs-scheduler>`_.
As a workaround, you may use the `GHC RTS <https://ghc.gitlab.haskell.org/ghc/doc/users_guide/runtime_control.html#runtime-system-rts-options>`_ ``-N`` option. For instance, to limit it to 2 CPU cores, do:
.. code::
# Set environment variable GHCRTS set to "-N2"
docker run --rm -p 3000:3000 \
-e PGRST_DB_URI="postgres://app_user:password@10.0.0.10/postgres" \
-e GHCRTS="-N2"
postgrest/postgrest
.. _build_source:
Building from Source
@@ -1,62 +0,0 @@
.. _debugging_performance_pg_stat_statements:
Debugging Performance with pg_stat_statements
=============================================
This how-to shows how to get a query identifier through PostgREST and then use it to inspect the same query in ``pg_stat_statements``.
.. important::
- :ref:`db-plan-enabled` must be enabled in PostgREST.
- PostgreSQL 14 or newer with ``pg_stat_statements`` available.
Get the Query Identifier from PostgREST
---------------------------------------
Request the plan in JSON format with the ``verbose`` option:
.. code-block:: bash
curl "http://localhost:3000/projects?select=id,name&order=id" \
-H "Accept: application/vnd.pgrst.plan+json; options=verbose"
The response will contain a top-level ``Query Identifier`` field:
.. code-block:: json
[
{
"Plan": {
"Node Type": "Aggregate"
},
"Query Identifier": -432192689578025496
}
]
Look up the query in pg_stat_statements
---------------------------------------
Use that identifier against ``pg_stat_statements``:
.. code-block:: postgres
select
calls,
total_exec_time,
mean_exec_time,
rows,
query
from pg_stat_statements
where queryid = -432192689578025496;
.. csv-table::
:header: "calls", "total_exec_time", "mean_exec_time", "rows", "query"
"13", "0.6355850000000001", "0.04889115384615385", "13", "WITH pgrst_source AS (...)"
This lets you correlate a PostgREST request with PostgreSQL runtime statistics such as:
- how often the query ran
- total and average execution time
- how many rows it produced
- the normalized SQL text recorded by PostgreSQL
@@ -318,6 +318,144 @@ You can insert a new product using a JSON object for the ``extra_info`` column:
To query and filter the data see :ref:`json_columns` for a complete reference.
.. _ww_postgis:
PostGIS
-------
You can use the string representation for `PostGIS <https://postgis.net/>`_ data types such as ``geometry`` or ``geography`` (you need to `install PostGIS <https://postgis.net/documentation/getting_started/>`_ first).
.. code-block:: postgres
-- Activate the postgis module in the current database
create extension if not exists postgis;
create table coverage (
id int primary key,
name text unique,
area geometry
);
To add areas in polygon format, you can use string representation:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 1, "name": "small", "area": "SRID=4326;POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" },
{ "id": 2, "name": "big", "area": "SRID=4326;POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))" }
]
EOF
Now, when you request the information, PostgREST will automatically cast the ``area`` column into a ``Polygon`` geometry type. Although this is useful, you may need the whole output to be in `GeoJSON <https://geojson.org/>`_ format out of the box, which can be done by including the ``Accept: application/geo+json`` in the request. This will work for PostGIS versions 3.0.0 and up and will return the output as a `FeatureCollection Object <https://www.rfc-editor.org/rfc/rfc7946#section-3.3>`_:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-H "Accept: application/geo+json"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
If you need to add an extra property, like the area in square units by using ``st_area(area)``, you could add a generated column to the table and it will appear in the ``properties`` key of each ``Feature``.
.. code-block:: postgres
alter table coverage
add square_units double precision generated always as ( st_area(area) ) stored;
In the case that you are using older PostGIS versions, then creating a function is your best option:
.. code-block:: postgres
create or replace function coverage_geo_collection() returns json as $$
select
json_build_object(
'type', 'FeatureCollection',
'features', json_agg(
json_build_object(
'type', 'Feature',
'geometry', st_AsGeoJSON(c.area)::json,
'properties', json_build_object('id', c.id, 'name', c.name)
)
)
)
from coverage c;
$$ language sql;
Now this query will return the same results:
.. code-block:: bash
curl "http://localhost:3000/rpc/coverage_geo_collection"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Ranges
------
@@ -471,20 +609,3 @@ You can use other comparative filters and also all the `PostgreSQL special date/
"due_date": "2022-02-27T06:00:00-05:00"
}
]
.. raw:: html
<script type="text/javascript">
let hash = window.location.hash;
const redirects = {
// PostGIS
'#postgis': '../integrations/postgis.html#postgis',
};
let willRedirectTo = redirects[hash];
if (willRedirectTo) {
window.location.href = willRedirectTo;
}
</script>
+3 -6
View File
@@ -119,14 +119,11 @@ Releases
PostgREST follows ``MAJOR.PATCH`` two-part versioning:
- ``MAJOR``: feature release, may deprecate or remove things.
- ``PATCH``: fix/security release only, no features and no behavior changes.
- ``PATCH``: fix/security release only; no features, no behavior changes.
MAJOR releases are published twice a year, with their scope and target dates tracked through `GitHub milestones <https://github.com/PostgREST/postgrest/milestones>`_.
PATCH releases are published on an as-needed basis.
Starting from ``v14.0``, only even-numbered MAJOR versions will be released, reserving odd-numbered MAJOR versions for development.
Starting from ``v14.0``, only even-numbered MAJOR versions are released, reserving odd-numbered MAJOR versions for development.
All releases are published on `PostgREST's GitHub release page <https://github.com/PostgREST/postgrest/releases>`_, along with the corresponding upgrade guides.
All the releases are published on `PostgREST's GitHub release page <https://github.com/PostgREST/postgrest/releases>`_.
Tutorials
---------
-36
View File
@@ -1,36 +0,0 @@
NixOS
=====
Nixpkgs contains a `NixOS module to run PostgREST <https://search.nixos.org/options?channel=unstable&query=services.postgrest&type=options>`_, which can be enabled with ``services.postgrest.enable = true``.
A PostgreSQL server can be enabled on the same machine with ``services.postgresql.enable = true``. Connections will use the name of the system user as user and database names by default, in this case ``postgrest``.
A minimal example could look like this:
.. code-block:: nix
{
pkgs,
...
}:
{
services.postgresql = {
enable = true;
initialScript = pkgs.writeText "init.sql" ''
CREATE ROLE postgrest LOGIN NOINHERIT;
CREATE ROLE anon ROLE postgrest;
'';
};
services.postgrest = {
enable = true;
settings.db-anon-role = "anon";
settings.db-uri.dbname = "postgres";
};
}
This will expose the PostgREST server on localhost on the NixOS machine and allow anonymous access.
.. tip::
NixOS also allows to quickly spin up different PostgreSQL versions or even forks this way. For example, to test the current beta version of `OrioleDB <https://www.orioledb.com>`_, use ``services.postgresql.package = pkgs.orioledb``.
-154
View File
@@ -1,154 +0,0 @@
.. _ww_postgis:
PostGIS
=======
To work with `PostGIS <https://postgis.net/>`_ data types such as ``geometry`` or ``geography``, you'll need to `install PostGIS <https://postgis.net/documentation/getting_started/>`_ first.
.. code-block:: postgres
-- Activate the postgis module in the current database
create extension if not exists postgis;
create table coverage (
id int primary key,
name text unique,
area geometry
);
insert into coverage (id, name, area) values
(1, 'small', ST_GeomFromText('POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))',4326)),
(2, 'big', ST_GeomFromText('POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))', 4326);
.. _application/geo+json:
``application/geo+json``
------------------------
PostgREST supports the `standard <https://www.iana.org/assignments/media-types/application/geo+json>`_ ``application/geo+json`` media type which can be used to get the output in `GeoJSON <https://geojson.org/>`_ format. This will work for PostGIS versions 3.0.0 and up and will return the output as a `FeatureCollection Object <https://www.rfc-editor.org/rfc/rfc7946#section-3.3>`_:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-H "Accept: application/geo+json"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Using generated columns
-----------------------
If you need to add an extra property, like the area in square units by using ``st_area(area)``, you could add a generated column to the table and it will appear in the ``properties`` key of each ``Feature``.
.. code-block:: postgres
alter table coverage
add square_units double precision generated always as ( st_area(area) ) stored;
In the case that you are using older PostGIS versions, then creating a function is your best option:
.. code-block:: postgres
create or replace function coverage_geo_collection() returns json as $$
select
json_build_object(
'type', 'FeatureCollection',
'features', json_agg(
json_build_object(
'type', 'Feature',
'geometry', st_AsGeoJSON(c.area)::json,
'properties', json_build_object('id', c.id, 'name', c.name)
)
)
)
from coverage c;
$$ language sql;
Now this query will return the same results:
.. code-block:: bash
curl "http://localhost:3000/rpc/coverage_geo_collection"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Using string representation
---------------------------
To insert areas in polygon format, you can use string representation:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 3, "name": "strip", "area": "SRID=4326;POLYGON((0 0, 50 0, 50 2, 0 2, 0 0))" },
{ "id": 4, "name": "diamond", "area": "SRID=4326;POLYGON((5 0, 10 5, 5 10, 0 5, 5 0))" }
]
EOF
PostgREST will automatically cast the ``area`` column into a ``Polygon`` geometry type.
+2 -12
View File
@@ -1,11 +1,9 @@
personal_ws-1.1 en 0 utf-8
api
autoscaling
API's
APIs
APISIX
AST
async
aud
Auth
auth
@@ -16,7 +14,6 @@ BOM
Bytea
Cardano
cd
CDNs
centric
CLI
CMS
@@ -31,11 +28,10 @@ CSV
durations
DDL
DOM
DSL
DevOps
Dramatiq
dockerize
enum
ECS
Enums
Entra
eq
@@ -45,10 +41,7 @@ EveryLayout
filename
FreeBSD
fts
fullstack
GC
GeoJSON
GHC
Github
Google
grantor
@@ -77,6 +70,7 @@ isdistinct
JS
js
JSON
JSPath
JWK
JWT
jwt
@@ -99,7 +93,6 @@ namespaced
Nanos
neq
nginx
NixOS
nixpkgs
npm
nxl
@@ -145,13 +138,11 @@ Redux
refactor
reloadable
Reloadable
reuseport
requester's
RESTful
RLS
RPC
RSA
RTS
safeupdate
savepoint
schemas
@@ -197,7 +188,6 @@ verifier
versioning
Vondra
Vue
webapp
webhooks
websearch
Websockets
+2 -3
View File
@@ -3,9 +3,7 @@
Admin Server
############
PostgREST provides an admin server that can be enabled by setting :ref:`admin-server-port` or :ref:`admin-server-unix-socket`.
Multiple PostgREST instances can share the same public API host and port when :ref:`server-reuseport` is enabled. Admin ports are not shared: give each instance a different :ref:`admin-server-port`, otherwise the new instance will fail to start.
PostgREST provides an admin server that can be enabled by setting :ref:`admin-server-port`.
.. _health_check:
@@ -74,4 +72,5 @@ Provides the ``schema_cache`` endpoint that prints the runtime :ref:`schema_cach
"dbRepresentations": ["..."],
"dbRoutines": ["..."],
"dbTables": ["..."],
"dbTimezones": ["..."]
}
-1
View File
@@ -21,7 +21,6 @@ PostgREST exposes three database objects of a schema as resources: tables, views
api/aggregate_functions.rst
api/openapi.rst
api/preferences.rst
api/vary_header.rst
api/*
.. raw:: html
-20
View File
@@ -69,26 +69,6 @@ If the function doesn't modify the database, it will also run under the GET meth
The function parameter names match the JSON object keys in the POST case, for the GET case they match the query parameters ``?a=1&b=2``.
If the function is defined to have default values for the parameters then arguments for these parameters can be omitted in the request. For instance:
.. code-block:: postgres
CREATE FUNCTION greet_user(username TEXT DEFAULT 'guest')
RETURNS TEXT AS $$
SELECT 'Hello ' || username || '!';
$$ LANGUAGE SQL IMMUTABLE;
.. code-block:: bash
curl -i "http://localhost:3000/rpc/greet_user"
.. code-block:: http
HTTP/1.1 200 OK
Context-Type: application/json; charset=utf-8
"Hello guest!"
.. _function_single_json:
Functions with an array of JSON objects
+1 -1
View File
@@ -15,7 +15,7 @@ Using these domains, :ref:`functions <functions>` can become handlers and `user-
.. important::
- PostgREST vendor media types (``application/vnd.pgrst.plan``, ``application/vnd.pgrst.object`` and ``application/vnd.pgrst.array``) cannot be overridden.
- PostgREST vendor media types (``application/vnd.pgrst.plan``, ``application/vnd.pgrst.object`` and ``application/vnd.pgrst.array``) cannot be overriden.
- Long media types like ``application/vnd.openxmlformats-officedocument.wordprocessingml.document`` cannot be expressed as domains since they surpass `PostgreSQL identifier length <https://www.postgresql.org/docs/current/limits.html#LIMITS-TABLE>`_.
For these you can use the :ref:`any_handler`.
+24 -29
View File
@@ -62,12 +62,8 @@ The server ignores unrecognized or unfulfillable preferences by default. You can
Timezone
========
.. important::
The ``timezone`` preference allows you to change the `PostgreSQL timezone <https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-TIMEZONE>`_. It accepts all time zones in `pg_timezone_names <https://www.postgresql.org/docs/current/view-pg-timezone-names.html>`_.
``handling=lenient`` is ignored for ``timezone``. Invalid time zones always return an error.
The ``timezone`` preference allows you to change the `PostgreSQL timezone <https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-TIMEZONE>`_.
It accepts all time zones in `pg_timezone_names <https://www.postgresql.org/docs/current/view-pg-timezone-names.html>`_ and numeric offsets.
.. code-block:: bash
@@ -88,36 +84,35 @@ It accepts all time zones in `pg_timezone_names <https://www.postgresql.org/docs
{"t":"2023-10-18T09:37:59.611-07:00"}
]
Offsets are also accepted:
.. code-block:: bash
curl -i "http://localhost:3000/timestamps" \
-H "Prefer: timezone=05:30"
.. code-block:: http
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Preference-Applied: timezone=05:30
.. code-block:: json
[
{"t":"2023-10-18T17:07:59.611+05:30"},
{"t":"2023-10-18T19:07:59.611+05:30"},
{"t":"2023-10-18T21:07:59.611+05:30"}
]
You can also use negative offsets like ``-03:00``.
For an invalid time zone, PostgREST returns a database error.
For an invalid time zone, PostgREST returns values with the default time zone (configured on ``postgresql.conf`` or as a setting on the :ref:`authenticator <roles>`).
.. code-block:: bash
curl -i "http://localhost:3000/timestamps" \
-H "Prefer: timezone=Jupiter/Red_Spot"
.. code-block:: http
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
.. code-block:: json
[
{"t":"2023-10-18T12:37:59.611+00:00"},
{"t":"2023-10-18T14:37:59.611+00:00"},
{"t":"2023-10-18T16:37:59.611+00:00"}
]
Note that there's no ``Preference-Applied`` in the response.
However, with ``handling=strict``, an invalid time zone preference will throw an :ref:`error <pgrst122>`.
.. code-block:: bash
curl -i "http://localhost:3000/timestamps" \
-H "Prefer: handling=strict, timezone=Jupiter/Red_Spot"
.. code-block:: http
HTTP/1.1 400 Bad Request
+1 -1
View File
@@ -1244,7 +1244,7 @@ You can order the correlated arrays explicitly. For example, to order by the fil
.. warning::
Aliasing spread columns is recommended since JSON allows duplicate keys. Example:
Aliasing spreaded columns is recommended since JSON allows duplicate keys. Example:
.. code-block:: bash
@@ -51,7 +51,7 @@ Builtin handlers are offered for common standard media types.
* ``text/csv`` and ``application/json``, for all API endpoints. See :ref:`tables_views` and :ref:`functions`.
* ``application/openapi+json``, for the root endpoint. See :ref:`open-api`.
* ``application/geo+json``, see :ref:`application/geo+json`.
* ``application/geo+json``, see :ref:`ww_postgis`.
* ``*/*``, resolves to ``application/json`` for API endpoints and to ``application/openapi+json`` for the root endpoint.
The following vendor media types handlers are also supported.
-4
View File
@@ -5,10 +5,6 @@ Schemas
PostgREST can expose a single or multiple schema's tables, views and functions. The :ref:`active database role <roles>` must have the usage privilege on the schemas to access them.
.. important::
``pg_catalog`` and ``information_schema`` are not allowed in :ref:`db-schemas`. This is done to prevent leaking sensitive information and hence they cannot be accessed directly. If you wish to expose objects of these schemas, expose another schema that contains wrapper views or functions over ``pg_catalog`` or ``information_schema`` objects.
Single schema
-------------
+1 -1
View File
@@ -639,7 +639,7 @@ However, it can work with surrogate primary keys (e.g. ``id serial primary key``
.. code-block:: bash
curl "http://localhost:3000/employees?columns=id,name,salary" \
curl "http://localhost:3000/employees?colums=id,name,salary" \
-X POST -H "Content-Type: application/json" \
-H "Prefer: resolution=merge-duplicates, missing=default" \
-d @- << EOF
+1 -1
View File
@@ -51,7 +51,7 @@ You can request table/columns with spaces in them by percent encoding the spaces
Reserved characters
~~~~~~~~~~~~~~~~~~~
If filters include PostgREST reserved characters(``,``, ``.``, ``:``, ``*``, ``(``, ``)``) you'll have to surround them in percent encoded double quotes ``%22`` for correct processing.
If filters include PostgREST reserved characters(``,``, ``.``, ``:``, ``()``) you'll have to surround them in percent encoded double quotes ``%22`` for correct processing.
Here ``Hebdon,John`` and ``Williams,Mary`` are values.
-16
View File
@@ -1,16 +0,0 @@
.. _vary_header:
Vary Header
===========
In order to assist caching proxies and CDNs, PostgREST includes a ``Vary`` header of value
``Accept, Prefer, Range`` in its responses which should fit most of the bills. As any other
response header, it's available for override
by updating ``response.headers`` GUC variable accordingly, for example:
.. code-block:: postgres
-- Override the Vary header to include Accept, Prefer and X-Test-Vary headers
perform set_config('response.headers', '[{"Vary": "Accept, Prefer, X-Test-Vary"}]', true);
In this case PostgREST will use provided value verbatim.
+21 -12
View File
@@ -217,38 +217,47 @@ It's recommended to leave the JWT cache enabled as our load tests indicate ~20%
- If the ``jwt-secret`` is changed and the config is reloaded, the JWT cache will reset.
- JWTs that pass :ref:`jwt_signature` are cached, regardless if they pass :ref:`jwt_claims_validation`. We do this to ensure responses stays fast under common failure cases (such as expired JWTs).
- You can use the :ref:`server-timing_header` to see the performance benefit of JWT caching.
- You can use the :ref:`server-timing_header` to see the peformance benefit of JWT caching.
.. _jwt_role_extract:
JWT Role Extraction
-------------------
A JSON Path (`RFC 9535 <https://www.rfc-editor.org/rfc/rfc9535.html>`_) can be specified for the location of the :code:`role` key in the JWT claims. It's configured by :ref:`jwt-role-claim-key`. This can be used to consume a JWT provided by a third party service like Auth0, Okta, Microsoft Entra or Keycloak.
A JSPath DSL that specifies the location of the :code:`role` key in the JWT claims. It's configured by :ref:`jwt-role-claim-key`. This can be used to consume a JWT provided by a third party service like Auth0, Okta, Microsoft Entra or Keycloak.
You can quickly try out JSON Path by visiting https://serdejsonpath.live.
The DSL follows the `JSONPath <https://goessner.net/articles/JsonPath/>`_ expression grammar with extended string comparison operators. Supported operators are:
- ``==`` selects the first array element that exactly matches the right operand
- ``!=`` selects the first array element that does not match the right operand
- ``^==`` selects the first array element that starts with the right operand
- ``==^`` selects the first array element that ends with the right operand
- ``*==`` selects the first array element that contains the right operand
Usage examples:
.. code:: bash
# {"postgrest":{"roles": ["other", "author"]}}
jwt-role-claim-key = "$$.postgrest.roles[1]"
# the DSL accepts characters that are alphanumerical or one of "_$@" as keys
jwt-role-claim-key = ".postgrest.roles[1]"
# {"https://www.example.com/role": { "key": "author" }}
# non-alphanumerical characters can go inside single quotes
jwt-role-claim-key = "$$['https://www.example.com/role'].key"
# non-alphanumerical characters can go inside quotes(escaped in the config value)
jwt-role-claim-key = ".\"https://www.example.com/role\".key"
# {"postgrest":{"roles": ["other", "author"]}}
# filter based on equality or regular expression
jwt-role-claim-key = "$$.postgrest.roles[?(@ == 'author')]"
jwt-role-claim-key = "$$.postgrest.roles[?search(@, '^au')]"
# `@` represents the current element in the array
# all the these match the string "author"
jwt-role-claim-key = ".postgrest.roles[?(@ == \"author\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ != \"other\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ ^== \"aut\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ ==^ \"hor\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ *== \"utho\")]"
.. note::
- If JSON Path query returns multiple values, the first one gets selected.
- Only when using the :ref:`file_config`, all ``$`` characters in the value must be escaped with an additional ``$`` char. For :ref:`env_variables_config` and :ref:`in_db_config`, only use a single ``$`` char.
- In our implementation, only the `search()` function from `JSON Path Functions <https://www.rfc-editor.org/rfc/rfc9535.html#name-function-extensions>`_ is available for filtering.
The string comparison operators are implemented as a custom extension to the JSPath and does not strictly follow the `RFC 9535 <https://www.rfc-editor.org/rfc/rfc9535.html>`_.
JWT Security
------------
+5 -154
View File
@@ -176,46 +176,6 @@ admin-server-port
Specifies the port for the :ref:`admin_server`. Cannot be equal to :ref:`server-port`.
.. _admin-server-unix-socket:
admin-server-unix-socket
------------------------
=============== =================================
**Type** String
**Default** `n/a`
**Reloadable** N
**Environment** PGRST_ADMIN_SERVER_UNIX_SOCKET
**In-Database** `n/a`
=============== =================================
`Unix domain socket <https://en.wikipedia.org/wiki/Unix_domain_socket>`_ where to bind the :ref:`admin_server`.
If specified, this takes precedence over :ref:`admin-server-port`. Example:
.. code:: bash
admin-server-unix-socket = "/tmp/pgrst-admin.sock"
.. _admin-server-unix-socket-mode:
admin-server-unix-socket-mode
-----------------------------
=============== ===================================
**Type** String
**Default** 660
**Reloadable** N
**Environment** PGRST_ADMIN_SERVER_UNIX_SOCKET_MODE
**In-Database** `n/a`
=============== ===================================
`Unix file mode <https://en.wikipedia.org/wiki/File_system_permissions>`_ to be set for the socket specified in :ref:`admin-server-unix-socket`
Needs to be a valid octal between 600 and 777.
.. code:: bash
admin-server-unix-socket-mode = "660"
.. _app.settings.*:
app.settings.*
@@ -235,33 +195,6 @@ app.settings.*
The :code:`current_setting` function has `an optional boolean second <https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-SET>`_ argument to avoid it from raising an error if the value was not defined. Default values to :code:`app.settings` can then be given by combining this argument with :code:`coalesce` and :code:`nullif` : :code:`coalesce(nullif(current_setting('app.settings.my_custom_variable', true), ''), 'default value')`. The use of :code:`nullif` is necessary because if set in a transaction, the setting is sometimes not "rolled back" to :code:`null`. See also :ref:`this section <guc_req_headers_cookies_claims>` for more information on this behaviour.
.. _client-error-verbosity:
client-error-verbosity
----------------------
=============== =======================
**Type** String
**Default** verbose
**Reloadable** Y
**Environment** PGRST_CLIENT_ERROR_VERBOSITY
**In-Database** pgrst.client_error_verbosity
=============== =======================
Specifies the verbosity of PostgREST errors. See :ref:`client_error_verbosity`.
.. code:: bash
# Return error "code", "message", "details" and "hint"
client-error-verbosity = "verbose"
# Return only "code" and "message"
client-error-verbosity = "minimal"
.. note::
This setting only affects client side error messages. Server side logs are not affected by this setting.
.. _db-aggregates-enabled:
db-aggregates-enabled
@@ -331,7 +264,7 @@ db-channel-enabled
When this is set to :code:`true`, the notification channel specified in :ref:`db-channel` is enabled.
You should set this to ``false`` when using PostgreSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
You should set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
.. _db-config:
@@ -546,7 +479,7 @@ db-prepared-statements
When disabled, the generated queries will be parameterized (invulnerable to SQL injection) but they will not be prepared (cached in the database session). Not using prepared statements will noticeably decrease performance, so it's recommended to always have this setting enabled.
You should only set this to ``false`` when using PostgreSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
You should only set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
.. _db-root-spec:
@@ -593,7 +526,7 @@ db-tx-end
**In-Database** pgrst.db_tx_end
=============== =================================
Specifies how to terminate the database transactions. See :ref:`prefer_tx`.
Specifies how to terminate the database transactions.
.. code:: bash
@@ -679,7 +612,7 @@ jwt-role-claim-key
=============== =================================
**Type** String
**Default** $.role
**Default** .role
**Reloadable** Y
**Environment** PGRST_JWT_ROLE_CLAIM_KEY
**In-Database** pgrst.jwt_role_claim_key
@@ -689,10 +622,6 @@ jwt-role-claim-key
See :ref:`jwt_role_extract` on how to specify key paths and usage examples.
.. warning::
Only when using :ref:`file_config`, the ``$`` char needs to be escaped, so use ``$$`` and PostgREST will interpret it as a single ``$`` character.
.. _jwt-secret:
jwt-secret
@@ -752,7 +681,7 @@ log-level
=============== =================================
**Type** String
**Default** error
**Reloadable** Y
**Reloadable** N
**Environment** PGRST_LOG_LEVEL
**In-Database** `n/a`
=============== =================================
@@ -928,50 +857,6 @@ server-port
The TCP port to bind the web server. Use ``0`` to automatically assign a port.
.. _server-reuseport:
server-reuseport
----------------
=============== =================================
**Type** Bool
**Default** false
**Reloadable** N
**Environment** PGRST_SERVER_REUSEPORT
**In-Database** `n/a`
=============== =================================
Enables ``SO_REUSEPORT`` on the TCP server socket. This allows multiple
PostgREST processes to bind to the same :ref:`server-host` and
:ref:`server-port` when the operating system supports it.
For example, two PostgREST processes can use the same configuration:
.. code:: ini
server-host = "127.0.0.1"
server-port = 3000
server-reuseport = true
New connections are then distributed by the operating system between the
running PostgREST processes. This can be used to start a replacement process
before stopping the old one, or to run several PostgREST processes behind one
port.
If ``server-reuseport`` is disabled, starting another PostgREST process on
the same host and port will fail with the usual address-in-use error.
Enabling this setting on an operating system that does not support
``SO_REUSEPORT`` is a configuration error. PostgREST will fail to start
instead of falling back to a normal TCP socket.
When running multiple PostgREST instances on the same :ref:`server-port`, use
a different ``admin-server-port`` for each instance. Admin ports are not shared
between instances, so readiness checks always target one specific PostgREST
instance.
This setting does not apply when :ref:`server-unix-socket` is used.
.. _server-trace-header:
server-trace-header
@@ -1042,37 +927,3 @@ server-unix-socket-mode
.. code:: bash
server-unix-socket-mode = "660"
.. _url-use-legacy-target-names:
url-use-legacy-target-names
---------------------------
=============== =================================
**Type** Boolean
**Default** True
**Reloadable** Y
**Environment** PGRST_URL_USE_LEGACY_TARGET_NAMES
**In-Database** pgrst.url_use_legacy_target_names
=============== =================================
When active, it allows using the the name of an embedded table in filters, orders or limits even if it has an alias:
.. code:: bash
curl "http://localhost:3000/table?select=alias:target(*)&target.order=id" -i
.. code:: text
Warning: 299 PostgRESTv16 "Embedded resource was referenced by relation name even though it has an alias. This is deprecated and will stop working in a future release. Update `target` to `alias` in query string filters, orders or limits."
[...]
Note that the response includes a deprecation message in the ``Warning`` header.
This will also show in the PostgREST logs:
.. code::
28/May/2026:20:33:22 -0500: WARNING: Embedded resource was referenced by relation name even though it has an alias. This is deprecated and will stop working in a future release.
28/May/2026:20:33:22 -0500: Update filters, orders or limits that use `target` to `alias` in `GET /table?select=alias:target(*)&target.order=id`
This feature will be removed in a future release, so you should start using the ``alias`` in these cases.
-2
View File
@@ -47,8 +47,6 @@ Under a busy system, the :ref:`db-pool-max-idletime` won't be reached and the co
To avoid this problem and save resources, a connection max lifetime (:ref:`db-pool-max-lifetime`) is enforced.
After the max lifetime is reached, connections from the pool will be released and new ones will be created. This doesn't affect running requests, only unused connections will be released.
.. _pool_timeout:
Acquisition Timeout
-------------------
+1 -36
View File
@@ -199,7 +199,7 @@ Related to the HTTP request elements.
| | | :ref:`switching schemas <multiple-schemas>` is not present |
| PGRST106 | | in the :ref:`db-schemas` configuration variable. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst107: | 406 | The ``Accept`` media type sent in the request is invalid. |
| .. _pgrst107: | 415 | The ``Content-Type`` sent in the request is invalid. |
| | | |
| PGRST107 | | |
+---------------+-------------+-------------------------------------------------------------+
@@ -473,38 +473,3 @@ For example, doing a request on a table with high count (say 30_000_000), we get
Proxy-Status: PostgREST; error=57014
The PostgreSQL error code ``57014`` (`ref <https://www.postgresql.org/docs/current/errcodes-appendix.html>`_) reveals that the error is due to a short ``statement_timeout`` value.
.. _client_error_verbosity:
Client Error Verbosity
======================
For HTTP clients, the error verbosity can be set via :ref:`client-error-verbosity` config.
With ``verbose``, it returns ``code``, ``message``, ``details`` and ``hint``.
.. code:: bash
curl "localhost:3000/itemsxx"
.. code-block:: json
{
"code": "PGRST205",
"message": "Could not find the table 'public.itemsxx' in the schema cache",
"details": "Perhaps you meant the table 'public.items'",
"hint": null
}
With ``minimal``, just ``code`` and ``message`` is returned.
.. code:: bash
curl "localhost:3000/itemsxx"
.. code-block:: json
{
"code": "PGRST205",
"message": "Could not find the table 'public.itemsxx' in the schema cache"
}
-18
View File
@@ -1,18 +0,0 @@
.. _http_server:
HTTP Server
###########
The HTTP server is provided by `Warp <https://aosabook.org/en/posa/warp.html>`_.
Graceful shutdown
-----------------
PostgREST uses Warp's graceful shutdown, when a ``SIGTERM`` is received:
- It stops accepting new requests.
- Allows requests that are already in progress to finish.
- Closes idle ``Keep-Alive`` connections instead of waiting for them to expire.
- Responses sent during shutdown indicate that the connection should not be reused (e.g. for HTTP/1.x, it sends ``Connection: close``).
This allows PostgREST to shut down promptly without interrupting in-flight requests. Useful for zero-downtime upgrades and autoscaling/load-balancing under cloud environments (AWS ECS, Kubernetes).
+1 -3
View File
@@ -46,9 +46,7 @@ This will cause the :ref:`connection_pool` to connect to the read replica host a
.. note::
- Under the hood, PostgREST forces `target_session_attrs=read-write <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS>`_ for the ``LISTEN`` session.
So if you specify ``target_session_attrs=read-only`` as mentioned above, PostgREST will override it for the ``LISTEN``.
- ``read-only`` is only available on libpq >= 14, if you use a lower version you will get an error like ``invalid target_session_attrs value: \"read-only\"``.
Under the hood, PostgREST forces `target_session_attrs=read-write <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS>`_ for the ``LISTEN`` session.
.. _listener_automatic_recovery:
+1 -42
View File
@@ -39,7 +39,7 @@ For diagnostic information about the server itself, PostgREST logs to ``stderr``
06/May/2024:08:16:11 -0500: Listening for database notifications on the "pgrst" channel
06/May/2024:08:16:11 -0500: Config reloaded
06/May/2024:08:16:11 -0500: Schema cache queried in 3.8 milliseconds
06/May/2024:08:16:11 -0500: Schema cache loaded 15 Relations, 8 Relationships, 8 RPCs, 0 Domain Representations, 4 Media Type Handlers
06/May/2024:08:16:11 -0500: Schema cache loaded 15 Relations, 8 Relationships, 8 Functions, 0 Domain Representations, 4 Media Type Handlers
06/May/2024:14:11:27 -0500: Received a config reload message on the "pgrst" channel
06/May/2024:14:11:27 -0500: Config reloaded
@@ -238,45 +238,6 @@ pgrst_jwt_cache_evictions_total
The total number of JWT cache evictions.
GHC Runtime Metrics
-------------------
PostgREST can also expose GHC runtime system metrics. These use the ``ghc_*``
prefix and include
`GHC RTS statistics <https://ghc.gitlab.haskell.org/ghc/doc/libraries/base-4.22.0.0-inplace/GHC-Stats.html#g:1>`_
for runtime allocation, garbage collection, memory, and CPU/elapsed time.
These are useful for monitoring PostgREST process health and diagnosing memory
pressure or GC behavior.
To expose these metrics, enable GHC RTS statistics when starting PostgREST:
.. code-block:: bash
postgrest +RTS -T -RTS
When enabled, the admin ``/metrics`` endpoint includes samples such as:
.. code-block:: text
# HELP ghc_gcs_total Total number of GCs
# TYPE ghc_gcs_total counter
ghc_gcs_total 1
# HELP ghc_allocated_bytes_total Total bytes allocated
# TYPE ghc_allocated_bytes_total counter
ghc_allocated_bytes_total 12345678
Other available GHC runtime metrics include:
- ``ghc_gcs_total``
- ``ghc_major_gcs_total``
- ``ghc_allocated_bytes_total``
- ``ghc_max_live_bytes``
- ``ghc_max_mem_in_use_bytes``
- ``ghc_mutator_cpu_seconds_total``
- ``ghc_gc_cpu_seconds_total``
- ``ghc_elapsed_seconds_total``
Traces
======
@@ -430,8 +391,6 @@ By default the plan is assumed to generate the JSON representation of a resource
The other available parameters are ``analyze``, ``verbose``, ``settings``, ``buffers`` and ``wal``, which correspond to the `EXPLAIN command options <https://www.postgresql.org/docs/current/sql-explain.html>`_. To use the ``analyze`` and ``wal`` parameters for example, you would add them like ``Accept: application/vnd.pgrst.plan; options=analyze|wal``.
For a workflow that takes the ``Query Identifier`` from a verbose PostgREST plan and uses it to inspect the same query in ``pg_stat_statements``, see :ref:`debugging_performance_pg_stat_statements`.
Note that akin to the EXPLAIN command, the changes will be committed when using the ``analyze`` option. To avoid this, you can use the :ref:`db-tx-end` and the ``Prefer: tx=rollback`` header.
Securing the Execution Plan
+2 -8
View File
@@ -3,16 +3,10 @@
Schema Cache
============
PostgREST requires metadata from the database to provide a REST API that abstracts SQL details. One example of this is the interface for :ref:`resource_embedding`.
PostgREST requires metadata from the database schema to provide a REST API that abstracts SQL details. One example of this is the interface for :ref:`resource_embedding`.
Getting this metadata requires expensive queries. To avoid repeating this work, PostgREST uses a schema cache.
.. note::
- Schema cache queries have been optimized over time to stay fast, even on complex databases. You can see a summary of their execution time in :ref:`pgrst_logging` and :ref:`metrics`.
- If the schema cache queries are slow, the most likely cause is *system catalog bloat*, see `issue#3212 <https://github.com/PostgREST/postgrest/issues/3212>`_ for more details.
- You can turn the :ref:`log-level` to ``debug`` to see the time of each schema cache query.
.. _schema_reloading:
Schema Cache Reloading
@@ -24,7 +18,7 @@ You can do this with UNIX signals or with PostgreSQL notifications. It's also po
.. note::
- If the schema cache fails to reload (e.g. due to a ``statement_timeout`` or :ref:`pool timeout <pool_timeout>`), PostgREST will continue serving requests in a "best effort" basis.
- Requests will wait until the schema cache reload is done. This to prevent client errors due to an stale schema cache.
- If you are using the :ref:`in_db_config`, a schema cache reload will :ref:`reload the configuration<config_reloading>` as well.
.. _schema_reloading_signals:
+1 -1
View File
@@ -221,7 +221,7 @@ Notice that the ``response.headers`` should be set to an *array* of single-key o
.. note::
PostgREST provided headers such as ``Content-Type``, ``Location``, etc. can be overridden this way. Note that irrespective of overridden ``Content-Type`` response header, the content will still be converted to JSON, unless you use :ref:`custom_media`.
PostgREST provided headers such as ``Content-Type``, ``Location``, etc. can be overriden this way. Note that irrespective of overridden ``Content-Type`` response header, the content will still be converted to JSON, unless you use :ref:`custom_media`.
.. _guc_resp_status:
+4 -4
View File
@@ -1,7 +1,7 @@
# This file is auto-generated by postgrest-nixpkgs-upgrade
sphinx==9.1.0
sphinx==8.2.3
sphinx-copybutton==0.5.2
sphinx-rtd-dark-mode==1.3.0
sphinx-rtd-theme==3.1.0
sphinx-tabs==3.5.0
sphinxext-opengraph==0.13.0
sphinx-rtd-theme==3.0.2
sphinx-tabs==3.4.7
sphinxext-opengraph==0.9.1
+1 -1
View File
@@ -22,7 +22,7 @@ Step 1. Install PostgreSQL
If you're already familiar with using PostgreSQL and have it installed on your system you can use the existing installation (see :ref:`pg-dependency` for minimum requirements). For this tutorial we'll describe how to use the database in Docker because database configuration is otherwise too complicated for a simple tutorial.
If Docker is not installed, you can get it `here <https://www.docker.com/get-started>`_. Make sure that Docker service is `started <https://docs.docker.com/engine/daemon/start/#start-the-daemon-using-operating-system-utilities>`_. Next, let's pull and start the database image:
If Docker is not installed, you can get it `here <https://www.docker.com/get-started>`_. Next, let's pull and start the database image:
.. code-block:: bash
+1 -1
View File
@@ -172,7 +172,7 @@ Go back to :ref:`tut1_step3` and change the payload to
.. code-block:: bash
payload=$(echo -n "{\"role\":\"todo_user\",\"exp\":123456789}" | _base64)
payload=$(echo -n "{\"role\":\"todo_user\",\"exp\":\"123456789\"}" | _base64)
echo -n "$header.$payload.$signature"
Generated
+4 -4
View File
@@ -2,16 +2,16 @@
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1784115452,
"narHash": "sha256-BoYPdqk6jlKXy+DyUzyGV/CtRGfAhk2MmIgBhsemTGI=",
"lastModified": 1752006229,
"narHash": "sha256-BeuAPwNM2RBc5bvUTb0j4GRs2yBkDeRCw/8Y3v9Xesc=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "35d3407a3816f3b341d8cf1d60abaf2b7b8166ac",
"rev": "c80edd02003fe3d8af527215a3ac069be9cfd47f",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixpkgs-unstable",
"ref": "nixpkgs-25.05-darwin",
"repo": "nixpkgs",
"type": "github"
}
+1 -5
View File
@@ -2,7 +2,7 @@
description = "REST API for any Postgres database";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-25.05-darwin";
};
nixConfig = {
@@ -46,9 +46,5 @@
meta.description = "REST API for any Postgres database";
};
});
devShells = genSystems (postgrest: {
default = import ./shell.nix { inherit postgrest; };
});
};
}
+63 -8
View File
@@ -70,16 +70,55 @@ The PostgREST utilities available in `nix-shell` all have names that begin with
`<tab>`) in `nix-shell` to see all that are available:
```bash
# Note: The utilities listed here might not be up to date.
[nix-shell]$ postgrest-<tab>
postgrest-build
postgrest-cabal-update
postgrest-check
postgrest-clean
postgrest-commitlint
postgrest-build postgrest-parallel-curl
postgrest-check postgrest-profiled-run
postgrest-clean postgrest-push-cachix
postgrest-commitlint postgrest-release
postgrest-coverage postgrest-repl
postgrest-coverage-draft-overlay postgrest-run
postgrest-docs-build postgrest-style
postgrest-docs-check postgrest-style-check
postgrest-docs-dictcheck postgrest-test-big-schema
postgrest-docs-linkcheck postgrest-test-doctests
postgrest-docs-render postgrest-test-io
postgrest-docs-serve postgrest-test-memory
postgrest-docs-spellcheck postgrest-test-replica
postgrest-dump-minimal-imports postgrest-test-spec
postgrest-dump-schema postgrest-test-spec-idempotence
postgrest-gen-ctags postgrest-watch
postgrest-gen-jwt postgrest-with-all
postgrest-gen-secret postgrest-with-git
postgrest-git-hooks postgrest-with-pgrst
postgrest-hsie-graph-modules postgrest-with-pg-13
postgrest-hsie-graph-symbols postgrest-with-pg-14
postgrest-hsie-minimal-imports postgrest-with-pg-15
postgrest-lint postgrest-with-pg-16
postgrest-loadtest postgrest-with-pg-17
postgrest-loadtest-against postgrest-with-slow-pg
postgrest-loadtest-report postgrest-with-slow-postgrest
postgrest-nixpkgs-upgrade
...
[nix-shell]$
```
Most of these commands provide a `--help` output, make sure to check it out.
The `docker` module has large dependencies to be build before the shell becomes
available, which could take an especially long time if the cachix binary cache
is not used. You can activate it by passing a flag to `nix-shell` with
`nix-shell --arg docker true`. This will make the respective utilities available:
```bash
$ nix-shell --arg docker true
[nix-shell]$ postgrest-docker-<tab>
postgrest-docker-load
...
```
Note that `postgrest-docker-load` is now also available.
To run one-off commands, you can also use `nix-shell --run <command>`, which
will launch the Nix shell, run that one command and exit. Note that the tab
@@ -95,6 +134,16 @@ $ nix-shell --run "postgrest-foo --bar"
```
A third option is to install utilities that you use very often locally:
```bash
$ nix-env -f default.nix -iA devTools
# `postgrest-style` can now be run directly:
$ postgrest-style
```
If you use `nix-shell` very often, you might like to use
https://github.com/xzfc/cached-nix-shell, which skips evaluating all our Nix
expressions if nothing changed, reducing startup time for the shell
@@ -125,7 +174,7 @@ $ nix-shell --run "postgrest-with-all postgrest-test-spec"
# Run the tests against a specific version of PostgreSQL (use tab-completion in
# nix-shell to see all available versions):
$ nix-shell --run "postgrest-with-pg-17 postgrest-test-spec"
$ nix-shell --run "postgrest-with-pg-13 postgrest-test-spec"
```
@@ -160,7 +209,13 @@ The loadtests ensure that performance doesn't drop on a change. Underlyingly the
[nix-shell]$ postgrest-loadtest
# You can loadtest comparing to a different branch
[nix-shell]$ postgrest-loadtest-against main
[nix-shell]$ postgrest-loadtest-against master
# You can simulate latency client/postgrest and postgrest/database
[nix-shell]$ PGRST_DELAY=5ms PGDELAY=5ms postgrest-loadtest
# You can build postgrest directly with cabal for faster iteration
[nix-shell]$ PGRST_BUILD_CABAL=1 postgrest-loadtest
# Produce a markdown report to be used on CI
[nix-shell]$ postgrest-loadtest-report
+24 -6
View File
@@ -16,8 +16,11 @@ The following checklist guides you through the complete process in more detail.
## Upgrade the pinned version of `nixpkgs`
The pinned version of [`nixpkgs`](https://github.com/NixOS/nixpkgs) is defined
in [`flake.nix`](../flake.nix). To upgrade it, you can use a small utility
script defined in [`nix/tools/nixpkgsTools.nix`](tools/nixpkgsTools.nix):
in [`nix/nixpkgs-version.nix`](nixpkgs-version.nix). The pin refers directly to
a GitHub tarball for the given revision, which is more efficient than pulling
the complete Git repository. To upgrade it to the current `main` of
`nixpkgs`, you can use a small utility script defined in
[`nix/nixpkgs-update.nix`](nixpkgs-update.nix):
```bash
# From the root of the repository, enter nix-shell
@@ -27,12 +30,21 @@ nix-shell
postgrest-nixpkgs-upgrade
# Exit the nix-shell with Ctrl-d
```
## Review overlays
Check whether the individual [overlays](overlays) are still required.
## Check if patches are still required and update them as needed
We track a number of PostgREST-specific patches in [`nix/patches`](patches).
Check whether the pull-requests/issues linked in the
[`default.nix`](patches/default.nix) have progressed and remove/modify the
patches if they did. If conflicting changes occurred, you might have to rebase
the respective patches.
## Build everything
Using the PostgREST binary Nix cache is recommended. Install
@@ -46,19 +58,25 @@ errors, this is probably due to one of our patches. Try to fix them and re-run
## Update the PostgREST binary cache
If you have access to the PostgREST cachix project, you can push the
If you have access to the PostgREST cachix signing key, you can push the
artifacts that you built locally to the binary cache. This will accelerate the
CI builds and tests, sometimes dramatically. This might sometimes even be
required to avoid build timeouts in CI.
You'll need to login with your token with `cachix authtoken <token>`.
You'll need to set the `CACHIX_SIGNING_KEY` before proceeding, e.g. by creating
a file containing `export CACHIX_SIGNING_KEY=...` and sourcing that file, which
avoids having the secret in your shell history.
To push all new artifacts to Cachix, run:
```
nix-store -qR --include-outputs $$(nix-instantiate) | cachix push postgrest
# Or, equivalently
nix-shell --run postgrest-push-cachix
```
The `postgrest-push-cachix` command will query the nix-store to list all
dependencies and build artifacts of PostgREST. It will then push
The `nix-store` command will query the nix-store to list all dependencies and
build artifacts of PostgREST. The `cachix` command will efficiently push
everything that is not yet cached to the binary cache.
+7 -19
View File
@@ -4,7 +4,6 @@
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE TupleSections #-}
{-# LANGUAGE TypeApplications #-}
{-# LANGUAGE TypeFamilies #-}
-- | Haskell Imports and Exports tool
@@ -34,15 +33,13 @@ import Data.Function ((&))
import Data.List (intercalate)
import Data.Maybe (catMaybes, mapMaybe)
import Data.Text (Text)
import GHC.Driver.Errors.Types (GhcMessage)
import GHC.Generics (Generic)
import GHC.Hs.Extension (GhcPs)
import GHC.Types.Error (Messages, defaultDiagnosticOpts, getMessages)
import GHC.Types.Error (getMessages)
import GHC.Types.Name.Occurrence (occNameString)
import GHC.Types.Name.Reader (rdrNameOcc)
import GHC.Unit.Module (moduleNameString)
import GHC.Unit.Module.Name (moduleNameString)
import GHC.Utils.Error (pprMsgEnvelopeBagWithLoc)
import GHC.Utils.Outputable (showSDocUnsafe)
import System.Directory.Recursive (getFilesRecursive)
import System.Exit (exitFailure)
@@ -201,7 +198,7 @@ sourceSymbols source = do
return $ concatMap (importSymbols source filepath . GHC.unLoc) hsmodImports
-- | Parse a Haskell module
parseModule :: FilePath -> IO (GHC.HsModule GhcPs)
parseModule :: FilePath -> IO GHC.HsModule
parseModule filepath = do
result <- ExactPrint.parseModule GHC.Paths.libdir filepath
case result of
@@ -209,13 +206,7 @@ parseModule filepath = do
return $ GHC.unLoc hsmod
Left errs ->
fail $ "Errors with " <> show filepath <> ":\n "
<> formatParseErrors errs
formatParseErrors :: Messages GhcMessage -> String
formatParseErrors errs =
intercalate "\n "
. fmap showSDocUnsafe
$ pprMsgEnvelopeBagWithLoc (defaultDiagnosticOpts @GhcMessage) (getMessages errs)
<> show (pprMsgEnvelopeBagWithLoc $ getMessages errs)
-- | Symbols imported in an import declaration.
--
@@ -223,12 +214,9 @@ formatParseErrors errs =
-- only one item is returned.
importSymbols :: FilePath -> FilePath -> GHC.ImportDecl GhcPs -> [ImportedSymbol]
importSymbols source filepath GHC.ImportDecl{..} =
case ideclImportList of
Just (importListInterpretation, syms) ->
symbol (if importListInterpretation == GHC.EverythingBut then Hiding else Explicit)
. Just
. GHC.unLoc
<$> GHC.unLoc syms
case ideclHiding of
Just (hiding, syms) ->
symbol (if hiding then Hiding else Explicit) . Just . GHC.unLoc <$> GHC.unLoc syms
Nothing ->
[ symbol Wildcard Nothing ]
where
+7 -7
View File
@@ -5,10 +5,10 @@ project. It's available in PostgREST's `nix-shell` by default.
## Dumping imports
Given source code in the directories `src/library` and `src/executable`, for example, you can run:
Given source code in the directories `src` and `main`, for example, you can run:
```
hsie dump-imports src/library src/executable
hsie dump-imports src main
```
This dumps all imports of the modules in the given directory to a CSV file,
@@ -18,7 +18,7 @@ To dump to a JSON file (e.g., to further process with `jq`), add the `--json`
flag:
```
hsie dump-imports --json src/library src/executable
hsie dump-imports --json src main
```
## Graphing imports
@@ -27,7 +27,7 @@ The tool can generate `graphviz` graphs of module and symbol imports by printing
a file to `stdout` that can directly be rendered with `dot`:
```
hsie graph-modules src/library src/executable | dot -Tpng -o modules.png
hsie graph-modules src main | dot -Tpng -o modules.png
```
The command `graph-modules` prints a graph of which modules insert which other
@@ -39,7 +39,7 @@ To check whether modules are imported under consistent aliases in your project,
run:
```
hsie check-aliases src/library src/executable
hsie check-aliases main src
```
This will exit with a non-zero exit code if any inconsistent aliases are found.
@@ -48,13 +48,13 @@ The following command checks whether any modules are imported as wildcards, i.e.
not qualified and without specifying symbols.
```
hsie check-wildcards src/library src/executable
hsie check-wildcards main src
```
To whitelist certain modules to be imported as wildcards, use `--ok`:
```
hsie check-wildcards src/library src/executable --ok Protolude --ok Test.Module
hsie check-wildcards main src --ok Protolude --ok Test.Module
```
## Current limitations
@@ -104,7 +104,7 @@ let
''
+ lib.optionalString withTmpDir ''
tmpdir="$(${coreutils}/bin/mktemp -d --tmpdir=/tmp ${name}-XXX)"
tmpdir="$(${coreutils}/bin/mktemp -d --tmpdir ${name}-XXX)"
# we keep the tmpdir when an error occurs for debugging
trap 'echo Temporary directory kept at: $tmpdir' ERR
+1
View File
@@ -3,4 +3,5 @@
checked-shell-script = import ./checked-shell-script;
gitignore = import ./gitignore.nix;
haskell-packages = import ./haskell-packages.nix;
slocat = import ./slocat.nix;
}
+19 -46
View File
@@ -47,64 +47,37 @@ let
# - To modify and try packages locally, see "Working with locally modified Haskell packages" in the Nix README.
# Before upgrading fuzzyset to 0.3, check: https://github.com/PostgREST/postgrest/issues/3329
# jailbreak, because hspec limit for tests
fuzzyset = prev.fuzzyset_0_2_4;
# TODO: Remove once available in nixpkgs
auto-update =
# TODO: Remove once available in nixpkgs haskellPackages
configurator-pg =
prev.callHackageDirect
{
pkg = "auto-update";
ver = "0.2.7";
sha256 = "sha256-fHX/OqF/cB9rbpGpLUtA29bcEJS43HUWHcK55yUxKoo=";
pkg = "configurator-pg";
ver = "0.2.11";
sha256 = "sha256-mtGtNawDJgz2ZIEVca+IYXVu4oNw9xsfJiYWAqAbbgc=";
}
{ };
# TODO: Remove once available in nixpkgs
aeson-jsonpath =
# TODO: Remove once available in nixpkgs haskellPackages
streaming-commons =
prev.callHackageDirect
{
pkg = "aeson-jsonpath";
ver = "0.4.2.0";
sha256 = "sha256-K+3brf1zjSSjojtSCXFrip5rrP7AO/S4zndAxAnvEfc=";
pkg = "streaming-commons";
ver = "0.2.3.1";
sha256 = "sha256-Gl2eaJcWe1sxmcE/octWlH9uSnERguf+5H66K4fV87s=";
}
{ };
http2 =
prev.callHackageDirect
{
pkg = "http2";
ver = "5.4.0";
sha256 = "sha256-PeEWVd61bQ8G7LvfLeXklzXqNJFaAjE2ecRMWJZESPE=";
}
{ };
http-semantics =
prev.callHackageDirect
{
pkg = "http-semantics";
ver = "0.4.0";
sha256 = "sha256-rh0z51EKvsu5rQd5n2z3fSRjjEObouNZSBPO9NFYOF0=";
}
{ };
network-run =
prev.callHackageDirect
{
pkg = "network-run";
ver = "0.5.0";
sha256 = "sha256-vbXh+CzxDsGApjqHxCYf/ijpZtUCApFbkcF5gyN0THU=";
}
{ };
warp =
lib.dontCheck
(prev.callHackageDirect
{
pkg = "warp";
ver = "3.4.14";
sha256 = "sha256-RnoOUlC6dOP0sK/tYAJCX1oLzVFG1GILUY+yVbmvW8Y=";
}
{ });
# Downgrade hasql and related packages while we are still on GHC 9.4 for the static build.
hasql = lib.dontCheck (lib.doJailbreak prev.hasql_1_6_4_4);
hasql-dynamic-statements = lib.dontCheck prev.hasql-dynamic-statements_0_3_1_5;
hasql-implicits = lib.dontCheck prev.hasql-implicits_0_1_1_3;
hasql-notifications = lib.dontCheck prev.hasql-notifications_0_2_2_2;
hasql-pool = lib.dontCheck prev.hasql-pool_1_0_1;
hasql-transaction = lib.dontCheck prev.hasql-transaction_1_1_0_1;
postgresql-binary = lib.dontCheck (lib.doJailbreak prev.postgresql-binary_0_13_1_3);
};
in
{
+13
View File
@@ -0,0 +1,13 @@
_: prev:
{
slocat = prev.buildGoModule {
name = "slocat";
src = prev.fetchFromGitHub {
owner = "robx";
repo = "slocat";
rev = "52e7512c6029fd00483e41ccce260a3b4b9b3b64";
sha256 = "sha256-qn6luuh5wqREu3s8RfuMCP5PKdS2WdwPrujRYTpfzQ8=";
};
vendorHash = null;
};
}
+2 -10
View File
@@ -51,7 +51,7 @@ let
docs = "Run PostgREST after building it interactively with cabal-install";
args =
[
"ARG_USE_ENV([PGRST_DB_ANON_ROLE], [], [PostgREST anonymous role. (default: 'postgrest_test_anonymous')])"
"ARG_USE_ENV([PGRST_DB_ANON_ROLE], [postgrest_test_anonymous], [PostgREST anonymous role])"
"ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])"
"ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool timeout])"
"ARG_USE_ENV([PGRST_JWT_SECRET], [reallyreallyreallyreallyverysafe], [PostgREST JWT secret])"
@@ -62,10 +62,6 @@ let
withEnv = postgrest.env;
}
''
# when there's a default, argbash conflates empty string with unset, so we do this workaround to be able to do `PGRST_DB_ANON_ROLE="" <command>` for manual testing
if [[ ! ''${PGRST_DB_ANON_ROLE+x} ]]; then
PGRST_DB_ANON_ROLE="postgrest_test_anonymous"
fi
export PGRST_DB_ANON_ROLE
export PGRST_DB_POOL
export PGRST_DB_POOL_ACQUISITION_TIMEOUT
@@ -84,7 +80,7 @@ let
docs = "Run a profiled build of postgREST. This will generate a postgrest.prof file that can be used to do optimization.";
args =
[
"ARG_USE_ENV([PGRST_DB_ANON_ROLE], [], [PostgREST anonymous role. (default: 'postgrest_test_anonymous')])"
"ARG_USE_ENV([PGRST_DB_ANON_ROLE], [postgrest_test_anonymous], [PostgREST anonymous role])"
"ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])"
"ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool timeout])"
"ARG_USE_ENV([PGRST_JWT_SECRET], [reallyreallyreallyreallyverysafe], [PostgREST JWT secret])"
@@ -94,10 +90,6 @@ let
withEnv = postgrest.env;
}
''
# when there's a default, argbash conflates empty string with unset, so we do this workaround to be able to do `PGRST_DB_ANON_ROLE="" <command>` for manual testing
if [[ ! ''${PGRST_DB_ANON_ROLE+x} ]]; then
PGRST_DB_ANON_ROLE="postgrest_test_anonymous"
fi
export PGRST_DB_ANON_ROLE
export PGRST_DB_POOL
export PGRST_DB_POOL_ACQUISITION_TIMEOUT
+157 -4
View File
@@ -5,13 +5,15 @@
, curl
, devCabalOptions
, entr
, fd
, git
, graphviz
, hsie
, nix
, silver-searcher
, stdenv
, style
, tests
, withTools
, haskellPackages
, ctags
, openssl
@@ -39,7 +41,7 @@ let
}
''
while true; do
(! ${fd}/bin/fd -H -E .git | ${entr}/bin/entr -dr "$_arg_command" "''${_arg_leftovers[@]}")
(! ${silver-searcher}/bin/ag -l . | ${entr}/bin/entr -dr "$_arg_command" "''${_arg_leftovers[@]}")
done
'';
@@ -89,6 +91,156 @@ let
${style}/bin/postgrest-style-check
'';
gitHooks =
let
name = "postgrest-git-hooks";
in
checkedShellScript
{
inherit name;
docs =
''
Enable or disable git pre-commit and pre-push hooks.
Basic is faster and will only run:
- pre-commit: postgrest-style
- pre-push: postgrest-lint
Full takes a lot more time and will run:
- pre-commit: postgrest-style && postgrest-lint
- pre-push: postgrest-check
Changes made by postgrest-style will be staged automatically.
Example usage:
postgrest-git-hooks disable
postgrest-git-hooks enable basic
postgrest-git-hooks enable full
The "run" operation and "--hook" argument are only used internally.
'';
args =
[
"ARG_POSITIONAL_SINGLE([operation], [Operation])"
"ARG_TYPE_GROUP_SET([OPERATION], [OPERATION], [operation], [disable,enable,run])"
"ARG_POSITIONAL_SINGLE([mode], [Mode], [basic])"
"ARG_TYPE_GROUP_SET([MODE], [MODE], [mode], [basic,full])"
"ARG_OPTIONAL_SINGLE([hook], , [Hook], [pre-commit])"
"ARG_TYPE_GROUP_SET([HOOK], [HOOK], [hook], [pre-commit,pre-push])"
];
positionalCompletion =
''
if test "$prev" == "${name}"; then
COMPREPLY=( $(compgen -W "enable disable" -- "$cur") )
elif test "$prev" == "enable" || test "$prev" == "disable"; then
COMPREPLY=( $(compgen -W "basic full" -- "$cur") )
fi
'';
workingDir = "/";
}
''
if [ run != "$_arg_operation" ]; then
# Remove all hooks first and ignore failures because the file might be missing.
# This assumes that we're only adding lines that include "postgrest-git-hooks"
# to the hook file.
sed -i -e '/postgrest-git-hooks/d' .git/hooks/pre-{commit,push} 2> /dev/null || true
if [ disable != "$_arg_operation" ]; then
# The nix-shell && + nix-shell || pattern makes sure we can run the hook
# in a pure nix-shell, where nix-shell itself is not available, too.
# The $(nix-shell --run "command -v ...") pattern ensures we only need to enable
# the hooks once and still run the latest of our hook scripts, even when we
# update them in the repo.
echo 'command -v nix-shell > /dev/null || postgrest-git-hooks --hook=pre-commit run' "$_arg_mode" \
>> .git/hooks/pre-commit
# shellcheck disable=SC2016
echo 'command -v nix-shell > /dev/null && $(nix-shell --quiet -Q --run "command -v postgrest-git-hooks") --hook=pre-commit run' "$_arg_mode" \
>> .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
echo 'command -v nix-shell > /dev/null || postgrest-git-hooks --hook=pre-push run' "$_arg_mode" \
>> .git/hooks/pre-push
# shellcheck disable=SC2016
echo 'command -v nix-shell > /dev/null && $(nix-shell --quiet -Q --run "command -v postgrest-git-hooks") --hook=pre-push run' "$_arg_mode" \
>> .git/hooks/pre-push
chmod +x .git/hooks/pre-push
fi
else
# When run from a git hook, the GIT_ environment variables conflict with our withGit helper.
# The following unsets all GIT_ variables.
unset "''${!GIT_@}"
# shellcheck disable=SC2317
function restore () {
ref="$(git stash list --format=format:%gD --grep "$1" -n1)"
# this will avoid merge conflicts when applying the stash
${git}/bin/git restore --source="$ref" .
# restore untracked files, too. could fail with no files
if [ "$(git show --numstat --format=oneline "$ref^3" | wc -l)" -gt 1 ]; then
${git}/bin/git restore --overlay --source="$ref^3" .
fi
${git}/bin/git stash drop "$ref"
}
case "$_arg_mode" in
basic)
case "$_arg_hook" in
pre-commit)
# To be able to automatically add only changes from postgrest-style to the staging area,
# we need to run postgrest-style twice. Otherwise we'd risk merge conflicts when popping
# the stash afterwards.
${style}/bin/postgrest-style
stash="postgrest-git-hooks-$RANDOM"
${git}/bin/git stash push --include-untracked --keep-index -m "$stash"
if [ "$(git stash list --grep $stash)" ]; then
# Only create the stash pop trap, if we actually created a stash.
# Otherwise stash pop will cause havoc.
trap 'restore "$stash"' EXIT
fi
${style}/bin/postgrest-style
${git}/bin/git add .
;;
pre-push)
# Create a clean working tree without any uncomitted changes.
${withTools.withGit} HEAD ${style}/bin/postgrest-lint
;;
esac
;;
full)
case "$_arg_hook" in
pre-commit)
# To be able to automatically add only changes from postgrest-style to the staging area,
# we need to run postgrest-style twice. Otherwise we'd risk merge conflicts when popping
# the stash afterwards.
${style}/bin/postgrest-style
stash="postgrest-git-hooks-$RANDOM"
${git}/bin/git stash push --include-untracked --keep-index -m "$stash"
if [ "$(git stash list --grep $stash)" ]; then
# Only create the stash pop trap, if we actually created a stash.
# Otherwise stash pop will cause havoc.
trap 'restore "$stash"' EXIT
fi
${style}/bin/postgrest-style
${git}/bin/git add .
${style}/bin/postgrest-lint
;;
pre-push)
# Create a clean working tree without any uncomitted changes.
${withTools.withGit} HEAD ${check}
;;
esac
;;
esac
fi
'';
dumpMinimalImports =
checkedShellScript
{
@@ -129,10 +281,10 @@ let
{
name = "postgrest-hsie-graph-modules";
docs = "Create a PNG graph of modules imported within the codebase.";
args = [ "ARG_OPTIONAL_SINGLE([outfile], [o], [Output filename], [postgrest-module-graph.png])" ];
args = [ "ARG_POSITIONAL_SINGLE([outfile], [Output filename])" ];
}
''
${hsie} graph-modules src/library src/executable | ${graphviz}/bin/dot -Tpng -o "$_arg_outfile"
${hsie} graph-modules main src | ${graphviz}/bin/dot -Tpng -o "$_arg_outfile"
'';
hsieGraphSymbols =
@@ -243,6 +395,7 @@ buildToolbox
inherit
check
dumpMinimalImports
gitHooks
hsieGraphModules
hsieGraphSymbols
hsieMinimalImports
+2 -6
View File
@@ -43,7 +43,7 @@ let
}
if [ "$_arg_language" == "" ]; then
# clean previous build, otherwise some errors might be suppressed
# clean previous build, otherwise some errors might be supressed
rm -rf "../.docs-build/html/default"
if [ -d languages ]; then
@@ -54,7 +54,7 @@ let
build html "../.docs-build/html/default"
else
# clean previous build, otherwise some errors might be suppressed
# clean previous build, otherwise some errors might be supressed
rm -rf "../.docs-build/html/$_arg_language"
# update and build specific locale, can be used to create new locale
@@ -122,8 +122,6 @@ let
workingDir = "/docs";
}
''
echo "Checking spelling mistakes..."
export LC_ALL=C
FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')
@@ -146,8 +144,6 @@ let
workingDir = "/docs";
}
''
echo "Detecting obsolete dictionary entries..."
export LC_ALL=C
FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')
+91 -44
View File
@@ -12,39 +12,42 @@
# from an array
import time
import argparse
import sys
import random
import jwcrypto.jwt as jwt
import jwt
import jwcrypto.jwk as jwk
from typing import Optional
from pathlib import Path
URL = "http://postgrest"
secret_key = "reallyreallyreallyreallyverysafe"
secret_key = b"reallyreallyreallyreallyverysafe"
key = jwk.JWK.generate(kty="RSA", size=4096)
private_key = jwt.algorithms.RSAAlgorithm.from_jwk(key.export_private())
public_key = key.export_public()
def generate_target(
now: int,
key: jwt.JWK,
) -> list[str]:
"""Generate a target using an HS256 or RS256 JWT"""
headers = {
def generate_jwt(now: int, exp_inc: Optional[int], is_hs: bool) -> str:
"""Generate an HS256 or RS256 JWT"""
payload = {
"sub": f"user_{random.getrandbits(32)}",
"iat": now,
}
claims = {
"role": "postgrest_test_author",
}
headers["alg"] = "RS256" if key.get("kty") == "RSA" else "HS256"
if exp_inc is not None:
payload["exp"] = now + exp_inc
token = jwt.JWT(headers, claims)
token.make_signed_token(key)
k = secret_key if is_hs else private_key
alg = "HS256" if is_hs else "RS256"
return jwt.encode(payload, k, alg)
return [
f"OPTIONS {URL}/authors_only?{headers["alg"]}",
f"Authorization: Bearer {token.serialize()}",
"", # blank line to separate requests
]
def append_targets(lines: list[str], token: str):
lines.append(f"OPTIONS {URL}/authors_only")
lines.append(f"Authorization: Bearer {token}")
lines.append("") # blank line to separate requests
def main():
@@ -52,47 +55,91 @@ def main():
description="Generate Vegeta targets with unique JWTs"
)
parser.add_argument(
"generated_path",
metavar="GENERATED_PATH",
help="Path to write the generated files",
"output",
help="Path to write the generated targets file",
)
parser.add_argument(
"--worst",
dest="worst",
action=argparse.BooleanOptionalAction,
default=False,
help="Generate worst case targets for a JWT cache",
)
parser.add_argument(
"--rsa",
dest="jwk_path",
metavar="JWK_PATH",
type=Path,
default=None,
help="Path for generating a RSA JWK file to sign tokens with",
)
args = parser.parse_args()
targets_path = args.generated_path / "gen_targets.http"
is_hs = args.jwk_path is None
hs = jwt.JWK.from_password(secret_key)
rsa = jwt.JWK.generate(kty="RSA", size=4096)
nsamples = 1000
if is_hs:
ntargets = 200000
else:
# The asymmetric targets take too long to compute so we reduce them
ntargets = 50000
jwks = jwt.JWKSet()
jwks.add(hs)
jwks.add(rsa)
jwks_path = args.generated_path / "gen_jwks.json"
# Technically, this exports the private keys, because HS does not have the concept
# of a public key. This is not a problem for tests, though, PostgREST can verify
# tokens with the private key just as well.
jwks_path.write_text(jwks.export())
print(f"Created JWKSet on {jwks_path}")
ntargets = 1000
if not is_hs:
try:
with open(args.jwk_path, "w") as jwk:
jwk.write(public_key)
print(f"Created {args.jwk_path} file containing the RSA JWK")
except IOError as e:
print(f"Error writing to {args.jwk_path}: {e}", file=sys.stderr)
sys.exit(1)
print(f"Generating {ntargets} targets...")
now = int(time.time())
start_time = time.time()
now = int(start_time)
lines = []
for i in range(ntargets):
target = generate_target(now, hs if i % 2 == 0 else rsa)
lines.extend(target)
# We want to ensure 401 Unauthorized responses don't happen during
# JWT validation, this can happen when the jwt `exp` is too short.
# At the same time, we want to ensure the `exp` is not too big,
# so expires will occur and postgREST needs to
# clean cached expired JWTs
if args.worst:
# estimated time takes to build and run postgrest itself
build_run_postgrest_time = 2
# estimated time it takes to generate the targets file
# the division numbers are tuned by hand
if is_hs: # hs generation is much faster
gen_time = ntargets // 66666
else: # asymmetric is slower so the time is higher
gen_time = ntargets // 220
with open(targets_path, "w") as f:
f.write("\n".join(lines))
# estimated exp time so some JWTs will expire
inc = build_run_postgrest_time + gen_time
for i in range(ntargets):
token = generate_jwt(now, inc + i // 1000, is_hs)
append_targets(lines, token)
else:
tokens = [generate_jwt(now, None, is_hs) for _ in range(nsamples)]
for i in range(ntargets):
token = random.choice(tokens)
append_targets(lines, token)
try:
with open(args.output, "w") as f:
f.write("\n".join(lines))
except IOError as e:
print(f"Error writing to {args.output}: {e}", file=sys.stderr)
sys.exit(1)
elapsed = time.time() - start_time
print(f"Created {ntargets} targets", end=" ")
print(f"in {args.output} ({elapsed:.2f}s)")
if __name__ == "__main__":
+98 -142
View File
@@ -1,11 +1,7 @@
{ buildToolbox
, checkedShellScript
, git
, jq
, libfaketime
, python3
, python3Packages
, runCommand
, vegeta
, withTools
, writers
@@ -22,8 +18,6 @@ let
];
}
''
echo "Starting vegeta loadtest..."
# ARG_USE_ENV only adds defaults or docs for environment variables
# We manually implement a required check here
# See also: https://github.com/matejak/argbash/issues/80
@@ -46,75 +40,86 @@ let
docs = "Run the vegeta loadtests with PostgREST.";
args = [
"ARG_OPTIONAL_SINGLE([output], [o], [Filename to dump json output to], [./loadtest/result.bin])"
"ARG_OPTIONAL_SINGLE([testdir], [t], [Directory to load tests and fixtures from], [./test/load])"
"ARG_OPTIONAL_SINGLE([kind], [k], [Kind of loadtest], [mixed])"
"ARG_TYPE_GROUP_SET([KIND], [KIND], [kind], [mixed,jwt-cache,jwt-cache-worst])"
"ARG_TYPE_GROUP_SET([KIND], [KIND], [kind], [mixed,jwt-hs,jwt-hs-cache,jwt-hs-cache-worst,jwt-rsa,jwt-rsa-cache,jwt-rsa-cache-worst])"
"ARG_OPTIONAL_SINGLE([monitor], [m], [Monitoring file], [./loadtest/result.csv])"
"ARG_LEFTOVERS([additional vegeta arguments])"
];
workingDir = "/";
}
''
# previously required settings to make this work with older branches
export PGRST_DB_ANON_ROLE="postgrest_test_anonymous"
export PGRST_DB_URI="postgresql://"
export PGRST_DB_SCHEMAS="test"
export PGRST_DB_CONFIG="false"
export PGRST_DB_POOL="1"
export PGRST_DB_SCHEMAS="test"
export PGRST_DB_TX_END="rollback-allow-override"
export PGRST_LOG_LEVEL="crit"
export PGRST_JWT_SECRET="reallyreallyreallyreallyverysafe"
# set previous PGRST_JWT_CACHE_MAX_LIFETIME configuration so that
# load test works across branches
# TODO clean once PGRST_JWT_CACHE_MAX_ENTRIES merged and released
export PGRST_JWT_CACHE_MAX_LIFETIME="86400"
mkdir -p "$(dirname "$_arg_output")"
abs_output="$(realpath "$_arg_output")"
case "$_arg_kind" in
jwt-cache)
export PGRST_JWT_SECRET="@${generatedTargets}/gen_jwks.json"
# shellcheck disable=SC2145
${withTools.withPg} -f test/load/fixtures.sql \
${withTools.withPgrst} --faketime '2000-01-01 00:00:00' -m "$_arg_monitor" \
sh -c "cd test/load && \
${runner} -targets ${generatedTargets}/gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
jwt-hs)
${genTargetsHS} "$_arg_testdir"/gen_targets.http
export PGRST_JWT_CACHE_MAX_ENTRIES="0"
export PGRST_JWT_CACHE_MAX_LIFETIME="0"
;;
# here we sleep purposefully to check how much memory does the schema cache consume in the final report
mixed)
# shellcheck disable=SC2145
${withTools.withPg} -f test/load/fixtures.sql \
${withTools.withPgrst} --timeout 2 --sleep 5 -m "$_arg_monitor" \
sh -c "cd test/load && \
${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
jwt-hs-cache)
${genTargetsHS} "$_arg_testdir"/gen_targets.http
;;
jwt-hs-cache-worst)
${genTargetsHS} --worst "$_arg_testdir"/gen_targets.http
;;
jwt-rsa)
${genTargetsHS} --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_CACHE_MAX_ENTRIES="0"
export PGRST_JWT_CACHE_MAX_LIFETIME="0"
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
;;
jwt-rsa-cache)
${genTargetsHS} --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
;;
jwt-rsa-cache-worst)
${genTargetsHS} --worst --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
;;
*)
;;
esac
${vegeta}/bin/vegeta report -type=text "$_arg_output"
if [ "$_arg_kind" != "mixed" ]; then
# fail in case 401 happened on jwt loadtests
unauthorized_count="$(${vegeta}/bin/vegeta report -type=json "$_arg_output" \
| ${jq}/bin/jq -r '.status_codes["401"] // 0')"
if [ "$unauthorized_count" -gt 0 ]; then
last_unauthorized_body="$(${vegeta}/bin/vegeta encode "$_arg_output" \
| ${jq}/bin/jq -rn '
reduce inputs as $item (null;
if $item.code == 401 then $item else . end
)
| if . == null then
empty
else
(.body | @base64d)
end
')"
echo "loadtest failed: found $unauthorized_count 401 Unauthorized responses" >&2
if [ -n "$last_unauthorized_body" ]; then
printf '%s\n' "Last 401 response body:" >&2
printf '%s\n' "$last_unauthorized_body" >&2
fi
exit 1
fi
if [ "$_arg_kind" == "mixed" ]; then
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withSlowPg} \
${withTools.withPgrst} -m "$_arg_monitor" \
${withTools.withSlowPgrst} \
sh -c "cd \"$_arg_testdir\" && \
${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
else
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
sh -c "cd \"$_arg_testdir\" && \
${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
fi
${vegeta}/bin/vegeta report -type=text "$_arg_output"
'';
loadtestAgainst =
@@ -129,9 +134,6 @@ let
Run the vegeta loadtest against every target branch and HEAD:
- once on the every <target-#> branch
- once in the current worktree
Note that the Nix tooling is always taken from the HEAD branch, while the PostgREST binary is taken from the target branch.
For a discussion on why this is set up like this, see https://github.com/PostgREST/postgrest/pull/5013#discussion_r3431508441.
'';
args = [
"ARG_POSITIONAL_INF([target], [Commit-ish reference to compare with], 1)"
@@ -146,40 +148,8 @@ let
workingDir = "/";
}
''
# Build postgrest for every target and HEAD.
# Keeps a reference to the postgrest binary and faketime lib for every branch to run later.
declare -A pgrst faketime
for tgt in "''${_arg_target[@]}" HEAD; do
# not using withTmpDir here, because we don't want to keep the directory on error
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
${git}/bin/git worktree add -f "$tmpdir" "$tgt" > /dev/null
pushd "$tmpdir" > /dev/null
build_start=$SECONDS
echo -n "${name}: Building postgrest (nix) on $tgt... "
# Using lib.getBin to also make this work with older checkouts, where .bin was not a thing, yet.
nix-build --no-out-link -E 'with import ./. {}; pkgs.lib.getBin postgrestPackage' > build.log 2>&1 || {
echo "failed, output:"
cat build.log
exit 1
}
pgrst[$tgt]="$(nix-build --no-out-link -E 'with import ./. {}; pkgs.lib.getBin postgrestPackage')/bin/postgrest"
# To avoid glibc mismatches with back-branches, we need to take libfaketime from the target branch.
faketime[$tgt]="$(nix-build --no-out-link -A pkgs.libfaketime)/lib/libfaketime.so.1"
build_end=$((SECONDS - build_start))
printf "done in %ss.\n" "$build_end"
popd > /dev/null
${git}/bin/git worktree remove -f "$tmpdir" > /dev/null
rm -rf "$tmpdir"
done
# Run loadtest for every target and HEAD.
# Running the tests is separated from building them to reduce the chances of
# other processes skewing the results between two runs.
for tgt in "''${_arg_target[@]}" HEAD; do
# run loadtest for every target
for tgt in "''${_arg_target[@]}"; do
cat << EOF
@@ -187,7 +157,12 @@ let
EOF
FAKETIME_LIB="''${faketime[$tgt]}" PGRST_CMD="''${pgrst[$tgt]}" ${loadtest} -k "$_arg_kind" -m "loadtest/$tgt.csv" --output "loadtest/$tgt.bin"
# Runs the test files from the current working tree
# to make sure both tests are run with the same files.
# Save the results in the current working tree, too,
# otherwise they'd be lost in the temporary working tree
# created by withTools.withGit.
${withTools.withGit} "$tgt" ${loadtest} -k "$_arg_kind" -m "$PWD/loadtest/$tgt.csv" --output "$PWD/loadtest/$tgt.bin" --testdir "$PWD/test/load"
cat << EOF
@@ -196,6 +171,22 @@ let
EOF
done
# run loadtest once on HEAD
cat << EOF
Running "$_arg_kind" loadtest on HEAD...
EOF
${loadtest} -k "$_arg_kind" -m "$PWD/loadtest/head.csv" --output "$PWD/loadtest/head.bin" --testdir "$PWD/test/load"
cat << EOF
Done running on HEAD.
EOF
'';
reporter =
@@ -205,14 +196,12 @@ let
docs = "Create a named json report for a single result file.";
args = [
"ARG_POSITIONAL_SINGLE([file], [Filename of result to create report for])"
"ARG_OPTIONAL_SINGLE([percentile], [p], [Percentile to report latency for], 50)"
"ARG_LEFTOVERS([additional vegeta arguments])"
];
workingDir = "/";
}
''
${vegeta}/bin/vegeta encode "$_arg_file" \
| ${jq}/bin/jq --arg percentile "$_arg_percentile" --slurp 'map(select(.url != "")) | group_by("\(.code) \(.method) \(.url)") | map({("\(.[0].code) \(.[0].method) \(.[0].url)" | sub("http://postgrest";"")): map(.latency) | sort | .[(length-1) * ($percentile | tonumber) / 100 | floor] / 10e3 }) | .[]' \
${vegeta}/bin/vegeta report -type=json "$_arg_file" \
| ${jq}/bin/jq --arg branch "$(basename "$_arg_file" .bin)" '. + {branch: $branch}'
'';
@@ -225,31 +214,12 @@ let
import sys
import pandas as pd
def evaluate_change(df):
try:
return ((df['HEAD'] / df['main'] - 1) * 100) \
.map(lambda r: "{icon} {ratio:.1f} %".format(
ratio=r,
# Hardcoded failure threshold for CI is 5% here.
icon="" if r < 5 else ":x:"
))
except KeyError:
return None
pd.read_json(sys.stdin) \
.rename(columns={'latency': sys.argv[1]}) \
.set_index(sys.argv[1]) \
.drop(['branch']) \
.set_index('param') \
.drop(['branch', 'earliest', 'end', 'latest']) \
.fillna("") \
.convert_dtypes() \
.assign(change=evaluate_change) \
.to_markdown(
sys.stdout,
floatfmt='.1f',
colglobalalign='right',
colalign=('left',)
)
.to_markdown(sys.stdout, floatfmt='.0f')
'';
@@ -260,46 +230,32 @@ let
docs = "Create a report of all loadtest reports as markdown.";
args = [
"ARG_OPTIONAL_SINGLE([group], [g], [Marker to group results])"
"ARG_OPTIONAL_SINGLE([percentile], [p], [Percentile to report latency for], 50)"
];
workingDir = "/";
}
''
echo -e "## Loadtest results $_arg_group (P$_arg_percentile)\n"
marker=''${_arg_group:+"($_arg_group)"}
find loadtest -type f -iname '*.bin' -exec ${reporter} -p "$_arg_percentile" {} \; \
| ${jq}/bin/jq '[paths(scalars) as $path | {latency: $path | join("."), (.branch): getpath($path)}]' \
| ${jq}/bin/jq --slurp 'flatten | group_by(.latency) | map(add)' \
| ${toMarkdown} "P$_arg_percentile latency [μs]"
'';
echo -e "## Loadtest results $marker\n"
report-load =
checkedShellScript
{
name = "postgrest-loadtest-report-load";
docs = "Create a report of all CPU/MEM usage as markdown.";
args = [
"ARG_OPTIONAL_SINGLE([group], [g], [Marker to group results])"
];
workingDir = "/";
}
''
echo -e "\n\n## Loadtest elapsed seconds vs CPU/MEM usage $_arg_group\n"
find loadtest -type f -iname '*.bin' -exec ${reporter} {} \; \
| ${jq}/bin/jq '[paths(scalars) as $path | {param: $path | join("."), (.branch): getpath($path)}]' \
| ${jq}/bin/jq --slurp 'flatten | group_by(.param) | map(add)' \
| ${toMarkdown}
echo -e "\n\n## Loadtest elapsed seconds vs CPU/MEM usage $marker\n"
find loadtest -type f -iname '*.csv' \
| sort -m \
| ${mergeMonitorResults}
'';
generatedTargets =
runCommand "postgrest-loadtest-targets"
genTargetsHS =
writers.writePython3 "postgrest-gen-loadtest-targets-hs"
{
nativeBuildInputs = [ (python3.withPackages (pyps: [ pyps.jwcrypto ])) ];
libraries = [ python3Packages.pyjwt python3Packages.jwcrypto ];
}
''
mkdir -p "$out"
${libfaketime}/bin/faketime '2000-01-01 00:00:00' python3 ${./generate_targets.py} "$out"
'';
(builtins.readFile ./generate_targets.py);
mergeMonitorResults =
writers.writePython3 "postgrest-merge-monitor-results"
@@ -310,5 +266,5 @@ let
in
buildToolbox {
name = "postgrest-loadtest";
tools = { inherit loadtest loadtestAgainst report report-load; };
tools = { inherit loadtest loadtestAgainst report; };
}
+1 -1
View File
@@ -17,7 +17,7 @@ let
nix flake update
echo "# This file is auto-generated by postgrest-nixpkgs-upgrade" > docs/requirements.txt
cat "$(nix-build --no-out-link -A docs.requirements)" >> docs/requirements.txt
cat "$(nix-build -A docs.requirements)" >> docs/requirements.txt
'';
in
+6 -7
View File
@@ -7,6 +7,7 @@ let
{
name = "postgrest-release";
docs = "Patch postgrest.cabal, CHANGELOG.md, commit and push all in one go.";
args = [ "ARG_OPTIONAL_BOOLEAN([major], [m], [Bump to new major version (only applies on main branch).])" ];
workingDir = "/";
}
''
@@ -19,6 +20,7 @@ let
git diff --exit-code HEAD postgrest.cabal > /dev/null
trap "" ERR
# TODO: Support C+D bumps when implementing hackage releases
bump () {
current_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
# shellcheck disable=SC2034
@@ -45,10 +47,7 @@ let
echo "Updating docs/conf.py ..."
sed -i -E "s/^(version = ).*$/\1\"$new_docs_version\"/" docs/conf.py > /dev/null
echo "Updating Haskell source file links ..."
sed -i -E "s#(github\.com/PostgREST/postgrest/blob)/main/#\1/$new_version/#g" docs/explanations/architecture.rst
git add postgrest.cabal docs/conf.py docs/explanations/architecture.rst > /dev/null
git add postgrest.cabal docs/conf.py > /dev/null
}
today_date_for_changelog="$(date '+%Y-%m-%d')"
@@ -63,19 +62,19 @@ let
git add CHANGELOG.md > /dev/null
echo "Committing ..."
git commit -m "chore: bump version to $new_version" > /dev/null
git commit -m "bump version to $new_version" > /dev/null
if [[ "$current_branch" == "main" ]]; then
bump devel
# The order of operations is important here:
# - bump devel is run and $A is updated to the new version
# - bump devel is run and $A is upated to the new version
# - the branch is created with the new A, but the commit before the devel bump
# - the devel bump is committed
git branch "v$A"
echo "Committing (devel bump)..."
git commit -m "chore: bump version to $new_version" > /dev/null
git commit -m "bump version to $new_version" > /dev/null
fi
trap "echo Remote not found. Please push manually ..." ERR
+12 -11
View File
@@ -3,13 +3,13 @@
, buildToolbox
, checkedShellScript
, deadnix
, fd
, git
, hlint
, hsie
, nixpkgs-fmt
, python3Packages
, ruff
, silver-searcher
, statix
, stylish-haskell
, writeText
@@ -21,27 +21,27 @@ let
name = "postgrest-style";
docs = "Automatically format Haskell, Nix and Python files.";
workingDir = "/";
withTmpDir = true;
}
''
# Format Nix files
${statix}/bin/statix fix
${nixpkgs-fmt}/bin/nixpkgs-fmt .
${nixpkgs-fmt}/bin/nixpkgs-fmt . > /dev/null 2> /dev/null
# Format Haskell files
${fd}/bin/fd '\.l?hs$' \
# --vimgrep fixes a bug in ag: https://github.com/ggreer/the_silver_searcher/issues/753
${silver-searcher}/bin/ag -l --vimgrep -g '\.l?hs$' . \
| xargs ${stylish-haskell}/bin/stylish-haskell -i
# Format Python files
TMPDIR="$tmpdir" ${black}/bin/black .
${black}/bin/black . 2> /dev/null
'';
# Script to check whether any uncommitted changes result from postgrest-style
# Script to check whether any uncommited changes result from postgrest-style
styleCheck =
checkedShellScript
{
name = "postgrest-style-check";
docs = "Check whether postgrest-style results in any uncommitted changes.";
docs = "Check whether postgrest-style results in any uncommited changes.";
workingDir = "/";
}
''
@@ -82,17 +82,18 @@ let
# ruff has gaps in scanning for unused code, so we use vulture
echo "Scanning python files for unused code..."
${fd}/bin/fd '\.l?py$' \
| xargs ${python3Packages.vulture}/bin/vulture --exclude docs/conf.py --min-confidence 80
${silver-searcher}/bin/ag -l --vimgrep -g '\.l?py$' . \
| xargs ${python3Packages.vulture}/bin/vulture --exclude docs/conf.py
echo "Linting python files..."
${ruff}/bin/ruff check .
echo "Checking consistency of import aliases in Haskell code..."
${hsie} check-aliases src/library src/executable
${hsie} check-aliases main src
echo "Linting Haskell files..."
${fd}/bin/fd '\.l?hs$' \
# --vimgrep fixes a bug in ag: https://github.com/ggreer/the_silver_searcher/issues/753
${silver-searcher}/bin/ag -l --vimgrep -g '\.l?hs$' . \
| xargs ${hlint}/bin/hlint --hint=${hlintConfig}
'';
+4 -4
View File
@@ -7,9 +7,9 @@
, glibcLocales ? null
, gnugrep
, hpc-codecov
, hostPlatform
, jq
, lib
, nginx
, postgrest
, python3
, runtimeShell
@@ -55,6 +55,8 @@ let
withEnv = postgrest.env;
}
''
# This makes nix-env -iA tests.doctests.bin work.
export NIX_GHC=${postgrest.env.NIX_GHC}
${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:doctests
'';
@@ -92,7 +94,6 @@ let
args = [ "ARG_LEFTOVERS([pytest arguments])" ];
workingDir = "/";
withEnv = postgrest.env;
withPath = [ nginx ];
}
''
${cabal-install}/bin/cabal v2-build ${devCabalOptions} exe:postgrest
@@ -155,11 +156,10 @@ let
redirectTixFiles = false;
withEnv = postgrest.env;
withTmpDir = true;
withPath = [ nginx ];
}
(
# required for `hpc markup` in CI; glibcLocales is not available e.g. on Darwin
lib.optionalString (stdenv.isLinux && stdenv.hostPlatform.libc == "glibc") ''
lib.optionalString (stdenv.isLinux && hostPlatform.libc == "glibc") ''
export LOCALE_ARCHIVE="${glibcLocales}/lib/locale/locale-archive"
'' +
+165 -58
View File
@@ -1,25 +1,20 @@
{ buildToolbox
, checkedShellScript
, curl
, git
, lib
, libfaketime
, postgresqlVersions
, postgrest
, python3Packages
, slocat
, writeText
, writers
}:
let
withTmpDb =
{ name, postgresql, config ? "" }:
{ name, postgresql }:
let
commandName = "postgrest-with-${name}";
postgresqlConf = writeText "postgresql.conf" ("
autovacuum = false
listen_addresses = ''
log_statement = all
shared_preload_libraries=pg_stat_statements
" + config);
in
checkedShellScript
{
@@ -30,7 +25,7 @@ let
"ARG_OPTIONAL_SINGLE([fixtures], [f], [SQL file to load fixtures from])"
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
"ARG_USE_ENV([PGUSER], [Postgrest_Test_Authenticator], [Authenticator PG role])" # user is written in mixed case to implicitly test that it is being properly quoted in schema cache queries
"ARG_USE_ENV([PGUSER], [postgrest_test_authenticator], [Authenticator PG role])"
"ARG_USE_ENV([PGDATABASE], [postgres], [PG database name])"
"ARG_USE_ENV([PGRST_DB_SCHEMAS], [test], [Schema to expose])"
"ARG_USE_ENV([PGTZ], [utc], [Timezone to use])"
@@ -78,10 +73,6 @@ let
TZ=$PGTZ initdb --no-locale --encoding=UTF8 --nosync -U postgres --auth=trust \
>> "$setuplog"
# Append our own config to the one initdb created to avoid replacing
# default values created by the latter.
cat ${postgresqlConf} >> "$tmpdir/db/postgresql.conf"
log "Starting the database cluster..."
# Instead of listening on a local port, we will listen on a unix domain socket.
@@ -90,7 +81,7 @@ let
# On MacOS, it's 104 chars
# See: https://serverfault.com/questions/641347/check-if-a-path-exceeds-maximum-for-unix-domain-socket
pg_ctl -l "$tmpdir/db.log" -w start -o "-F -c hba_file=$HBA_FILE -k $PGHOST " \
pg_ctl -l "$tmpdir/db.log" -w start -o "-F -c listen_addresses=\"\" -c hba_file=$HBA_FILE -k $PGHOST -c log_statement=\"all\" " \
>> "$setuplog"
log "Creating a minimally privileged $PGUSER connection role..."
@@ -115,8 +106,7 @@ let
log "Starting replica on $replica_host"
# We set a low max_standby_streaming_delay to make the replication conflict fail faster in tests (otherwise it waits for the default 30s)
pg_ctl -D "$replica_dir" -l "$replica_dblog" -w start -o "-F -c hba_file=$HBA_FILE -k $replica_host -c max_standby_streaming_delay=\"3s\" " \
pg_ctl -D "$replica_dir" -l "$replica_dblog" -w start -o "-F -c listen_addresses=\"\" -c hba_file=$HBA_FILE -k $replica_host -c log_statement=\"all\" " \
>> "$setuplog"
>&2 echo "${commandName}: Replica enabled. You can connect to it with: psql 'postgres:///$PGDATABASE?host=$replica_host' -U postgres"
@@ -127,7 +117,7 @@ let
export PGRST_DB_URI="postgres:///$PGDATABASE?host=$PGREPLICAHOST,$PGHOST"
fi
# shellcheck disable=SC2329
# shellcheck disable=SC2317
stop () {
log "Stopping the database cluster..."
pg_ctl stop --mode=immediate >> "$setuplog"
@@ -142,12 +132,9 @@ let
fi
if test "$_arg_fixtures"; then
load_start=$SECONDS
>&2 printf "${commandName}: Loading fixtures under the postgres role..."
log "Loading fixtures under the postgres role..."
psql -U postgres -v PGUSER="$PGUSER" -v ON_ERROR_STOP=1 -f "$_arg_fixtures" >> "$setuplog"
psql -U postgres -v ON_ERROR_STOP=1 -c "VACUUM ANALYZE;" >> "$setuplog"
load_end=$((SECONDS - load_start))
>&2 printf " done in %ss. Running command...\n" "$load_end"
log "Done. Running command..."
fi
("$_arg_command" "''${_arg_leftovers[@]}")
@@ -196,6 +183,134 @@ let
withPg = withTmpDb (builtins.head postgresqlVersions);
withSlowPg =
checkedShellScript
{
name = "postgrest-with-slow-pg";
docs = "Run the given command with simulated high latency postgresql";
args =
[
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
"ARG_USE_ENV([PGHOST], [], [PG host (socket name)])"
"ARG_USE_ENV([PGDELAY], [0ms], [extra PG latency (duration)])"
];
positionalCompletion = "_command";
workingDir = "/";
redirectTixFiles = false;
withTmpDir = true;
}
''
delay="''${PGDELAY:-0ms}"
echo "delaying data to/from postgres by $delay"
REALPGHOST="$PGHOST"
export PGHOST="$tmpdir/socket"
mkdir -p "$PGHOST"
${slocat}/bin/slocat -delay "$delay" -src "$PGHOST/.s.PGSQL.5432" -dst "$REALPGHOST/.s.PGSQL.5432" &
SLOCAT_PID=$!
# shellcheck disable=SC2317
stop_slocat() {
kill "$SLOCAT_PID" || true
wait "$SLOCAT_PID" || true
}
trap stop_slocat EXIT
sleep 1 # should wait for socket file to appear instead
("$_arg_command" "''${_arg_leftovers[@]}")
'';
withSlowPgrst =
checkedShellScript
{
name = "postgrest-with-slow-postgrest";
docs = "Run the given command with simulated high latency postgrest";
args =
[
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
"ARG_USE_ENV([PGRST_SERVER_UNIX_SOCKET], [], [PostgREST host (socket name)])"
"ARG_USE_ENV([PGRST_DELAY], [0ms], [extra PostgREST latency (duration)])"
];
positionalCompletion = "_command";
workingDir = "/";
redirectTixFiles = false;
withTmpDir = true;
}
''
delay="''${PGRST_DELAY:-0ms}"
echo "delaying data to/from PostgREST by $delay"
REAL_PGRST_SERVER_UNIX_SOCKET="$PGRST_SERVER_UNIX_SOCKET"
export PGRST_SERVER_UNIX_SOCKET="$tmpdir/postgrest.socket"
${slocat}/bin/slocat -delay "$delay" -src "$PGRST_SERVER_UNIX_SOCKET" -dst "$REAL_PGRST_SERVER_UNIX_SOCKET" &
SLOCAT_PID=$!
# shellcheck disable=SC2317
stop_slocat() {
kill "$SLOCAT_PID" || true
wait "$SLOCAT_PID" || true
}
trap stop_slocat EXIT
sleep 1 # should wait for socket file to appear instead
("$_arg_command" "''${_arg_leftovers[@]}")
'';
withGit =
let
name = "postgrest-with-git";
in
checkedShellScript
{
inherit name;
docs =
''
Create a new worktree of the postgrest repo in a temporary directory and
check out <commit>, then run <command> with arguments inside the temporary folder.
'';
args =
[
"ARG_POSITIONAL_SINGLE([commit], [Commit-ish reference to run command with])"
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
];
positionalCompletion =
''
if test "$prev" == "${name}"; then
__gitcomp_nl "$(__git_refs)"
else
_command_offset 2
fi
'';
workingDir = "/";
}
''
# not using withTmpDir here, because we don't want to keep the directory on error
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
${git}/bin/git worktree add -f "$tmpdir" "$_arg_commit" > /dev/null
cd "$tmpdir"
("$_arg_command" "''${_arg_leftovers[@]}")
${git}/bin/git worktree remove -f "$tmpdir" > /dev/null
'';
legacyConfig =
writeText "legacy.conf"
''
# Using this config file to support older postgrest versions for `postgrest-loadtest-against`
db-uri="$(PGRST_DB_URI)"
db-schema="$(PGRST_DB_SCHEMAS)"
db-anon-role="$(PGRST_DB_ANON_ROLE)"
db-pool="$(PGRST_DB_POOL)"
server-unix-socket="$(PGRST_SERVER_UNIX_SOCKET)"
log-level="$(PGRST_LOG_LEVEL)"
'';
waitForPgrstReady =
checkedShellScript
{
@@ -237,23 +352,15 @@ let
'';
withPgrst =
let
commandName = "postgrest-with-pgrst";
in
checkedShellScript
{
name = commandName;
name = "postgrest-with-pgrst";
docs = "Build and run PostgREST and run <command> with PGRST_SERVER_UNIX_SOCKET set.";
args =
[
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
"ARG_OPTIONAL_SINGLE([faketime], [f], [Fake the system time when starting PostgREST. This is useful to test expiry of JWT, for example in loadtests])"
"ARG_OPTIONAL_SINGLE([monitor], [m], [Enable CPU and memory monitoring of the PostgREST process and output to the designated file as markdown])"
"ARG_OPTIONAL_SINGLE([timeout], [t], [Maximum time to wait for PostgREST to be ready], [5])"
"ARG_OPTIONAL_SINGLE([sleep], [s], [Sleep time after PostgREST is ready, this is useful for monitoring])"
"ARG_USE_ENV([FAKETIME_LIB], [${libfaketime}/lib/libfaketime.so.1], [Faketime Library to preload])"
"ARG_USE_ENV([PGRST_CMD], [postgrest-run], [PostgREST executable to run])"
];
positionalCompletion = "_command";
workingDir = "/";
@@ -263,25 +370,30 @@ let
''
export PGRST_SERVER_UNIX_SOCKET="$tmpdir"/postgrest.socket
if [ "''${PGRST_CMD}" == "postgrest-run" ]; then
build_start=$SECONDS
echo -n "${commandName}: Building postgrest (cabal)... "
postgrest-build
build_end=$((SECONDS - build_start))
printf "done in %ss.\n" "$build_end"
fi
ver=$($PGRST_CMD --version)
echo -n "${commandName}: Starting $ver... "
if [[ -n "$_arg_faketime" ]]; then
LD_PRELOAD="$FAKETIME_LIB" FAKETIME="$_arg_faketime" "$PGRST_CMD" > "$tmpdir"/run.log 2>&1 &
rm -f result
if [ -z "''${PGRST_BUILD_CABAL:-}" ]; then
echo -n "Building postgrest (nix)... "
# Using lib.getBin to also make this work with older checkouts, where .bin was not a thing, yet.
nix-build -E 'with import ./. {}; pkgs.lib.getBin postgrestPackage' > "$tmpdir"/build.log 2>&1 || {
echo "failed, output:"
cat "$tmpdir"/build.log
exit 1
}
PGRST_CMD=$(echo ./result*/bin/postgrest)
else
$PGRST_CMD > "$tmpdir"/run.log 2>&1 &
echo -n "Building postgrest (cabal)... "
postgrest-build
PGRST_CMD=postgrest-run
fi
echo "done."
ver=$($PGRST_CMD ${legacyConfig} --version)
echo -n "Starting $ver... "
$PGRST_CMD ${legacyConfig} > "$tmpdir"/run.log 2>&1 &
pid=$!
# shellcheck disable=SC2329
# shellcheck disable=SC2317
cleanup() {
# Send INT to all postgrest processes.
# Workaround to trigger dumping postgrest.prof for postgrest-profiled-run
@@ -295,25 +407,17 @@ let
}
trap cleanup EXIT
wait_start=$SECONDS
timeout -s TERM "$_arg_timeout" ${waitForPgrstReady} || {
timeout -s TERM 5 ${waitForPgrstReady} || {
echo "timed out, output:"
cat "$tmpdir"/run.log
exit 1
}
wait_duration=$((SECONDS - wait_start))
printf "done in %ss.\n" "$wait_duration"
echo "${commandName}: You can tail the server logs with: tail -f $tmpdir/run.log"
echo "done."
if [[ -n "$_arg_monitor" ]]; then
${monitorPid} "$pid" > "$_arg_monitor" &
fi
if [[ -n "$_arg_sleep" ]]; then
sleep "$_arg_sleep"
fi
("$_arg_command" "''${_arg_leftovers[@]}")
'';
@@ -329,8 +433,11 @@ buildToolbox
name = "postgrest-with";
tools = {
inherit
withGit
withPgAll
withPgrst;
withPgrst
withSlowPg
withSlowPgrst;
} // builtins.listToAttrs (
# Create a `postgrest-with-pg-` for each PostgreSQL version
builtins.map (pg: { inherit (pg) name; value = withTmpDb pg; }) postgresqlVersions
+49 -68
View File
@@ -1,27 +1,28 @@
cabal-version: 3.0
name: postgrest
version: 17
version: 14.9
synopsis: REST API for any Postgres database
description: Reads the schema of a PostgreSQL database and creates RESTful routes
for tables, views, and functions, supporting all HTTP methods that security
permits.
license: MIT
license-file: LICENSE
author: Joe Nelson, Adam Baker, Steve Chavez, Wolfgang Walther
author: Joe Nelson, Adam Baker, Steve Chavez
maintainer: Steve Chavez <stevechavezast@gmail.com>
category: Executable, PostgreSQL, Network APIs
homepage: https://postgrest.org
bug-reports: https://github.com/PostgREST/postgrest/issues
build-type: Simple
extra-source-files: CHANGELOG.md
cabal-version: >= 1.10
tested-with:
-- nix
GHC == 9.4.8
-- cabal on Ubuntu
-- stack on FreeBSD, MacOS, Ubuntu, Windows
, GHC == 9.10.3
, GHC == 9.6.7
-- cabal on Ubuntu
-- nix
, GHC == 9.12.3
, GHC == 9.8.4
source-repository head
type: git
@@ -38,16 +39,13 @@ flag hpc
description: Enable HPC (dev only)
library
default-language: GHC2021
default-language: Haskell2010
default-extensions: OverloadedStrings
NoImplicitPrelude
hs-source-dirs: src/library
hs-source-dirs: src
exposed-modules: PostgREST.Admin
PostgREST.App
PostgREST.AppState
PostgREST.AppState.Pool
PostgREST.AppState.Reload
PostgREST.AppState.Types
PostgREST.Auth
PostgREST.Auth.Jwt
PostgREST.Auth.JwtCache
@@ -57,7 +55,6 @@ library
PostgREST.Client
PostgREST.Config
PostgREST.Config.Database
PostgREST.Debounce
PostgREST.Config.JSPath
PostgREST.Config.PgVersion
PostgREST.Config.Proxy
@@ -69,10 +66,9 @@ library
PostgREST.SchemaCache.Representations
PostgREST.SchemaCache.Table
PostgREST.Error
PostgREST.Error.Types
PostgREST.Listener
PostgREST.Logger
PostgREST.MainTx
PostgREST.Logger.Apache
PostgREST.MediaType
PostgREST.Metrics
PostgREST.Network
@@ -85,7 +81,6 @@ library
PostgREST.Plan
PostgREST.Plan.CallPlan
PostgREST.Plan.MutatePlan
PostgREST.Plan.Negotiate
PostgREST.Plan.ReadPlan
PostgREST.Plan.Types
PostgREST.RangeQuery
@@ -101,32 +96,27 @@ library
PostgREST.Response.Performance
PostgREST.TimeIt
PostgREST.Version
build-depends: base >= 4.9 && < 4.22
build-depends: base >= 4.9 && < 4.20
, HTTP >= 4000.3.7 && < 4000.5
, Ranged-sets >= 0.3 && < 0.6
, Ranged-sets >= 0.3 && < 0.5
, aeson >= 2.0.3 && < 2.3
, aeson-jsonpath >= 0.4.2 && < 0.5
, auto-update >= 0.2.7 && < 0.3
, auto-update >= 0.1.4 && < 0.3
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13
, case-insensitive >= 1.2 && < 1.3
, cassava >= 0.4.5 && < 0.6
, configurator-pg >= 0.2.11 && < 0.3
, containers >= 0.5.7 && < 0.8
, containers >= 0.5.7 && < 0.7
, cookie >= 0.4.2 && < 0.6
-- crypton 1.1.0 moved from `memory` to `ram`, which jose-jwt fails to build with right now.
-- should be possible to remove this once jose-jwt had a new release.
, crypton < 1.1.0
, directory >= 1.2.6 && < 1.4
, either >= 4.4.1 && < 5.1
, extra >= 1.7.0 && < 2.0
, fast-logger >= 3.2.0 && < 3.3
, fuzzyset >= 0.2.4 && < 0.3
, hasql >= 1.9 && <= 1.9.3.1
, hasql-dynamic-statements >= 0.3.1 && <= 0.3.1.8
, hasql-notifications >= 0.2.4.0 && < 0.3
, hasql-pool >= 1.1 && <= 1.3.0.4
, hasql-transaction >= 1.0.1 && <= 1.2.1
, hasql >= 1.6.1.1 && < 1.7
, hasql-dynamic-statements >= 0.3.1 && < 0.4
, hasql-notifications >= 0.2.2.2 && < 0.2.3
, hasql-pool >= 1.0.1 && < 1.1
, hasql-transaction >= 1.0.1 && < 1.2
, http-client >= 0.7.19 && < 0.8
, http-types >= 0.12.2 && < 0.13
, insert-ordered-containers >= 0.2.2 && < 0.3
@@ -139,9 +129,9 @@ library
, network-uri >= 2.6.1 && < 2.8
, optparse-applicative >= 0.13 && < 0.19
, parsec >= 3.1.11 && < 3.2
-- Technically unused, can be removed after updating to hasql >= 1.7
, postgresql-libpq >= 0.10
, prometheus-client >= 1.1.1 && < 1.2.0
, prometheus-metrics-ghc >= 1.0.1.2 && < 1.2
, protolude >= 0.3.1 && < 0.4
, regex-tdfa >= 1.2.2 && < 1.4
, retry >= 0.7.4 && < 0.10
@@ -149,30 +139,30 @@ library
, streaming-commons >= 0.2.3.1 && < 0.3
, swagger2 >= 2.4 && < 2.9
, text >= 1.2.2 && < 2.2
, time >= 1.6 && < 1.15
, time >= 1.6 && < 1.13
, unordered-containers >= 0.2.8 && < 0.3
, unix-compat >= 0.5.4 && < 0.8
, vault >= 0.3.1.5 && < 0.4
, vector >= 0.11 && < 0.14
, wai >= 3.2.1 && < 3.3
, wai-cors >= 0.2.5 && < 0.3
, wai-extra >= 3.1.8 && < 3.2
-- We already depend on wai-logger >= 2.3.7 indirectly via wai-extra,
-- but we want to depend on 2.4.0 which fixes 'unknownSocket' log output
-- for unix sockets; this is tested in test/io/test_log.py. See
-- for unix sockets; this is tested in test/io/test_io.py. See
-- https://github.com/kazu-yamamoto/logger/commit/3a71ca70afdbb93d4ecf0083eeba1fbbbcab3fc3
, wai-logger >= 2.4.0
, warp >= 3.4.14 && < 3.5
, warp >= 3.3.19 && < 3.5
, stm >= 2.5 && < 3
, stm-hamt >= 1.2 && < 2
, focus >= 1.0 && < 2
, some >= 1.0.4.1 && < 2
, uuid >= 1.3 && < 2
-- -fno-spec-constr may help keep compile time memory use in check,
-- see https://gitlab.haskell.org/ghc/ghc/issues/16017#note_219304
-- -optP-Wno-nonportable-include-path
-- prevents build failures on case-insensitive filesystems (macos),
-- see https://github.com/commercialhaskell/stack/issues/3918
ghc-options: -j -Werror -Wall -fwarn-identities
ghc-options: -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
if flag(dev)
@@ -181,27 +171,22 @@ library
ghc-options: -fhpc -hpcdir .hpc
else
ghc-options: -O2
if impl(ghc >= 9.12)
-- Makes GHC consider cross-module specialization for polymorphic functions
-- without explicitly needing to add INLINE, INLINABLE or SPECIALIZE pragmas.
-- Slightly increases the binary size but improves performance considerably.
ghc-options: -fexpose-overloaded-unfoldings -fspecialise-aggressively
if !os(windows)
build-depends:
unix
executable postgrest
default-language: GHC2021
default-language: Haskell2010
default-extensions: OverloadedStrings
NoImplicitPrelude
hs-source-dirs: src/executable
hs-source-dirs: main
main-is: Main.hs
build-depends: base >= 4.9 && < 4.22
, containers >= 0.5.7 && < 0.8
build-depends: base >= 4.9 && < 4.20
, containers >= 0.5.7 && < 0.7
, postgrest
, protolude >= 0.3.1 && < 0.4
ghc-options: -j -threaded -rtsopts "-with-rtsopts=-N -I0 -qg"
ghc-options: -threaded -rtsopts "-with-rtsopts=-N -I0 -qg"
-O2 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
@@ -216,7 +201,7 @@ executable postgrest
test-suite spec
type: exitcode-stdio-1.0
default-language: GHC2021
default-language: Haskell2010
default-extensions: OverloadedStrings
QuasiQuotes
NoImplicitPrelude
@@ -226,12 +211,10 @@ test-suite spec
Feature.Auth.AudienceJwtSecretSpec
Feature.Auth.AuthSpec
Feature.Auth.BinaryJwtSecretSpec
Feature.Auth.JwtCacheSpec
Feature.Auth.NoAnonSpec
Feature.Auth.NoJwtSecretSpec
Feature.ConcurrentSpec
Feature.CorsSpec
Feature.HttpHeaderSpec
Feature.ExtraSearchPathSpec
Feature.NoSuperuserSpec
Feature.ObservabilitySpec
@@ -257,10 +240,7 @@ test-suite spec
Feature.Query.PgSafeUpdateSpec
Feature.Query.PlanSpec
Feature.Query.PostGISSpec
Feature.Query.Preferences.HandlingSpec
Feature.Query.Preferences.MaxAffectedSpec
Feature.Query.Preferences.TimezoneSpec
Feature.Query.PreparedStatementsSpec
Feature.Query.PreferencesSpec
Feature.Query.QueryLimitedSpec
Feature.Query.QuerySpec
Feature.Query.RangeSpec
@@ -276,16 +256,16 @@ test-suite spec
Feature.RollbackSpec
Feature.RpcPreRequestGucsSpec
SpecHelper
build-depends: base >= 4.9 && < 4.22
build-depends: base >= 4.9 && < 4.20
, aeson >= 2.0.3 && < 2.3
, aeson-qq >= 0.8.1 && < 0.9
, async >= 2.1.1 && < 2.3
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13
, case-insensitive >= 1.2 && < 1.3
, containers >= 0.5.7 && < 0.8
, hasql-pool >= 1.0.1 && <= 1.3.0.4
, hasql-transaction >= 1.0.1 && <= 1.2.1
, containers >= 0.5.7 && < 0.7
, hasql-pool >= 1.0.1 && < 1.1
, hasql-transaction >= 1.0.1 && < 1.2
, heredoc >= 0.2 && < 0.3
, hspec >= 2.3 && < 2.12
, hspec-expectations >= 0.8.4 && < 0.9
@@ -303,11 +283,10 @@ test-suite spec
, regex-tdfa >= 1.2.2 && < 1.4
, scientific >= 0.3.4 && < 0.4
, text >= 1.2.2 && < 2.2
, time >= 1.6 && < 1.15
, transformers-base >= 0.4.4 && < 0.5
, wai >= 3.2.1 && < 3.3
, wai-extra >= 3.0.19 && < 3.2
ghc-options: -j -threaded -O0 -Werror -Wall -fwarn-identities
ghc-options: -threaded -O0 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
-fno-warn-missing-signatures
-fwrite-ide-info
@@ -316,7 +295,7 @@ test-suite spec
test-suite observability
type: exitcode-stdio-1.0
default-language: GHC2021
default-language: Haskell2010
default-extensions: OverloadedStrings
QuasiQuotes
NoImplicitPrelude
@@ -325,12 +304,11 @@ test-suite observability
other-modules: ObsHelper
Observation.JwtCache
Observation.MetricsSpec
Observation.SchemaCacheSpec
build-depends: base >= 4.9 && < 4.22
build-depends: base >= 4.9 && < 4.20
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13
, hasql-pool >= 1.0.1 && <= 1.3.0.4
, hasql-transaction >= 1.0.1 && <= 1.2.1
, hasql-pool >= 1.0.1 && < 1.1
, hasql-transaction >= 1.0.1 && < 1.2
, hspec >= 2.3 && < 2.12
, hspec-expectations >= 0.8.4 && < 0.9
, hspec-wai >= 0.10 && < 0.12
@@ -342,7 +320,7 @@ test-suite observability
, protolude >= 0.3.1 && < 0.4
, text >= 1.2.2 && < 2.2
, wai >= 3.2.1 && < 3.3
ghc-options: -j -threaded -O0 -Werror -Wall -fwarn-identities
ghc-options: -threaded -O0 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
-fwrite-ide-info
-- https://github.com/PostgREST/postgrest/issues/387
@@ -350,12 +328,15 @@ test-suite observability
test-suite doctests
type: exitcode-stdio-1.0
default-language: GHC2021
default-language: Haskell2010
default-extensions: OverloadedStrings
NoImplicitPrelude
hs-source-dirs: test/doc
main-is: Main.hs
build-depends: base >= 4.9 && < 4.22
, doctest-parallel >= 0.4
build-depends: base >= 4.9 && < 4.20
, doctest >= 0.8
, postgrest
, pretty-simple
ghc-options: -j -threaded -O0 -Werror -Wall -fwarn-identities
, protolude >= 0.3.1 && < 0.4
ghc-options: -threaded -O0 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
+6 -5
View File
@@ -7,9 +7,11 @@
# We highly recommend that use the PostgREST binary cache by installing cachix
# (https://app.cachix.org/) and running `cachix use postgrest`.
{ docker ? false
, postgrest ? import ./default.nix { }
}:
let
postgrest =
import ./default.nix { };
inherit (postgrest) pkgs;
inherit (pkgs) lib;
@@ -35,7 +37,10 @@ lib.overrideDerivation postgrest.env (
buildInputs =
base.buildInputs ++ [
pkgs.cabal-install
pkgs.cabal2nix
pkgs.git
pkgs.postgresql
pkgs.update-nix-fetchgit
postgrest.hsie.bin
]
++ toolboxes;
@@ -44,10 +49,6 @@ lib.overrideDerivation postgrest.env (
''
export HISTFILE=.history
# Bypass proxy for all hosts, it prevents HTTP client failures used in test
# suites. See: https://github.com/PostgREST/postgrest/issues/4633 for more info
export NO_PROXY=*
source ${pkgs.bash-completion}/etc/profile.d/bash_completion.sh
source ${pkgs.git}/share/git/contrib/completion/git-completion.bash
source ${postgrest.hsie.bash-completion}
@@ -7,10 +7,11 @@ import qualified Network.HTTP.Types.Status as HTTP
import qualified Network.Wai as Wai
import qualified Network.Wai.Handler.Warp as Warp
import Control.Monad.Extra (whenJust)
import Control.Monad.Extra (whenJust)
import Network.Socket hiding (addrFamily)
import Network.Socket.ByteString
import PostgREST.AppState (AppState, getConfig, killApp)
import PostgREST.Config (AppConfig (..))
import PostgREST.AppState (AppState)
import PostgREST.MediaType (MediaType (..), toContentType)
import PostgREST.Metrics (metricsToText)
import PostgREST.Network (resolveSocketToAddress)
@@ -21,39 +22,30 @@ import qualified PostgREST.AppState as AppState
import qualified Network.Socket as NS
import Protolude
runAdmin :: AppState -> Maybe NS.Socket -> IO Bool -> Warp.Settings -> IO ()
runAdmin appState maybeAdminSocket checkMainAppLive settings = do
conf <- getConfig appState
runAdmin :: AppState -> Maybe NS.Socket -> NS.Socket -> Warp.Settings -> IO ()
runAdmin appState maybeAdminSocket socketREST settings = do
whenJust maybeAdminSocket $ \adminSocket -> do
address <- resolveSocketToAddress adminSocket
void . forkIO $ handle onError $
Warp.runSettingsSocket (adminServerSettings conf address) adminSocket adminApp
observer $ AdminStartObs address
void . forkIO $ Warp.runSettingsSocket settings adminSocket adminApp
where
adminApp = admin appState checkMainAppLive
adminApp = admin appState socketREST
observer = AppState.getObserver appState
adminServerSettings config addr=
settings
& Warp.setBeforeMainLoop (observer $ AdminStartObs addr)
& maybe identity Warp.setPort (configAdminServerPort config)
onError ex = do
observer $ AdminServerCrashedObs ex
killApp appState -- Admin server crash is deemed unrecoverable, so we kill postgrest
-- | PostgREST admin application
admin :: AppState.AppState -> IO Bool -> Wai.Application
admin appState checkMainAppLive req respond = do
isMainAppLive <- checkMainAppLive
admin :: AppState.AppState -> NS.Socket -> Wai.Application
admin appState socketREST req respond = do
isMainAppReachable <- isRight <$> reachMainApp socketREST
isLoaded <- AppState.isLoaded appState
isPending <- AppState.isPending appState
case Wai.pathInfo req of
["live"] ->
respond $ Wai.responseLBS (if isMainAppLive then HTTP.status200 else HTTP.status500) [] mempty
respond $ Wai.responseLBS (if isMainAppReachable then HTTP.status200 else HTTP.status500) [] mempty
["ready"] ->
let
status | isPending = HTTP.status503
| not isMainAppLive = HTTP.status500
status | not isMainAppReachable = HTTP.status500
| isPending = HTTP.status503
| isLoaded = HTTP.status200
| otherwise = HTTP.status500
in
@@ -66,3 +58,18 @@ admin appState checkMainAppLive req respond = do
respond $ Wai.responseLBS HTTP.status200 [toContentType MTTextPlain] mets -- Content-Type is required for prometheus compliance
_ ->
respond $ Wai.responseLBS HTTP.status404 [] mempty
-- Try to connect to the main app socket
-- Note that it doesn't even send a valid HTTP request, we just want to check that the main app is accepting connections
reachMainApp :: Socket -> IO (Either IOException ())
reachMainApp appSock = do
sockAddr <- getSocketName appSock
sock <- socket (addrFamily sockAddr) Stream defaultProtocol
try $ do
connect sock sockAddr
withSocketsDo $ bracket (pure sock) close sendEmpty
where
sendEmpty sock = void $ send sock mempty
addrFamily (SockAddrInet _ _) = AF_INET
addrFamily (SockAddrInet6 {}) = AF_INET6
addrFamily (SockAddrUnix _) = AF_UNIX
@@ -8,7 +8,6 @@ module PostgREST.ApiRequest
( ApiRequest(..)
, userApiRequest
, userPreferences
, userBearerAuth
) where
import qualified Data.CaseInsensitive as CI
@@ -17,28 +16,32 @@ import qualified Data.List.NonEmpty as NonEmptyList
import qualified Data.Set as S
import qualified Data.Text.Encoding as T
import Data.List (lookup)
import Data.Ranged.Ranges (emptyRange, rangeIntersection,
rangeIsEmpty)
import Network.HTTP.Types.Header (RequestHeaders, hAuthorization, hCookie)
import Network.Wai (Request (..))
import Network.Wai.Middleware.HttpAuth (extractBearerAuth)
import Network.Wai.Parse (parseHttpAccept)
import Web.Cookie (parseCookies)
import Data.List (lookup)
import Data.Ranged.Ranges (emptyRange, rangeIntersection,
rangeIsEmpty)
import Network.HTTP.Types.Header (RequestHeaders, hCookie)
import Network.Wai (Request (..))
import Network.Wai.Parse (parseHttpAccept)
import Web.Cookie (parseCookies)
import PostgREST.ApiRequest.Payload (getPayload)
import PostgREST.ApiRequest.QueryParams (QueryParams (..))
import PostgREST.ApiRequest.Types (Action (..), DbAction (..),
InvokeMethod (..), Mutation (..),
Payload (..), RequestBody,
Resource (..))
import PostgREST.Config (AppConfig (..), OpenAPIMode (..))
import PostgREST.Error (ApiRequestError (..), RangeError (..))
InvokeMethod (..),
Mutation (..), Payload (..),
RequestBody, Resource (..))
import PostgREST.Config (AppConfig (..),
OpenAPIMode (..))
import PostgREST.Config.Database (TimezoneNames)
import PostgREST.Error (ApiRequestError (..),
RangeError (..))
import PostgREST.MediaType (MediaType (..))
import PostgREST.RangeQuery (NonnegRange, allRange,
convertToLimitZeroRange, hasLimitZero,
convertToLimitZeroRange,
hasLimitZero,
rangeRequested)
import PostgREST.SchemaCache.Identifiers (FieldName, QualifiedIdentifier (..),
import PostgREST.SchemaCache.Identifiers (FieldName,
QualifiedIdentifier (..),
Schema)
import qualified PostgREST.ApiRequest.Preferences as Preferences
@@ -61,7 +64,7 @@ data ApiRequest = ApiRequest {
, iPayload :: Maybe Payload -- ^ Data sent by client and used for mutation actions
, iPreferences :: Preferences.Preferences -- ^ Prefer header values
, iQueryParams :: QueryParams.QueryParams
, iColumns :: S.Set FieldName -- ^ parsed columns from &columns parameter and payload
, iColumns :: S.Set FieldName -- ^ parsed colums from &columns parameter and payload
, iHeaders :: [(ByteString, ByteString)] -- ^ HTTP request headers
, iCookies :: [(ByteString, ByteString)] -- ^ Request Cookies
, iPath :: ByteString -- ^ Raw request path
@@ -108,12 +111,8 @@ userApiRequest conf prefs req reqBody = do
actIsInvokeSafe x = case x of {ActDb (ActRoutine _ (InvRead _)) -> True; _ -> False}
-- | Parses the Prefer header
userPreferences :: AppConfig -> Request -> Preferences.Preferences
userPreferences conf req = Preferences.fromHeaders (configDbTxAllowOverride conf) $ requestHeaders req
-- | Obtains the Bearer Auth
userBearerAuth :: Request -> Maybe ByteString
userBearerAuth req = extractBearerAuth =<< lookup hAuthorization (requestHeaders req)
userPreferences :: AppConfig -> Request -> TimezoneNames -> Preferences.Preferences
userPreferences conf req timezones = Preferences.fromHeaders (configDbTxAllowOverride conf) timezones $ requestHeaders req
getResource :: AppConfig -> [Text] -> Either ApiRequestError Resource
getResource AppConfig{configOpenApiMode, configDbRootSpec} = \case
@@ -21,21 +21,20 @@ module PostgREST.ApiRequest.Preferences
, shouldCount
, shouldExplainCount
, prefAppliedHeader
, toHeaderValue
) where
import qualified Data.ByteString.Char8 as BS
import qualified Data.Map as Map
import qualified Data.Set as S
import qualified Network.HTTP.Types.Header as HTTP
import PostgREST.Config.Database (TimezoneNames)
import Protolude
-- $setup
-- Setup for doctests
-- >>> :set -XStandaloneDeriving
-- >>> import Text.Pretty.Simple (pPrint)
-- >>> import qualified Data.Set as S
-- >>> import Protolude
-- >>> deriving instance Show PreferResolution
-- >>> deriving instance Show PreferRepresentation
-- >>> deriving instance Show PreferCount
@@ -63,8 +62,10 @@ data Preferences
-- |
-- Parse HTTP headers based on RFC7240[1] to identify preferences.
--
-- >>> let sc = S.fromList ["America/Los_Angeles"]
--
-- One header with comma-separated values can be used to set multiple preferences:
-- >>> pPrint $ fromHeaders True [("Prefer", "resolution=ignore-duplicates, count=exact, timezone=America/Los_Angeles, max-affected=100")]
-- >>> pPrint $ fromHeaders True sc [("Prefer", "resolution=ignore-duplicates, count=exact, timezone=America/Los_Angeles, max-affected=100")]
-- Preferences
-- { preferResolution = Just IgnoreDuplicates
-- , preferRepresentation = Nothing
@@ -81,7 +82,7 @@ data Preferences
--
-- Multiple headers can also be used:
--
-- >>> pPrint $ fromHeaders True [("Prefer", "resolution=ignore-duplicates"), ("Prefer", "count=exact"), ("Prefer", "missing=null"), ("Prefer", "handling=lenient"), ("Prefer", "invalid"), ("Prefer", "max-affected=5999")]
-- >>> pPrint $ fromHeaders True sc [("Prefer", "resolution=ignore-duplicates"), ("Prefer", "count=exact"), ("Prefer", "missing=null"), ("Prefer", "handling=lenient"), ("Prefer", "invalid"), ("Prefer", "max-affected=5999")]
-- Preferences
-- { preferResolution = Just IgnoreDuplicates
-- , preferRepresentation = Nothing
@@ -97,13 +98,13 @@ data Preferences
--
-- If a preference is set more than once, only the first is used:
--
-- >>> preferTransaction $ fromHeaders True [("Prefer", "tx=commit, tx=rollback")]
-- >>> preferTransaction $ fromHeaders True sc [("Prefer", "tx=commit, tx=rollback")]
-- Just Commit
--
-- This is also the case across multiple headers:
--
-- >>> :{
-- preferResolution . fromHeaders True $
-- preferResolution . fromHeaders True sc $
-- [ ("Prefer", "resolution=ignore-duplicates")
-- , ("Prefer", "resolution=merge-duplicates")
-- ]
@@ -113,7 +114,7 @@ data Preferences
--
-- Preferences can be separated by arbitrary amounts of space, lower-case header is also recognized:
--
-- >>> pPrint $ fromHeaders True [("prefer", "count=exact, tx=commit ,return=representation , missing=default, handling=strict, anything")]
-- >>> pPrint $ fromHeaders True sc [("prefer", "count=exact, tx=commit ,return=representation , missing=default, handling=strict, anything")]
-- Preferences
-- { preferResolution = Nothing
-- , preferRepresentation = Just Full
@@ -126,8 +127,8 @@ data Preferences
-- , invalidPrefs = [ "anything" ]
-- }
--
fromHeaders :: Bool -> [HTTP.Header] -> Preferences
fromHeaders allowTxDbOverride headers =
fromHeaders :: Bool -> TimezoneNames -> [HTTP.Header] -> Preferences
fromHeaders allowTxDbOverride acceptedTzNames headers =
Preferences
{ preferResolution = parsePrefs [MergeDuplicates, IgnoreDuplicates]
, preferRepresentation = parsePrefs [Full, None, HeadersOnly]
@@ -135,7 +136,7 @@ fromHeaders allowTxDbOverride headers =
, preferTransaction = if allowTxDbOverride then parsePrefs [Commit, Rollback] else Nothing
, preferMissing = parsePrefs [ApplyDefaults, ApplyNulls]
, preferHandling = parsePrefs [Strict, Lenient]
, preferTimezone = PreferTimezone <$> timezonePref
, preferTimezone = if isTimezonePrefAccepted then PreferTimezone <$> timezonePref else Nothing
, preferMaxAffected = PreferMaxAffected <$> maxAffectedPref
, invalidPrefs = filter isUnacceptable prefs
}
@@ -155,11 +156,12 @@ fromHeaders allowTxDbOverride headers =
listStripPrefix prefix prefList = listToMaybe $ mapMaybe (BS.stripPrefix prefix) prefList
timezonePref = listStripPrefix "timezone=" prefs
isTimezonePrefAccepted = ((S.member . decodeUtf8 <$> timezonePref) <*> pure acceptedTzNames) == Just True
maxAffectedPref = listStripPrefix "max-affected=" prefs >>= readMaybe . BS.unpack
isUnacceptable p = p `notElem` acceptedPrefs &&
isNothing (BS.stripPrefix "timezone=" p) &&
(isNothing (BS.stripPrefix "timezone=" p) || not isTimezonePrefAccepted) &&
isNothing (BS.stripPrefix "max-affected=" p)
parsePrefs :: ToHeaderValue a => [a] -> Maybe a
@@ -4,22 +4,12 @@
--
-- This module is in charge of parsing all the querystring values in an url, e.g.
-- the select, id, order in `/projects?select=id,name&id=eq.1&order=id,name.desc`.
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE TupleSections #-}
module PostgREST.ApiRequest.QueryParams
( parse
, QueryParams(..)
, pFieldForest
, pFieldName
, pFieldSelect
, pJsonPath
, pLogicTree
, pOpExpr
, pOrder
, pRelationSelect
, pRequestFilter
, pRequestRange
, pSingleVal
, pSpreadRelationSelect
) where
import qualified Data.ByteString.Char8 as BS
@@ -38,38 +28,40 @@ import Data.List (init, last)
import Data.Ranged.Boundaries (Boundary (..))
import Data.Ranged.Ranges (Range (..))
import Data.Tree (Tree (..))
import Text.Parsec.Error (errorMessages, showErrorMessages)
import Text.ParserCombinators.Parsec (GenParser, ParseError, Parser, anyChar,
between, char, choice, digit, eof,
errorPos, letter, lookAhead, many1,
noneOf, notFollowedBy, oneOf, optionMaybe,
sepBy, sepBy1, string, try, (<?>))
import Text.Parsec.Error (errorMessages,
showErrorMessages)
import Text.ParserCombinators.Parsec (GenParser, ParseError, Parser,
anyChar, between, char, choice,
digit, eof, errorPos, letter,
lookAhead, many1, noneOf,
notFollowedBy, oneOf,
optionMaybe, sepBy, sepBy1,
string, try, (<?>))
import PostgREST.RangeQuery (NonnegRange, allRange, rangeGeq,
rangeLimit, rangeOffset,
restrictRange)
import PostgREST.RangeQuery (NonnegRange, allRange,
rangeGeq, rangeLimit,
rangeOffset, restrictRange)
import PostgREST.SchemaCache.Identifiers (FieldName)
import PostgREST.ApiRequest.Types (AggregateFunction (..), EmbedParam (..),
EmbedPath, Field, Filter (..),
FtsOperator (..), Hint, IsVal (..),
JoinType (..), JsonOperand (..),
JsonOperation (..), JsonPath, ListVal,
LogicOperator (..), LogicTree (..),
OpExpr (..), OpQuantifier (..),
Operation (..), OrderDirection (..),
import PostgREST.ApiRequest.Types (AggregateFunction (..),
EmbedParam (..), EmbedPath, Field,
Filter (..), FtsOperator (..),
Hint, IsVal (..), JoinType (..),
JsonOperand (..),
JsonOperation (..), JsonPath,
ListVal, LogicOperator (..),
LogicTree (..), OpExpr (..),
OpQuantifier (..), Operation (..),
OrderDirection (..),
OrderNulls (..), OrderTerm (..),
QuantOperator (..), SelectItem (..),
QuantOperator (..),
SelectItem (..),
SimpleOperator (..), SingleVal)
import PostgREST.Error (QPError (..))
import Protolude hiding (Sum, try)
-- $setup
-- >>> import qualified Text.ParserCombinators.Parsec as P
-- >>> import Protolude hiding (Sum, try)
data QueryParams =
QueryParams
{ qsCanonical :: ByteString
@@ -42,7 +42,8 @@ module PostgREST.ApiRequest.Types
import qualified Data.ByteString.Lazy as LBS
import qualified Data.Set as S
import PostgREST.SchemaCache.Identifiers (FieldName, QualifiedIdentifier (..),
import PostgREST.SchemaCache.Identifiers (FieldName,
QualifiedIdentifier (..),
Schema)
import Protolude
+274
View File
@@ -0,0 +1,274 @@
{-|
Module : PostgREST.App
Description : PostgREST main application
This module is in charge of mapping HTTP requests to PostgreSQL queries.
Some of its functionality includes:
- Mapping HTTP request methods to proper SQL statements. For example, a GET request is translated to executing a SELECT query in a read-only TRANSACTION.
- Producing HTTP Headers according to RFCs.
- Content Negotiation
-}
{-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE ViewPatterns #-}
module PostgREST.App
( postgrest
, run
) where
import GHC.IO.Exception (IOErrorType (..))
import System.IO.Error (ioeGetErrorType)
import Control.Monad.Except (liftEither)
import Data.Either.Combinators (mapLeft, whenLeft)
import Data.Maybe (fromJust)
import Data.String (IsString (..))
import Network.Wai.Handler.Warp (defaultSettings, setHost,
setOnException, setPort,
setServerName)
import qualified Data.Text.Encoding as T
import qualified Network.Wai as Wai
import qualified Network.Wai.Handler.Warp as Warp
import qualified PostgREST.Admin as Admin
import qualified PostgREST.ApiRequest as ApiRequest
import qualified PostgREST.AppState as AppState
import qualified PostgREST.Auth as Auth
import qualified PostgREST.Cors as Cors
import qualified PostgREST.Error as Error
import qualified PostgREST.Listener as Listener
import qualified PostgREST.Logger as Logger
import qualified PostgREST.MainTx as MainTx
import qualified PostgREST.Plan as Plan
import qualified PostgREST.Query as Query
import qualified PostgREST.Response as Response
import qualified PostgREST.Unix as Unix (installSignalHandlers)
import PostgREST.ApiRequest (ApiRequest (..))
import PostgREST.AppState (AppState)
import PostgREST.Auth.Types (AuthResult (..))
import PostgREST.Config (AppConfig (..), LogLevel (..))
import PostgREST.Error (Error)
import PostgREST.Network (resolveSocketToAddress)
import PostgREST.Observation (Observation (..))
import PostgREST.Response.Performance (ServerTiming (..),
serverTimingHeader)
import PostgREST.SchemaCache (SchemaCache (..))
import PostgREST.TimeIt (timeItT)
import PostgREST.Version (docsVersion, prettyVersion)
import qualified Data.ByteString.Char8 as BS
import qualified Data.List as L
import Data.Streaming.Network (bindPortTCP,
bindRandomPortTCP)
import qualified Data.Text as T
import qualified Network.HTTP.Types as HTTP
import qualified Network.Socket as NS
import PostgREST.Unix (createAndBindDomainSocket)
import Protolude hiding (Handler)
type Handler = ExceptT Error
run :: AppState -> IO ()
run appState = do
conf@AppConfig{..} <- AppState.getConfig appState
AppState.schemaCacheLoader appState -- Loads the initial SchemaCache
(mainSocket, adminSocket) <- initSockets conf
Unix.installSignalHandlers observer (AppState.getMainThreadId appState) (AppState.schemaCacheLoader appState) (AppState.readInDbConfig False appState)
Listener.runListener appState
Admin.runAdmin appState adminSocket mainSocket (serverSettings conf)
let app = postgrest configLogLevel appState (AppState.schemaCacheLoader appState)
do
address <- resolveSocketToAddress mainSocket
observer $ AppServerAddressObs address
Warp.runSettingsSocket (serverSettings conf & setOnException onWarpException) mainSocket app
where
observer = AppState.getObserver appState
onWarpException :: Maybe Wai.Request -> SomeException -> IO ()
onWarpException _ ex =
when (shouldDisplayException ex) $
observer $ WarpServerObs $ show ex
-- Similar to wai defaultShouldDisplayException in
-- https://github.com/yesodweb/wai//blob/8c3882c60f6abe043889fc20c7efd3fa9747fa4a/warp/Network/Wai/Handler/Warp/Settings.hs#L251-L258
-- but without omitting AsyncException since it's important to log for ThreadKilled, StackOverflow and other cases.
-- We want to reuse this to avoid flooding the logs for some transient failure cases.
shouldDisplayException :: SomeException -> Bool
shouldDisplayException se
| Just (_ :: Warp.InvalidRequest) <- fromException se = False
| Just (ioeGetErrorType -> et) <- fromException se, et == ResourceVanished || et == InvalidArgument = False
| otherwise = True
serverSettings :: AppConfig -> Warp.Settings
serverSettings AppConfig{..} =
defaultSettings
& setHost (fromString $ toS configServerHost)
& setPort configServerPort
& setServerName ("postgrest/" <> prettyVersion)
-- | PostgREST application
postgrest :: LogLevel -> AppState.AppState -> IO () -> Wai.Application
postgrest logLevel appState connWorker =
traceHeaderMiddleware appState .
Cors.middleware appState .
Auth.middleware appState .
Logger.middleware logLevel Auth.getRole $
-- fromJust can be used, because the auth middleware will **always** add
-- some AuthResult to the vault.
\req respond -> case fromJust $ Auth.getResult req of
Left err -> respond $ Error.errorResponseFor err
Right authResult -> do
appConf <- AppState.getConfig appState -- the config must be read again because it can reload
maybeSchemaCache <- AppState.getSchemaCache appState
let
eitherResponse :: IO (Either Error Wai.Response)
eitherResponse =
runExceptT $ postgrestResponse appState appConf maybeSchemaCache authResult req
response <- either Error.errorResponseFor identity <$> eitherResponse
-- Launch the connWorker when the connection is down. The postgrest
-- function can respond successfully (with a stale schema cache) before
-- the connWorker is done. However, when there's an empty schema cache
-- postgrest responds with the error `PGRST002`; this means that the schema
-- cache is still loading, so we don't launch the connWorker here because
-- it would duplicate the loading process, e.g. https://github.com/PostgREST/postgrest/issues/3704
-- TODO: this process may be unnecessary when the Listener is enabled. Revisit once https://github.com/PostgREST/postgrest/issues/1766 is done
when (isServiceUnavailable response && isJust maybeSchemaCache) connWorker
resp <- do
delay <- AppState.getNextDelay appState
return $ addRetryHint delay response
respond resp
postgrestResponse
:: AppState.AppState
-> AppConfig
-> Maybe SchemaCache
-> AuthResult
-> Wai.Request
-> Handler IO Wai.Response
postgrestResponse appState conf@AppConfig{..} maybeSchemaCache authResult@AuthResult{..} req = do
let observer = AppState.getObserver appState
sCache <-
case maybeSchemaCache of
Just sCache ->
return sCache
Nothing -> do
lift $ observer SchemaCacheEmptyObs
throwError Error.NoSchemaCacheError
body <- lift $ Wai.strictRequestBody req
let jwtTime = if configServerTimingEnabled then Auth.getJwtDur req else Nothing
timezones = dbTimezones sCache
prefs = ApiRequest.userPreferences conf req timezones
(parseTime, apiReq@ApiRequest{..}) <- withTiming $ liftEither . mapLeft Error.ApiRequestError $ ApiRequest.userApiRequest conf prefs req body
(planTime, plan) <- withTiming $ liftEither $ Plan.actionPlan iAction conf apiReq sCache
let mainQ = Query.mainQuery plan conf apiReq authResult configDbPreRequest
tx = MainTx.mainTx mainQ conf authResult apiReq plan sCache
obsQuery s = when configLogQuery $ observer $ QueryObs mainQ s
(txTime, txResult) <- withTiming $ do
case tx of
MainTx.NoDbTx r -> pure r
MainTx.DbTx{..} -> do
dbRes <- lift $ AppState.usePool appState (dqTransaction dqIsoLevel dqTxMode $ runExceptT dqDbHandler)
let eitherResp = join $ mapLeft (Error.PgErr . Error.PgError (Just authRole /= configDbAnonRole)) dbRes
-- TODO: we use obsQuery twice, one here and one below because in case of an error with the usePool above, the request will finish here and return an error message.
-- This is because of a combination of ExceptT + our Error module which has Wai.responseLBS.
-- This needs refactoring so only the below obsQuery is used.
lift $ whenLeft eitherResp $ obsQuery . Error.status
liftEither eitherResp
(respTime, resp) <- withTiming $ do
let response = Response.actionResponse txResult apiReq (T.decodeUtf8 prettyVersion, docsVersion) conf sCache iSchema iNegotiatedByProfile
status' = either Error.status Response.pgrstStatus response
-- TODO: see above obsQuery, only this obsQuery should remain after refactoring (because the QueryObs depends on the status)
lift $ obsQuery status'
liftEither response
return $ toWaiResponse (ServerTiming jwtTime parseTime planTime txTime respTime) resp
where
toWaiResponse :: ServerTiming -> Response.PgrstResponse -> Wai.Response
toWaiResponse timing (Response.PgrstResponse st hdrs bod) = Wai.responseLBS st (hdrs ++ ([serverTimingHeader timing | configServerTimingEnabled])) bod
withTiming :: Handler IO a -> Handler IO (Maybe Double, a)
withTiming f = if configServerTimingEnabled
then do
(t, r) <- timeItT f
pure (Just t, r)
else do
r <- f
pure (Nothing, r)
traceHeaderMiddleware :: AppState -> Wai.Middleware
traceHeaderMiddleware appState app req respond = do
conf <- AppState.getConfig appState
case configServerTraceHeader conf of
Nothing -> app req respond
Just hdr ->
let hdrVal = L.lookup hdr $ Wai.requestHeaders req in
app req (respond . Wai.mapResponseHeaders ([(hdr, fromMaybe mempty hdrVal)] ++))
addRetryHint :: Int -> Wai.Response -> Wai.Response
addRetryHint delay response = do
let h = ("Retry-After", BS.pack $ show delay)
Wai.mapResponseHeaders (\hs -> if isServiceUnavailable response then h:hs else hs) response
isServiceUnavailable :: Wai.Response -> Bool
isServiceUnavailable response = Wai.responseStatus response == HTTP.status503
type AppSockets = (NS.Socket, Maybe NS.Socket)
initSockets :: AppConfig -> IO AppSockets
initSockets AppConfig{..} = do
let
cfg'usp = configServerUnixSocket
cfg'uspm = configServerUnixSocketMode
cfg'host = configServerHost
cfg'port = configServerPort
cfg'adminHost = configAdminServerHost
cfg'adminPort = configAdminServerPort
sock <- case cfg'usp of
-- I'm not using `streaming-commons`' bindPath function here because it's not defined for Windows,
-- but we need to have runtime error if we try to use it in Windows, not compile time error
Just path -> createAndBindDomainSocket path cfg'uspm
Nothing -> do
(_, sock) <-
if cfg'port /= 0
then do
sock <- bindPortTCP cfg'port (fromString $ T.unpack cfg'host)
pure (cfg'port, sock)
else do
-- explicitly bind to a random port, returning bound port number
(num, sock) <- bindRandomPortTCP (fromString $ T.unpack cfg'host)
pure (num, sock)
pure sock
adminSock <- case cfg'adminPort of
Just adminPort -> do
adminSock <- bindPortTCP adminPort (fromString $ T.unpack cfg'adminHost)
pure $ Just adminSock
Nothing -> pure Nothing
pure (sock, adminSock)
+423
View File
@@ -0,0 +1,423 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE RecordWildCards #-}
module PostgREST.AppState
( AppState
, destroy
, getConfig
, getSchemaCache
, getMainThreadId
, getPgVersion
, getNextDelay
, getNextListenerDelay
, getTime
, getJwtCacheState
, init
, initWithPool
, putConfig -- For tests TODO refactoring
, putNextListenerDelay
, putSchemaCache
, putPgVersion
, putIsListenerOn
, usePool
, readInDbConfig
, schemaCacheLoader
, getObserver
, isLoaded
, isPending
) where
import qualified Data.ByteString.Char8 as BS
import Data.Either.Combinators (whenLeft)
import qualified Hasql.Pool as SQL
import qualified Hasql.Pool.Config as SQL
import qualified Hasql.Session as SQL
import qualified Hasql.Transaction.Sessions as SQL
import qualified Network.HTTP.Types.Status as HTTP
import qualified PostgREST.Auth.JwtCache as JwtCache
import qualified PostgREST.Error as Error
import qualified PostgREST.Logger as Logger
import qualified PostgREST.Metrics as Metrics
import PostgREST.Observation
import PostgREST.TimeIt (timeItT)
import PostgREST.Version (prettyVersion)
import Control.AutoUpdate (defaultUpdateSettings, mkAutoUpdate,
updateAction)
import Control.Debounce
import Control.Retry (RetryPolicy, RetryStatus (..), capDelay,
exponentialBackoff, retrying,
rsPreviousDelay)
import Data.IORef (IORef, atomicWriteIORef, newIORef,
readIORef)
import Data.Time.Clock (UTCTime, getCurrentTime)
import PostgREST.Auth.JwtCache (JwtCacheState, update)
import PostgREST.Config (AppConfig (..),
addFallbackAppName,
readAppConfig)
import PostgREST.Config.Database (queryDbSettings,
queryPgVersion,
queryRoleSettings)
import PostgREST.Config.PgVersion (PgVersion (..),
minimumPgVersion)
import PostgREST.SchemaCache (SchemaCache (..),
querySchemaCache,
showSummary)
import PostgREST.SchemaCache.Identifiers (quoteQi)
import Protolude
data AppState = AppState
-- | Database connection pool
{ statePool :: SQL.Pool
-- | Database server version
, statePgVersion :: IORef PgVersion
-- | Schema cache
, stateSchemaCache :: IORef (Maybe SchemaCache)
-- | The schema cache status
, stateSCacheStatus :: IORef SchemaCacheStatus
-- | State of the LISTEN channel
, stateIsListenerOn :: IORef Bool
-- | starts the connection worker with a debounce
, debouncedSCacheLoader :: IO ()
-- | Config that can change at runtime
, stateConf :: IORef AppConfig
-- | Time used for verifying JWT expiration
, stateGetTime :: IO UTCTime
-- | Used for killing the main thread in case a subthread fails
, stateMainThreadId :: ThreadId
-- | Keeps track of the next delay for db connection retry
, stateNextDelay :: IORef Int
-- | Keeps track of the next delay for the listener
, stateNextListenerDelay :: IORef Int
-- | Observation handler
, stateObserver :: ObservationHandler
-- | JWT Cache
, stateJwtCache :: JwtCache.JwtCacheState
, stateLogger :: Logger.LoggerState
, stateMetrics :: Metrics.MetricsState
}
-- | Schema cache status
data SchemaCacheStatus
= SCLoaded
| SCPending
deriving Eq
init :: AppConfig -> IO AppState
init conf@AppConfig{configLogLevel, configDbPoolSize} = do
loggerState <- Logger.init
metricsState <- Metrics.init configDbPoolSize
let observer = liftA2 (>>) (Logger.observationLogger loggerState configLogLevel) (Metrics.observationMetrics metricsState)
observer $ AppStartObs prettyVersion
pool <- initPool conf observer
initWithPool pool conf loggerState metricsState observer --{ stateSocketREST = sock, stateSocketAdmin = adminSock}
initWithPool :: SQL.Pool -> AppConfig -> Logger.LoggerState -> Metrics.MetricsState -> ObservationHandler -> IO AppState
initWithPool pool conf loggerState metricsState observer = do
appState <- AppState pool
<$> newIORef minimumPgVersion -- assume we're in a supported version when starting, this will be corrected on a later step
<*> newIORef Nothing
<*> newIORef SCPending
<*> newIORef False
<*> pure (pure ())
<*> newIORef conf
<*> mkAutoUpdate defaultUpdateSettings { updateAction = getCurrentTime }
<*> myThreadId
<*> newIORef 0
<*> newIORef 1
<*> pure observer
<*> JwtCache.init conf observer
<*> pure loggerState
<*> pure metricsState
deb <-
let decisecond = 100000 in
mkDebounce defaultDebounceSettings
{ debounceAction = retryingSchemaCacheLoad appState
, debounceFreq = decisecond
, debounceEdge = leadingEdge -- runs the worker at the start and the end
}
return appState { debouncedSCacheLoader = deb}
destroy :: AppState -> IO ()
destroy = destroyPool
initPool :: AppConfig -> ObservationHandler -> IO SQL.Pool
initPool AppConfig{..} observer = do
SQL.acquire $ SQL.settings
[ SQL.size configDbPoolSize
, SQL.acquisitionTimeout $ fromIntegral configDbPoolAcquisitionTimeout
, SQL.agingTimeout $ fromIntegral configDbPoolMaxLifetime
, SQL.idlenessTimeout $ fromIntegral configDbPoolMaxIdletime
, SQL.staticConnectionSettings (toUtf8 $ addFallbackAppName prettyVersion configDbUri)
, SQL.observationHandler $ observer . HasqlPoolObs
]
-- | Run an action with a database connection.
usePool :: AppState -> SQL.Session a -> IO (Either SQL.UsageError a)
usePool AppState{stateObserver=observer, stateMainThreadId=mainThreadId, ..} sess = do
observer PoolRequest
res <- SQL.use statePool sess
observer PoolRequestFullfilled
whenLeft res (\case
SQL.AcquisitionTimeoutUsageError ->
observer $ PoolAcqTimeoutObs SQL.AcquisitionTimeoutUsageError
err@(SQL.ConnectionUsageError e) ->
let failureMessage = BS.unpack $ fromMaybe mempty e in
when (("FATAL: password authentication failed" `isInfixOf` failureMessage) || ("no password supplied" `isInfixOf` failureMessage)) $ do
observer $ ExitDBFatalError ServerAuthError err
killThread mainThreadId
err@(SQL.SessionUsageError (SQL.QueryError tpl _ (SQL.ResultError resultErr))) -> do
case resultErr of
SQL.UnexpectedResult{} -> do
observer $ ExitDBFatalError ServerPgrstBug err
killThread mainThreadId
SQL.RowError{} -> do
observer $ ExitDBFatalError ServerPgrstBug err
killThread mainThreadId
SQL.UnexpectedAmountOfRows{} -> do
observer $ ExitDBFatalError ServerPgrstBug err
killThread mainThreadId
-- Check for a syntax error (42601 is the pg code) only for queries that don't have `WITH pgrst_source` as prefix.
-- This would mean the error is on our schema cache queries, so we treat it as fatal.
-- TODO have a better way to mark this as a schema cache query
SQL.ServerError "42601" _ _ _ _ ->
unless ("WITH pgrst_source" `BS.isPrefixOf` tpl) $ do
observer $ ExitDBFatalError ServerPgrstBug err
killThread mainThreadId
-- Check for a "prepared statement <name> already exists" error (Code 42P05: duplicate_prepared_statement).
-- This would mean that a connection pooler in transaction mode is being used
-- while prepared statements are enabled in the PostgREST configuration,
-- both of which are incompatible with each other.
SQL.ServerError "42P05" _ _ _ _ -> do
observer $ ExitDBFatalError ServerError42P05 err
killThread mainThreadId
-- Check for a "transaction blocks not allowed in statement pooling mode" error (Code 08P01: protocol_violation).
-- This would mean that a connection pooler in statement mode is being used which is not supported in PostgREST.
SQL.ServerError "08P01" "transaction blocks not allowed in statement pooling mode" _ _ _ -> do
observer $ ExitDBFatalError ServerError08P01 err
killThread mainThreadId
SQL.ServerError{} ->
when (Error.status (Error.PgError False err) >= HTTP.status500) $
observer $ QueryErrorCodeHighObs err
err@(SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ClientError _))) ->
-- An error on the client-side, usually indicates problems wth connection
observer $ QueryErrorCodeHighObs err
)
return res
-- | Flush the connection pool so that any future use of the pool will
-- use connections freshly established after this call.
flushPool :: AppState -> IO ()
flushPool AppState{..} = SQL.release statePool
-- | Destroy the pool on shutdown.
destroyPool :: AppState -> IO ()
destroyPool AppState{..} = SQL.release statePool
getPgVersion :: AppState -> IO PgVersion
getPgVersion = readIORef . statePgVersion
putPgVersion :: AppState -> PgVersion -> IO ()
putPgVersion = atomicWriteIORef . statePgVersion
getSchemaCache :: AppState -> IO (Maybe SchemaCache)
getSchemaCache = readIORef . stateSchemaCache
putSchemaCache :: AppState -> Maybe SchemaCache -> IO ()
putSchemaCache appState = atomicWriteIORef (stateSchemaCache appState)
schemaCacheLoader :: AppState -> IO ()
schemaCacheLoader = debouncedSCacheLoader
getNextDelay :: AppState -> IO Int
getNextDelay = readIORef . stateNextDelay
getNextListenerDelay :: AppState -> IO Int
getNextListenerDelay = readIORef . stateNextListenerDelay
putNextListenerDelay :: AppState -> Int -> IO ()
putNextListenerDelay = atomicWriteIORef . stateNextListenerDelay
getConfig :: AppState -> IO AppConfig
getConfig = readIORef . stateConf
putConfig :: AppState -> AppConfig -> IO ()
putConfig = atomicWriteIORef . stateConf
getTime :: AppState -> IO UTCTime
getTime = stateGetTime
getJwtCacheState :: AppState -> JwtCacheState
getJwtCacheState = stateJwtCache
getMainThreadId :: AppState -> ThreadId
getMainThreadId = stateMainThreadId
isConnEstablished :: AppState -> IO Bool
isConnEstablished appState = do
AppConfig{..} <- getConfig appState
if configDbChannelEnabled then -- if the listener is enabled, we can be sure the connection is up
readIORef $ stateIsListenerOn appState
else -- otherwise the only way to check the connection is to make a query
isRight <$> usePool appState (SQL.sql "SELECT 1")
putIsListenerOn :: AppState -> Bool -> IO ()
putIsListenerOn = atomicWriteIORef . stateIsListenerOn
isLoaded :: AppState -> IO Bool
isLoaded x = do
scacheStatus <- readIORef $ stateSCacheStatus x
connEstablished <- isConnEstablished x
return $ scacheStatus == SCLoaded && connEstablished
isPending :: AppState -> IO Bool
isPending x = do
scacheStatus <- readIORef $ stateSCacheStatus x
connEstablished <- isConnEstablished x
return $ scacheStatus == SCPending || not connEstablished
putSCacheStatus :: AppState -> SchemaCacheStatus -> IO ()
putSCacheStatus = atomicWriteIORef . stateSCacheStatus
getObserver :: AppState -> ObservationHandler
getObserver = stateObserver
-- | Try to load the schema cache and retry if it fails.
--
-- This is done by repeatedly: 1) flushing the pool, 2) querying the version and validating that the postgres version is supported by us, and 3) loading the schema cache.
-- It's necessary to flush the pool:
--
-- + Because connections cache the pg catalog(see #2620)
-- + For rapid recovery. Otherwise, the pool idle or lifetime timeout would have to be reached for new healthy connections to be acquired.
retryingSchemaCacheLoad :: AppState -> IO ()
retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThreadId=mainThreadId} =
void $ retrying retryPolicy shouldRetry (\RetryStatus{rsIterNumber, rsPreviousDelay} -> do
when (rsIterNumber > 0) $ do
let delay = fromMaybe 0 rsPreviousDelay `div` oneSecondInUs
observer $ ConnectionRetryObs delay
putNextListenerDelay appState delay
flushPool appState
(,) <$> qPgVersion <*> (qInDbConfig *> qSchemaCache)
)
where
qPgVersion :: IO (Maybe PgVersion)
qPgVersion = do
AppConfig{..} <- getConfig appState
pgVersion <- usePool appState (queryPgVersion False) -- No need to prepare the query here, as the connection might not be established
case pgVersion of
Left e -> do
observer $ QueryPgVersionError e
unless configDbPoolAutomaticRecovery $ do
observer ExitDBNoRecoveryObs
killThread mainThreadId
return Nothing
Right actualPgVersion ->
if actualPgVersion < minimumPgVersion then do
observer $ ExitUnsupportedPgVersion actualPgVersion minimumPgVersion
killThread mainThreadId
return Nothing
else do
observer $ DBConnectedObs $ pgvFullName actualPgVersion
observer $ PoolInit configDbPoolSize
putPgVersion appState actualPgVersion
return $ Just actualPgVersion
qInDbConfig :: IO ()
qInDbConfig = do
AppConfig{..} <- getConfig appState
when configDbConfig $ readInDbConfig False appState
qSchemaCache :: IO (Maybe SchemaCache)
qSchemaCache = do
conf@AppConfig{..} <- getConfig appState
(resultTime, result) <-
let transaction = if configDbPreparedStatements then SQL.transaction else SQL.unpreparedTransaction in
timeItT $ usePool appState (transaction SQL.ReadCommitted SQL.Read $ querySchemaCache conf)
case result of
Left e -> do
putSCacheStatus appState SCPending
putSchemaCache appState Nothing
observer $ SchemaCacheErrorObs configDbSchemas configDbExtraSearchPath e
return Nothing
Right sCache -> do
-- IMPORTANT: While the pending schema cache state starts from running the above querySchemaCache, only at this stage we block API requests due to the usage of an
-- IORef on putSchemaCache. This is why SCacheStatus is put at SCPending here to signal the Admin server (using isPending) that we're on a recovery state.
putSCacheStatus appState SCPending
putSchemaCache appState $ Just sCache
observer $ SchemaCacheQueriedObs resultTime
(t, _) <- timeItT $ observer $ SchemaCacheSummaryObs $ showSummary sCache
observer $ SchemaCacheLoadedObs t
putSCacheStatus appState SCLoaded
return $ Just sCache
shouldRetry :: RetryStatus -> (Maybe PgVersion, Maybe SchemaCache) -> IO Bool
shouldRetry _ (pgVer, sCache) = do
AppConfig{..} <- getConfig appState
let itShould = configDbPoolAutomaticRecovery && (isNothing pgVer || isNothing sCache)
return itShould
retryPolicy :: RetryPolicy
retryPolicy =
let delayMicroseconds = 32*oneSecondInUs {-32 seconds-} in
capDelay delayMicroseconds $ exponentialBackoff oneSecondInUs
oneSecondInUs = 1000000 -- one second in microseconds
-- | Reads the in-db config and reads the config file again
-- | We don't retry reading the in-db config after it fails immediately, because it could have user errors. We just report the error and continue.
readInDbConfig :: Bool -> AppState -> IO ()
readInDbConfig startingUp appState@AppState{stateObserver=observer} = do
conf <- getConfig appState
pgVer <- getPgVersion appState
dbSettings <-
if configDbConfig conf then do
qDbSettings <- usePool appState (queryDbSettings (quoteQi <$> configDbPreConfig conf) (configDbPreparedStatements conf))
case qDbSettings of
Left e -> do
observer $ ConfigReadErrorObs e
pure mempty
Right x -> pure x
else
pure mempty
(roleSettings, roleIsolationLvl) <-
if configDbConfig conf then do
rSettings <- usePool appState (queryRoleSettings pgVer (configDbPreparedStatements conf))
case rSettings of
Left e -> do
observer $ QueryRoleSettingsErrorObs e
pure (mempty, mempty)
Right x -> pure x
else
pure mempty
readAppConfig dbSettings (configFilePath conf) (Just $ configDbUri conf) roleSettings roleIsolationLvl >>= \case
Left err ->
if startingUp then
panic err -- die on invalid config if the program is starting up
else
observer $ ConfigInvalidObs err
Right newConf -> do
putConfig appState newConf
-- After the config has reloaded, jwt-secret might have changed, so
-- if it has changed, it is important to invalidate the jwt cache
-- entries, because they were cached using the old secret
update (getJwtCacheState appState) newConf
if startingUp then
pass
else
observer ConfigSucceededObs
+77
View File
@@ -0,0 +1,77 @@
{-# LANGUAGE RecordWildCards #-}
{-|
Module : PostgREST.Auth
Description : PostgREST authentication functions.
This module provides functions to deal with the JWT authentication (http://jwt.io).
It also can be used to define other authentication functions,
in the future Oauth, LDAP and similar integrations can be coded here.
Authentication should always be implemented in an external service.
In the test suite there is an example of simple login function that can be used for a
very simple authentication system inside the PostgreSQL database.
-}
module PostgREST.Auth
( getResult
, getJwtDur
, getRole
, middleware
) where
import qualified Data.ByteString as BS
import qualified Data.Vault.Lazy as Vault
import qualified Network.HTTP.Types.Header as HTTP
import qualified Network.Wai as Wai
import qualified Network.Wai.Middleware.HttpAuth as Wai
import Data.List (lookup)
import PostgREST.TimeIt (timeItT)
import System.IO.Unsafe (unsafePerformIO)
import PostgREST.AppState (AppState, getConfig, getJwtCacheState,
getTime)
import PostgREST.Auth.Jwt (parseClaims)
import PostgREST.Auth.JwtCache (lookupJwtCache)
import PostgREST.Auth.Types (AuthResult (..))
import PostgREST.Config (AppConfig (..))
import PostgREST.Error (Error (..))
import Protolude
-- | Validate authorization header
-- Parse and store JWT claims for future use in the request.
middleware :: AppState -> Wai.Middleware
middleware appState app req respond = do
conf@AppConfig{..} <- getConfig appState
time <- getTime appState
let token = Wai.extractBearerAuth =<< lookup HTTP.hAuthorization (Wai.requestHeaders req)
parseJwt = runExceptT $ lookupJwtCache jwtCacheState token >>= parseClaims conf time
jwtCacheState = getJwtCacheState appState
-- If ServerTimingEnabled -> calculate JWT validation time
req' <- if configServerTimingEnabled then do
(dur, authResult) <- timeItT parseJwt
pure $ req { Wai.vault = Wai.vault req & Vault.insert authResultKey authResult & Vault.insert jwtDurKey dur }
else do
authResult <- parseJwt
pure $ req { Wai.vault = Wai.vault req & Vault.insert authResultKey authResult }
app req' respond
authResultKey :: Vault.Key (Either Error AuthResult)
authResultKey = unsafePerformIO Vault.newKey
{-# NOINLINE authResultKey #-}
getResult :: Wai.Request -> Maybe (Either Error AuthResult)
getResult = Vault.lookup authResultKey . Wai.vault
jwtDurKey :: Vault.Key Double
jwtDurKey = unsafePerformIO Vault.newKey
{-# NOINLINE jwtDurKey #-}
getJwtDur :: Wai.Request -> Maybe Double
getJwtDur = Vault.lookup jwtDurKey . Wai.vault
getRole :: Wai.Request -> Maybe BS.ByteString
getRole req = authRole <$> (rightToMaybe =<< getResult req)
@@ -4,6 +4,8 @@ Description : PostgREST JWT support functions.
This module provides functions to deal with JWT parsing and validation (http://jwt.io).
-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE ImpredicativeTypes #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
@@ -14,10 +16,14 @@ module PostgREST.Auth.Jwt
, parseClaims) where
import qualified Data.Aeson as JSON
import qualified Data.Aeson.Key as K
import qualified Data.Aeson.KeyMap as KM
import qualified Data.ByteString as BS
import qualified Data.ByteString.Internal as BS
import qualified Data.ByteString.Lazy.Char8 as LBS
import qualified Data.Scientific as Sci
import qualified Data.Text as T
import qualified Data.Vector as V
import qualified Jose.Jwk as JWT
import qualified Jose.Jwt as JWT
@@ -27,11 +33,12 @@ import Data.Text ()
import Data.Time.Clock (UTCTime, nominalDiffTimeToSeconds)
import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds)
import PostgREST.Auth.Types (AuthResult (..))
import PostgREST.Config (AppConfig (..), audMatchesCfg)
import PostgREST.Config.JSPath (evaluateJSPath)
import PostgREST.Error (Error (..), JwtClaimsError (..),
JwtDecodeError (..), JwtError (..))
import PostgREST.Auth.Types (AuthResult (..))
import PostgREST.Config (AppConfig (..), FilterExp (..), JSPath,
JSPathExp (..), audMatchesCfg)
import PostgREST.Error (Error (..),
JwtClaimsError (AudClaimNotStringOrArray, ExpClaimNotNumber, IatClaimNotNumber, JWTExpired, JWTIssuedAtFuture, JWTNotInAudience, JWTNotYetValid, NbfClaimNotNumber, ParsingClaimsFailed),
JwtDecodeError (..), JwtError (..))
import Data.Aeson ((.:?))
import Data.Aeson.Types (parseMaybe)
@@ -88,10 +95,13 @@ checkForErrors time audMatches = mconcat
parseToken :: (MonadError Error m, MonadIO m) => JwkSet -> ByteString -> m JWT.JwtContent
parseToken _ "" = throwError $ JwtErr $ JwtDecodeErr EmptyAuthHeader
parseToken secret tkn = do
-- secret <- liftEither . maybeToRight (JwtErr JwtSecretMissing) $ configJWKS
tknWith3Parts <- hasThreeParts tkn
eitherContent <- liftIO $ JWT.decode (JWT.keys secret) Nothing tknWith3Parts
liftEither . mapLeft (JwtErr . jwtDecodeError) $ eitherContent
--liftEither $ mapLeft JwtErr $ verifyClaims content
where
--hasThreeParts :: ByteString -> Either Error ByteString
hasThreeParts token = case length $ BS.split (BS.c2w '.') token of
3 -> pure token
n -> throwError $ JwtErr $ JwtDecodeErr $ UnexpectedParts n
@@ -112,12 +122,30 @@ parseClaims cfg@AppConfig{configJwtRoleClaimKey, configDbAnonRole} time mclaims
validateClaims time (audMatchesCfg cfg) mclaims
-- role defaults to anon if not specified in jwt
role <- liftEither . maybeToRight (JwtErr JwtTokenRequired) $
unquoted <$> evaluateJSPath (Just $ JSON.Object mclaims) configJwtRoleClaimKey <|> configDbAnonRole
unquoted <$> walkJSPath (Just $ JSON.Object mclaims) configJwtRoleClaimKey <|> configDbAnonRole
pure AuthResult
{ authClaims = mclaims
{ authClaims = mclaims & KM.insert "role" (JSON.toJSON $ decodeUtf8 role)
, authRole = role
}
where
walkJSPath :: Maybe JSON.Value -> JSPath -> Maybe JSON.Value
walkJSPath x [] = x
walkJSPath (Just (JSON.Object o)) (JSPKey key:rest) = walkJSPath (KM.lookup (K.fromText key) o) rest
walkJSPath (Just (JSON.Array ar)) (JSPIdx idx:rest) = walkJSPath (ar V.!? idx) rest
walkJSPath (Just (JSON.Array ar)) [JSPFilter (EqualsCond txt)] = findFirstMatch (==) txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (NotEqualsCond txt)] = findFirstMatch (/=) txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (StartsWithCond txt)] = findFirstMatch T.isPrefixOf txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (EndsWithCond txt)] = findFirstMatch T.isSuffixOf txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (ContainsCond txt)] = findFirstMatch T.isInfixOf txt ar
walkJSPath _ _ = Nothing
findFirstMatch matchWith pattern = foldr checkMatch Nothing
where
checkMatch (JSON.String txt) acc
| pattern `matchWith` txt = Just $ JSON.String txt
| otherwise = acc
checkMatch _ acc = acc
unquoted :: JSON.Value -> BS.ByteString
unquoted (JSON.String t) = encodeUtf8 t
unquoted v = LBS.toStrict $ JSON.encode v

Some files were not shown because too many files have changed in this diff Show More