Compare commits

..
Author SHA1 Message Date
steve-chavez 16e2c73a95 bump version to 14.9 2026-04-10 18:25:04 -05:00
steve-chavez fb31654277 fix: remove red herring warp logs on default log-level
The logs added on e95e815483 are red
herrings under normal operation.

This moves them to `log-level=debug` and removes "error" from the
message prefix.

Fixes https://github.com/PostgREST/postgrest/issues/4799
2026-04-10 18:16:19 -05:00
steve-chavez 9230f3f49a Revert "refactor: Simplify schema cache loading triggering logic"
This reverts commit 810023a47c.
2026-04-10 18:16:19 -05:00
steve-chavez 96bbc86756 Revert "refactor: clarify debouncer function"
This reverts commit 577ed4dd4c.
2026-04-10 18:16:19 -05:00
renovate[bot]andWolfgang Walther dd394d6f81 chore(deps): update haskell-actions/setup action to v2.10.4 2026-04-09 08:48:45 +00:00
Steve ChavezandTaimoor Zaeem 577ed4dd4c refactor: clarify debouncer function
(cherry picked from commit 3d98f8d65b)
2026-04-08 10:46:38 +05:00
Michał KłeczekandTaimoor Zaeem 810023a47c refactor: Simplify schema cache loading triggering logic
DISCLAIMER:
This commit was authored entirely by a human without the assistance of LLMs.

Using debouncer to trigger schema cache loading makes it difficult to understand when exactly it is triggered.

(cherry picked from commit a4c1d945ee)
2026-04-08 10:46:38 +05:00
renovate[bot]andWolfgang Walther 687ebf0850 chore(deps): update ubuntu:noble docker digest to 84e77de 2026-04-07 08:10:14 +00:00
Michał KłeczekandSteve Chavez b9c8562641 add: Log pg version details of listener connection
Follow-up to #4617 adding more information to log entry produced upon successful listener connection establishement.
2026-04-06 11:11:16 -05:00
Michał KłeczekandSteve Chavez 34a767a5cc add: Log actual host and port of listener connection
Diagnosing problems with listener channel notifications not being handled properly by PostgREST connected to read replicas is difficult. Issues might be related to lost connections and listener not being connected to the right host after failover or database server restarts.
This patch adds logging of actual host:port used by libpq connection opened by the listener. It should make it easier to find out if PostgREST is connected to the right host.
2026-04-06 11:11:16 -05:00
steve-chavezandTaimoor Zaeem 5eac8bd203 docs: clarify set operators need views/functions
Closes https://github.com/PostgREST/postgrest/issues/4780.
2026-04-06 11:18:42 +05:00
steve-chavez 9f722e0799 bump version to 14.8 2026-04-03 16:51:55 -05:00
Artur Bento de CarvalhoandSteve Chavez 0d4d1dca51 fix: use int32/int64 formats for integer types
Fixed integer type mapping in OpenAPI 2.0: replaced the invalid integer format with int32/int64 and added the toSwaggerFormat function to map PostgreSQL types to valid OpenAPI 2.0 formats:

smallint -> int32
integer -> int32
bigint -> int64
2026-04-03 16:36:55 -05:00
Laurence IslaandSteve Chavez 250747aadc add(logs): Include a HINT when the LISTEN channel breaks due to a Postgres bug
The HINT shows a SQL command that solves the issue.
2026-04-03 16:36:55 -05:00
Michał KłeczekandTaimoor Zaeem aae929a718 test: Schema cache load debouncing
test: adjust replicateM to 100
(cherry picked from commit 328598eaed)
2026-04-03 08:00:57 +05:00
renovate[bot]andWolfgang Walther 77dde73057 chore(deps): update docker/login-action action to v4.1.0 2026-04-02 20:06:52 +00:00
Michal KleczekandSteve Chavez afb95a5268 refactor(test): provide means to validate metrics and observations
Some helpers are provided for introspecting metrics already (used in JWT cache tests). This change provides facilities to additionally validate emited Observation events.
A new Spec module is also implemented, adding basic tests of schema cache reloading - their main goal is to excercise the new infrastructure.
2026-04-02 13:34:38 -05:00
Michał KłeczekandTaimoor Zaeem 8262faa235 refactor: move socket creation and management to App module
Right now listening sockets initialization, management and usage is split between App, AppState and Admin modules: they are created in AppState.init and remembered in AppState but used only in App and Admin.

It has several negative consequences:
- sockets are initialized even if not needed (eg. command line invocations like dump-config or dump-schema)
- it is impossible to start listening on a socket after initial schema cache load because it requires AppState

This change decouples listen socket management from AppState. Sockets are created only when needed (ie. not in command line tools invocation) and passed to admin application and to Warp by the App module.
2026-04-02 12:27:31 +05:00
Michał KłeczekandTaimoor Zaeem 1d40fe5d93 test: Fix flakiness of test_second_request_for_non_existent_table_should_be_quick
Changed divider in assertion (response.elapsed.total_seconds() < first_duration / divider) to 2 (from 10).

(cherry picked from commit 886df84e87)
2026-04-01 15:14:45 +05:00
renovate[bot]andWolfgang Walther 3a4dc5eff3 chore(deps): update codecov/codecov-action action to v6 2026-03-27 11:02:38 +00:00
renovate[bot]andWolfgang Walther 55912515c0 chore(deps): update all dependencies 2026-03-27 08:45:13 +00:00
postgrest-ci[bot]andGitHub 863d1c9e9b docs: explain schema cache reload behavior with NOTIFY debouncing 2026-03-26 11:00:40 +05:00
steve-chavez 806579e659 bump version to 14.7 2026-03-20 14:22:31 -05:00
steve-chavez 1856434a74 fix: not logging termination unix signals
Under container environments like ECS, it's hard to know when PostgREST
is being terminated.
2026-03-20 13:38:40 -05:00
Taimoor ZaeemandSteve Chavez dade15acc4 nix(test): add test suite for observability tests
- Create separate test suite for observability tests

- Create wrapper script `postgrest-test-observability`

- Add to CI and `postgrest-check`

- Move JWT cache tests under observability tests

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-03-20 09:13:47 -05:00
Michał KłeczekandSteve Chavez 79531a612d test(spec): Move metrics state helpers from JwtCacheSpec to SpecHelpers
Refactoring: State validation helpers used in JwtCacheSpec moved to SpecHelper
to make them available in other Spec modules.

(cherry picked from commit 85a313a8cc)
2026-03-19 14:30:18 -05:00
renovate[bot]andWolfgang Walther 9d0e13d961 chore(deps): update ubuntu:noble docker digest to 186072b 2026-03-19 09:15:59 +00:00
renovate[bot]andWolfgang Walther 3e61dc024e chore(deps): update cachix/cachix-action action to v17 2026-03-18 21:24:50 +00:00
renovate[bot]andWolfgang Walther 1bdf773417 chore(deps): update all dependencies 2026-03-18 20:01:41 +00:00
renovate[bot]andWolfgang Walther 55868351f3 chore(deps): update actions/cache action to v5.0.4 2026-03-18 19:09:06 +00:00
renovate[bot]andWolfgang Walther 8c3cb76fbd chore(deps): update ubuntu:noble docker digest to 0d39fcc 2026-03-17 10:22:17 +00:00
renovate[bot]andWolfgang Walther 3bdb69bae7 chore(deps): update actions/create-github-app-token action to v3 2026-03-14 18:06:45 +00:00
renovate[bot]andWolfgang Walther 1ed3c2c197 chore(deps): update all dependencies 2026-03-13 17:23:12 +00:00
renovate[bot]andWolfgang Walther dde145b685 chore(deps): update all dependencies 2026-03-13 17:19:11 +00:00
Wolfgang Walther 720705e32b docs: fix prometheus text format link
Reported by linkcheck.
2026-03-11 09:45:21 +01:00
steve-chavez 6813ceaf31 bump version to 14.6 2026-03-06 16:30:04 -05:00
Taimoor Zaeemandsteve-chavez 73c8ae0bbc fix(error): leaking table and function names when calculating hint
Increase similarity score to 0.75 from 0.33 for table and functions
error hint.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-03-06 16:16:05 -05:00
Laurence Isla 6f356e32b6 chore: update sponsors
* The list of sponsors is now ordered by total contribution value (highest first)
* Removed former backers from sponsor pages
2026-03-03 23:11:22 -05:00
Laurence Isla c4772184e4 chore: update sponsor 2026-03-03 23:11:22 -05:00
dshukertjrandWolfgang Walther 0602080acb docs: Update the Supabase logo to a correct one 2026-02-19 10:54:29 +01:00
steve-chavez dc67e4d3f5 chore: fix changelog entries 2026-02-13 13:40:38 -05:00
steve-chavez d89826b982 bump version to 14.5 2026-02-12 20:12:52 -05:00
steve-chavez 94350fdde6 fix: don't hide async exceptions in logs
Fixes #4646. Using the repro on #4646, this now produces the log:

```
11/Feb/2026:09:40:08 -0500: Warp server error: stack overflow
```

When:
```
$ curl localhost:3000/
curl: (52) Empty reply from server
```

(cherry picked from commit e95e815483)
2026-02-12 20:11:44 -05:00
Taimoor ZaeemandWolfgang Walther 230eb3630c docs(install): update postgresql minimum supported version
PostgREST dropped support for PostgreSQL version 12 however,
it was not reflected in the docs.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-02-12 10:23:29 +01:00
steve-chavez 5485b8ca9a bump version to 14.4 2026-01-29 14:13:49 -05:00
steve-chavez 895e9c536c chore: remove wrong entry in CHANGELOG 2026-01-29 13:24:20 -05:00
Michal Kleczekandsteve-chavez 16c767134c fix: listener running with exception masked after first failure 2026-01-29 13:22:46 -05:00
Laurence IslaandSteve Chavez 0a8b836435 fix: filtering the returned representation whenn using or/and filters on mutations
(cherry picked from commit 1682677297)
2026-01-29 09:16:08 -05:00
Michal KleczekandSteve Chavez 5796f86100 fix: ensure Listener connections are released
retryingListen function potentially leaks database connections. This patch ensures the connections are released in case of listen/notify errors.

(cherry picked from commit 00c7cb1a22)
2026-01-28 18:26:15 -05:00
Wolfgang Walther 101eac1cce docs: fix links
datrium.com doesn't exist anymore, while euronodes.com seems to only
fail SSL in CI.
2026-01-28 09:57:23 +01:00
renovate[bot]andWolfgang Walther 1ae14afdf2 chore(deps): update haskell-actions/setup action to v2.10.2 2026-01-11 17:36:28 +00:00
Wolfgang Walther 69090bd224 ci: pin backport action to version instead of default branch 2026-01-11 18:34:43 +01:00
renovate[bot]andWolfgang Walther 5d5160fbd7 chore(deps): update haskell-actions/setup action to v2.10.1 2026-01-05 19:01:32 +00:00
steve-chavez 545f45d9de bump version to 14.3 2026-01-03 16:45:56 +08:00
Taimoor Zaeemandsteve-chavez eb55e73645 chore: move changelog entry to unreleased section
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-01-03 16:44:10 +08:00
Michał KłeczekandSteve Chavez e252a4900c fix: Performance and high memory usage of relation hint calculation
* Calculation of hint message when requested relation is not present in schema cache requires creation of a FuzzySet (to use fuzzy search to find candidate tables). For schemas with many tables it is costly.
This patch introduces dbTablesFuzzyIndex in SchemaCache to memoize the FuzzySet creation.

* Additionally, because of FuzzySet large memory requirements, this patch introduces a limit of 500 relations per schema, above which FuzzySet is not created and hint calculation disabled.

(cherry picked from commit e592d568c6)
2026-01-03 15:18:37 +08:00
Taimoor ZaeemandSteve Chavez 01bdb05c89 nix: add config file for hlint
Adds a config file for hlint containing arguments and
custom warnings.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2025-12-23 11:51:05 -05:00
122ed4d02e refactor: fix definition of Ord instance for Routine type (#4577)
The `Ord` instance definition for type `Routine` had a logical
error when comparing two routines. The error did not affect any
end users. However, for correctness and completeness reasons, this
commit fixes the error.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Co-authored-by: Joel Jacobson <joel@compiler.org>
2025-12-23 11:51:05 -05:00
renovate[bot]andWolfgang Walther 7ff6755af7 chore(deps): update docker/setup-buildx-action action to v3.12.0 2025-12-20 20:24:23 +00:00
steve-chavez 29d6d35020 bump version to 14.2 2025-12-18 21:30:38 -05:00
steve-chavez 4d35eb02e7 fix: inaccurate Server-Timing durations
The transaction duration was notably off, doing:

```
curl localhost:3000/rpc/sleep?seconds=5 -i
```

Shows `46.1` for the `transaction;dur`, with this fix we obtain
`5007.3`.

Fixes https://github.com/PostgREST/postgrest/issues/4522

This also fixes inaccurate "schema cache queried" logs,
see https://github.com/PostgREST/postgrest/issues/4551.

(cherry picked from commit 013f078bc4)
2025-12-18 18:18:09 -05:00
renovate[bot]andWolfgang Walther e126956c1b chore(deps): update haskell-actions/setup action to v2.9.1 2025-12-17 18:01:25 +00:00
Joel JakobssonandSteve Chavez ce7871c047 fix: hasSingleUnnamedParam incorrectly matching named parameters
The hasSingleUnnamedParam function was only checking the parameter type
but not whether the parameter actually had no name. This caused functions
with a single NAMED parameter (e.g., `foo(data json)`) to incorrectly
match the single-param fallback mode.

The result was a confusing PostgreSQL error 42883 "function does not exist"
instead of a clean PGRST202 error explaining that no matching function
was found.

Added ppName == mempty check so functions with named parameters don't
incorrectly match the single-param fallback.

(cherry picked from commit fd6a3bdccf)
2025-12-16 17:24:20 -05:00
Taimoor ZaeemandSteve Chavez a59e6d97c5 refactor: create function to handle response preferences
Centralizes handling of preferences that are used to
create `Preference-Applied` header which is returned on
responses.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 29c2aa015f)
2025-12-16 11:20:33 -05:00
renovate[bot]andWolfgang Walther 40fec696a6 chore(deps): update korthout/backport-action action to v4.0.1 2025-12-15 20:17:46 +00:00
renovate[bot]andWolfgang Walther 9eb36f1244 chore(deps): update all dependencies 2025-12-13 11:31:10 +00:00
Laurence IslaandSteve Chavez 720eb8e528 chore(changelog): move incorrectly placed fixes 2025-12-12 14:47:26 -05:00
Laurence IslaandSteve Chavez aec95f7944 refactor: use only Lazy.ByteString to calculate the response body length 2025-12-12 14:47:26 -05:00
Laurence IslaandSteve Chavez 1c33d2dd38 fix: add missing Content-Length to empty HTTP 201 responses
For when a preference other than return=representation is requested.
2025-12-12 14:47:26 -05:00
Laurence IslaandSteve Chavez 7be638f0f2 fix: regression that truncates error message when offset is out of bounds
- Happens when offset > the number of rows and when "Prefer: count=exact" header is sent
- Regression introduced in commit 57ef998
2025-12-12 14:47:26 -05:00
Taimoor ZaeemandSteve Chavez d4b8109522 fix: misleading logs on unsupported postgresql versions
Postgrest fails on unsupported pg versions. However before killing
the thread, it continues to print a few more log messages which
were misleading. This commit fixes this by making sure that the
no log message should be printed after the unsupported pg version
observation and kill the thread immediately.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 0fa2f569a7)
2025-12-10 20:24:14 -05:00
renovate[bot]andWolfgang Walther f2b96b1d41 chore(deps): update codecov/codecov-action action to v5.5.2 2025-12-10 12:24:00 +00:00
renovate[bot]andWolfgang Walther cc43d356c1 chore(deps): update actions/create-github-app-token action to v2.2.1 2025-12-06 11:00:16 +00:00
renovate[bot]andWolfgang Walther 8bcbbd1bfd chore(deps): update all dependencies 2025-12-03 13:52:35 +00:00
renovate[bot]andWolfgang Walther ec4d2e2af3 chore(deps): update actions/checkout action to v6 2025-12-03 13:52:23 +00:00
Taimoor ZaeemandSteve Chavez e8cb0e33eb test(io): fix freeport function to prevent failures
Sometimes, a healthcheck related test fails as occurred in
https://github.com/PostgREST/postgrest/actions/runs/19771357953/job/56655949002.
This happens due to freeport function accidently picking up a used port.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 50eec773bf)
2025-11-30 13:38:55 -05:00
Taimoor ZaeemandSteve Chavez 05074f41c2 test(io): move fixtures to fixtures/ directory
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 35de13eebd)
2025-11-28 13:23:20 -05:00
Taimoor ZaeemandSteve Chavez c9c617cce1 test(io): rename fixtures.sql to load.sql
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 3baa6b0063)
2025-11-28 13:23:20 -05:00
Taimoor ZaeemandWolfgang Walther e429974f31 nix(shell): remove postgrest/ directory prefix when running pg (#4502)
When running postgres from nix-shell, nix creates a directory
structure like `postgrest/postgrest-with-pg-17-XXX` in the `/tmp`
directory. This commit removes the extra `postgrest/` prefix to
shorten length of absolute path length of filenames.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 802cce9a28)
2025-11-26 08:33:23 +00:00
Taimoor ZaeemandSteve Chavez 88538c1357 test(io): move authentication related tests to test_auth.py
The `test_io.py` module is too bloated (2100+ lines). To
logically group related tests, as a first step, this commit
separates authentication related IO tests into `test_auth.py`
module.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit c561a3749c)
2025-11-22 12:45:09 -05:00
Taimoor ZaeemandSteve Chavez 1cbe6b7c5f test(io): remove stale jwt cache test
Removes a test related to jwt cache which is stale
since #4084.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 379eaec8e0)
2025-11-19 16:48:25 -05:00
Taimoor ZaeemandWolfgang Walther f97200948f nix: shorten postgrest-with-postgresql-xx scripts
Renames these scripts to `postgrest-with-pg-xx`. The renaming
helps reduce the length of temporary filenames. This is needed
to ensure that socket file names remain under the maximum
allowed length of 107 chars.

Closes #4461.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 462798dc2d)
2025-11-18 18:27:17 +00:00
Taimoor ZaeemandWolfgang Walther 0bacce6909 docs: update server-host config in docker-compose example
The docker-compose example did not work with multiple containers
when PGRST_SERVER_HOST is set to `localhost`. This updates the
value to `0.0.0.0` allowing other containers to connect.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 08c6deba12)
2025-11-18 18:13:26 +00:00
renovate[bot]andWolfgang Walther 4c3ac2f087 chore(deps): update actions/checkout action to v5.0.1 2025-11-18 10:06:11 +00:00
Wolfgang Walther 1c6f215bd7 ci/test: remove macos x86 flake check
See previous commit.
2025-11-18 11:05:43 +01:00
Wolfgang Walther 94fb0a489a ci: remove macos x86 builds
The x86 GitHub runner will not be available anymore, soon.

We might be able to re-introduce this, once we can build a static
executable via Nix on darwin, too.
2025-11-18 11:01:41 +01:00
Taimoor ZaeemandSteve Chavez e4e1b626a6 test(io): move resource embedding tests to test_io.py
- Adds fixtures to `test/io/fixtures.sql` to test resource
  embedding related queries.

- Moves the resource embedding related tests that no longer
  require big schema from `test_big_schema.py` to `test_io.py`.

Closes #4417.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 91abcd49e1)
2025-11-17 13:24:49 -05:00
renovate[bot]andWolfgang Walther 810341f635 chore(deps): update ubuntu:noble docker digest to c35e29c 2025-11-15 13:52:10 +00:00
Taimoor ZaeemandWolfgang Walther 4a6f0b4a18 chore(cabal): remove unused haskell dependencies
This should reduce setup time for build process.

- cache: introduced in #2928, defunct since #4084
- clock: introduced in #2928, defunct since #4084
- heredoc: introduced in #714, defunct since #4390
- iproute: introduced in #3560, defunct since #4288

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 99217433b1)
2025-11-10 14:24:48 +00:00
steve-chavez b3b4e5ff35 bump version to 14.1 2025-11-05 09:09:22 -05:00
Taimoor ZaeemandWolfgang Walther 84f437b6c9 chore(changelog): update versioning scheme description
The changelog description mentioned that we follow semantic
versioning but from now on we don't. Hence updated the description
to reflect new versioning policy.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit eb908c696f)
2025-11-04 07:25:35 +00:00
renovate[bot]andWolfgang Walther a3c8065e58 chore(deps): update actions/checkout digest to 71cf226 2025-11-03 20:47:35 +00:00
Taimoor ZaeemandWolfgang Walther c797c09e22 fix: server-host !6 incorrectly binds to IPv4 address
Updates streaming-commons to version 0.2.3.1. This resolves #3202.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 58efc2680e)
2025-11-03 11:40:09 +00:00
Taimoor ZaeemandWolfgang Walther fc6fbe9748 chore(changelog): fix typo in changelog entry
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 8180905890)
2025-11-01 09:18:20 +00:00
Taimoor ZaeemandSteve Chavez 4aa712b9d8 fix: db-pre-config function failing with pg reserved words
When db-pre-config is accidentally set to a pg reserved word
like "true", it fails with a confusing error. The function
names should be properly quoted to avoid such errors. This commit
resolves this by quoting the pre-config function name.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit a688878236)
2025-10-30 14:32:37 -05:00
Taimoor ZaeemandSteve Chavez 939061baff refactor: move escapeIdent function to Identifiers.hs
Moves the functions `escapeIdent` and `trimNullChars` to
SchemaCache/Identifiers.hs module.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 66f84c5903)
2025-10-30 12:24:49 -05:00
Taimoor ZaeemandSteve Chavez 6150d53592 refactor: sort exports of Identifiers.hs and SqlFragments.hs
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 5d9b169380)
2025-10-30 12:24:49 -05:00
Wolfgang Walther d245e07df5 ci: fix tag job with new release workflow
A single component version is the development version, everything with
more components is not. Thus, we only need to check for a single dot.
2025-10-25 10:21:51 +02:00
renovate[bot]andWolfgang Walther e913efb8cb chore(deps): update all dependencies 2025-10-25 08:12:30 +00:00
179 changed files with 1868 additions and 5557 deletions
-2
View File
@@ -1,2 +0,0 @@
# Ignore blame for commit that moved protolude files under src/protolude
d4949c633e8172d0e4dd8f5c991eaaae6b48fbb0
+5
View File
@@ -0,0 +1,5 @@
# TODO: Remove this once a new actionlint release has been cut
# and made its way to us through nixpkgs.
self-hosted-runner:
labels:
- ubuntu-24.04-arm
@@ -112,7 +112,7 @@ runs:
echo "artifacts=${artifacts}" >> "$GITHUB_OUTPUT" echo "artifacts=${artifacts}" >> "$GITHUB_OUTPUT"
- name: Save artifact to GitHub Actions - name: Save artifact to GitHub Actions
if: steps.find-task.outputs.task_found if: steps.find-task.outputs.task_found
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with: with:
name: ${{ inputs.upload }} name: ${{ inputs.upload }}
path: ${{ steps.download.outputs.artifacts }} path: ${{ steps.download.outputs.artifacts }}
+7 -6
View File
@@ -8,6 +8,7 @@ inputs:
required: true required: true
save-prs: save-prs:
description: Whether to additionally store the cache in a pull request, too. Should only be used for very small caches. description: Whether to additionally store the cache in a pull request, too. Should only be used for very small caches.
type: boolean
prefix: prefix:
description: Cache key prefix to be used in both primary key and restore-keys. description: Cache key prefix to be used in both primary key and restore-keys.
required: true required: true
@@ -18,17 +19,17 @@ inputs:
runs: runs:
using: composite using: composite
steps: steps:
- uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 - uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }} if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with: with:
path: ${{ inputs.path }} path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ runner.arch }}-${{ inputs.prefix }}-${{ inputs.suffix }} key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: | restore-keys: |
${{ runner.os }}-${{ runner.arch }}-${{ inputs.prefix }}- ${{ runner.os }}-${{ inputs.prefix }}-
- uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 - uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }} if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with: with:
path: ${{ inputs.path }} path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ runner.arch }}-${{ inputs.prefix }}-${{ inputs.suffix }} key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: | restore-keys: |
${{ runner.os }}-${{ runner.arch }}-${{ inputs.prefix }}- ${{ runner.os }}-${{ inputs.prefix }}-
-3
View File
@@ -4,9 +4,6 @@ codecov:
comment: false comment: false
github_checks:
annotations: true
coverage: coverage:
status: status:
project: project:
+41
View File
@@ -13,6 +13,9 @@
}, },
"packageRules": [ "packageRules": [
{ {
"matchBaseBranches": [
"/^v[0-9]+/"
],
"matchManagers": [ "matchManagers": [
"haskell-cabal" "haskell-cabal"
], ],
@@ -23,6 +26,44 @@
"/^v[0-9]+/" "/^v[0-9]+/"
], ],
"groupName": "all dependencies" "groupName": "all dependencies"
},
{
"matchManagers": [
"haskell-cabal"
],
"matchPackageNames": [
"base",
"bytestring",
"containers",
"directory",
"mtl",
"parsec",
"process",
"text"
],
"groupName": "GHC dependencies"
},
{
"matchManagers": [
"haskell-cabal"
],
"matchPackageNames": [
"hasql",
"hasql-dynamic-statements",
"hasql-notifications",
"hasql-transaction",
"hasql-pool"
],
"groupName": "hasql"
},
{
"matchManagers": [
"haskell-cabal"
],
"matchPackageNames": [
"fuzzyset"
],
"allowedVersions": "<0.3"
} }
] ]
} }
+2 -2
View File
@@ -28,7 +28,7 @@ jobs:
# This actions creates the github token using the postgrest app secrets # This actions creates the github token using the postgrest app secrets
- name: Create Github App Token - name: Create Github App Token
id: app-token id: app-token
uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1 uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0
with: with:
app-id: ${{ vars.POSTGREST_CI_APP_ID }} app-id: ${{ vars.POSTGREST_CI_APP_ID }}
private-key: ${{ secrets.POSTGREST_CI_PRIVATE_KEY }} private-key: ${{ secrets.POSTGREST_CI_PRIVATE_KEY }}
@@ -45,7 +45,7 @@ jobs:
# Backport action that creates the PR with given settings # Backport action that creates the PR with given settings
- name: Create backport PR - name: Create backport PR
uses: korthout/backport-action@7c3f6cd5843cac11bc59a04a1b7699af93261670 # v4.5 uses: korthout/backport-action@3c06f323a58619da1e8522229ebc8d5de2633e46 # v4.3.0
with: with:
github_token: ${{ steps.app-token.outputs.token }} github_token: ${{ steps.app-token.outputs.token }}
pull_description: 'Backport for #${pull_number}.' pull_description: 'Backport for #${pull_number}.'
+34 -40
View File
@@ -16,7 +16,6 @@ on:
- .github/* - .github/*
- '*.nix' - '*.nix'
- nix/** - nix/**
- flake.lock
- .cirrus.yml - .cirrus.yml
- cabal.project* - cabal.project*
- postgrest.cabal - postgrest.cabal
@@ -31,18 +30,8 @@ concurrency:
jobs: jobs:
static: static:
strategy: name: Nix - Linux x86-64 static
fail-fast: false runs-on: ubuntu-24.04
matrix:
include:
- name: Linux aarch64
runs-on: ubuntu-24.04-arm
artifact: aarch64
- name: Linux x86-64
runs-on: ubuntu-24.04
artifact: x86-64
name: Nix - ${{ matrix.name }} static
runs-on: ${{ matrix.runs-on }}
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Setup Nix Environment - name: Setup Nix Environment
@@ -53,19 +42,19 @@ jobs:
- name: Build static executable - name: Build static executable
run: nix-build -A postgrestStatic -A postgrestStatic.tests run: nix-build -A postgrestStatic -A postgrestStatic.tests
- name: Save built executable as artifact - name: Save built executable as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with: with:
name: postgrest-linux-static-${{ matrix.artifact }} name: postgrest-linux-static-x86-64
path: result/bin/postgrest path: result/bin/postgrest
if-no-files-found: error if-no-files-found: error
- name: Build Docker image - name: Build Docker image
run: nix-build -A docker.image --out-link postgrest-docker-${{ matrix.artifact }}.tar.gz run: nix-build -A docker.image --out-link postgrest-docker.tar.gz
- name: Save built Docker image as artifact - name: Save built Docker image as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with: with:
name: postgrest-docker-${{ matrix.artifact }} name: postgrest-docker-x86-64
path: postgrest-docker-${{ matrix.artifact }}.tar.gz path: postgrest-docker.tar.gz
if-no-files-found: error if-no-files-found: error
@@ -78,14 +67,19 @@ jobs:
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Install nix-build-uncached - name: Install gnu sed
run: nix-env -f default.nix -iA nix-build-uncached run: brew install gnu-sed
- name: Build everything (default.nix) - name: Build everything
run: nix-build-uncached run: |
# The --dry-run will give us a list of derivations to download from cachix and
- name: Build everything (shell.nix) # derivations to build. We only take those that would have to be built and then build
run: nix-build-uncached shell.nix # those explicitly. This has the advantage that pure verification will not include
# a download anymore, making it much faster. If something needs to be built, only
# the dependencies required to do so will be downloaded, but not everything.
nix-build --dry-run 2>&1 \
| gsed -e '1,/derivations will be built:$/d' -e '/paths will be fetched/Q' \
| xargs nix-build
stack: stack:
@@ -93,6 +87,15 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
include: include:
- name: Linux aarch64
runs-on: ubuntu-24.04-arm
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-ubuntu-aarch64
deps: sudo apt-get update && sudo apt-get install libpq-dev
- name: MacOS aarch64 - name: MacOS aarch64
runs-on: macos-14 runs-on: macos-14
cache: | cache: |
@@ -102,15 +105,6 @@ jobs:
artifact: postgrest-macos-aarch64 artifact: postgrest-macos-aarch64
deps: brew link --force libpq deps: brew link --force libpq
- name: MacOS x86-64
runs-on: macos-15-intel
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-macos-x86-64
deps: brew link --force libpq
- name: Windows - name: Windows
runs-on: windows-2022 runs-on: windows-2022
cache: | cache: |
@@ -124,10 +118,10 @@ jobs:
runs-on: ${{ matrix.runs-on }} runs-on: ${{ matrix.runs-on }}
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: haskell-actions/setup@cd0d9bdd65b20557f41bea4dbe43d0b5fbbfe553 # v2.11.0 - uses: haskell-actions/setup@de26526e12bc780fb9d384c1fb61c0bf02e3a40d # v2.10.4
with: with:
# This must match the version in stack.yaml's resolver # This must match the version in stack.yaml's resolver
ghc-version: 9.10.3 ghc-version: 9.6.7
enable-stack: true enable-stack: true
stack-no-global: true stack-no-global: true
stack-setup-ghc: true stack-setup-ghc: true
@@ -152,7 +146,7 @@ jobs:
- name: Strip Executable - name: Strip Executable
run: strip result/postgrest* run: strip result/postgrest*
- name: Save built executable as artifact - name: Save built executable as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with: with:
name: ${{ matrix.artifact }} name: ${{ matrix.artifact }}
path: | path: |
@@ -177,13 +171,13 @@ jobs:
cabal: cabal:
strategy: strategy:
matrix: matrix:
ghc: ['9.10.3', '9.12.3'] ghc: ['9.6.7', '9.8.4']
fail-fast: false fail-fast: false
name: Cabal - Linux x86-64 - GHC ${{ matrix.ghc }} name: Cabal - Linux x86-64 - GHC ${{ matrix.ghc }}
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: haskell-actions/setup@cd0d9bdd65b20557f41bea4dbe43d0b5fbbfe553 # v2.11.0 - uses: haskell-actions/setup@de26526e12bc780fb9d384c1fb61c0bf02e3a40d # v2.10.4
with: with:
ghc-version: ${{ matrix.ghc }} ghc-version: ${{ matrix.ghc }}
- name: Cache .cabal - name: Cache .cabal
-1
View File
@@ -14,7 +14,6 @@ on:
- .github/actions/setup-nix/** - .github/actions/setup-nix/**
- default.nix - default.nix
- nix/** - nix/**
- flake.lock
- docs/** - docs/**
- '!**.md' - '!**.md'
+32 -27
View File
@@ -49,7 +49,7 @@ jobs:
echo "Relevant extract from CHANGELOG.md:" echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md cat CHANGES.md
- name: Save CHANGES.md as artifact - name: Save CHANGES.md as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with: with:
name: release-changes name: release-changes
path: CHANGES.md path: CHANGES.md
@@ -75,26 +75,23 @@ jobs:
mkdir -p release-bundle mkdir -p release-bundle
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-aarch64.tar.xz" \
-C artifacts/postgrest-linux-static-aarch64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-x86-64.tar.xz" \ tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-x86-64.tar.xz" \
-C artifacts/postgrest-linux-static-x86-64 postgrest -C artifacts/postgrest-linux-static-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-aarch64.tar.xz" \ tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-aarch64.tar.xz" \
-C artifacts/postgrest-macos-aarch64 postgrest -C artifacts/postgrest-macos-aarch64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-x86-64.tar.xz" \
-C artifacts/postgrest-macos-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-freebsd-x86-64.tar.xz" \ tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-freebsd-x86-64.tar.xz" \
-C artifacts/postgrest-freebsd-x86-64 postgrest -C artifacts/postgrest-freebsd-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-ubuntu-aarch64.tar.xz" \
-C artifacts/postgrest-ubuntu-aarch64 postgrest
zip --junk-paths "release-bundle/postgrest-${GITHUB_REF_NAME}-windows-x86-64.zip" \ zip --junk-paths "release-bundle/postgrest-${GITHUB_REF_NAME}-windows-x86-64.zip" \
artifacts/postgrest-windows-x86-64/postgrest.exe artifacts/postgrest-windows-x86-64/postgrest.exe
- name: Save release bundle - name: Save release bundle
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with: with:
name: release-bundle name: release-bundle
path: release-bundle path: release-bundle
@@ -139,41 +136,49 @@ jobs:
DOCKER_REPO: ${{ vars.DOCKER_REPO }} DOCKER_REPO: ${{ vars.DOCKER_REPO }}
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Download aarch64 Docker image
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-docker-aarch64
- name: Download x86-64 Docker image - name: Download x86-64 Docker image
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with: with:
name: postgrest-docker-x86-64 name: postgrest-docker-x86-64
- name: Download aarch64 binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-ubuntu-aarch64
- uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0 - uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0 - uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with: with:
username: ${{ vars.DOCKER_USER }} username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }} password: ${{ secrets.DOCKER_PASS }}
- name: Build aarch64 Docker image
run: |
# This only pushes the image via digest, not a tag. This will not appear
# in the image list on Docker Hub, yet. It will be later added to the main
# tag's manifest.
docker buildx build \
-t "$DOCKER_REPO/postgrest" \
--platform linux/arm64 \
--output push-by-digest=true,type=image,push=true \
--metadata-file metadata.json \
.
echo "SHA256_ARM=$(jq -r '."containerimage.digest"' metadata.json)" >> "$GITHUB_ENV"
- name: Publish images on Docker Hub - name: Publish images on Docker Hub
run: | run: |
docker load -i postgrest-docker-aarch64.tar.gz docker load -i postgrest-docker.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-arm64"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-arm64"
docker load -i postgrest-docker-x86-64.tar.gz docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-amd64" docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-amd64" docker buildx imagetools create --append \
-t "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}" \
docker manifest create "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}" \ "$DOCKER_REPO/postgrest@$SHA256_ARM"
"$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-arm64" \
"$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-amd64"
docker manifest push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
# Only tag 'latest' for full releases # Only tag 'latest' for full releases
if [ "${GITHUB_REF_NAME}" != "devel" ]; then if [ "${GITHUB_REF_NAME}" != "devel" ]; then
echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..." echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..."
docker manifest create "$DOCKER_REPO/postgrest:latest" \ docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
"$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-arm64" \ docker push "$DOCKER_REPO"/postgrest:latest
"$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}-linux-amd64" docker buildx imagetools create --append \
docker manifest push "$DOCKER_REPO/postgrest:latest" -t "$DOCKER_REPO/postgrest:latest" \
"$DOCKER_REPO/postgrest@$SHA256_ARM"
else else
echo "Skipping push to 'latest' tag for pre-release..." echo "Skipping push to 'latest' tag for pre-release..."
fi fi
+3 -5
View File
@@ -17,7 +17,6 @@ on:
- .github/actions/setup-nix/** - .github/actions/setup-nix/**
- default.nix - default.nix
- nix/** - nix/**
- flake.lock
- .stylish-haskell.yaml - .stylish-haskell.yaml
- cabal.project - cabal.project
- postgrest.cabal - postgrest.cabal
@@ -49,7 +48,7 @@ jobs:
- run: postgrest-cabal-update - run: postgrest-cabal-update
- name: Run coverage (IO tests and Spec tests against latest supported PostgreSQL) - name: Run coverage (IO tests and Spec tests against PostgreSQL 15)
run: postgrest-coverage run: postgrest-coverage
- name: Upload coverage to codecov - name: Upload coverage to codecov
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0 uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
@@ -70,7 +69,7 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
pgVersion: [14, 15, 16, 17, 18] pgVersion: [13, 14, 15, 16, 17]
name: PG ${{ matrix.pgVersion }} name: PG ${{ matrix.pgVersion }}
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
defaults: defaults:
@@ -125,7 +124,7 @@ jobs:
loadtest: loadtest:
strategy: strategy:
matrix: matrix:
kind: ['mixed', 'errors', 'jwt-hs', 'jwt-hs-cache', 'jwt-hs-cache-worst', 'jwt-rsa', 'jwt-rsa-cache', 'jwt-rsa-cache-worst'] kind: ['mixed', 'jwt-hs', 'jwt-hs-cache', 'jwt-hs-cache-worst', 'jwt-rsa', 'jwt-rsa-cache', 'jwt-rsa-cache-worst']
name: Loadtest name: Loadtest
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
steps: steps:
@@ -157,7 +156,6 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
runs-on: runs-on:
- macos-15-intel # x86_64-darwin
- macos-14 # aarch64-darwin - macos-14 # aarch64-darwin
- ubuntu-24.04 # x86_64-linux - ubuntu-24.04 # x86_64-linux
- ubuntu-24.04-arm # aarch64-linux - ubuntu-24.04-arm # aarch64-linux
-1
View File
@@ -26,4 +26,3 @@ loadtest
.docs-build .docs-build
gen_targets.http gen_targets.http
gen_jwk.json gen_jwk.json
gen_private.json
+1 -1
View File
@@ -7,4 +7,4 @@ python:
build: build:
os: ubuntu-24.04 os: ubuntu-24.04
tools: tools:
python: "3.12" python: "3.11"
+3 -42
View File
@@ -4,45 +4,6 @@ All notable changes to this project will be documented in this file. From versio
## Unreleased ## Unreleased
### Added
- Log error when `db-schemas` config contains schema `pg_catalog` or `information_schema` by @taimoorzaeem in #4359
- Add string slicing operator for `jwt-role-claim-key` by @taimoorzaeem in #4599
- Optimize requests with `Prefer: count=exact` that do not use ranges or `db-max-rows` by @laurenceisla in #3957
+ Removed unnecessary double count when building the `Content-Range`.
- Add config `client-error-verbosity` to customize error verbosity by @taimoorzaeem in #4088, #3980, #3824
- Add `Vary` header to responses by @develop7 in #4609
- Add config `db-timezone-enabled` for optional querying of timezones by @taimoorzaeem in #4751
- Log schema cache queries timings on `log-level=debug` by @steve-chavez in #4805
### Fixed
- Shutdown should wait for in flight requests by @mkleczek in #4702
- Fix login with uppercase and mixed case role names by @taimoorzaeem in #4678
- Remove automatic transaction retries on `40001 (serialization_failure)` errors to prevent replication lag by @laurenceisla in #3673
- Fix unexpected results when embedding and filtering the same table more than once by @laurenceisla in #4075
### Changed
- Drop support for PostgreSQL EOL version 13 by @wolfgangwalther in #4193
- All responses now include a `Vary` header by @develop7 in #4609
- Log error when `db-schemas` config contains schema `pg_catalog` or `information_schema` by @taimoorzaeem in #4359
+ Now fails at startup. Prior to this, it failed with `PGRST205` on requests related to these schemas.
- Build a static executable for aarch64-linux by @wolfgangwalther in #4193
- Build the minimal docker image for aarch64-linux by @wolfgangwalther in #4193
- The name of an embedded table can no longer be used in filters if it has an alias by @laurenceisla in #4075
+ e.g. `?select=alias:table(*)&table.id=eq.1` is not possible anymore, use `?select=alias:table(*)&alias.id=eq.1` instead.
## [14.10] - 2026-04-16
### Added
- Log when the pool is released during schema cache reload on `log-level=debug` by @mkleczek in #4668
### Fixed
- Fix unnecessary connection pool flushes during schema cache reloading by @mkleczek in #4645
## [14.9] - 2026-04-10 ## [14.9] - 2026-04-10
### Added ### Added
@@ -87,7 +48,7 @@ All notable changes to this project will be documented in this file. From versio
- Ensure Listener connections are released by @mkleczek in #4614 - Ensure Listener connections are released by @mkleczek in #4614
- Fix incorrectly filtering the returned representation for PATCH requests when using `or/and` filters by @laurenceisla in #3707 - Fix incorrectly filtering the returned representation for PATCH requests when using `or/and` filters by @laurenceisla in #3707
- Fix listener running with exception masked after first failure by @mkleczek in #4615 - Fix listener running with exception masked after first failure by @mkleczek #4615
## [14.3] - 2026-01-03 ## [14.3] - 2026-01-03
@@ -691,7 +652,7 @@ All notable changes to this project will be documented in this file. From versio
### Added ### Added
- #1933, #2109, Add a minimal health check endpoint - @steve-chavez - #1933, #2109, Add a minimal health check endpoint - @steve-chavez
+ For enabling this, the `admin-server-port` config must be set explicitly + For enabling this, the `admin-server-port` config must be set explictly
+ A `<host>:<admin_server_port>/live` endpoint is available for checking if postgrest is running on its port/socket. 200 OK = alive, 503 = dead. + A `<host>:<admin_server_port>/live` endpoint is available for checking if postgrest is running on its port/socket. 200 OK = alive, 503 = dead.
+ A `<host>:<admin_server_port>/ready` endpoint is available for checking a correct internal state(the database connection plus the schema cache). 200 OK = ready, 503 = not ready. + A `<host>:<admin_server_port>/ready` endpoint is available for checking a correct internal state(the database connection plus the schema cache). 200 OK = ready, 503 = not ready.
- #1988, Add the current user to the request log on stdout - @DavidLindbom, @wolfgangwalther - #1988, Add the current user to the request log on stdout - @DavidLindbom, @wolfgangwalther
@@ -1174,7 +1135,7 @@ All notable changes to this project will be documented in this file. From versio
- Customize content negotiation per route - @begriffs - Customize content negotiation per route - @begriffs
- Allow using nulls order without explicit order direction - @steve-chavez - Allow using nulls order without explicit order direction - @steve-chavez
- Fatal error on postgres unsupported version, format supported version in error message - @steve-chavez - Fatal error on postgres unsupported version, format supported version in error message - @steve-chavez
- Prevent database memory consumption by prepared statements caches - @ruslantalpa - Prevent database memory cosumption by prepared statements caches - @ruslantalpa
- Use specific columns in the RETURNING section - @ruslantalpa - Use specific columns in the RETURNING section - @ruslantalpa
- Fix columns alias for RETURNING - @steve-chavez - Fix columns alias for RETURNING - @steve-chavez
+14 -21
View File
@@ -1,12 +1,17 @@
# Contributing to PostgREST # Contributing to PostgREST
## AI Policy **First:** if you're unsure or afraid of _anything_, just ask or
submit the issue or pull request anyways. You won't be yelled at
for giving your best effort. The worst that can happen is that
you'll be politely asked to change something. We appreciate any
sort of contributions, and don't want a wall of rules to get in the
way of that.
We adhere to [Gentoo's AI policy](https://wiki.gentoo.org/wiki/Project:Council/AI_policy): However, for those individuals who want a bit more guidance on the
best way to contribute to the project, read on. This document will
> It is expressly forbidden to contribute [...] any content that has been created with the assistance of Natural Language Processing artificial intelligence tools. This motion can be revisited, should a case been made over such a tool that does not pose copyright, ethical and quality concerns. cover what we're looking for. By addressing all the points we're
looking for, it raises the chances we can quickly merge or address
You can find more about its rationale [here](https://wiki.gentoo.org/wiki/Project:Council/AI_policy#Rationale). your contributions.
## Issues ## Issues
@@ -35,12 +40,12 @@ For questions on how to use PostgREST, please use
We have a fully nix-based development environment with many tools for a smooth development workflow available. We have a fully nix-based development environment with many tools for a smooth development workflow available.
Check the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md) on how to set it up and use it. Check the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md) on how to set it up and use it.
### Haskell Conventions
* All contributions must pass the tests before being merged. When * All contributions must pass the tests before being merged. When
you create a pull request your code will automatically be tested. you create a pull request your code will automatically be tested.
* All fixes or features must have a test proving the improvement. * All code must also pass [hlint](http://community.haskell.org/~ndm/hlint/) and [stylish-haskell](https://github.com/jaspervdj/stylish-haskell)
* All code must also pass a [linter](http://community.haskell.org/~ndm/hlint/) and [styler](https://github.com/jaspervdj/stylish-haskell)
with no warnings. This helps enforce a uniform style for all committers. Continuous integration will check this as well on every with no warnings. This helps enforce a uniform style for all committers. Continuous integration will check this as well on every
pull request. There are useful tools in the nix-shell that help with checking this locally. You can run `postgrest-check` to do this manually but pull request. There are useful tools in the nix-shell that help with checking this locally. You can run `postgrest-check` to do this manually but
we recommend adding it to `.git/hooks/pre-commit` as `nix-shell --run postgrest-check` to automatically check this before doing a commit. we recommend adding it to `.git/hooks/pre-commit` as `nix-shell --run postgrest-check` to automatically check this before doing a commit.
@@ -48,15 +53,3 @@ Check the [development docs](https://github.com/PostgREST/postgrest/blob/main/ni
### Running Tests ### Running Tests
For instructions on running tests, see the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md#testing). For instructions on running tests, see the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md#testing).
### Structuring commits in pull requests
To simplify reviews, make it easy to split pull requests if deemed necessary, and to maintain clean and meaningful history of changes, you will be asked to update your PR if it does not follow the below rules:
* It must be possible to merge the PR branch into target using `git merge --ff-only`, ie. the source branch must be rebased on top of target.
* No merge commits in the source branch.
* All commits in the source branch must be self contained, meaning: it should be possible to treat each commit as a separate PR.
* Commits in the source branch must contain only related changes (related means the changes target a single problem/goal). For example, any refactorings should be isolated from the actual change implementation into separate commits.
* Tests, documentation, and changelog updates should be contained in the same commits as the actual code changes they relate to. An exception to this rule is when test or documentation changes are made in separate PR.
* Commit messages must be prefixed with one of the prefixes defined in [the list used by commit verification scripts](https://github.com/PostgREST/postgrest/blob/main/nix/tools/gitTools.nix#L11).
* Commit messages should contain a longer description of the purpose of the changes contained in the commit and, for non-trivial changes, a description of the changes themselves.
+21
View File
@@ -0,0 +1,21 @@
# PostgREST Docker Hub image for aarch64.
# The x86-64 is a single-static-binary image built via Nix, see:
# nix/tools/docker/README.md
FROM ubuntu:noble@sha256:84e77dee7d1bc93fb029a45e3c6cb9d8aa4831ccfcc7103d36e876938d28895b AS postgrest
RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY postgrest /usr/bin/postgrest
RUN chmod +x /usr/bin/postgrest
EXPOSE 3000
USER 1000
# Use the array form to avoid running the command using bash, which does not handle `SIGTERM` properly.
# See https://docs.docker.com/compose/faq/#why-do-my-services-take-10-seconds-to-recreate-or-stop
CMD ["postgrest"]
+2 -1
View File
@@ -1,4 +1,5 @@
Copyright (c) 2014-2026 The PostgREST contributors Copyright (c) 2014 Joe Nelson
Copyright (c) 2019 Steve Chavez
Permission is hereby granted, free of charge, to any person obtaining Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the a copy of this software and associated documentation files (the
+1 -1
View File
@@ -125,7 +125,7 @@ and limited with - range headers. More about
## Data Integrity ## Data Integrity
Rather than relying on an Object Relational Mapper and custom Rather than relying on an Object Relational Mapper and custom
imperative coding, this system requires you to put declarative constraints imperative coding, this system requires you put declarative constraints
directly into your database. Hence no application can corrupt your directly into your database. Hence no application can corrupt your
data (including your API server). data (including your API server).
-2
View File
@@ -1,4 +1,2 @@
packages: postgrest.cabal packages: postgrest.cabal
tests: true tests: true
allow-newer:
hasql:postgresql-libpq
+1 -1
View File
@@ -1 +1 @@
index-state: hackage.haskell.org 2026-04-18T18:42:36Z index-state: hackage.haskell.org 2025-10-29T04:02:18Z
+3 -5
View File
@@ -1,6 +1,6 @@
{ system ? builtins.currentSystem { system ? builtins.currentSystem
, compiler ? "ghc9123" , compiler ? "ghc948"
, # Commit of the Nixpkgs repository that we want to use. , # Commit of the Nixpkgs repository that we want to use.
# It defaults to reading the inputs from flake.lock, which serves # It defaults to reading the inputs from flake.lock, which serves
@@ -44,6 +44,7 @@ let
allOverlays.checked-shell-script allOverlays.checked-shell-script
allOverlays.gitignore allOverlays.gitignore
(allOverlays.haskell-packages { inherit compiler; }) (allOverlays.haskell-packages { inherit compiler; })
allOverlays.slocat
]; ];
# Evaluated expression of the Nixpkgs repository. # Evaluated expression of the Nixpkgs repository.
@@ -52,11 +53,11 @@ let
postgresqlVersions = postgresqlVersions =
[ [
{ name = "pg-18"; postgresql = pkgs.postgresql_18.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-17"; postgresql = pkgs.postgresql_17.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "pg-17"; postgresql = pkgs.postgresql_17.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-16"; postgresql = pkgs.postgresql_16.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "pg-16"; postgresql = pkgs.postgresql_16.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "pg-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "pg-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
]; ];
haskellPackages = pkgs.haskell.packages."${compiler}"; haskellPackages = pkgs.haskell.packages."${compiler}";
@@ -107,9 +108,6 @@ rec {
inherit (pkgs.haskell.packages."${compiler}") ghcWithPackages; inherit (pkgs.haskell.packages."${compiler}") ghcWithPackages;
}; };
# Used by CI on MacOS
inherit (pkgs) nix-build-uncached;
### Tools ### Tools
cabalTools = cabalTools =
+2
View File
@@ -61,3 +61,5 @@ The image is built from scratch using
no commands are listed in the image history. See the [PostgREST no commands are listed in the image history. See the [PostgREST
repository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for repository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for
details on the build process and how to inspect the image. details on the build process and how to inspect the image.
This does not apply to the arm64 variant, which is based on Ubuntu.
+3 -3
View File
@@ -48,14 +48,14 @@ source_suffix = ".rst"
# The master toctree document. # The master toctree document.
master_doc = "index" master_doc = "index"
# This is overridden by readthedocs with the version tag anyway # This is overriden by readthedocs with the version tag anyway
version = "devel" version = "14"
# To avoid repetition in <title> we set this to an empty string. # To avoid repetition in <title> we set this to an empty string.
release = "" release = ""
# General information about the project. # General information about the project.
project = "PostgREST " + version project = "PostgREST " + version
author = "The PostgREST contributors" author = "Joe Nelson, Steve Chavez"
copyright = "2017, " + author copyright = "2017, " + author
# The language for content autogenerated by Sphinx. Refer to documentation # The language for content autogenerated by Sphinx. Refer to documentation
+1 -2
View File
@@ -6,7 +6,7 @@ Community Tutorials
* `Building a Contacts List with PostgREST and Vue.js <https://www.youtube.com/watch?v=iHtsALtD5-U>`_ - * `Building a Contacts List with PostgREST and Vue.js <https://www.youtube.com/watch?v=iHtsALtD5-U>`_ -
In this video series, DigitalOcean shows how to build and deploy an Nginx + PostgREST(using a managed PostgreSQL database) + Vue.js webapp in an Ubuntu server droplet. In this video series, DigitalOcean shows how to build and deploy an Nginx + PostgREST(using a managed PostgreSQL database) + Vue.js webapp in an Ubuntu server droplet.
* `PostgREST + Auth0: Create REST API in minutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest-1-introduction/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service. * `PostgREST + Auth0: Create REST API in mintutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service.
* `"CodeLess" backend using postgres, postgrest and oauth2 authentication with keycloak <https://www.mathieupassenaud.fr/codeless_backend/>`_ - * `"CodeLess" backend using postgres, postgrest and oauth2 authentication with keycloak <https://www.mathieupassenaud.fr/codeless_backend/>`_ -
A step-by-step tutorial for using PostgREST with KeyCloak(hosted on a managed service). A step-by-step tutorial for using PostgREST with KeyCloak(hosted on a managed service).
@@ -37,7 +37,6 @@ Example Apps
* `archtika <https://github.com/thiloho/archtika>`_ - self-hosted CMS * `archtika <https://github.com/thiloho/archtika>`_ - self-hosted CMS
* `delibrium-postgrest <https://gitlab.com/delibrium/delibrium-postgrest/>`_ - example school API and front-end in Vue.js * `delibrium-postgrest <https://gitlab.com/delibrium/delibrium-postgrest/>`_ - example school API and front-end in Vue.js
* `ETH-transactions-storage <https://github.com/Adamant-im/ETH-transactions-storage>`_ - indexer for Ethereum to get transaction list by ETH address * `ETH-transactions-storage <https://github.com/Adamant-im/ETH-transactions-storage>`_ - indexer for Ethereum to get transaction list by ETH address
* `fullstack template <https://github.com/jenstroeger/fullstack-webapp-template>`_ - a complete fullstack webapp template using PG as db and message queue, Python and Dramatiq to implement async jobs, db migrations, test runners, and more.
* `general <https://github.com/PierreRochard/general>`_ - example auth back-end * `general <https://github.com/PierreRochard/general>`_ - example auth back-end
* `guild-operators <https://github.com/cardano-community/koios-artifacts/tree/main/files/grest>`_ - example queries and functions that the Cardano Community uses for their Guild Operators' Repository * `guild-operators <https://github.com/cardano-community/koios-artifacts/tree/main/files/grest>`_ - example queries and functions that the Cardano Community uses for their Guild Operators' Repository
* `PostGUI <https://github.com/priyank-purohit/PostGUI>`_ - React Material UI admin panel * `PostGUI <https://github.com/priyank-purohit/PostGUI>`_ - React Material UI admin panel
+1 -1
View File
@@ -163,7 +163,7 @@ Another option is to define the function with the :code:`SECURITY DEFINER` optio
.. code-block:: postgres .. code-block:: postgres
-- login as a user which has privileges on the private schemas -- login as a user wich has privileges on the private schemas
-- create a sample function -- create a sample function
create or replace function login(email text, pass text, out token text) as $$ create or replace function login(email text, pass text, out token text) as $$
+1 -18
View File
@@ -16,7 +16,7 @@ Supported PostgreSQL versions
============================= =============================
=============== ================================= =============== =================================
**Supported** PostgreSQL >= 14 **Supported** PostgreSQL >= 13
=============== ================================= =============== =================================
PostgREST works with all PostgreSQL versions still `officially supported <https://www.postgresql.org/support/versioning/>`_. PostgREST works with all PostgreSQL versions still `officially supported <https://www.postgresql.org/support/versioning/>`_.
@@ -181,23 +181,6 @@ If you want to have a visual overview of your API in your browser you can add sw
With this you can see the swagger-ui in your browser on port 8080. With this you can see the swagger-ui in your browser on port 8080.
.. _docker_cpu_contraint:
Docker Resource Constraints
---------------------------
PostgREST does not support ``--cpus`` `constraint option <https://docs.docker.com/engine/containers/resource_constraints/#configure-the-default-cfs-scheduler>`_.
As a workaround, you may use the `GHC RTS <https://ghc.gitlab.haskell.org/ghc/doc/users_guide/runtime_control.html#runtime-system-rts-options>`_ ``-N`` option. For instance, to limit it to 2 CPU cores, do:
.. code::
# Set environment variable GHCRTS set to "-N2"
docker run --rm -p 3000:3000 \
-e PGRST_DB_URI="postgres://app_user:password@10.0.0.10/postgres" \
-e GHCRTS="-N2"
postgrest/postgrest
.. _build_source: .. _build_source:
Building from Source Building from Source
-5
View File
@@ -4,7 +4,6 @@ API's
APIs APIs
APISIX APISIX
AST AST
async
aud aud
Auth Auth
auth auth
@@ -15,7 +14,6 @@ BOM
Bytea Bytea
Cardano Cardano
cd cd
CDNs
centric centric
CLI CLI
CMS CMS
@@ -32,7 +30,6 @@ DDL
DOM DOM
DSL DSL
DevOps DevOps
Dramatiq
dockerize dockerize
enum enum
Enums Enums
@@ -44,7 +41,6 @@ EveryLayout
filename filename
FreeBSD FreeBSD
fts fts
fullstack
GeoJSON GeoJSON
Github Github
Google Google
@@ -192,7 +188,6 @@ verifier
versioning versioning
Vondra Vondra
Vue Vue
webapp
webhooks webhooks
websearch websearch
Websockets Websockets
-1
View File
@@ -21,7 +21,6 @@ PostgREST exposes three database objects of a schema as resources: tables, views
api/aggregate_functions.rst api/aggregate_functions.rst
api/openapi.rst api/openapi.rst
api/preferences.rst api/preferences.rst
api/vary_header.rst
api/* api/*
.. raw:: html .. raw:: html
-20
View File
@@ -69,26 +69,6 @@ If the function doesn't modify the database, it will also run under the GET meth
The function parameter names match the JSON object keys in the POST case, for the GET case they match the query parameters ``?a=1&b=2``. The function parameter names match the JSON object keys in the POST case, for the GET case they match the query parameters ``?a=1&b=2``.
If the function is defined to have default values for the parameters then arguments for these parameters can be omitted in the request. For instance:
.. code-block:: postgres
CREATE FUNCTION greet_user(username TEXT DEFAULT 'guest')
RETURNS TEXT AS $$
SELECT 'Hello ' || username || '!';
$$ LANGUAGE SQL IMMUTABLE;
.. code-block:: bash
curl -i "http://localhost:3000/rpc/greet_user"
.. code-block:: http
HTTP/1.1 200 OK
Context-Type: application/json; charset=utf-8
"Hello guest!"
.. _function_single_json: .. _function_single_json:
Functions with an array of JSON objects Functions with an array of JSON objects
+1 -1
View File
@@ -15,7 +15,7 @@ Using these domains, :ref:`functions <functions>` can become handlers and `user-
.. important:: .. important::
- PostgREST vendor media types (``application/vnd.pgrst.plan``, ``application/vnd.pgrst.object`` and ``application/vnd.pgrst.array``) cannot be overridden. - PostgREST vendor media types (``application/vnd.pgrst.plan``, ``application/vnd.pgrst.object`` and ``application/vnd.pgrst.array``) cannot be overriden.
- Long media types like ``application/vnd.openxmlformats-officedocument.wordprocessingml.document`` cannot be expressed as domains since they surpass `PostgreSQL identifier length <https://www.postgresql.org/docs/current/limits.html#LIMITS-TABLE>`_. - Long media types like ``application/vnd.openxmlformats-officedocument.wordprocessingml.document`` cannot be expressed as domains since they surpass `PostgreSQL identifier length <https://www.postgresql.org/docs/current/limits.html#LIMITS-TABLE>`_.
For these you can use the :ref:`any_handler`. For these you can use the :ref:`any_handler`.
-4
View File
@@ -117,10 +117,6 @@ However, with ``handling=strict``, an invalid time zone preference will throw an
HTTP/1.1 400 Bad Request HTTP/1.1 400 Bad Request
.. note::
This feature requires querying `pg_timezone_names <https://www.postgresql.org/docs/current/view-pg-timezone-names.html>`_ during :ref:`schema_cache` load. If this is not desired, you can disable the feature with :ref:`db-timezone-enabled`.
.. _prefer_return: .. _prefer_return:
Return Representation Return Representation
+1 -1
View File
@@ -1244,7 +1244,7 @@ You can order the correlated arrays explicitly. For example, to order by the fil
.. warning:: .. warning::
Aliasing spread columns is recommended since JSON allows duplicate keys. Example: Aliasing spreaded columns is recommended since JSON allows duplicate keys. Example:
.. code-block:: bash .. code-block:: bash
-4
View File
@@ -5,10 +5,6 @@ Schemas
PostgREST can expose a single or multiple schema's tables, views and functions. The :ref:`active database role <roles>` must have the usage privilege on the schemas to access them. PostgREST can expose a single or multiple schema's tables, views and functions. The :ref:`active database role <roles>` must have the usage privilege on the schemas to access them.
.. important::
``pg_catalog`` and ``information_schema`` are not allowed in :ref:`db-schemas`. This is done to prevent leaking sensitive information and hence they cannot be accessed directly. If you wish to expose objects of these schemas, expose another schema that contains wrapper views or functions over ``pg_catalog`` or ``information_schema`` objects.
Single schema Single schema
------------- -------------
+1 -1
View File
@@ -639,7 +639,7 @@ However, it can work with surrogate primary keys (e.g. ``id serial primary key``
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/employees?columns=id,name,salary" \ curl "http://localhost:3000/employees?colums=id,name,salary" \
-X POST -H "Content-Type: application/json" \ -X POST -H "Content-Type: application/json" \
-H "Prefer: resolution=merge-duplicates, missing=default" \ -H "Prefer: resolution=merge-duplicates, missing=default" \
-d @- << EOF -d @- << EOF
+1 -1
View File
@@ -51,7 +51,7 @@ You can request table/columns with spaces in them by percent encoding the spaces
Reserved characters Reserved characters
~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
If filters include PostgREST reserved characters(``,``, ``.``, ``:``, ``*``, ``(``, ``)``) you'll have to surround them in percent encoded double quotes ``%22`` for correct processing. If filters include PostgREST reserved characters(``,``, ``.``, ``:``, ``()``) you'll have to surround them in percent encoded double quotes ``%22`` for correct processing.
Here ``Hebdon,John`` and ``Williams,Mary`` are values. Here ``Hebdon,John`` and ``Williams,Mary`` are values.
-16
View File
@@ -1,16 +0,0 @@
.. _vary_header:
Vary Header
===========
In order to assist caching proxies and CDNs, PostgREST includes a ``Vary`` header of value
``Accept, Prefer, Range`` in its responses which should fit most of the bills. As any other
response header, it's available for override
by updating ``response.headers`` GUC variable accordingly, for example:
.. code-block:: postgres
-- Override the Vary header to include Accept, Prefer and X-Test-Vary headers
perform set_config('response.headers', '[{"Vary": "Accept, Prefer, X-Test-Vary"}]', true);
In this case PostgREST will use provided value verbatim.
+1 -17
View File
@@ -217,7 +217,7 @@ It's recommended to leave the JWT cache enabled as our load tests indicate ~20%
- If the ``jwt-secret`` is changed and the config is reloaded, the JWT cache will reset. - If the ``jwt-secret`` is changed and the config is reloaded, the JWT cache will reset.
- JWTs that pass :ref:`jwt_signature` are cached, regardless if they pass :ref:`jwt_claims_validation`. We do this to ensure responses stays fast under common failure cases (such as expired JWTs). - JWTs that pass :ref:`jwt_signature` are cached, regardless if they pass :ref:`jwt_claims_validation`. We do this to ensure responses stays fast under common failure cases (such as expired JWTs).
- You can use the :ref:`server-timing_header` to see the performance benefit of JWT caching. - You can use the :ref:`server-timing_header` to see the peformance benefit of JWT caching.
.. _jwt_role_extract: .. _jwt_role_extract:
@@ -234,17 +234,6 @@ The DSL follows the `JSONPath <https://goessner.net/articles/JsonPath/>`_ expres
- ``==^`` selects the first array element that ends with the right operand - ``==^`` selects the first array element that ends with the right operand
- ``*==`` selects the first array element that contains the right operand - ``*==`` selects the first array element that contains the right operand
The selected role value can also be sliced using the slice operator ``[a:b]``. It is similar to `slice operator in python <https://docs.python.org/3/library/functions.html#slice>`_. Negative index values are also supported. The syntax is as:
- ``[a:b]`` take slice from index ``a`` up to ``b``
- ``[a:]`` take slice from index ``a`` to end
- ``[:b]`` take slice from start to index ``b``
- ``[:]`` select everything, no slicing
.. important::
Make sure that you are not taking a slice where the start index comes after the end index like ``[11:2]``. The result of this would be empty string and so no role would get selected.
Usage examples: Usage examples:
.. code:: bash .. code:: bash
@@ -266,11 +255,6 @@ Usage examples:
jwt-role-claim-key = ".postgrest.roles[?(@ ==^ \"hor\")]" jwt-role-claim-key = ".postgrest.roles[?(@ ==^ \"hor\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ *== \"utho\")]" jwt-role-claim-key = ".postgrest.roles[?(@ *== \"utho\")]"
# {"postgrest":{"wlcg": ["/groupa", "/groupb/"]}}
# skip the "/" character using slice operator
jwt-role-claim-key = ".postgrest.wlcg[0][1:]"
jwt-role-claim-key = ".postgrest.wlcg[1][1:-1]"
.. note:: .. note::
The string comparison operators are implemented as a custom extension to the JSPath and does not strictly follow the `RFC 9535 <https://www.rfc-editor.org/rfc/rfc9535.html>`_. The string comparison operators are implemented as a custom extension to the JSPath and does not strictly follow the `RFC 9535 <https://www.rfc-editor.org/rfc/rfc9535.html>`_.
+3 -45
View File
@@ -195,33 +195,6 @@ app.settings.*
The :code:`current_setting` function has `an optional boolean second <https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-SET>`_ argument to avoid it from raising an error if the value was not defined. Default values to :code:`app.settings` can then be given by combining this argument with :code:`coalesce` and :code:`nullif` : :code:`coalesce(nullif(current_setting('app.settings.my_custom_variable', true), ''), 'default value')`. The use of :code:`nullif` is necessary because if set in a transaction, the setting is sometimes not "rolled back" to :code:`null`. See also :ref:`this section <guc_req_headers_cookies_claims>` for more information on this behaviour. The :code:`current_setting` function has `an optional boolean second <https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-SET>`_ argument to avoid it from raising an error if the value was not defined. Default values to :code:`app.settings` can then be given by combining this argument with :code:`coalesce` and :code:`nullif` : :code:`coalesce(nullif(current_setting('app.settings.my_custom_variable', true), ''), 'default value')`. The use of :code:`nullif` is necessary because if set in a transaction, the setting is sometimes not "rolled back" to :code:`null`. See also :ref:`this section <guc_req_headers_cookies_claims>` for more information on this behaviour.
.. _client-error-verbosity:
client-error-verbosity
----------------------
=============== =======================
**Type** String
**Default** verbose
**Reloadable** Y
**Environment** PGRST_CLIENT_ERROR_VERBOSITY
**In-Database** pgrst.client_error_verbosity
=============== =======================
Specifies the verbosity of PostgREST errors. See :ref:`client_error_verbosity`.
.. code:: bash
# Return error "code", "message", "details" and "hint"
client-error-verbosity = "verbose"
# Return only "code" and "message"
client-error-verbosity = "minimal"
.. note::
This setting only affects client side error messages. Server side logs are not affected by this setting.
.. _db-aggregates-enabled: .. _db-aggregates-enabled:
db-aggregates-enabled db-aggregates-enabled
@@ -291,7 +264,7 @@ db-channel-enabled
When this is set to :code:`true`, the notification channel specified in :ref:`db-channel` is enabled. When this is set to :code:`true`, the notification channel specified in :ref:`db-channel` is enabled.
You should set this to ``false`` when using PostgreSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information. You should set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
.. _db-config: .. _db-config:
@@ -506,7 +479,7 @@ db-prepared-statements
When disabled, the generated queries will be parameterized (invulnerable to SQL injection) but they will not be prepared (cached in the database session). Not using prepared statements will noticeably decrease performance, so it's recommended to always have this setting enabled. When disabled, the generated queries will be parameterized (invulnerable to SQL injection) but they will not be prepared (cached in the database session). Not using prepared statements will noticeably decrease performance, so it's recommended to always have this setting enabled.
You should only set this to ``false`` when using PostgreSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information. You should only set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
.. _db-root-spec: .. _db-root-spec:
@@ -540,21 +513,6 @@ db-schemas
The list of database schemas to expose to clients. See :ref:`schemas`. The list of database schemas to expose to clients. See :ref:`schemas`.
.. _db-timezone-enabled:
db-timezone-enabled
-------------------
=============== =================================
**Type** Boolean
**Default** True
**Reloadable** Y
**Environment** PGRST_DB_TIMEZONE_ENABLED
**In-Database** pgrst.db_timezone_enabled
=============== =================================
Enables the use of :ref:`prefer_timezone` preference header. Disabled when set to ``false``.
.. _db-tx-end: .. _db-tx-end:
db-tx-end db-tx-end
@@ -568,7 +526,7 @@ db-tx-end
**In-Database** pgrst.db_tx_end **In-Database** pgrst.db_tx_end
=============== ================================= =============== =================================
Specifies how to terminate the database transactions. See :ref:`prefer_tx`. Specifies how to terminate the database transactions.
.. code:: bash .. code:: bash
-35
View File
@@ -473,38 +473,3 @@ For example, doing a request on a table with high count (say 30_000_000), we get
Proxy-Status: PostgREST; error=57014 Proxy-Status: PostgREST; error=57014
The PostgreSQL error code ``57014`` (`ref <https://www.postgresql.org/docs/current/errcodes-appendix.html>`_) reveals that the error is due to a short ``statement_timeout`` value. The PostgreSQL error code ``57014`` (`ref <https://www.postgresql.org/docs/current/errcodes-appendix.html>`_) reveals that the error is due to a short ``statement_timeout`` value.
.. _client_error_verbosity:
Client Error Verbosity
======================
For HTTP clients, the error verbosity can be set via :ref:`client-error-verbosity` config.
With ``verbose``, it returns ``code``, ``message``, ``details`` and ``hint``.
.. code:: bash
curl "localhost:3000/itemsxx"
.. code-block:: json
{
"code": "PGRST205",
"message": "Could not find the table 'public.itemsxx' in the schema cache",
"details": "Perhaps you meant the table 'public.items'",
"hint": null
}
With ``minimal``, just ``code`` and ``message`` is returned.
.. code:: bash
curl "localhost:3000/itemsxx"
.. code-block:: json
{
"code": "PGRST205",
"message": "Could not find the table 'public.itemsxx' in the schema cache"
}
+1 -3
View File
@@ -46,9 +46,7 @@ This will cause the :ref:`connection_pool` to connect to the read replica host a
.. note:: .. note::
- Under the hood, PostgREST forces `target_session_attrs=read-write <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS>`_ for the ``LISTEN`` session. Under the hood, PostgREST forces `target_session_attrs=read-write <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS>`_ for the ``LISTEN`` session.
So if you specify ``target_session_attrs=read-only`` as mentioned above, PostgREST will override it for the ``LISTEN``.
- ``read-only`` is only available on libpq >= 14, if you use a lower version you will get an error like ``invalid target_session_attrs value: \"read-only\"``.
.. _listener_automatic_recovery: .. _listener_automatic_recovery:
+1 -7
View File
@@ -3,16 +3,10 @@
Schema Cache Schema Cache
============ ============
PostgREST requires metadata from the database to provide a REST API that abstracts SQL details. One example of this is the interface for :ref:`resource_embedding`. PostgREST requires metadata from the database schema to provide a REST API that abstracts SQL details. One example of this is the interface for :ref:`resource_embedding`.
Getting this metadata requires expensive queries. To avoid repeating this work, PostgREST uses a schema cache. Getting this metadata requires expensive queries. To avoid repeating this work, PostgREST uses a schema cache.
.. note::
- Schema cache queries have been optimized over time to stay fast, even on complex databases. You can see a summary of their execution time in :ref:`pgrst_logging` and :ref:`metrics`.
- If the schema cache queries are slow, the most likely cause is *system catalog bloat*, see `issue#3212 <https://github.com/PostgREST/postgrest/issues/3212>`_ for more details.
- You can turn the :ref:`log-level` to ``debug`` to see the time of each schema cache query.
.. _schema_reloading: .. _schema_reloading:
Schema Cache Reloading Schema Cache Reloading
+1 -1
View File
@@ -221,7 +221,7 @@ Notice that the ``response.headers`` should be set to an *array* of single-key o
.. note:: .. note::
PostgREST provided headers such as ``Content-Type``, ``Location``, etc. can be overridden this way. Note that irrespective of overridden ``Content-Type`` response header, the content will still be converted to JSON, unless you use :ref:`custom_media`. PostgREST provided headers such as ``Content-Type``, ``Location``, etc. can be overriden this way. Note that irrespective of overridden ``Content-Type`` response header, the content will still be converted to JSON, unless you use :ref:`custom_media`.
.. _guc_resp_status: .. _guc_resp_status:
+4 -4
View File
@@ -1,7 +1,7 @@
# This file is auto-generated by postgrest-nixpkgs-upgrade # This file is auto-generated by postgrest-nixpkgs-upgrade
sphinx==9.1.0 sphinx==8.2.3
sphinx-copybutton==0.5.2 sphinx-copybutton==0.5.2
sphinx-rtd-dark-mode==1.3.0 sphinx-rtd-dark-mode==1.3.0
sphinx-rtd-theme==3.1.0 sphinx-rtd-theme==3.0.2
sphinx-tabs==3.5.0 sphinx-tabs==3.4.7
sphinxext-opengraph==0.13.0 sphinxext-opengraph==0.9.1
+1 -1
View File
@@ -22,7 +22,7 @@ Step 1. Install PostgreSQL
If you're already familiar with using PostgreSQL and have it installed on your system you can use the existing installation (see :ref:`pg-dependency` for minimum requirements). For this tutorial we'll describe how to use the database in Docker because database configuration is otherwise too complicated for a simple tutorial. If you're already familiar with using PostgreSQL and have it installed on your system you can use the existing installation (see :ref:`pg-dependency` for minimum requirements). For this tutorial we'll describe how to use the database in Docker because database configuration is otherwise too complicated for a simple tutorial.
If Docker is not installed, you can get it `here <https://www.docker.com/get-started>`_. Make sure that Docker service is `started <https://docs.docker.com/engine/daemon/start/#start-the-daemon-using-operating-system-utilities>`_. Next, let's pull and start the database image: If Docker is not installed, you can get it `here <https://www.docker.com/get-started>`_. Next, let's pull and start the database image:
.. code-block:: bash .. code-block:: bash
Generated
+4 -4
View File
@@ -2,16 +2,16 @@
"nodes": { "nodes": {
"nixpkgs": { "nixpkgs": {
"locked": { "locked": {
"lastModified": 1776949667, "lastModified": 1752006229,
"narHash": "sha256-GMSVw35Q+294GlrTUKlx087E31z7KurReQ1YHSKp5iw=", "narHash": "sha256-BeuAPwNM2RBc5bvUTb0j4GRs2yBkDeRCw/8Y3v9Xesc=",
"owner": "nixos", "owner": "nixos",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "01fbdeef22b76df85ea168fbfe1bfd9e63681b30", "rev": "c80edd02003fe3d8af527215a3ac069be9cfd47f",
"type": "github" "type": "github"
}, },
"original": { "original": {
"owner": "nixos", "owner": "nixos",
"ref": "nixpkgs-unstable", "ref": "nixpkgs-25.05-darwin",
"repo": "nixpkgs", "repo": "nixpkgs",
"type": "github" "type": "github"
} }
+1 -5
View File
@@ -2,7 +2,7 @@
description = "REST API for any Postgres database"; description = "REST API for any Postgres database";
inputs = { inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable"; nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-25.05-darwin";
}; };
nixConfig = { nixConfig = {
@@ -46,9 +46,5 @@
meta.description = "REST API for any Postgres database"; meta.description = "REST API for any Postgres database";
}; };
}); });
devShells = genSystems (postgrest: {
default = import ./shell.nix { inherit postgrest; };
});
}; };
} }
+6 -6
View File
@@ -91,11 +91,11 @@ postgrest-gen-ctags postgrest-watch
postgrest-gen-jwt postgrest-with-all postgrest-gen-jwt postgrest-with-all
postgrest-gen-secret postgrest-with-git postgrest-gen-secret postgrest-with-git
postgrest-git-hooks postgrest-with-pgrst postgrest-git-hooks postgrest-with-pgrst
postgrest-hsie-graph-modules postgrest-with-pg-14 postgrest-hsie-graph-modules postgrest-with-pg-13
postgrest-hsie-graph-symbols postgrest-with-pg-15 postgrest-hsie-graph-symbols postgrest-with-pg-14
postgrest-hsie-minimal-imports postgrest-with-pg-16 postgrest-hsie-minimal-imports postgrest-with-pg-15
postgrest-lint postgrest-with-pg-17 postgrest-lint postgrest-with-pg-16
postgrest-loadtest postgrest-with-pg-18 postgrest-loadtest postgrest-with-pg-17
postgrest-loadtest-against postgrest-with-slow-pg postgrest-loadtest-against postgrest-with-slow-pg
postgrest-loadtest-report postgrest-with-slow-postgrest postgrest-loadtest-report postgrest-with-slow-postgrest
postgrest-nixpkgs-upgrade postgrest-nixpkgs-upgrade
@@ -174,7 +174,7 @@ $ nix-shell --run "postgrest-with-all postgrest-test-spec"
# Run the tests against a specific version of PostgreSQL (use tab-completion in # Run the tests against a specific version of PostgreSQL (use tab-completion in
# nix-shell to see all available versions): # nix-shell to see all available versions):
$ nix-shell --run "postgrest-with-pg-17 postgrest-test-spec" $ nix-shell --run "postgrest-with-pg-13 postgrest-test-spec"
``` ```
+24 -6
View File
@@ -16,8 +16,11 @@ The following checklist guides you through the complete process in more detail.
## Upgrade the pinned version of `nixpkgs` ## Upgrade the pinned version of `nixpkgs`
The pinned version of [`nixpkgs`](https://github.com/NixOS/nixpkgs) is defined The pinned version of [`nixpkgs`](https://github.com/NixOS/nixpkgs) is defined
in [`flake.nix`](../flake.nix). To upgrade it, you can use a small utility in [`nix/nixpkgs-version.nix`](nixpkgs-version.nix). The pin refers directly to
script defined in [`nix/tools/nixpkgsTools.nix`](tools/nixpkgsTools.nix): a GitHub tarball for the given revision, which is more efficient than pulling
the complete Git repository. To upgrade it to the current `main` of
`nixpkgs`, you can use a small utility script defined in
[`nix/nixpkgs-update.nix`](nixpkgs-update.nix):
```bash ```bash
# From the root of the repository, enter nix-shell # From the root of the repository, enter nix-shell
@@ -27,12 +30,21 @@ nix-shell
postgrest-nixpkgs-upgrade postgrest-nixpkgs-upgrade
# Exit the nix-shell with Ctrl-d # Exit the nix-shell with Ctrl-d
``` ```
## Review overlays ## Review overlays
Check whether the individual [overlays](overlays) are still required. Check whether the individual [overlays](overlays) are still required.
## Check if patches are still required and update them as needed
We track a number of PostgREST-specific patches in [`nix/patches`](patches).
Check whether the pull-requests/issues linked in the
[`default.nix`](patches/default.nix) have progressed and remove/modify the
patches if they did. If conflicting changes occurred, you might have to rebase
the respective patches.
## Build everything ## Build everything
Using the PostgREST binary Nix cache is recommended. Install Using the PostgREST binary Nix cache is recommended. Install
@@ -46,19 +58,25 @@ errors, this is probably due to one of our patches. Try to fix them and re-run
## Update the PostgREST binary cache ## Update the PostgREST binary cache
If you have access to the PostgREST cachix project, you can push the If you have access to the PostgREST cachix signing key, you can push the
artifacts that you built locally to the binary cache. This will accelerate the artifacts that you built locally to the binary cache. This will accelerate the
CI builds and tests, sometimes dramatically. This might sometimes even be CI builds and tests, sometimes dramatically. This might sometimes even be
required to avoid build timeouts in CI. required to avoid build timeouts in CI.
You'll need to login with your token with `cachix authtoken <token>`. You'll need to set the `CACHIX_SIGNING_KEY` before proceeding, e.g. by creating
a file containing `export CACHIX_SIGNING_KEY=...` and sourcing that file, which
avoids having the secret in your shell history.
To push all new artifacts to Cachix, run: To push all new artifacts to Cachix, run:
``` ```
nix-store -qR --include-outputs $$(nix-instantiate) | cachix push postgrest
# Or, equivalently
nix-shell --run postgrest-push-cachix nix-shell --run postgrest-push-cachix
``` ```
The `postgrest-push-cachix` command will query the nix-store to list all The `nix-store` command will query the nix-store to list all dependencies and
dependencies and build artifacts of PostgREST. It will then push build artifacts of PostgREST. The `cachix` command will efficiently push
everything that is not yet cached to the binary cache. everything that is not yet cached to the binary cache.
+7 -20
View File
@@ -4,7 +4,6 @@
{-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RecordWildCards #-} {-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE TupleSections #-} {-# LANGUAGE TupleSections #-}
{-# LANGUAGE TypeApplications #-}
{-# LANGUAGE TypeFamilies #-} {-# LANGUAGE TypeFamilies #-}
-- | Haskell Imports and Exports tool -- | Haskell Imports and Exports tool
@@ -34,16 +33,13 @@ import Data.Function ((&))
import Data.List (intercalate) import Data.List (intercalate)
import Data.Maybe (catMaybes, mapMaybe) import Data.Maybe (catMaybes, mapMaybe)
import Data.Text (Text) import Data.Text (Text)
import GHC.Driver.Errors.Types (GhcMessage)
import GHC.Generics (Generic) import GHC.Generics (Generic)
import GHC.Hs.Extension (GhcPs) import GHC.Hs.Extension (GhcPs)
import GHC.Types.Error (Messages, defaultDiagnosticOpts, import GHC.Types.Error (getMessages)
getMessages)
import GHC.Types.Name.Occurrence (occNameString) import GHC.Types.Name.Occurrence (occNameString)
import GHC.Types.Name.Reader (rdrNameOcc) import GHC.Types.Name.Reader (rdrNameOcc)
import GHC.Unit.Module (moduleNameString) import GHC.Unit.Module.Name (moduleNameString)
import GHC.Utils.Error (pprMsgEnvelopeBagWithLoc) import GHC.Utils.Error (pprMsgEnvelopeBagWithLoc)
import GHC.Utils.Outputable (showSDocUnsafe)
import System.Directory.Recursive (getFilesRecursive) import System.Directory.Recursive (getFilesRecursive)
import System.Exit (exitFailure) import System.Exit (exitFailure)
@@ -202,7 +198,7 @@ sourceSymbols source = do
return $ concatMap (importSymbols source filepath . GHC.unLoc) hsmodImports return $ concatMap (importSymbols source filepath . GHC.unLoc) hsmodImports
-- | Parse a Haskell module -- | Parse a Haskell module
parseModule :: FilePath -> IO (GHC.HsModule GhcPs) parseModule :: FilePath -> IO GHC.HsModule
parseModule filepath = do parseModule filepath = do
result <- ExactPrint.parseModule GHC.Paths.libdir filepath result <- ExactPrint.parseModule GHC.Paths.libdir filepath
case result of case result of
@@ -210,13 +206,7 @@ parseModule filepath = do
return $ GHC.unLoc hsmod return $ GHC.unLoc hsmod
Left errs -> Left errs ->
fail $ "Errors with " <> show filepath <> ":\n " fail $ "Errors with " <> show filepath <> ":\n "
<> formatParseErrors errs <> show (pprMsgEnvelopeBagWithLoc $ getMessages errs)
formatParseErrors :: Messages GhcMessage -> String
formatParseErrors errs =
intercalate "\n "
. fmap showSDocUnsafe
$ pprMsgEnvelopeBagWithLoc (defaultDiagnosticOpts @GhcMessage) (getMessages errs)
-- | Symbols imported in an import declaration. -- | Symbols imported in an import declaration.
-- --
@@ -224,12 +214,9 @@ formatParseErrors errs =
-- only one item is returned. -- only one item is returned.
importSymbols :: FilePath -> FilePath -> GHC.ImportDecl GhcPs -> [ImportedSymbol] importSymbols :: FilePath -> FilePath -> GHC.ImportDecl GhcPs -> [ImportedSymbol]
importSymbols source filepath GHC.ImportDecl{..} = importSymbols source filepath GHC.ImportDecl{..} =
case ideclImportList of case ideclHiding of
Just (importListInterpretation, syms) -> Just (hiding, syms) ->
symbol (if importListInterpretation == GHC.EverythingBut then Hiding else Explicit) symbol (if hiding then Hiding else Explicit) . Just . GHC.unLoc <$> GHC.unLoc syms
. Just
. GHC.unLoc
<$> GHC.unLoc syms
Nothing -> Nothing ->
[ symbol Wildcard Nothing ] [ symbol Wildcard Nothing ]
where where
@@ -104,7 +104,7 @@ let
'' ''
+ lib.optionalString withTmpDir '' + lib.optionalString withTmpDir ''
tmpdir="$(${coreutils}/bin/mktemp -d --tmpdir=/tmp ${name}-XXX)" tmpdir="$(${coreutils}/bin/mktemp -d --tmpdir ${name}-XXX)"
# we keep the tmpdir when an error occurs for debugging # we keep the tmpdir when an error occurs for debugging
trap 'echo Temporary directory kept at: $tmpdir' ERR trap 'echo Temporary directory kept at: $tmpdir' ERR
+1
View File
@@ -3,4 +3,5 @@
checked-shell-script = import ./checked-shell-script; checked-shell-script = import ./checked-shell-script;
gitignore = import ./gitignore.nix; gitignore = import ./gitignore.nix;
haskell-packages = import ./haskell-packages.nix; haskell-packages = import ./haskell-packages.nix;
slocat = import ./slocat.nix;
} }
+19 -25
View File
@@ -47,43 +47,37 @@ let
# - To modify and try packages locally, see "Working with locally modified Haskell packages" in the Nix README. # - To modify and try packages locally, see "Working with locally modified Haskell packages" in the Nix README.
# Before upgrading fuzzyset to 0.3, check: https://github.com/PostgREST/postgrest/issues/3329 # Before upgrading fuzzyset to 0.3, check: https://github.com/PostgREST/postgrest/issues/3329
# jailbreak, because hspec limit for tests
fuzzyset = prev.fuzzyset_0_2_4; fuzzyset = prev.fuzzyset_0_2_4;
http2 = # TODO: Remove once available in nixpkgs haskellPackages
configurator-pg =
prev.callHackageDirect prev.callHackageDirect
{ {
pkg = "http2"; pkg = "configurator-pg";
ver = "5.4.0"; ver = "0.2.11";
sha256 = "sha256-PeEWVd61bQ8G7LvfLeXklzXqNJFaAjE2ecRMWJZESPE="; sha256 = "sha256-mtGtNawDJgz2ZIEVca+IYXVu4oNw9xsfJiYWAqAbbgc=";
} }
{ }; { };
http-semantics = # TODO: Remove once available in nixpkgs haskellPackages
streaming-commons =
prev.callHackageDirect prev.callHackageDirect
{ {
pkg = "http-semantics"; pkg = "streaming-commons";
ver = "0.4.0"; ver = "0.2.3.1";
sha256 = "sha256-rh0z51EKvsu5rQd5n2z3fSRjjEObouNZSBPO9NFYOF0="; sha256 = "sha256-Gl2eaJcWe1sxmcE/octWlH9uSnERguf+5H66K4fV87s=";
} }
{ }; { };
network-run = # Downgrade hasql and related packages while we are still on GHC 9.4 for the static build.
prev.callHackageDirect hasql = lib.dontCheck (lib.doJailbreak prev.hasql_1_6_4_4);
{ hasql-dynamic-statements = lib.dontCheck prev.hasql-dynamic-statements_0_3_1_5;
pkg = "network-run"; hasql-implicits = lib.dontCheck prev.hasql-implicits_0_1_1_3;
ver = "0.5.0"; hasql-notifications = lib.dontCheck prev.hasql-notifications_0_2_2_2;
sha256 = "sha256-vbXh+CzxDsGApjqHxCYf/ijpZtUCApFbkcF5gyN0THU="; hasql-pool = lib.dontCheck prev.hasql-pool_1_0_1;
} hasql-transaction = lib.dontCheck prev.hasql-transaction_1_1_0_1;
{ }; postgresql-binary = lib.dontCheck (lib.doJailbreak prev.postgresql-binary_0_13_1_3);
warp =
lib.dontCheck (prev.callHackageDirect
{
pkg = "warp";
ver = "3.4.13";
sha256 = "sha256-jmr8kpeSPDkOhT0i9PhozZapX4nUs92cOX7POAGb7/M=";
}
{ });
}; };
in in
{ {
+13
View File
@@ -0,0 +1,13 @@
_: prev:
{
slocat = prev.buildGoModule {
name = "slocat";
src = prev.fetchFromGitHub {
owner = "robx";
repo = "slocat";
rev = "52e7512c6029fd00483e41ccce260a3b4b9b3b64";
sha256 = "sha256-qn6luuh5wqREu3s8RfuMCP5PKdS2WdwPrujRYTpfzQ8=";
};
vendorHash = null;
};
}
+3 -3
View File
@@ -172,7 +172,7 @@ let
# The following unsets all GIT_ variables. # The following unsets all GIT_ variables.
unset "''${!GIT_@}" unset "''${!GIT_@}"
# shellcheck disable=SC2329 # shellcheck disable=SC2317
function restore () { function restore () {
ref="$(git stash list --format=format:%gD --grep "$1" -n1)" ref="$(git stash list --format=format:%gD --grep "$1" -n1)"
# this will avoid merge conflicts when applying the stash # this will avoid merge conflicts when applying the stash
@@ -205,7 +205,7 @@ let
${git}/bin/git add . ${git}/bin/git add .
;; ;;
pre-push) pre-push)
# Create a clean working tree without any uncommitted changes. # Create a clean working tree without any uncomitted changes.
${withTools.withGit} HEAD ${style}/bin/postgrest-lint ${withTools.withGit} HEAD ${style}/bin/postgrest-lint
;; ;;
esac esac
@@ -232,7 +232,7 @@ let
${style}/bin/postgrest-lint ${style}/bin/postgrest-lint
;; ;;
pre-push) pre-push)
# Create a clean working tree without any uncommitted changes. # Create a clean working tree without any uncomitted changes.
${withTools.withGit} HEAD ${check} ${withTools.withGit} HEAD ${check}
;; ;;
esac esac
+2 -2
View File
@@ -43,7 +43,7 @@ let
} }
if [ "$_arg_language" == "" ]; then if [ "$_arg_language" == "" ]; then
# clean previous build, otherwise some errors might be suppressed # clean previous build, otherwise some errors might be supressed
rm -rf "../.docs-build/html/default" rm -rf "../.docs-build/html/default"
if [ -d languages ]; then if [ -d languages ]; then
@@ -54,7 +54,7 @@ let
build html "../.docs-build/html/default" build html "../.docs-build/html/default"
else else
# clean previous build, otherwise some errors might be suppressed # clean previous build, otherwise some errors might be supressed
rm -rf "../.docs-build/html/$_arg_language" rm -rf "../.docs-build/html/$_arg_language"
# update and build specific locale, can be used to create new locale # update and build specific locale, can be used to create new locale
-52
View File
@@ -1,52 +0,0 @@
# Generate RSA JWK/public material for loadtests.
import argparse
import sys
from pathlib import Path
import jwcrypto.jwk as jwk
def main():
parser = argparse.ArgumentParser(
description="Generate RSA JWK/private key pair for loadtests"
)
parser.add_argument(
"--rsa",
dest="jwk_path",
metavar="JWK_PATH",
type=Path,
required=True,
help="Path to write the RSA JWK file",
)
parser.add_argument(
"--private-key",
dest="private_key_path",
metavar="PRIVATE_KEY_PATH",
type=Path,
required=True,
help="Path to write the RSA private key file",
)
args = parser.parse_args()
key = jwk.JWK.generate(kty="RSA", size=4096)
private_jwk, public_jwk = key.export_private(), key.export_public()
try:
args.jwk_path.write_text(public_jwk)
print(f"Created RSA JWK on {args.jwk_path}")
except OSError as e:
print(f"Error writing to {args.jwk_path}:{e}", file=sys.stderr)
sys.exit(1)
try:
args.private_key_path.write_text(private_jwk)
print(f"Created private key on {args.private_key_path}")
except OSError as e:
print(f"Error writing to {args.private_key_path}:{e}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
+25 -106
View File
@@ -12,24 +12,23 @@
# from an array # from an array
import time import time
import argparse import argparse
import subprocess
import sys import sys
import random import random
import jwt import jwt
import jwcrypto.jwk as jwk
from typing import Optional from typing import Optional
from pathlib import Path from pathlib import Path
from enum import Enum
URL = "http://postgrest" URL = "http://postgrest"
secret_key = b"reallyreallyreallyreallyverysafe" secret_key = b"reallyreallyreallyreallyverysafe"
key = jwk.JWK.generate(kty="RSA", size=4096)
private_key = jwt.algorithms.RSAAlgorithm.from_jwk(key.export_private())
public_key = key.export_public()
def generate_jwt(
now: int, def generate_jwt(now: int, exp_inc: Optional[int], is_hs: bool) -> str:
exp_inc: Optional[int],
rsa_private_key: Optional[jwt.algorithms.RSAAlgorithm],
) -> str:
"""Generate an HS256 or RS256 JWT""" """Generate an HS256 or RS256 JWT"""
payload = { payload = {
"sub": f"user_{random.getrandbits(32)}", "sub": f"user_{random.getrandbits(32)}",
@@ -40,72 +39,25 @@ def generate_jwt(
if exp_inc is not None: if exp_inc is not None:
payload["exp"] = now + exp_inc payload["exp"] = now + exp_inc
if rsa_private_key is None: k = secret_key if is_hs else private_key
key = secret_key alg = "HS256" if is_hs else "RS256"
alg = "HS256" return jwt.encode(payload, k, alg)
else:
key = rsa_private_key
alg = "RS256"
return jwt.encode(payload, key, alg)
HTTP_METHODS = ( def append_targets(lines: list[str], token: str):
"GET", lines.append(f"OPTIONS {URL}/authors_only")
"OPTIONS",
)
HttpMethod = Enum(
"HttpMethod",
{method: method for method in HTTP_METHODS},
type=str,
module=__name__,
)
def append_targets(lines: list[str], token: str, http_method: HttpMethod):
lines.append(f"{http_method.value} {URL}/authors_only")
lines.append(f"Authorization: Bearer {token}") lines.append(f"Authorization: Bearer {token}")
lines.append("") # blank line to separate requests lines.append("") # blank line to separate requests
# we use this to chain commands on loadtest.nix
def run_command(command: list[str]):
if not command:
return
if command[0] == "--":
command = command[1:]
if not command:
return
try:
subprocess.run(command, check=True)
except subprocess.CalledProcessError as exc:
print(
f"Error executing command {' '.join(command)}: {exc}",
file=sys.stderr,
)
sys.exit(exc.returncode)
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description="Generate Vegeta targets with unique JWTs" description="Generate Vegeta targets with unique JWTs"
) )
parser.add_argument( parser.add_argument(
"targets_path", "output",
metavar="TARGETS_PATH",
help="Path to write the generated targets file", help="Path to write the generated targets file",
) )
parser.add_argument(
"--private-key",
dest="private_key_path",
metavar="PRIVATE_KEY_PATH",
type=Path,
default=None,
help="Path to the RSA private key file (required when --rsa is used)",
)
parser.add_argument( parser.add_argument(
"--worst", "--worst",
dest="worst", dest="worst",
@@ -119,31 +71,14 @@ def main():
metavar="JWK_PATH", metavar="JWK_PATH",
type=Path, type=Path,
default=None, default=None,
help="Path to an existing RSA JWK file used for signing tokens", help="Path for generating a RSA JWK file to sign tokens with",
)
parser.add_argument(
"--method",
dest="http_method",
choices=list(HTTP_METHODS),
required=True,
help="HTTP method for the vegeta targets",
)
parser.add_argument(
"command",
nargs=argparse.REMAINDER,
help="Command (and arguments) to run after generating the targets",
) )
args = parser.parse_args() args = parser.parse_args()
rsa_private_key: Optional[jwt.algorithms.RSAAlgorithm] = None
is_hs = args.jwk_path is None is_hs = args.jwk_path is None
http_method = HttpMethod(args.http_method)
nsamples = 1000 nsamples = 1000
if is_hs: if is_hs:
ntargets = 200000 ntargets = 200000
else: else:
@@ -151,25 +86,12 @@ def main():
ntargets = 50000 ntargets = 50000
if not is_hs: if not is_hs:
if args.private_key_path is None:
parser.error("--rsa requires the --private-key option")
try: try:
private_key_data = args.private_key_path.read_text() with open(args.jwk_path, "w") as jwk:
except OSError as e: jwk.write(public_key)
err = ( print(f"Created {args.jwk_path} file containing the RSA JWK")
f"Error reading RSA private key from {args.private_key_path}: " except IOError as e:
f"{e}. Generate RSA materials first with gen_rsa_materials.py." print(f"Error writing to {args.jwk_path}: {e}", file=sys.stderr)
)
print(err, file=sys.stderr)
sys.exit(1)
try:
rsa_private_key = jwt.algorithms.RSAAlgorithm.from_jwk(private_key_data)
except Exception as exc: # broad exception to capture parsing errors
err = (
f"Error loading RSA private key from {args.private_key_path}: " f"{exc}"
)
print(err, file=sys.stderr)
sys.exit(1) sys.exit(1)
print(f"Generating {ntargets} targets...") print(f"Generating {ntargets} targets...")
@@ -188,7 +110,6 @@ def main():
if args.worst: if args.worst:
# estimated time takes to build and run postgrest itself # estimated time takes to build and run postgrest itself
build_run_postgrest_time = 2 build_run_postgrest_time = 2
# estimated time it takes to generate the targets file # estimated time it takes to generate the targets file
# the division numbers are tuned by hand # the division numbers are tuned by hand
if is_hs: # hs generation is much faster if is_hs: # hs generation is much faster
@@ -200,27 +121,25 @@ def main():
inc = build_run_postgrest_time + gen_time inc = build_run_postgrest_time + gen_time
for i in range(ntargets): for i in range(ntargets):
token = generate_jwt(now, inc + i // 1000, rsa_private_key) token = generate_jwt(now, inc + i // 1000, is_hs)
append_targets(lines, token, http_method) append_targets(lines, token)
else: else:
tokens = [generate_jwt(now, None, rsa_private_key) for _ in range(nsamples)] tokens = [generate_jwt(now, None, is_hs) for _ in range(nsamples)]
for i in range(ntargets): for i in range(ntargets):
token = random.choice(tokens) token = random.choice(tokens)
append_targets(lines, token, http_method) append_targets(lines, token)
try: try:
with open(args.targets_path, "w") as f: with open(args.output, "w") as f:
f.write("\n".join(lines)) f.write("\n".join(lines))
except IOError as e: except IOError as e:
print(f"Error writing to {args.targets_path}: {e}", file=sys.stderr) print(f"Error writing to {args.output}: {e}", file=sys.stderr)
sys.exit(1) sys.exit(1)
elapsed = time.time() - start_time elapsed = time.time() - start_time
print(f"Created {ntargets} targets", end=" ") print(f"Created {ntargets} targets", end=" ")
print(f"in {args.targets_path} ({elapsed:.2f}s)") print(f"in {args.output} ({elapsed:.2f}s)")
run_command(args.command)
if __name__ == "__main__": if __name__ == "__main__":
+32 -105
View File
@@ -18,8 +18,6 @@ let
]; ];
} }
'' ''
echo "Starting vegeta loadtest..."
# ARG_USE_ENV only adds defaults or docs for environment variables # ARG_USE_ENV only adds defaults or docs for environment variables
# We manually implement a required check here # We manually implement a required check here
# See also: https://github.com/matejak/argbash/issues/80 # See also: https://github.com/matejak/argbash/issues/80
@@ -44,9 +42,7 @@ let
"ARG_OPTIONAL_SINGLE([output], [o], [Filename to dump json output to], [./loadtest/result.bin])" "ARG_OPTIONAL_SINGLE([output], [o], [Filename to dump json output to], [./loadtest/result.bin])"
"ARG_OPTIONAL_SINGLE([testdir], [t], [Directory to load tests and fixtures from], [./test/load])" "ARG_OPTIONAL_SINGLE([testdir], [t], [Directory to load tests and fixtures from], [./test/load])"
"ARG_OPTIONAL_SINGLE([kind], [k], [Kind of loadtest], [mixed])" "ARG_OPTIONAL_SINGLE([kind], [k], [Kind of loadtest], [mixed])"
"ARG_OPTIONAL_SINGLE([method],, [HTTP method used for the jwt loadtests], [OPTIONS])" "ARG_TYPE_GROUP_SET([KIND], [KIND], [kind], [mixed,jwt-hs,jwt-hs-cache,jwt-hs-cache-worst,jwt-rsa,jwt-rsa-cache,jwt-rsa-cache-worst])"
"ARG_TYPE_GROUP_SET([KIND], [KIND], [kind], [mixed,errors,jwt-hs,jwt-hs-cache,jwt-hs-cache-worst,jwt-rsa,jwt-rsa-cache,jwt-rsa-cache-worst])"
"ARG_TYPE_GROUP_SET([METHOD], [METHOD], [method], [OPTIONS,GET])"
"ARG_OPTIONAL_SINGLE([monitor], [m], [Monitoring file], [./loadtest/result.csv])" "ARG_OPTIONAL_SINGLE([monitor], [m], [Monitoring file], [./loadtest/result.csv])"
"ARG_LEFTOVERS([additional vegeta arguments])" "ARG_LEFTOVERS([additional vegeta arguments])"
]; ];
@@ -63,127 +59,67 @@ let
export PGRST_DB_TX_END="rollback-allow-override" export PGRST_DB_TX_END="rollback-allow-override"
export PGRST_LOG_LEVEL="crit" export PGRST_LOG_LEVEL="crit"
export PGRST_JWT_SECRET="reallyreallyreallyreallyverysafe" export PGRST_JWT_SECRET="reallyreallyreallyreallyverysafe"
# set previous PGRST_JWT_CACHE_MAX_LIFETIME configuration so that
# load test works across branches
# TODO clean once PGRST_JWT_CACHE_MAX_ENTRIES merged and released
export PGRST_JWT_CACHE_MAX_LIFETIME="86400"
mkdir -p "$(dirname "$_arg_output")" mkdir -p "$(dirname "$_arg_output")"
abs_output="$(realpath "$_arg_output")" abs_output="$(realpath "$_arg_output")"
case "$_arg_kind" in case "$_arg_kind" in
jwt-hs) jwt-hs)
${genTargetsHS} "$_arg_testdir"/gen_targets.http
export PGRST_JWT_CACHE_MAX_ENTRIES="0" export PGRST_JWT_CACHE_MAX_ENTRIES="0"
export PGRST_JWT_CACHE_MAX_LIFETIME="0"
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
${withGenTargets} --method "$_arg_method" "$_arg_testdir"/gen_targets.http \
sh -c "cd \"$_arg_testdir\" && \
${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
;; ;;
jwt-hs-cache) jwt-hs-cache)
# shellcheck disable=SC2145 ${genTargetsHS} "$_arg_testdir"/gen_targets.http
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
${withGenTargets} --method "$_arg_method" "$_arg_testdir"/gen_targets.http \
sh -c "cd \"$_arg_testdir\" && \
${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
;; ;;
jwt-hs-cache-worst) jwt-hs-cache-worst)
# shellcheck disable=SC2145 ${genTargetsHS} --worst "$_arg_testdir"/gen_targets.http
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
${withGenTargets} --method "$_arg_method" --worst "$_arg_testdir"/gen_targets.http \
sh -c "cd \"$_arg_testdir\" && \
${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
;; ;;
jwt-rsa) jwt-rsa)
${genTargetsHS} --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_CACHE_MAX_ENTRIES="0" export PGRST_JWT_CACHE_MAX_ENTRIES="0"
export PGRST_JWT_CACHE_MAX_LIFETIME="0"
${genRsaMaterials} --rsa="$_arg_testdir"/gen_jwk.json --private-key="$_arg_testdir"/gen_private.json
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json" export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
${withGenTargets} --method "$_arg_method" --rsa="$_arg_testdir"/gen_jwk.json --private-key="$_arg_testdir"/gen_private.json "$_arg_testdir"/gen_targets.http \
sh -c "cd \"$_arg_testdir\" && \
${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
;; ;;
jwt-rsa-cache) jwt-rsa-cache)
${genRsaMaterials} --rsa="$_arg_testdir"/gen_jwk.json --private-key="$_arg_testdir"/gen_private.json ${genTargetsHS} --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json" export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
${withGenTargets} --method "$_arg_method" --rsa="$_arg_testdir"/gen_jwk.json --private-key="$_arg_testdir"/gen_private.json "$_arg_testdir"/gen_targets.http \
sh -c "cd \"$_arg_testdir\" && \
${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
;; ;;
jwt-rsa-cache-worst) jwt-rsa-cache-worst)
${genTargetsHS} --worst --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json" export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
${genRsaMaterials} --rsa="$_arg_testdir"/gen_jwk.json --private-key="$_arg_testdir"/gen_private.json
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
${withGenTargets} --method "$_arg_method" --worst --rsa="$_arg_testdir"/gen_jwk.json --private-key="$_arg_testdir"/gen_private.json "$_arg_testdir"/gen_targets.http \
sh -c "cd \"$_arg_testdir\" && \
${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
;; ;;
mixed) *)
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
sh -c "cd \"$_arg_testdir\" && \
${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
;;
# here we sleep purposefully to check how much memory does the schema cache consume in the final report
errors)
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/errors.sql \
${withTools.withPgrst} --timeout 2 --sleep 5 -m "$_arg_monitor" \
sh -c "cd \"$_arg_testdir\" && \
${runner} -targets errors.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
;; ;;
esac esac
${vegeta}/bin/vegeta report -type=text "$_arg_output" if [ "$_arg_kind" == "mixed" ]; then
# shellcheck disable=SC2145
if [ "$_arg_kind" != "errors" ]; then ${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
# fail in case 401 happened on jwt loadtests ${withTools.withSlowPg} \
unauthorized_count="$(${vegeta}/bin/vegeta report -type=json "$_arg_output" \ ${withTools.withPgrst} -m "$_arg_monitor" \
| ${jq}/bin/jq -r '.status_codes["401"] // 0')" ${withTools.withSlowPgrst} \
sh -c "cd \"$_arg_testdir\" && \
if [ "$unauthorized_count" -gt 0 ]; then ${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
last_unauthorized_body="$(${vegeta}/bin/vegeta encode "$_arg_output" \ else
| ${jq}/bin/jq -rn ' # shellcheck disable=SC2145
reduce inputs as $item (null; ${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
if $item.code == 401 then $item else . end ${withTools.withPgrst} -m "$_arg_monitor" \
) sh -c "cd \"$_arg_testdir\" && \
| if . == null then ${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
empty
else
(.body | @base64d)
end
')"
echo "loadtest failed: found $unauthorized_count 401 Unauthorized responses" >&2
if [ -n "$last_unauthorized_body" ]; then
printf '%s\n' "Last 401 response body:" >&2
printf '%s\n' "$last_unauthorized_body" >&2
fi
exit 1
fi
fi fi
${vegeta}/bin/vegeta report -type=text "$_arg_output"
''; '';
loadtestAgainst = loadtestAgainst =
@@ -314,22 +250,13 @@ let
| ${mergeMonitorResults} | ${mergeMonitorResults}
''; '';
withGenTargets = genTargetsHS =
writers.writePython3 "postgrest-with-gen-loadtest-targets" writers.writePython3 "postgrest-gen-loadtest-targets-hs"
{ {
libraries = [ python3Packages.pyjwt python3Packages.jwcrypto ]; libraries = [ python3Packages.pyjwt python3Packages.jwcrypto ];
doCheck = false; # postgrest-style conflicts with this
} }
(builtins.readFile ./generate_targets.py); (builtins.readFile ./generate_targets.py);
genRsaMaterials =
writers.writePython3 "postgrest-gen-rsa-materials"
{
libraries = [ python3Packages.jwcrypto ];
doCheck = false; # postgrest-style conflicts with this
}
(builtins.readFile ./gen_rsa_materials.py);
mergeMonitorResults = mergeMonitorResults =
writers.writePython3 "postgrest-merge-monitor-results" writers.writePython3 "postgrest-merge-monitor-results"
{ {
+1 -1
View File
@@ -68,7 +68,7 @@ let
bump devel bump devel
# The order of operations is important here: # The order of operations is important here:
# - bump devel is run and $A is updated to the new version # - bump devel is run and $A is upated to the new version
# - the branch is created with the new A, but the commit before the devel bump # - the branch is created with the new A, but the commit before the devel bump
# - the devel bump is committed # - the devel bump is committed
git branch "v$A" git branch "v$A"
+6 -8
View File
@@ -29,20 +29,19 @@ let
# Format Haskell files # Format Haskell files
# --vimgrep fixes a bug in ag: https://github.com/ggreer/the_silver_searcher/issues/753 # --vimgrep fixes a bug in ag: https://github.com/ggreer/the_silver_searcher/issues/753
# TODO: fix style issues in src/protolude and include it ${silver-searcher}/bin/ag -l --vimgrep -g '\.l?hs$' . \
${silver-searcher}/bin/ag -l --vimgrep -g '\.l?hs$' --ignore-dir=src/protolude . \
| xargs ${stylish-haskell}/bin/stylish-haskell -i | xargs ${stylish-haskell}/bin/stylish-haskell -i
# Format Python files # Format Python files
${black}/bin/black . 2> /dev/null ${black}/bin/black . 2> /dev/null
''; '';
# Script to check whether any uncommitted changes result from postgrest-style # Script to check whether any uncommited changes result from postgrest-style
styleCheck = styleCheck =
checkedShellScript checkedShellScript
{ {
name = "postgrest-style-check"; name = "postgrest-style-check";
docs = "Check whether postgrest-style results in any uncommitted changes."; docs = "Check whether postgrest-style results in any uncommited changes.";
workingDir = "/"; workingDir = "/";
} }
'' ''
@@ -84,18 +83,17 @@ let
# ruff has gaps in scanning for unused code, so we use vulture # ruff has gaps in scanning for unused code, so we use vulture
echo "Scanning python files for unused code..." echo "Scanning python files for unused code..."
${silver-searcher}/bin/ag -l --vimgrep -g '\.l?py$' . \ ${silver-searcher}/bin/ag -l --vimgrep -g '\.l?py$' . \
| xargs ${python3Packages.vulture}/bin/vulture --exclude docs/conf.py --min-confidence 80 | xargs ${python3Packages.vulture}/bin/vulture --exclude docs/conf.py
echo "Linting python files..." echo "Linting python files..."
${ruff}/bin/ruff check . ${ruff}/bin/ruff check .
echo "Checking consistency of import aliases in Haskell code..." echo "Checking consistency of import aliases in Haskell code..."
${hsie} check-aliases main src/PostgREST ${hsie} check-aliases main src
echo "Linting Haskell files..." echo "Linting Haskell files..."
# --vimgrep fixes a bug in ag: https://github.com/ggreer/the_silver_searcher/issues/753 # --vimgrep fixes a bug in ag: https://github.com/ggreer/the_silver_searcher/issues/753
# TODO: fix lint issues in src/protolude and include it ${silver-searcher}/bin/ag -l --vimgrep -g '\.l?hs$' . \
${silver-searcher}/bin/ag -l --vimgrep -g '\.l?hs$' --ignore-dir=src/protolude . \
| xargs ${hlint}/bin/hlint --hint=${hlintConfig} | xargs ${hlint}/bin/hlint --hint=${hlintConfig}
''; '';
+2 -1
View File
@@ -7,6 +7,7 @@
, glibcLocales ? null , glibcLocales ? null
, gnugrep , gnugrep
, hpc-codecov , hpc-codecov
, hostPlatform
, jq , jq
, lib , lib
, postgrest , postgrest
@@ -158,7 +159,7 @@ let
} }
( (
# required for `hpc markup` in CI; glibcLocales is not available e.g. on Darwin # required for `hpc markup` in CI; glibcLocales is not available e.g. on Darwin
lib.optionalString (stdenv.isLinux && stdenv.hostPlatform.libc == "glibc") '' lib.optionalString (stdenv.isLinux && hostPlatform.libc == "glibc") ''
export LOCALE_ARCHIVE="${glibcLocales}/lib/locale/locale-archive" export LOCALE_ARCHIVE="${glibcLocales}/lib/locale/locale-archive"
'' + '' +
+104 -47
View File
@@ -6,6 +6,7 @@
, postgresqlVersions , postgresqlVersions
, postgrest , postgrest
, python3Packages , python3Packages
, slocat
, writeText , writeText
, writers , writers
}: }:
@@ -24,7 +25,7 @@ let
"ARG_OPTIONAL_SINGLE([fixtures], [f], [SQL file to load fixtures from])" "ARG_OPTIONAL_SINGLE([fixtures], [f], [SQL file to load fixtures from])"
"ARG_POSITIONAL_SINGLE([command], [Command to run])" "ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])" "ARG_LEFTOVERS([command arguments])"
"ARG_USE_ENV([PGUSER], [Postgrest_Test_Authenticator], [Authenticator PG role])" # user is written in mixed case to implicitly test that it is being properly quoted in schema cache queries "ARG_USE_ENV([PGUSER], [postgrest_test_authenticator], [Authenticator PG role])"
"ARG_USE_ENV([PGDATABASE], [postgres], [PG database name])" "ARG_USE_ENV([PGDATABASE], [postgres], [PG database name])"
"ARG_USE_ENV([PGRST_DB_SCHEMAS], [test], [Schema to expose])" "ARG_USE_ENV([PGRST_DB_SCHEMAS], [test], [Schema to expose])"
"ARG_USE_ENV([PGTZ], [utc], [Timezone to use])" "ARG_USE_ENV([PGTZ], [utc], [Timezone to use])"
@@ -105,8 +106,7 @@ let
log "Starting replica on $replica_host" log "Starting replica on $replica_host"
# We set a low max_standby_streaming_delay to make the replication conflict fail faster in tests (otherwise it waits for the default 30s) pg_ctl -D "$replica_dir" -l "$replica_dblog" -w start -o "-F -c listen_addresses=\"\" -c hba_file=$HBA_FILE -k $replica_host -c log_statement=\"all\" " \
pg_ctl -D "$replica_dir" -l "$replica_dblog" -w start -o "-F -c listen_addresses=\"\" -c hba_file=$HBA_FILE -k $replica_host -c log_statement=\"all\" -c max_standby_streaming_delay=\"3s\" " \
>> "$setuplog" >> "$setuplog"
>&2 echo "${commandName}: Replica enabled. You can connect to it with: psql 'postgres:///$PGDATABASE?host=$replica_host' -U postgres" >&2 echo "${commandName}: Replica enabled. You can connect to it with: psql 'postgres:///$PGDATABASE?host=$replica_host' -U postgres"
@@ -117,7 +117,7 @@ let
export PGRST_DB_URI="postgres:///$PGDATABASE?host=$PGREPLICAHOST,$PGHOST" export PGRST_DB_URI="postgres:///$PGDATABASE?host=$PGREPLICAHOST,$PGHOST"
fi fi
# shellcheck disable=SC2329 # shellcheck disable=SC2317
stop () { stop () {
log "Stopping the database cluster..." log "Stopping the database cluster..."
pg_ctl stop --mode=immediate >> "$setuplog" pg_ctl stop --mode=immediate >> "$setuplog"
@@ -132,11 +132,9 @@ let
fi fi
if test "$_arg_fixtures"; then if test "$_arg_fixtures"; then
load_start=$SECONDS log "Loading fixtures under the postgres role..."
>&2 printf "${commandName}: Loading fixtures under the postgres role..."
psql -U postgres -v PGUSER="$PGUSER" -v ON_ERROR_STOP=1 -f "$_arg_fixtures" >> "$setuplog" psql -U postgres -v PGUSER="$PGUSER" -v ON_ERROR_STOP=1 -f "$_arg_fixtures" >> "$setuplog"
load_end=$((SECONDS - load_start)) log "Done. Running command..."
>&2 printf " done in %ss. Running command...\n" "$load_end"
fi fi
("$_arg_command" "''${_arg_leftovers[@]}") ("$_arg_command" "''${_arg_leftovers[@]}")
@@ -185,6 +183,81 @@ let
withPg = withTmpDb (builtins.head postgresqlVersions); withPg = withTmpDb (builtins.head postgresqlVersions);
withSlowPg =
checkedShellScript
{
name = "postgrest-with-slow-pg";
docs = "Run the given command with simulated high latency postgresql";
args =
[
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
"ARG_USE_ENV([PGHOST], [], [PG host (socket name)])"
"ARG_USE_ENV([PGDELAY], [0ms], [extra PG latency (duration)])"
];
positionalCompletion = "_command";
workingDir = "/";
redirectTixFiles = false;
withTmpDir = true;
}
''
delay="''${PGDELAY:-0ms}"
echo "delaying data to/from postgres by $delay"
REALPGHOST="$PGHOST"
export PGHOST="$tmpdir/socket"
mkdir -p "$PGHOST"
${slocat}/bin/slocat -delay "$delay" -src "$PGHOST/.s.PGSQL.5432" -dst "$REALPGHOST/.s.PGSQL.5432" &
SLOCAT_PID=$!
# shellcheck disable=SC2317
stop_slocat() {
kill "$SLOCAT_PID" || true
wait "$SLOCAT_PID" || true
}
trap stop_slocat EXIT
sleep 1 # should wait for socket file to appear instead
("$_arg_command" "''${_arg_leftovers[@]}")
'';
withSlowPgrst =
checkedShellScript
{
name = "postgrest-with-slow-postgrest";
docs = "Run the given command with simulated high latency postgrest";
args =
[
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
"ARG_USE_ENV([PGRST_SERVER_UNIX_SOCKET], [], [PostgREST host (socket name)])"
"ARG_USE_ENV([PGRST_DELAY], [0ms], [extra PostgREST latency (duration)])"
];
positionalCompletion = "_command";
workingDir = "/";
redirectTixFiles = false;
withTmpDir = true;
}
''
delay="''${PGRST_DELAY:-0ms}"
echo "delaying data to/from PostgREST by $delay"
REAL_PGRST_SERVER_UNIX_SOCKET="$PGRST_SERVER_UNIX_SOCKET"
export PGRST_SERVER_UNIX_SOCKET="$tmpdir/postgrest.socket"
${slocat}/bin/slocat -delay "$delay" -src "$PGRST_SERVER_UNIX_SOCKET" -dst "$REAL_PGRST_SERVER_UNIX_SOCKET" &
SLOCAT_PID=$!
# shellcheck disable=SC2317
stop_slocat() {
kill "$SLOCAT_PID" || true
wait "$SLOCAT_PID" || true
}
trap stop_slocat EXIT
sleep 1 # should wait for socket file to appear instead
("$_arg_command" "''${_arg_leftovers[@]}")
'';
withGit = withGit =
let let
name = "postgrest-with-git"; name = "postgrest-with-git";
@@ -279,21 +352,15 @@ let
''; '';
withPgrst = withPgrst =
let
commandName = "postgrest-with-pgrst";
in
checkedShellScript checkedShellScript
{ {
name = commandName; name = "postgrest-with-pgrst";
docs = "Build and run PostgREST and run <command> with PGRST_SERVER_UNIX_SOCKET set."; docs = "Build and run PostgREST and run <command> with PGRST_SERVER_UNIX_SOCKET set.";
args = args =
[ [
"ARG_POSITIONAL_SINGLE([command], [Command to run])" "ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])" "ARG_LEFTOVERS([command arguments])"
"ARG_OPTIONAL_SINGLE([monitor], [m], [Enable CPU and memory monitoring of the PostgREST process and output to the designated file as markdown])" "ARG_OPTIONAL_SINGLE([monitor], [m], [Enable CPU and memory monitoring of the PostgREST process and output to the designated file as markdown])"
"ARG_OPTIONAL_SINGLE([timeout], [t], [Maximum time to wait for PostgREST to be ready], [5])"
"ARG_OPTIONAL_SINGLE([sleep], [s], [Sleep time after PostgREST is ready, this is useful for monitoring])"
"ARG_USE_ENV([PGRST_CMD], [], [PostgREST executable to run])"
]; ];
positionalCompletion = "_command"; positionalCompletion = "_command";
workingDir = "/"; workingDir = "/";
@@ -303,34 +370,30 @@ let
'' ''
export PGRST_SERVER_UNIX_SOCKET="$tmpdir"/postgrest.socket export PGRST_SERVER_UNIX_SOCKET="$tmpdir"/postgrest.socket
if [ -z "''${PGRST_CMD:-}" ]; then rm -f result
rm -f result if [ -z "''${PGRST_BUILD_CABAL:-}" ]; then
build_start=$SECONDS echo -n "Building postgrest (nix)... "
if [ -z "''${PGRST_BUILD_CABAL:-}" ]; then # Using lib.getBin to also make this work with older checkouts, where .bin was not a thing, yet.
echo -n "${commandName}: Building postgrest (nix)... " nix-build -E 'with import ./. {}; pkgs.lib.getBin postgrestPackage' > "$tmpdir"/build.log 2>&1 || {
# Using lib.getBin to also make this work with older checkouts, where .bin was not a thing, yet. echo "failed, output:"
nix-build -E 'with import ./. {}; pkgs.lib.getBin postgrestPackage' > "$tmpdir"/build.log 2>&1 || { cat "$tmpdir"/build.log
echo "failed, output:" exit 1
cat "$tmpdir"/build.log }
exit 1 PGRST_CMD=$(echo ./result*/bin/postgrest)
} else
PGRST_CMD=$(echo ./result*/bin/postgrest) echo -n "Building postgrest (cabal)... "
else postgrest-build
echo -n "${commandName}: Building postgrest (cabal)... " PGRST_CMD=postgrest-run
postgrest-build
PGRST_CMD=postgrest-run
fi
build_end=$((SECONDS - build_start))
printf "done in %ss.\n" "$build_end"
fi fi
echo "done."
ver=$($PGRST_CMD ${legacyConfig} --version) ver=$($PGRST_CMD ${legacyConfig} --version)
echo -n "${commandName}: Starting $ver... " echo -n "Starting $ver... "
$PGRST_CMD ${legacyConfig} > "$tmpdir"/run.log 2>&1 & $PGRST_CMD ${legacyConfig} > "$tmpdir"/run.log 2>&1 &
pid=$! pid=$!
# shellcheck disable=SC2329 # shellcheck disable=SC2317
cleanup() { cleanup() {
# Send INT to all postgrest processes. # Send INT to all postgrest processes.
# Workaround to trigger dumping postgrest.prof for postgrest-profiled-run # Workaround to trigger dumping postgrest.prof for postgrest-profiled-run
@@ -344,25 +407,17 @@ let
} }
trap cleanup EXIT trap cleanup EXIT
wait_start=$SECONDS timeout -s TERM 5 ${waitForPgrstReady} || {
timeout -s TERM "$_arg_timeout" ${waitForPgrstReady} || {
echo "timed out, output:" echo "timed out, output:"
cat "$tmpdir"/run.log cat "$tmpdir"/run.log
exit 1 exit 1
} }
wait_duration=$((SECONDS - wait_start)) echo "done."
printf "done in %ss.\n" "$wait_duration"
echo "${commandName}: You can tail the server logs with: tail -f $tmpdir/run.log"
if [[ -n "$_arg_monitor" ]]; then if [[ -n "$_arg_monitor" ]]; then
${monitorPid} "$pid" > "$_arg_monitor" & ${monitorPid} "$pid" > "$_arg_monitor" &
fi fi
if [[ -n "$_arg_sleep" ]]; then
sleep "$_arg_sleep"
fi
("$_arg_command" "''${_arg_leftovers[@]}") ("$_arg_command" "''${_arg_leftovers[@]}")
''; '';
@@ -380,7 +435,9 @@ buildToolbox
inherit inherit
withGit withGit
withPgAll withPgAll
withPgrst; withPgrst
withSlowPg
withSlowPgrst;
} // builtins.listToAttrs ( } // builtins.listToAttrs (
# Create a `postgrest-with-pg-` for each PostgreSQL version # Create a `postgrest-with-pg-` for each PostgreSQL version
builtins.map (pg: { inherit (pg) name; value = withTmpDb pg; }) postgresqlVersions builtins.map (pg: { inherit (pg) name; value = withTmpDb pg; }) postgresqlVersions
+34 -89
View File
@@ -1,27 +1,28 @@
cabal-version: 3.0
name: postgrest name: postgrest
version: 15 version: 14.9
synopsis: REST API for any Postgres database synopsis: REST API for any Postgres database
description: Reads the schema of a PostgreSQL database and creates RESTful routes description: Reads the schema of a PostgreSQL database and creates RESTful routes
for tables, views, and functions, supporting all HTTP methods that security for tables, views, and functions, supporting all HTTP methods that security
permits. permits.
license: MIT license: MIT
license-file: LICENSE license-file: LICENSE
author: Joe Nelson, Adam Baker, Steve Chavez, Wolfgang Walther author: Joe Nelson, Adam Baker, Steve Chavez
maintainer: Steve Chavez <stevechavezast@gmail.com> maintainer: Steve Chavez <stevechavezast@gmail.com>
category: Executable, PostgreSQL, Network APIs category: Executable, PostgreSQL, Network APIs
homepage: https://postgrest.org homepage: https://postgrest.org
bug-reports: https://github.com/PostgREST/postgrest/issues bug-reports: https://github.com/PostgREST/postgrest/issues
build-type: Simple build-type: Simple
extra-source-files: CHANGELOG.md extra-source-files: CHANGELOG.md
cabal-version: >= 1.10
tested-with: tested-with:
-- nix
GHC == 9.4.8
-- cabal on Ubuntu -- cabal on Ubuntu
-- stack on FreeBSD, MacOS, Ubuntu, Windows -- stack on FreeBSD, MacOS, Ubuntu, Windows
, GHC == 9.10.3 , GHC == 9.6.7
-- cabal on Ubuntu -- cabal on Ubuntu
-- nix , GHC == 9.8.4
, GHC == 9.12.3
source-repository head source-repository head
type: git type: git
@@ -54,7 +55,6 @@ library
PostgREST.Client PostgREST.Client
PostgREST.Config PostgREST.Config
PostgREST.Config.Database PostgREST.Config.Database
PostgREST.Debounce
PostgREST.Config.JSPath PostgREST.Config.JSPath
PostgREST.Config.PgVersion PostgREST.Config.PgVersion
PostgREST.Config.Proxy PostgREST.Config.Proxy
@@ -66,7 +66,6 @@ library
PostgREST.SchemaCache.Representations PostgREST.SchemaCache.Representations
PostgREST.SchemaCache.Table PostgREST.SchemaCache.Table
PostgREST.Error PostgREST.Error
PostgREST.Error.Types
PostgREST.Listener PostgREST.Listener
PostgREST.Logger PostgREST.Logger
PostgREST.MainTx PostgREST.MainTx
@@ -82,7 +81,6 @@ library
PostgREST.Plan PostgREST.Plan
PostgREST.Plan.CallPlan PostgREST.Plan.CallPlan
PostgREST.Plan.MutatePlan PostgREST.Plan.MutatePlan
PostgREST.Plan.Negotiate
PostgREST.Plan.ReadPlan PostgREST.Plan.ReadPlan
PostgREST.Plan.Types PostgREST.Plan.Types
PostgREST.RangeQuery PostgREST.RangeQuery
@@ -98,9 +96,9 @@ library
PostgREST.Response.Performance PostgREST.Response.Performance
PostgREST.TimeIt PostgREST.TimeIt
PostgREST.Version PostgREST.Version
build-depends: base >= 4.9 && < 4.22 build-depends: base >= 4.9 && < 4.20
, HTTP >= 4000.3.7 && < 4000.5 , HTTP >= 4000.3.7 && < 4000.5
, Ranged-sets >= 0.3 && < 0.6 , Ranged-sets >= 0.3 && < 0.5
, aeson >= 2.0.3 && < 2.3 , aeson >= 2.0.3 && < 2.3
, auto-update >= 0.1.4 && < 0.3 , auto-update >= 0.1.4 && < 0.3
, base64-bytestring >= 1 && < 1.3 , base64-bytestring >= 1 && < 1.3
@@ -108,20 +106,17 @@ library
, case-insensitive >= 1.2 && < 1.3 , case-insensitive >= 1.2 && < 1.3
, cassava >= 0.4.5 && < 0.6 , cassava >= 0.4.5 && < 0.6
, configurator-pg >= 0.2.11 && < 0.3 , configurator-pg >= 0.2.11 && < 0.3
, containers >= 0.5.7 && < 0.8 , containers >= 0.5.7 && < 0.7
, cookie >= 0.4.2 && < 0.6 , cookie >= 0.4.2 && < 0.6
-- crypton 1.1.0 moved from `memory` to `ram`, which jose-jwt fails to build with right now.
-- should be possible to remove this once jose-jwt had a new release.
, crypton < 1.1.0
, directory >= 1.2.6 && < 1.4 , directory >= 1.2.6 && < 1.4
, either >= 4.4.1 && < 5.1 , either >= 4.4.1 && < 5.1
, extra >= 1.7.0 && < 2.0 , extra >= 1.7.0 && < 2.0
, fuzzyset >= 0.2.4 && < 0.3 , fuzzyset >= 0.2.4 && < 0.3
, hasql >= 1.9 && <= 1.9.3.1 , hasql >= 1.6.1.1 && < 1.7
, hasql-dynamic-statements >= 0.3.1 && <= 0.3.1.8 , hasql-dynamic-statements >= 0.3.1 && < 0.4
, hasql-notifications >= 0.2.4.0 && < 0.3 , hasql-notifications >= 0.2.2.2 && < 0.2.3
, hasql-pool >= 1.1 && <= 1.3.0.4 , hasql-pool >= 1.0.1 && < 1.1
, hasql-transaction >= 1.0.1 && <= 1.2.1 , hasql-transaction >= 1.0.1 && < 1.2
, http-client >= 0.7.19 && < 0.8 , http-client >= 0.7.19 && < 0.8
, http-types >= 0.12.2 && < 0.13 , http-types >= 0.12.2 && < 0.13
, insert-ordered-containers >= 0.2.2 && < 0.3 , insert-ordered-containers >= 0.2.2 && < 0.3
@@ -134,16 +129,17 @@ library
, network-uri >= 2.6.1 && < 2.8 , network-uri >= 2.6.1 && < 2.8
, optparse-applicative >= 0.13 && < 0.19 , optparse-applicative >= 0.13 && < 0.19
, parsec >= 3.1.11 && < 3.2 , parsec >= 3.1.11 && < 3.2
-- Technically unused, can be removed after updating to hasql >= 1.7
, postgresql-libpq >= 0.10 , postgresql-libpq >= 0.10
, prometheus-client >= 1.1.1 && < 1.2.0 , prometheus-client >= 1.1.1 && < 1.2.0
, protolude , protolude >= 0.3.1 && < 0.4
, regex-tdfa >= 1.2.2 && < 1.4 , regex-tdfa >= 1.2.2 && < 1.4
, retry >= 0.7.4 && < 0.10 , retry >= 0.7.4 && < 0.10
, scientific >= 0.3.4 && < 0.4 , scientific >= 0.3.4 && < 0.4
, streaming-commons >= 0.2.3.1 && < 0.3 , streaming-commons >= 0.2.3.1 && < 0.3
, swagger2 >= 2.4 && < 2.9 , swagger2 >= 2.4 && < 2.9
, text >= 1.2.2 && < 2.2 , text >= 1.2.2 && < 2.2
, time >= 1.6 && < 1.15 , time >= 1.6 && < 1.13
, unordered-containers >= 0.2.8 && < 0.3 , unordered-containers >= 0.2.8 && < 0.3
, unix-compat >= 0.5.4 && < 0.8 , unix-compat >= 0.5.4 && < 0.8
, vault >= 0.3.1.5 && < 0.4 , vault >= 0.3.1.5 && < 0.4
@@ -156,7 +152,7 @@ library
-- for unix sockets; this is tested in test/io/test_io.py. See -- for unix sockets; this is tested in test/io/test_io.py. See
-- https://github.com/kazu-yamamoto/logger/commit/3a71ca70afdbb93d4ecf0083eeba1fbbbcab3fc3 -- https://github.com/kazu-yamamoto/logger/commit/3a71ca70afdbb93d4ecf0083eeba1fbbbcab3fc3
, wai-logger >= 2.4.0 , wai-logger >= 2.4.0
, warp >= 3.4.13 && < 3.5 , warp >= 3.3.19 && < 3.5
, stm >= 2.5 && < 3 , stm >= 2.5 && < 3
, stm-hamt >= 1.2 && < 2 , stm-hamt >= 1.2 && < 2
, focus >= 1.0 && < 2 , focus >= 1.0 && < 2
@@ -180,64 +176,16 @@ library
build-depends: build-depends:
unix unix
library protolude
visibility: private
default-language: Haskell2010
default-extensions: NoImplicitPrelude
FlexibleContexts
MultiParamTypeClasses
OverloadedStrings
hs-source-dirs: src/protolude
exposed-modules: Protolude
Protolude.Applicative
Protolude.Base
Protolude.Bifunctor
Protolude.Bool
Protolude.CallStack
Protolude.Conv
Protolude.ConvertText
Protolude.Debug
Protolude.Either
Protolude.Error
Protolude.Exceptions
Protolude.Functor
Protolude.List
Protolude.Monad
Protolude.Panic
Protolude.Partial
Protolude.Safe
Protolude.Semiring
Protolude.Show
Protolude.Unsafe
build-depends: array >= 0.4 && < 0.6
, async >= 2.0 && < 2.3
, base >= 4.6 && < 4.22
, bytestring >= 0.10.8 && < 0.13
, containers >= 0.5.7 && < 0.8
, deepseq >= 1.3 && < 1.6
, ghc-prim >= 0.3 && < 0.14
, hashable >= 1.2 && < 1.6
, mtl >= 2.1 && < 2.4
, mtl-compat >= 0.2 && < 0.3
, stm >= 2.5 && < 3
, text >= 1.2.2 && < 2.2
, transformers >= 0.2 && < 0.7
, transformers-compat >= 0.4 && < 0.8
-- Protolude has some partial functions, so
-- it is fine to disable that specific warning
ghc-options: -Werror -Wall -fwarn-identities -Wno-x-partial
-fno-spec-constr -optP-Wno-nonportable-include-path
executable postgrest executable postgrest
default-language: Haskell2010 default-language: Haskell2010
default-extensions: OverloadedStrings default-extensions: OverloadedStrings
NoImplicitPrelude NoImplicitPrelude
hs-source-dirs: main hs-source-dirs: main
main-is: Main.hs main-is: Main.hs
build-depends: base >= 4.9 && < 4.22 build-depends: base >= 4.9 && < 4.20
, containers >= 0.5.7 && < 0.8 , containers >= 0.5.7 && < 0.7
, postgrest , postgrest
, protolude , protolude >= 0.3.1 && < 0.4
ghc-options: -threaded -rtsopts "-with-rtsopts=-N -I0 -qg" ghc-options: -threaded -rtsopts "-with-rtsopts=-N -I0 -qg"
-O2 -Werror -Wall -fwarn-identities -O2 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path -fno-spec-constr -optP-Wno-nonportable-include-path
@@ -292,9 +240,7 @@ test-suite spec
Feature.Query.PgSafeUpdateSpec Feature.Query.PgSafeUpdateSpec
Feature.Query.PlanSpec Feature.Query.PlanSpec
Feature.Query.PostGISSpec Feature.Query.PostGISSpec
Feature.Query.Preferences.HandlingSpec Feature.Query.PreferencesSpec
Feature.Query.Preferences.MaxAffectedSpec
Feature.Query.Preferences.TimezoneSpec
Feature.Query.QueryLimitedSpec Feature.Query.QueryLimitedSpec
Feature.Query.QuerySpec Feature.Query.QuerySpec
Feature.Query.RangeSpec Feature.Query.RangeSpec
@@ -310,16 +256,16 @@ test-suite spec
Feature.RollbackSpec Feature.RollbackSpec
Feature.RpcPreRequestGucsSpec Feature.RpcPreRequestGucsSpec
SpecHelper SpecHelper
build-depends: base >= 4.9 && < 4.22 build-depends: base >= 4.9 && < 4.20
, aeson >= 2.0.3 && < 2.3 , aeson >= 2.0.3 && < 2.3
, aeson-qq >= 0.8.1 && < 0.9 , aeson-qq >= 0.8.1 && < 0.9
, async >= 2.1.1 && < 2.3 , async >= 2.1.1 && < 2.3
, base64-bytestring >= 1 && < 1.3 , base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13 , bytestring >= 0.10.8 && < 0.13
, case-insensitive >= 1.2 && < 1.3 , case-insensitive >= 1.2 && < 1.3
, containers >= 0.5.7 && < 0.8 , containers >= 0.5.7 && < 0.7
, hasql-pool >= 1.0.1 && <= 1.3.0.4 , hasql-pool >= 1.0.1 && < 1.1
, hasql-transaction >= 1.0.1 && <= 1.2.1 , hasql-transaction >= 1.0.1 && < 1.2
, heredoc >= 0.2 && < 0.3 , heredoc >= 0.2 && < 0.3
, hspec >= 2.3 && < 2.12 , hspec >= 2.3 && < 2.12
, hspec-expectations >= 0.8.4 && < 0.9 , hspec-expectations >= 0.8.4 && < 0.9
@@ -333,7 +279,7 @@ test-suite spec
, postgrest , postgrest
, process >= 1.4.2 && < 1.7 , process >= 1.4.2 && < 1.7
, prometheus-client >= 1.1.1 && < 1.2.0 , prometheus-client >= 1.1.1 && < 1.2.0
, protolude , protolude >= 0.3.1 && < 0.4
, regex-tdfa >= 1.2.2 && < 1.4 , regex-tdfa >= 1.2.2 && < 1.4
, scientific >= 0.3.4 && < 0.4 , scientific >= 0.3.4 && < 0.4
, text >= 1.2.2 && < 2.2 , text >= 1.2.2 && < 2.2
@@ -358,12 +304,11 @@ test-suite observability
other-modules: ObsHelper other-modules: ObsHelper
Observation.JwtCache Observation.JwtCache
Observation.MetricsSpec Observation.MetricsSpec
Observation.SchemaCacheSpec build-depends: base >= 4.9 && < 4.20
build-depends: base >= 4.9 && < 4.22
, base64-bytestring >= 1 && < 1.3 , base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13 , bytestring >= 0.10.8 && < 0.13
, hasql-pool >= 1.0.1 && <= 1.3.0.4 , hasql-pool >= 1.0.1 && < 1.1
, hasql-transaction >= 1.0.1 && <= 1.2.1 , hasql-transaction >= 1.0.1 && < 1.2
, hspec >= 2.3 && < 2.12 , hspec >= 2.3 && < 2.12
, hspec-expectations >= 0.8.4 && < 0.9 , hspec-expectations >= 0.8.4 && < 0.9
, hspec-wai >= 0.10 && < 0.12 , hspec-wai >= 0.10 && < 0.12
@@ -372,7 +317,7 @@ test-suite observability
, jose-jwt >= 0.9.6 && < 0.11 , jose-jwt >= 0.9.6 && < 0.11
, postgrest , postgrest
, prometheus-client >= 1.1.1 && < 1.2.0 , prometheus-client >= 1.1.1 && < 1.2.0
, protolude , protolude >= 0.3.1 && < 0.4
, text >= 1.2.2 && < 2.2 , text >= 1.2.2 && < 2.2
, wai >= 3.2.1 && < 3.3 , wai >= 3.2.1 && < 3.3
ghc-options: -threaded -O0 -Werror -Wall -fwarn-identities ghc-options: -threaded -O0 -Werror -Wall -fwarn-identities
@@ -388,10 +333,10 @@ test-suite doctests
NoImplicitPrelude NoImplicitPrelude
hs-source-dirs: test/doc hs-source-dirs: test/doc
main-is: Main.hs main-is: Main.hs
build-depends: base >= 4.9 && < 4.22 build-depends: base >= 4.9 && < 4.20
, doctest >= 0.8 , doctest >= 0.8
, postgrest , postgrest
, pretty-simple , pretty-simple
, protolude , protolude >= 0.3.1 && < 0.4
ghc-options: -threaded -O0 -Werror -Wall -fwarn-identities ghc-options: -threaded -O0 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path -fno-spec-constr -optP-Wno-nonportable-include-path
+6 -5
View File
@@ -7,9 +7,11 @@
# We highly recommend that use the PostgREST binary cache by installing cachix # We highly recommend that use the PostgREST binary cache by installing cachix
# (https://app.cachix.org/) and running `cachix use postgrest`. # (https://app.cachix.org/) and running `cachix use postgrest`.
{ docker ? false { docker ? false
, postgrest ? import ./default.nix { }
}: }:
let let
postgrest =
import ./default.nix { };
inherit (postgrest) pkgs; inherit (postgrest) pkgs;
inherit (pkgs) lib; inherit (pkgs) lib;
@@ -35,7 +37,10 @@ lib.overrideDerivation postgrest.env (
buildInputs = buildInputs =
base.buildInputs ++ [ base.buildInputs ++ [
pkgs.cabal-install pkgs.cabal-install
pkgs.cabal2nix
pkgs.git
pkgs.postgresql pkgs.postgresql
pkgs.update-nix-fetchgit
postgrest.hsie.bin postgrest.hsie.bin
] ]
++ toolboxes; ++ toolboxes;
@@ -44,10 +49,6 @@ lib.overrideDerivation postgrest.env (
'' ''
export HISTFILE=.history export HISTFILE=.history
# Bypass proxy for all hosts, it prevents HTTP client failures used in test
# suites. See: https://github.com/PostgREST/postgrest/issues/4633 for more info
export NO_PROXY=*
source ${pkgs.bash-completion}/etc/profile.d/bash_completion.sh source ${pkgs.bash-completion}/etc/profile.d/bash_completion.sh
source ${pkgs.git}/share/git/contrib/completion/git-completion.bash source ${pkgs.git}/share/git/contrib/completion/git-completion.bash
source ${postgrest.hsie.bash-completion} source ${postgrest.hsie.bash-completion}
+1 -1
View File
@@ -64,7 +64,7 @@ data ApiRequest = ApiRequest {
, iPayload :: Maybe Payload -- ^ Data sent by client and used for mutation actions , iPayload :: Maybe Payload -- ^ Data sent by client and used for mutation actions
, iPreferences :: Preferences.Preferences -- ^ Prefer header values , iPreferences :: Preferences.Preferences -- ^ Prefer header values
, iQueryParams :: QueryParams.QueryParams , iQueryParams :: QueryParams.QueryParams
, iColumns :: S.Set FieldName -- ^ parsed columns from &columns parameter and payload , iColumns :: S.Set FieldName -- ^ parsed colums from &columns parameter and payload
, iHeaders :: [(ByteString, ByteString)] -- ^ HTTP request headers , iHeaders :: [(ByteString, ByteString)] -- ^ HTTP request headers
, iCookies :: [(ByteString, ByteString)] -- ^ Request Cookies , iCookies :: [(ByteString, ByteString)] -- ^ Request Cookies
, iPath :: ByteString -- ^ Raw request path , iPath :: ByteString -- ^ Raw request path
+37 -51
View File
@@ -22,7 +22,6 @@ import GHC.IO.Exception (IOErrorType (..))
import System.IO.Error (ioeGetErrorType) import System.IO.Error (ioeGetErrorType)
import Control.Monad.Except (liftEither) import Control.Monad.Except (liftEither)
import Control.Monad.Extra (whenJust)
import Data.Either.Combinators (mapLeft, whenLeft) import Data.Either.Combinators (mapLeft, whenLeft)
import Data.Maybe (fromJust) import Data.Maybe (fromJust)
import Data.String (IsString (..)) import Data.String (IsString (..))
@@ -61,16 +60,15 @@ import PostgREST.SchemaCache (SchemaCache (..))
import PostgREST.TimeIt (timeItT) import PostgREST.TimeIt (timeItT)
import PostgREST.Version (docsVersion, prettyVersion) import PostgREST.Version (docsVersion, prettyVersion)
import qualified Data.ByteString.Char8 as BS import qualified Data.ByteString.Char8 as BS
import qualified Data.List as L import qualified Data.List as L
import Data.Streaming.Network (bindPortTCP, import Data.Streaming.Network (bindPortTCP,
bindRandomPortTCP) bindRandomPortTCP)
import qualified Data.Text as T import qualified Data.Text as T
import qualified Network.HTTP.Types as HTTP import qualified Network.HTTP.Types as HTTP
import qualified Network.HTTP.Types.Header as HTTP (hVary) import qualified Network.Socket as NS
import qualified Network.Socket as NS import PostgREST.Unix (createAndBindDomainSocket)
import PostgREST.Unix (createAndBindDomainSocket) import Protolude hiding (Handler)
import Protolude hiding (Handler)
type Handler = ExceptT Error type Handler = ExceptT Error
@@ -80,10 +78,8 @@ run appState = do
AppState.schemaCacheLoader appState -- Loads the initial SchemaCache AppState.schemaCacheLoader appState -- Loads the initial SchemaCache
(mainSocket, adminSocket) <- initSockets conf (mainSocket, adminSocket) <- initSockets conf
let closeSockets = do
whenJust adminSocket NS.close Unix.installSignalHandlers observer (AppState.getMainThreadId appState) (AppState.schemaCacheLoader appState) (AppState.readInDbConfig False appState)
NS.close mainSocket
Unix.installSignalHandlers observer closeSockets (AppState.schemaCacheLoader appState) (AppState.readInDbConfig False appState)
Listener.runListener appState Listener.runListener appState
@@ -130,31 +126,30 @@ postgrest logLevel appState connWorker =
Logger.middleware logLevel Auth.getRole $ Logger.middleware logLevel Auth.getRole $
-- fromJust can be used, because the auth middleware will **always** add -- fromJust can be used, because the auth middleware will **always** add
-- some AuthResult to the vault. -- some AuthResult to the vault.
\req respond -> do \req respond -> case fromJust $ Auth.getResult req of
appConf@AppConfig{..} <- AppState.getConfig appState -- the config must be read again because it can reload Left err -> respond $ Error.errorResponseFor err
case fromJust $ Auth.getResult req of Right authResult -> do
Left err -> respond $ Error.errorResponseFor configClientErrorVerbosity err appConf <- AppState.getConfig appState -- the config must be read again because it can reload
Right authResult -> do maybeSchemaCache <- AppState.getSchemaCache appState
maybeSchemaCache <- AppState.getSchemaCache appState
let let
eitherResponse :: IO (Either Error Wai.Response) eitherResponse :: IO (Either Error Wai.Response)
eitherResponse = eitherResponse =
runExceptT $ postgrestResponse appState appConf maybeSchemaCache authResult req runExceptT $ postgrestResponse appState appConf maybeSchemaCache authResult req
response <- either (Error.errorResponseFor configClientErrorVerbosity) identity <$> eitherResponse response <- either Error.errorResponseFor identity <$> eitherResponse
-- Launch the connWorker when the connection is down. The postgrest -- Launch the connWorker when the connection is down. The postgrest
-- function can respond successfully (with a stale schema cache) before -- function can respond successfully (with a stale schema cache) before
-- the connWorker is done. However, when there's an empty schema cache -- the connWorker is done. However, when there's an empty schema cache
-- postgrest responds with the error `PGRST002`; this means that the schema -- postgrest responds with the error `PGRST002`; this means that the schema
-- cache is still loading, so we don't launch the connWorker here because -- cache is still loading, so we don't launch the connWorker here because
-- it would duplicate the loading process, e.g. https://github.com/PostgREST/postgrest/issues/3704 -- it would duplicate the loading process, e.g. https://github.com/PostgREST/postgrest/issues/3704
-- TODO: this process may be unnecessary when the Listener is enabled. Revisit once https://github.com/PostgREST/postgrest/issues/1766 is done -- TODO: this process may be unnecessary when the Listener is enabled. Revisit once https://github.com/PostgREST/postgrest/issues/1766 is done
when (isServiceUnavailable response && isJust maybeSchemaCache) connWorker when (isServiceUnavailable response && isJust maybeSchemaCache) connWorker
resp <- do resp <- do
delay <- AppState.getNextDelay appState delay <- AppState.getNextDelay appState
return $ addRetryHint delay response return $ addRetryHint delay response
respond resp respond resp
postgrestResponse postgrestResponse
:: AppState.AppState :: AppState.AppState
@@ -180,7 +175,7 @@ postgrestResponse appState conf@AppConfig{..} maybeSchemaCache authResult@AuthRe
timezones = dbTimezones sCache timezones = dbTimezones sCache
prefs = ApiRequest.userPreferences conf req timezones prefs = ApiRequest.userPreferences conf req timezones
(parseTime, apiReq@ApiRequest{..}) <- withTiming $ liftEither . mapLeft Error.ApiRequestErr $ ApiRequest.userApiRequest conf prefs req body (parseTime, apiReq@ApiRequest{..}) <- withTiming $ liftEither . mapLeft Error.ApiRequestError $ ApiRequest.userApiRequest conf prefs req body
(planTime, plan) <- withTiming $ liftEither $ Plan.actionPlan iAction conf apiReq sCache (planTime, plan) <- withTiming $ liftEither $ Plan.actionPlan iAction conf apiReq sCache
let mainQ = Query.mainQuery plan conf apiReq authResult configDbPreRequest let mainQ = Query.mainQuery plan conf apiReq authResult configDbPreRequest
@@ -201,7 +196,7 @@ postgrestResponse appState conf@AppConfig{..} maybeSchemaCache authResult@AuthRe
liftEither eitherResp liftEither eitherResp
(respTime, resp) <- withTiming $ do (respTime, resp) <- withTiming $ do
let response = Response.actionResponse txResult apiReq (T.decodeUtf8 prettyVersion, docsVersion) conf sCache let response = Response.actionResponse txResult apiReq (T.decodeUtf8 prettyVersion, docsVersion) conf sCache iSchema iNegotiatedByProfile
status' = either Error.status Response.pgrstStatus response status' = either Error.status Response.pgrstStatus response
-- TODO: see above obsQuery, only this obsQuery should remain after refactoring (because the QueryObs depends on the status) -- TODO: see above obsQuery, only this obsQuery should remain after refactoring (because the QueryObs depends on the status)
@@ -212,17 +207,7 @@ postgrestResponse appState conf@AppConfig{..} maybeSchemaCache authResult@AuthRe
where where
toWaiResponse :: ServerTiming -> Response.PgrstResponse -> Wai.Response toWaiResponse :: ServerTiming -> Response.PgrstResponse -> Wai.Response
toWaiResponse timing (Response.PgrstResponse st hdrs bod) = toWaiResponse timing (Response.PgrstResponse st hdrs bod) = Wai.responseLBS st (hdrs ++ ([serverTimingHeader timing | configServerTimingEnabled])) bod
Wai.responseLBS st (hdrs ++ serverTimingHeaders timing ++ [varyHeader | not $ varyHeaderPresent hdrs]) bod
serverTimingHeaders :: ServerTiming -> [HTTP.Header]
serverTimingHeaders timing = [serverTimingHeader timing | configServerTimingEnabled]
varyHeader :: HTTP.Header
varyHeader = (HTTP.hVary, "Accept, Prefer, Range")
varyHeaderPresent :: [HTTP.Header] -> Bool
varyHeaderPresent = any (\(h, _v) -> h == HTTP.hVary)
withTiming :: Handler IO a -> Handler IO (Maybe Double, a) withTiming :: Handler IO a -> Handler IO (Maybe Double, a)
withTiming f = if configServerTimingEnabled withTiming f = if configServerTimingEnabled
@@ -286,3 +271,4 @@ initSockets AppConfig{..} = do
Nothing -> pure Nothing Nothing -> pure Nothing
pure (sock, adminSock) pure (sock, adminSock)
+65 -78
View File
@@ -1,7 +1,6 @@
{-# LANGUAGE LambdaCase #-} {-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-} {-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE RecordWildCards #-} {-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE RecursiveDo #-}
module PostgREST.AppState module PostgREST.AppState
( AppState ( AppState
@@ -46,6 +45,7 @@ import PostgREST.Version (prettyVersion)
import Control.AutoUpdate (defaultUpdateSettings, mkAutoUpdate, import Control.AutoUpdate (defaultUpdateSettings, mkAutoUpdate,
updateAction) updateAction)
import Control.Debounce
import Control.Retry (RetryPolicy, RetryStatus (..), capDelay, import Control.Retry (RetryPolicy, RetryStatus (..), capDelay,
exponentialBackoff, retrying, exponentialBackoff, retrying,
rsPreviousDelay) rsPreviousDelay)
@@ -55,14 +55,13 @@ import Data.Time.Clock (UTCTime, getCurrentTime)
import PostgREST.Auth.JwtCache (JwtCacheState, update) import PostgREST.Auth.JwtCache (JwtCacheState, update)
import PostgREST.Config (AppConfig (..), import PostgREST.Config (AppConfig (..),
readAppConfig, addFallbackAppName,
toConnectionSettings) readAppConfig)
import PostgREST.Config.Database (queryDbSettings, import PostgREST.Config.Database (queryDbSettings,
queryPgVersion, queryPgVersion,
queryRoleSettings) queryRoleSettings)
import PostgREST.Config.PgVersion (PgVersion (..), import PostgREST.Config.PgVersion (PgVersion (..),
minimumPgVersion) minimumPgVersion)
import PostgREST.Debounce (makeDebouncer)
import PostgREST.SchemaCache (SchemaCache (..), import PostgREST.SchemaCache (SchemaCache (..),
querySchemaCache, querySchemaCache,
showSummary) showSummary)
@@ -78,7 +77,7 @@ data AppState = AppState
-- | Schema cache -- | Schema cache
, stateSchemaCache :: IORef (Maybe SchemaCache) , stateSchemaCache :: IORef (Maybe SchemaCache)
-- | The schema cache status -- | The schema cache status
, stateSCacheStatus :: SchemaCacheStatus , stateSCacheStatus :: IORef SchemaCacheStatus
-- | State of the LISTEN channel -- | State of the LISTEN channel
, stateIsListenerOn :: IORef Bool , stateIsListenerOn :: IORef Bool
-- | starts the connection worker with a debounce -- | starts the connection worker with a debounce
@@ -101,11 +100,11 @@ data AppState = AppState
, stateMetrics :: Metrics.MetricsState , stateMetrics :: Metrics.MetricsState
} }
-- | Schema cache status. -- | Schema cache status
-- Empty means pending and full means loaded. data SchemaCacheStatus
newtype SchemaCacheStatus = SchemaCacheStatus = SCLoaded
{ getSCStatusMVar :: MVar () | SCPending
} deriving Eq
init :: AppConfig -> IO AppState init :: AppConfig -> IO AppState
init conf@AppConfig{configLogLevel, configDbPoolSize} = do init conf@AppConfig{configLogLevel, configDbPoolSize} = do
@@ -116,17 +115,17 @@ init conf@AppConfig{configLogLevel, configDbPoolSize} = do
observer $ AppStartObs prettyVersion observer $ AppStartObs prettyVersion
pool <- initPool conf observer pool <- initPool conf observer
initWithPool pool conf loggerState metricsState observer initWithPool pool conf loggerState metricsState observer --{ stateSocketREST = sock, stateSocketAdmin = adminSock}
initWithPool :: SQL.Pool -> AppConfig -> Logger.LoggerState -> Metrics.MetricsState -> ObservationHandler -> IO AppState initWithPool :: SQL.Pool -> AppConfig -> Logger.LoggerState -> Metrics.MetricsState -> ObservationHandler -> IO AppState
initWithPool pool conf loggerState metricsState observer = mdo initWithPool pool conf loggerState metricsState observer = do
appState <- AppState pool appState <- AppState pool
<$> newIORef minimumPgVersion -- assume we're in a supported version when starting, this will be corrected on a later step <$> newIORef minimumPgVersion -- assume we're in a supported version when starting, this will be corrected on a later step
<*> newIORef Nothing <*> newIORef Nothing
<*> newSchemaCacheStatus <*> newIORef SCPending
<*> newIORef False <*> newIORef False
<*> makeDebouncer (retryingSchemaCacheLoad appState *> threadDelay 100000) -- 100ms cooldown <*> pure (pure ())
<*> newIORef conf <*> newIORef conf
<*> mkAutoUpdate defaultUpdateSettings { updateAction = getCurrentTime } <*> mkAutoUpdate defaultUpdateSettings { updateAction = getCurrentTime }
<*> myThreadId <*> myThreadId
@@ -137,53 +136,48 @@ initWithPool pool conf loggerState metricsState observer = mdo
<*> pure loggerState <*> pure loggerState
<*> pure metricsState <*> pure metricsState
return appState deb <-
let decisecond = 100000 in
mkDebounce defaultDebounceSettings
{ debounceAction = retryingSchemaCacheLoad appState
, debounceFreq = decisecond
, debounceEdge = leadingEdge -- runs the worker at the start and the end
}
return appState { debouncedSCacheLoader = deb}
destroy :: AppState -> IO () destroy :: AppState -> IO ()
destroy = destroyPool destroy = destroyPool
initPool :: AppConfig -> ObservationHandler -> IO SQL.Pool initPool :: AppConfig -> ObservationHandler -> IO SQL.Pool
initPool cfg@AppConfig{..} observer = do initPool AppConfig{..} observer = do
SQL.acquire $ SQL.settings SQL.acquire $ SQL.settings
[ SQL.size configDbPoolSize [ SQL.size configDbPoolSize
, SQL.acquisitionTimeout $ fromIntegral configDbPoolAcquisitionTimeout , SQL.acquisitionTimeout $ fromIntegral configDbPoolAcquisitionTimeout
, SQL.agingTimeout $ fromIntegral configDbPoolMaxLifetime , SQL.agingTimeout $ fromIntegral configDbPoolMaxLifetime
, SQL.idlenessTimeout $ fromIntegral configDbPoolMaxIdletime , SQL.idlenessTimeout $ fromIntegral configDbPoolMaxIdletime
, SQL.staticConnectionSettings $ toConnectionSettings identity cfg , SQL.staticConnectionSettings (toUtf8 $ addFallbackAppName prettyVersion configDbUri)
, SQL.observationHandler $ observer . HasqlPoolObs , SQL.observationHandler $ observer . HasqlPoolObs
] ]
-- | Run an action with a database connection. -- | Run an action with a database connection.
usePool :: AppState -> SQL.Session a -> IO (Either SQL.UsageError a) usePool :: AppState -> SQL.Session a -> IO (Either SQL.UsageError a)
usePool AppState{stateObserver=observer, stateMainThreadId=mainThreadId, ..} sess = do usePool AppState{stateObserver=observer, stateMainThreadId=mainThreadId, ..} sess = do
observer PoolRequest observer PoolRequest
res <- SQL.use statePool sess res <- SQL.use statePool sess
observer PoolRequestFullfilled observer PoolRequestFullfilled
whenLeft res (\case whenLeft res (\case
SQL.AcquisitionTimeoutUsageError -> SQL.AcquisitionTimeoutUsageError ->
observer PoolAcqTimeoutObs observer $ PoolAcqTimeoutObs SQL.AcquisitionTimeoutUsageError
err@(SQL.ConnectionUsageError e) -> err@(SQL.ConnectionUsageError e) ->
let failureMessage = BS.unpack $ fromMaybe mempty e in let failureMessage = BS.unpack $ fromMaybe mempty e in
when (("FATAL: password authentication failed" `isInfixOf` failureMessage) || ("no password supplied" `isInfixOf` failureMessage)) $ do when (("FATAL: password authentication failed" `isInfixOf` failureMessage) || ("no password supplied" `isInfixOf` failureMessage)) $ do
observer $ ExitDBFatalError ServerAuthError err observer $ ExitDBFatalError ServerAuthError err
killThread mainThreadId killThread mainThreadId
err@(SQL.SessionUsageError (SQL.QueryError tpl _ (SQL.ResultError resultErr))) -> err@(SQL.SessionUsageError (SQL.QueryError tpl _ (SQL.ResultError resultErr))) -> do
handleResultError err tpl resultErr
err@(SQL.SessionUsageError (SQL.PipelineError (SQL.ResultError resultErr))) ->
-- Passing the empty template will not work for schema cache queries, see TODO further below.
handleResultError err mempty resultErr
err@(SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ClientError _))) ->
-- An error on the client-side, usually indicates problems with connection
observer $ QueryErrorCodeHighObs err
SQL.SessionUsageError (SQL.PipelineError (SQL.ClientError _)) -> pure ()
)
return res
where
handleResultError err tpl resultErr = do
case resultErr of case resultErr of
SQL.UnexpectedResult{} -> do SQL.UnexpectedResult{} -> do
observer $ ExitDBFatalError ServerPgrstBug err observer $ ExitDBFatalError ServerPgrstBug err
@@ -216,17 +210,19 @@ usePool AppState{stateObserver=observer, stateMainThreadId=mainThreadId, ..} ses
SQL.ServerError{} -> SQL.ServerError{} ->
when (Error.status (Error.PgError False err) >= HTTP.status500) $ when (Error.status (Error.PgError False err) >= HTTP.status500) $
observer $ QueryErrorCodeHighObs err observer $ QueryErrorCodeHighObs err
err@(SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ClientError _))) ->
-- An error on the client-side, usually indicates problems wth connection
observer $ QueryErrorCodeHighObs err
)
return res
-- | Flush the connection pool so that any future use of the pool will -- | Flush the connection pool so that any future use of the pool will
-- use connections freshly established after this call. -- use connections freshly established after this call.
-- | Emits PoolFlushed observation
flushPool :: AppState -> IO () flushPool :: AppState -> IO ()
flushPool AppState{..} = do flushPool AppState{..} = SQL.release statePool
SQL.release statePool
stateObserver PoolFlushed
-- | Destroy the pool on shutdown. -- | Destroy the pool on shutdown.
-- | Differs from flushPool in not emiting PoolFlushed observation.
destroyPool :: AppState -> IO () destroyPool :: AppState -> IO ()
destroyPool AppState{..} = SQL.release statePool destroyPool AppState{..} = SQL.release statePool
@@ -282,15 +278,18 @@ putIsListenerOn = atomicWriteIORef . stateIsListenerOn
isLoaded :: AppState -> IO Bool isLoaded :: AppState -> IO Bool
isLoaded x = do isLoaded x = do
scacheLoaded <- isSchemaCacheLoaded x scacheStatus <- readIORef $ stateSCacheStatus x
connEstablished <- isConnEstablished x connEstablished <- isConnEstablished x
return $ scacheLoaded && connEstablished return $ scacheStatus == SCLoaded && connEstablished
isPending :: AppState -> IO Bool isPending :: AppState -> IO Bool
isPending x = do isPending x = do
scacheLoaded <- isSchemaCacheLoaded x scacheStatus <- readIORef $ stateSCacheStatus x
connEstablished <- isConnEstablished x connEstablished <- isConnEstablished x
return $ not scacheLoaded || not connEstablished return $ scacheStatus == SCPending || not connEstablished
putSCacheStatus :: AppState -> SchemaCacheStatus -> IO ()
putSCacheStatus = atomicWriteIORef . stateSCacheStatus
getObserver :: AppState -> ObservationHandler getObserver :: AppState -> ObservationHandler
getObserver = stateObserver getObserver = stateObserver
@@ -308,6 +307,9 @@ retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThrea
when (rsIterNumber > 0) $ do when (rsIterNumber > 0) $ do
let delay = fromMaybe 0 rsPreviousDelay `div` oneSecondInUs let delay = fromMaybe 0 rsPreviousDelay `div` oneSecondInUs
observer $ ConnectionRetryObs delay observer $ ConnectionRetryObs delay
putNextListenerDelay appState delay
flushPool appState
(,) <$> qPgVersion <*> (qInDbConfig *> qSchemaCache) (,) <$> qPgVersion <*> (qInDbConfig *> qSchemaCache)
) )
@@ -315,7 +317,7 @@ retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThrea
qPgVersion :: IO (Maybe PgVersion) qPgVersion :: IO (Maybe PgVersion)
qPgVersion = do qPgVersion = do
AppConfig{..} <- getConfig appState AppConfig{..} <- getConfig appState
pgVersion <- usePool appState queryPgVersion pgVersion <- usePool appState (queryPgVersion False) -- No need to prepare the query here, as the connection might not be established
case pgVersion of case pgVersion of
Left e -> do Left e -> do
observer $ QueryPgVersionError e observer $ QueryPgVersionError e
@@ -343,27 +345,24 @@ retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThrea
qSchemaCache = do qSchemaCache = do
conf@AppConfig{..} <- getConfig appState conf@AppConfig{..} <- getConfig appState
(resultTime, result) <- (resultTime, result) <-
timeItT $ usePool appState (SQL.transactionNoRetry SQL.ReadCommitted SQL.Read $ querySchemaCache conf) let transaction = if configDbPreparedStatements then SQL.transaction else SQL.unpreparedTransaction in
timeItT $ usePool appState (transaction SQL.ReadCommitted SQL.Read $ querySchemaCache conf)
case result of case result of
Left e -> do Left e -> do
markSchemaCachePending appState putSCacheStatus appState SCPending
putSchemaCache appState Nothing putSchemaCache appState Nothing
observer $ SchemaCacheErrorObs configDbSchemas configDbExtraSearchPath e observer $ SchemaCacheErrorObs configDbSchemas configDbExtraSearchPath e
return Nothing return Nothing
Right sCache -> do Right sCache -> do
-- IMPORTANT: While the pending schema cache state starts from running the above querySchemaCache, only at this stage we block API requests due to the usage of an -- IMPORTANT: While the pending schema cache state starts from running the above querySchemaCache, only at this stage we block API requests due to the usage of an
-- IORef on putSchemaCache. This is why schema cache status is marked as pending here to signal the Admin server (using isPending) that we're on a recovery state. -- IORef on putSchemaCache. This is why SCacheStatus is put at SCPending here to signal the Admin server (using isPending) that we're on a recovery state.
markSchemaCachePending appState putSCacheStatus appState SCPending
putSchemaCache appState $ Just sCache putSchemaCache appState $ Just sCache
(loadTime, summary) <- timeItT (evaluate $ showSummary sCache) observer $ SchemaCacheQueriedObs resultTime
-- Flush the pool after loading the schema cache to reset any stale session cache entries (t, _) <- timeItT $ observer $ SchemaCacheSummaryObs $ showSummary sCache
-- We do it after successfully querying the schema cache (because this can fail and during retries we would flush the pool repeatedly unnecessarily) observer $ SchemaCacheLoadedObs t
-- and after marking sCacheStatus as pending, putSCacheStatus appState SCLoaded
flushPool appState
observer $ SchemaCacheQueriedObs resultTime $ dbQueryTimings sCache
observer $ SchemaCacheLoadedObs loadTime summary
markSchemaCacheLoaded appState
return $ Just sCache return $ Just sCache
shouldRetry :: RetryStatus -> (Maybe PgVersion, Maybe SchemaCache) -> IO Bool shouldRetry :: RetryStatus -> (Maybe PgVersion, Maybe SchemaCache) -> IO Bool
@@ -379,18 +378,6 @@ retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThrea
oneSecondInUs = 1000000 -- one second in microseconds oneSecondInUs = 1000000 -- one second in microseconds
newSchemaCacheStatus :: IO SchemaCacheStatus
newSchemaCacheStatus = SchemaCacheStatus <$> newEmptyMVar
markSchemaCachePending :: AppState -> IO ()
markSchemaCachePending = void . tryTakeMVar . getSCStatusMVar . stateSCacheStatus
markSchemaCacheLoaded :: AppState -> IO ()
markSchemaCacheLoaded = void . (`tryPutMVar` ()) . getSCStatusMVar . stateSCacheStatus
isSchemaCacheLoaded :: AppState -> IO Bool
isSchemaCacheLoaded = fmap not . isEmptyMVar . getSCStatusMVar . stateSCacheStatus
-- | Reads the in-db config and reads the config file again -- | Reads the in-db config and reads the config file again
-- | We don't retry reading the in-db config after it fails immediately, because it could have user errors. We just report the error and continue. -- | We don't retry reading the in-db config after it fails immediately, because it could have user errors. We just report the error and continue.
readInDbConfig :: Bool -> AppState -> IO () readInDbConfig :: Bool -> AppState -> IO ()
@@ -399,7 +386,7 @@ readInDbConfig startingUp appState@AppState{stateObserver=observer} = do
pgVer <- getPgVersion appState pgVer <- getPgVersion appState
dbSettings <- dbSettings <-
if configDbConfig conf then do if configDbConfig conf then do
qDbSettings <- usePool appState (queryDbSettings (quoteQi <$> configDbPreConfig conf)) qDbSettings <- usePool appState (queryDbSettings (quoteQi <$> configDbPreConfig conf) (configDbPreparedStatements conf))
case qDbSettings of case qDbSettings of
Left e -> do Left e -> do
observer $ ConfigReadErrorObs e observer $ ConfigReadErrorObs e
@@ -409,7 +396,7 @@ readInDbConfig startingUp appState@AppState{stateObserver=observer} = do
pure mempty pure mempty
(roleSettings, roleIsolationLvl) <- (roleSettings, roleIsolationLvl) <-
if configDbConfig conf then do if configDbConfig conf then do
rSettings <- usePool appState (queryRoleSettings pgVer) rSettings <- usePool appState (queryRoleSettings pgVer (configDbPreparedStatements conf))
case rSettings of case rSettings of
Left e -> do Left e -> do
observer $ QueryRoleSettingsErrorObs e observer $ QueryRoleSettingsErrorObs e
+32 -6
View File
@@ -16,10 +16,14 @@ module PostgREST.Auth.Jwt
, parseClaims) where , parseClaims) where
import qualified Data.Aeson as JSON import qualified Data.Aeson as JSON
import qualified Data.Aeson.Key as K
import qualified Data.Aeson.KeyMap as KM
import qualified Data.ByteString as BS import qualified Data.ByteString as BS
import qualified Data.ByteString.Internal as BS import qualified Data.ByteString.Internal as BS
import qualified Data.ByteString.Lazy.Char8 as LBS import qualified Data.ByteString.Lazy.Char8 as LBS
import qualified Data.Scientific as Sci import qualified Data.Scientific as Sci
import qualified Data.Text as T
import qualified Data.Vector as V
import qualified Jose.Jwk as JWT import qualified Jose.Jwk as JWT
import qualified Jose.Jwt as JWT import qualified Jose.Jwt as JWT
@@ -29,11 +33,12 @@ import Data.Text ()
import Data.Time.Clock (UTCTime, nominalDiffTimeToSeconds) import Data.Time.Clock (UTCTime, nominalDiffTimeToSeconds)
import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds) import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds)
import PostgREST.Auth.Types (AuthResult (..)) import PostgREST.Auth.Types (AuthResult (..))
import PostgREST.Config (AppConfig (..), audMatchesCfg) import PostgREST.Config (AppConfig (..), FilterExp (..), JSPath,
import PostgREST.Config.JSPath (walkJSPath) JSPathExp (..), audMatchesCfg)
import PostgREST.Error (Error (..), JwtClaimsError (..), import PostgREST.Error (Error (..),
JwtDecodeError (..), JwtError (..)) JwtClaimsError (AudClaimNotStringOrArray, ExpClaimNotNumber, IatClaimNotNumber, JWTExpired, JWTIssuedAtFuture, JWTNotInAudience, JWTNotYetValid, NbfClaimNotNumber, ParsingClaimsFailed),
JwtDecodeError (..), JwtError (..))
import Data.Aeson ((.:?)) import Data.Aeson ((.:?))
import Data.Aeson.Types (parseMaybe) import Data.Aeson.Types (parseMaybe)
@@ -90,10 +95,13 @@ checkForErrors time audMatches = mconcat
parseToken :: (MonadError Error m, MonadIO m) => JwkSet -> ByteString -> m JWT.JwtContent parseToken :: (MonadError Error m, MonadIO m) => JwkSet -> ByteString -> m JWT.JwtContent
parseToken _ "" = throwError $ JwtErr $ JwtDecodeErr EmptyAuthHeader parseToken _ "" = throwError $ JwtErr $ JwtDecodeErr EmptyAuthHeader
parseToken secret tkn = do parseToken secret tkn = do
-- secret <- liftEither . maybeToRight (JwtErr JwtSecretMissing) $ configJWKS
tknWith3Parts <- hasThreeParts tkn tknWith3Parts <- hasThreeParts tkn
eitherContent <- liftIO $ JWT.decode (JWT.keys secret) Nothing tknWith3Parts eitherContent <- liftIO $ JWT.decode (JWT.keys secret) Nothing tknWith3Parts
liftEither . mapLeft (JwtErr . jwtDecodeError) $ eitherContent liftEither . mapLeft (JwtErr . jwtDecodeError) $ eitherContent
--liftEither $ mapLeft JwtErr $ verifyClaims content
where where
--hasThreeParts :: ByteString -> Either Error ByteString
hasThreeParts token = case length $ BS.split (BS.c2w '.') token of hasThreeParts token = case length $ BS.split (BS.c2w '.') token of
3 -> pure token 3 -> pure token
n -> throwError $ JwtErr $ JwtDecodeErr $ UnexpectedParts n n -> throwError $ JwtErr $ JwtDecodeErr $ UnexpectedParts n
@@ -116,10 +124,28 @@ parseClaims cfg@AppConfig{configJwtRoleClaimKey, configDbAnonRole} time mclaims
role <- liftEither . maybeToRight (JwtErr JwtTokenRequired) $ role <- liftEither . maybeToRight (JwtErr JwtTokenRequired) $
unquoted <$> walkJSPath (Just $ JSON.Object mclaims) configJwtRoleClaimKey <|> configDbAnonRole unquoted <$> walkJSPath (Just $ JSON.Object mclaims) configJwtRoleClaimKey <|> configDbAnonRole
pure AuthResult pure AuthResult
{ authClaims = mclaims { authClaims = mclaims & KM.insert "role" (JSON.toJSON $ decodeUtf8 role)
, authRole = role , authRole = role
} }
where where
walkJSPath :: Maybe JSON.Value -> JSPath -> Maybe JSON.Value
walkJSPath x [] = x
walkJSPath (Just (JSON.Object o)) (JSPKey key:rest) = walkJSPath (KM.lookup (K.fromText key) o) rest
walkJSPath (Just (JSON.Array ar)) (JSPIdx idx:rest) = walkJSPath (ar V.!? idx) rest
walkJSPath (Just (JSON.Array ar)) [JSPFilter (EqualsCond txt)] = findFirstMatch (==) txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (NotEqualsCond txt)] = findFirstMatch (/=) txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (StartsWithCond txt)] = findFirstMatch T.isPrefixOf txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (EndsWithCond txt)] = findFirstMatch T.isSuffixOf txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (ContainsCond txt)] = findFirstMatch T.isInfixOf txt ar
walkJSPath _ _ = Nothing
findFirstMatch matchWith pattern = foldr checkMatch Nothing
where
checkMatch (JSON.String txt) acc
| pattern `matchWith` txt = Just $ JSON.String txt
| otherwise = acc
checkMatch _ acc = acc
unquoted :: JSON.Value -> BS.ByteString unquoted :: JSON.Value -> BS.ByteString
unquoted (JSON.String t) = encodeUtf8 t unquoted (JSON.String t) = encodeUtf8 t
unquoted v = LBS.toStrict $ JSON.encode v unquoted v = LBS.toStrict $ JSON.encode v
+1 -3
View File
@@ -6,9 +6,7 @@ import qualified Data.Aeson as JSON
import qualified Data.Aeson.KeyMap as KM import qualified Data.Aeson.KeyMap as KM
import qualified Data.ByteString as BS import qualified Data.ByteString as BS
-- | -- | Parse result for JWT Claims
-- Parse and store result for JWT Claims. Can be accessed in
-- db through GUCs (for RLS etc)
data AuthResult = AuthResult data AuthResult = AuthResult
{ authClaims :: KM.KeyMap JSON.Value { authClaims :: KM.KeyMap JSON.Value
, authRole :: BS.ByteString , authRole :: BS.ByteString
+3 -1
View File
@@ -62,7 +62,9 @@ dumpSchema :: AppState -> IO LBS.ByteString
dumpSchema appState = do dumpSchema appState = do
conf@AppConfig{..} <- AppState.getConfig appState conf@AppConfig{..} <- AppState.getConfig appState
result <- result <-
AppState.usePool appState (SQL.transactionNoRetry SQL.ReadCommitted SQL.Read $ querySchemaCache conf) let transaction = if configDbPreparedStatements then SQL.transaction else SQL.unpreparedTransaction in
AppState.usePool appState
(transaction SQL.ReadCommitted SQL.Read $ querySchemaCache conf)
case result of case result of
Left e -> do Left e -> do
let observer = AppState.getObserver appState let observer = AppState.getObserver appState
+22 -78
View File
@@ -9,7 +9,6 @@ Description : Manages PostgREST configuration type and parser.
{-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE RecordWildCards #-} {-# LANGUAGE RecordWildCards #-}
{-# OPTIONS_GHC -fno-warn-type-defaults #-} {-# OPTIONS_GHC -fno-warn-type-defaults #-}
{-# LANGUAGE NamedFieldPuns #-}
module PostgREST.Config module PostgREST.Config
( AppConfig (..) ( AppConfig (..)
@@ -28,25 +27,21 @@ module PostgREST.Config
, parseSecret , parseSecret
, addFallbackAppName , addFallbackAppName
, addTargetSessionAttrs , addTargetSessionAttrs
, toConnectionSettings
, exampleConfigFile , exampleConfigFile
, audMatchesCfg , audMatchesCfg
, Verbosity (..)
) where ) where
import qualified Data.Aeson as JSON import qualified Data.Aeson as JSON
import qualified Data.ByteString as BS import qualified Data.ByteString as BS
import qualified Data.ByteString.Base64 as B64 import qualified Data.ByteString.Base64 as B64
import qualified Data.CaseInsensitive as CI import qualified Data.CaseInsensitive as CI
import qualified Data.Configurator as C import qualified Data.Configurator as C
import qualified Data.Map.Strict as M import qualified Data.Map.Strict as M
import qualified Data.String as S import qualified Data.String as S
import qualified Data.Text as T import qualified Data.Text as T
import qualified Data.Text.Encoding as T import qualified Data.Text.Encoding as T
import qualified Hasql.Connection.Setting as SQL import qualified Jose.Jwa as JWT
import qualified Hasql.Connection.Setting.Connection as SQL import qualified Jose.Jwk as JWT
import qualified Jose.Jwa as JWT
import qualified Jose.Jwk as JWT
import Control.Monad (fail) import Control.Monad (fail)
import Data.Either.Combinators (mapLeft) import Data.Either.Combinators (mapLeft)
@@ -68,18 +63,16 @@ import PostgREST.Config.JSPath (FilterExp (..), JSPath,
pRoleClaimKey) pRoleClaimKey)
import PostgREST.Config.Proxy (Proxy (..), import PostgREST.Config.Proxy (Proxy (..),
isMalformedProxyUri, toURI) isMalformedProxyUri, toURI)
import PostgREST.SchemaCache.Identifiers (QualifiedIdentifier (..), import PostgREST.SchemaCache.Identifiers (QualifiedIdentifier, dumpQi,
toQi) toQi)
import PostgREST.Version (prettyVersion) import Protolude hiding (Proxy, toList)
import Protolude hiding (Proxy, toList)
audMatchesCfg :: AppConfig -> Text -> Bool audMatchesCfg :: AppConfig -> Text -> Bool
audMatchesCfg = maybe (const True) (==) . configJwtAudience audMatchesCfg = maybe (const True) (==) . configJwtAudience
data AppConfig = AppConfig data AppConfig = AppConfig
{ configAppSettings :: [(Text, Text)] { configAppSettings :: [(Text, Text)]
, configClientErrorVerbosity :: Verbosity
, configDbAggregates :: Bool , configDbAggregates :: Bool
, configDbAnonRole :: Maybe BS.ByteString , configDbAnonRole :: Maybe BS.ByteString
, configDbChannel :: Text , configDbChannel :: Text
@@ -99,7 +92,6 @@ data AppConfig = AppConfig
, configDbSchemas :: NonEmpty Text , configDbSchemas :: NonEmpty Text
, configDbConfig :: Bool , configDbConfig :: Bool
, configDbPreConfig :: Maybe QualifiedIdentifier , configDbPreConfig :: Maybe QualifiedIdentifier
, configDbTimezoneEnabled :: Bool
, configDbTxAllowOverride :: Bool , configDbTxAllowOverride :: Bool
, configDbTxRollbackAll :: Bool , configDbTxRollbackAll :: Bool
, configDbUri :: Text , configDbUri :: Text
@@ -142,15 +134,6 @@ dumpLogLevel = \case
LogInfo -> "info" LogInfo -> "info"
LogDebug -> "debug" LogDebug -> "debug"
data Verbosity
= Minimal
| Verbose
dumpClientErrorVerbosity :: Verbosity -> Text
dumpClientErrorVerbosity = \case
Minimal -> "minimal"
Verbose -> "verbose"
data OpenAPIMode = OAFollowPriv | OAIgnorePriv | OADisabled data OpenAPIMode = OAFollowPriv | OAIgnorePriv | OADisabled
deriving Eq deriving Eq
@@ -167,8 +150,7 @@ toText conf =
where where
-- apply conf to all pgrst settings -- apply conf to all pgrst settings
pgrstSettings = (\(k, v) -> (k, v conf)) <$> pgrstSettings = (\(k, v) -> (k, v conf)) <$>
[("client-error-verbosity", q . dumpClientErrorVerbosity . configClientErrorVerbosity) [("db-aggregates-enabled", T.toLower . show . configDbAggregates)
,("db-aggregates-enabled", T.toLower . show . configDbAggregates)
,("db-anon-role", q . T.decodeUtf8 . fromMaybe "" . configDbAnonRole) ,("db-anon-role", q . T.decodeUtf8 . fromMaybe "" . configDbAnonRole)
,("db-channel", q . configDbChannel) ,("db-channel", q . configDbChannel)
,("db-channel-enabled", T.toLower . show . configDbChannelEnabled) ,("db-channel-enabled", T.toLower . show . configDbChannelEnabled)
@@ -187,7 +169,6 @@ toText conf =
,("db-schemas", q . T.intercalate "," . toList . configDbSchemas) ,("db-schemas", q . T.intercalate "," . toList . configDbSchemas)
,("db-config", T.toLower . show . configDbConfig) ,("db-config", T.toLower . show . configDbConfig)
,("db-pre-config", q . maybe mempty dumpQi . configDbPreConfig) ,("db-pre-config", q . maybe mempty dumpQi . configDbPreConfig)
,("db-timezone-enabled", T.toLower . show . configDbTimezoneEnabled)
,("db-tx-end", q . showTxEnd) ,("db-tx-end", q . showTxEnd)
,("db-uri", q . configDbUri) ,("db-uri", q . configDbUri)
,("jwt-aud", q . fromMaybe mempty . configJwtAudience) ,("jwt-aud", q . fromMaybe mempty . configJwtAudience)
@@ -217,10 +198,6 @@ toText conf =
-- quote strings and replace " with \" -- quote strings and replace " with \"
q s = "\"" <> T.replace "\"" "\\\"" s <> "\"" q s = "\"" <> T.replace "\"" "\\\"" s <> "\""
dumpQi :: QualifiedIdentifier -> Text
dumpQi (QualifiedIdentifier s i) =
(if T.null s then mempty else s <> ".") <> i
showTxEnd c = case (configDbTxRollbackAll c, configDbTxAllowOverride c) of showTxEnd c = case (configDbTxRollbackAll c, configDbTxAllowOverride c) of
( False, False ) -> "commit" ( False, False ) -> "commit"
( False, True ) -> "commit-allow-override" ( False, True ) -> "commit-allow-override"
@@ -273,7 +250,6 @@ parser :: Maybe FilePath -> Environment -> [(Text, Text)] -> RoleSettings -> Rol
parser optPath env dbSettings roleSettings roleIsolationLvl = parser optPath env dbSettings roleSettings roleIsolationLvl =
AppConfig AppConfig
<$> parseAppSettings "app.settings" <$> parseAppSettings "app.settings"
<*> parseErrorVerbosity "client-error-verbosity"
<*> (fromMaybe False <$> optBool "db-aggregates-enabled") <*> (fromMaybe False <$> optBool "db-aggregates-enabled")
<*> (fmap encodeUtf8 <$> optString "db-anon-role") <*> (fmap encodeUtf8 <$> optString "db-anon-role")
<*> (fromMaybe "pgrst" <$> optString "db-channel") <*> (fromMaybe "pgrst" <$> optString "db-channel")
@@ -294,10 +270,10 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
<*> (fromMaybe True <$> optBool "db-prepared-statements") <*> (fromMaybe True <$> optBool "db-prepared-statements")
<*> (fmap toQi <$> optWithAlias (optString "db-root-spec") <*> (fmap toQi <$> optWithAlias (optString "db-root-spec")
(optString "root-spec")) (optString "root-spec"))
<*> parseDbSchemas "db-schemas" "db-schema" <*> (fromList . maybe ["public"] splitOnCommas <$> optWithAlias (optString "db-schemas")
(optString "db-schema"))
<*> (fromMaybe True <$> optBool "db-config") <*> (fromMaybe True <$> optBool "db-config")
<*> (fmap toQi <$> optString "db-pre-config") <*> (fmap toQi <$> optString "db-pre-config")
<*> (fromMaybe True <$> optBool "db-timezone-enabled")
<*> parseTxEnd "db-tx-end" snd <*> parseTxEnd "db-tx-end" snd
<*> parseTxEnd "db-tx-end" fst <*> parseTxEnd "db-tx-end" fst
<*> (fromMaybe "postgresql://" <$> optString "db-uri") <*> (fromMaybe "postgresql://" <$> optString "db-uri")
@@ -331,14 +307,6 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
<*> optInt "internal-schema-cache-load-sleep" <*> optInt "internal-schema-cache-load-sleep"
<*> optInt "internal-schema-cache-relationship-load-sleep" <*> optInt "internal-schema-cache-relationship-load-sleep"
where where
parseErrorVerbosity :: C.Key -> C.Parser C.Config Verbosity
parseErrorVerbosity k =
optString k >>= \case
Nothing -> pure Verbose -- default
Just "minimal" -> pure Minimal
Just "verbose" -> pure Verbose
Just _ -> fail "Invalid client-error-verbosity. Check your configuration."
parseAppSettings :: C.Key -> C.Parser C.Config [(Text, Text)] parseAppSettings :: C.Key -> C.Parser C.Config [(Text, Text)]
parseAppSettings key = addFromEnv . fmap (fmap coerceText) <$> C.subassocs key C.value parseAppSettings key = addFromEnv . fmap (fmap coerceText) <$> C.subassocs key C.value
where where
@@ -357,18 +325,6 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
Just asp | asp == serverPort -> fail "admin-server-port cannot be the same as server-port" Just asp | asp == serverPort -> fail "admin-server-port cannot be the same as server-port"
| otherwise -> pure $ Just asp | otherwise -> pure $ Just asp
parseDbSchemas :: C.Key -> C.Key -> C.Parser C.Config (NonEmpty Text)
parseDbSchemas k al =
optWithAlias (optString k) (optString al) >>= \case
Nothing -> pure $ fromList ["public"]
Just s
| "pg_catalog" `elem` schemas -> fail (errMsg "pg_catalog")
| "information_schema" `elem` schemas -> fail (errMsg "information_schema")
| otherwise -> pure $ fromList schemas
where
schemas = splitOnCommas s
errMsg x = "db-schemas does not allow schema: '" <> x <> "'"
parseSocketFileMode :: C.Key -> C.Parser C.Config FileMode parseSocketFileMode :: C.Key -> C.Parser C.Config FileMode
parseSocketFileMode k = parseSocketFileMode k =
optString k >>= \case optString k >>= \case
@@ -621,10 +577,10 @@ pgConnString conn | uriDesignator `T.isPrefixOf` conn || shortUriDesignator `T.i
-- >>> addFallbackAppName ver "postgres://admin2:?pass?special?@localhost:5432/postgres" -- >>> addFallbackAppName ver "postgres://admin2:?pass?special?@localhost:5432/postgres"
-- "postgres://admin2:?pass?special?@localhost:5432/postgres?fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29" -- "postgres://admin2:?pass?special?@localhost:5432/postgres?fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29"
-- --
-- >>> addFallbackAppName ver "postgresql://?dbname=postgres&host=/run/user/1000/postgrest/postgrest-with-postgresql-16-BuR/socket&user=some_protected_user&password=invalid_pass" -- addFallbackAppName ver "postgresql://?dbname=postgres&host=/run/user/1000/postgrest/postgrest-with-postgresql-16-BuR/socket&user=some_protected_user&password=invalid_pass"
-- "postgresql://?dbname=postgres&host=/run/user/1000/postgrest/postgrest-with-postgresql-16-BuR/socket&user=some_protected_user&password=invalid_pass&fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29" -- "postgresql://?dbname=postgres&host=/run/user/1000/postgrest/postgrest-with-postgresql-16-BuR/socket&user=some_protected_user&password=invalid_pass&fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29"
-- --
-- >>> addFallbackAppName ver "postgresql:///postgres?host=/run/user/1000/postgrest/postgrest-with-postgresql-16-BuR/socket&user=some_protected_user&password=invalid_pass" -- addFallbackAppName ver "postgresql:///postgres?host=/run/user/1000/postgrest/postgrest-with-postgresql-16-BuR/socket&user=some_protected_user&password=invalid_pass"
-- "postgresql:///postgres?host=/run/user/1000/postgrest/postgrest-with-postgresql-16-BuR/socket&user=some_protected_user&password=invalid_pass&fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29" -- "postgresql:///postgres?host=/run/user/1000/postgrest/postgrest-with-postgresql-16-BuR/socket&user=some_protected_user&password=invalid_pass&fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29"
addFallbackAppName :: ByteString -> Text -> Text addFallbackAppName :: ByteString -> Text -> Text
addFallbackAppName version dbUri = addConnStringOption dbUri "fallback_application_name" pgrstVer addFallbackAppName version dbUri = addConnStringOption dbUri "fallback_application_name" pgrstVer
@@ -651,12 +607,6 @@ addFallbackAppName version dbUri = addConnStringOption dbUri "fallback_applicati
addTargetSessionAttrs :: Text -> Text addTargetSessionAttrs :: Text -> Text
addTargetSessionAttrs dbUri = addConnStringOption dbUri "target_session_attrs" "read-write" addTargetSessionAttrs dbUri = addConnStringOption dbUri "target_session_attrs" "read-write"
toConnectionSettings :: (Text -> Text) -> AppConfig -> [SQL.Setting]
toConnectionSettings transformUri AppConfig{configDbUri, configDbPreparedStatements} =
[ SQL.connection $ SQL.string $ transformUri . addFallbackAppName prettyVersion $ configDbUri
, SQL.usePreparedStatements configDbPreparedStatements
]
addConnStringOption :: Text -> Text -> Text -> Text addConnStringOption :: Text -> Text -> Text -> Text
addConnStringOption dbUri key val = dbUri <> addConnStringOption dbUri key val = dbUri <>
case pgConnString dbUri of case pgConnString dbUri of
@@ -677,9 +627,6 @@ exampleConfigFile = S.unlines
[ "## Admin server used for checks. It's disabled by default unless a port is specified." [ "## Admin server used for checks. It's disabled by default unless a port is specified."
, "# admin-server-port = 3001" , "# admin-server-port = 3001"
, "" , ""
, "# PostgREST error json verbosity config"
, "# client-error-verbosity = \"verbose\""
, ""
, "## The database role to use when no client authentication is provided" , "## The database role to use when no client authentication is provided"
, "# db-anon-role = \"anon\"" , "# db-anon-role = \"anon\""
, "" , ""
@@ -729,19 +676,16 @@ exampleConfigFile = S.unlines
, "## The name of which database schema to expose to REST clients" , "## The name of which database schema to expose to REST clients"
, "db-schemas = \"public\"" , "db-schemas = \"public\""
, "" , ""
, "## Enable quering pg_timezone_names from db"
, "# db-timezone-enabled = true"
, ""
, "## How to terminate database transactions" , "## How to terminate database transactions"
, "## Possible values are:" , "## Possible values are:"
, "## commit (default)" , "## commit (default)"
, "## Transaction is always committed, this can not be overridden" , "## Transaction is always committed, this can not be overriden"
, "## commit-allow-override" , "## commit-allow-override"
, "## Transaction is committed, but can be overridden with Prefer tx=rollback header" , "## Transaction is committed, but can be overriden with Prefer tx=rollback header"
, "## rollback" , "## rollback"
, "## Transaction is always rolled back, this can not be overridden" , "## Transaction is always rolled back, this can not be overriden"
, "## rollback-allow-override" , "## rollback-allow-override"
, "## Transaction is rolled back, but can be overridden with Prefer tx=commit header" , "## Transaction is rolled back, but can be overriden with Prefer tx=commit header"
, "db-tx-end = \"commit\"" , "db-tx-end = \"commit\""
, "" , ""
, "## The standard connection URI format, documented at" , "## The standard connection URI format, documented at"
+13 -13
View File
@@ -46,7 +46,6 @@ dbSettingsNames :: [Text]
dbSettingsNames = dbSettingsNames =
(prefix <>) <$> (prefix <>) <$>
["db_aggregates_enabled" ["db_aggregates_enabled"
,"client_error_verbosity"
,"db_anon_role" ,"db_anon_role"
,"db_pre_config" ,"db_pre_config"
,"db_extra_search_path" ,"db_extra_search_path"
@@ -56,7 +55,6 @@ dbSettingsNames =
,"db_prepared_statements" ,"db_prepared_statements"
,"db_root_spec" ,"db_root_spec"
,"db_schemas" ,"db_schemas"
,"db_timezone_enabled"
,"db_tx_end" ,"db_tx_end"
,"db_hoisted_tx_settings" ,"db_hoisted_tx_settings"
,"jwt_aud" ,"jwt_aud"
@@ -72,8 +70,8 @@ dbSettingsNames =
,"server_timing_enabled" ,"server_timing_enabled"
] ]
queryPgVersion :: Session PgVersion queryPgVersion :: Bool -> Session PgVersion
queryPgVersion = statement mempty $ pgVersionStatement False queryPgVersion prepared = statement mempty $ pgVersionStatement prepared
pgVersionStatement :: Bool -> SQL.Statement () PgVersion pgVersionStatement :: Bool -> SQL.Statement () PgVersion
pgVersionStatement = SQL.Statement sql HE.noParams versionRow pgVersionStatement = SQL.Statement sql HE.noParams versionRow
@@ -92,9 +90,10 @@ pgVersionStatement = SQL.Statement sql HE.noParams versionRow
-- --
-- The example above will result in <prefix>jwt_aud = 'val' -- The example above will result in <prefix>jwt_aud = 'val'
-- A setting on the database only will have no effect: ALTER DATABASE postgres SET <prefix>jwt_aud = 'xx' -- A setting on the database only will have no effect: ALTER DATABASE postgres SET <prefix>jwt_aud = 'xx'
queryDbSettings :: Maybe Text -> Session [(Text, Text)] queryDbSettings :: Maybe Text -> Bool -> Session [(Text, Text)]
queryDbSettings preConfFunc = queryDbSettings preConfFunc prepared =
SQL.transactionNoRetry SQL.ReadCommitted SQL.Read $ SQL.statement dbSettingsNames $ SQL.Statement sql (arrayParam HE.text) decodeSettings True let transaction = if prepared then SQL.transaction else SQL.unpreparedTransaction in
transaction SQL.ReadCommitted SQL.Read $ SQL.statement dbSettingsNames $ SQL.Statement sql (arrayParam HE.text) decodeSettings prepared
where where
sql = encodeUtf8 [trimming| sql = encodeUtf8 [trimming|
WITH WITH
@@ -102,7 +101,7 @@ queryDbSettings preConfFunc =
SELECT setdatabase as database, SELECT setdatabase as database,
unnest(setconfig) as setting unnest(setconfig) as setting
FROM pg_catalog.pg_db_role_setting FROM pg_catalog.pg_db_role_setting
WHERE setrole = quote_ident(CURRENT_USER)::regrole::oid WHERE setrole = CURRENT_USER::regrole::oid
AND setdatabase IN (0, (SELECT oid FROM pg_catalog.pg_database WHERE datname = CURRENT_CATALOG)) AND setdatabase IN (0, (SELECT oid FROM pg_catalog.pg_database WHERE datname = CURRENT_CATALOG))
), ),
kv_settings AS ( kv_settings AS (
@@ -132,9 +131,10 @@ queryDbSettings preConfFunc =
|]::Text |]::Text
decodeSettings = HD.rowList $ (,) <$> column HD.text <*> column HD.text decodeSettings = HD.rowList $ (,) <$> column HD.text <*> column HD.text
queryRoleSettings :: PgVersion -> Session (RoleSettings, RoleIsolationLvl) queryRoleSettings :: PgVersion -> Bool -> Session (RoleSettings, RoleIsolationLvl)
queryRoleSettings pgVer = queryRoleSettings pgVer prepared =
SQL.transactionNoRetry SQL.ReadCommitted SQL.Read $ SQL.statement mempty $ SQL.Statement sql HE.noParams (processRows <$> rows) True let transaction = if prepared then SQL.transaction else SQL.unpreparedTransaction in
transaction SQL.ReadCommitted SQL.Read $ SQL.statement mempty $ SQL.Statement sql HE.noParams (processRows <$> rows) prepared
where where
sql = encodeUtf8 [trimming| sql = encodeUtf8 [trimming|
with with
@@ -142,7 +142,7 @@ queryRoleSettings pgVer =
select r.rolname, unnest(r.rolconfig) as setting select r.rolname, unnest(r.rolconfig) as setting
from pg_auth_members m from pg_auth_members m
join pg_roles r on r.oid = m.roleid join pg_roles r on r.oid = m.roleid
where member = quote_ident(current_user)::regrole::oid where member = current_user::regrole::oid
), ),
kv_settings AS ( kv_settings AS (
SELECT SELECT
@@ -167,7 +167,7 @@ queryRoleSettings pgVer =
|] |]
hasParameterPrivilege hasParameterPrivilege
| pgVer >= pgVersion150 = "or has_parameter_privilege(quote_ident(current_user)::regrole::oid, ps.name, 'set')" | pgVer >= pgVersion150 = "or has_parameter_privilege(current_user::regrole::oid, ps.name, 'set')"
| otherwise = "" | otherwise = ""
processRows :: [(Text, Maybe Text, [(Text, Text)])] -> (RoleSettings, RoleIsolationLvl) processRows :: [(Text, Maybe Text, [(Text, Text)])] -> (RoleSettings, RoleIsolationLvl)
+5 -55
View File
@@ -1,19 +1,12 @@
{-# OPTIONS_GHC -Wno-unused-do-bind #-} {-# OPTIONS_GHC -Wno-unused-do-bind #-}
{-# LANGUAGE LambdaCase #-}
module PostgREST.Config.JSPath module PostgREST.Config.JSPath
( JSPath ( JSPath
, JSPathExp(..) , JSPathExp(..)
, FilterExp(..) , FilterExp(..)
, dumpJSPath , dumpJSPath
, pRoleClaimKey , pRoleClaimKey
, walkJSPath
) where ) where
import qualified Data.Aeson as JSON
import qualified Data.Aeson.Key as K
import qualified Data.Aeson.KeyMap as KM
import qualified Data.Text as T
import qualified Data.Vector as V
import qualified Text.ParserCombinators.Parsec as P import qualified Text.ParserCombinators.Parsec as P
import Data.Either.Combinators (mapLeft) import Data.Either.Combinators (mapLeft)
@@ -29,10 +22,9 @@ type JSPath = [JSPathExp]
-- NOTE: We only accept one JSPFilter expr (at the end of input) -- NOTE: We only accept one JSPFilter expr (at the end of input)
-- | jspath expression -- | jspath expression
data JSPathExp data JSPathExp
= JSPKey Text -- .property or ."property-dash" = JSPKey Text -- .property or ."property-dash"
| JSPIdx Int -- [0] | JSPIdx Int -- [0]
| JSPSlice (Maybe Int) (Maybe Int) -- [0:5] or [0:] or [:5] or [:] | JSPFilter FilterExp -- [?(@ == "match")]
| JSPFilter FilterExp -- [?(@ == "match")]
data FilterExp data FilterExp
= EqualsCond Text = EqualsCond Text
@@ -45,7 +37,6 @@ dumpJSPath :: JSPathExp -> Text
-- TODO: this needs to be quoted properly for special chars -- TODO: this needs to be quoted properly for special chars
dumpJSPath (JSPKey k) = "." <> show k dumpJSPath (JSPKey k) = "." <> show k
dumpJSPath (JSPIdx i) = "[" <> show i <> "]" dumpJSPath (JSPIdx i) = "[" <> show i <> "]"
dumpJSPath (JSPSlice s e) = "[" <> maybe "" show s <> ":" <> maybe "" show e <> "]"
dumpJSPath (JSPFilter cond) = "[?(@" <> expr <> ")]" dumpJSPath (JSPFilter cond) = "[?(@" <> expr <> ")]"
where where
expr = expr =
@@ -56,35 +47,6 @@ dumpJSPath (JSPFilter cond) = "[?(@" <> expr <> ")]"
EndsWithCond text -> " ==^ " <> show text EndsWithCond text -> " ==^ " <> show text
ContainsCond text -> " *== " <> show text ContainsCond text -> " *== " <> show text
-- | Evaluate JSPath on a JSON
walkJSPath :: Maybe JSON.Value -> JSPath -> Maybe JSON.Value
walkJSPath x [] = x
walkJSPath (Just (JSON.Object o)) (JSPKey key:rest) = walkJSPath (KM.lookup (K.fromText key) o) rest
walkJSPath (Just (JSON.Array ar)) (JSPIdx idx:rest) = walkJSPath (ar V.!? idx) rest
walkJSPath (Just (JSON.String str)) (JSPSlice start end:rest) =
let
len = T.length str
norm :: Maybe Int -> Maybe Int -- Normalize negative indices to positive
norm = fmap (\i -> max 0 $ min len $ if i < 0 then len + i else i)
s = fromMaybe 0 $ norm start -- normalized start index
e = fromMaybe len $ norm end -- normalized end index
slicedString = if s >= e then T.empty else T.take (e-s) $ T.drop s str
in
walkJSPath (Just $ JSON.String slicedString) rest
walkJSPath (Just (JSON.Array ar)) (JSPFilter jspFilter:rest) = case jspFilter of
EqualsCond txt -> walkJSPath (findFirstMatch (==) txt ar) rest
NotEqualsCond txt -> walkJSPath (findFirstMatch (/=) txt ar) rest
StartsWithCond txt -> walkJSPath (findFirstMatch T.isPrefixOf txt ar) rest
EndsWithCond txt -> walkJSPath (findFirstMatch T.isSuffixOf txt ar) rest
ContainsCond txt -> walkJSPath (findFirstMatch T.isInfixOf txt ar) rest
where
findFirstMatch matchWith pattern = find (\case
JSON.String txt -> pattern `matchWith` txt
_ -> False)
walkJSPath _ _ = Nothing
-- Used for the config value "role-claim-key" -- Used for the config value "role-claim-key"
pRoleClaimKey :: Text -> Either Text JSPath pRoleClaimKey :: Text -> Either Text JSPath
@@ -95,7 +57,7 @@ pJSPath :: P.Parser JSPath
pJSPath = P.many1 pJSPathExp <* P.eof pJSPath = P.many1 pJSPathExp <* P.eof
pJSPathExp :: P.Parser JSPathExp pJSPathExp :: P.Parser JSPathExp
pJSPathExp = P.try pJSPKey <|> P.try pJSPFilter <|> P.try pJSPIdx <|> pJSPSlice pJSPathExp = pJSPKey <|> pJSPFilter <|> pJSPIdx
pJSPKey :: P.Parser JSPathExp pJSPKey :: P.Parser JSPathExp
pJSPKey = do pJSPKey = do
@@ -110,25 +72,13 @@ pJSPIdx = do
P.char ']' P.char ']'
return (JSPIdx num) <?> "pJSPIdx: JSPath array index" return (JSPIdx num) <?> "pJSPIdx: JSPath array index"
pJSPSlice :: P.Parser JSPathExp
pJSPSlice = do
P.char '['
startSign <- P.optionMaybe $ P.char '-'
startIndex <- P.optionMaybe (read <$> P.many1 P.digit)
P.char ':'
endSign <- P.optionMaybe $ P.char '-'
endIndex <- P.optionMaybe (read <$> P.many1 P.digit)
P.char ']'
let start' = if isJust startSign then ((-1) *) <$> startIndex else startIndex
end' = if isJust endSign then ((-1) *) <$> endIndex else endIndex
return (JSPSlice start' end') <?> "pJSPSlice: JSPath string slice"
pJSPFilter :: P.Parser JSPathExp pJSPFilter :: P.Parser JSPathExp
pJSPFilter = do pJSPFilter = do
P.try $ P.string "[?(" P.try $ P.string "[?("
condition <- pFilterConditionParser condition <- pFilterConditionParser
P.char ')' P.char ')'
P.char ']' P.char ']'
P.eof -- this should be the last jspath expression
return (JSPFilter condition) <?> "pJSPFilter: JSPath filter exp" return (JSPFilter condition) <?> "pJSPFilter: JSPath filter exp"
pFilterConditionParser :: P.Parser FilterExp pFilterConditionParser :: P.Parser FilterExp
+5 -5
View File
@@ -3,9 +3,9 @@
module PostgREST.Config.PgVersion module PostgREST.Config.PgVersion
( PgVersion(..) ( PgVersion(..)
, minimumPgVersion , minimumPgVersion
, pgVersion140
, pgVersion150 , pgVersion150
, pgVersion170 , pgVersion170
, pgVersion180
) where ) where
import qualified Data.Aeson as JSON import qualified Data.Aeson as JSON
@@ -25,7 +25,10 @@ instance Ord PgVersion where
-- | Tells the minimum PostgreSQL version required by this version of PostgREST -- | Tells the minimum PostgreSQL version required by this version of PostgREST
minimumPgVersion :: PgVersion minimumPgVersion :: PgVersion
minimumPgVersion = pgVersion140 minimumPgVersion = pgVersion130
pgVersion130 :: PgVersion
pgVersion130 = PgVersion 130000 "13.0" "13.0"
pgVersion140 :: PgVersion pgVersion140 :: PgVersion
pgVersion140 = PgVersion 140000 "14.0" "14.0" pgVersion140 = PgVersion 140000 "14.0" "14.0"
@@ -35,6 +38,3 @@ pgVersion150 = PgVersion 150000 "15.0" "15.0"
pgVersion170 :: PgVersion pgVersion170 :: PgVersion
pgVersion170 = PgVersion 170000 "17.0" "17.0" pgVersion170 = PgVersion 170000 "17.0" "17.0"
pgVersion180 :: PgVersion
pgVersion180 = PgVersion 180000 "18.0" "18.0"
-19
View File
@@ -1,19 +0,0 @@
module PostgREST.Debounce
( makeDebouncer) where
import Protolude
-- | Make a new debouncer action. An internal "worker" thread runs forever
-- ensuring "action" runs when the "trigger" is called. The "action" is only
-- executed once over a burst of calls.
makeDebouncer :: IO () -> IO (IO ())
makeDebouncer action = do
flag <- newEmptyMVar
let worker = forever $ do
takeMVar flag
action
trigger = void $ tryPutMVar flag ()
void $ forkIO worker
pure trigger
+191 -72
View File
@@ -42,7 +42,6 @@ import Network.HTTP.Types.Header (Header)
import PostgREST.MediaType (MediaType (..)) import PostgREST.MediaType (MediaType (..))
import qualified PostgREST.MediaType as MediaType import qualified PostgREST.MediaType as MediaType
import PostgREST.Config (Verbosity (..))
import PostgREST.SchemaCache (SchemaCache (SchemaCache, dbTablesFuzzyIndex)) import PostgREST.SchemaCache (SchemaCache (SchemaCache, dbTablesFuzzyIndex))
import PostgREST.SchemaCache.Identifiers (QualifiedIdentifier (..), import PostgREST.SchemaCache.Identifiers (QualifiedIdentifier (..),
Schema) Schema)
@@ -52,40 +51,22 @@ import PostgREST.SchemaCache.Relationship (Cardinality (..),
RelationshipsMap) RelationshipsMap)
import PostgREST.SchemaCache.Routine (Routine (..), import PostgREST.SchemaCache.Routine (Routine (..),
RoutineParam (..)) RoutineParam (..))
import PostgREST.Error.Types
import Protolude import Protolude
-- | Encode Error to ByteString class (ErrorBody a, JSON.ToJSON a) => PgrstError a where
errorPayload :: (ErrorBody a, ErrorHeaders a) => Verbosity -> a -> LByteString status :: a -> HTTP.Status
errorPayload verb = JSON.encode . toJsonPgrstError verb headers :: a -> [Header]
where
toJsonPgrstError :: (ErrorBody a, ErrorHeaders a) => Verbosity -> a -> JSON.Value
toJsonPgrstError Verbose err = JSON.object [
"code" .= code err
, "message" .= message err
, "details" .= details err
, "hint" .= hint err
]
toJsonPgrstError Minimal err = JSON.object [
"code" .= code err
, "message" .= message err
]
-- | Create HTTP response from Error errorPayload :: a -> LByteString
errorResponseFor :: (ErrorBody a, ErrorHeaders a) => Verbosity -> a -> Response errorPayload = JSON.encode
errorResponseFor verb err =
let
baseHeader = MediaType.toContentType MTApplicationJSON
cLHeader body = (,) "Content-Length" (show $ LBS.length body) :: Header
pSHeader code' = ("Proxy-Status", "PostgREST; error=" <> T.encodeUtf8 code')
in
responseLBS (status err) (baseHeader : cLHeader (errorPayload verb err) : pSHeader (code err) : headers err) $ errorPayload verb err
class ErrorHeaders a where errorResponseFor :: a -> Response
status :: a -> HTTP.Status errorResponseFor err =
headers :: a -> [Header] let
baseHeader = MediaType.toContentType MTApplicationJSON
cLHeader body = (,) "Content-Length" (show $ LBS.length body) :: Header
in
responseLBS (status err) (baseHeader : cLHeader (errorPayload err) : headers err) $ errorPayload err
class ErrorBody a where class ErrorBody a where
code :: a -> Text code :: a -> Text
@@ -93,7 +74,49 @@ class ErrorBody a where
details :: a -> Maybe JSON.Value details :: a -> Maybe JSON.Value
hint :: a -> Maybe JSON.Value hint :: a -> Maybe JSON.Value
instance ErrorHeaders ApiRequestError where data ApiRequestError
= AggregatesNotAllowed
| MediaTypeError [ByteString]
| InvalidBody ByteString
| InvalidFilters
| InvalidPreferences [ByteString]
| InvalidRange RangeError
| InvalidRpcMethod ByteString
| NotEmbedded Text
| NotImplemented Text
| PutLimitNotAllowedError
| QueryParamError QPError
| RelatedOrderNotToOne Text Text
| UnacceptableFilter Text
| UnacceptableSchema Text [Text]
| UnsupportedMethod ByteString
| GucHeadersError
| GucStatusError
| PutMatchingPkError
| SingularityError Integer
| PGRSTParseError RaiseError
| MaxAffectedViolationError Integer
| InvalidResourcePath
| OpenAPIDisabled
| MaxAffectedRpcViolation
deriving Show
data QPError = QPError Text Text
deriving Show
data RaiseError
= MsgParseError ByteString
| DetParseError ByteString
| NoDetail
deriving Show
data RangeError
= NegativeLimit
| LowerGTUpper
| OutOfBounds Text Text
deriving Show
instance PgrstError ApiRequestError where
status AggregatesNotAllowed{} = HTTP.status400 status AggregatesNotAllowed{} = HTTP.status400
status MediaTypeError{} = HTTP.status406 status MediaTypeError{} = HTTP.status406
status InvalidBody{} = HTTP.status400 status InvalidBody{} = HTTP.status400
@@ -217,7 +240,20 @@ instance ErrorBody ApiRequestError where
hint _ = Nothing hint _ = Nothing
instance ErrorHeaders SchemaCacheError where instance JSON.ToJSON ApiRequestError where
toJSON err = toJsonPgrstError
(code err) (message err) (details err) (hint err)
data SchemaCacheError
= AmbiguousRelBetween Text Text [Relationship]
| AmbiguousRpc [Routine]
| NoRelBetween Text Text (Maybe Text) Text RelationshipsMap
| NoRpc Text Text [Text] MediaType Bool [QualifiedIdentifier] [Routine]
| ColumnNotFound Text Text
| TableNotFound Text Text SchemaCache
deriving Show
instance PgrstError SchemaCacheError where
status AmbiguousRelBetween{} = HTTP.status300 status AmbiguousRelBetween{} = HTTP.status300
status AmbiguousRpc{} = HTTP.status300 status AmbiguousRpc{} = HTTP.status300
status NoRelBetween{} = HTTP.status400 status NoRelBetween{} = HTTP.status400
@@ -281,6 +317,18 @@ instance ErrorBody SchemaCacheError where
hint _ = Nothing hint _ = Nothing
instance JSON.ToJSON SchemaCacheError where
toJSON err = toJsonPgrstError
(code err) (message err) (details err) (hint err)
toJsonPgrstError :: Text -> Text -> Maybe JSON.Value -> Maybe JSON.Value -> JSON.Value
toJsonPgrstError code' message' details' hint' = JSON.object [
"code" .= code'
, "message" .= message'
, "details" .= details'
, "hint" .= hint'
]
-- | -- |
-- If no relationship is found then: -- If no relationship is found then:
-- --
@@ -299,6 +347,9 @@ instance ErrorBody SchemaCacheError where
-- >>> noRelBetweenHint "films" "role" "api" rels -- >>> noRelBetweenHint "films" "role" "api" rels
-- Just "Perhaps you meant 'roles' instead of 'role'." -- Just "Perhaps you meant 'roles' instead of 'role'."
-- --
-- >>> noRelBetweenHint "films" "role" "api" rels
-- Just "Perhaps you meant 'roles' instead of 'role'."
--
-- >>> noRelBetweenHint "films" "actors" "api" rels -- >>> noRelBetweenHint "films" "actors" "api" rels
-- Nothing -- Nothing
-- --
@@ -448,7 +499,12 @@ pgrstParseErrorHint err = case err of
MsgParseError _ -> "MESSAGE must be a JSON object with obligatory keys: 'code', 'message' and optional keys: 'details', 'hint'." MsgParseError _ -> "MESSAGE must be a JSON object with obligatory keys: 'code', 'message' and optional keys: 'details', 'hint'."
_ -> "DETAIL must be a JSON object with obligatory keys: 'status', 'headers' and optional key: 'status_text'." _ -> "DETAIL must be a JSON object with obligatory keys: 'status', 'headers' and optional key: 'status_text'."
instance ErrorHeaders PgError where data PgError = PgError Authenticated SQL.UsageError
deriving Show
type Authenticated = Bool
instance PgrstError PgError where
status (PgError authed usageError) = pgErrorStatus authed usageError status (PgError authed usageError) = pgErrorStatus authed usageError
headers (PgError _ (SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ResultError (SQL.ServerError "PGRST" m d _ _p))))) = headers (PgError _ (SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ResultError (SQL.ServerError "PGRST" m d _ _p))))) =
@@ -463,33 +519,44 @@ instance ErrorHeaders PgError where
then [("WWW-Authenticate", "Bearer") :: Header] then [("WWW-Authenticate", "Bearer") :: Header]
else mempty else mempty
proxyStatusHeader :: Text -> Header
proxyStatusHeader code' = ("Proxy-Status", "PostgREST; error=" <> T.encodeUtf8 code')
instance JSON.ToJSON PgError where
toJSON (PgError _ usageError) = toJsonPgrstError
(code usageError) (message usageError) (details usageError) (hint usageError)
instance ErrorBody PgError where instance ErrorBody PgError where
code (PgError _ usageError) = code usageError code (PgError _ usageError) = code usageError
message (PgError _ usageError) = message usageError message (PgError _ usageError) = message usageError
details (PgError _ usageError) = details usageError details (PgError _ usageError) = details usageError
hint (PgError _ usageError) = hint usageError hint (PgError _ usageError) = hint usageError
instance JSON.ToJSON SQL.UsageError where
toJSON err = toJsonPgrstError
(code err) (message err) (details err) (hint err)
instance ErrorBody SQL.UsageError where instance ErrorBody SQL.UsageError where
code (SQL.ConnectionUsageError _) = "PGRST000" code (SQL.ConnectionUsageError _) = "PGRST000"
code (SQL.SessionUsageError (SQL.PipelineError e)) = code e
code (SQL.SessionUsageError (SQL.QueryError _ _ e)) = code e code (SQL.SessionUsageError (SQL.QueryError _ _ e)) = code e
code SQL.AcquisitionTimeoutUsageError = "PGRST003" code SQL.AcquisitionTimeoutUsageError = "PGRST003"
message (SQL.ConnectionUsageError _) = "Database connection error. Retrying the connection." message (SQL.ConnectionUsageError _) = "Database connection error. Retrying the connection."
message (SQL.SessionUsageError (SQL.PipelineError e)) = message e
message (SQL.SessionUsageError (SQL.QueryError _ _ e)) = message e message (SQL.SessionUsageError (SQL.QueryError _ _ e)) = message e
message SQL.AcquisitionTimeoutUsageError = "Timed out acquiring connection from connection pool." message SQL.AcquisitionTimeoutUsageError = "Timed out acquiring connection from connection pool."
details (SQL.ConnectionUsageError e) = JSON.String . T.decodeUtf8 <$> e details (SQL.ConnectionUsageError e) = JSON.String . T.decodeUtf8 <$> e
details (SQL.SessionUsageError (SQL.PipelineError e)) = details e
details (SQL.SessionUsageError (SQL.QueryError _ _ e)) = details e details (SQL.SessionUsageError (SQL.QueryError _ _ e)) = details e
details SQL.AcquisitionTimeoutUsageError = Nothing details SQL.AcquisitionTimeoutUsageError = Nothing
hint (SQL.ConnectionUsageError _) = Nothing hint (SQL.ConnectionUsageError _) = Nothing
hint (SQL.SessionUsageError (SQL.PipelineError e)) = hint e
hint (SQL.SessionUsageError (SQL.QueryError _ _ e)) = hint e hint (SQL.SessionUsageError (SQL.QueryError _ _ e)) = hint e
hint SQL.AcquisitionTimeoutUsageError = Nothing hint SQL.AcquisitionTimeoutUsageError = Nothing
instance JSON.ToJSON SQL.CommandError where
toJSON err = toJsonPgrstError
(code err) (message err) (details err) (hint err)
instance ErrorBody SQL.CommandError where instance ErrorBody SQL.CommandError where
-- Special error raised with code PGRST, to allow full response control -- Special error raised with code PGRST, to allow full response control
code (SQL.ResultError (SQL.ServerError "PGRST" m d _ _)) = code (SQL.ResultError (SQL.ServerError "PGRST" m d _ _)) =
@@ -531,13 +598,8 @@ instance ErrorBody SQL.CommandError where
pgErrorStatus :: Bool -> SQL.UsageError -> HTTP.Status pgErrorStatus :: Bool -> SQL.UsageError -> HTTP.Status
pgErrorStatus _ (SQL.ConnectionUsageError _) = HTTP.status503 pgErrorStatus _ (SQL.ConnectionUsageError _) = HTTP.status503
pgErrorStatus _ SQL.AcquisitionTimeoutUsageError = HTTP.status504 pgErrorStatus _ SQL.AcquisitionTimeoutUsageError = HTTP.status504
pgErrorStatus _ (SQL.SessionUsageError (SQL.PipelineError (SQL.ClientError _))) = HTTP.status503
pgErrorStatus _ (SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ClientError _))) = HTTP.status503 pgErrorStatus _ (SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ClientError _))) = HTTP.status503
pgErrorStatus authed (SQL.SessionUsageError (SQL.PipelineError (SQL.ResultError rError))) = mapSQLtoHTTP authed rError pgErrorStatus authed (SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ResultError rError))) =
pgErrorStatus authed (SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ResultError rError))) = mapSQLtoHTTP authed rError
mapSQLtoHTTP :: Bool -> SQL.ResultError -> HTTP.Status
mapSQLtoHTTP authed rError =
case rError of case rError of
(SQL.ServerError c m d _ _) -> (SQL.ServerError c m d _ _) ->
case BS.unpack c of case BS.unpack c of
@@ -591,45 +653,86 @@ mapSQLtoHTTP authed rError =
_ -> HTTP.status500 _ -> HTTP.status500
instance ErrorHeaders Error where data Error
status (ApiRequestErr err) = status err = ApiRequestError ApiRequestError
status (SchemaCacheErr err) = status err | SchemaCacheErr SchemaCacheError
status (JwtErr err) = status err | JwtErr JwtError
status NoSchemaCacheError = HTTP.status503 | NoSchemaCacheError
status (PgErr err) = status err | PgErr PgError
deriving Show
headers (ApiRequestErr err) = headers err data JwtError
headers (SchemaCacheErr err) = headers err = JwtDecodeErr JwtDecodeError
headers (JwtErr err) = headers err | JwtSecretMissing
headers (PgErr err) = headers err | JwtTokenRequired
headers NoSchemaCacheError = mempty | JwtClaimsErr JwtClaimsError
deriving Show
data JwtDecodeError
= EmptyAuthHeader
| UnexpectedParts Int
| KeyError Text
| BadAlgorithm Text
| BadCrypto
| UnsupportedTokenType
| UnreachableDecodeError
deriving Show
data JwtClaimsError
= JWTExpired
| JWTNotYetValid
| JWTIssuedAtFuture
| JWTNotInAudience
| ParsingClaimsFailed
| ExpClaimNotNumber
| NbfClaimNotNumber
| IatClaimNotNumber
| AudClaimNotStringOrArray
deriving Show
instance PgrstError Error where
status (ApiRequestError err) = status err
status (SchemaCacheErr err) = status err
status (JwtErr err) = status err
status NoSchemaCacheError = HTTP.status503
status (PgErr err) = status err
headers (ApiRequestError err) = proxyStatusHeader (code err) : headers err
headers (SchemaCacheErr err) = proxyStatusHeader (code err) : headers err
headers (JwtErr err) = proxyStatusHeader (code err) : headers err
headers (PgErr err) = proxyStatusHeader (code err) : headers err
headers err@NoSchemaCacheError = proxyStatusHeader (code err) : mempty
instance JSON.ToJSON Error where
toJSON err = toJsonPgrstError
(code err) (message err) (details err) (hint err)
instance ErrorBody Error where instance ErrorBody Error where
code (ApiRequestErr err) = code err code (ApiRequestError err) = code err
code (SchemaCacheErr err) = code err code (SchemaCacheErr err) = code err
code (JwtErr err) = code err code (JwtErr err) = code err
code NoSchemaCacheError = "PGRST002" code NoSchemaCacheError = "PGRST002"
code (PgErr err) = code err code (PgErr err) = code err
message (ApiRequestErr err) = message err message (ApiRequestError err) = message err
message (SchemaCacheErr err) = message err message (SchemaCacheErr err) = message err
message (JwtErr err) = message err message (JwtErr err) = message err
message NoSchemaCacheError = "Could not query the database for the schema cache. Retrying." message NoSchemaCacheError = "Could not query the database for the schema cache. Retrying."
message (PgErr err) = message err message (PgErr err) = message err
details (ApiRequestErr err) = details err details (ApiRequestError err) = details err
details (SchemaCacheErr err) = details err details (SchemaCacheErr err) = details err
details (JwtErr err) = details err details (JwtErr err) = details err
details NoSchemaCacheError = Nothing details NoSchemaCacheError = Nothing
details (PgErr err) = details err details (PgErr err) = details err
hint (ApiRequestErr err) = hint err hint (ApiRequestError err) = hint err
hint (SchemaCacheErr err) = hint err hint (SchemaCacheErr err) = hint err
hint (JwtErr err) = hint err hint (JwtErr err) = hint err
hint NoSchemaCacheError = Nothing hint NoSchemaCacheError = Nothing
hint (PgErr err) = hint err hint (PgErr err) = hint err
instance ErrorHeaders JwtError where instance PgrstError JwtError where
status JwtDecodeErr{} = HTTP.unauthorized401 status JwtDecodeErr{} = HTTP.unauthorized401
status JwtSecretMissing = HTTP.status500 status JwtSecretMissing = HTTP.status500
status JwtTokenRequired = HTTP.unauthorized401 status JwtTokenRequired = HTTP.unauthorized401
@@ -640,6 +743,10 @@ instance ErrorHeaders JwtError where
headers e@(JwtClaimsErr _) = [invalidTokenHeader $ message e] headers e@(JwtClaimsErr _) = [invalidTokenHeader $ message e]
headers _ = mempty headers _ = mempty
instance JSON.ToJSON JwtError where
toJSON err = toJsonPgrstError
(code err) (message err) (details err) (hint err)
instance ErrorBody JwtError where instance ErrorBody JwtError where
code JwtSecretMissing = "PGRST300" code JwtSecretMissing = "PGRST300"
code (JwtDecodeErr _) = "PGRST301" code (JwtDecodeErr _) = "PGRST301"
@@ -683,6 +790,18 @@ requiredTokenHeader :: Header
requiredTokenHeader = ("WWW-Authenticate", "Bearer") requiredTokenHeader = ("WWW-Authenticate", "Bearer")
-- For parsing byteString to JSON Object, used for allowing full response control -- For parsing byteString to JSON Object, used for allowing full response control
data PgRaiseErrMessage = PgRaiseErrMessage {
getCode :: Text,
getMessage :: Text,
getDetails :: Maybe Text,
getHint :: Maybe Text
}
data PgRaiseErrDetails = PgRaiseErrDetails {
getStatus :: Int,
getStatusText :: Maybe Text,
getHeaders :: Map Text Text
}
instance JSON.FromJSON PgRaiseErrMessage where instance JSON.FromJSON PgRaiseErrMessage where
parseJSON (JSON.Object m) = parseJSON (JSON.Object m) =
-138
View File
@@ -1,138 +0,0 @@
{-|
Module : PostgREST.Error.Types
Description : PostgREST Error Data Types
-}
module PostgREST.Error.Types
( ApiRequestError(..)
, QPError(..)
, RangeError(..)
, RaiseError(..)
, SchemaCacheError(..)
, PgError(..)
, Error(..)
, JwtError (..)
, JwtDecodeError(..)
, JwtClaimsError(..)
, PgRaiseErrMessage(..)
, PgRaiseErrDetails(..)
) where
import qualified Hasql.Pool as SQL
import PostgREST.MediaType (MediaType (..))
import PostgREST.SchemaCache (SchemaCache (..))
import PostgREST.SchemaCache.Identifiers (QualifiedIdentifier (..))
import PostgREST.SchemaCache.Relationship (Relationship (..),
RelationshipsMap)
import PostgREST.SchemaCache.Routine (Routine (..))
import Protolude
data Error
= ApiRequestErr ApiRequestError
| SchemaCacheErr SchemaCacheError
| JwtErr JwtError
| NoSchemaCacheError
| PgErr PgError
deriving Show
-- API REQUEST ERRORS: PGRST1XX
data ApiRequestError
= AggregatesNotAllowed
| MediaTypeError [ByteString]
| InvalidBody ByteString
| InvalidFilters
| InvalidPreferences [ByteString]
| InvalidRange RangeError
| InvalidRpcMethod ByteString
| NotEmbedded Text
| NotImplemented Text
| PutLimitNotAllowedError
| QueryParamError QPError
| RelatedOrderNotToOne Text Text
| UnacceptableFilter Text
| UnacceptableSchema Text [Text]
| UnsupportedMethod ByteString
| GucHeadersError
| GucStatusError
| PutMatchingPkError
| SingularityError Integer
| PGRSTParseError RaiseError
| MaxAffectedViolationError Integer
| InvalidResourcePath
| OpenAPIDisabled
| MaxAffectedRpcViolation
deriving Show
data QPError = QPError Text Text
deriving Show
data RaiseError
= MsgParseError ByteString
| DetParseError ByteString
| NoDetail
deriving Show
data RangeError
= NegativeLimit
| LowerGTUpper
| OutOfBounds Text Text
deriving Show
-- SCHEMA CACHE ERRORS: PGRST2XX
data SchemaCacheError
= AmbiguousRelBetween Text Text [Relationship]
| AmbiguousRpc [Routine]
| NoRelBetween Text Text (Maybe Text) Text RelationshipsMap
| NoRpc Text Text [Text] MediaType Bool [QualifiedIdentifier] [Routine]
| ColumnNotFound Text Text
| TableNotFound Text Text SchemaCache
deriving Show
-- JWT ERRORS: PGRST3XX
data JwtError
= JwtDecodeErr JwtDecodeError
| JwtSecretMissing
| JwtTokenRequired
| JwtClaimsErr JwtClaimsError
deriving Show
data JwtDecodeError
= EmptyAuthHeader
| UnexpectedParts Int
| KeyError Text
| BadAlgorithm Text
| BadCrypto
| UnsupportedTokenType
| UnreachableDecodeError
deriving Show
data JwtClaimsError
= JWTExpired
| JWTNotYetValid
| JWTIssuedAtFuture
| JWTNotInAudience
| ParsingClaimsFailed
| ExpClaimNotNumber
| NbfClaimNotNumber
| IatClaimNotNumber
| AudClaimNotStringOrArray
deriving Show
-- PG ERRORS
type Authenticated = Bool
data PgError = PgError Authenticated SQL.UsageError
deriving Show
-- For parsing byteString to JSON Object, used for allowing full response control
data PgRaiseErrMessage = PgRaiseErrMessage {
getCode :: Text,
getMessage :: Text,
getDetails :: Maybe Text,
getHint :: Maybe Text
}
data PgRaiseErrDetails = PgRaiseErrDetails {
getStatus :: Int,
getStatusText :: Maybe Text,
getHeaders :: Map Text Text
}
+6 -8
View File
@@ -10,7 +10,9 @@ import qualified Hasql.Connection as SQL
import qualified Hasql.Notifications as SQL import qualified Hasql.Notifications as SQL
import PostgREST.AppState (AppState, getConfig) import PostgREST.AppState (AppState, getConfig)
import PostgREST.Config (AppConfig (..)) import PostgREST.Config (AppConfig (..))
import PostgREST.Observation (Observation (..)) import PostgREST.Observation (Observation (..),
isDbListenerBug)
import PostgREST.Version (prettyVersion)
import qualified PostgREST.AppState as AppState import qualified PostgREST.AppState as AppState
import qualified PostgREST.Config as Config import qualified PostgREST.Config as Config
@@ -18,7 +20,6 @@ import qualified PostgREST.Config as Config
import Control.Arrow ((&&&)) import Control.Arrow ((&&&))
import Data.Bitraversable (bisequence) import Data.Bitraversable (bisequence)
import Data.Either.Combinators (whenRight) import Data.Either.Combinators (whenRight)
import qualified Data.Text as T
import qualified Database.PostgreSQL.LibPQ as LibPQ import qualified Database.PostgreSQL.LibPQ as LibPQ
import qualified Hasql.Session as SQL import qualified Hasql.Session as SQL
import PostgREST.Config.Database (queryPgVersion) import PostgREST.Config.Database (queryPgVersion)
@@ -36,7 +37,7 @@ runListener appState = do
-- | This function never returns (but can throw) and return type enforces that. -- | This function never returns (but can throw) and return type enforces that.
retryingListen :: AppState -> IO Void retryingListen :: AppState -> IO Void
retryingListen appState = do retryingListen appState = do
cfg@AppConfig{..} <- AppState.getConfig appState AppConfig{..} <- AppState.getConfig appState
let let
dbChannel = toS configDbChannel dbChannel = toS configDbChannel
onError err = do onError err = do
@@ -61,8 +62,7 @@ retryingListen appState = do
-- Make sure we don't leak connections on errors -- Make sure we don't leak connections on errors
bracket bracket
-- acquire connection -- acquire connection
(SQL.acquire $ (SQL.acquire $ toUtf8 (Config.addTargetSessionAttrs $ Config.addFallbackAppName prettyVersion configDbUri))
Config.toConnectionSettings Config.addTargetSessionAttrs cfg)
-- release connection -- release connection
(`whenRight` releaseConnection) $ (`whenRight` releaseConnection) $
-- use connection -- use connection
@@ -70,7 +70,7 @@ retryingListen appState = do
Right db -> do Right db -> do
SQL.listen db $ SQL.toPgIdentifier dbChannel SQL.listen db $ SQL.toPgIdentifier dbChannel
(pqHost, pqPort) <- SQL.withLibPQConnection db $ bisequence . (LibPQ.host &&& LibPQ.port) (pqHost, pqPort) <- SQL.withLibPQConnection db $ bisequence . (LibPQ.host &&& LibPQ.port)
pgFullName <- SQL.run queryPgVersion db >>= either throwIO (pure . pgvFullName) pgFullName <- SQL.run (queryPgVersion False) db >>= either throwIO (pure . pgvFullName)
AppState.putIsListenerOn appState True AppState.putIsListenerOn appState True
@@ -106,5 +106,3 @@ retryingListen appState = do
AppState.schemaCacheLoader appState AppState.schemaCacheLoader appState
releaseConnection = void . forkIO . handle (observer . DBListenerConnectionCleanupFail) . SQL.release releaseConnection = void . forkIO . handle (observer . DBListenerConnectionCleanupFail) . SQL.release
isDbListenerBug e = "could not access status of transaction" `T.isInfixOf` show e
+49 -165
View File
@@ -1,6 +1,4 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE RecordWildCards #-} {-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE RecursiveDo #-}
{-| {-|
Module : PostgREST.Logger Module : PostgREST.Logger
Description : Logging based on the Observation.hs module. Access logs get sent to stdout and server diagnostic get sent to stderr. Description : Logging based on the Observation.hs module. Access logs get sent to stdout and server diagnostic get sent to stderr.
@@ -16,6 +14,7 @@ module PostgREST.Logger
import Control.AutoUpdate (defaultUpdateSettings, import Control.AutoUpdate (defaultUpdateSettings,
mkAutoUpdate, mkAutoUpdate,
updateAction) updateAction)
import Control.Debounce
import qualified Data.ByteString.Char8 as BS import qualified Data.ByteString.Char8 as BS
import qualified Data.Text.Encoding as T import qualified Data.Text.Encoding as T
import qualified Hasql.Decoders as HD import qualified Hasql.Decoders as HD
@@ -32,36 +31,37 @@ import qualified Network.Wai.Middleware.RequestLogger as Wai
import Network.HTTP.Types.Status (Status, status400, status500) import Network.HTTP.Types.Status (Status, status400, status500)
import System.IO.Unsafe (unsafePerformIO) import System.IO.Unsafe (unsafePerformIO)
import PostgREST.Config (LogLevel (..), Verbosity (..)) import PostgREST.Config (LogLevel (..))
import PostgREST.Debounce (makeDebouncer)
import PostgREST.Observation import PostgREST.Observation
import PostgREST.Query (MainQuery (..)) import PostgREST.Query (MainQuery (..))
import PostgREST.SchemaCache (queryTimingsWLabels)
import qualified Data.ByteString.Lazy as LBS import Protolude
import qualified Data.Text as T
import qualified Hasql.Connection as SQL
import qualified Hasql.Pool as SQL
import qualified Hasql.Pool.Observation as SQL
import Numeric (showFFloat)
import PostgREST.Config.PgVersion (pgvName)
import qualified PostgREST.Error as Error
import Protolude
data LoggerState = LoggerState data LoggerState = LoggerState
{ stateGetZTime :: IO ZonedTime -- ^ Time with time zone used for logs { stateGetZTime :: IO ZonedTime -- ^ Time with time zone used for logs
, stateLogDebouncePoolTimeout :: IO () -- ^ Logs with a debounce , stateLogDebouncePoolTimeout :: MVar (IO ()) -- ^ Logs with a debounce
} }
init :: IO LoggerState init :: IO LoggerState
init = mdo init = do
let
oneSecond = 1000000
loggerState = LoggerState zTime debouncePoolTimeout
zTime <- mkAutoUpdate defaultUpdateSettings { updateAction = getZonedTime } zTime <- mkAutoUpdate defaultUpdateSettings { updateAction = getZonedTime }
debouncePoolTimeout <- makeDebouncer $ LoggerState zTime <$> newEmptyMVar
logWithZTime loggerState (observationMessages PoolAcqTimeoutObs) *> threadDelay (5 * oneSecond)
pure loggerState logWithDebounce :: LoggerState -> IO () -> IO ()
logWithDebounce loggerState action = do
debouncer <- tryReadMVar $ stateLogDebouncePoolTimeout loggerState
case debouncer of
Just d -> d
Nothing -> do
newDebouncer <-
let oneSecond = 1000000 in
mkDebounce defaultDebounceSettings
{ debounceAction = action
, debounceFreq = 5*oneSecond
, debounceEdge = leadingEdge -- logs at the start and the end
}
putMVar (stateLogDebouncePoolTimeout loggerState) newDebouncer
newDebouncer
-- TODO stop using this middleware to reuse the same "observer" pattern for all our logs -- TODO stop using this middleware to reuse the same "observer" pattern for all our logs
middleware :: LogLevel -> (Wai.Request -> Maybe BS.ByteString) -> Wai.Middleware middleware :: LogLevel -> (Wai.Request -> Maybe BS.ByteString) -> Wai.Middleware
@@ -88,174 +88,58 @@ shouldLogResponse logLevel = case logLevel of
-- All observations are logged except some that depend on the log-level -- All observations are logged except some that depend on the log-level
observationLogger :: LoggerState -> LogLevel -> ObservationHandler observationLogger :: LoggerState -> LogLevel -> ObservationHandler
observationLogger loggerState logLevel obs = case obs of observationLogger loggerState logLevel obs = case obs of
PoolAcqTimeoutObs -> do o@(PoolAcqTimeoutObs _) -> do
when (logLevel >= LogError) $ when (logLevel >= LogError) $ do
stateLogDebouncePoolTimeout loggerState logWithDebounce loggerState $
logWithZTime loggerState $ observationMessage o
o@(QueryErrorCodeHighObs _) -> do o@(QueryErrorCodeHighObs _) -> do
when (logLevel >= LogError) $ do when (logLevel >= LogError) $ do
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
o@SchemaCacheEmptyObs -> o@SchemaCacheEmptyObs ->
when (logLevel >= LogError) $ do when (logLevel >= LogError) $ do
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
o@(HasqlPoolObs _) -> do o@(HasqlPoolObs _) -> do
when (logLevel >= LogDebug) $ do when (logLevel >= LogDebug) $ do
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
o@(QueryObs _ status) -> do QueryObs gq status -> do
when (shouldLogResponse logLevel status) $ when (shouldLogResponse logLevel status) $
logWithZTime loggerState $ observationMessages o logMainQ loggerState gq
o@PoolRequest -> o@PoolRequest ->
when (logLevel >= LogDebug) $ do when (logLevel >= LogDebug) $ do
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
o@PoolRequestFullfilled -> o@PoolRequestFullfilled ->
when (logLevel >= LogDebug) $ do when (logLevel >= LogDebug) $ do
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
o@PoolFlushed ->
when (logLevel >= LogDebug) $ do
logWithZTime loggerState $ observationMessages o
o@JwtCacheEviction -> o@JwtCacheEviction ->
when (logLevel >= LogDebug) $ do when (logLevel >= LogDebug) $ do
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
o@(JwtCacheLookup _) -> o@(JwtCacheLookup _) ->
when (logLevel >= LogDebug) $ do when (logLevel >= LogDebug) $ do
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
o@(WarpServerObs _) -> o@(WarpServerObs _) ->
when (logLevel >= LogDebug) $ do when (logLevel >= LogDebug) $ do
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
o -> o ->
logWithZTime loggerState $ observationMessages o logWithZTime loggerState $ observationMessage o
logWithZTime :: LoggerState -> [Text] -> IO () logWithZTime :: LoggerState -> Text -> IO ()
logWithZTime loggerState txts = do logWithZTime loggerState txt = do
zTime <- stateGetZTime loggerState zTime <- stateGetZTime loggerState
traverse_ (hPutStrLn stderr . (toS (formatTime defaultTimeLocale "%d/%b/%Y:%T %z: " zTime) <>)) txts hPutStrLn stderr $ toS (formatTime defaultTimeLocale "%d/%b/%Y:%T %z: " zTime) <> txt
logMainQ :: LoggerState -> MainQuery -> IO ()
logMainQ loggerState MainQuery{mqOpenAPI=(x, y, z),..} =
let snipts = renderSnippet <$> [mqTxVars, fromMaybe mempty mqPreReq, mqMain, x, y, z, fromMaybe mempty mqExplain]
-- Does not log SQL when it's empty (happens on OPTIONS requests and when the openapi queries are not generated)
logQ q = when (q /= mempty) $ logWithZTime loggerState $ showOnSingleLine '\n' $ T.decodeUtf8 q in
mapM_ logQ snipts
-- TODO: maybe patch upstream hasql-dynamic-statements so we have a less hackish way to convert -- TODO: maybe patch upstream hasql-dynamic-statements so we have a less hackish way to convert
-- the SQL.Snippet or maybe don't use hasql-dynamic-statements and resort to plain strings for the queries and use regular hasql -- the SQL.Snippet or maybe don't use hasql-dynamic-statements and resort to plain strings for the queries and use regular hasql
renderSnippet :: SQL.Snippet -> ByteString renderSnippet :: SQL.Snippet -> ByteString
renderSnippet snippet = renderSnippet snippet =
let SQL.Statement sql _ _ _ = SQL.dynamicallyParameterized snippet decoder False let SQL.Statement sql _ _ _ = SQL.dynamicallyParameterized snippet decoder prepared
decoder = HD.noResult -- unused decoder = HD.noResult -- unused
prepared = False -- unused
in in
sql sql
observationMessages :: Observation -> [Text]
observationMessages = \case
AdminStartObs address ->
pure $ "Admin server listening on " <> address
AppStartObs ver ->
pure $ "Starting PostgREST " <> T.decodeUtf8 ver <> "..."
AppServerAddressObs address ->
pure $ "API server listening on " <> address
DBConnectedObs ver ->
pure $ "Successfully connected to " <> ver
ExitUnsupportedPgVersion pgVer minPgVer ->
pure $ "Cannot run in this PostgreSQL version (" <> pgvName pgVer <> "), PostgREST needs at least " <> pgvName minPgVer
ExitDBNoRecoveryObs ->
pure "Automatic recovery disabled, exiting."
ExitDBFatalError ServerAuthError usageErr ->
pure $ "Failed to establish a connection. " <> jsonMessage usageErr
ExitDBFatalError ServerPgrstBug usageErr ->
pure $ "This is probably a bug in PostgREST, please report it at https://github.com/PostgREST/postgrest/issues. " <> jsonMessage usageErr
ExitDBFatalError ServerError42P05 usageErr ->
pure $ "If you are using connection poolers in transaction mode, try setting db-prepared-statements to false. " <> jsonMessage usageErr
ExitDBFatalError ServerError08P01 usageErr ->
pure $ "Connection poolers in statement mode are not supported." <> jsonMessage usageErr
SchemaCacheEmptyObs ->
pure $ T.decodeUtf8 . LBS.toStrict . Error.errorPayload Verbose $ Error.NoSchemaCacheError
SchemaCacheErrorObs dbSchemas extraPaths usageErr ->
pure $ "Failed to load the schema cache using "
<> "db-schemas=" <> T.intercalate "," (toList dbSchemas)
<> " and "
<> "db-extra-search-path=" <> T.intercalate "," extraPaths
<> ". " <> jsonMessage usageErr
SchemaCacheQueriedObs resultTime timings ->
[ "Schema cache queried in " <> showMillis resultTime <> " milliseconds " ] <>
let showTimings qt = [ T.intercalate ", " $ (\(l, v) -> T.decodeUtf8 l <> ": " <> v <> " ms") <$> queryTimingsWLabels qt ] in
maybe mempty showTimings timings
SchemaCacheLoadedObs resultTime summary ->
[
"Schema cache loaded " <> summary
, "Schema cache loaded in " <> showMillis resultTime <> " milliseconds"
]
ConnectionRetryObs delay ->
pure $ "Attempting to reconnect to the database in " <> (show delay::Text) <> " seconds..."
QueryPgVersionError usageErr ->
pure $ "Failed to query the PostgreSQL version. " <> jsonMessage usageErr
DBListenStart host port fullName channel -> do
pure $ "Listener connected to " <> fullName <> " on " <> show (fold $ host <> fmap (":" <>) port) <> " and listening for database notifications on the " <> show channel <> " channel"
DBListenFail channel listenErr ->
pure $ "Failed listening for database notifications on the " <> show channel <> " channel. " <>
either showListenerConnError showListenerException listenErr
DBListenRetry delay ->
pure $ "Retrying listening for database notifications in " <> (show delay::Text) <> " seconds..."
DBListenBugHint ->
pure "HINT: This is likely a bug in the notification queue, try executing the following to solve it: select pg_notification_queue_usage();"
DBListenerGotSCacheMsg channel ->
pure $ "Received a schema cache reload message on the " <> show channel <> " channel"
DBListenerGotConfigMsg channel ->
pure $ "Received a config reload message on the " <> show channel <> " channel"
DBListenerConnectionCleanupFail ex ->
pure $ "Failed during listener connection cleanup: " <> showOnSingleLine '\t' (show ex)
(QueryObs MainQuery{mqOpenAPI=(x, y, z),..} _) ->
let snipts = renderSnippet <$> [mqTxVars, fromMaybe mempty mqPreReq, mqMain, x, y, z, fromMaybe mempty mqExplain]
in
showOnSingleLine '\n' . T.decodeUtf8 <$> filter (/= mempty) snipts
ConfigReadErrorObs usageErr ->
pure $ "Failed to query database settings for the config parameters." <> jsonMessage usageErr
QueryRoleSettingsErrorObs usageErr ->
pure $ "Failed to query the role settings. " <> jsonMessage usageErr
QueryErrorCodeHighObs usageErr ->
pure $ jsonMessage usageErr
ConfigInvalidObs err ->
pure $ "Failed reloading config: " <> err
ConfigSucceededObs ->
pure "Config reloaded"
PoolInit poolSize ->
pure $ "Connection Pool initialized with a maximum size of " <> show poolSize <> " connections"
PoolAcqTimeoutObs -> pure $ jsonMessage SQL.AcquisitionTimeoutUsageError
HasqlPoolObs (SQL.ConnectionObservation uuid status) ->
pure $ "Connection " <> show uuid <> (
case status of
SQL.ConnectingConnectionStatus -> " is being established"
SQL.ReadyForUseConnectionStatus reason -> " is available due to " <> case reason of
SQL.EstablishedConnectionReadyForUseReason -> "connection establishment"
SQL.SessionFailedConnectionReadyForUseReason _ -> "session failure"
SQL.SessionSucceededConnectionReadyForUseReason -> "session success"
SQL.InUseConnectionStatus -> " is used"
SQL.TerminatedConnectionStatus reason -> " is terminated due to " <> case reason of
SQL.AgingConnectionTerminationReason -> "max lifetime"
SQL.IdlenessConnectionTerminationReason -> "max idletime"
SQL.ReleaseConnectionTerminationReason -> "release"
SQL.NetworkErrorConnectionTerminationReason _ -> "network error" -- usage error is already logged, no need to repeat the same message.
SQL.InitializationErrorTerminationReason _ -> "init failure"
)
PoolRequest ->
pure "Trying to borrow a connection from pool"
PoolRequestFullfilled ->
pure "Borrowed a connection from the pool"
PoolFlushed ->
pure "Database connection pool flushed"
JwtCacheLookup _ ->
pure "Looked up a JWT in JWT cache"
JwtCacheEviction ->
pure "Evicted entry from JWT cache"
TerminationUnixSignalObs signal ->
pure $ "Received termination unix signal " <> signal
WarpServerObs txt ->
pure $ "Warp server: " <> txt
where
showMillis :: Double -> Text
showMillis x = toS $ showFFloat (Just 1) x ""
jsonMessage err = T.decodeUtf8 . LBS.toStrict . Error.errorPayload Verbose $ Error.PgError False err
showListenerConnError :: SQL.ConnectionError -> Text
showListenerConnError = maybe "Connection error" (showOnSingleLine '\t' . T.decodeUtf8)
showListenerException :: SomeException -> Text
showListenerException = showOnSingleLine '\t' . show
showOnSingleLine :: Char -> Text -> Text
showOnSingleLine split txt = T.intercalate " " $ T.filter (/= split) <$> T.lines txt -- the errors from hasql-notifications come intercalated with "\t\n"
+5 -4
View File
@@ -96,8 +96,9 @@ data ResultSet
mainTx :: MainQuery -> AppConfig -> AuthResult -> ApiRequest -> ActionPlan -> SchemaCache -> MainTx mainTx :: MainQuery -> AppConfig -> AuthResult -> ApiRequest -> ActionPlan -> SchemaCache -> MainTx
mainTx _ _ _ _ (NoDb x) _ = NoDbTx $ NoDbResult x mainTx _ _ _ _ (NoDb x) _ = NoDbTx $ NoDbResult x
mainTx genQ@MainQuery{..} conf@AppConfig{..} AuthResult{..} apiReq (Db plan) sCache = mainTx genQ@MainQuery{..} conf@AppConfig{..} AuthResult{..} apiReq (Db plan) sCache =
DbTx isoLvl txMode dbHandler SQL.transactionNoRetry DbTx isoLvl txMode dbHandler transaction
where where
transaction = if configDbPreparedStatements then SQL.transaction else SQL.unpreparedTransaction
isoLvl = planIsoLvl conf authRole plan isoLvl = planIsoLvl conf authRole plan
txMode = planTxMode plan txMode = planTxMode plan
dbHandler = do dbHandler = do
@@ -221,7 +222,7 @@ failPut :: ResultSet -> DbHandler ()
failPut RSStandard{rsQueryTotal=queryTotal} = failPut RSStandard{rsQueryTotal=queryTotal} =
when (queryTotal /= 1) $ do when (queryTotal /= 1) $ do
lift SQL.condemn lift SQL.condemn
throwError $ Error.ApiRequestErr Error.PutMatchingPkError throwError $ Error.ApiRequestError Error.PutMatchingPkError
-- | -- |
-- Fail a response if a single JSON object was requested and not exactly one -- Fail a response if a single JSON object was requested and not exactly one
@@ -230,13 +231,13 @@ failNotSingular :: MediaType -> ResultSet -> DbHandler ()
failNotSingular mediaType RSStandard{rsQueryTotal=queryTotal} = failNotSingular mediaType RSStandard{rsQueryTotal=queryTotal} =
when (elem mediaType [MTVndSingularJSON True, MTVndSingularJSON False] && queryTotal /= 1) $ do when (elem mediaType [MTVndSingularJSON True, MTVndSingularJSON False] && queryTotal /= 1) $ do
lift SQL.condemn lift SQL.condemn
throwError $ Error.ApiRequestErr . Error.SingularityError $ toInteger queryTotal throwError $ Error.ApiRequestError . Error.SingularityError $ toInteger queryTotal
failExceedsMaxAffectedPref :: (Maybe PreferMaxAffected, Maybe PreferHandling) -> ResultSet -> DbHandler () failExceedsMaxAffectedPref :: (Maybe PreferMaxAffected, Maybe PreferHandling) -> ResultSet -> DbHandler ()
failExceedsMaxAffectedPref (Nothing,_) _ = pure () failExceedsMaxAffectedPref (Nothing,_) _ = pure ()
failExceedsMaxAffectedPref (Just (PreferMaxAffected n), handling) RSStandard{rsQueryTotal=queryTotal} = when ((queryTotal > n) && (handling == Just Strict)) $ do failExceedsMaxAffectedPref (Just (PreferMaxAffected n), handling) RSStandard{rsQueryTotal=queryTotal} = when ((queryTotal > n) && (handling == Just Strict)) $ do
lift SQL.condemn lift SQL.condemn
throwError $ Error.ApiRequestErr . Error.MaxAffectedViolationError $ toInteger queryTotal throwError $ Error.ApiRequestError . Error.MaxAffectedViolationError $ toInteger queryTotal
-- | Set a transaction to roll back if requested -- | Set a transaction to roll back if requested
optionalRollback :: AppConfig -> ApiRequest -> DbHandler () optionalRollback :: AppConfig -> ApiRequest -> DbHandler ()
+3 -3
View File
@@ -50,10 +50,10 @@ init configDbPoolSize = do
-- Only some observations are used as metrics -- Only some observations are used as metrics
observationMetrics :: MetricsState -> ObservationHandler observationMetrics :: MetricsState -> ObservationHandler
observationMetrics MetricsState{..} obs = case obs of observationMetrics MetricsState{..} obs = case obs of
PoolAcqTimeoutObs -> do (PoolAcqTimeoutObs _) -> do
incCounter poolTimeouts incCounter poolTimeouts
(HasqlPoolObs (SQL.ConnectionObservation _ status)) -> case status of (HasqlPoolObs (SQL.ConnectionObservation _ status)) -> case status of
SQL.ReadyForUseConnectionStatus _ -> do SQL.ReadyForUseConnectionStatus -> do
incGauge poolAvailable incGauge poolAvailable
SQL.InUseConnectionStatus -> do SQL.InUseConnectionStatus -> do
decGauge poolAvailable decGauge poolAvailable
@@ -64,7 +64,7 @@ observationMetrics MetricsState{..} obs = case obs of
incGauge poolWaiting incGauge poolWaiting
PoolRequestFullfilled -> PoolRequestFullfilled ->
decGauge poolWaiting decGauge poolWaiting
SchemaCacheLoadedObs resTime _ -> do SchemaCacheLoadedObs resTime -> do
withLabel schemaCacheLoads "SUCCESS" incCounter withLabel schemaCacheLoads "SUCCESS" incCounter
setGauge schemaCacheQueryTime resTime setGauge schemaCacheQueryTime resTime
SchemaCacheErrorObs{} -> do SchemaCacheErrorObs{} -> do
+129 -5
View File
@@ -1,4 +1,5 @@
{-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE LambdaCase #-}
{-| {-|
Module : PostgREST.Observation Module : PostgREST.Observation
Description : This module holds an Observation type which is the core of Observability for PostgREST. Description : This module holds an Observation type which is the core of Observability for PostgREST.
@@ -9,16 +10,24 @@ Description : This module holds an Observation type which is the core of Observa
module PostgREST.Observation module PostgREST.Observation
( Observation(..) ( Observation(..)
, ObsFatalError(..) , ObsFatalError(..)
, observationMessage
, ObservationHandler , ObservationHandler
, showOnSingleLine
, isDbListenerBug
) where ) where
import qualified Data.ByteString.Lazy as LBS
import Data.List.NonEmpty (toList)
import qualified Data.Text as T
import qualified Data.Text.Encoding as T
import qualified Hasql.Connection as SQL import qualified Hasql.Connection as SQL
import qualified Hasql.Pool as SQL import qualified Hasql.Pool as SQL
import qualified Hasql.Pool.Observation as SQL import qualified Hasql.Pool.Observation as SQL
import Network.HTTP.Types.Status (Status) import Network.HTTP.Types.Status (Status)
import Numeric (showFFloat)
import PostgREST.Config.PgVersion import PostgREST.Config.PgVersion
import qualified PostgREST.Error as Error
import PostgREST.Query (MainQuery) import PostgREST.Query (MainQuery)
import PostgREST.SchemaCache (QueryTimings)
import Protolude hiding (toList) import Protolude hiding (toList)
@@ -32,8 +41,9 @@ data Observation
| DBConnectedObs Text | DBConnectedObs Text
| SchemaCacheEmptyObs | SchemaCacheEmptyObs
| SchemaCacheErrorObs (NonEmpty Text) [Text] SQL.UsageError | SchemaCacheErrorObs (NonEmpty Text) [Text] SQL.UsageError
| SchemaCacheQueriedObs Double (Maybe QueryTimings) | SchemaCacheQueriedObs Double
| SchemaCacheLoadedObs Double Text | SchemaCacheSummaryObs Text
| SchemaCacheLoadedObs Double
| ConnectionRetryObs Int | ConnectionRetryObs Int
| DBListenStart (Maybe ByteString) (Maybe ByteString) Text Text -- host, port, version string, channel | DBListenStart (Maybe ByteString) (Maybe ByteString) Text Text -- host, port, version string, channel
| DBListenFail Text (Either SQL.ConnectionError SomeException) | DBListenFail Text (Either SQL.ConnectionError SomeException)
@@ -50,11 +60,10 @@ data Observation
| QueryErrorCodeHighObs SQL.UsageError | QueryErrorCodeHighObs SQL.UsageError
| QueryPgVersionError SQL.UsageError | QueryPgVersionError SQL.UsageError
| PoolInit Int | PoolInit Int
| PoolAcqTimeoutObs | PoolAcqTimeoutObs SQL.UsageError
| HasqlPoolObs SQL.Observation | HasqlPoolObs SQL.Observation
| PoolRequest | PoolRequest
| PoolRequestFullfilled | PoolRequestFullfilled
| PoolFlushed
| JwtCacheLookup Bool | JwtCacheLookup Bool
| JwtCacheEviction | JwtCacheEviction
| TerminationUnixSignalObs Text | TerminationUnixSignalObs Text
@@ -64,3 +73,118 @@ data Observation
data ObsFatalError = ServerAuthError | ServerPgrstBug | ServerError42P05 | ServerError08P01 data ObsFatalError = ServerAuthError | ServerPgrstBug | ServerError42P05 | ServerError08P01
type ObservationHandler = Observation -> IO () type ObservationHandler = Observation -> IO ()
observationMessage :: Observation -> Text
observationMessage = \case
AdminStartObs address ->
"Admin server listening on " <> address
AppStartObs ver ->
"Starting PostgREST " <> T.decodeUtf8 ver <> "..."
AppServerAddressObs address ->
"API server listening on " <> address
DBConnectedObs ver ->
"Successfully connected to " <> ver
ExitUnsupportedPgVersion pgVer minPgVer ->
"Cannot run in this PostgreSQL version (" <> pgvName pgVer <> "), PostgREST needs at least " <> pgvName minPgVer
ExitDBNoRecoveryObs ->
"Automatic recovery disabled, exiting."
ExitDBFatalError ServerAuthError usageErr ->
"Failed to establish a connection. " <> jsonMessage usageErr
ExitDBFatalError ServerPgrstBug usageErr ->
"This is probably a bug in PostgREST, please report it at https://github.com/PostgREST/postgrest/issues. " <> jsonMessage usageErr
ExitDBFatalError ServerError42P05 usageErr ->
"If you are using connection poolers in transaction mode, try setting db-prepared-statements to false. " <> jsonMessage usageErr
ExitDBFatalError ServerError08P01 usageErr ->
"Connection poolers in statement mode are not supported." <> jsonMessage usageErr
SchemaCacheEmptyObs ->
T.decodeUtf8 . LBS.toStrict . Error.errorPayload $ Error.NoSchemaCacheError
SchemaCacheErrorObs dbSchemas extraPaths usageErr ->
"Failed to load the schema cache using "
<> "db-schemas=" <> T.intercalate "," (toList dbSchemas)
<> " and "
<> "db-extra-search-path=" <> T.intercalate "," extraPaths
<> ". " <> jsonMessage usageErr
SchemaCacheQueriedObs resultTime ->
"Schema cache queried in " <> showMillis resultTime <> " milliseconds"
SchemaCacheSummaryObs summary ->
"Schema cache loaded " <> summary
SchemaCacheLoadedObs resultTime ->
"Schema cache loaded in " <> showMillis resultTime <> " milliseconds"
ConnectionRetryObs delay ->
"Attempting to reconnect to the database in " <> (show delay::Text) <> " seconds..."
QueryPgVersionError usageErr ->
"Failed to query the PostgreSQL version. " <> jsonMessage usageErr
DBListenStart host port fullName channel -> do
"Listener connected to " <> fullName <> " on " <> show (fold $ host <> fmap (":" <>) port) <> " and listening for database notifications on the " <> show channel <> " channel"
DBListenFail channel listenErr ->
"Failed listening for database notifications on the " <> show channel <> " channel. " <>
either showListenerConnError showListenerException listenErr
DBListenRetry delay ->
"Retrying listening for database notifications in " <> (show delay::Text) <> " seconds..."
DBListenBugHint ->
"HINT: This is likely a bug in the notification queue, try executing the following to solve it: select pg_notification_queue_usage();"
DBListenerGotSCacheMsg channel ->
"Received a schema cache reload message on the " <> show channel <> " channel"
DBListenerGotConfigMsg channel ->
"Received a config reload message on the " <> show channel <> " channel"
DBListenerConnectionCleanupFail ex ->
"Failed during listener connection cleanup: " <> showOnSingleLine '\t' (show ex)
QueryObs{} ->
mempty -- TODO pending refactor: The logic for printing the query cannot be done here. Join the observationMessage function into observationLogger to avoid this mempty.
ConfigReadErrorObs usageErr ->
"Failed to query database settings for the config parameters." <> jsonMessage usageErr
QueryRoleSettingsErrorObs usageErr ->
"Failed to query the role settings. " <> jsonMessage usageErr
QueryErrorCodeHighObs usageErr ->
jsonMessage usageErr
ConfigInvalidObs err ->
"Failed reloading config: " <> err
ConfigSucceededObs ->
"Config reloaded"
PoolInit poolSize ->
"Connection Pool initialized with a maximum size of " <> show poolSize <> " connections"
PoolAcqTimeoutObs usageErr ->
jsonMessage usageErr
HasqlPoolObs (SQL.ConnectionObservation uuid status) ->
"Connection " <> show uuid <> (
case status of
SQL.ConnectingConnectionStatus -> " is being established"
SQL.ReadyForUseConnectionStatus -> " is available"
SQL.InUseConnectionStatus -> " is used"
SQL.TerminatedConnectionStatus reason -> " is terminated due to " <> case reason of
SQL.AgingConnectionTerminationReason -> "max lifetime"
SQL.IdlenessConnectionTerminationReason -> "max idletime"
SQL.ReleaseConnectionTerminationReason -> "release"
SQL.NetworkErrorConnectionTerminationReason _ -> "network error" -- usage error is already logged, no need to repeat the same message.
)
PoolRequest ->
"Trying to borrow a connection from pool"
PoolRequestFullfilled ->
"Borrowed a connection from the pool"
JwtCacheLookup _ ->
"Looked up a JWT in JWT cache"
JwtCacheEviction ->
"Evicted entry from JWT cache"
TerminationUnixSignalObs signal ->
"Received termination unix signal " <> signal
WarpServerObs txt ->
"Warp server: " <> txt
where
showMillis :: Double -> Text
showMillis x = toS $ showFFloat (Just 1) x ""
jsonMessage err = T.decodeUtf8 . LBS.toStrict . Error.errorPayload $ Error.PgError False err
showListenerConnError :: SQL.ConnectionError -> Text
showListenerConnError = maybe "Connection error" (showOnSingleLine '\t' . T.decodeUtf8)
showListenerException :: SomeException -> Text
showListenerException = showOnSingleLine '\t' . show
showOnSingleLine :: Char -> Text -> Text
showOnSingleLine split txt = T.intercalate " " $ T.filter (/= split) <$> T.lines txt -- the errors from hasql-notifications come intercalated with "\t\n"
isDbListenerBug :: SomeException -> Bool
isDbListenerBug e = "could not access status of transaction" `T.isInfixOf` show e
+51 -17
View File
@@ -43,7 +43,6 @@ import PostgREST.Error (ApiRequestError (..),
Error (..), Error (..),
SchemaCacheError (..)) SchemaCacheError (..))
import PostgREST.MediaType (MediaType (..)) import PostgREST.MediaType (MediaType (..))
import PostgREST.Plan.Negotiate (negotiateContent)
import PostgREST.Query.SqlFragment (sourceCTEName) import PostgREST.Query.SqlFragment (sourceCTEName)
import PostgREST.RangeQuery (NonnegRange, allRange, import PostgREST.RangeQuery (NonnegRange, allRange,
convertToLimitZeroRange, convertToLimitZeroRange,
@@ -51,6 +50,7 @@ import PostgREST.RangeQuery (NonnegRange, allRange,
import PostgREST.SchemaCache (SchemaCache (..)) import PostgREST.SchemaCache (SchemaCache (..))
import PostgREST.SchemaCache.Identifiers (FieldName, import PostgREST.SchemaCache.Identifiers (FieldName,
QualifiedIdentifier (..), QualifiedIdentifier (..),
RelIdentifier (..),
Schema) Schema)
import PostgREST.SchemaCache.Relationship (Cardinality (..), import PostgREST.SchemaCache.Relationship (Cardinality (..),
Junction (..), Junction (..),
@@ -60,6 +60,8 @@ import PostgREST.SchemaCache.Relationship (Cardinality (..),
import PostgREST.SchemaCache.Representations (DataRepresentation (..), import PostgREST.SchemaCache.Representations (DataRepresentation (..),
RepresentationsMap) RepresentationsMap)
import PostgREST.SchemaCache.Routine (MediaHandler (..), import PostgREST.SchemaCache.Routine (MediaHandler (..),
MediaHandlerMap,
ResolvedHandler,
Routine (..), Routine (..),
RoutineMap, RoutineMap,
RoutineParam (..), RoutineParam (..),
@@ -172,8 +174,8 @@ wrappedReadPlan :: QualifiedIdentifier -> AppConfig -> SchemaCache -> ApiRequest
wrappedReadPlan identifier conf sCache apiRequest@ApiRequest{iPreferences=Preferences{..},..} headersOnly = do wrappedReadPlan identifier conf sCache apiRequest@ApiRequest{iPreferences=Preferences{..},..} headersOnly = do
qi <- findTable identifier sCache qi <- findTable identifier sCache
rPlan <- readPlan qi conf sCache apiRequest rPlan <- readPlan qi conf sCache apiRequest
(handler, mediaType) <- mapLeft ApiRequestErr $ negotiateContent conf apiRequest qi iAcceptMediaType (dbMediaHandlers sCache) (hasDefaultSelect rPlan) (handler, mediaType) <- mapLeft ApiRequestError $ negotiateContent conf apiRequest qi iAcceptMediaType (dbMediaHandlers sCache) (hasDefaultSelect rPlan)
if not (null invalidPrefs) && preferHandling == Just Strict then Left $ ApiRequestErr $ InvalidPreferences invalidPrefs else Right () if not (null invalidPrefs) && preferHandling == Just Strict then Left $ ApiRequestError $ InvalidPreferences invalidPrefs else Right ()
return $ WrappedReadPlan rPlan SQL.Read handler mediaType headersOnly qi return $ WrappedReadPlan rPlan SQL.Read handler mediaType headersOnly qi
mutateReadPlan :: Mutation -> ApiRequest -> QualifiedIdentifier -> AppConfig -> SchemaCache -> Either Error CrudPlan mutateReadPlan :: Mutation -> ApiRequest -> QualifiedIdentifier -> AppConfig -> SchemaCache -> Either Error CrudPlan
@@ -181,8 +183,8 @@ mutateReadPlan mutation apiRequest@ApiRequest{iPreferences=Preferences{..},..}
qi <- findTable identifier sCache qi <- findTable identifier sCache
rPlan <- readPlan qi conf sCache apiRequest rPlan <- readPlan qi conf sCache apiRequest
mPlan <- mutatePlan mutation qi apiRequest sCache rPlan mPlan <- mutatePlan mutation qi apiRequest sCache rPlan
if not (null invalidPrefs) && preferHandling == Just Strict then Left $ ApiRequestErr $ InvalidPreferences invalidPrefs else Right () if not (null invalidPrefs) && preferHandling == Just Strict then Left $ ApiRequestError $ InvalidPreferences invalidPrefs else Right ()
(handler, mediaType) <- mapLeft ApiRequestErr $ negotiateContent conf apiRequest qi iAcceptMediaType (dbMediaHandlers sCache) (hasDefaultSelect rPlan) (handler, mediaType) <- mapLeft ApiRequestError $ negotiateContent conf apiRequest qi iAcceptMediaType (dbMediaHandlers sCache) (hasDefaultSelect rPlan)
return $ MutateReadPlan rPlan mPlan SQL.Write handler mediaType mutation qi return $ MutateReadPlan rPlan mPlan SQL.Write handler mediaType mutation qi
callReadPlan :: QualifiedIdentifier -> AppConfig -> SchemaCache -> ApiRequest -> InvokeMethod -> Either Error CrudPlan callReadPlan :: QualifiedIdentifier -> AppConfig -> SchemaCache -> ApiRequest -> InvokeMethod -> Either Error CrudPlan
@@ -204,15 +206,15 @@ callReadPlan identifier conf sCache apiRequest@ApiRequest{iPreferences=Preferenc
(Inv, Routine.Immutable) -> SQL.Read (Inv, Routine.Immutable) -> SQL.Read
(Inv, Routine.Volatile) -> SQL.Write (Inv, Routine.Volatile) -> SQL.Write
cPlan = callPlan proc apiRequest paramKeys args rPlan cPlan = callPlan proc apiRequest paramKeys args rPlan
(handler, mediaType) <- mapLeft ApiRequestErr $ negotiateContent conf apiRequest relIdentifier iAcceptMediaType (dbMediaHandlers sCache) (hasDefaultSelect rPlan) (handler, mediaType) <- mapLeft ApiRequestError $ negotiateContent conf apiRequest relIdentifier iAcceptMediaType (dbMediaHandlers sCache) (hasDefaultSelect rPlan)
if not (null invalidPrefs) && preferHandling == Just Strict then Left $ ApiRequestErr $ InvalidPreferences invalidPrefs else Right () if not (null invalidPrefs) && preferHandling == Just Strict then Left $ ApiRequestError $ InvalidPreferences invalidPrefs else Right ()
failMaxAffectedRpcReturnsSingle (preferMaxAffected, preferHandling) proc failMaxAffectedRpcReturnsSingle (preferMaxAffected, preferHandling) proc
return $ CallReadPlan rPlan cPlan txMode proc handler mediaType invMethod identifier return $ CallReadPlan rPlan cPlan txMode proc handler mediaType invMethod identifier
where where
qsParams' = QueryParams.qsParams iQueryParams qsParams' = QueryParams.qsParams iQueryParams
failMaxAffectedRpcReturnsSingle :: (Maybe PreferMaxAffected, Maybe PreferHandling) -> Routine -> Either Error () failMaxAffectedRpcReturnsSingle :: (Maybe PreferMaxAffected, Maybe PreferHandling) -> Routine -> Either Error ()
failMaxAffectedRpcReturnsSingle (Just (PreferMaxAffected _), Just Strict) rout = if funcReturnsSingle rout then Left $ ApiRequestErr MaxAffectedRpcViolation else Right () failMaxAffectedRpcReturnsSingle (Just (PreferMaxAffected _), Just Strict) rout = if funcReturnsSingle rout then Left $ ApiRequestError MaxAffectedRpcViolation else Right ()
failMaxAffectedRpcReturnsSingle _ _ = Right () failMaxAffectedRpcReturnsSingle _ _ = Right ()
hasDefaultSelect :: ReadPlanTree -> Bool hasDefaultSelect :: ReadPlanTree -> Bool
@@ -225,7 +227,7 @@ inspectPlan apiRequest headersOnly schema = do
accepts = iAcceptMediaType apiRequest accepts = iAcceptMediaType apiRequest
mediaType <- if not . null $ L.intersect accepts producedMTs mediaType <- if not . null $ L.intersect accepts producedMTs
then Right MTOpenAPI then Right MTOpenAPI
else Left . ApiRequestErr . MediaTypeError $ MediaType.toMime <$> accepts else Left . ApiRequestError . MediaTypeError $ MediaType.toMime <$> accepts
return $ InspectPlan mediaType SQL.Read headersOnly schema return $ InspectPlan mediaType SQL.Read headersOnly schema
{-| {-|
@@ -784,7 +786,7 @@ hoistIntoRelSelectFields _ r = r
-- to order once it's aggregated if it's not selected in the inner query beforehand. -- to order once it's aggregated if it's not selected in the inner query beforehand.
addToManyOrderSelects :: ReadPlanTree -> Either Error ReadPlanTree addToManyOrderSelects :: ReadPlanTree -> Either Error ReadPlanTree
addToManyOrderSelects (Node rp@ReadPlan{order, select, relAggAlias, relSelect, relSpread = Just ToManySpread {}} forest) addToManyOrderSelects (Node rp@ReadPlan{order, select, relAggAlias, relSelect, relSpread = Just ToManySpread {}} forest)
| anyAggSel || anyAggRelSel = Left $ ApiRequestErr $ NotImplemented "Aggregates are not implemented for one-to-many or many-to-many spreads." | anyAggSel || anyAggRelSel = Left $ ApiRequestError $ NotImplemented "Aggregates are not implemented for one-to-many or many-to-many spreads."
| otherwise = Node rp { order = [], relSpread = newRelSpread } <$> addToManyOrderSelects `traverse` forest | otherwise = Node rp { order = [], relSpread = newRelSpread } <$> addToManyOrderSelects `traverse` forest
where where
newRelSpread = Just ToManySpread { stExtraSelect = addSprExtraSelects, stOrder = addSprOrder} newRelSpread = Just ToManySpread { stExtraSelect = addSprExtraSelects, stOrder = addSprOrder}
@@ -806,7 +808,7 @@ addToManyOrderSelects (Node rp forest) = Node rp <$> addToManyOrderSelects `trav
validateAggFunctions :: Bool -> ReadPlanTree -> Either Error ReadPlanTree validateAggFunctions :: Bool -> ReadPlanTree -> Either Error ReadPlanTree
validateAggFunctions aggFunctionsAllowed (Node rp@ReadPlan {select} forest) validateAggFunctions aggFunctionsAllowed (Node rp@ReadPlan {select} forest)
| not aggFunctionsAllowed && any (isJust . csAggFunction) select = Left $ ApiRequestErr AggregatesNotAllowed | not aggFunctionsAllowed && any (isJust . csAggFunction) select = Left $ ApiRequestError AggregatesNotAllowed
| otherwise = Node rp <$> traverse (validateAggFunctions aggFunctionsAllowed) forest | otherwise = Node rp <$> traverse (validateAggFunctions aggFunctionsAllowed) forest
-- | Lookup table in the schema cache before creating read plan -- | Lookup table in the schema cache before creating read plan
@@ -860,9 +862,9 @@ addRelatedOrders (Node rp@ReadPlan{order,from} forest) = do
name = fromMaybe relName relAlias in name = fromMaybe relName relAlias in
if isToOne == Just True if isToOne == Just True
then Right $ cot{coRelation=relAggAlias} then Right $ cot{coRelation=relAggAlias}
else Left $ ApiRequestErr $ RelatedOrderNotToOne (qiName from) name else Left $ ApiRequestError $ RelatedOrderNotToOne (qiName from) name
Nothing -> Nothing ->
Left $ ApiRequestErr $ NotEmbedded coRelation Left $ ApiRequestError $ NotEmbedded coRelation
-- | Searches for null filters on embeds, e.g. `projects=not.is.null` on `GET /clients?select=*,projects(*)&projects=not.is.null` -- | Searches for null filters on embeds, e.g. `projects=not.is.null` on `GET /clients?select=*,projects(*)&projects=not.is.null`
-- --
@@ -956,7 +958,7 @@ addRanges ApiRequest{..} rReq =
_ -> foldr addRangeToNode (Right rReq) =<< ranges _ -> foldr addRangeToNode (Right rReq) =<< ranges
where where
ranges :: Either Error [(EmbedPath, NonnegRange)] ranges :: Either Error [(EmbedPath, NonnegRange)]
ranges = first (ApiRequestErr . QueryParamError) $ QueryParams.pRequestRange `traverse` HM.toList iRange ranges = first (ApiRequestError . QueryParamError) $ QueryParams.pRequestRange `traverse` HM.toList iRange
addRangeToNode :: (EmbedPath, NonnegRange) -> Either Error ReadPlanTree -> Either Error ReadPlanTree addRangeToNode :: (EmbedPath, NonnegRange) -> Either Error ReadPlanTree -> Either Error ReadPlanTree
addRangeToNode = updateNode (\r (Node q f) -> Node q{range_=r} f) addRangeToNode = updateNode (\r (Node q f) -> Node q{range_=r} f)
@@ -990,13 +992,13 @@ updateNode f ([], a) rr = f a <$> rr
updateNode _ _ (Left e) = Left e updateNode _ _ (Left e) = Left e
updateNode f (targetNodeName:remainingPath, a) (Right (Node rootNode forest)) = updateNode f (targetNodeName:remainingPath, a) (Right (Node rootNode forest)) =
case findNode of case findNode of
Nothing -> Left $ ApiRequestErr $ NotEmbedded targetNodeName Nothing -> Left $ ApiRequestError $ NotEmbedded targetNodeName
Just target -> Just target ->
(\node -> Node rootNode $ node : delete target forest) <$> (\node -> Node rootNode $ node : delete target forest) <$>
updateNode f (remainingPath, a) (Right target) updateNode f (remainingPath, a) (Right target)
where where
findNode :: Maybe ReadPlanTree findNode :: Maybe ReadPlanTree
findNode = find (\(Node ReadPlan{relName, relAlias} _) -> fromMaybe relName relAlias == targetNodeName) forest findNode = find (\(Node ReadPlan{relName, relAlias} _) -> relName == targetNodeName || relAlias == Just targetNodeName) forest
mutatePlan :: Mutation -> QualifiedIdentifier -> ApiRequest -> SchemaCache -> ReadPlanTree -> Either Error MutatePlan mutatePlan :: Mutation -> QualifiedIdentifier -> ApiRequest -> SchemaCache -> ReadPlanTree -> Either Error MutatePlan
mutatePlan mutation qi ApiRequest{iPreferences=Preferences{..}, ..} SchemaCache{dbTables, dbRepresentations} readReq = mutatePlan mutation qi ApiRequest{iPreferences=Preferences{..}, ..} SchemaCache{dbTables, dbRepresentations} readReq =
@@ -1014,7 +1016,7 @@ mutatePlan mutation qi ApiRequest{iPreferences=Preferences{..}, ..} SchemaCache{
_ -> False) qsFiltersRoot _ -> False) qsFiltersRoot
then mapRight (\typedColumns -> Insert qi typedColumns body (Just (MergeDuplicates, pkCols)) combinedLogic returnings mempty False) typedColumnsOrError then mapRight (\typedColumns -> Insert qi typedColumns body (Just (MergeDuplicates, pkCols)) combinedLogic returnings mempty False) typedColumnsOrError
else else
Left $ ApiRequestErr InvalidFilters Left $ ApiRequestError InvalidFilters
MutationDelete -> Right $ Delete qi combinedLogic returnings MutationDelete -> Right $ Delete qi combinedLogic returnings
where where
ctx = ResolverContext dbTables dbRepresentations qi "json" ctx = ResolverContext dbTables dbRepresentations qi "json"
@@ -1121,3 +1123,35 @@ inferColsEmbedNeeds (Node ReadPlan{select} forest) pkCols
-- they are later concatenated with AND in the QueryBuilder -- they are later concatenated with AND in the QueryBuilder
addFilterToLogicForest :: CoercibleFilter -> [CoercibleLogicTree] -> [CoercibleLogicTree] addFilterToLogicForest :: CoercibleFilter -> [CoercibleLogicTree] -> [CoercibleLogicTree]
addFilterToLogicForest flt lf = CoercibleStmnt flt : lf addFilterToLogicForest flt lf = CoercibleStmnt flt : lf
-- | Do content negotiation. i.e. choose a media type based on the intersection of accepted/produced media types.
negotiateContent :: AppConfig -> ApiRequest -> QualifiedIdentifier -> [MediaType] -> MediaHandlerMap -> Bool -> Either ApiRequestError ResolvedHandler
negotiateContent conf ApiRequest{iAction=act, iPreferences=Preferences{preferRepresentation=rep}} identifier accepts produces defaultSelect =
case (act, firstAcceptedPick) of
(_, Nothing) -> Left . MediaTypeError $ map MediaType.toMime accepts
(ActDb (ActRelationMut _ _), Just (x, mt)) -> Right (if rep == Just Full then x else NoAgg, mt)
-- no need for an aggregate on HEAD https://github.com/PostgREST/postgrest/issues/2849
-- TODO: despite no aggregate, these are responding with a Content-Type, which is not correct.
(ActDb (ActRelationRead _ True), Just (_, mt)) -> Right (NoAgg, mt)
(ActDb (ActRoutine _ (InvRead True)), Just (_, mt)) -> Right (NoAgg, mt)
(_, Just (x, mt)) -> Right (x, mt)
where
firstAcceptedPick = listToMaybe $ mapMaybe matchMT accepts -- If there are multiple accepted media types, pick the first. This is usual in content negotiation.
matchMT mt = case mt of
-- all the vendored media types have special handling as they have media type parameters, they cannot be overridden
m@(MTVndSingularJSON strip) -> Just (BuiltinAggSingleJson strip, m)
m@MTVndArrayJSONStrip -> Just (BuiltinAggArrayJsonStrip, m)
m@(MTVndPlan (MTVndSingularJSON strip) _ _) -> mtPlanToNothing $ Just (BuiltinAggSingleJson strip, m)
m@(MTVndPlan MTVndArrayJSONStrip _ _) -> mtPlanToNothing $ Just (BuiltinAggArrayJsonStrip, m)
-- TODO the plan should have its own MediaHandler instead of relying on MediaType
m@(MTVndPlan mType _ _) -> mtPlanToNothing $ ((,) . fst <$> lookupHandler mType) <*> pure m
-- all the other media types can be overridden
x -> lookupHandler x
mtPlanToNothing x = if configDbPlanEnabled conf then x else Nothing -- don't find anything if the plan media type is not allowed
lookupHandler mt =
when' defaultSelect (HM.lookup (RelId identifier, MTAny) produces) <|> -- lookup for identifier and `*/*`
when' defaultSelect (HM.lookup (RelId identifier, mt) produces) <|> -- lookup for identifier and a particular media type
HM.lookup (RelAnyElement, mt) produces -- lookup for anyelement and a particular media type
when' :: Bool -> Maybe a -> Maybe a
when' True (Just a) = Just a
when' _ _ = Nothing
-82
View File
@@ -1,82 +0,0 @@
{-|
Module : PostgREST.Plan.Negotiate
Description : PostgREST Content Negotiation
This module contains logic for content negotiation.
RFC: https://datatracker.ietf.org/doc/html/rfc7231#section-3.4
-}
module PostgREST.Plan.Negotiate
( negotiateContent
) where
import qualified Data.HashMap.Strict as HM
import PostgREST.ApiRequest (ApiRequest (..))
import PostgREST.Config (AppConfig (..))
import PostgREST.Error (ApiRequestError (..))
import PostgREST.MediaType (MediaType (..))
import PostgREST.SchemaCache.Identifiers (QualifiedIdentifier (..),
RelIdentifier (..))
import PostgREST.SchemaCache.Routine (MediaHandler (..),
MediaHandlerMap,
ResolvedHandler)
import PostgREST.ApiRequest.Preferences
import PostgREST.ApiRequest.Types
import qualified PostgREST.MediaType as MediaType
import Protolude hiding (from)
-- We have two general cases of return values from database objects
-- (tables/views/functions):
--
-- 1. "un-mime-typed" values, in most of the cases this is a composite/row
-- value, for example for tables or views, but also often for functions.
-- It can be simple integer values or text or bytea as well.
--
-- For this, we need handlers to transform the "non-mime-typed" values
-- into "mimetypes". We have a default builtin handler that does
-- "application/json". We can add more handlers via aggregates.
--
-- 2. "mime-typed" values, which specifically return a domain type that is
-- associated to a certain mimetype. e.g, a function returning only
-- "image/png".
--
-- FIXME:
-- If the function returns a domain type - let's say image/png, we should
-- accept */*, image/*, and image/png.
-- Related issue: https://github.com/PostgREST/postgrest/issues/3391
-- | Do content negotiation. i.e. choose a media type based on the
-- intersection of accepted/produced media types.
negotiateContent :: AppConfig -> ApiRequest -> QualifiedIdentifier -> [MediaType] -> MediaHandlerMap -> Bool -> Either ApiRequestError ResolvedHandler
negotiateContent conf ApiRequest{iAction=act, iPreferences=Preferences{preferRepresentation=rep}} identifier accepts produces defaultSelect =
case (act, firstAcceptedPick) of
(_, Nothing) -> Left . MediaTypeError $ map MediaType.toMime accepts
(ActDb (ActRelationMut _ _), Just (x, mt)) -> Right (if rep == Just Full then x else NoAgg, mt)
-- no need for an aggregate on HEAD https://github.com/PostgREST/postgrest/issues/2849
-- TODO: despite no aggregate, these are responding with a Content-Type, which is not correct.
(ActDb (ActRelationRead _ True), Just (_, mt)) -> Right (NoAgg, mt)
(ActDb (ActRoutine _ (InvRead True)), Just (_, mt)) -> Right (NoAgg, mt)
(_, Just (x, mt)) -> Right (x, mt)
where
firstAcceptedPick = listToMaybe $ mapMaybe matchMT accepts -- If there are multiple accepted media types, pick the first. This is usual in content negotiation.
matchMT mt = case mt of
-- all the vendored media types have special handling as they have media type parameters, they cannot be overridden
m@(MTVndSingularJSON strip) -> Just (BuiltinAggSingleJson strip, m)
m@MTVndArrayJSONStrip -> Just (BuiltinAggArrayJsonStrip, m)
m@(MTVndPlan (MTVndSingularJSON strip) _ _) -> mtPlanToNothing $ Just (BuiltinAggSingleJson strip, m)
m@(MTVndPlan MTVndArrayJSONStrip _ _) -> mtPlanToNothing $ Just (BuiltinAggArrayJsonStrip, m)
-- TODO the plan should have its own MediaHandler instead of relying on MediaType
m@(MTVndPlan mType _ _) -> mtPlanToNothing $ ((,) . fst <$> lookupHandler mType) <*> pure m
-- all the other media types can be overridden
x -> lookupHandler x
mtPlanToNothing x = if configDbPlanEnabled conf then x else Nothing -- don't find anything if the plan media type is not allowed
lookupHandler mt =
when' defaultSelect (HM.lookup (RelId identifier, MTAny) produces) <|> -- lookup for identifier and `*/*`
when' defaultSelect (HM.lookup (RelId identifier, mt) produces) <|> -- lookup for identifier and a particular media type
HM.lookup (RelAnyElement, mt) produces -- lookup for anyelement and a particular media type
when' :: Bool -> Maybe a -> Maybe a
when' True (Just a) = Just a
when' _ _ = Nothing
+1 -1
View File
@@ -47,7 +47,7 @@ data CoercibleField = CoercibleField
, cfBaseType :: Text -- ^ The base type of the field in case of domains, or just the type otherwise (without modifiers in case of pg_catalog types) , cfBaseType :: Text -- ^ The base type of the field in case of domains, or just the type otherwise (without modifiers in case of pg_catalog types)
, cfTransform :: Maybe TransformerProc -- ^ The optional mapping from irType -> targetType. , cfTransform :: Maybe TransformerProc -- ^ The optional mapping from irType -> targetType.
, cfDefault :: Maybe Text , cfDefault :: Maybe Text
, cfFullRow :: Bool -- ^ True if the field represents the whole selected row. Used in spread rels: instead of COUNT(*), it does a COUNT(<row>) in order to not mix with other spread resources. , cfFullRow :: Bool -- ^ True if the field represents the whole selected row. Used in spread rels: instead of COUNT(*), it does a COUNT(<row>) in order to not mix with other spreaded resources.
} deriving (Eq, Show) } deriving (Eq, Show)
unknownField :: FieldName -> JsonPath -> CoercibleField unknownField :: FieldName -> JsonPath -> CoercibleField
+3 -3
View File
@@ -43,16 +43,16 @@ data MainQuery = MainQuery
mainQuery :: ActionPlan -> AppConfig -> ApiRequest -> AuthResult -> Maybe QualifiedIdentifier -> MainQuery mainQuery :: ActionPlan -> AppConfig -> ApiRequest -> AuthResult -> Maybe QualifiedIdentifier -> MainQuery
mainQuery (NoDb _) _ _ _ _ = MainQuery mempty Nothing mempty (mempty, mempty, mempty) mempty mainQuery (NoDb _) _ _ _ _ = MainQuery mempty Nothing mempty (mempty, mempty, mempty) mempty
mainQuery (Db plan) conf@AppConfig{..} apiReq@ApiRequest{iTopLevelRange=range, iPreferences=Preferences{..}} authRes preReq = mainQuery (Db plan) conf@AppConfig{..} apiReq@ApiRequest{iPreferences=Preferences{..}} authRes preReq =
let genQ = MainQuery (PreQuery.txVarQuery plan conf authRes apiReq) (PreQuery.preReqQuery <$> preReq) in let genQ = MainQuery (PreQuery.txVarQuery plan conf authRes apiReq) (PreQuery.preReqQuery <$> preReq) in
case plan of case plan of
DbCrud _ WrappedReadPlan{..} -> DbCrud _ WrappedReadPlan{..} ->
let countQuery = QueryBuilder.readPlanToCountQuery wrReadPlan in let countQuery = QueryBuilder.readPlanToCountQuery wrReadPlan in
genQ (Statements.mainRead wrReadPlan countQuery preferCount configDbMaxRows range pMedia wrHandler) (mempty, mempty, mempty) genQ (Statements.mainRead wrReadPlan countQuery preferCount configDbMaxRows pMedia wrHandler) (mempty, mempty, mempty)
(if shouldExplainCount preferCount then Just (Statements.postExplain countQuery) else Nothing) (if shouldExplainCount preferCount then Just (Statements.postExplain countQuery) else Nothing)
DbCrud _ MutateReadPlan{..} -> DbCrud _ MutateReadPlan{..} ->
genQ (Statements.mainWrite mrReadPlan mrMutatePlan pMedia mrHandler preferRepresentation preferResolution) (mempty, mempty, mempty) mempty genQ (Statements.mainWrite mrReadPlan mrMutatePlan pMedia mrHandler preferRepresentation preferResolution) (mempty, mempty, mempty) mempty
DbCrud _ CallReadPlan{..} -> DbCrud _ CallReadPlan{..} ->
genQ (Statements.mainCall crProc crCallPlan crReadPlan preferCount configDbMaxRows range pMedia crHandler) (mempty, mempty, mempty) mempty genQ (Statements.mainCall crProc crCallPlan crReadPlan preferCount pMedia crHandler) (mempty, mempty, mempty) mempty
MayUseDb InspectPlan{ipSchema=tSchema} -> MayUseDb InspectPlan{ipSchema=tSchema} ->
genQ mempty (SqlFragment.accessibleTables tSchema, SqlFragment.accessibleFuncs tSchema, SqlFragment.schemaDescription tSchema) mempty genQ mempty (SqlFragment.accessibleTables tSchema, SqlFragment.accessibleFuncs tSchema, SqlFragment.schemaDescription tSchema) mempty
+1 -5
View File
@@ -10,7 +10,6 @@ module PostgREST.Query.PreQuery
) where ) where
import qualified Data.Aeson as JSON import qualified Data.Aeson as JSON
import qualified Data.Aeson.KeyMap as KM
import qualified Data.ByteString.Lazy.Char8 as LBS import qualified Data.ByteString.Lazy.Char8 as LBS
import qualified Data.HashMap.Strict as HM import qualified Data.HashMap.Strict as HM
import qualified Hasql.DynamicStatements.Snippet as SQL hiding (sql) import qualified Hasql.DynamicStatements.Snippet as SQL hiding (sql)
@@ -47,10 +46,7 @@ txVarQuery dbActPlan AppConfig{..} AuthResult{..} ApiRequest{..} =
pathSql = setConfigWithConstantName ("request.path", iPath) pathSql = setConfigWithConstantName ("request.path", iPath)
headersSql = setConfigWithConstantNameJSON "request.headers" iHeaders headersSql = setConfigWithConstantNameJSON "request.headers" iHeaders
cookiesSql = setConfigWithConstantNameJSON "request.cookies" iCookies cookiesSql = setConfigWithConstantNameJSON "request.cookies" iCookies
claimsSql = [setConfigWithConstantName ("request.jwt.claims", LBS.toStrict $ JSON.encode claims)] claimsSql = [setConfigWithConstantName ("request.jwt.claims", LBS.toStrict $ JSON.encode authClaims)]
where
claims = authClaims & KM.insert "role" (JSON.String $ decodeUtf8 authRole) -- insert "role" to claims as well
roleSql = [setConfigWithConstantName ("role", authRole)] roleSql = [setConfigWithConstantName ("role", authRole)]
roleSettingsSql = setConfigWithDynamicName <$> HM.toList (fromMaybe mempty $ HM.lookup authRole configRoleSettings) roleSettingsSql = setConfigWithDynamicName <$> HM.toList (fromMaybe mempty $ HM.lookup authRole configRoleSettings)
appSettingsSql = setConfigWithDynamicName . join bimap toUtf8 <$> configAppSettings appSettingsSql = setConfigWithDynamicName . join bimap toUtf8 <$> configAppSettings
+1 -1
View File
@@ -264,7 +264,7 @@ readPlanToCountQuery (Node ReadPlan{from=mainQi, fromAlias=tblAlias, where_=logi
limitedQuery :: SQL.Snippet -> Maybe Integer -> SQL.Snippet limitedQuery :: SQL.Snippet -> Maybe Integer -> SQL.Snippet
limitedQuery query maxRows = query <> SQL.sql (maybe mempty (\x -> " LIMIT " <> BS.pack (show x)) maxRows) limitedQuery query maxRows = query <> SQL.sql (maybe mempty (\x -> " LIMIT " <> BS.pack (show x)) maxRows)
-- TODO refactor so this function is unneeded and ComputedRelationship QualifiedIdentifier comes from the ReadPlan type -- TODO refactor so this function is uneeded and ComputedRelationship QualifiedIdentifier comes from the ReadPlan type
getQualifiedIdentifier :: Maybe Relationship -> QualifiedIdentifier -> Maybe Alias -> QualifiedIdentifier getQualifiedIdentifier :: Maybe Relationship -> QualifiedIdentifier -> Maybe Alias -> QualifiedIdentifier
getQualifiedIdentifier rel mainQi tblAlias = case rel of getQualifiedIdentifier rel mainQi tblAlias = case rel of
Just ComputedRelationship{relFunction} -> QualifiedIdentifier mempty $ fromMaybe (qiName relFunction) tblAlias Just ComputedRelationship{relFunction} -> QualifiedIdentifier mempty $ fromMaybe (qiName relFunction) tblAlias
+9 -17
View File
@@ -23,7 +23,6 @@ module PostgREST.Query.SqlFragment
, locationF , locationF
, noLocationF , noLocationF
, orderF , orderF
, pageCountSelectF
, pgFmtColumn , pgFmtColumn
, pgFmtFilter , pgFmtFilter
, pgFmtIdent , pgFmtIdent
@@ -97,7 +96,6 @@ import PostgREST.SchemaCache.Routine (MediaHandler (..),
Routine (..), Routine (..),
funcReturnsScalar, funcReturnsScalar,
funcReturnsSetOfScalar, funcReturnsSetOfScalar,
funcReturnsSingle,
funcReturnsSingleComposite) funcReturnsSingleComposite)
import Protolude hiding (Sum, cast) import Protolude hiding (Sum, cast)
@@ -487,21 +485,15 @@ pgFmtGroup _ CoercibleSelectField{csAggFunction=Just _} = Nothing
pgFmtGroup _ CoercibleSelectField{csAlias=Just alias, csAggFunction=Nothing} = Just $ pgFmtIdent alias pgFmtGroup _ CoercibleSelectField{csAlias=Just alias, csAggFunction=Nothing} = Just $ pgFmtIdent alias
pgFmtGroup qi CoercibleSelectField{csField=fld, csAlias=Nothing, csAggFunction=Nothing} = Just $ pgFmtField qi fld pgFmtGroup qi CoercibleSelectField{csField=fld, csAlias=Nothing, csAggFunction=Nothing} = Just $ pgFmtField qi fld
countF :: SQL.Snippet -> SQL.Snippet -> Bool -> Maybe Integer -> NonnegRange -> (SQL.Snippet, SQL.Snippet) countF :: SQL.Snippet -> Bool -> (SQL.Snippet, SQL.Snippet)
countF countQuery pageCountSelect shouldCount maxRows range countF countQuery shouldCount =
| shouldCount = if isJust maxRows || range /= allRange if shouldCount
then ( ", pgrst_source_count AS (" <> countQuery <> ")" then (
, "(SELECT pg_catalog.count(*) FROM pgrst_source_count)" ) ", pgrst_source_count AS (" <> countQuery <> ")"
-- When there are no db-max-rows and limits/offsets, the total count will be the same as the page count, , "(SELECT pg_catalog.count(*) FROM pgrst_source_count)" )
-- so we use the same page count here to avoid doing a separate aggregated count. else (
else ( mempty, pageCountSelect ) mempty
| otherwise = ( mempty, "null::bigint" ) , "null::bigint")
pageCountSelectF :: Maybe Routine -> SQL.Snippet
pageCountSelectF rout =
if maybe False funcReturnsSingle rout
then "1"
else "pg_catalog.count(_postgrest_t)"
returningF :: QualifiedIdentifier -> [FieldName] -> SQL.Snippet returningF :: QualifiedIdentifier -> [FieldName] -> SQL.Snippet
returningF qi returnings = returningF qi returnings =
+13 -13
View File
@@ -20,8 +20,8 @@ import PostgREST.Plan.MutatePlan as MTPlan
import PostgREST.Plan.ReadPlan import PostgREST.Plan.ReadPlan
import PostgREST.Query.QueryBuilder import PostgREST.Query.QueryBuilder
import PostgREST.Query.SqlFragment import PostgREST.Query.SqlFragment
import PostgREST.RangeQuery (NonnegRange) import PostgREST.SchemaCache.Routine (MediaHandler (..), Routine,
import PostgREST.SchemaCache.Routine (MediaHandler (..), Routine) funcReturnsSingle)
import Protolude import Protolude
@@ -64,24 +64,23 @@ mainWrite rPlan mtplan mt handler rep resolution = mtSnippet mt snippet
_ -> (False,False, mempty); _ -> (False,False, mempty);
mainRead :: ReadPlanTree -> SQL.Snippet -> Maybe PreferCount -> Maybe Integer -> mainRead :: ReadPlanTree -> SQL.Snippet -> Maybe PreferCount -> Maybe Integer ->
NonnegRange -> MediaType -> MediaHandler -> SQL.Snippet MediaType -> MediaHandler -> SQL.Snippet
mainRead rPlan countQuery pCount maxRows range mt handler = mtSnippet mt snippet mainRead rPlan countQuery pCount maxRows mt handler = mtSnippet mt snippet
where where
snippet = snippet =
"WITH " <> sourceCTE <> " AS ( " <> selectQuery <> " ) " <> "WITH " <> sourceCTE <> " AS ( " <> selectQuery <> " ) " <>
countCTEF <> " " <> countCTEF <> " " <>
"SELECT " <> "SELECT " <>
countResultF <> " AS total_result_set, " <> countResultF <> " AS total_result_set, " <>
pageCountSelect <> " AS page_total, " <> "pg_catalog.count(_postgrest_t) AS page_total, " <>
handlerF Nothing handler <> " AS body, " <> handlerF Nothing handler <> " AS body, " <>
responseHeadersF <> " AS response_headers, " <> responseHeadersF <> " AS response_headers, " <>
responseStatusF <> " AS response_status, " <> responseStatusF <> " AS response_status, " <>
"''" <> " AS response_inserted " <> "''" <> " AS response_inserted " <>
"FROM ( SELECT * FROM " <> sourceCTE <> " ) _postgrest_t" "FROM ( SELECT * FROM " <> sourceCTE <> " ) _postgrest_t"
(countCTEF, countResultF) = countF countQ pageCountSelect (shouldCount pCount) maxRows range (countCTEF, countResultF) = countF countQ $ shouldCount pCount
selectQuery = readPlanToQuery rPlan selectQuery = readPlanToQuery rPlan
pageCountSelect = pageCountSelectF Nothing
countQ = countQ =
if pCount == Just EstimatedCount then if pCount == Just EstimatedCount then
-- LIMIT maxRows + 1 so we can determine below that maxRows was surpassed -- LIMIT maxRows + 1 so we can determine below that maxRows was surpassed
@@ -89,27 +88,28 @@ mainRead rPlan countQuery pCount maxRows range mt handler = mtSnippet mt snippet
else else
countQuery countQuery
mainCall :: Routine -> CallPlan -> ReadPlanTree -> Maybe PreferCount -> Maybe Integer -> mainCall :: Routine -> CallPlan -> ReadPlanTree -> Maybe PreferCount ->
NonnegRange-> MediaType -> MediaHandler -> SQL.Snippet MediaType -> MediaHandler -> SQL.Snippet
mainCall rout cPlan rPlan pCount maxRows range mt handler = mtSnippet mt snippet mainCall rout cPlan rPlan pCount mt handler = mtSnippet mt snippet
where where
snippet = snippet =
"WITH " <> sourceCTE <> " AS (" <> callProcQuery <> ") " <> "WITH " <> sourceCTE <> " AS (" <> callProcQuery <> ") " <>
countCTEF <> countCTEF <>
"SELECT " <> "SELECT " <>
countResultF <> " AS total_result_set, " <> countResultF <> " AS total_result_set, " <>
pageCountSelect <> " AS page_total, " <> (if funcReturnsSingle rout
then "1"
else "pg_catalog.count(_postgrest_t)") <> " AS page_total, " <>
handlerF (Just rout) handler <> " AS body, " <> handlerF (Just rout) handler <> " AS body, " <>
responseHeadersF <> " AS response_headers, " <> responseHeadersF <> " AS response_headers, " <>
responseStatusF <> " AS response_status, " <> responseStatusF <> " AS response_status, " <>
"''" <> " AS response_inserted " <> "''" <> " AS response_inserted " <>
"FROM (" <> selectQuery <> ") _postgrest_t" "FROM (" <> selectQuery <> ") _postgrest_t"
(countCTEF, countResultF) = countF countQuery pageCountSelect (shouldCount pCount) maxRows range (countCTEF, countResultF) = countF countQuery $ shouldCount pCount
selectQuery = readPlanToQuery rPlan selectQuery = readPlanToQuery rPlan
callProcQuery = callPlanToQuery cPlan callProcQuery = callPlanToQuery cPlan
countQuery = readPlanToCountQuery rPlan countQuery = readPlanToCountQuery rPlan
pageCountSelect = pageCountSelectF (Just rout)
-- This occurs after the main query runs, that's why it's prefixed with "post" -- This occurs after the main query runs, that's why it's prefixed with "post"
postExplain :: SQL.Snippet -> SQL.Snippet postExplain :: SQL.Snippet -> SQL.Snippet
+17 -17
View File
@@ -60,9 +60,9 @@ data PgrstResponse = PgrstResponse {
, pgrstBody :: LBS.ByteString , pgrstBody :: LBS.ByteString
} }
actionResponse :: DbResult -> ApiRequest -> (Text, Text) -> AppConfig -> SchemaCache -> Either Error.Error PgrstResponse actionResponse :: DbResult -> ApiRequest -> (Text, Text) -> AppConfig -> SchemaCache -> Schema -> Bool -> Either Error.Error PgrstResponse
actionResponse (DbCrudResult plan@WrappedReadPlan{pMedia, wrHdrsOnly=headersOnly, crudQi=identifier} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ AppConfig{..} _ = do actionResponse (DbCrudResult plan@WrappedReadPlan{pMedia, wrHdrsOnly=headersOnly, crudQi=identifier} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ _ _ = do
let let
(status, contentRange) = RangeQuery.rangeStatusHeader iTopLevelRange rsQueryTotal rsTableTotal (status, contentRange) = RangeQuery.rangeStatusHeader iTopLevelRange rsQueryTotal rsTableTotal
cLHeader = if headersOnly then mempty else [ contentLengthHeader bod ] cLHeader = if headersOnly then mempty else [ contentLengthHeader bod ]
@@ -79,7 +79,7 @@ actionResponse (DbCrudResult plan@WrappedReadPlan{pMedia, wrHdrsOnly=headersOnly
++ cLHeader ++ cLHeader
++ contentTypeHeaders pMedia ctxApiRequest ++ contentTypeHeaders pMedia ctxApiRequest
++ prefHeader ++ prefHeader
bod | status == HTTP.status416 = Error.errorPayload configClientErrorVerbosity $ Error.ApiRequestErr $ Error.InvalidRange $ bod | status == HTTP.status416 = Error.errorPayload $ Error.ApiRequestError $ Error.InvalidRange $
Error.OutOfBounds (show $ RangeQuery.rangeOffset iTopLevelRange) (maybe "0" show rsTableTotal) Error.OutOfBounds (show $ RangeQuery.rangeOffset iTopLevelRange) (maybe "0" show rsTableTotal)
| headersOnly = mempty | headersOnly = mempty
| otherwise = LBS.fromStrict rsBody | otherwise = LBS.fromStrict rsBody
@@ -88,7 +88,7 @@ actionResponse (DbCrudResult plan@WrappedReadPlan{pMedia, wrHdrsOnly=headersOnly
Right $ PgrstResponse ovStatus ovHeaders bod Right $ PgrstResponse ovStatus ovHeaders bod
actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationCreate, pMedia, crudQi=QualifiedIdentifier{..}} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ = do actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationCreate, pMedia, crudQi=QualifiedIdentifier{..}} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ _ _ = do
let let
prefHeader = prefAppliedHeader $ responsePreferences plan ctxApiRequest prefHeader = prefAppliedHeader $ responsePreferences plan ctxApiRequest
@@ -123,7 +123,7 @@ actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationCreate, pMed
Right $ PgrstResponse ovStatus ovHeaders bod Right $ PgrstResponse ovStatus ovHeaders bod
actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationUpdate, pMedia} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ = do actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationUpdate, pMedia} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ _ _ = do
let let
contentRangeHeader = contentRangeHeader =
Just . RangeQuery.contentRangeH 0 (rsQueryTotal - 1) $ Just . RangeQuery.contentRangeH 0 (rsQueryTotal - 1) $
@@ -144,7 +144,7 @@ actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationUpdate, pMed
Right $ PgrstResponse ovStatus ovHeaders body Right $ PgrstResponse ovStatus ovHeaders body
actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationSingleUpsert, pMedia} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ = do actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationSingleUpsert, pMedia} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ _ _ = do
let let
prefHeader = maybeToList . prefAppliedHeader $ responsePreferences plan ctxApiRequest prefHeader = maybeToList . prefAppliedHeader $ responsePreferences plan ctxApiRequest
lbsBody = LBS.fromStrict rsBody lbsBody = LBS.fromStrict rsBody
@@ -162,7 +162,7 @@ actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationSingleUpsert
Right $ PgrstResponse ovStatus ovHeaders body Right $ PgrstResponse ovStatus ovHeaders body
actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationDelete, pMedia} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ = do actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationDelete, pMedia} RSStandard{..}) ctxApiRequest@ApiRequest{..} _ _ _ _ _ = do
let let
contentRangeHeader = RangeQuery.contentRangeH 1 0 $ if shouldCount (preferCount iPreferences) then Just rsQueryTotal else Nothing contentRangeHeader = RangeQuery.contentRangeH 1 0 $ if shouldCount (preferCount iPreferences) then Just rsQueryTotal else Nothing
prefHeader = maybeToList . prefAppliedHeader $ responsePreferences plan ctxApiRequest prefHeader = maybeToList . prefAppliedHeader $ responsePreferences plan ctxApiRequest
@@ -178,12 +178,12 @@ actionResponse (DbCrudResult plan@MutateReadPlan{mrMutation=MutationDelete, pMed
Right $ PgrstResponse ovStatus ovHeaders body Right $ PgrstResponse ovStatus ovHeaders body
actionResponse (DbCrudResult plan@CallReadPlan{pMedia, crInvMthd=invMethod, crProc=proc} RSStandard {..}) ctxApiRequest@ApiRequest{..} _ AppConfig{..} _ = do actionResponse (DbCrudResult plan@CallReadPlan{pMedia, crInvMthd=invMethod, crProc=proc} RSStandard {..}) ctxApiRequest@ApiRequest{..} _ _ _ _ _ = do
let let
(status, contentRange) = (status, contentRange) =
RangeQuery.rangeStatusHeader iTopLevelRange rsQueryTotal rsTableTotal RangeQuery.rangeStatusHeader iTopLevelRange rsQueryTotal rsTableTotal
rsOrErrBody = if status == HTTP.status416 rsOrErrBody = if status == HTTP.status416
then Error.errorPayload configClientErrorVerbosity $ Error.ApiRequestErr $ Error.InvalidRange then Error.errorPayload $ Error.ApiRequestError $ Error.InvalidRange
$ Error.OutOfBounds (show $ RangeQuery.rangeOffset iTopLevelRange) (maybe "0" show rsTableTotal) $ Error.OutOfBounds (show $ RangeQuery.rangeOffset iTopLevelRange) (maybe "0" show rsTableTotal)
else LBS.fromStrict rsBody else LBS.fromStrict rsBody
isHeadMethod = invMethod == InvRead True isHeadMethod = invMethod == InvRead True
@@ -202,18 +202,18 @@ actionResponse (DbCrudResult plan@CallReadPlan{pMedia, crInvMthd=invMethod, crPr
Right $ PgrstResponse ovStatus ovHeaders body Right $ PgrstResponse ovStatus ovHeaders body
actionResponse (DbPlanResult media plan) ctxApiRequest _ _ _ = actionResponse (DbPlanResult media plan) ctxApiRequest _ _ _ _ _ =
let body = LBS.fromStrict plan in let body = LBS.fromStrict plan in
Right $ PgrstResponse HTTP.status200 (contentLengthHeader body : contentTypeHeaders media ctxApiRequest) body Right $ PgrstResponse HTTP.status200 (contentLengthHeader body : contentTypeHeaders media ctxApiRequest) body
actionResponse (MaybeDbResult InspectPlan{ipHdrsOnly=headersOnly} body) ApiRequest{..} versions conf sCache = actionResponse (MaybeDbResult InspectPlan{ipHdrsOnly=headersOnly} body) _ versions conf sCache schema negotiatedByProfile =
let let
rsBody = maybe mempty (\(x, y, z) -> if headersOnly then mempty else OpenAPI.encode versions conf sCache x y z) body rsBody = maybe mempty (\(x, y, z) -> if headersOnly then mempty else OpenAPI.encode versions conf sCache x y z) body
cLHeader = if headersOnly then mempty else [contentLengthHeader rsBody] cLHeader = if headersOnly then mempty else [contentLengthHeader rsBody]
in in
Right $ PgrstResponse HTTP.status200 (MediaType.toContentType MTOpenAPI : cLHeader ++ maybeToList (profileHeader iSchema iNegotiatedByProfile)) rsBody Right $ PgrstResponse HTTP.status200 (MediaType.toContentType MTOpenAPI : cLHeader ++ maybeToList (profileHeader schema negotiatedByProfile)) rsBody
actionResponse (NoDbResult (RelInfoPlan qi@QualifiedIdentifier{..})) _ _ _ sc@SchemaCache{dbTables} = actionResponse (NoDbResult (RelInfoPlan qi@QualifiedIdentifier{..})) _ _ _ sc@SchemaCache{dbTables} _ _ =
case HM.lookup qi dbTables of case HM.lookup qi dbTables of
Just tbl -> respondInfo $ allowH tbl Just tbl -> respondInfo $ allowH tbl
Nothing -> Left $ Error.SchemaCacheErr $ Error.TableNotFound qiSchema qiName sc Nothing -> Left $ Error.SchemaCacheErr $ Error.TableNotFound qiSchema qiName sc
@@ -227,11 +227,11 @@ actionResponse (NoDbResult (RelInfoPlan qi@QualifiedIdentifier{..})) _ _ _ sc@Sc
["PATCH" | tableUpdatable table] ++ ["PATCH" | tableUpdatable table] ++
["DELETE" | tableDeletable table] ["DELETE" | tableDeletable table]
actionResponse (NoDbResult (RoutineInfoPlan proc)) _ _ _ _ actionResponse (NoDbResult (RoutineInfoPlan proc)) _ _ _ _ _ _
| pdVolatility proc == Volatile = respondInfo "OPTIONS,POST" | pdVolatility proc == Volatile = respondInfo "OPTIONS,POST"
| otherwise = respondInfo "OPTIONS,GET,HEAD,POST" | otherwise = respondInfo "OPTIONS,GET,HEAD,POST"
actionResponse (NoDbResult SchemaInfoPlan) _ _ _ _ = respondInfo "OPTIONS,GET,HEAD" actionResponse (NoDbResult SchemaInfoPlan) _ _ _ _ _ _ = respondInfo "OPTIONS,GET,HEAD"
respondInfo :: ByteString -> Either Error.Error PgrstResponse respondInfo :: ByteString -> Either Error.Error PgrstResponse
respondInfo allowHeader = respondInfo allowHeader =
@@ -247,11 +247,11 @@ overrideStatusHeaders rsGucStatus rsGucHeaders pgrstStatus pgrstHeaders = do
decodeGucHeaders :: Maybe BS.ByteString -> Either Error.Error [GucHeader] decodeGucHeaders :: Maybe BS.ByteString -> Either Error.Error [GucHeader]
decodeGucHeaders = decodeGucHeaders =
maybe (Right []) $ first (const . Error.ApiRequestErr $ Error.GucHeadersError) . JSON.eitherDecode . LBS.fromStrict maybe (Right []) $ first (const . Error.ApiRequestError $ Error.GucHeadersError) . JSON.eitherDecode . LBS.fromStrict
decodeGucStatus :: Maybe Text -> Either Error.Error (Maybe HTTP.Status) decodeGucStatus :: Maybe Text -> Either Error.Error (Maybe HTTP.Status)
decodeGucStatus = decodeGucStatus =
maybe (Right Nothing) $ first (const . Error.ApiRequestErr $ Error.GucStatusError) . fmap (Just . toEnum . fst) . decimal maybe (Right Nothing) $ first (const . Error.ApiRequestError $ Error.GucStatusError) . fmap (Just . toEnum . fst) . decimal
contentLengthHeader :: LBS.ByteString -> HTTP.Header contentLengthHeader :: LBS.ByteString -> HTTP.Header
contentLengthHeader body = ("Content-Length", show (LBS.length body)) contentLengthHeader body = ("Content-Length", show (LBS.length body))
+33 -124
View File
@@ -24,14 +24,10 @@ module PostgREST.SchemaCache
, querySchemaCache , querySchemaCache
, showSummary , showSummary
, decodeFuncs , decodeFuncs
, QueryTimings(..)
, queryTimingsWLabels
) where ) where
import Data.Aeson ((.=)) import Data.Aeson ((.=))
import qualified Data.Aeson as JSON import qualified Data.Aeson as JSON
import qualified Data.ByteString.Char8 as BS
import qualified Data.HashMap.Strict as HM import qualified Data.HashMap.Strict as HM
import qualified Data.HashMap.Strict.InsOrd as HMI import qualified Data.HashMap.Strict.InsOrd as HMI
import qualified Data.Set as S import qualified Data.Set as S
@@ -44,8 +40,7 @@ import qualified Hasql.Transaction as SQL
import Data.Functor.Contravariant ((>$<)) import Data.Functor.Contravariant ((>$<))
import NeatInterpolation (trimming) import NeatInterpolation (trimming)
import PostgREST.Config (AppConfig (..), import PostgREST.Config (AppConfig (..))
LogLevel (..))
import PostgREST.Config.Database (TimezoneNames, import PostgREST.Config.Database (TimezoneNames,
toIsolationLevel) toIsolationLevel)
import PostgREST.SchemaCache.Identifiers (FieldName, import PostgREST.SchemaCache.Identifiers (FieldName,
@@ -90,11 +85,10 @@ data SchemaCache = SchemaCache
-- Since index construction can be expensive, we build it once and store in the SchemaCache -- Since index construction can be expensive, we build it once and store in the SchemaCache
-- Haskell lazy evaluation ensures it's only built on first use and memoized afterwards -- Haskell lazy evaluation ensures it's only built on first use and memoized afterwards
, dbTablesFuzzyIndex :: TablesFuzzyIndex , dbTablesFuzzyIndex :: TablesFuzzyIndex
, dbQueryTimings :: Maybe QueryTimings -- ^ cached time for the time each query took when debugging
} deriving (Show) } deriving (Show)
instance JSON.ToJSON SchemaCache where instance JSON.ToJSON SchemaCache where
toJSON (SchemaCache tabs rels routs reps hdlers tzs _ _) = JSON.object [ toJSON (SchemaCache tabs rels routs reps hdlers tzs _) = JSON.object [
"dbTables" .= JSON.toJSON tabs "dbTables" .= JSON.toJSON tabs
, "dbRelationships" .= JSON.toJSON rels , "dbRelationships" .= JSON.toJSON rels
, "dbRoutines" .= JSON.toJSON routs , "dbRoutines" .= JSON.toJSON routs
@@ -104,7 +98,7 @@ instance JSON.ToJSON SchemaCache where
] ]
showSummary :: SchemaCache -> Text showSummary :: SchemaCache -> Text
showSummary (SchemaCache tbls rels routs reps mediaHdlrs tzs _ _) = showSummary (SchemaCache tbls rels routs reps mediaHdlrs tzs _) =
T.intercalate ", " T.intercalate ", "
[ show (HM.size tbls) <> " Relations" [ show (HM.size tbls) <> " Relations"
, show (HM.size rels) <> " Relationships" , show (HM.size rels) <> " Relationships"
@@ -158,25 +152,18 @@ maxDbTablesForFuzzySearch = 500
querySchemaCache :: AppConfig -> SQL.Transaction SchemaCache querySchemaCache :: AppConfig -> SQL.Transaction SchemaCache
querySchemaCache conf@AppConfig{..} = do querySchemaCache conf@AppConfig{..} = do
SQL.sql "set local schema ''" -- This voids the search path. The following queries need this for getting the fully qualified name(schema.name) of every db object SQL.sql "set local schema ''" -- This voids the search path. The following queries need this for getting the fully qualified name(schema.name) of every db object
tabs <- sqlTimedStmt gucTbls conf allTables tabs <- SQL.statement conf $ allTables prepared
keyDeps <- sqlTimedStmt gucKDeps conf allViewsKeyDependencies keyDeps <- SQL.statement conf $ allViewsKeyDependencies prepared
m2oRels <- sqlTimedStmt gucRels mempty allM2OandO2ORels m2oRels <- SQL.statement mempty $ allM2OandO2ORels prepared
funcs <- sqlTimedStmt gucFuncs conf allFunctions funcs <- SQL.statement conf $ allFunctions prepared
cRels <- sqlTimedStmt gucCRels mempty allComputedRels cRels <- SQL.statement mempty $ allComputedRels prepared
reps <- sqlTimedStmt gucDReps conf dataRepresentations reps <- SQL.statement conf $ dataRepresentations prepared
mHdlers <- sqlTimedStmt gucMHdrs conf mediaHandlers mHdlers <- SQL.statement conf $ mediaHandlers prepared
tzones <- if configDbTimezoneEnabled tzones <- SQL.statement mempty $ timezones prepared
then sqlTimedStmt gucTzones mempty timezones
else pure S.empty
_ <- _ <-
let sleepCall = SQL.Statement "select pg_sleep($1 / 1000.0)" (param HE.int4) HD.noResult True in let sleepCall = SQL.Statement "select pg_sleep($1 / 1000.0)" (param HE.int4) HD.noResult prepared in
for_ configInternalSCQuerySleep (`SQL.statement` sleepCall) -- only used for testing for_ configInternalSCQuerySleep (`SQL.statement` sleepCall) -- only used for testing
qsTime <-
if isLogDebug
then Just <$> SQL.statement mempty (extractTimings configDbTimezoneEnabled)
else pure Nothing
let tabsWViewsPks = addViewPrimaryKeys tabs keyDeps let tabsWViewsPks = addViewPrimaryKeys tabs keyDeps
rels = addInverseRels $ addM2MRels tabsWViewsPks $ addViewM2OAndO2ORels keyDeps m2oRels rels = addInverseRels $ addM2MRels tabsWViewsPks $ addViewM2OAndO2ORels keyDeps m2oRels
@@ -194,13 +181,11 @@ querySchemaCache conf@AppConfig{..} = do
-- Only build fuzzy index for schemas with a reasonable number of tables -- Only build fuzzy index for schemas with a reasonable number of tables
-- Fuzzy.FuzzySet is memory heavy we just don't use it for large schemas -- Fuzzy.FuzzySet is memory heavy we just don't use it for large schemas
Fuzzy.fromList <$> HM.filter ((< maxDbTablesForFuzzySearch) . length) (HM.fromListWith (<>) ((qiSchema &&& pure . qiName) <$> HM.keys tabsWViewsPks)) Fuzzy.fromList <$> HM.filter ((< maxDbTablesForFuzzySearch) . length) (HM.fromListWith (<>) ((qiSchema &&& pure . qiName) <$> HM.keys tabsWViewsPks))
, dbQueryTimings = qsTime
} }
where where
schemas = toList configDbSchemas schemas = toList configDbSchemas
prepared = configDbPreparedStatements
delayEval confDelay result = maybe result (unsafePerformIO . (($> result) . (threadDelay . (1000 *) . fromIntegral))) confDelay delayEval confDelay result = maybe result (unsafePerformIO . (($> result) . (threadDelay . (1000 *) . fromIntegral))) confDelay
isLogDebug = configLogLevel == LogDebug
sqlTimedStmt = sqlTimedStatement isLogDebug
-- | overrides detected relationships with the computed relationships and gets the RelationshipsMap -- | overrides detected relationships with the computed relationships and gets the RelationshipsMap
getOverrideRelationshipsMap :: [Relationship] -> [Relationship] -> RelationshipsMap getOverrideRelationshipsMap :: [Relationship] -> [Relationship] -> RelationshipsMap
@@ -234,7 +219,6 @@ removeInternal schemas dbStruct =
, dbMediaHandlers = dbMediaHandlers dbStruct , dbMediaHandlers = dbMediaHandlers dbStruct
, dbTimezones = dbTimezones dbStruct , dbTimezones = dbTimezones dbStruct
, dbTablesFuzzyIndex = dbTablesFuzzyIndex dbStruct , dbTablesFuzzyIndex = dbTablesFuzzyIndex dbStruct
, dbQueryTimings = dbQueryTimings dbStruct
} }
where where
hasInternalJunction ComputedRelationship{} = False hasInternalJunction ComputedRelationship{} = False
@@ -363,8 +347,8 @@ decodeRepresentations =
-- 2. implicit -- 2. implicit
-- For the time being it must also be to/from JSON or text, although one can imagine a future where we support special -- For the time being it must also be to/from JSON or text, although one can imagine a future where we support special
-- cases like CSV specific representations. -- cases like CSV specific representations.
dataRepresentations :: SQL.Statement AppConfig RepresentationsMap dataRepresentations :: Bool -> SQL.Statement AppConfig RepresentationsMap
dataRepresentations = SQL.Statement sql mempty decodeRepresentations True dataRepresentations = SQL.Statement sql mempty decodeRepresentations
where where
sql = encodeUtf8 [trimming| sql = encodeUtf8 [trimming|
SELECT SELECT
@@ -385,8 +369,8 @@ dataRepresentations = SQL.Statement sql mempty decodeRepresentations True
OR (dst_t.typtype = 'd' AND c.castsource IN ('json'::regtype::oid , 'text'::regtype::oid))) OR (dst_t.typtype = 'd' AND c.castsource IN ('json'::regtype::oid , 'text'::regtype::oid)))
|] |]
allFunctions :: SQL.Statement AppConfig RoutineMap allFunctions :: Bool -> SQL.Statement AppConfig RoutineMap
allFunctions = SQL.Statement funcsSqlQuery params decodeFuncs True allFunctions = SQL.Statement funcsSqlQuery params decodeFuncs
where where
params = params =
(map escapeIdent . toList . configDbSchemas >$< arrayParam HE.text) <> (map escapeIdent . toList . configDbSchemas >$< arrayParam HE.text) <>
@@ -598,8 +582,8 @@ addViewPrimaryKeys tabs keyDeps =
takeFirstPK = mapMaybe (head . snd) takeFirstPK = mapMaybe (head . snd)
indexedDeps = HM.fromListWith (++) $ fmap ((keyDepType &&& keyDepView) &&& pure) keyDeps indexedDeps = HM.fromListWith (++) $ fmap ((keyDepType &&& keyDepView) &&& pure) keyDeps
allTables :: SQL.Statement AppConfig TablesMap allTables :: Bool -> SQL.Statement AppConfig TablesMap
allTables = SQL.Statement tablesSqlQuery params decodeTables True allTables = SQL.Statement tablesSqlQuery params decodeTables
where where
params = map escapeIdent . toList . configDbSchemas >$< arrayParam HE.text params = map escapeIdent . toList . configDbSchemas >$< arrayParam HE.text
@@ -746,9 +730,9 @@ tablesSqlQuery =
ORDER BY table_schema, table_name|] ORDER BY table_schema, table_name|]
-- | Gets many-to-one relationships and one-to-one(O2O) relationships, which are a refinement of the many-to-one's -- | Gets many-to-one relationships and one-to-one(O2O) relationships, which are a refinement of the many-to-one's
allM2OandO2ORels :: SQL.Statement () [Relationship] allM2OandO2ORels :: Bool -> SQL.Statement () [Relationship]
allM2OandO2ORels = allM2OandO2ORels =
SQL.Statement sql HE.noParams decodeRels True SQL.Statement sql HE.noParams decodeRels
where where
-- We use jsonb_agg for comparing the uniques/pks instead of array_agg to avoid the ERROR: cannot accumulate arrays of different dimensionality -- We use jsonb_agg for comparing the uniques/pks instead of array_agg to avoid the ERROR: cannot accumulate arrays of different dimensionality
sql = encodeUtf8 [trimming| sql = encodeUtf8 [trimming|
@@ -790,9 +774,9 @@ allM2OandO2ORels =
AND traint.conparentid = 0 AND traint.conparentid = 0
ORDER BY traint.conrelid, traint.conname|] ORDER BY traint.conrelid, traint.conname|]
allComputedRels :: SQL.Statement () [Relationship] allComputedRels :: Bool -> SQL.Statement () [Relationship]
allComputedRels = allComputedRels =
SQL.Statement sql HE.noParams (HD.rowList cRelRow) True SQL.Statement sql HE.noParams (HD.rowList cRelRow)
where where
sql = encodeUtf8 [trimming| sql = encodeUtf8 [trimming|
with with
@@ -836,9 +820,9 @@ allComputedRels =
column HD.bool column HD.bool
-- | Returns all the views' primary keys and foreign keys dependencies -- | Returns all the views' primary keys and foreign keys dependencies
allViewsKeyDependencies :: SQL.Statement AppConfig [ViewKeyDependency] allViewsKeyDependencies :: Bool -> SQL.Statement AppConfig [ViewKeyDependency]
allViewsKeyDependencies = allViewsKeyDependencies =
SQL.Statement sql params decodeViewKeyDeps True SQL.Statement sql params decodeViewKeyDeps
-- query explanation at: -- query explanation at:
-- * rationale: https://gist.github.com/wolfgangwalther/5425d64e7b0d20aad71f6f68474d9f19 -- * rationale: https://gist.github.com/wolfgangwalther/5425d64e7b0d20aad71f6f68474d9f19
-- * json transformation: https://gist.github.com/wolfgangwalther/3a8939da680c24ad767e93ad2c183089 -- * json transformation: https://gist.github.com/wolfgangwalther/3a8939da680c24ad767e93ad2c183089
@@ -947,7 +931,7 @@ allViewsKeyDependencies =
-- This leads to a smaller json result as well. -- This leads to a smaller json result as well.
-- Removal stops at `,` for used fields (see above) and `}` for the end of the current node. -- Removal stops at `,` for used fields (see above) and `}` for the end of the current node.
-- Nesting can't be parsed correctly with a regex, so we stop at `{` as well and -- Nesting can't be parsed correctly with a regex, so we stop at `{` as well and
-- add an empty key for the following node. -- add an empty key for the followig node.
), ' :[^}{,]+' , ',"":' , 'g' ), ' :[^}{,]+' , ',"":' , 'g'
-- For performance, the regex also added those empty keys when hitting a `,` or `}`. -- For performance, the regex also added those empty keys when hitting a `,` or `}`.
-- Those are removed next. -- Those are removed next.
@@ -1046,9 +1030,9 @@ initialMediaHandlers =
HM.insert (RelAnyElement, MediaType.MTGeoJSON ) (BuiltinOvAggGeoJson, MediaType.MTGeoJSON) HM.insert (RelAnyElement, MediaType.MTGeoJSON ) (BuiltinOvAggGeoJson, MediaType.MTGeoJSON)
HM.empty HM.empty
mediaHandlers :: SQL.Statement AppConfig MediaHandlerMap mediaHandlers :: Bool -> SQL.Statement AppConfig MediaHandlerMap
mediaHandlers = mediaHandlers =
SQL.Statement sql params decodeMediaHandlers True SQL.Statement sql params decodeMediaHandlers
where where
params = map escapeIdent . toList . configDbSchemas >$< arrayParam HE.text params = map escapeIdent . toList . configDbSchemas >$< arrayParam HE.text
sql = encodeUtf8 [trimming| sql = encodeUtf8 [trimming|
@@ -1122,16 +1106,10 @@ decodeMediaHandlers =
<*> (MediaType.decodeMediaType . encodeUtf8 <$> column HD.text) <*> (MediaType.decodeMediaType . encodeUtf8 <$> column HD.text)
<*> (MediaType.decodeMediaType . encodeUtf8 <$> column HD.text) <*> (MediaType.decodeMediaType . encodeUtf8 <$> column HD.text)
timezones :: SQL.Statement () TimezoneNames timezones :: Bool -> SQL.Statement () TimezoneNames
timezones = SQL.Statement sql HE.noParams decodeTimezones True timezones = SQL.Statement sql HE.noParams decodeTimezones
where where
sql = encodeUtf8 $ unlines sql = "SELECT name FROM pg_timezone_names"
-- This CTE wrapper is only added for clarifying the query under pg_stat_statements
["WITH pgrst_timezones AS ("
, " SELECT name FROM pg_timezone_names"
, ")"
, "SELECT * FROM pgrst_timezones"
]
decodeTimezones :: HD.Result TimezoneNames decodeTimezones :: HD.Result TimezoneNames
decodeTimezones = S.fromList <$> HD.rowList (column HD.text) decodeTimezones = S.fromList <$> HD.rowList (column HD.text)
@@ -1161,72 +1139,3 @@ nullableColumn = HD.column . HD.nullable
arrayColumn :: HD.Value a -> HD.Row [a] arrayColumn :: HD.Value a -> HD.Row [a]
arrayColumn = column . HD.listArray . HD.nonNullable arrayColumn = column . HD.listArray . HD.nonNullable
{-
- Times a sql statement inside a transaction, for this:
-
- 1. We start a timer: select set_config('pgrst.tmp_x', clock_timestamp()::text, false);
- 2. Run the statement: select ....
- 3. End the timer: select set_config('pgrst.tmp_x', (clock_timestamp() - current_setting('pgrst.tmp_x', false)::timestamptz)::text, false);
-
- We can do this for several statements inside the transaction. The timings are later captured at the end of the transaction with extractTimings.
-}
sqlTimedStatement :: Bool -> ByteString -> a -> SQL.Statement a b -> SQL.Transaction b
sqlTimedStatement isLogDebug guc params stmt =
if isLogDebug then
SQL.sql sFrag >> SQL.statement params stmt <* SQL.sql eFrag
else
SQL.statement params stmt
where
sFrag = "select set_config('pgrst." <> guc <> "', clock_timestamp()::text, true)"
eFrag = "select set_config('pgrst." <> guc <> "', (clock_timestamp() - current_setting('pgrst." <> guc <> "', false)::timestamptz)::text, true)"
-- Extract all the generated timings (see sqlTimedStatement) converting the value to milliseconds.
extractTimings :: Bool -> SQL.Statement () QueryTimings
extractTimings hasTimezones = SQL.Statement sql HE.noParams decodeThem True
where
qFrag setting = "extract('milliseconds' from current_setting('pgrst." <> setting <> "', false)::interval)::text"
sql = "SELECT " <> BS.intercalate ","
[ qFrag gucTbls, qFrag gucKDeps, qFrag gucRels
, qFrag gucFuncs, qFrag gucCRels, qFrag gucDReps
, qFrag gucMHdrs, if hasTimezones then qFrag gucTzones else "'0.0'"
]
decodeThem :: HD.Result QueryTimings
decodeThem = HD.singleRow $
QueryTimings
<$> column HD.text <*> column HD.text <*> column HD.text
<*> column HD.text <*> column HD.text <*> column HD.text
<*> column HD.text <*> column HD.text
data QueryTimings = QueryTimings
{ qtTables :: Text
, qtKeyDeps :: Text
, qtRels :: Text
, qtFuncs :: Text
, qtCRels :: Text
, qtDReps :: Text
, qtMHdrs :: Text
, qtTzones :: Text
} deriving (Show)
queryTimingsWLabels :: QueryTimings -> [(ByteString, Text)]
queryTimingsWLabels qt =
[ (gucTbls, qtTables qt)
, (gucKDeps, qtKeyDeps qt)
, (gucRels, qtRels qt)
, (gucFuncs, qtFuncs qt)
, (gucCRels, qtCRels qt)
, (gucDReps, qtDReps qt)
, (gucMHdrs, qtMHdrs qt)
, (gucTzones, qtTzones qt)
]
gucTbls, gucKDeps, gucRels, gucFuncs, gucCRels, gucDReps, gucMHdrs, gucTzones :: ByteString
gucTbls = "tables"
gucKDeps = "keydeps"
gucRels = "rels"
gucFuncs = "funcs"
gucCRels = "comprels"
gucDReps = "dreps"
gucMHdrs = "mhandlers"
gucTzones = "tzones"
+5 -6
View File
@@ -7,6 +7,7 @@ module PostgREST.SchemaCache.Identifiers
, RelIdentifier(..) , RelIdentifier(..)
, Schema , Schema
, TableName , TableName
, dumpQi
, escapeIdent , escapeIdent
, isAnyElement , isAnyElement
, quoteQi , quoteQi
@@ -37,12 +38,10 @@ instance Hashable QualifiedIdentifier
isAnyElement :: QualifiedIdentifier -> Bool isAnyElement :: QualifiedIdentifier -> Bool
isAnyElement y = QualifiedIdentifier "pg_catalog" "anyelement" == y isAnyElement y = QualifiedIdentifier "pg_catalog" "anyelement" == y
-- | dumpQi :: QualifiedIdentifier -> Text
-- Quote the qualified identifier when preparing the SQL. This avoids parse dumpQi (QualifiedIdentifier s i) =
-- errors by postgres, for example on pg reserved words like "true" or "select". (if T.null s then mempty else s <> ".") <> i
--
-- >>> quoteQi (QualifiedIdentifier "" "true")
-- "\"true\""
quoteQi :: QualifiedIdentifier -> Text quoteQi :: QualifiedIdentifier -> Text
quoteQi (QualifiedIdentifier s i) = quoteQi (QualifiedIdentifier s i) =
(if T.null s then mempty else escapeIdent s <> ".") <> escapeIdent i (if T.null s then mempty else escapeIdent s <> ".") <> escapeIdent i
+3 -2
View File
@@ -19,9 +19,10 @@ import System.Directory (removeFile)
import System.IO.Error (isDoesNotExistError) import System.IO.Error (isDoesNotExistError)
-- | Set signal handlers, only for systems with signals -- | Set signal handlers, only for systems with signals
installSignalHandlers :: Observation.ObservationHandler -> IO () -> IO () -> IO () -> IO () installSignalHandlers :: Observation.ObservationHandler -> ThreadId -> IO () -> IO () -> IO ()
#ifndef mingw32_HOST_OS #ifndef mingw32_HOST_OS
installSignalHandlers observer interrupt usr1 usr2 = do installSignalHandlers observer tid usr1 usr2 = do
let interrupt = throwTo tid UserInterrupt
install Signals.sigINT $ observer (Observation.TerminationUnixSignalObs "SIGINT") >> interrupt install Signals.sigINT $ observer (Observation.TerminationUnixSignalObs "SIGINT") >> interrupt
install Signals.sigTERM $ observer (Observation.TerminationUnixSignalObs "SIGTERM") >> interrupt install Signals.sigTERM $ observer (Observation.TerminationUnixSignalObs "SIGTERM") >> interrupt
install Signals.sigUSR1 usr1 install Signals.sigUSR1 usr1
-6
View File
@@ -8,12 +8,6 @@ import qualified Data.Text as T
import Protolude import Protolude
-- Somehow this is not defined in doctests, so when running them
-- on a file that includes Version.hs, compilation fails.
#ifndef VERSION_postgrest
#define VERSION_postgrest "0"
#endif
version :: [Text] version :: [Text]
version = T.splitOn "." VERSION_postgrest version = T.splitOn "." VERSION_postgrest
-19
View File
@@ -1,19 +0,0 @@
Copyright (c) 2016-2020, Stephen Diehl
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.
File diff suppressed because it is too large Load Diff
-38
View File
@@ -1,38 +0,0 @@
{-# LANGUAGE Safe #-}
{-# LANGUAGE NoImplicitPrelude #-}
module Protolude.Applicative
( orAlt,
orEmpty,
eitherA,
purer,
liftAA2,
(<<*>>),
)
where
import Control.Applicative
import Data.Bool (Bool)
import Data.Either (Either (Left, Right))
import Data.Function ((.))
import Data.Monoid (Monoid (mempty))
orAlt :: (Alternative f, Monoid a) => f a -> f a
orAlt f = f <|> pure mempty
orEmpty :: Alternative f => Bool -> a -> f a
orEmpty b a = if b then pure a else empty
eitherA :: (Alternative f) => f a -> f b -> f (Either a b)
eitherA a b = (Left <$> a) <|> (Right <$> b)
purer :: (Applicative f, Applicative g) => a -> f (g a)
purer = pure . pure
liftAA2 :: (Applicative f, Applicative g) => (a -> b -> c) -> f (g a) -> f (g b) -> f (g c)
liftAA2 = liftA2 . liftA2
infixl 4 <<*>>
(<<*>>) :: (Applicative f, Applicative g) => f (g (a -> b)) -> f (g a) -> f (g b)
(<<*>>) = liftA2 (<*>)
-225
View File
@@ -1,225 +0,0 @@
{-# LANGUAGE CPP #-}
{-# LANGUAGE MagicHash #-}
{-# LANGUAGE Unsafe #-}
{-# LANGUAGE BangPatterns #-}
{-# LANGUAGE NoImplicitPrelude #-}
{-# LANGUAGE ExplicitNamespaces #-}
module Protolude.Base (
module Base,
($!),
) where
-- Glorious Glasgow Haskell Compiler
#if defined(__GLASGOW_HASKELL__) && ( __GLASGOW_HASKELL__ >= 600 )
-- Base GHC types
import GHC.Num as Base (
Num(
(+),
(-),
(*),
negate,
abs,
signum,
fromInteger
)
, Integer
, subtract
)
import GHC.Enum as Base (
Bounded(minBound, maxBound)
, Enum(
succ,
pred,
toEnum,
fromEnum,
enumFrom,
enumFromThen,
enumFromTo,
enumFromThenTo
)
, boundedEnumFrom
, boundedEnumFromThen
)
import GHC.Real as Base (
(%)
, (/)
, Fractional
, Integral
, Ratio
, Rational
, Real
, RealFrac
, (^)
, (^%^)
, (^^)
, (^^%^^)
, ceiling
, denominator
, div
, divMod
#if MIN_VERSION_base(4,7,0)
, divZeroError
#endif
, even
, floor
, fromIntegral
, fromRational
, gcd
#if MIN_VERSION_base(4,9,0) && !MIN_VERSION_base(4,15,0)
#if defined(MIN_VERSION_integer_gmp)
, gcdInt'
, gcdWord'
#endif
#endif
, infinity
, integralEnumFrom
, integralEnumFromThen
, integralEnumFromThenTo
, integralEnumFromTo
, lcm
, mod
, notANumber
, numerator
, numericEnumFrom
, numericEnumFromThen
, numericEnumFromThenTo
, numericEnumFromTo
, odd
#if MIN_VERSION_base(4,7,0)
, overflowError
#endif
, properFraction
, quot
, quotRem
, ratioPrec
, ratioPrec1
#if MIN_VERSION_base(4,7,0)
, ratioZeroDenominatorError
#endif
, realToFrac
, recip
, reduce
, rem
, round
, showSigned
, toInteger
, toRational
, truncate
#if MIN_VERSION_base(4,12,0)
, underflowError
#endif
)
import GHC.Float as Base (
Float(F#)
, Double(D#)
, Floating (..)
, RealFloat(..)
, showFloat
, showSignedFloat
)
import GHC.Show as Base (
Show(showsPrec, show, showList)
)
import GHC.Exts as Base (
Constraint
, Ptr
, FunPtr
)
import GHC.Base as Base (
(++)
, seq
, asTypeOf
, ord
, maxInt
, minInt
, until
)
-- Exported for lifting into new functions.
import System.IO as Base (
print
, putStr
, putStrLn
)
import GHC.Types as Base (
Bool
, Char
, Int
, Word
, Ordering
, IO
#if ( __GLASGOW_HASKELL__ >= 710 )
, Coercible
#endif
)
#if ( __GLASGOW_HASKELL__ >= 710 )
import GHC.StaticPtr as Base (StaticPtr)
#endif
#if ( __GLASGOW_HASKELL__ >= 800 )
import GHC.OverloadedLabels as Base (
IsLabel(fromLabel)
)
import GHC.ExecutionStack as Base (
Location(Location, srcLoc, objectName, functionName)
, SrcLoc(SrcLoc, sourceColumn, sourceLine, sourceColumn)
, getStackTrace
, showStackTrace
)
import GHC.Stack as Base (
CallStack
, type HasCallStack
, callStack
, prettySrcLoc
, currentCallStack
, getCallStack
, prettyCallStack
, withFrozenCallStack
)
#endif
#if ( __GLASGOW_HASKELL__ >= 710 )
import GHC.TypeLits as Base (
Symbol
, SomeSymbol(SomeSymbol)
, Nat
, SomeNat(SomeNat)
, CmpNat
, KnownSymbol
, KnownNat
, natVal
, someNatVal
, symbolVal
, someSymbolVal
)
#endif
#if ( __GLASGOW_HASKELL__ >= 802 )
import GHC.Records as Base (
HasField(getField)
)
#endif
#if ( __GLASGOW_HASKELL__ >= 800 )
import Data.Kind as Base (
type Type
#if ( __GLASGOW_HASKELL__ < 805 )
, type (*)
#endif
, type Type
)
#endif
-- Default Prelude defines this at the toplevel module, so we do as well.
infixr 0 $!
($!) :: (a -> b) -> a -> b
f $! x = let !vx = x in f vx
#endif
-51
View File
@@ -1,51 +0,0 @@
{-# LANGUAGE Safe #-}
{-# LANGUAGE NoImplicitPrelude #-}
module Protolude.Bifunctor
( Bifunctor,
bimap,
first,
second,
)
where
import Control.Applicative (Const (Const))
import Data.Either (Either (Left, Right))
import Data.Function ((.), id)
class Bifunctor p where
{-# MINIMAL bimap | first, second #-}
bimap :: (a -> b) -> (c -> d) -> p a c -> p b d
bimap f g = first f . second g
first :: (a -> b) -> p a c -> p b c
first f = bimap f id
second :: (b -> c) -> p a b -> p a c
second = bimap id
instance Bifunctor (,) where
bimap f g ~(a, b) = (f a, g b)
instance Bifunctor ((,,) x1) where
bimap f g ~(x1, a, b) = (x1, f a, g b)
instance Bifunctor ((,,,) x1 x2) where
bimap f g ~(x1, x2, a, b) = (x1, x2, f a, g b)
instance Bifunctor ((,,,,) x1 x2 x3) where
bimap f g ~(x1, x2, x3, a, b) = (x1, x2, x3, f a, g b)
instance Bifunctor ((,,,,,) x1 x2 x3 x4) where
bimap f g ~(x1, x2, x3, x4, a, b) = (x1, x2, x3, x4, f a, g b)
instance Bifunctor ((,,,,,,) x1 x2 x3 x4 x5) where
bimap f g ~(x1, x2, x3, x4, x5, a, b) = (x1, x2, x3, x4, x5, f a, g b)
instance Bifunctor Either where
bimap f _ (Left a) = Left (f a)
bimap _ g (Right b) = Right (g b)
instance Bifunctor Const where
bimap f _ (Const a) = Const (f a)

Some files were not shown because too many files have changed in this diff Show More