Compare commits

..
879 Commits
Author SHA1 Message Date
Laurence Isla 519615d5fb bump version to 12.2.3 2024-08-01 11:46:52 -05:00
Laurence Isla de5a8954b2 changelog: add missing entry for 3670 2024-08-01 11:18:15 -05:00
64b5a59410 fix: schema cache loading before the in-db config (#3670)
Fixes #3660. Load the config after getting the pg version but before loading the schema.

The regression happened on f09655b.

Also remove schema cache load wrapper and separate db queries in different functions.

Co-authored-by: Laurence Isla <lau.isla.c@gmail.com>
2024-08-01 17:41:07 +02:00
Dan KurinandWolfgang Walther 417a96d137 fix: Remove OpenAPI format for rowFilter params (#3661) 2024-08-01 17:41:07 +02:00
Wolfgang Walther 53593824a8 fix: Embed One-to-One relationship with different column order properly 2024-07-13 22:27:04 +02:00
Wolfgang Walther e642086d27 chore: Fix changelog after 13868983 2024-07-13 17:16:06 +02:00
steve-chavezandWolfgang Walther e451700d5d chore: remove links to gitter
We'll now use github discussions for support.
2024-07-13 17:11:06 +02:00
Salim BandWolfgang Walther 13868983af fix: update OpenAPI externalDocs URL
fixes https://github.com/PostgREST/postgrest/issues/3091
2024-07-13 17:11:06 +02:00
steve-chavez db9da0b1ca bump version to 12.2.2 2024-07-10 23:07:23 -05:00
steve-chavez f09655b7a6 fix: schema cache retrying without backoff
Fixes https://github.com/PostgREST/postgrest/issues/3523.

Now if there's a failure when obtaining the pg version OR schema cache,
we do the same retrying process. This way we don't add two retries.

Refactors and renames the "connectionWorker" to "schemaCacheLoader".
This makes more sense since what we really want is the schema cache,
the version is the pre-requisite for ensuring our
schema cache queries work.

Additionally, we no longer log ` Attempting to connect to the database...`
at startup unnecessarily. This is only logged whenever there's a retry attempt.
2024-07-10 21:11:20 -05:00
steve-chavezandWolfgang Walther 8715e426c0 chore: remove paypal links
It was tied to a personal account and donations there have been too rare.
2024-07-10 21:13:23 +02:00
Wolfgang WaltherandWolfgang Walther 9a40bc266e fix: List correct enum options when multiple types with same name are present
The schema cache and OpenAPI output would currently list the first found
enum with the same name instead of the correct type. One other case
where this comes up is when a regular type and an enum type have the
same name. For example in the spec fixtures, we have an enum called
"bit". Every "bit" type, no matter whether it's that enum or the
built-in bit type, will show those enum options in the OpenApi output.

Not adding a test, because OpenAPI is supposed to go away in the future
anyway.
2024-07-09 09:09:22 +02:00
Wolfgang WaltherandWolfgang Walther e67461b991 fix: Show number of loaded timezones in log output
There is no reason to hide those, right?
2024-07-09 09:09:22 +02:00
Wolfgang WaltherandWolfgang Walther 420933ceba fix: Make --dump-schema work with in-database pgrst.db_schemas setting
This needs to be loaded from in-database configuration first, otherwise
the dump-schema output will be for the default (public) schema.
2024-07-09 09:09:22 +02:00
Laurence IslaandWolfgang Walther 8f8dbd3425 fix: nested empty embeds no longer return empty values and are correctly omitted 2024-07-09 09:09:22 +02:00
renovate[bot]andWolfgang Walther 8d7d7e4112 chore(deps): update all dependencies 2024-07-06 11:35:11 +02:00
Sandro BauerandWolfgang Walther 521acf61cb docs: fix rendering for inline code block in operator list 2024-07-02 12:28:24 +02:00
steve-chavez d3f7cba92e bump version to 12.2.1 2024-06-27 14:59:35 -05:00
Laurence IslaandSteve Chavez a71d9ea50f changelog: add missing entries for #3592 and #3616 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 5a1e9b01e4 nix: remove texlive dependencies from postgrest-docs-render 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez d3292d42de docs: use PlantUML instead of Latex to generate Schema Isolation image 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 572581cbb6 chore: organize diagrams in different folders 2024-06-27 14:08:16 -05:00
steve-chavez 9daddb549c docs: add listener recovery 2024-06-27 14:08:16 -05:00
steve-chavez 716fcbf147 fix: don't reload cache on every listener fail
Revert "prevent GSSAPI error between Listener and pool"

This reverts commit 4beac10d3d.
2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 5b62501ce7 docs: improve architecture diagram
- SVG format instead of PNG
- The components now have links to their reference in the Docs
- Supports dark mode
2024-06-27 14:08:16 -05:00
Wolfgang WaltherandSteve Chavez 7860e7287f docs: Fix punctuation in install.rst 2024-06-27 14:08:16 -05:00
Michal KleczekandSteve Chavez c958f94753 docs: add pg-notify-stdout to ecosystem 2024-06-27 14:08:16 -05:00
renovate[bot]andWolfgang Walther 94f0548344 chore(deps): update ubuntu:noble docker digest to 2e863c4 2024-06-19 19:22:47 +02:00
Wolfgang Walther 16d0e1ca8c fix: Treat pre-release and docs versions correctly for new release workflow
Since we changed our release workflow, we have adjusted:
- the docs to use postgrest.org/en/v12/ -style URLs, i.e. only using the
major component.
- the pre-release / devel versions to contain only two instead of four
version parts, i.e. currently 12.3.
2024-06-19 08:42:34 +02:00
Wolfgang Walther 9657d46c42 docs: Fix linkcheck
Some URLs are still forbidden for our linkcheck tool, so disabling them
again.

Others are permanently redirected, so adjusting them.
2024-06-19 08:40:58 +02:00
renovate[bot]andWolfgang Walther 70e07ccbc6 chore(deps): update dependency urllib3 to v2.2.2 2024-06-17 21:04:03 +02:00
renovate[bot]andWolfgang Walther cc5c4ae5ef chore(deps): update codecov/codecov-action action to v4.5.0 2024-06-13 19:51:15 +02:00
renovate[bot]andWolfgang Walther af8b62ba10 chore(deps): update all dependencies 2024-06-13 08:46:23 +02:00
Laurence Isla d4e7d23268 docs: fix example of listener failure on read replicas 2024-06-12 19:46:23 -05:00
Laurence Isla 4e084b664c docs: clarify what is logged when "log-level=debug" 2024-06-12 18:30:19 -05:00
Laurence Isla 91024b6eab docs: add missing logs to stderr
- Schema cache stats are now logged to stderr
- Log when the LISTEN channel gets a notification
2024-06-12 18:30:19 -05:00
Laurence IslaandSteve Chavez 0e4fcd7387 docs: add "Listener" page
Co-authored-by: Steve Chavez <stevechavezast@gmail.com>
2024-06-12 18:30:19 -05:00
Wolfgang Walther 49a7aa9933 bump docs version to 12.2 2024-06-12 07:47:17 +02:00
steve-chavez ec89f6b90c bump version to 12.2.0 2024-06-11 09:57:05 -05:00
steve-chavez 82aa58a08f docs: deprecate EOL pg versions 2024-06-11 09:43:09 -05:00
Laurence Isla db85faf3ed docs: wrap in double quotes the filters with reserved characters in logical operators 2024-06-05 17:59:44 -05:00
Wolfgang Walther a2d00e305a nix: Make postgrest-release work on remotes without .git suffix 2024-06-05 21:32:41 +02:00
steve-chavez 4beac10d3d prevent GSSAPI error between Listener and pool
Brings back the the signaling/waiting between the connection pool and
the Listener.

Prevents the GSSAPI error shown on https://github.com/PostgREST/postgrest/issues/3569
2024-06-05 13:58:34 -05:00
Wolfgang Walther 70a8a80491 Revert "fix: Build static postgrest with GSSAPI support"
This reverts commit c94aa9ccd9.
2024-06-05 19:59:07 +02:00
Wolfgang Walther b4d235f72a nix: refactor to remove TODO
We have meanwhile received the commit in question.
2024-06-05 19:53:29 +02:00
steve-chavez 1a8b6972a8 correct exponential backoff on Listener
Clears the limitation mentioned on

https://github.com/PostgREST/postgrest/pull/3536

The Listener no longer uses the https://hackage.haskell.org/package/retry
package and instead uses a much simpler IORef in AppState for the
delays.

Additionally it no longer uses exception throwing/catching, which
is rather messy and brings some
concerns(https://github.com/PostgREST/postgrest/issues/3569#issuecomment-2146013327).
2024-06-05 08:52:18 -05:00
Laurence Isla aaf2d2e430 docs: add example for double embedding when doing OR filtering across embeds 2024-06-04 16:51:28 -05:00
Joel JakobssonandGitHub a46bea16e2 nix: fix slocat overlay with nix 2.22
This seems to happen on nix 2.22 only, v2.21 in CI and v2.20 locally work fine. The error is:

vendor folder is empty, please set 'vendorHash = null;' in your expression
For full logs, run 'nix-store -l /nix/store/kxnnr344n7gsxzc6kycj19hs19rvddjj-slocat-go-modules.drv'.
error: 1 dependencies of derivation '/nix/store/x9w480k36a11i99m6zp12d5cjijsn3lm-slocat.drv' failed to build

Since the slocat module doesn't actually have any dependencies, this shouldn't matter much.
2024-06-03 10:24:35 +02:00
Laurence Isla 50b2d302d0 docs: use "curl --get" for better readability when necessary 2024-05-31 20:24:06 -05:00
steve-chavez 82a43c2767 nix: postgrest-gen-ctags use haskdogs 2024-05-25 15:21:44 -05:00
steve-chavez a51a74b3c1 docs: shorten CLI 2024-05-24 18:05:45 -05:00
steve-chavez 1c371d7340 docs: clarify architecture 2024-05-24 18:05:45 -05:00
steve-chavez 30ca64d849 docs: improve observability 2024-05-24 18:05:45 -05:00
Laurence Isla 9c165e3edb fix: log connection pool events on log-level="debug" instead of "info" 2024-05-24 17:19:31 -05:00
Laurence Isla c8612f1df1 test: use only the first line in the logs to check the server version 2024-05-24 17:19:31 -05:00
steve-chavez da9e497ef1 changelog: add architecture doc 2024-05-23 19:34:42 -05:00
steve-chavez 47e9a2d134 refactor: Listener to own module 2024-05-23 19:34:42 -05:00
Laurence Isla 4e0ffa6d0a changelog: fix link for deprecated feature 2024-05-22 19:16:01 -05:00
Laurence Isla f5767b8d86 docs: add "config" and "schema_cache" endpoints to the admin server 2024-05-22 19:16:01 -05:00
Taimoor ZaeemandSteve Chavez 8cbcf9867b feat: add config db-hoisted-tx-settings to allow only hoisted function settings 2024-05-21 19:50:03 -05:00
Taimoor ZaeemandSteve Chavez a1582a6136 test: clean test_role_settings in io-tests 2024-05-21 19:50:03 -05:00
Laurence IslaandSteve Chavez 1ca5b6f6ba docs: ignore linkcheck for blog.frankel.ch 2024-05-21 17:23:11 -05:00
Laurence IslaandGitHub aea563bd82 fix: remove verbosity from some error logs
Error logs starting with "An error occured..." are replaced with "Failed to..."
2024-05-21 16:44:28 -05:00
Laurence IslaandGitHub 5d3d09923f refactor: unDRY PGRST prefix in errors for better search/grep 2024-05-21 14:34:00 -05:00
Wolfgang WaltherandWolfgang Walther 71711bb935 ci: Prevent caching cabal and stack cache in PRs
This was supposed to happen in c4b0bd34 and 279febe2, but somehow didn't
work, yet.
2024-05-21 20:44:41 +02:00
Wolfgang Walther ad5bb38d70 ci: Don't cancel previous runs for tag pipelines
Those create annoying "cancelled" notifications which looks like CI on
main was failing. It's not, though.

By just disabling the cancel-in-progress setting, but keeping the group
intact, this should queue multiple tag jobs / tag workflows behind each
other and still avoids the underlying problem which occurs when running
them in parallel.
2024-05-21 20:43:18 +02:00
Wolfgang Walther 89eae607f5 ci: Run build workflow in PRs which change composite actions
This should have been added in c4b0bd34 for cache-on-main and way before
that for artifact-from-cirrus.
2024-05-21 19:57:39 +02:00
Wolfgang Walther dcfc6673b7 ci: Fix out-of-sync GHC version for stack jobs 2024-05-21 19:39:15 +02:00
Laurence IslaandGitHub 7671d63f06 refactor: add isParent flag to O2O relationships
It allows to identify the side with the FK when isParent == False
2024-05-21 12:11:27 -05:00
Laurence Isla 04d6f41f45 chore: update commit prefix info in the PR template 2024-05-21 09:03:12 -05:00
Taimoor ZaeemandSteve Chavez 5cc32c7f87 fix: fix incorrect 413 error on pg 54* errors 2024-05-20 18:59:33 -05:00
renovate[bot]andWolfgang Walther 11c9e8dac9 chore(deps): update cachix/cachix-action action to v15 2024-05-20 19:26:45 +02:00
renovate[bot]andWolfgang Walther b8436fd397 chore(deps): update codecov/codecov-action action to v4.4.1 2024-05-20 19:26:18 +02:00
renovate[bot]andWolfgang Walther 4156070838 chore(deps): update haskell-actions/setup action to v2.7.3 2024-05-20 19:26:07 +02:00
Wolfgang WaltherandWolfgang Walther 9e6a89ffb8 ci: Install GHC/stack via haskell-actions/setup consistently
This action makes sure to always have the correct GHC and/or stack
version installed in all environments. This solves problem where ghc or
stack might not be available on newer macos images anymore or where
ghcup is not available by default on our new custom github runner on
arm.
2024-05-20 15:43:00 +02:00
Wolfgang WaltherandWolfgang Walther d08e5959a0 ci: Reduce stack cache size
This removes the GHC install from stack caches to reduce size.
2024-05-20 15:43:00 +02:00
Wolfgang WaltherandWolfgang Walther 279febe26b ci: Split cabal and stack work caches from regular cache
This is a first step to split up the cabal and stack caches in separate
pieces. Here we split the work folder, which just contains the
postgrest-specific build artifacts, into a separate cache.

More fine-grained caching should give us better cache hits and much
fewer upload size in the regular case, improving CI performance.

Since the work file caches are very small (about 30-40 MB) they are
cached for PRs, too. This will allow the majority of PRs, which only
change source code files, but no dependencies, to still have cached
their build files for additional commits.
2024-05-20 15:43:00 +02:00
Wolfgang WaltherandWolfgang Walther c4b0bd347b ci: Only save caches on main and release branches
This restores caches on all branches and pull requests, but only stores
them on the main branch and release branches. This prevents those caches
from being evicted early when we hit the 10 GB limit quickly.
2024-05-20 15:43:00 +02:00
steve-chavez 7e61c9deb0 feat: force read-write for listener connection 2024-05-19 23:14:19 -05:00
steve-chavez 3cf565614d fix: listener retries with exponential backoff
Also corrects the admin ready response which now considers the listener
state.
2024-05-19 20:48:59 -05:00
steve-chavez bfa4e1bedb test: isolate pg_terminate_backend to appname
The pg_terminate_backend done in the io test:

`test_fail_with_automatic_recovery_disabled_and_terminated_using_query`

Can affect other connections.
2024-05-19 19:58:27 -05:00
steve-chavez d5a4c5609e refactor: greppable 57P01 error code 2024-05-19 19:58:27 -05:00
renovate[bot]andWolfgang Walther 80783a7ee8 chore(deps): update actions/checkout action to v4.1.6 2024-05-19 21:24:42 +02:00
steve-chavez c8ba505b99 docs: linkcheck ignore patreon 2024-05-19 13:44:23 -05:00
steve-chavez ae91fd89b3 test: assert empty response text on 204 2024-05-19 13:44:23 -05:00
steve-chavez 756aad7827 fix: listener silent fail on replica
Update hasql-notifications to include the fix on
https://github.com/diogob/hasql-notifications/issues/24.

Which now reveals the following error:

```
$ postgrest-with-postgresql-16 --replica -f test/spec/fixtures/load.sql postgrest-run

17/May/2024:18:35:38 -0500: Successfully connected to PostgreSQL 16.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 13.2.0, 64-bit
17/May/2024:18:35:38 -0500: Could not listen for notifications on the "pgrst" channel. ERROR:  cannot execute LISTEN during recovery
17/May/2024:18:35:38 -0500: Retrying listening for notifications...
```

This is still not good because the LISTEN channel will be retried
forever without a backoff.
2024-05-18 23:33:04 -05:00
steve-chavez aa75412932 nix: PGRST_DB_URI preference for tmp db replica
When using `postgrest-with-postgresql-* --replica`, the PGRST_DB_URI
will set the replica host as preference. This to be able to run
quick manual tests with postgrest running on a replica.
2024-05-18 23:33:04 -05:00
Laurence IslaandWolfgang Walther ea4d1596b7 ci: fix release not executing when skipping CI for docs and tests 2024-05-17 08:09:19 +02:00
steve-chavez 33b6ba8199 refactor: move checkIsFatal logic to usePool
The fatal logic is now inside `usePool`. It centralizes the
logic which is better for Locality of Behavior.

Removes:

- The need to do checkIsFatal on other parts of the code
- SCFatalFail/ConnFatalFail states which are no longer needed.
2024-05-16 17:40:25 -05:00
renovate[bot]andWolfgang Walther 9d763aef00 chore(deps): update codecov/codecov-action action to v4.4.0 2024-05-15 09:20:39 +02:00
steve-chavez d25df459ea feat: add metric label for scache load
localhost:3001/metrics now includes:

pgrst_schema_cache_loads_total{status="FAIL"} 352.0
pgrst_schema_cache_loads_total{status="SUCCESS"} 3.0

This allows testing the failure case on:

https://github.com/PostgREST/postgrest/issues/3424#issuecomment-2104904910
2024-05-14 17:57:21 -05:00
steve-chavez 05447bae33 nix: add postgrest-gen-jwt/secret for manual tests
```
$ postgrest-gen-secret
uMd97XSQzNkA1CWhMZ7u88Pj0RNyhrpo

$ postgrest-gen-jwt postgrest_test_author
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoicG9zdGdyZXN0X3Rlc3RfYXV0aG9yIn0.Xod-F15qsGL0WhdOCr2j3DdKuTw9QJERVgoFD3vGaWA
```

Also modifies postgrest-run to include a default PGRST_JWT_SECRET for
quicker manual tests.
2024-05-13 12:37:08 -05:00
Wolfgang WaltherandWolfgang Walther f6b2aa5a5d chore(deps): Update stackage extra deps 2024-05-13 08:08:18 +02:00
Wolfgang WaltherandWolfgang Walther 923a271105 chore(deps): Update stackage snapshot to lts-22.20
The freebsd build is kept at 21.7 / GHC 9.4.5, because no newer GHC
versioin is supported here.
2024-05-13 08:08:18 +02:00
Wolfgang Walther b8fe512def ci: Avoid trying to upload loadtest reports for tag pipelines
In this case the loadtest doesn't run anymore, so the job would fail.
2024-05-12 15:28:13 +02:00
Wolfgang Walther 271b3b273d ci: Disable tagging releases in forks 2024-05-12 15:10:28 +02:00
Wolfgang Walther 5c040e74f5 ci: Prevent running check, docs and test suites on tags
This doesn't make sense, because each tag is only pushed when those
pipelines have already passed. Thus, we can save time and avoid wasting
resources and don't run those again.
2024-05-12 15:01:20 +02:00
Wolfgang Walther 4a3fdc175c ci: Skip release jobs on outdated tags
This happens a commit is pushed to main while the pipelines have not
finished for the previous commit. In this case the devel-tag pipelines
will run concurrently, leading to unpredictable results for the devel
release.
2024-05-12 14:58:27 +02:00
Wolfgang Walther aef29d49ba docs: Mark db_tx_end as db-configurable 2024-05-12 11:53:59 +02:00
Wolfgang Walther c6f152f5b4 refactor: Remove left-over raw_media_types from Database.hs
This was removed in #2825.
2024-05-12 11:53:45 +02:00
Taimoor ZaeemandWolfgang Walther 7a5079542c docs: update docs for pg error 53400 2024-05-12 11:47:05 +02:00
Wolfgang WaltherandWolfgang Walther 71885bdba6 test: Avoid freeport() collisions in io tests
It's very unlikely, but it can (and did) happen that both the server and
admin ports have the same number returned from freeport(). This then
leads to a situation where PostgREST will accept the same port in both
cases, because the host "localhost" will allow binding to ipv4 or ipv6
respectively. This will make the IO tests fail.

This change makes sure that the admin port will never be the same as the
server port and thus avoids this problem.
2024-05-12 11:45:06 +02:00
Wolfgang WaltherandWolfgang Walther 8392357863 test: Fix internal_schema_cache_sleep after 747c78f6
The $subject commit broke internal_schema_cache_sleep for other tests.
This reverts the order change, but keeps the scaling by x1000 to ms and
thus changes other users of this setting to the new scale.
2024-05-12 11:44:43 +02:00
Laurence IslaandSteve Chavez 334c2710f6 changelog: update to 12.0.3 2024-05-11 13:30:50 -05:00
Wolfgang Walther 07222cff7b chore: Restrict macos to v12 2024-05-10 08:19:29 +02:00
David BaynardandWolfgang Walther 575dd4cf70 nix: Make default.nix extensible
Moving values defined with `let` to the function arguments (with
defaults) means other consumers of `default.nix` can customize these
values.

One example is a `flake.nix`, which can then supply the `nixpkgs` input.
2024-05-09 22:23:06 +02:00
Wolfgang Walther d03b321659 ci: Actually pass GHC_VERSION to arm build script 2024-05-09 21:58:37 +02:00
Wolfgang Walther 5580fe0040 ci: Make arm scripts fail on error 2024-05-09 21:46:10 +02:00
Wolfgang Walther 7d713b8c53 ci: Tag arm docker image properly for releases 2024-05-09 21:46:10 +02:00
Wolfgang Walther 2cb45bd5f8 ci: Extract changelog properly for releases 2024-05-09 21:46:10 +02:00
Andrei DziahelandWolfgang Walther 3026c1f308 fix: Parse accept header case-insensitively
The Accept header is parsed case-insensitively now, introducing proper
handling of media types specified in upper- and/or mixed-case.

Fixes #3478
2024-05-09 19:30:42 +02:00
Wolfgang Walther 1fa35cb3b8 test: Fix coverage & style check from c4295b3d 2024-05-09 18:50:00 +02:00
Wolfgang Walther 01a56db7d8 test: Make unicode insert test work in parallel mode
Changing the PK here will avoid duplicate conflicts with another test.

References #1799
2024-05-09 18:05:44 +02:00
Wolfgang Walther 577a7c7598 test: Move limited delete/update tests into separate spec file
This potentially allows to run the remaining tests in those files in
parallel mode.

References #1799
2024-05-09 18:05:44 +02:00
Wolfgang Walther c4295b3d63 test: Make PgSafeUpdateSpec parallel-ready 2024-05-09 18:05:44 +02:00
Wolfgang Walther aa94e436fa test: Missing space 2024-05-09 18:05:44 +02:00
Wolfgang Walther cf7a789e9e test: Improve failing test output for requestMutation 2024-05-09 18:05:44 +02:00
Wolfgang Walther a7ed5db78a test: Sort test suites in spec/Main.hs 2024-05-09 18:05:44 +02:00
Wolfgang WaltherandWolfgang Walther d903a8a115 nix: Adjust postgrest-release to new release workflow
This changes the postgrest-release tool to work with our new workflow.
It can be run on main and the v* release branches. When on a release
branch, it will bump a patch version and push to that branch only.

When on main, it will bump a minor version by default. To bump a major
version, pass --major. This first bump will be force-pushed to the
v<major> branch. A second bump to the current development version will
then be pushed to the main branch.

The tool will not tag commits anymore - this happens in CI
automatically.

References #3113
Resolves #3082
2024-05-09 18:05:04 +02:00
Wolfgang Walther a427fb67f6 ci: Fetch tags before checking whether tag exists 2024-05-09 14:53:38 +02:00
Wolfgang Walther 3afa5f6a36 ci: Move docker-hub-readme.md to base folder
This is not related to nix tooling anymore, because a github action
without any nix tooling is updating this now.
2024-05-09 14:35:52 +02:00
renovate[bot]andWolfgang Walther fdcadb3d53 chore(deps): update actions/checkout action to v4.1.5 2024-05-09 14:08:51 +02:00
Wolfgang WaltherandWolfgang Walther 747c78f6f4 test: Prevent test_admin_ready_includes_schema_cache_state from timing out
By increasing the delays in this test by factor 400x, postgrest will not
swamp pg with connection retries after the failed schema cache anymore.

This would happen because there is no backoff included after fatal
errors. Once it does, the io tests hang indefinitely in CI.
2024-05-09 13:36:03 +02:00
Wolfgang WaltherandWolfgang Walther e96e16fa27 test: Reset statement timeout after each test
The statement timeout needs to be cleaned up after each test that
modifies it instead of before the test. Otherwise the changed timeout
leaks into other tests.
2024-05-09 13:36:03 +02:00
steve-chavez 0060abeb01 feat: /live and /ready respond with 500 on failure
503 is still used by /ready to indicate a transient state
that can be recovered from.
2024-05-08 17:19:48 -05:00
steve-chavez f9e9740999 nix: add postgrest-ctags command
Generates ctags for Haskell and Python code.
2024-05-08 12:27:26 -05:00
steve-chavez 1b584f7e9c refactor: is ready Admin logic to AppState 2024-05-08 11:27:58 -05:00
Wolfgang WaltherandWolfgang Walther 1374178f27 ci: Improve performance for nix jobs in CI
Defaulting to max-jobs = auto should improve build times by using more
cores.

Setting always-allow-substitutes to true should cause all nix
derivations to be cached on cachix, which should improve performance of
the MacOS job dramatically, when no rebuilds need to happen.
2024-05-07 14:42:12 +02:00
Wolfgang Walther cb6151eb1b ci: Make artifact-from-cirrus action succeed when cirrus job doesn't start up in PR 2024-05-07 08:29:31 +02:00
Wolfgang Walther b006016d07 ci: Make "release / tag" job detect existing tags
The release / tag job has logic to decide whether to push a new tag on
stable branches, which depends on the all the tags being fetched. The
checkout action doesn't do that by default, so enable that.
2024-05-07 08:09:00 +02:00
steve-chavez df9b373465 docs: better place for application_name 2024-05-06 11:12:45 -05:00
steve-chavez b34c00c522 changelog: add missing entry for 3184 2024-05-06 11:12:45 -05:00
steve-chavez 1d4f31514f changelog: move 3340 from fixed to added
Since it was a feature
2024-05-06 11:12:45 -05:00
steve-chavez 7e91e5311d fix: not adding application_name on all URIs 2024-05-06 11:12:45 -05:00
Taimoor ZaeemandSteve Chavez 21bc48ad9a fix: fix wrong 503 Service Unavailable on pg error 53400 2024-05-06 07:54:11 -05:00
renovate[bot]andWolfgang Walther cabe744f01 chore(deps): update ubuntu docker tag to v24 2024-05-05 18:23:53 +02:00
Wolfgang Walther 463dfb552d chore(deps): update actions/checkout hash for release/prepare job
Accidentally committed an outdated version, this aligns it with all the
other references to the same action.
2024-05-05 17:48:44 +02:00
Wolfgang Walther a57d12b102 ci: Use the new DOCKER_ vars for docker-arm job 2024-05-05 17:11:22 +02:00
Wolfgang Walther d9ba9a8208 ci: Avoid devel release failure when no assets to clean up exist 2024-05-05 14:35:40 +02:00
Wolfgang Walther 8433f9812b ci: Replace assets of existing devel release properly 2024-05-05 13:38:58 +02:00
Wolfgang Walther c67f1c398c ci: Fix docker release job after 2f98d837 2024-05-05 13:33:48 +02:00
Wolfgang Walther 2f98d837ff ci: Push docker hub description automatically on main branch 2024-05-04 21:55:52 +02:00
Wolfgang Walther eec35d4569 ci: Avoid recreating devel release
Instead, the existing release is edited, which avoids notifications
for pre-releases.
2024-05-04 21:54:46 +02:00
Wolfgang Walther 9fe90bf9a8 ci: Fix pushing of devel-arm docker tag 2024-05-04 21:53:36 +02:00
Wolfgang Walther 58d8133454 ci: Enable docker push for releases 2024-05-04 21:14:40 +02:00
Wolfgang Walther fe0f2f708a ci: Use "devel" as the pre-release name instead of the commit title 2024-05-04 21:14:23 +02:00
Wolfgang WaltherandWolfgang Walther dd8d51ab71 ci: Automate patch releases and pre-releases
This work by automatically pushing a new tag on main and release
branches after each commit. The tag will be "devel" on main and the
version from postgrest.cabal for release branches. The release
workflow then runs as a tag pipeline, making the actual release.

For release branches, the tag will only be created if a tag for this
version doesn't exist, yet. This means to actually make a new patch
release, we still need to bump the version in postgrest.cabal. We
can automate this later as part of our backport-bot.

Resolves #2006
Resolves #2997
2024-05-04 20:42:45 +02:00
Wolfgang Walther cdcab34852 nix: Make postgrest-lint fail on dead nix code 2024-05-04 20:34:59 +02:00
Wolfgang Walther 1ca2d1f8be ci: Refactor get_cirrusci_freebsd script to GitHub action
This should make it more reliable and also easier to re-use, if we need
to.

Resolves #2555
2024-05-04 14:56:57 +02:00
Wolfgang Walther 5aa62d944d ci: Force running FreeBSD build in tag and push pipelines
This now behaves similar to other CI build jobs, which always run on the
main branches, but only conditionally on PRs, depending on which files
changed.
2024-05-04 14:56:57 +02:00
renovate[bot]andWolfgang Walther 5c29742d7f chore(deps): update nixbuild/nix-quick-install-action action to v28 2024-05-04 13:35:13 +02:00
renovate[bot]andWolfgang Walther cc7165ca10 chore(deps): update codecov/codecov-action action to v4.3.1 2024-05-04 12:21:53 +02:00
renovate[bot]andWolfgang Walther 277342bf8c chore(deps): update ubuntu:jammy docker digest to a6d2b38 2024-05-04 12:21:37 +02:00
steve-chavez d9a51f23ce test: sanity tests for primary and replica
* new --replica option to `postgrest-with-postgresql-*`
* new command `postgrest-test-replica`
* new sanity tests on test_replica.py
* add postgrest-test-replica to postgrest-check and postgrest-coverage
2024-04-30 18:35:40 -05:00
steve-chavez cdb877135b test: flush stdout so readline doesn't hang 2024-04-30 13:52:51 -05:00
Taimoor ZaeemandSteve Chavez 7e5fd317e6 docs: add docs page for CLI 2024-04-29 12:51:54 -05:00
Laurence IslaandGitHub 40bd9a7769 ci: DRY the cleanup job for ARM servers 2024-04-29 12:20:00 -05:00
renovate[bot]andWolfgang Walther 5e41121551 chore(deps): update ubuntu:jammy docker digest to 6d7b5d3 2024-04-26 08:34:18 +02:00
renovate[bot]andWolfgang Walther 46d8dfde42 chore(deps): update actions/checkout action to v4.1.4 2024-04-26 08:31:33 +02:00
Laurence Isla 85a4e35a2d docs: fix http status for PGRST121 error 2024-04-24 15:04:35 -05:00
renovate[bot]andWolfgang Walther 359a14533b chore(deps): update actions/download-artifact action to v4.1.7 2024-04-24 20:53:56 +02:00
Laurence IslaandGitHub 69bbce5b32 feat: improve PGRST121 error message
* Clarify the message field
* Show failed MESSAGE or DETAIL in the the PGRST121 error's details field
* Show the correct JSON format in the hint field
2024-04-24 13:33:10 -05:00
steve-chavez 3eff4670f0 docs: metrics for schema cache and connection pool 2024-04-23 19:08:37 -05:00
steve-chavez 357400b2b8 feat: schema cache metrics 2024-04-23 19:08:37 -05:00
steve-chavez 653c7955b2 feat: connection pool metrics in admin server 2024-04-23 19:08:37 -05:00
steve-chavez 29cd7d195c nix: add instructions for haskell-packages.nix 2024-04-23 19:08:37 -05:00
Laurence IslaandGitHub e788776ec3 ci: clean ARM server even when previous steps or jobs fail 2024-04-23 16:34:55 -05:00
steve-chavez 3e615bd0d9 feat: add log-level=debug 2024-04-22 21:36:29 -05:00
renovate[bot]andWolfgang Walther bddfa2782d chore(deps): update actions/checkout action to v4.1.3 2024-04-22 21:30:23 +02:00
Wolfgang Walther 6858693291 ci: Be explicit about the version of linux and windows runner images
Using the -latest tag is potentially prone to errors, because an update of the tag
could break our CI. This recently happend with macos-latest, which we downgraded
to macos-12 earlier.

Using an explicit version reference makes this problem much less likely - in fact
renovate will pick up new versions once they exist and will suggest updates for it.
Thus, we will see the failures in a related PR instead of randomly everywhere.
2024-04-22 21:20:30 +02:00
renovate[bot]andWolfgang Walther dcb9f92b80 chore(deps): update actions/upload-artifact action to v4.3.3 2024-04-22 21:14:58 +02:00
renovate[bot]andWolfgang Walther 6a38cc098d chore(deps): update actions/download-artifact action to v4.1.6 2024-04-22 21:14:44 +02:00
Wolfgang WaltherandWolfgang Walther 81ceac8ba3 ci: revert to macos-12
GitHub Actions updated the latest macos image to macos-14 [1], which made both
of our macos build jobs fail.

This reverts to macos-12, which should fix CI for the moment. We can then look
into migrating to macos-14 explicitly.

[1]: https://github.blog/changelog/2024-04-01-macos-14-sonoma-is-generally-available-and-the-latest-macos-runner-image/
2024-04-22 21:11:03 +02:00
Taimoor ZaeemandSteve Chavez 88abf600c4 fix: fix wrong http status on pg error 42P17 infinite recursion 2024-04-22 13:40:07 -05:00
Wolfgang WaltherandWolfgang Walther 80f83f0366 nix: Update to latest nixpkgs#master 2024-04-20 17:06:29 +02:00
Wolfgang WaltherandWolfgang Walther b6a50ab8ed nix: Allow changing to a fork in nixpkgs-version.nix 2024-04-20 17:06:29 +02:00
Wolfgang Walther 87a463f857 test: fix broken io tests after last commit
Forgot to update the snapshots.
2024-04-20 16:03:53 +02:00
Wolfgang Walther ed407350ad fix: hoist function settings with memory units properly
f9ee1f7e introduced the hoisting of function settings as transaction-scoped
settings. However, this currently doesn't work with memory units, which are
case-sensitive according to the docs [1]. This removes the lowercasing of
values to make them work.

This is not added to the CHANGELOG, because this feature was not released, yet.

[1]: https://www.postgresql.org/docs/current/config-setting.html#CONFIG-SETTING-NAMES-VALUES
2024-04-20 13:58:04 +02:00
renovate[bot]andWolfgang Walther de788c9832 chore(deps): update ubuntu docker tag to v22 2024-04-20 12:39:22 +02:00
Wolfgang Walther 07dc37e66f chore: limit docutils to <0.21.0
sphinx-rtd-theme currently does not support docutils 0.21.x.

This restriction can be lifted once the upstream issue has been resolved:
https://github.com/readthedocs/sphinx_rtd_theme/issues/1557
2024-04-20 12:38:24 +02:00
renovate[bot]andWolfgang Walther c2994fb0b1 chore(deps): update ubuntu:focal docker digest to 71b82b8 2024-04-20 12:33:30 +02:00
renovate[bot]andWolfgang Walther 6863fe1e26 chore(deps): update actions/download-artifact action to v4.1.5 2024-04-20 12:33:06 +02:00
renovate[bot]andWolfgang Walther dc6399abad chore(deps): update actions/upload-artifact action to v4.3.2 2024-04-20 12:32:55 +02:00
Laurence IslaandSteve Chavez 0d13b842a6 fix: OpenAPI now tags a FK correctly on O2O relationships 2024-04-18 17:08:14 -05:00
steve-chavez 1bf0c54dd6 feat: log connection pool events on log-level=info 2024-04-15 18:31:51 -05:00
Steve ChavezandGitHub 9d1dc783bf test: use postgrest.read_stdout in io tests (#3412)
It's easier to maintain this way in case there are new log lines.
2024-04-15 13:51:23 -05:00
steve-chavez 69c6ce9c38 refactor: use LogLevel in Logger
* remove Logger dependency on Auth.
2024-04-14 20:10:01 -05:00
steve-chavez c57ec52229 refactor: make stdout explicit on Logger
Otherwise it's hard to know we're logging to stdout.
2024-04-14 20:10:01 -05:00
Taimoor ZaeemandSteve Chavez 973201a8d3 fix: remove rejected mediatype application/vnd.pgrst.object+json from response 2024-04-13 16:07:38 -05:00
steve-chavez fbc4d565ca refactor: move debounce from AppState to Logger
Will allow to capture accurate timeout metrics.
2024-04-12 14:29:39 -05:00
steve-chavez 2de32fc108 refactor: observation handler to AppConfig
With this:

- Is no longer necessary to pass observer as an argument
  to every function that needs observations.
- We can invoke the observer on every function that uses AppConfig.
  However it'd be better to just call the observer in the upper modules
  (like on App.hs).
2024-04-12 14:29:39 -05:00
Laurence IslaandGitHub 460259548d perf: fix space leaks by downgrading fuzzyset to v0.2.4 2024-04-11 14:13:20 -05:00
steve-chavez b88191299f docs: remove schema cache dep from Query 2024-04-10 19:35:23 -05:00
Laurence Isla 4bcd6725fe chore: update sponsor logos 2024-04-10 18:37:57 -05:00
Laurence Isla 70396d3f6c docs: add dark mode
- Uses sphinx-rtd-dark-mode for sphinx-rtd-theme
- Adds dark mode logos for sponsors
2024-04-10 18:37:57 -05:00
steve-chavez c9136816a2 docs: move ARCHITECTURE.md to architecture.rst 2024-04-10 15:51:08 -05:00
steve-chavez 4428253efe docs: move health_check.rst to admin_server.rst 2024-04-10 15:51:08 -05:00
steve-chavez 47a4e2bfd0 docs: add proxy and config reloading to arch 2024-04-10 13:57:49 -05:00
renovate[bot]andWolfgang Walther 5880e62d7a chore(deps): update codecov/codecov-action action to v4.3.0 2024-04-10 09:10:27 +02:00
steve-chavez 2772b78013 docs: add config/cli/developer to arch 2024-04-08 15:51:30 -05:00
steve-chavez b91908b749 docs: add architecture description 2024-04-08 13:50:00 -05:00
steve-chavez ab73624366 docs: add architecture diagram 2024-04-08 11:57:03 -05:00
steve-chavez 5ab317caa0 refactor: one entrypoint for Plan/Response/Query
- deduplicates timing calculation for the different steps
- enabling query logging later on will be simpler
2024-04-04 08:58:55 -05:00
renovate[bot]andWolfgang Walther a66738b892 chore(deps): update codecov/codecov-action action to v4.2.0 2024-04-04 07:17:26 +02:00
steve-chavez 3d55f77bae Revert "fix: slow responses on schema cache reload"
This reverts commit 727ef465c1.

Also documents requests waiting for the schema cache.
2024-04-02 23:15:09 -05:00
steve-chavez d7c64a93f8 docs: update schema cache 2024-04-01 19:03:14 -05:00
steve-chavez 2543b8d724 fix: clarify PGRST204 error message 2024-04-01 19:03:14 -05:00
Wolfgang Walther 6de3ba543d nix: Use minimal set of texlive dependencies for docs-render 2024-03-31 22:05:02 +02:00
steve-chavez 06ff56d323 refactor: move isolation/settings logic to Plan.hs 2024-03-28 18:31:42 -05:00
steve-chavez c33ca4e60c refactor: dry timings calculation for openapi 2024-03-28 18:31:42 -05:00
steve-chavez b75cc853b4 chore: fix compilation
The App.hs module was missing NamedFieldPuns.
2024-03-27 19:18:17 -05:00
steve-chavez 745e7868b0 refactor: dry some timings calculation 2024-03-27 18:22:34 -05:00
Laurence Isla 378c11104b test: fix some in-db config values
To correctly test in-db override of config file values, the former must be different from the latter.
2024-03-27 15:59:35 -05:00
Laurence Isla 428a6fef63 fix: in-db config values not loading for pgrst.server_trace_header and pgrst.server_cors_allowed_origins 2024-03-27 15:59:35 -05:00
renovate[bot]andWolfgang Walther d02540ac44 chore(deps): update codecov/codecov-action action to v4.1.1 2024-03-26 19:49:51 +01:00
Laurence IslaandGitHub 3f162535b8 nix: add example on how to use a library locally 2024-03-26 11:39:07 -05:00
steve-chavez a5bb20bbf8 refactor: remove unreacheable 404 2024-03-25 18:17:01 +01:00
renovate[bot]andWolfgang Walther f1f01f1f5c chore(deps): update actions/cache action to v4.0.2 2024-03-19 21:58:16 +01:00
Wolfgang Walther ee359192fa ci: Fix duplicated version number for actions/cache 2024-03-19 21:57:30 +01:00
steve-chavez 941ea0f929 test: notify do nothing 2024-03-19 13:22:51 +03:30
steve-chavez ee8b3ef8fe fix: log on LISTEN notification 2024-03-19 13:22:51 +03:30
Ian WijmaandGitHub 47b70b8329 chore: fix link to docs in readme
Fixed the link to the archived documentation repo.
2024-03-16 11:41:44 +01:00
steve-chavez 727ef465c1 fix: slow responses on schema cache reload 2024-03-15 11:53:29 -05:00
renovate[bot]andWolfgang Walther 210cded560 chore(deps): update nixbuild/nix-quick-install-action action to v27 2024-03-15 17:13:24 +01:00
steve-chavez 11d8da046c fix: incorrect /ready response on slow schema load 2024-03-15 11:03:01 -05:00
steve-chavez 4b289b1c97 test: requests wait for schema cache load
* nix: add postgrest-test-big-schema command
2024-03-15 06:18:26 -05:00
steve-chavez 92ac7e574e chore: clarify concurrent notifications test 2024-03-14 23:17:05 -05:00
steve-chavez ec7ab271d6 chore: add postgrest roles to big_schema.sql 2024-03-14 11:21:17 -05:00
steve-chavez 4a2c851ae5 chore: update cabal/stack new hasql-notifications 2024-03-13 20:43:46 -05:00
steve-chavez 86e15dbb77 fix: upgrade hasql-notifications to show error 2024-03-13 11:14:11 -05:00
Steve ChavezandGitHub 00f5780415 fix: don't hide error on LISTEN channel failure (#3323) 2024-03-11 19:56:35 -05:00
Steve ChavezandGitHub 650249ed29 perf: remove json_typeof (#3316) 2024-03-08 16:02:28 -05:00
steve-chavez 9405a62de8 docs: update links to cache/config notify reload 2024-03-08 10:31:55 -05:00
steve-chavez 05cdbb34c2 test: insignificant JSON whitespace on writes 2024-03-07 17:14:16 -05:00
steve-chavez 8eb88ef218 test: reduce threshold on memory tests 2024-03-07 16:01:21 -05:00
Laurence IslaandWolfgang Walther cf7b9bad08 nix: only json output for postgrest-dump-schema 2024-03-07 16:50:14 +01:00
Laurence IslaandWolfgang Walther c276e9741e nix: fix postgrest-dump-schema with optional yaml output 2024-03-07 16:50:14 +01:00
renovate[bot]andWolfgang Walther 6a2677d985 chore(deps): update ubuntu:focal docker digest to 80ef4a4 2024-03-07 16:27:56 +01:00
Steve ChavezandGitHub 58999f5102 changelog: add how to detect o2o rels on v10.0.0 (#3312) 2024-03-06 20:21:58 -05:00
Steve ChavezandGitHub 2f91853cb1 docs: add index usage section (#3299) 2024-03-02 11:00:31 -05:00
Laurence IslaandWolfgang Walther b8b957667b docs: add enums to working with data types how-to 2024-03-02 12:14:13 +01:00
renovate[bot]andWolfgang Walther db56c3eecc chore(deps): update actions/download-artifact action to v4.1.4 2024-03-02 12:10:10 +01:00
Wolfgang Walther d5cb6b57ea chore: Prevent unnecessary rebases for renovate PRs
This saves a few CI cycles.
2024-03-01 08:57:47 +01:00
Wolfgang Walther f72b47c485 chore: Add full semver comment to used actions 2024-03-01 08:54:27 +01:00
renovate[bot]andWolfgang Walther 0eda5df644 chore(deps): update actions/cache digest to ab5e6d0 2024-03-01 08:11:53 +01:00
Steve ChavezandGitHub d3f15baa4a nix: add postgrest-profiled-run (#3292) 2024-02-29 16:44:10 -05:00
Wolfgang WaltherandWolfgang Walther 29ffb9329b ci: Build on FreeBSD only when build workflow runs
This prevents running the freebsd build when only tests or docs change.
2024-02-29 21:27:05 +01:00
Wolfgang Walther a1480c758e ci: Run postgrest-test-doctests without nix-shell
Resolves #3183
2024-02-27 10:02:42 +01:00
Laurence IslaandGitHub 2479e0d4a5 changelog: clarify breaking change about dropping legacy gucs 2024-02-26 20:13:28 -05:00
renovate[bot]andWolfgang Walther 4878719b90 chore(deps): update actions/download-artifact digest to 87c5514 2024-02-26 22:13:53 +01:00
Wolfgang Walther 629ace0103 ci: Replace actions cache for macos build job with smart cachix lookup 2024-02-26 22:06:01 +01:00
Wolfgang Walther 73199127bb chore: Remove cabal.project.non-nix
We don't have any source-repository-package stanzas in there anymore, so there's no
need to keep it around anymore.
2024-02-26 22:06:01 +01:00
Wolfgang Walther a090f28cea ci: Install only the required nix tools
A previous commit allowed to select each tool separately on the toolbox.

This commit makes use of that for CI to possibly speed up loading from cachix
a little bit. It will also cause fewer cache misses when nix code is changed.

Resolves #3183
2024-02-26 22:06:01 +01:00
Wolfgang Walther c7eb4036a1 nix: Export all tools on each toolbox
This allows targeting each tool separately for installs.
2024-02-26 22:06:01 +01:00
Wolfgang Walther 08901323a3 nix: Move parallelCurl to devTools
This is not a with-tool, because it doesn't wrap another command.
2024-02-26 22:06:01 +01:00
Wolfgang Walther 1db6c04a28 ci: Remove unused .github/release script 2024-02-26 22:06:01 +01:00
Wolfgang Walther e70f001cfc ci: Remove matrix build for GHC 9.4.8 via Cabal on Linux
This is to reduce storage requirements for GitHub Actions cache. We already build with GHC 9.4.x
via Nix on Linux x64, via stack on FreeBSD, MacOS and Windows and via Cabal on Linux ARM. That
should cover 9.4.x enough.
2024-02-26 22:06:01 +01:00
Wolfgang Walther 3b497a5db4 ci: Remove useless cache- prefix for cache names
This just makes the name longer than needed.
2024-02-26 22:06:01 +01:00
renovate[bot]andWolfgang Walther 5bf0b9ca14 chore(deps): update codecov/codecov-action action to v4.1.0 2024-02-26 22:00:48 +01:00
Wolfgang Walther e5aee49891 ci: Prevent Upload Reports job from running on release branches
This won't work, since the workflows on the backbranches are structured differently
and thus the job can't find the loadtest.md file anywhere.
2024-02-24 23:33:25 +01:00
renovate[bot]andWolfgang Walther 873ac6221e chore(deps): pin dependencies 2024-02-24 21:15:38 +01:00
renovate[bot]andWolfgang Walther f97c6e3db3 chore(deps): update dependency urllib3 to v2.2.1 2024-02-24 20:38:43 +01:00
Wolfgang WaltherandWolfgang Walther d6153be67a ci: Split Lint & Style from Test workflow
The Lint & Style job needs to run on all PRs, not only when something "test" related
changes. Otherwise not all workflow, nix or other files are style-checked and linted.
2024-02-24 20:38:25 +01:00
renovate[bot]andWolfgang Walther e6cde92f11 chore(deps): update dependency sphinxext-opengraph to v0.9.1 2024-02-24 20:01:04 +01:00
renovate[bot]andWolfgang Walther f7879bd4b1 chore(deps): update dependency docutils to v0.20.1 2024-02-24 20:00:51 +01:00
Wolfgang WaltherandWolfgang Walther 11accb760a ci: Pass cachix token to all nix jobs 2024-02-24 19:59:46 +01:00
Wolfgang WaltherandWolfgang Walther 44e041282c ci: Switch action-get-latest-tag to github-action-get-previous-tag
The former is not maintained anymore and produces deprecation warnings in CI.
2024-02-24 19:59:46 +01:00
renovate[bot]andWolfgang Walther f2483dc722 chore(deps): update codecov/codecov-action action to v4 2024-02-24 18:41:06 +01:00
renovate[bot]andWolfgang Walther c18ccf5232 chore: Switch from dependabot to renovate
This allows much better configuration and customization and will keep our
release branches up2date, too.
2024-02-24 18:10:30 +01:00
Wolfgang WaltherandWolfgang Walther 79e172a65c ci: Fix failing Upload Reports job 2024-02-24 17:27:01 +01:00
Wolfgang WaltherandWolfgang Walther b49e3b1e5b ci: Only run ARM-related jobs if related settings are present
This allows CI to properly run through in a fork, which doesn't have the SSH_ARM_xxx settings.
2024-02-24 17:27:01 +01:00
Wolfgang Walther 6caaee777b nix: Simplify static.nix
The enable-executable-static flag is set by default, so doesn't make a difference.

The pkg-config improvement for libpq was merged upstream, so we can use the same
here already.
2024-02-24 15:38:02 +01:00
Wolfgang Walther 86b2e59f6f chore: Add tested-with field to postgrest.cabal
This documents supported GHC versions. GHC 9.8.1 is currently commented
out to reflect the fact that PostgREST can't currently be built with it
straight from hackage - we still require some overrides in cabal.project
for that.
2024-02-24 14:07:55 +01:00
Wolfgang Walther 0738785d72 test: Improve dump-schema snapshot test formatting
Splitting the output into separate files and adding top-level newlines makes this
much better to read and understand when looking at diffs.

Inspired by #1699
2024-02-24 14:01:19 +01:00
Wolfgang WaltherandWolfgang Walther 2c2513b9cd deps: Use libpq v16 for freebsd build 2024-02-24 13:46:05 +01:00
Wolfgang WaltherandWolfgang Walther 9314dede2d ci: Build with GHC 9.8.2 2024-02-24 13:37:24 +01:00
Wolfgang Walther ccc386cf38 nix: Add deadnix to postgrest-lint and remove dead nix code 2024-02-24 13:08:04 +01:00
Wolfgang Walther caf0a70171 nix: Clarify pkg-config usage in static.nix a bit
This will make it easier to make the switch from pkgsCross.libpq to pkgsStatic.libpq,
once that is possible to build upstream - if ever.
2024-02-24 12:08:40 +01:00
Wolfgang Walther 22e8906c40 nix: Run faster tasks first in postgrest-lint
This avoid long waiting times when waiting for the result of shellcheck or actionlint.
2024-02-23 14:02:05 +01:00
Wolfgang WaltherandWolfgang Walther 0b3f6de015 ci: Merge docs-spellcheck and docs-dictcheck jobs
Both jobs run really quick, running them separate is just a waste of resources. Plus,
they are semantically closely related anyway.
2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther c59736b902 ci: Rename release jobs for consistency 2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther 19423b0c0a ci: Only run PR workflows when files have changed 2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther e9abb64fc0 ci: Split Build workflow from CI workflow 2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther 57ef98a316 ci: Split Test workflow from CI workflow 2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther 31a3864f85 ci: Remove files from arm server immediately on non-release push
While this duplicates code a little bit, it makes it much simpler to refactor later on.
2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther 073280ad32 ci: Require docs job to pass before making a release 2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther 333f8cf592 ci: Cancel all workflows consistently in pull requests without blocking main
The previous setup would cause multiple commits on main to be stuck in a pending
state, waiting for the previous run to be finished.

The new group specification is taken from:
https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value

This also adds the same settings for the docs workflow.
2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther f9fdf666ad ci: Use if without ${{ }} where possible 2024-02-22 20:12:18 +01:00
Wolfgang WaltherandWolfgang Walther c54e0d3353 ci: Use actions/download-artifact instead of dawidd6/action-download-artifact
The "default" action now supports downloading from different workflow runs, too, so
we might as well use it.
2024-02-22 20:12:18 +01:00
steve-chavez 3a3601cbeb feat: log schema cache load time 2024-02-21 18:16:37 -05:00
Wolfgang WaltherandWolfgang Walther 1a141c19df nix: Add postgrest-docs-render to render latex documents 2024-02-21 09:40:11 +01:00
Wolfgang WaltherandWolfgang Walther b8f35d880f docs: Fix broken redirects in api.rst 2024-02-21 09:40:11 +01:00
Wolfgang WaltherandWolfgang Walther 791b86cbf6 docs: Add note about not support Stored Procedures
Resolves https://github.com/PostgREST/postgrest-docs/issues/147
2024-02-21 09:40:11 +01:00
Wolfgang WaltherandWolfgang Walther 944b02fbb8 docs: Rename Stored Procedures to Functions consistently
This avoids confusing our RPCs with actual CREATE PROCEDURE, which we don't
support.

References https://github.com/PostgREST/postgrest-docs/issues/147
2024-02-21 09:40:11 +01:00
Wolfgang WaltherandWolfgang Walther 95b8751496 docs: Simplify SQL for user management how-tos
All those DROP IF EXISTS and CREATE IF NOT EXISTS etc. just give a lot more text to
read and understand. If in fact a user creates the same thing twice, they should be
able to understand the error message from postgres.
2024-02-21 09:40:11 +01:00
Wolfgang WaltherandWolfgang Walther 9b1ff2235a docs: Simplify auth examples with OUT parameter
The jwt_token type was not created consistently in all examples, which can
be confusing when following those. To return an object with a single key
named token, it's enough to have an OUT parameter to the function.

Resolves https://github.com/PostgREST/postgrest-docs/issues/280
2024-02-21 09:40:11 +01:00
Steve ChavezandGitHub 7c6c056e92 refactor: make observation messages pure (#3250)
removes the observation messages from the Logger
2024-02-20 18:42:38 -05:00
Laurence IslaandGitHub 229bc77886 docs: using or across embedded resources 2024-02-20 17:15:42 -05:00
Wolfgang WaltherandWolfgang Walther 8a21a9c34e fix: Dump media handlers and timezones with --dump-schema
Those were left out of the schema dump when the features were introduced, probably
because ByteString doesn't have a toJSON instance. Changing the type to Text solves
this easily.

Resolves #3237
2024-02-20 18:44:17 +01:00
Steve ChavezandGitHub 6d506df6f3 refactor: add observation module (#3232) 2024-02-20 12:29:33 -05:00
Taimoor ZaeemandGitHub 32e1900370 feat: dump schema cache through admin API (#3233) 2024-02-19 21:20:29 -05:00
Wolfgang WaltherandWolfgang Walther fac7acafa3 chore: Add docs/_build to .gitignore again to allow switching to old release branches easily 2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 645217333b docs: Display version number more prominently
Resolves https://github.com/PostgREST/postgrest-docs/issues/734
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther fcd29caecf docs: Use code-block postgres consistently 2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 553ac79d9b docs: Add example on how to automatically store mimetype in files table
Resolves https://github.com/PostgREST/postgrest-docs/issues/612
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther b2ad914435 docs: Improve introduction for datatypes how-to
Not all examples on this page are strictly using string representation, especially bytea.
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 8ee4e9f0bb docs: Sort datatypes how-to alphabetically
The examples in this sections were in seemingly random order before.
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 8f2b6b1652 docs: Add hint about schema reloading with db-config=true
Resolves https://github.com/PostgREST/postgrest-docs/issues/490
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther b1deb049c8 docs: Clarify behavior of jwt-aud and tokens without aud claim
Resolves https://github.com/PostgREST/postgrest-docs/issues/479
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 51db5ce26f docs: Add example for ordering by json field
Resolves https://github.com/PostgREST/postgrest-docs/issues/364
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther e7dde2806c docs: Change tutorials to use identity column instead of serial/sequence
Resolves https://github.com/PostgREST/postgrest-docs/issues/311
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 4325a4f9d5 docs: Add links to resource embedding from insert/update/delete sections
Resolves https://github.com/PostgREST/postgrest-docs/issues/277
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 77715dcabf docs: Rename remaining instances of computed columns to computed fields
Resolves https://github.com/PostgREST/postgrest-docs/issues/368
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther aaaf0572ce docs: Remove deprecated auth0 features
As mentioned in https://github.com/PostgREST/postgrest/discussions/3088 rules and hooks are not
available to new tenants anymore, so the note is not helpful anymore.

Resolves #3088
Resolves https://github.com/PostgREST/postgrest-docs/issues/85
Resolves https://github.com/PostgREST/postgrest-docs/issues/715
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 500aad3360 docs: Clarify secret wording in tutorial 1
Resolves https://github.com/PostgREST/postgrest-docs/issues/108
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 3d95c41115 docs: Remove note about risk of asymmetric keys for JWT auth
The obviously wrong statement is, that PostgREST does not support asymmetric keys, while it
does. Extending on this type of attack is not necessary, because it is in fact covered by
the paragraph before - reading the algorithm from the JWT header is the problem in that case,
too. We don't do that.

This leaves us with the sentence about how the chosen library is the most important part. While
that is correct, the hint about high quality libraries for use on the *client* side is mis-
leading: The important part here is the library we choose to implement PostgREST with, not the
client-side lib. Thus, removing the whole paragraph is the best thing to do here.

Resolves https://github.com/PostgREST/postgrest-docs/issues/123
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 6eaaa9aa9b docs: Remove note about possible memory leak on alpine
We don't have any justification to add this hint anywhere.

Resolves https://github.com/PostgREST/postgrest-docs/issues/399
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther f33bccbcdb docs: Use port 5432 in tutorial 0 to avoid connecting to the wrong server
Some users connect PostgREST to the wrong PostgreSQL instance - likely because they are
not even aware that another instance is running. By using the standard port 5432 instead
of 5433, we avoid this problem. The user will be made aware very early that they have
another postgresql instance running - and can solve the problem at this stage. If they
decide to change the port, they are much more likely to remember that in the later stages
of the tutorial, too.

Resolves https://github.com/PostgREST/postgrest-docs/issues/304
2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 6c886b1291 docs: Update output in tutorials to latest version 2024-02-19 21:54:15 +01:00
Wolfgang WaltherandWolfgang Walther 0e7af7b77b docs: Various formatting improvements 2024-02-19 21:54:15 +01:00
Wolfgang Walther 4a796e0758 nix: Show connection hints for postgrest-with-postgresql-xx on stdout
This allows running this together with postgrest-run --dump-schema and
piping the result into jq without syntax errors.
2024-02-19 17:49:16 +01:00
Wolfgang Walther 8ff9f3292e test: Hide "cast on domain" warnings in spec tests
Also removes a few other unused settings which originate from running pg_dump.
2024-02-19 17:49:16 +01:00
Wolfgang WaltherandWolfgang Walther 5804b754e3 chore(deps): Remove useless pins in cabal.project.non-nix
Those were mistakenly added to support GHC 9.8.1, before I understood hackage revisions.
2024-02-19 17:47:37 +01:00
Wolfgang Walther 16bc853114 nix: Change default socket location for libpq back to /var/run/postgresql
Commit 85fbb233 accidentally changed the default socket location in which libpq is
looking for postgresql unix sockets. This is changed in nixpkgs via patch. By imp-
orting the default patches, this is changed back to what it was before. Without
those patches it was changed from /run/postgresql to /tmp.

Not a bugfix, because it was not released, yet.
2024-02-19 16:24:17 +01:00
Wolfgang Walther 272c1cc1e2 nix: Fix docs-spellcheck with multiple references on a single line 2024-02-18 19:35:40 +01:00
Wolfgang Walther 72de95b069 docs: Fix spelling mistakes 2024-02-18 19:35:40 +01:00
Wolfgang Walther c196d406d4 nix: Make postgrest-watch postgrest-docs-check work
This moves the _build folder into the repo root, to avoid postgrest-watch ending in an
infinite loop of restarting the build.

Also, for repeated use during development, running linkcheck is not a good idea, this
will quickly result in rate-limiting requests from various servers.
2024-02-18 16:12:20 +01:00
Wolfgang WaltherandWolfgang Walther b435f1b2d4 nix: Move docs build and serve scripts into nix
This is long overdue. We expect everyone contributing to postgrest to use the nix tools,
so it makes no sense to carry around external tools anymore.
2024-02-18 13:10:00 +01:00
Wolfgang WaltherandWolfgang Walther 3750f35e3f chore: Deduplicate static files between core and docs 2024-02-18 13:10:00 +01:00
Wolfgang WaltherandWolfgang Walther c2a333efc4 chore: Bump version docs to 12.1-dev
See c004840e.
2024-02-18 13:10:00 +01:00
Laurence IslaandWolfgang Walther a1f2ecadda nix: Move docs tools into core infrastructure 2024-02-18 13:10:00 +01:00
Wolfgang WaltherandWolfgang Walther e110fdbd2c nix: Refactor checkedShellScript's inRootDir to workingDir
This allows more flexible control over the working directory. Values for workingDir must always start
with a / and will then be relative to the repo root.
2024-02-18 13:10:00 +01:00
Wolfgang Walther 75a86ba873 ci: Change names for release branches from rel-MAJOR.MINOR to vMAJOR
This naming scheme gives us the best support for readthedocs.

References #2814
2024-02-17 17:43:03 +01:00
Wolfgang Walther b2dda6a5d3 ci: Refactor conditions to check for branch events 2024-02-17 17:38:39 +01:00
Wolfgang Walther 9a145f6931 Merge branch 'docs/main' into main 2024-02-17 15:14:44 +01:00
Wolfgang Walther 311a38193b chore: Run postgrest-style on docs files
This prevents CI from failing after the merge.
2024-02-17 13:43:50 +01:00
Wolfgang Walther 4954859d68 chore: Prepare merge of postgrest-docs into postgrest main repo
This avoids some merge conflicts to allow git blame to detect renames properly.
2024-02-17 13:43:50 +01:00
Wolfgang Walther a7afcc4de9 chore: Remove accidentally committed submodule
This was accidentally added in b22bb74.
2024-02-17 13:13:00 +01:00
Wolfgang Walther 1b855d805e docs: Replace some permanent redirections with their target
And sort the ecosystemi's _devops list again.
2024-02-16 22:36:57 +01:00
Laurence IslaandWolfgang Walther 4059e1ea76 Add tutorial for Godot 4 + PostgREST in ecosystem 2024-02-16 22:14:05 +01:00
Laurence IslaandWolfgang Walther 93e50b1d90 docs: Enable non-broken link again 2024-02-16 22:11:00 +01:00
Wolfgang WaltherandWolfgang Walther 2ae96e7733 docs: Fix broken stackoverflow link 2024-02-16 22:04:07 +01:00
Wolfgang WaltherandWolfgang Walther 9167ae9462 chore: Remove leftover .test file 2024-02-16 22:04:07 +01:00
Wolfgang WaltherandWolfgang Walther 3664343989 chore: Add build tools for translations
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2024-02-16 22:04:07 +01:00
Wolfgang WaltherandWolfgang Walther 18d105f8c2 nix: Update nixpkgs to same version as core repo 2024-02-16 22:04:07 +01:00
Wolfgang Walther c004840e76 Bump to v12.1
Going forward, an uneven minor version will be a development version, while an even minor will be
considered a stable version to be released. This is similar to what GHC does and was discussed in
#3113.

This bump should have happened after branching off v12.0.0, but obviously we didn't know about it
back then. This will happen immediately after branching off a new release from now on.
2024-02-16 16:38:32 +01:00
Wolfgang Walther 00fbe9ff3e fix: Avoid casting to table type when select= and media type handler are used
Previously using a generic mimetype handler failed when any kind of select= was given, because
we tried to cast the select-result to the original table type. With this change, this cast is
only applied when select=* is given implicitly or explicitly. This is the only case where this
makes sense, because this guarantees that correct columns are selected in the correct order for
this cast to succeed.

Resolves #3160
2024-02-15 19:01:12 +01:00
Wolfgang Walther 2466f4e738 fix: Return 406 instead of 415 for non-acceptable media type
415 is for Content-Type and 406 for Accept headers.
2024-02-15 19:01:12 +01:00
Andrei DziahelandGitHub 3432f75ed4 fix: fixes server timings' precision (#3227)
* test: fix test_io accordingly
2024-02-14 12:26:34 -05:00
Wolfgang Walther b22bb748f2 nix: Remove left-over commented out code in postgrest-test-doctests
This seems to be commented out for a while already, but nobody reported
problems, so far.
2024-02-11 11:38:53 +01:00
steve-chavez 21e8ca5051 feat: log full pg version to stderr on connection 2024-02-10 19:21:55 -05:00
Wolfgang WaltherandWolfgang Walther 410fa9508b nix: Make postgrest-with-postgresql-xxx postgrest-test-io work better
The upside is that postgrest-with-postgresql-xxx postgrest-test-io works as expected
now. The downside is, that postgrest-with-postgresql-xxx psql now starts without
any schema. This now needs an explicit postgrest-with-postgresql-xxx -f path/to.sql
to do anything useful.

Resolves #2864
2024-02-10 20:51:55 +01:00
Wolfgang Walther dcfdd8dfc9 ci: Only try to download loadtest artifact after conclusion of CI workflow
Removing the condition when migrating the loadtest workflow was not helpufl, this
triggers the report job a few times per pipeline. The goal was to always download
the report, even when the overall pipeline fails because of some other jobs.
Explicitly checking for both success and failure should be enough.
2024-02-10 20:42:06 +01:00
Wolfgang Walther ca777315b0 ci: Consistently put two blank lines between jobs for better visual separation 2024-02-10 20:16:22 +01:00
Wolfgang Walther f513a71c7a ci: Use latest stable nix version in CI
This was pinned to 2.13.6 in https://github.com/PostgREST/postgrest/pull/2692#issuecomment-1448938894
because of a regression in nix 2.14. Latest is 2.16.x now, so maybe this bug is already fixed.
2024-02-10 20:16:22 +01:00
Wolfgang Walther 09c5cbece5 ci: Remove workaround for ghcup on github actions runner images
This was introduced in https://github.com/PostgREST/postgrest/pull/2655.
2024-02-10 20:16:22 +01:00
Wolfgang Walther 073320133d ci: Remove matrix configuration for Build-Cabal-Arm job
This avoids the "${{ matrix.ghc }}" display when the job is skipped in PR workflows.
2024-02-10 20:16:22 +01:00
Wolfgang Walther f17f23bd9f ci: Make stack build fail when lock file is out of date
When updating stack.yaml, we need to make sure to update stack.yaml.lock, too.

This check prevents them from getting out of sync by failing CI in this case.

This improves cachability.
2024-02-10 20:16:22 +01:00
Wolfgang Walther fd1efa4635 ci: Merge CI and Loadtest workflows
The reason why those workflows were split in the first place was just to obtain loadtest
results quicker, because the in the separated workflow, only the single loadtest job
needs to finish before the artifacts can be downloaded.

However, the disadvantage of this approach was, that the results were not as easily
accessible as they could be in a single workflow. Additionally, it's possible to depend
on the "prepopulate nix" job for efficiency if the loadtest runs in the main workflow.
2024-02-10 20:15:35 +01:00
Wolfgang Walther db16683ba2 ci: Simplify loadtest workflow
The two different PR and Merge jobs were introduced to be able to test the main branch
against the latest release. However, this is now included in the PR job, too, so no need
for the two separate jobs anymore.
2024-02-10 20:15:09 +01:00
Wolfgang Walther 8a3b0c60ad ci: Remove nix actions cache and prepopulate job
The nix actions cache currently leads to repeated "no space left on
devices" errors for jobs in CI.

The prepopulate job is useless without the nix actions cache, so it
will go away at the same time.
2024-02-10 12:35:56 +01:00
steve-chavez 5424be76d9 feat: log schema cache stats to stderr
adjust the memory tests
2024-02-09 16:18:04 -05:00
Wolfgang Walther 066fa8b6aa ci: Improve cache keys and their restore prefixes
The cache key for nix now depends on default.nix and shell.nix in the root folder and all
.patch files in the nix folder. Those may change the output of our nix derivations, so
must be included. At the same time, there is no reason to include the actions/setup-nix
folder. This would only lead to new caches being created every time we update one of the
dependent actions in this file. Finally, we never restore caches with a different id any-
more. There is no point in having the style job fall back to the static cache for example.

The cache keys for cabal can be more explicit: We only have one postgrest.cabal and one
relevant cabal.project file. We were missing the cabal.project.freeze file, though, which
affects the dependencies used, too.
2024-02-09 18:43:53 +01:00
Taimoor ZaeemandSteve Chavez f9ee1f7e73 feat: apply all function settings as transaction-scoped settings 2024-02-09 11:37:01 -05:00
Laurence IslaandGitHub 4a0b93c451 Fix technical_specs in sample DB using UNIQUE instead of PRIMARY KEY 2024-02-08 14:10:17 -05:00
Wolfgang Walther 49584728b6 nix: Remove broken docker-based nix development environment
It used to be possible to spin up a nix environment via docker container this way,
but the upstream nixos/nix image has changed and the docker build doesn't succeed
anymore. Since nobody complained about that, we can assume it is not being used
anyway.
2024-02-06 18:27:20 +01:00
Wolfgang Walther 868913d179 ci: Replace seed cachix workflow with new daemon mode
cachix-action v14 added a new daemon mode, which pushes new derivations to the store
as soon as they have been built. This replaces the seed cachix workflow nicely by just
pushing from all jobs directly.
2024-02-06 10:20:02 +01:00
Wolfgang Walther 74d8ddec36 nix: Make postgrest-push-cachix more efficient by passing each derivation only once 2024-02-06 10:20:02 +01:00
Wolfgang WaltherandWolfgang Walther ea02729ebd nix: Use -split-sections with pkgsStatic to make static executable smaller again
This works around https://github.com/NixOS/nixpkgs/issues/286285 to use -split-sections
in a cross-compiling scenario. This will reduce the size of the static executable and also
remove the remaining references to /nix/store/.. reducing closure size dramatically.

This also fixes the docker image blowing up in size since we switched to pkgsStatic.
2024-02-06 10:19:31 +01:00
Wolfgang WaltherandWolfgang Walther 9fe0f50709 ci: Add freebsd executable to releases
This was temporarily disabled, because of timeouts in Cirrus. This seems to work well again.
2024-02-06 10:19:31 +01:00
Wolfgang WaltherandWolfgang Walther 1c02e1e3a0 ci: Prevent creating artifact for dynamic ubuntu build
This is not used in the release process anyway, because we are using the static build.
2024-02-06 10:19:31 +01:00
Wolfgang WaltherandWolfgang Walther 7d096e5de5 ci: Update stack.yaml.lock 2024-02-06 10:19:31 +01:00
Wolfgang WaltherandWolfgang Walther 0eca2954b3 ci: Reduce file size of cabal and stack based builds
By passing -split-sections to all dependencies, GHC will link only the
modules we actually use and not the full package for each dependency.
This does neither work on MacOS nor Windows, thus we don't do it for
stack right now.

Stripping unused symbols in CI will further decrease the size of those files.
2024-02-06 10:19:31 +01:00
Wolfgang WaltherandWolfgang Walther cc9b725005 nix: Remove libkrb5 references from static executable 2024-02-06 10:19:31 +01:00
Wolfgang WaltherandWolfgang Walther 4a1b9a4609 nix: Remove libpq reference from static executable 2024-02-06 10:19:31 +01:00
Wolfgang WaltherandWolfgang Walther f51090f3d1 nix: Check static executable for /nix/store references
This makes the static build fail in case any references to the nix store
are left over. Those will increase the closure size of the nix derivation
massively and lead to a huge docker image.

At the same time, those references will not be functional on non-nix systems,
to which the static executable is distributed, anyway.
2024-02-06 10:19:31 +01:00
Ivan KasatenkoandGitHub 71887e4b78 feat: dump config through Admin API 2024-02-05 19:16:05 -05:00
Andrei DziahelandWolfgang Walther a45058ba02 ci: reuse more previous caches (Nix & Stack) 2024-02-05 21:51:32 +01:00
Wolfgang Walther 310d04065e test: Make failing loadtest fixtures pass again
PR #2358 added a bulk insert to the loadtest. However this broke the regular insert test,
which just returned 400 Bad Request because of a missing PK value since. Adding the new
id column in the payload to the ?columns= argument fixes that.
2024-02-05 21:40:21 +01:00
Wolfgang Walther 63ff6d1f15 nix: Fix postgrest-loadtest after vegeta update
The recent nixpkgs update gave us a new version of vegeta. This version includes a new
DNS cache features - which unfortunately doesn't play well with unix sockets. Disabling
the DNS cache makes requests succeed again.
2024-02-05 21:38:23 +01:00
steve-chavez 45cabacdcc fix: wrong subquery error returning as 400 status 2024-02-02 22:08:56 -05:00
dependabot[bot]andGitHub bbc0bda6f5 build(deps): bump LouisBrunner/checks-action from 1.6.2 to 2.0.0 (#3195) 2024-01-29 09:35:22 -05:00
dependabot[bot]andGitHub c1b46ffa1b build(deps): bump codecov/codecov-action from 3.1.4 to 3.1.5 (#3196) 2024-01-29 09:34:33 -05:00
Laurence IslaandGitHub 439db880dd Simplify htmx how-to functions using Pico CSS and Ionicons 2024-01-26 20:28:58 -05:00
Wolfgang WaltherandWolfgang Walther 43f552dbab ci: Improve caching for static and dynamic postgrest nix packages 2024-01-26 23:24:12 +01:00
Wolfgang WaltherandWolfgang Walther c43696c9a6 nix: Move postgrest-check-static inside derivation of static package
This will make the static build fail if we're not producing a static binary.
2024-01-26 23:24:12 +01:00
Andrei DziahelandWolfgang Walther 411cf430ad ci: revert cache-nix-action to v4
Upgrading c-n-action seems to break restoring Nix store rendering the cache useless (nix-community/cache-nix-action#27). This rolls the change back.
2024-01-26 18:40:45 +01:00
Wolfgang Walther 071a3d436f ci: Fix Seed-Cachix jobs on linux 2024-01-26 18:10:38 +01:00
Wolfgang WaltherandWolfgang Walther c94aa9ccd9 fix: Build static postgrest with GSSAPI support
Resolves #2815
2024-01-26 17:17:05 +01:00
Wolfgang WaltherandWolfgang Walther dcb6c5bb13 nix: Bump libpq version to link against v16 2024-01-26 17:17:05 +01:00
Wolfgang WaltherandWolfgang Walther 85fbb233a7 nix: Build only libpq instead of full postgresql package 2024-01-26 17:17:05 +01:00
Wolfgang WaltherandWolfgang Walther 259d97acee nix: Build static postgrest executable via pkgsStatic
This replaces the build via static-haskell-nix and will hopefully make
it possible to cross compile in the future.
2024-01-26 17:17:05 +01:00
Wolfgang WaltherandWolfgang Walther 3a639c7172 ci: Fix FreeBSD build on cirrus 2024-01-25 21:27:17 +01:00
Wolfgang WaltherandWolfgang Walther d75be243ac ci: Build with GHC 9.8.1 via Cabal 2024-01-25 21:27:17 +01:00
Wolfgang WaltherandWolfgang Walther 2ffb97991d ci: Build with GHC 9.6.4 via Cabal
This confirms that building with GHC 9.6.x works already, even if we can't
switch to it for the nix and stack builds, yet.
2024-01-25 21:27:17 +01:00
Wolfgang WaltherandWolfgang Walther 2e298824d6 deps: Replace postgresql-libpq fork with upstream v0.10
Our changes to reduce memory usage have been merged and released
upstream, so no need for the fork anymore.
2024-01-25 21:27:17 +01:00
steve-chavez 76b1c00935 docs: fix link to op modifiers 2024-01-22 23:49:59 -05:00
Andrei DziahelandWolfgang Walther d7246b4851 ci: nix-store --realise to tools:
Since we're building only tools now, we can just specify them directly
in the `tools:` parameter of `setup-nix` action.
2024-01-22 18:17:38 +01:00
Andrei DziahelandWolfgang Walther 286103daf2 ci: do not rebuild postgrest
specify outputs to cache excluding postgrest
2024-01-22 18:17:38 +01:00
Andrei DziahelandSteve Chavez 9c78751871 ci: fix c-n-a input names 2024-01-22 11:28:41 -05:00
dependabot[bot]andSteve Chavez 820917bcb6 build(deps): bump nix-community/cache-nix-action
Bumps [nix-community/cache-nix-action](https://github.com/nix-community/cache-nix-action) from 4.0.3 to 5.0.1.
- [Release notes](https://github.com/nix-community/cache-nix-action/releases)
- [Changelog](https://github.com/nix-community/cache-nix-action/blob/main/RELEASES.md)
- [Commits](https://github.com/nix-community/cache-nix-action/compare/v4.0.3...v5.0.1)

---
updated-dependencies:
- dependency-name: nix-community/cache-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-22 11:28:41 -05:00
dependabot[bot]andGitHub 02312804f0 build(deps): bump actions/cache from 3 to 4 (#3179) 2024-01-22 09:13:30 -05:00
Steve ChavezandGitHub 7a387c7a30 nix: add parallel-curl wrapper (#3172)
stock curl doesn't make it easy to make many parallel requests to a
single endpoint. The endpoint has to be repeated N times.

```
curl --parallel https://example.com https://example.com ..
```

This provides a wrapper for parallel curl. It's useful for testing
scenarios like pool timeouts.

```
parallel-curl N https://example.com
```
2024-01-18 20:03:01 -05:00
Wolfgang WaltherandWolfgang Walther 6682baebd2 nix: Update nixpkgs and build with GHC 9.4.8 2024-01-18 18:54:33 +01:00
Wolfgang WaltherandWolfgang Walther e4c60889ee nix: Make postgrest-nixpkgs-upgrade take the latest stable version
Previously this command upgraded to the latest unstable version of nixpkgs,
but this was often broken. Taking the latest stable branch should give
better results.
2024-01-18 18:54:33 +01:00
Steve ChavezandGitHub c070cb9502 Revert "build(deps): bump nix-community/cache-nix-action from 4.0.3 to 5.0.1 in /.github/actions/setup-nix" (#3167) 2024-01-17 17:55:38 -05:00
dependabot[bot]andGitHub a8165a89a0 build(deps): bump nix-community/cache-nix-action (#3135) 2024-01-17 17:25:10 -05:00
Laurence IslaandGitHub 9ef0e677e0 Use -g to prevent globbing in some curl examples 2024-01-17 12:39:42 -05:00
Laurence IslaandGitHub cb5d80aff8 Remove HTTP Snippets 2024-01-17 09:07:37 -05:00
Laurence IslaandGitHub be2778edb5 Replace the term 'password' with 'secret' to clarify the Tutorial 1 2024-01-17 09:06:20 -05:00
ParashoeandLaurence Isla 0e7d507e5e Spelling correction (#740)
Spell correct "callounter" to "callcounter"
2024-01-15 20:22:42 -05:00
Laurence IslaandGitHub 74a3217bee Update sponsor logo 2024-01-15 20:17:55 -05:00
Laurence IslaandGitHub e063a29212 Fix Sphinx version error in RTD (#739) 2024-01-15 20:09:54 -05:00
Laurence IslaandGitHub f5fca59b2e chore: update sponsor logo (#3161) 2024-01-15 20:09:42 -05:00
dependabot[bot]andGitHub 740fcdf5ab build(deps): bump cachix/cachix-action in /.github/actions/setup-nix (#3159) 2024-01-15 14:54:44 -05:00
dependabot[bot]andGitHub f864437055 Bump cachix/install-nix-action from 24 to 25 (#737) 2024-01-15 14:38:23 -05:00
steve-chavez 4bb98225d8 docs: remove Heroku integration 2024-01-10 17:23:13 -05:00
steve-chavez fdaeea8fd2 fix: misleading "Starting.." logs on scache reload 2024-01-09 20:37:25 -05:00
steve-chavez 6c3d7a946d deprecate: params=single-object preference 2024-01-04 17:00:38 -05:00
steve-chavez c29d876f90 docs: add params preference 2024-01-04 16:09:07 -05:00
steve-chavez 1e4efafd9e docs: add tx preference 2024-01-04 16:09:07 -05:00
steve-chavez 50ac91d1ae docs: testing of media handlers as a note
Otherwise users think they're required steps. See the opening comment on
https://github.com/PostgREST/postgrest/issues/3124.
2024-01-02 23:44:53 -05:00
Taimoor ZaeemandSteve Chavez f73845159c add documentation for max-affected preference 2023-12-30 23:07:45 -05:00
Laurence IslaandGitHub 0ce37f8fd6 Update sponsor image (#728) 2023-12-20 19:16:48 -05:00
Laurence IslaandGitHub b2095a8ef6 chore: update sponsor image 2023-12-20 19:16:31 -05:00
Laurence Isla d99e3ac805 changelog: update to 12.0.2 2023-12-20 18:26:05 -05:00
steve-chavez 3af63cb94c fix: empty row on handler function
Closes https://github.com/PostgREST/postgrest/issues/3126
2023-12-20 16:18:09 -05:00
steve-chavez 1680a6eed3 fix: aggregates not working for all schemas
Closes https://github.com/PostgREST/postgrest/issues/3124
2023-12-20 15:20:18 -05:00
Laurence IslaandSteve Chavez c97bd1f01b Add warning and fix to htmx how-to 2023-12-19 13:30:06 -05:00
steve-chavez 9e04bd7da6 docs: soap how-to is outdated
`*/*` can now be handled with media type handlers
2023-12-18 23:58:17 -05:00
Taimoor ZaeemandGitHub 0b77098460 feat: add max-affected preference to prefer header (#3083) 2023-12-18 18:35:23 -05:00
Laurence IslaandGitHub 3ec6526467 Add new sponsor 2023-12-18 12:14:52 -05:00
Laurence IslaandGitHub 4a1dee453b chore: add new sponsor 2023-12-18 12:14:01 -05:00
dependabot[bot]andGitHub 93bdae962e build(deps): bump actions/upload-artifact from 3 to 4 (#3123) 2023-12-18 09:09:18 -05:00
dependabot[bot]andGitHub 6a34018cfc build(deps): bump dawidd6/action-download-artifact from 2 to 3 (#3122) 2023-12-18 09:09:10 -05:00
dependabot[bot]andGitHub 2aa5373a27 build(deps): bump actions/download-artifact from 3 to 4 (#3121) 2023-12-18 09:09:02 -05:00
Laurence Isla f95a8e77a6 changelog: update to 12.0.1 2023-12-14 12:10:08 -05:00
steve-chavez a59cfafe10 fix: correct any media type handler 2023-12-12 17:45:14 -05:00
steve-chavez 6b9fde59cd fix: any handler sets a default application/json
Now it sets application/octet-stream as the generic type.
2023-12-12 17:44:07 -05:00
Laurence IslaandSteve Chavez 1e65a72264 Add missing in-db configuration for jwt-cache-max-lifetime 2023-12-12 16:36:24 -05:00
Laurence IslaandGitHub 4fb521cac6 fix: add jwt_cache_max_lifetime as an in-database configuration option (#3102) 2023-12-12 16:29:50 -05:00
steve-chavez cf258ef499 fix: add missing pgrst.server_timing_enabled 2023-12-12 10:58:22 -05:00
Laurence IslaandGitHub 0938e72c5c Add new sponsor (#723) 2023-12-12 10:39:05 -05:00
Laurence IslaandSteve Chavez e31363a325 chore: add new sponsor 2023-12-12 10:38:09 -05:00
Laurence IslaandGitHub 5bd7cda308 Priorize query parameters instead of headers in limits and pagination 2023-12-07 18:47:56 -05:00
Andrei DziahelandGitHub cbfff2804d fix: replace json parser error with generic msg (#3090) 2023-12-07 18:04:51 -05:00
Andrei DziahelandSteve Chavez 41ea4e6db6 ci: cancel in-progress loadtests for PRs as well 2023-12-07 16:01:32 -05:00
Andrei DziahelandSteve Chavez f9294c5e43 ci: test only head of branch 2023-12-07 16:01:32 -05:00
Laurence IslaandGitHub d24a3c82ef Add meta tags using an Open Graph extension 2023-12-07 15:52:57 -05:00
Laurence IslaandGitHub c6b71551f4 changelog: move missplaced unreleased fix (#3094) 2023-12-07 11:00:46 -05:00
Laurence IslaandGitHub d49e3d7132 fix: allow using special characters in json keys (#3081)
* increase memory size test
2023-12-06 17:28:51 -05:00
Laurence IslaandSteve Chavez f80a33cf32 Add missing curl requests in Preferences section 2023-12-06 17:22:31 -05:00
steve-chavez 8fd4fa0d09 fix: any media type should be bytea 2023-12-05 17:48:16 -05:00
Andrei DziahelandGitHub 72083361bc ci: similar jobs use a single cache for Nix stores
Jobs that use the setup-nix action will share a single cache, which is created by a previous warm up Job.
2023-12-05 13:57:18 -05:00
Laurence IslaandGitHub 48dee75d1a Separate Admin into Observability and Health Check (#716) 2023-12-04 16:32:02 -05:00
dependabot[bot]andGitHub 92573215b9 Bump cachix/install-nix-action from 23 to 24 (#714) 2023-12-04 16:31:07 -05:00
dependabot[bot]andGitHub 6277eb43f7 build(deps): bump cachix/cachix-action in /.github/actions/setup-nix (#3085) 2023-12-04 15:39:40 -05:00
steve-chavez b04a7f0080 fix server-timing header section
Also link it to jwt caching
2023-12-02 01:41:26 -05:00
steve-chavez 0c65bf494d changelog: update to 12.0.0 2023-12-01 19:37:11 -05:00
steve-chavez 70c0d88fa7 bring back schema isolation on its own page 2023-12-01 19:03:23 -05:00
steve-chavez 2b4e0cc10a move return representation to preferences page 2023-12-01 18:32:38 -05:00
steve-chavez 1268b0a258 split pagination/count into own page 2023-12-01 16:24:52 -05:00
steve-chavez 11f40c384a use the clearer START TRANSACTION 2023-12-01 16:24:52 -05:00
Tim AbdullaandGitHub 2bb9cee996 Add documentation for aggregate functions (#701) 2023-12-01 11:38:42 -05:00
steve-chavez a38eee399e chore: improve mt handlers snippets 2023-12-01 10:59:48 -05:00
steve-chavez ee4321623f chore: improve intro of media type handlers 2023-12-01 10:51:46 -05:00
steve-chavez 49903ca6db chore: wording of media type handlers 2023-12-01 10:08:12 -05:00
steve-chavez d15e357d2e reference: media type handlers 2023-11-30 23:27:06 -05:00
Laurence IslaandGitHub 62512268cd Add PostgREST installation using package managers to tut0 2023-11-30 18:20:54 -05:00
Laurence IslaandGitHub 84152b483a Remove db-use-legacy-gucs 2023-11-29 15:37:25 -05:00
steve-chavez 9fe11249f9 feat: custom SQL handler for the "*/*" media type 2023-11-28 23:14:38 -05:00
Laurence IslaandGitHub 7640de34e2 refactor: use a data type instead of Map for Server Timing 2023-11-28 18:26:56 -05:00
Andrei DziahelandSteve Chavez 31ce39ba36 ci: cabal+GHC: tweak caching
Makes it to cache only relevant directories, adds `dist-newstyle` to prevent needless rebuilding even harder and shortens cache key name by supplying `hashFiles`multiple arguments
2023-11-28 12:48:57 -05:00
Laurence IslaandGitHub ca5eb64deb chore: fix server-timing metric order and doctest 2023-11-28 12:45:39 -05:00
Andrei DziahelandSteve Chavez a72241ad4e ci: fixes job caches not producing after recent cabal upgrade
Recent cabal adopted XDG guidelines and stores data
across multiple directories under $HOME.
Creating ~/.cabal manually returns old behavior and allows caching single directory again.
2023-11-28 11:52:52 -05:00
steve-chavez b080f59bac test: server timing on root and options method 2023-11-28 10:25:25 -05:00
steve-chavez 558e9d40e8 changelog: join server-timing entries 2023-11-28 10:25:25 -05:00
steve-chavez 958339b8d3 chore: change server timing render to response
"render" is a loaded term than might be thought as the generation
of a full HTML page. While for this phase we only process the status
and the headers.

Changing it to "response" so is not misleading at least. Users can check the
docs for clarification.
2023-11-28 10:25:25 -05:00
steve-chavez 3d56f8435d chore: delete dead error codes 2023-11-28 10:25:25 -05:00
steve-chavez e620ae5e9c pin sphinx-tabs ver 2023-11-28 07:46:13 -05:00
steve-chavez bd6d8624ab fix: sphinx breaking change
see https://github.com/sphinx-doc/sphinx/issues/10474
2023-11-28 07:46:13 -05:00
steve-chavez 03e384d55b remove release notes 2023-11-28 07:46:13 -05:00
Laurence IslaandSteve Chavez dfa875c8c7 fix: do not log internal db errors like 'acquisition timeout' when log-level=crit 2023-11-27 23:06:40 -05:00
Laurence IslaandSteve Chavez 33891e3a73 feat: log all internal database errors to stderr 2023-11-27 23:06:40 -05:00
Laurence IslaandSteve Chavez 8483459d59 chore: add missing --example for server-timing-enabled 2023-11-24 17:41:16 -05:00
Andrei DziahelandGitHub 7b5c2f03fc introducing server-timing-header config parameter (#700) 2023-11-24 16:50:28 -05:00
Laurence IslaandSteve Chavez b538ab9823 feat: add timing for the api request parse 2023-11-24 16:29:25 -05:00
Laurence IslaandSteve Chavez 850b15fe13 fix: change timing name from 'query' to 'transaction' 2023-11-24 16:29:25 -05:00
Andrei DziahelandSteve Chavez 9eda78eda5 server-port can be 0
Introduces special value of 0 for port to be assigned randomy
2023-11-23 22:35:28 -05:00
Andrei DziahelandGitHub df97a5071f Display an actual TCP port app is bound to (#3034) 2023-11-23 18:17:16 -05:00
Tim AbdullaandGitHub 1c60b50e2e Add aggregate functions (#2925)
The aggregate functions SUM(), MAX(), MIN(), AVG(), and COUNT() are now supported.
2023-11-23 14:03:03 -05:00
Andrei DziahelandGitHub c3301a1653 feat: implement server-timing-enabled config parameter (#3064) 2023-11-22 17:57:50 -05:00
Andrei DziahelandGitHub abcb21c69a ci: cache Nix store with cache-nix-action (#2992)
* ci: try nix-community/cache-nix-action

* ci: add cache-id param to setup-nix action

* ci: tidy up cache keys for non-nix jobs

* ci: merge-nix-caches-linux job

* ci: merge caches other way around

* ci: reduce number of caches

Should prevent disk overflow

* ci: change cache id prefix for merge

* ci: comment out cache merging job

* ci: revert cache id prefix

* ci: setup-nix: use latest cache-nix-action

Among others, makes action logs look more tidy (see https://github.com/nix-community/cache-nix-action/commit/17d19d3d8be918757589635bc1a6830be0b129d2)

* ci: use test-pg cache key for loadtest
2023-11-21 18:56:23 -05:00
steve-chavez 475b4601ca split role settings from function settings
Also add doc for GRANT SET ON PARAMETER, see
https://github.com/PostgREST/postgrest/pull/3058.
2023-11-21 17:49:35 -05:00
Taimoor ZaeemandSteve Chavez 49e3d2c4b4 add documentation for statement_timeout set on functions 2023-11-21 17:49:35 -05:00
steve-chavez f7bf2157f3 feat: apply super settings on impersonated roles
If they have GRANT SET ON PARAMETER <setting> TO authenticator
2023-11-21 10:54:56 -05:00
Taimoor ZaeemandGitHub 125f10a60f feat: add statement_timeout set on functions (#3056) 2023-11-17 12:36:51 -05:00
Taimoor ZaeemandGitHub 5502dce556 add documentation for timezone preference (#698) 2023-11-15 08:24:25 -05:00
Anwar KnyaneandGitHub 084e8c2950 make the jwt generation easier :) (#697) 2023-11-14 11:31:33 -05:00
Taimoor ZaeemandGitHub 3c1a7f2641 feat: add timezone in Prefer header (#3024)
* increase reloading timeout in io-tests
* increase memory test by 1M
2023-11-13 14:16:27 -05:00
omahsandGitHub f10d4139fe nix: fix typos in README 2023-11-08 11:43:42 -05:00
Laurence IslaandGitHub 7e7e494b31 Remove former sponsors (#694) 2023-11-07 17:24:46 -05:00
Laurence IslaandSteve Chavez 99b705d44f chore: remove former sponsors 2023-11-07 16:25:47 -05:00
Laurence IslaandGitHub e0586c3afa add Neon as new sponsor (#693) 2023-11-06 22:49:53 -05:00
Laurence IslaandGitHub 379d9d6298 chore: add Neon as sponsor 2023-11-06 20:02:58 -05:00
Laurence IslaandSteve Chavez 2c35257eec chore: add prefix info to the PR template 2023-11-03 19:42:06 -05:00
steve-chavez ed90774f53 test: query the empty string 2023-11-03 12:58:54 -05:00
Laurence Isla 96ca177c31 transaction-scoped settings are now shown clearly in the postgres logs 2023-11-02 15:50:22 -05:00
Laurence Isla 226400a5bc break:remove the db-use-legacy-gucs config
BREAKING CHANGE

All PostgreSQL versions will use JSON GUCs for headers, cookies and JWT claims.
2023-11-02 15:50:22 -05:00
Taimoor ZaeemandGitHub 57ceabb260 add documentation for server-cors-allowed-origins config (#688) 2023-10-30 11:08:12 -05:00
b85ee953b7 doc: media type handlers (#689)
* reference: media type handlers

* Use function instead of computed field in editable task

---------

Co-authored-by: Laurence Isla <lau.isla.c@gmail.com>
2023-10-28 12:00:37 -05:00
Laurence IslaandGitHub b235227119 allow all origins when server-cors-allowed-origins is an empty string 2023-10-27 21:30:56 -05:00
Taimoor ZaeemandGitHub 5c9c7f4ff4 fix: HTTP status responses for upserts
* PUT returns 201 instead of 200 when rows are inserted
* POST with "Prefer: resolution=merge-duplicates" returns 200 instead of 201 when no rows are inserted
2023-10-26 20:49:20 -05:00
steve-chavez 82b38341bf feat: sql handlers for custom media types
* test text/html and drop HtmlRawOutputSpec.hs
* all tests passing, removed all pendingWith
* make functions compatible with pg <= 12
* move custom media types tests to own spec
* anyelement aggregate
* apply aggregates without a final function
* overriding works
* overriding anyelement with particular agg
* cannot override vendored media types
* plan spec works with custom aggregate
* renamed media types to make clear which ones are overridable
* correct content negotiation with same weight
* text/tab-separated-values media type
* text/csv with BOM plus content-disposition header
2023-10-26 09:04:02 -05:00
steve-chavez 4a90e9fbd9 break: db-root-spec default app/openapi+json media
BREAKING CHANGE

Can be done later with custom media types
2023-10-26 09:04:02 -05:00
steve-chavez 14d030b96c break: remove binary field logic, raw-media-types
BREAKING CHANGE

Can be done later with custom media types
2023-10-26 09:04:02 -05:00
Wolfgang Walther 6920a88dc0 nix: Fix postgrest-release on rel-branches pushing to the right branch 2023-10-25 19:09:21 +02:00
Wolfgang Walther 966f8df33f changelog: update to 11.2.2 2023-10-25 19:08:04 +02:00
Wolfgang Walther 98bd1de1cd nix: Allow running postgrest-release on rel- branches 2023-10-25 16:29:03 +02:00
Taimoor ZaeemandGitHub d94286c185 feat: add config to specify CORS origins (#2986) 2023-10-24 09:38:25 -05:00
Taimoor ZaeemandGitHub 618f93dec1 test: fix typos in batch upsert tests 2023-10-21 16:29:43 -05:00
Laurence IslaandGitHub 317619bf62 fix: regression by reverting fix that returned 206 when first-pos=length in Range header 2023-10-21 16:28:08 -05:00
steve-chavez eb238ad678 test: DRY scache/config sleep in io tests
Also increase schema cache sleep to 0.2 as 0.1 tends to fail in CI.
2023-10-20 17:53:54 -05:00
steve-chavez 54786a6c04 fix: unnecessary count() on RPC returning single 2023-10-19 15:54:22 -05:00
steve-chavez 00f3cb3746 test: prove RPC returning single always gives 1 2023-10-19 15:54:22 -05:00
Andrei DziahelandGitHub e977032847 build: add cabal.project.freeze (#3004) 2023-10-19 09:26:46 -05:00
dependabot[bot]andSteve Chavez 5c314e3f02 Bump urllib3 from 2.0.6 to 2.0.7
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.0.6...2.0.7)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-18 13:09:04 -05:00
Taimoor ZaeemandGitHub f10b4c3268 test: add tests for batch upserts (#3005) 2023-10-13 09:47:26 -03:00
Andrei DziahelandGitHub dc01c748ae feat: add more perf counters to Server-Timing (#2983) 2023-10-12 09:55:30 -03:00
Taimoor ZaeemandGitHub bae8dc3283 add documentation for JWT caching (#683) 2023-10-10 23:22:41 -03:00
Laurence IslaandGitHub 056c748c5f refactor: DRY and enforce error format 2023-10-10 11:42:36 -05:00
Steve ChavezandGitHub 93f2b0093c references: remark NOLOGIN impersonated roles 2023-10-07 19:20:48 -03:00
Taimoor ZaeemandGitHub 5ba370fe0a add documentation for handling preference (#684) 2023-10-07 00:23:41 -03:00
Kam Ting HoiandGitHub 818387f24e fix: range request with 0 rows and 0 offset return status 416 (#2991) 2023-10-07 00:10:17 -03:00
steve-chavez a9d6c318fd nix: instructions for static binary 2023-10-06 19:54:05 -03:00
steve-chavez 2aa58164ab refactor: move errors to ApiRequestError 2023-10-06 01:16:45 -03:00
steve-chavez df08d7f3ff refactor: rename JSONParseError to PGRSTParseError
The error name is too generic otherwise.
2023-10-06 01:16:45 -03:00
steve-chavez 3dd292be46 refactor: DRY application/json on Error.hs
Small refactor to help with:

https://github.com/PostgREST/postgrest/issues/2901
2023-10-06 01:16:45 -03:00
Laurence IslaandGitHub e29260d754 Version not set when invalid URI or key/value format 2023-10-05 15:18:25 -05:00
steve-chavez d9857e1c54 releases: add v11.2.1 2023-10-04 12:25:09 -03:00
steve-chavez 0703f27d2b changelog: update to 11.2.1 2023-10-04 12:12:30 -03:00
dependabot[bot]andSteve Chavez 8a7889b9b1 Bump urllib3 from 2.0.2 to 2.0.6
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.0.2 to 2.0.6.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.0.2...2.0.6)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-03 23:41:04 -03:00
Laurence IslaandGitHub af0e369c65 fix: regression that rejects URI connection strings with certain unescaped characters in the password 2023-10-03 10:57:21 -05:00
Laurence IslaandGitHub 910950dbac fix: RPCs not embedding correctly when using overloaded functions for computed relationships 2023-10-02 16:00:33 -05:00
Andrei DziahelandSteve Chavez 4c44782d15 refactor: complete purifying Response module 2023-09-30 09:35:50 -03:00
steve-chavez 3b1eb51744 ci: fix MacOS CI 2023-09-29 19:50:15 -03:00
steve-chavez cf7ee67dfe fix: arrow filter on RPC returning TABLE+composite 2023-09-28 20:26:38 -03:00
steve-chavez 290d90609b fix: unnecessary set default_transaction_isolation 2023-09-27 17:48:48 -03:00
Taimoor ZaeemandSteve Chavez 3c1cdd434a feat: add handling=strict/lenient for Prefer header 2023-09-27 15:00:52 -03:00
steve-chavez 2825ac059e refactor: make Response module pure 2023-09-27 11:50:22 -03:00
Taimoor ZaeemandGitHub a6e3eda5b2 feat: implement JWT caching (#2928) 2023-09-25 14:46:55 -03:00
steve-chavez 10231bc1a8 reorganize errors page 2023-09-23 20:29:01 -03:00
Laurence IslaandSteve Chavez 90e3a5e29f Add test option for PostgreSQL 16 2023-09-21 15:38:46 -03:00
Andrei DziahelandSteve Chavez add4dfeed5 ci: loadtest PRs against latest releases
Leveraging the previously-improved postgrest-loadtest-against
introduces loadtesting PRS against latest release as well.
2023-09-21 14:08:53 -03:00
Andrei DziahelandSteve Chavez fca039a54d nix:postgrest-loadtest-against multiple branches
Improve postgrest-loadtest-against script to accept multiple refs
to perform loadtests against *all* of them
2023-09-21 14:08:53 -03:00
Laurence IslaandSteve Chavez 1b4dae5a7a Add missing changelog entries 2023-09-19 00:15:48 -03:00
Laurence IslaandGitHub 37a3f818dd fix: error when requesting "Prefer: count=<type>" with null filters on embedded resources 2023-09-16 15:02:59 -05:00
Andrei DziahelandSteve Chavez c3169b7dc2 ci: less strict conditions for loadtest inmain 2023-09-15 16:35:20 -03:00
Taimoor ZaeemandSteve Chavez d64b71cbf0 refactor: rename test module NullsStrip.hs to NullsStripSpec.hs 2023-09-15 14:39:12 -03:00
Taimoor ZaeemandSteve Chavez c195eece65 feat: add Server-Timing header with JWT duration 2023-09-15 14:39:12 -03:00
Laurence IslaandGitHub fa182c216e fix: bug when Null Filtering on embedded resources
When doing Null Filtering, the to-one embed resources were not included if they had a NULL value in any of the selected fields.
2023-09-14 21:30:48 -05:00
Andrei DziahelandSteve Chavez 30474c424c ci: loadtest commits to main against latest release
Introduces loadtest job for main branch
that runs loadtest against latest release
2023-09-14 17:10:48 -03:00
Laurence IslaandGitHub 2888f351d1 Remove former sponsor 2023-09-12 22:41:39 -05:00
Taimoor ZaeemandGitHub 69eaef461f nix: add postgrest-docs-check (#676) 2023-09-12 23:18:17 -03:00
steve-chavez 8d3c9f8435 refactor: move media type logic to Plan module 2023-09-12 19:24:26 -03:00
steve-chavez d8a91453d1 fix: inconsistent Preference-Applied
* Don't apply `tx=commit` if the transaction doesn't commit
* Apply `count=exact`
* Also simplifies the Preference-Applied logic, removing the need for
  some functions.
2023-09-12 13:52:49 -03:00
dependabot[bot]andGitHub 50c87fec88 Bump actions/checkout from 3 to 4 (#678) 2023-09-11 18:33:29 -03:00
Laurence IslaandGitHub 4ee5410c3d Remove former sponsor 2023-09-11 15:19:10 -05:00
dependabot[bot]andGitHub 3f5e840baf build(deps): bump cachix/install-nix-action (#2935) 2023-09-11 03:00:29 -03:00
dependabot[bot]andGitHub 2358b6670f build(deps): bump actions/checkout from 3 to 4 (#2936) 2023-09-11 02:34:35 -03:00
Taimoor ZaeemandGitHub 96b8fccd2a allow add-headers-with-raise (#675) 2023-09-06 13:00:01 -03:00
Laurence IslaandGitHub d719cbf59d Fix and add images to htmx how-to 2023-09-05 12:38:51 -05:00
steve-chavez a6f1eefe55 fix linkcheck 2023-09-04 19:23:07 -03:00
steve-chavez 3dead2ac8b clarify access mode 2023-09-04 19:23:07 -03:00
steve-chavez 5a1ed11924 note on current_setting inconsistency
Closes #362.
2023-09-04 19:23:07 -03:00
dependabot[bot]andSteve Chavez 46eae105b7 Bump cachix/install-nix-action from 22 to 23
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 22 to 23.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v22...v23)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-04 17:14:42 -03:00
Taimoor ZaeemandSteve Chavez 8eed576826 fix: range request with first position same as length return status 206 2023-09-04 15:50:00 -03:00
Laurence IslaandSteve Chavez d643aab261 Add html-htmx how-to 2023-09-02 20:26:55 -05:00
Taimoor ZaeemandGitHub 07fef25591 feat: allow full response control when raising exceptions 2023-09-01 14:02:03 -05:00
Haowu GeandSteve Chavez 9352a72195 Add a regular account to reduce operational security risks 2023-08-29 14:22:11 -05:00
Taimoor ZaeemandSteve Chavez 7dc6e2b899 fix: duplicate headers in response 2023-08-25 13:52:50 -05:00
Taimoor ZaeemandSteve Chavez ff3a3a9100 add db-pool-automatic-recovery 2023-08-25 13:34:48 -05:00
Taimoor ZaeemandGitHub 57fa2719dd feat: add db-pool-automatic-recovery configuration to disable connection retrying 2023-08-23 21:08:04 -05:00
Diogo BiazusandGitHub b8b3145c5c fix: schema cache and configuration reloading with NOTIFY not working on Windows 2023-08-21 10:25:44 -05:00
steve-chavez 531a183b44 nix: postgrest-coverage notice 2023-08-17 13:43:48 -05:00
steve-chavez 739f056b0a nix: add postgrest-repl command 2023-08-17 13:43:48 -05:00
Steve Chavez 32b77cae6c Revert "nix: Update nixpkgs to 2023-08-04"
This reverts commit 2434724edd.
2023-08-15 23:55:35 -05:00
Laurence IslaandSteve Chavez 46548dd09b Fix typo and version 2023-08-14 22:54:39 -05:00
Taimoor ZaeemandSteve Chavez 2434724edd nix: Update nixpkgs to 2023-08-04 2023-08-14 15:00:20 -05:00
Taimoor ZaeemandSteve Chavez 87d6a0d0fe fix: application/vnd.pgrst.array not accepted as a valid mediatype 2023-08-11 12:41:04 -05:00
steve-chavez c820efb64a bump version to 11.2.0 2023-08-10 08:51:58 -05:00
steve-chavez cacbbc31fe add setof record functions 2023-08-10 08:49:55 -05:00
steve-chavez 78d9880607 add nulls=stripped 2023-08-10 08:49:55 -05:00
steve-chavez b33bfb656d add preference-applied for return=* 2023-08-10 08:49:55 -05:00
steve-chavez daad21e9ec add note to impersonated role settings 2023-08-10 08:49:55 -05:00
steve-chavez c229bee681 add disamb error 2023-08-10 08:49:55 -05:00
steve-chavez 9ed7ff8846 clarify FK joins 2023-08-10 08:49:55 -05:00
Laurence IslaandSteve Chavez 562f699770 Revert disamb using FK instead of computed rels 2023-08-10 08:49:55 -05:00
Laurence IslaandSteve Chavez 3f112553b0 Add missing ERD/SQL tabs and fix some requests 2023-08-10 08:49:55 -05:00
steve-chavez 637165ba5b use the foreign key join term 2023-08-10 08:49:55 -05:00
steve-chavez 095d43ff75 add dynamic schemas example
Closes #644
2023-08-10 08:49:55 -05:00
steve-chavez 49d4b9b7db fix dict 2023-08-10 08:49:55 -05:00
steve-chavez afee9b73a4 add return=minimal and organize other returns
Closes #624
2023-08-10 08:49:55 -05:00
steve-chavez 459f3f8173 add apache apisix tutorial 2023-08-10 08:49:55 -05:00
steve-chavez b0aaf7afe8 mention header names are lowercased
Closes #627
2023-08-10 08:49:55 -05:00
steve-chavez 54bfefdb55 move heroku outside Installation 2023-08-10 08:49:55 -05:00
steve-chavez a6f93f55b1 clear supported pg versions
closes #592
2023-08-10 08:49:55 -05:00
steve-chavez 291b6f3587 add different ways to specify the connection string
Closes #576
2023-08-10 08:49:55 -05:00
steve-chavez 43a8745e62 link connection string from connection pool
Closes #647
2023-08-10 08:49:55 -05:00
steve-chavez e82dc41a6a add link to github discussions
closes #401
2023-08-10 08:49:55 -05:00
steve-chavez 1b036dfab9 release page for v11.2.0
Closes #644
2023-08-10 08:49:55 -05:00
steve-chavez e30bf53afa deprecate: target embedding disambiguation 2023-08-10 08:14:23 -05:00
steve-chavez 5ce020d5bc fix: impersonated role applying superuser settings 2023-08-09 20:05:22 -05:00
Taimoor ZaeemandGitHub fbf9bf21c2 feat: add optional nulls=stripped parameter for mediatypes applicatio… (#2894) 2023-08-09 18:49:22 -05:00
Taimoor ZaeemandSteve Chavez d490bf09fd feat: add Preference-Applied header in response for Prefer: return=representation/minimal/headers-only 2023-08-05 02:21:50 -05:00
Laurence IslaandGitHub aa53623aac fix: error when a function returns RECORD or SETOF RECORD 2023-08-03 20:09:55 -05:00
steve-chavez 0fce7ca361 refactor: Response rm Paths_postgrest dependency
This allows using doctests on the Response.hs module
2023-08-02 16:28:48 -05:00
Taimoor ZaeemandGitHub d0a71de2da chore: improve details field of the singular error response (#2876) 2023-08-01 19:37:54 -05:00
steve-chavez fefd0e2e83 dry config and make it responsive
* shorten connection pool
* move securing execution plan
* shorten config intro page
2023-07-31 19:22:26 -05:00
Laurence IslaandGitHub 40c2bcd4a1 fix: paramaters of type character and bit not ignoring length
- Fixes the error "value too long for type character(1)" when the char length of the parameter was bigger than one.
2023-07-31 18:36:37 -05:00
dependabot[bot]andGitHub 0dc67bed0b build(deps): bump LouisBrunner/checks-action from 1.6.1 to 1.6.2 (#2883) 2023-07-31 00:17:36 -05:00
steve-chavez 2d00d7d248 fix: null filtering on embed when column=relation 2023-07-27 22:43:13 -05:00
Taimoor ZaeemandGitHub 2977d09779 chore: maintenance to the Nix development and build guide 2023-07-26 20:28:59 -05:00
Laurence IslaandGitHub 630e0a1691 fix: character and bit columns with fixed length not inserting/updating properly
Fixes the error "value too long for type character(1)" when the char length of the column was bigger than one.
2023-07-21 11:30:01 -05:00
steve-chavez 220683cef0 add HEAD optimization 2023-07-20 18:09:47 -05:00
steve-chavez 4c4a1de832 to_jsonb index no longer required on json/jsonb 2023-07-20 18:09:47 -05:00
a6aa2adf04 Remove embedding disambiguation in favor of computed rels (#660)
* Add SQL snippets for ERDs
---------

Co-authored-by: steve-chavez <stevechavezast@gmail.com>
2023-07-20 09:02:33 -05:00
steve-chavez 28d5278d62 fix: use index on jsonb/jsonb arrow filter/order 2023-07-12 14:51:21 -05:00
Steve ChavezandGitHub c74b9c4efa add version querying (#657) 2023-07-10 17:04:23 -05:00
steve-chavez e332f038ef fix: HEAD unnecessarily executing aggregates 2023-07-10 15:05:51 -05:00
steve-chavez 905fcb05cc refactor: add ResultAggregate type 2023-07-10 15:05:51 -05:00
Laurence IslaandGitHub 774d015eb5 feat: add the --version CLI option to print the version information 2023-07-07 18:19:02 -05:00
Laurence IslaandGitHub e752224f14 feat: show PostgREST version in the logs 2023-07-07 10:28:27 -05:00
steve-chavez add10bd10c test: doctest for addNullEmbedFilters 2023-07-06 16:52:46 -05:00
steve-chavez 52d3026133 refactor: remove unneeded FilterNullEmbed 2023-07-06 16:52:46 -05:00
Laurence IslaandGitHub 5cdafb23e7 ci: fix ARM docker containers not stopping immediately 2023-07-06 11:20:32 -05:00
steve-chavez 20b06efabe chore: show on ReadPlan and MediaType 2023-07-05 17:35:54 -05:00
Grantandsteve-chavez 4706472df9 Update connection_pool.rst typo 2023-07-05 12:13:23 -05:00
Laurence IslaandGitHub 7508230760 feat: allow to verify the PostgREST version through SQL 2023-07-03 18:31:32 -05:00
ab8ccf43a6 add domain representations (#655)
* add domain representations

* Update docs/references/api/domain_representations.rst

Co-authored-by: Alexander Ljungberg <aljungberg@wireload.net>

* Update docs/references/api/domain_representations.rst

Co-authored-by: Alexander Ljungberg <aljungberg@wireload.net>

* Update docs/references/api/domain_representations.rst

Co-authored-by: Alexander Ljungberg <aljungberg@wireload.net>

* Update docs/references/api/domain_representations.rst

Co-authored-by: Alexander Ljungberg <aljungberg@wireload.net>

* Update docs/references/api/domain_representations.rst

Co-authored-by: Alexander Ljungberg <aljungberg@wireload.net>

* Update docs/references/api/domain_representations.rst

Co-authored-by: Alexander Ljungberg <aljungberg@wireload.net>

* add custom domain subheading and TOC

this way it can be seen at the beggining there are advantages over views
and computed fields.

* important note for scache reloading

* clarify advantages

* why not create a type?

---------

Co-authored-by: Alexander Ljungberg <aljungberg@wireload.net>
2023-07-02 14:51:47 -05:00
steve-chavez a17dd41d6b fix: Prefer: missing=default with DOMAIN defaults 2023-06-30 17:46:16 -05:00
Alexander LjungbergandSteve Chavez 0a1564ba5a feat: data representations allow custom parsing and formatting of API fields.
See PR #2523. Most notable code changes:

- Load data representation casts into schema cache.
- Data representations for reads, filters, inserts, updates, views, over joins.
- `CoercibleField` represents name references in queries where coercion may be needed.
- `ResolverContext` help facilitate field resolution during planning.
- Planner 'resolves' names in the API query and pairs them with any implicit conversions to be used in the query builder stage.
- Tests for all of the above.
- More consistent naming (TypedX -> CoercibleX).

New: unit tests for more data representation use cases; helpful as examples as well.

New: update CHANGELOG with data representations feature description.

Fixed failing idempotence test.

New: replace date formatter test with one that does something.

Fixup: inadvertent CHANGELOG change after rebase.

Cleanup: `tfName` -> `cfName` and related.

Document what IRType means.

Formatting.

New: use a subquery to interpret `IN` literals requiring data rep transformation.

- With the previous method, very long queries such as `ANY (ARRAY[test.color('000100'), test.color('CAFE12'), test.color('01E240'), ...` could be generated. Consider the case where the parser function name is 45 characters and there's a hundred literals. That's 4.5kB of SQL just for the function name alone!
- New version uses `unnest`: `ANY (SELECT test.color(unnest('{000100,CAFE12,01E240,...}'::text[]))` to produce a much shorter query.
- This is likely to be more performant and either way much more readable and debuggable in the logs.
2023-06-29 15:01:58 -05:00
steve-chavez 1a9e105227 add headings to computed fields 2023-06-28 12:12:27 -05:00
Steve Chavez 3a41d55be6 Fix inserting with defaults 2023-06-27 15:37:22 -05:00
steve-chavez d59f841d15 add request body handling 2023-06-26 17:51:21 -05:00
steve-chavez 06a3898107 nix: add HISTFILE 2023-06-26 17:51:21 -05:00
steve-chavez 078c6ec08c refactor: remove SqlFragment and use SQL.Snippet 2023-06-22 21:46:16 -05:00
Steve ChavezandGitHub 991b151b69 improve connection pool wording 2023-06-22 14:44:26 -05:00
Steve ChavezandGitHub 14de9f1dfd references: add computed fields page (#650) 2023-06-22 09:14:23 -05:00
Laurence IslaandGitHub 83cf15fb7e ci: specify GHC_VERSION for ARM build on CI config
* change cabal version from recommended to latest
2023-06-21 16:54:15 -05:00
steve-chavez 40dc46ed2e fix: compilation on Ubuntu, GHC 9.0.2 compat 2023-06-21 16:22:30 -05:00
steve-chavez d9261fa674 ci: check against GHC 9.0.2 2023-06-21 16:22:30 -05:00
JoonasandSteve Chavez 9780790cee Change example getallfilms to stable instead of immutable 2023-06-21 09:51:41 -05:00
steve-chavez 856d450775 refactor: stricter plan media type 2023-06-20 19:12:17 -05:00
Laurence IslaandGitHub c1a8661ab3 ci: Update dependabot to look for dependencies in nested folder
- Bump cachix/install-nix-action from 21 to 22

- Add actions/setup-nix to dependabot dependencies
2023-06-19 12:33:42 -05:00
dependabot[bot]andGitHub ab986b77d8 Bump cachix/install-nix-action from 21 to 22 (#648) 2023-06-19 10:22:57 -05:00
steve-chavez aa15f4782e refactor: remove MTPlanAttrs to doctest 2023-06-18 15:27:25 -05:00
steve-chavez 77cd9387d4 Revert "refactor: plan media type"
This reverts commit 3cd3a3f8c6.
2023-06-18 15:27:25 -05:00
steve-chavez 3cd3a3f8c6 refactor: plan media type 2023-06-17 23:00:32 -05:00
steve-chavez 78821a8fe7 refactor: clarify content negotiation 2023-06-16 17:36:44 -05:00
steve-chavez 5c372df487 refactor: asJsonF/asJsonSingleF Routine param 2023-06-16 17:36:44 -05:00
Laurence IslaandGitHub 772046242e Add postgrest-docs-dictcheck to CI 2023-06-14 14:55:28 -05:00
Laurence IslaandGitHub fdf85773e7 Fix jwt_secret in example 2023-06-14 14:25:54 -05:00
steve-chavez fad47324c3 fix: OPTIONS not accepting all available media 2023-06-13 22:00:21 -05:00
Laurence IslaandGitHub 396afa106e Remove old or unmaintained links from the ecosystem page 2023-06-12 19:36:06 -05:00
dependabot[bot]andGitHub 11a9849152 build(deps): bump LouisBrunner/checks-action from 1.6.0 to 1.6.1 (#2817) 2023-06-12 10:34:48 -05:00
David BaynardandGitHub 018427e2b6 Note content-type for singular objects (#639)
* Clarify vnd.pgrst.object+json is also Content-Type

The previous docs explained how the `Accept` header could be set to
`application/vnd.pgrst.object+json`. This change indicates the resulting
response `Content-Type` begins likewise.

While robust clients will correctly identify that
`application/vnd.pgrst.object+json` data are json, many others which
naively check the `Content-Type` is `application/json` will fail to
recognize what should be a valid `Content-Type`.
2023-06-09 15:23:04 -05:00
steve-chavez ee6b9c5224 pin version to 11.0.1 2023-06-07 19:56:05 -05:00
steve-chavez b1f50537e9 clarify procs intro 2023-06-07 19:56:05 -05:00
steve-chavez fb59b82c35 references: updated automatic recovery
* use bullets were possible and join paragraphs.
* add bug fixes to the unreleased page
2023-06-07 19:56:05 -05:00
steve-chavez 188461af7d references: prepared statements mention 2023-06-07 19:56:05 -05:00
steve-chavez c62d24c993 references: add dynamic pool 2023-06-07 19:56:05 -05:00
steve-chavez 1f13e43abe bump version to 11.1.0 2023-06-07 19:03:31 -05:00
steve-chavez fac797c766 changelog: clarify 2781 2023-06-07 18:28:01 -05:00
steve-chavez 07cb0b582e feat: recover on pg_terminate_backend 2023-06-06 14:41:37 -05:00
steve-chavez d54a2f48de refactor: move checkIsFatal to AppState 2023-06-06 14:41:37 -05:00
steve-chavez bcce7b1c53 fix: stop retrying on "no password supplied" 2023-06-06 14:41:37 -05:00
steve-chavez 8d1961ce07 refactor: cache the isolation level 2023-06-06 14:41:37 -05:00
steve-chavez 9a19dff83e refactor: move debounceLogAcquisitionTimeout
Move it to AppState
2023-06-06 14:41:37 -05:00
paulocrandLaurence Isla e7d63a2b27 Fix boolean configuration example 2023-06-05 16:06:37 -05:00
steve-chavez a7ff2294ac references: move raise error to errors page 2023-06-03 00:20:21 -05:00
steve-chavez 54b9a0b8b3 test: fix hello function for pg 9.6 2023-06-02 17:45:25 -05:00
steve-chavez 9a3d453bf4 test: remove big_schema.sql from io tests
It was unused(test skipped) and made the scache reload slower, causing
test failure on CI.
2023-06-02 17:45:25 -05:00
steve-chavez 4f6c466031 nix: postgrest-run uses similar config to tests
Otherwise manual tests can be deceiving
2023-06-02 17:45:25 -05:00
Steve ChavezandGitHub a852b766eb Fix dropping schema cache reload notifications
* test: bad schema reload
* refactor: DRY using the "extra" lib
* refactor: move worker funtions inside AppState
* Also rename Workers.hs to Admin.hs
2023-06-02 12:22:17 -05:00
Laurence IslaandGitHub 48365188ad Add redirect to many-to-many relationships from embedding through join tables 2023-05-31 12:33:08 -05:00
dependabot[bot]andSteve Chavez d87d80108a Bump cachix/install-nix-action from 20 to 21
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 20 to 21.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v20...v21)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-29 09:56:01 -05:00
steve-chavez d59e2498be add pre-config function 2023-05-28 10:23:37 -05:00
steve-chavez 1c15d3d35b remove note and put it at the start 2023-05-28 10:23:37 -05:00
steve-chavez 232c6dc702 dry config settings
Environment variables names and in-db configs on the table.
2023-05-28 10:23:37 -05:00
steve-chavez 14be3fb671 feat: add pre-config function
Allows using the in-database configuration without SUPERUSER
2023-05-28 10:21:13 -05:00
steve-chavez 8a3686d86b refactor: whitelist in-db config settings 2023-05-28 10:21:13 -05:00
steve-chavez 8d706a342c fix broken links 2023-05-27 22:23:44 -05:00
steve-chavez fff0eae77e Fix in-db config settings 2023-05-27 22:23:44 -05:00
steve-chavez 5e19594ea4 auth: explain why jwt is used 2023-05-27 22:23:44 -05:00
steve-chavez 281d7a9223 remove db functions section on index 2023-05-27 22:23:44 -05:00
Steve Chavez 2e6ebca885 Fix link to archlinux pkg 2023-05-27 22:23:44 -05:00
steve-chavez 009250006e feat: add db-pool-timeout alias
For db-pool-max-idletime
2023-05-22 18:34:18 -03:00
steve-chavez 38ad8c04e1 nix: withTmpDb set search_path to test
It gets repetitive to set the search_path every time you go in the test
db with psql.
2023-05-22 13:46:24 -03:00
dependabot[bot]andGitHub 54cbf147e3 build(deps): bump codecov/codecov-action from 3.1.3 to 3.1.4 (#2794) 2023-05-22 11:34:18 -05:00
RobertandGitHub f9f0f79fa9 feat: use hasql-pool-0.10, add db-pool-max-idletime (#2786)
- new option db-pool-max-idletime limits the maximal idle time of a
  postgresql connection, defaults to 30 seconds
- this essentially restores the db-pool-timeout option from versions
  <=10, with more specific name
2023-05-16 23:00:40 +02:00
Laurence IslaandGitHub 05bd033539 Add JavaScript redirect from the old api.html to new pages and sections
Uses the hashtag in the url to redirect to the sections where the content was moved to
2023-05-12 21:48:01 -05:00
Laurence IslaandGitHub ce82b0c79b Re-add missing legacy gucs information 2023-05-12 21:27:12 -05:00
steve-chavez 911634ed32 move impersonated settings 2023-05-12 01:14:19 -03:00
steve-chavez 946b113912 add function inlining 2023-05-11 21:57:54 -03:00
steve-chavez d7907f4aa4 pin version to 11.0.1 2023-05-11 01:37:50 -03:00
steve-chavez ed147ce5c8 add bug fixes
simply link to github releases page.
bug fixes do not add anything for docs.
2023-05-11 01:37:50 -03:00
steve-chavez 789cedff08 add isolation level 2023-05-11 01:37:50 -03:00
steve-chavez e0b0e98816 add impersonated settings 2023-05-11 01:37:50 -03:00
steve-chavez ea7a2bd531 add bulk insert defaults 2023-05-11 01:37:50 -03:00
steve-chavez 84b01dea99 add recursive m2m disamb 2023-05-11 01:37:50 -03:00
steve-chavez f02be3fb76 add spread embed 2023-05-11 01:37:50 -03:00
steve-chavez 9595ed845e add null filtering embeds 2023-05-11 01:37:50 -03:00
steve-chavez d15dc680af add top-level ordering 2023-05-11 01:37:50 -03:00
steve-chavez 8ae6b012f4 add empty embed filter 2023-05-11 01:37:50 -03:00
steve-chavez 7832ddba52 add any/all modifiers 2023-05-11 01:37:50 -03:00
steve-chavez e813375c7d openapi title override 2023-05-11 01:37:50 -03:00
steve-chavez 1de06f5332 add openapi overriding 2023-05-11 01:37:50 -03:00
steve-chavez cb7d70c197 add trace header 2023-05-11 01:37:50 -03:00
steve-chavez 0bd43663fc add isdistinct 2023-05-11 01:37:50 -03:00
steve-chavez c56108b8a9 add breaking changes 2023-05-11 01:37:50 -03:00
steve-chavez dec1e03a0e add v11 release notes with docs improvs 2023-05-11 01:37:50 -03:00
Laurence IslaandLaurence Isla d593bcff80 Fix remaining info about pg-safeupdate in nginx docs 2023-05-10 15:35:22 -05:00
steve-chavez c5d6526d99 move pg-safeupdate to own integration 2023-05-10 14:21:31 -03:00
steve-chavez fc635d18ae split the API reference 2023-05-10 14:21:31 -03:00
steve-chavez c82b23940b add missing mention to cache reload on recovery 2023-05-10 12:52:39 -03:00
steve-chavez a530030b71 add integrations
* Move Nginx and systemd to integration
* Move External JWT Generatio to integration
* greenplum short notice
2023-05-08 14:33:29 -03:00
steve-chavez 637a6e281d Move part of admin to references
- Add integrations section with Nginx and SystemD
- Move Nginx and SystemD sections from admin on integrations
2023-05-08 13:17:33 -03:00
steve-chavez 79620396ed remove manual inclusion of pages on index
* drop unavailable doc pages from release notes
* rename releases pages so they order in TOC
2023-05-08 08:58:09 -03:00
Steve ChavezandGitHub 36300da16d connection pool reference (#616)
* add link in index to external jwt
* change pgrst errors location for better reading
2023-05-07 10:25:12 -05:00
steve-chavez a867d79c42 nix: withTmpDb psql notice can't mutate the db
Use the postgres role instead of the authenticator on the notice
2023-05-05 20:03:09 -03:00
steve-chavez 082e9b14e9 transactions reference doesn't consider anon reqs
Change authenticated to user impersonation.
2023-05-05 12:04:46 -03:00
steve-chavez 2e1dc0738a readthedocs doesn't build anymore
Pin the python version to solve it.
2023-05-05 12:04:46 -03:00
steve-chavez 9a3973be35 add transactions reference 2023-05-05 12:04:46 -03:00
steve-chavez 560ec79a90 clarify schema cache and errors page 2023-05-05 12:04:46 -03:00
steve-chavez 9de94274cf shorten some paragraphs in errors page 2023-05-03 17:47:54 -03:00
steve-chavez 837df8b045 schema cache title is redundant on navbar
Use a raw html title for correcting this
2023-05-03 17:47:54 -03:00
steve-chavez 4fb13ebae6 add tables to db authz 2023-05-03 17:47:54 -03:00
steve-chavez 8ec925c33e clarify view security invoker feature
Move RULEs limitation to Insertions
2023-05-03 17:47:54 -03:00
steve-chavez d1d159d1e0 join schema structure with db authz 2023-05-03 17:47:54 -03:00
steve-chavez 4f4ae27a2d clarify custom validation 2023-05-03 08:45:05 -03:00
steve-chavez 764f0a0352 link authn to db authz 2023-05-03 00:38:23 -03:00
steve-chavez 473035575c database as single source of truth philosophy
make it clear at the index
2023-05-03 00:06:17 -03:00
steve-chavez 638aa8369d fix broken link 2023-05-03 00:06:17 -03:00
steve-chavez 55cc616a5e Authentication as a reference
* Creates a Database Authorization topic
* Moves JWT from Auth0 into Ecosystem
* Moves SQL User Management to a how-to
2023-05-03 00:06:17 -03:00
steve-chavez 4197d2f739 bump version to 11.0.1 2023-04-27 21:30:40 -05:00
steve-chavez c10ba8e214 fix: skew of 30 seconds for JWT validation 2023-04-27 21:06:06 -05:00
steve-chavez 887948d259 fix: missing=default error msg on generated column 2023-04-27 19:27:09 -05:00
steve-chavez c63786733a fix: no schema privilege for schema cache
This was due to the usage of pg_get_serial_sequence
2023-04-27 19:27:09 -05:00
steve-chavez 4fe696dd96 nix: PGRST_DB_ANON_ROLE default for postgrest-run
Makes manually testing a feature easier
2023-04-27 19:27:09 -05:00
steve-chavez 67936b343f test: prove that authenticator is not a superuser 2023-04-27 19:27:09 -05:00
steve-chavez b0e395f495 nix: no SUPERUSER for connection role
Change :USER to :PGUSER in SQL scripts
2023-04-27 19:27:09 -05:00
Steve ChavezandLaurence Isla 4e49310d6e Clarify pool connection features 2023-04-25 20:27:02 -05:00
424eac4b3e Add documentation for v10.2.0
Co-authored-by: Steve Chavez <stevechavezast@gmail.com>
2023-04-25 20:27:02 -05:00
dependabot[bot]andGitHub 3b55a27ef3 build(deps): bump codecov/codecov-action from 3.1.2 to 3.1.3 (#2765) 2023-04-24 11:21:49 -05:00
Laurence IslaandGitHub f55ced0306 Add a warning when working with computed relationships 2023-04-21 17:04:31 -05:00
Steve Chavez 516a4daacf snippet for computed rel overload 2023-04-18 17:16:51 -05:00
Steve Chavez 3ff9b81c7d Remove public from schema cache event trigger 2023-04-18 13:25:01 -05:00
dependabot[bot]andGitHub 43da81c30c build(deps): bump codecov/codecov-action from 3.1.1 to 3.1.2 (#2757) 2023-04-18 12:25:50 -05:00
steve-chavez dd2f5511d8 bump version to 11.0.0 2023-04-16 14:22:38 -05:00
steve-chavez 49c349c846 chore: CHANGELOG to include release 10.2.0 2023-04-16 12:18:10 -05:00
steve-chavez aaf77902f6 feat: isolation level for roles/functions 2023-04-15 18:05:04 -05:00
steve-chavez 4c555cbd5d refactor: authRole to ByteString 2023-04-15 18:05:04 -05:00
steve-chavez 3e53796120 correct missing=default with GENERATED BY column 2023-04-14 22:12:34 -05:00
Laurence IslaandGitHub 0a2b7064c7 fix: PATCH requests not recognizing embedded filters
- Already fixed in #2618
- Adds tests and CHANGELOG
2023-04-14 16:42:55 -05:00
Laurence IslaandGitHub f588bb6daf Fix info on updates without filters 2023-04-13 18:38:07 -05:00
steve-chavez ce378e6b3a refactor: clarify Scalar type
It now contains the type of the scalar. This way we can discriminate the
void type in a more obvious way.
2023-04-13 18:17:43 -05:00
steve-chavez feadf59bb3 refactor: rename Proc module to Routine
Assuming that functions=procedures is wrong since pg11, which introduced
real stored procedures.
2023-04-13 18:17:43 -05:00
Laurence IslaandGitHub ad7d80a430 ci: clean files from arm server only on build failure/cancel for a version release 2023-04-13 16:29:26 -05:00
steve-chavez e572d1d1a2 feat: configurable role settings 2023-04-10 14:27:08 -05:00
steve-chavez c06237cc56 fix: db settings and pg version query not prepared 2023-04-10 14:27:08 -05:00
steve-chavez c656a870f4 bump version to 10.2.0.20230407 2023-04-07 14:02:14 -05:00
steve-chavez 1b625cb77a feat: any/all modifiers for operators
Only for the eq,like,ilike,gt,gte,lt,lte,match,imatch operators
2023-04-07 13:10:57 -05:00
Robert Vollmert 394bd22148 feat: use hasql-pool-0.9, add db-pool-max-lifetime (fixes #2638)
- db-pool-acquisition-timeout is no longer optional, defaults to 10s
- new option db-pool-max-lifetime limits the maximal lifetime of a
  postgresql connection, defaults to 30m
2023-04-06 22:40:22 +02:00
Robert Vollmert 963416ae29 test: stabilize log probe in test_pool_acquisition_timeout
There's two lines of log output, and their order is not deterministic.
2023-04-06 22:40:22 +02:00
Laurence IslaandGitHub d3b10e7b2a feat: Allow filtering by IS DISTINCT FROM using the isdistinct operator 2023-04-06 13:02:03 -05:00
Steve ChavezandGitHub acf62320ef RPC returning table alias now works for pg 11/12 (#2737)
The new LATERAL query used for calling the function, introduced on
https://github.com/PostgREST/postgrest/pull/2677, failed on functions
that returned a domain like `CREATE DOMAIN projects_domain AS projects`.

Work around that by changing the query conditionally, by
obtaining a bool that represents the composite alias on the SchemaCache
and only do this on pg 11 and 12.
2023-04-05 15:34:41 -05:00
steve-chavez 16f2849724 refactor: wrap proc logic in asJsonF
* add explicit logic for returning setof scalars
2023-04-04 19:55:10 -05:00
steve-chavez d945e8c06a refactor: remove Maybe from RetType for Proc type 2023-04-04 19:55:10 -05:00
steve-chavez bc1fb67df0 drop: Prefer: params=multiple-objects on RPC
BREAKING CHANGE

A function with a JSON array or object parameter should be used instead
2023-04-04 19:55:10 -05:00
Wolfgang Walther 1442e02f5f chore: Allow seed-cachix job to keep running when part of the matrix fail 2023-04-04 18:52:52 +02:00
Wolfgang WaltherandWolfgang Walther 68d2d834ba nix: Update nixpkgs to 2023-03-25 2023-04-04 18:51:34 +02:00
Steve ChavezandGitHub eb777cf823 Update BACKERS.md 2023-04-03 11:09:19 -05:00
Laurence IslaandGitHub 1aed55be68 Ignore the Range header when the method is different than GET
fix: bug when using Range header on PATCH/DELETE
  - Fix the "message": "syntax error at or near \"RETURNING\"" error
  - Fix doing a limited update/delete when an order query parameter was present

breaking: The Range header is now only considered on GET requests and is ignored for any other method
  - Other methods should use the `limit/offset` query parameters for sub-ranges
  - PUT requests no longer return an error when this header is present
2023-04-03 10:40:25 -05:00
steve-chavez 032f07f3f0 refactor: remove character varying casting 2023-03-29 13:17:15 -05:00
steve-chavez 7629eff51d undefined-keys=apply-defaults to missing=default 2023-03-29 05:33:38 -05:00
steve-chavez 46fd856fc6 test: tidy inlining plan test 2023-03-29 05:33:38 -05:00
Laurence IslaandGitHub caaa9a3944 ci: Update ARM build script to use GHC 9.2.4 2023-03-24 20:23:43 -05:00
Steve ChavezandGitHub 439a96c578 feat: undefined json keys as defaults w/ Prefer:undefined-keys 2023-03-22 02:44:14 -05:00
Robert Vollmert e731241b97 test: optionally build postgrest with cabal in postgrest-loadtest
By default, postgrest-with-pgrst builds postgrest as a nix package,
which means that source changes cause a full rebuild. With this
change, running the loadtest as

PGRST_BUILD_CABAL=1 postgrest-loadtest

rebuilds directly using cabal, like postgrest-build. Note that
results between nix and cabal builds aren't necessarily comparable
due to differing build parameters.
2023-03-20 15:00:11 +01:00
Robert Vollmert 216dc833fd test: add optional latency to postgrest-loadtest
This pipes data between client/postgrest and between
postgrest/database through a proxy that adds delay
(github.com/robx/slocat).
2023-03-20 15:00:11 +01:00
Robert Vollmert a8e02f766b perf: Allow preparing pre-request SQL statement
This will prepare the pre-request statement (if prepared statements
are enabled) with a possible minor performance benefit -- the load
test is not entirely conclusive.
2023-03-20 13:40:43 +01:00
steve-chavez cae1c67b00 refactor: add iPreferences to ApiRequest
It avoids adding a new iPrefer.. to the ApiRequest every time a
preference is added.
2023-03-09 14:40:32 -05:00
Steve ChavezandGitHub b05ea14122 nix: add notices for postgrest-with-* commands (#2697)
For knowing where to connect and how to get logs
2023-03-07 12:42:41 -05:00
Steve ChavezandGitHub 666114f81d refactor: remove SchemaCache from ApiRequest (#2695) 2023-03-06 13:22:35 -05:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
6da204d554 Bump cachix/install-nix-action from 19 to 20 (#597)
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 19 to 20.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v19...v20)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-06 11:17:46 -05:00
Steve ChavezandGitHub 3e99995e6a feat: make db-root-spec stable (#2694) 2023-03-01 12:23:19 -05:00
steve-chavez 8213c58452 ci: pin Nix version to avoid error
To 2.13.3
2023-02-28 17:30:54 -05:00
steve-chavez ee036f8397 feat: add server-trace-header config for tracing 2023-02-28 17:30:54 -05:00
steve-chavez 5b6421d03a test: fix sporadic failure on PlanSpec 2023-02-28 17:30:54 -05:00
dependabot[bot]andSteve Chavez ba00cb98f4 build(deps): bump actions/upload-artifact from 2.3.1 to 3.1.2
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2.3.1 to 3.1.2.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2.3.1...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-27 12:00:46 -05:00
dependabot[bot]andSteve Chavez 6184921edc build(deps): bump actions/checkout from 2.4.0 to 3.3.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 2.4.0 to 3.3.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2.4.0...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-27 10:59:38 -05:00
Robert Vollmert 9e0ea113fb refactor: split pre-request function out of setPgLocals 2023-02-27 15:12:32 +01:00
steve-chavez 0d5d209bfb perf: use LATERAL instead of CTE for json body
* add pgbench test for INSERT and RPC
* use LATERAL for RPC and INSERT
* add tests for inlining
2023-02-25 15:01:06 -05:00
steve-chavez 253f2bf537 ci: remove GHC 8.10.7 2023-02-25 14:19:52 -05:00
steve-chavez 2a2889020a nix: fix PGTZ on postgrest-with-* commands 2023-02-20 17:23:11 -05:00
Steve ChavezandGitHub 6a79de67ce fix: log to stderr on AcquisitionTimeoutUsageError (#2667)
* refactor: remove uneeded type on checkIsFatal
* dry with a logPgrstError function
2023-02-17 17:38:31 -05:00
steve-chavez c49932d3a8 nix: UTC by default for postgrest-with-postgresql 2023-02-14 21:30:45 -05:00
RobertandGitHub 95d71281d6 refactor: fix inaccurate variable name (#2662) 2023-02-14 17:09:48 +01:00
Steve ChavezandGitHub cf8ced1014 remove wrong claim about CSV being faster (#595) 2023-02-13 22:29:38 -05:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
adeb622d31 Bump cachix/install-nix-action from 18 to 19 (#593)
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 18 to 19.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v18...v19)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 12:34:33 -05:00
Steve ChavezandGitHub 87b383316d fix: PGRST003 status code to 504 (#594) 2023-02-13 11:06:22 -05:00
Ian EybergandGitHub 6089a46235 adding link to ops unikernel package (#586) 2023-02-09 18:43:11 -05:00
steve-chavez a1e2fe308f bump version to 10.2.0.20230209 2023-02-09 17:18:31 -05:00
Laurence IslaandGitHub 6b65a5b151 Add HTTP status codes to PGRST errors (#590) 2023-02-09 16:48:17 -05:00
steve-chavez a9d66d1dac ci: fix ghcup 2023-02-09 16:46:27 -05:00
steve-chavez f4135bb3ff nix: correct postgrest-release pre 2023-02-09 16:46:27 -05:00
steve-chavez d7868235d8 refactor: add getRanges func to ApiRequest 2023-02-09 16:46:27 -05:00
Laurence IslaandGitHub 60f4446fd8 Change inaccurate error codes to new ones
- Code for column is not found is PGRST204
- Code for timed out when acquiring connection to db is PGRST003
2023-02-09 14:57:51 -05:00
Laurence IslaandGitHub 378a225a9b Add db-pool-acquisition-timeout configuration parameter 2023-02-09 14:56:38 -05:00
Laurence IslaandGitHub c893ac15dc Add the missing 'get' path item for RPCs to the OpenAPI output 2023-02-09 12:04:41 -05:00
dependabot[bot]andSteve Chavez ecada119f8 build(deps): bump LouisBrunner/checks-action from 1.5.0 to 1.6.0
Bumps [LouisBrunner/checks-action](https://github.com/LouisBrunner/checks-action) from 1.5.0 to 1.6.0.
- [Release notes](https://github.com/LouisBrunner/checks-action/releases)
- [Commits](https://github.com/LouisBrunner/checks-action/compare/v1.5.0...v1.6.0)

---
updated-dependencies:
- dependency-name: LouisBrunner/checks-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-06 09:21:31 -05:00
steve-chavez 57e32ca0a3 bump version to 10.2.0.20230203 2023-02-03 21:16:43 -05:00
Laurence IslaandGitHub eb3f7c696a Fix release date for v10.1.2 in CHANGELOG 2023-02-02 14:50:42 -05:00
Laurence IslaandSteve Chavez 09249f97ab Update CHANGELOG to include release 10.1.2 2023-02-02 14:40:34 -05:00
Laurence IslaandGitHub 754282c385 Add missing fixes to the changelog 2023-02-01 22:15:43 -05:00
steve-chavez 52ba757ebe nix: postgrest-release bumps pre minor 2023-01-31 19:11:45 -05:00
steve-chavez 7aadaa44e8 refactor: remove Target from infoResponse 2023-01-30 00:23:24 -05:00
steve-chavez 0139cd8261 refactor: move txMode to Plan 2023-01-30 00:23:24 -05:00
Steve ChavezandGitHub 082b418334 Add db-plan-enabled recommendation (#588)
* fix broken links
2023-01-28 20:20:48 -05:00
steve-chavez 7874bee879 fix: NOTIFY pgrst not reoading the catalog cache 2023-01-28 10:30:29 -05:00
RobertandGitHub 12ad7d0585 test: fix result path for load test (#2628) 2023-01-25 08:56:52 +01:00
AnthonyFisiandGitHub 9065ed65fa feat: customizable OpenAPI title 2023-01-21 19:34:55 -05:00
Tuan LeandSteve Chavez 8aa79086d2 fix: consider authentication failure as a fatal error 2023-01-20 13:51:28 -05:00
Laurence IslaandGitHub 775c006806 fix: FK pointing to VIEW instead of TABLE in OpenAPI output 2023-01-20 09:37:15 -05:00
steve-chavez be6c30a0fb refactor: dry RPC param parsing 2023-01-18 12:26:13 -05:00
RobertandGitHub 8f80cd1469 Update nixpkgs, dependencies (#2612)
* relax upper bounds on HTTP, hspec, lens-aeson, optparse-applicative (fixes #2580)
* upgrade stackage snapshot to latest LTS, with GHC 9.2.5
* bump nixpkgs to 2023-01-12
* fix complaints due to updated linters
2023-01-16 17:12:54 +01:00
RobertandGitHub 5fd6b3956e ci: Add workflow to seed cachix (#2615)
Fixes #2609.
2023-01-16 16:56:48 +01:00
RobertandGitHub f353711ed2 ci: build with nix on macos (and push to cachix) (#2610)
The idea is to ensure the macos nix build works, and to
build a cache in cachix to help local development on macos.
2023-01-12 11:45:10 +01:00
Robert Vollmert 0a56d6ce88 ci: authenticate GitHub API request (fixes #2607) 2023-01-10 18:45:50 +01:00
Alexander LjungbergandGitHub 43ad6d6aa0 feat: validate ?columns mutation targets based on schema cache (#2542)
This returns an error for trying to update or insert into invalid columns, without hitting the database. This change also switches from `json_populate_recordset` for these operations `json_to_recordset` which should make no functional difference except allowing future flexibility.
2023-01-07 22:08:30 -05:00
Franz-Josef FärberandSteve Chavez 8be87ed4ac Minor improvements to sql-user-management-using-postgres-users-and-passwords.rst 2023-01-04 02:22:41 -03:00
RobertandGitHub 5a0f83ecb8 bump postgresql-libpq (#2599)
For https://github.com/PostgREST/postgresql-libpq/pull/2.
2022-12-30 15:40:17 +01:00
Laurence IslaandGitHub 4ddd5b4a76 Add upsert headers for POST requests to the OpenAPI output 2022-12-27 16:16:41 -05:00
Laurence IslaandGitHub 1f69757835 fix: Add required OpenAPI items object when the paramater is an array 2022-12-20 12:30:28 -05:00
steve-chavez e0df95844b bump version to 10.1.1.20221215 2022-12-15 23:06:32 -05:00
Laurence IslaandGitHub 5a660e0cbe Add suggestions with fuzzy text search when no relationship is found (#2583) 2022-12-15 18:33:15 -05:00
steve-chavez 566c6fe53f correct bad null filter restriction
When embedding using the column name.

Also make related order similarly strict to avoid the same kind of issues.
2022-12-15 18:02:25 -05:00
steve-chavez d93bb95d83 bump version to 10.1.1.20221212 2022-12-12 17:34:46 -05:00
steve-chavez 3773cce246 feat: null filters on embedded resources 2022-12-12 17:14:54 -05:00
steve-chavez 52c011f896 refactor: remove Bool from readPlanToQuery 2022-12-12 17:14:54 -05:00
steve-chavez 81cd9d4b15 refactor: delete unused TestTypes 2022-12-12 13:31:02 -05:00
steve-chavez 781fa592ca refactor: delete QueryCost, instead use PlanSpec 2022-12-12 13:31:02 -05:00
steve-chavez 1065021348 refactor: add planCost and planHdr for tests 2022-12-12 13:31:02 -05:00
Laurence IslaandGitHub aecc53d8f9 fix: clarify error messages for functions
Move explanation on single unnamed parameters to the error details
2022-12-09 14:34:38 -05:00
steve-chavez a5465e20d0 correct spread embed to '...' 2022-12-05 19:38:32 -05:00
Laurence IslaandGitHub 9e567216e9 feat: hint function names/parameters on error 2022-12-01 17:17:45 -05:00
Steve ChavezandGitHub 5e9dba5292 feat: Allow embedding without selecting any column (#2574) 2022-11-25 18:37:57 -05:00
steve-chavez f7009635d6 test: move failed order parsing to doctest 2022-11-24 09:03:35 -05:00
steve-chavez e5c77385ae test: move failed or/and parsing to doctest 2022-11-24 09:03:35 -05:00
steve-chavez 3103060f4b test: move failed json parsing to doctest 2022-11-24 09:03:35 -05:00
steve-chavez 315b01ebf7 test: spread embed includes junction cols 2022-11-18 18:11:14 -05:00
steve-chavez 25f65065f4 test: spread embed disambiguates recursive m2m 2022-11-18 18:11:14 -05:00
steve-chavez 60c0c11c1d test: spread embed disambiguates 2 fks on junction 2022-11-18 18:11:14 -05:00
steve-chavez cb99270a8f fix: bad M2M embed on RPC 2022-11-18 17:49:24 -05:00
steve-chavez cca0b5ae66 messages: better error for related order 2022-11-16 21:26:16 -05:00
steve-chavez 2aa0e091bb feat: add spread embeds 2022-11-16 21:26:16 -05:00
fjf2002andGitHub f6e3b77643 how-to: SQL User Management using postgres’ users and passwords (#581) 2022-11-16 17:10:18 -05:00
Laurence IslaandSteve Chavez f3831427e5 Fix broken links and dictcheck 2022-11-14 22:14:04 -05:00
Laurence IslaandSteve Chavez 485e4db4d7 Fix auth page due to anonymous requests not allowed when db-anon-role is not set 2022-11-14 22:14:04 -05:00
Laurence IslaandSteve Chavez a6af266ab7 Fix step 3 of the tutorial 0 2022-11-14 22:14:04 -05:00
steve-chavez 78d45b4e32 feat: add related orders 2022-11-08 17:28:50 -05:00
steve-chavez ef42d1c87f test: add doctest for pOrderTerm 2022-11-08 17:28:50 -05:00
Andrea BernicchiaandGitHub 50eb42bf46 LICENSE file added (#579) 2022-11-02 12:23:35 -05:00
Andrea Bernicchiaandsteve-chavez 8a98b4ed26 Update heroku installation docs
* Heroku installation docs aligned with https://github.com/PostgREST/postgrest-heroku/pull/40
* Update install.rst
2022-10-27 16:38:25 -05:00
dependabot[bot]andSteve Chavez 73f759ef3a Bump cachix/install-nix-action from 17 to 18
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 17 to 18.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v17...v18)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-17 21:06:34 -05:00
Laurence IslaandGitHub fdb975c87c Add information on preflight requests in CORS subsection 2022-09-21 19:09:50 -05:00
Laurence IslaandSteve Chavez d507f2f0f4 Shorten explanations for the working with types section 2022-09-19 21:55:31 -05:00
mdr1384andSteve Chavez 022101ae3c Fix example code in computed relationships
Looks like a copy-paste error - the `films.id` column should be associated with the `premieres.film_id` column.
2022-09-15 18:33:14 -05:00
319 changed files with 25459 additions and 12648 deletions
+42
View File
@@ -0,0 +1,42 @@
freebsd_instance:
image_family: freebsd-14-0
build_task:
# Don't change this name without adjusting .github/workflows/build.yaml
name: Build FreeBSD (Stack)
install_script: pkg install -y postgresql16-client hs-stack git
only_if: |
$CIRRUS_TAG != '' || $CIRRUS_BRANCH == 'main' || $CIRRUS_BRANCH =~ 'v*' ||
changesInclude(
'.github/workflows/build.yaml',
'.github/actions/artifact-from-cirrus/**',
'.cirrus.yml',
'postgrest.cabal',
'stack.yaml*',
'**.hs'
)
stack_cache:
folders: /.stack
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
stack_work_cache:
folders: .stack-work
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
- find main src -type f -iname '*.hs' -exec md5sum "{}" +
build_script: |
stack build -j 1 --local-bin-path . --copy-bins --stack-yaml stack-21.7.yaml
strip postgrest
bin_artifacts:
path: postgrest
+14
View File
@@ -3,4 +3,18 @@ When submitting a new feature or fix:
- Add a new entry to the CHANGELOG - https://github.com/PostgREST/postgrest/blob/main/CHANGELOG.md#unreleased
- If relevant, update the docs
- Use a prefix for the PR title or commits, e.g. "fix: description of the fix".
+ `fix`, bug fixes
+ `feat`, new features added
+ `perf`, performance improvements
+ `docs`, updating the documentation
+ `nix`, related to the Nix development environment
+ `ci`, related to the Continuous Integration modules
+ `test`, related to the testing modules
+ `refactor`, refactoring code
+ `deprecate`, deprecating a feature
+ `changelog`, updating the CHANGELOG
+ `chore`, maintenance (build process, updating sponsors, etc.)
+ Other prefixes may be used if necessary
- If there's a breaking change, add `BREAKING CHANGE` and an explanation to your commit message
-->
@@ -0,0 +1,119 @@
name: Artifact from Cirrus
description: Waits for a specific Cirrus CI run to complete, then downloads the artifact and uploads it to the current workflow. This will silently succeed if Cirrus CI did not schedule a task within 2 minutes.
inputs:
download:
description: Name of Artifact to download from Cirrus CI
required: true
task:
description: Name of Cirrus Task
required: true
token:
description: GitHub Token
required: true
upload:
description: Name of Artifact to upload on GitHub Actions
required: true
runs:
using: composite
steps:
- shell: bash
run: echo "GH_TOKEN=${{ inputs.token }}" >> "$GITHUB_ENV"
- name: Wait for Check Suite to be created
id: check-suite
env:
# GITHUB_SHA does weird things for pull request, so we roll our own:
COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
shell: bash
run: |
get_check_runs_url() {
gh api "repos/{owner}/{repo}/commits/${COMMIT}/check-suites" \
| jq -r '.check_suites[] | select(.app.slug == "cirrus-ci") | .check_runs_url'
}
for _ in $(seq 1 12); do
check_runs_url="$(get_check_runs_url)"
if [ -z "$check_runs_url" ]; then
echo "Cirrus CI task has not started, yet. Waiting..."
sleep 10
else
echo "check_runs_url=$check_runs_url" >> "$GITHUB_OUTPUT"
exit 0
fi
done
>&2 echo "Cirrus CI check suite not found. Is Cirrus CI enabled for this repo?"
- name: Find task by name
id: find-task
if: steps.check-suite.outputs.check_runs_url
shell: bash
run: |
get_number_of_tasks() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -r '.check_runs | map(select(.name == "${{ inputs.task }}")) | length'
}
tasks="$(get_number_of_tasks)"
case "$tasks" in
0)
echo "Task not found, assuming it's skipped intentionally..."
exit 0
;;
1)
echo "task_found=1" >> "$GITHUB_OUTPUT"
exit 0
;;
*)
>&2 echo "More than 1 task with the same name found. Don't know what to do..."
exit 1
;;
esac
- name: Wait for Cirrus CI to complete task
if: steps.find-task.outputs.task_found
shell: bash
run: |
get_conclusion() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -r '.check_runs[] | select(.name == "${{ inputs.task }}" and .status == "completed") | .conclusion'
}
while true; do
conclusion="$(get_conclusion)"
if [ -z "$conclusion" ]; then
echo "Cirrus CI task has not completed, yet. Waiting..."
sleep 30
else
if [ "$conclusion" == "success" ]; then
break
else
exit 1
fi
fi
done
- name: Download artifact from Cirrus CI
if: steps.find-task.outputs.task_found
id: download
shell: bash
run: |
get_external_id() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -er '.check_runs[] | select(.name == "${{ inputs.task }}") | .external_id'
}
archive="$(mktemp)"
artifacts="$(mktemp -d)"
until curl --no-progress-meter --fail -o "${archive}" \
"https://api.cirrus-ci.com/v1/artifact/task/$(get_external_id)/${{ inputs.download }}.zip"
do
# This happens when a tag is pushed on the same commit. In this case the
# job is immediately marked as "completed" for us, so we end up here after a few
# seconds - but the actual Cirrus CI task is still running and didn't produce its artifact, yet.
echo "Artifact not found on Cirrus CI, yet. Waiting..."
sleep 30
done
unzip "${archive}" -d "${artifacts}"
echo "artifacts=${artifacts}" >> "$GITHUB_OUTPUT"
- name: Save artifact to GitHub Actions
if: steps.find-task.outputs.task_found
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ inputs.upload }}
path: ${{ steps.download.outputs.artifacts }}
if-no-files-found: error
+35
View File
@@ -0,0 +1,35 @@
name: Cache on main
description: Stores caches on main and release branches only, but restores them on all branches.
inputs:
path:
description: Path(s) to cache
required: true
save-prs:
description: Whether to additionally store the cache in a pull request, too. Should only be used for very small caches.
type: boolean
prefix:
description: Cache key prefix to be used in both primary key and restore-keys.
required: true
suffix:
description: Cache key suffix to be used only in primary key.
required: true
runs:
using: composite
steps:
- uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with:
path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: |
${{ runner.os }}-${{ inputs.prefix }}-
- uses: actions/cache/restore@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with:
path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: |
${{ runner.os }}-${{ inputs.prefix }}-
+25
View File
@@ -0,0 +1,25 @@
name: Setup Nix
description: Installs nix, sets up cachix and installs a subset of tooling.
inputs:
authToken:
description: Token to pass to cachix
tools:
description: Tools to install with nix-env -iA <tools>
runs:
using: composite
steps:
- uses: nixbuild/nix-quick-install-action@60e9c39264d4714139af3cdf15f691b19eec3530 # v28
with:
nix_conf: |-
always-allow-substitutes = true
max-jobs = auto
- uses: cachix/cachix-action@ad2ddac53f961de1989924296a1f236fcfbaa4fc # v15
with:
name: postgrest
authToken: ${{ inputs.authToken }}
- if: ${{ inputs.tools }}
run: nix-env -f default.nix -iA ${{ inputs.tools }}
shell: bash
+18
View File
@@ -0,0 +1,18 @@
codecov:
branch: main
require_ci_to_pass: false
comment: false
coverage:
status:
project:
default:
target: auto
threshold: 1%
only_pulls: false
patch:
default:
target: auto
threshold: 1%
only_pulls: true
+25
View File
@@ -0,0 +1,25 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:best-practices"
],
"baseBranches": [
"main",
"/^v[0-9]+/"
],
"rebaseWhen": "conflicted",
"packageRules": [
{
"matchBaseBranches": [ "/^v[0-9]+/" ],
"groupName": "all dependencies"
},
{
"matchPackageNames": ["docutils"],
"allowedVersions": "<0.21.0"
},
{
"matchPackageNames": ["macos"],
"allowedVersions": "<13"
}
]
}
+72
View File
@@ -0,0 +1,72 @@
#!/bin/bash
set -Eeuo pipefail
# This script builds PostgREST in a remote ARM server
[ -z "$1" ] && { echo "Missing 1st argument: PostgREST github commit SHA"; exit 1; }
[ -z "$2" ] && { echo "Missing 2nd argument: Build environment directory name"; exit 1; }
[ -z "$3" ] && { echo "Missing 3rd argument: GHC version"; exit 1; }
PGRST_GITHUB_COMMIT="$1"
SCRIPT_DIR="$2"
DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"
# latest is a shortcut documented on https://www.haskell.org/ghcup/guide/#tags-and-shortcuts
CABAL_VERSION="latest"
GHC_VERSION="$3"
install_packages() {
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install -y git build-essential curl libffi-dev libffi7 libgmp-dev libgmp10 libncurses-dev libncurses5 libtinfo5 llvm libnuma-dev zlib1g-dev libpq-dev jq gcc
sudo apt-get clean
}
install_ghcup() {
export BOOTSTRAP_HASKELL_NONINTERACTIVE=1
export BOOTSTRAP_HASKELL_MINIMAL=1
curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh
source ~/.ghcup/env
}
install_cabal() {
ghcup upgrade
ghcup install cabal $CABAL_VERSION
ghcup set cabal $CABAL_VERSION
}
install_ghc() {
ghcup upgrade
ghcup install ghc $GHC_VERSION
ghcup set ghc $GHC_VERSION
}
install_packages
# Add ghcup to the PATH for this session
[ -f ~/.ghcup/env ] && source ~/.ghcup/env
ghcup --version || install_ghcup
ghcup set cabal $CABAL_VERSION || install_cabal
ghcup set ghc $GHC_VERSION || install_ghc
cd ~/$SCRIPT_DIR
# Clone the repository and build the project
git clone https://github.com/PostgREST/postgrest.git
cd postgrest
git checkout $PGRST_GITHUB_COMMIT
cabal v2-update && cabal v2-build
# Strip unused symbols from executable
PGRST_BIN=$(cabal exec which postgrest | tail -1)
strip $PGRST_BIN
# Copy the built binary to the Dockerfile directory
cp $PGRST_BIN ~/$DOCKER_BUILD_DIR
# Move and compress the built binary
mkdir -p ~/$SCRIPT_DIR/result
mv $PGRST_BIN ~/$SCRIPT_DIR/result
cd ~/$SCRIPT_DIR
tar -cJf result.tar.xz result
+18
View File
@@ -0,0 +1,18 @@
# PostgREST docker hub image
FROM ubuntu:noble@sha256:2e863c44b718727c860746568e1d54afd13b2fa71b160f5cd9058fc436217b30 AS postgrest
RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY postgrest /usr/bin/postgrest
EXPOSE 3000
USER 1000
# Use the array form to avoid running the command using bash, which does not handle `SIGTERM` properly.
# See https://docs.docker.com/compose/faq/#why-do-my-services-take-10-seconds-to-recreate-or-stop
CMD ["postgrest"]
+50
View File
@@ -0,0 +1,50 @@
#!/bin/bash
set -Eeuo pipefail
# This script publishes the Docker ARM images to Docker Hub.
[ -z "$1" ] && { echo "Missing 1st argument: PostgREST github commit SHA"; exit 1; }
[ -z "$2" ] && { echo "Missing 2nd argument: Docker repo"; exit 1; }
[ -z "$3" ] && { echo "Missing 3rd argument: Docker username"; exit 1; }
[ -z "$4" ] && { echo "Missing 4th argument: Docker password"; exit 1; }
[ -z "$5" ] && { echo "Missing 5th argument: Build environment directory name"; exit 1; }
[ -z "$6" ] && { echo "Missing 6th argument: PostgREST version"; exit 1; }
PGRST_GITHUB_COMMIT="$1"
DOCKER_REPO="$2"
DOCKER_USER="$3"
DOCKER_PASS="$4"
SCRIPT_DIR="$5"
PGRST_VERSION="$6"
DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"
clean_env()
{
sudo docker logout
}
# Login to Docker
sudo docker logout
{ echo $DOCKER_PASS | sudo docker login -u $DOCKER_USER --password-stdin; } || { echo "Couldn't login to docker"; exit 1; }
trap clean_env sigint sigterm exit
# Move to the docker build environment
cd ~/$DOCKER_BUILD_DIR
# Push final images to Docker hub
# NOTE: This command publishes a separate ARM image because the builds cannot
# be added to the manifest if they are not in the registry beforehand.
# This image must be manually deleted from Docker Hub at the end of the process.
sudo docker buildx build --build-arg PGRST_GITHUB_COMMIT=$PGRST_GITHUB_COMMIT \
-t $DOCKER_REPO/postgrest:$PGRST_VERSION-arm \
--push .
# Add the arm images to the manifest
# NOTE: This assumes that there already is a `postgrest:<version>` image
# for the amd64 architecture pushed to Docker Hub
sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:$PGRST_VERSION $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
[ "$PGRST_VERSION" != "devel" ] && sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:latest $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
sudo docker logout
+203
View File
@@ -0,0 +1,203 @@
name: Build
on:
workflow_call:
secrets:
CACHIX_AUTH_TOKEN:
required: false
pull_request:
branches:
- main
- v[0-9]+
paths:
- .github/workflows/build.yaml
- .github/actions/**
- .github/scripts/**
- .github/*
- '*.nix'
- nix/**
- .cirrus.yml
- cabal.project*
- postgrest.cabal
- stack.yaml*
- '**.hs'
- '!**.md'
concurrency:
# Terminate all previous runs of the same workflow for pull requests
group: build-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
static:
name: Nix - Linux static
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Build static executable
run: nix-build -A postgrestStatic
- name: Save built executable as artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-linux-static-x64
path: result/bin/postgrest
if-no-files-found: error
- name: Build Docker image
run: nix-build -A docker.image --out-link postgrest-docker.tar.gz
- name: Save built Docker image as artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-docker-x64
path: postgrest-docker.tar.gz
if-no-files-found: error
macos:
name: Nix - MacOS
runs-on: macos-12
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Install gnu sed
run: brew install gnu-sed
- name: Build everything
run: |
# The --dry-run will give us a list of derivations to download from cachix and
# derivations to build. We only take those that would have to be built and then build
# those explicitly. This has the advantage that pure verification will not include
# a download anymore, making it much faster. If something needs to be built, only
# the dependencies required to do so will be downloaded, but not everything.
nix-build --dry-run 2>&1 \
| gsed -e '1,/derivations will be built:$/d' -e '/paths will be fetched/Q' \
| xargs nix-build
stack:
strategy:
fail-fast: false
matrix:
include:
- name: Linux
runs-on: ubuntu-22.04
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
# no artifact for Linux, because we use the static build
- name: MacOS
runs-on: macos-12
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-macos-x64
- name: Windows
runs-on: windows-2022
cache: |
~\AppData\Roaming\stack\pantry
~\AppData\Local\Programs\stack\pantry
~\AppData\Roaming\stack\snapshots
~\AppData\Local\Programs\stack\snapshots
~\AppData\Roaming\stack\stack.sqlite3
~\AppData\Local\Programs\stack\stack.sqlite3
deps: Add-Content $env:GITHUB_PATH $env:PGBIN
artifact: postgrest-windows-x64
name: Stack - ${{ matrix.name }}
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: haskell-actions/setup@ec49483bfc012387b227434aba94f59a6ecd0900 # v2.7.5
with:
# This must match the version in stack.yaml's resolver
ghc-version: 9.6.5
enable-stack: true
stack-no-global: true
stack-setup-ghc: true
- name: Cache ~/.stack
uses: ./.github/actions/cache-on-main
with:
path: ${{ matrix.cache }}
prefix: stack
suffix: ${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
- name: Cache .stack-work
uses: ./.github/actions/cache-on-main
with:
path: .stack-work
save-prs: true
prefix: stack-work-${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
suffix: ${{ hashFiles('main/**/*.hs', 'src/**/*.hs') }}
- name: Install dependencies
if: matrix.deps
run: ${{ matrix.deps }}
- name: Build with Stack
run: stack build --lock-file error-on-write --local-bin-path result --copy-bins
- name: Strip Executable
run: strip result/postgrest*
- name: Save built executable as artifact
if: matrix.artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ matrix.artifact }}
path: |
result/postgrest
result/postgrest.exe
if-no-files-found: error
freebsd:
name: Stack - FreeBSD from CirrusCI
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: ./.github/actions/artifact-from-cirrus
with:
token: ${{ github.token }}
task: Build FreeBSD (Stack)
download: bin
upload: postgrest-freebsd-x64
cabal:
strategy:
matrix:
ghc: ['9.6.4', '9.8.2']
fail-fast: false
name: Cabal - Linux GHC ${{ matrix.ghc }}
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: haskell-actions/setup@ec49483bfc012387b227434aba94f59a6ecd0900 # v2.7.5
with:
ghc-version: ${{ matrix.ghc }}
- name: Cache .cabal
uses: ./.github/actions/cache-on-main
with:
path: |
~/.cabal/packages
~/.cabal/store
prefix: cabal-${{ matrix.ghc }}
suffix: ${{ hashFiles('postgrest.cabal', 'cabal.project', 'cabal.project.freeze') }}
- name: Cache dist-newstyle
uses: ./.github/actions/cache-on-main
with:
path: dist-newstyle
save-prs: true
prefix: cabal-${{ matrix.ghc }}-dist-newstyle-${{ hashFiles('postgrest.cabal', 'cabal.project', 'cabal.project.freeze') }}
suffix: ${{ hashFiles('**/*.hs') }}
- name: Install dependencies
run: cabal build --only-dependencies --enable-tests --enable-benchmarks
- name: Build
run: cabal build --enable-tests --enable-benchmarks all
+32
View File
@@ -0,0 +1,32 @@
name: Check
on:
workflow_call:
secrets:
CACHIX_AUTH_TOKEN:
required: false
pull_request:
branches:
- main
- v[0-9]+
concurrency:
# Terminate all previous runs of the same workflow for pull requests
group: style-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
lint-style:
name: Lint & Style
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: style.lint.bin style.styleCheck.bin
- name: Run linter (check locally with `nix-shell --run postgrest-lint`)
run: postgrest-lint
- name: Run style check (auto-format with `nix-shell --run postgrest-style`)
run: postgrest-style-check
+362
View File
@@ -0,0 +1,362 @@
name: CI
on:
push:
branches:
- main
- v[0-9]+
tags:
- devel
- v*
concurrency:
# Terminate all previous runs of the same workflow for the same tag, but never
# cancel for the main / release branch pipelines.
group: ci-${{ (github.ref_type == 'tag' && github.ref) || github.run_id }}
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
jobs:
check:
name: Check
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/check.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
docs:
name: Docs
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/docs.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
test:
name: Test
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/test.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
build:
name: Build
uses: ./.github/workflows/build.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
arm:
name: Build / Cabal - aarch64 GHC 9.4.8
if: vars.SSH_ARM_ENABLED
runs-on: ubuntu-22.04
outputs:
remotepath: ${{ steps.Remote-Dir.outputs.remotepath }}
env:
GITHUB_COMMIT: ${{ github.sha }}
GHC_VERSION: '9.4.8'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- id: Remote-Dir
name: Unique directory name for the remote build
run: echo "remotepath=postgrest-build-$(uuidgen)" >> "$GITHUB_OUTPUT"
- name: Copy script files to the remote server
uses: appleboy/scp-action@master
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
source: ".github/scripts/arm/*"
target: ${{ steps.Remote-Dir.outputs.remotepath }}
strip_components: 3
- name: Build ARM
uses: appleboy/ssh-action@master
env:
REMOTE_DIR: ${{ steps.Remote-Dir.outputs.remotepath }}
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
command_timeout: 120m
script_stop: true
envs: GITHUB_COMMIT,REMOTE_DIR,GHC_VERSION
script: bash ~/$REMOTE_DIR/build.sh "$GITHUB_COMMIT" "$REMOTE_DIR" "$GHC_VERSION"
- name: Download binaries from remote server
uses: nicklasfrahm/scp-action@main
with:
direction: download
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
source: "${{ steps.Remote-Dir.outputs.remotepath }}/result.tar.xz"
target: "result.tar.xz"
- name: Extract downloaded binaries
run: tar -xvf result.tar.xz && rm result.tar.xz
- name: Save aarch64 executable as artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-ubuntu-aarch64
path: result/postgrest
if-no-files-found: error
tag:
name: Release / Tag
concurrency:
# Never tag outdated commits on the main branch by skipping superseded commits
group: ci-tag-${{ (github.ref == 'refs/heads/main' && github.ref) || github.run_id }}
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
if: |
vars.RELEASE_ENABLED &&
startsWith(github.ref, 'refs/heads/') &&
needs.docs.result == 'success' &&
needs.test.result == 'success' &&
needs.build.result == 'success' &&
(needs.arm.result == 'skipped' || success())
permissions:
contents: write
runs-on: ubuntu-22.04
needs:
- docs
- test
- build
- arm
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ssh-key: ${{ secrets.POSTGREST_SSH_KEY }}
- name: Tag latest commit
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
if [[ "$cabal_version" == *.*.* ]]; then
git fetch --tags
if [ -z "$(git tag --list "v$cabal_version")" ]; then
git tag "v$cabal_version"
git push origin "v$cabal_version"
fi
else
git tag -f "devel"
git push -f origin "devel"
fi
prepare:
name: Release / Prepare
if: |
startsWith(github.ref, 'refs/tags/') &&
needs.build.result == 'success' &&
(needs.arm.result == 'skipped' || success())
runs-on: ubuntu-22.04
needs:
- build
- arm
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
if [ "${GITHUB_REF_NAME}" != "devel" ] && [ "${GITHUB_REF_NAME}" != "v$cabal_version" ]; then
echo "Tagged version ($GITHUB_REF_NAME) does not match the one in postgrest.cabal (v$cabal_version). Aborting release..."
exit 1
fi
- name: Identify changes from CHANGELOG.md
run: |
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
echo "Getting unreleased changes..."
sed -n "1,/## Unreleased/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
else
version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
echo "Propper release, getting changes for version $version ..."
sed -n "1,/## \[$version\]/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
fi
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Save CHANGES.md as artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: release-changes
path: CHANGES.md
if-no-files-found: error
github:
name: Release / GitHub
permissions:
contents: write
runs-on: ubuntu-22.04
needs:
- prepare
if: success() || needs.prepare.result == 'success'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Download all artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: artifacts
- name: Create release bundle with archives for all builds
run: |
find artifacts -type f -iname postgrest -exec chmod +x {} \;
mkdir -p release-bundle
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-x64.tar.xz" \
-C artifacts/postgrest-linux-static-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-x64.tar.xz" \
-C artifacts/postgrest-macos-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-freebsd-x64.tar.xz" \
-C artifacts/postgrest-freebsd-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-ubuntu-aarch64.tar.xz" \
-C artifacts/postgrest-ubuntu-aarch64 postgrest
zip "release-bundle/postgrest-${GITHUB_REF_NAME}-windows-x64.zip" \
artifacts/postgrest-windows-x64/postgrest.exe
- name: Save release bundle
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: release-bundle
path: release-bundle
if-no-files-found: error
- name: Publish release on GitHub
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "Releasing version ${GITHUB_REF_NAME} on GitHub..."
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
# To replace the existing release, we must first delete the old assets,
# then modify the release, then add the new assets.
gh release view devel --json assets \
| jq -r '.assets[] | .name' \
| xargs -rn1 \
gh release delete-asset -y devel
gh release edit devel \
-t devel \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
--prerelease
gh release upload --clobber devel release-bundle/*
else
gh release create "${GITHUB_REF_NAME}" \
-t "${GITHUB_REF_NAME}" \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
release-bundle/*
fi
docker:
name: Release / Docker Hub
runs-on: ubuntu-22.04
needs:
- prepare
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
(success() || needs.prepare.result == 'success')
env:
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Download Docker image
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: postgrest-docker-x64
- name: Publish images on Docker Hub
run: |
docker login -u "$DOCKER_USER" -p "$DOCKER_PASS"
docker load -i postgrest-docker.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
# Only tag 'latest' for full releases
if [ "${GITHUB_REF_NAME}" != "devel" ]; then
echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..."
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
docker push "$DOCKER_REPO"/postgrest:latest
else
echo "Skipping push to 'latest' tag for pre-release..."
fi
docker-description:
name: Release / Docker Hub Description
runs-on: ubuntu-22.04
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
github.ref == 'refs/tags/devel'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: peter-evans/dockerhub-description@e98e4d1628a5f3be2be7c231e50981aee98723ae # v4.0.0
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
repository: ${{ vars.DOCKER_REPO }}/postgrest
short-description: ${{ github.event.repository.description }}
readme-filepath: ./docker-hub-readme.md
docker-arm:
name: Release / Docker Hub Arm
runs-on: ubuntu-22.04
needs:
- arm
- docker
env:
GITHUB_COMMIT: ${{ github.sha }}
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Publish images for ARM builds on Docker Hub
uses: appleboy/ssh-action@master
env:
REMOTE_DIR: ${{ needs.arm.outputs.remotepath }}
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
script_stop: true
envs: GITHUB_COMMIT,DOCKER_REPO,DOCKER_USER,DOCKER_PASS,REMOTE_DIR,GITHUB_REF_NAME
script: bash ~/$REMOTE_DIR/docker-publish.sh "$GITHUB_COMMIT" "$DOCKER_REPO" "$DOCKER_USER" "$DOCKER_PASS" "$REMOTE_DIR" "$GITHUB_REF_NAME"
clean-arm:
name: Build / Cleanup
needs:
- arm
- docker-arm
if: ${{ always() && vars.SSH_ARM_ENABLED }}
runs-on: ubuntu-22.04
env:
REMOTE_DIR: ${{ needs.arm.outputs.remotepath }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Remove uploaded files from server
uses: appleboy/ssh-action@master
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
envs: REMOTE_DIR
script: rm -rf $REMOTE_DIR
+48 -31
View File
@@ -1,50 +1,67 @@
name: Docs
on:
push:
branches:
- main
- v[0-9]+
workflow_call:
secrets:
CACHIX_AUTH_TOKEN:
required: false
pull_request:
branches:
- main
- v[0-9]+
- main
- v[0-9]+
paths:
- .github/workflows/docs.yaml
- .github/actions/setup-nix/**
- default.nix
- nix/**
- docs/**
- '!**.md'
concurrency:
# Terminate all previous runs of the same workflow for pull requests
group: docs-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
build:
name: Build docs
runs-on: ubuntu-24.04
name: Build
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31
- run: nix-env -f docs/default.nix -iA build
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.build.bin
- run: postgrest-docs-build
- run: git diff --exit-code HEAD locales || echo "Please commit changes to the locales/ folder after running postgrest-docs-build."
spellcheck:
name: Run spellcheck
runs-on: ubuntu-24.04
name: Spellcheck
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31
- run: nix-env -f docs/default.nix -iA spellcheck
- run: postgrest-docs-spellcheck
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.spellcheck.bin docs.dictcheck.bin
- name: Run spellcheck
run: postgrest-docs-spellcheck
- name: Run dictcheck
run: postgrest-docs-dictcheck
dictcheck:
name: Run dictcheck
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31
- run: nix-env -f docs/default.nix -iA dictcheck
- run: postgrest-docs-dictcheck
linkcheck:
name: Run linkcheck
name: Linkcheck
if: github.base_ref == 'main'
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31
- run: nix-env -f docs/default.nix -iA linkcheck
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.linkcheck.bin
- run: postgrest-docs-linkcheck
+39
View File
@@ -0,0 +1,39 @@
name: Upload Reports
on:
workflow_run:
workflows:
- CI
- Test
branches-ignore:
- devel
- v[0-9]+
- v[0-9]+.[0-9]+.[0-9]+
types:
- completed
jobs:
loadtest:
name: Loadtest
permissions:
checks: write
runs-on: ubuntu-22.04
if: github.event.workflow_run.conclusion == 'success' || github.event.workflow_run.conclusion == 'failure'
steps:
- name: Download from Artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
github-token: ${{ github.token }}
run-id: ${{ github.event.workflow_run.id }}
name: loadtest.md
path: artifacts
- name: Upload to GitHub Checks
uses: LouisBrunner/checks-action@6b626ffbad7cc56fd58627f774b9067e6118af23 # v2.0.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
sha: ${{ github.event.workflow_run.head_sha }}
name: Loadtest Results
conclusion: neutral
output: |
{"summary":""}
output_text_description_file: artifacts/loadtest.md
+138
View File
@@ -0,0 +1,138 @@
name: Test
on:
workflow_call:
secrets:
CACHIX_AUTH_TOKEN:
required: false
CODECOV_TOKEN:
required: false
pull_request:
branches:
- main
- v[0-9]+
paths:
- .github/workflows/test.yaml
- .github/workflows/report.yaml
- .github/actions/setup-nix/**
- default.nix
- nix/**
- .stylish-haskell.yaml
- cabal.project
- postgrest.cabal
- '**.hs'
- test/**
- '!**.md'
concurrency:
# Terminate all previous runs of the same workflow for pull requests
group: test-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
coverage:
name: Coverage
runs-on: ubuntu-22.04
defaults:
run:
# Hack for enabling color output, see:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: tests.coverage.bin tests.testDoctests.bin tests.testSpecIdempotence.bin
- name: Run coverage (IO tests and Spec tests against PostgreSQL 15)
run: postgrest-coverage
- name: Upload coverage to codecov
uses: codecov/codecov-action@e28ff129e5465c2c0dcc6f003fc735cb6ae0c673 # v4.5.0
with:
files: ./coverage/codecov.json
token: ${{ secrets.CODECOV_TOKEN }}
- name: Run doctests
if: always()
run: postgrest-test-doctests
- name: Check the spec tests for idempotence
if: always()
run: postgrest-test-spec-idempotence
postgres:
strategy:
fail-fast: false
matrix:
pgVersion: ["9_6", 10, 11, 12, 13, 14, 15, 16]
name: PG ${{ matrix.pgVersion }}
runs-on: ubuntu-22.04
defaults:
run:
# Hack for enabling color output, see:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: tests.testSpec.bin tests.testIO.bin tests.testBigSchema.bin withTools.postgresql-${{ matrix.pgVersion }}.bin
- name: Run spec tests
if: always()
run: postgrest-with-postgresql-${{ matrix.pgVersion }} postgrest-test-spec
- name: Run IO tests
if: always()
run: postgrest-with-postgresql-${{ matrix.pgVersion }} postgrest-test-io -vv
- name: Run IO tests on a big schema
if: always()
run: postgrest-with-postgresql-${{ matrix.pgVersion }} postgrest-test-big-schema -vv
memory:
name: Memory
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: memory.test.bin
- name: Run memory tests
run: postgrest-test-memory
loadtest:
name: Loadtest
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
fetch-depth: 0
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: loadtest.loadtestAgainst.bin loadtest.report.bin
- uses: WyriHaximus/github-action-get-previous-tag@04e8485ecb6487243907e330d522ff60f02283ce # v1.4.0
id: get-latest-tag
with:
prefix: v
- name: Run loadtest
run: |
postgrest-loadtest-against main ${{ steps.get-latest-tag.outputs.tag }}
postgrest-loadtest-report > loadtest/loadtest.md
- name: Upload report
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: loadtest.md
path: loadtest/loadtest.md
if-no-files-found: error
-70
View File
@@ -1,70 +0,0 @@
# Architecture
This document describes the high-level architecture of PostgREST.
## Bird's Eye View
```haskell
postgrest :: Request -> Either Error SQLStatement -> Response
```
On the highest level, PostgREST processes an HTTP request, if it's accepted it builds a SQL statement for it, executes it, and produces a response.
## Code Map
This section talks briefly about various important modules.
The starting point of the program is `main/Main.hs`, which calls `src/PostgREST/CLI.hs` which then calls `src/PostgREST/App.hs`.
`App.hs` is then in charge of composing the different modules.
### ApiRequest.hs
PostgREST operates over two types of resources: database relations(tables or views) and database functions; providing different representations(depending on the media type)
for them.
This module is in charge of representing the operation over an `ApiRequest` type. It parses the URL querystring following PostgREST syntax, the request headers, and the request body
(if possible it avoids parsing the body and sends it directly to the db).
A request might be rejected at this level if it's invalid, e.g. providing an unknown media type to PostgREST or using an unknown HTTP method.
### Plan.hs
Using the Schema Cache, this module enables more complex functionality(like resource embedding) by enriching the ApiRequest. It generates Plan types(`ReadPlan`, `MutatePlan`)
that then will be used to generate a SQL statement.
A request might be rejected at this level if it's invalid, e.g. by doing resource embedding on a nonexistent resource.
An OPTIONS request doesn't require a plan to be generated.
### Query.hs
This module constructs single SQL statements that can be parametrized and prepared. Only at this stage a PostgreSQL connection from the pool is used.
A query might fail(and be rollbacked) at this level if it doesn't comply to certain conditions, e.g. by not returning a single row when a ``Accept: application/vnd.pgrst.object`` header is specified.
An OPTIONS request doesn't require a query to be executed.
### Response.hs
This module constructs the HTTP response body with the right headers.
It builds the OpenAPI response using the schema cache.
### Auth.hs
This module provides functions to deal with JWT authorization.
### Workers.hs
This spawns threads which are used to execute concurrent jobs.
Jobs include connection recovery, a listener for the PostgreSQL LISTEN command, and an admin server.
### SchemaCache.hs
This queries the PostgreSQL system catalogs and caches the metadata into a SchemaCache type,
### AppState.hs
The state of the App which is kept across requests.
+27 -10
View File
@@ -4,30 +4,30 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
## Sponsors
<table>
<table align="center">
<tbody>
<tr>
<td align="center" valign="middle">
<a href="https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest" target="_blank">
<img width="222px" src="static/cybertec-new.png">
<img width="296px" src="static/cybertec.svg">
</a>
</td>
<td align="center" valign="middle">
<a href="https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo" target="_blank">
<img width="296px" src="static/2ndquadrant.png">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tryretool.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/retool.png">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/neon.jpg">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/gnuhost.png">
<a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/code-build.png">
</a>
</td>
<td align="center" valign="middle">
@@ -36,8 +36,8 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
</a>
</td>
<td align="center" valign="middle">
<a href="https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/oblivious.jpg">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
</a>
</td>
</tr>
@@ -46,12 +46,14 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
## Lead Backers
- [Roboflow](https://github.com/roboflow)
- Evans Fernandes
- [Jan Sommer](https://github.com/nerfpops)
- [Franz Gusenbauer](https://www.igutech.at/)
## Backers
- Zac Miller
- Tsingson Qin
- Michel Pelletier
- Jay Hannah
@@ -76,6 +78,21 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
<img width="222px" src="static/timescaledb.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tryretool.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img max-width="222px" height="88" src="static/retool.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo" target="_blank">
<img width="222px" src="static/2ndquadrant.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="222px" src="static/oblivious.jpg">
</a>
</td>
</tr>
</tbody>
</table>
+305
View File
@@ -3,6 +3,305 @@
All notable changes to this project will be documented in this file.
This project adheres to [Semantic Versioning](http://semver.org/).
## Unreleased
## [12.2.3] - 2024-08-01
### Fixed
- #3091, Broken link in OpenAPI description `externalDocs` - @salim-b
- #3659, Embed One-to-One relationship with different column order properly - @wolfgangwalther
- #3504, Remove `format` from `rowFilter` parameters in OpenAPI - @dantheman2865
- #3660, Fix regression that loaded the schema cache before the in-database configuration - @steve-chavez, @laurenceisla
## [12.2.2] - 2024-07-10
### Fixed
- #3093, Nested empty embeds no longer show empty values and are correctly omitted - @laurenceisla
- #3644, Make --dump-schema work with in-database pgrst.db_schemas setting - @wolfgangwalther
- #3644, Show number of timezones in schema cache load report - @wolfgangwalther
- #3644, List correct enum options in OpenApi output when multiple types with same name are present - @wolfgangwalther
- #3523, Fix schema cache loading retry without backoff - @steve-chavez
## [12.2.1] - 2024-06-27
### Fixed
- #3147, Don't reload schema cache on every listener failure - @steve-chavez
### Documentation
- #3592, Architecture diagram now supports dark mode and has links - @laurenceisla
- #3616, The schema isolation diagram now supports dark mode and uses well-known schemas - @laurenceisla
## [12.2.0] - 2024-06-11
### Added
- #2887, Add Preference `max-affected` to limit affected resources - @taimoorzaeem
- #3171, Add an ability to dump config via admin API - @skywriter
- #3171, #3046, Log schema cache stats to stderr - @steve-chavez
- #3210, Dump schema cache through admin API - @taimoorzaeem
- #2676, Performance improvement on bulk json inserts, around 10% increase on requests per second by removing `json_typeof` from write queries - @steve-chavez
- #3435, Add log-level=debug, for development purposes - @steve-chavez
- #1526, Add `/metrics` endpoint on admin server - @steve-chavez
- Exposes connection pool metrics, schema cache metrics
- #3404, Show the failed MESSAGE or DETAIL in the `details` field of the `PGRST121` (could not parse RAISE 'PGRST') error - @laurenceisla
- #3404, Show extra information in the `PGRST121` (could not parse RAISE 'PGRST') error - @laurenceisla
+ Shows the failed MESSAGE or DETAIL in the `details` field
+ Shows the correct JSON format in the `hints` field
- #3340, Log when the LISTEN channel gets a notification - @steve-chavez
- #3184, Log full pg version to stderr on connection - @steve-chavez
- #3242. Add config `db-hoisted-tx-settings` to apply only hoisted function settings - @taimoorzaeem
- #3214, #3229 Log connection pool events on log-level=debug - @steve-chavez, @laurenceisla
### Fixed
- #3237, Dump media handlers and timezones with --dump-schema - @wolfgangwalther
- #3323, #3324, Don't hide error on LISTEN channel failure - @steve-chavez
- #3330, Incorrect admin server `/ready` response on slow schema cache loads - @steve-chavez
- #3345, Fix in-database configuration values not loading for `pgrst.server_trace_header` and `pgrst.server_cors_allowed_origins` - @laurenceisla
- #3404, Clarify the `PGRST121` (could not parse RAISE 'PGRST') error message - @laurenceisla
- #3267, Fix wrong `503 Service Unavailable` on pg error `53400` - @taimoorzaeem
- #2985, Fix not adding `application_name` on all connection strings - @steve-chavez
- #3424, Admin `/live` and `/ready` now differentiates a failure as 500 status - @steve-chavez
+ 503 status is still given when postgREST is in a recovering state
- #3478, Media Types are parsed case insensitively - @develop7
- #3533, #3536, Fix listener silently failing on read replica - @steve-chavez
+ If the LISTEN connection fails, it's retried with exponential backoff
- #3414, Force listener to connect to read-write instances using `target_session_attrs` - @steve-chavez
- #3255, Fix incorrect `413 Request Entity Too Large` on pg errors `54*` - @taimoorzaeem
- #3549, Remove verbosity from error logs starting with "An error occurred..." and replacing it with "Failed to..." - @laurenceisla
### Deprecated
- Support for PostgreSQL versions 9.6, 10 and 11 is deprecated. From this on version onwards, PostgREST will only support non-end-of-life PostgreSQL versions. See https://www.postgresql.org/support/versioning/.
- `Prefer: params=single-object` is deprecated. Use [a function with a single unnamed JSON parameter](https://postgrest.org/en/latest/references/api/functions.html#function-single-json) instead. - @steve-chavez
### Documentation
- #3289, Add dark mode. Can be toggled by a button in the bottom right corner. - @laurenceisla
- #3384, Add architecture diagram and documentation - @steve-chavez
## [12.0.3] - 2024-05-09
### Fixed
- #3149, Misleading "Starting PostgREST.." logs on schema cache reloading - @steve-chavez
- #3205, Fix wrong subquery error returning a status of 400 Bad Request - @steve-chavez
- #3224, Return status code 406 for non-accepted media type instead of code 415 - @wolfgangwalther
- #3160, Fix using select= query parameter for custom media type handlers - @wolfgangwalther
- #3361, Clarify PGRST204(column not found) error message - @steve-chavez
- #3373, Remove rejected mediatype `application/vnd.pgrst.object+json` from response - @taimoorzaeem
- #3418, Fix OpenAPI not tagging a FK column correctly on O2O relationships - @laurenceisla
- #3256, Fix wrong http status for pg error `42P17 infinite recursion` - @taimoorzaeem
## [12.0.2] - 2023-12-20
### Fixed
- #3124, Fix table's media type handlers not working for all schemas - @steve-chavez
- #3126, Fix empty row on media type handler function - @steve-chavez
## [12.0.1] - 2023-12-12
### Fixed
- #3054, Fix not allowing special characters in JSON keys - @laurenceisla
- #2344, Replace JSON parser error with a clearer generic message - @develop7
- #3100, Add missing in-database configuration option for `jwt-cache-max-lifetime` - @laurenceisla
- #3089, The any media type handler now sets `Content-Type: application/octet-stream` by default instead of `Content-Type: application/json` - @steve-chavez
## [12.0.0] - 2023-12-01
### Added
- #1614, Add `db-pool-automatic-recovery` configuration to disable connection retrying - @taimoorzaeem
- #2492, Allow full response control when raising exceptions - @taimoorzaeem, @laurenceisla
- #2771, #2983, #3062, #3055 Add `Server-Timing` response header - @taimoorzaeem, @develop7, @laurenceisla
- #2698, Add config `jwt-cache-max-lifetime` and implement JWT caching - @taimoorzaeem
- #2943, Add `handling=strict/lenient` for Prefer header - @taimoorzaeem
- #2441, Add config `server-cors-allowed-origins` to specify CORS origins - @taimoorzaeem
- #2825, SQL handlers for custom media types - @steve-chavez
+ Solves #1548, #2699, #2763, #2170, #1462, #1102, #1374, #2901
- #2799, Add timezone in Prefer header - @taimoorzaeem
- #3001, Add `statement_timeout` set on functions - @taimoorzaeem
- #3045, Apply superuser settings on impersonated roles if they have PostgreSQL 15 `GRANT SET ON PARAMETER` privilege - @steve-chavez
- #915, Add support for aggregate functions - @timabdulla
+ The aggregate functions SUM(), MAX(), MIN(), AVG(), and COUNT() are now supported.
+ It's disabled by default, you can enable it with `db-aggregates-enabled`.
- #3057, Log all internal database errors to stderr - @laurenceisla
### Fixed
- #3015, Fix unnecessary count() on RPC returning single - @steve-chavez
- #1070, Fix HTTP status responses for upserts - @taimoorzaeem
+ `PUT` returns `201` instead of `200` when rows are inserted
+ `POST` with `Prefer: resolution=merge-duplicates` returns `200` instead of `201` when no rows are inserted
- #3019, Transaction-Scoped Settings are now shown clearly in the Postgres logs - @laurenceisla
+ Shows `set_config('pgrst.setting_name', $1)` instead of `setconfig($1, $2)`
+ Does not apply to role settings and `app.settings.*`
- #2420, Fix bogus message when listening on port 0 - @develop7
- #3067, Fix Acquision Timeout errors logging to stderr when `log-level=crit` - @laurenceisla
### Changed
- Removed [raw-media-types config](https://postgrest.org/en/v11.1/references/configuration.html#raw-media-types) - @steve-chavez
- Removed `application/octet-stream`, `text/plain`, `text/xml` [builtin support for scalar results](https://postgrest.org/en/v11.1/references/api/resource_representation.html#scalar-function-response-format) - @steve-chavez
- Removed default `application/openapi+json` media type for [db-root-spec](https://postgrest.org/en/v11.1/references/configuration.html#db-root-spec) - @steve-chavez
- Removed [db-use-legacy-gucs](https://postgrest.org/en/v11.2/references/configuration.html#db-use-legacy-gucs) - @laurenceisla
+ All PostgreSQL versions now use GUCs in JSON format for [Headers, Cookies and JWT claims](https://postgrest.org/en/v12/references/transactions.html#request-headers-cookies-and-jwt-claims).
## [11.2.2] - 2023-10-25
### Fixed
- #2824, Fix regression by reverting fix that returned 206 when first position = length in a `Range` header - @laurenceisla, @strengthless
## [11.2.1] - 2023-10-03
### Fixed
- #2899, Fix `application/vnd.pgrst.array` not accepted as a valid mediatype - @taimoorzaeem
- #2524, Fix schema cache and configuration reloading with `NOTIFY` not working on Windows - @diogob, @laurenceisla
- #2915, Fix duplicate headers in response - @taimoorzaeem
- #2824, Fix range request with first position same as length return status 206 - @taimoorzaeem
- #2939, Fix wrong `Preference-Applied` with `Prefer: tx=commit` when transaction is rollbacked - @steve-chavez
- #2939, Fix `count=exact` not being included in `Preference-Applied` - @steve-chavez
- #2800, Fix not including to-one embed resources that had a `NULL` value in any of the selected fields when doing null filtering on them - @laurenceisla
- #2846, Fix error when requesting `Prefer: count=<type>` and doing null filtering on embedded resources - @laurenceisla
- #2959, Fix setting `default_transaction_isolation` unnecessarily - @steve-chavez
- #2929, Fix arrow filtering on RPC returning dynamic TABLE with composite type - @steve-chavez
- #2963, Fix RPCs not embedding correctly when using overloaded functions for computed relationships - @laurenceisla
- #2970, Fix regression that rejects URI connection strings with certain unescaped characters in the password - @laurenceisla, @steve-chavez
## [11.2.0] - 2023-08-10
### Added
- #2523, Data representations - @aljungberg
+ Allows for flexible API output formatting and input parsing on a per-column type basis using regular SQL functions configured in the database
+ Enables greater flexibility in the form and shape of your APIs, both for output and input, making PostgREST a more versatile general-purpose API server
+ Examples include base64 encode/decode your binary data (like a `bytea` column containing an image), choose whether to present a timestamp column as seconds since the Unix epoch or as an ISO 8601 string, or represent fixed precision decimals as strings, not doubles, to preserve precision
+ ...and accept the same in `POST/PUT/PATCH` by configuring the reverse transformation(s)
+ Other use-cases include custom representation of enums, arrays, nested objects, CSS hex colour strings, gzip compressed fields, metric to imperial conversions, and much more
+ Works when using the `select` parameter to select only a subset of columns, embedding through complex joins, renaming fields, with views and computed columns
+ Works when filtering on a formatted column without extra indexes by parsing to the canonical representation
+ Works for data `RETURNING` operations, such as requesting the full body in a POST/PUT/PATCH with `Prefer: return=representation`
+ Works for batch updates and inserts
+ Completely optional, define the functions in the database and they will be used automatically everywhere
+ Data representations preserve the ability to write to the original column and require no extra storage or complex triggers (compared to using `GENERATED ALWAYS` columns)
+ Note: data representations require Postgres 10 (Postgres 11 if using `IN` predicates); data representations are not implemented for RPC
- #2647, Allow to verify the PostgREST version in SQL: `select distinct application_name from pg_stat_activity`. - @laurenceisla
- #2856, Add the `--version` CLI option that prints the version information - @laurenceisla
- #1655, Improve `details` field of the singular error response - @taimoorzaeem
- #740, Add `Preference-Applied` in response for `Prefer: return=representation/headers-only/minimal` - @taimoorzaeem
- #1601, Add optional `nulls=stripped` parameter for mediatypes `application/vnd.pgrst.array+json` and `application/vnd.pgrst.object+json` - @taimoorzaeem
### Fixed
- #2821, Fix OPTIONS not accepting all available media types - @steve-chavez
- #2834, Fix compilation on Ubuntu by being compatible with GHC 9.0.2 - @steve-chavez
- #2840, Fix `Prefer: missing=default` with DOMAIN default values - @steve-chavez
- #2849, Fix HEAD unnecessarily executing aggregates - @steve-chavez
- #2594, Fix unused index on jsonb/jsonb arrow filter and order (``/bets?data->>contractId=eq.1`` and ``/bets?order=data->>contractId``) - @steve-chavez
- #2861, Fix character and bit columns with fixed length not inserting/updating properly - @laurenceisla
+ Fixes the error "value too long for type character(1)" when the char length of the column was bigger than one.
- #2862, Fix null filtering on embedded resource when using a column name equal to the relation name - @steve-chavez
- #1586, Fix function parameters of type character and bit not ignoring length - @laurenceisla
+ Fixes the error "value too long for type character(1)" when the char length of the parameter was bigger than one.
- #2881, Fix error when a function returns `RECORD` or `SET OF RECORD` - @laurenceisla
- #2896, Fix applying superuser settings for impersonated role - @steve-chavez
### Deprecated
- #2863, Deprecate resource embedding target disambiguation - @steve-chavez
+ The `/table?select=*,other!fk(*)` must be used to disambiguate
+ The server aids in choosing the `!fk` by sending a `hint` on the error whenever an ambiguous request happens.
## [11.1.0] - 2023-06-07
### Added
- #2786, Limit idle postgresql connection lifetime - @robx
+ New option `db-pool-max-idletime` (default 30s).
+ This is equivalent to the old option `db-pool-timeout` of PostgREST 10.0.0.
+ A config alias for `db-pool-timeout` is included.
- #2703, Add pre-config function - @steve-chavez
+ New config option `db-pre-config`(empty by default)
+ Allows using the in-database configuration without SUPERUSER
- #2781, When `db-channel-enabled` is false, start automatic connection recovery on a new request when pool connections are closed with `pg_terminate_backend` - @steve-chavez
+ Mitigates the lack of LISTEN/NOTIFY for schema cache reloading on read replicas.
### Fixed
- #2791, Fix dropping schema cache reload notifications - @steve-chavez
- #2801, Stop retrying connection when "no password supplied" - @steve-chavez
## [11.0.1] - 2023-04-27
### Fixed
- #2762, Fixes "permission denied for schema" error during schema cache load - @steve-chavez
- #2756, Fix bad error message on generated columns when using `Prefer: missing=default` - @steve-chavez
- #1139, Allow a 30 second skew for JWT validation - @steve-chavez
+ It used to be 1 second, which was too strict
## [11.0.0] - 2023-04-16
### Added
- #1414, Add related orders - @steve-chavez
+ On a many-to-one or one-to-one relationship, you can order a parent by a child column `/projects?select=*,clients(*)&order=clients(name).desc.nullsfirst`
- #1233, #1907, #2566, Allow spreading embedded resources - @steve-chavez
+ On a many-to-one or one-to-one relationship, you can unnest a json object with `/projects?select=*,...clients(client_name:name)`
+ Allows including the join table columns when resource embedding
+ Allows disambiguating a recursive m2m embed
+ Allows disambiguating an embed that has a many-to-many relationship using two foreign keys on a junction
- #2340, Allow embedding without selecting any column - @steve-chavez
- #2563, Allow `is.null` or `not.is.null` on an embedded resource - @steve-chavez
+ Offers a more flexible replacement for `!inner`, e.g. `/projects?select=*,clients(*)&clients=not.is.null`
+ Allows doing an anti join, e.g. `/projects?select=*,clients(*)&clients=is.null`
+ Allows using or across related tables conditions
- #1100, Customizable OpenAPI title - @AnthonyFisi
- #2506, Add `server-trace-header` for tracing HTTP requests. - @steve-chavez
+ When the client sends the request header specified in the config it will be included in the response headers.
- #2694, Make `db-root-spec` stable. - @steve-chavez
+ This can be used to override the OpenAPI spec with a custom database function
- #1567, On bulk inserts, missing values can get the column DEFAULT by using the `Prefer: missing=default` header - @steve-chavez
- #2501, Allow filtering by`IS DISTINCT FROM` using the `isdistinct` operator, e.g. `/people?alias=isdistinct.foo`
- #1569, Allow `any/all` modifiers on the `eq,like,ilike,gt,gte,lt,lte,match,imatch` operators, e.g. `/tbl?id=eq(any).{1,2,3}` - @steve-chavez
- This converts the input into an array type
- #2561, Configurable role settings - @steve-chavez
- Database roles that are members of the connection role get their settings applied, e.g. doing
`ALTER ROLE anon SET statement_timeout TO '5s'` will result in that `statement_timeout` getting applied for that role.
- Works when switching roles when a JWT is sent
- Settings can be reloaded with `NOTIFY pgrst, 'reload config'`.
- #2468, Configurable transaction isolation level with `default_transaction_isolation` - @steve-chavez
- Can be set per function `create function .. set default_transaction_isolation = 'repeatable read'`
- Or per role `alter role .. set default_transaction_isolation = 'serializable'`
### Fixed
- #2651, Add the missing `get` path item for RPCs to the OpenAPI output - @laurenceisla
- #2648, Fix inaccurate error codes with new ones - @laurenceisla
+ `PGRST204`: Column is not found
+ `PGRST003`: Timed out when acquiring connection to db
- #1652, Fix function call with arguments not inlining - @steve-chavez
- #2705, Fix bug when using the `Range` header on `PATCH/DELETE` - @laurenceisla
+ Fix the`"message": "syntax error at or near \"RETURNING\""` error
+ Fix doing a limited update/delete when an `order` query parameter was present
- #2742, Fix db settings and pg version queries not getting prepared - @steve-chavez
- #2618, Fix `PATCH` requests not recognizing embedded filters and using the top-level resource instead - @steve-chavez
### Changed
- #2705, The `Range` header is now only considered on `GET` requests and is ignored for any other method - @laurenceisla
+ Other methods should use the `limit/offset` query parameters for sub-ranges
+ `PUT` requests no longer return an error when this header is present (using `limit/offset` still triggers the error)
- #2733, Remove bulk RPC call with the `Prefer: params=multiple-objects` header. A function with a JSON array or object parameter should be used instead.
## [10.2.0] - 2023-04-12
### Added
@@ -36,6 +335,7 @@ This project adheres to [Semantic Versioning](http://semver.org/).
- #2548, Fix regression when embedding views with partial references to multi column FKs - @wolfgangwalther
- #2558, Fix regression when requesting limit=0 and `db-max-row` is set - @laurenceisla
- #2542, Return a clear error without hitting the database when trying to update or insert an unknown column with `?columns` - @aljungberg
## [10.1.0] - 2022-10-28
@@ -155,6 +455,11 @@ This project adheres to [Semantic Versioning](http://semver.org/).
- #2312, Using `Prefer: return=representation` no longer returns a `Location` header - @laurenceisla
- #1984, For the cases where one to one relationships are detected, json objects will be returned instead of json arrays of length 1
+ If you wish to override this behavior, you can use computed relationships to return arrays again
+ You can get the newly detected one-to-one relationships by using the `--dump-schema` option and filtering with [jq](https://github.com/jqlang/jq).
```
./postgrest --dump-schema \
| jq '[.dbRelationships | .[] | .[1] | .[] | select(.relCardinality.tag == "O2O" and .relFTableIsView == false and .relTableIsView == false) | del(.relFTableIsView,.relTableIsView,.tag,.relIsSelf)]'
```
## [9.0.1] - 2022-06-03
+12 -12
View File
@@ -1,4 +1,4 @@
![Logo](static/bigger-logo.png "Logo")
![Logo](static/postgrest.png "Logo")
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854)](https://www.patreon.com/postgrest)
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://postgrest.org)
@@ -13,30 +13,30 @@ API than you are likely to write from scratch.
## Sponsors
<table>
<table align="center">
<tbody>
<tr>
<td align="center" valign="middle">
<a href="https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest" target="_blank">
<img width="222px" src="static/cybertec-new.png">
<img width="296px" src="static/cybertec.svg">
</a>
</td>
<td align="center" valign="middle">
<a href="https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo" target="_blank">
<img width="296px" src="static/2ndquadrant.png">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tryretool.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/retool.png">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/neon.jpg">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/gnuhost.png">
<a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/code-build.png">
</a>
</td>
<td align="center" valign="middle">
@@ -45,8 +45,8 @@ API than you are likely to write from scratch.
</a>
</td>
<td align="center" valign="middle">
<a href="https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/oblivious.jpg">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
</a>
</td>
</tr>
@@ -66,7 +66,7 @@ Big thanks to our sponsors! You can join them by supporting PostgREST on [Patreo
```
## [Documentation](http://postgrest.org)
Latest documentation is at [postgrest.org](http://postgrest.org). You can contribute to the docs in [PostgREST/postgrest-docs](https://github.com/PostgREST/postgrest-docs).
Latest documentation is at [postgrest.org](http://postgrest.org). You can contribute to the docs in [PostgREST/postgrest/docs](https://github.com/PostgREST/postgrest/tree/main/docs).
## Performance
+4
View File
@@ -0,0 +1,4 @@
packages: postgrest.cabal
tests: true
package *
ghc-options: -split-sections
+1
View File
@@ -0,0 +1 @@
index-state: hackage.haskell.org 2024-05-17T23:41:49Z
-20
View File
@@ -1,20 +0,0 @@
-- Settings to allow building with plain cabal. If this was
-- named just cabal.project, it would interfere with the default
-- nix build.
packages: .
-- Example of depending on a forked repository (the same dependency
-- would be mentioned in nix/overlays/haskell-packages.nix and
-- stack.yaml, and should refer to a main branch commit of the
-- repository.
--
-- source-repository-package
-- type: git
-- location: https://github.com/PostgREST/hasql-pool.git
-- tag: 4d462c4d47d762effefc7de6c85eaed55f144f1d
source-repository-package
type: git
location: https://github.com/PostgREST/postgresql-libpq.git
tag: 890a0a16cf57dd401420fdc6c7d576fb696003bc
+35 -36
View File
@@ -1,12 +1,23 @@
{ system ? builtins.currentSystem }:
{ system ? builtins.currentSystem
, compiler ? "ghc948"
, # Commit of the Nixpkgs repository that we want to use.
nixpkgsVersion ? import nix/nixpkgs-version.nix
, # Nix files that describe the Nixpkgs repository. We evaluate the expression
# using `import` below.
nixpkgs ? let inherit (nixpkgsVersion) owner repo rev tarballHash; in
builtins.fetchTarball {
url = "https://github.com/${owner}/${repo}/archive/${rev}.tar.gz";
sha256 = tarballHash;
}
}:
let
name =
"postgrest";
compiler =
"ghc924";
# PostgREST source files, filtered based on the rules in the .gitignore files
# and file extensions. We want to include as litte as possible, as the files
# added here will increase the space used in the Nix store and trigger the
@@ -16,18 +27,6 @@ let
(pkgs.gitignoreSource ./.)
[ ".cabal" ".hs" ".lhs" "LICENSE" ];
# Commit of the Nixpkgs repository that we want to use.
nixpkgsVersion =
import nix/nixpkgs-version.nix;
# Nix files that describe the Nixpkgs repository. We evaluate the expression
# using `import` below.
nixpkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${nixpkgsVersion.rev}.tar.gz";
sha256 = nixpkgsVersion.tarballHash;
};
allOverlays =
import nix/overlays;
@@ -36,11 +35,12 @@ let
allOverlays.build-toolbox
allOverlays.checked-shell-script
allOverlays.gitignore
allOverlays.postgis
(allOverlays.postgresql-default { inherit patches; })
allOverlays.postgresql-libpq
allOverlays.postgresql-legacy
allOverlays.postgresql-future
allOverlays.postgis
(allOverlays.haskell-packages { inherit compiler; })
allOverlays.slocat
];
# Evaluated expression of the Nixpkgs repository.
@@ -49,33 +49,28 @@ let
postgresqlVersions =
[
{ name = "postgresql-16"; postgresql = pkgs.postgresql_16.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-12"; postgresql = pkgs.postgresql_12.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-11"; postgresql = pkgs.postgresql_11.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-10"; postgresql = pkgs.postgresql_10.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-9.6"; postgresql = pkgs.postgresql_9_6.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-9_6"; postgresql = pkgs.postgresql_9_6.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
];
patches =
pkgs.callPackage nix/patches { };
# Dynamic derivation for PostgREST
postgrest =
pkgs.haskell.packages."${compiler}".callCabal2nix name src { };
# Function that derives a fully static Haskell package based on
# nh2/static-haskell-nix
staticHaskellPackage =
import nix/static-haskell-package.nix { inherit nixpkgs system compiler patches allOverlays; };
staticHaskellPackage = import nix/static.nix { inherit compiler name pkgs src; };
# Options passed to cabal in dev tools and tests
devCabalOptions =
"-f dev --test-show-detail=direct";
profiledHaskellPackages =
pkgs.haskell.packages."${compiler}".extend (self: super:
pkgs.haskell.packages."${compiler}".extend (_: super:
{
mkDerivation =
args:
@@ -94,10 +89,6 @@ rec {
postgrestPackage =
lib.dontCheck postgrest;
# Static executable.
postgrestStatic =
lib.justStaticExecutables (lib.dontCheck (staticHaskellPackage name src));
# Profiled dynamic executable.
postgrestProfiled =
lib.enableExecutableProfiling (
@@ -119,13 +110,16 @@ rec {
cabalTools =
pkgs.callPackage nix/tools/cabalTools.nix { inherit devCabalOptions postgrest; };
withTools =
pkgs.callPackage nix/tools/withTools.nix { inherit postgresqlVersions postgrest; };
# Development tools.
devTools =
pkgs.callPackage nix/tools/devTools.nix { inherit tests style devCabalOptions hsie withTools; };
# Docker images and loading script.
docker =
pkgs.callPackage nix/tools/docker { postgrest = postgrestStatic; };
# Documentation tools.
docs =
pkgs.callPackage nix/tools/docs.nix { };
# Load testing tools.
loadtest =
@@ -155,7 +149,12 @@ rec {
inherit (pkgs.haskell.packages."${compiler}") hpc-codecov;
inherit (pkgs.haskell.packages."${compiler}") weeder;
};
} // pkgs.lib.optionalAttrs pkgs.stdenv.isLinux rec {
# Static executable.
inherit (staticHaskellPackage) postgrestStatic;
inherit (staticHaskellPackage) packagesStatic;
withTools =
pkgs.callPackage nix/tools/withTools.nix { inherit devCabalOptions postgresqlVersions postgrest; };
# Docker images and loading script.
docker =
pkgs.callPackage nix/tools/docker { postgrest = postgrestStatic; };
}
@@ -10,30 +10,30 @@ write from scratch.
## Sponsors
<table>
<table align="center">
<tbody>
<tr>
<td align="center" valign="middle">
<a href="https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest" target="_blank">
<img width="222px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/cybertec-new.png">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/cybertec.svg">
</a>
</td>
<td align="center" valign="middle">
<a href="https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/2ndquadrant.png">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tryretool.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/retool.png">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/neon.jpg">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/gnuhost.png">
<a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/code-build.png">
</a>
</td>
<td align="center" valign="middle">
@@ -42,8 +42,8 @@ write from scratch.
</a>
</td>
<td align="center" valign="middle">
<a href="https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/oblivious.jpg">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/tembo.png">
</a>
</td>
</tr>
@@ -56,13 +56,15 @@ To learn how to use this container, see the [PostgREST Docker
documentation](https://postgrest.org/en/stable/install.html#docker).
You can configure the PostgREST image by setting
[enviroment variables](https://postgrest.org/en/stable/configuration.html).
[environment variables](https://postgrest.org/en/stable/configuration.html).
# How this image is built
The image is built from scratch using
[Nix](https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools) instead of a
`Dockerfile`, which yields a higly secure and optimized image. This is also why
`Dockerfile`, which yields a highly secure and optimized image. This is also why
no commands are listed in the image history. See the [PostgREST
respository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for
details on the build process and how to inspect the image.
This does not apply to the arm64 variant, which is based on Ubuntu.
+19 -11
View File
@@ -2,19 +2,27 @@
PostgREST docs use the reStructuredText format, check this [cheatsheet](https://github.com/ralsina/rst-cheatsheet/blob/master/rst-cheatsheet.rst) to get acquainted with it.
To build the docs locally, use [nix](https://nixos.org/nix/):
```bash
nix-shell
```
Once in the nix-shell you have the following commands available:
- `postgrest-docs-build`: Build the docs.
- `postgrest-docs-serve`: Build the docs and start a livereload server on `http://localhost:5500`.
- `postgrest-docs-spellcheck`: Run aspell.
To build the docs locally, see [the Nix development readme](/nix/README.md#documentation).
## Documentation structure
This documentation is structured according to tutorials-howtos-topics-references. For more details on the rationale of this structure,
see https://www.divio.com/blog/documentation.
## Translating
To create `.po` files for translation into a new language pass the language code as the first argument to `postgrest-docs-build`.
Example to add German/de:
```
postgrest-docs-build de
```
The livereload server also supports a language/locale argument to show the translated docs during translation:
```
postgrest-docs-serve de
```
Spellcheck is currently only available for the default language.
+10 -26
View File
@@ -5,36 +5,20 @@ The ER diagrams were created with https://github.com/BurntSushi/erd/.
You can go download erd from https://github.com/BurntSushi/erd/releases and then do:
```bash
./erd_static-x86-64 -i film.er -o ../_static/film.png
./erd_static-x86-64 -i ./er/film.er -o ../_static/film.png
```
## LaTeX
The fonts used belong to the GNU FreeFont family. You can download them here: http://ftp.gnu.org/gnu/freefont/
The schema structure diagram is done with LaTeX. You can use a GUI like https://www.mathcha.io/editor to create the .tex file.
## UML
Then use this command to generate the png file.
The UML diagrams are created with https://plantuml.com/.
PlantUML only creates one diagram per file.
That's why we need to create another one for dark mode.
For example, for the file [uml/arch.uml](uml/arch.uml) there's [uml/dark/arch-dark.uml](uml/dark/arch-dark.uml) which includes the first one:
```bash
pdflatex --shell-escape -halt-on-error db.tex
## and move it to the static folder(it's not easy to do it in one go with the pdflatex)
mv db.png ../_static/
```
LaTeX is used because it's a tweakable plain text format.
You can install the full latex suite with `nix`:
```
nix-env -iA texlive.combined.scheme-full
```
To tweak the file with a live reload environment use:
```bash
# open the pdf(zathura used as an example)
zathura db.pdf &
# live reload with entr
echo db.tex | entr pdflatex --shell-escape -halt-on-error db.tex
plantuml -tsvg uml/arch.uml -o ../../_static
plantuml -tsvg -darkmode uml/dark/arch-dark.uml -o ../../../_static
```
-71
View File
@@ -1,71 +0,0 @@
\documentclass[convert]{standalone}
\usepackage{amsmath}
\usepackage{tikz}
\usepackage{mathdots}
\usepackage{yhmath}
\usepackage{cancel}
\usepackage{color}
\usepackage{siunitx}
\usepackage{array}
\usepackage{multirow}
\usepackage{amssymb}
\usepackage{gensymb}
\usepackage{tabularx}
\usepackage{booktabs}
\usetikzlibrary{fadings}
\usetikzlibrary{patterns}
\usetikzlibrary{shadows.blur}
\usetikzlibrary{shapes}
\begin{document}
\newcommand\customScale{0.35}
\begin{tikzpicture}[x=0.75pt,y=0.75pt,yscale=-1,xscale=1, scale=\customScale, every node/.style={scale=\customScale}]
%Shape: Can [id:dp7234864758664346]
\draw [fill={rgb, 255:red, 47; green, 97; blue, 144 } ,fill opacity=1 ] (497.5,51.5) -- (497.5,255.5) .. controls (497.5,275.66) and (423.18,292) .. (331.5,292) .. controls (239.82,292) and (165.5,275.66) .. (165.5,255.5) -- (165.5,51.5) .. controls (165.5,31.34) and (239.82,15) .. (331.5,15) .. controls (423.18,15) and (497.5,31.34) .. (497.5,51.5) .. controls (497.5,71.66) and (423.18,88) .. (331.5,88) .. controls (239.82,88) and (165.5,71.66) .. (165.5,51.5) ;
%Shape: Rectangle [id:dp7384065579958246]
\draw [fill={rgb, 255:red, 236; green, 227; blue, 227 } ,fill opacity=1 ] (189,115) -- (252.5,115) -- (252.5,155) -- (189,155) -- cycle ;
%Shape: Rectangle [id:dp24763906430298177]
\draw [fill={rgb, 255:red, 236; green, 227; blue, 227 } ,fill opacity=1 ] (292,118) -- (362,118) -- (362,158) -- (292,158) -- cycle ;
%Shape: Rectangle [id:dp3775601612537265]
\draw [fill={rgb, 255:red, 236; green, 227; blue, 227 } ,fill opacity=1 ] (397,114) -- (467,114) -- (467,154) -- (397,154) -- cycle ;
%Shape: Rectangle [id:dp7071457022893852]
\draw [fill={rgb, 255:red, 248; green, 231; blue, 28 } ,fill opacity=1 ] (269,199) -- (397.5,199) -- (397.5,273) -- (269,273) -- cycle ;
%Straight Lines [id:da8846759047437789]
\draw (268,234) -- (226.44,155.77) ;
\draw [shift={(225.5,154)}, rotate = 422.02] [color={rgb, 255:red, 0; green, 0; blue, 0 } ][line width=0.75] (10.93,-3.29) .. controls (6.95,-1.4) and (3.31,-0.3) .. (0,0) .. controls (3.31,0.3) and (6.95,1.4) .. (10.93,3.29) ;
%Straight Lines [id:da6908444738113828]
\draw (309.5,198) -- (307.6,161) ;
\draw [shift={(307.5,159)}, rotate = 447.06] [color={rgb, 255:red, 0; green, 0; blue, 0 } ][line width=0.75] (10.93,-3.29) .. controls (6.95,-1.4) and (3.31,-0.3) .. (0,0) .. controls (3.31,0.3) and (6.95,1.4) .. (10.93,3.29) ;
%Straight Lines [id:da7168757864413169]
\draw (398.5,233) -- (431.72,154.84) ;
\draw [shift={(432.5,153)}, rotate = 473.03] [color={rgb, 255:red, 0; green, 0; blue, 0 } ][line width=0.75] (10.93,-3.29) .. controls (6.95,-1.4) and (3.31,-0.3) .. (0,0) .. controls (3.31,0.3) and (6.95,1.4) .. (10.93,3.29) ;
%Up Down Arrow [id:dp14059754167108496]
\draw [fill={rgb, 255:red, 126; green, 211; blue, 33 } ,fill opacity=1 ] (312.5,288.5) -- (330,273) -- (347.5,288.5) -- (338.75,288.5) -- (338.75,319.5) -- (347.5,319.5) -- (330,335) -- (312.5,319.5) -- (321.25,319.5) -- (321.25,288.5) -- cycle ;
% Text Node
\draw (201,129) node [anchor=north west][inner sep=0.75pt] [align=left] {tables};
% Text Node
\draw (307,130) node [anchor=north west][inner sep=0.75pt] [align=left ] {tables};
% Text Node
\draw (414,127) node [anchor=north west][inner sep=0.75pt] [align=left] {tables};
% Text Node
\draw (272,203) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 0; green, 0; blue, 0 } ,opacity=1 ] [align=center] { \\ views \\ + \\ \ \ stored procedures};
% Text Node
\draw (322,178) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 255; green, 255; blue, 255 } ,opacity=1 ] [align=left] {\large\textbf{api}};
% Text Node
\draw (190,97) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 255; green, 255; blue, 255 } ,opacity=1 ] [align=left] {\large\textbf{internal}};
% Text Node
\draw (300,99) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 255; green, 255; blue, 255 } ,opacity=1 ] [align=left] {\large\textbf{private}};
% Text Node
\draw (417,101) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 255; green, 255; blue, 255 } ,opacity=1 ] [align=left] {\large\textbf{core}};
% Text Node
\draw (358,306) node [anchor=north west][inner sep=0.75pt] [align=left] {REST};
\end{tikzpicture}
\end{document}
+15
View File
@@ -0,0 +1,15 @@
entity {font: "FreeSans"}
relationship {font: "FreeMono"}
[Box_Office]
*bo_date
*+film_id
gross_revenue
[Films]
*id
+director_id
title
`...`
Box_Office +--1 Films
+12
View File
@@ -0,0 +1,12 @@
# Build using: -e ortho
entity {font: "FreeSans"}
relationship {font: "FreeMono"}
[Employees]
*id
first_name
last_name
+supervisor_id
Employees 1--* Employees
@@ -1,3 +1,6 @@
entity {font: "FreeSans"}
relationship {font: "FreeSerif"}
[Films]
*id
+director_id
@@ -31,6 +34,12 @@ year
*+film_id
rank
[Technical_Specs]
*+film_id
runtime
camera
sound
Roles *--1 Actors
Roles *--1 Films
@@ -38,3 +47,5 @@ Nominations *--1 Competitions
Nominations *--1 Films
Films *--1 Directors
Films 1--1 Technical_Specs
@@ -1,3 +1,8 @@
# Build using: -e ortho
entity {font: "FreeSans"}
relationship {font: "FreeMono"}
[Addresses]
*id
name
+16
View File
@@ -0,0 +1,16 @@
entity {font: "FreeSans"}
relationship {font: "FreeMono"}
[Premieres]
*id
location
date
+film_id
[Films]
*id
+director_id
title
`...`
Premieres *--1 Films
+12
View File
@@ -0,0 +1,12 @@
# Build using: -e ortho
entity {font: "FreeSans"}
relationship {font: "FreeMono"}
[Presidents]
*id
first_name
last_name
+predecessor_id
Presidents 1--? Presidents
+18
View File
@@ -0,0 +1,18 @@
# Build using: -e ortho
entity {font: "FreeSans"}
relationship {font: "FreeMono"}
[Users]
*id
first_name
last_name
username
[Subscriptions]
*+subscriber_id
*+subscribed_id
type
Users 1--* Subscriptions
Subscriptions *--1 Users
+84
View File
@@ -0,0 +1,84 @@
@startuml
skinparam backgroundColor transparent
package "PostgREST" {
() HTTP as HTTPAPI
HTTPAPI - [Auth]
[Auth] -r.> [ApiRequest]
[ApiRequest] -r.> [Plan]
[Plan] -r.> [Query]
[Query] - () "Connection Pool" : "\t"
[Plan] -u-> [Schema Cache]:uses
[Schema Cache] <- () Listener : reloads
() HTTP as HTTPADMIN
[Admin] -r- () HTTPADMIN
[Config] -l- () CLI
[Config] <-r~ Listener
HTTPADMIN -[hidden]r- CLI
[Schema Cache] -l[hidden]- [Config]
[Schema Cache] -l[hidden]- [Admin]
[Schema Cache] -l[hidden]- CLI
}
database "PostgreSQL" {
node Authorization {
rectangle "Roles, GRANT, RLS"
}
node "API schema" as API {
rectangle "Functions, Views"
}
rectangle "Tables, extensions" as tbs
API -d- tbs
API -l[hidden]- Authorization
}
:user:
hexagon Proxy
:user: .r-> Proxy
HTTPAPI <.l- Proxy
:operator: .d-> HTTPADMIN
:operator: .d-> CLI
PostgreSQL <.developer : "\t"
Listener -r.> "PostgreSQL"
"Connection Pool" -r.> "PostgreSQL" : "\t\t"
note bottom of Auth
Authenticates the user request
end note
note bottom of ApiRequest
Parses the URL syntax
end note
note bottom of Query
Generates the SQL
end note
note top of Listener
LISTEN session
end note
url of Admin is [[../references/admin_server.html#admin-server]]
url of API is [[../explanations/schema_isolation.html]]
url of Auth is [[../references/auth.html#authn]]
url of Authorization is [[../explanations/db_authz.html]]
url of CLI is [[../references/cli.html#cli]]
url of "Connection Pool" is [[../references/connection_pool.html]]
url of Config is [[../references/configuration.html#configuration]]
url of HTTPADMIN is [[https://aosabook.org/en/posa/warp.html]]
url of HTTPAPI is [[https://aosabook.org/en/posa/warp.html]]
url of Listener is [[../references/listener.html#listener]]
url of Proxy is [[../explanations/nginx.html]]
url of "Schema Cache" is [[../references/schema_cache.html#schema-cache]]
@enduml
+3
View File
@@ -0,0 +1,3 @@
@startuml
!include ../arch.uml
@enduml
+3
View File
@@ -0,0 +1,3 @@
@startuml
!include ../sch-iso.uml
@enduml
+29
View File
@@ -0,0 +1,29 @@
@startuml
skinparam backgroundColor transparent
skinparam linetype ortho
skinparam node {
backgroundColor transparent
borderThickness 1
}
database "PostgreSQL" {
node public {
rectangle tables_public as "tables"
}
node extensions as "**extensions**" {
}
node API as "<size:20>api" {
rectangle vf_api as "views + functions"
}
tables_public <-- vf_api
extensions <-- vf_api
}
vf_api <-[thickness=3]-> () PostgREST
@enduml
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 88 KiB

+1
View File
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 23 KiB

+1
View File
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 24 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

+88
View File
@@ -28,6 +28,7 @@ div.line-block {
#sponsors img{
margin: 10px;
width: 13em; /* ".. image::" does not apply width properly to SVGs */
}
#thanks{
@@ -65,3 +66,90 @@ div.line-block {
.wy-table-responsive {
overflow: visible !important;
}
#tutorials span.caption-text {
display: none;
}
#references span.caption-text {
display: none;
}
#explanations span.caption-text {
display: none;
}
#how-tos span.caption-text {
display: none;
}
#ecosystem span.caption-text {
display: none;
}
#integrations span.caption-text {
display: none;
}
#api span.caption-text {
display: none;
}
/* Tweaks for dark mode from extension: sphinx-rtd-dark-theme */
html[data-theme="dark"] .highlight {
background-color: #17181c !important;
}
html[data-theme="dark"] .sphinx-tabs-tab {
color: var(--dark-link-color);
}
html[data-theme="dark"] .sphinx-tabs-panel {
border: 1px solid #404040;
border-top: 0;
background: #141414;
}
html[data-theme="dark"] .sphinx-tabs-tab[aria-selected="true"] {
border: 1px solid #404040;
border-bottom: 1px solid #141414;
background-color: #141414;
}
html[data-theme="dark"] [role="tablist"] {
border-bottom: 1px solid #404040;
}
html[data-theme="dark"] .btn-neutral {
color: white !important;
}
html[data-theme="dark"] .img-dark {
display: inline;
}
html:not([data-theme="dark"]) .img-dark {
display: none;
}
html[data-theme="dark"] .img-light {
display: none;
}
html:not([data-theme="dark"]) .img-light {
display: inline;
}
html[data-theme="dark"] .img-translucent img {
background-color: #cccccc;
}
.img-translucent img {
transition: background-color 0.3s;
margin-bottom: 24px;
}
.svg-container-md {
max-width: 400px;
}
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 345 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.0 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 8.3 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 50 KiB

After

Width:  |  Height:  |  Size: 56 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 142 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 16 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 9.0 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 62 KiB

+1
View File
@@ -0,0 +1 @@
<?xml version="1.0" encoding="us-ascii" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentStyleType="text/css" height="411px" preserveAspectRatio="none" style="width:315px;height:411px;" version="1.1" viewBox="0 0 315 411" width="315px" zoomAndPan="magnify"><defs/><g><!--cluster PostgreSQL--><g id="cluster_PostgreSQL"><path d="M6,16 C6,6 152,6 152,6 C152,6 298,6 298,16 L298,309.14 C298,319.14 152,319.14 152,319.14 C152,319.14 6,319.14 6,309.14 L6,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><path d="M6,16 C6,26 152,26 152,26 C152,26 298,26 298,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="81.375" x="111.3125" y="42.9659">PostgreSQL</text></g><!--cluster public--><g id="cluster_public"><polygon fill="none" points="30,77,40,67,150,67,150,155.07,140,165.07,30,165.07,30,77" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="140" x2="150" y1="77" y2="67"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="30" x2="140" y1="77" y2="77"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="140" x2="140" y1="77" y2="165.07"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="44.625" x="63.6875" y="94.9659">public</text></g><!--cluster API--><g id="cluster_API"><polygon fill="none" points="68,199.07,78,189.07,236,189.07,236,285.14,226,295.14,68,295.14,68,199.07" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="226" x2="236" y1="199.07" y2="189.07"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="68" x2="226" y1="199.07" y2="199.07"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="226" x2="226" y1="199.07" y2="295.14"/><text fill="#FFFFFF" font-family="sans-serif" font-size="20" font-weight="bold" lengthAdjust="spacing" textLength="29.875" x="133.0625" y="223.45">api</text></g><!--entity tables_public--><g id="elem_tables_public"><rect fill="#313139" height="39.0679" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="61" x="69.5" y="110"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="41" x="79.5" y="134.9659">tables</text></g><!--entity extensions--><g id="elem_extensions"><polygon fill="none" points="165.81,115,175.81,105,282.185,105,282.185,144.0679,272.185,154.0679,165.81,154.0679,165.81,115" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="272.185" x2="282.185" y1="115" y2="105"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="165.81" x2="272.185" y1="115" y2="115"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="272.185" x2="272.185" y1="115" y2="154.0679"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="76.375" x="180.81" y="139.9659">extensions</text></g><!--entity vf_api--><g id="elem_vf_api"><rect fill="#313139" height="39.0679" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="136" x="84" y="240.07"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="116" x="94" y="265.0359">views + functions</text></g><!--entity PostgREST--><g id="elem_PostgREST"><ellipse cx="152" cy="371.14" fill="#313139" rx="8" ry="8" style="stroke:#E7E7E7;stroke-width:0.5;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="70" x="117" y="403.1059">PostgREST</text></g><!--reverse link tables_public to vf_api--><g id="link_tables_public_vf_api"><path d="M107.25,155.29 C107.25,179.74 107.25,215.13 107.25,239.67 " fill="none" id="tables_public-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="107.25,149.29,103.25,158.29,107.25,154.29,111.25,158.29,107.25,149.29" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--reverse link extensions to vf_api--><g id="link_extensions_vf_api"><path d="M192.91,160.31 C192.91,185.07 192.91,216.68 192.91,239.61 " fill="none" id="extensions-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="192.91,154.31,188.91,163.31,192.91,159.31,196.91,163.31,192.91,154.31" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--link vf_api to PostgREST--><g id="link_vf_api_PostgREST"><path d="M152,285.55 C152,309.98 152,338.34 152,356.18 " fill="none" id="vf_api-PostgREST" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="152,279.55,148,288.55,152,284.55,156,288.55,152,279.55" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="152,362.18,156,353.18,152,357.18,148,353.18,152,362.18" style="stroke:#E7E7E7;stroke-width:3.0;"/></g><!--SRC=[KypCIyufJKbLqDFJBqxEqCqipjShpSq10000]--></g></svg>

After

Width:  |  Height:  |  Size: 4.8 KiB

+1
View File
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 5.1 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.4 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 89 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

-385
View File
@@ -1,385 +0,0 @@
.. _admin:
Hardening PostgREST
===================
PostgREST is a fast way to construct a RESTful API. Its default behavior is great for scaffolding in development. When it's time to go to production it works great too, as long as you take precautions. PostgREST is a small sharp tool that focuses on performing the API-to-database mapping. We rely on a reverse proxy like Nginx for additional safeguards.
The first step is to create an Nginx configuration file that proxies requests to an underlying PostgREST server.
.. code-block:: nginx
http {
# ...
# upstream configuration
upstream postgrest {
server localhost:3000;
}
# ...
server {
# ...
# expose to the outside world
location /api/ {
default_type application/json;
proxy_hide_header Content-Location;
add_header Content-Location /api/$upstream_http_content_location;
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_pass http://postgrest/;
}
# ...
}
}
.. note::
For ubuntu, if you already installed nginx through :code:`apt` you can add this to the config file in
:code:`/etc/nginx/sites-enabled/default`.
.. _block_fulltable:
Block Full-Table Operations
---------------------------
Each table in the admin-selected schema gets exposed as a top level route. Client requests are executed by certain database roles depending on their authentication. All HTTP verbs are supported that correspond to actions permitted to the role. For instance if the active role can drop rows of the table then the DELETE verb is allowed for clients. Here's an API request to delete old rows from a hypothetical logs table:
.. tabs::
.. code-tab:: http
DELETE /logs?time=lt.1991-08-06 HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/logs?time=lt.1991-08-06" -X DELETE
However it's very easy to delete the **entire table** by omitting the query parameter!
.. tabs::
.. code-tab:: http
DELETE /logs HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/logs" -X DELETE
This can happen accidentally such as by switching a request from a GET to a DELETE. To protect against accidental operations use the `pg-safeupdate <https://github.com/eradman/pg-safeupdate>`_ PostgreSQL extension. It raises an error if UPDATE or DELETE are executed without specifying conditions. To install it you can use the `PGXN <https://pgxn.org/>`_ network:
.. code-block:: bash
sudo -E pgxn install safeupdate
# then add this to postgresql.conf:
# shared_preload_libraries='safeupdate';
This does not protect against malicious actions, since someone can add a url parameter that does not affect the result set. To prevent this you must turn to database permissions, forbidding the wrong people from deleting rows, and using `row-level security <https://www.postgresql.org/docs/current/ddl-rowsecurity.html>`_ if finer access control is required.
Count-Header DoS
----------------
For convenience to client-side pagination controls PostgREST supports counting and reporting total table size in its response. As described in :ref:`limits`, responses ordinarily include a range but leave the total unspecified like
.. code-block:: http
HTTP/1.1 200 OK
Range-Unit: items
Content-Range: 0-14/*
However including the request header :code:`Prefer: count=exact` calculates and includes the full count:
.. code-block:: http
HTTP/1.1 206 Partial Content
Range-Unit: items
Content-Range: 0-14/3573458
This is fine in small tables, but count performance degrades in big tables due to the MVCC architecture of PostgreSQL. For very large tables it can take a very long time to retrieve the results which allows a denial of service attack. The solution is to strip this header from all requests:
.. code-block:: postgres
-- Pending nginx config: Remove any prefer header which contains the word count
.. _https:
HTTPS
-----
PostgREST aims to do one thing well: add an HTTP interface to a PostgreSQL database. To keep the code small and focused we do not implement HTTPS. Use a reverse proxy such as NGINX to add this, `here's how <https://nginx.org/en/docs/http/configuring_https_servers.html>`_. Note that some Platforms as a Service like Heroku also add SSL automatically in their load balancer.
Rate Limiting
-------------
Nginx supports "leaky bucket" rate limiting (see `official docs <https://nginx.org/en/docs/http/ngx_http_limit_req_module.html>`_). Using standard Nginx configuration, routes can be grouped into *request zones* for rate limiting. For instance we can define a zone for login attempts:
.. code-block:: nginx
limit_req_zone $binary_remote_addr zone=login:10m rate=1r/s;
This creates a shared memory zone called "login" to store a log of IP addresses that access the rate limited urls. The space reserved, 10 MB (:code:`10m`) will give us enough space to store a history of 160k requests. We have chosen to allow only allow one request per second (:code:`1r/s`).
Next we apply the zone to certain routes, like a hypothetical stored procedure called :code:`login`.
.. code-block:: nginx
location /rpc/login/ {
# apply rate limiting
limit_req zone=login burst=5;
}
The burst argument tells Nginx to start dropping requests if more than five queue up from a specific IP.
Nginx rate limiting is general and indiscriminate. To rate limit each authenticated request individually you will need to add logic in a :ref:`Custom Validation <custom_validation>` function.
.. _external_connection_poolers:
Using External Connection Poolers
---------------------------------
PostgREST manages its :ref:`own pool of connections <db-pool>` and uses prepared statements by default in order to increase performance. However, this setting is incompatible with external connection poolers such as PgBouncer working in transaction pooling mode. In this case, you need to set the :ref:`db-prepared-statements` config option to ``false``. On the other hand, session pooling is fully compatible with PostgREST, while statement pooling is not compatible at all.
.. note::
If prepared statements are enabled, PostgREST will quit after detecting that transaction or statement pooling is being used.
You should also set the :ref:`db-channel-enabled` config option to ``false``, due to the ``LISTEN`` command not being compatible with transaction pooling, although it should not give any errors if it's left enabled by default.
Debugging
=========
Server Version
--------------
When debugging a problem it's important to verify the PostgREST version. At any time you can make a request to the running server and determine exactly which version is deployed. Look for the :code:`Server` HTTP response header, which contains the version number.
Errors
------
See the :doc:`Errors <errors>` reference page for detailed information on the errors that PostgREST returns.
.. _pgrst_logging:
Logging
-------
PostgREST logs basic request information to ``stdout``, including the authenticated user if available, the requesting IP address and user agent, the URL requested, and HTTP response status.
.. code::
127.0.0.1 - user [26/Jul/2021:01:56:38 -0500] "GET /clients HTTP/1.1" 200 - "" "curl/7.64.0"
127.0.0.1 - anonymous [26/Jul/2021:01:56:48 -0500] "GET /unexistent HTTP/1.1" 404 - "" "curl/7.64.0"
For diagnostic information about the server itself, PostgREST logs to ``stderr``.
.. code::
12/Jun/2021:17:47:39 -0500: Attempting to connect to the database...
12/Jun/2021:17:47:39 -0500: Listening on port 3000
12/Jun/2021:17:47:39 -0500: Connection successful
12/Jun/2021:17:47:39 -0500: Config re-loaded
12/Jun/2021:17:47:40 -0500: Schema cache loaded
.. note::
When running it in an SSH session you must detach it from stdout or it will be terminated when the session closes. The easiest technique is redirecting the output to a log file or to the syslog:
.. code-block:: bash
ssh foo@example.com \
'postgrest foo.conf </dev/null >/var/log/postgrest.log 2>&1 &'
# another option is to pipe the output into "logger -t postgrest"
PostgREST logging provides limited information for debugging server errors. It's helpful to get full information about both client requests and the corresponding SQL commands executed against the underlying database.
HTTP Requests
-------------
A great way to inspect incoming HTTP requests including headers and query parameters is to sniff the network traffic on the port where PostgREST is running. For instance on a development server bound to port 3000 on localhost, run this:
.. code:: bash
# sudo access is necessary for watching the network
sudo ngrep -d lo0 port 3000
The options to ngrep vary depending on the address and host on which you've bound the server. The binding is described in the :ref:`configuration` section. The ngrep output isn't particularly pretty, but it's legible.
.. _automatic_recovery:
Automatic Connection Recovery
-----------------------------
When PostgREST loses the connection to the database, it retries the connection using capped exponential backoff, with 32 seconds being the maximum backoff time.
This retry behavior is triggered immediately after the connection is lost if :ref:`db-channel-enabled` is set to true(the default), otherwise it will be activated once a request is made.
To notify the client when the next reconnection attempt will be, PostgREST responds with ``503 Service Unavailable`` and the ``Retry-After: x`` header, where ``x`` is the number of seconds programmed for the next retry.
Database Logs
-------------
Once you've verified that requests are as you expect, you can get more information about the server operations by watching the database logs. By default PostgreSQL does not keep these logs, so you'll need to make the configuration changes below. Find :code:`postgresql.conf` inside your PostgreSQL data directory (to find that, issue the command :code:`show data_directory;`). Either find the settings scattered throughout the file and change them to the following values, or append this block of code to the end of the configuration file.
.. code:: sql
# send logs where the collector can access them
log_destination = "stderr"
# collect stderr output to log files
logging_collector = on
# save logs in pg_log/ under the pg data directory
log_directory = "pg_log"
# (optional) new log file per day
log_filename = "postgresql-%Y-%m-%d.log"
# log every kind of SQL statement
log_statement = "all"
Restart the database and watch the log file in real-time to understand how HTTP requests are being translated into SQL commands.
.. note::
On Docker you can enable the logs by using a custom ``init.sh``:
.. code:: bash
#!/bin/sh
echo "log_statement = 'all'" >> /var/lib/postgresql/data/postgresql.conf
After that you can start the container and check the logs with ``docker logs``.
.. code:: bash
docker run -v "$(pwd)/init.sh":"/docker-entrypoint-initdb.d/init.sh" -d postgres
docker logs -f <container-id>
Schema Reloading
----------------
Changing the schema while the server is running can lead to errors due to a stale schema cache. To learn how to refresh the cache see :ref:`schema_reloading`.
.. _health_check:
Health Check
------------
You can enable a minimal health check to verify if PostgREST is available for client requests and to check the status of its internal state.
To do this, set the configuration variable :ref:`admin-server-port` to the port number of your preference. Two endpoints ``live`` and ``ready`` will then be available.
The ``live`` endpoint verifies if PostgREST is running on its configured port. A request will return ``200 OK`` if PostgREST is alive or ``503`` otherwise.
The ``ready`` endpoint also checks the state of both the Database Connection and the :ref:`schema_cache`. A request will return ``200 OK`` if it is ready or ``503`` if not.
For instance, to verify if PostgREST is running at ``localhost:3000`` while the ``admin-server-port`` is set to ``3001``:
.. tabs::
.. code-tab:: http
GET localhost:3001/live HTTP/1.1
.. code-tab:: bash Curl
curl -I "http://localhost:3001/live"
.. code-block:: http
HTTP/1.1 200 OK
If you have a machine with multiple network interfaces and multiple PostgREST instances in the same port, you need to specify a unique :ref:`hostname <server-host>` in the configuration of each PostgREST instance for the health check to work correctly. Don't use the special values(``!4``, ``*``, etc) in this case because the health check could report a false positive.
Daemonizing
===========
For Linux distributions that use **systemd** (Ubuntu, Debian, Archlinux) you can create a daemon in the following way.
First, create postgrest configuration in ``/etc/postgrest/config``
.. code-block:: ini
db-uri = "postgres://<your_user>:<your_password>@localhost:5432/<your_db>"
db-schemas = "<your_exposed_schema>"
db-anon-role = "<your_anon_role>"
jwt-secret = "<your_secret>"
Then create the systemd service file in ``/etc/systemd/system/postgrest.service``
.. code-block:: ini
[Unit]
Description=REST API for any PostgreSQL database
After=postgresql.service
[Service]
ExecStart=/bin/postgrest /etc/postgrest/config
ExecReload=/bin/kill -SIGUSR1 $MAINPID
[Install]
WantedBy=multi-user.target
After that, you can enable the service at boot time and start it with:
.. code-block:: bash
systemctl enable postgrest
systemctl start postgrest
## For reloading the service
## systemctl restart postgrest
.. _file_descriptors:
File Descriptors
----------------
File descriptors are kernel resources that are used by HTTP connections (among others). File descriptors are limited per process. The kernel default limit is 1024, which is increased in some Linux distributions.
When under heavy traffic, PostgREST can reach this limit and start showing ``No file descriptors available`` errors. To clear these errors, you can increase the process' file descriptor limit.
.. code-block:: ini
[Service]
LimitNOFILE=10000
Alternate URL Structure
=======================
As discussed in :ref:`singular_plural`, there are no special URL forms for singular resources in PostgREST, only operators for filtering. Thus there are no URLs like :code:`/people/1`. It would be specified instead as
.. tabs::
.. code-tab:: http
GET /people?id=eq.1 HTTP/1.1
Accept: application/vnd.pgrst.object+json
.. code-tab:: bash Curl
curl "http://localhost:3000/people?id=eq.1" \
-H "Accept: application/vnd.pgrst.object+json"
This allows compound primary keys and makes the intent for singular response independent of a URL convention.
Nginx rewrite rules allow you to simulate the familiar URL convention. The following example adds a rewrite rule for all table endpoints, but you'll want to restrict it to those tables that have a numeric simple primary key named "id."
.. code-block:: nginx
# support /endpoint/:id url style
location ~ ^/([a-z_]+)/([0-9]+) {
# make the response singular
proxy_set_header Accept 'application/vnd.pgrst.object+json';
# assuming an upstream named "postgrest"
proxy_pass http://postgrest/$1?id=eq.$2;
}
.. TODO
.. Administration
.. API Versioning
.. HTTP Caching
.. Upgrading
-3113
View File
File diff suppressed because it is too large Load Diff
-496
View File
@@ -1,496 +0,0 @@
.. _roles:
Overview of Role System
=======================
PostgREST is designed to keep the database at the center of API security. All authorization happens through database roles and permissions. It is PostgREST's job to **authenticate** requests -- i.e. verify that a client is who they say they are -- and then let the database **authorize** client actions.
Authentication Sequence
-----------------------
There are three types of roles used by PostgREST, the **authenticator**, **anonymous** and **user** roles. The database administrator creates these roles and configures PostgREST to use them.
.. image:: _static/security-roles.png
The authenticator should be created :code:`NOINHERIT` and configured in the database to have very limited access. It is a chameleon whose job is to "become" other users to service authenticated HTTP requests. The picture below shows how the server handles authentication. If auth succeeds, it switches into the user role specified by the request, otherwise it switches into the anonymous role (if it's set in :ref:`db-anon-role`).
.. image:: _static/security-anon-choice.png
Here are the technical details. We use `JSON Web Tokens <https://jwt.io/>`_ to authenticate API requests. As you'll recall a JWT contains a list of cryptographically signed claims. All claims are allowed but PostgREST cares specifically about a claim called role.
.. code:: json
{
"role": "user123"
}
When a request contains a valid JWT with a role claim PostgREST will switch to the database role with that name for the duration of the HTTP request.
.. code:: sql
SET LOCAL ROLE user123;
Note that the database administrator must allow the authenticator role to switch into this user by previously executing
.. code:: sql
GRANT user123 TO authenticator;
If the client included no JWT (or one without a role claim) then PostgREST switches into the anonymous role whose actual database-specific name, like that of with the authenticator role, is specified in the PostgREST server configuration file. The database administrator must set anonymous role permissions correctly to prevent anonymous users from seeing or changing things they shouldn't.
Users and Groups
----------------
PostgreSQL manages database access permissions using the concept of roles. A role can be thought of as either a database user, or a group of database users, depending on how the role is set up.
Roles for Each Web User
~~~~~~~~~~~~~~~~~~~~~~~
PostgREST can accommodate either viewpoint. If you treat a role as a single user then the JWT-based role switching described above does most of what you need. When an authenticated user makes a request PostgREST will switch into the role for that user, which in addition to restricting queries, is available to SQL through the :code:`current_user` variable.
You can use row-level security to flexibly restrict visibility and access for the current user. Here is an `example <https://www.2ndquadrant.com/en/blog/application-users-vs-row-level-security/>`_ from Tomas Vondra, a chat table storing messages sent between users. Users can insert rows into it to send messages to other users, and query it to see messages sent to them by other users.
.. code-block:: postgres
CREATE TABLE chat (
message_uuid UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
message_time TIMESTAMP NOT NULL DEFAULT now(),
message_from NAME NOT NULL DEFAULT current_user,
message_to NAME NOT NULL,
message_subject VARCHAR(64) NOT NULL,
message_body TEXT
);
ALTER TABLE chat ENABLE ROW LEVEL SECURITY;
We want to enforce a policy that ensures a user can see only those messages sent by them or intended for them. Also we want to prevent a user from forging the message_from column with another person's name.
PostgreSQL allows us to set this policy with row-level security:
.. code-block:: postgres
CREATE POLICY chat_policy ON chat
USING ((message_to = current_user) OR (message_from = current_user))
WITH CHECK (message_from = current_user)
Anyone accessing the generated API endpoint for the chat table will see exactly the rows they should, without our needing custom imperative server-side coding.
.. warning::
Roles are namespaced per-cluster rather than per-database so they may be prone to collision.
Web Users Sharing Role
~~~~~~~~~~~~~~~~~~~~~~
Alternately database roles can represent groups instead of (or in addition to) individual users. You may choose that all signed-in users for a web app share the role webuser. You can distinguish individual users by including extra claims in the JWT such as email.
.. code:: json
{
"role": "webuser",
"email": "john@doe.com"
}
SQL code can access claims through GUC variables set by PostgREST per request. For instance to get the email claim, call this function:
For PostgreSQL server version >= 14
.. code:: sql
current_setting('request.jwt.claims', true)::json->>'email';
For PostgreSQL server version < 14
.. code:: sql
current_setting('request.jwt.claim.email', true);
This allows JWT generation services to include extra information and your database code to react to it. For instance the RLS example could be modified to use this current_setting rather than current_user. The second 'true' argument tells current_setting to return NULL if the setting is missing from the current configuration.
Hybrid User-Group Roles
~~~~~~~~~~~~~~~~~~~~~~~
You can mix the group and individual role policies. For instance we could still have a webuser role and individual users which inherit from it:
.. code-block:: postgres
CREATE ROLE webuser NOLOGIN;
-- grant this role access to certain tables etc
CREATE ROLE user000 NOLOGIN;
GRANT webuser TO user000;
-- now user000 can do whatever webuser can
GRANT user000 TO authenticator;
-- allow authenticator to switch into user000 role
-- (the role itself has nologin)
.. _custom_validation:
Custom Validation
-----------------
PostgREST honors the :code:`exp` claim for token expiration, rejecting expired tokens. However it does not enforce any extra constraints. An example of an extra constraint would be to immediately revoke access for a certain user. The configuration file parameter :code:`db-pre-request` specifies a stored procedure to call immediately after the authenticator switches into a new role and before the main query itself runs.
Here's an example. In the config file specify a stored procedure:
.. code:: ini
db-pre-request = "public.check_user"
In the function you can run arbitrary code to check the request and raise an exception to block it if desired.
.. code-block:: postgres
CREATE OR REPLACE FUNCTION check_user() RETURNS void AS $$
BEGIN
IF current_user = 'evil_user' THEN
RAISE EXCEPTION 'No, you are evil'
USING HINT = 'Stop being so evil and maybe you can log in';
END IF;
END
$$ LANGUAGE plpgsql;
.. _client_auth:
Client Auth
===========
To make an authenticated request the client must include an :code:`Authorization` HTTP header with the value :code:`Bearer <jwt>`. For instance:
.. tabs::
.. code-tab:: http
GET /foo HTTP/1.1
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiamRvZSIsImV4cCI6MTQ3NTUxNjI1MH0.GYDZV3yM0gqvuEtJmfpplLBXSGYnke_Pvnl0tbKAjB4
.. code-tab:: bash Curl
curl "http://localhost:3000/foo" \
-H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiamRvZSIsImV4cCI6MTQ3NTUxNjI1MH0.GYDZV3yM0gqvuEtJmfpplLBXSGYnke_Pvnl0tbKAjB4"
The ``Bearer`` header value can be used with or without capitalization(``bearer``).
JWT Generation
--------------
You can create a valid JWT either from inside your database or via an external service. Each token is cryptographically signed with a secret key. In the case of symmetric cryptography the signer and verifier share the same secret passphrase. In asymmetric cryptography the signer uses the private key and the verifier the public key. PostgREST supports both symmetric and asymmetric cryptography.
JWT from SQL
~~~~~~~~~~~~
You can create JWT tokens in SQL using the `pgjwt extension <https://github.com/michelp/pgjwt>`_. It's simple and requires only pgcrypto. If you're on an environment like Amazon RDS which doesn't support installing new extensions, you can still manually run the `SQL inside pgjwt <https://github.com/michelp/pgjwt/blob/master/pgjwt--0.1.1.sql>`_ (you'll need to replace ``@extschema@`` with another schema or just delete it) which creates the functions you will need.
Next write a stored procedure that returns the token. The one below returns a token with a hard-coded role, which expires five minutes after it was issued. Note this function has a hard-coded secret as well.
.. code-block:: postgres
CREATE TYPE jwt_token AS (
token text
);
CREATE FUNCTION jwt_test() RETURNS public.jwt_token AS $$
SELECT public.sign(
row_to_json(r), 'reallyreallyreallyreallyverysafe'
) AS token
FROM (
SELECT
'my_role'::text as role,
extract(epoch from now())::integer + 300 AS exp
) r;
$$ LANGUAGE sql;
PostgREST exposes this function to clients via a POST request to ``/rpc/jwt_test``.
.. note::
To avoid hard-coding the secret in stored procedures, save it as a property of the database.
.. code-block:: postgres
-- run this once
ALTER DATABASE mydb SET "app.jwt_secret" TO 'reallyreallyreallyreallyverysafe';
-- then all functions can refer to app.jwt_secret
SELECT sign(
row_to_json(r), current_setting('app.jwt_secret')
) AS token
FROM ...
JWT from Auth0
~~~~~~~~~~~~~~
An external service like `Auth0 <https://auth0.com/>`_ can do the hard work transforming OAuth from Github, Twitter, Google etc into a JWT suitable for PostgREST. Auth0 can also handle email signup and password reset flows.
To use Auth0, create `an application <https://auth0.com/docs/get-started/applications>`_ for your app and `an API <https://auth0.com/docs/get-started/apis>`_ for your PostgREST server. Auth0 supports both HS256 and RS256 scheme for the issued tokens for APIs. For simplicity, you may first try HS256 scheme while creating your API on Auth0. Your application should use your PostgREST API's `API identifier <https://auth0.com/docs/get-started/apis/api-settings>`_ by setting it with the `audience parameter <https://auth0.com/docs/secure/tokens/access-tokens/get-access-tokens#control-access-token-audience>`_ during the authorization request. This will ensure that Auth0 will issue an access token for your PostgREST API. For PostgREST to verify the access token, you will need to set ``jwt-secret`` on PostgREST config file with your API's signing secret.
.. note::
Our code requires a database role in the JWT. To add it you need to save the database role in Auth0 `app metadata <https://auth0.com/docs/manage-users/user-accounts/metadata/manage-metadata-rules>`_. Then, you will need to write `a rule <https://auth0.com/docs/customize/rules>`_ that will extract the role from the user's app_metadata and set it as a `custom claim <https://auth0.com/docs/get-started/apis/scopes/sample-use-cases-scopes-and-claims#add-custom-claims-to-a-token>`_ in the access token. Note that, you may use Auth0's `core authorization feature <https://auth0.com/docs/manage-users/access-control/rbac>`_ for more complex use cases. Metadata solution is mentioned here for simplicity.
.. code:: javascript
function (user, context, callback) {
// Follow the documentations at
// https://postgrest.org/en/latest/configuration.html#db-role-claim-key
// to set a custom role claim on PostgREST
// and use it as custom claim attribute in this rule
const myRoleClaim = 'https://myapp.com/role';
user.app_metadata = user.app_metadata || {};
context.accessToken[myRoleClaim] = user.app_metadata.role;
callback(null, user, context);
}
.. _asym_keys:
Asymmetric Keys
~~~~~~~~~~~~~~~
As described in the :ref:`configuration` section, PostgREST accepts a ``jwt-secret`` config file parameter. If it is set to a simple string value like "reallyreallyreallyreallyverysafe" then PostgREST interprets it as an HMAC-SHA256 passphrase. However you can also specify a literal JSON Web Key (JWK) or set. For example, you can use an RSA-256 public key encoded as a JWK:
.. code-block:: json
{
"alg":"RS256",
"e":"AQAB",
"key_ops":["verify"],
"kty":"RSA",
"n":"9zKNYTaYGfGm1tBMpRT6FxOYrM720GhXdettc02uyakYSEHU2IJz90G_MLlEl4-WWWYoS_QKFupw3s7aPYlaAjamG22rAnvWu-rRkP5sSSkKvud_IgKL4iE6Y2WJx2Bkl1XUFkdZ8wlEUR6O1ft3TS4uA-qKifSZ43CahzAJyUezOH9shI--tirC028lNg767ldEki3WnVr3zokSujC9YJ_9XXjw2hFBfmJUrNb0-wldvxQbFU8RPXip-GQ_JPTrCTZhrzGFeWPvhA6Rqmc3b1PhM9jY7Dur1sjYWYVyXlFNCK3c-6feo5WlRfe1aCWmwZQh6O18eTmLeT4nWYkDzQ"
}
.. note::
This could also be a JSON Web Key Set (JWKS) if it was contained within an array assigned to a `keys` member, e.g. ``{ keys: [jwk1, jwk2] }``.
Just pass it in as a single line string, escaping the quotes:
.. code-block:: ini
jwt-secret = "{ \"alg\":\"RS256\", … }"
To generate such a public/private key pair use a utility like `latchset/jose <https://github.com/latchset/jose>`_.
.. code-block:: bash
jose jwk gen -i '{"alg": "RS256"}' -o rsa.jwk
jose jwk pub -i rsa.jwk -o rsa.jwk.pub
# now rsa.jwk.pub contains the desired JSON object
You can specify the literal value as we saw earlier, or reference a filename to load the JWK from a file:
.. code-block:: ini
jwt-secret = "@rsa.jwk.pub"
JWT security
~~~~~~~~~~~~
There are at least three types of common critiques against using JWT: 1) against the standard itself, 2) against using libraries with known security vulnerabilities, and 3) against using JWT for web sessions. We'll briefly explain each critique, how PostgREST deals with it, and give recommendations for appropriate user action.
The critique against the `JWT standard <https://datatracker.ietf.org/doc/html/rfc7519>`_ is voiced in detail `elsewhere on the web <https://web.archive.org/web/20230123041631/https://paragonie.com/blog/2017/03/jwt-json-web-tokens-is-bad-standard-that-everyone-should-avoid>`_. The most relevant part for PostgREST is the so-called :code:`alg=none` issue. Some servers implementing JWT allow clients to choose the algorithm used to sign the JWT. In this case, an attacker could set the algorithm to :code:`none`, remove the need for any signature at all and gain unauthorized access. The current implementation of PostgREST, however, does not allow clients to set the signature algorithm in the HTTP request, making this attack irrelevant. The critique against the standard is that it requires the implementation of the :code:`alg=none` at all.
Critiques against JWT libraries are only relevant to PostgREST via the library it uses. As mentioned above, not allowing clients to choose the signature algorithm in HTTP requests removes the greatest risk. Another more subtle attack is possible where servers use asymmetric algorithms like RSA for signatures. Once again this is not relevant to PostgREST since it is not supported. Curious readers can find more information in `this article <https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/>`_. Recommendations about high quality libraries for usage in API clients can be found on `jwt.io <https://jwt.io/>`_.
The last type of critique focuses on the misuse of JWT for maintaining web sessions. The basic recommendation is to `stop using JWT for sessions <http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-for-sessions/>`_ because most, if not all, solutions to the problems that arise when you do, `do not work <http://cryto.net/~joepie91/blog/2016/06/19/stop-using-jwt-for-sessions-part-2-why-your-solution-doesnt-work/>`_. The linked articles discuss the problems in depth but the essence of the problem is that JWT is not designed to be secure and stateful units for client-side storage and therefore not suited to session management.
PostgREST uses JWT mainly for authentication and authorization purposes and encourages users to do the same. For web sessions, using cookies over HTTPS is good enough and well catered for by standard web frameworks.
Schema Isolation
================
You can isolate your api schema from internal implementation details, as explained in :ref:`schema_isolation`. For an example of wrapping a private table with a public view see the :ref:`public_ui` section below.
.. _sql_user_management:
SQL User Management
===================
Storing Users and Passwords
---------------------------
As mentioned, an external service can provide user management and coordinate with the PostgREST server using JWT. It's also possible to support logins entirely through SQL. It's a fair bit of work, so get ready.
The following table, functions, and triggers will live in a :code:`basic_auth` schema that you shouldn't expose publicly in the API. The public views and functions will live in a different schema which internally references this internal information.
First we'll need a table to keep track of our users:
.. code:: sql
-- We put things inside the basic_auth schema to hide
-- them from public view. Certain public procs/views will
-- refer to helpers and tables inside.
create schema if not exists basic_auth;
create table if not exists
basic_auth.users (
email text primary key check ( email ~* '^.+@.+\..+$' ),
pass text not null check (length(pass) < 512),
role name not null check (length(role) < 512)
);
We would like the role to be a foreign key to actual database roles, however PostgreSQL does not support these constraints against the :code:`pg_roles` table. We'll use a trigger to manually enforce it.
.. code-block:: plpgsql
create or replace function
basic_auth.check_role_exists() returns trigger as $$
begin
if not exists (select 1 from pg_roles as r where r.rolname = new.role) then
raise foreign_key_violation using message =
'unknown database role: ' || new.role;
return null;
end if;
return new;
end
$$ language plpgsql;
drop trigger if exists ensure_user_role_exists on basic_auth.users;
create constraint trigger ensure_user_role_exists
after insert or update on basic_auth.users
for each row
execute procedure basic_auth.check_role_exists();
Next we'll use the pgcrypto extension and a trigger to keep passwords safe in the :code:`users` table.
.. code-block:: plpgsql
create extension if not exists pgcrypto;
create or replace function
basic_auth.encrypt_pass() returns trigger as $$
begin
if tg_op = 'INSERT' or new.pass <> old.pass then
new.pass = crypt(new.pass, gen_salt('bf'));
end if;
return new;
end
$$ language plpgsql;
drop trigger if exists encrypt_pass on basic_auth.users;
create trigger encrypt_pass
before insert or update on basic_auth.users
for each row
execute procedure basic_auth.encrypt_pass();
With the table in place we can make a helper to check a password against the encrypted column. It returns the database role for a user if the email and password are correct.
.. code-block:: plpgsql
create or replace function
basic_auth.user_role(email text, pass text) returns name
language plpgsql
as $$
begin
return (
select role from basic_auth.users
where users.email = user_role.email
and users.pass = crypt(user_role.pass, users.pass)
);
end;
$$;
.. _public_ui:
Public User Interface
---------------------
In the previous section we created an internal table to store user information. Here we create a login function which takes an email address and password and returns JWT if the credentials match a user in the internal table.
Permissions
~~~~~~~~~~~
Your database roles need access to the schema, tables, views and functions in order to service HTTP requests.
Recall from the `Overview of Role System`_ that PostgREST uses special roles to process requests, namely the authenticator and
anonymous roles. Below is an example of permissions that allow anonymous users to create accounts and attempt to log in.
.. code-block:: postgres
-- the names "anon" and "authenticator" are configurable and not
-- sacred, we simply choose them for clarity
create role anon noinherit;
create role authenticator noinherit;
grant anon to authenticator;
Then, add ``db-anon-role`` to the configuration file to allow anonymous requests.
.. code:: ini
db-anon-role = "anon"
Logins
~~~~~~
As described in `JWT from SQL`_, we'll create a JWT inside our login function. Note that you'll need to adjust the secret key which is hard-coded in this example to a secure (at least thirty-two character) secret of your choosing.
.. code-block:: postgres
-- add type
CREATE TYPE basic_auth.jwt_token AS (
token text
);
-- login should be on your exposed schema
create or replace function
login(email text, pass text) returns basic_auth.jwt_token as $$
declare
_role name;
result basic_auth.jwt_token;
begin
-- check email and password
select basic_auth.user_role(email, pass) into _role;
if _role is null then
raise invalid_password using message = 'invalid user or password';
end if;
select sign(
row_to_json(r), 'reallyreallyreallyreallyverysafe'
) as token
from (
select _role as role, login.email as email,
extract(epoch from now())::integer + 60*60 as exp
) r
into result;
return result;
end;
$$ language plpgsql security definer;
grant execute on function login(text,text) to anon;
Since the above :code:`login` function is defined as `security definer <https://www.postgresql.org/docs/current/sql-createfunction.html#id-1.9.3.67.10.2>`_,
the anonymous user :code:`anon` doesn't need permission to read the :code:`basic_auth.users` table. It doesn't even need permission to access the :code:`basic_auth` schema.
:code:`grant execute on function` is included for clarity but it might not be needed, see :ref:`func_privs` for more details.
An API request to call this function would look like:
.. tabs::
.. code-tab:: http
POST /rpc/login HTTP/1.1
{ "email": "foo@bar.com", "pass": "foobar" }
.. code-tab:: bash Curl
curl "http://localhost:3000/rpc/login" \
-X POST -H "Content-Type: application/json" \
-d '{ "email": "foo@bar.com", "pass": "foobar" }'
The response would look like the snippet below. Try decoding the token at `jwt.io <https://jwt.io/>`_. (It was encoded with a secret of :code:`reallyreallyreallyreallyverysafe` as specified in the SQL code above. You'll want to change this secret in your app!)
.. code:: json
{
"token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImZvb0BiYXIuY29tIiwicGFzcyI6ImZvb2JhciJ9.37066TTRlh-1hXhnA9oO9Pj6lgL6zFuJU0iCHhuCFno"
}
Alternatives
~~~~~~~~~~~~
See the how-to :ref:`sql-user-management-using-postgres-users-and-passwords` for a similar way that completely avoids the table :code:`basic_auth.users`.
+31 -5
View File
@@ -28,7 +28,12 @@ import os
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ["sphinx_tabs.tabs", "sphinx_copybutton"]
extensions = [
"sphinx_tabs.tabs",
"sphinx_copybutton",
"sphinxext.opengraph",
"sphinx_rtd_dark_mode",
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
@@ -45,7 +50,7 @@ source_suffix = ".rst"
master_doc = "index"
# This is overriden by readthedocs with the version tag anyway
version = "10.2"
version = "12.2"
# To avoid repetition in <title> we set this to an empty string.
release = ""
@@ -59,7 +64,7 @@ copyright = "2017, " + author
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
language = "en"
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
@@ -70,7 +75,7 @@ language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "shared/*.rst"]
# The reST default role (used for this markup: `text`) to use for all
# documents.
@@ -287,7 +292,28 @@ def setup(app):
app.add_css_file("css/custom.css")
user_agent = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:135.0) Gecko/20100101 Firefox/135.0"
# taken from https://github.com/sphinx-doc/sphinx/blob/82dad44e5bd3776ecb6fd8ded656bc8151d0e63d/sphinx/util/requests.py#L42
user_agent = "Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0"
linkcheck_ignore = [
r"https://www.patreon.com/postgrest",
r"https://blog.frankel.ch/poor-man-api",
]
# sphinx-tabs configuration
sphinx_tabs_disable_tab_closing = True
# sphinx_rtd_dark_mode configuration
default_dark_mode = False
# sphinxext-opengraph configuration
ogp_image = "_images/logo.png"
ogp_use_first_image = True
ogp_enable_meta_description = True
ogp_description_length = 300
## RTD sets html_baseurl, ensures we use the correct env for canonical URLs
## Useful to generate correct meta tags for Open Graph
## Refs: https://github.com/readthedocs/readthedocs.org/issues/10226, https://github.com/urllib3/urllib3/pull/3064
html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "/")
-728
View File
@@ -1,728 +0,0 @@
.. _configuration:
Configuration
=============
Without configuration, PostgREST won't be able to serve requests. At the minimum it needs either :ref:`a role to serve anonymous requests with <db-anon-role>` - or :ref:`a secret to use for JWT authentication <jwt-secret>`. Config parameters can be provided via :ref:`file_config`, via :ref:`env_variables_config` or through :ref:`in_db_config`.
To connect to a database it uses a `libpq connection string <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_. The connection string can be set in the configuration file or via environment variable or can be read from an external file. See :ref:`db-uri` for details. Any parameter that is not set in the connection string is read from `libpq environment variables <https://www.postgresql.org/docs/current/libpq-envars.html>`_. The default connection string is ``postgresql://``, which reads **all** parameters from the environment.
The user with whom PostgREST connects to the database is also known as the authenticator role. For more information about the anonymous vs authenticator roles see :ref:`roles`.
Config parameters are read in the following order:
1. From the config file.
2. From environment variables, overriding values from the config file.
3. From the database, overriding values from both the config file and environment variables.
.. _file_config:
Config File
-----------
PostgREST can read a config file. There is no predefined location for this file, you must specify the file path as the one and only argument to the server:
.. code:: bash
./postgrest /path/to/postgrest.conf
.. note::
Configuration can be reloaded without restarting the server. See :ref:`config_reloading`.
The configuration file must contain a set of key value pairs:
.. code::
# postgrest.conf
# The standard connection URI format, documented at
# https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING
db-uri = "postgres://user:pass@host:5432/dbname"
# The database role to use when no client authentication is provided.
# Should differ from authenticator
db-anon-role = "anon"
# The secret to verify the JWT for authenticated requests with.
# Needs to be 32 characters minimum.
jwt-secret = "reallyreallyreallyreallyverysafe"
jwt-secret-is-base64 = False
# Port the postgrest process is listening on for http requests
server-port = 80
You can run ``postgrest --example`` to display all possible configuration parameters and how to use them in a configuration file.
.. _env_variables_config:
Environment Variables
---------------------
You can also set these :ref:`configuration parameters <config_full_list>` using environment variables. They are capitalized, have a ``PGRST_`` prefix, and use underscores. For example: ``PGRST_DB_URI`` corresponds to ``db-uri`` and ``PGRST_APP_SETTINGS_*`` to ``app.settings.*``.
.. _in_db_config:
In-Database Configuration
-------------------------
By adding settings to the **authenticator** role (see :ref:`roles`), you can make the database the single source of truth for PostgREST's configuration.
This is enabled by :ref:`db-config`.
For example, you can configure :ref:`db-schemas` and :ref:`jwt-secret` like this:
.. code:: postgresql
ALTER ROLE authenticator SET pgrst.db_schemas = "tenant1, tenant2, tenant3"
ALTER ROLE authenticator IN DATABASE <your_database_name> SET pgrst.jwt_secret = "REALLYREALLYREALLYREALLYVERYSAFE"
You can use both database-specific settings with `IN DATABASE` and cluster-wide settings without it. Database-specific settings will override cluster-wide settings if both are used for the same parameter.
Note that underscores(``_``) need to be used instead of dashes(``-``) for the in-database config parameters.
.. important::
For altering a role in this way, you need a SUPERUSER. You might not be able to use this configuration mode on cloud-hosted databases.
When using both the configuration file and the in-database configuration, the latter takes precedence.
.. danger::
If direct connections to the database are allowed, then it's not safe to use the in-db configuration for storing the :ref:`jwt-secret`.
The settings of every role are PUBLIC - they can be viewed by any user that queries the ``pg_catalog.pg_db_role_setting`` table.
In this case you should keep the :ref:`jwt-secret` in the configuration file or as environment variables.
.. _config_reloading:
Configuration Reloading
=======================
It's possible to reload PostgREST's configuration without restarting the server. You can do this :ref:`via signal <config_reloading_signal>` or :ref:`via notification <config_reloading_notify>`.
It's not possible to change :ref:`env_variables_config` for a running process and reloading a Docker container configuration will not work. In these cases, you need to restart the PostgREST server or use :ref:`in_db_config` as an alternative.
.. important::
The following settings will not be reloaded. You will need to restart PostgREST to change those.
* :ref:`admin-server-port`
* :ref:`db-uri`
* :ref:`db-pool`
* :ref:`db-pool-acquisition-timeout`
* :ref:`db-pool-max-lifetime`
* :ref:`server-host`
* :ref:`server-port`
* :ref:`server-unix-socket`
* :ref:`server-unix-socket-mode`
.. _config_reloading_signal:
Reload with signal
------------------
To reload the configuration via signal, send a SIGUSR2 signal to the server process.
.. code:: bash
killall -SIGUSR2 postgrest
.. _config_reloading_notify:
Reload with NOTIFY
------------------
To reload the configuration from within the database, you can use a NOTIFY command.
.. code:: postgresql
NOTIFY pgrst, 'reload config'
The ``"pgrst"`` notification channel is enabled by default. For configuring the channel, see :ref:`db-channel` and :ref:`db-channel-enabled`.
.. _config_full_list:
List of parameters
==================
=========================== ======= ================= ==========
Name Type Default Reloadable
=========================== ======= ================= ==========
admin-server-port Int
app.settings.* String Y
db-anon-role String Y
db-channel String pgrst Y
db-channel-enabled Boolean True Y
db-config Boolean True Y
db-extra-search-path String public Y
db-max-rows Int ∞ Y
db-plan-enabled Boolean False Y
db-pool Int 10
db-pool-acquisition-timeout Int 10
db-pool-max-lifetime Int 1800
db-pre-request String Y
db-prepared-statements Boolean True Y
db-schemas String public Y
db-tx-end String commit
db-uri String postgresql://
db-use-legacy-gucs Boolean True Y
jwt-aud String Y
jwt-role-claim-key String .role Y
jwt-secret String Y
jwt-secret-is-base64 Boolean False Y
log-level String error Y
openapi-mode String follow-privileges Y
openapi-security-active Boolean False Y
openapi-server-proxy-uri String Y
raw-media-types String Y
server-host String !4
server-port Int 3000
server-unix-socket String
server-unix-socket-mode String 660
=========================== ======= ================= ==========
.. _admin-server-port:
admin-server-port
-----------------
=============== =======================
**Environment** PGRST_ADMIN_SERVER_PORT
**In-Database** `n/a`
=============== =======================
Specifies the port for the :ref:`health_check` endpoints.
.. _app.settings.*:
app.settings.*
--------------
=============== ====================
**Environment** PGRST_APP_SETTINGS_*
**In-Database** pgrst.app_settings_*
=============== ====================
Arbitrary settings that can be used to pass in secret keys directly as strings, or via OS environment variables. For instance: :code:`app.settings.jwt_secret = "$(MYAPP_JWT_SECRET)"` will take :code:`MYAPP_JWT_SECRET` from the environment and make it available to postgresql functions as :code:`current_setting('app.settings.jwt_secret')`.
.. _db-anon-role:
db-anon-role
------------
=============== ==================
**Environment** PGRST_DB_ANON_ROLE
**In-Database** `n/a`
=============== ==================
The database role to use when executing commands on behalf of unauthenticated clients. For more information, see :ref:`roles`.
When unset anonymous access will be blocked.
.. _db-channel:
db-channel
----------
=============== ================
**Environment** PGRST_DB_CHANNEL
**In-Database** `n/a`
=============== ================
The name of the notification channel that PostgREST uses for :ref:`schema_reloading` and configuration reloading.
.. _db-channel-enabled:
db-channel-enabled
------------------
=============== ========================
**Environment** PGRST_DB_CHANNEL_ENABLED
**In-Database** `n/a`
=============== ========================
When this is set to :code:`true`, the notification channel specified in :ref:`db-channel` is enabled.
You should set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
.. _db-config:
db-config
---------
=============== ===============
**Environment** PGRST_DB_CONFIG
**In-Database** `n/a`
=============== ===============
Enables the in-database configuration.
.. _db-extra-search-path:
db-extra-search-path
--------------------
=============== ==========================
**Environment** PGRST_DB_EXTRA_SEARCH_PATH
**In-Database** pgrst.db_extra_search_path
=============== ==========================
Extra schemas to add to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request. These schemas tables, views and stored procedures **don't get API endpoints**, they can only be referred from the database objects inside your :ref:`db-schemas`.
This parameter was meant to make it easier to use **PostgreSQL extensions** (like PostGIS) that are outside of the :ref:`db-schemas`.
Multiple schemas can be added in a comma-separated string, e.g. ``public, extensions``.
.. _db-max-rows:
db-max-rows
-----------
*For backwards compatibility, this config parameter is also available without prefix as "max-rows".*
=============== =================
**Environment** PGRST_DB_MAX_ROWS
**In-Database** pgrst.db_max_rows
=============== =================
A hard limit to the number of rows PostgREST will fetch from a view, table, or stored procedure. Limits payload size for accidental or malicious requests.
.. _db-plan-enabled:
db-plan-enabled
---------------
=============== =====================
**Environment** PGRST_DB_PLAN_ENABLED
**In-Database** pgrst.db_plan_enabled
=============== =====================
When this is set to :code:`true`, the execution plan of a request can be retrieved by using the :code:`Accept: application/vnd.pgrst.plan` header. See :ref:`explain_plan`.
It's recommended to use this in testing environments only since it reveals internal database details.
However, if you choose to use it in production you can add a :ref:`db-pre-request` to filter the requests that can use this feature.
For example, to only allow requests from an IP address to get the execution plans:
.. code-block:: postgresql
-- Assuming a proxy(Nginx, Cloudflare, etc) passes an "X-Forwarded-For" header(https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For)
create or replace function filter_plan_requests()
returns void as $$
declare
headers json := current_setting('request.headers', true)::json;
client_ip text := coalesce(headers->>'x-forwarded-for', '');
accept text := coalesce(headers->>'accept', '');
begin
if accept like 'application/vnd.pgrst.plan%' and client_ip != '144.96.121.73' then
raise insufficient_privilege using
message = 'Not allowed to use application/vnd.pgrst.plan';
end if;
end; $$ language plpgsql;
-- set this function on your postgrest.conf
-- db-pre-request = filter_plan_requests
.. _db-pool:
db-pool
-------
=============== =================
**Environment** PGRST_DB_POOL
**In-Database** `n/a`
=============== =================
Number of connections to keep open in PostgREST's database pool. Having enough here for the maximum expected simultaneous client connections can improve performance. Note it's pointless to set this higher than the :code:`max_connections` GUC in your database.
.. _db-pool-acquisition-timeout:
db-pool-acquisition-timeout
---------------------------
=============== =================
**Environment** PGRST_DB_POOL_ACQUISITION_TIMEOUT
**In-Database** `n/a`
=============== =================
Specifies the maximum time in seconds that the request will wait for the pool to free up a connection slot to the database. If it times out without acquiring a connection, then the request is aborted and a ``504`` error is returned.
.. _db-pool-max-lifetime:
db-pool-max-lifetime
--------------------
=============== =================
**Environment** PGRST_DB_POOL_MAX_LIFETIME
**In-Database** `n/a`
=============== =================
Specifies the maximum time in seconds of an existing connection in the pool. When this lifetime is reached, then the connection will be closed and returned to the pool.
.. _db-pre-request:
db-pre-request
--------------
*For backwards compatibility, this config parameter is also available without prefix as "pre-request".*
=============== =================
**Environment** PGRST_DB_PRE_REQUEST
**In-Database** pgrst.db_pre_request
=============== =================
A schema-qualified stored procedure name to call right after switching roles for a client request. This provides an opportunity to modify SQL variables or raise an exception to prevent the request from completing.
.. _db-prepared-statements:
db-prepared-statements
----------------------
=============== =================
**Environment** PGRST_DB_PREPARED_STATEMENTS
**In-Database** pgrst.db_prepared_statements
=============== =================
Enables or disables prepared statements.
When disabled, the generated queries will be parameterized (invulnerable to SQL injection) but they will not be prepared (cached in the database session). Not using prepared statements will noticeably decrease performance, so it's recommended to always have this setting enabled.
You should only set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
.. _db-schemas:
db-schemas
----------
*For backwards compatibility, this config parameter is also available in singular as "db-schema".*
=============== =================
**Environment** PGRST_DB_SCHEMAS
**In-Database** pgrst.db_schemas
=============== =================
The database schema to expose to REST clients. Tables, views and stored procedures in this schema will get API endpoints.
.. code:: bash
db-schemas = "api"
This schema gets added to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request.
List of schemas
~~~~~~~~~~~~~~~
You can also specify a list of schemas that can be used for **schema-based multitenancy** and **api versioning** by :ref:`multiple-schemas`. Example:
.. code:: bash
db-schemas = "tenant1, tenant2"
If you don't :ref:`Switch Schemas <multiple-schemas>`, the first schema in the list(``tenant1`` in this case) is chosen as the default schema.
*Only the chosen schema* gets added to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request.
.. warning::
Never expose private schemas in this way. See :ref:`schema_isolation`.
.. _db-tx-end:
db-tx-end
---------
=============== =================
**Environment** PGRST_DB_TX_END
**In-Database** pgrst.db_tx_end
=============== =================
Specifies how to terminate the database transactions.
.. code:: bash
# The transaction is always committed
db-tx-end = "commit"
# The transaction is committed unless a "Prefer: tx=rollback" header is sent
db-tx-end = "commit-allow-override"
# The transaction is always rolled back
db-tx-end = "rollback"
# The transaction is rolled back unless a "Prefer: tx=commit" header is sent
db-tx-end = "rollback-allow-override"
.. _db-uri:
db-uri
------
=============== =================
**Environment** PGRST_DB_URI
**In-Database** `n/a`
=============== =================
The standard connection PostgreSQL `URI format <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_. Symbols and unusual characters in the password or other fields should be percent encoded to avoid a parse error. If enforcing an SSL connection to the database is required you can use `sslmode <https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS>`_ in the URI, for example ``postgres://user:pass@host:5432/dbname?sslmode=require``.
When running PostgREST on the same machine as PostgreSQL, it is also possible to connect to the database using a `Unix socket <https://en.wikipedia.org/wiki/Unix_domain_socket>`_ and the `Peer Authentication method <https://www.postgresql.org/docs/current/auth-peer.html>`_ as an alternative to TCP/IP communication and authentication with a password, this also grants higher performance. To do this you can omit the host and the password, e.g. ``postgres://user@/dbname``, see the `libpq connection string <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_ documentation for more details.
Choosing a value for this parameter beginning with the at sign such as ``@filename`` (e.g. ``@./configs/my-config``) loads the connection string out of an external file.
.. _db-use-legacy-gucs:
db-use-legacy-gucs
------------------
=============== =================
**Environment** PGRST_DB_USE_LEGACY_GUCS
**In-Database** pgrst.db_use_legacy_gucs
=============== =================
Determine if GUC request settings for headers, cookies and jwt claims use the :ref:`legacy names <guc_legacy_names>` (string with dashes, invalid starting from PostgreSQL v14) with text values instead of the :ref:`new names <guc_req_headers_cookies_claims>` (string without dashes, valid on all PostgreSQL versions) with json values.
On PostgreSQL versions 14 and above, this parameter is ignored.
.. _jwt-aud:
jwt-aud
-------
=============== =================
**Environment** PGRST_JWT_AUD
**In-Database** pgrst.jwt_aud
=============== =================
Specifies the `JWT audience claim <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.3>`_. If this claim is present in the client provided JWT then you must set this to the same value as in the JWT, otherwise verifying the JWT will fail.
.. _jwt-role-claim-key:
jwt-role-claim-key
------------------
*For backwards compatibility, this config parameter is also available without prefix as "role-claim-key".*
=============== =================
**Environment** PGRST_JWT_ROLE_CLAIM_KEY
**In-Database** pgrst.jwt_role_claim_key
=============== =================
A JSPath DSL that specifies the location of the :code:`role` key in the JWT claims. This can be used to consume a JWT provided by a third party service like Auth0, Okta or Keycloak. Usage examples:
.. code:: bash
# {"postgrest":{"roles": ["other", "author"]}}
# the DSL accepts characters that are alphanumerical or one of "_$@" as keys
jwt-role-claim-key = ".postgrest.roles[1]"
# {"https://www.example.com/role": { "key": "author }}
# non-alphanumerical characters can go inside quotes(escaped in the config value)
jwt-role-claim-key = ".\"https://www.example.com/role\".key"
.. _jwt-secret:
jwt-secret
----------
=============== =================
**Environment** PGRST_JWT_SECRET
**In-Database** pgrst.jwt_secret
=============== =================
The secret or `JSON Web Key (JWK) (or set) <https://datatracker.ietf.org/doc/html/rfc7517>`_ used to decode JWT tokens clients provide for authentication. For security the key must be **at least 32 characters long**. If this parameter is not specified then PostgREST refuses authentication requests. Choosing a value for this parameter beginning with the at sign such as :code:`@filename` loads the secret out of an external file. This is useful for automating deployments. Note that any binary secrets must be base64 encoded. Both symmetric and asymmetric cryptography are supported. For more info see :ref:`asym_keys`.
Choosing a value for this parameter beginning with the at sign such as ``@filename`` (e.g. ``@./configs/my-config``) loads the secret out of an external file.
.. warning::
Only when using the :ref:`file_config`, if the ``jwt-secret`` contains a ``$`` character by itself it will give errors. In this case, use ``$$`` and PostgREST will interpret it as a single ``$`` character.
.. _jwt-secret-is-base64:
jwt-secret-is-base64
--------------------
=============== =================
**Environment** PGRST_JWT_SECRET_IS_BASE64
**In-Database** pgrst.jwt_secret_is_base64
=============== =================
When this is set to :code:`true`, the value derived from :code:`jwt-secret` will be treated as a base64 encoded secret.
.. _log-level:
log-level
---------
=============== =================
**Environment** PGRST_LOG_LEVEL
**In-Database** `n/a`
=============== =================
Specifies the level of information to be logged while running PostgREST.
.. code:: bash
# Only startup and db connection recovery messages are logged
log-level = "crit"
# All the "crit" level events plus server errors (status 5xx) are logged
log-level = "error"
# All the "error" level events plus request errors (status 4xx) are logged
log-level = "warn"
# All the "warn" level events plus all requests (every status code) are logged
log-level = "info"
Because currently there's no buffering for logging, the levels with minimal logging(``crit/error``) will increase throughput.
.. _openapi-mode:
openapi-mode
------------
=============== =================
**Environment** PGRST_OPENAPI_MODE
**In-Database** pgrst.openapi_mode
=============== =================
Specifies how the OpenAPI output should be displayed.
.. code:: bash
# Follows the privileges of the JWT role claim (or from db-anon-role if the JWT is not sent)
# Shows information depending on the permissions that the role making the request has
openapi-mode = "follow-privileges"
# Ignores the privileges of the JWT role claim (or from db-anon-role if the JWT is not sent)
# Shows all the exposed information, regardless of the permissions that the role making the request has
openapi-mode = "ignore-privileges"
# Disables the OpenApi output altogether.
# Throws a `404 Not Found` error when accessing the API root path
openapi-mode = "disabled"
.. _openapi-security-active:
openapi-security-active
-----------------------
=============== =============================
**Environment** PGRST_OPENAPI_SECURITY_ACTIVE
**In-Database** pgrst.openapi_security_active
=============== =============================
When this is set to :code:`true`, security options are included in the :ref:`OpenAPI output <open-api>`.
.. _openapi-server-proxy-uri:
openapi-server-proxy-uri
------------------------
=============== =================
**Environment** PGRST_OPENAPI_SERVER_PROXY_URI
**In-Database** pgrst.openapi_server_proxy_uri
=============== =================
Overrides the base URL used within the OpenAPI self-documentation hosted at the API root path. Use a complete URI syntax :code:`scheme:[//[user:password@]host[:port]][/]path[?query][#fragment]`. Ex. :code:`https://postgrest.com`
.. code:: json
{
"swagger": "2.0",
"info": {
"version": "0.4.3.0",
"title": "PostgREST API",
"description": "This is a dynamic API generated by PostgREST"
},
"host": "postgrest.com:443",
"basePath": "/",
"schemes": [
"https"
]
}
.. _raw-media-types:
raw-media-types
---------------
=============== =================
**Environment** PGRST_RAW_MEDIA_TYPES
**In-Database** pgrst.raw_media_types
=============== =================
This serves to extend the `Media Types <https://en.wikipedia.org/wiki/Media_type>`_ that PostgREST currently accepts through an ``Accept`` header.
These media types can be requested by following the same rules as the ones defined in :ref:`scalar_return_formats`.
As an example, the below config would allow you to request an **image** and a **XML** file by doing a request with ``Accept: image/png``
or ``Accept: font/woff2``, respectively.
.. code:: bash
raw-media-types="image/png, font/woff2"
.. _server-host:
server-host
-----------
=============== =================
**Environment** PGRST_SERVER_HOST
**In-Database** `n/a`
=============== =================
Where to bind the PostgREST web server. In addition to the usual address options, PostgREST interprets these reserved addresses with special meanings:
* :code:`*` - any IPv4 or IPv6 hostname
* :code:`*4` - any IPv4 or IPv6 hostname, IPv4 preferred
* :code:`!4` - any IPv4 hostname
* :code:`*6` - any IPv4 or IPv6 hostname, IPv6 preferred
* :code:`!6` - any IPv6 hostname
.. _server-port:
server-port
-----------
=============== =================
**Environment** PGRST_SERVER_PORT
**In-Database** `n/a`
=============== =================
The TCP port to bind the web server.
.. _server-unix-socket:
server-unix-socket
------------------
=============== =================
**Environment** PGRST_SERVER_UNIX_SOCKET
**In-Database** `n/a`
=============== =================
`Unix domain socket <https://en.wikipedia.org/wiki/Unix_domain_socket>`_ where to bind the PostgREST web server.
If specified, this takes precedence over :ref:`server-port`. Example:
.. code:: bash
server-unix-socket = "/tmp/pgrst.sock"
.. _server-unix-socket-mode:
server-unix-socket-mode
-----------------------
=============== =================
**Environment** PGRST_SERVER_UNIX_SOCKET_MODE
**In-Database** `n/a`
=============== =================
`Unix file mode <https://en.wikipedia.org/wiki/File_system_permissions>`_ to be set for the socket specified in :ref:`server-unix-socket`
Needs to be a valid octal between 600 and 777.
.. code:: bash
server-unix-socket-mode = "660"
-91
View File
@@ -1,91 +0,0 @@
let
# Commit of the Nixpkgs repository that we want to use.
nixpkgsVersion = {
date = "2021-06-02";
rev = "84aa23742f6c72501f9cc209f29c438766f5352d";
tarballHash = "0h7xl6q0yjrbl9vm3h6lkxw692nm8bg3wy65gm95a2mivhrdjpxp";
};
# Nix files that describe the Nixpkgs repository. We evaluate the expression
# using `import` below.
pkgs = import
(fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${nixpkgsVersion.rev}.tar.gz";
sha256 = nixpkgsVersion.tarballHash;
})
{ };
sphinxTabsPkg = ps: ps.callPackage ./extensions/sphinx-tabs.nix { };
sphinxCopybuttonPkg = ps: ps.callPackage ./extensions/sphinx-copybutton.nix { };
python = pkgs.python3.withPackages (ps: [ ps.sphinx ps.sphinx_rtd_theme ps.livereload (sphinxTabsPkg ps) (sphinxCopybuttonPkg ps) ]);
in
{
inherit pkgs;
build =
pkgs.writeShellScriptBin "postgrest-docs-build"
''
set -euo pipefail
cd "$(${pkgs.git}/bin/git rev-parse --show-toplevel)/docs"
# clean previous build, otherwise some errors might be supressed
rm -rf _build
${python}/bin/sphinx-build --color -W -b html -a -n . _build
'';
serve =
pkgs.writeShellScriptBin "postgrest-docs-serve"
''
set -euo pipefail
cd "$(${pkgs.git}/bin/git rev-parse --show-toplevel)/docs"
# livereload_docs.py needs to find "sphinx-build"
PATH=${python}/bin:$PATH
${python}/bin/python livereload_docs.py
'';
spellcheck =
pkgs.writeShellScriptBin "postgrest-docs-spellcheck"
''
set -euo pipefail
cd "$(${pkgs.git}/bin/git rev-parse --show-toplevel)/docs"
FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')
cat $FILES \
| grep -v '^\(\.\.\| \)' \
| sed 's/`.*`//g' \
| ${pkgs.aspell}/bin/aspell -d ${pkgs.aspellDicts.en}/lib/aspell/en_US -p ./postgrest.dict list \
| sort -f \
| tee misspellings
test ! -s misspellings
'';
# dictcheck detects obsolete entries in postgrest.dict, that are not used anymore
dictcheck =
pkgs.writeShellScriptBin "postgrest-docs-dictcheck"
''
set -euo pipefail
cd "$(${pkgs.git}/bin/git rev-parse --show-toplevel)/docs"
FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')
cat postgrest.dict \
| tail -n+2 \
| tr '\n' '\0' \
| xargs -0 -n 1 -i \
sh -c "grep \"{}\" $FILES > /dev/null || echo \"{}\""
'';
linkcheck =
pkgs.writeShellScriptBin "postgrest-docs-linkcheck"
''
set -euo pipefail
cd "$(${pkgs.git}/bin/git rev-parse --show-toplevel)/docs"
${python}/bin/sphinx-build --color -b linkcheck . _build
'';
}
+17 -47
View File
@@ -6,16 +6,24 @@ Community Tutorials
* `Building a Contacts List with PostgREST and Vue.js <https://www.youtube.com/watch?v=iHtsALtD5-U>`_ -
In this video series, DigitalOcean shows how to build and deploy an Nginx + PostgREST(using a managed PostgreSQL database) + Vue.js webapp in an Ubuntu server droplet.
* `PostgREST + Auth0: Create REST API in minutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest-1-introduction/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service.
* `PostgREST + Auth0: Create REST API in mintutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service.
* `PostgREST + PostGIS API tutorial in 5 minutes <https://gis-ops.com/postgrest-postgis-api-tutorial-geospatial-api-in-5-minutes/>`_ -
In this tutorial, GIS • OPS shows how to perform PostGIS calculations through PostgREST :ref:`s_procs` interface.
In this tutorial, GIS • OPS shows how to perform PostGIS calculations through PostgREST :ref:`functions` interface.
* `"CodeLess" backend using postgres, postgrest and oauth2 authentication with keycloak <https://www.mathieupassenaud.fr/codeless_backend/>`_ -
A step-by-step tutorial for using PostgREST with KeyCloak(hosted on a managed service).
* `How PostgreSQL triggers work when called with a PostgREST PATCH HTTP request <https://blog.fgribreau.com/2020/11/how-postgresql-triggers-works-when.html>`_ - A tutorial to see how the old and new values are set or not when doing a PATCH request to PostgREST.
* `REST Data Service on YugabyteDB / PostgreSQL <https://dev.to/yugabyte/rest-data-service-on-yugabytedb-postgresql-5f2h>`_
* `Build data-driven applications with Workers and PostgreSQL <https://developers.cloudflare.com/workers/tutorials/postgres/>`_ - A tutorial on how to integrate with PostgREST and PostgreSQL using Cloudflare Workers.
* `A poor man's API <https://blog.frankel.ch/poor-man-api>`_ - Shows how to integrate PostgREST with Apache APISIX as an alternative to Nginx.
.. * `Accessing a PostgreSQL database in Godot 4 via PostgREST <https://peterkingsbury.com/2022/08/16/godot-postgresql-postgrest/>`_
.. _templates:
Templates
@@ -29,28 +37,10 @@ Templates
Example Apps
------------
* `chronicle <https://github.com/srid/chronicle>`_ - tracking a tree of personal memories
* `code-du-travail-backoffice <https://github.com/SocialGouv/code-du-travail-backoffice>`_ - data administration portal for the official French Labor Code and Agreements
* `delibrium-postgrest <https://gitlab.com/delibrium/delibrium-postgrest/>`_ - example school API and front-end in Vue.js
* `elm-workshop <https://github.com/diogob/elm-workshop>`_ - building a simple database query UI
* `ember-postgrest-dynamic-ui <https://github.com/benoror/ember-postgrest-dynamic-ui>`_ - generating Ember forms to edit data
* `ETH-transactions-storage <https://github.com/Adamant-im/ETH-transactions-storage>`_ - indexer for Ethereum to get transaction list by ETH address
* `ext-postgrest-crud <https://github.com/timwis/ext-postgrest-crud>`_ - browser-based spreadsheet
* `general <https://github.com/PierreRochard/general>`_ - example auth back-end
* `goodfilm <https://github.com/tyrchen/goodfilm>`_ - example film API
* `guild-operators <https://github.com/cardano-community/koios-artifacts/tree/main/files/grest>`_ - example queries and functions that the Cardano Community uses for their Guild Operators' Repository
* `handsontable-postgrest <https://github.com/timwis/handsontable-postgrest>`_ - an excel-like database table editor
* `heritage-near-me <https://github.com/CodeforAustralia/heritage-near-me>`_ - Elm and PostgREST with PostGIS
* `ng-admin-postgrest <https://github.com/marmelab/ng-admin-postgrest>`_ - automatic database admin panel
* `pgrst-dev-setup <https://github.com/Qu4tro/pgrst-dev-setup>`_ - docker-compose and tmuxp setup for experimentation.
* `postgres-postgrest-cloudflared-example <https://github.com/cloudflare/postgres-postgrest-cloudflared-example>`_ - docker-compose setup exposing PostgREST using cloudfared
* `postgrest-demo <https://github.com/SMRxT/postgrest-demo>`_ - multi-tenant logging system
* `postgrest-example <https://github.com/begriffs/postgrest-example>`_ - sqitch versioning for API
* `postgrest-sessions-example <https://github.com/monacoremo/postgrest-sessions-example>`_ - example for cookie-based sessions
* `postgrest-translation-proxy <https://github.com/NikolayS/postgrest-translation-proxy>`_ - calling to external translation service
* `postgrest-ui <https://github.com/tatut/postgrest-ui>`_ - ClojureScript UI components for PostgREST
* `postgrest-vercel <https://github.com/seveibar/postgrest-vercel>`_ - run PostgREST on Vercel (Serverless/AWS Lambda)
* `PostgrestSkeleton <https://github.com/Recmo/PostgrestSkeleton>`_ - Docker Compose, PostgREST, Nginx and Auth0
* `PostGUI <https://github.com/priyank-purohit/PostGUI>`_ - React Material UI admin panel
* `prospector <https://github.com/sfcta/prospector>`_ - data warehouse and visualization platform
@@ -61,24 +51,22 @@ DevOps
* `cloudgov-demo-postgrest <https://github.com/GSA/cloudgov-demo-postgrest>`_ - demo for a federally-compliant REST API on cloud.gov
* `cloudstark/helm-charts <https://github.com/cloudstark/helm-charts/tree/master/postgrest>`_ - helm chart to deploy PostgREST to a Kubernetes cluster via a Deployment and Service
* `cyril-sabourault/postgrest-cloud-run <https://github.com/cyril-sabourault/postgrest-cloud-run>`_ - expose a PostgreSQL database on Cloud SQL using Cloud Run
* `eyberg/postgrest <https://repo.ops.city/v2/packages/eyberg/postgrest/10.1.1/x86_64/show>`_ - run PostgREST as a Nanos unikernel
* `jbkarle/postgrest <https://github.com/jbkarle/postgrest>`_ - helm chart with a demo database for development and test purposes
* `Limezest/postgrest-cloud-run <https://github.com/Limezest/postgrest-cloud-run>`_ - expose a PostgreSQL database on Cloud SQL using Cloud Run
.. _eco_external_notification:
External Notification
---------------------
These are PostgreSQL bridges that propagate LISTEN/NOTIFY to external queues for further processing. This allows stored procedures to initiate actions outside the database such as sending emails.
These are PostgreSQL bridges that propagate LISTEN/NOTIFY to external queues for further processing. This allows functions to initiate actions outside the database such as sending emails.
* `pg-bridge <https://github.com/matthewmueller/pg-bridge>`_ - Amazon SNS
* `pg-kinesis-bridge <https://github.com/daurnimator/pg-kinesis-bridge>`_ - Amazon Kinesis
* `pg-notify-stdout <https://github.com/mkleczek/pg-notify-stdout>`_ - writes notifications to standard output (use in shell scripts etc.)
* `pg-notify-webhook <https://github.com/vbalasu/pg-notify-webhook>`_ - trigger webhooks from PostgreSQL's LISTEN/NOTIFY
* `pgsql-listen-exchange <https://github.com/gmr/pgsql-listen-exchange>`_ - RabbitMQ
* `postgres-websockets <https://github.com/diogob/postgres-websockets>`_ - expose web sockets for PostgreSQL's LISTEN/NOTIFY
* `postgresql-to-amqp <https://github.com/FGRibreau/postgresql-to-amqp>`_ - AMQP
* `postgresql2websocket <https://github.com/frafra/postgresql2websocket>`_ - Websockets
* `skeeter <https://github.com/SpiderOak/skeeter>`_ - ZeroMQ
.. _eco_extensions:
@@ -88,41 +76,23 @@ Extensions
* `aiodata <https://github.com/Exahilosys/aiodata>`_ - Python, event-based proxy and caching client.
* `pg-safeupdate <https://github.com/eradman/pg-safeupdate>`_ - prevent full-table updates or deletes
* `postgrest-auth (criles25) <https://github.com/criles25/postgrest-auth>`_ - email based auth/signup
* `postgrest-node <https://github.com/seveibar/postgrest-node>`_ - Run a PostgREST server in Node.js via npm module
* `postgrest-oauth <https://github.com/nblumoe/postgrest-oauth>`_ - OAuth2 WAI middleware
* `postgrest-oauth/api <https://github.com/postgrest-oauth/api>`_ - OAuth2 server
* `PostgREST-writeAPI <https://github.com/ppKrauss/PostgREST-writeAPI>`_ - generate Nginx rewrite rules to fit an OpenAPI spec
* `spas <https://github.com/srid/spas>`_ - allow file uploads and basic auth
.. _clientside_libraries:
Client-Side Libraries
---------------------
* `aor-postgrest-client <https://github.com/tomberek/aor-postgrest-client>`_ - JS, admin-on-rest
* `elm-postgrest <https://github.com/john-kelly/elm-postgrest>`_ - Elm
* `general-angular <https://github.com/PierreRochard/general-angular>`_ - TypeScript, generate UI from API description
* `jarvus-postgrest-apikit <https://github.com/JarvusInnovations/jarvus-postgrest-apikit>`_ - JS, Sencha framework
* `mithril-postgrest <https://github.com/catarse/mithril-postgrest>`_ - JS, Mithril
* `ng-postgrest <https://github.com/team142/ng-postgrest>`_ - Angular app for browsing, editing data exposed over PostgREST.
* `postgrest-client <https://github.com/calebmer/postgrest-client>`_ - JS
* `postgrest-csharp <https://github.com/supabase-community/postgrest-csharp>`_ - C#
* `postgrest-dart <https://github.com/supabase-community/postgrest-dart>`_ - Dart
* `postgrest-ex <https://github.com/J0/postgrest-ex>`_ - Elixir
* `postgrest-dart <https://github.com/supabase/postgrest-dart>`_ - Dart
* `postgrest-ex <https://github.com/supabase-community/postgrest-ex>`_ - Elixir
* `postgrest-go <https://github.com/supabase-community/postgrest-go>`_ - Go
* `postgrest-js <https://github.com/supabase/postgrest-js>`_ - TypeScript/JavaScript
* `postgrest-kt <https://github.com/supabase-community/postgrest-kt>`_ - Kotlin
* `postgrest-py <https://github.com/supabase/postgrest-py>`_ - Python
* `postgrest-request <https://github.com/lewisjared/postgrest-request>`_ - JS, SuperAgent
* `postgrest-py <https://github.com/supabase-community/postgrest-py>`_ - Python
* `postgrest-rs <https://github.com/supabase-community/postgrest-rs>`_ - Rust
* `postgrest-sharp-client <https://github.com/thejettdurham/postgrest-sharp-client>`_ (needs maintainer) - C#, RestSharp
* `postgrest-swift <https://github.com/supabase-community/postgrest-swift>`_ - Swift
* `postgrest-url <https://github.com/hugomrdias/postgrest-url>`_ - JS, just for generating query URLs
* `postgrest_python_requests_client <https://github.com/davidthewatson/postgrest_python_requests_client>`_ - Python
* `postgrester <https://github.com/ivangabriele/postgrester>`_ - JS + Typescript
* `postgrestR <https://github.com/clesiemo3/postgrestR>`_ - R
* `py-postgrest <https://github.com/Kong/py-postgrest>`_ - Python
* `redux-postgrest <https://github.com/andytango/redux-postgrest>`_ - TypeScript/JS, client integrated with (React) Redux.
* `vue-postgrest <https://github.com/technowledgy/vue-postgrest>`_ - Vue.js
+95
View File
@@ -0,0 +1,95 @@
Architecture
############
This page describes the architecture of PostgREST.
Bird's Eye View
===============
You can click on the components to navigate to their respective documentation.
.. container:: img-dark
.. See https://github.com/sphinx-doc/sphinx/issues/2240#issuecomment-187366626
.. raw:: html
<object width="100%" data="../_static/arch-dark.svg" type="image/svg+xml"></object>
.. container:: img-light
.. raw:: html
<object width="100%" data="../_static/arch.svg" type="image/svg+xml"></object>
Code Map
========
This section talks briefly about various important modules.
Main
----
The starting point of the program is `Main.hs <https://github.com/PostgREST/postgrest/blob/main/main/Main.hs>`_.
CLI
---
Main then calls `CLI.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/CLI.hs>`_, which is in charge of :ref:`cli`.
App
---
`App.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/App.hs>`_ is then in charge of composing the different modules.
Auth
----
`Auth.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Auth.hs>`_ is in charge of :ref:`authn`.
Api Request
-----------
`ApiRequest.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/ApiRequest.hs>`_ is in charge of parsing the URL query string (following PostgREST syntax), the request headers, and the request body.
A request might be rejected at this level if it's invalid. For example when providing an unknown media type to PostgREST or using an unknown HTTP method.
Plan
----
Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Plan.hs>`_ fills in out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request.
A request might be rejected at this level if it's invalid. For example when doing resource embedding on a nonexistent resource.
Query
-----
`Query.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Query.hs>`_ generates the SQL queries (parametrized and prepared) required to satisfy the user request.
Only at this stage a connection from the pool might be used.
Schema Cache
------------
`SchemaCache.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/SchemaCache.hs>`_ is in charge of :ref:`schema_cache`.
Config
------
`Config.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Config.hs>`_ is in charge of :ref:`configuration`.
Admin
-----
`Admin.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Admin.hs>`_ is in charge of the :ref:`admin_server`.
HTTP
----
The HTTP server is provided by `Warp <https://aosabook.org/en/posa/warp.html>`_.
Listener
--------
`Listener.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Listener.hs>`_ is in charge of the :ref:`listener`.
+198
View File
@@ -0,0 +1,198 @@
.. _db_authz:
Database Authorization
######################
Database authorization is the process of granting and verifying database access permissions. PostgreSQL manages permissions using the concept of roles.
Users and Groups
================
A role can be thought of as either a database user, or a group of database users, depending on how the role is set up.
Roles for Each Web User
-----------------------
PostgREST can accommodate either viewpoint. If you treat a role as a single user then the :ref:`jwt_impersonation` does most of what you need. When an authenticated user makes a request PostgREST will switch into the database role for that user, which in addition to restricting queries, is available to SQL through the :code:`current_user` variable.
You can use row-level security to flexibly restrict visibility and access for the current user. Here is an `example <https://www.2ndquadrant.com/en/blog/application-users-vs-row-level-security/>`_ from Tomas Vondra, a chat table storing messages sent between users. Users can insert rows into it to send messages to other users, and query it to see messages sent to them by other users.
.. code-block:: postgres
CREATE TABLE chat (
message_uuid UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
message_time TIMESTAMP NOT NULL DEFAULT now(),
message_from NAME NOT NULL DEFAULT current_user,
message_to NAME NOT NULL,
message_subject VARCHAR(64) NOT NULL,
message_body TEXT
);
ALTER TABLE chat ENABLE ROW LEVEL SECURITY;
We want to enforce a policy that ensures a user can see only those messages sent by them or intended for them. Also we want to prevent a user from forging the ``message_from`` column with another person's name.
PostgreSQL allows us to set this policy with row-level security:
.. code-block:: postgres
CREATE POLICY chat_policy ON chat
USING ((message_to = current_user) OR (message_from = current_user))
WITH CHECK (message_from = current_user)
Anyone accessing the generated API endpoint for the chat table will see exactly the rows they should, without our needing custom imperative server-side coding.
.. warning::
Roles are namespaced per-cluster rather than per-database so they may be prone to collision.
Web Users Sharing Role
----------------------
Alternately database roles can represent groups instead of (or in addition to) individual users. You may choose that all signed-in users for a web app share the role ``webuser``. You can distinguish individual users by including extra claims in the JWT such as email.
.. code:: json
{
"role": "webuser",
"email": "john@doe.com"
}
SQL code can access claims through PostgREST :ref:`tx_settings`. For instance to get the email claim, call this function:
.. code:: sql
current_setting('request.jwt.claims', true)::json->>'email';
.. note::
For PostgreSQL < 14
.. code:: sql
current_setting('request.jwt.claim.email', true);
This allows JWT generation services to include extra information and your database code to react to it. For instance the RLS example could be modified to use this ``current_setting`` rather than ``current_user``. The second ``'true'`` argument tells ``current_setting`` to return NULL if the setting is missing from the current configuration.
Hybrid User-Group Roles
-----------------------
You can mix the group and individual role policies. For instance we could still have a webuser role and individual users which inherit from it:
.. code-block:: postgres
CREATE ROLE webuser NOLOGIN;
-- grant this role access to certain tables etc
CREATE ROLE user000 NOLOGIN;
GRANT webuser TO user000;
-- now user000 can do whatever webuser can
GRANT user000 TO authenticator;
-- allow authenticator to switch into user000 role
-- (the role itself has nologin)
Schemas
=======
You must explicitly allow roles to access the exposed schemas in :ref:`db-schemas`.
.. code-block:: postgres
GRANT USAGE ON SCHEMA api TO webuser;
Tables
======
To let web users access tables you must grant them privileges for the operations you want them to do.
.. code-block:: postgres
GRANT
SELECT
, INSERT
, UPDATE(message_body)
, DELETE
ON chat TO webuser;
You can also choose on which table columns the operation is valid. In the above example, the web user can only update the ``message_body`` column.
.. _func_privs:
Functions
=========
By default, when a function is created, the privilege to execute it is not restricted by role. The function access is ``PUBLIC`` — executable by all roles (more details at `PostgreSQL Privileges page <https://www.postgresql.org/docs/current/ddl-priv.html>`_). This is not ideal for an API schema. To disable this behavior, you can run the following SQL statement:
.. code-block:: postgres
ALTER DEFAULT PRIVILEGES REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;
This will change the privileges for all functions created in the future in all schemas. Currently there is no way to limit it to a single schema. In our opinion it's a good practice anyway.
.. note::
It is however possible to limit the effect of this clause only to functions you define. You can put the above statement at the beginning of the API schema definition, and then at the end reverse it with:
.. code-block:: postgres
ALTER DEFAULT PRIVILEGES GRANT EXECUTE ON FUNCTIONS TO PUBLIC;
This will work because the :code:`alter default privileges` statement has effect on function created *after* it is executed. See `PostgreSQL alter default privileges <https://www.postgresql.org/docs/current/sql-alterdefaultprivileges.html>`_ for more details.
After that, you'll need to grant EXECUTE privileges on functions explicitly:
.. code-block:: postgres
GRANT EXECUTE ON FUNCTION login TO anonymous;
GRANT EXECUTE ON FUNCTION signup TO anonymous;
You can also grant execute on all functions in a schema to a higher privileged role:
.. code-block:: postgres
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA api TO web_user;
Security definer
----------------
A function is executed with the privileges of the user who calls it. This means that the user has to have all permissions to do the operations the function performs.
If the function accesses private database objects, your :ref:`API roles <roles>` won't be able to successfully execute the function.
Another option is to define the function with the :code:`SECURITY DEFINER` option. Then only one permission check will take place, the permission to call the function, and the operations in the function will have the authority of the user who owns the function itself.
.. code-block:: postgres
-- login as a user wich has privileges on the private schemas
-- create a sample function
create or replace function login(email text, pass text, out token text) as $$
begin
-- access to a private schema called 'auth'
select auth.user_role(email, pass) into _role;
-- other operations
-- ...
end;
$$ language plpgsql security definer;
Note the ``SECURITY DEFINER`` keywords at the end of the function. See `PostgreSQL documentation <https://www.postgresql.org/docs/current/sql-createfunction.html#SQL-CREATEFUNCTION-SECURITY>`_ for more details.
Views
=====
Views are invoked with the privileges of the view owner, much like functions with the ``SECURITY DEFINER`` option. When created by a SUPERUSER role, all `row-level security <https://www.postgresql.org/docs/current/ddl-rowsecurity.html>`_ policies will be bypassed.
If you're on PostgreSQL >= 15, this behavior can be changed by specifying the ``security_invoker`` option.
.. code-block:: postgres
CREATE VIEW sample_view WITH (security_invoker = true) AS
SELECT * FROM sample_table;
On PostgreSQL < 15, you can create a non-SUPERUSER role and make this role the view's owner.
.. code-block:: postgres
CREATE ROLE api_views_owner NOSUPERUSER NOBYPASSRLS;
ALTER VIEW sample_view OWNER TO api_views_owner;
@@ -1,59 +1,31 @@
.. _install:
Installation
============
############
The release page has `pre-compiled binaries for Mac OS X, Windows, Linux and FreeBSD <https://github.com/PostgREST/postgrest/releases/latest>`_ .
The release page has `pre-compiled binaries for macOS, Windows, Linux and FreeBSD <https://github.com/PostgREST/postgrest/releases/latest>`_.
The Linux binary is a static executable that can be run on any Linux distribution.
You can also use your OS package manager.
.. tabs::
.. include:: ../shared/installation.rst
.. group-tab:: Mac OSX
.. _pg-dependency:
You can install PostgREST from the `Homebrew official repo <https://formulae.brew.sh/formula/postgrest>`_.
Supported PostgreSQL versions
=============================
.. code:: bash
=============== =================================
**Supported** PostgreSQL >= 9.6
=============== =================================
brew install postgrest
PostgREST works with all PostgreSQL versions starting from 9.6.
.. group-tab:: FreeBSD
.. note::
You can install PostgREST from the `official ports <https://www.freshports.org/www/hs-postgrest>`_.
Support for PostgreSQL versions 9.6, 10 and 11 is deprecated. From this on version onwards, PostgREST will only support non-end-of-life PostgreSQL versions.
.. code:: bash
pkg install hs-postgrest
.. group-tab:: Linux
.. tabs::
.. tab:: Arch Linux
You can install PostgREST from the `community repo <https://archlinux.org/packages/community/x86_64/postgrest>`_.
.. code:: bash
pacman -S postgrest
.. tab:: Nix
You can install PostgREST from nixpkgs.
.. code:: bash
nix-env -i haskellPackages.postgrest
.. group-tab:: Windows
You can install PostgREST using `Chocolatey <https://community.chocolatey.org/packages/postgrest>`_ or `Scoop <https://github.com/ScoopInstaller/Scoop>`_.
.. code:: bash
choco install postgrest
scoop install postgrest
See https://www.postgresql.org/support/versioning/.
Running PostgREST
=================
@@ -95,19 +67,12 @@ For a complete reference of the configuration file, see :ref:`configuration`.
If you see a dialog box like this on Windows, it may be that the :code:`pg_config` program is not in your system path.
.. image:: _static/win-err-dialog.png
.. image:: ../_static/win-err-dialog.png
It usually lives in :code:`C:\Program Files\PostgreSQL\<version>\bin`. See this `article <https://www.howtogeek.com/118594/how-to-edit-your-system-path-for-easy-command-line-access/>`_ about how to modify the system path.
To test that the system path is set correctly, run ``pg_config`` from the command line. You should see it output a list of paths.
.. _pg-dependency:
PostgreSQL dependency
---------------------
To use PostgREST you will need an underlying database. We require PostgreSQL 9.6 or greater. You can use something like `Amazon RDS <https://aws.amazon.com/rds/>`_ but installing your own locally is cheaper and more convenient for development. You can also run PostgreSQL in a :ref:`docker container<pg-in-docker>`.
Docker
======
@@ -204,14 +169,15 @@ If you want to have a visual overview of your API in your browser you can add sw
.. code-block:: yaml
swagger:
image: swaggerapi/swagger-ui
ports:
- "8080:8080"
expose:
- "8080"
environment:
API_URL: http://localhost:3000/
# in services:
swagger:
image: swaggerapi/swagger-ui
ports:
- "8080:8080"
expose:
- "8080"
environment:
API_URL: http://localhost:3000/
With this you can see the swagger-ui in your browser on port 8080.
@@ -222,10 +188,6 @@ Building from Source
When a pre-built binary does not exist for your system you can build the project from source.
.. note::
We discourage building and using PostgREST on **Alpine Linux** because of a reported GHC memory leak on that platform.
You can build PostgREST from source with `Stack <https://github.com/commercialhaskell/stack>`_. It will install any necessary Haskell dependencies on your system.
* `Install Stack <https://docs.haskellstack.org/en/stable/#how-to-install-stack>`_ for your platform
@@ -237,7 +199,7 @@ You can build PostgREST from source with `Stack <https://github.com/commercialha
Ubuntu/Debian libpq-dev, libgmp-dev, zlib1g-dev
CentOS/Fedora/Red Hat postgresql-devel, zlib-devel, gmp-devel
BSD postgresql12-client
OS X libpq, gmp
macOS libpq, gmp
===================== =======================================
* Build and install binary
@@ -256,126 +218,3 @@ You can build PostgREST from source with `Stack <https://github.com/commercialha
- `--install-ghc` flag is only needed for the first build and can be omitted in the subsequent builds.
* Check that the server is installed: :code:`postgrest --help`.
.. _deploy_heroku:
Deploying to Heroku
===================
1. Log into Heroku using the `Heroku CLI <https://devcenter.heroku.com/articles/heroku-cli>`_:
.. code-block:: bash
# If you have multiple Heroku accounts, use flag '--interactive' to switch between them
heroku login --interactive
2. Create a new Heroku app using the PostgREST buildpack:
.. code-block:: bash
mkdir ${YOUR_APP_NAME}
cd ${YOUR_APP_NAME}
git init .
heroku apps:create ${YOUR_APP_NAME} --buildpack https://github.com/PostgREST/postgrest-heroku.git
heroku git:remote -a ${YOUR_APP_NAME}
3. Create a new Heroku PostgreSQL add-on attached to the app and keep notes of the assigned add-on name (e.g. :code:`postgresql-curly-58902`) referred later as ${HEROKU_PG_DB_NAME}
.. code-block:: bash
heroku addons:create heroku-postgresql:standard-0 -a ${YOUR_APP_NAME}
# wait until the add-on is available
heroku pg:wait -a ${YOUR_APP_NAME}
4. Create the necessary user roles according to the
`PostgREST documentation <https://postgrest.org/en/stable/auth.html>`_:
.. code-block:: bash
heroku pg:credentials:create --name api_user -a ${YOUR_APP_NAME}
# use the following command to ensure the new credential state is active before attaching it
heroku pg:credentials -a ${YOUR_APP_NAME}
heroku addons:attach ${HEROKU_PG_DB_NAME} --credential api_user -a ${YOUR_APP_NAME}
5. Connect to the PostgreSQL database and create some sample data:
.. code-block:: bash
heroku psql -a ${YOUR_APP_NAME}
.. code-block:: postgres
# from the psql command prompt execute the following commands:
create schema api;
create table api.todos (
id serial primary key,
done boolean not null default false,
task text not null,
due timestamptz
);
insert into api.todos (task) values
('finish tutorial 0'), ('pat self on back');
grant usage on schema api to api_user;
grant select on api.todos to api_user;
6. Create the :code:`Procfile`:
.. code-block:: bash
web: PGRST_SERVER_HOST=0.0.0.0 PGRST_SERVER_PORT=${PORT} PGRST_DB_URI=${PGRST_DB_URI:-${DATABASE_URL}} ./postgrest-${POSTGREST_VER}
..
Set the following environment variables on Heroku:
.. code-block:: bash
heroku config:set POSTGREST_VER=10.0.0
heroku config:set PGRST_DB_SCHEMA=api
heroku config:set PGRST_DB_ANON_ROLE=api_user
..
PGRST_DB_URI can be set if an external database is used or if it's different from the default Heroku DATABASE_URL. This latter is used if nothing is provided.
POSTGREST_VER is mandatory to select and build the required PostgREST release.
See https://postgrest.org/en/stable/configuration.html#environment-variables for the full list of environment variables.
7. Build and deploy your app:
.. code-block:: bash
git add Procfile
git commit -m "PostgREST on Heroku"
git push heroku master
..
Your Heroku app should be live at :code:`${YOUR_APP_NAME}.herokuapp.com`
8. Test your app
From a terminal display the application logs:
.. code-block:: bash
heroku logs -t
..
From a different terminal retrieve with curl the records previously created:
.. code-block:: bash
curl https://${YOUR_APP_NAME}.herokuapp.com/todos
..
and test that any attempt to modify the table via a read-only user is not allowed:
.. code-block:: bash
curl https://${YOUR_APP_NAME}.herokuapp.com/todos -X POST \
-H "Content-Type: application/json" \
-d '{"task": "do bad thing"}'
+102
View File
@@ -0,0 +1,102 @@
.. _nginx:
Nginx
=====
PostgREST is a fast way to construct a RESTful API. Its default behavior is great for scaffolding in development. When it's time to go to production it works great too, as long as you take precautions.
PostgREST is a small sharp tool that focuses on performing the API-to-database mapping. We rely on a reverse proxy like Nginx for additional safeguards.
The first step is to create an Nginx configuration file that proxies requests to an underlying PostgREST server.
.. code-block:: nginx
http {
# ...
# upstream configuration
upstream postgrest {
server localhost:3000;
}
# ...
server {
# ...
# expose to the outside world
location /api/ {
default_type application/json;
proxy_hide_header Content-Location;
add_header Content-Location /api/$upstream_http_content_location;
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_pass http://postgrest/;
}
# ...
}
}
.. note::
For ubuntu, if you already installed nginx through :code:`apt` you can add this to the config file in
:code:`/etc/nginx/sites-enabled/default`.
.. _https:
HTTPS
-----
PostgREST aims to do one thing well: add an HTTP interface to a PostgreSQL database. To keep the code small and focused we do not implement HTTPS. Use a reverse proxy such as NGINX to add this, `here's how <https://nginx.org/en/docs/http/configuring_https_servers.html>`_.
Rate Limiting
-------------
Nginx supports "leaky bucket" rate limiting (see `official docs <https://nginx.org/en/docs/http/ngx_http_limit_req_module.html>`_). Using standard Nginx configuration, routes can be grouped into *request zones* for rate limiting. For instance we can define a zone for login attempts:
.. code-block:: nginx
limit_req_zone $binary_remote_addr zone=login:10m rate=1r/s;
This creates a shared memory zone called "login" to store a log of IP addresses that access the rate limited urls. The space reserved, 10 MB (:code:`10m`) will give us enough space to store a history of 160k requests. We have chosen to allow only allow one request per second (:code:`1r/s`).
Next we apply the zone to certain routes, like a hypothetical function called :code:`login`.
.. code-block:: nginx
location /rpc/login/ {
# apply rate limiting
limit_req zone=login burst=5;
}
The burst argument tells Nginx to start dropping requests if more than five queue up from a specific IP.
Nginx rate limiting is general and indiscriminate. To rate limit each authenticated request individually you will need to add logic in a :ref:`Custom Validation <custom_validation>` function.
Alternate URL Structure
-----------------------
As discussed in :ref:`singular_plural`, there are no special URL forms for singular resources in PostgREST, only operators for filtering. Thus there are no URLs like :code:`/people/1`. It would be specified instead as
.. code-block:: bash
curl "http://localhost:3000/people?id=eq.1" \
-H "Accept: application/vnd.pgrst.object+json"
This allows compound primary keys and makes the intent for singular response independent of a URL convention.
Nginx rewrite rules allow you to simulate the familiar URL convention. The following example adds a rewrite rule for all table endpoints, but you'll want to restrict it to those tables that have a numeric simple primary key named "id."
.. code-block:: nginx
# support /endpoint/:id url style
location ~ ^/([a-z_]+)/([0-9]+) {
# make the response singular
proxy_set_header Accept 'application/vnd.pgrst.object+json';
# assuming an upstream named "postgrest"
proxy_pass http://postgrest/$1?id=eq.$2;
}
.. TODO
.. Administration
.. API Versioning
.. HTTP Caching
.. Upgrading
+25
View File
@@ -0,0 +1,25 @@
.. _schema_isolation:
Schema Isolation
================
A PostgREST instance exposes all the tables, views, and functions of a single `PostgreSQL schema <https://www.postgresql.org/docs/current/ddl-schemas.html>`_ (a namespace of database objects). This means private data or implementation details can go inside different private schemas and be invisible to HTTP clients.
It is recommended that you don't expose tables on your API schema. Instead expose views and functions which insulate the internal details from the outside world.
This allows you to change the internals of your schema and maintain backwards compatibility. It also keeps your code easier to refactor, and provides a natural way to do API versioning.
.. container:: svg-container-md
.. container:: img-dark
.. See https://github.com/sphinx-doc/sphinx/issues/2240#issuecomment-187366626
.. raw:: html
<object width="100%" data="../_static/sch-iso-dark.svg" type="image/svg+xml"></object>
.. container:: img-light
.. raw:: html
<object width="100%" data="../_static/sch-iso.svg" type="image/svg+xml"></object>
-33
View File
@@ -1,33 +0,0 @@
{ lib
, buildPythonPackage
, fetchFromGitHub
, sphinx
}:
buildPythonPackage rec {
pname = "sphinx-copybutton";
version = "0.4.0";
src = fetchFromGitHub {
owner = "executablebooks";
repo = "sphinx-copybutton";
rev = "v${version}";
sha256 = "sha256-vrEIvQeP7AMXSme1PBp0ox5k8Q1rz+1cbHIO+o17Jqc=";
fetchSubmodules = true;
};
propagatedBuildInputs = [
sphinx
];
doCheck = false; # no tests
pythonImportsCheck = [ "sphinx_copybutton" ];
meta = with lib; {
description = "A small sphinx extension to add a \"copy\" button to code blocks";
homepage = "https://github.com/executablebooks/sphinx-copybutton";
license = licenses.mit;
maintainers = with maintainers; [ Luflosi ];
};
}
-29
View File
@@ -1,29 +0,0 @@
{ lib
, buildPythonPackage
, fetchPypi
, sphinx
}:
buildPythonPackage rec {
pname = "sphinx-tabs";
version = "3.2.0";
src = fetchPypi {
inherit pname version;
sha256 = "sha256:1970aahi6sa7c37cpz8nwgdb2xzf21rk6ykdd1m6w9wvxla7j4rk";
};
propagatedBuildInputs = [
sphinx
];
doCheck = false;
pythonImportsCheck = [ "sphinx_tabs" ];
meta = with lib; {
description = "Create tabbed content in Sphinx documentation when building HTML";
homepage = "https://sphinx-tabs.readthedocs.io";
license = licenses.mit;
};
}
+12 -33
View File
@@ -5,22 +5,20 @@ Create a SOAP endpoint
:author: `fjf2002 <https://github.com/fjf2002>`_
PostgREST now has XML support. With a bit of work, SOAP endpoints become possible.
Please note that PostgREST supports just ``text/xml`` MIME type in request/response headers ``Content-Type`` and ``Accept``.
If you have to use other MIME types such as ``application/soap+xml``, you could manipulate the headers in your reverse proxy.
PostgREST supports :ref:`custom_media`. With a bit of work, SOAP endpoints become possible.
Minimal Example
---------------
This example will simply return the request body, inside a tag ``therequestbodywas``.
Add the following function to your PostgreSQL database:
.. code-block:: postgres
CREATE OR REPLACE FUNCTION my_soap_endpoint(xml) RETURNS xml AS $$
create domain "text/xml" as pg_catalog.xml;
CREATE OR REPLACE FUNCTION my_soap_endpoint(xml) RETURNS "text/xml" AS $$
DECLARE
nsarray CONSTANT text[][] := ARRAY[
ARRAY['soapenv', 'http://schemas.xmlsoap.org/soap/envelope/']
@@ -79,25 +77,6 @@ and should roughly look like:
</soapenv:Body>
</soapenv:Envelope>
Unfortunately the ``Accept: text/xml`` header is currently mandatory concerning PostgREST, otherwise it will respond
with a ``Content-Type: application/json`` header and enclose the response with quotes.
(You can check the returned headers by adding ``-v`` to the curl call.)
If your SOAP clients do not send the ``Accept: text/xml`` header, you can fix that in your nginx reverse proxy
by adding something like ...
.. code-block:: nginx
set $accept $http_accept;
if ($contentType ~ "^text/xml($|;)") {
set $accept "text/xml";
}
proxy_set_header Accept $accept;
to your ``location`` nginx configuration.
(The given example sets the ``Accept`` header for each request of Content-Type ``text/xml``.)
A more elaborate example
------------------------
@@ -121,7 +100,7 @@ potentially disclosing internals to the client, but instead handle the errors di
xmlelement(NAME "soapenv:Body", body)
);
$function$;
-- helper function
CREATE OR REPLACE FUNCTION _soap_exception(
faultcode text,
@@ -137,9 +116,9 @@ potentially disclosing internals to the client, but instead handle the errors di
)
);
$function$;
CREATE OR REPLACE FUNCTION fraction_to_decimal(xml)
RETURNS xml
RETURNS "text/xml"
LANGUAGE plpgsql
AS $function$
DECLARE
@@ -207,14 +186,14 @@ The output should roughly look like:
</soapenv:Body>
</soapenv:Envelope>
References
----------
For more information concerning PostgREST, cf.
- :ref:`s_proc_single_unnamed`
- :ref:`scalar_return_formats`
- :ref:`Nginx reverse proxy <admin>`
- :ref:`function_single_unnamed`
- :ref:`custom_media`. See :ref:`any_handler`, if you need to support an ``application/soap+xml`` media type or if you want to respond with XML without sending a media type.
- :ref:`Nginx reverse proxy <nginx>`
For SOAP reference, visit
@@ -0,0 +1,326 @@
.. _providing_html_htmx:
Providing HTML Content Using Htmx
=================================
:author: `Laurence Isla <https://github.com/laurenceisla>`_
This how-to shows a way to return HTML content and use the `htmx library <https://htmx.org/>`_ to handle the AJAX requests.
Htmx expects an HTML response and uses it to replace an element inside the DOM (see the `htmx introduction <https://htmx.org/docs/#introduction>`_ in the docs).
.. image:: ../_static/how-tos/htmx-demo.gif
.. warning::
This is a proof of concept showing what can be achieved using both technologies.
We are working on `plmustache <https://github.com/PostgREST/plmustache>`_ which will further improve the HTML aspect of this how-to.
Preparatory Configuration
-------------------------
We will make a to-do app based on the :ref:`tut0`, so make sure to complete it before continuing.
To simplify things, we won't be using authentication, so grant all permissions on the ``todos`` table to the ``web_anon`` user.
.. code-block:: postgres
grant all on api.todos to web_anon;
grant usage, select on sequence api.todos_id_seq to web_anon;
Next, add the ``text/html`` as a :ref:`custom_media`. With this, PostgREST can identify the request made by your web browser (with the ``Accept: text/html`` header)
and return a raw HTML document file.
.. code-block:: postgres
create domain "text/html" as text;
Creating an HTML Response
-------------------------
Let's create a function that returns a basic HTML file, using `Pico CSS <https://picocss.com>`_ for styling and
`Ionicons <https://ionic.io/ionicons>`_ to show some icons later.
.. code-block:: postgres
create or replace function api.index() returns "text/html" as $$
select $html$
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>PostgREST + HTMX To-Do List</title>
<!-- Pico CSS for CSS styling -->
<link href="https://cdn.jsdelivr.net/npm/@picocss/pico@next/css/pico.min.css" rel="stylesheet" />
</head>
<body>
<main class="container">
<article>
<h5 style="text-align: center;">
PostgREST + HTMX To-Do List
</h5>
</article>
</main>
<!-- Script for Ionicons icons -->
<script type="module" src="https://unpkg.com/ionicons@7.1.0/dist/ionicons/ionicons.esm.js"></script>
<script nomodule src="https://unpkg.com/ionicons@7.1.0/dist/ionicons/ionicons.js"></script>
</body>
</html>
$html$;
$$ language sql;
The web browser will open the web page at ``http://localhost:3000/rpc/index``.
.. image:: ../_static/how-tos/htmx-simple.jpg
.. _html_htmx_list_create:
Listing and Creating To-Dos
---------------------------
Now, let's show a list of the to-dos already inserted in the database.
For that, we'll also need a function to help us sanitize the HTML content that may be present in the task.
.. code-block:: postgres
create or replace function api.sanitize_html(text) returns text as $$
select replace(replace(replace(replace(replace($1, '&', '&amp;'), '"', '&quot;'),'>', '&gt;'),'<', '&lt;'), '''', '&apos;')
$$ language sql;
create or replace function api.html_todo(api.todos) returns text as $$
select format($html$
<div>
<%2$s>
%3$s
</%2$s>
</div>
$html$,
$1.id,
case when $1.done then 's' else 'span' end,
api.sanitize_html($1.task)
);
$$ language sql stable;
create or replace function api.html_all_todos() returns text as $$
select coalesce(
string_agg(api.html_todo(t), '<hr/>' order by t.id),
'<p><em>There is nothing else to do.</em></p>'
)
from api.todos t;
$$ language sql;
These two functions are used to build the to-do list template. We won't use them as PostgREST endpoints.
- The ``api.html_todo`` function uses the table ``api.todos`` as a parameter and formats each item into a list element ``<li>``.
The PostgreSQL `format <https://www.postgresql.org/docs/current/functions-string.html#FUNCTIONS-STRING-FORMAT>`_ is useful to that end.
It replaces the values according to the position in the template, e.g. ``%1$s`` will be replaced with the value of ``$1.id`` (the first parameter).
- The ``api.html_all_todos`` function returns the ``<ul>`` wrapper for all the list elements.
It uses `string_arg <https://www.postgresql.org/docs/current/functions-aggregate.html>`_ to concatenate all the to-dos in a single text value.
It also returns an alternative message, instead of a list, when the ``api.todos`` table is empty.
Next, let's add an endpoint to register a to-do in the database and modify the ``/rpc/index`` page accordingly.
.. code-block:: postgres
create or replace function api.add_todo(_task text) returns "text/html" as $$
insert into api.todos(task) values (_task);
select api.html_all_todos();
$$ language sql;
create or replace function api.index() returns "text/html" as $$
select $html$
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>PostgREST + HTMX To-Do List</title>
<!-- Pico CSS for CSS styling -->
<link href="https://cdn.jsdelivr.net/npm/@picocss/pico@next/css/pico.min.css" rel="stylesheet"/>
<!-- htmx for AJAX requests -->
<script src="https://unpkg.com/htmx.org"></script>
</head>
<body>
<main class="container"
style="max-width: 600px"
hx-headers='{"Accept": "text/html"}'>
<article>
<h5 style="text-align: center;">
PostgREST + HTMX To-Do List
</h5>
<form hx-post="/rpc/add_todo"
hx-target="#todo-list-area"
hx-trigger="submit"
hx-on="htmx:afterRequest: this.reset()">
<input type="text" name="_task" placeholder="Add a todo...">
</form>
<div id="todo-list-area">
$html$
|| api.html_all_todos() ||
$html$
<div>
</article>
</main>
<!-- Script for Ionicons icons -->
<script type="module" src="https://unpkg.com/ionicons@7.1.0/dist/ionicons/ionicons.esm.js"></script>
<script nomodule src="https://unpkg.com/ionicons@7.1.0/dist/ionicons/ionicons.js"></script>
</body>
</html>
$html$;
$$ language sql;
- The ``/rpc/add_todo`` endpoint allows us to add a new to-do using the ``_task`` parameter and returns an ``html`` with all the to-dos in the database.
- The ``/rpc/index`` now adds the ``hx-headers='{"Accept": "text/html"}'`` tag to the ``<body>``.
This will make sure that all htmx elements inside the body send this header, otherwise PostgREST won't recognize it as HTML.
There is also a ``<form>`` element that uses the htmx library. Let's break it down:
+ ``hx-post="/rpc/add_todo"``: sends an AJAX POST request to the ``/rpc/add_todo`` endpoint, with the value of the ``_task`` from the ``<input>`` element.
+ ``hx-target="#todo-list-area"``: the HTML content returned from the request will go inside ``<div id="todo-list-area"></div>`` (which is the list of to-dos).
+ ``hx-trigger="submit"``: htmx will do this request when submitting the form (by pressing enter while inside the ``<input>``).
+ ``hx-on="htmx:afterRequest: this.reset()">``: this is a Javascript command that clears the form `after the request is done <https://htmx.org/events/#htmx:afterRequest>`_.
With this, the ``http://localhost:3000/rpc/index`` page lists all the todos and adds new ones by submitting tasks in the input element.
Don't forget to refresh the :ref:`schema cache <schema_reloading>`.
.. image:: ../_static/how-tos/htmx-insert.gif
Editing and Deleting To-Dos
---------------------------
Now, let's modify ``api.html_todo`` and make it more functional.
.. code-block:: postgres
create or replace function api.html_todo(api.todos) returns text as $$
select format($html$
<div class="grid">
<div id="todo-edit-area-%1$s">
<form id="edit-task-state-%1$s"
hx-post="/rpc/change_todo_state"
hx-vals='{"_id": %1$s, "_done": %4$s}'
hx-target="#todo-list-area"
hx-trigger="click">
<%2$s style="cursor: pointer">
%3$s
</%2$s>
</form>
</div>
<div style="text-align: right">
<button class="outline"
hx-get="/rpc/html_editable_task"
hx-vals='{"_id": "%1$s"}'
hx-target="#todo-edit-area-%1$s"
hx-trigger="click">
<span>
<ion-icon name="create"></ion-icon>
</span>
</button>
<button class="outline contrast"
hx-post="/rpc/delete_todo"
hx-vals='{"_id": %1$s}'
hx-target="#todo-list-area"
hx-trigger="click">
<span>
<ion-icon name="trash" style="color: #f87171"></ion-icon>
</span>
</button>
</div>
</div>
$html$,
$1.id,
case when $1.done then 's' else 'span' end,
api.sanitize_html($1.task),
(not $1.done)::text
);
$$ language sql stable;
Let's deconstruct the new htmx features added:
- The ``<form>`` element is configured as follows:
+ ``hx-post="/rpc/change_todo_state"``: does an AJAX POST request to that endpoint. It will toggle the ``done`` state of the to-do.
+ ``hx-vals='{"_id": %1$s, "_done": %4$s}'``: adds the parameters to the request.
This is an alternative to using hidden inputs inside the ``<form>``.
+ ``hx-trigger="click"``: htmx does the request after clicking on the element.
- For the first ``<button>``:
+ ``hx-get="/rpc/html_editable_task"``: it does an AJAX GET request to that endpoint.
It returns an HTML with an input that will allow us to edit the task.
+ ``hx-target="#todo-edit-area"``: the returned HTML will replace the element with this id.
In this case, this replaces an individual task, not the whole list.
+ ``hx-vals='{"id": "eq.%1$s"}'``: adds the query parameters to the GET request.
Note that this needs the ``eq.`` operator because it represents a table column not a function parameter.
- For the second ``<button>``:
+ ``hx-post="/rpc/delete_todo"``: this post request will delete the corresponding to-do.
Clicking on the first button will enable the task editing.
That's why we create the ``api.html_editable_task`` function as an endpoint:
.. code-block:: postgres
create or replace function api.html_editable_task(_id int) returns "text/html" as $$
select format ($html$
<form id="edit-task-%1$s"
hx-post="/rpc/change_todo_task"
hx-headers='{"Accept": "text/html"}'
hx-vals='{"_id": %1$s}'
hx-target="#todo-list-area"
hx-trigger="submit,focusout">
<input id="task-%1$s" type="text" name="_task" value="%2$s" autofocus>
</form>
$html$,
id,
api.sanitize_html(task)
)
from api.todos
where id = _id;
$$ language sql;
In this example, this will return an input field that allows us to edit the corresponding to-do task.
Finally, let's add the endpoints that will modify and delete the to-dos in the database.
.. code-block:: postgres
create or replace function api.change_todo_state(_id int, _done boolean) returns "text/html" as $$
update api.todos set done = _done where id = _id;
select api.html_all_todos();
$$ language sql;
create or replace function api.change_todo_task(_id int, _task text) returns "text/html" as $$
update api.todos set task = _task where id = _id;
select api.html_all_todos();
$$ language sql;
create or replace function api.delete_todo(_id int) returns "text/html" as $$
delete from api.todos where id = _id;
select api.html_all_todos();
$$ language sql;
All of those functions return an HTML list of to-dos that will replace the outdated one:
- The ``api.change_todo_state`` function updates the ``done`` column using the ``_id`` and the ``_done`` values from the request.
- The ``api.delete_todo`` function deletes a to-do using the ``_id`` value from the request.
- The ``api.change_todo_task`` function modifies the ``task`` column using the ``_id`` and the ``_task`` value from the request.
After refreshing the :ref:`schema cache <schema_reloading>`, the page at ``http://localhost:3000/rpc/index`` will allow us to edit, delete and complete any to-do.
.. image:: ../_static/how-tos/htmx-edit-delete.gif
With that, we completed the to-do list functionality.
+40 -12
View File
@@ -26,18 +26,42 @@ First, we need a public table for storing the files.
, blob bytea
);
Let's assume this table contains an image of two cute kittens with id 42.
We can retrieve this image in binary format from our PostgREST API by requesting :code:`/files?select=blob&id=eq.42` with the :code:`Accept: application/octet-stream` header.
Unfortunately, putting the URL into the :code:`src` of an :code:`<img>` tag will not work.
That's because browsers do not send the required :code:`Accept: application/octet-stream` header.
Let's assume this table contains an image of two cute kittens with id 42. We can retrieve this image in binary format from our PostgREST API by using :ref:`custom_media`:
.. code-block:: postgres
create domain "application/octet-stream" as bytea;
create or replace function file(id int) returns "application/octet-stream" as $$
select blob from files where id = file.id;
$$ language sql;
Now we can request the RPC endpoint :code:`/rpc/file?id=42` with the :code:`Accept: application/octet-stream` header.
.. code-block:: bash
curl "localhost:3000/rpc/file?id=42" -H "Accept: application/octet-stream"
Unfortunately, putting the URL into the :code:`src` of an :code:`<img>` tag will not work. That's because browsers do not send the required :code:`Accept: application/octet-stream` header.
Instead, the :code:`Accept: image/webp` header is sent by many web browsers by default.
Luckily we can change the accepted media type in the function like so:
.. code-block:: postgres
create domain "image/webp" as bytea;
create or replace function file(id int) returns "image/webp" as $$
select blob from files where id = file.id;
$$ language sql;
Luckily we can specify the accepted media types in the :ref:`raw-media-types` configuration variable.
In this case, the :code:`Accept: image/webp` header is sent by many web browsers by default, so let's add it to the configuration variable, like this: :code:`raw-media-types="image/webp"`.
Now, the image will be displayed in the HTML page:
.. code-block:: html
<img src="http://localhost:3000/files?select=blob&id=eq.42" alt="Cute Kittens"/>
<img src="http://localhost:3000/file?id=42" alt="Cute Kittens"/>
Improved Version
----------------
@@ -57,16 +81,20 @@ First, in addition to the minimal example, we need to store the media types and
.. code-block:: postgres
alter table files
add column type text,
add column type text generated always as (byteamagic_mime(substr(blob, 0, 4100))) stored,
add column name text;
Next, we set up an RPC endpoint that sets the content type and filename.
This uses the :code:`byteamagic_mime()` function from the `pg_byteamagic extension <https://github.com/nmandery/pg_byteamagic>`_ to automatically generate the type in the :code:`files` table. To guess the type of a file, it's generally enough to look at the beginning of the file, which is more efficient.
Next, we set modify the function to set the content type and filename.
We use this opportunity to configure some basic, client-side caching.
For production, you probably want to configure additional caches, e.g. on the :ref:`reverse proxy <admin>`.
For production, you probably want to configure additional caches, e.g. on the :ref:`reverse proxy <nginx>`.
.. code-block:: postgres
create function file(id int) returns bytea as
create domain "*/*" as bytea;
create function file(id int) returns "*/*" as
$$
declare headers text;
declare blob bytea;
@@ -79,7 +107,7 @@ For production, you probably want to configure additional caches, e.g. on the :r
from files where files.id = file.id into headers;
perform set_config('response.headers', headers, true);
select files.blob from files where files.id = file.id into blob;
if found
if FOUND -- special var, see https://www.postgresql.org/docs/current/plpgsql-statements.html#PLPGSQL-STATEMENTS-DIAGNOSTICS
then return(blob);
else raise sqlstate 'PT404' using
message = 'NOT FOUND',
@@ -31,7 +31,7 @@ As in :ref:`sql_user_management`, we create a :code:`basic_auth` schema:
-- We put things inside the basic_auth schema to hide
-- them from public view. Certain public procs/views will
-- refer to helpers and tables inside.
CREATE SCHEMA IF NOT EXISTS basic_auth;
CREATE SCHEMA basic_auth;
As in :ref:`sql_user_management`, we create the :code:`pgcrypto` and :code:`pgjwt` extensions. Here we prefer to put the extensions in its own schemas:
@@ -40,7 +40,7 @@ As in :ref:`sql_user_management`, we create the :code:`pgcrypto` and :code:`pgjw
CREATE SCHEMA ext_pgcrypto;
ALTER SCHEMA ext_pgcrypto OWNER TO postgres;
CREATE EXTENSION IF NOT EXISTS pgcrypto WITH SCHEMA ext_pgcrypto;
CREATE EXTENSION pgcrypto WITH SCHEMA ext_pgcrypto;
Concerning the `pgjwt extension <https://github.com/michelp/pgjwt>`_, please cf. to :ref:`client_auth`.
@@ -49,17 +49,17 @@ Concerning the `pgjwt extension <https://github.com/michelp/pgjwt>`_, please cf.
CREATE SCHEMA ext_pgjwt;
ALTER SCHEMA ext_pgjwt OWNER TO postgres;
CREATE EXTENSION IF NOT EXISTS pgjwt WITH SCHEMA ext_pgjwt;
CREATE EXTENSION pgjwt WITH SCHEMA ext_pgjwt;
In order to be able to work with postgres' SCRAM-SHA-256 password hashes, we also need the PBKDF2 key derivation function. Luckily there is `a PL/pgSQL implementation on stackoverflow <https://stackoverflow.com/q/47162200/2337147>`_:
In order to be able to work with postgres' SCRAM-SHA-256 password hashes, we also need the PBKDF2 key derivation function. Luckily there is `a PL/pgSQL implementation on stackoverflow <https://stackoverflow.com/a/72805848>`_:
.. code-block:: plpgsql
.. code-block:: postgres
CREATE FUNCTION basic_auth.pbkdf2(salt bytea, pw text, count integer, desired_length integer, algorithm text) RETURNS bytea
LANGUAGE plpgsql IMMUTABLE
AS $$
DECLARE
DECLARE
hash_length integer;
block_count integer;
output bytea;
@@ -97,7 +97,7 @@ In order to be able to work with postgres' SCRAM-SHA-256 password hashes, we als
--
FOR j IN 2 .. count LOOP
the_last := ext_pgcrypto.HMAC(the_last, pw::bytea, algorithm);
-- xor the two
FOR k IN 1 .. length(xorsum) LOOP
xorsum := set_byte(xorsum, k - 1, get_byte(xorsum, k - 1) # get_byte(the_last, k - 1));
@@ -117,10 +117,10 @@ In order to be able to work with postgres' SCRAM-SHA-256 password hashes, we als
ALTER FUNCTION basic_auth.pbkdf2(salt bytea, pw text, count integer, desired_length integer, algorithm text) OWNER TO postgres;
Analogous to :ref:`sql_user_management` creates the function :code:`basic_auth.user_role`, we create a helper function to check the user's password, here with another name and signature (since we want the username, not an email address).
Analogous to how :ref:`sql_user_management` creates the function :code:`basic_auth.user_role`, we create a helper function to check the user's password, here with another name and signature (since we want the username, not an email address).
But contrary to :ref:`sql_user_management`, this function does not use a dedicated :code:`users` table with passwords, but instead utilizes the built-in table `pg_catalog.pg_authid <https://www.postgresql.org/docs/current/catalog-pg-authid.html>`_:
.. code-block:: plpgsql
.. code-block:: postgres
CREATE FUNCTION basic_auth.check_user_pass(username text, password text) RETURNS name
LANGUAGE sql
@@ -160,22 +160,17 @@ Logins
As described in :ref:`client_auth`, we'll create a JWT token inside our login function. Note that you'll need to adjust the secret key which is hard-coded in this example to a secure (at least thirty-two character) secret of your choosing.
.. code-block:: plpgsql
CREATE TYPE basic_auth.jwt_token AS (
token text
);
.. code-block:: postgres
-- if you are not using psql, you need to replace :DBNAME with the current database's name.
ALTER DATABASE :DBNAME SET "app.jwt_secret" to 'reallyreallyreallyreallyverysafe';
CREATE FUNCTION public.login(username text, password text) RETURNS basic_auth.jwt_token
CREATE FUNCTION public.login(username text, password text, OUT token text)
LANGUAGE plpgsql security definer
AS $$
DECLARE
_role name;
result basic_auth.jwt_token;
BEGIN
-- check email and password
SELECT basic_auth.check_user_pass(username, password) INTO _role;
@@ -190,8 +185,7 @@ As described in :ref:`client_auth`, we'll create a JWT token inside our login fu
SELECT login.username as role,
extract(epoch FROM now())::integer + 60*60 AS exp
) r
INTO result;
RETURN result;
INTO token;
END;
$$;
@@ -210,8 +204,6 @@ anonymous roles. Below is an example of permissions that allow anonymous users t
.. code-block:: postgres
-- the names "anon" and "authenticator" are configurable and not
-- sacred, we simply choose them for clarity
CREATE ROLE anon NOINHERIT;
CREATE role authenticator NOINHERIT LOGIN PASSWORD 'secret';
GRANT anon TO authenticator;
@@ -261,19 +253,11 @@ Test at the REST level
~~~~~~~~~~~~~~~~~~~~~~
An API request to call this function would look like:
.. tabs::
.. code-block:: bash
.. code-tab:: http
POST /rpc/login HTTP/1.1
{ "username": "foo", "password": "bar" }
.. code-tab:: bash Curl
curl "http://localhost:3000/rpc/login" \
-X POST -H "Content-Type: application/json" \
-d '{ "username": "foo", "password": "bar" }'
curl "http://localhost:3000/rpc/login" \
-X POST -H "Content-Type: application/json" \
-d '{ "username": "foo", "password": "bar" }'
The response would look like the snippet below. Try decoding the token at `jwt.io <https://jwt.io/>`_. (It was encoded with a secret of :code:`reallyreallyreallyreallyverysafe` as specified in the SQL code above. You'll want to change this secret in your app!)
@@ -298,31 +282,18 @@ Let's add a table, intended for the :code:`foo` user:
Now try to get the table's contents with:
.. tabs::
.. code-tab:: http
.. code-block:: bash
GET /foobar HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/foobar"
curl "http://localhost:3000/foobar"
This should fail --- of course, we haven't specified the user, thus PostgREST falls back to the :code:`anon` user and denies access.
Add an :code:`Authorization` header. Please use the token value from the login function call above instead of the one provided below.
.. tabs::
.. code-block:: bash
.. code-tab:: http
GET /foobar HTTP/1.1
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiZm9vIiwiZXhwIjoxNjY4MTkyMjAyfQ.zzdHCBjfkqDQLQ8D7CHO3cIALF6KBCsfPTWgwhCiHCY
.. code-tab:: bash Curl
curl "http://localhost:3000/foobar" \
-H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiZm9vIiwiZXhwIjoxNjY4MTkyMjAyfQ.zzdHCBjfkqDQLQ8D7CHO3cIALF6KBCsfPTWgwhCiHCY"
curl "http://localhost:3000/foobar" \
-H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiZm9vIiwiZXhwIjoxNjY4MTkyMjAyfQ.zzdHCBjfkqDQLQ8D7CHO3cIALF6KBCsfPTWgwhCiHCY"
This will fail again --- we get :code:`Permission denied to set role`. We forgot to allow the authenticator role to switch into this user by executing:
+206
View File
@@ -0,0 +1,206 @@
.. _sql_user_management:
SQL User Management
===================
As mentioned on :ref:`jwt_generation`, an external service can provide user management and coordinate with the PostgREST server using JWT. Its also possible to support logins entirely through SQL. Its a fair bit of work, so get ready.
Storing Users and Passwords
---------------------------
The following table, functions, and triggers will live in a :code:`basic_auth` schema that you shouldn't expose publicly in the API. The public views and functions will live in a different schema which internally references this internal information.
First we'll need a table to keep track of our users:
.. code:: sql
-- We put things inside the basic_auth schema to hide
-- them from public view. Certain public procs/views will
-- refer to helpers and tables inside.
create table
basic_auth.users (
email text primary key check ( email ~* '^.+@.+\..+$' ),
pass text not null check (length(pass) < 512),
role name not null check (length(role) < 512)
);
We would like the role to be a foreign key to actual database roles, however PostgreSQL does not support these constraints against the :code:`pg_roles` table. We'll use a trigger to manually enforce it.
.. code-block:: postgres
create function
basic_auth.check_role_exists() returns trigger as $$
begin
if not exists (select 1 from pg_roles as r where r.rolname = new.role) then
raise foreign_key_violation using message =
'unknown database role: ' || new.role;
return null;
end if;
return new;
end
$$ language plpgsql;
create constraint trigger ensure_user_role_exists
after insert or update on basic_auth.users
for each row
execute procedure basic_auth.check_role_exists();
Next we'll use the pgcrypto extension and a trigger to keep passwords safe in the :code:`users` table.
.. code-block:: postgres
create extension pgcrypto;
create function
basic_auth.encrypt_pass() returns trigger as $$
begin
if tg_op = 'INSERT' or new.pass <> old.pass then
new.pass = crypt(new.pass, gen_salt('bf'));
end if;
return new;
end
$$ language plpgsql;
create trigger encrypt_pass
before insert or update on basic_auth.users
for each row
execute procedure basic_auth.encrypt_pass();
With the table in place we can make a helper to check a password against the encrypted column. It returns the database role for a user if the email and password are correct.
.. code-block:: postgres
create function
basic_auth.user_role(email text, pass text) returns name
language plpgsql
as $$
begin
return (
select role from basic_auth.users
where users.email = user_role.email
and users.pass = crypt(user_role.pass, users.pass)
);
end;
$$;
.. _public_ui:
Public User Interface
---------------------
In the previous section we created an internal table to store user information. Here we create a login function which takes an email address and password and returns JWT if the credentials match a user in the internal table.
Permissions
~~~~~~~~~~~
Your database roles need access to the schema, tables, views and functions in order to service HTTP requests.
Recall from the :ref:`roles` that PostgREST uses special roles to process requests, namely the authenticator and
anonymous roles. Below is an example of permissions that allow anonymous users to create accounts and attempt to log in.
.. code-block:: postgres
create role anon noinherit;
create role authenticator noinherit;
grant anon to authenticator;
Then, add ``db-anon-role`` to the configuration file to allow anonymous requests.
.. code:: ini
db-anon-role = "anon"
JWT from SQL
~~~~~~~~~~~~
You can create JWT tokens in SQL using the `pgjwt extension <https://github.com/michelp/pgjwt>`_. It's simple and requires only pgcrypto. If you're on an environment like Amazon RDS which doesn't support installing new extensions, you can still manually run the `SQL inside pgjwt <https://github.com/michelp/pgjwt/blob/master/pgjwt--0.1.1.sql>`_ (you'll need to replace ``@extschema@`` with another schema or just delete it) which creates the functions you will need.
Next write a function that returns the token. The one below returns a token with a hard-coded role, which expires five minutes after it was issued. Note this function has a hard-coded secret as well.
.. code-block:: postgres
CREATE FUNCTION jwt_test(OUT token text) AS $$
SELECT public.sign(
row_to_json(r), 'reallyreallyreallyreallyverysafe'
) AS token
FROM (
SELECT
'my_role'::text as role,
extract(epoch from now())::integer + 300 AS exp
) r;
$$ LANGUAGE sql;
PostgREST exposes this function to clients via a POST request to ``/rpc/jwt_test``.
.. note::
To avoid hard-coding the secret in functions, save it as a property of the database.
.. code-block:: postgres
-- run this once
ALTER DATABASE mydb SET "app.jwt_secret" TO 'reallyreallyreallyreallyverysafe';
-- then all functions can refer to app.jwt_secret
SELECT sign(
row_to_json(r), current_setting('app.jwt_secret')
) AS token
FROM ...
Logins
~~~~~~
As described in `JWT from SQL`_, we'll create a JWT inside our login function. Note that you'll need to adjust the secret key which is hard-coded in this example to a secure (at least thirty-two character) secret of your choosing.
.. code-block:: postgres
-- login should be on your exposed schema
create function
login(email text, pass text, out token text) as $$
declare
_role name;
begin
-- check email and password
select basic_auth.user_role(email, pass) into _role;
if _role is null then
raise invalid_password using message = 'invalid user or password';
end if;
select sign(
row_to_json(r), 'reallyreallyreallyreallyverysafe'
) as token
from (
select _role as role, login.email as email,
extract(epoch from now())::integer + 60*60 as exp
) r
into token;
end;
$$ language plpgsql security definer;
grant execute on function login(text,text) to anon;
Since the above :code:`login` function is defined as `security definer <https://www.postgresql.org/docs/current/sql-createfunction.html#id-1.9.3.67.10.2>`_,
the anonymous user :code:`anon` doesn't need permission to read the :code:`basic_auth.users` table. It doesn't even need permission to access the :code:`basic_auth` schema.
:code:`grant execute on function` is included for clarity but it might not be needed, see :ref:`func_privs` for more details.
An API request to call this function would look like:
.. code-block:: bash
curl "http://localhost:3000/rpc/login" \
-X POST -H "Content-Type: application/json" \
-d '{ "email": "foo@bar.com", "pass": "foobar" }'
The response would look like the snippet below. Try decoding the token at `jwt.io <https://jwt.io/>`_. (It was encoded with a secret of :code:`reallyreallyreallyreallyverysafe` as specified in the SQL code above. You'll want to change this secret in your app!)
.. code:: json
{
"token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImZvb0BiYXIuY29tIiwicGFzcyI6ImZvb2JhciJ9.37066TTRlh-1hXhnA9oO9Pj6lgL6zFuJU0iCHhuCFno"
}
Alternatives
~~~~~~~~~~~~
See the how-to :ref:`sql-user-management-using-postgres-users-and-passwords` for a similar way that completely avoids the table :code:`basic_auth.users`.
@@ -5,133 +5,13 @@ Working with PostgreSQL data types
:author: `Laurence Isla <https://github.com/laurenceisla>`_
PostgREST makes use of PostgreSQL string representations to work with data types. Thanks to this, you can use special values, such as ``now`` for timestamps, ``yes`` for booleans or time values including the time zones. This page describes how you can take advantage of these string representations to perform operations on different PostgreSQL data types.
PostgREST makes use of PostgreSQL string representations to work with data types. Thanks to this, you can use special values, such as ``now`` for timestamps, ``yes`` for booleans or time values including the time zones. This page describes how you can take advantage of these string representations and some alternatives to perform operations on different PostgreSQL data types.
.. contents::
:local:
:depth: 1
Timestamps
----------
You can use the **time zone** to filter or send data if needed.
.. code-block:: postgres
create table reports (
id int primary key
, due_date timestamptz
);
Suppose you are located in Sydney and want create a report with the date in the local time zone. Your request should look like this:
.. tabs::
.. code-tab:: http
POST /reports HTTP/1.1
Content-Type: application/json
[{ "id": 1, "due_date": "2022-02-24 11:10:15 Australia/Sydney" },
{ "id": 2, "due_date": "2022-02-27 22:00:00 Australia/Sydney" }]
.. code-tab:: bash Curl
curl "http://localhost:3000/reports" \
-X POST -H "Content-Type: application/json" \
-d '[{ "id": 1, "due_date": "2022-02-24 11:10:15 Australia/Sydney" },{ "id": 2, "due_date": "2022-02-27 22:00:00 Australia/Sydney" }]'
Someone located in Cairo can retrieve the data using their local time, too:
.. tabs::
.. code-tab:: http
GET /reports?due_date=eq.2022-02-24+02:10:15+Africa/Cairo HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/reports?due_date=eq.2022-02-24+02:10:15+Africa/Cairo"
.. code-block:: json
[
{
"id": 1,
"due_date": "2022-02-23T19:10:15-05:00"
}
]
The response has the date in the time zone configured by the server: ``UTC -05:00``.
You can use other comparative filters and also all the `PostgreSQL special date/time input values <https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-DATETIME-SPECIAL-TABLE>`_ as illustrated in this example:
.. tabs::
.. code-tab:: http
GET /reports?or=(and(due_date.gte.today,due_date.lte.tomorrow),and(due_date.gt.-infinity,due_date.lte.epoch)) HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/reports?or=(and(due_date.gte.today,due_date.lte.tomorrow),and(due_date.gt.-infinity,due_date.lte.epoch))"
.. code-block:: json
[
{
"id": 2,
"due_date": "2022-02-27T06:00:00-05:00"
}
]
JSON
----
To work with a ``json`` type column, you can handle the value as a JSON object.
.. code-block:: postgres
create table products (
id int primary key,
name text unique,
extra_info json
);
You can insert a new product using a JSON object for the ``extra_info`` column:
.. tabs::
.. code-tab:: http
POST /products HTTP/1.1
Content-Type: application/json
{
"id": 1,
"name": "Canned fish",
"extra_info": {
"expiry_date": "2025-12-31",
"exportable": true
}
}
.. code-tab:: bash Curl
curl "http://localhost:3000/products" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 1,
"name": "Canned fish",
"extra_info": {
"expiry_date": "2025-12-31",
"exportable": true
}
}
EOF
To query and filter the data see :ref:`json_columns` for a complete reference.
.. NOTE: Titles are ordered alphabetically. New entries should respect this order.
Arrays
------
@@ -149,61 +29,33 @@ To handle `array types <https://www.postgresql.org/docs/current/arrays.html>`_ y
You can insert a new value using string representation.
.. tabs::
.. code-tab:: http
POST /movies HTTP/1.1
Content-Type: application/json
.. code-block:: bash
curl "http://localhost:3000/movies" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 1,
"title": "Paddington",
"tags": "{family,comedy,not streamable}",
"performance_times": "{12:40,15:00,20:00}"
}
.. code-tab:: bash Curl
curl "http://localhost:3000/movies" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 1,
"title": "Paddington",
"tags": "{family,comedy,not streamable}",
"performance_times": "{12:40,15:00,20:00}"
}
EOF
EOF
Or you could send the same data using JSON array format:
.. tabs::
.. code-tab:: http
POST /movies HTTP/1.1
Content-Type: application/json
.. code-block:: bash
curl "http://localhost:3000/movies" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 1,
"title": "Paddington",
"tags": ["family", "comedy", "not streamable"],
"performance_times": ["12:40", "15:00", "20:00"]
}
.. code-tab:: bash Curl
curl "http://localhost:3000/movies" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 1,
"title": "Paddington",
"tags": ["family", "comedy", "not streamable"],
"performance_times": ["12:40", "15:00", "20:00"]
}
EOF
EOF
To query the data you can use arrow operators. See :ref:`composite_array_columns`.
@@ -220,38 +72,21 @@ Similarly to one-dimensional arrays, both the string representation and JSON arr
You can now update the item using JSON array format:
.. tabs::
.. code-tab:: http
PATCH /movies?id=eq.1 HTTP/1.1
Content-Type: application/json
.. code-block:: bash
curl "http://localhost:3000/movies?id=eq.1" \
-X PATCH -H "Content-Type: application/json" \
-d @- << EOF
{
"cinema_floor_auditorium": [ [ [1,2], [6,7] ], [ [3,5], [8,9] ] ]
}
.. code-tab:: bash Curl
curl "http://localhost:3000/movies?id=eq.1" \
-X PATCH -H "Content-Type: application/json" \
-d @- << EOF
{
"cinema_floor_auditorium": [ [ [1,2], [6,7] ], [ [3,5], [8,9] ] ]
}
EOF
EOF
Then, for example, to query the auditoriums that are located in the first cinema (position 0 in the array) and on the second floor (position 1 in the next inner array), we can use the arrow operators this way:
.. tabs::
.. code-block:: bash
.. code-tab:: http
GET /movies?select=title,auditorium:cinema_floor_auditorium->0->1&id=eq.1 HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/movies?select=title,auditorium:cinema_floor_auditorium->0->1&id=eq.1"
curl "http://localhost:3000/movies?select=title,auditorium:cinema_floor_auditorium->0->1&id=eq.1"
.. code-block:: json
@@ -262,211 +97,6 @@ Then, for example, to query the auditoriums that are located in the first cinema
}
]
Composite Types
---------------
With PostgREST, you have two options to handle `composite type columns <https://www.postgresql.org/docs/current/rowtypes.html>`_.
.. code-block:: postgres
create type dimension as (
length decimal(6,2),
width decimal (6,2),
height decimal (6,2),
unit text
);
create table products (
id int primary key,
size dimension
);
insert into products (id, size)
values (1, '(5.0,5.0,10.0,"cm")');
On one hand you can insert values using string representation.
.. tabs::
.. code-tab:: http
POST /products HTTP/1.1
Content-Type: application/json
{ "id": 2, "size": "(0.7,0.5,1.8,\"m\")" }
.. code-tab:: bash Curl
curl "http://localhost:3000/products" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{ "id": 2, "size": "(0.7,0.5,1.8,\"m\")" }
EOF
Or you could insert the same data in JSON format.
.. tabs::
.. code-tab:: http
POST /products HTTP/1.1
Content-Type: application/json
{
"id": 2,
"size": {
"length": 0.7,
"width": 0.5,
"height": 1.8,
"unit": "m"
}
}
.. code-tab:: bash Curl
curl "http://localhost:3000/products" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 2,
"size": {
"length": 0.7,
"width": 0.5,
"height": 1.8,
"unit": "m"
}
}
EOF
You can also query the data using arrow operators. See :ref:`composite_array_columns`.
Ranges
------
PostgREST allows you to handle `ranges <https://www.postgresql.org/docs/current/rangetypes.html>`_.
.. code-block:: postgres
create table events (
id int primary key,
name text unique,
duration tsrange
);
To insert a new event, specify the ``duration`` value as a string representation of the ``tsrange`` type:
.. tabs::
.. code-tab:: http
POST /events HTTP/1.1
Content-Type: application/json
{
"id": 1,
"name": "New Year's Party",
"duration": "['2022-12-31 11:00','2023-01-01 06:00']"
}
.. code-tab:: bash Curl
curl "http://localhost:3000/events" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 1,
"name": "New Year's Party",
"duration": "['2022-12-31 11:00','2023-01-01 06:00']"
}
EOF
You can use range :ref:`operators <operators>` to filter the data. But, in this case, requesting a filter like ``events?duration=cs.2023-01-01`` will return an error, because PostgreSQL needs an explicit cast from string to timestamp. A workaround is to use a range starting and ending in the same date:
.. tabs::
.. code-tab:: http
GET /events?duration=cs.[2023-01-01,2023-01-01] HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/events?duration=cs.\[2023-01-01,2023-01-01\]"
.. code-block:: json
[
{
"id": 1,
"name": "New Year's Party",
"duration": "[\"2022-12-31 11:00:00\",\"2023-01-01 06:00:00\"]"
}
]
.. _casting_range_to_json:
Casting a Range to a JSON Object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As you may have noticed, the ``tsrange`` value is returned as a string literal. To return it as a JSON value, first you need to create a function that will do the conversion from a ``tsrange`` type:
.. code-block:: postgres
create or replace function tsrange_to_json(tsrange) returns json as $$
select json_build_object(
'lower', lower($1)
, 'upper', upper($1)
, 'lower_inc', lower_inc($1)
, 'upper_inc', upper_inc($1)
);
$$ language sql;
Then, create the cast using this function:
.. code-block:: postgres
create cast (tsrange as json) with function tsrange_to_json(tsrange) as assignment;
Finally, do the request :ref:`casting the range column <casting_columns>`:
.. tabs::
.. code-tab:: http
GET /events?select=id,name,duration::json HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/events?select=id,name,duration::json"
.. code-block:: json
[
{
"id": 1,
"name": "New Year's Party",
"duration": {
"lower": "2022-12-31T11:00:00",
"upper": "2023-01-01T06:00:00",
"lower_inc": true,
"upper_inc": true
}
}
]
.. note::
If you don't want to modify casts for built-in types, an option would be to `create a custom type <https://www.postgresql.org/docs/current/sql-createtype.html>`_
for your own ``tsrange`` and add its own cast.
.. code-block:: postgres
create type mytsrange as range (subtype = timestamp, subtype_diff = tsrange_subdiff);
-- define column types and casting function analogously to the above example
-- ...
create cast (mytsrange as json) with function mytsrange_to_json(mytsrange) as assignment;
Bytea
-----
@@ -491,49 +121,26 @@ Let's download the PostgREST logo for our test.
Now, to send the file ``postgrest-logo.png`` we need to set the ``Content-Type: application/octet-stream`` header in the request:
.. tabs::
.. code-block:: bash
.. code-tab:: http
curl "http://localhost:3000/rpc/upload_binary" \
-X POST -H "Content-Type: application/octet-stream" \
--data-binary "@postgrest-logo.png"
POST /rpc/upload_binary HTTP/1.1
Content-Type: application/octet-stream
To get the image from the database, use :ref:`custom_media` like so:
postgrest-logo.png
.. code-block:: postgres
.. code-tab:: bash Curl
create domain "image/png" as bytea;
curl "http://localhost:3000/rpc/upload_binary" \
-X POST -H "Content-Type: application/octet-stream" \
--data-binary "@postgrest-logo.png"
create or replace get_image(id int) returns "image/png" as $$
select file from files where id = $1;
$$ language sql;
To get the image from the database, set the ``Accept: application/octet-stream`` header and select only the
``bytea`` type column.
.. code-block:: bash
.. tabs::
.. code-tab:: http
GET /files?select=file&id=eq.1 HTTP/1.1
Accept: application/octet-stream
.. code-tab:: bash Curl
curl "http://localhost:3000/files?select=file&id=eq.1" \
-H "Accept: application/octet-stream"
Use more accurate headers according to the type of the files by using the :ref:`raw-media-types` configuration. For example, adding the ``raw-media-types="image/png"`` setting to the configuration file will allow you to use the ``Accept: image/png`` header:
.. tabs::
.. code-tab:: http
GET /files?select=file&id=eq.1 HTTP/1.1
Accept: image/png
.. code-tab:: bash Curl
curl "http://localhost:3000/files?select=file&id=eq.1" \
-H "Accept: image/png"
curl "http://localhost:3000/get_image?id=1" \
-H "Accept: image/png"
See :ref:`providing_img` for a step-by-step example on how to handle images in HTML.
@@ -541,6 +148,104 @@ See :ref:`providing_img` for a step-by-step example on how to handle images in H
Be careful when saving binaries in the database, having a separate storage service for these is preferable in most cases. See `Storing Binary files in the Database <https://wiki.postgresql.org/wiki/BinaryFilesInDB>`_.
Composite Types
---------------
With PostgREST, you have two options to handle `composite type columns <https://www.postgresql.org/docs/current/rowtypes.html>`_.
.. code-block:: postgres
create type dimension as (
length decimal(6,2),
width decimal (6,2),
height decimal (6,2),
unit text
);
create table products (
id int primary key,
size dimension
);
insert into products (id, size)
values (1, '(5.0,5.0,10.0,"cm")');
On one hand you can insert values using string representation.
.. code-block:: bash
curl "http://localhost:3000/products" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{ "id": 2, "size": "(0.7,0.5,1.8,\"m\")" }
EOF
Or you could insert the same data in JSON format.
.. code-block:: bash
curl "http://localhost:3000/products" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 2,
"size": {
"length": 0.7,
"width": 0.5,
"height": 1.8,
"unit": "m"
}
}
EOF
You can also query the data using arrow operators. See :ref:`composite_array_columns`.
Enums
-----
You can handle `Enumerated Types <https://www.postgresql.org/docs/current/datatype-enum.html>`_ using string representations:
.. code-block:: postgres
create type letter_size as enum ('s','m','l','xl');
create table products (
id int primary key generated always as identity,
name text,
size letter_size
);
To insert or update the value use a string:
.. code-block:: bash
curl -X POST "http://localhost:3000/products" \
-H "Content-Type: application/json" \
-d @- << EOF
{ "name": "t-shirt", "size": "l" }
EOF
You can then query and filter the enum using the compatible :ref:`operators <operators>`.
For example, to get all the products larger than `m` and ordering them by their size:
.. code-block:: bash
curl "http://localhost:3000/products?select=name,size&size=gt.m&order=size"
.. code-block:: json
[
{
"name": "t-shirt",
"size": "l"
},
{
"name": "hoodie",
"size": "xl"
}
]
hstore
------
@@ -558,53 +263,67 @@ You can work with data types belonging to additional supplied modules such as `h
The ``name`` column will have the name of the country in different formats. You can insert values using the string representation for that data type:
.. tabs::
.. code-tab:: http
POST /countries HTTP/1.1
Content-Type: application/json
.. code-block:: bash
curl "http://localhost:3000/countries" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 1, "name": "common => Egypt, official => \"Arab Republic of Egypt\", native => مصر" },
{ "id": 2, "name": "common => Germany, official => \"Federal Republic of Germany\", native => Deutschland" }
]
.. code-tab:: bash Curl
curl "http://localhost:3000/countries" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 1, "name": "common => Egypt, official => \"Arab Republic of Egypt\", native => مصر" },
{ "id": 2, "name": "common => Germany, official => \"Federal Republic of Germany\", native => Deutschland" }
]
EOF
EOF
Notice that the use of ``"`` in the value of the ``name`` column needs to be escaped using a backslash ``\``.
You can also query and filter the value of a ``hstore`` column using the arrow operators, as you would do for a :ref:`JSON column<json_columns>`. For example, if you want to get the native name of Egypt:
.. tabs::
.. code-block:: bash
.. code-tab:: http
GET /countries?select=name->>native&name->>common=like.Egypt HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/countries?select=name->>native&name->>common=like.Egypt"
curl "http://localhost:3000/countries?select=name->>native&name->>common=like.Egypt"
.. code-block:: json
[{ "native": "مصر" }]
JSON
----
To work with a ``json`` type column, you can handle the value as a JSON object.
.. code-block:: postgres
create table products (
id int primary key,
name text unique,
extra_info json
);
You can insert a new product using a JSON object for the ``extra_info`` column:
.. code-block:: bash
curl "http://localhost:3000/products" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 1,
"name": "Canned fish",
"extra_info": {
"expiry_date": "2025-12-31",
"exportable": true
}
}
EOF
To query and filter the data see :ref:`json_columns` for a complete reference.
.. _ww_postgis:
PostGIS
-------
You can use the string representation for `PostGIS <https://postgis.net/>`_ data types such as ``geometry`` or ``geography`` (you need to `install PostGIS <https://postgis.net/install/>`_ first).
You can use the string representation for `PostGIS <https://postgis.net/>`_ data types such as ``geometry`` or ``geography`` (you need to `install PostGIS <https://postgis.net/documentation/getting_started/>`_ first).
.. code-block:: postgres
@@ -619,42 +338,23 @@ You can use the string representation for `PostGIS <https://postgis.net/>`_ data
To add areas in polygon format, you can use string representation:
.. tabs::
.. code-tab:: http
POST /coverage HTTP/1.1
Content-Type: application/json
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 1, "name": "small", "area": "SRID=4326;POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" },
{ "id": 2, "name": "big", "area": "SRID=4326;POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))" }
]
.. code-tab:: bash Curl
curl "http://localhost:3000/coverage" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 1, "name": "small", "area": "SRID=4326;POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" },
{ "id": 2, "name": "big", "area": "SRID=4326;POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))" }
]
EOF
EOF
Now, when you request the information, PostgREST will automatically cast the ``area`` column into a ``Polygon`` geometry type. Although this is useful, you may need the whole output to be in `GeoJSON <https://geojson.org/>`_ format out of the box, which can be done by including the ``Accept: application/geo+json`` in the request. This will work for PostGIS versions 3.0.0 and up and will return the output as a `FeatureCollection Object <https://www.rfc-editor.org/rfc/rfc7946#section-3.3>`_:
.. tabs::
.. code-block:: bash
.. code-tab:: http
GET /coverage HTTP/1.1
Accept: application/geo+json
.. code-tab:: bash Curl
curl "http://localhost:3000/coverage" \
-H "Accept: application/geo+json"
curl "http://localhost:3000/coverage" \
-H "Accept: application/geo+json"
.. code-block:: json
@@ -718,15 +418,9 @@ In the case that you are using older PostGIS versions, then creating a function
Now this query will return the same results:
.. tabs::
.. code-block:: bash
.. code-tab:: http
GET /rpc/coverage_geo_collection HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/rpc/coverage_geo_collection"
curl "http://localhost:3000/rpc/coverage_geo_collection"
.. code-block:: json
@@ -761,3 +455,157 @@ Now this query will return the same results:
}
]
}
Ranges
------
PostgREST allows you to handle `ranges <https://www.postgresql.org/docs/current/rangetypes.html>`_.
.. code-block:: postgres
create table events (
id int primary key,
name text unique,
duration tsrange
);
To insert a new event, specify the ``duration`` value as a string representation of the ``tsrange`` type:
.. code-block:: bash
curl "http://localhost:3000/events" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
{
"id": 1,
"name": "New Year's Party",
"duration": "['2022-12-31 11:00','2023-01-01 06:00']"
}
EOF
You can use range :ref:`operators <operators>` to filter the data. But, in this case, requesting a filter like ``events?duration=cs.2023-01-01`` will return an error, because PostgreSQL needs an explicit cast from string to timestamp. A workaround is to use a range starting and ending in the same date:
.. code-block:: bash
curl "http://localhost:3000/events?duration=cs.\[2023-01-01,2023-01-01\]"
.. code-block:: json
[
{
"id": 1,
"name": "New Year's Party",
"duration": "[\"2022-12-31 11:00:00\",\"2023-01-01 06:00:00\"]"
}
]
.. _casting_range_to_json:
Casting a Range to a JSON Object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As you may have noticed, the ``tsrange`` value is returned as a string literal. To return it as a JSON value, first you need to create a function that will do the conversion from a ``tsrange`` type:
.. code-block:: postgres
create or replace function tsrange_to_json(tsrange) returns json as $$
select json_build_object(
'lower', lower($1)
, 'upper', upper($1)
, 'lower_inc', lower_inc($1)
, 'upper_inc', upper_inc($1)
);
$$ language sql;
Then, create the cast using this function:
.. code-block:: postgres
create cast (tsrange as json) with function tsrange_to_json(tsrange) as assignment;
Finally, do the request :ref:`casting the range column <casting_columns>`:
.. code-block:: bash
curl "http://localhost:3000/events?select=id,name,duration::json"
.. code-block:: json
[
{
"id": 1,
"name": "New Year's Party",
"duration": {
"lower": "2022-12-31T11:00:00",
"upper": "2023-01-01T06:00:00",
"lower_inc": true,
"upper_inc": true
}
}
]
.. note::
If you don't want to modify casts for built-in types, an option would be to `create a custom type <https://www.postgresql.org/docs/current/sql-createtype.html>`_
for your own ``tsrange`` and add its own cast.
.. code-block:: postgres
create type mytsrange as range (subtype = timestamp, subtype_diff = tsrange_subdiff);
-- define column types and casting function analogously to the above example
-- ...
create cast (mytsrange as json) with function mytsrange_to_json(mytsrange) as assignment;
Timestamps
----------
You can use the **time zone** to filter or send data if needed.
.. code-block:: postgres
create table reports (
id int primary key
, due_date timestamptz
);
Suppose you are located in Sydney and want create a report with the date in the local time zone. Your request should look like this:
.. code-block:: bash
curl "http://localhost:3000/reports" \
-X POST -H "Content-Type: application/json" \
-d '[{ "id": 1, "due_date": "2022-02-24 11:10:15 Australia/Sydney" },{ "id": 2, "due_date": "2022-02-27 22:00:00 Australia/Sydney" }]'
Someone located in Cairo can retrieve the data using their local time, too:
.. code-block:: bash
curl "http://localhost:3000/reports?due_date=eq.2022-02-24+02:10:15+Africa/Cairo"
.. code-block:: json
[
{
"id": 1,
"due_date": "2022-02-23T19:10:15-05:00"
}
]
The response has the date in the time zone configured by the server: ``UTC -05:00`` (see :ref:`prefer_timezone`).
You can use other comparative filters and also all the `PostgreSQL special date/time input values <https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-DATETIME-SPECIAL-TABLE>`_ as illustrated in this example:
.. code-block:: bash
curl "http://localhost:3000/reports?or=(and(due_date.gte.today,due_date.lte.tomorrow),and(due_date.gt.-infinity,due_date.lte.epoch))"
.. code-block:: json
[
{
"id": 2,
"due_date": "2022-02-27T06:00:00-05:00"
}
]
+105 -132
View File
@@ -5,7 +5,7 @@ PostgREST Documentation
.. container:: image-container
.. figure:: _static/logo.png
.. figure:: ../static/postgrest.png
.. image:: https://img.shields.io/github/stars/postgrest/postgrest.svg?style=social
:target: https://github.com/PostgREST/postgrest
@@ -28,48 +28,71 @@ Sponsors
.. container:: image-container
.. image:: _static/cybertec-new.png
:target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
:width: 13em
.. container:: img-dark
.. image:: _static/2ndquadrant.png
:target: https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo
:width: 13em
.. image:: ../static/cybertec-dark.svg
:target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
.. image:: _static/retool.png
:target: https://retool.com/?utm_source=sponsor&utm_campaign=postgrest
:width: 13em
.. container:: img-light
.. image:: _static/gnuhost.png
:target: https://euronodes.com/?utm_source=sponsor&utm_campaign=postgrest
:width: 13em
.. image:: ../static/cybertec.svg
:target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
.. image:: _static/supabase.png
:target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
:width: 13em
.. image:: ../static/gnuhost.png
:target: https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest
.. image:: _static/oblivious.jpg
:target: https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest
:width: 13em
.. container:: img-dark
.. image:: ../static/neon-dark.jpg
:target: https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-light
.. image:: ../static/neon.jpg
:target: https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest
|
.. container:: img-dark
.. image:: ../static/code-build-dark.png
:target: https://code.build/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-light
.. image:: ../static/code-build.png
:target: https://code.build/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-dark
.. image:: ../static/supabase-dark.png
:target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
.. container:: img-light
.. image:: ../static/supabase.png
:target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
.. image:: ../static/tembo.png
:target: https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest
.. The static/empty.png(created with `convert -size 320x95 xc:#fcfcfc empty.png`) is an ugly workaround
to create space and center the logos. It's not easy to layout with restructuredText.
.. .. image:: _static/empty.png
:target: #sponsors
:width: 13em
|
Motivation
----------
Database as Single Source of Truth
----------------------------------
Using PostgREST is an alternative to manual CRUD programming. Custom API servers suffer problems. Writing business logic often duplicates, ignores or hobbles database structure. Object-relational mapping is a leaky abstraction leading to slow imperative code. The PostgREST philosophy establishes a single declarative source of truth: the data itself.
Declarative Programming
-----------------------
It's easier to ask PostgreSQL to join data for you and let its query planner figure out the details than to loop through rows yourself. It's easier to assign permissions to db objects than to add guards in controllers. (This is especially true for cascading permissions in data dependencies.) It's easier to set constraints than to litter code with sanity checks.
It's easier to ask PostgreSQL to join data for you and let its query planner figure out the details than to loop through rows yourself. It's easier to assign permissions to database objects than to add guards in controllers. (This is especially true for cascading permissions in data dependencies.) It's easier to set constraints than to litter code with sanity checks.
Leak-proof Abstraction
----------------------
@@ -86,21 +109,10 @@ Getting Support
The project has a friendly and growing community. For discussions, use the Github `discussions page <https://github.com/PostgREST/postgrest/discussions>`_. You can also report or search for bugs/features on the Github `issues <https://github.com/PostgREST/postgrest/issues>`_ page.
.. toctree::
:glob:
:caption: Release Notes
:titlesonly:
:hidden:
Release Notes
-------------
v10.2.0 <releases/v10.2.0>
v10.0.0 <releases/v10.0.0>
v9.0.1 <releases/v9.0.1>
v9.0.0 <releases/v9.0.0>
releases/v8.0.0
releases/v7.0.1
releases/v7.0.0
releases/v6.0.2
releases/v5.2.0
The release notes are published on `PostgREST's GitHub release page <https://github.com/PostgREST/postgrest/releases>`_.
Tutorials
---------
@@ -110,104 +122,74 @@ Are you new to PostgREST? This is the place to start!
.. toctree::
:glob:
:caption: Tutorials
:hidden:
:maxdepth: 1
tutorials/*
- :doc:`tutorials/tut0`
- :doc:`tutorials/tut1`
Also have a look at :ref:`install` and :ref:`community_tutorials`.
Also have a look at :doc:`Installation <install>` and :ref:`community_tutorials`.
Reference guides
----------------
References
----------
Technical references for PostgREST's functionality.
.. toctree::
:caption: API
:hidden:
:glob:
:caption: References
:name: references
:maxdepth: 1
api.rst
references/auth.rst
references/api.rst
references/cli.rst
references/transactions.rst
references/connection_pool.rst
references/schema_cache.rst
references/errors.rst
references/configuration.rst
references/observability.rst
references/*
.. toctree::
:caption: Configuration
:hidden:
configuration.rst
.. toctree::
:caption: Schema Cache
:hidden:
schema_cache.rst
.. toctree::
:caption: Errors
:hidden:
errors.rst
- :doc:`API <api>`
- :doc:`configuration`
- :doc:`Schema Cache <schema_cache>`
- :doc:`Errors <errors>`
Topic guides
Explanations
------------
Explanations of some key concepts in PostgREST.
Key concepts in PostgREST.
.. toctree::
:caption: Authentication
:hidden:
:glob:
:caption: Explanations
:name: explanations
:maxdepth: 1
auth.rst
explanations/*
.. toctree::
:caption: Schema Structure
:hidden:
How-tos
-------
schema_structure.rst
.. toctree::
:caption: Administration
:hidden:
admin.rst
.. toctree::
:caption: Installation
:hidden:
install.rst
- :doc:`Authentication <auth>`
- :doc:`Schema Structure <schema_structure>`
- :doc:`Administration <admin>`
- :doc:`Installation <install>`
.. _how_tos:
How-to guides
-------------
These are recipes that'll help you address specific use-cases.
Recipes that'll help you address specific use-cases.
.. toctree::
:glob:
:caption: How-to guides
:hidden:
:name: how-tos
:maxdepth: 1
how-tos/working-with-postgresql-data-types
how-tos/providing-images-for-img
how-tos/create-soap-endpoint
how-tos/sql-user-management-using-postgres-users-and-passwords
how-tos/sql-user-*
how-tos/working-*
how-tos/*
- :doc:`how-tos/providing-images-for-img`
- :doc:`how-tos/working-with-postgresql-data-types`
- :doc:`how-tos/create-soap-endpoint`
- :doc:`how-tos/sql-user-management-using-postgres-users-and-passwords`
.. _intgrs:
Integrations
------------
.. toctree::
:glob:
:caption: Integrations
:name: integrations
:maxdepth: 1
integrations/*
Ecosystem
---------
@@ -216,27 +198,11 @@ PostgREST has a growing ecosystem of examples, libraries, and experiments. Here
.. toctree::
:caption: Ecosystem
:hidden:
:name: ecosystem
:maxdepth: 1
ecosystem.rst
* :ref:`community_tutorials`
* :ref:`templates`
* :ref:`eco_example_apps`
* :ref:`devops`
* :ref:`eco_external_notification`
* :ref:`eco_extensions`
* :ref:`clientside_libraries`
Release Notes
-------------
Changes among versions.
- :doc:`releases/v9.0.0`
- :doc:`releases/v8.0.0`
In Production
-------------
@@ -246,12 +212,19 @@ Here are some companies that use PostgREST in production.
* `Datrium <https://www.datrium.com>`_
* `Drip Depot <https://www.dripdepot.com>`_
* `Image-charts <https://www.image-charts.com>`_
* `Moat <https://www.oracle.com/advertising/>`_
* `Netwo <https://www.netwo.io>`_
* `Nimbus <https://www.nimbusforwork.com>`_
* `Nimbus <https://www.nimbusfacility.com/sg/home>`_
- See how Nimbus uses PostgREST in `Paul Copplestone's blog post <https://paul.copplest.one/blog/nimbus-tech-2019-04.html>`_.
* `OpenBooking <https://www.openbooking.ch>`_
* `OpenBooking <https://openbooking.ch>`_
* `Redsmin <https://www.redsmin.com>`_
* `Sompani <https://www.sompani.com>`_
* `Supabase <https://supabase.com>`_
.. Failing links
* `eGull <http://www.egull.co>`_
* `MotionDynamic - Fast highly dynamic video generation at scale <https://motiondynamic.tech>`_
Testimonials
------------
+6
View File
@@ -0,0 +1,6 @@
Greenplum
#########
`Greenplum <https://greenplum.org/>`_ has been reported to work by adding ``LOGIN`` to the :ref:`anonymous and user roles <roles>`.
For more details, see https://github.com/PostgREST/postgrest/issues/2021.
+11
View File
@@ -0,0 +1,11 @@
.. _external_jwt:
External JWT Generation
-----------------------
JWT from Auth0
~~~~~~~~~~~~~~
An external service like `Auth0 <https://auth0.com/>`_ can do the hard work transforming OAuth from Github, Twitter, Google etc into a JWT suitable for PostgREST. Auth0 can also handle email signup and password reset flows.
To use Auth0, create `an application <https://auth0.com/docs/get-started/applications>`_ for your app and `an API <https://auth0.com/docs/get-started/apis>`_ for your PostgREST server. Auth0 supports both HS256 and RS256 scheme for the issued tokens for APIs. For simplicity, you may first try HS256 scheme while creating your API on Auth0. Your application should use your PostgREST API's `API identifier <https://auth0.com/docs/get-started/apis/api-settings>`_ by setting it with the `audience parameter <https://auth0.com/docs/secure/tokens/access-tokens/get-access-tokens#control-access-token-audience>`_ during the authorization request. This will ensure that Auth0 will issue an access token for your PostgREST API. For PostgREST to verify the access token, you will need to set ``jwt-secret`` on PostgREST config file with your API's signing secret.
+30
View File
@@ -0,0 +1,30 @@
pg-safeupdate
#############
.. _block_fulltable:
Block Full-Table Operations
---------------------------
If the :ref:`active role <user_impersonation>` can delete table rows then the DELETE verb is allowed for clients. Here's an API request to delete old rows from a hypothetical logs table:
.. code-block:: bash
curl "http://localhost:3000/logs?time=lt.1991-08-06" -X DELETE
Note that it's very easy to delete the **entire table** by omitting the query parameter!
.. code-block:: bash
curl "http://localhost:3000/logs" -X DELETE
This can happen accidentally such as by switching a request from a GET to a DELETE. To protect against accidental operations use the `pg-safeupdate <https://github.com/eradman/pg-safeupdate>`_ PostgreSQL extension. It raises an error if UPDATE or DELETE are executed without specifying conditions. To install it you can use the `PGXN <https://pgxn.org/>`_ network:
.. code-block:: bash
sudo -E pgxn install safeupdate
# then add this to postgresql.conf:
# shared_preload_libraries='safeupdate';
This does not protect against malicious actions, since someone can add a url parameter that does not affect the result set. To prevent this you must turn to database permissions, forbidding the wrong people from deleting rows, and using `row-level security <https://www.postgresql.org/docs/current/ddl-rowsecurity.html>`_ if finer access control is required.
+59
View File
@@ -0,0 +1,59 @@
systemd
=======
For Linux distributions that use **systemd** (Ubuntu, Debian, Arch Linux) you can create a daemon in the following way.
First, create postgrest configuration in ``/etc/postgrest/config``
.. code-block:: ini
db-uri = "postgres://<your_user>:<your_password>@localhost:5432/<your_db>"
db-schemas = "<your_exposed_schema>"
db-anon-role = "<your_anon_role>"
jwt-secret = "<your_secret>"
Create a dedicated ``postgrest`` user with:
.. code-block:: ini
sudo useradd -M -U -d /nonexistent -s /usr/sbin/nologin postgrest
Then create the systemd service file in ``/etc/systemd/system/postgrest.service``
.. code-block:: ini
[Unit]
Description=REST API for any PostgreSQL database
After=postgresql.service
[Service]
User=postgrest
Group=postgrest
ExecStart=/bin/postgrest /etc/postgrest/config
ExecReload=/bin/kill -SIGUSR1 $MAINPID
[Install]
WantedBy=multi-user.target
After that, you can enable the service at boot time and start it with:
.. code-block:: bash
systemctl enable postgrest
systemctl start postgrest
## For reloading the service
## systemctl restart postgrest
.. _file_descriptors:
File Descriptors
----------------
File descriptors are kernel resources that are used by HTTP connections (among others). File descriptors are limited per process. The kernel default limit is 1024, which is increased in some Linux distributions.
When under heavy traffic, PostgREST can reach this limit and start showing ``No file descriptors available`` errors. To clear these errors, you can increase the process' file descriptor limit.
.. code-block:: ini
[Service]
LimitNOFILE=10000
-11
View File
@@ -1,11 +0,0 @@
#!/usr/bin/env python
from livereload import Server, shell
from subprocess import call
## Build docs at startup
call(["sphinx-build", "-b", "html", "-a", "-n", ".", "_build"])
server = Server()
server.watch("**/*.rst", shell("sphinx-build -b html -a -n . _build"))
# For custom port and host
# server.serve(root='_build/', host='192.168.1.2')
server.serve(root="_build/")
+42 -59
View File
@@ -1,113 +1,107 @@
personal_ws-1.1 en 0 utf-8
Adossi
AMQP
api
API's
Archlinux
APIs
APISIX
aud
Auth
auth
authenticator
backoff
balancer
Beles
booleans
Bouscal
buildpack
BOM
Bytea
Cardano
casted
cd
centric
changelog
ClojureScript
cloudfared
CLI
coercible
conf
Cloudflare
config
cors
CORS
CPUs
cryptographically
CSV
Daemonizing
durations
DDL
DOM
DevOps
DiBiase
dockerize
DoS
enum
Enums
eq
ETH
Ethereum
EveryLayout
Fenko
Fernandes
filename
FreeBSD
fts
GC
GeoJSON
GHC
Github
Google
grantor
GraphQL
Greenplum
gte
GUC
GUCs
gucs
Gumbs
Haskell
Heroku
HMAC
htmx
Htmx
Homebrew
hstore
HTTP
HTTPS
HV
Ibarluzea
Inlining
inlined
Integrations
idletime
IDLETIME
ilike
imatch
io
IP
isdistinct
JS
js
JSON
JWK
JWT
jwt
JWTs
Kinesis
Kofi
Kubernetes
localhost
login
lookups
Logins
LIBPQ
logins
lon
lt
lte
middleware
macOS
misprediction
Mithril
multi
MVCC
namespace
namespaced
Nanos
neq
nginx
ngrep
nixpkgs
npm
nxl
nxr
OAuth
onwards
Observability
OpenAPI
openapi
ORM
ov
parametrized
passphrase
Pawel
PBKDF
Pelletier
Petr
PgBouncer
pgcrypto
pgjwt
@@ -121,7 +115,6 @@ phraseto
plainto
plfts
poolers
POSIX
PostGIS
PostgreSQL
PostgreSQL's
@@ -131,79 +124,69 @@ postgrest
PostgREST's
pre
preflight
plpgsql
psql
Qin
RabbitMQ
Rafaj
RDS
reallyreallyreallyreallyverysafe
Rechkemmer
reconnection
Redux
refactor
reloadable
Reloadable
Remo
requester's
RESTful
RestSharp
RLS
RPC
RSA
Saleeba
safeupdate
savepoint
schemas
Sencha
Serverless
Severin
schema's
SHA
signup
SIGUSR
sl
SNS
sqitch
spreaded
Spreaded
SQL
sql
SQLSTATE
sr
SSL
stateful
stdout
Stolarz
subselect
SuperAgent
supervisees
SvelteKit
SwaggerUI
syslog
systemd
Tcl
tmuxp
todo
todos
Tsingson
tos
tsquery
tx
Tyll
TypeScript
UI
ui
unicode
unikernel
unix
updatable
unfulfillable
Untyped
UPSERT
Upsert
upsert
uri
url
urlencoded
urls
variadic
Vercel
verifier
versioning
Vondra
Vue
WAI
webhooks
websearch
Websockets
webuser
wfts
ZeroMQ
www
+93
View File
@@ -0,0 +1,93 @@
.. _admin_server:
Admin Server
############
PostgREST provides an admin server that can be enabled by setting :ref:`admin-server-port`.
.. _health_check:
Health Check
============
You can enable a health check to verify if PostgREST is available for client requests. Also to check the status of its internal state.
Two endpoints ``live`` and ``ready`` will then be available.
.. important::
If you have a machine with multiple network interfaces and multiple PostgREST instances in the same port, you need to specify a unique :ref:`hostname <server-host>`
in the configuration of each PostgREST instance for the health check to work correctly. Don't use the special values(``!4``, ``*``, etc) in this case because the health check
could report a false positive.
Live
----
The ``live`` endpoint verifies if PostgREST is running on its configured port. A request will return ``200 OK`` if PostgREST is alive or ``500`` otherwise.
For instance, to verify if PostgREST is running while the ``admin-server-port`` is set to ``3001``:
.. code-block:: bash
curl -I "http://localhost:3001/live"
.. code-block:: http
HTTP/1.1 200 OK
Ready
-----
Additionally to the ``live`` check, the ``ready`` endpoint checks the state of the :ref:`connection_pool` and the :ref:`schema_cache`. A request will return ``200 OK`` if both are good or ``503`` if not.
.. code-block:: bash
curl -I "http://localhost:3001/ready"
.. code-block:: http
HTTP/1.1 200 OK
PostgREST will try to recover from the ``503`` state with :ref:`automatic_recovery`.
Metrics
=======
Provides :ref:`metrics`.
Runtime Configuration
=====================
Provides a ``config`` endpoint that returns the runtime :ref:`configuration`.
.. code-block:: bash
curl "http://localhost:3001/config"
.. code-block::
db-aggregates-enabled = false
db-anon-role = "web_anon"
db-channel = "pgrst"
db-channel-enabled = false
...
Runtime Schema Cache
====================
Provides the ``schema_cache`` endpoint that prints the runtime :ref:`schema_cache`.
.. code-block:: bash
curl "http://localhost:3001/schema_cache"
.. code-block:: json
{
"dbMediaHandlers": ["..."],
"dbRelationships": ["..."],
"dbRepresentations": ["..."],
"dbRoutines": ["..."],
"dbTables": ["..."],
"dbTimezones": ["..."]
}
+124
View File
@@ -0,0 +1,124 @@
.. _api:
API
###
PostgREST exposes three database objects of a schema as resources: tables, views and functions.
.. toctree::
:glob:
:maxdepth: 1
api/tables_views.rst
api/functions.rst
api/schemas.rst
api/computed_fields.rst
api/domain_representations.rst
api/pagination_count.rst
api/resource_embedding.rst
api/resource_representation.rst
api/media_type_handlers.rst
api/aggregate_functions.rst
api/openapi.rst
api/preferences.rst
api/*
.. raw:: html
<script type="text/javascript">
let hash = window.location.hash;
const redirects = {
// Tables and Views
'#horizontal-filtering-rows': 'api/tables_views.html#horizontal-filtering',
'#operators': 'api/tables_views.html#operators',
'#logical-operators': 'api/tables_views.html#logical-operators',
'#pattern-matching': 'api/tables_views.html#pattern-matching',
'#full-text-search': 'api/tables_views.html#full-text-search',
'#vertical-filtering-columns': 'api/tables_views.html#vertical-filtering',
'#renaming-columns': 'api/tables_views.html#renaming-columns',
'#casting-columns': 'api/tables_views.html#casting-columns',
'#json-columns': 'api/tables_views.html#json-columns',
'#composite-array-columns': 'api/tables_views.html#composite-array-columns',
'#computed-virtual-columns': 'api/computed_fields.html',
'#ordering': 'api/tables_views.html#ordering',
'#limits-and-pagination': 'api/pagination_count.html',
'#exact-count': 'api/pagination_count.html#exact-count',
'#planned-count': 'api/pagination_count.html#planned-count',
'#estimated-count': 'api/pagination_count.html#estimated-count',
'#updates': 'api/tables_views.html#update',
'#insertions': 'api/tables_views.html#insert',
'#bulk-insert': 'api/tables_views.html#bulk-insert',
'#specifying-columns': 'api/tables_views.html#specifying-columns',
'#upsert': 'api/tables_views.html#upsert',
'#on-conflict': 'api/tables_views.html#on-conflict',
'#put': 'api/tables_views.html#put',
'#deletions': 'api/tables_views.html#delete',
'#limited-updates-deletions': 'api/tables_views.html#limited-update-delete',
// Functions
'#stored-procedures': 'api/functions.html',
'#calling-functions-with-a-single-json-parameter': 'api/functions.html#functions-with-a-single-json-parameter',
'#calling-functions-with-a-single-unnamed-parameter': 'api/functions.html#functions-with-a-single-unnamed-parameter',
'#calling-functions-with-array-parameters': 'api/functions.html#functions-with-array-parameters',
'#calling-variadic-functions': 'api/functions.html#variadic-functions',
'#scalar-functions': 'api/functions.html#scalar-functions',
'#function-filters': 'api/functions.html#table-valued-functions',
'#overloaded-functions': 'api/functions.html#overloaded-functions',
// Schemas
'#switching-schemas': 'api/schemas.html',
// Resource Embedding
'#resource-embedding': 'api/resource_embedding.html#resource-embedding',
'#many-to-one-relationships': 'api/resource_embedding.html#many-to-one-relationships',
'#one-to-many-relationships': 'api/resource_embedding.html#one-to-many-relationships',
'#many-to-many-relationships': 'api/resource_embedding.html#many-to-many-relationships',
'#one-to-one-relationships': 'api/resource_embedding.html#one-to-one-relationships',
'#computed-relationships': 'api/resource_embedding.html#computed-relationships',
'#nested-embedding': 'api/resource_embedding.html#nested-embedding',
'#embedded-filters': 'api/resource_embedding.html#embedded-filters',
'#embedding-with-top-level-filtering': 'api/resource_embedding.html#top-level-filtering',
'#embedding-partitioned-tables': 'api/resource_embedding.html#foreign-key-joins-on-partitioned-tables',
'#embedding-views': 'api/resource_embedding.html#foreign-key-joins-on-views',
'#embedding-chains-of-views': 'api/resource_embedding.html#foreign-key-joins-on-chains-of-views',
'#embedding-on-stored-procedures': 'api/resource_embedding.html#foreign-key-joins-on-table-valued-functions',
'#embedding-after-insertions-updates-deletions': 'api/resource_embedding.html#foreign-key-joins-on-writes',
'#embedding-disambiguation': 'api/resource_embedding.html#foreign-key-joins-on-multiple-foreign-key-relationships',
'#target-disambiguation': 'api/resource_embedding.html#foreign-key-joins-on-multiple-foreign-key-relationships',
'#hint-disambiguation': 'api/resource_embedding.html#foreign-key-joins-on-multiple-foreign-key-relationships',
"#embedding-through-join-tables": "api/resource_embedding.html#many-to-many-relationships",
// OpenAPI
'#openapi-support': 'api/openapi.html',
// Resource Representation
'#response-format': 'api/resource_representation.html#response-format',
'#singular-or-plural': 'api/resource_representation.html#singular-or-plural',
'#response-formats-for-scalar-responses': 'api/functions.html#scalar-functions',
// CORS
'#cors': 'api/cors.html',
// OPTIONS
'#options': 'api/options.html',
// URL Grammar
'#custom-queries': 'api/url_grammar.html#custom-queries',
'#unicode-support': 'api/url_grammar.html#unicode-support',
'#table-columns-with-spaces': 'api/url_grammar.html#table-columns-with-spaces',
'#reserved-characters': 'api/url_grammar.html#reserved-characters',
// Transactions
'#immutable-and-stable-functions': 'transactions.html#access-mode',
'#http-context': 'transactions.html#transaction-scoped-settings',
'#accessing-request-headers-cookies-and-jwt-claims': 'transactions.html#request-headers-cookies-and-jwt-claims',
'#legacy-guc-variable-names': 'transactions.html#transaction-scoped-settings',
'#accessing-request-path-and-method': 'transactions.html#request-path-and-method',
'#setting-response-headers': 'transactions.html#response-headers',
'#setting-headers-via-pre-request': 'transactions.html#setting-headers-via-pre-request',
'#setting-response-status-code': 'transactions.html#response-status-code',
'#raise-errors-with-http-status-codes': 'errors.html#raise-errors-with-http-status-codes',
// Admin
'#execution-plan': 'observability.html#execution-plan',
// Deprecated
'#bulk-call': '../releases/v11.0.1.html#breaking-changes',
};
let willRedirectTo = redirects[hash];
if (willRedirectTo) {
window.location.href = willRedirectTo;
}
</script>
+288
View File
@@ -0,0 +1,288 @@
.. _aggregate_functions:
Aggregate Functions
###################
Aggregate functions allow you to summarize data by performing calculations across groups of rows. For instance, if you have an ``orders`` table that has an ``amount`` column, you could use an aggregate function to get the sum of the ``amount`` column, either for all rows, or for each group of rows that share specific values, for instance all rows that share the same ``order_date``.
.. note::
Aggregate functions are *disabled* by default in PostgREST, as without appropriate safeguards, aggregate functions can create performance problems. See :ref:`db-aggregates-enabled` for further details.
PostgREST supports the following aggregate functions: ``avg()``, ``count()``, ``max()``, ``min()``, and ``sum()``. Please refer to the `section on aggregate functions in the PostgreSQL documentation <https://www.postgresql.org/docs/current/functions-aggregate.html>`_ for a detailed explanation of these functions.
To use an aggregate function, you append the function to a value in the ``select`` parameter, like so:
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum()"
With the above query, PostgREST will return a single row with a single column named ``sum`` that contains the sum of all the values in the ``amount`` column:
.. code-block:: json
[
{
"sum": 1234.56
}
]
You can use multiple aggregate functions by just adding more columns with aggregate functions to the ``select`` parameter.
To group by other columns, you simply add those columns to the ``select`` parameter. For instance:
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum(),amount.avg(),order_date"
This will return a row for each unique value in the ``order_date`` column, with the sum and average of the ``amount`` column for all rows that share the same ``order_date``:
.. code-block:: json
[
{
"sum": 1234.56,
"avg": 123.45,
"order_date": "2023-01-01"
},
{
"sum": 2345.67,
"avg": 234.56,
"order_date": "2023-01-02"
}
]
.. note::
Aggregate functions work alongside other PostgREST features, like :ref:`h_filter`, :ref:`json_columns`, and :ref:`ordering`. Please note at this time aggregate functions are not compatible with :ref:`domain_reps`. Additionally, PostgreSQL's ``HAVING`` clause and ordering by aggregated columns are not yet supported.
The Case of ``count()``
===========================
.. note::
Before the addition of aggregate functions, it was possible to count by adding ``count`` (without parentheses) to the ``select`` parameter. While this is still supported, it may be deprecated in the future, and thus use of this legacy feature is **not recommended.** Please use ``count()`` (with parentheses) instead.
``count()`` is treated specially, as it can be used without an associated column. Take for example the following query:
.. code-block:: bash
curl "http://localhost:3000/orders?select=count(),order_date"
This would return a row for each unique value in the ``order_date`` column, with the count of all rows that share the same ``order_date``:
.. code-block:: json
[
{
"count": 4,
"order_date": "2023-01-01"
},
{
"count": 2,
"order_date": "2023-01-02"
}
]
When ``count()`` is used with an associated column, its behavior is slightly different: It will return the count of all values that are not ``NULL``. This is due to how PostgreSQL itself implements the ``count()`` function.
Renaming and Casting
====================
Renaming Aggregates
-------------------
Just like with other columns, you can rename aggregated columns too. See :ref:`renaming_columns` for details.
Renaming columns is especially helpful in the context of aggregate functions, as by default a column with an aggregate function applied will take on the name of the applied aggregate function. You may want to provide a more semantically meaningful name or prevent collisions when using multiple aggregate functions of the same type.
Casting Aggregates
------------------
When applying an aggregate function to a column, you are able to cast both the value of the input to the aggregate function *and* the value of the output from the aggregate function. In both cases, the syntax works as described in :ref:`casting_columns`, with the only difference being the placement of the cast.
Casting the Value of the Input
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For instance, imagine that the ``orders`` table has a JSON column, ``order_details``, and this column contains a JSON object that has a key, ``tax_amount``. Let's say you want to get the sum of the tax amount for every order. You can use the ``->`` or ``->>`` operators to extract the value with this key (see :ref:`json_columns`), but these operators will return values of the types JSON and ``text`` respectively, and neither of these types can be used with ``sum()``.
Therefore, you will need to first cast the input value to a type that is compatible with ``sum()`` (e.g. ``numeric``). Casting the input value is done in exactly the same way as casting any other value:
.. code-block:: bash
curl "http://localhost:3000/orders?select=order_details->tax_amount::numeric.sum()"
With this, you will receive the sum of the casted ``tax_amount`` value:
.. code-block:: json
[
{
"sum": 1234.56
}
]
Casting the Value of the Output
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now let's return to an example involving the ``amount`` column of the ``orders`` table. Imagine that we want to get the rounded average of the ``amount`` column. One way to do this is to use the ``avg()`` aggregate function and then to cast the output value of the function to ``int``. To cast the value of the output of the function, we simply place the cast *after* the aggregate function:
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.avg()::int"
You will then receive the rounded average as the result:
.. code-block:: json
[
{
"avg": 201
}
]
Of course, you can use both input and output casts at the same time, if you so desire.
Using Aggregate Functions with Resource Embedding
=================================================
Aggregate functions can be used in conjunction with :ref:`resource_embedding`. You can use embedded resources as grouping columns, use aggregate functions within the context of an embedded resource, or use columns from a spreaded resource as grouping columns or as inputs to aggregate functions.
Using Embedded Resources as Grouping Columns
--------------------------------------------
Using an embedded resource as a grouping column allows you to use data from an association to group the results of an aggregation.
For example, imagine that the ``orders`` table from the examples above is related to a ``customers`` table. If you want to get the sum of the ``amount`` column grouped by the ``name`` column from the ``customers`` table, you can include the customer name, using the standard :ref:`resource_embedding` syntax, and perform a sum on the ``amount`` column.
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum(),customers(name)"
You will then get the summed amount, along with the embedded customer resource:
.. code-block:: json
[
{
"sum": 100,
"customers": {
"name": "Customer A"
}
},
{
"sum": 200,
"customers": {
"name": "Customer B"
}
}
]
.. note::
The previous example uses a has-one association to demonstrate this functionality, but you may also use has-many associations as grouping columns, although there are few obvious use cases for this.
Using Aggregate Functions Within the Context of an Embedded Resource
--------------------------------------------------------------------
When embedding a resource, you can apply aggregate functions to columns from the associated resource to perform aggregations within the context of an embedded resource.
Continuing with the example relationship between ``orders`` and ``customers`` from the previous section, imagine that you want to fetch the ``name``, ``city``, and ``state`` for each customer, along with the sum of amount of the customer's orders, grouped by the order date. This can be done in the following way:
.. code-block:: bash
curl "http://localhost:3000/customers?select=name,city,state,orders(amount.sum(),order_date)"
.. code-block:: json
[
{
"name": "Customer A",
"city": "New York",
"state": "NY",
"orders": [
{
"sum": 215.22,
"order_date": "2023-09-01"
},
{
"sum": 905.73,
"order_date": "2023-09-02"
}
]
},
{
"name": "Customer B",
"city": "Los Angeles",
"state": "CA",
"orders": [
{
"sum": 329.71,
"order_date": "2023-09-01"
},
{
"sum": 425.87,
"order_date": "2023-09-03"
}
]
}
]
In this example, the ``amount`` column is summed and grouped by the ``order_date`` *within* the context of the embedded resource. That is, the ``name``, ``city``, and ``state`` from the ``customers`` table have no bearing on the aggregation performed in the context of the ``orders`` association; instead, each aggregation can be seen as being performed independently on just the orders belonging to a particular customer, using only the data from the embedded resource for both grouping and aggregation.
Using Columns from a Spreaded Resource
--------------------------------------
When you :ref:`spread an embedded resource <spread_embed>`, the columns from the spreaded resource are treated as if they were columns of the top-level resource, both when using them as grouping columns and when applying aggregate functions to them.
Grouping with Columns from a Spreaded Resource
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For instance, assume you want to sum the ``amount`` column from the ``orders`` table, using the ``city`` and ``state`` columns from the ``customers`` table as grouping columns. To achieve this, you may select these two columns from the ``customers`` table and spread them; they will then be used as grouping columns:
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum(),...customers(city,state)
The result will be the same as if ``city`` and ``state`` were columns from the ``orders`` table:
.. code-block:: json
[
{
"sum": 2000.29,
"city": "New York",
"state": "NY"
},
{
"sum": 9241.21,
"city": "Los Angeles",
"state": "CA"
}
]
Aggregate Functions with Columns from a Spreaded Resource
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now imagine that the ``customers`` table has a ``joined_date`` column that represents the date that the customer joined. You want to get both the most recent and the oldest ``joined_date`` for customers that placed an order on every distinct order date. This can be expressed as follows:
.. code-block:: bash
curl "http://localhost:3000/orders?select=order_date,...customers(joined_date.max(),joined_date.min())
As columns from a spreaded resource are treated as if they were columns from the top-level resource, the ``max()`` and ``min()`` are applied *within* the context of the top-level, rather than within the context of the embedded resource, as in the previous section.
The result will be the same as if the aggregations were applied to columns from the top-level:
.. code-block:: json
[
{
"order_date": "2023-11-01",
"max": "2023-10-15",
"min": "2013-10-01"
},
{
"order_date": "2023-11-02",
"max": "2023-10-30",
"min": "2016-02-11"
}
]
+75
View File
@@ -0,0 +1,75 @@
.. _computed_cols:
Computed Fields
###############
Computed fields are virtual columns that are not stored in a table. PostgreSQL makes it possible to implement them using functions on table types.
.. code-block:: postgres
CREATE TABLE people (
first_name text
, last_name text
, job text
);
-- a computed field that combines data from two columns
CREATE FUNCTION full_name(people)
RETURNS text AS $$
SELECT $1.first_name || ' ' || $1.last_name;
$$ LANGUAGE SQL;
Horizontal Filtering on Computed Fields
=======================================
:ref:`h_filter` can be applied to computed fields. For example, we can do a :ref:`fts` on :code:`full_name`:
.. code-block:: postgres
-- (optional) you can add an index on the computed field to speed up the query
CREATE INDEX people_full_name_idx ON people
USING GIN (to_tsvector('english', full_name(people)));
.. code-block:: bash
curl "http://localhost:3000/people?full_name=fts.Beckett"
.. code-block:: json
[
{"first_name": "Samuel", "last_name": "Beckett", "job": "novelist"}
]
Vertical Filtering on Computed Fields
=====================================
Computed fields won't appear on the response by default but you can use :ref:`v_filter` to include them:
.. code-block:: bash
curl "http://localhost:3000/people?select=full_name,job"
.. code-block:: json
[
{"full_name": "Samuel Beckett", "job": "novelist"}
]
Ordering on Computed Fields
===========================
:ref:`ordering` on computed fields is also possible:
.. code-block:: bash
curl "http://localhost:3000/people?order=full_name.desc"
.. important::
Computed fields must be created in the :ref:`exposed schema <db-schemas>` or in a schema in the :ref:`extra search path <db-extra-search-path>` to be used in this way. When placing the computed field in the :ref:`exposed schema <db-schemas>` you can use an **unnamed** parameter, as in the example above, to prevent it from being exposed as an :ref:`RPC <functions>` under ``/rpc``.
.. note::
- PostgreSQL 12 introduced `generated columns <https://www.postgresql.org/docs/12/ddl-generated-columns.html>`_, which can also compute a value based on other columns. However they're stored, not virtual.
- "computed fields" are documented on https://www.postgresql.org/docs/current/rowtypes.html#ROWTYPES-USAGE (search for "computed fields")
- On previous PostgREST versions this feature was documented with the name of "computed columns".
+40
View File
@@ -0,0 +1,40 @@
.. _cors:
CORS
####
By default, PostgREST sets highly permissive cross origin resource sharing, that is why it accepts Ajax requests from any domain. This behavior can be configured by using :ref:`server_cors_allowed_origins`.
It also handles `preflight requests <https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request>`_ done by the browser, which are cached using the returned ``Access-Control-Max-Age: 86400`` header (86400 seconds = 24 hours). This is useful to reduce the latency of the subsequent requests.
A ``POST`` preflight request would look like this:
.. code-block:: bash
curl -i "http://localhost:3000/items" \
-X OPTIONS \
-H "Origin: http://example.com" \
-H "Access-Control-Request-Method: POST" \
-H "Access-Control-Request-Headers: Content-Type"
.. code-block:: http
HTTP/1.1 200 OK
Access-Control-Allow-Origin: http://example.com
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, PATCH, PUT, DELETE, OPTIONS, HEAD
Access-Control-Allow-Headers: Authorization, Content-Type, Accept, Accept-Language, Content-Language
Access-Control-Max-Age: 86400
.. _allowed_origins:
Allowed Origins
===============
With the following config setting, PostgREST will accept CORS requests from domains :code:`http://example.com` and :code:`http://example2.com`.
.. code-block::
server-cors-allowed-origins="http://example.com, http://example2.com"
@@ -0,0 +1,176 @@
.. _domain_reps:
Domain Representations
######################
Domain Representations separates "how the data is presented" from "how the data is stored". It works by creating `domains <https://www.postgresql.org/docs/current/sql-createdomain.html>`_ and `casts <https://www.postgresql.org/docs/current/sql-createcast.html>`_, the latter act on the former to present and receive the data in different formats.
.. contents::
:depth: 1
:local:
:backlinks: none
Custom Domain
=============
Suppose you want to use a ``uuid`` type for a primary key and want to present it shortened to web users.
For this, let's create a domain based on ``uuid``.
.. code-block:: postgres
create domain app_uuid as uuid;
-- and use it as our table PK.
create table profiles(
id app_uuid
, name text
);
-- some data for the example
insert into profiles values ('846c4ffd-92ce-4de7-8d11-8e29929f4ec4', 'John Doe');
Domain Response Format
======================
We can shorten the ``uuid`` with ``base64`` encoding. Let's use JSON as our response format for this example.
To change the domain format for JSON, create a function that converts ``app_uuid`` to ``json``.
.. code-block:: postgres
-- the name of the function is arbitrary
CREATE OR REPLACE FUNCTION json(app_uuid) RETURNS json AS $$
select to_json(encode(uuid_send($1),'base64'));
$$ LANGUAGE SQL IMMUTABLE;
-- check it works
select json('846c4ffd-92ce-4de7-8d11-8e29929f4ec4'::app_uuid);
json
----------------------------
"hGxP/ZLOTeeNEY4pkp9OxA=="
Then create a CAST to tell PostgREST to convert it automatically whenever a JSON response is requested.
.. code-block:: postgres
CREATE CAST (app_uuid AS json) WITH FUNCTION json(app_uuid) AS IMPLICIT;
With this you can obtain the data in the shortened format.
.. code-block:: bash
curl "http://localhost:3000/profiles" \
-H "Accept: application/json"
.. code-block:: json
[{"id":"hGxP/ZLOTeeNEY4pkp9OxA==","name":"John Doe"}]
.. note::
- Casts on domains are ignored by PostgreSQL, their interpretation is left to the application. We're discussing the possibility of including the Domain Representations behavior on `pgsql-hackers <https://www.postgresql.org/message-id/flat/CAGRrpzZKa%2BGu91j1SOvN3tM1f-7Gh_w441c5nAX1QqdH3Q31Lg%40mail.gmail.com>`_.
- It would make more sense to use ``base58`` encoding as it's URL friendly but for simplicity we use ``base64`` (supported natively in PostgreSQL).
.. important::
After creating a cast over a domain, you must refresh PostgREST schema cache. See :ref:`schema_reloading`.
Domain Filter Format
====================
For :ref:`h_filter` to work with the shortened format, you need a different conversion.
PostgREST considers the URL query string to be, in the most generic sense, ``text``. So let's create a function that converts ``text`` to ``app_uuid``.
.. code-block:: postgres
-- the name of the function is arbitrary
CREATE OR REPLACE FUNCTION app_uuid(text) RETURNS app_uuid AS $$
select substring(decode($1,'base64')::text from 3)::uuid;
$$ LANGUAGE SQL IMMUTABLE;
-- plus a CAST to tell PostgREST to use this function
CREATE CAST (text AS app_uuid) WITH FUNCTION app_uuid(text) AS IMPLICIT;
Now you can filter as usual.
.. code-block:: bash
curl "http://localhost:3000/profiles?id=eq.ZLOTeeNEY4pkp9OxA==" \
-H "Accept: application/json"
.. code-block:: json
[{"id":"hGxP/ZLOTeeNEY4pkp9OxA==","name":"John Doe"}]
.. note::
If there's no CAST from ``text`` to ``app_uuid`` defined, the filter will still work with the native uuid format (``846c4ffd-92ce-4de7-8d11-8e29929f4ec4``).
Domain Request Body Format
==========================
To accept the shortened format in a JSON request body, for example when creating a new record, define a ``json`` to ``app_uuid`` conversion.
.. code-block:: postgres
-- the name of the function is arbitrary
CREATE OR REPLACE FUNCTION app_uuid(json) RETURNS public.app_uuid AS $$
-- here we reuse the previous app_uuid(text) function
select app_uuid($1 #>> '{}');
$$ LANGUAGE SQL IMMUTABLE;
CREATE CAST (json AS public.app_uuid) WITH FUNCTION app_uuid(json) AS IMPLICIT;
Now we can :ref:`insert` (or :ref:`update`) as usual.
.. code-block:: bash
curl "http://localhost:3000/profiles" \
-H "Prefer: return=representation" \
-H "Content-Type: application/json" \
-d @- <<JSON
{"id":"zH7HbFJUTfy/GZpwuirpuQ==","name":"Jane Doe"}
JSON
The response:
.. code-block:: json
[{"id":"zH7HbFJUTfy/GZpwuirpuQ==","name":"Jane Doe"}]
Note that on the database side we have our regular ``uuid`` format.
.. code-block:: postgres
select * from profiles;
id | name
--------------------------------------+----------
846c4ffd-92ce-4de7-8d11-8e29929f4ec4 | John Doe
cc7ec76c-5254-4dfc-bf19-9a70ba2ae9b9 | Jane Doe
(2 rows)
.. note::
If there's no CAST from ``json`` to ``app_uuid`` defined, the request body will still work with the native uuid format (``cc7ec76c-5254-4dfc-bf19-9a70ba2ae9b9``).
Advantages over Views
=====================
`Views <https://www.postgresql.org/docs/current/sql-createview.html>`_ also allow us to change the format of the underlying type. However they come with drawbacks that increase complexity.
1) Formatting the column in the view makes it `non-updatable <https://www.postgresql.org/docs/current/sql-createview.html#SQL-CREATEVIEW-UPDATABLE-VIEWS>`_ since Postgres doesn't know how to reverse the transform. This can be worked around using INSTEAD OF triggers.
2) When filtering by this column, we get full table scans for the same reason (also applies to :ref:`computed_cols`) . The performance loss here can be avoided with a computed index, or using a materialized generated column.
3) If the formatted column is used as a foreign key, PostgREST can no longer detect that relationship and :ref:`resource_embedding` breaks. This can be worked around with :ref:`computed_relationships`.
Domain Representations avoid all the above drawbacks. Their only drawback is that for existing tables, you have to change the column types. But this should be a fast operation since domains are binary coercible with their underlying types. A table rewrite won't be required.
.. note::
Why not create a `base type <https://www.postgresql.org/docs/current/sql-createtype.html#id-1.9.3.94.5.8>`_ instead? ``CREATE TYPE app_uuid (INTERNALLENGTH = 22, INPUT = app_uuid_parser, OUTPUT = app_uuid_formatter)``.
Creating base types need superuser, which is restricted on cloud hosted databases. Additionally this way lets “how the data is presented” dictate “how the data is stored” which would be backwards.
+344
View File
@@ -0,0 +1,344 @@
.. _functions:
Functions as RPC
================
*"A single resource can be the equivalent of a database function, with the power to abstract state changes over any number of storage items"* -- `Roy T. Fielding <https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven#comment-743>`_
Functions can perform any operation allowed by PostgreSQL (read data, modify data, :ref:`raise errors <raise_error>`, and even DDL operations). Every function in the :ref:`exposed schema <schemas>` and accessible by the :ref:`active database role <roles>` is executable under the :code:`/rpc` prefix.
If they return table types, functions can:
- Use all the same :ref:`read filters as Tables and Views <read>` (horizontal/vertical filtering, counts, limits, etc.).
- Use :ref:`Resource Embedding <function_embed>`, if the returned table type has relationships to other tables.
.. note::
Why the ``/rpc`` prefix? PostgreSQL allows a table or view to have the same name as a function. The prefix allows us to avoid routes collisions.
.. warning::
`Stored Procedures <https://www.postgresql.org/docs/current/xproc.html>`_ are not supported.
Calling with POST
-----------------
To supply arguments in an API call, include a JSON object in the request payload. Each key/value of the object will become an argument.
For instance, assume we have created this function in the database.
.. code-block:: postgres
CREATE FUNCTION add_them(a integer, b integer)
RETURNS integer AS $$
SELECT a + b;
$$ LANGUAGE SQL IMMUTABLE;
.. important::
Whenever you create or change a function you must refresh PostgREST's schema cache. See the section :ref:`schema_reloading`.
The client can call it by posting an object like
.. code-block:: bash
curl "http://localhost:3000/rpc/add_them" \
-X POST -H "Content-Type: application/json" \
-d '{ "a": 1, "b": 2 }'
.. code-block:: json
3
.. note::
PostgreSQL converts identifier names to lowercase unless you quote them like:
.. code-block:: postgres
CREATE FUNCTION "someFunc"("someParam" text) ...
Calling with GET
----------------
If the function doesn't modify the database, it will also run under the GET method (see :ref:`access_mode`).
.. code-block:: bash
curl "http://localhost:3000/rpc/add_them?a=1&b=2"
The function parameter names match the JSON object keys in the POST case, for the GET case they match the query parameters ``?a=1&b=2``.
.. _function_single_json:
Functions with a single unnamed JSON parameter
----------------------------------------------
If you want the JSON request body to be sent as a single argument, you can create a function with a single unnamed ``json`` or ``jsonb`` parameter.
For this the ``Content-Type: application/json`` header must be included in the request.
.. code-block:: postgres
CREATE FUNCTION mult_them(json) RETURNS int AS $$
SELECT ($1->>'x')::int * ($1->>'y')::int
$$ LANGUAGE SQL;
.. code-block:: bash
curl "http://localhost:3000/rpc/mult_them" \
-X POST -H "Content-Type: application/json" \
-d '{ "x": 4, "y": 2 }'
.. code-block:: json
8
.. note::
If an overloaded function has a single ``json`` or ``jsonb`` unnamed parameter, PostgREST will call this function as a fallback provided that no other overloaded function is found with the parameters sent in the POST request.
.. warning::
Sending the JSON request body as a single argument is also possible with :ref:`Prefer: params=single-object <prefer_params>` but this method is **deprecated**.
.. _function_single_unnamed:
Functions with a single unnamed parameter
-----------------------------------------
You can make a POST request to a function with a single unnamed parameter to send raw ``bytea``, ``text`` or ``xml`` data.
To send raw XML, the parameter type must be ``xml`` and the header ``Content-Type: text/xml`` must be included in the request.
To send raw binary, the parameter type must be ``bytea`` and the header ``Content-Type: application/octet-stream`` must be included in the request.
.. code-block:: postgres
CREATE TABLE files(blob bytea);
CREATE FUNCTION upload_binary(bytea) RETURNS void AS $$
INSERT INTO files(blob) VALUES ($1);
$$ LANGUAGE SQL;
.. code-block:: bash
curl "http://localhost:3000/rpc/upload_binary" \
-X POST -H "Content-Type: application/octet-stream" \
--data-binary "@file_name.ext"
.. code-block:: http
HTTP/1.1 200 OK
[ ... ]
To send raw text, the parameter type must be ``text`` and the header ``Content-Type: text/plain`` must be included in the request.
.. _functions_array:
Functions with array parameters
-------------------------------
You can call a function that takes an array parameter:
.. code-block:: postgres
create function plus_one(arr int[]) returns int[] as $$
SELECT array_agg(n + 1) FROM unnest($1) AS n;
$$ language sql;
.. code-block:: bash
curl "http://localhost:3000/rpc/plus_one" \
-X POST -H "Content-Type: application/json" \
-d '{"arr": [1,2,3,4]}'
.. code-block:: json
[2,3,4,5]
For calling the function with GET, you can pass the array as an `array literal <https://www.postgresql.org/docs/current/arrays.html#ARRAYS-INPUT>`_,
as in ``{1,2,3,4}``. Note that the curly brackets have to be urlencoded(``{`` is ``%7B`` and ``}`` is ``%7D``).
.. code-block:: bash
curl "http://localhost:3000/rpc/plus_one?arr=%7B1,2,3,4%7D'"
.. note::
For versions prior to PostgreSQL 10, to pass a PostgreSQL native array on a POST payload, you need to quote it and use an array literal:
.. code-block:: bash
curl "http://localhost:3000/rpc/plus_one" \
-X POST -H "Content-Type: application/json" \
-d '{ "arr": "{1,2,3,4}" }'
In these versions we recommend using function parameters of type JSON to accept arrays from the client.
.. _functions_variadic:
Variadic functions
------------------
You can call a variadic function by passing a JSON array in a POST request:
.. code-block:: postgres
create function plus_one(variadic v int[]) returns int[] as $$
SELECT array_agg(n + 1) FROM unnest($1) AS n;
$$ language sql;
.. code-block:: bash
curl "http://localhost:3000/rpc/plus_one" \
-X POST -H "Content-Type: application/json" \
-d '{"v": [1,2,3,4]}'
.. code-block:: json
[2,3,4,5]
In a GET request, you can repeat the same parameter name:
.. code-block:: bash
curl "http://localhost:3000/rpc/plus_one?v=1&v=2&v=3&v=4"
Repeating also works in POST requests with ``Content-Type: application/x-www-form-urlencoded``:
.. code-block:: bash
curl "http://localhost:3000/rpc/plus_one" \
-X POST -H "Content-Type: application/x-www-form-urlencoded" \
-d 'v=1&v=2&v=3&v=4'
.. _table_functions:
Table-Valued Functions
----------------------
A function that returns a table type can be filtered using the same filters as :ref:`tables and views <tables_views>`. They can also use :ref:`Resource Embedding <function_embed>`.
.. code-block:: postgres
CREATE FUNCTION best_films_2017() RETURNS SETOF films ..
.. code-block:: bash
curl "http://localhost:3000/rpc/best_films_2017?select=title,director:directors(*)"
.. code-block:: bash
curl "http://localhost:3000/rpc/best_films_2017?rating=gt.8&order=title.desc"
.. _function_inlining:
Function Inlining
~~~~~~~~~~~~~~~~~
A function that follows the `rules for inlining <https://wiki.postgresql.org/wiki/Inlining_of_SQL_functions#Inlining_conditions_for_table_functions>`_ will also inline :ref:`filters <h_filter>`, :ref:`order <ordering>` and :ref:`limits <limits>`.
For example, for the following function:
.. code-block:: postgres
create function getallprojects() returns setof projects
language sql stable
as $$
select * from projects;
$$;
Let's get its :ref:`explain_plan` when calling it with filters applied:
.. code-block:: bash
curl "http://localhost:3000/rpc/getallprojects?id=eq.1" \
-H "Accept: application/vnd.pgrst.plan"
.. code-block:: postgres
Aggregate (cost=8.18..8.20 rows=1 width=112)
-> Index Scan using projects_pkey on projects (cost=0.15..8.17 rows=1 width=40)
Index Cond: (id = 1)
Notice there's no "Function Scan" node in the plan, which tells us it has been inlined.
.. _scalar_functions:
Scalar functions
----------------
PostgREST will detect if the function is scalar or table-valued and will shape the response format accordingly:
.. code-block:: bash
curl "http://localhost:3000/rpc/add_them?a=1&b=2"
.. code-block:: json
3
.. code-block:: bash
curl "http://localhost:3000/rpc/best_films_2017"
.. code-block:: json
[
{ "title": "Okja", "rating": 7.4},
{ "title": "Call me by your name", "rating": 8},
{ "title": "Blade Runner 2049", "rating": 8.1}
]
To manually choose a return format such as binary, see :ref:`custom_media`.
.. _untyped_functions:
Untyped functions
-----------------
Functions that return ``record`` or ``SETOF record`` are supported:
.. code-block:: postgres
create function projects_setof_record() returns setof record as $$
select * from projects;
$$ language sql;
.. code-block:: bash
curl "http://localhost:3000/rpc/projects_setof_record"
.. code-block:: json
[{"id":1,"name":"Windows 7","client_id":1},
{"id":2,"name":"Windows 10","client_id":1},
{"id":3,"name":"IOS","client_id":2}]
However note that they will fail when trying to use :ref:`v_filter` and :ref:`h_filter` on them.
So while they can be used for quick tests, it's recommended to always choose a strict return type for the function.
Overloaded functions
--------------------
You can call overloaded functions with different number of arguments.
.. code-block:: postgres
CREATE FUNCTION rental_duration(customer_id integer) ..
CREATE FUNCTION rental_duration(customer_id integer, from_date date) ..
.. code-block:: bash
curl "http://localhost:3000/rpc/rental_duration?customer_id=232"
.. code-block:: bash
curl "http://localhost:3000/rpc/rental_duration?customer_id=232&from_date=2018-07-01"
.. important::
Overloaded functions with the same argument names but different types are not supported.
+318
View File
@@ -0,0 +1,318 @@
.. _custom_media:
Media Type Handlers
###################
Media Type Handlers allow PostgREST to deliver custom media types. These handlers extend the :ref:`builtin ones <builtin_media>` and can also override them.
Media types are expressed as type aliases using `domains <https://www.postgresql.org/docs/current/sql-createdomain.html>`_ and their name must comply to `RFC 6838 requirements <https://datatracker.ietf.org/doc/html/rfc6838#section-4.2>`_.
.. code-block:: postgres
CREATE DOMAIN "application/json" AS json;
Using these domains, :ref:`functions <functions>` can become handlers and `user-defined aggregates <https://www.postgresql.org/docs/current/xaggr.html>`_ can serve as handlers for :ref:`tables_views` and :ref:`table_functions`.
.. important::
- PostgREST vendor media types (``application/vnd.pgrst.plan``, ``application/vnd.pgrst.object`` and ``application/vnd.pgrst.array``) cannot be overriden.
- Long media types like ``application/vnd.openxmlformats-officedocument.wordprocessingml.document`` cannot be expressed as domains since they surpass `PostgreSQL identifier length <https://www.postgresql.org/docs/current/limits.html#LIMITS-TABLE>`_.
For these you can use the :ref:`any_handler`.
Handler Function
================
As an example, let's obtain the `TWKB <https://postgis.net/docs/ST_AsTWKB.html>`_ compressed binary format for a PostGIS geometry.
.. code-block:: postgres
create extension postgis;
create table lines (
id int primary key
, name text
, geom geometry(LINESTRING, 4326)
);
insert into lines values (1, 'line-1', 'LINESTRING(1 1,5 5)'::geometry), (2, 'line-2', 'LINESTRING(2 2,6 6)'::geometry);
For this you can create a vendor media type.
.. code-block:: postgres
create domain "application/vnd.twkb" as bytea;
And use it as a return type on a function, to make it a handler.
.. code-block:: postgres
create or replace function get_line (id int)
returns "application/vnd.twkb" as $$
select st_astwkb(geom) from lines where id = get_line.id;
$$ language sql;
.. note::
For PostgreSQL <= 12, you'll need a cast on the function body :code:`st_astwkb(geom)::"application/vnd.twkb"`.
Now you can request the ``TWKB`` output like so:
.. code-block:: bash
curl 'localhost:3000/rpc/get_line?id=1' -i \
-H "Accept: application/vnd.twkb"
HTTP/1.1 200 OK
Content-Type: application/vnd.twkb
# binary output
Note that PostgREST will automatically set the ``Content-Type`` to ``application/vnd.twkb``.
Handlers for Tables/Views
=========================
To benefit from a compressed format like ``TWKB``, it makes more sense to obtain many rows instead of one. Let's allow that by adding a handler for the table.
User-defined aggregates can be turned into handlers by using domain media types as the return type of their transition or final functions.
Let's create a transition function for this example.
.. code-block:: postgres
create or replace function twkb_handler_transition (state bytea, next lines)
returns "application/vnd.twkb" as $$
select state || st_astwkb(next.geom);
$$ language sql;
Now we'll use it on a new aggregate defined for the ``lines`` table.
.. code-block:: postgres
create or replace aggregate twkb_agg (lines) (
initcond = ''
, stype = "application/vnd.twkb"
, sfunc = twkb_handler_transition
);
.. note::
You can test see this aggregate working with:
.. code-block:: psql
SELECT twkb_agg(l) from lines l;
twkb_agg
---------------------------------------------------------------
\xa20002c09a0cc09a0c80ea3080ea30a2000280b51880b51880ea3080ea30
(1 row)
Now you can request the table endpoint with the ``twkb`` media type:
.. code-block:: bash
curl 'localhost:3000/lines' -i \
-H "Accept: application/vnd.twkb"
HTTP/1.1 200 OK
Content-Type: application/vnd.twkb
# binary output
If you have a table-valued function returning the same table type, the handler can also act upon on it.
.. code-block:: postgres
create or replace function get_lines ()
returns setof lines as $$
select * from lines;
$$ language sql;
.. code-block:: bash
curl 'localhost:3000/get_lines' -i \
-H "Accept: application/vnd.twkb"
HTTP/1.1 200 OK
Content-Type: application/vnd.twkb
# binary output
Overriding a Builtin Handler
============================
Let's override the existing ``text/csv`` handler for the table to provide a more complex CSV output.
It'll include a `Byte order mark (BOM) <https://en.wikipedia.org/wiki/Byte_order_mark>`_ plus a ``Content-Disposition`` header to set a name for the downloaded file.
Create a domain for the standard ``text/csv`` media type.
.. code-block:: postgres
create domain "text/csv" as text;
And a transition function that returns the domain.
.. code-block:: postgres
create or replace function bom_csv_trans (state text, next lines)
returns "text/csv" as $$
select state || next.id::text || ',' || next.name || ',' || next.geom::text || E'\n';
$$ language sql;
This time we'll add a final function. This will add the CSV header, the BOM and the ``Content-Disposition`` header.
.. code-block:: postgres
create or replace function bom_csv_final (data "text/csv")
returns "text/csv" as $$
-- set the Content-Disposition header
select set_config('response.headers', '[{"Content-Disposition": "attachment; filename=\"lines.csv\""}]', true);
select
-- EFBBBF is the BOM in UTF8 https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8
convert_from (decode (E'EFBBBF', 'hex'),'UTF8') ||
-- the header for the CSV
(E'id,name,geom\n' || data);
$$ language sql;
Now use the transition and final function as part of the new aggregate.
.. code-block:: postgres
create or replace aggregate bom_csv_agg (lines) (
initcond = ''
, stype = "text/csv"
, sfunc = bom_csv_trans
, finalfunc = bom_csv_final
);
.. note::
You can test this with:
.. code-block:: psql
select bom_csv_agg(l) from lines l;
bom_csv_agg
-----------------------------------------------------------------------------------------------------
id,name,geom +
1,line-1,0102000020E610000002000000000000000000F03F000000000000F03F00000000000014400000000000001440+
2,line-2,0102000020E6100000020000000000000000000040000000000000004000000000000018400000000000001840+
(1 row)
And request it like:
.. code-block:: bash
curl 'localhost:3000/lines' -i \
-H "Accept: text/csv"
HTTP/1.1 200 OK
Content-Type: text/csv
Content-Disposition: attachment; filename="lines.csv"
id,name,geom
1,line-1,0102000020E610000002000000000000000000F03F000000000000F03F00000000000014400000000000001440
2,line-2,0102000020E6100000020000000000000000000040000000000000004000000000000018400000000000001840
.. _any_handler:
The "Any" Handler
=================
For more flexibility, you can also define a catch-all handler by using a domain named ``*/*`` (any media type). This handler obeys the following rules:
- It responds to all media types and even to requests that don't include an ``Accept`` header.
- It sets the ``Content-Type`` header to ``application/octet-stream`` by default, but this can be overridden inside the function with :ref:`guc_resp_hdrs`.
- It overrides all other handlers (:ref:`builtin <builtin_media>` or custom), so it's better to do it for an isolated function or view.
Let's define an any handler for a view that will always respond with ``XML`` output. It will accept ``text/xml``, ``application/xml``, ``*/*`` and reject other media types.
.. code-block:: postgres
create domain "*/*" as bytea;
-- we'll use an .xml suffix for the view to be clear its output is always XML
create view "lines.xml" as
select * from lines;
-- transition function
create or replace function lines_xml_trans (state "*/*", next "lines.xml")
returns "*/*" as $$
select state || xmlelement(name line, xmlattributes(next.id as id, next.name as name), next.geom)::text::bytea || E'\n' ;
$$ language sql;
-- final function
create or replace function lines_xml_final (data "*/*")
returns "*/*" as $$
declare
-- get the Accept header
req_accept text := current_setting('request.headers', true)::json->>'accept';
begin
-- when we need to override the default Content-Type (application/octet-stream) set by PostgREST
if req_accept = '*/*' then
perform set_config('response.headers', json_build_array(json_build_object('Content-Type', 'text/xml'))::text, true);
elsif req_accept IN ('application/xml', 'text/xml') then
perform set_config('response.headers', json_build_array(json_build_object('Content-Type', req_accept))::text, true);
else
-- we'll reject other non XML media types, we need to reject manually since */* will command PostgREST to accept all media types
raise sqlstate 'PT415' using message = 'Unsupported Media Type';
end if;
return data;
end; $$ language plpgsql;
-- new aggregate
create or replace aggregate lines_xml_agg ("lines.xml") (
stype = "*/*"
, sfunc = lines_xml_trans
, finalfunc = lines_xml_final
);
Test it on SQL:
.. code-block:: psql
select (encode(lines_xml_agg(x), 'escape'))::xml from "lines.xml" x;
encode
------------------------------------------------------------------------------------------------------------------------------
<line id="1" name="line-1">0102000020E610000002000000000000000000F03F000000000000F03F00000000000014400000000000001440</line>+
<line id="2" name="line-2">0102000020E6100000020000000000000000000040000000000000004000000000000018400000000000001840</line>+
Now we can omit the ``Accept`` header and it will respond with XML.
.. code-block:: bash
curl 'localhost:3000/lines.xml' -i
HTTP/1.1 200 OK
Content-Type: text/xml
<line id="1" name="line-1">0102000020E610000002000000000000000000F03F000000000000F03F00000000000014400000000000001440</line>
<line id="2" name="line-2">0102000020E6100000020000000000000000000040000000000000004000000000000018400000000000001840</line>
And it will accept only XML media types.
.. code-block:: bash
curl 'localhost:3000/lines.xml' -i \
-H "Accept: text/xml"
HTTP/1.1 200 OK
Content-Type: text/xml
.. code-block:: bash
curl 'localhost:3000/lines.xml' -i \
-H "Accept: application/xml"
HTTP/1.1 200 OK
Content-Type: text/xml
.. code-block:: bash
curl 'localhost:3000/lines.xml' -i \
-H "Accept: unknown/media"
HTTP/1.1 415 Unsupported Media Type
+98
View File
@@ -0,0 +1,98 @@
.. _open-api:
OpenAPI
=======
PostgREST automatically serves a full `OpenAPI <https://www.openapis.org/>`_ description on the root path. This provides a list of all endpoints (tables, foreign tables, views, functions), along with supported HTTP verbs and example payloads.
.. note::
By default, this output depends on the permissions of the role that is contained in the JWT role claim (or the :ref:`db-anon-role` if no JWT is sent). If you need to show all the endpoints disregarding the role's permissions, set the :ref:`openapi-mode` config to :code:`ignore-privileges`.
For extra customization, the OpenAPI output contains a "description" field for every `SQL comment <https://www.postgresql.org/docs/current/sql-comment.html>`_ on any database object. For instance,
.. code-block:: postgres
COMMENT ON SCHEMA mammals IS
'A warm-blooded vertebrate animal of a class that is distinguished by the secretion of milk by females for the nourishment of the young';
COMMENT ON TABLE monotremes IS
'Freakish mammals lay the best eggs for breakfast';
COMMENT ON COLUMN monotremes.has_venomous_claw IS
'Sometimes breakfast is not worth it';
These unsavory comments will appear in the generated JSON as the fields, ``info.description``, ``definitions.monotremes.description`` and ``definitions.monotremes.properties.has_venomous_claw.description``.
Also if you wish to generate a ``summary`` field you can do it by having a multiple line comment, the ``summary`` will be the first line and the ``description`` the lines that follow it:
.. code-block:: postgres
COMMENT ON TABLE entities IS
$$Entities summary
Entities description that
spans
multiple lines$$;
Similarly, you can override the API title by commenting the schema.
.. code-block:: postgres
COMMENT ON SCHEMA api IS
$$FooBar API
A RESTful API that serves FooBar data.$$;
If you need to include the ``security`` and ``securityDefinitions`` options, set the :ref:`openapi-security-active` configuration to ``true``.
You can use a tool like `Swagger UI <https://swagger.io/tools/swagger-ui/>`_ to create beautiful documentation from the description and to host an interactive web-based dashboard. The dashboard allows developers to make requests against a live PostgREST server, and provides guidance with request headers and example request bodies.
.. important::
The OpenAPI information can go out of date as the schema changes under a running server. See :ref:`schema_reloading`.
.. _override_openapi:
Overriding Full OpenAPI Response
--------------------------------
You can override the whole default response with a function result. To do this, set the function on :ref:`db-root-spec`.
.. code:: bash
db-root-spec = "root"
.. code:: postgres
create or replace function root() returns json as $_$
declare
openapi json = $$
{
"swagger": "2.0",
"info":{
"title":"Overridden",
"description":"This is a my own API"
}
}
$$;
begin
return openapi;
end
$_$ language plpgsql;
.. code-block:: bash
curl http://localhost:3000
.. code-block:: http
HTTP/1.1 200 OK
{
"swagger": "2.0",
"info":{
"title":"Overridden",
"description":"This is a my own API"
}
}
+46
View File
@@ -0,0 +1,46 @@
.. _options_requests:
OPTIONS method
==============
You can verify which HTTP methods are allowed on endpoints for tables and views by using an OPTIONS request. These methods are allowed depending on what operations *can* be done on the table or view, not on the database permissions assigned to them.
For a table named ``people``, OPTIONS would show:
.. code-block:: bash
curl "http://localhost:3000/people" -X OPTIONS -i
.. code-block:: http
HTTP/1.1 200 OK
Allow: OPTIONS,GET,HEAD,POST,PUT,PATCH,DELETE
For a view, the methods are determined by the presence of INSTEAD OF TRIGGERS.
.. table::
:widths: auto
+--------------------+-------------------------------------------------------------------------------------------------+
| Method allowed | View's requirements |
+====================+=================================================================================================+
| OPTIONS, GET, HEAD | None (Always allowed) |
+--------------------+-------------------------------------------------------------------------------------------------+
| POST | INSTEAD OF INSERT TRIGGER |
+--------------------+-------------------------------------------------------------------------------------------------+
| PUT | INSTEAD OF INSERT TRIGGER, INSTEAD OF UPDATE TRIGGER, also requires the presence of a |
| | primary key |
+--------------------+-------------------------------------------------------------------------------------------------+
| PATCH | INSTEAD OF UPDATE TRIGGER |
+--------------------+-------------------------------------------------------------------------------------------------+
| DELETE | INSTEAD OF DELETE TRIGGER |
+--------------------+-------------------------------------------------------------------------------------------------+
| All the above methods are allowed for |
| `auto-updatable views <https://www.postgresql.org/docs/current/sql-createview.html#SQL-CREATEVIEW-UPDATABLE-VIEWS>`_ |
+--------------------+-------------------------------------------------------------------------------------------------+
For functions, the methods depend on their volatility. ``VOLATILE`` functions allow only ``OPTIONS,POST``, whereas the rest also permit ``GET,HEAD``.
.. important::
Whenever you add or remove tables or views, or modify a view's INSTEAD OF TRIGGERS on the database, you must refresh PostgREST's schema cache for OPTIONS requests to work properly. See the section :ref:`schema_reloading`.

Some files were not shown because too many files have changed in this diff Show More