Compare commits

..
38 Commits
Author SHA1 Message Date
Laurence Isla 519615d5fb bump version to 12.2.3 2024-08-01 11:46:52 -05:00
Laurence Isla de5a8954b2 changelog: add missing entry for 3670 2024-08-01 11:18:15 -05:00
64b5a59410 fix: schema cache loading before the in-db config (#3670)
Fixes #3660. Load the config after getting the pg version but before loading the schema.

The regression happened on f09655b.

Also remove schema cache load wrapper and separate db queries in different functions.

Co-authored-by: Laurence Isla <lau.isla.c@gmail.com>
2024-08-01 17:41:07 +02:00
Dan KurinandWolfgang Walther 417a96d137 fix: Remove OpenAPI format for rowFilter params (#3661) 2024-08-01 17:41:07 +02:00
Wolfgang Walther 53593824a8 fix: Embed One-to-One relationship with different column order properly 2024-07-13 22:27:04 +02:00
Wolfgang Walther e642086d27 chore: Fix changelog after 13868983 2024-07-13 17:16:06 +02:00
steve-chavezandWolfgang Walther e451700d5d chore: remove links to gitter
We'll now use github discussions for support.
2024-07-13 17:11:06 +02:00
Salim BandWolfgang Walther 13868983af fix: update OpenAPI externalDocs URL
fixes https://github.com/PostgREST/postgrest/issues/3091
2024-07-13 17:11:06 +02:00
steve-chavez db9da0b1ca bump version to 12.2.2 2024-07-10 23:07:23 -05:00
steve-chavez f09655b7a6 fix: schema cache retrying without backoff
Fixes https://github.com/PostgREST/postgrest/issues/3523.

Now if there's a failure when obtaining the pg version OR schema cache,
we do the same retrying process. This way we don't add two retries.

Refactors and renames the "connectionWorker" to "schemaCacheLoader".
This makes more sense since what we really want is the schema cache,
the version is the pre-requisite for ensuring our
schema cache queries work.

Additionally, we no longer log ` Attempting to connect to the database...`
at startup unnecessarily. This is only logged whenever there's a retry attempt.
2024-07-10 21:11:20 -05:00
steve-chavezandWolfgang Walther 8715e426c0 chore: remove paypal links
It was tied to a personal account and donations there have been too rare.
2024-07-10 21:13:23 +02:00
Wolfgang WaltherandWolfgang Walther 9a40bc266e fix: List correct enum options when multiple types with same name are present
The schema cache and OpenAPI output would currently list the first found
enum with the same name instead of the correct type. One other case
where this comes up is when a regular type and an enum type have the
same name. For example in the spec fixtures, we have an enum called
"bit". Every "bit" type, no matter whether it's that enum or the
built-in bit type, will show those enum options in the OpenApi output.

Not adding a test, because OpenAPI is supposed to go away in the future
anyway.
2024-07-09 09:09:22 +02:00
Wolfgang WaltherandWolfgang Walther e67461b991 fix: Show number of loaded timezones in log output
There is no reason to hide those, right?
2024-07-09 09:09:22 +02:00
Wolfgang WaltherandWolfgang Walther 420933ceba fix: Make --dump-schema work with in-database pgrst.db_schemas setting
This needs to be loaded from in-database configuration first, otherwise
the dump-schema output will be for the default (public) schema.
2024-07-09 09:09:22 +02:00
Laurence IslaandWolfgang Walther 8f8dbd3425 fix: nested empty embeds no longer return empty values and are correctly omitted 2024-07-09 09:09:22 +02:00
renovate[bot]andWolfgang Walther 8d7d7e4112 chore(deps): update all dependencies 2024-07-06 11:35:11 +02:00
Sandro BauerandWolfgang Walther 521acf61cb docs: fix rendering for inline code block in operator list 2024-07-02 12:28:24 +02:00
steve-chavez d3f7cba92e bump version to 12.2.1 2024-06-27 14:59:35 -05:00
Laurence IslaandSteve Chavez a71d9ea50f changelog: add missing entries for #3592 and #3616 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 5a1e9b01e4 nix: remove texlive dependencies from postgrest-docs-render 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez d3292d42de docs: use PlantUML instead of Latex to generate Schema Isolation image 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 572581cbb6 chore: organize diagrams in different folders 2024-06-27 14:08:16 -05:00
steve-chavez 9daddb549c docs: add listener recovery 2024-06-27 14:08:16 -05:00
steve-chavez 716fcbf147 fix: don't reload cache on every listener fail
Revert "prevent GSSAPI error between Listener and pool"

This reverts commit 4beac10d3d.
2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 5b62501ce7 docs: improve architecture diagram
- SVG format instead of PNG
- The components now have links to their reference in the Docs
- Supports dark mode
2024-06-27 14:08:16 -05:00
Wolfgang WaltherandSteve Chavez 7860e7287f docs: Fix punctuation in install.rst 2024-06-27 14:08:16 -05:00
Michal KleczekandSteve Chavez c958f94753 docs: add pg-notify-stdout to ecosystem 2024-06-27 14:08:16 -05:00
renovate[bot]andWolfgang Walther 94f0548344 chore(deps): update ubuntu:noble docker digest to 2e863c4 2024-06-19 19:22:47 +02:00
Wolfgang Walther 16d0e1ca8c fix: Treat pre-release and docs versions correctly for new release workflow
Since we changed our release workflow, we have adjusted:
- the docs to use postgrest.org/en/v12/ -style URLs, i.e. only using the
major component.
- the pre-release / devel versions to contain only two instead of four
version parts, i.e. currently 12.3.
2024-06-19 08:42:34 +02:00
Wolfgang Walther 9657d46c42 docs: Fix linkcheck
Some URLs are still forbidden for our linkcheck tool, so disabling them
again.

Others are permanently redirected, so adjusting them.
2024-06-19 08:40:58 +02:00
renovate[bot]andWolfgang Walther 70e07ccbc6 chore(deps): update dependency urllib3 to v2.2.2 2024-06-17 21:04:03 +02:00
renovate[bot]andWolfgang Walther cc5c4ae5ef chore(deps): update codecov/codecov-action action to v4.5.0 2024-06-13 19:51:15 +02:00
renovate[bot]andWolfgang Walther af8b62ba10 chore(deps): update all dependencies 2024-06-13 08:46:23 +02:00
Laurence Isla d4e7d23268 docs: fix example of listener failure on read replicas 2024-06-12 19:46:23 -05:00
Laurence Isla 4e084b664c docs: clarify what is logged when "log-level=debug" 2024-06-12 18:30:19 -05:00
Laurence Isla 91024b6eab docs: add missing logs to stderr
- Schema cache stats are now logged to stderr
- Log when the LISTEN channel gets a notification
2024-06-12 18:30:19 -05:00
Laurence IslaandSteve Chavez 0e4fcd7387 docs: add "Listener" page
Co-authored-by: Steve Chavez <stevechavezast@gmail.com>
2024-06-12 18:30:19 -05:00
Wolfgang Walther 49a7aa9933 bump docs version to 12.2 2024-06-12 07:47:17 +02:00
264 changed files with 7780 additions and 13777 deletions
+42
View File
@@ -0,0 +1,42 @@
freebsd_instance:
image_family: freebsd-14-0
build_task:
# Don't change this name without adjusting .github/workflows/build.yaml
name: Build FreeBSD (Stack)
install_script: pkg install -y postgresql16-client hs-stack git
only_if: |
$CIRRUS_TAG != '' || $CIRRUS_BRANCH == 'main' || $CIRRUS_BRANCH =~ 'v*' ||
changesInclude(
'.github/workflows/build.yaml',
'.github/actions/artifact-from-cirrus/**',
'.cirrus.yml',
'postgrest.cabal',
'stack.yaml*',
'**.hs'
)
stack_cache:
folders: /.stack
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
stack_work_cache:
folders: .stack-work
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
- find main src -type f -iname '*.hs' -exec md5sum "{}" +
build_script: |
stack build -j 1 --local-bin-path . --copy-bins --stack-yaml stack-21.7.yaml
strip postgrest
bin_artifacts:
path: postgrest
-9
View File
@@ -1,9 +0,0 @@
root = true
[*]
charset = utf-8
end_of_line = lf
indent_size = 2
indent_style = space
insert_final_newline = true
trim_trailing_whitespace = true
+55
View File
@@ -0,0 +1,55 @@
# Contributing to PostgREST
**First:** if you're unsure or afraid of _anything_, just ask or
submit the issue or pull request anyways. You won't be yelled at
for giving your best effort. The worst that can happen is that
you'll be politely asked to change something. We appreciate any
sort of contributions, and don't want a wall of rules to get in the
way of that.
However, for those individuals who want a bit more guidance on the
best way to contribute to the project, read on. This document will
cover what we're looking for. By addressing all the points we're
looking for, it raises the chances we can quickly merge or address
your contributions.
## Issues
For questions on how to use PostgREST, please use
[GitHub discussions](https://github.com/PostgREST/postgrest/discussions).
### Reporting an Issue
* Make sure you test against the latest [stable release](https://github.com/PostgREST/postgrest/releases/latest)
and also against the latest [nightly release](https://github.com/PostgREST/postgrest/releases/tag/nightly).
It is possible we already fixed the bug you're experiencing.
* Provide steps to reproduce the issue, including your OS version and
the specific database schema that you are using.
* Please include SQL logs for issues involving runtime problems. To obtain logs first
[enable logging all statements](http://www.microhowto.info/howto/log_all_queries_to_a_postgresql_server.html),
then [find your logs](http://blog.endpoint.com/2014/11/dear-postgresql-where-are-my-logs.html).
* If your database schema has changed while the PostgREST server is running,
[send the server a `SIGUSR1` signal](http://postgrest.org/en/latest/admin.html#schema-reloading) or restart it to ensure the schema cache
is not stale. This sometimes fixes apparent bugs.
## Code
We have a fully nix-based development environment with many tools for a smooth development workflow available.
Check the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md) on how to set it up and use it.
### Haskell Conventions
* All contributions must pass the tests before being merged. When
you create a pull request your code will automatically be tested.
* All code must also pass [hlint](http://community.haskell.org/~ndm/hlint/) and [stylish-haskell](https://github.com/jaspervdj/stylish-haskell)
with no warnings. This helps enforce a uniform style for all committers. Continuous integration will check this as well on every
pull request. There are useful tools in the nix-shell that help with checking this locally. You can run `postgrest-check` to do this manually but
we recommend adding it to `.git/hooks/pre-commit` as `nix-shell --run postgrest-check` to automatically check this before doing a commit.
### Running Tests
For instructions on running tests, see the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md#testing).
+17
View File
@@ -0,0 +1,17 @@
<!--
Before reporting a bug:
If your database schema has changed while the PostgREST server is running,
send the server a SIGUSR1 signal or restart it(http://postgrest.org/en/stable/admin.html#schema-reloading)
to ensure the schema cache is not stale. This sometimes fixes apparent bugs.
-->
### Environment
* PostgreSQL version: (if using docker, specify the image)
* PostgREST version: (if using docker, specify the image)
* Operating system:
### Description of issue
(Expected behavior vs actual behavior)
(Steps to reproduce: Include a minimal SQL definition plus how you make the request to PostgREST and the response body)
-28
View File
@@ -1,28 +0,0 @@
---
name: Bug report
about: Create a bug report to help us improve
type: Bug
title: ''
labels: ''
assignees: ''
---
<!--
Before reporting a bug:
If your database schema has changed while the PostgREST server is running,
send the server a SIGUSR1 signal or restart it (http://postgrest.org/en/stable/admin.html#schema-reloading) to ensure the schema cache is not stale. This sometimes fixes apparent bugs.
-->
### Environment
* PostgreSQL version: (if using docker, specify the image)
* PostgREST version: (if using docker, specify the image)
* Operating system:
### Description of issue
Describe the behavior you expected vs the actual behavior. Include:
- A minimal SQL definition.
- How you make the request to PostgREST (curl command preferred).
- The PostgREST response.
-1
View File
@@ -1 +0,0 @@
blank_issues_enabled: false
-17
View File
@@ -1,17 +0,0 @@
---
name: Feature request
about: Suggest an enhancement for this project
type: Feature
title: ''
labels: ''
assignees: ''
---
## Problem
A clear and concise description of what the problem is.
## Solution
A clear and concise description of what you want to happen.
+11 -12
View File
@@ -4,18 +4,17 @@ When submitting a new feature or fix:
- Add a new entry to the CHANGELOG - https://github.com/PostgREST/postgrest/blob/main/CHANGELOG.md#unreleased - Add a new entry to the CHANGELOG - https://github.com/PostgREST/postgrest/blob/main/CHANGELOG.md#unreleased
- If relevant, update the docs - If relevant, update the docs
- Use a prefix for the PR title or commits, e.g. "fix: description of the fix". - Use a prefix for the PR title or commits, e.g. "fix: description of the fix".
+ `add`, Add a new feature + `fix`, bug fixes
+ `amend`, To amend an unrealease commit + `feat`, new features added
+ `change`, Breaking changes + `perf`, performance improvements
+ `chore`, Maintenance, update sponsors, changelog, readme etc + `docs`, updating the documentation
+ `ci`, CI configuration files and scripts + `nix`, related to the Nix development environment
+ `docs`, Documentation + `ci`, related to the Continuous Integration modules
+ `fix`, Bug fix + `test`, related to the testing modules
+ `nix`, Related to Nix + `refactor`, refactoring code
+ `perf`, Performance improvements + `deprecate`, deprecating a feature
+ `refactor`, Refactoring code + `changelog`, updating the CHANGELOG
+ `remove`, Remove a feature or fix + `chore`, maintenance (build process, updating sponsors, etc.)
+ `test`, Adding tests
+ Other prefixes may be used if necessary + Other prefixes may be used if necessary
- If there's a breaking change, add `BREAKING CHANGE` and an explanation to your commit message - If there's a breaking change, add `BREAKING CHANGE` and an explanation to your commit message
--> -->
-8
View File
@@ -1,8 +0,0 @@
# TODO: Remove this once a new actionlint release has been cut
# and made its way to us through nixpkgs.
self-hosted-runner:
labels:
- macos-15-intel
- macos-26
- ubuntu-24.04-arm
- ubuntu-slim
@@ -0,0 +1,119 @@
name: Artifact from Cirrus
description: Waits for a specific Cirrus CI run to complete, then downloads the artifact and uploads it to the current workflow. This will silently succeed if Cirrus CI did not schedule a task within 2 minutes.
inputs:
download:
description: Name of Artifact to download from Cirrus CI
required: true
task:
description: Name of Cirrus Task
required: true
token:
description: GitHub Token
required: true
upload:
description: Name of Artifact to upload on GitHub Actions
required: true
runs:
using: composite
steps:
- shell: bash
run: echo "GH_TOKEN=${{ inputs.token }}" >> "$GITHUB_ENV"
- name: Wait for Check Suite to be created
id: check-suite
env:
# GITHUB_SHA does weird things for pull request, so we roll our own:
COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
shell: bash
run: |
get_check_runs_url() {
gh api "repos/{owner}/{repo}/commits/${COMMIT}/check-suites" \
| jq -r '.check_suites[] | select(.app.slug == "cirrus-ci") | .check_runs_url'
}
for _ in $(seq 1 12); do
check_runs_url="$(get_check_runs_url)"
if [ -z "$check_runs_url" ]; then
echo "Cirrus CI task has not started, yet. Waiting..."
sleep 10
else
echo "check_runs_url=$check_runs_url" >> "$GITHUB_OUTPUT"
exit 0
fi
done
>&2 echo "Cirrus CI check suite not found. Is Cirrus CI enabled for this repo?"
- name: Find task by name
id: find-task
if: steps.check-suite.outputs.check_runs_url
shell: bash
run: |
get_number_of_tasks() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -r '.check_runs | map(select(.name == "${{ inputs.task }}")) | length'
}
tasks="$(get_number_of_tasks)"
case "$tasks" in
0)
echo "Task not found, assuming it's skipped intentionally..."
exit 0
;;
1)
echo "task_found=1" >> "$GITHUB_OUTPUT"
exit 0
;;
*)
>&2 echo "More than 1 task with the same name found. Don't know what to do..."
exit 1
;;
esac
- name: Wait for Cirrus CI to complete task
if: steps.find-task.outputs.task_found
shell: bash
run: |
get_conclusion() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -r '.check_runs[] | select(.name == "${{ inputs.task }}" and .status == "completed") | .conclusion'
}
while true; do
conclusion="$(get_conclusion)"
if [ -z "$conclusion" ]; then
echo "Cirrus CI task has not completed, yet. Waiting..."
sleep 30
else
if [ "$conclusion" == "success" ]; then
break
else
exit 1
fi
fi
done
- name: Download artifact from Cirrus CI
if: steps.find-task.outputs.task_found
id: download
shell: bash
run: |
get_external_id() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -er '.check_runs[] | select(.name == "${{ inputs.task }}") | .external_id'
}
archive="$(mktemp)"
artifacts="$(mktemp -d)"
until curl --no-progress-meter --fail -o "${archive}" \
"https://api.cirrus-ci.com/v1/artifact/task/$(get_external_id)/${{ inputs.download }}.zip"
do
# This happens when a tag is pushed on the same commit. In this case the
# job is immediately marked as "completed" for us, so we end up here after a few
# seconds - but the actual Cirrus CI task is still running and didn't produce its artifact, yet.
echo "Artifact not found on Cirrus CI, yet. Waiting..."
sleep 30
done
unzip "${archive}" -d "${artifacts}"
echo "artifacts=${artifacts}" >> "$GITHUB_OUTPUT"
- name: Save artifact to GitHub Actions
if: steps.find-task.outputs.task_found
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ inputs.upload }}
path: ${{ steps.download.outputs.artifacts }}
if-no-files-found: error
+2 -2
View File
@@ -19,14 +19,14 @@ inputs:
runs: runs:
using: composite using: composite
steps: steps:
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 - uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }} if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with: with:
path: ${{ inputs.path }} path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }} key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: | restore-keys: |
${{ runner.os }}-${{ inputs.prefix }}- ${{ runner.os }}-${{ inputs.prefix }}-
- uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 - uses: actions/cache/restore@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }} if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with: with:
path: ${{ inputs.path }} path: ${{ inputs.path }}
-35
View File
@@ -1,35 +0,0 @@
name: Run anywhere
description: Runs the same code either in a VM or on the bare machine
inputs:
vm:
description: Which VM to run on.
envs:
description: List of relevant environment variables, which might need to be copied into the VM.
prepare:
description: Code to run in a prepare step, e.g. installing dependencies.
run:
description: Code to run as the main action.
required: true
runs:
using: composite
steps:
- if: ${{ inputs.vm == 'freebsd' }}
uses: vmactions/freebsd-vm@83b151f58c6047089f4c80eb5ba2039d158ce093 # v1.5.3
with:
envs: ${{ inputs.envs }}
prepare: ${{ inputs.prepare }}
# Work around https://github.com/vmactions/freebsd-vm/issues/59
run: |
pw user add -n action -m
su action -c '${{ inputs.run }}'
- if: ${{ inputs.vm == '' }}
name: Prepare
shell: ${{ runner.os == 'Windows' && 'pwsh' || 'bash' }}
run: ${{ inputs.prepare }}
- if: ${{ inputs.vm == '' }}
name: Run
shell: ${{ runner.os == 'Windows' && 'pwsh' || 'bash' }}
run: ${{ inputs.run }}
+2 -3
View File
@@ -11,16 +11,15 @@ inputs:
runs: runs:
using: composite using: composite
steps: steps:
- uses: nixbuild/nix-quick-install-action@9f63be77f412a248c9d9a65a4c82cf066cdf8f0c # v35 - uses: nixbuild/nix-quick-install-action@60e9c39264d4714139af3cdf15f691b19eec3530 # v28
with: with:
nix_conf: |- nix_conf: |-
always-allow-substitutes = true always-allow-substitutes = true
max-jobs = auto max-jobs = auto
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17 - uses: cachix/cachix-action@ad2ddac53f961de1989924296a1f236fcfbaa4fc # v15
with: with:
name: postgrest name: postgrest
authToken: ${{ inputs.authToken }} authToken: ${{ inputs.authToken }}
skipPush: ${{ inputs.authToken == '' }}
- if: ${{ inputs.tools }} - if: ${{ inputs.tools }}
run: nix-env -f default.nix -iA ${{ inputs.tools }} run: nix-env -f default.nix -iA ${{ inputs.tools }}
shell: bash shell: bash
+6 -50
View File
@@ -3,67 +3,23 @@
"extends": [ "extends": [
"config:best-practices" "config:best-practices"
], ],
"baseBranchPatterns": [ "baseBranches": [
"main", "main",
"/^v[0-9]+/" "/^v[0-9]+/"
], ],
"rebaseWhen": "conflicted", "rebaseWhen": "conflicted",
"pip_requirements": {
"enabled": false
},
"packageRules": [ "packageRules": [
{ {
"matchBaseBranches": [ "matchBaseBranches": [ "/^v[0-9]+/" ],
"/^v[0-9]+/"
],
"matchManagers": [
"haskell-cabal"
],
"enabled": false
},
{
"matchBaseBranches": [
"/^v[0-9]+/"
],
"groupName": "all dependencies" "groupName": "all dependencies"
}, },
{ {
"matchManagers": [ "matchPackageNames": ["docutils"],
"haskell-cabal" "allowedVersions": "<0.21.0"
],
"matchPackageNames": [
"base",
"bytestring",
"containers",
"directory",
"mtl",
"parsec",
"process",
"text"
],
"groupName": "GHC dependencies"
}, },
{ {
"matchManagers": [ "matchPackageNames": ["macos"],
"haskell-cabal" "allowedVersions": "<13"
],
"matchPackageNames": [
"hasql",
"hasql-dynamic-statements",
"hasql-notifications",
"hasql-transaction",
"hasql-pool"
],
"groupName": "hasql"
},
{
"matchManagers": [
"haskell-cabal"
],
"matchPackageNames": [
"fuzzyset"
],
"allowedVersions": "<0.3"
} }
] ]
} }
+72
View File
@@ -0,0 +1,72 @@
#!/bin/bash
set -Eeuo pipefail
# This script builds PostgREST in a remote ARM server
[ -z "$1" ] && { echo "Missing 1st argument: PostgREST github commit SHA"; exit 1; }
[ -z "$2" ] && { echo "Missing 2nd argument: Build environment directory name"; exit 1; }
[ -z "$3" ] && { echo "Missing 3rd argument: GHC version"; exit 1; }
PGRST_GITHUB_COMMIT="$1"
SCRIPT_DIR="$2"
DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"
# latest is a shortcut documented on https://www.haskell.org/ghcup/guide/#tags-and-shortcuts
CABAL_VERSION="latest"
GHC_VERSION="$3"
install_packages() {
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install -y git build-essential curl libffi-dev libffi7 libgmp-dev libgmp10 libncurses-dev libncurses5 libtinfo5 llvm libnuma-dev zlib1g-dev libpq-dev jq gcc
sudo apt-get clean
}
install_ghcup() {
export BOOTSTRAP_HASKELL_NONINTERACTIVE=1
export BOOTSTRAP_HASKELL_MINIMAL=1
curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh
source ~/.ghcup/env
}
install_cabal() {
ghcup upgrade
ghcup install cabal $CABAL_VERSION
ghcup set cabal $CABAL_VERSION
}
install_ghc() {
ghcup upgrade
ghcup install ghc $GHC_VERSION
ghcup set ghc $GHC_VERSION
}
install_packages
# Add ghcup to the PATH for this session
[ -f ~/.ghcup/env ] && source ~/.ghcup/env
ghcup --version || install_ghcup
ghcup set cabal $CABAL_VERSION || install_cabal
ghcup set ghc $GHC_VERSION || install_ghc
cd ~/$SCRIPT_DIR
# Clone the repository and build the project
git clone https://github.com/PostgREST/postgrest.git
cd postgrest
git checkout $PGRST_GITHUB_COMMIT
cabal v2-update && cabal v2-build
# Strip unused symbols from executable
PGRST_BIN=$(cabal exec which postgrest | tail -1)
strip $PGRST_BIN
# Copy the built binary to the Dockerfile directory
cp $PGRST_BIN ~/$DOCKER_BUILD_DIR
# Move and compress the built binary
mkdir -p ~/$SCRIPT_DIR/result
mv $PGRST_BIN ~/$SCRIPT_DIR/result
cd ~/$SCRIPT_DIR
tar -cJf result.tar.xz result
@@ -1,8 +1,6 @@
# PostgREST Docker Hub image for aarch64. # PostgREST docker hub image
# The x86-64 is a single-static-binary image built via Nix, see:
# nix/tools/docker/README.md
FROM ubuntu:resolute@sha256:678c6550cc43645e08669028bc177f50be4e7c5b8cca677067b1914d4afc7a03 AS postgrest FROM ubuntu:noble@sha256:2e863c44b718727c860746568e1d54afd13b2fa71b160f5cd9058fc436217b30 AS postgrest
RUN apt-get update -y \ RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \ && apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
@@ -10,7 +8,6 @@ RUN apt-get update -y \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
COPY postgrest /usr/bin/postgrest COPY postgrest /usr/bin/postgrest
RUN chmod +x /usr/bin/postgrest
EXPOSE 3000 EXPOSE 3000
+50
View File
@@ -0,0 +1,50 @@
#!/bin/bash
set -Eeuo pipefail
# This script publishes the Docker ARM images to Docker Hub.
[ -z "$1" ] && { echo "Missing 1st argument: PostgREST github commit SHA"; exit 1; }
[ -z "$2" ] && { echo "Missing 2nd argument: Docker repo"; exit 1; }
[ -z "$3" ] && { echo "Missing 3rd argument: Docker username"; exit 1; }
[ -z "$4" ] && { echo "Missing 4th argument: Docker password"; exit 1; }
[ -z "$5" ] && { echo "Missing 5th argument: Build environment directory name"; exit 1; }
[ -z "$6" ] && { echo "Missing 6th argument: PostgREST version"; exit 1; }
PGRST_GITHUB_COMMIT="$1"
DOCKER_REPO="$2"
DOCKER_USER="$3"
DOCKER_PASS="$4"
SCRIPT_DIR="$5"
PGRST_VERSION="$6"
DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"
clean_env()
{
sudo docker logout
}
# Login to Docker
sudo docker logout
{ echo $DOCKER_PASS | sudo docker login -u $DOCKER_USER --password-stdin; } || { echo "Couldn't login to docker"; exit 1; }
trap clean_env sigint sigterm exit
# Move to the docker build environment
cd ~/$DOCKER_BUILD_DIR
# Push final images to Docker hub
# NOTE: This command publishes a separate ARM image because the builds cannot
# be added to the manifest if they are not in the registry beforehand.
# This image must be manually deleted from Docker Hub at the end of the process.
sudo docker buildx build --build-arg PGRST_GITHUB_COMMIT=$PGRST_GITHUB_COMMIT \
-t $DOCKER_REPO/postgrest:$PGRST_VERSION-arm \
--push .
# Add the arm images to the manifest
# NOTE: This assumes that there already is a `postgrest:<version>` image
# for the amd64 architecture pushed to Docker Hub
sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:$PGRST_VERSION $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
[ "$PGRST_VERSION" != "devel" ] && sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:latest $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
sudo docker logout
-52
View File
@@ -1,52 +0,0 @@
name: Backport
on:
pull_request_target:
types:
- closed
- labeled
jobs:
backport:
name: Backport
runs-on: ubuntu-slim
# It triggers only when PR is already merged on either:
#
# - The merge event itself (action != labeled) or
# - A label event with the right label (backport ...).
#
# The result will be that we can add the label before or after merge,
# but the workflow will only run once the PR had been merged.
if: >
github.event.pull_request.merged &&
(
github.event.action != 'labeled' ||
startsWith(github.event.label.name, 'backport')
)
steps:
# This actions creates the github token using the postgrest app secrets
- name: Create Github App Token
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ vars.POSTGREST_CI_APP_ID }}
private-key: ${{ secrets.POSTGREST_CI_PRIVATE_KEY }}
permission-contents: write
permission-pull-requests: write
permission-workflows: write # required when backporting CI changes
# This is required for backport action to cherry-pick the PR
- name: Fetch PR ref
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha }}
token: ${{ steps.app-token.outputs.token }}
# Backport action that creates the PR with given settings
- name: Create backport PR
uses: korthout/backport-action@2e830a1d0b8269505846ddd407a70876913ad1f8 # v4.6
with:
github_token: ${{ steps.app-token.outputs.token }}
pull_description: 'Backport for #${pull_number}.'
pull_title: '${target_branch}: ${pull_title}'
+79 -65
View File
@@ -16,7 +16,6 @@ on:
- .github/* - .github/*
- '*.nix' - '*.nix'
- nix/** - nix/**
- flake.lock
- .cirrus.yml - .cirrus.yml
- cabal.project* - cabal.project*
- postgrest.cabal - postgrest.cabal
@@ -31,51 +30,56 @@ concurrency:
jobs: jobs:
static: static:
name: Nix - Linux x86-64 static name: Nix - Linux static
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Build static executable - name: Build static executable
run: nix-build -A postgrestStatic -A postgrestStatic.tests run: nix-build -A postgrestStatic
- name: Save built executable as artifact - name: Save built executable as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with: with:
name: postgrest-linux-static-x86-64 name: postgrest-linux-static-x64
path: result/bin/postgrest path: result/bin/postgrest
if-no-files-found: error if-no-files-found: error
- name: Build Docker image - name: Build Docker image
run: nix-build -A docker.image --out-link postgrest-docker.tar.gz run: nix-build -A docker.image --out-link postgrest-docker.tar.gz
- name: Save built Docker image as artifact - name: Save built Docker image as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with: with:
name: postgrest-docker-x86-64 name: postgrest-docker-x64
path: postgrest-docker.tar.gz path: postgrest-docker.tar.gz
if-no-files-found: error if-no-files-found: error
macos: macos:
name: Nix - MacOS name: Nix - MacOS
runs-on: macos-26 runs-on: macos-12
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Install nix-build-uncached - name: Install gnu sed
run: nix-env -f default.nix -iA nix-build-uncached run: brew install gnu-sed
- name: Build everything (default.nix) - name: Build everything
run: nix-build-uncached run: |
# The --dry-run will give us a list of derivations to download from cachix and
- name: Build everything (shell.nix) # derivations to build. We only take those that would have to be built and then build
run: nix-build-uncached shell.nix # those explicitly. This has the advantage that pure verification will not include
# a download anymore, making it much faster. If something needs to be built, only
# the dependencies required to do so will be downloaded, but not everything.
nix-build --dry-run 2>&1 \
| gsed -e '1,/derivations will be built:$/d' -e '/paths will be fetched/Q' \
| xargs nix-build
stack: stack:
@@ -83,71 +87,68 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
include: include:
- name: FreeBSD x86-64 - name: Linux
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
vm: freebsd cache: |
artifact: postgrest-freebsd-x86-64 ~/.stack/pantry
deps: pkg install -y git postgresql16-client hs-stack ~/.stack/snapshots
~/.stack/stack.sqlite3
# no artifact for Linux, because we use the static build
- name: Linux aarch64 - name: MacOS
runs-on: ubuntu-24.04-arm runs-on: macos-12
artifact: postgrest-ubuntu-aarch64 cache: |
deps: sudo apt-get update && sudo apt-get install libpq-dev ~/.stack/pantry
~/.stack/snapshots
- name: MacOS aarch64 ~/.stack/stack.sqlite3
runs-on: macos-14 artifact: postgrest-macos-x64
artifact: postgrest-macos-aarch64
deps: brew link --force libpq
- name: MacOS x86-64
runs-on: macos-15-intel
artifact: postgrest-macos-x86-64
deps: brew link --force libpq
- name: Windows - name: Windows
runs-on: windows-2022 runs-on: windows-2022
cache: |
~\AppData\Roaming\stack\pantry
~\AppData\Local\Programs\stack\pantry
~\AppData\Roaming\stack\snapshots
~\AppData\Local\Programs\stack\snapshots
~\AppData\Roaming\stack\stack.sqlite3
~\AppData\Local\Programs\stack\stack.sqlite3
deps: Add-Content $env:GITHUB_PATH $env:PGBIN deps: Add-Content $env:GITHUB_PATH $env:PGBIN
artifact: postgrest-windows-x86-64 artifact: postgrest-windows-x64
name: Stack - ${{ matrix.name }} name: Stack - ${{ matrix.name }}
runs-on: ${{ matrix.runs-on }} runs-on: ${{ matrix.runs-on }}
env:
# Putting .stack in the working directory helps with moving this in and out of the FreeBSD VM.
STACK_ROOT: ${{ github.workspace }}/.stack
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- if: ${{ !matrix.vm }} - uses: haskell-actions/setup@ec49483bfc012387b227434aba94f59a6ecd0900 # v2.7.5
uses: haskell-actions/setup@6037f33647c3f17758a2356c80fc4a53d7e0685d # v2.12.0
with: with:
# This must match the version in stack.yaml's resolver # This must match the version in stack.yaml's resolver
ghc-version: 9.6.7 ghc-version: 9.6.5
enable-stack: true enable-stack: true
stack-no-global: true stack-no-global: true
stack-setup-ghc: true stack-setup-ghc: true
- name: Cache .stack - name: Cache ~/.stack
uses: ./.github/actions/cache-on-main uses: ./.github/actions/cache-on-main
with: with:
path: .stack path: ${{ matrix.cache }}
prefix: ${{ matrix.vm }}${{ matrix.vm && '-' }}stack prefix: stack
suffix: ${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }} suffix: ${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
- name: Cache .stack-work - name: Cache .stack-work
uses: ./.github/actions/cache-on-main uses: ./.github/actions/cache-on-main
with: with:
path: .stack-work path: .stack-work
save-prs: true save-prs: true
prefix: ${{ matrix.vm }}${{ matrix.vm && '-' }}stack-work-${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }} prefix: stack-work-${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
suffix: ${{ hashFiles('main/**/*.hs', 'src/**/*.hs') }} suffix: ${{ hashFiles('main/**/*.hs', 'src/**/*.hs') }}
- name: Install dependencies
if: matrix.deps
run: ${{ matrix.deps }}
- name: Build with Stack - name: Build with Stack
uses: ./.github/actions/run-anywhere run: stack build --lock-file error-on-write --local-bin-path result --copy-bins
with: - name: Strip Executable
vm: ${{ matrix.vm }} run: strip result/postgrest*
envs: STACK_ROOT
prepare: ${{ matrix.deps }}
run: |
stack build --lock-file error-on-write --local-bin-path result --copy-bins
strip result/postgrest*
- name: Save built executable as artifact - name: Save built executable as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 if: matrix.artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with: with:
name: ${{ matrix.artifact }} name: ${{ matrix.artifact }}
path: | path: |
@@ -156,16 +157,29 @@ jobs:
if-no-files-found: error if-no-files-found: error
freebsd:
name: Stack - FreeBSD from CirrusCI
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: ./.github/actions/artifact-from-cirrus
with:
token: ${{ github.token }}
task: Build FreeBSD (Stack)
download: bin
upload: postgrest-freebsd-x64
cabal: cabal:
strategy: strategy:
matrix: matrix:
ghc: ['9.6.7', '9.8.4'] ghc: ['9.6.4', '9.8.2']
fail-fast: false fail-fast: false
name: Cabal - Linux x86-64 - GHC ${{ matrix.ghc }} name: Cabal - Linux GHC ${{ matrix.ghc }}
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: haskell-actions/setup@6037f33647c3f17758a2356c80fc4a53d7e0685d # v2.12.0 - uses: haskell-actions/setup@ec49483bfc012387b227434aba94f59a6ecd0900 # v2.7.5
with: with:
ghc-version: ${{ matrix.ghc }} ghc-version: ${{ matrix.ghc }}
- name: Cache .cabal - name: Cache .cabal
@@ -174,8 +188,8 @@ jobs:
path: | path: |
~/.cabal/packages ~/.cabal/packages
~/.cabal/store ~/.cabal/store
prefix: cabal-${{ matrix.ghc }}-${{ hashFiles('cabal.project.freeze') }} prefix: cabal-${{ matrix.ghc }}
suffix: ${{ hashFiles('postgrest.cabal', 'cabal.project') }} suffix: ${{ hashFiles('postgrest.cabal', 'cabal.project', 'cabal.project.freeze') }}
- name: Cache dist-newstyle - name: Cache dist-newstyle
uses: ./.github/actions/cache-on-main uses: ./.github/actions/cache-on-main
with: with:
+2 -23
View File
@@ -18,9 +18,9 @@ concurrency:
jobs: jobs:
lint-style: lint-style:
name: Lint & Style name: Lint & Style
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
@@ -30,24 +30,3 @@ jobs:
run: postgrest-lint run: postgrest-lint
- name: Run style check (auto-format with `nix-shell --run postgrest-style`) - name: Run style check (auto-format with `nix-shell --run postgrest-style`)
run: postgrest-style-check run: postgrest-style-check
commit:
if: github.event_name != 'push' # we don't run this on a push, a failure on push disrupts the release workflow
name: Commit
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 100 # fetch history (last 100 commits) instead of default shallow clone history, this is deemed enough for a PR history
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: gitTools.commitCheck.bin
- name: Run commitlint (check locally with `nix-shell --run postgrest-commitlint`)
run: |
# Fetch target branch explicitly
git fetch origin ${{ github.base_ref }}
# Run commitlint
postgrest-commitlint --from origin/${{ github.base_ref }} --to HEAD
+299 -6
View File
@@ -5,10 +5,21 @@ on:
branches: branches:
- main - main
- v[0-9]+ - v[0-9]+
tags:
- devel
- v*
concurrency:
# Terminate all previous runs of the same workflow for the same tag, but never
# cancel for the main / release branch pipelines.
group: ci-${{ (github.ref_type == 'tag' && github.ref) || github.run_id }}
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
jobs: jobs:
check: check:
name: Check name: Check
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/check.yaml uses: ./.github/workflows/check.yaml
secrets: secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }} CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
@@ -16,6 +27,7 @@ jobs:
docs: docs:
name: Docs name: Docs
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/docs.yaml uses: ./.github/workflows/docs.yaml
secrets: secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }} CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
@@ -23,6 +35,7 @@ jobs:
test: test:
name: Test name: Test
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/test.yaml uses: ./.github/workflows/test.yaml
secrets: secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }} CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
@@ -36,27 +49,94 @@ jobs:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }} CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
arm:
name: Build / Cabal - aarch64 GHC 9.4.8
if: vars.SSH_ARM_ENABLED
runs-on: ubuntu-22.04
outputs:
remotepath: ${{ steps.Remote-Dir.outputs.remotepath }}
env:
GITHUB_COMMIT: ${{ github.sha }}
GHC_VERSION: '9.4.8'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- id: Remote-Dir
name: Unique directory name for the remote build
run: echo "remotepath=postgrest-build-$(uuidgen)" >> "$GITHUB_OUTPUT"
- name: Copy script files to the remote server
uses: appleboy/scp-action@master
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
source: ".github/scripts/arm/*"
target: ${{ steps.Remote-Dir.outputs.remotepath }}
strip_components: 3
- name: Build ARM
uses: appleboy/ssh-action@master
env:
REMOTE_DIR: ${{ steps.Remote-Dir.outputs.remotepath }}
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
command_timeout: 120m
script_stop: true
envs: GITHUB_COMMIT,REMOTE_DIR,GHC_VERSION
script: bash ~/$REMOTE_DIR/build.sh "$GITHUB_COMMIT" "$REMOTE_DIR" "$GHC_VERSION"
- name: Download binaries from remote server
uses: nicklasfrahm/scp-action@main
with:
direction: download
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
source: "${{ steps.Remote-Dir.outputs.remotepath }}/result.tar.xz"
target: "result.tar.xz"
- name: Extract downloaded binaries
run: tar -xvf result.tar.xz && rm result.tar.xz
- name: Save aarch64 executable as artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-ubuntu-aarch64
path: result/postgrest
if-no-files-found: error
tag: tag:
name: Tag name: Release / Tag
concurrency: concurrency:
# Never tag outdated commits on the main branch by skipping superseded commits # Never tag outdated commits on the main branch by skipping superseded commits
group: ci-tag-${{ (github.ref == 'refs/heads/main' && github.ref) || github.run_id }} group: ci-tag-${{ (github.ref == 'refs/heads/main' && github.ref) || github.run_id }}
cancel-in-progress: true # TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
if: vars.RELEASE_ENABLED cancel-in-progress: false
runs-on: ubuntu-slim if: |
vars.RELEASE_ENABLED &&
startsWith(github.ref, 'refs/heads/') &&
needs.docs.result == 'success' &&
needs.test.result == 'success' &&
needs.build.result == 'success' &&
(needs.arm.result == 'skipped' || success())
permissions:
contents: write
runs-on: ubuntu-22.04
needs: needs:
- docs - docs
- test - test
- build - build
- arm
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with: with:
ssh-key: ${{ secrets.POSTGREST_SSH_KEY }} ssh-key: ${{ secrets.POSTGREST_SSH_KEY }}
- name: Tag latest commit - name: Tag latest commit
run: | run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)" cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
if [[ "$cabal_version" == *.* ]]; then if [[ "$cabal_version" == *.*.* ]]; then
git fetch --tags git fetch --tags
if [ -z "$(git tag --list "v$cabal_version")" ]; then if [ -z "$(git tag --list "v$cabal_version")" ]; then
@@ -67,3 +147,216 @@ jobs:
git tag -f "devel" git tag -f "devel"
git push -f origin "devel" git push -f origin "devel"
fi fi
prepare:
name: Release / Prepare
if: |
startsWith(github.ref, 'refs/tags/') &&
needs.build.result == 'success' &&
(needs.arm.result == 'skipped' || success())
runs-on: ubuntu-22.04
needs:
- build
- arm
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
if [ "${GITHUB_REF_NAME}" != "devel" ] && [ "${GITHUB_REF_NAME}" != "v$cabal_version" ]; then
echo "Tagged version ($GITHUB_REF_NAME) does not match the one in postgrest.cabal (v$cabal_version). Aborting release..."
exit 1
fi
- name: Identify changes from CHANGELOG.md
run: |
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
echo "Getting unreleased changes..."
sed -n "1,/## Unreleased/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
else
version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
echo "Propper release, getting changes for version $version ..."
sed -n "1,/## \[$version\]/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
fi
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Save CHANGES.md as artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: release-changes
path: CHANGES.md
if-no-files-found: error
github:
name: Release / GitHub
permissions:
contents: write
runs-on: ubuntu-22.04
needs:
- prepare
if: success() || needs.prepare.result == 'success'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Download all artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: artifacts
- name: Create release bundle with archives for all builds
run: |
find artifacts -type f -iname postgrest -exec chmod +x {} \;
mkdir -p release-bundle
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-x64.tar.xz" \
-C artifacts/postgrest-linux-static-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-x64.tar.xz" \
-C artifacts/postgrest-macos-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-freebsd-x64.tar.xz" \
-C artifacts/postgrest-freebsd-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-ubuntu-aarch64.tar.xz" \
-C artifacts/postgrest-ubuntu-aarch64 postgrest
zip "release-bundle/postgrest-${GITHUB_REF_NAME}-windows-x64.zip" \
artifacts/postgrest-windows-x64/postgrest.exe
- name: Save release bundle
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: release-bundle
path: release-bundle
if-no-files-found: error
- name: Publish release on GitHub
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "Releasing version ${GITHUB_REF_NAME} on GitHub..."
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
# To replace the existing release, we must first delete the old assets,
# then modify the release, then add the new assets.
gh release view devel --json assets \
| jq -r '.assets[] | .name' \
| xargs -rn1 \
gh release delete-asset -y devel
gh release edit devel \
-t devel \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
--prerelease
gh release upload --clobber devel release-bundle/*
else
gh release create "${GITHUB_REF_NAME}" \
-t "${GITHUB_REF_NAME}" \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
release-bundle/*
fi
docker:
name: Release / Docker Hub
runs-on: ubuntu-22.04
needs:
- prepare
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
(success() || needs.prepare.result == 'success')
env:
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Download Docker image
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: postgrest-docker-x64
- name: Publish images on Docker Hub
run: |
docker login -u "$DOCKER_USER" -p "$DOCKER_PASS"
docker load -i postgrest-docker.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
# Only tag 'latest' for full releases
if [ "${GITHUB_REF_NAME}" != "devel" ]; then
echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..."
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
docker push "$DOCKER_REPO"/postgrest:latest
else
echo "Skipping push to 'latest' tag for pre-release..."
fi
docker-description:
name: Release / Docker Hub Description
runs-on: ubuntu-22.04
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
github.ref == 'refs/tags/devel'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: peter-evans/dockerhub-description@e98e4d1628a5f3be2be7c231e50981aee98723ae # v4.0.0
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
repository: ${{ vars.DOCKER_REPO }}/postgrest
short-description: ${{ github.event.repository.description }}
readme-filepath: ./docker-hub-readme.md
docker-arm:
name: Release / Docker Hub Arm
runs-on: ubuntu-22.04
needs:
- arm
- docker
env:
GITHUB_COMMIT: ${{ github.sha }}
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Publish images for ARM builds on Docker Hub
uses: appleboy/ssh-action@master
env:
REMOTE_DIR: ${{ needs.arm.outputs.remotepath }}
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
script_stop: true
envs: GITHUB_COMMIT,DOCKER_REPO,DOCKER_USER,DOCKER_PASS,REMOTE_DIR,GITHUB_REF_NAME
script: bash ~/$REMOTE_DIR/docker-publish.sh "$GITHUB_COMMIT" "$DOCKER_REPO" "$DOCKER_USER" "$DOCKER_PASS" "$REMOTE_DIR" "$GITHUB_REF_NAME"
clean-arm:
name: Build / Cleanup
needs:
- arm
- docker-arm
if: ${{ always() && vars.SSH_ARM_ENABLED }}
runs-on: ubuntu-22.04
env:
REMOTE_DIR: ${{ needs.arm.outputs.remotepath }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Remove uploaded files from server
uses: appleboy/ssh-action@master
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
envs: REMOTE_DIR
script: rm -rf $REMOTE_DIR
+18 -5
View File
@@ -14,7 +14,6 @@ on:
- .github/actions/setup-nix/** - .github/actions/setup-nix/**
- default.nix - default.nix
- nix/** - nix/**
- flake.lock
- docs/** - docs/**
- '!**.md' - '!**.md'
@@ -26,9 +25,9 @@ concurrency:
jobs: jobs:
build: build:
name: Build name: Build
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
@@ -40,9 +39,9 @@ jobs:
spellcheck: spellcheck:
name: Spellcheck name: Spellcheck
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
@@ -52,3 +51,17 @@ jobs:
run: postgrest-docs-spellcheck run: postgrest-docs-spellcheck
- name: Run dictcheck - name: Run dictcheck
run: postgrest-docs-dictcheck run: postgrest-docs-dictcheck
linkcheck:
name: Linkcheck
if: github.base_ref == 'main'
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.linkcheck.bin
- run: postgrest-docs-linkcheck
-43
View File
@@ -1,43 +0,0 @@
name: Linkcheck
on:
schedule:
- cron: '1 2 * * 3'
workflow_dispatch:
jobs:
linkcheck:
name: Linkcheck
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.linkcheck.bin
- name: Run Linkcheck
id: linkcheck
run: postgrest-docs-linkcheck
# This actions creates the github token using the postgrest app secrets
- name: Create Github App Token (Runs only on linkcheck failure)
id: app-token
if: ${{ failure() && steps.linkcheck.outcome == 'failure' }} # only create the token on linkcheck failure
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ vars.POSTGREST_CI_APP_ID }}
private-key: ${{ secrets.POSTGREST_CI_PRIVATE_KEY }}
permission-issues: write # required for commenting on issues
- name: Notify on linkcheck failure by commenting on GH Issue 4106
if: ${{ failure() && steps.linkcheck.outcome == 'failure' }}
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
with:
token: ${{ steps.app-token.outputs.token }}
issue-number: 4106
body: |
**Linkcheck Job Failed!**
A broken link was detected in the docs. Please check the [failed run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details.
-183
View File
@@ -1,183 +0,0 @@
name: Release
on:
push:
tags:
- devel
- v*
concurrency:
# Terminate all previous runs of the same workflow for the same tag.
group: release-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
name: Build
uses: ./.github/workflows/build.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
github:
name: GitHub
permissions:
contents: write
runs-on: ubuntu-slim
needs:
- build
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
if [ "${GITHUB_REF_NAME}" != "devel" ] && [ "${GITHUB_REF_NAME}" != "v$cabal_version" ]; then
echo "Tagged version ($GITHUB_REF_NAME) does not match the one in postgrest.cabal (v$cabal_version). Aborting release..."
exit 1
fi
- name: Identify changes from CHANGELOG.md
run: |
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
echo "Getting unreleased changes..."
sed -n "1,/## Unreleased/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
else
version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
echo "Propper release, getting changes for version $version ..."
sed -n "1,/## \[$version\]/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
fi
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Download all artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
path: artifacts
- name: Create release bundle with archives for all builds
run: |
find artifacts -type f -iname postgrest -exec chmod +x {} \;
mkdir -p release-bundle
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-x86-64.tar.xz" \
-C artifacts/postgrest-linux-static-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-aarch64.tar.xz" \
-C artifacts/postgrest-macos-aarch64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-x86-64.tar.xz" \
-C artifacts/postgrest-macos-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-freebsd-x86-64.tar.xz" \
-C artifacts/postgrest-freebsd-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-ubuntu-aarch64.tar.xz" \
-C artifacts/postgrest-ubuntu-aarch64 postgrest
zip --junk-paths "release-bundle/postgrest-${GITHUB_REF_NAME}-windows-x86-64.zip" \
artifacts/postgrest-windows-x86-64/postgrest.exe
- name: Save release bundle
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-bundle
path: release-bundle
if-no-files-found: error
- name: Publish release on GitHub
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "Releasing version ${GITHUB_REF_NAME} on GitHub..."
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
# To replace the existing release, we must first delete the old assets,
# then modify the release, then add the new assets.
gh release view devel --json assets \
| jq -r '.assets[] | .name' \
| xargs -rn1 \
gh release delete-asset -y devel
gh release edit devel \
-t devel \
--verify-tag \
-F CHANGES.md \
--prerelease
gh release upload --clobber devel release-bundle/*
else
gh release create "${GITHUB_REF_NAME}" \
-t "${GITHUB_REF_NAME}" \
--verify-tag \
-F CHANGES.md \
release-bundle/*
fi
docker:
name: Docker Hub
runs-on: ubuntu-24.04-arm
needs:
- github
if: |
vars.DOCKER_REPO && vars.DOCKER_USER
env:
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Download x86-64 Docker image
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-docker-x86-64
- name: Download aarch64 binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-ubuntu-aarch64
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
- name: Build aarch64 Docker image
run: |
# This only pushes the image via digest, not a tag. This will not appear
# in the image list on Docker Hub, yet. It will be later added to the main
# tag's manifest.
docker buildx build \
-t "$DOCKER_REPO/postgrest" \
--platform linux/arm64 \
--output push-by-digest=true,type=image,push=true \
--metadata-file metadata.json \
.
echo "SHA256_ARM=$(jq -r '."containerimage.digest"' metadata.json)" >> "$GITHUB_ENV"
- name: Publish images on Docker Hub
run: |
docker load -i postgrest-docker.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker buildx imagetools create --append \
-t "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}" \
"$DOCKER_REPO/postgrest@$SHA256_ARM"
# Only tag 'latest' for full releases
if [ "${GITHUB_REF_NAME}" != "devel" ]; then
echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..."
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
docker push "$DOCKER_REPO"/postgrest:latest
docker buildx imagetools create --append \
-t "$DOCKER_REPO/postgrest:latest" \
"$DOCKER_REPO/postgrest@$SHA256_ARM"
else
echo "Skipping push to 'latest' tag for pre-release..."
fi
- uses: peter-evans/dockerhub-description@1b9a80c056b620d92cedb9d9b5a223409c68ddfa # v5.0.0
if: github.ref == 'refs/tags/devel'
name: Docker Hub Description
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
repository: ${{ vars.DOCKER_REPO }}/postgrest
short-description: ${{ github.event.repository.description }}
readme-filepath: ./docker-hub-readme.md
+39
View File
@@ -0,0 +1,39 @@
name: Upload Reports
on:
workflow_run:
workflows:
- CI
- Test
branches-ignore:
- devel
- v[0-9]+
- v[0-9]+.[0-9]+.[0-9]+
types:
- completed
jobs:
loadtest:
name: Loadtest
permissions:
checks: write
runs-on: ubuntu-22.04
if: github.event.workflow_run.conclusion == 'success' || github.event.workflow_run.conclusion == 'failure'
steps:
- name: Download from Artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
github-token: ${{ github.token }}
run-id: ${{ github.event.workflow_run.id }}
name: loadtest.md
path: artifacts
- name: Upload to GitHub Checks
uses: LouisBrunner/checks-action@6b626ffbad7cc56fd58627f774b9067e6118af23 # v2.0.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
sha: ${{ github.event.workflow_run.head_sha }}
name: Loadtest Results
conclusion: neutral
output: |
{"summary":""}
output_text_description_file: artifacts/loadtest.md
+28 -68
View File
@@ -17,7 +17,6 @@ on:
- .github/actions/setup-nix/** - .github/actions/setup-nix/**
- default.nix - default.nix
- nix/** - nix/**
- flake.lock
- .stylish-haskell.yaml - .stylish-haskell.yaml
- cabal.project - cabal.project
- postgrest.cabal - postgrest.cabal
@@ -33,26 +32,24 @@ concurrency:
jobs: jobs:
coverage: coverage:
name: Coverage name: Coverage
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
defaults: defaults:
run: run:
# Hack for enabling color output, see: # Hack for enabling color output, see:
# https://github.com/actions/runner/issues/241#issuecomment-842566950 # https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}" shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: tests.coverage.bin tests.testDoctests.bin tests.testSpecIdempotence.bin cabalTools.update.bin tools: tests.coverage.bin tests.testDoctests.bin tests.testSpecIdempotence.bin
- run: postgrest-cabal-update
- name: Run coverage (IO tests and Spec tests against PostgreSQL 15) - name: Run coverage (IO tests and Spec tests against PostgreSQL 15)
run: postgrest-coverage run: postgrest-coverage
- name: Upload coverage to codecov - name: Upload coverage to codecov
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0 uses: codecov/codecov-action@e28ff129e5465c2c0dcc6f003fc735cb6ae0c673 # v4.5.0
with: with:
files: ./coverage/codecov.json files: ./coverage/codecov.json
token: ${{ secrets.CODECOV_TOKEN }} token: ${{ secrets.CODECOV_TOKEN }}
@@ -70,109 +67,72 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
# Latest version is tested via `coverage` above. pgVersion: ["9_6", 10, 11, 12, 13, 14, 15, 16]
pgVersion: [13, 14, 15, 16]
name: PG ${{ matrix.pgVersion }} name: PG ${{ matrix.pgVersion }}
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
defaults: defaults:
run: run:
# Hack for enabling color output, see: # Hack for enabling color output, see:
# https://github.com/actions/runner/issues/241#issuecomment-842566950 # https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}" shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: tests.testSpec.bin tests.testObservability.bin tests.testIO.bin tests.testBigSchema.bin withTools.pg-${{ matrix.pgVersion }}.bin cabalTools.update.bin tools: tests.testSpec.bin tests.testIO.bin tests.testBigSchema.bin withTools.postgresql-${{ matrix.pgVersion }}.bin
- run: postgrest-cabal-update
- name: Run spec tests - name: Run spec tests
if: always() if: always()
run: postgrest-with-pg-${{ matrix.pgVersion }} postgrest-test-spec run: postgrest-with-postgresql-${{ matrix.pgVersion }} postgrest-test-spec
- name: Run observability tests
if: always()
run: postgrest-with-pg-${{ matrix.pgVersion }} postgrest-test-observability
- name: Run IO tests - name: Run IO tests
if: always() if: always()
run: postgrest-with-pg-${{ matrix.pgVersion }} postgrest-test-io -vv run: postgrest-with-postgresql-${{ matrix.pgVersion }} postgrest-test-io -vv
- name: Run IO tests on a big schema - name: Run IO tests on a big schema
if: always() if: always()
run: postgrest-with-pg-${{ matrix.pgVersion }} postgrest-test-big-schema -vv run: postgrest-with-postgresql-${{ matrix.pgVersion }} postgrest-test-big-schema -vv
memory: memory:
name: Memory name: Memory
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: tests.testMemory.bin cabalTools.update.bin tools: memory.test.bin
- run: postgrest-cabal-update
- name: Run memory tests - name: Run memory tests
run: postgrest-test-memory run: postgrest-test-memory
loadtest: loadtest:
strategy:
fail-fast: false
matrix:
kind: ['mixed', 'jwt-hs', 'jwt-hs-cache', 'jwt-hs-cache-worst', 'jwt-rsa', 'jwt-rsa-cache', 'jwt-rsa-cache-worst']
name: Loadtest name: Loadtest
runs-on: ubuntu-24.04 runs-on: ubuntu-22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with: with:
fetch-depth: 0 fetch-depth: 0
- name: Setup Nix Environment - name: Setup Nix Environment
uses: ./.github/actions/setup-nix uses: ./.github/actions/setup-nix
with: with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: loadtest.loadtestAgainst.bin loadtest.report.bin cabalTools.update.bin tools: loadtest.loadtestAgainst.bin loadtest.report.bin
- uses: WyriHaximus/github-action-get-previous-tag@04e8485ecb6487243907e330d522ff60f02283ce # v1.4.0
- run: postgrest-cabal-update id: get-latest-tag
with:
prefix: v
- name: Run loadtest - name: Run loadtest
env:
TARGET_BRANCH: ${{ github.base_ref || github.ref_name }}
run: | run: |
if [ "$TARGET_BRANCH" = "main" ]; then postgrest-loadtest-against main ${{ steps.get-latest-tag.outputs.tag }}
latest_tag=$(git tag --sort=-creatordate --list "v*" | head -n1) postgrest-loadtest-report > loadtest/loadtest.md
else - name: Upload report
latest_tag=$(git tag --merged HEAD --sort=-creatordate "v*" | head -n1) uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
fi
postgrest-loadtest-against -k ${{ matrix.kind }} "$TARGET_BRANCH" "$latest_tag"
postgrest-loadtest-report -g ${{ matrix.kind }} >> "$GITHUB_STEP_SUMMARY"
flake:
strategy:
fail-fast: false
matrix:
runs-on:
- macos-15-intel # x86_64-darwin
- macos-14 # aarch64-darwin
- ubuntu-24.04 # x86_64-linux
- ubuntu-24.04-arm # aarch64-linux
name: Flake Check
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 name: loadtest.md
- name: Setup Nix Environment path: loadtest/loadtest.md
uses: ./.github/actions/setup-nix if-no-files-found: error
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Run flake check
run: |
nix flake check
+1 -4
View File
@@ -14,7 +14,7 @@ site
.#* .#*
*.swp *.swp
result* result*
dist-* dist-newstyle
postgrest.hp postgrest.hp
postgrest.prof postgrest.prof
__pycache__ __pycache__
@@ -24,6 +24,3 @@ coverage
loadtest loadtest
.history .history
.docs-build .docs-build
gen_targets.http
gen_jwk.json
.ghc.environment.*
+1 -1
View File
@@ -5,6 +5,6 @@ python:
install: install:
- requirements: docs/requirements.txt - requirements: docs/requirements.txt
build: build:
os: ubuntu-24.04 os: ubuntu-22.04
tools: tools:
python: "3.11" python: "3.11"
+13 -18
View File
@@ -13,26 +13,31 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank"> <a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/supabase.svg"> <img width="296px" src="static/gnuhost.png">
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://www.euronodes.com/postgrest" target="_blank"> <a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/euronodes.svg"> <img width="296px" src="static/neon.jpg">
</a> </a>
</td> </td>
</tr> </tr>
<tr></tr> <tr></tr>
<tr> <tr>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank"> <a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/neon.jpg"> <img width="296px" src="static/code-build.png">
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://www.bytebase.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank"> <a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank">
<img width="296px" src="static/bytebase.svg"> <img width="296px" src="static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
</a> </a>
</td> </td>
</tr> </tr>
@@ -88,16 +93,6 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
<img width="222px" src="static/oblivious.jpg"> <img width="222px" src="static/oblivious.jpg">
</a> </a>
</td> </td>
<td align="center" valign="middle">
<a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="222px" src="static/code-build.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
</a>
</td>
</tr> </tr>
</tbody> </tbody>
</table> </table>
+2 -334
View File
@@ -1,342 +1,10 @@
# Change Log # Change Log
All notable changes to this project will be documented in this file. From version `14.0` onwards PostgREST follows a `MAJOR.PATCH` two-part versioning. Only even-numbered MAJOR versions will be released, reserving odd-numbered MAJOR versions for development. All notable changes to this project will be documented in this file.
This project adheres to [Semantic Versioning](http://semver.org/).
## Unreleased ## Unreleased
## [14.17] - 2026-08-13
### Fixed
- JWT validation uses wrong current time due to a bug in auto-update by @mkleczek in #5159
## [14.16] - 2026-07-27
### Fixed
- Fix admin server crashing without a way to recover by @taimoorzaeem in #5096
## [14.15] - 2026-07-13
### Fixed
- Fix admin server dying silently by @Vlix, @mkleczek, @steve-chavez in #5012
## [14.14] - 2026-06-29
### Fixed
- Fix admin server not logging cause of failure by @taimoorzaeem in #5012
## [14.13] - 2026-06-04
### Fixed
- Fix connection retrying message in `PGRST000` error by @netqo in #4980
+ Remove redundant "Retrying the connection." from message because it is logged separately
- Fix request failures when `work_mem` is set on a role by @laurenceisla in #4955
## [14.12] - 2026-05-20
### Fixed
- Fix race condition in pool_available metric causing negative values during network instability by @mkleczek in #4622
## [14.11] - 2026-05-04
### Fixed
- Fix login with uppercase and mixed case role names by @taimoorzaeem in #4678
- Restore Listener query shape so it can be found in `pg_stat_activity` by @mkleczek in #4857 #4859
- The LISTEN channel now automatically recovers when it stops working due to a PostgreSQL bug @laurenceisla in #3147
- Fix misleading "Functions" name on schema cache summary in startup logs by @taimoorzaeem in #4821
## [14.10] - 2026-04-16
### Added
- Log when the pool is released during schema cache reload on `log-level=debug` by @mkleczek in #4668
### Fixed
- Fix unnecessary connection pool flushes during schema cache reloading by @mkleczek in #4645
## [14.9] - 2026-04-10
### Added
- Log host, port and pg version of listener database connection by @mkleczek in #4617 #4618
### Fixed
- Remove red herring warp logs on default log-level, only emit them on `log-level=debug` by @steve-chavez in #4799
## [14.8] - 2026-04-03
### Added
- Log a `HINT` when the LISTEN channel stops working due to a PostgreSQL bug by @laurenceisla in #4581
### Fixed
- Fix invalid OpenAPI 2.0 format for integer types (`smallint`, `integer`, `bigint`) by @arturbent0 in #4641
## [14.7] - 2026-03-20
### Fixed
- Fix not logging SIGTERM and SIGINT by @steve-chavez in #4728
## [14.6] - 2026-03-06
### Fixed
- Fix leaking table and function names when calculating error hint by @taimoorzaeem in #4675
## [14.5] - 2026-02-12
### Fixed
- Don't hide async exceptions in logs by @stevechavez in #4646
## [14.4] - 2026-01-29
### Fixed
- Ensure Listener connections are released by @mkleczek in #4614
- Fix incorrectly filtering the returned representation for PATCH requests when using `or/and` filters by @laurenceisla in #3707
- Fix listener running with exception masked after first failure by @mkleczek #4615
## [14.3] - 2026-01-03
### Fixed
- Fix performance and high memory usage of relation hint calculation by @mkleczek in #4462, #4463
## [14.2] - 2025-12-18
### Fixed
- Fix `hasSingleUnnamedParam` incorrectly matching functions with named parameters by @joelonsql in #4553
+ Functions with a single named parameter (e.g., `foo(data json)`) no longer incorrectly match the single-param fallback, returning a clean `PGRST202` error instead of a confusing PostgreSQL `42883` error.
- Fix misleading logs on unsupported PostgreSQL versions by @taimoorzaeem in #4519
- Fix regression where the `PGRST103` error response was truncated by @laurenceisla in #4455
+ Happened when an `offset` was greater than the rows requested and `Prefer: count=exact` was sent.
- Fix not returning `Content-Length` on empty HTTP `201` responses by @laurenceisla in #4518
- Fix inaccurate Server-Timing header durations by @steve-chavez in #4522
- Fix inaccurate "Schema cache queried" logs by @steve-chavez in #4522
## [14.1] - 2025-11-05
## Fixed
- Fix `db-pre-config` function failing when function names are pg reserved words by @taimoorzaeem in #4380
- Fix `server-host=!6` incorrectly binds to IPv4 address by @taimoorzaeem in #3202
## [14.0] - 2025-10-24
### Added
- Bounded JWT cache using the SIEVE algorithm by @mkleczek in #4084
+ It now uses a fixed size cache instead of arbitrary sized cache.
- Add `--ready` flag for postgrest healthcheck by @taimoorzaeem in #4239
### Fixed
- Fix not logging OpenAPI queries when `log-query=main-query` is enabled by @steve-chavez in #4226
- Fix not logging explain query when `log-query=main-query` is enabled by @steve-chavez in #4319
- Fix not logging transaction variables and db-pre-request function when `log-query=main-query` is enabled by @steve-chavez in #3934
- Fix not logging the JSON message to stderr on a `PGRST002` error by @laurenceisla in #4129
- Fix reloading the Schema Cache unnecessarily on a `PGRST002` error by @laurenceisla in #4367
- Fix schema cache loading taking a long time for large schemas by @mkleczek in #4360, #3704
### Changed
- Drop support for PostgreSQL EOL version 12 by @wolfgangwalther in #3865
- From now on PostgREST will follow a `MAJOR.PATCH` two-part versioning. Only even-numbered MAJOR versions will be released, reserving odd-numbered MAJOR versions for development.
- Replaced `jwt-cache-max-lifetime` config with `jwt-cache-max-entries` by @mkleczek in #4084
- `log-query` config now takes a boolean instead of a string value by @steve-chavez in #3934
## [13.0.8] - 2025-10-24
### Fixed
- Fix loading utf-8 config files with `ASCII` locale set by @taimoorzaeem in #4386
## [13.0.7] - 2025-09-14
### Added
- Improve the `PGRST106` error when the requested schema is invalid by @laurenceisla in #4089
+ It now shows the invalid schema in the `message` field.
+ The exposed schemas are now listed in the `hint` instead of the `message` field.
- Improve error details of `PGRST301` error by @taimoorzaeem in #4051
## [13.0.6] - 2025-08-30
### Fixed
- Fix logging the Haskell type instead of the listener error message directly by @laurenceisla in #3588
- Fix format of `IPv6` address logged at PostgREST startup by @taimoorzaeem in #4291
- Fix empty enum in `preferParams` OpenAPI parameter by @laurenceisla in #4292
## [13.0.5] - 2025-08-24
### Fixed
- Fix OpenAPI broken docs link by @taimoorzaeem in #4080
- Fix OpenAPI specification incorrectly exposing GET methods for volatile functions by @joelonsql in #4174
- Fix empty spread embeddings return unexpected SQL error by @taimoorzaeem in #3887
- Fix `/metrics` endpoint not responding with `Content-Type` header by @taimoorzaeem in #4271
## [13.0.4] - 2025-06-17
### Fixed
- Fix regression that makes full-text search not work on domain types based on `tsvector` by @laurenceisla in #4135
- Fix `jwt-aud` config not failing when set to an invalid URI by @taimoorzaeem in #4132
## [13.0.3] - 2025-06-16
- Fix `max-affected` preference not failing with RPC when `handling=strict` by @taimoorzaeem in #4100
- Fix a property definition's type in OpenAPI not showing the correct base type of a recursive domain by @laurenceisla in #4136
### Fixed
## [13.0.2] - 2025-06-02
### Fixed
- Fix regression that makes `ORDER BY` with nulls-order not work alongside limits by @laurenceisla in #4109
## [13.0.1] - 2025-06-01
### Fixed
- Fix jwt error returning HTTP status `400` for invalid role by @taimoorzaeem in #3601
- Fix `db-extra-search-path` cannot be set to nothing by @taimoorzaeem in #4074
+ It can now be disabled by setting it to empty string.
+ Schema Cache load error is now logged including `db-schemas` and `db-extra-search-path` config values.
## [13.0.0] - 2025-05-08
### Added
- #3558, Add the `admin-server-host` config to set the host for the admin server - @develop7
- #3607, Log to stderr when the JWT secret is less than 32 characters long - @laurenceisla
- #2858, Performance improvements when calling RPCs via GET using indexes in more cases - @wolfgangwalther
- #3560, Log resolved host in "Listening on ..." messages - @develop7
- #3727, Log maximum pool size - @steve-chavez
- #1536, Add string comparison feature for jwt-role-claim-key - @taimoorzaeem
- #3747, Allow `not_null` value for the `is` operator - @taimoorzaeem
- #2255, Apply `to_tsvector()` explicitly to the full-text search filtered column (excluding `tsvector` types) - @laurenceisla
- #1578, Log the main SQL query to stderr at the current `log-level` when `log-query=main-query` - @laurenceisla
- #3903, Log connection pool borrows on `log-level=debug` - @taimoorzaeem
- #3041, Allow spreading one-to-many and many-to-many embedded resources - @laurenceisla
+ The selected columns in the embedded resources are aggregated into arrays
+ Aggregates are not supported
- #2967, Add `Proxy-Status` header for better error response - @taimoorzaeem
- #4016, Add `Content-Length` response header - @laurenceisla
### Fixed
- #3693, Prevent spread embedding to allow aggregates when they are disabled - @laurenceisla
- #3693, A nested spread embedding now correctly groups by the fields of its top parent relationship - @laurenceisla
- #3693, Fix spread embedding errors when using the `count()` aggregate without a field - @laurenceisla
+ Fixed `"column reference <col> is ambiguous"` error when selecting `?select=...table(col,count())`
+ Fixed `"column <json_aggregate>.<alias> does not exist"` error when selecting `?select=...table(aias:count())`
- #3727, Clarify "listening" logs - @steve-chavez
- #3795, Clarify `Accept: vnd.pgrst.object` error message - @steve-chavez
- #3697, #3602, Handle queries on non-existing table gracefully - @taimoorzaeem
- #3600, #3926, Improve JWT errors - @taimoorzaeem
- #3013, Fix `order=` with POST, PATCH, PUT and DELETE requests - @taimoorzaeem
- #3965, Fix filter on unselected columns in a table-valued function - @taimoorzaeem
- #4052, Fix schema cache load duplicate objects with different object type but same oid - @taimoorzaeem
### Changed
- #2052, Dropped support for PostgreSQL 9.6 - @wolfgangwalther
- #2052, Dropped support for PostgreSQL 10 - @wolfgangwalther
- #2052, Dropped support for PostgreSQL 11 - @wolfgangwalther
- #3508, PostgREST now fails to start when `server-port` and `admin-server-port` config options are the same - @develop7
- #3607, PostgREST now fails to start when the JWT secret is less than 32 characters long - @laurenceisla
- #3644, Fail schema cache lookup with invalid `db-schemas` or `db-extra-search-path` config - @wolfgangwalther
- Previously, this would silently return 200 - OK on the root endpoint, but don't provide any usable endpoints.
- Note: This also applies when deleting the `public` schema - both config options default to that.
- #3757, Remove support for `Prefer: params=single-object` - @joelonsql
+ This preference was deprecated in favor of Functions with an array of JSON objects
- #3013, Drop support for Limited updates/deletes
+ The feature was complicated and largely unused.
- #3956, Drop `/config` endpoint of admin server - @steve-chavez
+ The endpoint was at risk of being left unprotected when exposing it.
+ The accompanying `admin-server-config-enabled` config was also dropped.
- #3598, PostgREST now validates the `kid` parameter of the JWT - @wolfgangwalther
+ If the JWT contains a ``kid`` parameter, then PostgREST will look for the JSON Web Key in the `jwt-secret`.
+ If the JWT doesn't contain a `kid`, the behavior should be backwards compatible. PostgREST will try each key in the `jwt-secret` one by one until it finds one that works.
- #3697, #3602, Querying non-existent table now returns `PGRST205` error instead of empty json - @taimoorzaeem
- #3600, #3926, Improve JWT errors - @taimoorzaeem
+ Return `PGRST301` error when `Bearer` in auth header is sent empty
+ Diagnostic error messages instead of exposed internals
+ Return new `PGRST303` error when jwt claims decoding fails
- #3906, Return `PGRST125` and `PGRST126` errors instead of empty json - @taimoorzaeem
## [12.2.12] - 2025-05-01
### Fixed
- #3956, Fix exposing admin server `/config` by default - @steve-chavez
+ The above endpoint is now disabled unless the `admin-server-config-enabled` config is set to `true`
## [12.2.11] - 2025-04-22
### Fixed
- #4030, Fix regression with parameter `charset=utf-8` in mediatype - @taimoorzaeem
## [12.2.10] - 2025-04-18
### Fixed
- #3889, Fix: JWT cache purging on every request decreases performance - @mkleczek
## [12.2.9] - 2025-04-16
### Fixed
- #3498, Fix incorrect parsing of the `for` parameter of the `application/vnd.pgrst.plan` media type - @taimoorzaeem
- #4014, Fix JWT cache allows old tokens after the jwt-secret is changed in a config reload - @taimoorzaeem
## [12.2.8] - 2025-02-10
### Fixed
- #3841, Log `503` client error to stderr - @taimoorzaeem
## [12.2.7] - 2025-02-03
### Fixed
- #2524, Fix schema reloading notice on windows - @diogob
## [12.2.6] - 2025-01-29
### Fixed
- #3788, Fix jwt cache does not remove expired entries - @taimoorzaeem
## [12.2.5] - 2025-01-20
### Fixed
- #3867, Fix startup for arm64 docker image - @wolfgangwalther
## [12.2.4] - 2025-01-18
### Fixed
- #3779, Always log the schema cache load time - @steve-chavez
- #3706, Fix insert with `missing=default` uses default value of domain instead of column - @taimoorzaeem
## [12.2.3] - 2024-08-01 ## [12.2.3] - 2024-08-01
### Fixed ### Fixed
+2 -54
View File
@@ -1,55 +1,3 @@
# Contributing to PostgREST This repository follows the same contribution guidelines as the main PostgREST repository contribution guidelines:
**First:** if you're unsure or afraid of _anything_, just ask or https://github.com/PostgREST/postgrest/blob/main/.github/CONTRIBUTING.md
submit the issue or pull request anyways. You won't be yelled at
for giving your best effort. The worst that can happen is that
you'll be politely asked to change something. We appreciate any
sort of contributions, and don't want a wall of rules to get in the
way of that.
However, for those individuals who want a bit more guidance on the
best way to contribute to the project, read on. This document will
cover what we're looking for. By addressing all the points we're
looking for, it raises the chances we can quickly merge or address
your contributions.
## Issues
For questions on how to use PostgREST, please use
[GitHub discussions](https://github.com/PostgREST/postgrest/discussions).
### Reporting an Issue
* Make sure you test against the latest [stable release](https://github.com/PostgREST/postgrest/releases/latest)
and also against the latest [devel release](https://github.com/PostgREST/postgrest/releases/tag/devel).
It is possible we already fixed the bug you're experiencing.
* Provide steps to reproduce the issue, including your OS version and
the specific database schema that you are using.
* Please include SQL logs for issues involving runtime problems. To obtain logs first
[enable logging all statements](http://www.microhowto.info/howto/log_all_queries_to_a_postgresql_server.html),
then [find your logs](http://blog.endpoint.com/2014/11/dear-postgresql-where-are-my-logs.html).
* If your database schema has changed while the PostgREST server is running,
[send the server a `SIGUSR1` signal](http://postgrest.org/en/latest/admin.html#schema-reloading) or restart it to ensure the schema cache
is not stale. This sometimes fixes apparent bugs.
## Code
We have a fully nix-based development environment with many tools for a smooth development workflow available.
Check the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md) on how to set it up and use it.
### Haskell Conventions
* All contributions must pass the tests before being merged. When
you create a pull request your code will automatically be tested.
* All code must also pass [hlint](http://community.haskell.org/~ndm/hlint/) and [stylish-haskell](https://github.com/jaspervdj/stylish-haskell)
with no warnings. This helps enforce a uniform style for all committers. Continuous integration will check this as well on every
pull request. There are useful tools in the nix-shell that help with checking this locally. You can run `postgrest-check` to do this manually but
we recommend adding it to `.git/hooks/pre-commit` as `nix-shell --run postgrest-check` to automatically check this before doing a commit.
### Running Tests
For instructions on running tests, see the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md#testing).
+15 -14
View File
@@ -22,26 +22,31 @@ API than you are likely to write from scratch.
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank"> <a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/supabase.svg"> <img width="296px" src="static/gnuhost.png">
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://www.euronodes.com/postgrest" target="_blank"> <a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/euronodes.svg"> <img width="296px" src="static/neon.jpg">
</a> </a>
</td> </td>
</tr> </tr>
<tr></tr> <tr></tr>
<tr> <tr>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank"> <a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/neon.jpg"> <img width="296px" src="static/code-build.png">
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://www.bytebase.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank"> <a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank">
<img width="296px" src="static/bytebase.svg"> <img width="296px" src="static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
</a> </a>
</td> </td>
</tr> </tr>
@@ -52,8 +57,8 @@ Big thanks to our sponsors! You can join them by supporting PostgREST on [Patreo
## Usage ## Usage
1. See the docs for [how to install PostgREST on your platform](https://docs.postgrest.org/en/stable/explanations/install.html). You can also [use Docker](https://docs.postgrest.org/en/stable/explanations/install.html#docker). 1. Download the binary ([latest release](https://github.com/PostgREST/postgrest/releases/latest))
for your platform.
2. Invoke for help: 2. Invoke for help:
```bash ```bash
@@ -142,10 +147,6 @@ You can help PostgREST ongoing maintenance and development by making a regular d
Every donation will be spent on making PostgREST better for the whole community. Every donation will be spent on making PostgREST better for the whole community.
## Contributing
Contributions are always welcome and appreciated. Please see the [Contributing guidelines](https://github.com/PostgREST/postgrest/blob/main/CONTRIBUTING.md).
## Thanks ## Thanks
The PostgREST organization is grateful to: The PostgREST organization is grateful to:
+2
View File
@@ -1,2 +1,4 @@
packages: postgrest.cabal packages: postgrest.cabal
tests: true tests: true
package *
ghc-options: -split-sections
+1 -1
View File
@@ -1 +1 @@
index-state: hackage.haskell.org 2026-08-10T16:58:32Z index-state: hackage.haskell.org 2024-05-17T23:41:49Z
+39 -45
View File
@@ -3,15 +3,7 @@
, compiler ? "ghc948" , compiler ? "ghc948"
, # Commit of the Nixpkgs repository that we want to use. , # Commit of the Nixpkgs repository that we want to use.
# It defaults to reading the inputs from flake.lock, which serves nixpkgsVersion ? import nix/nixpkgs-version.nix
# as a compatibility layer for non-flake builds / default.nix / shell.nix.
nixpkgsVersion ? let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
{
inherit (lock.nodes.nixpkgs.locked) owner repo rev;
tarballHash = lock.nodes.nixpkgs.locked.narHash;
}
, # Nix files that describe the Nixpkgs repository. We evaluate the expression , # Nix files that describe the Nixpkgs repository. We evaluate the expression
# using `import` below. # using `import` below.
@@ -27,7 +19,7 @@ let
"postgrest"; "postgrest";
# PostgREST source files, filtered based on the rules in the .gitignore files # PostgREST source files, filtered based on the rules in the .gitignore files
# and file extensions. We want to include as little as possible, as the files # and file extensions. We want to include as litte as possible, as the files
# added here will increase the space used in the Nix store and trigger the # added here will increase the space used in the Nix store and trigger the
# build of new Nix derivations when changed. # build of new Nix derivations when changed.
src = src =
@@ -43,6 +35,10 @@ let
allOverlays.build-toolbox allOverlays.build-toolbox
allOverlays.checked-shell-script allOverlays.checked-shell-script
allOverlays.gitignore allOverlays.gitignore
allOverlays.postgresql-libpq
allOverlays.postgresql-legacy
allOverlays.postgresql-future
allOverlays.postgis
(allOverlays.haskell-packages { inherit compiler; }) (allOverlays.haskell-packages { inherit compiler; })
allOverlays.slocat allOverlays.slocat
]; ];
@@ -53,23 +49,19 @@ let
postgresqlVersions = postgresqlVersions =
[ [
{ name = "pg-17"; postgresql = pkgs.postgresql_17.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "postgresql-16"; postgresql = pkgs.postgresql_16.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-16"; postgresql = pkgs.postgresql_16.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "postgresql-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "postgresql-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "postgresql-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "pg-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); } { name = "postgresql-12"; postgresql = pkgs.postgresql_12.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-11"; postgresql = pkgs.postgresql_11.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-10"; postgresql = pkgs.postgresql_10.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-9_6"; postgresql = pkgs.postgresql_9_6.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
]; ];
haskellPackages = pkgs.haskell.packages."${compiler}";
# Dynamic derivation for PostgREST # Dynamic derivation for PostgREST
postgrest = pkgs.lib.pipe (haskellPackages.callCabal2nix name src { }) [ postgrest =
# To allow ghc-datasize to be used. pkgs.haskell.packages."${compiler}".callCabal2nix name src { };
lib.disableLibraryProfiling
# We are never going to use dynamic haskell libraries anyway. "Dynamic" refers to how
# non-haskell deps are linked. All haskell dependencies are always statically linked.
lib.disableSharedLibraries
];
staticHaskellPackage = import nix/static.nix { inherit compiler name pkgs src; }; staticHaskellPackage = import nix/static.nix { inherit compiler name pkgs src; };
@@ -77,6 +69,15 @@ let
devCabalOptions = devCabalOptions =
"-f dev --test-show-detail=direct"; "-f dev --test-show-detail=direct";
profiledHaskellPackages =
pkgs.haskell.packages."${compiler}".extend (_: super:
{
mkDerivation =
args:
super.mkDerivation (args // { enableLibraryProfiling = true; });
}
);
inherit (pkgs.haskell) lib; inherit (pkgs.haskell) lib;
in in
rec { rec {
@@ -84,21 +85,17 @@ rec {
# Derivation for the PostgREST Haskell package, including the executable, # Derivation for the PostgREST Haskell package, including the executable,
# libraries and documentation. We disable running the test suite on Nix # libraries and documentation. We disable running the test suite on Nix
# builds, as they require a database to be set up. We split the binary # builds, as they require a database to be set up.
# into a separate output, so that the default distribution via flake.nix postgrestPackage =
# has a much smaller closure size. lib.dontCheck postgrest;
postgrestPackage = pkgs.lib.pipe postgrest [
lib.dontCheck
lib.enableSeparateBinOutput
(haskellPackages.generateOptparseApplicativeCompletions [ "postgrest" ])
];
# Profiled dynamic executable. # Profiled dynamic executable.
postgrestProfiled = pkgs.lib.pipe postgrestPackage [ postgrestProfiled =
lib.enableExecutableProfiling lib.enableExecutableProfiling (
lib.enableLibraryProfiling lib.dontHaddock (
lib.dontHaddock lib.dontCheck (profiledHaskellPackages.callCabal2nix name src { })
]; )
);
inherit (postgrest) env; inherit (postgrest) env;
@@ -108,9 +105,6 @@ rec {
inherit (pkgs.haskell.packages."${compiler}") ghcWithPackages; inherit (pkgs.haskell.packages."${compiler}") ghcWithPackages;
}; };
# Used by CI on MacOS
inherit (pkgs) nix-build-uncached;
### Tools ### Tools
cabalTools = cabalTools =
@@ -127,21 +121,21 @@ rec {
docs = docs =
pkgs.callPackage nix/tools/docs.nix { }; pkgs.callPackage nix/tools/docs.nix { };
# Git tools.
gitTools =
pkgs.callPackage nix/tools/gitTools.nix { };
# Load testing tools. # Load testing tools.
loadtest = loadtest =
pkgs.callPackage nix/tools/loadtest.nix { inherit withTools; }; pkgs.callPackage nix/tools/loadtest.nix { inherit withTools; };
# Script for running memory tests.
memory =
pkgs.callPackage nix/tools/memory.nix { inherit postgrestProfiled withTools; };
# Utility for updating the pinned version of Nixpkgs. # Utility for updating the pinned version of Nixpkgs.
nixpkgsTools = nixpkgsTools =
pkgs.callPackage nix/tools/nixpkgsTools.nix { }; pkgs.callPackage nix/tools/nixpkgsTools.nix { };
# Scripts for publishing new releases. # Scripts for publishing new releases.
release = release =
pkgs.callPackage nix/tools/release.nix { }; pkgs.callPackage nix/tools/release { };
# Linting and styling tools. # Linting and styling tools.
style = style =
+14 -9
View File
@@ -19,26 +19,31 @@ write from scratch.
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank"> <a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/supabase.svg"> <img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/gnuhost.png">
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://www.euronodes.com/postgrest" target="_blank"> <a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/euronodes.svg"> <img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/neon.jpg">
</a> </a>
</td> </td>
</tr> </tr>
<tr></tr> <tr></tr>
<tr> <tr>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank"> <a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/neon.jpg"> <img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/code-build.png">
</a> </a>
</td> </td>
<td align="center" valign="middle"> <td align="center" valign="middle">
<a href="https://www.bytebase.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank"> <a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/bytebase.svg"> <img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/tembo.png">
</a> </a>
</td> </td>
</tr> </tr>
@@ -59,7 +64,7 @@ The image is built from scratch using
[Nix](https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools) instead of a [Nix](https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools) instead of a
`Dockerfile`, which yields a highly secure and optimized image. This is also why `Dockerfile`, which yields a highly secure and optimized image. This is also why
no commands are listed in the image history. See the [PostgREST no commands are listed in the image history. See the [PostgREST
repository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for respository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for
details on the build process and how to inspect the image. details on the build process and how to inspect the image.
This does not apply to the arm64 variant, which is based on Ubuntu. This does not apply to the arm64 variant, which is based on Ubuntu.
+5 -17
View File
@@ -40,14 +40,9 @@ database "PostgreSQL" {
:user: :user:
hexagon Proxy hexagon Proxy
:user: .r-> Proxy : request with JWT :user: .r-> Proxy
HTTPAPI <.l- Proxy HTTPAPI <.l- Proxy
hexagon ExternalAuth
ExternalAuth -u[hidden]- Proxy
:user: .r-> ExternalAuth : login
:user: <.r- ExternalAuth : JWT
:operator: .d-> HTTPADMIN :operator: .d-> HTTPADMIN
:operator: .d-> CLI :operator: .d-> CLI
@@ -56,18 +51,15 @@ PostgreSQL <.developer : "\t"
Listener -r.> "PostgreSQL" Listener -r.> "PostgreSQL"
"Connection Pool" -r.> "PostgreSQL" : "\t\t" "Connection Pool" -r.> "PostgreSQL" : "\t\t"
note bottom of Auth note bottom of Auth
Validates the JWT Authenticates the user request
end note end note
note bottom of ApiRequest note bottom of ApiRequest
Parses the URL syntax Parses the URL syntax
end note end note
note bottom of Plan
Generates internal AST
end note
note bottom of Query note bottom of Query
Generates the SQL Generates the SQL
end note end note
@@ -76,19 +68,15 @@ note top of Listener
LISTEN session LISTEN session
end note end note
url of ExternalAuth is [[../explanations/external_auth.html]]
url of Admin is [[../references/admin_server.html#admin-server]] url of Admin is [[../references/admin_server.html#admin-server]]
url of API is [[../explanations/schema_isolation.html]] url of API is [[../explanations/schema_isolation.html]]
url of Auth is [[../references/auth.html#authn]] url of Auth is [[../references/auth.html#authn]]
url of ApiRequest is [[../explanations/architecture.html#api-request]]
url of Plan is [[../explanations/architecture.html#plan]]
url of Query is [[../explanations/architecture.html#query]]
url of Authorization is [[../explanations/db_authz.html]] url of Authorization is [[../explanations/db_authz.html]]
url of CLI is [[../references/cli.html#cli]] url of CLI is [[../references/cli.html#cli]]
url of "Connection Pool" is [[../references/connection_pool.html]] url of "Connection Pool" is [[../references/connection_pool.html]]
url of Config is [[../references/configuration.html#configuration]] url of Config is [[../references/configuration.html#configuration]]
url of HTTPADMIN is [[../explanations/architecture.html#http]] url of HTTPADMIN is [[https://aosabook.org/en/posa/warp.html]]
url of HTTPAPI is [[../explanations/architecture.html#http]] url of HTTPAPI is [[https://aosabook.org/en/posa/warp.html]]
url of Listener is [[../references/listener.html#listener]] url of Listener is [[../references/listener.html#listener]]
url of Proxy is [[../explanations/nginx.html]] url of Proxy is [[../explanations/nginx.html]]
url of "Schema Cache" is [[../references/schema_cache.html#schema-cache]] url of "Schema Cache" is [[../references/schema_cache.html#schema-cache]]
+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 28 KiB

After

Width:  |  Height:  |  Size: 23 KiB

+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 24 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 156 B

After

Width:  |  Height:  |  Size: 468 B

+1 -1
View File
@@ -1 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentStyleType="text/css" height="391px" preserveAspectRatio="none" style="width:328px;height:391px;" version="1.1" viewBox="0 0 328 391" width="328px" zoomAndPan="magnify"><defs/><g><!--cluster PostgreSQL--><g id="cluster_PostgreSQL"><path d="M6,16 C6,6 158.5,6 158.5,6 C158.5,6 311,6 311,16 L311,293.59 C311,303.59 158.5,303.59 158.5,303.59 C158.5,303.59 6,303.59 6,293.59 L6,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><path d="M6,16 C6,26 158.5,26 158.5,26 C158.5,26 311,26 311,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="92.2305" x="112.3848" y="40.9951">PostgreSQL</text></g><!--cluster public--><g id="cluster_public"><polygon fill="none" points="30,74,40,64,153,64,153,146.29,143,156.29,30,156.29,30,74" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="143" x2="153" y1="74" y2="64"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="30" x2="143" y1="74" y2="74"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="143" x2="143" y1="74" y2="156.29"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="47.9063" x="63.5469" y="89.9951">public</text></g><!--cluster API--><g id="cluster_API"><polygon fill="none" points="70,190.29,80,180.29,246,180.29,246,269.59,236,279.59,70,279.59,70,190.29" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="236" x2="246" y1="190.29" y2="180.29"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="70" x2="236" y1="190.29" y2="190.29"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="236" x2="236" y1="190.29" y2="279.59"/><text fill="#FFFFFF" font-family="sans-serif" font-size="20" font-weight="bold" lengthAdjust="spacing" textLength="34.668" x="136.666" y="211.8545">api</text></g><!--entity tables_public--><g id="elem_tables_public"><rect fill="#313139" height="36.2969" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="62.752" x="71.62" y="104"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="42.752" x="81.62" y="126.9951">tables</text></g><!--entity extensions--><g id="elem_extensions"><polygon fill="none" points="169.14,109,179.14,99,294.8695,99,294.8695,135.2969,284.8695,145.2969,169.14,145.2969,169.14,109" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="284.8695" x2="294.8695" y1="109" y2="99"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="169.14" x2="284.8695" y1="109" y2="109"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="284.8695" x2="284.8695" y1="109" y2="145.2969"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="85.7295" x="184.14" y="131.9951">extensions</text></g><!--entity vf_api--><g id="elem_vf_api"><rect fill="#313139" height="36.2969" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="144.6465" x="85.68" y="227.29"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="124.6465" x="95.68" y="250.2851">views + functions</text></g><!--entity PostgREST--><g id="elem_PostgREST"><ellipse cx="158" cy="352.59" fill="#313139" rx="8" ry="8" style="stroke:#E7E7E7;stroke-width:0.5;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="74.6895" x="120.6553" y="382.5851">PostgREST</text></g><!--reverse link tables_public to vf_api--><g id="link_tables_public_vf_api"><path d="M110.03,146.6 C110.03,169.85 110.03,203.55 110.03,226.86 " fill="none" id="tables_public-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="110.03,140.6,106.03,149.6,110.03,145.6,114.03,149.6,110.03,140.6" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--reverse link extensions to vf_api--><g id="link_extensions_vf_api"><path d="M199.73,151.63 C199.73,175.24 199.73,205.18 199.73,226.88 " fill="none" id="extensions-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="199.73,145.63,195.73,154.63,199.73,150.63,203.73,154.63,199.73,145.63" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--link vf_api to PostgREST--><g id="link_vf_api_PostgREST"><path d="M158,269.62 C158,292.9 158,320.34 158,337.81 " fill="none" id="vf_api-PostgREST" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="158,263.62,154,272.62,158,268.62,162,272.62,158,263.62" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="158,343.81,162,334.81,158,338.81,154,334.81,158,343.81" style="stroke:#E7E7E7;stroke-width:3.0;"/></g><!--SRC=[KypCIyufJKbLqDFJBqxEqCqipjShpSq10000]--></g></svg> <?xml version="1.0" encoding="us-ascii" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentStyleType="text/css" height="411px" preserveAspectRatio="none" style="width:315px;height:411px;" version="1.1" viewBox="0 0 315 411" width="315px" zoomAndPan="magnify"><defs/><g><!--cluster PostgreSQL--><g id="cluster_PostgreSQL"><path d="M6,16 C6,6 152,6 152,6 C152,6 298,6 298,16 L298,309.14 C298,319.14 152,319.14 152,319.14 C152,319.14 6,319.14 6,309.14 L6,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><path d="M6,16 C6,26 152,26 152,26 C152,26 298,26 298,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="81.375" x="111.3125" y="42.9659">PostgreSQL</text></g><!--cluster public--><g id="cluster_public"><polygon fill="none" points="30,77,40,67,150,67,150,155.07,140,165.07,30,165.07,30,77" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="140" x2="150" y1="77" y2="67"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="30" x2="140" y1="77" y2="77"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="140" x2="140" y1="77" y2="165.07"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="44.625" x="63.6875" y="94.9659">public</text></g><!--cluster API--><g id="cluster_API"><polygon fill="none" points="68,199.07,78,189.07,236,189.07,236,285.14,226,295.14,68,295.14,68,199.07" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="226" x2="236" y1="199.07" y2="189.07"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="68" x2="226" y1="199.07" y2="199.07"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="226" x2="226" y1="199.07" y2="295.14"/><text fill="#FFFFFF" font-family="sans-serif" font-size="20" font-weight="bold" lengthAdjust="spacing" textLength="29.875" x="133.0625" y="223.45">api</text></g><!--entity tables_public--><g id="elem_tables_public"><rect fill="#313139" height="39.0679" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="61" x="69.5" y="110"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="41" x="79.5" y="134.9659">tables</text></g><!--entity extensions--><g id="elem_extensions"><polygon fill="none" points="165.81,115,175.81,105,282.185,105,282.185,144.0679,272.185,154.0679,165.81,154.0679,165.81,115" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="272.185" x2="282.185" y1="115" y2="105"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="165.81" x2="272.185" y1="115" y2="115"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="272.185" x2="272.185" y1="115" y2="154.0679"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="76.375" x="180.81" y="139.9659">extensions</text></g><!--entity vf_api--><g id="elem_vf_api"><rect fill="#313139" height="39.0679" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="136" x="84" y="240.07"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="116" x="94" y="265.0359">views + functions</text></g><!--entity PostgREST--><g id="elem_PostgREST"><ellipse cx="152" cy="371.14" fill="#313139" rx="8" ry="8" style="stroke:#E7E7E7;stroke-width:0.5;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="70" x="117" y="403.1059">PostgREST</text></g><!--reverse link tables_public to vf_api--><g id="link_tables_public_vf_api"><path d="M107.25,155.29 C107.25,179.74 107.25,215.13 107.25,239.67 " fill="none" id="tables_public-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="107.25,149.29,103.25,158.29,107.25,154.29,111.25,158.29,107.25,149.29" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--reverse link extensions to vf_api--><g id="link_extensions_vf_api"><path d="M192.91,160.31 C192.91,185.07 192.91,216.68 192.91,239.61 " fill="none" id="extensions-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="192.91,154.31,188.91,163.31,192.91,159.31,196.91,163.31,192.91,154.31" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--link vf_api to PostgREST--><g id="link_vf_api_PostgREST"><path d="M152,285.55 C152,309.98 152,338.34 152,356.18 " fill="none" id="vf_api-PostgREST" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="152,279.55,148,288.55,152,284.55,156,288.55,152,279.55" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="152,362.18,156,353.18,152,357.18,148,353.18,152,362.18" style="stroke:#E7E7E7;stroke-width:3.0;"/></g><!--SRC=[KypCIyufJKbLqDFJBqxEqCqipjShpSq10000]--></g></svg>

Before

Width:  |  Height:  |  Size: 4.8 KiB

After

Width:  |  Height:  |  Size: 4.8 KiB

+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 5.1 KiB

After

Width:  |  Height:  |  Size: 5.1 KiB

+5 -14
View File
@@ -12,6 +12,7 @@
# All configuration values have a default; values that are commented out # All configuration values have a default; values that are commented out
# serve to show the default. # serve to show the default.
import sys
import os import os
# If extensions (or modules to document with autodoc) are in another directory, # If extensions (or modules to document with autodoc) are in another directory,
@@ -49,7 +50,7 @@ source_suffix = ".rst"
master_doc = "index" master_doc = "index"
# This is overriden by readthedocs with the version tag anyway # This is overriden by readthedocs with the version tag anyway
version = "14" version = "12.2"
# To avoid repetition in <title> we set this to an empty string. # To avoid repetition in <title> we set this to an empty string.
release = "" release = ""
@@ -113,7 +114,7 @@ html_theme = "sphinx_rtd_theme"
# Theme options are theme-specific and customize the look and feel of a theme # Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the # further. For a list of options available for each theme, see the
# documentation. # documentation.
html_theme_options = {} html_theme_options = {"display_version": False}
# Add any paths that contain custom themes here, relative to this directory. # Add any paths that contain custom themes here, relative to this directory.
# html_theme_path = [] # html_theme_path = []
@@ -291,22 +292,12 @@ def setup(app):
app.add_css_file("css/custom.css") app.add_css_file("css/custom.css")
user_agent = ( # taken from https://github.com/sphinx-doc/sphinx/blob/82dad44e5bd3776ecb6fd8ded656bc8151d0e63d/sphinx/util/requests.py#L42
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:135.0) Gecko/20100101 Firefox/135.0" user_agent = "Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0"
)
linkcheck_ignore = [ linkcheck_ignore = [
# 403 only in CI / GitHub Actions
r"https://www.patreon.com/postgrest", r"https://www.patreon.com/postgrest",
r"https://blog.frankel.ch/poor-man-api", r"https://blog.frankel.ch/poor-man-api",
r"https://www.cybertec-postgresql.com/.*",
# Odd SSL error
r"https://www.dripdepot.com",
r"https://www.euronodes.com",
# New GitHub UI delays comment load, so anchor fails
r"https://github.com/.*#issuecomment",
# Random 500 Internal Server Error
r"https://jwt.io",
] ]
# sphinx-tabs configuration # sphinx-tabs configuration
+5 -3
View File
@@ -6,7 +6,10 @@ Community Tutorials
* `Building a Contacts List with PostgREST and Vue.js <https://www.youtube.com/watch?v=iHtsALtD5-U>`_ - * `Building a Contacts List with PostgREST and Vue.js <https://www.youtube.com/watch?v=iHtsALtD5-U>`_ -
In this video series, DigitalOcean shows how to build and deploy an Nginx + PostgREST(using a managed PostgreSQL database) + Vue.js webapp in an Ubuntu server droplet. In this video series, DigitalOcean shows how to build and deploy an Nginx + PostgREST(using a managed PostgreSQL database) + Vue.js webapp in an Ubuntu server droplet.
* `PostgREST + Auth0: Create REST API in minutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest-1-introduction/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service. * `PostgREST + Auth0: Create REST API in mintutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service.
* `PostgREST + PostGIS API tutorial in 5 minutes <https://gis-ops.com/postgrest-postgis-api-tutorial-geospatial-api-in-5-minutes/>`_ -
In this tutorial, GIS • OPS shows how to perform PostGIS calculations through PostgREST :ref:`functions` interface.
* `"CodeLess" backend using postgres, postgrest and oauth2 authentication with keycloak <https://www.mathieupassenaud.fr/codeless_backend/>`_ - * `"CodeLess" backend using postgres, postgrest and oauth2 authentication with keycloak <https://www.mathieupassenaud.fr/codeless_backend/>`_ -
A step-by-step tutorial for using PostgREST with KeyCloak(hosted on a managed service). A step-by-step tutorial for using PostgREST with KeyCloak(hosted on a managed service).
@@ -34,7 +37,6 @@ Templates
Example Apps Example Apps
------------ ------------
* `archtika <https://github.com/thiloho/archtika>`_ - self-hosted CMS
* `delibrium-postgrest <https://gitlab.com/delibrium/delibrium-postgrest/>`_ - example school API and front-end in Vue.js * `delibrium-postgrest <https://gitlab.com/delibrium/delibrium-postgrest/>`_ - example school API and front-end in Vue.js
* `ETH-transactions-storage <https://github.com/Adamant-im/ETH-transactions-storage>`_ - indexer for Ethereum to get transaction list by ETH address * `ETH-transactions-storage <https://github.com/Adamant-im/ETH-transactions-storage>`_ - indexer for Ethereum to get transaction list by ETH address
* `general <https://github.com/PierreRochard/general>`_ - example auth back-end * `general <https://github.com/PierreRochard/general>`_ - example auth back-end
@@ -88,7 +90,7 @@ Client-Side Libraries
* `postgrest-go <https://github.com/supabase-community/postgrest-go>`_ - Go * `postgrest-go <https://github.com/supabase-community/postgrest-go>`_ - Go
* `postgrest-js <https://github.com/supabase/postgrest-js>`_ - TypeScript/JavaScript * `postgrest-js <https://github.com/supabase/postgrest-js>`_ - TypeScript/JavaScript
* `postgrest-kt <https://github.com/supabase-community/postgrest-kt>`_ - Kotlin * `postgrest-kt <https://github.com/supabase-community/postgrest-kt>`_ - Kotlin
* `postgrest-py <https://github.com/supabase/postgrest-py>`_ - Python * `postgrest-py <https://github.com/supabase-community/postgrest-py>`_ - Python
* `postgrest-rs <https://github.com/supabase-community/postgrest-rs>`_ - Rust * `postgrest-rs <https://github.com/supabase-community/postgrest-rs>`_ - Rust
* `postgrest-swift <https://github.com/supabase-community/postgrest-swift>`_ - Swift * `postgrest-swift <https://github.com/supabase-community/postgrest-swift>`_ - Swift
* `redux-postgrest <https://github.com/andytango/redux-postgrest>`_ - TypeScript/JS, client integrated with (React) Redux. * `redux-postgrest <https://github.com/andytango/redux-postgrest>`_ - TypeScript/JS, client integrated with (React) Redux.
+11 -11
View File
@@ -31,58 +31,58 @@ This section talks briefly about various important modules.
Main Main
---- ----
The starting point of the program is `Main.hs <https://github.com/PostgREST/postgrest/blob/v14/main/Main.hs>`_. The starting point of the program is `Main.hs <https://github.com/PostgREST/postgrest/blob/main/main/Main.hs>`_.
CLI CLI
--- ---
Main then calls `CLI.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/CLI.hs>`_, which is in charge of :ref:`cli`. Main then calls `CLI.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/CLI.hs>`_, which is in charge of :ref:`cli`.
App App
--- ---
`App.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/App.hs>`_ is then in charge of composing the different modules. `App.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/App.hs>`_ is then in charge of composing the different modules.
Auth Auth
---- ----
`Auth.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Auth.hs>`_ is in charge of :ref:`authn`. `Auth.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Auth.hs>`_ is in charge of :ref:`authn`.
Api Request Api Request
----------- -----------
`ApiRequest.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/ApiRequest.hs>`_ is in charge of parsing the URL query string (following PostgREST syntax), the request headers, and the request body. `ApiRequest.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/ApiRequest.hs>`_ is in charge of parsing the URL query string (following PostgREST syntax), the request headers, and the request body.
A request might be rejected at this level if it's invalid. For example when providing an unknown media type to PostgREST or using an unknown HTTP method. A request might be rejected at this level if it's invalid. For example when providing an unknown media type to PostgREST or using an unknown HTTP method.
Plan Plan
---- ----
Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Plan.hs>`_ generates an internal AST, filling out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request. Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Plan.hs>`_ fills in out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request.
A request might be rejected at this level if it's invalid. For example when doing resource embedding on a nonexistent resource. A request might be rejected at this level if it's invalid. For example when doing resource embedding on a nonexistent resource.
Query Query
----- -----
`Query.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Query.hs>`_ generates the SQL queries (parametrized and prepared) required to satisfy the user request. `Query.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Query.hs>`_ generates the SQL queries (parametrized and prepared) required to satisfy the user request.
Only at this stage a connection from the pool might be used. Only at this stage a connection from the pool might be used.
Schema Cache Schema Cache
------------ ------------
`SchemaCache.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/SchemaCache.hs>`_ is in charge of :ref:`schema_cache`. `SchemaCache.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/SchemaCache.hs>`_ is in charge of :ref:`schema_cache`.
Config Config
------ ------
`Config.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Config.hs>`_ is in charge of :ref:`configuration`. `Config.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Config.hs>`_ is in charge of :ref:`configuration`.
Admin Admin
----- -----
`Admin.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Admin.hs>`_ is in charge of the :ref:`admin_server`. `Admin.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Admin.hs>`_ is in charge of the :ref:`admin_server`.
HTTP HTTP
---- ----
@@ -92,4 +92,4 @@ The HTTP server is provided by `Warp <https://aosabook.org/en/posa/warp.html>`_.
Listener Listener
-------- --------
`Listener.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Listener.hs>`_ is in charge of the :ref:`listener`. `Listener.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Listener.hs>`_ is in charge of the :ref:`listener`.
+2 -2
View File
@@ -13,9 +13,9 @@ A role can be thought of as either a database user, or a group of database users
Roles for Each Web User Roles for Each Web User
----------------------- -----------------------
PostgREST can accommodate either viewpoint. If you treat a role as a single user then :ref:`user_impersonation` does most of what you need. When an authenticated user makes a request PostgREST will switch into the database role for that user, which in addition to restricting queries, is available to SQL through the :code:`current_user` variable. PostgREST can accommodate either viewpoint. If you treat a role as a single user then the :ref:`jwt_impersonation` does most of what you need. When an authenticated user makes a request PostgREST will switch into the database role for that user, which in addition to restricting queries, is available to SQL through the :code:`current_user` variable.
You can use row-level security to flexibly restrict visibility and access for the current user. Here is an `example <https://www.enterprisedb.com:443/blog/application-users-vs-row-level-security>`_ from Tomas Vondra, a chat table storing messages sent between users. Users can insert rows into it to send messages to other users, and query it to see messages sent to them by other users. You can use row-level security to flexibly restrict visibility and access for the current user. Here is an `example <https://www.2ndquadrant.com/en/blog/application-users-vs-row-level-security/>`_ from Tomas Vondra, a chat table storing messages sent between users. Users can insert rows into it to send messages to other users, and query it to see messages sent to them by other users.
.. code-block:: postgres .. code-block:: postgres
+7 -7
View File
@@ -16,11 +16,16 @@ Supported PostgreSQL versions
============================= =============================
=============== ================================= =============== =================================
**Supported** PostgreSQL >= 13 **Supported** PostgreSQL >= 9.6
=============== ================================= =============== =================================
PostgREST works with all PostgreSQL versions still `officially supported <https://www.postgresql.org/support/versioning/>`_. PostgREST works with all PostgreSQL versions starting from 9.6.
.. note::
Support for PostgreSQL versions 9.6, 10 and 11 is deprecated. From this on version onwards, PostgREST will only support non-end-of-life PostgreSQL versions.
See https://www.postgresql.org/support/versioning/.
Running PostgREST Running PostgREST
================= =================
@@ -75,12 +80,8 @@ You can get the `official PostgREST Docker image <https://hub.docker.com/r/postg
.. code-block:: bash .. code-block:: bash
# pull the latest version
docker pull postgrest/postgrest docker pull postgrest/postgrest
# to pull a particular version, use one of the versions on https://hub.docker.com/r/postgrest/postgrest/tags
docker pull postgrest/postgrest:<version>
To configure the container image, use :ref:`env_variables_config`. To configure the container image, use :ref:`env_variables_config`.
There are two ways to run the PostgREST container: with an existing external database, or through docker-compose. There are two ways to run the PostgREST container: with an existing external database, or through docker-compose.
@@ -146,7 +147,6 @@ To avoid having to install the database at all, you can run both it and the serv
ports: ports:
- "3000:3000" - "3000:3000"
environment: environment:
PGRST_SERVER_HOST: 0.0.0.0 # necessary for `postgrest --ready` flag to work
PGRST_DB_URI: postgres://app_user:password@db:5432/app_db PGRST_DB_URI: postgres://app_user:password@db:5432/app_db
PGRST_OPENAPI_SERVER_PROXY_URI: http://127.0.0.1:3000 PGRST_OPENAPI_SERVER_PROXY_URI: http://127.0.0.1:3000
depends_on: depends_on:
@@ -43,7 +43,7 @@ As in :ref:`sql_user_management`, we create the :code:`pgcrypto` and :code:`pgjw
CREATE EXTENSION pgcrypto WITH SCHEMA ext_pgcrypto; CREATE EXTENSION pgcrypto WITH SCHEMA ext_pgcrypto;
Concerning the `pgjwt extension <https://github.com/michelp/pgjwt>`_, please cf. to :ref:`jwt-from-sql`. Concerning the `pgjwt extension <https://github.com/michelp/pgjwt>`_, please cf. to :ref:`client_auth`.
.. code-block:: postgres .. code-block:: postgres
@@ -157,7 +157,7 @@ Here we use the username instead of the email address to identify a user.
Logins Logins
~~~~~~ ~~~~~~
As described in :ref:`jwt-from-sql`, we'll create a JWT token inside our login function. Note that you'll need to adjust the secret key which is hard-coded in this example to a secure (at least thirty-two character) secret of your choosing. As described in :ref:`client_auth`, we'll create a JWT token inside our login function. Note that you'll need to adjust the secret key which is hard-coded in this example to a secure (at least thirty-two character) secret of your choosing.
.. code-block:: postgres .. code-block:: postgres
+1 -3
View File
@@ -3,7 +3,7 @@
SQL User Management SQL User Management
=================== ===================
As mentioned on :ref:`jwt_generation`, an external service can provide user management and coordinate with the PostgREST server using JWT. It's also possible to support logins entirely through SQL. It's a fair bit of work, so get ready. As mentioned on :ref:`jwt_generation`, an external service can provide user management and coordinate with the PostgREST server using JWT. Its also possible to support logins entirely through SQL. Its a fair bit of work, so get ready.
Storing Users and Passwords Storing Users and Passwords
--------------------------- ---------------------------
@@ -110,8 +110,6 @@ Then, add ``db-anon-role`` to the configuration file to allow anonymous requests
db-anon-role = "anon" db-anon-role = "anon"
.. _jwt-from-sql:
JWT from SQL JWT from SQL
~~~~~~~~~~~~ ~~~~~~~~~~~~
@@ -318,6 +318,144 @@ You can insert a new product using a JSON object for the ``extra_info`` column:
To query and filter the data see :ref:`json_columns` for a complete reference. To query and filter the data see :ref:`json_columns` for a complete reference.
.. _ww_postgis:
PostGIS
-------
You can use the string representation for `PostGIS <https://postgis.net/>`_ data types such as ``geometry`` or ``geography`` (you need to `install PostGIS <https://postgis.net/documentation/getting_started/>`_ first).
.. code-block:: postgres
-- Activate the postgis module in the current database
create extension if not exists postgis;
create table coverage (
id int primary key,
name text unique,
area geometry
);
To add areas in polygon format, you can use string representation:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 1, "name": "small", "area": "SRID=4326;POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" },
{ "id": 2, "name": "big", "area": "SRID=4326;POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))" }
]
EOF
Now, when you request the information, PostgREST will automatically cast the ``area`` column into a ``Polygon`` geometry type. Although this is useful, you may need the whole output to be in `GeoJSON <https://geojson.org/>`_ format out of the box, which can be done by including the ``Accept: application/geo+json`` in the request. This will work for PostGIS versions 3.0.0 and up and will return the output as a `FeatureCollection Object <https://www.rfc-editor.org/rfc/rfc7946#section-3.3>`_:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-H "Accept: application/geo+json"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
If you need to add an extra property, like the area in square units by using ``st_area(area)``, you could add a generated column to the table and it will appear in the ``properties`` key of each ``Feature``.
.. code-block:: postgres
alter table coverage
add square_units double precision generated always as ( st_area(area) ) stored;
In the case that you are using older PostGIS versions, then creating a function is your best option:
.. code-block:: postgres
create or replace function coverage_geo_collection() returns json as $$
select
json_build_object(
'type', 'FeatureCollection',
'features', json_agg(
json_build_object(
'type', 'Feature',
'geometry', st_AsGeoJSON(c.area)::json,
'properties', json_build_object('id', c.id, 'name', c.name)
)
)
)
from coverage c;
$$ language sql;
Now this query will return the same results:
.. code-block:: bash
curl "http://localhost:3000/rpc/coverage_geo_collection"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Ranges Ranges
------ ------
@@ -471,20 +609,3 @@ You can use other comparative filters and also all the `PostgreSQL special date/
"due_date": "2022-02-27T06:00:00-05:00" "due_date": "2022-02-27T06:00:00-05:00"
} }
] ]
.. raw:: html
<script type="text/javascript">
let hash = window.location.hash;
const redirects = {
// PostGIS
'#postgis': '../integrations/postgis.html#postgis',
};
let willRedirectTo = redirects[hash];
if (willRedirectTo) {
window.location.href = willRedirectTo;
}
</script>
+31 -51
View File
@@ -38,53 +38,49 @@ Sponsors
.. image:: ../static/cybertec.svg .. image:: ../static/cybertec.svg
:target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest :target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
.. container:: img-dark .. image:: ../static/gnuhost.png
:target: https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest
.. image:: ../static/supabase-dark.svg
:target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
.. container:: img-light
.. image:: ../static/supabase.svg
:target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
.. container:: img-dark .. container:: img-dark
.. image:: ../static/euronodes.svg .. image:: ../static/neon-dark.jpg
:target: https://www.euronodes.com/postgrest :target: https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-light .. container:: img-light
.. image:: ../static/euronodes.svg .. image:: ../static/neon.jpg
:target: https://www.euronodes.com/postgrest :target: https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest
| |
.. container:: img-dark .. container:: img-dark
.. image:: ../static/neon-dark.jpg .. image:: ../static/code-build-dark.png
:target: https://neon.com/?utm_source=sponsor&utm_campaign=postgrest :target: https://code.build/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-light .. container:: img-light
.. image:: ../static/neon.jpg .. image:: ../static/code-build.png
:target: https://neon.com/?utm_source=sponsor&utm_campaign=postgrest :target: https://code.build/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-dark .. container:: img-dark
.. image:: ../static/bytebase-dark.svg .. image:: ../static/supabase-dark.png
:target: https://www.bytebase.com/?utm_source=sponsor&utm_campaign=postgrest :target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
.. container:: img-light .. container:: img-light
.. image:: ../static/bytebase.svg .. image:: ../static/supabase.png
:target: https://www.bytebase.com/?utm_source=sponsor&utm_campaign=postgrest :target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
.. image:: ../static/tembo.png
:target: https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest
.. The static/empty.png(created with `convert -size 320x95 xc:#fcfcfc empty.png`) is an ugly workaround .. The static/empty.png(created with `convert -size 320x95 xc:#fcfcfc empty.png`) is an ugly workaround
to create space and center the logos. It's not easy to layout with restructuredText. to create space and center the logos. It's not easy to layout with restructuredText.
.. image:: _static/empty.png .. .. image:: _static/empty.png
:target: #sponsors :target: #sponsors
| |
@@ -113,17 +109,10 @@ Getting Support
The project has a friendly and growing community. For discussions, use the Github `discussions page <https://github.com/PostgREST/postgrest/discussions>`_. You can also report or search for bugs/features on the Github `issues <https://github.com/PostgREST/postgrest/issues>`_ page. The project has a friendly and growing community. For discussions, use the Github `discussions page <https://github.com/PostgREST/postgrest/discussions>`_. You can also report or search for bugs/features on the Github `issues <https://github.com/PostgREST/postgrest/issues>`_ page.
Releases Release Notes
-------- -------------
PostgREST follows ``MAJOR.PATCH`` two-part versioning: The release notes are published on `PostgREST's GitHub release page <https://github.com/PostgREST/postgrest/releases>`_.
- ``MAJOR``: feature release, may deprecate or remove things.
- ``PATCH``: fix/security release only; no features, no behavior changes.
Starting from ``v14.0``, only even-numbered MAJOR versions will be released, reserving odd-numbered MAJOR versions for development.
All the releases are published on `PostgREST's GitHub release page <https://github.com/PostgREST/postgrest/releases>`_.
Tutorials Tutorials
--------- ---------
@@ -220,14 +209,22 @@ In Production
Here are some companies that use PostgREST in production. Here are some companies that use PostgREST in production.
* `Catarse <https://www.catarse.me>`_ * `Catarse <https://www.catarse.me>`_
* `Datrium <https://www.datrium.com>`_
* `Drip Depot <https://www.dripdepot.com>`_ * `Drip Depot <https://www.dripdepot.com>`_
* `Image-charts <https://www.image-charts.com>`_ * `Image-charts <https://www.image-charts.com>`_
* `Moat <https://www.oracle.com/advertising/>`_
* `Netwo <https://www.netwo.io>`_ * `Netwo <https://www.netwo.io>`_
* `Nimbus <https://www.nimbusfacility.com/sg/home>`_ * `Nimbus <https://www.nimbusfacility.com/sg/home>`_
- See how Nimbus uses PostgREST in `Paul Copplestone's blog post <https://paul.copplest.one/blog/nimbus-tech-2019-04.html>`_. - See how Nimbus uses PostgREST in `Paul Copplestone's blog post <https://paul.copplest.one/blog/nimbus-tech-2019-04.html>`_.
* `OpenBooking <https://openbooking.ch>`_ * `OpenBooking <https://openbooking.ch>`_
* `Redsmin <https://www.redsmin.com>`_
* `Sompani <https://www.sompani.com>`_
* `Supabase <https://supabase.com>`_ * `Supabase <https://supabase.com>`_
.. Failing links
* `eGull <http://www.egull.co>`_
* `MotionDynamic - Fast highly dynamic video generation at scale <https://motiondynamic.tech>`_
Testimonials Testimonials
------------ ------------
@@ -271,21 +268,4 @@ Testimonials
Contributing Contributing
------------ ------------
Please see the `Contributing guidelines <https://github.com/PostgREST/postgrest/blob/main/CONTRIBUTING.md>`_ in the main PostgREST repository. Please see the `Contributing guidelines <https://github.com/PostgREST/postgrest/blob/main/.github/CONTRIBUTING.md>`_ in the main PostgREST repository.
.. raw:: html
<script type="text/javascript">
let hash = window.location.hash;
const redirects = {
// Tables and Views
'#release-notes': '#releases',
};
let willRedirectTo = redirects[hash];
if (willRedirectTo) {
window.location.href = willRedirectTo;
}
</script>
+6
View File
@@ -0,0 +1,6 @@
Greenplum
#########
`Greenplum <https://greenplum.org/>`_ has been reported to work by adding ``LOGIN`` to the :ref:`anonymous and user roles <roles>`.
For more details, see https://github.com/PostgREST/postgrest/issues/2021.
@@ -1,6 +1,6 @@
.. _external_auth: .. _external_jwt:
External Authentication External JWT Generation
----------------------- -----------------------
JWT from Auth0 JWT from Auth0
-154
View File
@@ -1,154 +0,0 @@
.. _ww_postgis:
PostGIS
=======
To work with `PostGIS <https://postgis.net/>`_ data types such as ``geometry`` or ``geography``, you'll need to `install PostGIS <https://postgis.net/documentation/getting_started/>`_ first.
.. code-block:: postgres
-- Activate the postgis module in the current database
create extension if not exists postgis;
create table coverage (
id int primary key,
name text unique,
area geometry
);
insert into coverage (id, name, area) values
(1, 'small', ST_GeomFromText('POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))',4326)),
(2, 'big', ST_GeomFromText('POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))', 4326);
.. _application/geo+json:
``application/geo+json``
------------------------
PostgREST supports the `standard <https://www.iana.org/assignments/media-types/application/geo+json>`_ ``application/geo+json`` media type which can be used to get the output in `GeoJSON <https://geojson.org/>`_ format. This will work for PostGIS versions 3.0.0 and up and will return the output as a `FeatureCollection Object <https://www.rfc-editor.org/rfc/rfc7946#section-3.3>`_:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-H "Accept: application/geo+json"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Using generated columns
-----------------------
If you need to add an extra property, like the area in square units by using ``st_area(area)``, you could add a generated column to the table and it will appear in the ``properties`` key of each ``Feature``.
.. code-block:: postgres
alter table coverage
add square_units double precision generated always as ( st_area(area) ) stored;
In the case that you are using older PostGIS versions, then creating a function is your best option:
.. code-block:: postgres
create or replace function coverage_geo_collection() returns json as $$
select
json_build_object(
'type', 'FeatureCollection',
'features', json_agg(
json_build_object(
'type', 'Feature',
'geometry', st_AsGeoJSON(c.area)::json,
'properties', json_build_object('id', c.id, 'name', c.name)
)
)
)
from coverage c;
$$ language sql;
Now this query will return the same results:
.. code-block:: bash
curl "http://localhost:3000/rpc/coverage_geo_collection"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Using string representation
---------------------------
To insert areas in polygon format, you can use string representation:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 3, "name": "strip", "area": "SRID=4326;POLYGON((0 0, 50 0, 50 2, 0 2, 0 0))" },
{ "id": 4, "name": "diamond", "area": "SRID=4326;POLYGON((5 0, 10 5, 5 10, 0 5, 5 0))" }
]
EOF
PostgREST will automatically cast the ``area`` column into a ``Polygon`` geometry type.
+5 -11
View File
@@ -3,7 +3,6 @@ api
API's API's
APIs APIs
APISIX APISIX
AST
aud aud
Auth Auth
auth auth
@@ -13,10 +12,10 @@ booleans
BOM BOM
Bytea Bytea
Cardano Cardano
casted
cd cd
centric centric
CLI CLI
CMS
coercible coercible
conf conf
Cloudflare Cloudflare
@@ -28,12 +27,10 @@ CSV
durations durations
DDL DDL
DOM DOM
DSL
DevOps DevOps
dockerize dockerize
enum enum
Enums Enums
Entra
eq eq
ETH ETH
Ethereum Ethereum
@@ -46,6 +43,7 @@ Github
Google Google
grantor grantor
GraphQL GraphQL
Greenplum
gte gte
GUC GUC
Haskell Haskell
@@ -70,11 +68,9 @@ isdistinct
JS JS
js js
JSON JSON
JSPath
JWK JWK
JWT JWT
jwt jwt
Keycloak
Kubernetes Kubernetes
localhost localhost
login login
@@ -98,11 +94,10 @@ npm
nxl nxl
nxr nxr
OAuth OAuth
ORM
Observability Observability
Okta
OpenAPI OpenAPI
openapi openapi
ORM
ov ov
parametrized parametrized
passphrase passphrase
@@ -151,6 +146,8 @@ SHA
signup signup
SIGUSR SIGUSR
sl sl
spreaded
Spreaded
SQL SQL
sql sql
SQLSTATE SQLSTATE
@@ -174,7 +171,6 @@ unikernel
unix unix
updatable updatable
unfulfillable unfulfillable
unselected
Untyped Untyped
UPSERT UPSERT
Upsert Upsert
@@ -194,5 +190,3 @@ Websockets
webuser webuser
wfts wfts
www www
debouncing
deduplicates
+18 -1
View File
@@ -12,7 +12,7 @@ Health Check
You can enable a health check to verify if PostgREST is available for client requests. Also to check the status of its internal state. You can enable a health check to verify if PostgREST is available for client requests. Also to check the status of its internal state.
Two endpoints ``live`` and ``ready`` will then be available. Both these endpoints reply with a status code and empty response body. Two endpoints ``live`` and ``ready`` will then be available.
.. important:: .. important::
@@ -55,6 +55,23 @@ Metrics
Provides :ref:`metrics`. Provides :ref:`metrics`.
Runtime Configuration
=====================
Provides a ``config`` endpoint that returns the runtime :ref:`configuration`.
.. code-block:: bash
curl "http://localhost:3001/config"
.. code-block::
db-aggregates-enabled = false
db-anon-role = "web_anon"
db-channel = "pgrst"
db-channel-enabled = false
...
Runtime Schema Cache Runtime Schema Cache
==================== ====================
+103 -72
View File
@@ -3,20 +3,20 @@
Aggregate Functions Aggregate Functions
################### ###################
PostgREST supports the following aggregate functions: ``avg()``, ``count()``, ``max()``, ``min()``, and ``sum()``. Aggregate functions allow you to summarize data by performing calculations across groups of rows. For instance, if you have an ``orders`` table that has an ``amount`` column, you could use an aggregate function to get the sum of the ``amount`` column, either for all rows, or for each group of rows that share specific values, for instance all rows that share the same ``order_date``.
Please refer to the `section on aggregate functions in the PostgreSQL documentation <https://www.postgresql.org/docs/current/functions-aggregate.html>`_ for a detailed explanation of these functions.
.. note:: .. note::
Aggregate functions are *disabled* by default in PostgREST, because they can create performance problems without appropriate safeguards. Aggregate functions are *disabled* by default in PostgREST, as without appropriate safeguards, aggregate functions can create performance problems. See :ref:`db-aggregates-enabled` for further details.
See :ref:`db-aggregates-enabled` for further details.
To use an aggregate function, append it to a column in the ``select`` parameter, like so: PostgREST supports the following aggregate functions: ``avg()``, ``count()``, ``max()``, ``min()``, and ``sum()``. Please refer to the `section on aggregate functions in the PostgreSQL documentation <https://www.postgresql.org/docs/current/functions-aggregate.html>`_ for a detailed explanation of these functions.
To use an aggregate function, you append the function to a value in the ``select`` parameter, like so:
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum()" curl "http://localhost:3000/orders?select=amount.sum()"
This will return a ``sum`` of all the values of the ``amount`` column in a single row: With the above query, PostgREST will return a single row with a single column named ``sum`` that contains the sum of all the values in the ``amount`` column:
.. code-block:: json .. code-block:: json
@@ -26,29 +26,15 @@ This will return a ``sum`` of all the values of the ``amount`` column in a singl
} }
] ]
You can ``select`` multiple aggregate functions at the same time (you may need to :ref:`rename them <renaming_columns>` to disambiguate). You can use multiple aggregate functions by just adding more columns with aggregate functions to the ``select`` parameter.
.. code-block:: bash To group by other columns, you simply add those columns to the ``select`` parameter. For instance:
curl "http://localhost:3000/orders?select=total_amount:amount.sum(),avg_amount:amount.avg(),total_quantity:quantity.sum()"
.. note::
Aggregate functions work alongside other PostgREST features, like :ref:`h_filter`, :ref:`json_columns`, and :ref:`ordering`.
However they are not compatible with :ref:`domain_reps` for the moment.
Additionally, PostgreSQL's ``HAVING`` clause and ordering by aggregated columns are not yet supported.
Automatic ``GROUP BY``
======================
In SQL, a ``GROUP BY`` clause is required to aggregate the selected columns.
However, PostgREST handles grouping automatically if the columns are already present in the ``select`` parameter.
For instance:
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum(),amount.avg(),order_date" curl "http://localhost:3000/orders?select=amount.sum(),amount.avg(),order_date"
This will get the sum and average of the amounts grouped by each unique value in the ``order_date`` column: This will return a row for each unique value in the ``order_date`` column, with the sum and average of the ``amount`` column for all rows that share the same ``order_date``:
.. code-block:: json .. code-block:: json
@@ -65,55 +51,67 @@ This will get the sum and average of the amounts grouped by each unique value in
} }
] ]
The ``count()`` Aggregate .. note::
========================= Aggregate functions work alongside other PostgREST features, like :ref:`h_filter`, :ref:`json_columns`, and :ref:`ordering`. Please note at this time aggregate functions are not compatible with :ref:`domain_reps`. Additionally, PostgreSQL's ``HAVING`` clause and ordering by aggregated columns are not yet supported.
The Case of ``count()``
===========================
.. note:: .. note::
Before the addition of aggregate functions, it was possible to count by adding ``count`` (without parentheses) to the ``select`` parameter. Before the addition of aggregate functions, it was possible to count by adding ``count`` (without parentheses) to the ``select`` parameter. While this is still supported, it may be deprecated in the future, and thus use of this legacy feature is **not recommended.** Please use ``count()`` (with parentheses) instead.
While this is still supported, it may be deprecated in the future, and thus use of this legacy feature is **not recommended**.
Please use ``count()`` (with parentheses) instead.
``count()`` is a special case because it can be used with or without an aggregated column. For example:
``count()`` is treated specially, as it can be used without an associated column. Take for example the following query:
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/orders?select=count(),observation_count:observation.count(),order_date" curl "http://localhost:3000/orders?select=count(),order_date"
This would return a row for each unique value in the ``order_date`` column, with the count of all rows that share the same ``order_date``:
.. code-block:: json .. code-block:: json
[ [
{ {
"count": 4, "count": 4,
"observation_count": 2,
"order_date": "2023-01-01" "order_date": "2023-01-01"
}, },
{ {
"count": 2, "count": 2,
"observation_count": 1,
"order_date": "2023-01-02" "order_date": "2023-01-02"
} }
] ]
Note that there is a difference between the result of ``count()`` and ``observation.count()``. When ``count()`` is used with an associated column, its behavior is slightly different: It will return the count of all values that are not ``NULL``. This is due to how PostgreSQL itself implements the ``count()`` function.
The former counts the whole row, while the latter counts the non ``NULL`` values of the ``observation`` column (both grouped by ``order_date``).
This is due to how PostgreSQL itself implements the ``count()`` function. Renaming and Casting
====================
Renaming Aggregates
-------------------
Just like with other columns, you can rename aggregated columns too. See :ref:`renaming_columns` for details.
Renaming columns is especially helpful in the context of aggregate functions, as by default a column with an aggregate function applied will take on the name of the applied aggregate function. You may want to provide a more semantically meaningful name or prevent collisions when using multiple aggregate functions of the same type.
Casting Aggregates Casting Aggregates
================== ------------------
It is :ref:`possible to cast <casting_columns>` the aggregated column or the aggregate itself, or both at the same time. When applying an aggregate function to a column, you are able to cast both the value of the input to the aggregate function *and* the value of the output from the aggregate function. In both cases, the syntax works as described in :ref:`casting_columns`, with the only difference being the placement of the cast.
Casting the Aggregated Column Casting the Value of the Input
----------------------------- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For example, let's say that ``orders`` has an ``order_details`` :ref:`JSON column <json_columns>` with a ``tax_amount`` key. For instance, imagine that the ``orders`` table has a JSON column, ``order_details``, and this column contains a JSON object that has a key, ``tax_amount``. Let's say you want to get the sum of the tax amount for every order. You can use the ``->`` or ``->>`` operators to extract the value with this key (see :ref:`json_columns`), but these operators will return values of the types JSON and ``text`` respectively, and neither of these types can be used with ``sum()``.
We cannot sum ``tax_amount`` directly because using ``->`` or ``->>`` will return the data in ``json`` or ``text`` format.
So we need to cast it to a compatible type (e.g. ``numeric``) right before the aggregate function: Therefore, you will need to first cast the input value to a type that is compatible with ``sum()`` (e.g. ``numeric``). Casting the input value is done in exactly the same way as casting any other value:
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/orders?select=order_details->tax_amount::numeric.sum()" curl "http://localhost:3000/orders?select=order_details->tax_amount::numeric.sum()"
With this, you will receive the sum of the casted ``tax_amount`` value:
.. code-block:: json .. code-block:: json
[ [
@@ -122,15 +120,17 @@ So we need to cast it to a compatible type (e.g. ``numeric``) right before the a
} }
] ]
Casting the Aggregate Casting the Value of the Output
--------------------- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For instance, if we wanted to round the average of the ``amount`` column, we could do so by casting ``avg()`` to an ``int``: Now let's return to an example involving the ``amount`` column of the ``orders`` table. Imagine that we want to get the rounded average of the ``amount`` column. One way to do this is to use the ``avg()`` aggregate function and then to cast the output value of the function to ``int``. To cast the value of the output of the function, we simply place the cast *after* the aggregate function:
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/orders?select=amount.avg()::int" curl "http://localhost:3000/orders?select=amount.avg()::int"
You will then receive the rounded average as the result:
.. code-block:: json .. code-block:: json
[ [
@@ -139,22 +139,27 @@ For instance, if we wanted to round the average of the ``amount`` column, we cou
} }
] ]
Aggregates and Resource Embedding Of course, you can use both input and output casts at the same time, if you so desire.
=================================
You can group an aggregate function by an :ref:`embedded resource <resource_embedding>` and also use the aggregates inside them.
Grouping by an Embedded Resource Using Aggregate Functions with Resource Embedding
-------------------------------- =================================================
Similar to grouping by columns, aggregate functions can also be grouped by embedded resources. Aggregate functions can be used in conjunction with :ref:`resource_embedding`. You can use embedded resources as grouping columns, use aggregate functions within the context of an embedded resource, or use columns from a spreaded resource as grouping columns or as inputs to aggregate functions.
For example, let's say that the ``orders`` table is related to a ``customers`` table.
To get the sum of the ``amount`` column grouped by the ``name`` column from the ``customers`` table, we would do the following: Using Embedded Resources as Grouping Columns
--------------------------------------------
Using an embedded resource as a grouping column allows you to use data from an association to group the results of an aggregation.
For example, imagine that the ``orders`` table from the examples above is related to a ``customers`` table. If you want to get the sum of the ``amount`` column grouped by the ``name`` column from the ``customers`` table, you can include the customer name, using the standard :ref:`resource_embedding` syntax, and perform a sum on the ``amount`` column.
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum(),customers(name)" curl "http://localhost:3000/orders?select=amount.sum(),customers(name)"
You will then get the summed amount, along with the embedded customer resource:
.. code-block:: json .. code-block:: json
[ [
@@ -172,16 +177,15 @@ To get the sum of the ``amount`` column grouped by the ``name`` column from the
} }
] ]
The previous example uses a "to-one" relationship, but this can be done on "to-many" relationships as well (although there are few obvious use cases). .. note::
The previous example uses a has-one association to demonstrate this functionality, but you may also use has-many associations as grouping columns, although there are few obvious use cases for this.
This also works in a similar way for :ref:`spread embedded resources <spread_embed>`. Using Aggregate Functions Within the Context of an Embedded Resource
For example, ``select=amount.sum(),...customers(name)`` would sum the ``amount`` grouped by the ``name`` column. --------------------------------------------------------------------
Using Aggregates Inside Embedded Resources When embedding a resource, you can apply aggregate functions to columns from the associated resource to perform aggregations within the context of an embedded resource.
------------------------------------------
Using the relationship from the previous example, let's take all the ``customers`` and embed their ``orders``. Continuing with the example relationship between ``orders`` and ``customers`` from the previous section, imagine that you want to fetch the ``name``, ``city``, and ``state`` for each customer, along with the sum of amount of the customer's orders, grouped by the order date. This can be done in the following way:
If we also want to get the total ``amount`` grouped by the ``order_date`` of the ``orders``, we would do the following:
.. code-block:: bash .. code-block:: bash
@@ -222,20 +226,51 @@ If we also want to get the total ``amount`` grouped by the ``order_date`` of the
} }
] ]
Note that the aggregate is done within the embedded resource ``orders``. In this example, the ``amount`` column is summed and grouped by the ``order_date`` *within* the context of the embedded resource. That is, the ``name``, ``city``, and ``state`` from the ``customers`` table have no bearing on the aggregation performed in the context of the ``orders`` association; instead, each aggregation can be seen as being performed independently on just the orders belonging to a particular customer, using only the data from the embedded resource for both grouping and aggregation.
It is not affected by any of the columns from the top-level relationship ``customers``.
Aggregates in To-One Spreads Using Columns from a Spreaded Resource
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ --------------------------------------
All the aggregates inside a :ref:`one-to-one or many-to-one spread embedded resource <spread_to_one_embed>` will be hoisted to the top-level relationship. When you :ref:`spread an embedded resource <spread_embed>`, the columns from the spreaded resource are treated as if they were columns of the top-level resource, both when using them as grouping columns and when applying aggregate functions to them.
In other words, it will behave as if the aggregate was done in the top-level relationship itself. For example:
Grouping with Columns from a Spreaded Resource
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For instance, assume you want to sum the ``amount`` column from the ``orders`` table, using the ``city`` and ``state`` columns from the ``customers`` table as grouping columns. To achieve this, you may select these two columns from the ``customers`` table and spread them; they will then be used as grouping columns:
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/orders?select=order_date,...customers(subscription_date.max(),subscription_date.min()) curl "http://localhost:3000/orders?select=amount.sum(),...customers(city,state)
This will take the ``max`` and ``min`` subscription date of every customer and group it by the ``order_date`` column: The result will be the same as if ``city`` and ``state`` were columns from the ``orders`` table:
.. code-block:: json
[
{
"sum": 2000.29,
"city": "New York",
"state": "NY"
},
{
"sum": 9241.21,
"city": "Los Angeles",
"state": "CA"
}
]
Aggregate Functions with Columns from a Spreaded Resource
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now imagine that the ``customers`` table has a ``joined_date`` column that represents the date that the customer joined. You want to get both the most recent and the oldest ``joined_date`` for customers that placed an order on every distinct order date. This can be expressed as follows:
.. code-block:: bash
curl "http://localhost:3000/orders?select=order_date,...customers(joined_date.max(),joined_date.min())
As columns from a spreaded resource are treated as if they were columns from the top-level resource, the ``max()`` and ``min()`` are applied *within* the context of the top-level, rather than within the context of the embedded resource, as in the previous section.
The result will be the same as if the aggregations were applied to columns from the top-level:
.. code-block:: json .. code-block:: json
@@ -251,7 +286,3 @@ This will take the ``max`` and ``min`` subscription date of every customer and g
"min": "2016-02-11" "min": "2016-02-11"
} }
] ]
.. note::
Aggregates inside to-many spreads are not supported
@@ -173,4 +173,4 @@ Domain Representations avoid all the above drawbacks. Their only drawback is tha
Why not create a `base type <https://www.postgresql.org/docs/current/sql-createtype.html#id-1.9.3.94.5.8>`_ instead? ``CREATE TYPE app_uuid (INTERNALLENGTH = 22, INPUT = app_uuid_parser, OUTPUT = app_uuid_formatter)``. Why not create a `base type <https://www.postgresql.org/docs/current/sql-createtype.html#id-1.9.3.94.5.8>`_ instead? ``CREATE TYPE app_uuid (INTERNALLENGTH = 22, INPUT = app_uuid_parser, OUTPUT = app_uuid_formatter)``.
Creating base types need superuser, which is restricted on cloud hosted databases. Additionally this way lets "how the data is presented" dictate "how the data is stored" which would be backwards. Creating base types need superuser, which is restricted on cloud hosted databases. Additionally this way lets how the data is presented dictate how the data is stored which would be backwards.
+4 -51
View File
@@ -71,40 +71,6 @@ The function parameter names match the JSON object keys in the POST case, for th
.. _function_single_json: .. _function_single_json:
Functions with an array of JSON objects
----------------------------------------------
If you want to pass multiple JSON objects to a Postgres function (an array of objects), you can create a function with a parameter of type ``json`` or ``jsonb``.
Within the curl request, this JSON must be embedded in an object where they key matches the same name as the function's ``json`` or ``jsonb`` parameter.
This will allow you to loop over the array of JSON objects within the Postgres function.
This practice may allow you to reduce the number of ``curl`` requests required to accomplish a task.
For instance, assume we have created this function in the database.
.. code-block:: postgres
CREATE FUNCTION update_data(p_json jsonb)
RETURNS void AS $$
DECLARE
json_item json;
BEGIN
FOR json_item IN SELECT jsonb_array_elements(p_json) LOOP
UPDATE data_table SET data_text_column = (json_item->>'data_text')::text
WHERE data_int_column = (json_item->>'data_int')::integer;
END LOOP;
END;
$$ LANGUAGE SQL IMMUTABLE;
A ``curl`` request using the POST method would look like the following:
.. code-block:: bash
curl "http://localhost:3000/rpc/update_data" \
-X POST -H "Content-Type: application/json" \
-d '{ "p_json": [ { "data_text": "one", "data_int": "1" }, { "data_text": "two", "data_int": "2" } ] }'
Functions with a single unnamed JSON parameter Functions with a single unnamed JSON parameter
---------------------------------------------- ----------------------------------------------
@@ -131,6 +97,10 @@ For this the ``Content-Type: application/json`` header must be included in the r
If an overloaded function has a single ``json`` or ``jsonb`` unnamed parameter, PostgREST will call this function as a fallback provided that no other overloaded function is found with the parameters sent in the POST request. If an overloaded function has a single ``json`` or ``jsonb`` unnamed parameter, PostgREST will call this function as a fallback provided that no other overloaded function is found with the parameters sent in the POST request.
.. warning::
Sending the JSON request body as a single argument is also possible with :ref:`Prefer: params=single-object <prefer_params>` but this method is **deprecated**.
.. _function_single_unnamed: .. _function_single_unnamed:
Functions with a single unnamed parameter Functions with a single unnamed parameter
@@ -294,23 +264,6 @@ Let's get its :ref:`explain_plan` when calling it with filters applied:
Notice there's no "Function Scan" node in the plan, which tells us it has been inlined. Notice there's no "Function Scan" node in the plan, which tells us it has been inlined.
Horizontal Filtering
~~~~~~~~~~~~~~~~~~~~
Table-valued functions support horizontal filtering on selected and unselected columns.
For example, the following RPC with filter on unselected column returns:
.. code-block:: bash
curl "http://localhost:3000/rpc/getallprojects?select=id,client_id&name=like.OSX"
.. code-block:: json
[
{ "id": 4, "client_id": 2 }
]
.. _scalar_functions: .. _scalar_functions:
Scalar functions Scalar functions
-3
View File
@@ -19,9 +19,6 @@ For extra customization, the OpenAPI output contains a "description" field for e
COMMENT ON TABLE monotremes IS COMMENT ON TABLE monotremes IS
'Freakish mammals lay the best eggs for breakfast'; 'Freakish mammals lay the best eggs for breakfast';
COMMENT ON VIEW monotremes_v IS
'Only the platypus is publicly visible';
COMMENT ON COLUMN monotremes.has_venomous_claw IS COMMENT ON COLUMN monotremes.has_venomous_claw IS
'Sometimes breakfast is not worth it'; 'Sometimes breakfast is not worth it';
+20 -64
View File
@@ -12,9 +12,10 @@ The following preferences are supported.
- ``Prefer: return``. See :ref:`prefer_return`. - ``Prefer: return``. See :ref:`prefer_return`.
- ``Prefer: count``. See :ref:`prefer_count`. - ``Prefer: count``. See :ref:`prefer_count`.
- ``Prefer: resolution``. See :ref:`prefer_resolution`. - ``Prefer: resolution``. See :ref:`prefer_resolution`.
- ``Prefer: missing``. See :ref:`prefer_missing`. - ``Prefer: missing``. See :ref:`bulk_insert_default`.
- ``Prefer: max-affected``, See :ref:`prefer_max_affected`. - ``Prefer: max-affected``, See :ref:`prefer_max_affected`.
- ``Prefer: tx``. See :ref:`prefer_tx`. - ``Prefer: tx``. See :ref:`prefer_tx`.
- ``Prefer: params``. See :ref:`prefer_params`.
.. _prefer_handling: .. _prefer_handling:
@@ -196,53 +197,12 @@ The ``tx`` preference can be set to specify if the :ref:`transaction <transactio
{"id": 35, "name": "Project X"} {"id": 35, "name": "Project X"}
.. _prefer_missing:
Missing
=======
When doing ``POST`` and ``PATCH`` requests, any missing columns in the payload will be inserted as ``null`` value by default. To use the ``DEFAULT`` column value instead, use the ``Prefer: missing=default`` header.
Having:
.. code-block:: postgres
create table foo (
id bigint generated by default as identity primary key
, bar text
, baz int default 100
);
A request:
.. code-block:: bash
curl "http://localhost:3000/foo?columns=id,bar,baz" \
-H "Content-Type: application/json" \
-H "Prefer: missing=default, return=representation" \
-d @- << EOF
[
{ "bar": "val1" },
{ "bar": "val2", "baz": 15 }
]
EOF
Will result in:
.. code-block:: json
[
{ "id": 1, "bar": "val1", "baz": 100 },
{ "id": 2, "bar": "val2", "baz": 15 }
]
.. _prefer_max_affected: .. _prefer_max_affected:
Max Affected Max Affected
============ ============
You can set a limit to the amount of resources affected in a request by sending ``max-affected`` preference. This feature works in combination with ``handling=strict`` preference. ``max-affected`` would be ignored with lenient handling. The "affected resources" are the number of rows returned by ``DELETE`` and ``PATCH`` requests. You can set a limit to the amount of resources affected in a request by sending ``max-affected`` preference. This feature works in combination with ``handling=strict`` preference. ``max-affected`` would be ignored with lenient handling. The "affected resources" are the number of rows returned by ``DELETE`` and ``PATCH`` requests. This is also supported through ``RPC`` calls.
To illustrate the use of this preference, consider the following scenario where the ``items`` table contains 14 rows. To illustrate the use of this preference, consider the following scenario where the ``items`` table contains 14 rows.
@@ -265,34 +225,30 @@ To illustrate the use of this preference, consider the following scenario where
"hint": null "hint": null
} }
With :ref:`RPC <functions>`, the preference is honored completely on the basis of the number of rows returned in the result set of the function. This can be useful for complex mutation queries using `data-modifying statements <https://www.postgresql.org/docs/current/queries-with.html#QUERIES-WITH-MODIFYING>`_. A simple example: .. _prefer_params:
Single JSON object as Function Parameter
----------------------------------------
.. warning::
Using this preference is **deprecated** in favor of :ref:`function_single_json`.
:code:`Prefer: params=single-object` allows sending the JSON request body as the single argument of a :ref:`function <functions>`.
.. code-block:: postgres .. code-block:: postgres
CREATE FUNCTION test.delete_items() CREATE FUNCTION mult_them(param json) RETURNS int AS $$
RETURNS SETOF items AS $$ SELECT (param->>'x')::int * (param->>'y')::int
DELETE FROM items WHERE id < 15 RETURNING *;
$$ LANGUAGE SQL; $$ LANGUAGE SQL;
.. code-block:: bash .. code-block:: bash
curl -i "http://localhost:3000/rpc/delete_items" \ curl "http://localhost:3000/rpc/mult_them" \
-H "Content-Type: application/json" \ -X POST -H "Content-Type: application/json" \
-H "Prefer: handling=strict, max-affected=10" -H "Prefer: params=single-object" \
-d '{ "x": 4, "y": 2 }'
.. code-block:: http
HTTP/1.1 400 Bad Request
.. code-block:: json .. code-block:: json
{ 8
"code": "PGRST124",
"message": "Query result exceeds max-affected preference constraint",
"details": "The query affects 14 rows",
"hint": null
}
.. note::
It is important for functions to return ``SETOF`` or ``TABLE`` when called with ``max-affected`` preference. A violation of this would cause a :ref:`PGRST128 <pgrst128>` error.
+24 -174
View File
@@ -143,7 +143,7 @@ Since the table name is plural, we can be more accurate by making it singular wi
One-to-many relationships One-to-many relationships
------------------------- -------------------------
The **foreign key reference** establishes the inverse one-to-many relationship. In this case, ``films`` returns as a JSON array because of the "to-many" end. The **foreign key reference** establishes the inverse one-to-many relationship. In this case, ``films`` returns as a JSON array because of the to-many end.
.. code-block:: bash .. code-block:: bash
@@ -209,25 +209,18 @@ The join table is also detected if the composite key has additional columns.
One-to-one relationships One-to-one relationships
------------------------ ------------------------
One-to-one relationships are detected in two ways. (We'll use the ``films`` and ``technical_specs`` tables from the :ref:`sample film database <erd_film>` as an example). One-to-one relationships are detected in two ways.
- When the foreign key is also a primary key. - When the foreign key is a primary key as specified in the :ref:`sample film database <erd_film>`.
- When the foreign key has a unique constraint.
.. code-block:: postgres .. code-block:: postgres
create table technical_specs( create table technical_specs(
film_id int references films(id) primary key film_id int references films(id) unique,
-- ... runtime time,
); camera text,
sound text
- Or when the foreign key has a unique constraint.
.. code-block:: postgres
create table technical_specs(
id int primary key
, film_id int references films(id) unique
-- ...
); );
.. code-block:: bash .. code-block:: bash
@@ -251,12 +244,6 @@ Computed Relationships
You can manually define relationships by using functions. This is useful for database objects that can't define foreign keys, like `Foreign Data Wrappers <https://wiki.postgresql.org/wiki/Foreign_data_wrappers>`_. You can manually define relationships by using functions. This is useful for database objects that can't define foreign keys, like `Foreign Data Wrappers <https://wiki.postgresql.org/wiki/Foreign_data_wrappers>`_.
Computed relationships have good performance as their intended design enable `function inlining <https://wiki.postgresql.org/wiki/Inlining_of_SQL_functions#Inlining_conditions_for_table_functions>`_.
.. important::
- Always use ``SETOF`` when creating computed relationships. Functions can return a table without using ``SETOF``, but bear in mind that PostgreSQL will not inline them. e.g. ``RETURNS <table_name>`` is not inlinable.
Assuming there's a foreign table ``premieres`` that we want to relate to ``films``. Assuming there's a foreign table ``premieres`` that we want to relate to ``films``.
.. code-block:: postgres .. code-block:: postgres
@@ -289,10 +276,6 @@ The name of the function ``film`` is arbitrary and can be used to do the embeddi
".." ".."
] ]
.. warning::
- Make sure to correctly label the ``to-one`` part of the relationship. When using the ``ROWS 1`` estimation, PostgREST will expect a single row to be returned. If that is not the case, it will unnest the embedding and return repeated values for the top level resource.
Now let's define the opposite one-to-many relationship. Now let's define the opposite one-to-many relationship.
.. code-block:: postgres .. code-block:: postgres
@@ -341,6 +324,12 @@ Thanks to overloaded functions, you can use the same function name for different
Computed relationships have good performance as their intended design enable `function inlining <https://wiki.postgresql.org/wiki/Inlining_of_SQL_functions#Inlining_conditions_for_table_functions>`_. Computed relationships have good performance as their intended design enable `function inlining <https://wiki.postgresql.org/wiki/Inlining_of_SQL_functions#Inlining_conditions_for_table_functions>`_.
.. warning::
- Always use ``SETOF`` when creating computed relationships. Functions can return a table without using ``SETOF``, but bear in mind that PostgreSQL will not inline them.
- Make sure to correctly label the ``to-one`` part of the relationship. When using the ``ROWS 1`` estimation, PostgREST will expect a single row to be returned. If that is not the case, it will unnest the embedding and return repeated values for the top level resource.
.. _embed_disamb: .. _embed_disamb:
.. _target_disamb: .. _target_disamb:
.. _hint_disamb: .. _hint_disamb:
@@ -938,12 +927,7 @@ Filters can also be applied on nested embedded resources:
.. code-block:: bash .. code-block:: bash
# curl "http://localhost:3000/films?select=*,roles(*,actors(*))&roles.actors.order=last_name&roles.actors.first_name=like.*Tom*" curl "http://localhost:3000/films?select=*,roles(*,actors(*))&roles.actors.order=last_name&roles.actors.first_name=like.*Tom*"
curl --get "http://localhost:3000/films" \
-d "select=*,roles(*,actors(*))" \
-d "roles.actors.order=last_name" \
-d "roles.actors.first_name=like.*Tom*"
The result will show the nested actors named Tom and order them by last name. Aliases can also be used instead of the resource names to filter the nested tables. The result will show the nested actors named Tom and order them by last name. Aliases can also be used instead of the resource names to filter the nested tables.
@@ -1154,19 +1138,14 @@ For example, to arrange the films in descending order using the director's last
Spread embedded resource Spread embedded resource
======================== ========================
You can modify the shape of the embedded resources by using the spread syntax (``...``). On many-to-one and one-to-one relationships, you can "spread" the embedded resource. That is, remove the surrounding JSON object for the embedded resource columns.
.. _spread_to_one_embed:
Spread To-One relationships
---------------------------
Spread on resources forming :ref:`one-to-one <one-to-one>` and :ref:`many-to-one <many-to-one>` relationships, will lift the embedded columns to the top object.
.. code-block:: bash .. code-block:: bash
# curl "http://localhost:3000/films?select=title,...directors(director_last_name:last_name)&title=like.*Workers*"
curl --get "http://localhost:3000/films" \ curl --get "http://localhost:3000/films" \
-d "select=title,...directors(director_first_name:first_name, director_last_name:last_name)" \ -d "select=title,...directors(director_last_name:last_name)" \
-d "title=like.*Workers*" -d "title=like.*Workers*"
.. code-block:: json .. code-block:: json
@@ -1174,148 +1153,17 @@ Spread on resources forming :ref:`one-to-one <one-to-one>` and :ref:`many-to-one
[ [
{ {
"title": "Workers Leaving The Lumière Factory In Lyon", "title": "Workers Leaving The Lumière Factory In Lyon",
"director_first_name": "Louis",
"director_last_name": "Lumière" "director_last_name": "Lumière"
} }
] ]
Note that there is no wrapping ``"directors"`` object, unlike regularly embedding :ref:`many-to-one <many-to-one>` relationships. Also note that embedded columns can be aliased normally. Note that there is no ``"directors"`` object. Also the embed columns can be aliased normally.
.. _spread_to_many_embed: You can use this to get the columns of a join table in a many-to-many relationship. For instance, to get films and its actors, but including the ``character`` column from the roles table:
Spread To-Many relationships
----------------------------
Spread on resources forming :ref:`one-to-many <one-to-many>` and :ref:`many-to-many <many-to-many>` relationships, will convert the embedded columns into correlated arrays.
.. code-block:: bash .. code-block:: bash
curl --get "http://localhost:3000/directors" \ # curl "http://localhost:3000/films?select=title,actors:roles(character,...actors(first_name,last_name))&title=like.*Lighthouse*"
-d "select=first_name,...films(film_titles:title,film_years:year)" \
-d "first_name=like.Quentin*"
.. code-block:: json
[
{
"first_name": "Quentin",
"film_titles": [
"Pulp Fiction",
"Reservoir Dogs"
],
"film_years": [
1994,
1992
]
}
]
Note that ``films`` is no longer an array of objects, unlike regularly embedding :ref:`one-to-many`. The embedded columns become arrays and they're correlated-in the above result, we can say that "Pulp Fiction" premiered in 1994 and "Reservoir Dogs" in 1992.
Order in spread to-many
~~~~~~~~~~~~~~~~~~~~~~~
In the above example, the order of the values inside the correlated arrays is unspecified, but all the values are guaranteed to be in the same unspecified order.
You can order the correlated arrays explicitly. For example, to order by the film year:
.. code-block:: bash
curl --get "http://localhost:3000/directors" \
-d "select=first_name,...films(film_titles:title,film_years:year)" \
-d "first_name=like.Quentin*" \
-d "films.order=year"
.. code-block:: json
[
{
"first_name": "Quentin",
"film_titles": [
"Reservoir Dogs",
"Pulp Fiction"
],
"film_years": [
1992,
1994
]
}
]
.. warning::
Aliasing spreaded columns is recommended since JSON allows duplicate keys. Example:
.. code-block:: bash
curl --get "localhost:3000/projects" \
-d "select=id,name,...clients(id,name)"
.. code-block:: json
[{"id":1,"name":"Windows 7","id":1,"name":"Microsoft"},
{"id":2,"name":"Windows 10","id":1,"name":"Microsoft"},
{"id":3,"name":"IOS","id":2,"name":"Apple"},
{"id":4,"name":"OSX","id":2,"name":"Apple"},
{"id":5,"name":"Orphan","id":null,"name":null}]
This can be a problem in Javascript objects, since only the last duplicated key will be considered. To solve it do:
.. code-block:: bash
curl --get "localhost:3000/projects" \
-d "select=id,name,...clients(client_id:id,client_name:name)"
Multiple Spreads
----------------
You can use multiple spreads at any level. For example, let's spread ``technical_specs`` and ``roles`` into ``films`` and then spread ``films`` into ``directors``:
.. code-block:: bash
curl --get "http://localhost:3000/directors" \
-d "select=first_name,...films(film_titles:title,film_years:year,...technical_specs(film_runtimes:runtime),...roles(film_characters:character))" \
-d "first_name=like.Quentin*" \
-d "films.order=year" \
-d "films.roles.order=character"
.. code-block:: json
[
{
"first_name": "Quentin",
"film_titles": [
"Reservoir Dogs",
"Pulp Fiction"
],
"film_years": [
1992,
1994
],
"film_runtimes": [
"01:39:00",
"02:29:00"
]
"film_characters": [
[ "Mr. Pink", "Mr. White" ],
[ "Mia Wallace", "Vincent Vega" ]
]
}
]
Note that:
- All the ``film_*`` arrays are correlated-"Reservoir Dogs" premiered in 1992, its runtime is 1:39:00 and it has the following characters: ``[ "Mr. Pink", "Mr. White" ]``.
- The ``film_*`` arrays are ordered by ``year`` (due to ``films.order=year``).
- The bottom level array ``film_characters`` is ordered (due to ``films.roles.order=character``).
Spread a join table
-------------------
Spread can be used to move the columns of a join table in a :ref:`many-to-many <many-to-many>` to the top object. For instance, to get the ``character`` column of the ``roles`` join table into ``actors``:
.. code-block:: bash
curl --get "http://localhost:3000/films" \ curl --get "http://localhost:3000/films" \
-d "select=title,actors:roles(character,...actors(first_name,last_name))" \ -d "select=title,actors:roles(character,...actors(first_name,last_name))" \
@@ -1336,4 +1184,6 @@ Spread can be used to move the columns of a join table in a :ref:`many-to-many <
} }
] ]
.. note::
The spread operator ``...`` is borrowed from the Javascript `spread syntax <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax>`_.
@@ -16,32 +16,6 @@ Use the Accept request header to specify the acceptable format (or formats) for
curl "http://localhost:3000/people" \ curl "http://localhost:3000/people" \
-H "Accept: application/json" -H "Accept: application/json"
.. note::
The ordering of columns in the response isn't guaranteed to align with the order specified in the ``select`` clause. For example, with resource embedding:
.. code-block:: bash
http://localhost:3000/films?select=directors(last_name,id),title
We may get:
.. code-block:: bash
[
{
"title": "title",
"directors": {
"id": 5,
"last_name": "name"
}
}
]
This is in line with the `JSON schema spec <https://json-schema.org/draft/2020-12/json-schema-core#name-instance-data-model>`_:
*"object: An unordered set of properties mapping a string to an instance"*
.. _builtin_media: .. _builtin_media:
Builtin Media Type Handlers Builtin Media Type Handlers
@@ -51,7 +25,7 @@ Builtin handlers are offered for common standard media types.
* ``text/csv`` and ``application/json``, for all API endpoints. See :ref:`tables_views` and :ref:`functions`. * ``text/csv`` and ``application/json``, for all API endpoints. See :ref:`tables_views` and :ref:`functions`.
* ``application/openapi+json``, for the root endpoint. See :ref:`open-api`. * ``application/openapi+json``, for the root endpoint. See :ref:`open-api`.
* ``application/geo+json``, see :ref:`application/geo+json`. * ``application/geo+json``, see :ref:`ww_postgis`.
* ``*/*``, resolves to ``application/json`` for API endpoints and to ``application/openapi+json`` for the root endpoint. * ``*/*``, resolves to ``application/json`` for API endpoints and to ``application/openapi+json`` for the root endpoint.
The following vendor media types handlers are also supported. The following vendor media types handlers are also supported.
@@ -100,15 +74,17 @@ This returns
{ "id": 1 } { "id": 1 }
with a :code:`Content-Type: application/vnd.pgrst.object+json`.
When a singular response is requested but no entries are found, the server responds with an error message and 406 Not Acceptable status code rather than the usual empty array and 200 status: When a singular response is requested but no entries are found, the server responds with an error message and 406 Not Acceptable status code rather than the usual empty array and 200 status:
.. code-block:: json .. code-block:: json
{ {
"code": "PGRST116", "message": "JSON object requested, multiple (or no) rows returned",
"message": "Cannot coerce the result to a single JSON object", "details": "Results contain 0 rows, application/vnd.pgrst.object+json requires 1 row",
"details": "The result contains 0 rows", "hint": null,
"hint": null "code": "PGRST505"
} }
.. note:: .. note::
+77 -67
View File
@@ -72,7 +72,7 @@ imatch :code:`~*` ~* operator, see :ref:`pattern_matching`
in :code:`IN` one of a list of values, e.g. :code:`?a=in.(1,2,3)` in :code:`IN` one of a list of values, e.g. :code:`?a=in.(1,2,3)`
also supports commas in quoted strings like also supports commas in quoted strings like
:code:`?a=in.("hi,there","yes,you")` :code:`?a=in.("hi,there","yes,you")`
is :code:`IS` checking for exact equality (null,not_null,true,false,unknown) is :code:`IS` checking for exact equality (null,true,false,unknown)
isdistinct :code:`IS DISTINCT FROM` not equal, treating :code:`NULL` as a comparable value isdistinct :code:`IS DISTINCT FROM` not equal, treating :code:`NULL` as a comparable value
fts :code:`@@` :ref:`fts` using to_tsquery fts :code:`@@` :ref:`fts` using to_tsquery
plfts :code:`@@` :ref:`fts` using plainto_tsquery plfts :code:`@@` :ref:`fts` using plainto_tsquery
@@ -175,56 +175,25 @@ To ensure best performance on larger data sets, an `appropriate index <https://w
Full-Text Search Full-Text Search
~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~
The :code:`fts` operator has a number of options to support flexible textual queries, namely the choice of plain vs phrase search and the language used for stemming. The :code:`fts` filter mentioned above has a number of options to support flexible textual queries, namely the choice of plain vs phrase search and the language used for stemming. Suppose that :code:`tsearch` is a table with column :code:`my_tsv`, of type `tsvector <https://www.postgresql.org/docs/current/datatype-textsearch.html>`_. The following examples illustrate the possibilities.
The following examples illustrate the possibilities, assuming column :code:`my_tsv` is of type `tsvector <https://www.postgresql.org/docs/current/datatype-textsearch.html>`_.
.. code-block:: bash .. code-block:: bash
curl --get "http://localhost:3000/people" \ curl "http://localhost:3000/tsearch?my_tsv=fts(french).amusant"
-d "my_tsv=fts(french).amusant"
.. code-block:: bash .. code-block:: bash
curl --get "http://localhost:3000/people" \ curl "http://localhost:3000/tsearch?my_tsv=plfts.The%20Fat%20Cats"
-d "my_tsv=plfts.The%20Fat%20Cats"
.. code-block:: bash .. code-block:: bash
curl --get "http://localhost:3000/people" \ curl "http://localhost:3000/tsearch?my_tsv=not.phfts(english).The%20Fat%20Cats"
-d "my_tsv=not.phfts(english).The%20Fat%20Cats"
.. code-block:: bash .. code-block:: bash
curl --get "http://localhost:3000/people" \ curl "http://localhost:3000/tsearch?my_tsv=not.wfts(french).amusant"
-d "my_tsv=not.wfts(french).amusant"
.. _fts_to_tsvector: Using `websearch_to_tsquery` requires PostgreSQL of version at least 11.0 and will raise an error in earlier versions of the database.
Automatic ``tsvector`` conversion
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If the filtered column is not of type ``tsvector``, then it will be automatically converted using `to_tsvector() <https://www.postgresql.org/docs/current/functions-textsearch.html#TEXTSEARCH-FUNCTIONS-TABLE>`_.
This allows using the ``fts`` operator on ``text`` and ``json`` types out of the box.
.. code-block:: bash
curl --get "http://localhost:3000/people" \
-d "my_text_column=fts(french).amusant"
.. code-block:: bash
curl --get "http://localhost:3000/people" \
-d "my_json_column=not.phfts(english).The%20Fat%20Cats"
.. important::
To ensure this operation is fast, you need to create an index on the expression:
.. code-block:: postgres
CREATE INDEX idx_people_col ON people
USING GIN (to_tsvector('french', my_text_column));
.. _v_filter: .. _v_filter:
@@ -556,6 +525,45 @@ To bulk insert JSON post an array of objects having all-matching keys
] ]
EOF EOF
.. _bulk_insert_default:
Bulk Insert with Default Values
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Any missing columns in the payload will be inserted as ``null`` values. To use the ``DEFAULT`` column value instead, use the ``Prefer: missing=default`` header.
Having:
.. code-block:: postgres
create table foo (
id bigint generated by default as identity primary key
, bar text
, baz int default 100
);
A request:
.. code-block:: bash
curl "http://localhost:3000/foo?columns=id,bar,baz" \
-H "Content-Type: application/json" \
-H "Prefer: missing=default, return=representation" \
-d @- << EOF
[
{ "bar": "val1" },
{ "bar": "val2", "baz": 15 }
]
EOF
Will result in:
.. code-block:: json
[
{ "id": 1, "bar": "val1", "baz": 100 },
{ "id": 2, "bar": "val2", "baz": 15 }
]
.. _specify_columns: .. _specify_columns:
@@ -598,13 +606,7 @@ To update a row or rows in a table, use the PATCH verb. Use :ref:`h_filter` to s
-X PATCH -H "Content-Type: application/json" \ -X PATCH -H "Content-Type: application/json" \
-d '{ "category": "child" }' -d '{ "category": "child" }'
Updates also support: Updates also support :ref:`prefer_return`, :ref:`resource_embedding` and :ref:`v_filter`.
- :ref:`prefer_return`
- :ref:`resource_embedding`
- :ref:`v_filter`
- :ref:`Missing Preference <prefer_missing>`
- :ref:`specify_columns`
.. warning:: .. warning::
@@ -621,34 +623,18 @@ You can make an upsert with :code:`POST` and the :code:`Prefer: resolution=merge
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3000/products" \ curl "http://localhost:3000/employees" \
-X POST -H "Content-Type: application/json" \ -X POST -H "Content-Type: application/json" \
-H "Prefer: resolution=merge-duplicates" \ -H "Prefer: resolution=merge-duplicates" \
-d @- << EOF -d @- << EOF
[ [
{ "sku": "CL2031", "name": "Existing T-shirt", "price": 35 }, { "id": 1, "name": "Old employee 1", "salary": 30000 },
{ "sku": "CL2040", "name": "Existing Hoodie", "price": 60 }, { "id": 2, "name": "Old employee 2", "salary": 42000 },
{ "sku": "AC1022", "name": "New Cap", "price": 30 } { "id": 3, "name": "New employee 3", "salary": 50000 }
] ]
EOF EOF
By default, upsert operates based on the primary key columns, so you must specify all of them. By default, upsert operates based on the primary key columns, you must specify all of them. You can also choose to ignore the duplicates with :code:`Prefer: resolution=ignore-duplicates`. This works best when the primary key is natural, but it's also possible to use it if the primary key is surrogate (example: "id serial primary key"). For more details read `this issue <https://github.com/PostgREST/postgrest/issues/1118>`_.
You can also choose to ignore the duplicates with :code:`Prefer: resolution=ignore-duplicates`.
Upsert works best when the primary key is natural (e.g. ``sku``).
However, it can work with surrogate primary keys (e.g. ``id serial primary key``), if you also do a :ref:`bulk_insert` with :ref:`prefer_missing`:
.. code-block:: bash
curl "http://localhost:3000/employees?colums=id,name,salary" \
-X POST -H "Content-Type: application/json" \
-H "Prefer: resolution=merge-duplicates, missing=default" \
-d @- << EOF
[
{ "id": 1, "name": "Existing employee 1", "salary": 30000 },
{ "id": 2, "name": "Existing employee 2", "salary": 42000 },
{ "name": "New employee 3", "salary": 50000 }
]
EOF
.. important:: .. important::
After creating a table or changing its primary key, you must refresh PostgREST schema cache for upsert to work properly. To learn how to refresh the cache see :ref:`schema_reloading`. After creating a table or changing its primary key, you must refresh PostgREST schema cache for upsert to work properly. To learn how to refresh the cache see :ref:`schema_reloading`.
@@ -714,6 +700,31 @@ Deletions also support :ref:`prefer_return`, :ref:`resource_embedding` and :ref:
Beware of accidentally deleting all rows in a table. To learn to prevent that see :ref:`block_fulltable`. Beware of accidentally deleting all rows in a table. To learn to prevent that see :ref:`block_fulltable`.
.. _limited_update_delete:
Limited Update/Delete
=====================
You can limit the amount of affected rows by :ref:`update` or :ref:`delete` with the ``limit`` query parameter. For this, you must add an explicit ``order`` on a unique column(s).
.. code-block:: bash
curl -X PATCH "/users?limit=10&order=id&last_login=lt.2020-01-01" \
-H "Content-Type: application/json" \
-d '{ "status": "inactive" }'
.. code-block:: bash
curl -X DELETE "http://localhost:3000/users?limit=10&order=id&status=eq.inactive"
If your table has no unique columns, you can use the `ctid <https://www.postgresql.org/docs/current/ddl-system-columns.html>`_ system column.
Using ``offset`` to target a different subset of rows is also possible.
.. note::
There is no native ``UPDATE...LIMIT`` or ``DELETE...LIMIT`` support in PostgreSQL; the generated query simulates that behavior and is based on `this Crunchy Data blog post <https://www.crunchydata.com/blog/simulating-update-or-delete-with-limit-in-postgres-ctes-to-the-rescue>`_.
.. raw:: html .. raw:: html
<script type="text/javascript"> <script type="text/javascript">
@@ -728,7 +739,6 @@ Deletions also support :ref:`prefer_return`, :ref:`resource_embedding` and :ref:
'#estimated-count': 'pagination_count.html#estimated-count', '#estimated-count': 'pagination_count.html#estimated-count',
'#prefer-return-headers-only': 'preferences.html#headers-only', '#prefer-return-headers-only': 'preferences.html#headers-only',
'#prefer-return-representation': 'preferences.html#full', '#prefer-return-representation': 'preferences.html#full',
'#bulk-insert-default': 'preferences.html#prefer-missing',
}; };
let willRedirectTo = redirects[hash]; let willRedirectTo = redirects[hash];
+1 -1
View File
@@ -14,7 +14,7 @@ Custom Queries
The PostgREST URL grammar limits the kinds of queries clients can perform. It prevents arbitrary, potentially poorly constructed and slow client queries. It's good for quality of service, but means database administrators must create custom views and functions to provide richer endpoints. The most common causes for custom endpoints are The PostgREST URL grammar limits the kinds of queries clients can perform. It prevents arbitrary, potentially poorly constructed and slow client queries. It's good for quality of service, but means database administrators must create custom views and functions to provide richer endpoints. The most common causes for custom endpoints are
* SET operators like `UNION, INTERSECT and EXCEPT <https://www.postgresql.org/docs/current/queries-union.html>`_. * Table unions
* More complicated joins than those provided by :ref:`resource_embedding`. * More complicated joins than those provided by :ref:`resource_embedding`.
* Geo-spatial queries that require an argument, like "points near (lat,lon)" * Geo-spatial queries that require an argument, like "points near (lat,lon)"
+27 -145
View File
@@ -31,7 +31,7 @@ The authenticator role is used for connecting to the database and should be conf
.. _user_impersonation: .. _user_impersonation:
User Impersonation User Impersonation
~~~~~~~~~~~~~~~~~~ ------------------
The picture below shows how the server handles authentication. If auth succeeds, it switches into the user role specified by the request, otherwise it switches into the anonymous role (if it's set in :ref:`db-anon-role`). The picture below shows how the server handles authentication. If auth succeeds, it switches into the user role specified by the request, otherwise it switches into the anonymous role (if it's set in :ref:`db-anon-role`).
@@ -43,13 +43,12 @@ This role switching mechanism is called **user impersonation**. In PostgreSQL it
The impersonated roles will have their settings applied. See :ref:`impersonated_settings`. The impersonated roles will have their settings applied. See :ref:`impersonated_settings`.
.. _jwt_auth: .. _jwt_impersonation:
JWT Authentication JWT-Based User Impersonation
------------------ ----------------------------
We use `JSON Web Tokens <https://datatracker.ietf.org/doc/html/rfc7519/>`_ to authenticate API requests, this allows us to be stateless and not require database lookups for verification. We use `JSON Web Tokens <https://jwt.io/>`_ to authenticate API requests, this allows us to be stateless and not require database lookups for verification. As you'll recall a JWT contains a list of cryptographically signed claims. All claims are allowed but PostgREST cares specifically about a claim called role.
As you'll recall a JWT contains a list of cryptographically signed claims. All claims are allowed but PostgREST cares specifically about a claim called role (configurable with :ref:`jwt_role_extract`).
.. code:: json .. code:: json
@@ -73,10 +72,17 @@ Note that the database administrator must allow the authenticator role to switch
If the client included no JWT (or one without a role claim) then PostgREST switches into the anonymous role. The database administrator must set the anonymous role permissions correctly to prevent anonymous users from seeing or changing things they shouldn't. If the client included no JWT (or one without a role claim) then PostgREST switches into the anonymous role. The database administrator must set the anonymous role permissions correctly to prevent anonymous users from seeing or changing things they shouldn't.
.. _bearer_auth: .. _jwt_generation:
Bearer Authentication JWT Generation
~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
You can create a valid JWT either from inside your database (see :ref:`sql_user_management`) or via an external service (see :ref:`external_jwt`).
.. _client_auth:
Client Auth
~~~~~~~~~~~
To make an authenticated request the client must include an :code:`Authorization` HTTP header with the value :code:`Bearer <jwt>`. For instance: To make an authenticated request the client must include an :code:`Authorization` HTTP header with the value :code:`Bearer <jwt>`. For instance:
@@ -87,29 +93,24 @@ To make an authenticated request the client must include an :code:`Authorization
The ``Bearer`` header value can be used with or without capitalization(``bearer``). The ``Bearer`` header value can be used with or without capitalization(``bearer``).
.. _jwt_generation: .. _jwt_caching:
JWT Generation JWT Caching
~~~~~~~~~~~~~~ -----------
You can create a valid JWT either from inside your database (see :ref:`sql_user_management`) or via an external service (see :ref:`external_auth`). PostgREST validates ``JWTs`` on every request. We can cache ``JWTs`` to avoid this performance overhead.
.. _jwt_signature: To enable JWT caching, the config :code:`jwt-cache-max-lifetime` is to be set. It is the maximum number of seconds for which the cache stores the JWT validation results. The cache uses the :code:`exp` claim to set the cache entry lifetime. If the JWT does not have an :code:`exp` claim, it uses the config value. See :ref:`jwt-cache-max-lifetime` for more details.
JWT Signature Verification .. note::
--------------------------
PostgREST supports both symmetric and asymmetric keys for verifying the signature of the token. You can use the :ref:`server-timing_header` to see the effect of JWT caching.
Symmetric Keys Symmetric Keys
~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
In the case of symmetric cryptography the signer and verifier share the same secret passphrase, which can be configured with :ref:`jwt-secret`. Each token is cryptographically signed with a secret key. In the case of symmetric cryptography the signer and verifier share the same secret passphrase, which can be configured with :ref:`jwt-secret`.
If it is set to a simple string then PostgREST interprets it as an HMAC-SHA256 passphrase. If it is set to a simple string value like “reallyreallyreallyreallyverysafe” then PostgREST interprets it as an HMAC-SHA256 passphrase.
.. code-block:: ini
jwt-secret = "reallyreallyreallyreallyverysafe"
.. _asym_keys: .. _asym_keys:
@@ -155,112 +156,13 @@ You can specify the literal value as we saw earlier, or reference a filename to
jwt-secret = "@rsa.jwk.pub" jwt-secret = "@rsa.jwk.pub"
``kid`` verification
^^^^^^^^^^^^^^^^^^^^
PostgREST has built-in verification of the `key ID parameter <https://www.rfc-editor.org/rfc/rfc7517#section-4.5>`_, useful when working with a JSON Web Key Set.
It goes as follows:
- If the JWT contains a ``kid`` parameter, then PostgREST will look for the JSON Web Key in the :ref:`jwt-secret`.
+ If no key has a matching ``kid`` (or if they don't have one defined), the token will be rejected with a :ref:`401 Unauthorized <pgrst301>` error.
+ If a key matches the ``kid`` value then it will validate the token against that key accordingly.
- If the JWT doesn't have a ``kid``, PostgREST will try each key in the :ref:`jwt-secret` one by one until it finds one that works.
.. _jwt_claims_validation:
JWT Claims Validation JWT Claims Validation
--------------------- ~~~~~~~~~~~~~~~~~~~~~
Time-Based claims validation PostgREST honors the :code:`exp` claim for token expiration, rejecting expired tokens.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The time-based JWT claims specified in `RFC 7519 <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.4>`_ are validated:
- ``exp`` Expiration Time
- ``iat`` Issued At
- ``nbf`` Not Before
We allow a 30-second clock skew when validating the above claims. In other words, we give an extra 30 seconds before the JWT is rejected if there is a slight discrepancy in the timestamps.
.. _jwt_aud:
``aud`` validation
~~~~~~~~~~~~~~~~~~
PostgREST has built-in validation of the `JWT audience claim <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.3>`_.
It works this way:
- If :ref:`jwt-aud` is not set (the default), PostgREST identifies with all audiences and allows the JWT for any ``aud`` claim.
- If :ref:`jwt-aud` is set to a specific audience, PostgREST will check if this audience is present in the ``aud`` claim:
+ If the ``aud`` value is a JSON string, it will match it to the :ref:`jwt-aud`.
+ If the ``aud`` value is a JSON array of strings, it will search every element for a match.
+ If the match fails or if the ``aud`` value is not a string or array of strings, then the token will be rejected with a :ref:`401 Unauthorized <pgrst303>` error.
+ If the ``aud`` key **is not present** or if its value is ``null`` or ``[]``, PostgREST will interpret this token as allowed for all audiences and will complete the request.
.. _jwt_caching:
JWT Cache
---------
JWT signature validation (specially :ref:`asym_keys` such as RSA) is slow, we can cache ``JWT`` validation results to avoid this performance overhead.
The JWT cache is bounded and uses the `SIEVE algorithm <https://cachemon.github.io/SIEVE-website>`_ for efficient eviction. The cache is enabled by default and can be configured with :ref:`jwt-cache-max-entries`.
It's recommended to leave the JWT cache enabled as our load tests indicate ~20% more throughput for simple GET requests when using it. This while reducing CPU utilization in exchange for a bit more memory.
:ref:`jwt_cache_metrics` are available.
.. note::
- If the ``jwt-secret`` is changed and the config is reloaded, the JWT cache will reset.
- JWTs that pass :ref:`jwt_signature` are cached, regardless if they pass :ref:`jwt_claims_validation`. We do this to ensure responses stays fast under common failure cases (such as expired JWTs).
- You can use the :ref:`server-timing_header` to see the peformance benefit of JWT caching.
.. _jwt_role_extract:
JWT Role Extraction
-------------------
A JSPath DSL that specifies the location of the :code:`role` key in the JWT claims. It's configured by :ref:`jwt-role-claim-key`. This can be used to consume a JWT provided by a third party service like Auth0, Okta, Microsoft Entra or Keycloak.
The DSL follows the `JSONPath <https://goessner.net/articles/JsonPath/>`_ expression grammar with extended string comparison operators. Supported operators are:
- ``==`` selects the first array element that exactly matches the right operand
- ``!=`` selects the first array element that does not match the right operand
- ``^==`` selects the first array element that starts with the right operand
- ``==^`` selects the first array element that ends with the right operand
- ``*==`` selects the first array element that contains the right operand
Usage examples:
.. code:: bash
# {"postgrest":{"roles": ["other", "author"]}}
# the DSL accepts characters that are alphanumerical or one of "_$@" as keys
jwt-role-claim-key = ".postgrest.roles[1]"
# {"https://www.example.com/role": { "key": "author" }}
# non-alphanumerical characters can go inside quotes(escaped in the config value)
jwt-role-claim-key = ".\"https://www.example.com/role\".key"
# {"postgrest":{"roles": ["other", "author"]}}
# `@` represents the current element in the array
# all the these match the string "author"
jwt-role-claim-key = ".postgrest.roles[?(@ == \"author\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ != \"other\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ ^== \"aut\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ ==^ \"hor\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ *== \"utho\")]"
.. note::
The string comparison operators are implemented as a custom extension to the JSPath and does not strictly follow the `RFC 9535 <https://www.rfc-editor.org/rfc/rfc9535.html>`_.
JWT Security JWT Security
------------ ~~~~~~~~~~~~
There are at least three types of common critiques against using JWT: 1) against the standard itself, 2) against using libraries with known security vulnerabilities, and 3) against using JWT for web sessions. We'll briefly explain each critique, how PostgREST deals with it, and give recommendations for appropriate user action. There are at least three types of common critiques against using JWT: 1) against the standard itself, 2) against using libraries with known security vulnerabilities, and 3) against using JWT for web sessions. We'll briefly explain each critique, how PostgREST deals with it, and give recommendations for appropriate user action.
@@ -296,23 +198,3 @@ doing custom logic based on the web user info.
END IF; END IF;
END END
$$ LANGUAGE plpgsql; $$ LANGUAGE plpgsql;
.. raw:: html
<script type="text/javascript">
let hash = window.location.hash;
const redirects = {
'#jwt-based-user-impersonation': '#jwt-authentication',
'#client-auth': '#bearer-authentication',
'#jwt-caching': '#jwt-cache',
'#jwk-kid-validation': '#kid-verification',
'#jwt-aud-claim-validation': '#aud-validation',
};
let willRedirectTo = redirects[hash];
if (willRedirectTo) {
window.location.href = willRedirectTo;
}
</script>
+8 -46
View File
@@ -3,47 +3,23 @@
CLI CLI
=== ===
PostgREST provides a CLI with the options listed below: PostgREST provides a CLI with the commands listed below:
.. code:: text
Usage: postgrest [-v|--version] [-e|--example] [--dump-config | --dump-schema | --ready]
[FILENAME]
PostgREST / create a REST API to an existing Postgres
database
Available options:
-h,--help Show this help text
-v,--version Show the version information
-e,--example Show an example configuration file
--dump-config Dump loaded configuration and exit
--dump-schema Dump loaded schema as JSON and exit (for debugging,
output structure is unstable)
--ready Checks the health of PostgREST by doing a request on
the admin server /ready endpoint
FILENAME Path to configuration file
FILENAME
--------
Runs PostgREST with the given :ref:`file_config`.
Help Help
---- ----
.. code:: bash .. code:: bash
$ postgrest --help $ postgrest [-h|--help]
Shows all the options available. Shows all the commands available.
Version Version
------- -------
.. code:: bash .. code:: bash
$ postgrest --version $ postgrest [-v|--version]
Prints the PostgREST version. Prints the PostgREST version.
@@ -52,16 +28,16 @@ Example
.. code:: bash .. code:: bash
$ postgrest --example $ postgrest [-e|--example]
Shows example configuration settings. Shows example configuration options.
Dump Config Dump Config
----------- -----------
.. code:: bash .. code:: bash
$ postgrest --dump-config $ postgrest [--dump-config]
Dumps the loaded :ref:`configuration` values, considering the configuration file, environment variables and :ref:`in_db_config`. Dumps the loaded :ref:`configuration` values, considering the configuration file, environment variables and :ref:`in_db_config`.
@@ -70,20 +46,6 @@ Dump Schema
.. code:: bash .. code:: bash
$ postgrest --dump-schema $ postgrest [--dump-schema]
Dumps the schema cache in JSON format. Dumps the schema cache in JSON format.
Ready Flag
----------
Makes a request to the ``/ready`` endpoint of the :ref:`admin_server`. It exits with a return code of ``0`` on success and ``1`` on failure.
.. code-block:: bash
$ postgrest --ready
OK: http://localhost:3001/ready
.. note::
The ``--ready`` flag cannot be used when :ref:`server-host` is configured with special hostnames. We suggest to change it to ``localhost``.
+25 -55
View File
@@ -146,21 +146,6 @@ To reload the configuration from within the database, you can use the ``NOTIFY``
List of parameters List of parameters
================== ==================
.. _admin-server-host:
admin-server-host
-----------------
=============== =======================
**Type** String
**Default** `server-host` value
**Reloadable** N
**Environment** PGRST_ADMIN_SERVER_HOST
**In-Database** `n/a`
=============== =======================
Specifies the host for the :ref:`admin_server`. Defaults to :ref:`server-host` value.
.. _admin-server-port: .. _admin-server-port:
admin-server-port admin-server-port
@@ -174,7 +159,7 @@ admin-server-port
**In-Database** `n/a` **In-Database** `n/a`
=============== ======================= =============== =======================
Specifies the port for the :ref:`admin_server`. Cannot be equal to :ref:`server-port`. Specifies the port for the :ref:`admin_server`.
.. _app.settings.*: .. _app.settings.*:
@@ -191,10 +176,6 @@ app.settings.*
Arbitrary settings that can be used to pass in secret keys directly as strings, or via OS environment variables. For instance: :code:`app.settings.jwt_secret = "$(MYAPP_JWT_SECRET)"` will take :code:`MYAPP_JWT_SECRET` from the environment and make it available to PostgreSQL functions as :code:`current_setting('app.settings.jwt_secret')`. Arbitrary settings that can be used to pass in secret keys directly as strings, or via OS environment variables. For instance: :code:`app.settings.jwt_secret = "$(MYAPP_JWT_SECRET)"` will take :code:`MYAPP_JWT_SECRET` from the environment and make it available to PostgreSQL functions as :code:`current_setting('app.settings.jwt_secret')`.
When using the environment variable `PGRST_APP_SETTINGS_*` form, the remainder of the variable is used as the new name. Case is not important : :code:`PGRST_APP_SETTINGS_MY_ENV_VARIABLE=some_value` can be accessed in postgres as :code:`current_setting('app.settings.my_env_variable')`.
The :code:`current_setting` function has `an optional boolean second <https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-SET>`_ argument to avoid it from raising an error if the value was not defined. Default values to :code:`app.settings` can then be given by combining this argument with :code:`coalesce` and :code:`nullif` : :code:`coalesce(nullif(current_setting('app.settings.my_custom_variable', true), ''), 'default value')`. The use of :code:`nullif` is necessary because if set in a transaction, the setting is sometimes not "rolled back" to :code:`null`. See also :ref:`this section <guc_req_headers_cookies_claims>` for more information on this behaviour.
.. _db-aggregates-enabled: .. _db-aggregates-enabled:
db-aggregates-enabled db-aggregates-enabled
@@ -315,10 +296,6 @@ db-extra-search-path
Multiple schemas can be added in a comma-separated string, e.g. ``public, extensions``. Multiple schemas can be added in a comma-separated string, e.g. ``public, extensions``.
.. important::
We default this config to ``public`` because it is the most common schema used to install PostgreSQL extensions such as :ref:`PostGIS <ww_postgis>`. You can disable this by setting this config to ``""``.
.. _db-hoisted-tx-settings: .. _db-hoisted-tx-settings:
db-hoisted-tx-settings db-hoisted-tx-settings
@@ -409,7 +386,7 @@ db-pool-max-idletime
**In-Database** `n/a` **In-Database** `n/a`
=============== ================================= =============== =================================
*For backwards compatibility, this config parameter is also available as "db-pool-timeout".* *For backwards compatibility, this config parameter is also available as db-pool-timeout.*
Time in seconds to close idle pool connections. Time in seconds to close idle pool connections.
@@ -603,7 +580,11 @@ jwt-aud
**In-Database** pgrst.jwt_aud **In-Database** pgrst.jwt_aud
=============== ================================= =============== =================================
Specifies an audience for the JWT ``aud`` claim. See :ref:`jwt_aud`. Specifies the `JWT audience claim <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.3>`_. If this claim is present in the client provided JWT then you must set this to the same value as in the JWT, otherwise verifying the JWT will fail.
.. warning::
Using this setting will only reject tokens with a different audience claim. Tokens **without** audience claim will still be accepted.
.. _jwt-role-claim-key: .. _jwt-role-claim-key:
@@ -620,7 +601,17 @@ jwt-role-claim-key
*For backwards compatibility, this config parameter is also available without prefix as "role-claim-key".* *For backwards compatibility, this config parameter is also available without prefix as "role-claim-key".*
See :ref:`jwt_role_extract` on how to specify key paths and usage examples. A JSPath DSL that specifies the location of the :code:`role` key in the JWT claims. This can be used to consume a JWT provided by a third party service like Auth0, Okta or Keycloak. Usage examples:
.. code:: bash
# {"postgrest":{"roles": ["other", "author"]}}
# the DSL accepts characters that are alphanumerical or one of "_$@" as keys
jwt-role-claim-key = ".postgrest.roles[1]"
# {"https://www.example.com/role": { "key": "author }}
# non-alphanumerical characters can go inside quotes(escaped in the config value)
jwt-role-claim-key = ".\"https://www.example.com/role\".key"
.. _jwt-secret: .. _jwt-secret:
@@ -658,20 +649,20 @@ jwt-secret-is-base64
When this is set to :code:`true`, the value derived from :code:`jwt-secret` will be treated as a base64 encoded secret. When this is set to :code:`true`, the value derived from :code:`jwt-secret` will be treated as a base64 encoded secret.
.. _jwt-cache-max-entries: .. _jwt-cache-max-lifetime:
jwt-cache-max-entries jwt-cache-max-lifetime
---------------------- ----------------------
=============== ================================= =============== =================================
**Type** Int **Type** Int
**Default** 1000 **Default** 0
**Reloadable** Y **Reloadable** Y
**Environment** PGRST_JWT_CACHE_MAX_ENTRIES **Environment** PGRST_JWT_CACHE_MAX_LIFETIME
**In-Database** pgrst.jwt_cache_max_entries **In-Database** pgrst.jwt_cache_max_lifetime
=============== ================================= =============== =================================
Maximum number of entries in JWT cache. The value :code:`0` disables JWT caching. See :ref:`jwt_caching`. Maximum number of seconds of lifetime for cached entries. The default :code:`0` disables caching. See :ref:`jwt_caching`.
.. _log-level: .. _log-level:
@@ -708,21 +699,6 @@ log-level
Because currently there's no buffering for logging, the levels with minimal logging(``crit/error``) will increase throughput. Because currently there's no buffering for logging, the levels with minimal logging(``crit/error``) will increase throughput.
.. _log-query:
log-query
---------
=============== =================================
**Type** Boolean
**Default** False
**Reloadable** Y
**Environment** PGRST_LOG_QUERY
**In-Database** `n/a`
=============== =================================
Logs the SQL query for the corresponding request at the current :ref:`log-level`. See :ref:`sql_query_logs`.
.. _openapi-mode: .. _openapi-mode:
openapi-mode openapi-mode
@@ -836,12 +812,6 @@ server-host
* :code:`*6` - any IPv4 or IPv6 hostname, IPv6 preferred * :code:`*6` - any IPv4 or IPv6 hostname, IPv6 preferred
* :code:`!6` - any IPv6 hostname * :code:`!6` - any IPv6 hostname
Examples:
.. code:: bash
server-host = "127.0.0.1"
.. _server-port: .. _server-port:
server-port server-port
@@ -885,7 +855,7 @@ server-timing-enabled
**In-Database** pgrst.server_timing_enabled **In-Database** pgrst.server_timing_enabled
=============== ================================= =============== =================================
Enables the `Server-Timing <https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Server-Timing>`_ header. Enables the `Server-Timing <https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing>`_ header.
See :ref:`server-timing_header`. See :ref:`server-timing_header`.
.. _server-unix-socket: .. _server-unix-socket:
+2 -2
View File
@@ -15,7 +15,7 @@ Dynamic Connection Pool
To conserve system resources, PostgREST uses a dynamic connection pool. This enables the number of connections in the pool to increase and decrease depending on request traffic. To conserve system resources, PostgREST uses a dynamic connection pool. This enables the number of connections in the pool to increase and decrease depending on request traffic.
- If all the connections are being used, a new connection is added. The pool can grow until it reaches the :ref:`db-pool` size. Note that it's pointless to set this higher than the ``max_connections`` setting in your database. - If all the connections are being used, a new connection is added. The pool can grow until it reaches the :ref:`db-pool` size. Note that its pointless to set this higher than the ``max_connections`` setting in your database.
- If a connection is unused for a period of time (:ref:`db-pool-max-idletime`), it will be released. - If a connection is unused for a period of time (:ref:`db-pool-max-idletime`), it will be released.
- For connecting to the database, the :ref:`authenticator <roles>` role is used. You can configure this using :ref:`db-uri`. - For connecting to the database, the :ref:`authenticator <roles>` role is used. You can configure this using :ref:`db-uri`.
@@ -106,4 +106,4 @@ Also set :ref:`db-channel-enabled` to ``false`` since ``LISTEN`` is not compatib
.. note:: .. note::
It's not recommended to use an external connection pooler. `Our benchmarks <https://github.com/PostgREST/postgrest/issues/2294#issuecomment-1139148672>`_ indicate it provides much lower performance than PostgREST built-in pool. Its not recommended to use an external connection pooler. `Our benchmarks <https://github.com/PostgREST/postgrest/issues/2294#issuecomment-1139148672>`_ indicate it provides much lower performance than PostgREST built-in pool.
+15 -61
View File
@@ -5,8 +5,6 @@ Errors
PostgREST error messages follow the PostgreSQL error structure. It includes ``MESSAGE``, ``DETAIL``, ``HINT``, ``ERRCODE`` and will add an HTTP status code to the response. PostgREST error messages follow the PostgreSQL error structure. It includes ``MESSAGE``, ``DETAIL``, ``HINT``, ``ERRCODE`` and will add an HTTP status code to the response.
.. _postgresql_errors:
Errors from PostgreSQL Errors from PostgreSQL
====================== ======================
@@ -207,6 +205,14 @@ Related to the HTTP request elements.
| | | specified in the ``select`` part of the query string. | | | | specified in the ``select`` part of the query string. |
| PGRST108 | | See :ref:`embed_filters`. | | PGRST108 | | See :ref:`embed_filters`. |
+---------------+-------------+-------------------------------------------------------------+ +---------------+-------------+-------------------------------------------------------------+
| .. _pgrst109: | 400 | Restricting a Deletion or an Update using limits must |
| | | include the ordering of a unique column. |
| PGRST109 | | See :ref:`limited_update_delete`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst110: | 400 | When restricting a Deletion or an Update using limits |
| | | modifies more rows than the maximum specified in the limit. |
| PGRST110 | | See :ref:`limited_update_delete`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst111: | 500 | An invalid ``response.headers`` was set. | | .. _pgrst111: | 500 | An invalid ``response.headers`` was set. |
| | | See :ref:`guc_resp_hdrs`. | | | | See :ref:`guc_resp_hdrs`. |
| PGRST111 | | | | PGRST111 | | |
@@ -235,6 +241,10 @@ Related to the HTTP request elements.
| | | there is no many-to-one or one-to-one relationship between | | | | there is no many-to-one or one-to-one relationship between |
| PGRST118 | | them. | | PGRST118 | | them. |
+---------------+-------------+-------------------------------------------------------------+ +---------------+-------------+-------------------------------------------------------------+
| .. _pgrst119: | 400 | Could not use the spread operator on the related table |
| | | because there is no many-to-one or one-to-one relationship |
| PGRST119 | | between them. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst120: | 400 | An embedded resource can only be filtered using the | | .. _pgrst120: | 400 | An embedded resource can only be filtered using the |
| | | ``is.null`` or ``not.is.null`` :ref:`operators <operators>`.| | | | ``is.null`` or ``not.is.null`` :ref:`operators <operators>`.|
| PGRST120 | | | | PGRST120 | | |
@@ -247,31 +257,6 @@ Related to the HTTP request elements.
| | | ``Prefer: handling=strict``. See :ref:`prefer_handling`. | | | | ``Prefer: handling=strict``. See :ref:`prefer_handling`. |
| PGRST122 | | | | PGRST122 | | |
+---------------+-------------+-------------------------------------------------------------+ +---------------+-------------+-------------------------------------------------------------+
| .. _pgrst123: | 400 | Aggregate functions are disabled. |
| | | See :ref:`db-aggregates-enabled`. |
| PGRST123 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst124: | 400 | ``max-affected`` preference is violated. |
| | | See :ref:`prefer_max_affected`. |
| PGRST124 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst125: | 404 | Invalid path is specified in request URL. |
| | | |
| PGRST125 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst126: | 404 | Open API config is disabled but API root path is |
| | | accessed. See :ref:`openapi-mode`. |
| PGRST126 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst127: | 400 | The feature specified in the ``details`` field is not |
| | | implemented. |
| PGRST127 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst128: | 400 | ``max-affected`` preference is violated with ``RPC`` call. |
| | | See :ref:`prefer_max_affected`. |
| PGRST128 | | |
+---------------+-------------+-------------------------------------------------------------+
.. _pgrst2**: .. _pgrst2**:
@@ -305,10 +290,6 @@ Related to a :ref:`schema_cache`. Most of the time, these errors are solved by :
| | | in the ``columns`` query parameter is not found. | | | | in the ``columns`` query parameter is not found. |
| PGRST204 | | | | PGRST204 | | |
+---------------+-------------+-------------------------------------------------------------+ +---------------+-------------+-------------------------------------------------------------+
| .. _pgrst205: | 404 | Caused when the :ref:`table specified <tables_views>` in |
| | | the URI is not found. |
| PGRST205 | | |
+---------------+-------------+-------------------------------------------------------------+
.. _pgrst3**: .. _pgrst3**:
@@ -324,18 +305,14 @@ Related to the authentication process using JWT. You can follow the :ref:`tut1`
| | | configuration. | | | | configuration. |
| PGRST300 | | | | PGRST300 | | |
+---------------+-------------+-------------------------------------------------------------+ +---------------+-------------+-------------------------------------------------------------+
| .. _pgrst301: | 401 | Provided JWT couldn't be decoded or it is invalid. | | .. _pgrst301: | 401 | Any error related to the verification of the JWT, |
| | | | | | | which means that the JWT provided is invalid in some way. |
| PGRST301 | | | | PGRST301 | | |
+---------------+-------------+-------------------------------------------------------------+ +---------------+-------------+-------------------------------------------------------------+
| .. _pgrst302: | 401 | Attempted to do a request without | | .. _pgrst302: | 401 | Attempted to do a request without |
| | | :ref:`bearer_auth` when the anonymous role | | | | :ref:`authentication <client_auth>` when the anonymous role |
| PGRST302 | | is disabled by not setting it in :ref:`db-anon-role`. | | PGRST302 | | is disabled by not setting it in :ref:`db-anon-role`. |
+---------------+-------------+-------------------------------------------------------------+ +---------------+-------------+-------------------------------------------------------------+
| .. _pgrst303: | 401 | :ref:`JWT claims validation <jwt_claims_validation>` |
| | | or parsing failed. |
| PGRST303 | | |
+---------------+-------------+-------------------------------------------------------------+
.. The Internal Errors Group X** is always at the end .. The Internal Errors Group X** is always at the end
@@ -354,8 +331,6 @@ Internal errors. If you encounter any of these, you may have stumbled on a Postg
| PGRSTX00 | | | | PGRSTX00 | | |
+---------------+-------------+-------------------------------------------------------------+ +---------------+-------------+-------------------------------------------------------------+
.. _custom_errors:
Custom Errors Custom Errors
============= =============
@@ -452,24 +427,3 @@ For non standard HTTP status, you can optionally add ``status_text`` to describe
detail = '{"status":419,"status_text":"Page Expired","headers":{"X-Powered-By":"Nerd Rage"}}'; detail = '{"status":419,"status_text":"Page Expired","headers":{"X-Powered-By":"Nerd Rage"}}';
If PostgREST can't parse the JSON objects ``message`` and ``detail``, it will throw a ``PGRST121`` error. See :ref:`Errors from PostgREST<pgrst1**>`. If PostgREST can't parse the JSON objects ``message`` and ``detail``, it will throw a ``PGRST121`` error. See :ref:`Errors from PostgREST<pgrst1**>`.
.. _proxy-status_header:
Proxy-Status Header
===================
For error cases, the standard `Proxy-Status <https://www.rfc-editor.org/rfc/rfc9209.html#name-the-proxy-status-http-field>`_ header is returned with the error code. The error code comes from either :ref:`PostgREST <pgrst_errors>`, :ref:`PostgreSQL <postgresql_errors>` or :ref:`Custom <custom_errors>` errors. This is useful when doing ``HEAD`` requests where the HTTP status is not descriptive enough.
For example, doing a request on a table with high count (say 30_000_000), we get:
.. code-block:: http
HEAD /table HTTP/1.1
Prefer: count=exact
.. code-block:: http
HTTP/1.1 500 Internal Server Error
Proxy-Status: PostgREST; error=57014
The PostgreSQL error code ``57014`` (`ref <https://www.postgresql.org/docs/current/errcodes-appendix.html>`_) reveals that the error is due to a short ``statement_timeout`` value.
+2 -4
View File
@@ -4,7 +4,7 @@ Listener
######## ########
PostgREST uses `LISTEN <https://www.postgresql.org/docs/current/sql-listen.html>`_ to reload its :ref:`Schema Cache <schema_reloading_notify>` and :ref:`Configuration <config_reloading_notify>` via `NOTIFY <https://www.postgresql.org/docs/current/sql-notify.html>`_. PostgREST uses `LISTEN <https://www.postgresql.org/docs/current/sql-listen.html>`_ to reload its :ref:`Schema Cache <schema_reloading_notify>` and :ref:`Configuration <config_reloading_notify>` via `NOTIFY <https://www.postgresql.org/docs/current/sql-notify.html>`_.
This is useful in environments where you can't send SIGUSR1 or SIGUSR2 Unix Signals. This is useful in environments where you cant send SIGUSR1 or SIGUSR2 Unix Signals.
Like on cloud managed containers or on Windows systems. Like on cloud managed containers or on Windows systems.
.. code:: postgresql .. code:: postgresql
@@ -46,9 +46,7 @@ This will cause the :ref:`connection_pool` to connect to the read replica host a
.. note:: .. note::
- Under the hood, PostgREST forces `target_session_attrs=read-write <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS>`_ for the ``LISTEN`` session. Under the hood, PostgREST forces `target_session_attrs=read-write <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS>`_ for the ``LISTEN`` session.
So if you specify ``target_session_attrs=read-only`` as mentioned above, PostgREST will override it for the ``LISTEN``.
- ``read-only`` is only available on libpq >= 14, if you use a lower version you will get an error like ``invalid target_session_attrs value: \"read-only\"``.
.. _listener_automatic_recovery: .. _listener_automatic_recovery:
+13 -108
View File
@@ -15,14 +15,12 @@ Observability allows measuring a system's current state based on the data it gen
Logs Logs
==== ====
PostgREST logs basic request information to ``stdout``, including the authenticated user if available, the requesting IP address and user agent, the URL requested, the HTTP response status and the response body size in bytes if available. PostgREST logs basic request information to ``stdout``, including the authenticated user if available, the requesting IP address and user agent, the URL requested, and HTTP response status.
With :ref:`log-level` set to ``info``, we get:
.. code:: .. code::
127.0.0.1 - user [26/Jul/2021:01:56:38 -0500] "GET /clients HTTP/1.1" 200 56 "" "curl/7.64.0" 127.0.0.1 - user [26/Jul/2021:01:56:38 -0500] "GET /clients HTTP/1.1" 200 - "" "curl/7.64.0"
127.0.0.1 - anonymous [26/Jul/2021:01:56:48 -0500] "GET /unexistent HTTP/1.1" 404 162 "" "curl/7.64.0" 127.0.0.1 - anonymous [26/Jul/2021:01:56:48 -0500] "GET /unexistent HTTP/1.1" 404 - "" "curl/7.64.0"
For diagnostic information about the server itself, PostgREST logs to ``stderr``: For diagnostic information about the server itself, PostgREST logs to ``stderr``:
@@ -33,50 +31,22 @@ For diagnostic information about the server itself, PostgREST logs to ``stderr``
.. code:: .. code::
06/May/2024:08:16:11 -0500: Starting PostgREST 12.1... 06/May/2024:08:16:11 -0500: Starting PostgREST 12.1...
06/May/2024:08:16:11 -0500: Attempting to connect to the database...
06/May/2024:08:16:11 -0500: Successfully connected to PostgreSQL 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit 06/May/2024:08:16:11 -0500: Successfully connected to PostgreSQL 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit
06/May/2024:08:16:11 -0500: Connection Pool initialized with a maximum size of 10 connections 06/May/2024:08:16:11 -0500: Listening on port 3000
06/May/2024:08:16:11 -0500: API server listening on port 3000 06/May/2024:08:16:11 -0500: Listening for notifications on the "pgrst" channel
06/May/2024:08:16:11 -0500: Listening for database notifications on the "pgrst" channel
06/May/2024:08:16:11 -0500: Config reloaded 06/May/2024:08:16:11 -0500: Config reloaded
06/May/2024:08:16:11 -0500: Schema cache queried in 3.8 milliseconds 06/May/2024:08:16:11 -0500: Schema cache queried in 3.8 milliseconds
06/May/2024:08:16:11 -0500: Schema cache loaded 15 Relations, 8 Relationships, 8 RPCs, 0 Domain Representations, 4 Media Type Handlers 06/May/2024:08:16:11 -0500: Schema cache loaded 15 Relations, 8 Relationships, 8 Functions, 0 Domain Representations, 4 Media Type Handlers
06/May/2024:14:11:27 -0500: Received a config reload message on the "pgrst" channel 06/May/2024:14:11:27 -0500: Received a config reload message on the "pgrst" channel
06/May/2024:14:11:27 -0500: Config reloaded 06/May/2024:14:11:27 -0500: Config reloaded
.. note::
Logs are based on the ``log-level`` setting. See :ref:`log-level`.
.. _sql_query_logs:
SQL Query Logs
--------------
To log the SQL queries executed for a request, set the :ref:`log-query` to ``true``. It will be logged based on the current :ref:`log-level` setting.
.. code-block:: bash
log-level = "warn"
log-query = "true"
The SQL queries will only be logged on ``400`` HTTP errors and up.
So, if the user requests a resource without sufficient privileges:
.. code-block:: bash
curl "localhost:3000/protected_table"
This will be logged by PostgREST:
.. code::
17/Feb/2025:17:28:15 -0500: WITH pgrst_source AS ( SELECT "public"."protected_table".* FROM "public"."protected_table" ) SELECT null::bigint AS total_result_set, pg_catalog.count(_postgrest_t) AS page_total, coalesce(json_agg(_postgrest_t), '[]') AS body, nullif(current_setting('response.headers', true), '') AS response_headers, nullif(current_setting('response.status', true), '') AS response_status, '' AS response_inserted FROM ( SELECT * FROM pgrst_source ) _postgrest_t
127.0.0.1 - web_anon [17/Feb/2025:17:28:15 -0500] "GET /protected_table HTTP/1.1" 401 99 "" "curl/8.7.1"
Database Logs Database Logs
------------- -------------
Additionally, to find all the SQL operations, you can watch the database logs. By default PostgreSQL does not keep these logs, so you'll need to make the configuration changes below. Currently PostgREST doesn't log the SQL commands executed against the underlying database.
To find the SQL operations, you can watch the database logs. By default PostgreSQL does not keep these logs, so you'll need to make the configuration changes below.
Find :code:`postgresql.conf` inside your PostgreSQL data directory (to find that, issue the command :code:`show data_directory;`). Either find the settings scattered throughout the file and change them to the following values, or append this block of code to the end of the configuration file. Find :code:`postgresql.conf` inside your PostgreSQL data directory (to find that, issue the command :code:`show data_directory;`). Either find the settings scattered throughout the file and change them to the following values, or append this block of code to the end of the configuration file.
@@ -120,17 +90,12 @@ Restart the database and watch the log file in real-time to understand how HTTP
Metrics Metrics
======= =======
The ``metrics`` endpoint on the :ref:`admin_server` endpoint provides metrics in `Prometheus text format <https://prometheus.io/docs/instrumenting/exposition_formats/#prometheus-text-format>`_. The ``metrics`` endpoint on the :ref:`admin_server` endpoint provides metrics in `Prometheus text format <https://prometheus.io/docs/instrumenting/exposition_formats/#text-based-format>`_.
.. code-block:: bash .. code-block:: bash
curl "http://localhost:3001/metrics" curl "http://localhost:3001/metrics"
.. code-block:: http
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
# HELP pgrst_schema_cache_query_time_seconds The query time in seconds of the last schema cache load # HELP pgrst_schema_cache_query_time_seconds The query time in seconds of the last schema cache load
# TYPE pgrst_schema_cache_query_time_seconds gauge # TYPE pgrst_schema_cache_query_time_seconds gauge
pgrst_schema_cache_query_time_seconds 1.5937927e-2 pgrst_schema_cache_query_time_seconds 1.5937927e-2
@@ -204,40 +169,6 @@ pgrst_db_pool_max
Max pool connections. Max pool connections.
.. _jwt_cache_metrics:
JWT Cache Metrics
-----------------
Metrics related to the :ref:`jwt_caching`.
pgrst_jwt_cache_requests_total
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
======== =======
**Type** Counter
======== =======
The total number of JWT cache lookups.
pgrst_jwt_cache_hits_total
~~~~~~~~~~~~~~~~~~~~~~~~~~
======== =======
**Type** Counter
======== =======
The total number of JWT cache hits.
pgrst_jwt_cache_evictions_total
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
======== =======
**Type** Counter
======== =======
The total number of JWT cache evictions.
Traces Traces
====== ======
@@ -273,17 +204,12 @@ You can enable tracing HTTP requests by setting :ref:`server-trace-header`. Spec
HTTP/1.1 200 OK HTTP/1.1 200 OK
X-Request-Id: 123 X-Request-Id: 123
Proxy-Status Header
-------------------
See :ref:`proxy-status_header`.
.. _server-timing_header: .. _server-timing_header:
Server-Timing Header Server-Timing Header
-------------------- --------------------
You can enable the `Server-Timing <https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Server-Timing>`_ header by setting :ref:`server-timing-enabled` on. You can enable the `Server-Timing <https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing>`_ header by setting :ref:`server-timing-enabled` on.
This header communicates metrics of the different phases in the request-response cycle. This header communicates metrics of the different phases in the request-response cycle.
.. code-block:: bash .. code-block:: bash
@@ -297,7 +223,7 @@ This header communicates metrics of the different phases in the request-response
Server-Timing: jwt;dur=14.9, parse;dur=71.1, plan;dur=109.0, transaction;dur=353.2, response;dur=4.4 Server-Timing: jwt;dur=14.9, parse;dur=71.1, plan;dur=109.0, transaction;dur=353.2, response;dur=4.4
- All the durations (``dur``) are in milliseconds. - All the durations (``dur``) are in milliseconds.
- The ``jwt`` stage is when :ref:`jwt_auth` is done. This duration can be lowered with :ref:`jwt_caching`. - The ``jwt`` stage is when :ref:`jwt_impersonation` is done. This duration can be lowered with :ref:`jwt_caching`.
- On the ``parse`` stage, the :ref:`url_grammar` is parsed. - On the ``parse`` stage, the :ref:`url_grammar` is parsed.
- On the ``plan`` stage, the :ref:`schema_cache` is used to generate the :ref:`main_query` of the transaction. - On the ``plan`` stage, the :ref:`schema_cache` is used to generate the :ref:`main_query` of the transaction.
- The ``transaction`` stage corresponds to the database transaction. See :ref:`transactions`. - The ``transaction`` stage corresponds to the database transaction. See :ref:`transactions`.
@@ -307,27 +233,6 @@ This header communicates metrics of the different phases in the request-response
We're working on lowering the duration of the ``parse`` and ``plan`` stages on https://github.com/PostgREST/postgrest/issues/2816. We're working on lowering the duration of the ``parse`` and ``plan`` stages on https://github.com/PostgREST/postgrest/issues/2816.
.. _content-length_header:
Content-Length Header
---------------------
You can verify the response body size in bytes in the `Content-Length header <https://httpwg.org/specs/rfc9110.html#field.content-length>`_.
.. code-block:: bash
curl -i 'localhost:3000/users'
.. code-block:: http
HTTP/1.1 200 OK
Content-Length: 104
Note that this header won't be returned on ``HEAD`` requests for optimization purposes (see :ref:`head_req`).
This is in line with `RFC 9110 <https://httpwg.org/specs/rfc9110.html#field.content-length>`_.
The body size is also present in the :ref:`PostgREST logs <pgrst_logging>`.
.. _explain_plan: .. _explain_plan:
Execution plan Execution plan
-13
View File
@@ -53,19 +53,6 @@ To reload the schema cache from within the database, you can use the ``NOTIFY``
NOTIFY pgrst, 'reload schema' NOTIFY pgrst, 'reload schema'
Debouncing
~~~~~~~~~~
PostgREST does not reload the schema cache for each notification when several ``NOTIFY pgrst`` events are generated quickly after one another.
There are two cases to consider: when notifications are sent within a single transaction and when they are sent across multiple transactions.
In the first case, PostgreSQL deduplicates identical ``NOTIFY`` events within the same transaction. This means that even if multiple ``NOTIFY pgrst`` statements are executed before a ``COMMIT``, only a single notification is delivered to PostgREST.
In the second case, when notifications are sent from separate transactions in a short time span, PostgREST applies a debouncing mechanism to avoid excessive schema cache reloads.
Instead of reloading the schema cache for each notification, events are grouped within a small time window of 100 milliseconds. The reload function is executed once immediately when the first notification is received and once more after the burst of events settles, resulting in at most two executions within that time window.
.. _auto_schema_reloading: .. _auto_schema_reloading:
Automatic Schema Cache Reloading Automatic Schema Cache Reloading
+5 -4
View File
@@ -284,10 +284,11 @@ However, starting from PostgreSQL 15, you can grant privileges for these setting
GRANT SET ON PARAMETER <setting> TO <authenticator>; GRANT SET ON PARAMETER <setting> TO <authenticator>;
Hoisted Function Settings Function Settings
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~
PostgREST can "hoist" function settings to transaction-scoped settings. This allows functions settings to override the impersonated and connection role settings. In addition to :ref:`impersonated_settings`, PostgREST will also apply function settings as transaction-scoped settings. This allows functions settings to override
the impersonated and connection role settings.
.. code-block:: postgres .. code-block:: postgres
@@ -302,7 +303,7 @@ When calling the above function (see :ref:`functions`), the statement timeout wi
.. note:: .. note::
Only the settings in :ref:`db-hoisted-tx-settings` will be hoisted. Only the transactions that are hoisted by config :ref:`db-hoisted-tx-settings` will be applied.
.. _main_query: .. _main_query:
+8 -6
View File
@@ -1,7 +1,9 @@
# This file is auto-generated by postgrest-nixpkgs-upgrade docutils==0.20.1
sphinx==8.2.3 sphinx-copybutton
sphinx-copybutton==0.5.2 sphinx-intl
sphinx-rtd-dark-mode==1.3.0 sphinx-rtd-theme>=0.5.1
sphinx-rtd-theme==3.0.2 sphinx-rtd-dark-mode>=1.3.0
sphinx-tabs==3.4.7 sphinx-tabs>=3.2.0
sphinx>=5.0.2
sphinxext-opengraph==0.9.1 sphinxext-opengraph==0.9.1
urllib3==2.2.2
+2 -13
View File
@@ -28,24 +28,13 @@
pacman -S postgrest pacman -S postgrest
.. tab:: Nix via nixpkgs .. tab:: Nix
You can install PostgREST from nixpkgs. You can install PostgREST from nixpkgs.
.. code:: bash .. code:: bash
nix-env -i postgrest nix-env -i haskellPackages.postgrest
.. tab:: Nix via flake
You can install PostgREST via flake.
.. code:: nix
{
inputs.postgrest.url = "github:postgrest/postgrest";
# ...
}
.. group-tab:: Windows .. group-tab:: Windows
+6 -2
View File
@@ -213,8 +213,12 @@ You should see something similar to:
.. code-block:: text .. code-block:: text
Starting PostgREST 12.0.2... Starting PostgREST 12.0.2...
Successfully connected to PostgreSQL 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit Attempting to connect to the database...
API server listening on port 3000 Connection successful
Listening on port 3000
Config reloaded
Listening for notifications on the pgrst channel
Schema cache loaded
It's now ready to serve web requests. There are many nice graphical API exploration tools you can use, but for this tutorial we'll use :code:`curl` because it's likely to be installed on your system already. Open a new terminal (leaving the one open that PostgREST is running inside). Try doing an HTTP request for the todos. It's now ready to serve web requests. There are many nice graphical API exploration tools you can use, but for this tutorial we'll use :code:`curl` because it's likely to be installed on your system already. Open a new terminal (leaving the one open that PostgREST is running inside). Try doing an HTTP request for the todos.
+11 -33
View File
@@ -52,31 +52,17 @@ Check that the :code:`tutorial.conf` (created in the previous tutorial) has the
If the PostgREST server is still running from the previous tutorial, restart it to load the updated configuration file. If the PostgREST server is still running from the previous tutorial, restart it to load the updated configuration file.
.. _tut1_step3:
Step 3. Sign a Token Step 3. Sign a Token
-------------------- --------------------
Ordinarily your own code in the database or in another server will create and sign authentication tokens, but for this tutorial we will make one "by hand" using ``bash`` and ``openssl``. Ordinarily your own code in the database or in another server will create and sign authentication tokens, but for this tutorial we will make one "by hand." Go to `jwt.io <https://jwt.io/#debugger-io>`_ and fill in the fields like this:
.. code:: bash .. figure:: ../_static/tuts/tut1-jwt-io.png
:alt: jwt.io interface
#!/bin/bash How to create a token at https://jwt.io
set -e
JWT_SECRET='test_secret_that_is_at_least_32_characters_long' **Remember to fill in the secret you generated rather than the word "secret".** After you have filled in the secret and payload, the encoded data on the left will update. Copy the encoded token.
_base64 () { openssl base64 -e -A | tr '+/' '-_' | tr -d '='; }
header=$(echo -n '{"alg":"HS256","typ":"JWT"}' | _base64)
payload=$(echo -n "{\"role\":\"todo_user\"}" | _base64)
signature=$(echo -n "$header.$payload" | openssl dgst -sha256 -hmac "$JWT_SECRET" -binary | _base64)
echo -n "$header.$payload.$signature"
**Remember to fill in the secret you generated rather than keeping the "test_secret_that_is_at_least_32_characters_long".** After you have filled in the secret and payload, the encoded data on the left will update. Copy the encoded token.
.. note:: .. note::
@@ -159,22 +145,14 @@ To observe expiration in action, we'll add an :code:`exp` claim of five minutes
select extract(epoch from now() + '5 minutes'::interval) :: integer; select extract(epoch from now() + '5 minutes'::interval) :: integer;
Or in ``bash``: Go back to jwt.io and change the payload to
.. code-block:: json
.. code-block:: bash {
"role": "todo_user",
exp=$(( EPOCHSECONDS + 5*60 )) # five minutes "exp": 123456789
}
echo $exp
Go back to :ref:`tut1_step3` and change the payload to
.. code-block:: bash
payload=$(echo -n "{\"role\":\"todo_user\",\"exp\":123456789}" | _base64)
echo -n "$header.$payload.$signature"
**NOTE**: Don't forget to change the dummy epoch value :code:`123456789` in the snippet above to the epoch value returned by the :code:`psql` command. **NOTE**: Don't forget to change the dummy epoch value :code:`123456789` in the snippet above to the epoch value returned by the :code:`psql` command.
Generated
-27
View File
@@ -1,27 +0,0 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1752006229,
"narHash": "sha256-BeuAPwNM2RBc5bvUTb0j4GRs2yBkDeRCw/8Y3v9Xesc=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "c80edd02003fe3d8af527215a3ac069be9cfd47f",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixpkgs-25.05-darwin",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
-50
View File
@@ -1,50 +0,0 @@
{
description = "REST API for any Postgres database";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-25.05-darwin";
};
nixConfig = {
extra-substituters = "https://postgrest.cachix.org";
extra-trusted-public-keys = "postgrest.cachix.org-1:icgW4R15fz1+LqvhPjt4EnX/r19AaqxiVV+1olwlZtI=";
};
outputs = { nixpkgs, ... }:
let
systems = [
"aarch64-darwin"
"aarch64-linux"
"x86_64-darwin"
"x86_64-linux"
];
pgrstFor = system: import ./default.nix {
inherit system;
nixpkgsVersion = {
owner = "nixos";
repo = "nixpkgs";
inherit (nixpkgs) rev;
tarballHash = nixpkgs.narHash;
};
};
genSystems = f: nixpkgs.lib.genAttrs systems (system: f (pgrstFor system));
in
{
packages = genSystems (attrs: {
default = attrs.postgrestPackage.bin;
profiled = attrs.postgrestProfiled.bin;
} // nixpkgs.lib.optionalAttrs (attrs ? postgrestStatic) {
static = attrs.postgrestStatic;
});
apps = genSystems (attrs: {
default = {
type = "app";
program = "${attrs.postgrestStatic or attrs.postgrestPackage.bin}/bin/postgrest";
meta.description = "REST API for any Postgres database";
};
});
};
}
+32 -38
View File
@@ -72,53 +72,45 @@ The PostgREST utilities available in `nix-shell` all have names that begin with
```bash ```bash
# Note: The utilities listed here might not be up to date. # Note: The utilities listed here might not be up to date.
[nix-shell]$ postgrest-<tab> [nix-shell]$ postgrest-<tab>
postgrest-build postgrest-parallel-curl postgrest-build postgrest-test-spec
postgrest-check postgrest-profiled-run postgrest-check postgrest-watch
postgrest-clean postgrest-push-cachix postgrest-clean postgrest-with-all
postgrest-commitlint postgrest-release postgrest-coverage postgrest-with-postgresql-10
postgrest-coverage postgrest-repl postgrest-lint postgrest-with-postgresql-11
postgrest-coverage-draft-overlay postgrest-run postgrest-run postgrest-with-postgresql-12
postgrest-docs-build postgrest-style postgrest-style postgrest-with-postgresql-13
postgrest-docs-check postgrest-style-check postgrest-style-check postgrest-with-postgresql-9.6
postgrest-docs-dictcheck postgrest-test-big-schema postgrest-test-io
postgrest-docs-linkcheck postgrest-test-doctests
postgrest-docs-render postgrest-test-io
postgrest-docs-serve postgrest-test-memory
postgrest-docs-spellcheck postgrest-test-replica
postgrest-dump-minimal-imports postgrest-test-spec
postgrest-dump-schema postgrest-test-spec-idempotence
postgrest-gen-ctags postgrest-watch
postgrest-gen-jwt postgrest-with-all
postgrest-gen-secret postgrest-with-git
postgrest-git-hooks postgrest-with-pgrst
postgrest-hsie-graph-modules postgrest-with-pg-13
postgrest-hsie-graph-symbols postgrest-with-pg-14
postgrest-hsie-minimal-imports postgrest-with-pg-15
postgrest-lint postgrest-with-pg-16
postgrest-loadtest postgrest-with-pg-17
postgrest-loadtest-against postgrest-with-slow-pg
postgrest-loadtest-report postgrest-with-slow-postgrest
postgrest-nixpkgs-upgrade
... ...
[nix-shell]$ [nix-shell]$
``` ```
The `docker` module has large dependencies to be build before the shell becomes Some additional modules like `memory`, `docker` and `release`
have large dependencies that would need to be built before the shell becomes
available, which could take an especially long time if the cachix binary cache available, which could take an especially long time if the cachix binary cache
is not used. You can activate it by passing a flag to `nix-shell` with is not used. You can activate those by passing a flag to `nix-shell` with
`nix-shell --arg docker true`. This will make the respective utilities available: `nix-shell --arg <module> true`. This will make the respective utilities available:
```bash ```bash
$ nix-shell --arg docker true $ nix-shell --arg memory true
[nix-shell]$ postgrest-docker-<tab> [nix-shell]$ postgrest-<tab>
postgrest-docker-load postgrest-build postgrest-test-spec
postgrest-check postgrest-watch
postgrest-clean postgrest-with-all
postgrest-coverage postgrest-with-postgresql-10
postgrest-lint postgrest-with-postgresql-11
postgrest-run postgrest-with-postgresql-12
postgrest-style postgrest-with-postgresql-13
postgrest-style-check postgrest-with-postgresql-9.6
postgrest-test-io
postgrest-test-memory
... ...
``` ```
Note that `postgrest-docker-load` is now also available. Note that `postgrest-test-memory` is now also available.
To run one-off commands, you can also use `nix-shell --run <command>`, which To run one-off commands, you can also use `nix-shell --run <command>`, which
will launch the Nix shell, run that one command and exit. Note that the tab will launch the Nix shell, run that one command and exit. Note that the tab
@@ -174,7 +166,7 @@ $ nix-shell --run "postgrest-with-all postgrest-test-spec"
# Run the tests against a specific version of PostgreSQL (use tab-completion in # Run the tests against a specific version of PostgreSQL (use tab-completion in
# nix-shell to see all available versions): # nix-shell to see all available versions):
$ nix-shell --run "postgrest-with-pg-13 postgrest-test-spec" $ nix-shell --run "postgrest-with-postgresql-13 postgrest-test-spec"
``` ```
@@ -284,7 +276,7 @@ Tools like `postgrest-build`, `postgrest-run`, `postgrest-repl` etc. are simple
also run in CI, with the exception of the IO and Memory checks that need to be run also run in CI, with the exception of the IO and Memory checks that need to be run
separately. separately.
`postgrest-with-pg-*` take a command as an argument and will run it `postgrest-with-postgresql-*` take a command as an argument and will run it
with a temporary database. `postgrest-with-all` will run the command against with a temporary database. `postgrest-with-all` will run the command against
all supported PostgreSQL versions. Tests run without `postgrest-with-*` are all supported PostgreSQL versions. Tests run without `postgrest-with-*` are
run against the latest PostgreSQL version by default. run against the latest PostgreSQL version by default.
@@ -379,8 +371,10 @@ that).
We also use `default.nix` to load our pinned version of the `nixpkgs` We also use `default.nix` to load our pinned version of the `nixpkgs`
repository. This set of packages will always be the same, independently from repository. This set of packages will always be the same, independently from
where or when you use it. The pinned version is taken from `flake.lock` and where or when you use it. The pinned version can be upgraded with the small
can be updated with `postgrest-nixpkgs-upgrade`. `nixpkgs-upgrade` utility. Running `nixpkgs-upgrade > nix/nixpkgs-version.nix`
in `nix-shell` will upgrade the pinned version to the latest `nixpkgs-unstable`
version.
### `shell.nix` ### `shell.nix`
@@ -0,0 +1,46 @@
{ buildPythonPackage
, fetchFromGitHub
, lib
, nose
, sphinx
, sphinx-rtd-theme
}:
buildPythonPackage rec {
pname = "sphinx-rtd-dark-mode";
version = "1.3.0";
format = "setuptools";
src = fetchFromGitHub {
owner = "MrDogeBro";
repo = "sphinx_rtd_dark_mode";
rev = "refs/tags/v${version}";
hash = "sha256-N5KG2Wqn9wfGNY3VH4FnBce1aZUbnvVmwD10Loe0Qn4=";
};
propagatedBuildInputs = [
sphinx-rtd-theme
];
nativeCheckInputs = [
nose
sphinx
];
checkPhase = ''
runHook preCheck
nosetests tests
runHook postCheck
'';
pythonImportsCheck = [
"sphinx_rtd_dark_mode"
];
meta = with lib; {
description = "Adds a toggleable dark mode to the Read the Docs theme for Sphinx.";
homepage = "https://github.com/MrDogeBro/sphinx_rtd_dark_mode";
changelog = "https://github.com/MrDogeBro/sphinx_rtd_dark_mode/releases/tag/v${version}";
license = licenses.mit;
};
}
+57
View File
@@ -0,0 +1,57 @@
# Creating a separate libpq package is is discussed in
# https://github.com/NixOS/nixpkgs/issues/61580, but nixpkgs has not moved
# forward, yet.
# This package is passed to postgresql-libpq (haskell) which needs to be
# cross-compiled to the static build and possibly other architectures as
# as well. To reduce the number of dependencies that need to be built with
# it, this derivation focuses on building the client libraries only. No
# server, no tests.
{ stdenv
, lib
, openssl
, zlib
, postgresql
, pkg-config
, tzdata
}:
stdenv.mkDerivation {
pname = "libpq";
inherit (postgresql) src version patches;
configureFlags = [
"--without-gssapi"
"--without-icu"
"--without-readline"
"--with-openssl"
"--with-system-tzdata=${tzdata}/share/zoneinfo"
"--sysconfdir=/etc/postgresql"
];
nativeBuildInputs = [ pkg-config tzdata ];
buildInputs = [ openssl zlib ];
buildFlags = [ "submake-libpq" "submake-libpgport" ];
installPhase = ''
runHook preInstall
make -C src/bin/pg_config install
make -C src/common install
make -C src/include install
make -C src/interfaces/libpq install
make -C src/port install
rm -rfv $out/share
runHook postInstall
'';
outputs = [ "out" ];
meta = with lib; {
homepage = "https://www.postgresql.org";
description = "Client API library for PostgreSQL";
license = licenses.postgresql;
};
}
+8
View File
@@ -0,0 +1,8 @@
# Pinned version of Nixpkgs, generated with postgrest-nixpkgs-upgrade.
{
owner = "NixOS";
repo = "nixpkgs";
date = "2024-04-20";
rev = "92d295f588631b0db2da509f381b4fb1e74173c5";
tarballHash = "162w28y4i5c8g5qhjvs827qxphf2a8n4c8fwhcywzl1j1a35h2im";
}
@@ -6,7 +6,6 @@
, coreutils , coreutils
, git , git
, lib , lib
, moreutils
, runCommand , runCommand
, shellcheck , shellcheck
, stdenv , stdenv
@@ -57,7 +56,7 @@ let
# Example: This way `postgrest-watch -h` will return the help output for watch, while # Example: This way `postgrest-watch -h` will return the help output for watch, while
# `postgrest-watch postgrest-test-spec -h` will return the help output for test-spec. # `postgrest-watch postgrest-test-spec -h` will return the help output for test-spec.
# Taken from: https://github.com/matejak/argbash/issues/114#issuecomment-557108274 # Taken from: https://github.com/matejak/argbash/issues/114#issuecomment-557108274
sed '/_positionals_count + 1/a\\t\t\t\tset -- "''${@:1:1}" "--" "''${@:2}"' $out | ${moreutils}/bin/sponge $out sed '/_positionals_count + 1/a\\t\t\t\tset -- "''${@:1:1}" "--" "''${@:2}"' -i $out
''; '';
bash-completion = bash-completion =
@@ -67,7 +66,7 @@ let
'' ''
+ lib.optionalString (positionalCompletion != "") '' + lib.optionalString (positionalCompletion != "") ''
sed 's#COMPREPLY.*compgen -o bashdefault .*$#${escape positionalCompletion}#' $out | ${moreutils}/bin/sponge $out sed 's#COMPREPLY.*compgen -o bashdefault .*$#${escape positionalCompletion}#' -i $out
'' ''
); );
@@ -104,7 +103,8 @@ let
'' ''
+ lib.optionalString withTmpDir '' + lib.optionalString withTmpDir ''
tmpdir="$(${coreutils}/bin/mktemp -d --tmpdir ${name}-XXX)" mkdir -p "''${TMPDIR:-/tmp}/postgrest"
tmpdir="$(${coreutils}/bin/mktemp -d --tmpdir postgrest/${name}-XXX)"
# we keep the tmpdir when an error occurs for debugging # we keep the tmpdir when an error occurs for debugging
trap 'echo Temporary directory kept at: $tmpdir' ERR trap 'echo Temporary directory kept at: $tmpdir' ERR
+4
View File
@@ -3,5 +3,9 @@
checked-shell-script = import ./checked-shell-script; checked-shell-script = import ./checked-shell-script;
gitignore = import ./gitignore.nix; gitignore = import ./gitignore.nix;
haskell-packages = import ./haskell-packages.nix; haskell-packages = import ./haskell-packages.nix;
postgis = import ./postgis.nix;
postgresql-libpq = import ./postgresql-libpq.nix;
postgresql-legacy = import ./postgresql-legacy.nix;
postgresql-future = import ./postgresql-future.nix;
slocat = import ./slocat.nix; slocat = import ./slocat.nix;
} }
+20 -90
View File
@@ -48,101 +48,31 @@ let
# Before upgrading fuzzyset to 0.3, check: https://github.com/PostgREST/postgrest/issues/3329 # Before upgrading fuzzyset to 0.3, check: https://github.com/PostgREST/postgrest/issues/3329
# jailbreak, because hspec limit for tests # jailbreak, because hspec limit for tests
fuzzyset = prev.fuzzyset_0_2_4; fuzzyset = lib.doJailbreak
(prev.callHackageDirect
# TODO: Remove once available in nixpkgs
auto-update =
prev.callHackageDirect
{ {
pkg = "auto-update"; pkg = "fuzzyset";
ver = "0.2.7";
sha256 = "sha256-fHX/OqF/cB9rbpGpLUtA29bcEJS43HUWHcK55yUxKoo=";
}
{ };
# TODO: Remove once available in nixpkgs haskellPackages
configurator-pg =
prev.callHackageDirect
{
pkg = "configurator-pg";
ver = "0.2.11";
sha256 = "sha256-mtGtNawDJgz2ZIEVca+IYXVu4oNw9xsfJiYWAqAbbgc=";
}
{ };
# TODO: Remove once available in nixpkgs haskellPackages
streaming-commons =
prev.callHackageDirect
{
pkg = "streaming-commons";
ver = "0.2.3.1";
sha256 = "sha256-Gl2eaJcWe1sxmcE/octWlH9uSnERguf+5H66K4fV87s=";
}
{ };
http2 =
prev.callHackageDirect
{
pkg = "http2";
ver = "5.4.0";
sha256 = "sha256-PeEWVd61bQ8G7LvfLeXklzXqNJFaAjE2ecRMWJZESPE=";
}
{ };
http-semantics =
prev.callHackageDirect
{
pkg = "http-semantics";
ver = "0.4.0";
sha256 = "sha256-rh0z51EKvsu5rQd5n2z3fSRjjEObouNZSBPO9NFYOF0=";
}
{ };
jose-jwt =
prev.callHackageDirect
{
pkg = "jose-jwt";
ver = "0.9.6";
sha256 = "sha256-FhBz5wzyNrDvmjHWOeNAHuVMyJUVSlm+DeQQuITSjaI=";
}
{ };
time-manager =
prev.callHackageDirect
{
pkg = "time-manager";
ver = "0.2.4"; ver = "0.2.4";
sha256 = "sha256-sAt/331YLQ2IU3z90aKYSq1nxoazv87irsuJp7ZG3pw="; sha256 = "sha256-lpkrTFcR0B4rT/P6x7ui31Twgq7BBj6KIvjKyqXKdpc=";
} }
{ }; { });
network-run =
prev.callHackageDirect
{
pkg = "network-run";
ver = "0.5.0";
sha256 = "sha256-vbXh+CzxDsGApjqHxCYf/ijpZtUCApFbkcF5gyN0THU=";
}
{ };
warp =
lib.dontCheck
(prev.callHackageDirect
{
pkg = "warp";
ver = "3.4.14";
sha256 = "sha256-RnoOUlC6dOP0sK/tYAJCX1oLzVFG1GILUY+yVbmvW8Y=";
}
{ });
# Downgrade hasql and related packages while we are still on GHC 9.4 for the static build.
hasql = lib.dontCheck (lib.doJailbreak prev.hasql_1_6_4_4);
hasql-dynamic-statements = lib.dontCheck prev.hasql-dynamic-statements_0_3_1_5;
hasql-implicits = lib.dontCheck prev.hasql-implicits_0_1_1_3;
hasql-notifications = lib.dontCheck prev.hasql-notifications_0_2_2_2;
hasql-pool = lib.dontCheck prev.hasql-pool_1_0_1; hasql-pool = lib.dontCheck prev.hasql-pool_1_0_1;
hasql-transaction = lib.dontCheck prev.hasql-transaction_1_1_0_1;
postgresql-binary = lib.dontCheck (lib.doJailbreak prev.postgresql-binary_0_13_1_3); postgresql-libpq = lib.dontCheck
(prev.postgresql-libpq.override {
postgresql = super.libpq;
});
hasql-notifications = lib.dontCheck (prev.callHackageDirect
{
pkg = "hasql-notifications";
ver = "0.2.2.0";
sha256 = "sha256-73OQ9/su2qvO7HavF3xuuNWLXSXyB9reBUQDaHys06I=";
}
{ }
);
}; };
in in
{ {
+28
View File
@@ -0,0 +1,28 @@
final: prev:
let
postgis_3_2_3 = rec {
version = "3.2.3";
src = final.fetchurl {
url = "https://download.osgeo.org/postgis/source/postgis-${version}.tar.gz";
sha256 = "sha256-G02LXHVuWrpZ77wYM7Iu/k1lYneO7KVvpJf+susTZow=";
};
meta.broken = false;
};
in
{
postgresql_11 = prev.postgresql_11.override { this = final.postgresql_11; } // {
pkgs = prev.postgresql_11.pkgs // {
postgis = prev.postgresql_11.pkgs.postgis.overrideAttrs (_: postgis_3_2_3);
};
};
postgresql_10 = prev.postgresql_10.override { this = final.postgresql_11; } // {
pkgs = prev.postgresql_10.pkgs // {
postgis = prev.postgresql_10.pkgs.postgis.overrideAttrs (_: postgis_3_2_3);
};
};
postgresql_9_6 = prev.postgresql_9_6.override { this = final.postgresql_11; } // {
pkgs = prev.postgresql_9_6.pkgs // {
postgis = prev.postgresql_9_6.pkgs.postgis.overrideAttrs (_: postgis_3_2_3);
};
};
}
+19
View File
@@ -0,0 +1,19 @@
_: _:
# Overlay that adds future versions of PostgreSQL that are supported by
# PostgREST.
{
## Example for including a postgresql version from a specific nixpks commit:
##
# postgresql_16 =
# let
# rev = "5148520bfab61f99fd25fb9ff7bfbb50dad3c9db";
# tarballHash = "1dfjmz65h8z4lk845724vypzmf3dbgsdndjpj8ydlhx6c7rpcq3p";
#
# pinnedPkgs =
# builtins.fetchTarball {
# url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
# sha256 = tarballHash;
# };
# in
# (import pinnedPkgs { }).pkgs.postgresql_16;
}
+49
View File
@@ -0,0 +1,49 @@
_: _:
# Overlay that adds legacy versions of PostgreSQL that are supported by
# PostgREST.
{
# PostgreSQL 9.6 was removed from Nixpkgs with
# https://github.com/NixOS/nixpkgs/commit/757dd008b2f2926fc0f7688fa8189f930ea47521
# We pin its parent commit to get the last version that was available.
postgresql_9_6 =
let
rev = "571cbf3d1db477058303cef8754fb85a14e90eb7";
tarballHash = "0q74wn418i1bn5sssacmw8ykpmqvzr0s93sj6pbs3rf6bf134fkz";
pinnedPkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
sha256 = tarballHash;
};
in
(import pinnedPkgs { }).pkgs.postgresql_9_6;
# PostgreSQL 10 was removed from Nixpkgs with
# https://github.com/NixOS/nixpkgs/commit/aa1483114bb329fee7e1266100b8d8921ed4723f
# We pin its parent commit to get the last version that was available.
postgresql_10 =
let
rev = "79661ba7e2fb96ebefbb537458a5bbae9dc5bd1a";
tarballHash = "0rn796pfn4sg90ai9fdnwmr10a2s835p1arazzgz46h6s5cxvq97";
pinnedPkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
sha256 = tarballHash;
};
in
(import pinnedPkgs { }).pkgs.postgresql_10;
# PostgreSQL 11 was removed from Nixpkgs with
# https://github.com/NixOS/nixpkgs/commit/1220a4d4dd1a4590780a5e1c18d1333a121be366
# We pin its parent commit to get the last version that was available.
postgresql_11 =
let
rev = "f5458516e42cc5cb4123cc2d93f45c240548aa18";
tarballHash = "1h03621sxfhw4z6ya74k6c2lyx3z7pvf2jcg4vs7i01yz2m6w3cv";
pinnedPkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
sha256 = tarballHash;
};
in
(import pinnedPkgs { }).pkgs.postgresql_11;
}
+6
View File
@@ -0,0 +1,6 @@
_: super:
{
libpq = super.callPackage ../libpq.nix {
postgresql = super.postgresql_16;
};
}
+52 -15
View File
@@ -4,29 +4,66 @@
, src , src
}: }:
let let
# This builds a static PostgREST executable based on pkgsStatic. # This builds a static PostgREST exectuable based on pkgsStatic.
inherit (pkgs) pkgsStatic; # pkgsStatic is based on musl, so is a kind of cross-compilation.
# We still make this explicit here via pkgsCross, because we need
# to get postgresql/libpq for musl, too.
pkgsCross = pkgs.pkgsCross.musl64;
inherit (pkgsCross) pkgsStatic;
inherit (pkgsStatic.haskell) lib; inherit (pkgsStatic.haskell) lib;
packagesStatic = pkgsStatic.haskell.packages.native-bignum."${compiler}"; packagesStatic =
pkgsStatic.haskell.packages."${compiler}".override (old: {
ghc = pkgsStatic.pkgsBuildHost.haskell.compiler."${compiler}".override {
# Using the bundled libffi generally works better for cross-compiling
libffi = null;
# Building sphinx fails on some platforms
enableDocs = false;
# Cross compiling with native bignum works better than with gmp
enableNativeBignum = true;
};
overrides = pkgs.lib.composeExtensions old.overrides (_: prev: {
postgresql-libpq = (lib.overrideCabal prev.postgresql-libpq {
# postgresql doesn't build in the fully static overlay - but the default
# derivation is built with static libraries anyway.
libraryPkgconfigDepends = [ pkgsCross.libpq ];
}).overrideAttrs (_: prevAttrs: {
buildInputs = prevAttrs.buildInputs ++ [ pkgsStatic.openssl ];
});
});
});
makeExecutableStatic = drv: pkgs.lib.pipe drv [ makeExecutableStatic = drv: pkgs.lib.pipe drv [
lib.compose.justStaticExecutables lib.compose.justStaticExecutables
# To successfully compile a redistributable, fully static executable we need to: # To successfully compile a redistributable, fully static executable we need to:
# 1. avoid any references to /nix/store to prevent blowing up the closure size. # 1. make executable really statically linked.
(drv: drv.overrideAttrs { # 2. avoid any references to /nix/store to prevent blowing up the closure size.
allowedReferences = [ # 3. be able to run the executable.
pkgsStatic.openssl.etc # When checking for references, we ignore the following:
]; # - eeee... are removed references which don't actually exist
}) # - openssl-etc references are purposely designed to be very small
(lib.compose.overrideCabal (drv: {
postFixup = drv.postFixup + ''
exe="$out/bin/postgrest"
# 2. be able to run the executable. if ! (file "$exe" | grep 'statically linked') then
(drv: drv.overrideAttrs { echo "not a static executable, ldd output:"
passthru.tests.version = pkgsStatic.testers.testVersion { ldd "$exe"
package = drv; exit 1
}; fi
})
echo "Checking for references to /nix/store..."
(${pkgsStatic.binutils}/bin/strings "$exe" \
| grep -v /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee \
| grep -v -etc/etc/ssl \
| grep /nix/store || exit 0 && exit 1)
echo "No references to /nix/store found"
"$exe" --help
'';
}))
]; ];
in in
-42
View File
@@ -33,17 +33,6 @@ let
exec ${cabal-install}/bin/cabal v2-clean exec ${cabal-install}/bin/cabal v2-clean
''; '';
update =
checkedShellScript
{
name = "postgrest-cabal-update";
docs = "Update cabal's package list from hackage.haskell.org";
workingDir = "/";
}
''
exec ${cabal-install}/bin/cabal v2-update
'';
run = run =
checkedShellScript checkedShellScript
{ {
@@ -55,7 +44,6 @@ let
"ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])" "ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])"
"ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool timeout])" "ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool timeout])"
"ARG_USE_ENV([PGRST_JWT_SECRET], [reallyreallyreallyreallyverysafe], [PostgREST JWT secret])" "ARG_USE_ENV([PGRST_JWT_SECRET], [reallyreallyreallyreallyverysafe], [PostgREST JWT secret])"
"ARG_USE_ENV([PGRST_ADMIN_SERVER_PORT], [3001], [PostgREST admin server port])"
"ARG_LEFTOVERS([PostgREST arguments])" "ARG_LEFTOVERS([PostgREST arguments])"
]; ];
workingDir = "/"; workingDir = "/";
@@ -66,39 +54,11 @@ let
export PGRST_DB_POOL export PGRST_DB_POOL
export PGRST_DB_POOL_ACQUISITION_TIMEOUT export PGRST_DB_POOL_ACQUISITION_TIMEOUT
export PGRST_JWT_SECRET export PGRST_JWT_SECRET
export PGRST_ADMIN_SERVER_PORT
exec ${cabal-install}/bin/cabal v2-run ${devCabalOptions} --verbose=0 -- \ exec ${cabal-install}/bin/cabal v2-run ${devCabalOptions} --verbose=0 -- \
postgrest "''${_arg_leftovers[@]}" postgrest "''${_arg_leftovers[@]}"
''; '';
runProfiled =
checkedShellScript
{
name = "postgrest-profiled-run";
docs = "Run a profiled build of postgREST. This will generate a postgrest.prof file that can be used to do optimization.";
args =
[
"ARG_USE_ENV([PGRST_DB_ANON_ROLE], [postgrest_test_anonymous], [PostgREST anonymous role])"
"ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])"
"ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool timeout])"
"ARG_USE_ENV([PGRST_JWT_SECRET], [reallyreallyreallyreallyverysafe], [PostgREST JWT secret])"
"ARG_LEFTOVERS([PostgREST arguments])"
];
workingDir = "/";
withEnv = postgrest.env;
}
''
export PGRST_DB_ANON_ROLE
export PGRST_DB_POOL
export PGRST_DB_POOL_ACQUISITION_TIMEOUT
export PGRST_JWT_SECRET
exec ${cabal-install}/bin/cabal --builddir="dist-prof" v2-run --enable-profiling --disable-shared exe:postgrest -- \
+RTS -p -h -RTS "''${_arg_leftovers[@]}"
'';
repl = repl =
checkedShellScript checkedShellScript
{ {
@@ -119,9 +79,7 @@ buildToolbox
inherit inherit
build build
clean clean
update
run run
runProfiled
repl; repl;
}; };
} }
+4 -19
View File
@@ -10,7 +10,6 @@
, hsie , hsie
, nix , nix
, silver-searcher , silver-searcher
, stdenv
, style , style
, tests , tests
, withTools , withTools
@@ -54,14 +53,10 @@ let
Requires authentication with `cachix authtoken ...`. Requires authentication with `cachix authtoken ...`.
''; '';
args =
[
"ARG_OPTIONAL_SINGLE([system], , [System], [${stdenv.system}])"
];
workingDir = "/"; workingDir = "/";
} }
'' ''
${nix}/bin/nix-instantiate --argstr system "$_arg_system" \ ${nix}/bin/nix-instantiate \
| xargs ${nix}/bin/nix-store -qR --include-outputs \ | xargs ${nix}/bin/nix-store -qR --include-outputs \
| ${cachix}/bin/cachix push postgrest | ${cachix}/bin/cachix push postgrest
''; '';
@@ -82,7 +77,6 @@ let
} }
'' ''
${tests}/bin/postgrest-test-spec ${tests}/bin/postgrest-test-spec
${tests}/bin/postgrest-test-observability
${tests}/bin/postgrest-test-doctests ${tests}/bin/postgrest-test-doctests
${tests}/bin/postgrest-test-io ${tests}/bin/postgrest-test-io
${tests}/bin/postgrest-test-big-schema ${tests}/bin/postgrest-test-big-schema
@@ -339,29 +333,20 @@ let
checkedShellScript checkedShellScript
{ {
name = "postgrest-gen-jwt"; name = "postgrest-gen-jwt";
docs = '' docs = "Generate a JWT";
Generate a JWT. Example: postgrest-gen-jwt --exp 10 postgrest_test_author
# This can be used to quickly prove a JWT expiry
$ curl localhost:3000/authors_only -H "Authorization: Bearer \$(postgrest-gen-jwt --exp -31 postgrest_test_author)"
'';
args = [ args = [
"ARG_POSITIONAL_SINGLE([role], [role for the jwt payload])" "ARG_POSITIONAL_SINGLE([role], [role for the jwt payload])"
"ARG_OPTIONAL_SINGLE([secret],, [secret used to sign the JWT], [reallyreallyreallyreallyverysafe])" "ARG_OPTIONAL_SINGLE([secret],, [secret used to sign the JWT], [reallyreallyreallyreallyverysafe])"
"ARG_OPTIONAL_SINGLE([exp],, [seconds for JWT expiry, it accepts negative values], [3600])"
]; ];
} }
'' ''
# Based on https://stackoverflow.com/questions/59002949/how-to-create-a-json-web-token-jwt-using-openssl-shell-commands # From https://stackoverflow.com/questions/59002949/how-to-create-a-json-web-token-jwt-using-openssl-shell-commands
# Construct the header # Construct the header
jwt_header=$(echo -n '{"alg":"HS256","typ":"JWT"}' | base64 | sed s/\+/-/g | sed 's/\//_/g' | sed -E s/=+$//) jwt_header=$(echo -n '{"alg":"HS256","typ":"JWT"}' | base64 | sed s/\+/-/g | sed 's/\//_/g' | sed -E s/=+$//)
# Construct the exp value
expiry=$((EPOCHSECONDS + _arg_exp))
# Construct the payload # Construct the payload
payload=$(echo -n "{\"role\": \"$_arg_role\", \"exp\": $expiry}" | base64 | sed s/\+/-/g |sed 's/\//_/g' | sed -E s/=+$//) payload=$(echo -n "{\"role\":\"$_arg_role\"}" | base64 | sed s/\+/-/g |sed 's/\//_/g' | sed -E s/=+$//)
# Convert secret to hex # Convert secret to hex
hexsecret=$(echo -n "$_arg_secret" | xxd -p | paste -sd "") hexsecret=$(echo -n "$_arg_secret" | xxd -p | paste -sd "")
+8 -26
View File
@@ -2,28 +2,24 @@
, aspellDicts , aspellDicts
, buildToolbox , buildToolbox
, checkedShellScript , checkedShellScript
, lib
, plantuml
, python3 , python3
, python3Packages , python3Packages
, writeTextFile
, writers , writers
, plantuml
}: }:
let let
selectPythonPackages = ps: [ selectPythonPackages = ps: [
ps.sphinx ps.sphinx
ps.sphinx-copybutton ps.sphinx_rtd_theme
ps.sphinx-rtd-dark-mode ps.livereload
ps.sphinx-rtd-theme
ps.sphinx-tabs ps.sphinx-tabs
ps.sphinx-copybutton
ps.sphinxext-opengraph ps.sphinxext-opengraph
(ps.callPackage ../docs-extensions/sphinx-rtd-dark-mode.nix { })
# TODO: Remove override once new sphinx-intl version (> 2.1.0) is released and available in nixpkgs
(ps.sphinx-intl.overrideAttrs (drv: { nativeBuildInputs = drv.nativeBuildInputs ++ [ ps.six ]; }))
]; ];
requirements = writeTextFile {
name = "requirements.txt";
text = lib.concatMapStringsSep "\n" (pkg: "${pkg.pname}==${pkg.version}") (selectPythonPackages python3Packages);
};
python = python3.withPackages selectPythonPackages; python = python3.withPackages selectPythonPackages;
build = build =
@@ -35,9 +31,6 @@ let
workingDir = "/docs"; workingDir = "/docs";
} }
'' ''
# https://github.com/sphinx-doc/sphinx/issues/11739
export LC_ALL=C
function build() { function build() {
${python}/bin/sphinx-build --color -W -a -n . -b "$@" ${python}/bin/sphinx-build --color -W -a -n . -b "$@"
} }
@@ -80,7 +73,7 @@ let
server = server =
writers.writePython3 writers.writePython3
"postgrest-docs-server" "postgrest-docs-server"
{ libraries = selectPythonPackages python3Packages ++ [ python3Packages.livereload ]; } { libraries = selectPythonPackages python3Packages; }
'' ''
import sys import sys
from livereload import Server, shell from livereload import Server, shell
@@ -122,10 +115,6 @@ let
workingDir = "/docs"; workingDir = "/docs";
} }
'' ''
echo "Checking spelling mistakes..."
export LC_ALL=C
FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ') FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')
# shellcheck disable=SC2086 disable=SC2016 # shellcheck disable=SC2086 disable=SC2016
@@ -146,10 +135,6 @@ let
workingDir = "/docs"; workingDir = "/docs";
} }
'' ''
echo "Detecting obsolete dictionary entries..."
export LC_ALL=C
FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ') FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')
tail -n+2 postgrest.dict \ tail -n+2 postgrest.dict \
@@ -168,8 +153,6 @@ let
workingDir = "/docs"; workingDir = "/docs";
} }
'' ''
export LC_ALL=C
${python}/bin/sphinx-build --color -b linkcheck . ../.docs-build ${python}/bin/sphinx-build --color -b linkcheck . ../.docs-build
''; '';
@@ -200,5 +183,4 @@ buildToolbox
serve serve
spellcheck; spellcheck;
}; };
extra = { inherit requirements; };
} }
-146
View File
@@ -1,146 +0,0 @@
# generates a file to be used by the vegeta load testing tool
# It includes a worst case scenario for the JWT cache:
# - all requests will have a unique JWT so no cache hits
# - all jwts have an expiration that will be long enough to be
# valid at time of request but short enough that already
# validated jwts will expire later during the loadtest run
# - the above guarantees JWT cache purging will happen
# - we want this to track resource consumption in the worst case
# And a more normal scenario where non-expiring JWTs are picked
# from an array
import time
import argparse
import sys
import random
import jwt
import jwcrypto.jwk as jwk
from typing import Optional
from pathlib import Path
URL = "http://postgrest"
secret_key = b"reallyreallyreallyreallyverysafe"
key = jwk.JWK.generate(kty="RSA", size=4096)
private_key = jwt.algorithms.RSAAlgorithm.from_jwk(key.export_private())
public_key = key.export_public()
def generate_jwt(now: int, exp_inc: Optional[int], is_hs: bool) -> str:
"""Generate an HS256 or RS256 JWT"""
payload = {
"sub": f"user_{random.getrandbits(32)}",
"iat": now,
"role": "postgrest_test_author",
}
if exp_inc is not None:
payload["exp"] = now + exp_inc
k = secret_key if is_hs else private_key
alg = "HS256" if is_hs else "RS256"
return jwt.encode(payload, k, alg)
def append_targets(lines: list[str], token: str):
lines.append(f"OPTIONS {URL}/authors_only")
lines.append(f"Authorization: Bearer {token}")
lines.append("") # blank line to separate requests
def main():
parser = argparse.ArgumentParser(
description="Generate Vegeta targets with unique JWTs"
)
parser.add_argument(
"output",
help="Path to write the generated targets file",
)
parser.add_argument(
"--worst",
dest="worst",
action=argparse.BooleanOptionalAction,
default=False,
help="Generate worst case targets for a JWT cache",
)
parser.add_argument(
"--rsa",
dest="jwk_path",
metavar="JWK_PATH",
type=Path,
default=None,
help="Path for generating a RSA JWK file to sign tokens with",
)
args = parser.parse_args()
is_hs = args.jwk_path is None
nsamples = 1000
if is_hs:
ntargets = 200000
else:
# The asymmetric targets take too long to compute so we reduce them
ntargets = 50000
if not is_hs:
try:
with open(args.jwk_path, "w") as jwk:
jwk.write(public_key)
print(f"Created {args.jwk_path} file containing the RSA JWK")
except IOError as e:
print(f"Error writing to {args.jwk_path}: {e}", file=sys.stderr)
sys.exit(1)
print(f"Generating {ntargets} targets...")
start_time = time.time()
now = int(start_time)
lines = []
# We want to ensure 401 Unauthorized responses don't happen during
# JWT validation, this can happen when the jwt `exp` is too short.
# At the same time, we want to ensure the `exp` is not too big,
# so expires will occur and postgREST needs to
# clean cached expired JWTs
if args.worst:
# estimated time takes to build and run postgrest itself
build_run_postgrest_time = 2
# estimated time it takes to generate the targets file
# the division numbers are tuned by hand
if is_hs: # hs generation is much faster
gen_time = ntargets // 66666
else: # asymmetric is slower so the time is higher
gen_time = ntargets // 220
# estimated exp time so some JWTs will expire
inc = build_run_postgrest_time + gen_time
for i in range(ntargets):
token = generate_jwt(now, inc + i // 1000, is_hs)
append_targets(lines, token)
else:
tokens = [generate_jwt(now, None, is_hs) for _ in range(nsamples)]
for i in range(ntargets):
token = random.choice(tokens)
append_targets(lines, token)
try:
with open(args.output, "w") as f:
f.write("\n".join(lines))
except IOError as e:
print(f"Error writing to {args.output}: {e}", file=sys.stderr)
sys.exit(1)
elapsed = time.time() - start_time
print(f"Created {ntargets} targets", end=" ")
print(f"in {args.output} ({elapsed:.2f}s)")
if __name__ == "__main__":
main()
-60
View File
@@ -1,60 +0,0 @@
{ buildToolbox
, checkedShellScript
, commitlint
, writeText
}:
let
# Rules format: [<severity>, <"always"/"never">, <value>]
commitlintConfig = writeText "commitlint.config.mjs" ''
export default {
rules: {
"type-enum": [2, "always", [
'add', // Add a new feature
'amend', // To amend an unrealease commit
'change', // Breaking changes
'chore', // Update sponsors, changelog, readme etc
'ci', // CI configuration files and scripts
'docs', // Documentation
'fix', // Bug fix
'nix', // Related to Nix
'perf', // Performance improvements
'refactor', // Refactoring code
'remove', // Remove a feature or fix
'test', // Adding tests
]],
'subject-case': [2, 'never', ['pascal-case', 'start-case']],
'subject-empty': [2, 'never'],
'subject-full-stop': [2, 'never', '.'],
'subject-max-length': [2, 'always', 80],
'subject-min-length': [2, 'always', 5],
'scope-case': [2, 'always', 'lower-case'],
'body-leading-blank': [2, 'always'],
},
};
'';
commitCheck =
checkedShellScript
{
name = "postgrest-commitlint";
docs = "Script to validate commit messages";
workingDir = "/";
args = [
"ARG_OPTIONAL_SINGLE([from],, [commit ref start from], [main])"
"ARG_OPTIONAL_SINGLE([to],, [commit ref end at], [HEAD])"
];
}
''
# Run commitlint with the given configuration
${commitlint}/bin/commitlint --config ${commitlintConfig} --from "$_arg_from" --to "$_arg_to"
'';
in
buildToolbox
{
name = "postgrest-commitlint";
tools = { inherit commitCheck; };
}
+10 -95
View File
@@ -41,9 +41,6 @@ let
args = [ args = [
"ARG_OPTIONAL_SINGLE([output], [o], [Filename to dump json output to], [./loadtest/result.bin])" "ARG_OPTIONAL_SINGLE([output], [o], [Filename to dump json output to], [./loadtest/result.bin])"
"ARG_OPTIONAL_SINGLE([testdir], [t], [Directory to load tests and fixtures from], [./test/load])" "ARG_OPTIONAL_SINGLE([testdir], [t], [Directory to load tests and fixtures from], [./test/load])"
"ARG_OPTIONAL_SINGLE([kind], [k], [Kind of loadtest], [mixed])"
"ARG_TYPE_GROUP_SET([KIND], [KIND], [kind], [mixed,jwt-hs,jwt-hs-cache,jwt-hs-cache-worst,jwt-rsa,jwt-rsa-cache,jwt-rsa-cache-worst])"
"ARG_OPTIONAL_SINGLE([monitor], [m], [Monitoring file], [./loadtest/result.csv])"
"ARG_LEFTOVERS([additional vegeta arguments])" "ARG_LEFTOVERS([additional vegeta arguments])"
]; ];
workingDir = "/"; workingDir = "/";
@@ -58,67 +55,16 @@ let
export PGRST_DB_POOL="1" export PGRST_DB_POOL="1"
export PGRST_DB_TX_END="rollback-allow-override" export PGRST_DB_TX_END="rollback-allow-override"
export PGRST_LOG_LEVEL="crit" export PGRST_LOG_LEVEL="crit"
export PGRST_JWT_SECRET="reallyreallyreallyreallyverysafe"
# set previous PGRST_JWT_CACHE_MAX_LIFETIME configuration so that
# load test works across branches
# TODO clean once PGRST_JWT_CACHE_MAX_ENTRIES merged and released
export PGRST_JWT_CACHE_MAX_LIFETIME="86400"
mkdir -p "$(dirname "$_arg_output")" mkdir -p "$(dirname "$_arg_output")"
abs_output="$(realpath "$_arg_output")" abs_output="$(realpath "$_arg_output")"
case "$_arg_kind" in # shellcheck disable=SC2145
jwt-hs) ${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${genTargetsHS} "$_arg_testdir"/gen_targets.http ${withTools.withSlowPg} \
export PGRST_JWT_CACHE_MAX_ENTRIES="0" ${withTools.withPgrst} \
export PGRST_JWT_CACHE_MAX_LIFETIME="0" ${withTools.withSlowPgrst} \
;; sh -c "cd \"$_arg_testdir\" && ${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
jwt-hs-cache)
${genTargetsHS} "$_arg_testdir"/gen_targets.http
;;
jwt-hs-cache-worst)
${genTargetsHS} --worst "$_arg_testdir"/gen_targets.http
;;
jwt-rsa)
${genTargetsHS} --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_CACHE_MAX_ENTRIES="0"
export PGRST_JWT_CACHE_MAX_LIFETIME="0"
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
;;
jwt-rsa-cache)
${genTargetsHS} --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
;;
jwt-rsa-cache-worst)
${genTargetsHS} --worst --rsa="$_arg_testdir"/gen_jwk.json "$_arg_testdir"/gen_targets.http
export PGRST_JWT_SECRET="@$_arg_testdir/gen_jwk.json"
;;
*)
;;
esac
if [ "$_arg_kind" == "mixed" ]; then
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withSlowPg} \
${withTools.withPgrst} -m "$_arg_monitor" \
${withTools.withSlowPgrst} \
sh -c "cd \"$_arg_testdir\" && \
${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
else
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} -m "$_arg_monitor" \
sh -c "cd \"$_arg_testdir\" && \
${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
fi
${vegeta}/bin/vegeta report -type=text "$_arg_output" ${vegeta}/bin/vegeta report -type=text "$_arg_output"
''; '';
@@ -137,7 +83,6 @@ let
''; '';
args = [ args = [
"ARG_POSITIONAL_INF([target], [Commit-ish reference to compare with], 1)" "ARG_POSITIONAL_INF([target], [Commit-ish reference to compare with], 1)"
"ARG_OPTIONAL_SINGLE([kind], [k], [Kind of loadtest], [mixed])"
]; ];
positionalCompletion = positionalCompletion =
'' ''
@@ -148,12 +93,11 @@ let
workingDir = "/"; workingDir = "/";
} }
'' ''
# run loadtest for every target
for tgt in "''${_arg_target[@]}"; do for tgt in "''${_arg_target[@]}"; do
cat << EOF cat << EOF
Running "$_arg_kind" loadtest on "$tgt"... Running loadtest on "$tgt"...
EOF EOF
@@ -162,7 +106,7 @@ let
# Save the results in the current working tree, too, # Save the results in the current working tree, too,
# otherwise they'd be lost in the temporary working tree # otherwise they'd be lost in the temporary working tree
# created by withTools.withGit. # created by withTools.withGit.
${withTools.withGit} "$tgt" ${loadtest} -k "$_arg_kind" -m "$PWD/loadtest/$tgt.csv" --output "$PWD/loadtest/$tgt.bin" --testdir "$PWD/test/load" ${withTools.withGit} "$tgt" ${loadtest} --output "$PWD/loadtest/$tgt.bin" --testdir "$PWD/test/load"
cat << EOF cat << EOF
@@ -172,15 +116,13 @@ let
done done
# run loadtest once on HEAD
cat << EOF cat << EOF
Running "$_arg_kind" loadtest on HEAD... Running loadtest on HEAD...
EOF EOF
${loadtest} -k "$_arg_kind" -m "$PWD/loadtest/head.csv" --output "$PWD/loadtest/head.bin" --testdir "$PWD/test/load" ${loadtest} --output "$PWD/loadtest/head.bin" --testdir "$PWD/test/load"
cat << EOF cat << EOF
@@ -217,7 +159,6 @@ let
pd.read_json(sys.stdin) \ pd.read_json(sys.stdin) \
.set_index('param') \ .set_index('param') \
.drop(['branch', 'earliest', 'end', 'latest']) \ .drop(['branch', 'earliest', 'end', 'latest']) \
.fillna("") \
.convert_dtypes() \ .convert_dtypes() \
.to_markdown(sys.stdout, floatfmt='.0f') .to_markdown(sys.stdout, floatfmt='.0f')
''; '';
@@ -228,41 +169,15 @@ let
{ {
name = "postgrest-loadtest-report"; name = "postgrest-loadtest-report";
docs = "Create a report of all loadtest reports as markdown."; docs = "Create a report of all loadtest reports as markdown.";
args = [
"ARG_OPTIONAL_SINGLE([group], [g], [Marker to group results])"
];
workingDir = "/"; workingDir = "/";
} }
'' ''
marker=''${_arg_group:+"($_arg_group)"}
echo -e "## Loadtest results $marker\n"
find loadtest -type f -iname '*.bin' -exec ${reporter} {} \; \ find loadtest -type f -iname '*.bin' -exec ${reporter} {} \; \
| ${jq}/bin/jq '[paths(scalars) as $path | {param: $path | join("."), (.branch): getpath($path)}]' \ | ${jq}/bin/jq '[paths(scalars) as $path | {param: $path | join("."), (.branch): getpath($path)}]' \
| ${jq}/bin/jq --slurp 'flatten | group_by(.param) | map(add)' \ | ${jq}/bin/jq --slurp 'flatten | group_by(.param) | map(add)' \
| ${toMarkdown} | ${toMarkdown}
echo -e "\n\n## Loadtest elapsed seconds vs CPU/MEM usage $marker\n"
find loadtest -type f -iname '*.csv' \
| sort -m \
| ${mergeMonitorResults}
''; '';
genTargetsHS =
writers.writePython3 "postgrest-gen-loadtest-targets-hs"
{
libraries = [ python3Packages.pyjwt python3Packages.jwcrypto ];
}
(builtins.readFile ./generate_targets.py);
mergeMonitorResults =
writers.writePython3 "postgrest-merge-monitor-results"
{
libraries = [ python3Packages.pandas python3Packages.tabulate ];
}
(builtins.readFile ./merge_monitor_result.py);
in in
buildToolbox { buildToolbox {
name = "postgrest-loadtest"; name = "postgrest-loadtest";
+51
View File
@@ -0,0 +1,51 @@
# The memory tests have large dependencies (a profiled build of PostgREST)
# and are run less often than the spec tests, so we don't include them in
# the default test environment. We make them available through a separate module.
# TODO both of these require reentering the nix-shell if you make a change to the code
{ buildToolbox
, checkedShellScript
, curl
, postgrestProfiled
, withTools
}:
let
test =
checkedShellScript
{
name = "postgrest-test-memory";
docs = "Run the memory tests.";
workingDir = "/";
withPath = [ postgrestProfiled curl ];
}
''
${withTools.withPg} -f test/spec/fixtures/load.sql test/memory/memory-tests.sh
'';
runProfiled =
checkedShellScript
{
name = "postgrest-profiled-run";
docs = "Run a profiled build of postgREST. This will generate a postgrest.prof file that can be used to do optimization. Note: if you make a change to the code, you must reenter the nix-shell for an updated profiled build.";
args =
[
"ARG_USE_ENV([PGRST_DB_ANON_ROLE], [postgrest_test_anonymous], [PostgREST anonymous role])"
"ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])"
"ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool timeout])"
"ARG_LEFTOVERS([PostgREST arguments])"
];
workingDir = "/";
withPath = [ postgrestProfiled ];
}
''
export PGRST_DB_ANON_ROLE
export PGRST_DB_POOL
export PGRST_DB_POOL_ACQUISITION_TIMEOUT
postgrest +RTS -p -h -RTS "''${_arg_leftovers[@]}"
'';
in
buildToolbox
{
name = "postgrest-memory";
tools = { inherit test runProfiled; };
}
-44
View File
@@ -1,44 +0,0 @@
import os
import sys
import pandas as pd
KEY = "Elapsed seconds"
BASE_METRICS = ["CPU (%)", "Real (MB)"]
branch_order = []
merged = None
paths = [p.strip() for p in sys.stdin.read().split() if p.strip()]
for csv_path in paths:
# br is branch (variable shortened to pass linter)
br = os.path.splitext(os.path.basename(csv_path))[0]
branch_order.append(br)
df = pd.read_csv(csv_path)
if KEY not in df.columns:
sys.exit(f"{csv_path} is missing the {KEY} column")
for m in BASE_METRICS:
if m not in df.columns:
sys.exit(f"Error: '{csv_path}' missing required column '{m}'.")
# add branch marker to every metric column
df = df.rename(columns={c: f"{c} [{br}]" for c in df.columns if c != KEY})
# outer join so missing rows appear
merged = df if merged is None else merged.merge(df, on=KEY, how="outer")
# Re-order columns so related metrics are adjacent
ordered_cols = [KEY]
for metric in BASE_METRICS:
for br in branch_order:
col_name = f"{metric} [{br}]"
if col_name in merged.columns:
ordered_cols.append(col_name)
merged = merged[ordered_cols]
# replace nan with empty string
merged = merged.fillna("")
merged.to_markdown(sys.stdout, index=False, tablefmt="github")

Some files were not shown because too many files have changed in this diff Show More