Compare commits

..
30 Commits
Author SHA1 Message Date
steve-chavez db9da0b1ca bump version to 12.2.2 2024-07-10 23:07:23 -05:00
steve-chavez f09655b7a6 fix: schema cache retrying without backoff
Fixes https://github.com/PostgREST/postgrest/issues/3523.

Now if there's a failure when obtaining the pg version OR schema cache,
we do the same retrying process. This way we don't add two retries.

Refactors and renames the "connectionWorker" to "schemaCacheLoader".
This makes more sense since what we really want is the schema cache,
the version is the pre-requisite for ensuring our
schema cache queries work.

Additionally, we no longer log ` Attempting to connect to the database...`
at startup unnecessarily. This is only logged whenever there's a retry attempt.
2024-07-10 21:11:20 -05:00
steve-chavezandWolfgang Walther 8715e426c0 chore: remove paypal links
It was tied to a personal account and donations there have been too rare.
2024-07-10 21:13:23 +02:00
Wolfgang WaltherandWolfgang Walther 9a40bc266e fix: List correct enum options when multiple types with same name are present
The schema cache and OpenAPI output would currently list the first found
enum with the same name instead of the correct type. One other case
where this comes up is when a regular type and an enum type have the
same name. For example in the spec fixtures, we have an enum called
"bit". Every "bit" type, no matter whether it's that enum or the
built-in bit type, will show those enum options in the OpenApi output.

Not adding a test, because OpenAPI is supposed to go away in the future
anyway.
2024-07-09 09:09:22 +02:00
Wolfgang WaltherandWolfgang Walther e67461b991 fix: Show number of loaded timezones in log output
There is no reason to hide those, right?
2024-07-09 09:09:22 +02:00
Wolfgang WaltherandWolfgang Walther 420933ceba fix: Make --dump-schema work with in-database pgrst.db_schemas setting
This needs to be loaded from in-database configuration first, otherwise
the dump-schema output will be for the default (public) schema.
2024-07-09 09:09:22 +02:00
Laurence IslaandWolfgang Walther 8f8dbd3425 fix: nested empty embeds no longer return empty values and are correctly omitted 2024-07-09 09:09:22 +02:00
renovate[bot]andWolfgang Walther 8d7d7e4112 chore(deps): update all dependencies 2024-07-06 11:35:11 +02:00
Sandro BauerandWolfgang Walther 521acf61cb docs: fix rendering for inline code block in operator list 2024-07-02 12:28:24 +02:00
steve-chavez d3f7cba92e bump version to 12.2.1 2024-06-27 14:59:35 -05:00
Laurence IslaandSteve Chavez a71d9ea50f changelog: add missing entries for #3592 and #3616 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 5a1e9b01e4 nix: remove texlive dependencies from postgrest-docs-render 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez d3292d42de docs: use PlantUML instead of Latex to generate Schema Isolation image 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 572581cbb6 chore: organize diagrams in different folders 2024-06-27 14:08:16 -05:00
steve-chavez 9daddb549c docs: add listener recovery 2024-06-27 14:08:16 -05:00
steve-chavez 716fcbf147 fix: don't reload cache on every listener fail
Revert "prevent GSSAPI error between Listener and pool"

This reverts commit 4beac10d3d.
2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 5b62501ce7 docs: improve architecture diagram
- SVG format instead of PNG
- The components now have links to their reference in the Docs
- Supports dark mode
2024-06-27 14:08:16 -05:00
Wolfgang WaltherandSteve Chavez 7860e7287f docs: Fix punctuation in install.rst 2024-06-27 14:08:16 -05:00
Michal KleczekandSteve Chavez c958f94753 docs: add pg-notify-stdout to ecosystem 2024-06-27 14:08:16 -05:00
renovate[bot]andWolfgang Walther 94f0548344 chore(deps): update ubuntu:noble docker digest to 2e863c4 2024-06-19 19:22:47 +02:00
Wolfgang Walther 16d0e1ca8c fix: Treat pre-release and docs versions correctly for new release workflow
Since we changed our release workflow, we have adjusted:
- the docs to use postgrest.org/en/v12/ -style URLs, i.e. only using the
major component.
- the pre-release / devel versions to contain only two instead of four
version parts, i.e. currently 12.3.
2024-06-19 08:42:34 +02:00
Wolfgang Walther 9657d46c42 docs: Fix linkcheck
Some URLs are still forbidden for our linkcheck tool, so disabling them
again.

Others are permanently redirected, so adjusting them.
2024-06-19 08:40:58 +02:00
renovate[bot]andWolfgang Walther 70e07ccbc6 chore(deps): update dependency urllib3 to v2.2.2 2024-06-17 21:04:03 +02:00
renovate[bot]andWolfgang Walther cc5c4ae5ef chore(deps): update codecov/codecov-action action to v4.5.0 2024-06-13 19:51:15 +02:00
renovate[bot]andWolfgang Walther af8b62ba10 chore(deps): update all dependencies 2024-06-13 08:46:23 +02:00
Laurence Isla d4e7d23268 docs: fix example of listener failure on read replicas 2024-06-12 19:46:23 -05:00
Laurence Isla 4e084b664c docs: clarify what is logged when "log-level=debug" 2024-06-12 18:30:19 -05:00
Laurence Isla 91024b6eab docs: add missing logs to stderr
- Schema cache stats are now logged to stderr
- Log when the LISTEN channel gets a notification
2024-06-12 18:30:19 -05:00
Laurence IslaandSteve Chavez 0e4fcd7387 docs: add "Listener" page
Co-authored-by: Steve Chavez <stevechavezast@gmail.com>
2024-06-12 18:30:19 -05:00
Wolfgang Walther 49a7aa9933 bump docs version to 12.2 2024-06-12 07:47:17 +02:00
198 changed files with 5087 additions and 7788 deletions
+2 -2
View File
@@ -1,5 +1,5 @@
freebsd_instance:
image_family: freebsd-14-3
image_family: freebsd-14-0
build_task:
# Don't change this name without adjusting .github/workflows/build.yaml
@@ -35,7 +35,7 @@ build_task:
- find main src -type f -iname '*.hs' -exec md5sum "{}" +
build_script: |
stack build -j 1 --local-bin-path . --copy-bins
stack build -j 1 --local-bin-path . --copy-bins --stack-yaml stack-21.7.yaml
strip postgrest
bin_artifacts:
-9
View File
@@ -1,9 +0,0 @@
root = true
[*]
charset = utf-8
end_of_line = lf
indent_size = 2
indent_style = space
insert_final_newline = true
trim_trailing_whitespace = true
+17
View File
@@ -0,0 +1,17 @@
<!--
Before reporting a bug:
If your database schema has changed while the PostgREST server is running,
send the server a SIGUSR1 signal or restart it(http://postgrest.org/en/stable/admin.html#schema-reloading)
to ensure the schema cache is not stale. This sometimes fixes apparent bugs.
-->
### Environment
* PostgreSQL version: (if using docker, specify the image)
* PostgREST version: (if using docker, specify the image)
* Operating system:
### Description of issue
(Expected behavior vs actual behavior)
(Steps to reproduce: Include a minimal SQL definition plus how you make the request to PostgREST and the response body)
-28
View File
@@ -1,28 +0,0 @@
---
name: Bug report
about: Create a bug report to help us improve
type: Bug
title: ''
labels: ''
assignees: ''
---
<!--
Before reporting a bug:
If your database schema has changed while the PostgREST server is running,
send the server a SIGUSR1 signal or restart it (http://postgrest.org/en/stable/admin.html#schema-reloading) to ensure the schema cache is not stale. This sometimes fixes apparent bugs.
-->
### Environment
* PostgreSQL version: (if using docker, specify the image)
* PostgREST version: (if using docker, specify the image)
* Operating system:
### Description of issue
Describe the behavior you expected vs the actual behavior. Include:
- A minimal SQL definition.
- How you make the request to PostgREST (curl command preferred).
- The PostgREST response.
-1
View File
@@ -1 +0,0 @@
blank_issues_enabled: false
-17
View File
@@ -1,17 +0,0 @@
---
name: Feature request
about: Suggest an enhancement for this project
type: Feature
title: ''
labels: ''
assignees: ''
---
## Problem
A clear and concise description of what the problem is.
## Solution
A clear and concise description of what you want to happen.
-5
View File
@@ -1,5 +0,0 @@
# TODO: Remove this once a new actionlint release has been cut
# and made its way to us through nixpkgs.
self-hosted-runner:
labels:
- ubuntu-24.04-arm
@@ -112,7 +112,7 @@ runs:
echo "artifacts=${artifacts}" >> "$GITHUB_OUTPUT"
- name: Save artifact to GitHub Actions
if: steps.find-task.outputs.task_found
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ inputs.upload }}
path: ${{ steps.download.outputs.artifacts }}
+2 -2
View File
@@ -19,14 +19,14 @@ inputs:
runs:
using: composite
steps:
- uses: actions/cache@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
- uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with:
path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: |
${{ runner.os }}-${{ inputs.prefix }}-
- uses: actions/cache/restore@0400d5f644dc74513175e3cd8d07132dd4860809 # v4.2.4
- uses: actions/cache/restore@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with:
path: ${{ inputs.path }}
+2 -3
View File
@@ -11,16 +11,15 @@ inputs:
runs:
using: composite
steps:
- uses: nixbuild/nix-quick-install-action@1f095fee853b33114486cfdeae62fa099cda35a9 # v33
- uses: nixbuild/nix-quick-install-action@60e9c39264d4714139af3cdf15f691b19eec3530 # v28
with:
nix_conf: |-
always-allow-substitutes = true
max-jobs = auto
- uses: cachix/cachix-action@0fc020193b5a1fa3ac4575aa3a7d3aa6a35435ad # v16
- uses: cachix/cachix-action@ad2ddac53f961de1989924296a1f236fcfbaa4fc # v15
with:
name: postgrest
authToken: ${{ inputs.authToken }}
skipPush: ${{ inputs.authToken == '' }}
- if: ${{ inputs.tools }}
run: nix-env -f default.nix -iA ${{ inputs.tools }}
shell: bash
+4 -19
View File
@@ -8,33 +8,18 @@
"/^v[0-9]+/"
],
"rebaseWhen": "conflicted",
"pip_requirements": {
"enabled": false
},
"packageRules": [
{
"matchBaseBranches": [ "/^v[0-9]+/" ],
"matchManagers": ["haskell-cabal"],
"enabled": false
},
{
"matchBaseBranches": [ "/^v[0-9]+/" ],
"groupName": "all dependencies"
},
{
"matchManagers": ["haskell-cabal"],
"matchPackageNames": ["base", "bytestring", "containers", "directory", "mtl", "parsec", "process", "text"],
"groupName": "GHC dependencies"
"matchPackageNames": ["docutils"],
"allowedVersions": "<0.21.0"
},
{
"matchManagers": ["haskell-cabal"],
"matchPackageNames": ["hasql", "hasql-dynamic-statements", "hasql-notifications", "hasql-transaction", "hasql-pool"],
"groupName": "hasql"
},
{
"matchManagers": ["haskell-cabal"],
"matchPackageNames": ["fuzzyset"],
"allowedVersions": "<0.3"
"matchPackageNames": ["macos"],
"allowedVersions": "<13"
}
]
}
+72
View File
@@ -0,0 +1,72 @@
#!/bin/bash
set -Eeuo pipefail
# This script builds PostgREST in a remote ARM server
[ -z "$1" ] && { echo "Missing 1st argument: PostgREST github commit SHA"; exit 1; }
[ -z "$2" ] && { echo "Missing 2nd argument: Build environment directory name"; exit 1; }
[ -z "$3" ] && { echo "Missing 3rd argument: GHC version"; exit 1; }
PGRST_GITHUB_COMMIT="$1"
SCRIPT_DIR="$2"
DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"
# latest is a shortcut documented on https://www.haskell.org/ghcup/guide/#tags-and-shortcuts
CABAL_VERSION="latest"
GHC_VERSION="$3"
install_packages() {
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install -y git build-essential curl libffi-dev libffi7 libgmp-dev libgmp10 libncurses-dev libncurses5 libtinfo5 llvm libnuma-dev zlib1g-dev libpq-dev jq gcc
sudo apt-get clean
}
install_ghcup() {
export BOOTSTRAP_HASKELL_NONINTERACTIVE=1
export BOOTSTRAP_HASKELL_MINIMAL=1
curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh
source ~/.ghcup/env
}
install_cabal() {
ghcup upgrade
ghcup install cabal $CABAL_VERSION
ghcup set cabal $CABAL_VERSION
}
install_ghc() {
ghcup upgrade
ghcup install ghc $GHC_VERSION
ghcup set ghc $GHC_VERSION
}
install_packages
# Add ghcup to the PATH for this session
[ -f ~/.ghcup/env ] && source ~/.ghcup/env
ghcup --version || install_ghcup
ghcup set cabal $CABAL_VERSION || install_cabal
ghcup set ghc $GHC_VERSION || install_ghc
cd ~/$SCRIPT_DIR
# Clone the repository and build the project
git clone https://github.com/PostgREST/postgrest.git
cd postgrest
git checkout $PGRST_GITHUB_COMMIT
cabal v2-update && cabal v2-build
# Strip unused symbols from executable
PGRST_BIN=$(cabal exec which postgrest | tail -1)
strip $PGRST_BIN
# Copy the built binary to the Dockerfile directory
cp $PGRST_BIN ~/$DOCKER_BUILD_DIR
# Move and compress the built binary
mkdir -p ~/$SCRIPT_DIR/result
mv $PGRST_BIN ~/$SCRIPT_DIR/result
cd ~/$SCRIPT_DIR
tar -cJf result.tar.xz result
@@ -1,8 +1,6 @@
# PostgREST Docker Hub image for aarch64.
# The x86-64 is a single-static-binary image built via Nix, see:
# nix/tools/docker/README.md
# PostgREST docker hub image
FROM ubuntu:noble@sha256:7c06e91f61fa88c08cc74f7e1b7c69ae24910d745357e0dfe1d2c0322aaf20f9 AS postgrest
FROM ubuntu:noble@sha256:2e863c44b718727c860746568e1d54afd13b2fa71b160f5cd9058fc436217b30 AS postgrest
RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
@@ -10,7 +8,6 @@ RUN apt-get update -y \
&& rm -rf /var/lib/apt/lists/*
COPY postgrest /usr/bin/postgrest
RUN chmod +x /usr/bin/postgrest
EXPOSE 3000
+50
View File
@@ -0,0 +1,50 @@
#!/bin/bash
set -Eeuo pipefail
# This script publishes the Docker ARM images to Docker Hub.
[ -z "$1" ] && { echo "Missing 1st argument: PostgREST github commit SHA"; exit 1; }
[ -z "$2" ] && { echo "Missing 2nd argument: Docker repo"; exit 1; }
[ -z "$3" ] && { echo "Missing 3rd argument: Docker username"; exit 1; }
[ -z "$4" ] && { echo "Missing 4th argument: Docker password"; exit 1; }
[ -z "$5" ] && { echo "Missing 5th argument: Build environment directory name"; exit 1; }
[ -z "$6" ] && { echo "Missing 6th argument: PostgREST version"; exit 1; }
PGRST_GITHUB_COMMIT="$1"
DOCKER_REPO="$2"
DOCKER_USER="$3"
DOCKER_PASS="$4"
SCRIPT_DIR="$5"
PGRST_VERSION="$6"
DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"
clean_env()
{
sudo docker logout
}
# Login to Docker
sudo docker logout
{ echo $DOCKER_PASS | sudo docker login -u $DOCKER_USER --password-stdin; } || { echo "Couldn't login to docker"; exit 1; }
trap clean_env sigint sigterm exit
# Move to the docker build environment
cd ~/$DOCKER_BUILD_DIR
# Push final images to Docker hub
# NOTE: This command publishes a separate ARM image because the builds cannot
# be added to the manifest if they are not in the registry beforehand.
# This image must be manually deleted from Docker Hub at the end of the process.
sudo docker buildx build --build-arg PGRST_GITHUB_COMMIT=$PGRST_GITHUB_COMMIT \
-t $DOCKER_REPO/postgrest:$PGRST_VERSION-arm \
--push .
# Add the arm images to the manifest
# NOTE: This assumes that there already is a `postgrest:<version>` image
# for the amd64 architecture pushed to Docker Hub
sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:$PGRST_VERSION $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
[ "$PGRST_VERSION" != "devel" ] && sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:latest $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
sudo docker logout
+37 -44
View File
@@ -30,10 +30,10 @@ concurrency:
jobs:
static:
name: Nix - Linux x86-64 static
runs-on: ubuntu-24.04
name: Nix - Linux static
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -42,27 +42,27 @@ jobs:
- name: Build static executable
run: nix-build -A postgrestStatic
- name: Save built executable as artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-linux-static-x86-64
name: postgrest-linux-static-x64
path: result/bin/postgrest
if-no-files-found: error
- name: Build Docker image
run: nix-build -A docker.image --out-link postgrest-docker.tar.gz
- name: Save built Docker image as artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-docker-x86-64
name: postgrest-docker-x64
path: postgrest-docker.tar.gz
if-no-files-found: error
macos:
name: Nix - MacOS
runs-on: macos-14
runs-on: macos-12
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -87,50 +87,42 @@ jobs:
fail-fast: false
matrix:
include:
- name: Linux aarch64
runs-on: ubuntu-24.04-arm
- name: Linux
runs-on: ubuntu-22.04
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-ubuntu-aarch64
deps: sudo apt-get update && sudo apt-get install libpq-dev
# no artifact for Linux, because we use the static build
- name: MacOS aarch64
runs-on: macos-14
- name: MacOS
runs-on: macos-12
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-macos-aarch64
deps: brew link --force libpq
- name: MacOS x86-64
runs-on: macos-13
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-macos-x86-64
deps: brew link --force libpq
artifact: postgrest-macos-x64
- name: Windows
runs-on: windows-2022
cache: |
C:\sr\pantry
C:\sr\snapshots
C:\sr\stack.sqlite3
~\AppData\Roaming\stack\pantry
~\AppData\Local\Programs\stack\pantry
~\AppData\Roaming\stack\snapshots
~\AppData\Local\Programs\stack\snapshots
~\AppData\Roaming\stack\stack.sqlite3
~\AppData\Local\Programs\stack\stack.sqlite3
deps: Add-Content $env:GITHUB_PATH $env:PGBIN
artifact: postgrest-windows-x86-64
artifact: postgrest-windows-x64
name: Stack - ${{ matrix.name }}
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: haskell-actions/setup@7909071ceec0344debcc968c6c7a96a52e8dd0d7 # v2.8.1
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: haskell-actions/setup@ec49483bfc012387b227434aba94f59a6ecd0900 # v2.7.5
with:
# This must match the version in stack.yaml's resolver
ghc-version: 9.6.7
ghc-version: 9.6.5
enable-stack: true
stack-no-global: true
stack-setup-ghc: true
@@ -155,7 +147,8 @@ jobs:
- name: Strip Executable
run: strip result/postgrest*
- name: Save built executable as artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: matrix.artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ matrix.artifact }}
path: |
@@ -166,27 +159,27 @@ jobs:
freebsd:
name: Stack - FreeBSD from CirrusCI
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: ./.github/actions/artifact-from-cirrus
with:
token: ${{ github.token }}
task: Build FreeBSD (Stack)
download: bin
upload: postgrest-freebsd-x86-64
upload: postgrest-freebsd-x64
cabal:
strategy:
matrix:
ghc: ['9.6.7', '9.8.4']
ghc: ['9.6.4', '9.8.2']
fail-fast: false
name: Cabal - Linux x86-64 - GHC ${{ matrix.ghc }}
runs-on: ubuntu-24.04
name: Cabal - Linux GHC ${{ matrix.ghc }}
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: haskell-actions/setup@7909071ceec0344debcc968c6c7a96a52e8dd0d7 # v2.8.1
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: haskell-actions/setup@ec49483bfc012387b227434aba94f59a6ecd0900 # v2.7.5
with:
ghc-version: ${{ matrix.ghc }}
- name: Cache .cabal
@@ -195,8 +188,8 @@ jobs:
path: |
~/.cabal/packages
~/.cabal/store
prefix: cabal-${{ matrix.ghc }}-${{ hashFiles('cabal.project.freeze') }}
suffix: ${{ hashFiles('postgrest.cabal', 'cabal.project') }}
prefix: cabal-${{ matrix.ghc }}
suffix: ${{ hashFiles('postgrest.cabal', 'cabal.project', 'cabal.project.freeze') }}
- name: Cache dist-newstyle
uses: ./.github/actions/cache-on-main
with:
+2 -2
View File
@@ -18,9 +18,9 @@ concurrency:
jobs:
lint-style:
name: Lint & Style
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
+296 -4
View File
@@ -5,10 +5,21 @@ on:
branches:
- main
- v[0-9]+
tags:
- devel
- v*
concurrency:
# Terminate all previous runs of the same workflow for the same tag, but never
# cancel for the main / release branch pipelines.
group: ci-${{ (github.ref_type == 'tag' && github.ref) || github.run_id }}
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
jobs:
check:
name: Check
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/check.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
@@ -16,6 +27,7 @@ jobs:
docs:
name: Docs
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/docs.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
@@ -23,6 +35,7 @@ jobs:
test:
name: Test
if: startsWith(github.ref, 'refs/heads/')
uses: ./.github/workflows/test.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
@@ -36,21 +49,87 @@ jobs:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
arm:
name: Build / Cabal - aarch64 GHC 9.4.8
if: vars.SSH_ARM_ENABLED
runs-on: ubuntu-22.04
outputs:
remotepath: ${{ steps.Remote-Dir.outputs.remotepath }}
env:
GITHUB_COMMIT: ${{ github.sha }}
GHC_VERSION: '9.4.8'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- id: Remote-Dir
name: Unique directory name for the remote build
run: echo "remotepath=postgrest-build-$(uuidgen)" >> "$GITHUB_OUTPUT"
- name: Copy script files to the remote server
uses: appleboy/scp-action@master
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
source: ".github/scripts/arm/*"
target: ${{ steps.Remote-Dir.outputs.remotepath }}
strip_components: 3
- name: Build ARM
uses: appleboy/ssh-action@master
env:
REMOTE_DIR: ${{ steps.Remote-Dir.outputs.remotepath }}
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
command_timeout: 120m
script_stop: true
envs: GITHUB_COMMIT,REMOTE_DIR,GHC_VERSION
script: bash ~/$REMOTE_DIR/build.sh "$GITHUB_COMMIT" "$REMOTE_DIR" "$GHC_VERSION"
- name: Download binaries from remote server
uses: nicklasfrahm/scp-action@main
with:
direction: download
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
source: "${{ steps.Remote-Dir.outputs.remotepath }}/result.tar.xz"
target: "result.tar.xz"
- name: Extract downloaded binaries
run: tar -xvf result.tar.xz && rm result.tar.xz
- name: Save aarch64 executable as artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-ubuntu-aarch64
path: result/postgrest
if-no-files-found: error
tag:
name: Tag
name: Release / Tag
concurrency:
# Never tag outdated commits on the main branch by skipping superseded commits
group: ci-tag-${{ (github.ref == 'refs/heads/main' && github.ref) || github.run_id }}
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
if: vars.RELEASE_ENABLED
runs-on: ubuntu-24.04
if: |
vars.RELEASE_ENABLED &&
startsWith(github.ref, 'refs/heads/') &&
needs.docs.result == 'success' &&
needs.test.result == 'success' &&
needs.build.result == 'success' &&
(needs.arm.result == 'skipped' || success())
permissions:
contents: write
runs-on: ubuntu-22.04
needs:
- docs
- test
- build
- arm
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ssh-key: ${{ secrets.POSTGREST_SSH_KEY }}
- name: Tag latest commit
@@ -68,3 +147,216 @@ jobs:
git tag -f "devel"
git push -f origin "devel"
fi
prepare:
name: Release / Prepare
if: |
startsWith(github.ref, 'refs/tags/') &&
needs.build.result == 'success' &&
(needs.arm.result == 'skipped' || success())
runs-on: ubuntu-22.04
needs:
- build
- arm
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
if [ "${GITHUB_REF_NAME}" != "devel" ] && [ "${GITHUB_REF_NAME}" != "v$cabal_version" ]; then
echo "Tagged version ($GITHUB_REF_NAME) does not match the one in postgrest.cabal (v$cabal_version). Aborting release..."
exit 1
fi
- name: Identify changes from CHANGELOG.md
run: |
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
echo "Getting unreleased changes..."
sed -n "1,/## Unreleased/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
else
version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
echo "Propper release, getting changes for version $version ..."
sed -n "1,/## \[$version\]/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
fi
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Save CHANGES.md as artifact
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: release-changes
path: CHANGES.md
if-no-files-found: error
github:
name: Release / GitHub
permissions:
contents: write
runs-on: ubuntu-22.04
needs:
- prepare
if: success() || needs.prepare.result == 'success'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Download all artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: artifacts
- name: Create release bundle with archives for all builds
run: |
find artifacts -type f -iname postgrest -exec chmod +x {} \;
mkdir -p release-bundle
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-x64.tar.xz" \
-C artifacts/postgrest-linux-static-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-x64.tar.xz" \
-C artifacts/postgrest-macos-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-freebsd-x64.tar.xz" \
-C artifacts/postgrest-freebsd-x64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-ubuntu-aarch64.tar.xz" \
-C artifacts/postgrest-ubuntu-aarch64 postgrest
zip "release-bundle/postgrest-${GITHUB_REF_NAME}-windows-x64.zip" \
artifacts/postgrest-windows-x64/postgrest.exe
- name: Save release bundle
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: release-bundle
path: release-bundle
if-no-files-found: error
- name: Publish release on GitHub
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "Releasing version ${GITHUB_REF_NAME} on GitHub..."
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
# To replace the existing release, we must first delete the old assets,
# then modify the release, then add the new assets.
gh release view devel --json assets \
| jq -r '.assets[] | .name' \
| xargs -rn1 \
gh release delete-asset -y devel
gh release edit devel \
-t devel \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
--prerelease
gh release upload --clobber devel release-bundle/*
else
gh release create "${GITHUB_REF_NAME}" \
-t "${GITHUB_REF_NAME}" \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
release-bundle/*
fi
docker:
name: Release / Docker Hub
runs-on: ubuntu-22.04
needs:
- prepare
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
(success() || needs.prepare.result == 'success')
env:
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Download Docker image
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: postgrest-docker-x64
- name: Publish images on Docker Hub
run: |
docker login -u "$DOCKER_USER" -p "$DOCKER_PASS"
docker load -i postgrest-docker.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
# Only tag 'latest' for full releases
if [ "${GITHUB_REF_NAME}" != "devel" ]; then
echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..."
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
docker push "$DOCKER_REPO"/postgrest:latest
else
echo "Skipping push to 'latest' tag for pre-release..."
fi
docker-description:
name: Release / Docker Hub Description
runs-on: ubuntu-22.04
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
github.ref == 'refs/tags/devel'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: peter-evans/dockerhub-description@e98e4d1628a5f3be2be7c231e50981aee98723ae # v4.0.0
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
repository: ${{ vars.DOCKER_REPO }}/postgrest
short-description: ${{ github.event.repository.description }}
readme-filepath: ./docker-hub-readme.md
docker-arm:
name: Release / Docker Hub Arm
runs-on: ubuntu-22.04
needs:
- arm
- docker
env:
GITHUB_COMMIT: ${{ github.sha }}
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Publish images for ARM builds on Docker Hub
uses: appleboy/ssh-action@master
env:
REMOTE_DIR: ${{ needs.arm.outputs.remotepath }}
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
script_stop: true
envs: GITHUB_COMMIT,DOCKER_REPO,DOCKER_USER,DOCKER_PASS,REMOTE_DIR,GITHUB_REF_NAME
script: bash ~/$REMOTE_DIR/docker-publish.sh "$GITHUB_COMMIT" "$DOCKER_REPO" "$DOCKER_USER" "$DOCKER_PASS" "$REMOTE_DIR" "$GITHUB_REF_NAME"
clean-arm:
name: Build / Cleanup
needs:
- arm
- docker-arm
if: ${{ always() && vars.SSH_ARM_ENABLED }}
runs-on: ubuntu-22.04
env:
REMOTE_DIR: ${{ needs.arm.outputs.remotepath }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Remove uploaded files from server
uses: appleboy/ssh-action@master
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
envs: REMOTE_DIR
script: rm -rf $REMOTE_DIR
+18 -4
View File
@@ -25,9 +25,9 @@ concurrency:
jobs:
build:
name: Build
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -39,9 +39,9 @@ jobs:
spellcheck:
name: Spellcheck
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -51,3 +51,17 @@ jobs:
run: postgrest-docs-spellcheck
- name: Run dictcheck
run: postgrest-docs-dictcheck
linkcheck:
name: Linkcheck
if: github.base_ref == 'main'
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.linkcheck.bin
- run: postgrest-docs-linkcheck
-17
View File
@@ -1,17 +0,0 @@
name: Linkcheck
on:
schedule:
- cron: '1 2 * * 3'
jobs:
linkcheck:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.linkcheck.bin
- run: postgrest-docs-linkcheck
-205
View File
@@ -1,205 +0,0 @@
name: Release
on:
push:
tags:
- devel
- v*
concurrency:
# Terminate all previous runs of the same workflow for the same tag.
group: release-${{ github.ref }}
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
jobs:
build:
name: Build
uses: ./.github/workflows/build.yaml
secrets:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
prepare:
name: Prepare
runs-on: ubuntu-24.04
needs:
- build
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
if [ "${GITHUB_REF_NAME}" != "devel" ] && [ "${GITHUB_REF_NAME}" != "v$cabal_version" ]; then
echo "Tagged version ($GITHUB_REF_NAME) does not match the one in postgrest.cabal (v$cabal_version). Aborting release..."
exit 1
fi
- name: Identify changes from CHANGELOG.md
run: |
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
echo "Getting unreleased changes..."
sed -n "1,/## Unreleased/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
else
version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
echo "Propper release, getting changes for version $version ..."
sed -n "1,/## \[$version\]/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
fi
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Save CHANGES.md as artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: release-changes
path: CHANGES.md
if-no-files-found: error
github:
name: GitHub
permissions:
contents: write
runs-on: ubuntu-24.04
needs:
- prepare
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Download all artifacts
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
with:
path: artifacts
- name: Create release bundle with archives for all builds
run: |
find artifacts -type f -iname postgrest -exec chmod +x {} \;
mkdir -p release-bundle
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-linux-static-x86-64.tar.xz" \
-C artifacts/postgrest-linux-static-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-aarch64.tar.xz" \
-C artifacts/postgrest-macos-aarch64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-macos-x86-64.tar.xz" \
-C artifacts/postgrest-macos-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-freebsd-x86-64.tar.xz" \
-C artifacts/postgrest-freebsd-x86-64 postgrest
tar cJvf "release-bundle/postgrest-${GITHUB_REF_NAME}-ubuntu-aarch64.tar.xz" \
-C artifacts/postgrest-ubuntu-aarch64 postgrest
zip --junk-paths "release-bundle/postgrest-${GITHUB_REF_NAME}-windows-x86-64.zip" \
artifacts/postgrest-windows-x86-64/postgrest.exe
- name: Save release bundle
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: release-bundle
path: release-bundle
if-no-files-found: error
- name: Publish release on GitHub
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "Releasing version ${GITHUB_REF_NAME} on GitHub..."
if [ "${GITHUB_REF_NAME}" == "devel" ]; then
# To replace the existing release, we must first delete the old assets,
# then modify the release, then add the new assets.
gh release view devel --json assets \
| jq -r '.assets[] | .name' \
| xargs -rn1 \
gh release delete-asset -y devel
gh release edit devel \
-t devel \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
--prerelease
gh release upload --clobber devel release-bundle/*
else
gh release create "${GITHUB_REF_NAME}" \
-t "${GITHUB_REF_NAME}" \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
release-bundle/*
fi
docker:
name: Docker Hub
runs-on: ubuntu-24.04-arm
needs:
- prepare
if: |
vars.DOCKER_REPO && vars.DOCKER_USER
env:
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Download x86-64 Docker image
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
with:
name: postgrest-docker-x86-64
- name: Download aarch64 binary
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
with:
name: postgrest-ubuntu-aarch64
- uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
- name: Build aarch64 Docker image
run: |
# This only pushes the image via digest, not a tag. This will not appear
# in the image list on Docker Hub, yet. It will be later added to the main
# tag's manifest.
docker buildx build \
-t "$DOCKER_REPO/postgrest" \
--platform linux/arm64 \
--output push-by-digest=true,type=image,push=true \
--metadata-file metadata.json \
.
echo "SHA256_ARM=$(jq -r '."containerimage.digest"' metadata.json)" >> "$GITHUB_ENV"
- name: Publish images on Docker Hub
run: |
docker load -i postgrest-docker.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker push "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}"
docker buildx imagetools create --append \
-t "$DOCKER_REPO/postgrest:${GITHUB_REF_NAME}" \
"$DOCKER_REPO/postgrest@$SHA256_ARM"
# Only tag 'latest' for full releases
if [ "${GITHUB_REF_NAME}" != "devel" ]; then
echo "Pushing to 'latest' tag for full release of ${GITHUB_REF_NAME} ..."
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
docker push "$DOCKER_REPO"/postgrest:latest
docker buildx imagetools create --append \
-t "$DOCKER_REPO/postgrest:latest" \
"$DOCKER_REPO/postgrest@$SHA256_ARM"
else
echo "Skipping push to 'latest' tag for pre-release..."
fi
docker-description:
name: Docker Hub Description
runs-on: ubuntu-24.04
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
github.ref == 'refs/tags/devel'
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: peter-evans/dockerhub-description@432a30c9e07499fd01da9f8a49f0faf9e0ca5b77 # v4.0.2
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
repository: ${{ vars.DOCKER_REPO }}/postgrest
short-description: ${{ github.event.repository.description }}
readme-filepath: ./docker-hub-readme.md
+39
View File
@@ -0,0 +1,39 @@
name: Upload Reports
on:
workflow_run:
workflows:
- CI
- Test
branches-ignore:
- devel
- v[0-9]+
- v[0-9]+.[0-9]+.[0-9]+
types:
- completed
jobs:
loadtest:
name: Loadtest
permissions:
checks: write
runs-on: ubuntu-22.04
if: github.event.workflow_run.conclusion == 'success' || github.event.workflow_run.conclusion == 'failure'
steps:
- name: Download from Artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
github-token: ${{ github.token }}
run-id: ${{ github.event.workflow_run.id }}
name: loadtest.md
path: artifacts
- name: Upload to GitHub Checks
uses: LouisBrunner/checks-action@6b626ffbad7cc56fd58627f774b9067e6118af23 # v2.0.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
sha: ${{ github.event.workflow_run.head_sha }}
name: Loadtest Results
conclusion: neutral
output: |
{"summary":""}
output_text_description_file: artifacts/loadtest.md
+22 -45
View File
@@ -32,14 +32,14 @@ concurrency:
jobs:
coverage:
name: Coverage
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
defaults:
run:
# Hack for enabling color output, see:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -49,7 +49,7 @@ jobs:
- name: Run coverage (IO tests and Spec tests against PostgreSQL 15)
run: postgrest-coverage
- name: Upload coverage to codecov
uses: codecov/codecov-action@fdcc8476540edceab3de004e990f80d881c6cc00 # v5.5.0
uses: codecov/codecov-action@e28ff129e5465c2c0dcc6f003fc735cb6ae0c673 # v4.5.0
with:
files: ./coverage/codecov.json
token: ${{ secrets.CODECOV_TOKEN }}
@@ -67,16 +67,16 @@ jobs:
strategy:
fail-fast: false
matrix:
pgVersion: [12, 13, 14, 15, 16, 17]
pgVersion: ["9_6", 10, 11, 12, 13, 14, 15, 16]
name: PG ${{ matrix.pgVersion }}
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
defaults:
run:
# Hack for enabling color output, see:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -98,26 +98,23 @@ jobs:
memory:
name: Memory
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: tests.testMemory.bin
tools: memory.test.bin
- name: Run memory tests
run: postgrest-test-memory
loadtest:
strategy:
matrix:
kind: ['mixed', 'jwt']
name: Loadtest
runs-on: ubuntu-24.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
fetch-depth: 0
- name: Setup Nix Environment
@@ -125,37 +122,17 @@ jobs:
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: loadtest.loadtestAgainst.bin loadtest.report.bin
- uses: WyriHaximus/github-action-get-previous-tag@04e8485ecb6487243907e330d522ff60f02283ce # v1.4.0
id: get-latest-tag
with:
prefix: v
- name: Run loadtest
env:
TARGET_BRANCH: ${{ github.base_ref || github.ref_name }}
run: |
if [ "$TARGET_BRANCH" = "main" ]; then
latest_tag=$(git tag --sort=-creatordate --list "v*" | head -n1)
else
latest_tag=$(git tag --merged HEAD --sort=-creatordate "v*" | head -n1)
fi
postgrest-loadtest-against -k ${{ matrix.kind }} "$TARGET_BRANCH" "$latest_tag"
postgrest-loadtest-report >> "$GITHUB_STEP_SUMMARY"
flake:
strategy:
fail-fast: false
matrix:
runs-on:
- macos-13 # x86_64-darwin
- macos-14 # aarch64-darwin
- ubuntu-24.04 # x86_64-linux
- ubuntu-24.04-arm # aarch64-linux
name: Flake Check
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
postgrest-loadtest-against main ${{ steps.get-latest-tag.outputs.tag }}
postgrest-loadtest-report > loadtest/loadtest.md
- name: Upload report
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
fetch-depth: 0
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Run flake check
run: |
nix flake check
name: loadtest.md
path: loadtest/loadtest.md
if-no-files-found: error
+1 -2
View File
@@ -14,7 +14,7 @@ site
.#*
*.swp
result*
dist-*
dist-newstyle
postgrest.hp
postgrest.prof
__pycache__
@@ -24,4 +24,3 @@ coverage
loadtest
.history
.docs-build
gen_targets.http
+1 -1
View File
@@ -5,6 +5,6 @@ python:
install:
- requirements: docs/requirements.txt
build:
os: ubuntu-24.04
os: ubuntu-22.04
tools:
python: "3.11"
+11 -11
View File
@@ -13,21 +13,21 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
</a>
</td>
<td align="center" valign="middle">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/neon.jpg">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/neon.jpg">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://www.euronodes.com/postgrest" target="_blank">
<img width="296px" src="static/euronodes.svg">
<a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/code-build.png">
</a>
</td>
<td align="center" valign="middle">
@@ -35,6 +35,11 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
<img width="296px" src="static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
</a>
</td>
</tr>
</tbody>
</table>
@@ -88,11 +93,6 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
<img width="222px" src="static/oblivious.jpg">
</a>
</td>
<td align="center" valign="middle">
<a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="222px" src="static/code-build.png">
</a>
</td>
</tr>
</tbody>
</table>
-171
View File
@@ -5,177 +5,6 @@ This project adheres to [Semantic Versioning](http://semver.org/).
## Unreleased
## [13.0.6] - 2025-08-30
### Fixed
- Fix logging the Haskell type instead of the listener error message directly by @laurenceisla in #3588
- Fix format of `IPv6` address logged at PostgREST startup by @taimoorzaeem in #4291
- Fix empty enum in `preferParams` OpenAPI parameter by @laurenceisla in #4292
## [13.0.5] - 2025-08-24
### Fixed
- Fix OpenAPI broken docs link by @taimoorzaeem in #4048
- Fix OpenAPI specification incorrectly exposing GET methods for volatile functions by @joelonsql in #4174
- Fix empty spread embeddings return unexpected SQL error by @taimoorzaeem in #3887
- Fix `/metrics` endpoint not responding with `Content-Type` header by @taimoorzaeem in #4271
## [13.0.4] - 2025-06-17
### Fixed
- Fix regression that makes full-text search not work on domain types based on `tsvector` by @laurenceisla in #4135
- Fix `jwt-aud` config not failing when set to an invalid URI by @taimoorzaeem in #4132
## [13.0.3] - 2025-06-16
### Fixed
- Fix `max-affected` preference not failing with RPC when `handling=strict` by @taimoorzaeem in #4100
- Fix a property definition's type in OpenAPI not showing the correct base type of a recursive domain by @laurenceisla in #4136
## [13.0.2] - 2025-06-02
### Fixed
- Fix regression that makes `ORDER BY` with nulls-order not work alongside limits by @laurenceisla in #4109
## [13.0.1] - 2025-06-01
### Fixed
- Fix jwt error returning HTTP status `400` for invalid role by @taimoorzaeem in #3601
- Fix `db-extra-search-path` cannot be set to nothing by @taimoorzaeem in #4074
+ It can now be disabled by setting it to empty string.
+ Schema Cache load error is now logged including `db-schemas` and `db-extra-search-path` config values.
## [13.0.0] - 2025-05-08
### Added
- #3558, Add the `admin-server-host` config to set the host for the admin server - @develop7
- #3607, Log to stderr when the JWT secret is less than 32 characters long - @laurenceisla
- #2858, Performance improvements when calling RPCs via GET using indexes in more cases - @wolfgangwalther
- #3560, Log resolved host in "Listening on ..." messages - @develop7
- #3727, Log maximum pool size - @steve-chavez
- #1536, Add string comparison feature for jwt-role-claim-key - @taimoorzaeem
- #3747, Allow `not_null` value for the `is` operator - @taimoorzaeem
- #2255, Apply `to_tsvector()` explicitly to the full-text search filtered column (excluding `tsvector` types) - @laurenceisla
- #1578, Log the main SQL query to stderr at the current `log-level` when `log-query=main-query` - @laurenceisla
- #3903, Log connection pool borrows on `log-level=debug` - @taimoorzaeem
- #3041, Allow spreading one-to-many and many-to-many embedded resources - @laurenceisla
+ The selected columns in the embedded resources are aggregated into arrays
+ Aggregates are not supported
- #2967, Add `Proxy-Status` header for better error response - @taimoorzaeem
- #4016, Add `Content-Length` response header - @laurenceisla
### Fixed
- #3693, Prevent spread embedding to allow aggregates when they are disabled - @laurenceisla
- #3693, A nested spread embedding now correctly groups by the fields of its top parent relationship - @laurenceisla
- #3693, Fix spread embedding errors when using the `count()` aggregate without a field - @laurenceisla
+ Fixed `"column reference <col> is ambiguous"` error when selecting `?select=...table(col,count())`
+ Fixed `"column <json_aggregate>.<alias> does not exist"` error when selecting `?select=...table(aias:count())`
- #3727, Clarify "listening" logs - @steve-chavez
- #3795, Clarify `Accept: vnd.pgrst.object` error message - @steve-chavez
- #3697, #3602, Handle queries on non-existing table gracefully - @taimoorzaeem
- #3600, #3926, Improve JWT errors - @taimoorzaeem
- #3013, Fix `order=` with POST, PATCH, PUT and DELETE requests - @taimoorzaeem
- #3965, Fix filter on unselected columns in a table-valued function - @taimoorzaeem
- #4052, Fix schema cache load duplicate objects with different object type but same oid - @taimoorzaeem
### Changed
- #2052, Dropped support for PostgreSQL 9.6 - @wolfgangwalther
- #2052, Dropped support for PostgreSQL 10 - @wolfgangwalther
- #2052, Dropped support for PostgreSQL 11 - @wolfgangwalther
- #3508, PostgREST now fails to start when `server-port` and `admin-server-port` config options are the same - @develop7
- #3607, PostgREST now fails to start when the JWT secret is less than 32 characters long - @laurenceisla
- #3644, Fail schema cache lookup with invalid `db-schemas` or `db-extra-search-path` config - @wolfgangwalther
- Previously, this would silently return 200 - OK on the root endpoint, but don't provide any usable endpoints.
- Note: This also applies when deleting the `public` schema - both config options default to that.
- #3757, Remove support for `Prefer: params=single-object` - @joelonsql
+ This preference was deprecated in favor of Functions with an array of JSON objects
- #3013, Drop support for Limited updates/deletes
+ The feature was complicated and largely unused.
- #3956, Drop `/config` endpoint of admin server - @steve-chavez
+ The endpoint was at risk of being left unprotected when exposing it.
+ The accompanying `admin-server-config-enabled` config was also dropped.
- #3697, #3602, Querying non-existent table now returns `PGRST205` error instead of empty json - @taimoorzaeem
- #3600, #3926, Improve JWT errors - @taimoorzaeem
+ Return `PGRST301` error when `Bearer` in auth header is sent empty
+ Diagnostic error messages instead of exposed internals
+ Return new `PGRST303` error when jwt claims decoding fails
- #3906, Return `PGRST125` and `PGRST126` errors instead of empty json - @taimoorzaeem
## [12.2.12] - 2025-05-01
### Fixed
- #3956, Fix exposing admin server `/config` by default - @steve-chavez
+ The above endpoint is now disabled unless the `admin-server-config-enabled` config is set to `true`
## [12.2.11] - 2025-04-22
### Fixed
- #4030, Fix regression with parameter `charset=utf-8` in mediatype - @taimoorzaeem
## [12.2.10] - 2025-04-18
### Fixed
- #3889, Fix: JWT cache purging on every request decreases performance - @mkleczek
## [12.2.9] - 2025-04-16
### Fixed
- #3498, Fix incorrect parsing of the `for` parameter of the `application/vnd.pgrst.plan` media type - @taimoorzaeem
- #4014, Fix JWT cache allows old tokens after the jwt-secret is changed in a config reload - @taimoorzaeem
## [12.2.8] - 2025-02-10
### Fixed
- #3841, Log `503` client error to stderr - @taimoorzaeem
## [12.2.7] - 2025-02-03
### Fixed
- #2524, Fix schema reloading notice on windows - @diogob
## [12.2.6] - 2025-01-29
### Fixed
- #3788, Fix jwt cache does not remove expired entries - @taimoorzaeem
## [12.2.5] - 2025-01-20
### Fixed
- #3867, Fix startup for arm64 docker image - @wolfgangwalther
## [12.2.4] - 2025-01-18
### Fixed
- #3779, Always log the schema cache load time - @steve-chavez
- #3706, Fix insert with `missing=default` uses default value of domain instead of column - @taimoorzaeem
## [12.2.3] - 2024-08-01
### Fixed
- #3091, Broken link in OpenAPI description `externalDocs` - @salim-b
- #3659, Embed One-to-One relationship with different column order properly - @wolfgangwalther
- #3504, Remove `format` from `rowFilter` parameters in OpenAPI - @dantheman2865
- #3660, Fix regression that loaded the schema cache before the in-database configuration - @steve-chavez, @laurenceisla
## [12.2.2] - 2024-07-10
### Fixed
+3
View File
@@ -0,0 +1,3 @@
This repository follows the same contribution guidelines as the main PostgREST repository contribution guidelines:
https://github.com/PostgREST/postgrest/blob/main/.github/CONTRIBUTING.md
+12 -6
View File
@@ -1,6 +1,7 @@
![Logo](static/postgrest.png "Logo")
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854)](https://www.patreon.com/postgrest)
[![Join the chat at https://gitter.im/begriffs/postgrest](https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/begriffs/postgrest)
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://postgrest.org)
[![Docker Stars](https://img.shields.io/docker/pulls/postgrest/postgrest.svg)](https://hub.docker.com/r/postgrest/postgrest/)
[![Build Status](https://github.com/postgrest/postgrest/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/PostgREST/postgrest/actions?query=branch%3Amain)
@@ -22,21 +23,21 @@ API than you are likely to write from scratch.
</a>
</td>
<td align="center" valign="middle">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/neon.jpg">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/neon.jpg">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://www.euronodes.com/postgrest" target="_blank">
<img width="296px" src="static/euronodes.svg">
<a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/code-build.png">
</a>
</td>
<td align="center" valign="middle">
@@ -44,6 +45,11 @@ API than you are likely to write from scratch.
<img width="296px" src="static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/tembo.png">
</a>
</td>
</tr>
</tbody>
</table>
+1 -1
View File
@@ -1 +1 @@
index-state: hackage.haskell.org 2025-07-08T15:50:10Z
index-state: hackage.haskell.org 2024-05-17T23:41:49Z
+35 -25
View File
@@ -3,15 +3,7 @@
, compiler ? "ghc948"
, # Commit of the Nixpkgs repository that we want to use.
# It defaults to reading the inputs from flake.lock, which serves
# as a compatibility layer for non-flake builds / default.nix / shell.nix.
nixpkgsVersion ? let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
{
inherit (lock.nodes.nixpkgs.locked) owner repo rev;
tarballHash = lock.nodes.nixpkgs.locked.narHash;
}
nixpkgsVersion ? import nix/nixpkgs-version.nix
, # Nix files that describe the Nixpkgs repository. We evaluate the expression
# using `import` below.
@@ -27,7 +19,7 @@ let
"postgrest";
# PostgREST source files, filtered based on the rules in the .gitignore files
# and file extensions. We want to include as little as possible, as the files
# and file extensions. We want to include as litte as possible, as the files
# added here will increase the space used in the Nix store and trigger the
# build of new Nix derivations when changed.
src =
@@ -44,6 +36,9 @@ let
allOverlays.checked-shell-script
allOverlays.gitignore
allOverlays.postgresql-libpq
allOverlays.postgresql-legacy
allOverlays.postgresql-future
allOverlays.postgis
(allOverlays.haskell-packages { inherit compiler; })
allOverlays.slocat
];
@@ -54,28 +49,34 @@ let
postgresqlVersions =
[
{ name = "postgresql-17"; postgresql = pkgs.postgresql_17.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-16"; postgresql = pkgs.postgresql_16.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-12"; postgresql = pkgs.postgresql_12.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-11"; postgresql = pkgs.postgresql_11.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-10"; postgresql = pkgs.postgresql_10.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-9_6"; postgresql = pkgs.postgresql_9_6.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
];
# Dynamic derivation for PostgREST
postgrest = pkgs.lib.pipe (pkgs.haskell.packages."${compiler}".callCabal2nix name src { }) [
# To allow ghc-datasize to be used.
lib.disableLibraryProfiling
# We are never going to use dynamic haskell libraries anyway. "Dynamic" refers to how
# non-haskell deps are linked. All haskell dependencies are always statically linked.
lib.disableSharedLibraries
];
postgrest =
pkgs.haskell.packages."${compiler}".callCabal2nix name src { };
staticHaskellPackage = import nix/static.nix { inherit compiler name pkgs src; };
# Options passed to cabal in dev tools and tests
devCabalOptions =
"-f dev --test-show-detail=direct --disable-shared";
"-f dev --test-show-detail=direct";
profiledHaskellPackages =
pkgs.haskell.packages."${compiler}".extend (_: super:
{
mkDerivation =
args:
super.mkDerivation (args // { enableLibraryProfiling = true; });
}
);
inherit (pkgs.haskell) lib;
in
@@ -89,11 +90,12 @@ rec {
lib.dontCheck postgrest;
# Profiled dynamic executable.
postgrestProfiled = pkgs.lib.pipe postgrestPackage [
lib.enableExecutableProfiling
lib.enableLibraryProfiling
lib.dontHaddock
];
postgrestProfiled =
lib.enableExecutableProfiling (
lib.dontHaddock (
lib.dontCheck (profiledHaskellPackages.callCabal2nix name src { })
)
);
inherit (postgrest) env;
@@ -123,9 +125,17 @@ rec {
loadtest =
pkgs.callPackage nix/tools/loadtest.nix { inherit withTools; };
# Script for running memory tests.
memory =
pkgs.callPackage nix/tools/memory.nix { inherit postgrestProfiled withTools; };
# Utility for updating the pinned version of Nixpkgs.
nixpkgsTools =
pkgs.callPackage nix/tools/nixpkgsTools.nix { };
# Scripts for publishing new releases.
release =
pkgs.callPackage nix/tools/release.nix { };
pkgs.callPackage nix/tools/release { };
# Linting and styling tools.
style =
+13 -7
View File
@@ -1,5 +1,6 @@
# PostgREST
[![Join the chat at https://gitter.im/begriffs/postgrest](https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/begriffs/postgrest)
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854)](https://www.patreon.com/postgrest)
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://postgrest.org)
[![Build Status](https://github.com/postgrest/postgrest/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/PostgREST/postgrest/actions?query=branch%3Amain)
@@ -19,21 +20,21 @@ write from scratch.
</a>
</td>
<td align="center" valign="middle">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/neon.jpg">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/tembo.png">
<a href="https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/neon.jpg">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://www.euronodes.com/postgrest" target="_blank">
<img width="296px" src="static/euronodes.svg">
<a href="https://code.build/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/code-build.png">
</a>
</td>
<td align="center" valign="middle">
@@ -41,6 +42,11 @@ write from scratch.
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/tembo.png">
</a>
</td>
</tr>
</tbody>
</table>
@@ -59,7 +65,7 @@ The image is built from scratch using
[Nix](https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools) instead of a
`Dockerfile`, which yields a highly secure and optimized image. This is also why
no commands are listed in the image history. See the [PostgREST
repository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for
respository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for
details on the build process and how to inspect the image.
This does not apply to the arm64 variant, which is based on Ubuntu.
+2 -9
View File
@@ -60,10 +60,6 @@ note bottom of ApiRequest
Parses the URL syntax
end note
note bottom of Plan
Generates internal AST
end note
note bottom of Query
Generates the SQL
end note
@@ -75,15 +71,12 @@ end note
url of Admin is [[../references/admin_server.html#admin-server]]
url of API is [[../explanations/schema_isolation.html]]
url of Auth is [[../references/auth.html#authn]]
url of ApiRequest is [[../explanations/architecture.html#api-request]]
url of Plan is [[../explanations/architecture.html#plan]]
url of Query is [[../explanations/architecture.html#query]]
url of Authorization is [[../explanations/db_authz.html]]
url of CLI is [[../references/cli.html#cli]]
url of "Connection Pool" is [[../references/connection_pool.html]]
url of Config is [[../references/configuration.html#configuration]]
url of HTTPADMIN is [[../explanations/architecture.html#http]]
url of HTTPAPI is [[../explanations/architecture.html#http]]
url of HTTPADMIN is [[https://aosabook.org/en/posa/warp.html]]
url of HTTPAPI is [[https://aosabook.org/en/posa/warp.html]]
url of Listener is [[../references/listener.html#listener]]
url of Proxy is [[../explanations/nginx.html]]
url of "Schema Cache" is [[../references/schema_cache.html#schema-cache]]
+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 23 KiB

+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 27 KiB

After

Width:  |  Height:  |  Size: 24 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 156 B

After

Width:  |  Height:  |  Size: 468 B

+1 -1
View File
@@ -1 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentStyleType="text/css" height="391px" preserveAspectRatio="none" style="width:328px;height:391px;" version="1.1" viewBox="0 0 328 391" width="328px" zoomAndPan="magnify"><defs/><g><!--cluster PostgreSQL--><g id="cluster_PostgreSQL"><path d="M6,16 C6,6 158.5,6 158.5,6 C158.5,6 311,6 311,16 L311,293.59 C311,303.59 158.5,303.59 158.5,303.59 C158.5,303.59 6,303.59 6,293.59 L6,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><path d="M6,16 C6,26 158.5,26 158.5,26 C158.5,26 311,26 311,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="92.2305" x="112.3848" y="40.9951">PostgreSQL</text></g><!--cluster public--><g id="cluster_public"><polygon fill="none" points="30,74,40,64,153,64,153,146.29,143,156.29,30,156.29,30,74" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="143" x2="153" y1="74" y2="64"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="30" x2="143" y1="74" y2="74"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="143" x2="143" y1="74" y2="156.29"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="47.9063" x="63.5469" y="89.9951">public</text></g><!--cluster API--><g id="cluster_API"><polygon fill="none" points="70,190.29,80,180.29,246,180.29,246,269.59,236,279.59,70,279.59,70,190.29" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="236" x2="246" y1="190.29" y2="180.29"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="70" x2="236" y1="190.29" y2="190.29"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="236" x2="236" y1="190.29" y2="279.59"/><text fill="#FFFFFF" font-family="sans-serif" font-size="20" font-weight="bold" lengthAdjust="spacing" textLength="34.668" x="136.666" y="211.8545">api</text></g><!--entity tables_public--><g id="elem_tables_public"><rect fill="#313139" height="36.2969" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="62.752" x="71.62" y="104"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="42.752" x="81.62" y="126.9951">tables</text></g><!--entity extensions--><g id="elem_extensions"><polygon fill="none" points="169.14,109,179.14,99,294.8695,99,294.8695,135.2969,284.8695,145.2969,169.14,145.2969,169.14,109" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="284.8695" x2="294.8695" y1="109" y2="99"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="169.14" x2="284.8695" y1="109" y2="109"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="284.8695" x2="284.8695" y1="109" y2="145.2969"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="85.7295" x="184.14" y="131.9951">extensions</text></g><!--entity vf_api--><g id="elem_vf_api"><rect fill="#313139" height="36.2969" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="144.6465" x="85.68" y="227.29"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="124.6465" x="95.68" y="250.2851">views + functions</text></g><!--entity PostgREST--><g id="elem_PostgREST"><ellipse cx="158" cy="352.59" fill="#313139" rx="8" ry="8" style="stroke:#E7E7E7;stroke-width:0.5;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="74.6895" x="120.6553" y="382.5851">PostgREST</text></g><!--reverse link tables_public to vf_api--><g id="link_tables_public_vf_api"><path d="M110.03,146.6 C110.03,169.85 110.03,203.55 110.03,226.86 " fill="none" id="tables_public-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="110.03,140.6,106.03,149.6,110.03,145.6,114.03,149.6,110.03,140.6" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--reverse link extensions to vf_api--><g id="link_extensions_vf_api"><path d="M199.73,151.63 C199.73,175.24 199.73,205.18 199.73,226.88 " fill="none" id="extensions-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="199.73,145.63,195.73,154.63,199.73,150.63,203.73,154.63,199.73,145.63" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--link vf_api to PostgREST--><g id="link_vf_api_PostgREST"><path d="M158,269.62 C158,292.9 158,320.34 158,337.81 " fill="none" id="vf_api-PostgREST" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="158,263.62,154,272.62,158,268.62,162,272.62,158,263.62" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="158,343.81,162,334.81,158,338.81,154,334.81,158,343.81" style="stroke:#E7E7E7;stroke-width:3.0;"/></g><!--SRC=[KypCIyufJKbLqDFJBqxEqCqipjShpSq10000]--></g></svg>
<?xml version="1.0" encoding="us-ascii" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentStyleType="text/css" height="411px" preserveAspectRatio="none" style="width:315px;height:411px;" version="1.1" viewBox="0 0 315 411" width="315px" zoomAndPan="magnify"><defs/><g><!--cluster PostgreSQL--><g id="cluster_PostgreSQL"><path d="M6,16 C6,6 152,6 152,6 C152,6 298,6 298,16 L298,309.14 C298,319.14 152,319.14 152,319.14 C152,319.14 6,319.14 6,309.14 L6,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><path d="M6,16 C6,26 152,26 152,26 C152,26 298,26 298,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="81.375" x="111.3125" y="42.9659">PostgreSQL</text></g><!--cluster public--><g id="cluster_public"><polygon fill="none" points="30,77,40,67,150,67,150,155.07,140,165.07,30,165.07,30,77" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="140" x2="150" y1="77" y2="67"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="30" x2="140" y1="77" y2="77"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="140" x2="140" y1="77" y2="165.07"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="44.625" x="63.6875" y="94.9659">public</text></g><!--cluster API--><g id="cluster_API"><polygon fill="none" points="68,199.07,78,189.07,236,189.07,236,285.14,226,295.14,68,295.14,68,199.07" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="226" x2="236" y1="199.07" y2="189.07"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="68" x2="226" y1="199.07" y2="199.07"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="226" x2="226" y1="199.07" y2="295.14"/><text fill="#FFFFFF" font-family="sans-serif" font-size="20" font-weight="bold" lengthAdjust="spacing" textLength="29.875" x="133.0625" y="223.45">api</text></g><!--entity tables_public--><g id="elem_tables_public"><rect fill="#313139" height="39.0679" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="61" x="69.5" y="110"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="41" x="79.5" y="134.9659">tables</text></g><!--entity extensions--><g id="elem_extensions"><polygon fill="none" points="165.81,115,175.81,105,282.185,105,282.185,144.0679,272.185,154.0679,165.81,154.0679,165.81,115" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="272.185" x2="282.185" y1="115" y2="105"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="165.81" x2="272.185" y1="115" y2="115"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="272.185" x2="272.185" y1="115" y2="154.0679"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="76.375" x="180.81" y="139.9659">extensions</text></g><!--entity vf_api--><g id="elem_vf_api"><rect fill="#313139" height="39.0679" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="136" x="84" y="240.07"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="116" x="94" y="265.0359">views + functions</text></g><!--entity PostgREST--><g id="elem_PostgREST"><ellipse cx="152" cy="371.14" fill="#313139" rx="8" ry="8" style="stroke:#E7E7E7;stroke-width:0.5;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="70" x="117" y="403.1059">PostgREST</text></g><!--reverse link tables_public to vf_api--><g id="link_tables_public_vf_api"><path d="M107.25,155.29 C107.25,179.74 107.25,215.13 107.25,239.67 " fill="none" id="tables_public-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="107.25,149.29,103.25,158.29,107.25,154.29,111.25,158.29,107.25,149.29" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--reverse link extensions to vf_api--><g id="link_extensions_vf_api"><path d="M192.91,160.31 C192.91,185.07 192.91,216.68 192.91,239.61 " fill="none" id="extensions-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="192.91,154.31,188.91,163.31,192.91,159.31,196.91,163.31,192.91,154.31" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--link vf_api to PostgREST--><g id="link_vf_api_PostgREST"><path d="M152,285.55 C152,309.98 152,338.34 152,356.18 " fill="none" id="vf_api-PostgREST" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="152,279.55,148,288.55,152,284.55,156,288.55,152,279.55" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="152,362.18,156,353.18,152,357.18,148,353.18,152,362.18" style="stroke:#E7E7E7;stroke-width:3.0;"/></g><!--SRC=[KypCIyufJKbLqDFJBqxEqCqipjShpSq10000]--></g></svg>

Before

Width:  |  Height:  |  Size: 4.8 KiB

After

Width:  |  Height:  |  Size: 4.8 KiB

+1 -1
View File
File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 5.1 KiB

After

Width:  |  Height:  |  Size: 5.1 KiB

+4 -12
View File
@@ -50,7 +50,7 @@ source_suffix = ".rst"
master_doc = "index"
# This is overriden by readthedocs with the version tag anyway
version = "13.0"
version = "12.2"
# To avoid repetition in <title> we set this to an empty string.
release = ""
@@ -114,7 +114,7 @@ html_theme = "sphinx_rtd_theme"
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {}
html_theme_options = {"display_version": False}
# Add any paths that contain custom themes here, relative to this directory.
# html_theme_path = []
@@ -292,20 +292,12 @@ def setup(app):
app.add_css_file("css/custom.css")
user_agent = (
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:135.0) Gecko/20100101 Firefox/135.0"
)
# taken from https://github.com/sphinx-doc/sphinx/blob/82dad44e5bd3776ecb6fd8ded656bc8151d0e63d/sphinx/util/requests.py#L42
user_agent = "Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0"
linkcheck_ignore = [
# 403 only in CI / GitHub Actions
r"https://www.patreon.com/postgrest",
r"https://blog.frankel.ch/poor-man-api",
# Odd SSL error
r"https://www.dripdepot.com",
# New GitHub UI delays comment load, so anchor fails
r"https://github.com/.*#issuecomment",
# Random 500 Internal Server Error
r"https://jwt.io",
]
# sphinx-tabs configuration
+4 -2
View File
@@ -8,6 +8,9 @@ Community Tutorials
* `PostgREST + Auth0: Create REST API in mintutes, and add social login using Auth0 <https://samkhawase.com/blog/postgrest/>`_ - A step-by-step tutorial to show how to dockerize and integrate Auth0 to PostgREST service.
* `PostgREST + PostGIS API tutorial in 5 minutes <https://gis-ops.com/postgrest-postgis-api-tutorial-geospatial-api-in-5-minutes/>`_ -
In this tutorial, GIS • OPS shows how to perform PostGIS calculations through PostgREST :ref:`functions` interface.
* `"CodeLess" backend using postgres, postgrest and oauth2 authentication with keycloak <https://www.mathieupassenaud.fr/codeless_backend/>`_ -
A step-by-step tutorial for using PostgREST with KeyCloak(hosted on a managed service).
@@ -34,7 +37,6 @@ Templates
Example Apps
------------
* `archtika <https://github.com/thiloho/archtika>`_ - selfhosted CMS
* `delibrium-postgrest <https://gitlab.com/delibrium/delibrium-postgrest/>`_ - example school API and front-end in Vue.js
* `ETH-transactions-storage <https://github.com/Adamant-im/ETH-transactions-storage>`_ - indexer for Ethereum to get transaction list by ETH address
* `general <https://github.com/PierreRochard/general>`_ - example auth back-end
@@ -88,7 +90,7 @@ Client-Side Libraries
* `postgrest-go <https://github.com/supabase-community/postgrest-go>`_ - Go
* `postgrest-js <https://github.com/supabase/postgrest-js>`_ - TypeScript/JavaScript
* `postgrest-kt <https://github.com/supabase-community/postgrest-kt>`_ - Kotlin
* `postgrest-py <https://github.com/supabase/postgrest-py>`_ - Python
* `postgrest-py <https://github.com/supabase-community/postgrest-py>`_ - Python
* `postgrest-rs <https://github.com/supabase-community/postgrest-rs>`_ - Rust
* `postgrest-swift <https://github.com/supabase-community/postgrest-swift>`_ - Swift
* `redux-postgrest <https://github.com/andytango/redux-postgrest>`_ - TypeScript/JS, client integrated with (React) Redux.
+1 -1
View File
@@ -58,7 +58,7 @@ A request might be rejected at this level if it's invalid. For example when prov
Plan
----
Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Plan.hs>`_ generates an internal AST, filling out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request.
Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Plan.hs>`_ fills in out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request.
A request might be rejected at this level if it's invalid. For example when doing resource embedding on a nonexistent resource.
+1 -1
View File
@@ -15,7 +15,7 @@ Roles for Each Web User
PostgREST can accommodate either viewpoint. If you treat a role as a single user then the :ref:`jwt_impersonation` does most of what you need. When an authenticated user makes a request PostgREST will switch into the database role for that user, which in addition to restricting queries, is available to SQL through the :code:`current_user` variable.
You can use row-level security to flexibly restrict visibility and access for the current user. Here is an `example <https://www.enterprisedb.com:443/blog/application-users-vs-row-level-security>`_ from Tomas Vondra, a chat table storing messages sent between users. Users can insert rows into it to send messages to other users, and query it to see messages sent to them by other users.
You can use row-level security to flexibly restrict visibility and access for the current user. Here is an `example <https://www.2ndquadrant.com/en/blog/application-users-vs-row-level-security/>`_ from Tomas Vondra, a chat table storing messages sent between users. Users can insert rows into it to send messages to other users, and query it to see messages sent to them by other users.
.. code-block:: postgres
+7 -2
View File
@@ -16,11 +16,16 @@ Supported PostgreSQL versions
=============================
=============== =================================
**Supported** PostgreSQL >= 12
**Supported** PostgreSQL >= 9.6
=============== =================================
PostgREST works with all PostgreSQL versions still `officially supported <https://www.postgresql.org/support/versioning/>`_.
PostgREST works with all PostgreSQL versions starting from 9.6.
.. note::
Support for PostgreSQL versions 9.6, 10 and 11 is deprecated. From this on version onwards, PostgREST will only support non-end-of-life PostgreSQL versions.
See https://www.postgresql.org/support/versioning/.
Running PostgREST
=================
+25 -12
View File
@@ -16,6 +16,9 @@ PostgREST Documentation
.. image:: https://img.shields.io/docker/pulls/postgrest/postgrest.svg
:target: https://hub.docker.com/r/postgrest/postgrest/
.. image:: https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg
:target: https://gitter.im/begriffs/postgrest
.. image:: https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854
:target: https://www.patreon.com/postgrest
@@ -38,30 +41,30 @@ Sponsors
.. image:: ../static/cybertec.svg
:target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
.. image:: ../static/gnuhost.png
:target: https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-dark
.. image:: ../static/neon-dark.jpg
:target: https://neon.com/?utm_source=sponsor&utm_campaign=postgrest
:target: https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-light
.. image:: ../static/neon.jpg
:target: https://neon.com/?utm_source=sponsor&utm_campaign=postgrest
.. image:: ../static/tembo.png
:target: https://www.tembo.io/?utm_source=sponsor&utm_campaign=postgrest
:target: https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest
|
.. container:: img-dark
.. image:: ../static/euronodes.svg
:target: https://www.euronodes.com/postgrest
.. image:: ../static/code-build-dark.png
:target: https://code.build/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-light
.. image:: ../static/euronodes.svg
:target: https://www.euronodes.com/postgrest
.. image:: ../static/code-build.png
:target: https://code.build/?utm_source=sponsor&utm_campaign=postgrest
.. container:: img-dark
@@ -73,11 +76,14 @@ Sponsors
.. image:: ../static/supabase.png
:target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
.. image:: ../static/tembo.png
:target: https://tembo.io/?utm_source=sponsor&utm_campaign=postgrest
.. The static/empty.png(created with `convert -size 320x95 xc:#fcfcfc empty.png`) is an ugly workaround
to create space and center the logos. It's not easy to layout with restructuredText.
.. image:: _static/empty.png
:target: #sponsors
.. .. image:: _static/empty.png
:target: #sponsors
|
@@ -104,7 +110,7 @@ PostgREST has a focused scope. It works well with other tools like Nginx. This f
Getting Support
----------------
The project has a friendly and growing community. For discussions, use the Github `discussions page <https://github.com/PostgREST/postgrest/discussions>`_. You can also report or search for bugs/features on the Github `issues <https://github.com/PostgREST/postgrest/issues>`_ page.
The project has a friendly and growing community. For discussions, use the Github `discussions page <https://github.com/PostgREST/postgrest/discussions>`_ or join our `chat room <https://gitter.im/begriffs/postgrest>`_. You can also report or search for bugs/features on the Github `issues <https://github.com/PostgREST/postgrest/issues>`_ page.
Release Notes
-------------
@@ -209,12 +215,19 @@ Here are some companies that use PostgREST in production.
* `Datrium <https://www.datrium.com>`_
* `Drip Depot <https://www.dripdepot.com>`_
* `Image-charts <https://www.image-charts.com>`_
* `Moat <https://www.oracle.com/advertising/>`_
* `Netwo <https://www.netwo.io>`_
* `Nimbus <https://www.nimbusfacility.com/sg/home>`_
- See how Nimbus uses PostgREST in `Paul Copplestone's blog post <https://paul.copplest.one/blog/nimbus-tech-2019-04.html>`_.
* `OpenBooking <https://openbooking.ch>`_
* `Redsmin <https://www.redsmin.com>`_
* `Sompani <https://www.sompani.com>`_
* `Supabase <https://supabase.com>`_
.. Failing links
* `eGull <http://www.egull.co>`_
* `MotionDynamic - Fast highly dynamic video generation at scale <https://motiondynamic.tech>`_
Testimonials
------------
+6
View File
@@ -0,0 +1,6 @@
Greenplum
#########
`Greenplum <https://greenplum.org/>`_ has been reported to work by adding ``LOGIN`` to the :ref:`anonymous and user roles <roles>`.
For more details, see https://github.com/PostgREST/postgrest/issues/2021.
@@ -1,6 +1,6 @@
.. _external_auth:
.. _external_jwt:
External Authentication
External JWT Generation
-----------------------
JWT from Auth0
+5 -9
View File
@@ -3,7 +3,6 @@ api
API's
APIs
APISIX
AST
aud
Auth
auth
@@ -13,10 +12,10 @@ booleans
BOM
Bytea
Cardano
casted
cd
centric
CLI
CMS
coercible
conf
Cloudflare
@@ -28,12 +27,10 @@ CSV
durations
DDL
DOM
DSL
DevOps
dockerize
enum
Enums
Entra
eq
ETH
Ethereum
@@ -46,6 +43,7 @@ Github
Google
grantor
GraphQL
Greenplum
gte
GUC
Haskell
@@ -70,11 +68,9 @@ isdistinct
JS
js
JSON
JSPath
JWK
JWT
jwt
Keycloak
Kubernetes
localhost
login
@@ -98,11 +94,10 @@ npm
nxl
nxr
OAuth
ORM
Observability
Okta
OpenAPI
openapi
ORM
ov
parametrized
passphrase
@@ -151,6 +146,8 @@ SHA
signup
SIGUSR
sl
spreaded
Spreaded
SQL
sql
SQLSTATE
@@ -174,7 +171,6 @@ unikernel
unix
updatable
unfulfillable
unselected
Untyped
UPSERT
Upsert
+18 -1
View File
@@ -12,7 +12,7 @@ Health Check
You can enable a health check to verify if PostgREST is available for client requests. Also to check the status of its internal state.
Two endpoints ``live`` and ``ready`` will then be available. Both these endpoints reply with a status code and empty response body.
Two endpoints ``live`` and ``ready`` will then be available.
.. important::
@@ -55,6 +55,23 @@ Metrics
Provides :ref:`metrics`.
Runtime Configuration
=====================
Provides a ``config`` endpoint that returns the runtime :ref:`configuration`.
.. code-block:: bash
curl "http://localhost:3001/config"
.. code-block::
db-aggregates-enabled = false
db-anon-role = "web_anon"
db-channel = "pgrst"
db-channel-enabled = false
...
Runtime Schema Cache
====================
+103 -72
View File
@@ -3,20 +3,20 @@
Aggregate Functions
###################
PostgREST supports the following aggregate functions: ``avg()``, ``count()``, ``max()``, ``min()``, and ``sum()``.
Please refer to the `section on aggregate functions in the PostgreSQL documentation <https://www.postgresql.org/docs/current/functions-aggregate.html>`_ for a detailed explanation of these functions.
Aggregate functions allow you to summarize data by performing calculations across groups of rows. For instance, if you have an ``orders`` table that has an ``amount`` column, you could use an aggregate function to get the sum of the ``amount`` column, either for all rows, or for each group of rows that share specific values, for instance all rows that share the same ``order_date``.
.. note::
Aggregate functions are *disabled* by default in PostgREST, because they can create performance problems without appropriate safeguards.
See :ref:`db-aggregates-enabled` for further details.
Aggregate functions are *disabled* by default in PostgREST, as without appropriate safeguards, aggregate functions can create performance problems. See :ref:`db-aggregates-enabled` for further details.
To use an aggregate function, append it to a column in the ``select`` parameter, like so:
PostgREST supports the following aggregate functions: ``avg()``, ``count()``, ``max()``, ``min()``, and ``sum()``. Please refer to the `section on aggregate functions in the PostgreSQL documentation <https://www.postgresql.org/docs/current/functions-aggregate.html>`_ for a detailed explanation of these functions.
To use an aggregate function, you append the function to a value in the ``select`` parameter, like so:
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum()"
This will return a ``sum`` of all the values of the ``amount`` column in a single row:
With the above query, PostgREST will return a single row with a single column named ``sum`` that contains the sum of all the values in the ``amount`` column:
.. code-block:: json
@@ -26,29 +26,15 @@ This will return a ``sum`` of all the values of the ``amount`` column in a singl
}
]
You can ``select`` multiple aggregate functions at the same time (you may need to :ref:`rename them <renaming_columns>` to disambiguate).
You can use multiple aggregate functions by just adding more columns with aggregate functions to the ``select`` parameter.
.. code-block:: bash
curl "http://localhost:3000/orders?select=total_amount:amount.sum(),avg_amount:amount.avg(),total_quantity:quantity.sum()"
.. note::
Aggregate functions work alongside other PostgREST features, like :ref:`h_filter`, :ref:`json_columns`, and :ref:`ordering`.
However they are not compatible with :ref:`domain_reps` for the moment.
Additionally, PostgreSQL's ``HAVING`` clause and ordering by aggregated columns are not yet supported.
Automatic ``GROUP BY``
======================
In SQL, a ``GROUP BY`` clause is required to aggregate the selected columns.
However, PostgREST handles grouping automatically if the columns are already present in the ``select`` parameter.
For instance:
To group by other columns, you simply add those columns to the ``select`` parameter. For instance:
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum(),amount.avg(),order_date"
This will get the sum and average of the amounts grouped by each unique value in the ``order_date`` column:
This will return a row for each unique value in the ``order_date`` column, with the sum and average of the ``amount`` column for all rows that share the same ``order_date``:
.. code-block:: json
@@ -65,55 +51,67 @@ This will get the sum and average of the amounts grouped by each unique value in
}
]
The ``count()`` Aggregate
=========================
.. note::
Aggregate functions work alongside other PostgREST features, like :ref:`h_filter`, :ref:`json_columns`, and :ref:`ordering`. Please note at this time aggregate functions are not compatible with :ref:`domain_reps`. Additionally, PostgreSQL's ``HAVING`` clause and ordering by aggregated columns are not yet supported.
The Case of ``count()``
===========================
.. note::
Before the addition of aggregate functions, it was possible to count by adding ``count`` (without parentheses) to the ``select`` parameter.
While this is still supported, it may be deprecated in the future, and thus use of this legacy feature is **not recommended**.
Please use ``count()`` (with parentheses) instead.
Before the addition of aggregate functions, it was possible to count by adding ``count`` (without parentheses) to the ``select`` parameter. While this is still supported, it may be deprecated in the future, and thus use of this legacy feature is **not recommended.** Please use ``count()`` (with parentheses) instead.
``count()`` is a special case because it can be used with or without an aggregated column. For example:
``count()`` is treated specially, as it can be used without an associated column. Take for example the following query:
.. code-block:: bash
curl "http://localhost:3000/orders?select=count(),observation_count:observation.count(),order_date"
curl "http://localhost:3000/orders?select=count(),order_date"
This would return a row for each unique value in the ``order_date`` column, with the count of all rows that share the same ``order_date``:
.. code-block:: json
[
{
"count": 4,
"observation_count": 2,
"order_date": "2023-01-01"
},
{
"count": 2,
"observation_count": 1,
"order_date": "2023-01-02"
}
]
Note that there is a difference between the result of ``count()`` and ``observation.count()``.
The former counts the whole row, while the latter counts the non ``NULL`` values of the ``observation`` column (both grouped by ``order_date``).
This is due to how PostgreSQL itself implements the ``count()`` function.
When ``count()`` is used with an associated column, its behavior is slightly different: It will return the count of all values that are not ``NULL``. This is due to how PostgreSQL itself implements the ``count()`` function.
Renaming and Casting
====================
Renaming Aggregates
-------------------
Just like with other columns, you can rename aggregated columns too. See :ref:`renaming_columns` for details.
Renaming columns is especially helpful in the context of aggregate functions, as by default a column with an aggregate function applied will take on the name of the applied aggregate function. You may want to provide a more semantically meaningful name or prevent collisions when using multiple aggregate functions of the same type.
Casting Aggregates
==================
------------------
It is :ref:`possible to cast <casting_columns>` the aggregated column or the aggregate itself, or both at the same time.
When applying an aggregate function to a column, you are able to cast both the value of the input to the aggregate function *and* the value of the output from the aggregate function. In both cases, the syntax works as described in :ref:`casting_columns`, with the only difference being the placement of the cast.
Casting the Aggregated Column
-----------------------------
Casting the Value of the Input
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For example, let's say that ``orders`` has an ``order_details`` :ref:`JSON column <json_columns>` with a ``tax_amount`` key.
We cannot sum ``tax_amount`` directly because using ``->`` or ``->>`` will return the data in ``json`` or ``text`` format.
So we need to cast it to a compatible type (e.g. ``numeric``) right before the aggregate function:
For instance, imagine that the ``orders`` table has a JSON column, ``order_details``, and this column contains a JSON object that has a key, ``tax_amount``. Let's say you want to get the sum of the tax amount for every order. You can use the ``->`` or ``->>`` operators to extract the value with this key (see :ref:`json_columns`), but these operators will return values of the types JSON and ``text`` respectively, and neither of these types can be used with ``sum()``.
Therefore, you will need to first cast the input value to a type that is compatible with ``sum()`` (e.g. ``numeric``). Casting the input value is done in exactly the same way as casting any other value:
.. code-block:: bash
curl "http://localhost:3000/orders?select=order_details->tax_amount::numeric.sum()"
With this, you will receive the sum of the casted ``tax_amount`` value:
.. code-block:: json
[
@@ -122,15 +120,17 @@ So we need to cast it to a compatible type (e.g. ``numeric``) right before the a
}
]
Casting the Aggregate
---------------------
Casting the Value of the Output
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For instance, if we wanted to round the average of the ``amount`` column, we could do so by casting ``avg()`` to an ``int``:
Now let's return to an example involving the ``amount`` column of the ``orders`` table. Imagine that we want to get the rounded average of the ``amount`` column. One way to do this is to use the ``avg()`` aggregate function and then to cast the output value of the function to ``int``. To cast the value of the output of the function, we simply place the cast *after* the aggregate function:
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.avg()::int"
You will then receive the rounded average as the result:
.. code-block:: json
[
@@ -139,22 +139,27 @@ For instance, if we wanted to round the average of the ``amount`` column, we cou
}
]
Aggregates and Resource Embedding
=================================
Of course, you can use both input and output casts at the same time, if you so desire.
You can group an aggregate function by an :ref:`embedded resource <resource_embedding>` and also use the aggregates inside them.
Grouping by an Embedded Resource
--------------------------------
Using Aggregate Functions with Resource Embedding
=================================================
Similar to grouping by columns, aggregate functions can also be grouped by embedded resources.
For example, let's say that the ``orders`` table is related to a ``customers`` table.
To get the sum of the ``amount`` column grouped by the ``name`` column from the ``customers`` table, we would do the following:
Aggregate functions can be used in conjunction with :ref:`resource_embedding`. You can use embedded resources as grouping columns, use aggregate functions within the context of an embedded resource, or use columns from a spreaded resource as grouping columns or as inputs to aggregate functions.
Using Embedded Resources as Grouping Columns
--------------------------------------------
Using an embedded resource as a grouping column allows you to use data from an association to group the results of an aggregation.
For example, imagine that the ``orders`` table from the examples above is related to a ``customers`` table. If you want to get the sum of the ``amount`` column grouped by the ``name`` column from the ``customers`` table, you can include the customer name, using the standard :ref:`resource_embedding` syntax, and perform a sum on the ``amount`` column.
.. code-block:: bash
curl "http://localhost:3000/orders?select=amount.sum(),customers(name)"
You will then get the summed amount, along with the embedded customer resource:
.. code-block:: json
[
@@ -172,16 +177,15 @@ To get the sum of the ``amount`` column grouped by the ``name`` column from the
}
]
The previous example uses a "to-one" relationship, but this can be done on "to-many" relationships as well (although there are few obvious use cases).
.. note::
The previous example uses a has-one association to demonstrate this functionality, but you may also use has-many associations as grouping columns, although there are few obvious use cases for this.
This also works in a similar way for :ref:`spread embedded resources <spread_embed>`.
For example, ``select=amount.sum(),...customers(name)`` would sum the ``amount`` grouped by the ``name`` column.
Using Aggregate Functions Within the Context of an Embedded Resource
--------------------------------------------------------------------
Using Aggregates Inside Embedded Resources
------------------------------------------
When embedding a resource, you can apply aggregate functions to columns from the associated resource to perform aggregations within the context of an embedded resource.
Using the relationship from the previous example, let's take all the ``customers`` and embed their ``orders``.
If we also want to get the total ``amount`` grouped by the ``order_date`` of the ``orders``, we would do the following:
Continuing with the example relationship between ``orders`` and ``customers`` from the previous section, imagine that you want to fetch the ``name``, ``city``, and ``state`` for each customer, along with the sum of amount of the customer's orders, grouped by the order date. This can be done in the following way:
.. code-block:: bash
@@ -222,20 +226,51 @@ If we also want to get the total ``amount`` grouped by the ``order_date`` of the
}
]
Note that the aggregate is done within the embedded resource ``orders``.
It is not affected by any of the columns from the top-level relationship ``customers``.
In this example, the ``amount`` column is summed and grouped by the ``order_date`` *within* the context of the embedded resource. That is, the ``name``, ``city``, and ``state`` from the ``customers`` table have no bearing on the aggregation performed in the context of the ``orders`` association; instead, each aggregation can be seen as being performed independently on just the orders belonging to a particular customer, using only the data from the embedded resource for both grouping and aggregation.
Aggregates in To-One Spreads
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Using Columns from a Spreaded Resource
--------------------------------------
All the aggregates inside a :ref:`one-to-one or many-to-one spread embedded resource <spread_to_one_embed>` will be hoisted to the top-level relationship.
In other words, it will behave as if the aggregate was done in the top-level relationship itself. For example:
When you :ref:`spread an embedded resource <spread_embed>`, the columns from the spreaded resource are treated as if they were columns of the top-level resource, both when using them as grouping columns and when applying aggregate functions to them.
Grouping with Columns from a Spreaded Resource
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For instance, assume you want to sum the ``amount`` column from the ``orders`` table, using the ``city`` and ``state`` columns from the ``customers`` table as grouping columns. To achieve this, you may select these two columns from the ``customers`` table and spread them; they will then be used as grouping columns:
.. code-block:: bash
curl "http://localhost:3000/orders?select=order_date,...customers(subscription_date.max(),subscription_date.min())
curl "http://localhost:3000/orders?select=amount.sum(),...customers(city,state)
This will take the ``max`` and ``min`` subscription date of every customer and group it by the ``order_date`` column:
The result will be the same as if ``city`` and ``state`` were columns from the ``orders`` table:
.. code-block:: json
[
{
"sum": 2000.29,
"city": "New York",
"state": "NY"
},
{
"sum": 9241.21,
"city": "Los Angeles",
"state": "CA"
}
]
Aggregate Functions with Columns from a Spreaded Resource
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now imagine that the ``customers`` table has a ``joined_date`` column that represents the date that the customer joined. You want to get both the most recent and the oldest ``joined_date`` for customers that placed an order on every distinct order date. This can be expressed as follows:
.. code-block:: bash
curl "http://localhost:3000/orders?select=order_date,...customers(joined_date.max(),joined_date.min())
As columns from a spreaded resource are treated as if they were columns from the top-level resource, the ``max()`` and ``min()`` are applied *within* the context of the top-level, rather than within the context of the embedded resource, as in the previous section.
The result will be the same as if the aggregations were applied to columns from the top-level:
.. code-block:: json
@@ -251,7 +286,3 @@ This will take the ``max`` and ``min`` subscription date of every customer and g
"min": "2016-02-11"
}
]
.. note::
Aggregates inside to-many spreads are not supported
+4 -51
View File
@@ -71,40 +71,6 @@ The function parameter names match the JSON object keys in the POST case, for th
.. _function_single_json:
Functions with an array of JSON objects
----------------------------------------------
If you want to pass multiple JSON objects to a Postgres function (an array of objects), you can create a function with a parameter of type ``json`` or ``jsonb``.
Within the curl request, this JSON must be embedded in an object where they key matches the same name as the function's ``json`` or ``jsonb`` parameter.
This will allow you to loop over the array of JSON objects within the Postgres function.
This practice may allow you to reduce the number of ``curl`` requests required to accomplish a task.
For instance, assume we have created this function in the database.
.. code-block:: postgres
CREATE FUNCTION update_data(p_json jsonb)
RETURNS void AS $$
DECLARE
json_item json;
BEGIN
FOR json_item IN SELECT jsonb_array_elements(p_json) LOOP
UPDATE data_table SET data_text_column = (json_item->>'data_text')::text
WHERE data_int_column = (json_item->>'data_int')::integer;
END LOOP;
END;
$$ LANGUAGE SQL IMMUTABLE;
A ``curl`` request using the POST method would look like the following:
.. code-block:: bash
curl "http://localhost:3000/rpc/update_data" \
-X POST -H "Content-Type: application/json" \
-d '{ "p_json": [ { "data_text": "one", "data_int": "1" }, { "data_text": "two", "data_int": "2" } ] }'
Functions with a single unnamed JSON parameter
----------------------------------------------
@@ -131,6 +97,10 @@ For this the ``Content-Type: application/json`` header must be included in the r
If an overloaded function has a single ``json`` or ``jsonb`` unnamed parameter, PostgREST will call this function as a fallback provided that no other overloaded function is found with the parameters sent in the POST request.
.. warning::
Sending the JSON request body as a single argument is also possible with :ref:`Prefer: params=single-object <prefer_params>` but this method is **deprecated**.
.. _function_single_unnamed:
Functions with a single unnamed parameter
@@ -294,23 +264,6 @@ Let's get its :ref:`explain_plan` when calling it with filters applied:
Notice there's no "Function Scan" node in the plan, which tells us it has been inlined.
Horizontal Filtering
~~~~~~~~~~~~~~~~~~~~
Table-valued functions support horizontal filtering on selected and unselected columns.
For example, the following RPC with filter on unselected column returns:
.. code-block:: bash
curl "http://localhost:3000/rpc/getallprojects?select=id,client_id&name=like.OSX"
.. code-block:: json
[
{ "id": 4, "client_id": 2 }
]
.. _scalar_functions:
Scalar functions
-3
View File
@@ -19,9 +19,6 @@ For extra customization, the OpenAPI output contains a "description" field for e
COMMENT ON TABLE monotremes IS
'Freakish mammals lay the best eggs for breakfast';
COMMENT ON VIEW monotremes_v IS
'Only the platypus is publicly visible';
COMMENT ON COLUMN monotremes.has_venomous_claw IS
'Sometimes breakfast is not worth it';
+20 -64
View File
@@ -12,9 +12,10 @@ The following preferences are supported.
- ``Prefer: return``. See :ref:`prefer_return`.
- ``Prefer: count``. See :ref:`prefer_count`.
- ``Prefer: resolution``. See :ref:`prefer_resolution`.
- ``Prefer: missing``. See :ref:`prefer_missing`.
- ``Prefer: missing``. See :ref:`bulk_insert_default`.
- ``Prefer: max-affected``, See :ref:`prefer_max_affected`.
- ``Prefer: tx``. See :ref:`prefer_tx`.
- ``Prefer: params``. See :ref:`prefer_params`.
.. _prefer_handling:
@@ -196,53 +197,12 @@ The ``tx`` preference can be set to specify if the :ref:`transaction <transactio
{"id": 35, "name": "Project X"}
.. _prefer_missing:
Missing
=======
When doing ``POST`` and ``PATCH`` requests, any missing columns in the payload will be inserted as ``null`` value by default. To use the ``DEFAULT`` column value instead, use the ``Prefer: missing=default`` header.
Having:
.. code-block:: postgres
create table foo (
id bigint generated by default as identity primary key
, bar text
, baz int default 100
);
A request:
.. code-block:: bash
curl "http://localhost:3000/foo?columns=id,bar,baz" \
-H "Content-Type: application/json" \
-H "Prefer: missing=default, return=representation" \
-d @- << EOF
[
{ "bar": "val1" },
{ "bar": "val2", "baz": 15 }
]
EOF
Will result in:
.. code-block:: json
[
{ "id": 1, "bar": "val1", "baz": 100 },
{ "id": 2, "bar": "val2", "baz": 15 }
]
.. _prefer_max_affected:
Max Affected
============
You can set a limit to the amount of resources affected in a request by sending ``max-affected`` preference. This feature works in combination with ``handling=strict`` preference. ``max-affected`` would be ignored with lenient handling. The "affected resources" are the number of rows returned by ``DELETE`` and ``PATCH`` requests.
You can set a limit to the amount of resources affected in a request by sending ``max-affected`` preference. This feature works in combination with ``handling=strict`` preference. ``max-affected`` would be ignored with lenient handling. The "affected resources" are the number of rows returned by ``DELETE`` and ``PATCH`` requests. This is also supported through ``RPC`` calls.
To illustrate the use of this preference, consider the following scenario where the ``items`` table contains 14 rows.
@@ -265,34 +225,30 @@ To illustrate the use of this preference, consider the following scenario where
"hint": null
}
With :ref:`RPC <functions>`, the preference is honored completely on the basis of the number of rows returned in the result set of the function. This can be useful for complex mutation queries using `data-modifying statements <https://www.postgresql.org/docs/current/queries-with.html#QUERIES-WITH-MODIFYING>`_. A simple example:
.. _prefer_params:
Single JSON object as Function Parameter
----------------------------------------
.. warning::
Using this preference is **deprecated** in favor of :ref:`function_single_json`.
:code:`Prefer: params=single-object` allows sending the JSON request body as the single argument of a :ref:`function <functions>`.
.. code-block:: postgres
CREATE FUNCTION test.delete_items()
RETURNS SETOF items AS $$
DELETE FROM items WHERE id < 15 RETURNING *;
CREATE FUNCTION mult_them(param json) RETURNS int AS $$
SELECT (param->>'x')::int * (param->>'y')::int
$$ LANGUAGE SQL;
.. code-block:: bash
curl -i "http://localhost:3000/rpc/delete_items" \
-H "Content-Type: application/json" \
-H "Prefer: handling=strict, max-affected=10"
.. code-block:: http
HTTP/1.1 400 Bad Request
curl "http://localhost:3000/rpc/mult_them" \
-X POST -H "Content-Type: application/json" \
-H "Prefer: params=single-object" \
-d '{ "x": 4, "y": 2 }'
.. code-block:: json
{
"code": "PGRST124",
"message": "Query result exceeds max-affected preference constraint",
"details": "The query affects 14 rows",
"hint": null
}
.. note::
It is important for functions to return ``SETOF`` or ``TABLE`` when called with ``max-affected`` preference. A violation of this would cause a :ref:`PGRST128 <pgrst128>` error.
8
+17 -163
View File
@@ -209,25 +209,18 @@ The join table is also detected if the composite key has additional columns.
One-to-one relationships
------------------------
One-to-one relationships are detected in two ways. (We'll use the ``films`` and ``technical_specs`` tables from the :ref:`sample film database <erd_film>` as an example).
One-to-one relationships are detected in two ways.
- When the foreign key is also a primary key.
- When the foreign key is a primary key as specified in the :ref:`sample film database <erd_film>`.
- When the foreign key has a unique constraint.
.. code-block:: postgres
create table technical_specs(
film_id int references films(id) primary key
-- ...
);
- Or when the foreign key has a unique constraint.
.. code-block:: postgres
create table technical_specs(
id int primary key
, film_id int references films(id) unique
-- ...
film_id int references films(id) unique,
runtime time,
camera text,
sound text
);
.. code-block:: bash
@@ -934,12 +927,7 @@ Filters can also be applied on nested embedded resources:
.. code-block:: bash
# curl "http://localhost:3000/films?select=*,roles(*,actors(*))&roles.actors.order=last_name&roles.actors.first_name=like.*Tom*"
curl --get "http://localhost:3000/films" \
-d "select=*,roles(*,actors(*))" \
-d "roles.actors.order=last_name" \
-d "roles.actors.first_name=like.*Tom*"
curl "http://localhost:3000/films?select=*,roles(*,actors(*))&roles.actors.order=last_name&roles.actors.first_name=like.*Tom*"
The result will show the nested actors named Tom and order them by last name. Aliases can also be used instead of the resource names to filter the nested tables.
@@ -1150,19 +1138,14 @@ For example, to arrange the films in descending order using the director's last
Spread embedded resource
========================
You can modify the shape of the embedded resources by using the spread syntax (``...``).
.. _spread_to_one_embed:
Spread To-One relationships
---------------------------
Spread on resources forming :ref:`one-to-one <one-to-one>` and :ref:`many-to-one <many-to-one>` relationships, will lift the embedded columns to the top object.
On many-to-one and one-to-one relationships, you can "spread" the embedded resource. That is, remove the surrounding JSON object for the embedded resource columns.
.. code-block:: bash
# curl "http://localhost:3000/films?select=title,...directors(director_last_name:last_name)&title=like.*Workers*"
curl --get "http://localhost:3000/films" \
-d "select=title,...directors(director_first_name:first_name, director_last_name:last_name)" \
-d "select=title,...directors(director_last_name:last_name)" \
-d "title=like.*Workers*"
.. code-block:: json
@@ -1170,148 +1153,17 @@ Spread on resources forming :ref:`one-to-one <one-to-one>` and :ref:`many-to-one
[
{
"title": "Workers Leaving The Lumière Factory In Lyon",
"director_first_name": "Louis",
"director_last_name": "Lumière"
}
]
Note that there is no wrapping ``"directors"`` object, unlike regularly embedding :ref:`many-to-one <many-to-one>` relationships. Also note that embedded columns can be aliased normally.
Note that there is no ``"directors"`` object. Also the embed columns can be aliased normally.
.. _spread_to_many_embed:
Spread To-Many relationships
----------------------------
Spread on resources forming :ref:`one-to-many <one-to-many>` and :ref:`many-to-many <many-to-many>` relationships, will convert the embedded columns into correlated arrays.
You can use this to get the columns of a join table in a many-to-many relationship. For instance, to get films and its actors, but including the ``character`` column from the roles table:
.. code-block:: bash
curl --get "http://localhost:3000/directors" \
-d "select=first_name,...films(film_titles:title,film_years:year)" \
-d "first_name=like.Quentin*"
.. code-block:: json
[
{
"first_name": "Quentin",
"film_titles": [
"Pulp Fiction",
"Reservoir Dogs"
],
"film_years": [
1994,
1992
]
}
]
Note that ``films`` is no longer an array of objects, unlike regularly embedding :ref:`one-to-many`. The embedded columns become arrays and they're correlated—in the above result, we can say that "Pulp Fiction" premiered in 1994 and "Reservoir Dogs" in 1992.
Order in spread to-many
~~~~~~~~~~~~~~~~~~~~~~~
In the above example, the order of the values inside the correlated arrays is unspecified, but all the values are guaranteed to be in the same unspecified order.
You can order the correlated arrays explicitly. For example, to order by the film year:
.. code-block:: bash
curl --get "http://localhost:3000/directors" \
-d "select=first_name,...films(film_titles:title,film_years:year)" \
-d "first_name=like.Quentin*" \
-d "films.order=year"
.. code-block:: json
[
{
"first_name": "Quentin",
"film_titles": [
"Reservoir Dogs",
"Pulp Fiction"
],
"film_years": [
1992,
1994
]
}
]
.. warning::
Aliasing spreaded columns is recommended since JSON allows duplicate keys. Example:
.. code-block:: bash
curl --get "localhost:3000/projects" \
-d "select=id,name,...clients(id,name)"
.. code-block:: json
[{"id":1,"name":"Windows 7","id":1,"name":"Microsoft"},
{"id":2,"name":"Windows 10","id":1,"name":"Microsoft"},
{"id":3,"name":"IOS","id":2,"name":"Apple"},
{"id":4,"name":"OSX","id":2,"name":"Apple"},
{"id":5,"name":"Orphan","id":null,"name":null}]
This can be a problem in Javascript objects, since only the last duplicated key will be considered. To solve it do:
.. code-block:: bash
curl --get "localhost:3000/projects" \
-d "select=id,name,...clients(client_id:id,client_name:name)"
Multiple Spreads
----------------
You can use multiple spreads at any level. For example, let's spread ``technical_specs`` and ``roles`` into ``films`` and then spread ``films`` into ``directors``:
.. code-block:: bash
curl --get "http://localhost:3000/directors" \
-d "select=first_name,...films(film_titles:title,film_years:year,...technical_specs(film_runtimes:runtime),...roles(film_characters:character))" \
-d "first_name=like.Quentin*" \
-d "films.order=year" \
-d "films.roles.order=character"
.. code-block:: json
[
{
"first_name": "Quentin",
"film_titles": [
"Reservoir Dogs",
"Pulp Fiction"
],
"film_years": [
1992,
1994
],
"film_runtimes": [
"01:39:00",
"02:29:00"
]
"film_characters": [
[ "Mr. Pink", "Mr. White" ],
[ "Mia Wallace", "Vincent Vega" ]
]
}
]
Note that:
- All the ``film_*`` arrays are correlated—"Reservoir Dogs" premiered in 1992, its runtime is 1:39:00 and it has the following characters: ``[ "Mr. Pink", "Mr. White" ]``.
- The ``film_*`` arrays are ordered by ``year`` (due to ``films.order=year``).
- The bottom level array ``film_characters`` is ordered (due to ``films.roles.order=character``).
Spread a join table
-------------------
Spread can be used to move the columns of a join table in a :ref:`many-to-many <many-to-many>` to the top object. For instance, to get the ``character`` column of the ``roles`` join table into ``actors``:
.. code-block:: bash
# curl "http://localhost:3000/films?select=title,actors:roles(character,...actors(first_name,last_name))&title=like.*Lighthouse*"
curl --get "http://localhost:3000/films" \
-d "select=title,actors:roles(character,...actors(first_name,last_name))" \
@@ -1332,4 +1184,6 @@ Spread can be used to move the columns of a join table in a :ref:`many-to-many <
}
]
.. note::
The spread operator ``...`` is borrowed from the Javascript `spread syntax <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax>`_.
@@ -16,32 +16,6 @@ Use the Accept request header to specify the acceptable format (or formats) for
curl "http://localhost:3000/people" \
-H "Accept: application/json"
.. note::
The ordering of columns in the response isn't guaranteed to align with the order specified in the ``select`` clause. For example, with resource embedding:
.. code-block:: bash
http://localhost:3000/films?select=directors(last_name,id),title
We may get:
.. code-block:: bash
[
{
"title": "title",
"directors": {
"id": 5,
"last_name": "name"
}
}
]
This is in line with the `JSON schema spec <https://json-schema.org/draft/2020-12/json-schema-core#name-instance-data-model>`_:
*"object: An unordered set of properties mapping a string to an instance"*
.. _builtin_media:
Builtin Media Type Handlers
@@ -100,15 +74,17 @@ This returns
{ "id": 1 }
with a :code:`Content-Type: application/vnd.pgrst.object+json`.
When a singular response is requested but no entries are found, the server responds with an error message and 406 Not Acceptable status code rather than the usual empty array and 200 status:
.. code-block:: json
{
"code": "PGRST116",
"message": "Cannot coerce the result to a single JSON object",
"details": "The result contains 0 rows",
"hint": null
"message": "JSON object requested, multiple (or no) rows returned",
"details": "Results contain 0 rows, application/vnd.pgrst.object+json requires 1 row",
"hint": null,
"code": "PGRST505"
}
.. note::
+77 -58
View File
@@ -72,7 +72,7 @@ imatch :code:`~*` ~* operator, see :ref:`pattern_matching`
in :code:`IN` one of a list of values, e.g. :code:`?a=in.(1,2,3)`
also supports commas in quoted strings like
:code:`?a=in.("hi,there","yes,you")`
is :code:`IS` checking for exact equality (null,not_null,true,false,unknown)
is :code:`IS` checking for exact equality (null,true,false,unknown)
isdistinct :code:`IS DISTINCT FROM` not equal, treating :code:`NULL` as a comparable value
fts :code:`@@` :ref:`fts` using to_tsquery
plfts :code:`@@` :ref:`fts` using plainto_tsquery
@@ -175,47 +175,25 @@ To ensure best performance on larger data sets, an `appropriate index <https://w
Full-Text Search
~~~~~~~~~~~~~~~~
The :code:`fts` operator has a number of options to support flexible textual queries, namely the choice of plain vs phrase search and the language used for stemming.
The following examples illustrate the possibilities, assuming column :code:`my_tsv` is of type `tsvector <https://www.postgresql.org/docs/current/datatype-textsearch.html>`_.
The :code:`fts` filter mentioned above has a number of options to support flexible textual queries, namely the choice of plain vs phrase search and the language used for stemming. Suppose that :code:`tsearch` is a table with column :code:`my_tsv`, of type `tsvector <https://www.postgresql.org/docs/current/datatype-textsearch.html>`_. The following examples illustrate the possibilities.
.. code-block:: bash
curl --get "http://localhost:3000/people" \
-d "my_tsv=fts(french).amusant"
curl "http://localhost:3000/tsearch?my_tsv=fts(french).amusant"
.. code-block:: bash
curl --get "http://localhost:3000/people" \
-d "my_tsv=plfts.The%20Fat%20Cats"
curl "http://localhost:3000/tsearch?my_tsv=plfts.The%20Fat%20Cats"
.. code-block:: bash
curl --get "http://localhost:3000/people" \
-d "my_tsv=not.phfts(english).The%20Fat%20Cats"
curl "http://localhost:3000/tsearch?my_tsv=not.phfts(english).The%20Fat%20Cats"
.. code-block:: bash
curl --get "http://localhost:3000/people" \
-d "my_tsv=not.wfts(french).amusant"
curl "http://localhost:3000/tsearch?my_tsv=not.wfts(french).amusant"
.. _fts_to_tsvector:
Automatic ``tsvector`` conversion
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If the filtered column is not of type ``tsvector``, then it will be automatically converted using `to_tsvector() <https://www.postgresql.org/docs/current/functions-textsearch.html#TEXTSEARCH-FUNCTIONS-TABLE>`_.
This allows using the ``fts`` operator on ``text`` and ``json`` types out of the box.
.. code-block:: bash
curl --get "http://localhost:3000/people" \
-d "my_text_column=fts(french).amusant"
.. code-block:: bash
curl --get "http://localhost:3000/people" \
-d "my_json_column=not.phfts(english).The%20Fat%20Cats"
Using `websearch_to_tsquery` requires PostgreSQL of version at least 11.0 and will raise an error in earlier versions of the database.
.. _v_filter:
@@ -547,6 +525,45 @@ To bulk insert JSON post an array of objects having all-matching keys
]
EOF
.. _bulk_insert_default:
Bulk Insert with Default Values
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Any missing columns in the payload will be inserted as ``null`` values. To use the ``DEFAULT`` column value instead, use the ``Prefer: missing=default`` header.
Having:
.. code-block:: postgres
create table foo (
id bigint generated by default as identity primary key
, bar text
, baz int default 100
);
A request:
.. code-block:: bash
curl "http://localhost:3000/foo?columns=id,bar,baz" \
-H "Content-Type: application/json" \
-H "Prefer: missing=default, return=representation" \
-d @- << EOF
[
{ "bar": "val1" },
{ "bar": "val2", "baz": 15 }
]
EOF
Will result in:
.. code-block:: json
[
{ "id": 1, "bar": "val1", "baz": 100 },
{ "id": 2, "bar": "val2", "baz": 15 }
]
.. _specify_columns:
@@ -589,13 +606,7 @@ To update a row or rows in a table, use the PATCH verb. Use :ref:`h_filter` to s
-X PATCH -H "Content-Type: application/json" \
-d '{ "category": "child" }'
Updates also support:
- :ref:`prefer_return`
- :ref:`resource_embedding`
- :ref:`v_filter`
- :ref:`Missing Preference <prefer_missing>`
- :ref:`specify_columns`
Updates also support :ref:`prefer_return`, :ref:`resource_embedding` and :ref:`v_filter`.
.. warning::
@@ -612,34 +623,18 @@ You can make an upsert with :code:`POST` and the :code:`Prefer: resolution=merge
.. code-block:: bash
curl "http://localhost:3000/products" \
curl "http://localhost:3000/employees" \
-X POST -H "Content-Type: application/json" \
-H "Prefer: resolution=merge-duplicates" \
-d @- << EOF
[
{ "sku": "CL2031", "name": "Existing T-shirt", "price": 35 },
{ "sku": "CL2040", "name": "Existing Hoodie", "price": 60 },
{ "sku": "AC1022", "name": "New Cap", "price": 30 }
{ "id": 1, "name": "Old employee 1", "salary": 30000 },
{ "id": 2, "name": "Old employee 2", "salary": 42000 },
{ "id": 3, "name": "New employee 3", "salary": 50000 }
]
EOF
By default, upsert operates based on the primary key columns, so you must specify all of them.
You can also choose to ignore the duplicates with :code:`Prefer: resolution=ignore-duplicates`.
Upsert works best when the primary key is natural (e.g. ``sku``).
However, it can work with surrogate primary keys (e.g. ``id serial primary key``), if you also do a :ref:`bulk_insert` with :ref:`prefer_missing`:
.. code-block:: bash
curl "http://localhost:3000/employees?colums=id,name,salary" \
-X POST -H "Content-Type: application/json" \
-H "Prefer: resolution=merge-duplicates, missing=default" \
-d @- << EOF
[
{ "id": 1, "name": "Existing employee 1", "salary": 30000 },
{ "id": 2, "name": "Existing employee 2", "salary": 42000 },
{ "name": "New employee 3", "salary": 50000 }
]
EOF
By default, upsert operates based on the primary key columns, you must specify all of them. You can also choose to ignore the duplicates with :code:`Prefer: resolution=ignore-duplicates`. This works best when the primary key is natural, but it's also possible to use it if the primary key is surrogate (example: "id serial primary key"). For more details read `this issue <https://github.com/PostgREST/postgrest/issues/1118>`_.
.. important::
After creating a table or changing its primary key, you must refresh PostgREST schema cache for upsert to work properly. To learn how to refresh the cache see :ref:`schema_reloading`.
@@ -705,6 +700,31 @@ Deletions also support :ref:`prefer_return`, :ref:`resource_embedding` and :ref:
Beware of accidentally deleting all rows in a table. To learn to prevent that see :ref:`block_fulltable`.
.. _limited_update_delete:
Limited Update/Delete
=====================
You can limit the amount of affected rows by :ref:`update` or :ref:`delete` with the ``limit`` query parameter. For this, you must add an explicit ``order`` on a unique column(s).
.. code-block:: bash
curl -X PATCH "/users?limit=10&order=id&last_login=lt.2020-01-01" \
-H "Content-Type: application/json" \
-d '{ "status": "inactive" }'
.. code-block:: bash
curl -X DELETE "http://localhost:3000/users?limit=10&order=id&status=eq.inactive"
If your table has no unique columns, you can use the `ctid <https://www.postgresql.org/docs/current/ddl-system-columns.html>`_ system column.
Using ``offset`` to target a different subset of rows is also possible.
.. note::
There is no native ``UPDATE...LIMIT`` or ``DELETE...LIMIT`` support in PostgreSQL; the generated query simulates that behavior and is based on `this Crunchy Data blog post <https://www.crunchydata.com/blog/simulating-update-or-delete-with-limit-in-postgres-ctes-to-the-rescue>`_.
.. raw:: html
<script type="text/javascript">
@@ -719,7 +739,6 @@ Deletions also support :ref:`prefer_return`, :ref:`resource_embedding` and :ref:
'#estimated-count': 'pagination_count.html#estimated-count',
'#prefer-return-headers-only': 'preferences.html#headers-only',
'#prefer-return-representation': 'preferences.html#full',
'#bulk-insert-default': 'preferences.html#prefer-missing',
};
let willRedirectTo = redirects[hash];
+2 -82
View File
@@ -77,7 +77,7 @@ If the client included no JWT (or one without a role claim) then PostgREST switc
JWT Generation
~~~~~~~~~~~~~~
You can create a valid JWT either from inside your database (see :ref:`sql_user_management`) or via an external service (see :ref:`external_auth`).
You can create a valid JWT either from inside your database (see :ref:`sql_user_management`) or via an external service (see :ref:`external_jwt`).
.. _client_auth:
@@ -156,90 +156,10 @@ You can specify the literal value as we saw earlier, or reference a filename to
jwt-secret = "@rsa.jwk.pub"
JWK ``kid`` validation
^^^^^^^^^^^^^^^^^^^^^^
PostgREST has built-in validation of the `key ID parameter <https://www.rfc-editor.org/rfc/rfc7517#section-4.5>`_, useful when working with a JWK Set.
It goes as follows:
- If the JWT contains a ``kid`` parameter, then PostgREST will look for the JWK in the :ref:`jwt-secret`.
+ If no JWK matches the same ``kid`` value (or if they do not have a ``kid``), then the token will be rejected with a :ref:`401 Unauthorized <pgrst301>` error.
+ If a JWK matches the ``kid`` value then it will validate the token against that JWK accordingly.
- If the JWT does not have a ``kid`` parameter, then PostgREST will validate the token against each JWK in the :ref:`jwt-secret`.
.. _jwt_claims_validation:
JWT Claims Validation
~~~~~~~~~~~~~~~~~~~~~
PostgREST honors the following `JWT claims <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.4>`_:
- ``exp`` Expiration Time
- ``iat`` Issued At
- ``nbf`` Not Before
- ``aud`` :ref:`Audience <jwt_aud_validation>`
.. note::
PostgREST allows for a 30-second clock skew when validating the ``exp``, ``iat`` and ``nbf`` claims.
In other words, it gives an extra 30 seconds before the token is rejected if there is a slight discrepancy in the timestamps.
.. _jwt_aud_validation:
JWT ``aud`` Claim Validation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PostgREST has built-in validation of the `JWT audience claim <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.3>`_.
It works this way:
- If :ref:`jwt-aud` is not set (the default), PostgREST identifies with all audiences and allows the JWT for any ``aud`` claim.
- If :ref:`jwt-aud` is set to a specific audience, PostgREST will check if this audience is present in the ``aud`` claim:
+ If the ``aud`` value is a JSON string, it will match it to the :ref:`jwt-aud`.
+ If the ``aud`` value is a JSON array of strings, it will search every element for a match.
+ If the match fails or if the ``aud`` value is not a string or array of strings, then the token will be rejected with a :ref:`401 Unauthorized <pgrst303>` error.
+ If the ``aud`` key **is not present** or if its value is ``null`` or ``[]``, PostgREST will interpret this token as allowed for all audiences and will complete the request.
.. _jwt_role_extract:
JWT Role Extraction
~~~~~~~~~~~~~~~~~~~
A JSPath DSL that specifies the location of the :code:`role` key in the JWT claims. It's configured by :ref:`jwt-role-claim-key`. This can be used to consume a JWT provided by a third party service like Auth0, Okta, Microsoft Entra or Keycloak.
The DSL follows the `JSONPath <https://goessner.net/articles/JsonPath/>`_ expression grammar with extended string comparison operators. Supported operators are:
- ``==`` selects the first array element that exactly matches the right operand
- ``!=`` selects the first array element that does not match the right operand
- ``^==`` selects the first array element that starts with the right operand
- ``==^`` selects the first array element that ends with the right operand
- ``*==`` selects the first array element that contains the right operand
Usage examples:
.. code:: bash
# {"postgrest":{"roles": ["other", "author"]}}
# the DSL accepts characters that are alphanumerical or one of "_$@" as keys
jwt-role-claim-key = ".postgrest.roles[1]"
# {"https://www.example.com/role": { "key": "author" }}
# non-alphanumerical characters can go inside quotes(escaped in the config value)
jwt-role-claim-key = ".\"https://www.example.com/role\".key"
# {"postgrest":{"roles": ["other", "author"]}}
# `@` represents the current element in the array
# all the these match the string "author"
jwt-role-claim-key = ".postgrest.roles[?(@ == \"author\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ != \"other\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ ^== \"aut\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ ==^ \"hor\")]"
jwt-role-claim-key = ".postgrest.roles[?(@ *== \"utho\")]"
.. note::
The string comparison operators are implemented as a custom extension to the JSPath and does not strictly follow the `RFC 9535 <https://www.rfc-editor.org/rfc/rfc9535.html>`_.
PostgREST honors the :code:`exp` claim for token expiration, rejecting expired tokens.
JWT Security
~~~~~~~~~~~~
+18 -57
View File
@@ -146,21 +146,6 @@ To reload the configuration from within the database, you can use the ``NOTIFY``
List of parameters
==================
.. _admin-server-host:
admin-server-host
-----------------
=============== =======================
**Type** String
**Default** `server-host` value
**Reloadable** N
**Environment** PGRST_ADMIN_SERVER_HOST
**In-Database** `n/a`
=============== =======================
Specifies the host for the :ref:`admin_server`. Defaults to :ref:`server-host` value.
.. _admin-server-port:
admin-server-port
@@ -174,7 +159,7 @@ admin-server-port
**In-Database** `n/a`
=============== =======================
Specifies the port for the :ref:`admin_server`. Cannot be equal to :ref:`server-port`.
Specifies the port for the :ref:`admin_server`.
.. _app.settings.*:
@@ -191,10 +176,6 @@ app.settings.*
Arbitrary settings that can be used to pass in secret keys directly as strings, or via OS environment variables. For instance: :code:`app.settings.jwt_secret = "$(MYAPP_JWT_SECRET)"` will take :code:`MYAPP_JWT_SECRET` from the environment and make it available to PostgreSQL functions as :code:`current_setting('app.settings.jwt_secret')`.
When using the environment variable `PGRST_APP_SETTINGS_*` form, the remainder of the variable is used as the new name. Case is not important : :code:`PGRST_APP_SETTINGS_MY_ENV_VARIABLE=some_value` can be accessed in postgres as :code:`current_setting('app.settings.my_env_variable')`.
The :code:`current_setting` function has `an optional boolean second <https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-SET>`_ argument to avoid it from raising an error if the value was not defined. Default values to :code:`app.settings` can then be given by combining this argument with :code:`coalesce` and :code:`nullif` : :code:`coalesce(nullif(current_setting('app.settings.my_custom_variable', true), ''), 'default value')`. The use of :code:`nullif` is necessary because if set in a transaction, the setting is sometimes not "rolled back" to :code:`null`. See also :ref:`this section <guc_req_headers_cookies_claims>` for more information on this behaviour.
.. _db-aggregates-enabled:
db-aggregates-enabled
@@ -315,10 +296,6 @@ db-extra-search-path
Multiple schemas can be added in a comma-separated string, e.g. ``public, extensions``.
.. important::
We default this config to ``public`` because it is the most common schema used to install PostgreSQL extensions such as :ref:`PostGIS <ww_postgis>`. You can disable this by setting this config to ``""``.
.. _db-hoisted-tx-settings:
db-hoisted-tx-settings
@@ -603,7 +580,11 @@ jwt-aud
**In-Database** pgrst.jwt_aud
=============== =================================
Specifies an audience for the JWT ``aud`` claim. See :ref:`jwt_aud_validation`.
Specifies the `JWT audience claim <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.3>`_. If this claim is present in the client provided JWT then you must set this to the same value as in the JWT, otherwise verifying the JWT will fail.
.. warning::
Using this setting will only reject tokens with a different audience claim. Tokens **without** audience claim will still be accepted.
.. _jwt-role-claim-key:
@@ -620,7 +601,17 @@ jwt-role-claim-key
*For backwards compatibility, this config parameter is also available without prefix as "role-claim-key".*
See :ref:`jwt_role_extract` on how to specify key paths and usage examples.
A JSPath DSL that specifies the location of the :code:`role` key in the JWT claims. This can be used to consume a JWT provided by a third party service like Auth0, Okta or Keycloak. Usage examples:
.. code:: bash
# {"postgrest":{"roles": ["other", "author"]}}
# the DSL accepts characters that are alphanumerical or one of "_$@" as keys
jwt-role-claim-key = ".postgrest.roles[1]"
# {"https://www.example.com/role": { "key": "author }}
# non-alphanumerical characters can go inside quotes(escaped in the config value)
jwt-role-claim-key = ".\"https://www.example.com/role\".key"
.. _jwt-secret:
@@ -708,30 +699,6 @@ log-level
Because currently there's no buffering for logging, the levels with minimal logging(``crit/error``) will increase throughput.
.. _log-query:
log-query
---------
=============== =================================
**Type** String
**Default** "disabled"
**Reloadable** Y
**Environment** PGRST_LOG_QUERY
**In-Database** `n/a`
=============== =================================
Logs the SQL query for the corresponding request at the current :ref:`log-level`.
See :ref:`sql_query_logs`.
.. code:: bash
# Logs the main SQL query
log-query = "main-query"
# Disables logging the SQL query
log-query = "disabled"
.. _openapi-mode:
openapi-mode
@@ -845,12 +812,6 @@ server-host
* :code:`*6` - any IPv4 or IPv6 hostname, IPv6 preferred
* :code:`!6` - any IPv6 hostname
Examples:
.. code:: bash
server-host = "127.0.0.1"
.. _server-port:
server-port
@@ -894,7 +855,7 @@ server-timing-enabled
**In-Database** pgrst.server_timing_enabled
=============== =================================
Enables the `Server-Timing <https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Server-Timing>`_ header.
Enables the `Server-Timing <https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing>`_ header.
See :ref:`server-timing_header`.
.. _server-unix-socket:
+14 -60
View File
@@ -5,8 +5,6 @@ Errors
PostgREST error messages follow the PostgreSQL error structure. It includes ``MESSAGE``, ``DETAIL``, ``HINT``, ``ERRCODE`` and will add an HTTP status code to the response.
.. _postgresql_errors:
Errors from PostgreSQL
======================
@@ -207,6 +205,14 @@ Related to the HTTP request elements.
| | | specified in the ``select`` part of the query string. |
| PGRST108 | | See :ref:`embed_filters`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst109: | 400 | Restricting a Deletion or an Update using limits must |
| | | include the ordering of a unique column. |
| PGRST109 | | See :ref:`limited_update_delete`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst110: | 400 | When restricting a Deletion or an Update using limits |
| | | modifies more rows than the maximum specified in the limit. |
| PGRST110 | | See :ref:`limited_update_delete`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst111: | 500 | An invalid ``response.headers`` was set. |
| | | See :ref:`guc_resp_hdrs`. |
| PGRST111 | | |
@@ -235,6 +241,10 @@ Related to the HTTP request elements.
| | | there is no many-to-one or one-to-one relationship between |
| PGRST118 | | them. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst119: | 400 | Could not use the spread operator on the related table |
| | | because there is no many-to-one or one-to-one relationship |
| PGRST119 | | between them. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst120: | 400 | An embedded resource can only be filtered using the |
| | | ``is.null`` or ``not.is.null`` :ref:`operators <operators>`.|
| PGRST120 | | |
@@ -247,31 +257,6 @@ Related to the HTTP request elements.
| | | ``Prefer: handling=strict``. See :ref:`prefer_handling`. |
| PGRST122 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst123: | 400 | Aggregate functions are disabled. |
| | | See :ref:`db-aggregates-enabled`. |
| PGRST123 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst124: | 400 | ``max-affected`` preference is violated. |
| | | See :ref:`prefer_max_affected`. |
| PGRST124 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst125: | 404 | Invalid path is specified in request URL. |
| | | |
| PGRST125 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst126: | 404 | Open API config is disabled but API root path is |
| | | accessed. See :ref:`openapi-mode`. |
| PGRST126 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst127: | 400 | The feature specified in the ``details`` field is not |
| | | implemented. |
| PGRST127 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst128: | 400 | ``max-affected`` preference is violated with ``RPC`` call. |
| | | See :ref:`prefer_max_affected`. |
| PGRST128 | | |
+---------------+-------------+-------------------------------------------------------------+
.. _pgrst2**:
@@ -305,10 +290,6 @@ Related to a :ref:`schema_cache`. Most of the time, these errors are solved by :
| | | in the ``columns`` query parameter is not found. |
| PGRST204 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst205: | 404 | Caused when the :ref:`table specified <tables_views>` in |
| | | the URI is not found. |
| PGRST205 | | |
+---------------+-------------+-------------------------------------------------------------+
.. _pgrst3**:
@@ -324,18 +305,14 @@ Related to the authentication process using JWT. You can follow the :ref:`tut1`
| | | configuration. |
| PGRST300 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst301: | 401 | Provided JWT couldn't be decoded or it is invalid. |
| | | |
| .. _pgrst301: | 401 | Any error related to the verification of the JWT, |
| | | which means that the JWT provided is invalid in some way. |
| PGRST301 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst302: | 401 | Attempted to do a request without |
| | | :ref:`authentication <client_auth>` when the anonymous role |
| PGRST302 | | is disabled by not setting it in :ref:`db-anon-role`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst303: | 401 | :ref:`JWT claims validation <jwt_claims_validation>` |
| | | or parsing failed. |
| PGRST303 | | |
+---------------+-------------+-------------------------------------------------------------+
.. The Internal Errors Group X** is always at the end
@@ -354,8 +331,6 @@ Internal errors. If you encounter any of these, you may have stumbled on a Postg
| PGRSTX00 | | |
+---------------+-------------+-------------------------------------------------------------+
.. _custom_errors:
Custom Errors
=============
@@ -452,24 +427,3 @@ For non standard HTTP status, you can optionally add ``status_text`` to describe
detail = '{"status":419,"status_text":"Page Expired","headers":{"X-Powered-By":"Nerd Rage"}}';
If PostgREST can't parse the JSON objects ``message`` and ``detail``, it will throw a ``PGRST121`` error. See :ref:`Errors from PostgREST<pgrst1**>`.
.. _proxy-status_header:
Proxy-Status Header
===================
For error cases, the standard `Proxy-Status <https://www.rfc-editor.org/rfc/rfc9209.html#name-the-proxy-status-http-field>`_ header is returned with the error code. The error code comes from either :ref:`PostgREST <pgrst_errors>`, :ref:`PostgreSQL <postgresql_errors>` or :ref:`Custom <custom_errors>` errors. This is useful when doing ``HEAD`` requests where the HTTP status is not descriptive enough.
For example, doing a request on a table with high count (say 30_000_000), we get:
.. code-block:: http
HEAD /table HTTP/1.1
Prefer: count=exact
.. code-block:: http
HTTP/1.1 500 Internal Server Error
Proxy-Status: PostgREST; error=57014
The PostgreSQL error code ``57014`` (`ref <https://www.postgresql.org/docs/current/errcodes-appendix.html>`_) reveals that the error is due to a short ``statement_timeout`` value.
+10 -73
View File
@@ -15,14 +15,12 @@ Observability allows measuring a system's current state based on the data it gen
Logs
====
PostgREST logs basic request information to ``stdout``, including the authenticated user if available, the requesting IP address and user agent, the URL requested, the HTTP response status and the response body size in bytes if available.
With :ref:`log-level` set to ``info``, we get:
PostgREST logs basic request information to ``stdout``, including the authenticated user if available, the requesting IP address and user agent, the URL requested, and HTTP response status.
.. code::
127.0.0.1 - user [26/Jul/2021:01:56:38 -0500] "GET /clients HTTP/1.1" 200 56 "" "curl/7.64.0"
127.0.0.1 - anonymous [26/Jul/2021:01:56:48 -0500] "GET /unexistent HTTP/1.1" 404 162 "" "curl/7.64.0"
127.0.0.1 - user [26/Jul/2021:01:56:38 -0500] "GET /clients HTTP/1.1" 200 - "" "curl/7.64.0"
127.0.0.1 - anonymous [26/Jul/2021:01:56:48 -0500] "GET /unexistent HTTP/1.1" 404 - "" "curl/7.64.0"
For diagnostic information about the server itself, PostgREST logs to ``stderr``:
@@ -33,52 +31,22 @@ For diagnostic information about the server itself, PostgREST logs to ``stderr``
.. code::
06/May/2024:08:16:11 -0500: Starting PostgREST 12.1...
06/May/2024:08:16:11 -0500: Attempting to connect to the database...
06/May/2024:08:16:11 -0500: Successfully connected to PostgreSQL 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit
06/May/2024:08:16:11 -0500: Connection Pool initialized with a maximum size of 10 connections
06/May/2024:08:16:11 -0500: API server listening on port 3000
06/May/2024:08:16:11 -0500: Listening for database notifications on the "pgrst" channel
06/May/2024:08:16:11 -0500: Listening on port 3000
06/May/2024:08:16:11 -0500: Listening for notifications on the "pgrst" channel
06/May/2024:08:16:11 -0500: Config reloaded
06/May/2024:08:16:11 -0500: Schema cache queried in 3.8 milliseconds
06/May/2024:08:16:11 -0500: Schema cache loaded 15 Relations, 8 Relationships, 8 Functions, 0 Domain Representations, 4 Media Type Handlers
06/May/2024:14:11:27 -0500: Received a config reload message on the "pgrst" channel
06/May/2024:14:11:27 -0500: Config reloaded
.. note::
Logs are based on the ``log-level`` setting. See :ref:`log-level`.
.. _sql_query_logs:
SQL Query Logs
--------------
To log the :ref:`main SQL query <main_query>` executed for a request, set the :ref:`log-query` to ``main-query``.
It will be logged based on the current :ref:`log-level` setting.
For example, with this configuration:
.. code-block:: bash
log-level = "warn"
log-query = "main-query"
The SQL queries will only be logged on ``400`` HTTP errors and up.
So, if the user requests a resource without sufficient privileges:
.. code-block:: bash
curl "localhost:3000/protected_table"
This will be logged by PostgREST:
.. code::
17/Feb/2025:17:28:15 -0500: WITH pgrst_source AS ( SELECT "public"."protected_table".* FROM "public"."protected_table" ) SELECT null::bigint AS total_result_set, pg_catalog.count(_postgrest_t) AS page_total, coalesce(json_agg(_postgrest_t), '[]') AS body, nullif(current_setting('response.headers', true), '') AS response_headers, nullif(current_setting('response.status', true), '') AS response_status, '' AS response_inserted FROM ( SELECT * FROM pgrst_source ) _postgrest_t
127.0.0.1 - web_anon [17/Feb/2025:17:28:15 -0500] "GET /protected_table HTTP/1.1" 401 99 "" "curl/8.7.1"
Database Logs
-------------
Additionally, to find all the SQL operations, you can watch the database logs. By default PostgreSQL does not keep these logs, so you'll need to make the configuration changes below.
Currently PostgREST doesn't log the SQL commands executed against the underlying database.
To find the SQL operations, you can watch the database logs. By default PostgreSQL does not keep these logs, so you'll need to make the configuration changes below.
Find :code:`postgresql.conf` inside your PostgreSQL data directory (to find that, issue the command :code:`show data_directory;`). Either find the settings scattered throughout the file and change them to the following values, or append this block of code to the end of the configuration file.
@@ -128,11 +96,6 @@ The ``metrics`` endpoint on the :ref:`admin_server` endpoint provides metrics in
curl "http://localhost:3001/metrics"
.. code-block:: http
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
# HELP pgrst_schema_cache_query_time_seconds The query time in seconds of the last schema cache load
# TYPE pgrst_schema_cache_query_time_seconds gauge
pgrst_schema_cache_query_time_seconds 1.5937927e-2
@@ -241,17 +204,12 @@ You can enable tracing HTTP requests by setting :ref:`server-trace-header`. Spec
HTTP/1.1 200 OK
X-Request-Id: 123
Proxy-Status Header
-------------------
See :ref:`proxy-status_header`.
.. _server-timing_header:
Server-Timing Header
--------------------
You can enable the `Server-Timing <https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Server-Timing>`_ header by setting :ref:`server-timing-enabled` on.
You can enable the `Server-Timing <https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing>`_ header by setting :ref:`server-timing-enabled` on.
This header communicates metrics of the different phases in the request-response cycle.
.. code-block:: bash
@@ -275,27 +233,6 @@ This header communicates metrics of the different phases in the request-response
We're working on lowering the duration of the ``parse`` and ``plan`` stages on https://github.com/PostgREST/postgrest/issues/2816.
.. _content-length_header:
Content-Length Header
---------------------
You can verify the response body size in bytes in the `Content-Length header <https://httpwg.org/specs/rfc9110.html#field.content-length>`_.
.. code-block:: bash
curl -i 'localhost:3000/users'
.. code-block:: http
HTTP/1.1 200 OK
Content-Length: 104
Note that this header won't be returned on ``HEAD`` requests for optimization purposes (see :ref:`head_req`).
This is in line with `RFC 9110 <https://httpwg.org/specs/rfc9110.html#field.content-length>`_.
The body size is also present in the :ref:`PostgREST logs <pgrst_logging>`.
.. _explain_plan:
Execution plan
+5 -4
View File
@@ -284,10 +284,11 @@ However, starting from PostgreSQL 15, you can grant privileges for these setting
GRANT SET ON PARAMETER <setting> TO <authenticator>;
Hoisted Function Settings
~~~~~~~~~~~~~~~~~~~~~~~~~
Function Settings
~~~~~~~~~~~~~~~~~
PostgREST can "hoist" function settings to transaction-scoped settings. This allows functions settings to override the impersonated and connection role settings.
In addition to :ref:`impersonated_settings`, PostgREST will also apply function settings as transaction-scoped settings. This allows functions settings to override
the impersonated and connection role settings.
.. code-block:: postgres
@@ -302,7 +303,7 @@ When calling the above function (see :ref:`functions`), the statement timeout wi
.. note::
Only the settings in :ref:`db-hoisted-tx-settings` will be hoisted.
Only the transactions that are hoisted by config :ref:`db-hoisted-tx-settings` will be applied.
.. _main_query:
+8 -6
View File
@@ -1,7 +1,9 @@
# This file is auto-generated by postgrest-nixpkgs-upgrade
sphinx==7.4.7
sphinx-copybutton==0.5.2
sphinx-rtd-dark-mode==1.3.0
sphinx-rtd-theme==3.0.2
sphinx-tabs==3.4.7
docutils==0.20.1
sphinx-copybutton
sphinx-intl
sphinx-rtd-theme>=0.5.1
sphinx-rtd-dark-mode>=1.3.0
sphinx-tabs>=3.2.0
sphinx>=5.0.2
sphinxext-opengraph==0.9.1
urllib3==2.2.2
+2 -13
View File
@@ -28,24 +28,13 @@
pacman -S postgrest
.. tab:: Nix via nixpkgs
.. tab:: Nix
You can install PostgREST from nixpkgs.
.. code:: bash
nix-env -i postgrest
.. tab:: Nix via flake
You can install PostgREST via flake.
.. code:: nix
{
inputs.postgrest.url = "github:postgrest/postgrest";
# ...
}
nix-env -i haskellPackages.postgrest
.. group-tab:: Windows
+15 -6
View File
@@ -17,7 +17,12 @@ To make an API we'll simply be building a database. All the endpoints and permis
By the end of this tutorial you'll have a working database, PostgREST server, and a simple single-user todo list API.
Step 1. Install PostgreSQL
Step 1. Relax, we'll help
-------------------------
As you begin the tutorial, pop open the project `chat room <https://gitter.im/begriffs/postgrest>`_ in another tab. There are a nice group of people active in the project and we'll help you out if you get stuck.
Step 2. Install PostgreSQL
--------------------------
If you're already familiar with using PostgreSQL and have it installed on your system you can use the existing installation (see :ref:`pg-dependency` for minimum requirements). For this tutorial we'll describe how to use the database in Docker because database configuration is otherwise too complicated for a simple tutorial.
@@ -43,7 +48,7 @@ This will run the Docker instance as a daemon and expose port 5432 to the host s
In this case, you will need to change the **first** of the two 5432 to something else, for example to :code:`5433:5432`. Remember to also adjust the port in your config file in Step 5!
Step 2. Install PostgREST
Step 3. Install PostgREST
-------------------------
Using a Package Manager
@@ -117,7 +122,7 @@ If everything is working correctly it will print out its version and the availab
</details>
</p>
Step 3. Create Database for API
Step 4. Create Database for API
-------------------------------
Connect to the SQL console (psql) inside the container. To do so, run this from your command line:
@@ -180,7 +185,7 @@ Now quit out of psql; it's time to start the API!
\q
Step 4. Run PostgREST
Step 5. Run PostgREST
---------------------
PostgREST can use a configuration file to tell it how to connect to the database. Create a file :code:`tutorial.conf` with this inside:
@@ -213,8 +218,12 @@ You should see something similar to:
.. code-block:: text
Starting PostgREST 12.0.2...
Successfully connected to PostgreSQL 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit
API server listening on port 3000
Attempting to connect to the database...
Connection successful
Listening on port 3000
Config reloaded
Listening for notifications on the pgrst channel
Schema cache loaded
It's now ready to serve web requests. There are many nice graphical API exploration tools you can use, but for this tutorial we'll use :code:`curl` because it's likely to be installed on your system already. Open a new terminal (leaving the one open that PostgREST is running inside). Try doing an HTTP request for the todos.
+11 -33
View File
@@ -52,31 +52,17 @@ Check that the :code:`tutorial.conf` (created in the previous tutorial) has the
If the PostgREST server is still running from the previous tutorial, restart it to load the updated configuration file.
.. _tut1_step3:
Step 3. Sign a Token
--------------------
Ordinarily your own code in the database or in another server will create and sign authentication tokens, but for this tutorial we will make one "by hand" using ``bash`` and ``openssl``.
Ordinarily your own code in the database or in another server will create and sign authentication tokens, but for this tutorial we will make one "by hand." Go to `jwt.io <https://jwt.io/#debugger-io>`_ and fill in the fields like this:
.. code:: bash
.. figure:: ../_static/tuts/tut1-jwt-io.png
:alt: jwt.io interface
#!/bin/bash
set -e
How to create a token at https://jwt.io
JWT_SECRET='test_secret_that_is_at_least_32_characters_long'
_base64 () { openssl base64 -e -A | tr '+/' '-_' | tr -d '='; }
header=$(echo -n '{"alg":"HS256","typ":"JWT"}' | _base64)
payload=$(echo -n "{\"role\":\"todo_user\"}" | _base64)
signature=$(echo -n "$header.$payload" | openssl dgst -sha256 -hmac "$JWT_SECRET" -binary | _base64)
echo -n "$header.$payload.$signature"
**Remember to fill in the secret you generated rather than keeping the "test_secret_that_is_at_least_32_characters_long".** After you have filled in the secret and payload, the encoded data on the left will update. Copy the encoded token.
**Remember to fill in the secret you generated rather than the word "secret".** After you have filled in the secret and payload, the encoded data on the left will update. Copy the encoded token.
.. note::
@@ -159,22 +145,14 @@ To observe expiration in action, we'll add an :code:`exp` claim of five minutes
select extract(epoch from now() + '5 minutes'::interval) :: integer;
Or in ``bash``:
Go back to jwt.io and change the payload to
.. code-block:: json
.. code-block:: bash
exp=$(( EPOCHSECONDS + 5*60 )) # five minutes
echo $exp
Go back to :ref:`tut1_step3` and change the payload to
.. code-block:: bash
payload=$(echo -n "{\"role\":\"todo_user\",\"exp\":\"123456789\"}" | _base64)
echo -n "$header.$payload.$signature"
{
"role": "todo_user",
"exp": 123456789
}
**NOTE**: Don't forget to change the dummy epoch value :code:`123456789` in the snippet above to the epoch value returned by the :code:`psql` command.
Generated
-27
View File
@@ -1,27 +0,0 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1731165248,
"narHash": "sha256-DiHFKIdBmMx5/DUARhVqaxvEIiy4EE6Eqs9Qs4oxme8=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "a90280100f41a10914edfe729a4053e60c92b8e3",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
-50
View File
@@ -1,50 +0,0 @@
{
description = "REST API for any Postgres database";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
};
nixConfig = {
extra-substituters = "https://postgrest.cachix.org";
extra-trusted-public-keys = "postgrest.cachix.org-1:icgW4R15fz1+LqvhPjt4EnX/r19AaqxiVV+1olwlZtI=";
};
outputs = { nixpkgs, ... }:
let
systems = [
"aarch64-darwin"
"aarch64-linux"
"x86_64-darwin"
"x86_64-linux"
];
pgrstFor = system: import ./default.nix {
inherit system;
nixpkgsVersion = {
owner = "nixos";
repo = "nixpkgs";
inherit (nixpkgs) rev;
tarballHash = nixpkgs.narHash;
};
};
genSystems = f: nixpkgs.lib.genAttrs systems (system: f (pgrstFor system));
in
{
packages = genSystems (attrs: {
default = attrs.postgrestPackage;
profiled = attrs.postgrestProfiled;
} // nixpkgs.lib.optionalAttrs (attrs ? postgrestStatic) {
static = attrs.postgrestStatic;
});
apps = genSystems (attrs: {
default = {
type = "app";
program = "${attrs.postgrestStatic or attrs.postgrestPackage}/bin/postgrest";
meta.description = "REST API for any Postgres database";
};
});
};
}
+30 -36
View File
@@ -72,53 +72,45 @@ The PostgREST utilities available in `nix-shell` all have names that begin with
```bash
# Note: The utilities listed here might not be up to date.
[nix-shell]$ postgrest-<tab>
postgrest-build postgrest-profiled-run
postgrest-check postgrest-push-cachix
postgrest-clean postgrest-release
postgrest-coverage postgrest-repl
postgrest-coverage-draft-overlay postgrest-run
postgrest-docs-build postgrest-style
postgrest-docs-check postgrest-style-check
postgrest-docs-dictcheck postgrest-test-big-schema
postgrest-docs-linkcheck postgrest-test-doctests
postgrest-docs-render postgrest-test-io
postgrest-docs-serve postgrest-test-memory
postgrest-docs-spellcheck postgrest-test-replica
postgrest-dump-minimal-imports postgrest-test-spec
postgrest-dump-schema postgrest-test-spec-idempotence
postgrest-gen-ctags postgrest-watch
postgrest-gen-jwt postgrest-with-all
postgrest-gen-secret postgrest-with-git
postgrest-git-hooks postgrest-with-pgrst
postgrest-hsie-graph-modules postgrest-with-postgresql-12
postgrest-hsie-graph-symbols postgrest-with-postgresql-13
postgrest-hsie-minimal-imports postgrest-with-postgresql-14
postgrest-lint postgrest-with-postgresql-15
postgrest-loadtest postgrest-with-postgresql-16
postgrest-loadtest-against postgrest-with-postgresql-17
postgrest-loadtest-report postgrest-with-slow-pg
postgrest-nixpkgs-upgrade postgrest-with-slow-postgrest
postgrest-parallel-curl
postgrest-build postgrest-test-spec
postgrest-check postgrest-watch
postgrest-clean postgrest-with-all
postgrest-coverage postgrest-with-postgresql-10
postgrest-lint postgrest-with-postgresql-11
postgrest-run postgrest-with-postgresql-12
postgrest-style postgrest-with-postgresql-13
postgrest-style-check postgrest-with-postgresql-9.6
postgrest-test-io
...
[nix-shell]$
```
The `docker` module has large dependencies to be build before the shell becomes
Some additional modules like `memory`, `docker` and `release`
have large dependencies that would need to be built before the shell becomes
available, which could take an especially long time if the cachix binary cache
is not used. You can activate it by passing a flag to `nix-shell` with
`nix-shell --arg docker true`. This will make the respective utilities available:
is not used. You can activate those by passing a flag to `nix-shell` with
`nix-shell --arg <module> true`. This will make the respective utilities available:
```bash
$ nix-shell --arg docker true
[nix-shell]$ postgrest-docker-<tab>
postgrest-docker-load
$ nix-shell --arg memory true
[nix-shell]$ postgrest-<tab>
postgrest-build postgrest-test-spec
postgrest-check postgrest-watch
postgrest-clean postgrest-with-all
postgrest-coverage postgrest-with-postgresql-10
postgrest-lint postgrest-with-postgresql-11
postgrest-run postgrest-with-postgresql-12
postgrest-style postgrest-with-postgresql-13
postgrest-style-check postgrest-with-postgresql-9.6
postgrest-test-io
postgrest-test-memory
...
```
Note that `postgrest-docker-load` is now also available.
Note that `postgrest-test-memory` is now also available.
To run one-off commands, you can also use `nix-shell --run <command>`, which
will launch the Nix shell, run that one command and exit. Note that the tab
@@ -379,8 +371,10 @@ that).
We also use `default.nix` to load our pinned version of the `nixpkgs`
repository. This set of packages will always be the same, independently from
where or when you use it. The pinned version is taken from `flake.lock` and
can be updated with `nix flake update`.
where or when you use it. The pinned version can be upgraded with the small
`nixpkgs-upgrade` utility. Running `nixpkgs-upgrade > nix/nixpkgs-version.nix`
in `nix-shell` will upgrade the pinned version to the latest `nixpkgs-unstable`
version.
### `shell.nix`
@@ -0,0 +1,46 @@
{ buildPythonPackage
, fetchFromGitHub
, lib
, nose
, sphinx
, sphinx-rtd-theme
}:
buildPythonPackage rec {
pname = "sphinx-rtd-dark-mode";
version = "1.3.0";
format = "setuptools";
src = fetchFromGitHub {
owner = "MrDogeBro";
repo = "sphinx_rtd_dark_mode";
rev = "refs/tags/v${version}";
hash = "sha256-N5KG2Wqn9wfGNY3VH4FnBce1aZUbnvVmwD10Loe0Qn4=";
};
propagatedBuildInputs = [
sphinx-rtd-theme
];
nativeCheckInputs = [
nose
sphinx
];
checkPhase = ''
runHook preCheck
nosetests tests
runHook postCheck
'';
pythonImportsCheck = [
"sphinx_rtd_dark_mode"
];
meta = with lib; {
description = "Adds a toggleable dark mode to the Read the Docs theme for Sphinx.";
homepage = "https://github.com/MrDogeBro/sphinx_rtd_dark_mode";
changelog = "https://github.com/MrDogeBro/sphinx_rtd_dark_mode/releases/tag/v${version}";
license = licenses.mit;
};
}
-4
View File
@@ -19,10 +19,6 @@ stdenv.mkDerivation {
pname = "libpq";
inherit (postgresql) src version patches;
__structuredAttrs = true;
env.CFLAGS = "-fdata-sections -ffunction-sections"
+ (if stdenv.cc.isClang then " -flto" else " -fmerge-constants -Wl,--gc-sections");
configureFlags = [
"--without-gssapi"
"--without-icu"
+8
View File
@@ -0,0 +1,8 @@
# Pinned version of Nixpkgs, generated with postgrest-nixpkgs-upgrade.
{
owner = "NixOS";
repo = "nixpkgs";
date = "2024-04-20";
rev = "92d295f588631b0db2da509f381b4fb1e74173c5";
tarballHash = "162w28y4i5c8g5qhjvs827qxphf2a8n4c8fwhcywzl1j1a35h2im";
}
+3
View File
@@ -3,6 +3,9 @@
checked-shell-script = import ./checked-shell-script;
gitignore = import ./gitignore.nix;
haskell-packages = import ./haskell-packages.nix;
postgis = import ./postgis.nix;
postgresql-libpq = import ./postgresql-libpq.nix;
postgresql-legacy = import ./postgresql-legacy.nix;
postgresql-future = import ./postgresql-future.nix;
slocat = import ./slocat.nix;
}
+18 -23
View File
@@ -48,36 +48,31 @@ let
# Before upgrading fuzzyset to 0.3, check: https://github.com/PostgREST/postgrest/issues/3329
# jailbreak, because hspec limit for tests
fuzzyset = prev.fuzzyset_0_2_4;
fuzzyset = lib.doJailbreak
(prev.callHackageDirect
{
pkg = "fuzzyset";
ver = "0.2.4";
sha256 = "sha256-lpkrTFcR0B4rT/P6x7ui31Twgq7BBj6KIvjKyqXKdpc=";
}
{ });
hasql-pool = lib.dontCheck (prev.callHackageDirect
{
pkg = "hasql-pool";
ver = "1.0.1";
sha256 = "sha256-Hf1f7lX0LWkjrb25SDBovCYPRdmUP1H6pAxzi7kT4Gg=";
}
{ });
hasql-pool = lib.dontCheck prev.hasql-pool_1_0_1;
postgresql-libpq = lib.dontCheck
(prev.postgresql-libpq.override {
postgresql = super.libpq;
});
hasql-notifications = lib.dontCheck (prev.callHackageDirect
{
pkg = "hasql-notifications";
ver = "0.2.2.2";
sha256 = "sha256-myKwlug7OgTa/qP6mHfCD+5Q8IhM17JvpJBfSo+M01k=";
ver = "0.2.2.0";
sha256 = "sha256-73OQ9/su2qvO7HavF3xuuNWLXSXyB9reBUQDaHys06I=";
}
{ });
{ }
);
# newer nixpkgs already has 0.10., so we fallback to default for forward compat
jose-jwt = prev.jose-jwt_0_10_0 or prev.jose-jwt;
postgresql-libpq = lib.dontCheck (prev.callHackageDirect
{
pkg = "postgresql-libpq";
ver = "0.10.1.0";
sha256 = "sha256-tXOMqCO8opMilI9rx0D+njqjIjbZsH168Bzb8Aq8Ff4=";
}
{
postgresql = super.libpq;
});
};
in
{
+28
View File
@@ -0,0 +1,28 @@
final: prev:
let
postgis_3_2_3 = rec {
version = "3.2.3";
src = final.fetchurl {
url = "https://download.osgeo.org/postgis/source/postgis-${version}.tar.gz";
sha256 = "sha256-G02LXHVuWrpZ77wYM7Iu/k1lYneO7KVvpJf+susTZow=";
};
meta.broken = false;
};
in
{
postgresql_11 = prev.postgresql_11.override { this = final.postgresql_11; } // {
pkgs = prev.postgresql_11.pkgs // {
postgis = prev.postgresql_11.pkgs.postgis.overrideAttrs (_: postgis_3_2_3);
};
};
postgresql_10 = prev.postgresql_10.override { this = final.postgresql_11; } // {
pkgs = prev.postgresql_10.pkgs // {
postgis = prev.postgresql_10.pkgs.postgis.overrideAttrs (_: postgis_3_2_3);
};
};
postgresql_9_6 = prev.postgresql_9_6.override { this = final.postgresql_11; } // {
pkgs = prev.postgresql_9_6.pkgs // {
postgis = prev.postgresql_9_6.pkgs.postgis.overrideAttrs (_: postgis_3_2_3);
};
};
}
+19
View File
@@ -0,0 +1,19 @@
_: _:
# Overlay that adds future versions of PostgreSQL that are supported by
# PostgREST.
{
## Example for including a postgresql version from a specific nixpks commit:
##
# postgresql_16 =
# let
# rev = "5148520bfab61f99fd25fb9ff7bfbb50dad3c9db";
# tarballHash = "1dfjmz65h8z4lk845724vypzmf3dbgsdndjpj8ydlhx6c7rpcq3p";
#
# pinnedPkgs =
# builtins.fetchTarball {
# url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
# sha256 = tarballHash;
# };
# in
# (import pinnedPkgs { }).pkgs.postgresql_16;
}
+49
View File
@@ -0,0 +1,49 @@
_: _:
# Overlay that adds legacy versions of PostgreSQL that are supported by
# PostgREST.
{
# PostgreSQL 9.6 was removed from Nixpkgs with
# https://github.com/NixOS/nixpkgs/commit/757dd008b2f2926fc0f7688fa8189f930ea47521
# We pin its parent commit to get the last version that was available.
postgresql_9_6 =
let
rev = "571cbf3d1db477058303cef8754fb85a14e90eb7";
tarballHash = "0q74wn418i1bn5sssacmw8ykpmqvzr0s93sj6pbs3rf6bf134fkz";
pinnedPkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
sha256 = tarballHash;
};
in
(import pinnedPkgs { }).pkgs.postgresql_9_6;
# PostgreSQL 10 was removed from Nixpkgs with
# https://github.com/NixOS/nixpkgs/commit/aa1483114bb329fee7e1266100b8d8921ed4723f
# We pin its parent commit to get the last version that was available.
postgresql_10 =
let
rev = "79661ba7e2fb96ebefbb537458a5bbae9dc5bd1a";
tarballHash = "0rn796pfn4sg90ai9fdnwmr10a2s835p1arazzgz46h6s5cxvq97";
pinnedPkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
sha256 = tarballHash;
};
in
(import pinnedPkgs { }).pkgs.postgresql_10;
# PostgreSQL 11 was removed from Nixpkgs with
# https://github.com/NixOS/nixpkgs/commit/1220a4d4dd1a4590780a5e1c18d1333a121be366
# We pin its parent commit to get the last version that was available.
postgresql_11 =
let
rev = "f5458516e42cc5cb4123cc2d93f45c240548aa18";
tarballHash = "1h03621sxfhw4z6ya74k6c2lyx3z7pvf2jcg4vs7i01yz2m6w3cv";
pinnedPkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
sha256 = tarballHash;
};
in
(import pinnedPkgs { }).pkgs.postgresql_11;
}
+2 -3
View File
@@ -1,7 +1,6 @@
_: super:
{
# Depending on which nixpkgs version is pinned, libpq might either be available already - or not.
libpq = super.libpq or (super.callPackage ../libpq.nix {
libpq = super.callPackage ../libpq.nix {
postgresql = super.postgresql_16;
});
};
}
+7 -10
View File
@@ -4,8 +4,12 @@
, src
}:
let
# This builds a static PostgREST executable based on pkgsStatic.
inherit (pkgs) pkgsStatic;
# This builds a static PostgREST exectuable based on pkgsStatic.
# pkgsStatic is based on musl, so is a kind of cross-compilation.
# We still make this explicit here via pkgsCross, because we need
# to get postgresql/libpq for musl, too.
pkgsCross = pkgs.pkgsCross.musl64;
inherit (pkgsCross) pkgsStatic;
inherit (pkgsStatic.haskell) lib;
packagesStatic =
@@ -21,16 +25,9 @@ let
overrides = pkgs.lib.composeExtensions old.overrides (_: prev: {
postgresql-libpq = (lib.overrideCabal prev.postgresql-libpq {
# TODO: This section can be simplified when this PR has made it's way to us:
# https://github.com/NixOS/nixpkgs/pull/286370
# Additionally, we need to use the default version in nixpkgs, otherwise the
# override will not be active as well.
# Using use-pkg-config flag, because pg_config won't work when cross-compiling
configureFlags = [ "-fuse-pkg-config" ];
# postgresql doesn't build in the fully static overlay - but the default
# derivation is built with static libraries anyway.
libraryPkgconfigDepends = [ pkgsStatic.libpq ];
librarySystemDepends = [ ];
libraryPkgconfigDepends = [ pkgsCross.libpq ];
}).overrideAttrs (_: prevAttrs: {
buildInputs = prevAttrs.buildInputs ++ [ pkgsStatic.openssl ];
});
-31
View File
@@ -15,7 +15,6 @@ let
withEnv = postgrest.env;
}
''
${cabal-install}/bin/cabal v2-update
exec ${cabal-install}/bin/cabal v2-build ${devCabalOptions} "''${_arg_leftovers[@]}"
'';
@@ -60,35 +59,6 @@ let
postgrest "''${_arg_leftovers[@]}"
'';
runProfiled =
checkedShellScript
{
name = "postgrest-profiled-run";
docs = "Run a profiled build of postgREST. This will generate a postgrest.prof file that can be used to do optimization.";
args =
[
"ARG_USE_ENV([PGRST_DB_ANON_ROLE], [postgrest_test_anonymous], [PostgREST anonymous role])"
"ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])"
"ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool timeout])"
"ARG_USE_ENV([PGRST_JWT_SECRET], [reallyreallyreallyreallyverysafe], [PostgREST JWT secret])"
"ARG_LEFTOVERS([PostgREST arguments])"
];
workingDir = "/";
withEnv = postgrest.env;
}
''
export PGRST_DB_ANON_ROLE
export PGRST_DB_POOL
export PGRST_DB_POOL_ACQUISITION_TIMEOUT
export PGRST_JWT_SECRET
${cabal-install}/bin/cabal v2-update
${cabal-install}/bin/cabal --builddir="dist-prof" v2-build --enable-profiling --disable-shared exe:postgrest
${cabal-install}/bin/cabal --builddir="dist-prof" v2-run -- \
postgrest +RTS -p -h -RTS "''${_arg_leftovers[@]}"
'';
repl =
checkedShellScript
{
@@ -110,7 +80,6 @@ buildToolbox
build
clean
run
runProfiled
repl;
};
}
+3 -13
View File
@@ -246,7 +246,6 @@ let
}
''
mkdir -p "$_arg_dumpdir"
${cabal-install}/bin/cabal v2-update
${cabal-install}/bin/cabal v2-build ${devCabalOptions} \
--builddir="$tmpdir" \
--ghc-option=-ddump-minimal-imports \
@@ -334,29 +333,20 @@ let
checkedShellScript
{
name = "postgrest-gen-jwt";
docs = ''
Generate a JWT. Example: postgrest-gen-jwt --exp 10 postgrest_test_author
# This can be used to quickly prove a JWT expiry
$ curl localhost:3000/authors_only -H "Authorization: Bearer \$(postgrest-gen-jwt --exp -31 postgrest_test_author)"
'';
docs = "Generate a JWT";
args = [
"ARG_POSITIONAL_SINGLE([role], [role for the jwt payload])"
"ARG_OPTIONAL_SINGLE([secret],, [secret used to sign the JWT], [reallyreallyreallyreallyverysafe])"
"ARG_OPTIONAL_SINGLE([exp],, [seconds for JWT expiry, it accepts negative values], [3600])"
];
}
''
# Based on https://stackoverflow.com/questions/59002949/how-to-create-a-json-web-token-jwt-using-openssl-shell-commands
# From https://stackoverflow.com/questions/59002949/how-to-create-a-json-web-token-jwt-using-openssl-shell-commands
# Construct the header
jwt_header=$(echo -n '{"alg":"HS256","typ":"JWT"}' | base64 | sed s/\+/-/g | sed 's/\//_/g' | sed -E s/=+$//)
# Construct the exp value
expiry=$((EPOCHSECONDS + _arg_exp))
# Construct the payload
payload=$(echo -n "{\"role\": \"$_arg_role\", \"exp\": $expiry}" | base64 | sed s/\+/-/g |sed 's/\//_/g' | sed -E s/=+$//)
payload=$(echo -n "{\"role\":\"$_arg_role\"}" | base64 | sed s/\+/-/g |sed 's/\//_/g' | sed -E s/=+$//)
# Convert secret to hex
hexsecret=$(echo -n "$_arg_secret" | xxd -p | paste -sd "")
+13 -31
View File
@@ -2,40 +2,23 @@
, aspellDicts
, buildToolbox
, checkedShellScript
, fetchPypi
, lib
, plantuml
, python3
, python3Packages
, writeTextFile
, writers
, plantuml
}:
let
selectPythonPackages = ps:
let
# TODO: Remove with next nixpkgs update
sphinx-rtd-theme = assert ps.sphinx-rtd-theme.version == "2.0.0"; ps.sphinx-rtd-theme.overrideAttrs rec {
version = "3.0.2";
src = fetchPypi {
pname = "sphinx_rtd_theme";
inherit version;
hash = "sha256-t0V7wl3acjsgsIamcLmVPIWeq2CioD7o6yuyPhduX4U=";
};
};
in
[
ps.sphinx
ps.sphinx-copybutton
(ps.sphinx-rtd-dark-mode.override { inherit sphinx-rtd-theme; })
sphinx-rtd-theme
ps.sphinx-tabs
ps.sphinxext-opengraph
];
requirements = writeTextFile {
name = "requirements.txt";
text = lib.concatMapStringsSep "\n" (pkg: "${pkg.pname}==${pkg.version}") (selectPythonPackages python3Packages);
};
selectPythonPackages = ps: [
ps.sphinx
ps.sphinx_rtd_theme
ps.livereload
ps.sphinx-tabs
ps.sphinx-copybutton
ps.sphinxext-opengraph
(ps.callPackage ../docs-extensions/sphinx-rtd-dark-mode.nix { })
# TODO: Remove override once new sphinx-intl version (> 2.1.0) is released and available in nixpkgs
(ps.sphinx-intl.overrideAttrs (drv: { nativeBuildInputs = drv.nativeBuildInputs ++ [ ps.six ]; }))
];
python = python3.withPackages selectPythonPackages;
@@ -90,7 +73,7 @@ let
server =
writers.writePython3
"postgrest-docs-server"
{ libraries = selectPythonPackages python3Packages ++ [ python3Packages.livereload ]; }
{ libraries = selectPythonPackages python3Packages; }
''
import sys
from livereload import Server, shell
@@ -200,5 +183,4 @@ buildToolbox
serve
spellcheck;
};
extra = { inherit requirements; };
}
-94
View File
@@ -1,94 +0,0 @@
# generates a file to be used by the vegeta load testing tool
import time
import hmac
import hashlib
import base64
import json
import argparse
import sys
import random
SECRET = b"reallyreallyreallyreallyverysafe"
URL = "http://postgrest"
JWT_DURATION = 120
TOTAL_TARGETS = 50000 # tuned by hand to reduce result variance
def base64url_encode(data: bytes) -> str:
"""URL-safe Base64 encode without padding."""
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
def generate_jwt(exp_inc: int) -> str:
"""Generate an HS256 JWT"""
# Header & payload
header = {"alg": "HS256", "typ": "JWT"}
now = int(time.time())
payload = {
"sub": f"user_{random.getrandbits(32)}",
"iat": now,
"exp": now + exp_inc,
"role": "postgrest_test_author",
}
# Encode to JSON and then to Base64URL
header_b = json.dumps(header, separators=(",", ":")).encode()
payload_b = json.dumps(payload, separators=(",", ":")).encode()
header_b64 = base64url_encode(header_b)
payload_b64 = base64url_encode(payload_b)
# Sign (HMACSHA256) the "<header>.<payload>" string
signing_input = f"{header_b64}.{payload_b64}".encode()
signature = hmac.new(SECRET, signing_input, hashlib.sha256).digest()
signature_b64 = base64url_encode(signature)
return f"{header_b64}.{payload_b64}.{signature_b64}"
# We want to ensure 401 Unauthorized responses don't happen during
# JWT validation, this can happen when the jwt `exp` is too short.
# At the same time, we want to ensure the `exp` is not too big,
# so expires will occur and postgREST will have to clean cached expired JWTs.
def estimate_adequate_jwt_exp_increase(iteration: int) -> int:
# estimated time takes to build and run postgrest itself
build_run_postgrest_time = 2
# estimated time it takes to generate the targets file
file_generation_time = TOTAL_TARGETS // (10**-5)
# estimated exp time so some JWTs will expire
dynamic_exp_inc = iteration // 1000
return build_run_postgrest_time + file_generation_time + dynamic_exp_inc
def main():
parser = argparse.ArgumentParser(
description="Generate Vegeta targets with unique JWTs"
)
parser.add_argument(
"output",
help="Path to write the generated targets file",
)
args = parser.parse_args()
lines = []
start_time = time.time()
for i in range(TOTAL_TARGETS):
token = generate_jwt(estimate_adequate_jwt_exp_increase(i))
lines.append(f"OPTIONS {URL}/authors_only")
lines.append(f"Authorization: Bearer {token}")
lines.append("") # blank line to separate requests
try:
with open(args.output, "w") as f:
f.write("\n".join(lines))
except IOError as e:
print(f"Error writing to {args.output}: {e}", file=sys.stderr)
sys.exit(1)
elapsed = time.time() - start_time
print(f"Created {TOTAL_TARGETS} targets in {args.output} ({elapsed:.2f}s)")
if __name__ == "__main__":
main()
+9 -32
View File
@@ -41,8 +41,6 @@ let
args = [
"ARG_OPTIONAL_SINGLE([output], [o], [Filename to dump json output to], [./loadtest/result.bin])"
"ARG_OPTIONAL_SINGLE([testdir], [t], [Directory to load tests and fixtures from], [./test/load])"
"ARG_OPTIONAL_SINGLE([kind], [k], [Kind of loadtest (mixed: repeat mixed requests, jwt: run once over many requests with unique jwts)], [mixed])"
"ARG_TYPE_GROUP_SET([KIND], [KIND], [kind], [mixed,jwt])"
"ARG_LEFTOVERS([additional vegeta arguments])"
];
workingDir = "/";
@@ -57,36 +55,17 @@ let
export PGRST_DB_POOL="1"
export PGRST_DB_TX_END="rollback-allow-override"
export PGRST_LOG_LEVEL="crit"
export PGRST_JWT_SECRET="reallyreallyreallyreallyverysafe"
export PGRST_JWT_CACHE_MAX_LIFETIME="86400"
mkdir -p "$(dirname "$_arg_output")"
abs_output="$(realpath "$_arg_output")"
case "$_arg_kind" in
jwt)
${genTargets} "$_arg_testdir"/gen_targets.http
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} \
sh -c "cd \"$_arg_testdir\" && ${runner} -lazy -targets gen_targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
${vegeta}/bin/vegeta report -type=text "$_arg_output"
;;
*)
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withSlowPg} \
${withTools.withPgrst} \
${withTools.withSlowPgrst} \
sh -c "cd \"$_arg_testdir\" && ${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
${vegeta}/bin/vegeta report -type=text "$_arg_output"
;;
esac
# shellcheck disable=SC2145
${withTools.withPg} -f "$_arg_testdir"/fixtures.sql \
${withTools.withSlowPg} \
${withTools.withPgrst} \
${withTools.withSlowPgrst} \
sh -c "cd \"$_arg_testdir\" && ${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
${vegeta}/bin/vegeta report -type=text "$_arg_output"
'';
loadtestAgainst =
@@ -104,7 +83,6 @@ let
'';
args = [
"ARG_POSITIONAL_INF([target], [Commit-ish reference to compare with], 1)"
"ARG_OPTIONAL_SINGLE([kind], [k], [Kind of loadtest], [mixed])"
];
positionalCompletion =
''
@@ -128,7 +106,7 @@ let
# Save the results in the current working tree, too,
# otherwise they'd be lost in the temporary working tree
# created by withTools.withGit.
${withTools.withGit} "$tgt" ${loadtest} -k "$_arg_kind" --output "$PWD/loadtest/$tgt.bin" --testdir "$PWD/test/load"
${withTools.withGit} "$tgt" ${loadtest} --output "$PWD/loadtest/$tgt.bin" --testdir "$PWD/test/load"
cat << EOF
@@ -144,7 +122,7 @@ let
EOF
${loadtest} -k "$_arg_kind" --output "$PWD/loadtest/head.bin" --testdir "$PWD/test/load"
${loadtest} --output "$PWD/loadtest/head.bin" --testdir "$PWD/test/load"
cat << EOF
@@ -200,7 +178,6 @@ let
| ${toMarkdown}
'';
genTargets = writers.writePython3 "postgrest-gen-loadtest-targets" { } (builtins.readFile ./generate_targets.py);
in
buildToolbox {
name = "postgrest-loadtest";
+51
View File
@@ -0,0 +1,51 @@
# The memory tests have large dependencies (a profiled build of PostgREST)
# and are run less often than the spec tests, so we don't include them in
# the default test environment. We make them available through a separate module.
# TODO both of these require reentering the nix-shell if you make a change to the code
{ buildToolbox
, checkedShellScript
, curl
, postgrestProfiled
, withTools
}:
let
test =
checkedShellScript
{
name = "postgrest-test-memory";
docs = "Run the memory tests.";
workingDir = "/";
withPath = [ postgrestProfiled curl ];
}
''
${withTools.withPg} -f test/spec/fixtures/load.sql test/memory/memory-tests.sh
'';
runProfiled =
checkedShellScript
{
name = "postgrest-profiled-run";
docs = "Run a profiled build of postgREST. This will generate a postgrest.prof file that can be used to do optimization. Note: if you make a change to the code, you must reenter the nix-shell for an updated profiled build.";
args =
[
"ARG_USE_ENV([PGRST_DB_ANON_ROLE], [postgrest_test_anonymous], [PostgREST anonymous role])"
"ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])"
"ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool timeout])"
"ARG_LEFTOVERS([PostgREST arguments])"
];
workingDir = "/";
withPath = [ postgrestProfiled ];
}
''
export PGRST_DB_ANON_ROLE
export PGRST_DB_POOL
export PGRST_DB_POOL_ACQUISITION_TIMEOUT
postgrest +RTS -p -h -RTS "''${_arg_leftovers[@]}"
'';
in
buildToolbox
{
name = "postgrest-memory";
tools = { inherit test runProfiled; };
}
+57
View File
@@ -0,0 +1,57 @@
{ buildToolbox
, checkedShellScript
, coreutils
, curl
, jq
, nix
}:
# Utility script for pinning the latest stable version of Nixpkgs.
# Instead of pinning Nixpkgs based on the huge Git repository, we reference a
# specific tarball that only contains the source of the revision that we want
# to pin.
let
name =
"postgrest-nixpkgs-upgrade";
refUrl =
"https://api.github.com/repos/nixos/nixpkgs/git/matching-refs/heads/nixpkgs-";
githubV3Header =
"Accept: application/vnd.github.v3+json";
tarballUrlBase =
"https://github.com/nixos/nixpkgs/archive/";
upgrade =
checkedShellScript
{
inherit name;
docs = "Pin the newest stable version of Nixpkgs.";
workingDir = "/";
}
''
# The list of refs is sorted. The first result will be nixpkgs-unstable, the second the latest stable branch.
commitHash="$(${curl}/bin/curl "${refUrl}" -H "${githubV3Header}" | ${jq}/bin/jq -r 'sort_by(.ref) | reverse | .[1].object.sha')"
tarballUrl="${tarballUrlBase}$commitHash.tar.gz"
tarballHash="$(${nix}/bin/nix-prefetch-url --unpack "$tarballUrl")"
currentDate="$(${coreutils}/bin/date --iso)"
cat > nix/nixpkgs-version.nix << EOF
# Pinned version of Nixpkgs, generated with ${name}.
{
owner = "NixOS";
repo = "nixpkgs";
date = "$currentDate";
rev = "$commitHash";
tarballHash = "$tarballHash";
}
EOF
'';
in
buildToolbox
{
name = "postgrest-nixpkgs";
tools = { inherit upgrade; };
}
@@ -29,28 +29,22 @@ let
case "$1" in
major)
new_version="$((major+1)).0.0"
new_docs_version="$((major+1)).0"
;;
minor)
new_version="$major.$((minor+1)).0"
new_docs_version="$major.$((minor+1))"
;;
patch)
new_version="$major.$minor.$((patch+1))"
new_docs_version="$major.$minor"
;;
devel)
new_version="$major.$((minor+1))"
new_docs_version="devel"
;;
esac
echo "Updating postgrest.cabal ..."
sed -i -E "s/^(version:\s+).*$/\1$new_version/" postgrest.cabal > /dev/null
echo "Updating docs/conf.py ..."
sed -i -E "s/^(version = ).*$/\1\"$new_docs_version\"/" docs/conf.py > /dev/null
git add postgrest.cabal docs/conf.py > /dev/null
git add postgrest.cabal > /dev/null
}
today_date_for_changelog="$(date '+%Y-%m-%d')"
+4 -30
View File
@@ -1,7 +1,6 @@
{ buildToolbox
, cabal-install
, checkedShellScript
, curl
, devCabalOptions
, ghc
, glibcLocales ? null
@@ -28,7 +27,6 @@ let
withEnv = postgrest.env;
}
''
${cabal-install}/bin/cabal v2-update
${withTools.withPg} -f test/spec/fixtures/load.sql \
${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:spec -- "''${_arg_leftovers[@]}"
'';
@@ -42,7 +40,6 @@ let
withEnv = postgrest.env;
}
''
${cabal-install}/bin/cabal v2-update
# This makes nix-env -iA tests.doctests.bin work.
export NIX_GHC=${postgrest.env.NIX_GHC}
${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:doctests
@@ -57,7 +54,6 @@ let
withEnv = postgrest.env;
}
''
${cabal-install}/bin/cabal v2-update
${withTools.withPg} -f test/spec/fixtures/load.sql \
${runtimeShell} -c " \
${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:spec && \
@@ -85,8 +81,7 @@ let
withEnv = postgrest.env;
}
''
${cabal-install}/bin/cabal v2-update
${cabal-install}/bin/cabal v2-build ${devCabalOptions} exe:postgrest
${cabal-install}/bin/cabal v2-build ${devCabalOptions}
${cabal-install}/bin/cabal v2-exec -- ${withTools.withPg} -f test/io/fixtures.sql \
${ioTestPython}/bin/pytest --ignore=test/io/test_big_schema.py --ignore=test/io/test_replica.py -v test/io "''${_arg_leftovers[@]}"
'';
@@ -101,8 +96,7 @@ let
withEnv = postgrest.env;
}
''
${cabal-install}/bin/cabal v2-update
${cabal-install}/bin/cabal v2-build ${devCabalOptions} exe:postgrest
${cabal-install}/bin/cabal v2-build ${devCabalOptions}
${cabal-install}/bin/cabal v2-exec -- ${withTools.withPg} -f test/io/big_schema.sql \
${ioTestPython}/bin/pytest -v test/io/test_big_schema.py "''${_arg_leftovers[@]}"
'';
@@ -117,8 +111,7 @@ let
withEnv = postgrest.env;
}
''
${cabal-install}/bin/cabal v2-update
${cabal-install}/bin/cabal v2-build ${devCabalOptions} exe:postgrest
${cabal-install}/bin/cabal v2-build ${devCabalOptions}
${cabal-install}/bin/cabal v2-exec -- ${withTools.withPg} --replica -f test/io/replica.sql \
${ioTestPython}/bin/pytest -v test/io/test_replica.py "''${_arg_leftovers[@]}"
'';
@@ -133,7 +126,6 @@ let
withPath = [ jq ];
}
''
${cabal-install}/bin/cabal v2-update
${withTools.withPg} -f test/spec/fixtures/load.sql \
${cabal-install}/bin/cabal v2-run ${devCabalOptions} --verbose=0 -- \
postgrest --dump-schema
@@ -162,7 +154,6 @@ let
rm -rf coverage/*
# build once before running all the tests
${cabal-install}/bin/cabal v2-update
${cabal-install}/bin/cabal v2-build ${devCabalOptions} exe:postgrest lib:postgrest test:spec
(
@@ -236,22 +227,6 @@ let
sed -i 's|^module \(.*\):|module \1/|g' test/coverage.overlay
'';
testMemory =
checkedShellScript
{
name = "postgrest-test-memory";
docs = "Run the memory tests.";
workingDir = "/";
withEnv = postgrest.env;
withPath = [ curl ];
}
''
${cabal-install}/bin/cabal v2-update
${cabal-install}/bin/cabal --builddir="dist-prof" v2-build --enable-profiling --disable-shared exe:postgrest
${cabal-install}/bin/cabal --builddir="dist-prof" v2-exec -- ${withTools.withPg} -f test/spec/fixtures/load.sql \
test/memory/memory-tests.sh
'';
in
buildToolbox
{
@@ -266,7 +241,6 @@ buildToolbox
testReplica
dumpSchema
coverage
coverageDraftOverlay
testMemory;
coverageDraftOverlay;
};
}
+24 -24
View File
@@ -1,5 +1,5 @@
name: postgrest
version: 13.0.6
version: 12.2.2
synopsis: REST API for any Postgres database
description: Reads the schema of a PostgreSQL database and creates RESTful routes
for tables, views, and functions, supporting all HTTP methods that security
@@ -16,13 +16,14 @@ extra-source-files: CHANGELOG.md
cabal-version: >= 1.10
tested-with:
-- nix
GHC == 9.4.8
-- cabal on Ubuntu
-- stack on FreeBSD, MacOS, Ubuntu, Windows
, GHC == 9.6.7
GHC == 9.4.5
-- nix, cabal on Ubuntu (arm)
, GHC == 9.4.8
-- cabal on Ubuntu
, GHC == 9.8.4
, GHC == 9.6.4
-- cabal on Ubuntu
, GHC == 9.8.2
source-repository head
type: git
@@ -47,8 +48,6 @@ library
PostgREST.App
PostgREST.AppState
PostgREST.Auth
PostgREST.Auth.JwtCache
PostgREST.Auth.Types
PostgREST.CLI
PostgREST.Config
PostgREST.Config.Database
@@ -67,7 +66,6 @@ library
PostgREST.Logger
PostgREST.MediaType
PostgREST.Metrics
PostgREST.Network
PostgREST.Observation
PostgREST.Query
PostgREST.Query.QueryBuilder
@@ -89,11 +87,12 @@ library
PostgREST.Response.GucHeader
PostgREST.Response.Performance
PostgREST.Version
other-modules: Paths_postgrest
build-depends: base >= 4.9 && < 4.20
, HTTP >= 4000.3.7 && < 4000.5
, Ranged-sets >= 0.3 && < 0.5
, aeson >= 2.0.3 && < 2.3
, auto-update >= 0.1.4 && < 0.3
, auto-update >= 0.1.4 && < 0.2
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13
, cache >= 0.1.3 && < 0.2.0
@@ -102,30 +101,30 @@ library
, clock >= 0.8.3 && < 0.9.0
, configurator-pg >= 0.2 && < 0.3
, containers >= 0.5.7 && < 0.7
, cookie >= 0.4.2 && < 0.6
, contravariant-extras >= 0.3.3 && < 0.4
, cookie >= 0.4.2 && < 0.5
, directory >= 1.2.6 && < 1.4
, either >= 4.4.1 && < 5.1
, extra >= 1.7.0 && < 2.0
, fuzzyset >= 0.2.4 && < 0.3
, gitrev >= 1.2 && < 1.4
, hasql >= 1.6.1.1 && < 1.7
, hasql-dynamic-statements >= 0.3.1 && < 0.4
, hasql-notifications >= 0.2.2.2 && < 0.2.3
, hasql-notifications >= 0.2.2.0 && < 0.3
, hasql-pool >= 1.0.1 && < 1.1
, hasql-transaction >= 1.0.1 && < 1.2
, hasql-transaction >= 1.0.1 && < 1.1
, heredoc >= 0.2 && < 0.3
, http-types >= 0.12.2 && < 0.13
, insert-ordered-containers >= 0.2.2 && < 0.3
, iproute >= 1.7.0 && < 1.8
, jose-jwt >= 0.9.6 && < 0.11
, lens >= 4.14 && < 5.4
, interpolatedstring-perl6 >= 1 && < 1.1
, jose >= 0.8.5.1 && < 0.12
, lens >= 4.14 && < 5.3
, lens-aeson >= 1.0.1 && < 1.3
, mtl >= 2.2.2 && < 2.4
, neat-interpolation >= 0.5 && < 0.6
, network >= 2.6 && < 3.3
, network >= 2.6 && < 3.2
, network-uri >= 2.6.1 && < 2.8
, optparse-applicative >= 0.13 && < 0.19
, parsec >= 3.1.11 && < 3.2
-- Technically unused, can be removed after updating to hasql >= 1.7
, postgresql-libpq >= 0.10
, prometheus-client >= 1.1.1 && < 1.2.0
, protolude >= 0.3.1 && < 0.4
@@ -149,7 +148,7 @@ library
-- for unix sockets; this is tested in test/io/test_io.py. See
-- https://github.com/kazu-yamamoto/logger/commit/3a71ca70afdbb93d4ecf0083eeba1fbbbcab3fc3
, wai-logger >= 2.4.0
, warp >= 3.3.19 && < 3.5
, warp >= 3.3.19 && < 3.4
-- -fno-spec-constr may help keep compile time memory use in check,
-- see https://gitlab.haskell.org/ghc/ghc/issues/16017#note_219304
-- -optP-Wno-nonportable-include-path
@@ -205,7 +204,7 @@ test-suite spec
Feature.Auth.AuthSpec
Feature.Auth.BinaryJwtSecretSpec
Feature.Auth.NoAnonSpec
Feature.Auth.NoJwtSecretSpec
Feature.Auth.NoJwtSpec
Feature.ConcurrentSpec
Feature.CorsSpec
Feature.ExtraSearchPathSpec
@@ -228,6 +227,7 @@ test-suite spec
Feature.Query.ErrorSpec
Feature.Query.InsertSpec
Feature.Query.JsonOperatorSpec
Feature.Query.LimitedMutationSpec
Feature.Query.MultipleSchemaSpec
Feature.Query.NullsStripSpec
Feature.Query.PgSafeUpdateSpec
@@ -253,19 +253,19 @@ test-suite spec
, aeson >= 2.0.3 && < 2.3
, aeson-qq >= 0.8.1 && < 0.9
, async >= 2.1.1 && < 2.3
, auto-update >= 0.1.4 && < 0.2
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13
, case-insensitive >= 1.2 && < 1.3
, containers >= 0.5.7 && < 0.7
, hasql-pool >= 1.0.1 && < 1.1
, hasql-transaction >= 1.0.1 && < 1.2
, hasql-transaction >= 1.0.1 && < 1.1
, heredoc >= 0.2 && < 0.3
, hspec >= 2.3 && < 2.12
, hspec-wai >= 0.10 && < 0.12
, hspec-wai-json >= 0.10 && < 0.12
, http-types >= 0.12.3 && < 0.13
, jose-jwt >= 0.9.6 && < 0.11
, lens >= 4.14 && < 5.4
, lens >= 4.14 && < 5.3
, lens-aeson >= 1.0.1 && < 1.3
, monad-control >= 1.0.1 && < 1.1
, postgrest
+5 -2
View File
@@ -7,6 +7,7 @@
# We highly recommend that use the PostgREST binary cache by installing cachix
# (https://app.cachix.org/) and running `cachix use postgrest`.
{ docker ? false
, memory ? false
}:
let
postgrest =
@@ -22,12 +23,14 @@ let
postgrest.devTools
postgrest.docs
postgrest.loadtest
postgrest.release
postgrest.nixpkgsTools
postgrest.style
postgrest.tests
postgrest.withTools
postgrest.release
]
++ lib.optional docker postgrest.docker;
++ lib.optional docker postgrest.docker
++ lib.optional memory postgrest.memory;
in
lib.overrideDerivation postgrest.env (
+9 -5
View File
@@ -11,17 +11,19 @@ import qualified Network.Wai.Handler.Warp as Warp
import Control.Monad.Extra (whenJust)
import qualified Data.ByteString.Lazy as LBS
import Network.Socket
import Network.Socket.ByteString
import PostgREST.AppState (AppState)
import PostgREST.Config (AppConfig (..))
import PostgREST.MediaType (MediaType (..), toContentType)
import PostgREST.Metrics (metricsToText)
import PostgREST.Network (resolveHost)
import PostgREST.Observation (Observation (..))
import qualified PostgREST.AppState as AppState
import qualified PostgREST.Config as Config
import Protolude
@@ -29,8 +31,7 @@ runAdmin :: AppState -> Warp.Settings -> IO ()
runAdmin appState settings = do
AppConfig{configAdminServerPort} <- AppState.getConfig appState
whenJust (AppState.getSocketAdmin appState) $ \adminSocket -> do
host <- resolveHost adminSocket
observer $ AdminStartObs host configAdminServerPort
observer $ AdminStartObs configAdminServerPort
void . forkIO $ Warp.runSettingsSocket settings adminSocket adminApp
where
adminApp = admin appState
@@ -54,12 +55,15 @@ admin appState req respond = do
| otherwise = HTTP.status500
in
respond $ Wai.responseLBS status [] mempty
["config"] -> do
config <- AppState.getConfig appState
respond $ Wai.responseLBS HTTP.status200 [] (LBS.fromStrict $ encodeUtf8 $ Config.toText config)
["schema_cache"] -> do
sCache <- AppState.getSchemaCache appState
respond $ Wai.responseLBS HTTP.status200 [] (maybe mempty JSON.encode sCache)
["metrics"] -> do
mets <- metricsToText
respond $ Wai.responseLBS HTTP.status200 [toContentType MTTextPlain] mets -- Content-Type is required for prometheus compliance
respond $ Wai.responseLBS HTTP.status200 [] mets
_ ->
respond $ Wai.responseLBS HTTP.status404 [] mempty
+14 -19
View File
@@ -4,6 +4,7 @@ Description : PostgREST functions to translate HTTP request to a domain type cal
-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
-- TODO: This module shouldn't depend on SchemaCache
module PostgREST.ApiRequest
( ApiRequest(..)
, InvokeMethod(..)
@@ -13,7 +14,6 @@ module PostgREST.ApiRequest
, DbAction(..)
, Payload(..)
, userApiRequest
, userPreferences
) where
import qualified Data.Aeson as JSON
@@ -44,16 +44,16 @@ import Network.Wai.Parse (parseHttpAccept)
import Web.Cookie (parseCookies)
import PostgREST.ApiRequest.QueryParams (QueryParams (..))
import PostgREST.ApiRequest.Types (ApiRequestError (..),
RangeError (..))
import PostgREST.Config (AppConfig (..),
OpenAPIMode (..))
import PostgREST.Config.Database (TimezoneNames)
import PostgREST.Error (ApiRequestError (..),
RangeError (..))
import PostgREST.MediaType (MediaType (..))
import PostgREST.RangeQuery (NonnegRange, allRange,
convertToLimitZeroRange,
hasLimitZero,
rangeRequested)
import PostgREST.SchemaCache (SchemaCache (..))
import PostgREST.SchemaCache.Identifiers (FieldName,
QualifiedIdentifier (..),
Schema)
@@ -128,8 +128,8 @@ data ApiRequest = ApiRequest {
}
-- | Examines HTTP request and translates it into user intent.
userApiRequest :: AppConfig -> Preferences.Preferences -> Request -> RequestBody -> Either ApiRequestError ApiRequest
userApiRequest conf prefs req reqBody = do
userApiRequest :: AppConfig -> Request -> RequestBody -> SchemaCache -> Either ApiRequestError ApiRequest
userApiRequest conf req reqBody sCache = do
resource <- getResource conf $ pathInfo req
(schema, negotiatedByProfile) <- getSchema conf hdrs method
act <- getAction resource schema method
@@ -141,7 +141,7 @@ userApiRequest conf prefs req reqBody = do
, iRange = ranges
, iTopLevelRange = topLevelRange
, iPayload = payload
, iPreferences = prefs
, iPreferences = Preferences.fromHeaders (configDbTxAllowOverride conf) (dbTimezones sCache) hdrs
, iQueryParams = qPrms
, iColumns = columns
, iHeaders = iHdrs
@@ -162,21 +162,15 @@ userApiRequest conf prefs req reqBody = do
contentMediaType = maybe MTApplicationJSON MediaType.decodeMediaType $ lookupHeader "content-type"
actIsInvokeSafe x = case x of {ActDb (ActRoutine _ (InvRead _)) -> True; _ -> False}
-- | Parses the Prefer header
userPreferences :: AppConfig -> Request -> TimezoneNames -> Preferences.Preferences
userPreferences conf req timezones = Preferences.fromHeaders (configDbTxAllowOverride conf) timezones $ requestHeaders req
getResource :: AppConfig -> [Text] -> Either ApiRequestError Resource
getResource AppConfig{configOpenApiMode, configDbRootSpec} = \case
[] ->
case (configOpenApiMode,configDbRootSpec) of
(OADisabled,_) -> Left OpenAPIDisabled
(_, Just qi) -> Right $ ResourceRoutine (qiName qi)
(_, Nothing) -> Right ResourceSchema
[] -> case configDbRootSpec of
Just (QualifiedIdentifier _ pathName) -> Right $ ResourceRoutine pathName
Nothing | configOpenApiMode == OADisabled -> Left NotFound
| otherwise -> Right ResourceSchema
[table] -> Right $ ResourceRelation table
["rpc", pName] -> Right $ ResourceRoutine pName
_ -> Left InvalidResourcePath
_ -> Left NotFound
getAction :: Resource -> Schema -> ByteString -> Either ApiRequestError Action
getAction resource schema method =
@@ -224,8 +218,9 @@ getSchema AppConfig{configDbSchemas} hdrs method = do
lookupHeader = flip lookup hdrs
getRanges :: ByteString -> QueryParams -> RequestHeaders -> Either ApiRequestError (NonnegRange, HM.HashMap Text NonnegRange)
getRanges method QueryParams{qsRanges} hdrs
getRanges method QueryParams{qsOrder,qsRanges} hdrs
| isInvalidRange = Left $ InvalidRange (if rangeIsEmpty headerRange then LowerGTUpper else NegativeLimit)
| method `elem` ["PATCH", "DELETE"] && not (null qsRanges) && null qsOrder = Left LimitNoOrderError
| method == "PUT" && topLevelRange /= allRange = Left PutLimitNotAllowedError
| otherwise = Right (topLevelRange, ranges)
where
+20 -2
View File
@@ -2,7 +2,7 @@
-- Module: PostgREST.ApiRequest.Preferences
-- Description: Track client preferences to be employed when processing requests
--
-- Track client preferences set in HTTP 'Prefer' headers according to RFC7240[1].
-- Track client prefences set in HTTP 'Prefer' headers according to RFC7240[1].
--
-- [1] https://datatracker.ietf.org/doc/html/rfc7240
--
@@ -12,6 +12,7 @@ module PostgREST.ApiRequest.Preferences
, PreferCount(..)
, PreferHandling(..)
, PreferMissing(..)
, PreferParameters(..)
, PreferRepresentation(..)
, PreferResolution(..)
, PreferTransaction(..)
@@ -36,6 +37,7 @@ import Protolude
-- >>> import Text.Pretty.Simple (pPrint)
-- >>> deriving instance Show PreferResolution
-- >>> deriving instance Show PreferRepresentation
-- >>> deriving instance Show PreferParameters
-- >>> deriving instance Show PreferCount
-- >>> deriving instance Show PreferTransaction
-- >>> deriving instance Show PreferMissing
@@ -49,6 +51,7 @@ data Preferences
= Preferences
{ preferResolution :: Maybe PreferResolution
, preferRepresentation :: Maybe PreferRepresentation
, preferParameters :: Maybe PreferParameters
, preferCount :: Maybe PreferCount
, preferTransaction :: Maybe PreferTransaction
, preferMissing :: Maybe PreferMissing
@@ -68,6 +71,7 @@ data Preferences
-- Preferences
-- { preferResolution = Just IgnoreDuplicates
-- , preferRepresentation = Nothing
-- , preferParameters = Nothing
-- , preferCount = Just ExactCount
-- , preferTransaction = Nothing
-- , preferMissing = Nothing
@@ -85,6 +89,7 @@ data Preferences
-- Preferences
-- { preferResolution = Just IgnoreDuplicates
-- , preferRepresentation = Nothing
-- , preferParameters = Nothing
-- , preferCount = Just ExactCount
-- , preferTransaction = Nothing
-- , preferMissing = Just ApplyNulls
@@ -117,6 +122,7 @@ data Preferences
-- Preferences
-- { preferResolution = Nothing
-- , preferRepresentation = Just Full
-- , preferParameters = Nothing
-- , preferCount = Just ExactCount
-- , preferTransaction = Just Commit
-- , preferMissing = Just ApplyDefaults
@@ -131,6 +137,7 @@ fromHeaders allowTxDbOverride acceptedTzNames headers =
Preferences
{ preferResolution = parsePrefs [MergeDuplicates, IgnoreDuplicates]
, preferRepresentation = parsePrefs [Full, None, HeadersOnly]
, preferParameters = parsePrefs [SingleObject]
, preferCount = parsePrefs [ExactCount, PlannedCount, EstimatedCount]
, preferTransaction = if allowTxDbOverride then parsePrefs [Commit, Rollback] else Nothing
, preferMissing = parsePrefs [ApplyDefaults, ApplyNulls]
@@ -144,6 +151,7 @@ fromHeaders allowTxDbOverride acceptedTzNames headers =
mapToHeadVal = map toHeaderValue
acceptedPrefs = mapToHeadVal [MergeDuplicates, IgnoreDuplicates] ++
mapToHeadVal [Full, None, HeadersOnly] ++
mapToHeadVal [SingleObject] ++
mapToHeadVal [ExactCount, PlannedCount, EstimatedCount] ++
mapToHeadVal [Commit, Rollback] ++
mapToHeadVal [ApplyDefaults, ApplyNulls] ++
@@ -171,7 +179,7 @@ fromHeaders allowTxDbOverride acceptedTzNames headers =
prefMap = Map.fromList . fmap (\pref -> (toHeaderValue pref, pref))
prefAppliedHeader :: Preferences -> Maybe HTTP.Header
prefAppliedHeader Preferences {preferResolution, preferRepresentation, preferCount, preferTransaction, preferMissing, preferHandling, preferTimezone, preferMaxAffected } =
prefAppliedHeader Preferences {preferResolution, preferRepresentation, preferParameters, preferCount, preferTransaction, preferMissing, preferHandling, preferTimezone, preferMaxAffected } =
if null prefsVals
then Nothing
else Just (HTTP.hPreferenceApplied, combined)
@@ -181,6 +189,7 @@ prefAppliedHeader Preferences {preferResolution, preferRepresentation, preferCou
toHeaderValue <$> preferResolution
, toHeaderValue <$> preferMissing
, toHeaderValue <$> preferRepresentation
, toHeaderValue <$> preferParameters
, toHeaderValue <$> preferCount
, toHeaderValue <$> preferTransaction
, toHeaderValue <$> preferHandling
@@ -222,6 +231,15 @@ instance ToHeaderValue PreferRepresentation where
toHeaderValue None = "return=minimal"
toHeaderValue HeadersOnly = "return=headers-only"
-- | How to pass parameters to stored procedures.
-- TODO: deprecated. Remove on next major version.
data PreferParameters
= SingleObject -- ^ Pass all parameters as a single json object to a stored procedure.
deriving Eq
instance ToHeaderValue PreferParameters where
toHeaderValue SingleObject = "params=single-object"
-- | How to determine the count of (expected) results
data PreferCount
= ExactCount -- ^ Exact count (slower).
+10 -12
View File
@@ -46,7 +46,7 @@ import PostgREST.SchemaCache.Identifiers (FieldName)
import PostgREST.ApiRequest.Types (AggregateFunction (..),
EmbedParam (..), EmbedPath, Field,
Filter (..), FtsOperator (..),
Hint, IsVal (..), JoinType (..),
Hint, JoinType (..),
JsonOperand (..),
JsonOperation (..), JsonPath,
ListVal, LogicOperator (..),
@@ -54,11 +54,10 @@ import PostgREST.ApiRequest.Types (AggregateFunction (..),
OpQuantifier (..), Operation (..),
OrderDirection (..),
OrderNulls (..), OrderTerm (..),
QuantOperator (..),
QPError (..), QuantOperator (..),
SelectItem (..),
SimpleOperator (..), SingleVal)
import PostgREST.Error (QPError (..))
SimpleOperator (..), SingleVal,
TrileanVal (..))
import Protolude hiding (Sum, try)
@@ -641,7 +640,7 @@ pOpExpr pSVal = do
pOperation = pIn <|> pIs <|> pIsDist <|> try pFts <|> try pSimpleOp <|> try pQuantOp <?> "operator (eq, gt, ...)"
pIn = In <$> (try (string "in" *> pDelimiter) *> pListVal)
pIs = Is <$> (try (string "is" *> pDelimiter) *> pIsVal)
pIs = Is <$> (try (string "is" *> pDelimiter) *> pTriVal)
pIsDist = IsDistinctFrom <$> (try (string "isdistinct" *> pDelimiter) *> pSVal)
@@ -654,12 +653,11 @@ pOpExpr pSVal = do
quant <- optionMaybe $ try (between (char '(') (char ')') (try (string "any" $> QuantAny) <|> string "all" $> QuantAll))
pDelimiter *> (OpQuant op quant <$> pSVal)
pIsVal = try (ciString "null" $> IsNull)
<|> try (ciString "not_null" $> IsNotNull)
<|> try (ciString "true" $> IsTriTrue)
<|> try (ciString "false" $> IsTriFalse)
<|> try (ciString "unknown" $> IsTriUnknown)
<?> "isVal: (null, not_null, true, false, unknown)"
pTriVal = try (ciString "null" $> TriNull)
<|> try (ciString "unknown" $> TriUnknown)
<|> try (ciString "true" $> TriTrue)
<|> try (ciString "false" $> TriFalse)
<?> "null or trilean value (unknown, true, false)"
pFts = do
op <- try (string "fts" $> FilterFts)
+64 -12
View File
@@ -5,6 +5,7 @@ module PostgREST.ApiRequest.Types
, Cast
, Depth
, EmbedParam(..)
, ApiRequestError(..)
, EmbedPath
, Field
, Filter(..)
@@ -13,7 +14,6 @@ module PostgREST.ApiRequest.Types
, JsonOperand(..)
, JsonOperation(..)
, JsonPath
, Language
, ListVal
, LogicOperator(..)
, LogicTree(..)
@@ -24,15 +24,23 @@ module PostgREST.ApiRequest.Types
, OrderDirection(..)
, OrderNulls(..)
, OrderTerm(..)
, QPError(..)
, RaiseError(..)
, RangeError(..)
, SingleVal
, IsVal(..)
, TrileanVal(..)
, SimpleOperator(..)
, QuantOperator(..)
, FtsOperator(..)
, SelectItem(..)
) where
import PostgREST.SchemaCache.Identifiers (FieldName)
import PostgREST.MediaType (MediaType (..))
import PostgREST.SchemaCache.Identifiers (FieldName,
QualifiedIdentifier)
import PostgREST.SchemaCache.Relationship (Relationship,
RelationshipsMap)
import PostgREST.SchemaCache.Routine (Routine (..))
import Protolude
@@ -60,6 +68,51 @@ data SelectItem
}
deriving (Eq, Show)
data ApiRequestError
= AggregatesNotAllowed
| AmbiguousRelBetween Text Text [Relationship]
| AmbiguousRpc [Routine]
| MediaTypeError [ByteString]
| InvalidBody ByteString
| InvalidFilters
| InvalidPreferences [ByteString]
| InvalidRange RangeError
| InvalidRpcMethod ByteString
| LimitNoOrderError
| NotFound
| NoRelBetween Text Text (Maybe Text) Text RelationshipsMap
| NoRpc Text Text [Text] Bool MediaType Bool [QualifiedIdentifier] [Routine]
| NotEmbedded Text
| PutLimitNotAllowedError
| QueryParamError QPError
| RelatedOrderNotToOne Text Text
| SpreadNotToOne Text Text
| UnacceptableFilter Text
| UnacceptableSchema [Text]
| UnsupportedMethod ByteString
| ColumnNotFound Text Text
| GucHeadersError
| GucStatusError
| OffLimitsChangesError Int64 Integer
| PutMatchingPkError
| SingularityError Integer
| PGRSTParseError RaiseError
| MaxAffectedViolationError Integer
deriving Show
data QPError = QPError Text Text
deriving Show
data RaiseError
= MsgParseError ByteString
| DetParseError ByteString
| NoDetail
deriving Show
data RangeError
= NegativeLimit
| LowerGTUpper
| OutOfBounds Text Text
deriving Show
type NodeName = Text
type Depth = Integer
@@ -122,7 +175,7 @@ data JsonOperation
deriving (Eq, Show, Ord)
-- | Represents the key(`->'key'`) or index(`->'1`::int`), the index is Text
-- because we reuse our escaping functions and let pg do the casting with
-- because we reuse our escaping functons and let pg do the casting with
-- '1'::int
data JsonOperand
= JKey { jVal :: Text }
@@ -165,7 +218,7 @@ data Operation
= Op SimpleOperator SingleVal
| OpQuant QuantOperator (Maybe OpQuantifier) SingleVal
| In ListVal
| Is IsVal
| Is TrileanVal
| IsDistinctFrom SingleVal
| Fts FtsOperator (Maybe Language) SingleVal
deriving (Eq, Show)
@@ -178,13 +231,12 @@ type SingleVal = Text
-- | Represents a list value in a filter, e.g. id=in.(val1,val2,val3)
type ListVal = [Text]
data IsVal
= IsNull
| IsNotNull
-- Trilean values
| IsTriTrue
| IsTriFalse
| IsTriUnknown
-- | Three-valued logic values
data TrileanVal
= TriTrue
| TriFalse
| TriNull
| TriUnknown
deriving (Eq, Show)
-- Operators that are quantifiable, i.e. they can be used with the any/all modifiers
+9 -27
View File
@@ -17,7 +17,7 @@ module PostgREST.App
import Control.Monad.Except (liftEither)
import Data.Either.Combinators (mapLeft, whenLeft)
import Data.Either.Combinators (mapLeft)
import Data.Maybe (fromJust)
import Data.String (IsString (..))
import Network.Wai.Handler.Warp (defaultSettings, setHost, setPort,
@@ -42,12 +42,10 @@ import qualified PostgREST.Unix as Unix (installSignalHandlers)
import PostgREST.ApiRequest (ApiRequest (..))
import PostgREST.AppState (AppState)
import PostgREST.Auth.Types (AuthResult (..))
import PostgREST.Config (AppConfig (..), LogLevel (..),
LogQuery (..))
import PostgREST.Auth (AuthResult (..))
import PostgREST.Config (AppConfig (..), LogLevel (..))
import PostgREST.Config.PgVersion (PgVersion (..))
import PostgREST.Error (Error)
import PostgREST.Network (resolveHost)
import PostgREST.Observation (Observation (..))
import PostgREST.Response.Performance (ServerTiming (..),
serverTimingHeader)
@@ -68,6 +66,8 @@ run appState = do
let observer = AppState.getObserver appState
conf@AppConfig{..} <- AppState.getConfig appState
observer $ AppStartObs prettyVersion
AppState.schemaCacheLoader appState -- Loads the initial SchemaCache
Unix.installSignalHandlers (AppState.getMainThreadId appState) (AppState.schemaCacheLoader appState) (AppState.readInDbConfig False appState)
@@ -82,8 +82,7 @@ run appState = do
observer $ AppServerUnixObs path
Nothing -> do
port <- NS.socketPort $ AppState.getSocketREST appState
host <- resolveHost $ AppState.getSocketREST appState
observer $ AppServerPortObs (fromJust host) port
observer $ AppServerPortObs port
Warp.runSettingsSocket (serverSettings conf) (AppState.getSocketREST appState) app
@@ -144,28 +143,11 @@ postgrestResponse appState conf@AppConfig{..} maybeSchemaCache pgVer authResult@
body <- lift $ Wai.strictRequestBody req
let jwtTime = if configServerTimingEnabled then Auth.getJwtDur req else Nothing
timezones = dbTimezones sCache
prefs = ApiRequest.userPreferences conf req timezones
(parseTime, apiReq@ApiRequest{..}) <- withTiming $ liftEither . mapLeft Error.ApiRequestError $ ApiRequest.userApiRequest conf prefs req body
(parseTime, apiReq@ApiRequest{..}) <- withTiming $ liftEither . mapLeft Error.ApiRequestError $ ApiRequest.userApiRequest conf req body sCache
(planTime, plan) <- withTiming $ liftEither $ Plan.actionPlan iAction conf apiReq sCache
let query = Query.query conf authResult apiReq plan sCache pgVer
logSQL = lift . AppState.getObserver appState . DBQuery (Query.getSQLQuery query)
(queryTime, queryResult) <- withTiming $ do
case query of
Query.NoDbQuery r -> pure r
Query.DbQuery{..} -> do
dbRes <- lift $ AppState.usePool appState (dqTransaction dqIsoLevel dqTxMode $ runExceptT dqDbHandler)
let eitherResp = mapLeft Error.PgErr . mapLeft (Error.PgError (Just authRole /= configDbAnonRole)) $ dbRes
when (configLogQuery /= LogQueryDisabled) $ whenLeft eitherResp $ logSQL . Error.status
liftEither eitherResp >>= liftEither
(respTime, resp) <- withTiming $ do
let response = Response.actionResponse queryResult apiReq (T.decodeUtf8 prettyVersion, docsVersion) conf sCache iSchema iNegotiatedByProfile
when (configLogQuery /= LogQueryDisabled) $ logSQL $ either Error.status Response.pgrstStatus response
liftEither response
(queryTime, queryResult) <- withTiming $ Query.runQuery appState conf authResult apiReq plan sCache pgVer (Just authRole /= configDbAnonRole)
(respTime, resp) <- withTiming $ liftEither $ Response.actionResponse queryResult apiReq (T.decodeUtf8 prettyVersion, docsVersion) conf sCache iSchema iNegotiatedByProfile
return $ toWaiResponse (ServerTiming jwtTime parseTime planTime queryTime respTime) resp
+42 -49
View File
@@ -4,6 +4,7 @@
module PostgREST.AppState
( AppState
, AuthResult(..)
, destroy
, getConfig
, getSchemaCache
@@ -12,7 +13,7 @@ module PostgREST.AppState
, getNextDelay
, getNextListenerDelay
, getTime
, getJwtCacheState
, getJwtCache
, getSocketREST
, getSocketAdmin
, init
@@ -30,7 +31,10 @@ module PostgREST.AppState
, isPending
) where
import qualified Data.Aeson as JSON
import qualified Data.Aeson.KeyMap as KM
import qualified Data.ByteString.Char8 as BS
import qualified Data.Cache as C
import Data.Either.Combinators (whenLeft)
import qualified Data.Text as T (unpack)
import qualified Hasql.Pool as SQL
@@ -39,7 +43,6 @@ import qualified Hasql.Session as SQL
import qualified Hasql.Transaction.Sessions as SQL
import qualified Network.HTTP.Types.Status as HTTP
import qualified Network.Socket as NS
import qualified PostgREST.Auth.JwtCache as JwtCache
import qualified PostgREST.Error as Error
import qualified PostgREST.Logger as Logger
import qualified PostgREST.Metrics as Metrics
@@ -57,7 +60,6 @@ import Data.IORef (IORef, atomicWriteIORef, newIORef,
readIORef)
import Data.Time.Clock (UTCTime, getCurrentTime)
import PostgREST.Auth.JwtCache (JwtCacheState)
import PostgREST.Config (AppConfig (..),
addFallbackAppName,
readAppConfig)
@@ -76,6 +78,11 @@ import Data.Streaming.Network (bindPortTCP, bindRandomPortTCP)
import Data.String (IsString (..))
import Protolude
data AuthResult = AuthResult
{ authClaims :: KM.KeyMap JSON.Value
, authRole :: BS.ByteString
}
data AppState = AppState
-- | Database connection pool
{ statePool :: SQL.Pool
@@ -99,14 +106,14 @@ data AppState = AppState
, stateNextDelay :: IORef Int
-- | Keeps track of the next delay for the listener
, stateNextListenerDelay :: IORef Int
-- | JWT Cache
, jwtCache :: C.Cache ByteString AuthResult
-- | Network socket for REST API
, stateSocketREST :: NS.Socket
-- | Network socket for the admin UI
, stateSocketAdmin :: Maybe NS.Socket
-- | Observation handler
, stateObserver :: ObservationHandler
-- | JWT Cache
, stateJwtCache :: JwtCache.JwtCacheState
, stateLogger :: Logger.LoggerState
, stateMetrics :: Metrics.MetricsState
}
@@ -125,16 +132,13 @@ init conf@AppConfig{configLogLevel, configDbPoolSize} = do
metricsState <- Metrics.init configDbPoolSize
let observer = liftA2 (>>) (Logger.observationLogger loggerState configLogLevel) (Metrics.observationMetrics metricsState)
observer $ AppStartObs prettyVersion
jwtCacheState <- JwtCache.init
pool <- initPool conf observer
(sock, adminSock) <- initSockets conf
state' <- initWithPool (sock, adminSock) pool conf jwtCacheState loggerState metricsState observer
state' <- initWithPool (sock, adminSock) pool conf loggerState metricsState observer
pure state' { stateSocketREST = sock, stateSocketAdmin = adminSock}
initWithPool :: AppSockets -> SQL.Pool -> AppConfig -> JwtCache.JwtCacheState -> Logger.LoggerState -> Metrics.MetricsState -> ObservationHandler -> IO AppState
initWithPool (sock, adminSock) pool conf jwtCacheState loggerState metricsState observer = do
initWithPool :: AppSockets -> SQL.Pool -> AppConfig -> Logger.LoggerState -> Metrics.MetricsState -> ObservationHandler -> IO AppState
initWithPool (sock, adminSock) pool conf loggerState metricsState observer = do
appState <- AppState pool
<$> newIORef minimumPgVersion -- assume we're in a supported version when starting, this will be corrected on a later step
@@ -147,17 +151,17 @@ initWithPool (sock, adminSock) pool conf jwtCacheState loggerState metricsState
<*> myThreadId
<*> newIORef 0
<*> newIORef 1
<*> C.newCache Nothing
<*> pure sock
<*> pure adminSock
<*> pure observer
<*> pure jwtCacheState
<*> pure loggerState
<*> pure metricsState
deb <-
let decisecond = 100000 in
mkDebounce defaultDebounceSettings
{ debounceAction = retryingSchemaCacheLoad appState
{ debounceAction = internalSchemaCacheLoad appState
, debounceFreq = decisecond
, debounceEdge = leadingEdge -- runs the worker at the start and the end
}
@@ -174,8 +178,7 @@ initSockets AppConfig{..} = do
cfg'uspm = configServerUnixSocketMode
cfg'host = configServerHost
cfg'port = configServerPort
cfg'adminHost = configAdminServerHost
cfg'adminPort = configAdminServerPort
cfg'adminport = configAdminServerPort
sock <- case cfg'usp of
-- I'm not using `streaming-commons`' bindPath function here because it's not defined for Windows,
@@ -193,16 +196,16 @@ initSockets AppConfig{..} = do
pure (num, sock)
pure sock
adminSock <- case cfg'adminPort of
adminSock <- case cfg'adminport of
Just adminPort -> do
adminSock <- bindPortTCP adminPort (fromString $ T.unpack cfg'adminHost)
adminSock <- bindPortTCP adminPort (fromString $ T.unpack cfg'host)
pure $ Just adminSock
Nothing -> pure Nothing
pure (sock, adminSock)
initPool :: AppConfig -> ObservationHandler -> IO SQL.Pool
initPool AppConfig{..} observer = do
initPool AppConfig{..} observer =
SQL.acquire $ SQL.settings
[ SQL.size configDbPoolSize
, SQL.acquisitionTimeout $ fromIntegral configDbPoolAcquisitionTimeout
@@ -262,9 +265,8 @@ usePool AppState{stateObserver=observer, stateMainThreadId=mainThreadId, ..} ses
SQL.ServerError{} ->
when (Error.status (Error.PgError False err) >= HTTP.status500) $
observer $ QueryErrorCodeHighObs err
err@(SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ClientError _))) ->
-- An error on the client-side, usually indicates problems wth connection
observer $ QueryErrorCodeHighObs err
SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ClientError _)) ->
pure ()
)
return res
@@ -311,8 +313,8 @@ putConfig = atomicWriteIORef . stateConf
getTime :: AppState -> IO UTCTime
getTime = stateGetTime
getJwtCacheState :: AppState -> JwtCacheState
getJwtCacheState = stateJwtCache
getJwtCache :: AppState -> C.Cache ByteString AuthResult
getJwtCache = jwtCache
getSocketREST :: AppState -> NS.Socket
getSocketREST = stateSocketREST
@@ -352,6 +354,13 @@ putSCacheStatus = atomicWriteIORef . stateSCacheStatus
getObserver :: AppState -> ObservationHandler
getObserver = stateObserver
internalSchemaCacheLoad :: AppState -> IO ()
internalSchemaCacheLoad appState = do
AppConfig{..} <- getConfig appState
void $ retryingSchemaCacheLoad appState
-- We cannot retry reading the in-db config after it fails immediately, because it could have user errors. We just report the error and continue.
when configDbConfig $ readInDbConfig False appState
-- | Try to load the schema cache and retry if it fails.
--
-- This is done by repeatedly: 1) flushing the pool, 2) querying the version and validating that the postgres version is supported by us, and 3) loading the schema cache.
@@ -359,17 +368,16 @@ getObserver = stateObserver
--
-- + Because connections cache the pg catalog(see #2620)
-- + For rapid recovery. Otherwise, the pool idle or lifetime timeout would have to be reached for new healthy connections to be acquired.
retryingSchemaCacheLoad :: AppState -> IO ()
retryingSchemaCacheLoad :: AppState -> IO (Maybe PgVersion, Maybe SchemaCache)
retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThreadId=mainThreadId} =
void $ retrying retryPolicy shouldRetry (\RetryStatus{rsIterNumber, rsPreviousDelay} -> do
retrying retryPolicy shouldRetry (\RetryStatus{rsIterNumber, rsPreviousDelay} -> do
when (rsIterNumber > 0) $ do
let delay = fromMaybe 0 rsPreviousDelay `div` oneSecondInUs
observer $ ConnectionRetryObs delay
putNextListenerDelay appState delay
flushPool appState
(,) <$> qPgVersion <*> (qInDbConfig *> qSchemaCache)
(,) <$> qPgVersion <*> qSchemaCache
)
where
qPgVersion :: IO (Maybe PgVersion)
@@ -388,15 +396,9 @@ retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThrea
observer $ ExitUnsupportedPgVersion actualPgVersion minimumPgVersion
killThread mainThreadId
observer $ DBConnectedObs $ pgvFullName actualPgVersion
observer $ PoolInit configDbPoolSize
putPgVersion appState actualPgVersion
return $ Just actualPgVersion
qInDbConfig :: IO ()
qInDbConfig = do
AppConfig{..} <- getConfig appState
when configDbConfig $ readInDbConfig False appState
qSchemaCache :: IO (Maybe SchemaCache)
qSchemaCache = do
conf@AppConfig{..} <- getConfig appState
@@ -407,7 +409,7 @@ retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThrea
Left e -> do
putSCacheStatus appState SCPending
putSchemaCache appState Nothing
observer $ SchemaCacheErrorObs configDbSchemas configDbExtraSearchPath e
observer $ SchemaCacheErrorObs e
return Nothing
Right sCache -> do
@@ -435,14 +437,13 @@ retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThrea
oneSecondInUs = 1000000 -- one second in microseconds
-- | Reads the in-db config and reads the config file again
-- | We don't retry reading the in-db config after it fails immediately, because it could have user errors. We just report the error and continue.
readInDbConfig :: Bool -> AppState -> IO ()
readInDbConfig startingUp appState@AppState{stateObserver=observer} = do
conf <- getConfig appState
AppConfig{..} <- getConfig appState
pgVer <- getPgVersion appState
dbSettings <-
if configDbConfig conf then do
qDbSettings <- usePool appState (queryDbSettings (dumpQi <$> configDbPreConfig conf) (configDbPreparedStatements conf))
if configDbConfig then do
qDbSettings <- usePool appState (queryDbSettings (dumpQi <$> configDbPreConfig) configDbPreparedStatements)
case qDbSettings of
Left e -> do
observer $ ConfigReadErrorObs e
@@ -451,8 +452,8 @@ readInDbConfig startingUp appState@AppState{stateObserver=observer} = do
else
pure mempty
(roleSettings, roleIsolationLvl) <-
if configDbConfig conf then do
rSettings <- usePool appState (queryRoleSettings pgVer (configDbPreparedStatements conf))
if configDbConfig then do
rSettings <- usePool appState (queryRoleSettings pgVer configDbPreparedStatements)
case rSettings of
Left e -> do
observer $ QueryRoleSettingsErrorObs e
@@ -460,7 +461,7 @@ readInDbConfig startingUp appState@AppState{stateObserver=observer} = do
Right x -> pure x
else
pure mempty
readAppConfig dbSettings (configFilePath conf) (Just $ configDbUri conf) roleSettings roleIsolationLvl >>= \case
readAppConfig dbSettings configFilePath (Just configDbUri) roleSettings roleIsolationLvl >>= \case
Left err ->
if startingUp then
panic err -- die on invalid config if the program is starting up
@@ -468,14 +469,6 @@ readInDbConfig startingUp appState@AppState{stateObserver=observer} = do
observer $ ConfigInvalidObs err
Right newConf -> do
putConfig appState newConf
-- After the config has reloaded, jwt-secret might have changed, so
-- if it has changed, it is important to invalidate the jwt cache
-- entries, because they were cached using the old secret
if configJwtSecret conf == configJwtSecret newConf then
pass
else
JwtCache.emptyCache (getJwtCacheState appState) -- atomic O(1) operation
if startingUp then
pass
else
+55 -98
View File
@@ -10,123 +10,75 @@ Authentication should always be implemented in an external service.
In the test suite there is an example of simple login function that can be used for a
very simple authentication system inside the PostgreSQL database.
-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE RecordWildCards #-}
module PostgREST.Auth
( getResult
( AuthResult (..)
, getResult
, getJwtDur
, getRole
, middleware
) where
import qualified Crypto.JWT as JWT
import qualified Data.Aeson as JSON
import qualified Data.Aeson.Key as K
import qualified Data.Aeson.KeyMap as KM
import qualified Data.Aeson.Types as JSON
import qualified Data.ByteString as BS
import qualified Data.ByteString.Internal as BS
import qualified Data.ByteString.Lazy.Char8 as LBS
import qualified Data.Cache as C
import qualified Data.Scientific as Sci
import qualified Data.Text as T
import qualified Data.Vault.Lazy as Vault
import qualified Data.Vector as V
import qualified Jose.Jwk as JWT
import qualified Jose.Jwt as JWT
import qualified Network.HTTP.Types.Header as HTTP
import qualified Network.Wai as Wai
import qualified Network.Wai.Middleware.HttpAuth as Wai
import Control.Lens (set)
import Control.Monad.Except (liftEither)
import Data.Either.Combinators (mapLeft)
import Data.List (lookup)
import Data.Time.Clock (UTCTime, nominalDiffTimeToSeconds)
import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds)
import System.Clock (TimeSpec (..))
import System.IO.Unsafe (unsafePerformIO)
import System.TimeIt (timeItT)
import PostgREST.AppState (AppState, getConfig, getJwtCacheState,
getTime)
import PostgREST.Auth.JwtCache (lookupJwtCache)
import PostgREST.Auth.Types (AuthResult (..))
import PostgREST.Config (AppConfig (..), FilterExp (..),
JSPath, JSPathExp (..))
import PostgREST.Error (Error (..), JwtError (..))
import PostgREST.AppState (AppState, AuthResult (..), getConfig,
getJwtCache, getTime)
import PostgREST.Config (AppConfig (..), JSPath, JSPathExp (..))
import PostgREST.Error (Error (..))
import Protolude
-- | Receives the JWT secret and audience (from config) and a JWT and returns a
-- JSON object of JWT claims.
parseToken :: AppConfig -> Maybe ByteString -> UTCTime -> ExceptT Error IO JSON.Value
parseToken _ Nothing _ = return JSON.emptyObject
parseToken _ (Just "") _ = throwE . JwtErr $ JwtDecodeError "Empty JWT is sent in Authorization header"
parseToken AppConfig{..} (Just tkn) time = do
secret <- liftEither . maybeToRight (JwtErr JwtSecretMissing) $ configJWKS
tknWith3Parts <- liftEither $ hasThreeParts tkn
eitherContent <- liftIO $ JWT.decode (JWT.keys secret) Nothing tknWith3Parts
content <- liftEither . mapLeft (JwtErr . jwtDecodeError) $ eitherContent
liftEither $ mapLeft JwtErr $ verifyClaims content
parseToken :: Monad m =>
AppConfig -> LByteString -> UTCTime -> ExceptT Error m JSON.Value
parseToken _ "" _ = return JSON.emptyObject
parseToken AppConfig{..} token time = do
secret <- liftEither . maybeToRight JwtTokenMissing $ configJWKS
eitherClaims <-
lift . runExceptT $
JWT.verifyClaimsAt validation secret time =<< JWT.decodeCompact token
liftEither . mapLeft jwtClaimsError $ JSON.toJSON <$> eitherClaims
where
hasThreeParts :: ByteString -> Either Error ByteString
hasThreeParts token = case length $ BS.split (BS.c2w '.') token of
3 -> Right token
n -> Left $ JwtErr $ JwtDecodeError ("Expected 3 parts in JWT; got " <> show n)
jwtDecodeError :: JWT.JwtError -> JwtError
-- The only errors we can get from JWT.decode function are:
-- BadAlgorithm
-- KeyError
-- BadCrypto
jwtDecodeError (JWT.KeyError _) = JwtDecodeError "No suitable key or wrong key type"
jwtDecodeError (JWT.BadAlgorithm _) = JwtDecodeError "Wrong or unsupported encoding algorithm"
jwtDecodeError JWT.BadCrypto = JwtDecodeError "JWT cryptographic operation failed"
-- Control never reaches here, the decode function only returns the above three
jwtDecodeError _ = JwtDecodeError "JWT couldn't be decoded"
validation =
JWT.defaultJWTValidationSettings audienceCheck & set JWT.allowedSkew 30
verifyClaims :: JWT.JwtContent -> Either JwtError JSON.Value
verifyClaims (JWT.Jws (_, claims)) = case JSON.decodeStrict claims of
Just jclaims@(JSON.Object mclaims) ->
verifyClaim mclaims "exp" isValidExpClaim "JWT expired" >>
verifyClaim mclaims "nbf" isValidNbfClaim "JWT not yet valid" >>
verifyClaim mclaims "iat" isValidIatClaim "JWT issued at future" >>
verifyClaim mclaims "aud" isValidAudClaim "JWT not in audience" >>
return jclaims
_ -> Left $ JwtClaimsError "Parsing claims failed"
-- TODO: We could enable JWE support here (encrypted tokens)
verifyClaims _ = Left $ JwtDecodeError "Unsupported token type"
audienceCheck :: JWT.StringOrURI -> Bool
audienceCheck = maybe (const True) (==) configJwtAudience
verifyClaim mclaims claim func err = do
isValid <- maybe (Right True) func (KM.lookup claim mclaims)
unless isValid $ Left $ JwtClaimsError err
allowedSkewSeconds = 30 :: Int64
now = floor . nominalDiffTimeToSeconds $ utcTimeToPOSIXSeconds time
sciToInt = fromMaybe 0 . Sci.toBoundedInteger
allStrings = all (\case (JSON.String _) -> True; _ -> False)
isValidExpClaim :: JSON.Value -> Either JwtError Bool
isValidExpClaim (JSON.Number secs) = Right $ now <= (sciToInt secs + allowedSkewSeconds)
isValidExpClaim _ = Left $ JwtClaimsError "The JWT 'exp' claim must be a number"
isValidNbfClaim :: JSON.Value -> Either JwtError Bool
isValidNbfClaim (JSON.Number secs) = Right $ now >= (sciToInt secs - allowedSkewSeconds)
isValidNbfClaim _ = Left $ JwtClaimsError "The JWT 'nbf' claim must be a number"
isValidIatClaim :: JSON.Value -> Either JwtError Bool
isValidIatClaim (JSON.Number secs) = Right $ now >= (sciToInt secs - allowedSkewSeconds)
isValidIatClaim _ = Left $ JwtClaimsError "The JWT 'iat' claim must be a number"
isValidAudClaim :: JSON.Value -> Either JwtError Bool
isValidAudClaim JSON.Null = Right True -- {"aud": null} is valid for all audiences
isValidAudClaim (JSON.String str) = Right $ maybe (const True) (==) configJwtAudience str
isValidAudClaim (JSON.Array arr)
| null arr = Right True -- {"aud": []} is valid for all audiences
| allStrings arr = Right $ maybe True (\a -> JSON.String a `elem` arr) configJwtAudience
isValidAudClaim _ = Left $ JwtClaimsError "The JWT 'aud' claim must be a string or an array of strings"
jwtClaimsError :: JWT.JWTError -> Error
jwtClaimsError JWT.JWTExpired = JwtTokenInvalid "JWT expired"
jwtClaimsError e = JwtTokenInvalid $ show e
parseClaims :: Monad m =>
AppConfig -> JSON.Value -> ExceptT Error m AuthResult
parseClaims AppConfig{..} jclaims@(JSON.Object mclaims) = do
-- role defaults to anon if not specified in jwt
role <- liftEither . maybeToRight (JwtErr JwtTokenRequired) $
role <- liftEither . maybeToRight JwtTokenRequired $
unquoted <$> walkJSPath (Just jclaims) configJwtRoleClaimKey <|> configDbAnonRole
return AuthResult
{ authClaims = mclaims & KM.insert "role" (JSON.toJSON $ decodeUtf8 role)
@@ -137,20 +89,8 @@ parseClaims AppConfig{..} jclaims@(JSON.Object mclaims) = do
walkJSPath x [] = x
walkJSPath (Just (JSON.Object o)) (JSPKey key:rest) = walkJSPath (KM.lookup (K.fromText key) o) rest
walkJSPath (Just (JSON.Array ar)) (JSPIdx idx:rest) = walkJSPath (ar V.!? idx) rest
walkJSPath (Just (JSON.Array ar)) [JSPFilter (EqualsCond txt)] = findFirstMatch (==) txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (NotEqualsCond txt)] = findFirstMatch (/=) txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (StartsWithCond txt)] = findFirstMatch T.isPrefixOf txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (EndsWithCond txt)] = findFirstMatch T.isSuffixOf txt ar
walkJSPath (Just (JSON.Array ar)) [JSPFilter (ContainsCond txt)] = findFirstMatch T.isInfixOf txt ar
walkJSPath _ _ = Nothing
findFirstMatch matchWith pattern = foldr checkMatch Nothing
where
checkMatch (JSON.String txt) acc
| pattern `matchWith` txt = Just $ JSON.String txt
| otherwise = acc
checkMatch _ acc = acc
unquoted :: JSON.Value -> BS.ByteString
unquoted (JSON.String t) = encodeUtf8 t
unquoted v = LBS.toStrict $ JSON.encode v
@@ -164,11 +104,10 @@ middleware appState app req respond = do
conf <- getConfig appState
time <- getTime appState
let token = Wai.extractBearerAuth =<< lookup HTTP.hAuthorization (Wai.requestHeaders req)
parseJwt = runExceptT $ parseToken conf token time >>= parseClaims conf
jwtCacheState = getJwtCacheState appState
let token = fromMaybe "" $ Wai.extractBearerAuth =<< lookup HTTP.hAuthorization (Wai.requestHeaders req)
parseJwt = runExceptT $ parseToken conf (LBS.fromStrict token) time >>= parseClaims conf
-- If ServerTimingEnabled -> calculate JWT validation time
-- If DbPlanEnabled -> calculate JWT validation time
-- If JwtCacheMaxLifetime -> cache JWT validation result
req' <- case (configServerTimingEnabled conf, configJwtCacheMaxLifetime conf) of
(True, 0) -> do
@@ -176,9 +115,7 @@ middleware appState app req respond = do
return $ req { Wai.vault = Wai.vault req & Vault.insert authResultKey authResult & Vault.insert jwtDurKey dur }
(True, maxLifetime) -> do
(dur, authResult) <- timeItT $ case token of
Just tkn -> lookupJwtCache jwtCacheState tkn maxLifetime parseJwt time
Nothing -> parseJwt
(dur, authResult) <- timeItT $ getJWTFromCache appState token maxLifetime parseJwt time
return $ req { Wai.vault = Wai.vault req & Vault.insert authResultKey authResult & Vault.insert jwtDurKey dur }
(False, 0) -> do
@@ -186,13 +123,33 @@ middleware appState app req respond = do
return $ req { Wai.vault = Wai.vault req & Vault.insert authResultKey authResult }
(False, maxLifetime) -> do
authResult <- case token of
Just tkn -> lookupJwtCache jwtCacheState tkn maxLifetime parseJwt time
Nothing -> parseJwt
authResult <- getJWTFromCache appState token maxLifetime parseJwt time
return $ req { Wai.vault = Wai.vault req & Vault.insert authResultKey authResult }
app req' respond
-- | Used to retrieve and insert JWT to JWT Cache
getJWTFromCache :: AppState -> ByteString -> Int -> IO (Either Error AuthResult) -> UTCTime -> IO (Either Error AuthResult)
getJWTFromCache appState token maxLifetime parseJwt utc = do
checkCache <- C.lookup (getJwtCache appState) token
authResult <- maybe parseJwt (pure . Right) checkCache
case (authResult,checkCache) of
(Right res, Nothing) -> C.insert' (getJwtCache appState) (getTimeSpec res maxLifetime utc) token res
_ -> pure ()
return authResult
-- Used to extract JWT exp claim and add to JWT Cache
getTimeSpec :: AuthResult -> Int -> UTCTime -> Maybe TimeSpec
getTimeSpec res maxLifetime utc = do
let expireJSON = KM.lookup "exp" (authClaims res)
utcToSecs = floor . nominalDiffTimeToSeconds . utcTimeToPOSIXSeconds
sciToInt = fromMaybe 0 . Sci.toBoundedInteger
case expireJSON of
Just (JSON.Number seconds) -> Just $ TimeSpec (sciToInt seconds - utcToSecs utc) 0
_ -> Just $ TimeSpec (fromIntegral maxLifetime :: Int64) 0
authResultKey :: Vault.Key (Either Error AuthResult)
authResultKey = unsafePerformIO Vault.newKey
{-# NOINLINE authResultKey #-}
-99
View File
@@ -1,99 +0,0 @@
{-|
Module : PostgREST.Auth.JwtCache
Description : PostgREST Jwt Authentication Result Cache.
This module provides functions to deal with the JWT cache
-}
{-# LANGUAGE NamedFieldPuns #-}
module PostgREST.Auth.JwtCache
( init
, JwtCacheState
, lookupJwtCache
, emptyCache
) where
import qualified Data.Aeson as JSON
import qualified Data.Aeson.KeyMap as KM
import qualified Data.Cache as C
import qualified Data.Scientific as Sci
import Control.Debounce
import Data.Time.Clock (UTCTime, nominalDiffTimeToSeconds)
import Data.Time.Clock.POSIX (utcTimeToPOSIXSeconds)
import System.Clock (TimeSpec (..))
import PostgREST.Auth.Types (AuthResult (..))
import PostgREST.Error (Error (..))
import Protolude
-- | JWT Cache and IO action that triggers purging old entries from the cache
data JwtCacheState = JwtCacheState
{ jwtCache :: C.Cache ByteString AuthResult
, purgeCache :: IO ()
}
-- | Initialize JwtCacheState
init :: IO JwtCacheState
init = do
cache <- C.newCache Nothing -- no default expiration
-- purgeExpired has O(n^2) complexity
-- so we wrap it in debounce to make sure it:
-- 1) is executed asynchronously
-- 2) only a single purge operation is running at a time
debounce <- mkDebounce defaultDebounceSettings
-- debounceFreq is set to default 1 second
{ debounceAction = C.purgeExpired cache
, debounceEdge = leadingEdge
}
pure $ JwtCacheState cache debounce
-- | Used to retrieve and insert JWT to JWT Cache
lookupJwtCache :: JwtCacheState -> ByteString -> Int -> IO (Either Error AuthResult) -> UTCTime -> IO (Either Error AuthResult)
lookupJwtCache JwtCacheState{jwtCache, purgeCache} token maxLifetime parseJwt utc = do
checkCache <- C.lookup jwtCache token
authResult <- maybe parseJwt (pure . Right) checkCache
case (authResult,checkCache) of
-- From comment:
-- https://github.com/PostgREST/postgrest/pull/3801#discussion_r1857987914
--
-- We purge expired cache entries on a cache miss
-- The reasoning is that:
--
-- 1. We expect it to be rare (otherwise there is no point of the cache)
-- 2. It makes sure the cache is not growing (as inserting new entries
-- does garbage collection)
-- 3. Since this is time expiration based cache there is no real risk of
-- starvation - sooner or later we are going to have a cache miss.
(Right res, Nothing) -> do -- cache miss
let timeSpec = getTimeSpec res maxLifetime utc
-- insert new cache entry
C.insert' jwtCache (Just timeSpec) token res
-- Execute IO action to purge the cache
-- It is assumed this action returns immidiately
-- so that request processing is not blocked.
purgeCache
_ -> pure ()
return authResult
-- Used to extract JWT exp claim and add to JWT Cache
getTimeSpec :: AuthResult -> Int -> UTCTime -> TimeSpec
getTimeSpec res maxLifetime utc = do
let expireJSON = KM.lookup "exp" (authClaims res)
utcToSecs = floor . nominalDiffTimeToSeconds . utcTimeToPOSIXSeconds
sciToInt = fromMaybe 0 . Sci.toBoundedInteger
case expireJSON of
Just (JSON.Number seconds) -> TimeSpec (sciToInt seconds - utcToSecs utc) 0
_ -> TimeSpec (fromIntegral maxLifetime :: Int64) 0
-- | Empty the cache (done when the config is reloaded)
emptyCache :: JwtCacheState -> IO ()
emptyCache JwtCacheState{jwtCache} = C.purge jwtCache
-13
View File
@@ -1,13 +0,0 @@
module PostgREST.Auth.Types
( AuthResult (..) )
where
import qualified Data.Aeson as JSON
import qualified Data.Aeson.KeyMap as KM
import qualified Data.ByteString as BS
-- | Parse result for JWT Claims
data AuthResult = AuthResult
{ authClaims :: KM.KeyMap JSON.Value
, authRole :: BS.ByteString
}
+1 -4
View File
@@ -60,7 +60,7 @@ dumpSchema appState = do
case result of
Left e -> do
let observer = AppState.getObserver appState
observer $ SchemaCacheErrorObs configDbSchemas configDbExtraSearchPath e
observer $ SchemaCacheErrorObs e
exitFailure
Right sCache -> return $ JSON.encode sCache
@@ -209,9 +209,6 @@ exampleConfigFile =
|## Logging level, the admitted values are: crit, error, warn, info and debug.
|log-level = "error"
|
|## Log the requested SQL query at the current log-level.
|log-query = "disabled"
|
|## Determine if the OpenAPI output should follow or ignore role privileges or be disabled entirely.
|## Admitted values: follow-privileges, ignore-privileges, disabled
|openapi-mode = "follow-privileges"
+50 -104
View File
@@ -15,9 +15,7 @@ module PostgREST.Config
, Environment
, JSPath
, JSPathExp(..)
, FilterExp(..)
, LogLevel(..)
, LogQuery(..)
, OpenAPIMode(..)
, Proxy(..)
, toText
@@ -30,25 +28,28 @@ module PostgREST.Config
, addTargetSessionAttrs
) where
import qualified Crypto.JOSE.Types as JOSE
import qualified Crypto.JWT as JWT
import qualified Data.Aeson as JSON
import qualified Data.ByteString as BS
import qualified Data.ByteString.Base64 as B64
import qualified Data.ByteString.Lazy as LBS
import qualified Data.CaseInsensitive as CI
import qualified Data.Configurator as C
import qualified Data.Map.Strict as M
import qualified Data.Text as T
import qualified Data.Text.Encoding as T
import qualified Jose.Jwa as JWT
import qualified Jose.Jwk as JWT
import Control.Lens (preview)
import Control.Monad (fail)
import Crypto.JWT (JWK, JWKSet, StringOrURI, stringOrUri)
import Data.Aeson (toJSON)
import Data.Either.Combinators (mapLeft)
import Data.List (lookup)
import Data.List.NonEmpty (fromList, toList)
import Data.Maybe (fromJust)
import Data.Scientific (floatingOrInteger)
import Jose.Jwk (Jwk, JwkSet)
import Network.URI (escapeURIString, isURI,
import Network.URI (escapeURIString,
isUnescapedInURIComponent)
import Numeric (readOct, showOct)
import System.Environment (getEnvironment)
@@ -56,9 +57,8 @@ import System.Posix.Types (FileMode)
import PostgREST.Config.Database (RoleIsolationLvl,
RoleSettings)
import PostgREST.Config.JSPath (FilterExp (..), JSPath,
JSPathExp (..), dumpJSPath,
pRoleClaimKey)
import PostgREST.Config.JSPath (JSPath, JSPathExp (..),
dumpJSPath, pRoleClaimKey)
import PostgREST.Config.Proxy (Proxy (..),
isMalformedProxyUri, toURI)
import PostgREST.SchemaCache.Identifiers (QualifiedIdentifier, dumpQi,
@@ -92,14 +92,13 @@ data AppConfig = AppConfig
, configDbTxRollbackAll :: Bool
, configDbUri :: Text
, configFilePath :: Maybe FilePath
, configJWKS :: Maybe JwkSet
, configJwtAudience :: Maybe Text
, configJWKS :: Maybe JWKSet
, configJwtAudience :: Maybe StringOrURI
, configJwtRoleClaimKey :: JSPath
, configJwtSecret :: Maybe BS.ByteString
, configJwtSecretIsBase64 :: Bool
, configJwtCacheMaxLifetime :: Int
, configLogLevel :: LogLevel
, configLogQuery :: LogQuery
, configOpenApiMode :: OpenAPIMode
, configOpenApiSecurityActive :: Bool
, configOpenApiServerProxyUri :: Maybe Text
@@ -110,7 +109,6 @@ data AppConfig = AppConfig
, configServerTimingEnabled :: Bool
, configServerUnixSocket :: Maybe FilePath
, configServerUnixSocketMode :: FileMode
, configAdminServerHost :: Text
, configAdminServerPort :: Maybe Int
, configRoleSettings :: RoleSettings
, configRoleIsoLvl :: RoleIsolationLvl
@@ -128,14 +126,6 @@ dumpLogLevel = \case
LogInfo -> "info"
LogDebug -> "debug"
data LogQuery = LogQueryMain | LogQueryDisabled
deriving (Eq)
dumpLogQuery :: LogQuery -> Text
dumpLogQuery = \case
LogQueryMain -> "main-query"
LogQueryDisabled -> "disabled"
data OpenAPIMode = OAFollowPriv | OAIgnorePriv | OADisabled
deriving Eq
@@ -173,13 +163,12 @@ toText conf =
,("db-pre-config", q . maybe mempty dumpQi . configDbPreConfig)
,("db-tx-end", q . showTxEnd)
,("db-uri", q . configDbUri)
,("jwt-aud", q . fromMaybe mempty . configJwtAudience)
,("jwt-aud", T.decodeUtf8 . LBS.toStrict . JSON.encode . maybe "" toJSON . configJwtAudience)
,("jwt-role-claim-key", q . T.intercalate mempty . fmap dumpJSPath . configJwtRoleClaimKey)
,("jwt-secret", q . T.decodeUtf8 . showJwtSecret)
,("jwt-secret-is-base64", T.toLower . show . configJwtSecretIsBase64)
,("jwt-cache-max-lifetime", show . configJwtCacheMaxLifetime)
,("log-level", q . dumpLogLevel . configLogLevel)
,("log-query", q . dumpLogQuery . configLogQuery)
,("openapi-mode", q . dumpOpenApiMode . configOpenApiMode)
,("openapi-security-active", T.toLower . show . configOpenApiSecurityActive)
,("openapi-server-proxy-uri", q . fromMaybe mempty . configOpenApiServerProxyUri)
@@ -190,7 +179,6 @@ toText conf =
,("server-timing-enabled", T.toLower . show . configServerTimingEnabled)
,("server-unix-socket", q . maybe mempty T.pack . configServerUnixSocket)
,("server-unix-socket-mode", q . T.pack . showSocketMode)
,("admin-server-host", q . configAdminServerHost)
,("admin-server-port", maybe "\"\"" show . configAdminServerPort)
]
@@ -235,18 +223,16 @@ readAppConfig dbSettings optPath prevDbUri roleSettings roleIsolationLvl = do
Left err ->
return . Left $ "Error in config " <> err
Right parsedConfig ->
mapLeft show <$> decodeLoadFiles parsedConfig
Right <$> decodeLoadFiles parsedConfig
where
-- Both C.ParseError and IOError are shown here
loadConfig :: FilePath -> IO (Either SomeException C.Config)
loadConfig = try . C.load
decodeLoadFiles :: AppConfig -> IO (Either IOException AppConfig)
decodeLoadFiles parsedConfig = try $
decodeJWKS =<<
decodeSecret =<<
readSecretFile =<<
readDbUriFile prevDbUri parsedConfig
decodeLoadFiles :: AppConfig -> IO AppConfig
decodeLoadFiles parsedConfig =
decodeJWKS <$>
(decodeSecret =<< readSecretFile =<< readDbUriFile prevDbUri parsedConfig)
parser :: Maybe FilePath -> Environment -> [(Text, Text)] -> RoleSettings -> RoleIsolationLvl -> C.Parser C.Config AppConfig
parser optPath env dbSettings roleSettings roleIsolationLvl =
@@ -256,8 +242,8 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
<*> (fmap encodeUtf8 <$> optString "db-anon-role")
<*> (fromMaybe "pgrst" <$> optString "db-channel")
<*> (fromMaybe True <$> optBool "db-channel-enabled")
<*> (maybe ["public"] splitOnCommasEmptyable <$> optStringEmptyable "db-extra-search-path")
<*> (maybe defaultHoistedAllowList splitOnCommas <$> optString "db-hoisted-tx-settings")
<*> (maybe ["public"] splitOnCommas <$> optValue "db-extra-search-path")
<*> (maybe defaultHoistedAllowList splitOnCommas <$> optValue "db-hoisted-tx-settings")
<*> optWithAlias (optInt "db-max-rows")
(optInt "max-rows")
<*> (fromMaybe False <$> optBool "db-plan-enabled")
@@ -272,8 +258,8 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
<*> (fromMaybe True <$> optBool "db-prepared-statements")
<*> (fmap toQi <$> optWithAlias (optString "db-root-spec")
(optString "root-spec"))
<*> (fromList . maybe ["public"] splitOnCommas <$> optWithAlias (optString "db-schemas")
(optString "db-schema"))
<*> (fromList . maybe ["public"] splitOnCommas <$> optWithAlias (optValue "db-schemas")
(optValue "db-schema"))
<*> (fromMaybe True <$> optBool "db-config")
<*> (fmap toQi <$> optString "db-pre-config")
<*> parseTxEnd "db-tx-end" snd
@@ -281,7 +267,7 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
<*> (fromMaybe "postgresql://" <$> optString "db-uri")
<*> pure optPath
<*> pure Nothing
<*> optStringOrURI "jwt-aud"
<*> parseJwtAudience "jwt-aud"
<*> parseRoleClaimKey "jwt-role-claim-key" "role-claim-key"
<*> (fmap encodeUtf8 <$> optString "jwt-secret")
<*> (fromMaybe False <$> optWithAlias
@@ -289,20 +275,17 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
(optBool "secret-is-base64"))
<*> (fromMaybe 0 <$> optInt "jwt-cache-max-lifetime")
<*> parseLogLevel "log-level"
<*> parseLogQuery "log-query"
<*> parseOpenAPIMode "openapi-mode"
<*> (fromMaybe False <$> optBool "openapi-security-active")
<*> parseOpenAPIServerProxyURI "openapi-server-proxy-uri"
<*> parseCORSAllowedOrigins "server-cors-allowed-origins"
<*> (defaultServerHost <$> optString "server-host")
<*> parseServerPort "server-port"
<*> (fromMaybe "!4" <$> optString "server-host")
<*> (fromMaybe 3000 <$> optInt "server-port")
<*> (fmap (CI.mk . encodeUtf8) <$> optString "server-trace-header")
<*> (fromMaybe False <$> optBool "server-timing-enabled")
<*> (fmap T.unpack <$> optString "server-unix-socket")
<*> parseSocketFileMode "server-unix-socket-mode"
<*> (defaultServerHost <$> optWithAlias (optString "admin-server-host")
(optString "server-host"))
<*> parseAdminServerPort "admin-server-port"
<*> optInt "admin-server-port"
<*> pure roleSettings
<*> pure roleIsolationLvl
<*> optInt "internal-schema-cache-sleep"
@@ -314,17 +297,6 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
fromEnv = M.mapKeys fromJust $ M.filterWithKey (\k _ -> isJust k) $ M.mapKeys normalize env
normalize k = ("app.settings." <>) <$> T.stripPrefix "PGRST_APP_SETTINGS_" (toS k)
parseServerPort :: C.Key -> C.Parser C.Config Int
parseServerPort k = fromMaybe 3000 <$> optInt k
parseAdminServerPort :: C.Key -> C.Parser C.Config (Maybe Int)
parseAdminServerPort k = do
serverPort <- parseServerPort "server-port"
optInt k >>= \case
Nothing -> pure Nothing
Just asp | asp == serverPort -> fail "admin-server-port cannot be the same as server-port"
| otherwise -> pure $ Just asp
parseSocketFileMode :: C.Key -> C.Parser C.Config FileMode
parseSocketFileMode k =
optString k >>= \case
@@ -354,6 +326,14 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
Just val | isMalformedProxyUri val -> fail "Malformed proxy uri, a correct example: https://example.com:8443/basePath"
| otherwise -> pure $ Just val
parseJwtAudience :: C.Key -> C.Parser C.Config (Maybe StringOrURI)
parseJwtAudience k =
optString k >>= \case
Nothing -> pure Nothing -- no audience in config file
Just aud -> case preview stringOrUri (T.unpack aud) of
Nothing -> fail "Invalid Jwt audience. Check your configuration."
aud' -> pure aud'
parseLogLevel :: C.Key -> C.Parser C.Config LogLevel
parseLogLevel k =
optString k >>= \case
@@ -365,14 +345,6 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
Just "debug" -> pure LogDebug
Just _ -> fail "Invalid logging level. Check your configuration."
parseLogQuery :: C.Key -> C.Parser C.Config LogQuery
parseLogQuery k =
optString k >>= \case
Nothing -> pure LogQueryDisabled
Just "disabled" -> pure LogQueryDisabled
Just "main-query" -> pure LogQueryMain
Just _ -> fail "Invalid SQL logging value. Check your configuration."
parseTxEnd :: C.Key -> ((Bool, Bool) -> Bool) -> C.Parser C.Config Bool
parseTxEnd k f =
optString k >>= \case
@@ -404,22 +376,8 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
optString :: C.Key -> C.Parser C.Config (Maybe Text)
optString k = mfilter (/= "") <$> overrideFromDbOrEnvironment C.optional k coerceText
optStringEmptyable :: C.Key -> C.Parser C.Config (Maybe Text)
optStringEmptyable k = overrideFromDbOrEnvironment C.optional k coerceText
optStringOrURI :: C.Key -> C.Parser C.Config (Maybe Text)
optStringOrURI k = do
stringOrURI <- mfilter (/= "") <$> overrideFromDbOrEnvironment C.optional k coerceText
-- If the string contains ':' then it should
-- be a valid URI according to RFC 3986
case stringOrURI of
Just s -> if T.isInfixOf ":" s then validateURI s else return (Just s)
Nothing -> return Nothing
where
validateURI :: Text -> C.Parser C.Config (Maybe Text)
validateURI s = if isURI (T.unpack s)
then return $ Just s
else fail "jwt-aud should be a string or a valid URI"
optValue :: C.Key -> C.Parser C.Config (Maybe C.Value)
optValue k = overrideFromDbOrEnvironment C.optional k identity
optInt :: (Read i, Integral i) => C.Key -> C.Parser C.Config (Maybe i)
optInt k = join <$> overrideFromDbOrEnvironment C.optional k coerceInt
@@ -459,18 +417,12 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
Nothing -> (> 0) <$> (readMaybe s :: Maybe Integer)
coerceBool _ = Nothing
splitOnCommas :: Text -> [Text]
splitOnCommas s = T.strip <$> T.splitOn "," s
splitOnCommasEmptyable :: Text -> [Text]
splitOnCommasEmptyable "" = []
splitOnCommasEmptyable s = T.strip <$> T.splitOn "," s
splitOnCommas :: C.Value -> [Text]
splitOnCommas (C.String s) = T.strip <$> T.splitOn "," s
splitOnCommas _ = []
defaultHoistedAllowList = ["statement_timeout","plan_filter.statement_cost_limit","default_transaction_isolation"]
defaultServerHost :: Maybe Text -> Text
defaultServerHost = fromMaybe "!4"
-- | Read the JWT secret from a file if configJwtSecret is actually a
-- filepath(has @ as its prefix). To check if the JWT secret is provided is
-- in fact a file path, it must be decoded as 'Text' to be processed.
@@ -495,30 +447,24 @@ decodeSecret conf@AppConfig{..} =
decodeB64 = B64.decode . encodeUtf8 . T.strip . replaceUrlChars . decodeUtf8
replaceUrlChars = T.replace "_" "/" . T.replace "-" "+" . T.replace "." "="
-- | Parse `jwt-secret` configuration option and turn into a JWKS.
-- | Parse `jwt-secret` configuration option and turn into a JWKSet.
--
-- There are three ways to specify `jwt-secret`: text secret, JSON Web Key
-- (JWK), or JSON Web Key Set (JWKS). The first two are converted into a JwkSet
-- (JWK), or JSON Web Key Set (JWKS). The first two are converted into a JWKSet
-- with one key and the last is converted as is.
decodeJWKS :: AppConfig -> IO AppConfig
decodeJWKS conf = do
jwks <- case configJwtSecret conf of
Just s -> either fail (pure . Just) $ parseSecret s
Nothing -> pure Nothing
return $ conf { configJWKS = jwks }
decodeJWKS :: AppConfig -> AppConfig
decodeJWKS conf =
conf { configJWKS = parseSecret <$> configJwtSecret conf }
parseSecret :: ByteString -> Either [Char] JwkSet
parseSecret :: ByteString -> JWKSet
parseSecret bytes =
case maybeJWKSet of
Just jwk -> Right jwk
Nothing -> maybe validateSecret (\jwk' -> Right $ JWT.JwkSet [jwk']) maybeJWK
fromMaybe (maybe secret (\jwk' -> JWT.JWKSet [jwk']) maybeJWK)
maybeJWKSet
where
maybeJWKSet = JSON.decodeStrict bytes :: Maybe JwkSet
maybeJWK = JSON.decodeStrict bytes :: Maybe Jwk
secret = JWT.JwkSet [JWT.SymmetricJwk bytes Nothing (Just JWT.Sig) (Just $ JWT.Signed JWT.HS256)]
validateSecret
| BS.length bytes < 32 = Left "The JWT secret must be at least 32 characters long."
| otherwise = Right secret
maybeJWKSet = JSON.decodeStrict bytes :: Maybe JWKSet
maybeJWK = JSON.decodeStrict bytes :: Maybe JWK
secret = JWT.JWKSet [JWT.fromKeyMaterial keyMaterial]
keyMaterial = JWT.OctKeyMaterial . JWT.OctKeyParameters $ JOSE.Base64Octets bytes
-- | Read database uri from a separate file if `db-uri` is a filepath.
readDbUriFile :: Maybe Text -> AppConfig -> IO AppConfig
+13 -14
View File
@@ -24,7 +24,7 @@ import qualified Hasql.Statement as SQL
import qualified Hasql.Transaction as SQL
import qualified Hasql.Transaction.Sessions as SQL
import NeatInterpolation (trimming)
import Text.InterpolatedString.Perl6 (q, qc)
import Protolude
@@ -95,7 +95,7 @@ queryDbSettings preConfFunc prepared =
let transaction = if prepared then SQL.transaction else SQL.unpreparedTransaction in
transaction SQL.ReadCommitted SQL.Read $ SQL.statement dbSettingsNames $ SQL.Statement sql (arrayParam HE.text) decodeSettings prepared
where
sql = encodeUtf8 [trimming|
sql = [qc|
WITH
role_setting AS (
SELECT setdatabase as database,
@@ -109,25 +109,25 @@ queryDbSettings preConfFunc prepared =
substr(setting, 1, strpos(setting, '=') - 1) as k,
substr(setting, strpos(setting, '=') + 1) as v
FROM role_setting
${preConfigF}
{preConfigF}
)
SELECT DISTINCT ON (key)
replace(k, '${prefix}', '') AS key,
replace(k, '{prefix}', '') AS key,
v AS value
FROM kv_settings
WHERE k = ANY($$1) AND v IS NOT NULL
WHERE k = ANY($1) AND v IS NOT NULL
ORDER BY key, database DESC NULLS LAST;
|]
preConfigF = case preConfFunc of
Nothing -> mempty
Just func -> [trimming|
Just func -> [qc|
UNION
SELECT
null as database,
x as k,
current_setting(x, true) as v
FROM unnest($$1) x
JOIN ${func}() _ ON TRUE
FROM unnest($1) x
JOIN {func}() _ ON TRUE
|]::Text
decodeSettings = HD.rowList $ (,) <$> column HD.text <*> column HD.text
@@ -136,7 +136,7 @@ queryRoleSettings pgVer prepared =
let transaction = if prepared then SQL.transaction else SQL.unpreparedTransaction in
transaction SQL.ReadCommitted SQL.Read $ SQL.statement mempty $ SQL.Statement sql HE.noParams (processRows <$> rows) prepared
where
sql = encodeUtf8 [trimming|
sql = [q|
with
role_setting as (
select r.rolname, unnest(r.rolconfig) as setting
@@ -161,15 +161,14 @@ queryRoleSettings pgVer prepared =
i.value as iso_lvl,
coalesce(array_agg(row(kv.key, kv.value)) filter (where key <> 'default_transaction_isolation'), '{}') as role_settings
from kv_settings kv
join pg_settings ps on ps.name = kv.key and (ps.context = 'user' ${hasParameterPrivilege})
join pg_settings ps on ps.name = kv.key |] <>
(if pgVer >= pgVersion150
then "and (ps.context = 'user' or has_parameter_privilege(current_user::regrole::oid, ps.name, 'set')) "
else "and ps.context = 'user' ") <> [q|
left join iso_setting i on i.rolname = kv.rolname
group by kv.rolname, i.value;
|]
hasParameterPrivilege
| pgVer >= pgVersion150 = "or has_parameter_privilege(current_user::regrole::oid, ps.name, 'set')"
| otherwise = ""
processRows :: [(Text, Maybe Text, [(Text, Text)])] -> (RoleSettings, RoleIsolationLvl)
processRows rs =
let

Some files were not shown because too many files have changed in this diff Show More