Compare commits

...
79 Commits
Author SHA1 Message Date
steve-chavez 064e5fea7b chore: bump version to 14.17 2026-08-13 10:48:30 -05:00
Michał KłeczekandSteve Chavez cbdafd6f8b fix: JWT validation uses wrong current time due to a bug in auto-update
Upgrade auto-update to 0.2.7 which contains a fix to a bug causing some threads not seeing updates to the cached values.

(cherry picked from commit a2a592a617)
2026-08-13 09:30:02 -05:00
renovate[bot]andWolfgang Walther b6ffe9bcf8 chore(deps): update vmactions/freebsd-vm action to v1.5.3 2026-08-07 07:40:59 +00:00
renovate[bot]andWolfgang Walther c57a10a8c4 chore(deps): update all dependencies 2026-08-04 21:04:54 +00:00
renovate[bot]andWolfgang Walther aee0d52b4f chore(deps): update docker/login-action action to v4.6.0 2026-07-30 20:41:24 +00:00
Taimoor Zaeem 673bbbf291 chore: bump version to 14.16 2026-07-27 11:19:37 +05:00
renovate[bot]andWolfgang Walther 17c39dbef6 chore(deps): update docker/login-action action to v4.5.1 2026-07-24 19:26:07 +00:00
renovate[bot]andWolfgang Walther 1a777919c6 chore(deps): update all dependencies 2026-07-24 18:51:21 +00:00
Taimoor Zaeem 85c8828c0d fix: kill postgrest main thread on admin server crash
Fixes #5096.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-07-21 22:00:56 +05:00
Taimoor Zaeem 6206f55673 docs: update PostgREST source file locations in architecture page
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-07-18 18:24:46 +05:00
Taimoor Zaeem 2aa023e2d7 chore: add .ghc.environment* to .gitignore
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-07-18 18:24:46 +05:00
renovate[bot]andWolfgang Walther 82315a45ff chore(deps): update ubuntu:resolute docker digest to 3131b4c 2026-07-17 17:42:26 +00:00
renovate[bot]andWolfgang Walther d9931d4165 chore(deps): update ubuntu:resolute docker digest to 651ba3f 2026-07-16 21:19:27 +00:00
steve-chavez 167a0a1800 chore: bump version to 14.15 2026-07-13 17:15:03 -05:00
Steve ChavezandGitHub 032ceb62c1 fix: admin server dies silently by upgrading warp
* fix: admin server dies silently by upgrading warp

Uses warp main branch https://github.com/yesodweb/wai/commit/ad041216b643f69a2a9c87cbf4c2988aa4633dd5
since there's no release yet.

* amend: use warp 3.4.14 for 3522917

Use a release instead of a git commit

* amend: update cabal index so cabal build succeeds

* chore: pin jose-jwt to build on CI
2026-07-13 15:45:54 -05:00
steve-chavez 71a894623c ci: add git to freebsd
Otherwise when using a git commit on stack the build fails because it
lacks the git dependency.
2026-07-10 21:49:58 -05:00
renovate[bot]andWolfgang Walther 3eab24400c chore(deps): update docker/login-action action to v4.4.0 2026-07-05 12:49:36 +00:00
renovate[bot]andWolfgang Walther 01593a8a2e chore(deps): update docker/setup-buildx-action action to v4.2.0 2026-07-02 20:08:01 +00:00
renovate[bot]andWolfgang Walther 099f053f50 chore(deps): update all dependencies 2026-07-02 08:56:42 +00:00
ncikandTaimoor Zaeem 46dd242c9c docs: fix example jwt expiration epoch
Issue: the example epoch date is wrapped in parentheses. This causes the server to return PGRST303: "The JWT 'exp' claim must be a number"

Fix: remove parentheses
(cherry picked from commit 69d21a82c9)
2026-07-02 13:46:04 +05:00
renovate[bot]andWolfgang Walther f06a03329f chore(deps): update korthout/backport-action action to v4.6 2026-06-30 06:24:42 +00:00
steve-chavez c1be852d55 chore: bump version to 14.14 2026-06-29 13:20:56 -05:00
Taimoor Zaeemandsteve-chavez ae65768569 fix(admin): log admin server exceptions and close admin socket
Admin server was crashing silently and requests hang indefinitely.
With this fix, we are now logging the admin server exceptions and also
close the socket afterwards so admin request don't hang.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-06-29 13:13:16 -05:00
renovate[bot]andWolfgang Walther c98a8c2a29 chore(deps): update vmactions/freebsd-vm action to v1.5.0 2026-06-29 17:45:43 +00:00
renovate[bot]andWolfgang Walther 1853a4a2b7 chore(deps): update vmactions/freebsd-vm action to v1.4.9 2026-06-27 09:41:00 +00:00
renovate[bot]andWolfgang Walther 64407a45fc chore(deps): update actions/cache action to v6.1.0 2026-06-26 19:48:31 +00:00
renovate[bot]andWolfgang Walther 2641bf48a8 chore(deps): update all dependencies 2026-06-25 20:10:24 +00:00
renovate[bot]andWolfgang Walther 050e54965f chore(deps): update all dependencies to v1.4.8 2026-06-20 21:57:15 +00:00
renovate[bot]andWolfgang Walther 5826ce50c8 chore(deps): update all dependencies 2026-06-19 19:08:29 +00:00
renovate[bot]andWolfgang Walther 9772e075f6 chore(deps): update all dependencies 2026-06-19 19:07:45 +00:00
steve-chavezandTaimoor Zaeem ef54d94cc1 amend: required membership for postgrest_test_anonymous
Previous test correction required GRANTing membership for
postgrest_test_anonymous.
2026-06-12 12:34:04 +05:00
steve-chavezandTaimoor Zaeem d67f68330a test: correct test_admin_ready_includes_schema_cache_state
There was an oversight on d556cea, `timeout_authenticator` doesn't have
any privileges on the tables so it shouldn't be making requests to the
API server or it will end up with 401 responses.
2026-06-12 12:34:04 +05:00
Wolfgang Walther 2e4b72d130 Revert "ci(ci): avoid running build jobs twice on pushes to main"
This reverts commit 840d3f99ed.

This was a nice idea in theory, but in practice this means that the
build jobs do not run at all on the default branch anymore. This means
the caches they push to won't be used by any other jobs, neither on the
tag pipelines, nor on PRs. This in turn makes all of these jobs really
slow.

Not helpful!
2026-06-10 15:04:34 +02:00
renovate[bot]andWolfgang Walther 4e68ffbc92 chore(deps): update all dependencies to v7 2026-06-07 11:01:20 +00:00
steve-chavez b2501edf8d chore: bump version to 14.13 2026-06-04 13:49:01 -05:00
Laurence IslaandTaimoor Zaeem 4dc7d84ff1 fix: request failures when work_mem is set on a role 2026-06-04 21:25:50 +05:00
Taimoor Zaeem 627e8c07ab test(io): set client_min_messages to warning in SQL fixtures
To suppress unwanted "NOTICE: ..." messages which pollute the log.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-06-04 15:23:43 +05:00
Taimoor Zaeem 3decedce1e test(io): refactor SQL fixtures
- Move privileges and grants to privileges.sql
- Move schema, tables and functions to schema.sql
- Move global database settings to database.sql

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-06-04 15:23:43 +05:00
Taimoor Zaeem c71bfacc1d test(io): move role fixtures to roles.sql
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-06-04 15:23:43 +05:00
netandSteve Chavez 4f69c407e3 fix: don't mention retrying in the database connection error message
The PGRST000 database connection error message was "Database connection
error. Retrying the connection.", but reconnection attempts are already
logged separately by the reconnection observation, and on fatal errors
(e.g. authentication failure) PostgREST does not retry at all. Drop the
"Retrying the connection." part, leaving "Database connection error.".
2026-06-03 13:36:33 -05:00
Wolfgang Walther 80d0caaff9 ci: remove deprecation warning
app-id was replaced with client-id
2026-06-03 10:09:47 +02:00
renovate[bot]andWolfgang Walther 0dbbf734b7 chore(deps): update all dependencies to v6.0.3 2026-06-03 07:39:54 +00:00
Taimoor Zaeem 8357488ded docs: restructure PostGIS page
Restructure PostGIS page with order:

- `Accept: application/geo+json` request example
- Generated column example
- String representation example

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 669fc9ede8)
2026-06-02 15:07:46 +05:00
Wolfgang Walther 7bf7da627b chore: fix actionlint with ubuntu-slim jobs 2026-06-02 10:39:04 +02:00
Wolfgang Walther b1864594e6 ci(ci): avoid running build jobs twice on pushes to main
It's enough to run the build jobs in the release pipeline, no need to
run them on the push event as well.
2026-06-02 10:15:14 +02:00
Wolfgang Walther e9464bc535 ci(release): only update docker description from devel release
This condition was accidentally removed when merging 4 release jobs into
2.
2026-06-02 10:15:14 +02:00
Wolfgang Walther 27734f5d51 ci(release): merge 4 jobs into 2
No need to do this in as many jobs. Splitting all the github stuff from
docker stuff into two jobs is enough. This still allows to conditionally
enable docker jobs in contributors repos, depending on whether the
relevant docker credentials are provided - but avoids using too many
concurrent runners.
2026-06-02 10:15:13 +02:00
Wolfgang Walther 8ce8939259 ci(release): run docker job on regular VM runner
Tried to run it on a -slim runner in fb24b6b3, but the result is that no
docker socket is available. This jobs needs one of the big runners.
2026-06-02 10:15:12 +02:00
Wolfgang Walther f71f70152a ci: avoid running pg18 tests twice
We already test PG18, the default, with the coverage job, so why run it
twice?
2026-06-02 10:15:12 +02:00
Wolfgang Walther 41b58579cb ci: use -slim runner for small jobs
No need to spin up full VM runners for small automation tasks, when we
can use single-CPU runners in containers instead.

https://docs.github.com/en/actions/reference/runners/github-hosted-runners#single-cpu-runners

(some of this will potentially not work, because dependencies in the
slim image might not be available - however, it makes no sense to create
this as a PR, because all jobs touched here run on branches only. Thus
pushing directly to main)
2026-06-02 10:15:11 +02:00
Wolfgang Walther 6d2fa4d3a7 ci: run other loadtests to completion when one of them fails
The most likely offender to fail early is jwt-rsa-cache-worst - which
then cancels *all* loadtests at once. This is unnecessary churn for CI.
Instead we can re-run only that one test later.
2026-06-02 10:15:10 +02:00
Wolfgang Walther 99905c7988 ci: test build of dev shell on darwin 2026-06-02 10:13:29 +02:00
Wolfgang Walther df38175645 ci: use nix-build-uncached to save useless downloads
There is no point to actually realize the derivations to disk, when they
are in cache. `nix-build-uncached` checks the cache and stops before
downloading dependencies if they are available. It will however build as
normal on a cache-miss. Should speed up CI when no Nix dependencies
change a fair bit.

We can only do this for the MacOS job, which only tests the builds. We
can not do this for other jobs which require the build results to be
present before uploading parts of them as artifacts.
2026-06-02 10:13:28 +02:00
Wolfgang Walther ebac332cb1 ci: trigger workflows when nixpkgs pin is updated
Otherwise we might break things...
2026-06-02 10:13:27 +02:00
Wolfgang Walther 6e573b911d ci: Use FreeBSD VM instead of cirrus 2026-06-02 10:10:22 +02:00
Taimoor Zaeem e49a17ea56 nix(docs): print message to console on script runs
It's not immediately visible that these scripts are run when
doing `postgrest-docs-check`.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit e1d547623e)
2026-05-28 14:28:37 +05:00
renovate[bot]andWolfgang Walther b4eec4bbfb chore(deps): update all dependencies 2026-05-22 20:51:50 +00:00
Michał KłeczekandTaimoor Zaeem 83e0df24f5 refactor(remove): schema cache load delay config
Remove internal schema cache load and relationship load sleep settings plus
the delay wrappers they enabled. Drop IO tests that depended on the removed
settings.
2026-05-21 12:01:10 +05:00
Michał KłeczekandTaimoor Zaeem b1a53e008c test: cover stale schema cache database error
Add an IO test that drops a table while schema cache reload is delayed. It verifies the stale cache path returns PostgreSQL 42P01 and the refreshed cache returns PGRST205.
2026-05-21 12:01:10 +05:00
steve-chavez 6200fbad58 chore: bump version to 14.12 2026-05-20 12:20:38 -05:00
steve-chavez b71345ab09 test: adjust test_positive_pool_metric
Adjusted so it waits for liveness instead of requiring sleeping
2026-05-20 10:29:12 -05:00
steve-chavez 9be5c41c05 test: change wait_for_readiness to an enum
This so we can wait for both readiness and liveness
2026-05-20 10:29:12 -05:00
Michał KłeczekandSteve Chavez ed2dc1fe86 fix: implement connection tracking in metrics
Right now metrics observation handler does not track database connections but updates a single Gauge based on HasqlPoolObs events. This is problematic because Hasql pool reports various connection events in multiple phases. The connection state machine is not simple and to precisely report the number of connections in various states, it is necessary to track their lifecycles.

This change adds a ConnTrack data structure and logic to track database connections lifecycles. At the moment it supports "connected" and "inUse" connection counts precisely. The "pgrst_db_pool_available" metric is implemented on top of ConnTrack instead of a simple Gauge.
2026-05-20 10:29:12 -05:00
steve-chavez f6e99117ab test: adjust sleep in test_positive_pool_metric
Otherwise it's not enough time for the pool metric to
reach a negative value.
2026-05-20 10:29:12 -05:00
steve-chavez 0be41a4941 test: negative pgrst_db_pool_available in metrics
Proves the failure on https://github.com/PostgREST/postgrest/issues/4622.

This doesn't require additional test infra, only nginx. Taking advantage
of the `stream {}` context which is also compatible with unix socket
besides TCP.
2026-05-20 10:29:12 -05:00
renovate[bot]andWolfgang Walther ae1e3d6c4d chore(deps): update codecov/codecov-action action to v6.0.1 2026-05-19 07:04:25 +00:00
Taimoor Zaeem f2932bffea test(io): move utility function to util.py
The function `match_log` should be in `util.py` so it can be reused
in other modules.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-05-19 10:01:51 +05:00
Taimoor ZaeemandSteve Chavez 15b83ff4da docs: add redirect of the moved postgis section
In 055921ea, we missed redirecting the old link to the new one.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit ab43ee1fbb)
2026-05-15 19:40:45 -05:00
renovate[bot]andWolfgang Walther a2d76b01b7 chore(deps): update dependency macos to v26 2026-05-15 20:38:43 +00:00
renovate[bot]andTaimoor Zaeem 16d4477c62 chore(deps): update actions/create-github-app-token action to v3.2.0 2026-05-14 10:14:25 +05:00
Taimoor Zaeem d75d2838fc ci: catch linkcheck failure
The scheduled linkcheck job fails without notification to maintainers.
To catch the failure, a github workflow step has been added which
comments on issue #4106 so that the maintainers get notified.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit f90d7d7296)
2026-05-14 09:56:19 +05:00
renovate[bot]andWolfgang Walther 4e08b2748c chore(deps): update actions/create-github-app-token action to v3.2.0 2026-05-13 21:01:16 +00:00
Taimoor Zaeem 86f03c5ded docs: move PostGIS under Integrations
For better visibility.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 055921eaf5)
2026-05-13 13:27:10 +05:00
renovate[bot]andWolfgang Walther eda5a10aa5 chore(deps): update all dependencies 2026-05-12 13:03:04 +00:00
Taimoor ZaeemandWolfgang Walther 51c51ea140 ci: cancel previous runs for tag pipelines
Reverts the change in ad5bb38d70.

https://github.com/orgs/community/discussions/13015 is now solved.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 96f12f83f0)
2026-05-07 07:10:27 +00:00
Taimoor Zaeem 3824e73c91 nix(chore): add commit prefix in release commit message
We should follow our commit convention.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 4c04572783)
2026-05-07 09:29:56 +05:00
renovate[bot]andWolfgang Walther d65288dfd6 chore(deps): update korthout/backport-action digest to bf97bcf 2026-05-06 19:47:08 +00:00
Michał KłeczekandTaimoor Zaeem 80ea1f4a1b refactor: make Observation Logger and Listener modules more cohesive
Observation module exports observationMessage and isDbListenerBug functions. The first one is used only in Logger module and the second one - only in Listener.

This change moves observationMessage function to Logger module and isDbListenerBug function to Listener module, making all three modules more cohesive and lessening dependencies.
It also gives the compiler more opportunities for intra-module inlining and optimization.
2026-05-07 00:36:36 +05:00
renovate[bot]andWolfgang Walther 23b0c8d82a chore(deps): update ubuntu:resolute docker digest to f3d2860 2026-05-05 15:53:44 +00:00
55 changed files with 1301 additions and 1016 deletions
-42
View File
@@ -1,42 +0,0 @@
freebsd_instance:
image_family: freebsd-14-3
build_task:
# Don't change this name without adjusting .github/workflows/build.yaml
name: Build FreeBSD (Stack)
install_script: pkg install -y postgresql16-client hs-stack git
only_if: |
$CIRRUS_TAG != '' || $CIRRUS_BRANCH == 'main' || $CIRRUS_BRANCH =~ 'v*' ||
changesInclude(
'.github/workflows/build.yaml',
'.github/actions/artifact-from-cirrus/**',
'.cirrus.yml',
'postgrest.cabal',
'stack.yaml*',
'**.hs'
)
stack_cache:
folders: /.stack
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
stack_work_cache:
folders: .stack-work
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
- find main src -type f -iname '*.hs' -exec md5sum "{}" +
build_script: |
stack build -j 1 --local-bin-path . --copy-bins
strip postgrest
bin_artifacts:
path: postgrest
+2
View File
@@ -3,4 +3,6 @@
self-hosted-runner:
labels:
- macos-15-intel
- macos-26
- ubuntu-24.04-arm
- ubuntu-slim
@@ -1,119 +0,0 @@
name: Artifact from Cirrus
description: Waits for a specific Cirrus CI run to complete, then downloads the artifact and uploads it to the current workflow. This will silently succeed if Cirrus CI did not schedule a task within 2 minutes.
inputs:
download:
description: Name of Artifact to download from Cirrus CI
required: true
task:
description: Name of Cirrus Task
required: true
token:
description: GitHub Token
required: true
upload:
description: Name of Artifact to upload on GitHub Actions
required: true
runs:
using: composite
steps:
- shell: bash
run: echo "GH_TOKEN=${{ inputs.token }}" >> "$GITHUB_ENV"
- name: Wait for Check Suite to be created
id: check-suite
env:
# GITHUB_SHA does weird things for pull request, so we roll our own:
COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
shell: bash
run: |
get_check_runs_url() {
gh api "repos/{owner}/{repo}/commits/${COMMIT}/check-suites" \
| jq -r '.check_suites[] | select(.app.slug == "cirrus-ci") | .check_runs_url'
}
for _ in $(seq 1 12); do
check_runs_url="$(get_check_runs_url)"
if [ -z "$check_runs_url" ]; then
echo "Cirrus CI task has not started, yet. Waiting..."
sleep 10
else
echo "check_runs_url=$check_runs_url" >> "$GITHUB_OUTPUT"
exit 0
fi
done
>&2 echo "Cirrus CI check suite not found. Is Cirrus CI enabled for this repo?"
- name: Find task by name
id: find-task
if: steps.check-suite.outputs.check_runs_url
shell: bash
run: |
get_number_of_tasks() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -r '.check_runs | map(select(.name == "${{ inputs.task }}")) | length'
}
tasks="$(get_number_of_tasks)"
case "$tasks" in
0)
echo "Task not found, assuming it's skipped intentionally..."
exit 0
;;
1)
echo "task_found=1" >> "$GITHUB_OUTPUT"
exit 0
;;
*)
>&2 echo "More than 1 task with the same name found. Don't know what to do..."
exit 1
;;
esac
- name: Wait for Cirrus CI to complete task
if: steps.find-task.outputs.task_found
shell: bash
run: |
get_conclusion() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -r '.check_runs[] | select(.name == "${{ inputs.task }}" and .status == "completed") | .conclusion'
}
while true; do
conclusion="$(get_conclusion)"
if [ -z "$conclusion" ]; then
echo "Cirrus CI task has not completed, yet. Waiting..."
sleep 30
else
if [ "$conclusion" == "success" ]; then
break
else
exit 1
fi
fi
done
- name: Download artifact from Cirrus CI
if: steps.find-task.outputs.task_found
id: download
shell: bash
run: |
get_external_id() {
gh api "${{ steps.check-suite.outputs.check_runs_url }}" \
| jq -er '.check_runs[] | select(.name == "${{ inputs.task }}") | .external_id'
}
archive="$(mktemp)"
artifacts="$(mktemp -d)"
until curl --no-progress-meter --fail -o "${archive}" \
"https://api.cirrus-ci.com/v1/artifact/task/$(get_external_id)/${{ inputs.download }}.zip"
do
# This happens when a tag is pushed on the same commit. In this case the
# job is immediately marked as "completed" for us, so we end up here after a few
# seconds - but the actual Cirrus CI task is still running and didn't produce its artifact, yet.
echo "Artifact not found on Cirrus CI, yet. Waiting..."
sleep 30
done
unzip "${archive}" -d "${artifacts}"
echo "artifacts=${artifacts}" >> "$GITHUB_OUTPUT"
- name: Save artifact to GitHub Actions
if: steps.find-task.outputs.task_found
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ inputs.upload }}
path: ${{ steps.download.outputs.artifacts }}
if-no-files-found: error
+2 -2
View File
@@ -19,14 +19,14 @@ inputs:
runs:
using: composite
steps:
- uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with:
path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: |
${{ runner.os }}-${{ inputs.prefix }}-
- uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
- uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && startsWith(github.ref, 'refs/pull/')) }}
with:
path: ${{ inputs.path }}
+35
View File
@@ -0,0 +1,35 @@
name: Run anywhere
description: Runs the same code either in a VM or on the bare machine
inputs:
vm:
description: Which VM to run on.
envs:
description: List of relevant environment variables, which might need to be copied into the VM.
prepare:
description: Code to run in a prepare step, e.g. installing dependencies.
run:
description: Code to run as the main action.
required: true
runs:
using: composite
steps:
- if: ${{ inputs.vm == 'freebsd' }}
uses: vmactions/freebsd-vm@83b151f58c6047089f4c80eb5ba2039d158ce093 # v1.5.3
with:
envs: ${{ inputs.envs }}
prepare: ${{ inputs.prepare }}
# Work around https://github.com/vmactions/freebsd-vm/issues/59
run: |
pw user add -n action -m
su action -c '${{ inputs.run }}'
- if: ${{ inputs.vm == '' }}
name: Prepare
shell: ${{ runner.os == 'Windows' && 'pwsh' || 'bash' }}
run: ${{ inputs.prepare }}
- if: ${{ inputs.vm == '' }}
name: Run
shell: ${{ runner.os == 'Windows' && 'pwsh' || 'bash' }}
run: ${{ inputs.run }}
+2 -2
View File
@@ -11,12 +11,12 @@ inputs:
runs:
using: composite
steps:
- uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
- uses: nixbuild/nix-quick-install-action@9f63be77f412a248c9d9a65a4c82cf066cdf8f0c # v35
with:
nix_conf: |-
always-allow-substitutes = true
max-jobs = auto
- uses: cachix/cachix-action@1eb2ef646ac0255473d23a5907ad7b04ce94065c # v17
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
with:
name: postgrest
authToken: ${{ inputs.authToken }}
+5 -5
View File
@@ -9,7 +9,7 @@ on:
jobs:
backport:
name: Backport
runs-on: ubuntu-24.04
runs-on: ubuntu-slim
# It triggers only when PR is already merged on either:
#
# - The merge event itself (action != labeled) or
@@ -28,9 +28,9 @@ jobs:
# This actions creates the github token using the postgrest app secrets
- name: Create Github App Token
id: app-token
uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ vars.POSTGREST_CI_APP_ID }}
client-id: ${{ vars.POSTGREST_CI_APP_ID }}
private-key: ${{ secrets.POSTGREST_CI_PRIVATE_KEY }}
permission-contents: write
permission-pull-requests: write
@@ -38,14 +38,14 @@ jobs:
# This is required for backport action to cherry-pick the PR
- name: Fetch PR ref
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha }}
token: ${{ steps.app-token.outputs.token }}
# Backport action that creates the PR with given settings
- name: Create backport PR
uses: korthout/backport-action@7c3f6cd5843cac11bc59a04a1b7699af93261670 # v4.5
uses: korthout/backport-action@2e830a1d0b8269505846ddd407a70876913ad1f8 # v4.6
with:
github_token: ${{ steps.app-token.outputs.token }}
pull_description: 'Backport for #${pull_number}.'
+37 -58
View File
@@ -16,6 +16,7 @@ on:
- .github/*
- '*.nix'
- nix/**
- flake.lock
- .cirrus.yml
- cabal.project*
- postgrest.cabal
@@ -33,7 +34,7 @@ jobs:
name: Nix - Linux x86-64 static
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -60,26 +61,21 @@ jobs:
macos:
name: Nix - MacOS
runs-on: macos-15
runs-on: macos-26
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Install gnu sed
run: brew install gnu-sed
- name: Install nix-build-uncached
run: nix-env -f default.nix -iA nix-build-uncached
- name: Build everything
run: |
# The --dry-run will give us a list of derivations to download from cachix and
# derivations to build. We only take those that would have to be built and then build
# those explicitly. This has the advantage that pure verification will not include
# a download anymore, making it much faster. If something needs to be built, only
# the dependencies required to do so will be downloaded, but not everything.
nix-build --dry-run 2>&1 \
| gsed -e '1,/derivations will be built:$/d' -e '/paths will be fetched/Q' \
| xargs nix-build
- name: Build everything (default.nix)
run: nix-build-uncached
- name: Build everything (shell.nix)
run: nix-build-uncached shell.nix
stack:
@@ -87,73 +83,69 @@ jobs:
fail-fast: false
matrix:
include:
- name: FreeBSD x86-64
runs-on: ubuntu-24.04
vm: freebsd
artifact: postgrest-freebsd-x86-64
deps: pkg install -y git postgresql16-client hs-stack
- name: Linux aarch64
runs-on: ubuntu-24.04-arm
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-ubuntu-aarch64
deps: sudo apt-get update && sudo apt-get install libpq-dev
- name: MacOS aarch64
runs-on: macos-14
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-macos-aarch64
deps: brew link --force libpq
- name: MacOS x86-64
runs-on: macos-15-intel
cache: |
~/.stack/pantry
~/.stack/snapshots
~/.stack/stack.sqlite3
artifact: postgrest-macos-x86-64
deps: brew link --force libpq
- name: Windows
runs-on: windows-2022
cache: |
C:\sr\pantry
C:\sr\snapshots
C:\sr\stack.sqlite3
deps: Add-Content $env:GITHUB_PATH $env:PGBIN
artifact: postgrest-windows-x86-64
name: Stack - ${{ matrix.name }}
runs-on: ${{ matrix.runs-on }}
env:
# Putting .stack in the working directory helps with moving this in and out of the FreeBSD VM.
STACK_ROOT: ${{ github.workspace }}/.stack
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: haskell-actions/setup@cd0d9bdd65b20557f41bea4dbe43d0b5fbbfe553 # v2.11.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- if: ${{ !matrix.vm }}
uses: haskell-actions/setup@6037f33647c3f17758a2356c80fc4a53d7e0685d # v2.12.0
with:
# This must match the version in stack.yaml's resolver
ghc-version: 9.6.7
enable-stack: true
stack-no-global: true
stack-setup-ghc: true
- name: Cache ~/.stack
- name: Cache .stack
uses: ./.github/actions/cache-on-main
with:
path: ${{ matrix.cache }}
prefix: stack
path: .stack
prefix: ${{ matrix.vm }}${{ matrix.vm && '-' }}stack
suffix: ${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
- name: Cache .stack-work
uses: ./.github/actions/cache-on-main
with:
path: .stack-work
save-prs: true
prefix: stack-work-${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
prefix: ${{ matrix.vm }}${{ matrix.vm && '-' }}stack-work-${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
suffix: ${{ hashFiles('main/**/*.hs', 'src/**/*.hs') }}
- name: Install dependencies
if: matrix.deps
run: ${{ matrix.deps }}
- name: Build with Stack
run: stack build --lock-file error-on-write --local-bin-path result --copy-bins
- name: Strip Executable
run: strip result/postgrest*
uses: ./.github/actions/run-anywhere
with:
vm: ${{ matrix.vm }}
envs: STACK_ROOT
prepare: ${{ matrix.deps }}
run: |
stack build --lock-file error-on-write --local-bin-path result --copy-bins
strip result/postgrest*
- name: Save built executable as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
@@ -164,19 +156,6 @@ jobs:
if-no-files-found: error
freebsd:
name: Stack - FreeBSD from CirrusCI
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: ./.github/actions/artifact-from-cirrus
with:
token: ${{ github.token }}
task: Build FreeBSD (Stack)
download: bin
upload: postgrest-freebsd-x86-64
cabal:
strategy:
matrix:
@@ -185,8 +164,8 @@ jobs:
name: Cabal - Linux x86-64 - GHC ${{ matrix.ghc }}
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: haskell-actions/setup@cd0d9bdd65b20557f41bea4dbe43d0b5fbbfe553 # v2.11.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: haskell-actions/setup@6037f33647c3f17758a2356c80fc4a53d7e0685d # v2.12.0
with:
ghc-version: ${{ matrix.ghc }}
- name: Cache .cabal
+2 -2
View File
@@ -20,7 +20,7 @@ jobs:
name: Lint & Style
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -36,7 +36,7 @@ jobs:
name: Commit
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 100 # fetch history (last 100 commits) instead of default shallow clone history, this is deemed enough for a PR history
- name: Setup Nix Environment
+3 -4
View File
@@ -41,16 +41,15 @@ jobs:
concurrency:
# Never tag outdated commits on the main branch by skipping superseded commits
group: ci-tag-${{ (github.ref == 'refs/heads/main' && github.ref) || github.run_id }}
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
cancel-in-progress: true
if: vars.RELEASE_ENABLED
runs-on: ubuntu-24.04
runs-on: ubuntu-slim
needs:
- docs
- test
- build
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ssh-key: ${{ secrets.POSTGREST_SSH_KEY }}
- name: Tag latest commit
+3 -2
View File
@@ -14,6 +14,7 @@ on:
- .github/actions/setup-nix/**
- default.nix
- nix/**
- flake.lock
- docs/**
- '!**.md'
@@ -27,7 +28,7 @@ jobs:
name: Build
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -41,7 +42,7 @@ jobs:
name: Spellcheck
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
+27 -2
View File
@@ -7,12 +7,37 @@ on:
jobs:
linkcheck:
name: Linkcheck
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: docs.linkcheck.bin
- run: postgrest-docs-linkcheck
- name: Run Linkcheck
id: linkcheck
run: postgrest-docs-linkcheck
# This actions creates the github token using the postgrest app secrets
- name: Create Github App Token (Runs only on linkcheck failure)
id: app-token
if: ${{ failure() && steps.linkcheck.outcome == 'failure' }} # only create the token on linkcheck failure
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
client-id: ${{ vars.POSTGREST_CI_APP_ID }}
private-key: ${{ secrets.POSTGREST_CI_PRIVATE_KEY }}
permission-issues: write # required for commenting on issues
- name: Notify on linkcheck failure by commenting on GH Issue 4106
if: ${{ failure() && steps.linkcheck.outcome == 'failure' }}
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
with:
token: ${{ steps.app-token.outputs.token }}
issue-number: 4106
body: |
**Linkcheck Job Failed!**
A broken link was detected in the docs. Please check the [failed run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details.
+15 -37
View File
@@ -9,8 +9,7 @@ on:
concurrency:
# Terminate all previous runs of the same workflow for the same tag.
group: release-${{ github.ref }}
# TODO: Enable this once https://github.com/orgs/community/discussions/13015 is solved
cancel-in-progress: false
cancel-in-progress: true
jobs:
build:
@@ -20,13 +19,15 @@ jobs:
CACHIX_AUTH_TOKEN: ${{ secrets.CACHIX_AUTH_TOKEN }}
prepare:
name: Prepare
runs-on: ubuntu-24.04
github:
name: GitHub
permissions:
contents: write
runs-on: ubuntu-slim
needs:
- build
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
@@ -48,23 +49,7 @@ jobs:
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Save CHANGES.md as artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-changes
path: CHANGES.md
if-no-files-found: error
github:
name: GitHub
permissions:
contents: write
runs-on: ubuntu-24.04
needs:
- prepare
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Download all artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
@@ -116,14 +101,14 @@ jobs:
gh release edit devel \
-t devel \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
-F CHANGES.md \
--prerelease
gh release upload --clobber devel release-bundle/*
else
gh release create "${GITHUB_REF_NAME}" \
-t "${GITHUB_REF_NAME}" \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
-F CHANGES.md \
release-bundle/*
fi
@@ -132,13 +117,13 @@ jobs:
name: Docker Hub
runs-on: ubuntu-24.04-arm
needs:
- prepare
- github
if: |
vars.DOCKER_REPO && vars.DOCKER_USER
env:
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Download x86-64 Docker image
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
@@ -147,8 +132,8 @@ jobs:
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: postgrest-ubuntu-aarch64
- uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
- uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
@@ -186,16 +171,9 @@ jobs:
echo "Skipping push to 'latest' tag for pre-release..."
fi
docker-description:
name: Docker Hub Description
runs-on: ubuntu-24.04
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
github.ref == 'refs/tags/devel'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: peter-evans/dockerhub-description@1b9a80c056b620d92cedb9d9b5a223409c68ddfa # v5.0.0
if: github.ref == 'refs/tags/devel'
name: Docker Hub Description
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
+10 -7
View File
@@ -17,6 +17,7 @@ on:
- .github/actions/setup-nix/**
- default.nix
- nix/**
- flake.lock
- .stylish-haskell.yaml
- cabal.project
- postgrest.cabal
@@ -39,7 +40,7 @@ jobs:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -51,7 +52,7 @@ jobs:
- name: Run coverage (IO tests and Spec tests against PostgreSQL 15)
run: postgrest-coverage
- name: Upload coverage to codecov
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
files: ./coverage/codecov.json
token: ${{ secrets.CODECOV_TOKEN }}
@@ -69,7 +70,8 @@ jobs:
strategy:
fail-fast: false
matrix:
pgVersion: [13, 14, 15, 16, 17]
# Latest version is tested via `coverage` above.
pgVersion: [13, 14, 15, 16]
name: PG ${{ matrix.pgVersion }}
runs-on: ubuntu-24.04
defaults:
@@ -78,7 +80,7 @@ jobs:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -108,7 +110,7 @@ jobs:
name: Memory
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -123,12 +125,13 @@ jobs:
loadtest:
strategy:
fail-fast: false
matrix:
kind: ['mixed', 'jwt-hs', 'jwt-hs-cache', 'jwt-hs-cache-worst', 'jwt-rsa', 'jwt-rsa-cache', 'jwt-rsa-cache-worst']
name: Loadtest
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- name: Setup Nix Environment
@@ -163,7 +166,7 @@ jobs:
name: Flake Check
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- name: Setup Nix Environment
+1
View File
@@ -26,3 +26,4 @@ loadtest
.docs-build
gen_targets.http
gen_jwk.json
.ghc.environment.*
+38
View File
@@ -4,6 +4,44 @@ All notable changes to this project will be documented in this file. From versio
## Unreleased
## [14.17] - 2026-08-13
### Fixed
- JWT validation uses wrong current time due to a bug in auto-update by @mkleczek in #5159
## [14.16] - 2026-07-27
### Fixed
- Fix admin server crashing without a way to recover by @taimoorzaeem in #5096
## [14.15] - 2026-07-13
### Fixed
- Fix admin server dying silently by @Vlix, @mkleczek, @steve-chavez in #5012
## [14.14] - 2026-06-29
### Fixed
- Fix admin server not logging cause of failure by @taimoorzaeem in #5012
## [14.13] - 2026-06-04
### Fixed
- Fix connection retrying message in `PGRST000` error by @netqo in #4980
+ Remove redundant "Retrying the connection." from message because it is logged separately
- Fix request failures when `work_mem` is set on a role by @laurenceisla in #4955
## [14.12] - 2026-05-20
### Fixed
- Fix race condition in pool_available metric causing negative values during network instability by @mkleczek in #4622
## [14.11] - 2026-05-04
### Fixed
+1 -1
View File
@@ -2,7 +2,7 @@
# The x86-64 is a single-static-binary image built via Nix, see:
# nix/tools/docker/README.md
FROM ubuntu:resolute@sha256:5e275723f82c67e387ba9e3c24baa0abdcb268917f276a0561c97bef9450d0b4 AS postgrest
FROM ubuntu:resolute@sha256:678c6550cc43645e08669028bc177f50be4e7c5b8cca677067b1914d4afc7a03 AS postgrest
RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
+1 -1
View File
@@ -1 +1 @@
index-state: hackage.haskell.org 2025-10-29T04:02:18Z
index-state: hackage.haskell.org 2026-08-10T16:58:32Z
+3
View File
@@ -108,6 +108,9 @@ rec {
inherit (pkgs.haskell.packages."${compiler}") ghcWithPackages;
};
# Used by CI on MacOS
inherit (pkgs) nix-build-uncached;
### Tools
cabalTools =
+11 -11
View File
@@ -31,58 +31,58 @@ This section talks briefly about various important modules.
Main
----
The starting point of the program is `Main.hs <https://github.com/PostgREST/postgrest/blob/main/main/Main.hs>`_.
The starting point of the program is `Main.hs <https://github.com/PostgREST/postgrest/blob/v14/main/Main.hs>`_.
CLI
---
Main then calls `CLI.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/CLI.hs>`_, which is in charge of :ref:`cli`.
Main then calls `CLI.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/CLI.hs>`_, which is in charge of :ref:`cli`.
App
---
`App.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/App.hs>`_ is then in charge of composing the different modules.
`App.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/App.hs>`_ is then in charge of composing the different modules.
Auth
----
`Auth.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Auth.hs>`_ is in charge of :ref:`authn`.
`Auth.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Auth.hs>`_ is in charge of :ref:`authn`.
Api Request
-----------
`ApiRequest.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/ApiRequest.hs>`_ is in charge of parsing the URL query string (following PostgREST syntax), the request headers, and the request body.
`ApiRequest.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/ApiRequest.hs>`_ is in charge of parsing the URL query string (following PostgREST syntax), the request headers, and the request body.
A request might be rejected at this level if it's invalid. For example when providing an unknown media type to PostgREST or using an unknown HTTP method.
Plan
----
Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Plan.hs>`_ generates an internal AST, filling out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request.
Using the Schema Cache, `Plan.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Plan.hs>`_ generates an internal AST, filling out-of-band SQL details (like an ``ON CONFLICT (pk)`` clause) required to complete the user request.
A request might be rejected at this level if it's invalid. For example when doing resource embedding on a nonexistent resource.
Query
-----
`Query.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Query.hs>`_ generates the SQL queries (parametrized and prepared) required to satisfy the user request.
`Query.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Query.hs>`_ generates the SQL queries (parametrized and prepared) required to satisfy the user request.
Only at this stage a connection from the pool might be used.
Schema Cache
------------
`SchemaCache.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/SchemaCache.hs>`_ is in charge of :ref:`schema_cache`.
`SchemaCache.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/SchemaCache.hs>`_ is in charge of :ref:`schema_cache`.
Config
------
`Config.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Config.hs>`_ is in charge of :ref:`configuration`.
`Config.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Config.hs>`_ is in charge of :ref:`configuration`.
Admin
-----
`Admin.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Admin.hs>`_ is in charge of the :ref:`admin_server`.
`Admin.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Admin.hs>`_ is in charge of the :ref:`admin_server`.
HTTP
----
@@ -92,4 +92,4 @@ The HTTP server is provided by `Warp <https://aosabook.org/en/posa/warp.html>`_.
Listener
--------
`Listener.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Listener.hs>`_ is in charge of the :ref:`listener`.
`Listener.hs <https://github.com/PostgREST/postgrest/blob/v14/src/PostgREST/Listener.hs>`_ is in charge of the :ref:`listener`.
@@ -318,144 +318,6 @@ You can insert a new product using a JSON object for the ``extra_info`` column:
To query and filter the data see :ref:`json_columns` for a complete reference.
.. _ww_postgis:
PostGIS
-------
You can use the string representation for `PostGIS <https://postgis.net/>`_ data types such as ``geometry`` or ``geography`` (you need to `install PostGIS <https://postgis.net/documentation/getting_started/>`_ first).
.. code-block:: postgres
-- Activate the postgis module in the current database
create extension if not exists postgis;
create table coverage (
id int primary key,
name text unique,
area geometry
);
To add areas in polygon format, you can use string representation:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 1, "name": "small", "area": "SRID=4326;POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" },
{ "id": 2, "name": "big", "area": "SRID=4326;POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))" }
]
EOF
Now, when you request the information, PostgREST will automatically cast the ``area`` column into a ``Polygon`` geometry type. Although this is useful, you may need the whole output to be in `GeoJSON <https://geojson.org/>`_ format out of the box, which can be done by including the ``Accept: application/geo+json`` in the request. This will work for PostGIS versions 3.0.0 and up and will return the output as a `FeatureCollection Object <https://www.rfc-editor.org/rfc/rfc7946#section-3.3>`_:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-H "Accept: application/geo+json"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
If you need to add an extra property, like the area in square units by using ``st_area(area)``, you could add a generated column to the table and it will appear in the ``properties`` key of each ``Feature``.
.. code-block:: postgres
alter table coverage
add square_units double precision generated always as ( st_area(area) ) stored;
In the case that you are using older PostGIS versions, then creating a function is your best option:
.. code-block:: postgres
create or replace function coverage_geo_collection() returns json as $$
select
json_build_object(
'type', 'FeatureCollection',
'features', json_agg(
json_build_object(
'type', 'Feature',
'geometry', st_AsGeoJSON(c.area)::json,
'properties', json_build_object('id', c.id, 'name', c.name)
)
)
)
from coverage c;
$$ language sql;
Now this query will return the same results:
.. code-block:: bash
curl "http://localhost:3000/rpc/coverage_geo_collection"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Ranges
------
@@ -609,3 +471,20 @@ You can use other comparative filters and also all the `PostgreSQL special date/
"due_date": "2022-02-27T06:00:00-05:00"
}
]
.. raw:: html
<script type="text/javascript">
let hash = window.location.hash;
const redirects = {
// PostGIS
'#postgis': '../integrations/postgis.html#postgis',
};
let willRedirectTo = redirects[hash];
if (willRedirectTo) {
window.location.href = willRedirectTo;
}
</script>
+154
View File
@@ -0,0 +1,154 @@
.. _ww_postgis:
PostGIS
=======
To work with `PostGIS <https://postgis.net/>`_ data types such as ``geometry`` or ``geography``, you'll need to `install PostGIS <https://postgis.net/documentation/getting_started/>`_ first.
.. code-block:: postgres
-- Activate the postgis module in the current database
create extension if not exists postgis;
create table coverage (
id int primary key,
name text unique,
area geometry
);
insert into coverage (id, name, area) values
(1, 'small', ST_GeomFromText('POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))',4326)),
(2, 'big', ST_GeomFromText('POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))', 4326);
.. _application/geo+json:
``application/geo+json``
------------------------
PostgREST supports the `standard <https://www.iana.org/assignments/media-types/application/geo+json>`_ ``application/geo+json`` media type which can be used to get the output in `GeoJSON <https://geojson.org/>`_ format. This will work for PostGIS versions 3.0.0 and up and will return the output as a `FeatureCollection Object <https://www.rfc-editor.org/rfc/rfc7946#section-3.3>`_:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-H "Accept: application/geo+json"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Using generated columns
-----------------------
If you need to add an extra property, like the area in square units by using ``st_area(area)``, you could add a generated column to the table and it will appear in the ``properties`` key of each ``Feature``.
.. code-block:: postgres
alter table coverage
add square_units double precision generated always as ( st_area(area) ) stored;
In the case that you are using older PostGIS versions, then creating a function is your best option:
.. code-block:: postgres
create or replace function coverage_geo_collection() returns json as $$
select
json_build_object(
'type', 'FeatureCollection',
'features', json_agg(
json_build_object(
'type', 'Feature',
'geometry', st_AsGeoJSON(c.area)::json,
'properties', json_build_object('id', c.id, 'name', c.name)
)
)
)
from coverage c;
$$ language sql;
Now this query will return the same results:
.. code-block:: bash
curl "http://localhost:3000/rpc/coverage_geo_collection"
.. code-block:: json
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[1,0],[1,1],[0,1],[0,0]]
]
},
"properties": {
"id": 1,
"name": "small"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[[0,0],[10,0],[10,10],[0,10],[0,0]]
]
},
"properties": {
"id": 2,
"name": "big"
}
}
]
}
Using string representation
---------------------------
To insert areas in polygon format, you can use string representation:
.. code-block:: bash
curl "http://localhost:3000/coverage" \
-X POST -H "Content-Type: application/json" \
-d @- << EOF
[
{ "id": 3, "name": "strip", "area": "SRID=4326;POLYGON((0 0, 50 0, 50 2, 0 2, 0 0))" },
{ "id": 4, "name": "diamond", "area": "SRID=4326;POLYGON((5 0, 10 5, 5 10, 0 5, 5 0))" }
]
EOF
PostgREST will automatically cast the ``area`` column into a ``Polygon`` geometry type.
@@ -51,7 +51,7 @@ Builtin handlers are offered for common standard media types.
* ``text/csv`` and ``application/json``, for all API endpoints. See :ref:`tables_views` and :ref:`functions`.
* ``application/openapi+json``, for the root endpoint. See :ref:`open-api`.
* ``application/geo+json``, see :ref:`ww_postgis`.
* ``application/geo+json``, see :ref:`application/geo+json`.
* ``*/*``, resolves to ``application/json`` for API endpoints and to ``application/openapi+json`` for the root endpoint.
The following vendor media types handlers are also supported.
+1 -1
View File
@@ -172,7 +172,7 @@ Go back to :ref:`tut1_step3` and change the payload to
.. code-block:: bash
payload=$(echo -n "{\"role\":\"todo_user\",\"exp\":\"123456789\"}" | _base64)
payload=$(echo -n "{\"role\":\"todo_user\",\"exp\":123456789}" | _base64)
echo -n "$header.$payload.$signature"
+65
View File
@@ -50,6 +50,16 @@ let
# jailbreak, because hspec limit for tests
fuzzyset = prev.fuzzyset_0_2_4;
# TODO: Remove once available in nixpkgs
auto-update =
prev.callHackageDirect
{
pkg = "auto-update";
ver = "0.2.7";
sha256 = "sha256-fHX/OqF/cB9rbpGpLUtA29bcEJS43HUWHcK55yUxKoo=";
}
{ };
# TODO: Remove once available in nixpkgs haskellPackages
configurator-pg =
prev.callHackageDirect
@@ -70,6 +80,61 @@ let
}
{ };
http2 =
prev.callHackageDirect
{
pkg = "http2";
ver = "5.4.0";
sha256 = "sha256-PeEWVd61bQ8G7LvfLeXklzXqNJFaAjE2ecRMWJZESPE=";
}
{ };
http-semantics =
prev.callHackageDirect
{
pkg = "http-semantics";
ver = "0.4.0";
sha256 = "sha256-rh0z51EKvsu5rQd5n2z3fSRjjEObouNZSBPO9NFYOF0=";
}
{ };
jose-jwt =
prev.callHackageDirect
{
pkg = "jose-jwt";
ver = "0.9.6";
sha256 = "sha256-FhBz5wzyNrDvmjHWOeNAHuVMyJUVSlm+DeQQuITSjaI=";
}
{ };
time-manager =
prev.callHackageDirect
{
pkg = "time-manager";
ver = "0.2.4";
sha256 = "sha256-sAt/331YLQ2IU3z90aKYSq1nxoazv87irsuJp7ZG3pw=";
}
{ };
network-run =
prev.callHackageDirect
{
pkg = "network-run";
ver = "0.5.0";
sha256 = "sha256-vbXh+CzxDsGApjqHxCYf/ijpZtUCApFbkcF5gyN0THU=";
}
{ };
warp =
lib.dontCheck
(prev.callHackageDirect
{
pkg = "warp";
ver = "3.4.14";
sha256 = "sha256-RnoOUlC6dOP0sK/tYAJCX1oLzVFG1GILUY+yVbmvW8Y=";
}
{ });
# Downgrade hasql and related packages while we are still on GHC 9.4 for the static build.
hasql = lib.dontCheck (lib.doJailbreak prev.hasql_1_6_4_4);
hasql-dynamic-statements = lib.dontCheck prev.hasql-dynamic-statements_0_3_1_5;
+4
View File
@@ -122,6 +122,8 @@ let
workingDir = "/docs";
}
''
echo "Checking spelling mistakes..."
export LC_ALL=C
FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')
@@ -144,6 +146,8 @@ let
workingDir = "/docs";
}
''
echo "Detecting obsolete dictionary entries..."
export LC_ALL=C
FILES=$(find . -type f -iname '*.rst' | tr '\n' ' ')
+2 -2
View File
@@ -62,7 +62,7 @@ let
git add CHANGELOG.md > /dev/null
echo "Committing ..."
git commit -m "bump version to $new_version" > /dev/null
git commit -m "chore: bump version to $new_version" > /dev/null
if [[ "$current_branch" == "main" ]]; then
bump devel
@@ -74,7 +74,7 @@ let
git branch "v$A"
echo "Committing (devel bump)..."
git commit -m "bump version to $new_version" > /dev/null
git commit -m "chore: bump version to $new_version" > /dev/null
fi
trap "echo Remote not found. Please push manually ..." ERR
+3
View File
@@ -10,6 +10,7 @@
, hostPlatform
, jq
, lib
, nginx
, postgrest
, python3
, runtimeShell
@@ -94,6 +95,7 @@ let
args = [ "ARG_LEFTOVERS([pytest arguments])" ];
workingDir = "/";
withEnv = postgrest.env;
withPath = [ nginx ];
}
''
${cabal-install}/bin/cabal v2-build ${devCabalOptions} exe:postgrest
@@ -156,6 +158,7 @@ let
redirectTixFiles = false;
withEnv = postgrest.env;
withTmpDir = true;
withPath = [ nginx ];
}
(
# required for `hpc markup` in CI; glibcLocales is not available e.g. on Darwin
+7 -6
View File
@@ -1,5 +1,5 @@
name: postgrest
version: 14.11
version: 14.17
synopsis: REST API for any Postgres database
description: Reads the schema of a PostgreSQL database and creates RESTful routes
for tables, views, and functions, supporting all HTTP methods that security
@@ -100,7 +100,7 @@ library
, HTTP >= 4000.3.7 && < 4000.5
, Ranged-sets >= 0.3 && < 0.5
, aeson >= 2.0.3 && < 2.3
, auto-update >= 0.1.4 && < 0.3
, auto-update >= 0.2.7 && < 0.3
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13
, case-insensitive >= 1.2 && < 1.3
@@ -120,7 +120,7 @@ library
, http-client >= 0.7.19 && < 0.8
, http-types >= 0.12.2 && < 0.13
, insert-ordered-containers >= 0.2.2 && < 0.3
, jose-jwt >= 0.9.6 && < 0.11
, jose-jwt >= 0.9.6 && < 0.10
, lens >= 4.14 && < 5.4
, lens-aeson >= 1.0.1 && < 1.3
, mtl >= 2.2.2 && < 2.4
@@ -152,11 +152,12 @@ library
-- for unix sockets; this is tested in test/io/test_io.py. See
-- https://github.com/kazu-yamamoto/logger/commit/3a71ca70afdbb93d4ecf0083eeba1fbbbcab3fc3
, wai-logger >= 2.4.0
, warp >= 3.3.19 && < 3.5
, warp >= 3.4.14 && < 3.5
, stm >= 2.5 && < 3
, stm-hamt >= 1.2 && < 2
, focus >= 1.0 && < 2
, some >= 1.0.4.1 && < 2
, uuid >= 1.3 && < 2
-- -fno-spec-constr may help keep compile time memory use in check,
-- see https://gitlab.haskell.org/ghc/ghc/issues/16017#note_219304
-- -optP-Wno-nonportable-include-path
@@ -272,7 +273,7 @@ test-suite spec
, hspec-wai >= 0.10 && < 0.12
, hspec-wai-json >= 0.10 && < 0.12
, http-types >= 0.12.3 && < 0.13
, jose-jwt >= 0.9.6 && < 0.11
, jose-jwt >= 0.9.6 && < 0.10
, lens >= 4.14 && < 5.4
, lens-aeson >= 1.0.1 && < 1.3
, monad-control >= 1.0.1 && < 1.1
@@ -315,7 +316,7 @@ test-suite observability
, hspec-wai >= 0.10 && < 0.12
, hspec-wai-json >= 0.10 && < 0.12
, http-types >= 0.12.3 && < 0.13
, jose-jwt >= 0.9.6 && < 0.11
, jose-jwt >= 0.9.6 && < 0.10
, postgrest
, prometheus-client >= 1.1.1 && < 1.2.0
, protolude >= 0.3.1 && < 0.4
+13 -3
View File
@@ -11,7 +11,8 @@ import Control.Monad.Extra (whenJust)
import Network.Socket hiding (addrFamily)
import Network.Socket.ByteString
import PostgREST.AppState (AppState)
import PostgREST.AppState (AppState, getConfig, getMainThreadId)
import PostgREST.Config (AppConfig (..))
import PostgREST.MediaType (MediaType (..), toContentType)
import PostgREST.Metrics (metricsToText)
import PostgREST.Network (resolveSocketToAddress)
@@ -24,13 +25,22 @@ import Protolude
runAdmin :: AppState -> Maybe NS.Socket -> NS.Socket -> Warp.Settings -> IO ()
runAdmin appState maybeAdminSocket socketREST settings = do
conf <- getConfig appState
whenJust maybeAdminSocket $ \adminSocket -> do
address <- resolveSocketToAddress adminSocket
observer $ AdminStartObs address
void . forkIO $ Warp.runSettingsSocket settings adminSocket adminApp
void . forkIO $ handle onError $
Warp.runSettingsSocket (adminServerSettings conf address) adminSocket adminApp
where
adminApp = admin appState socketREST
observer = AppState.getObserver appState
adminServerSettings config addr =
settings
& Warp.setBeforeMainLoop (observer $ AdminStartObs addr)
& maybe identity Warp.setPort (configAdminServerPort config)
onError ex = do
observer $ AdminServerCrashedObs ex
killThread (getMainThreadId appState) -- Admin server crash is deemed unrecoverable, so we kill postgrest
-- | PostgREST admin application
admin :: AppState.AppState -> NS.Socket -> Wai.Application
-4
View File
@@ -119,8 +119,6 @@ data AppConfig = AppConfig
, configRoleSettings :: RoleSettings
, configRoleIsoLvl :: RoleIsolationLvl
, configInternalSCQuerySleep :: Maybe Int32
, configInternalSCLoadSleep :: Maybe Int32
, configInternalSCRelLoadSleep :: Maybe Int32
}
data LogLevel = LogCrit | LogError | LogWarn | LogInfo | LogDebug
@@ -304,8 +302,6 @@ parser optPath env dbSettings roleSettings roleIsolationLvl =
<*> pure roleSettings
<*> pure roleIsolationLvl
<*> optInt "internal-schema-cache-query-sleep"
<*> optInt "internal-schema-cache-load-sleep"
<*> optInt "internal-schema-cache-relationship-load-sleep"
where
parseAppSettings :: C.Key -> C.Parser C.Config [(Text, Text)]
parseAppSettings key = addFromEnv . fmap (fmap coerceText) <$> C.subassocs key C.value
+4 -3
View File
@@ -16,6 +16,7 @@ import Control.Arrow ((***))
import PostgREST.Config.PgVersion (PgVersion (..), pgVersion150)
import qualified Data.HashMap.Strict as HM
import qualified Data.Text as T
import qualified Hasql.Decoders as HD
import qualified Hasql.Encoders as HE
@@ -32,8 +33,8 @@ type RoleSettings = (HM.HashMap ByteString (HM.HashMap ByteString ByteString
type RoleIsolationLvl = HM.HashMap ByteString SQL.IsolationLevel
type TimezoneNames = Set Text -- cache timezone names for prefer timezone=
toIsolationLevel :: (Eq a, IsString a) => a -> SQL.IsolationLevel
toIsolationLevel a = case a of
toIsolationLevel :: Text -> SQL.IsolationLevel
toIsolationLevel a = case T.toLower a of
"repeatable read" -> SQL.RepeatableRead
"serializable" -> SQL.Serializable
_ -> SQL.ReadCommitted
@@ -148,7 +149,7 @@ queryRoleSettings pgVer prepared =
SELECT
rolname,
substr(setting, 1, strpos(setting, '=') - 1) as key,
lower(substr(setting, strpos(setting, '=') + 1)) as value
substr(setting, strpos(setting, '=') + 1) as value
FROM role_setting
),
iso_setting AS (
+1 -1
View File
@@ -541,7 +541,7 @@ instance ErrorBody SQL.UsageError where
code (SQL.SessionUsageError (SQL.QueryError _ _ e)) = code e
code SQL.AcquisitionTimeoutUsageError = "PGRST003"
message (SQL.ConnectionUsageError _) = "Database connection error. Retrying the connection."
message (SQL.ConnectionUsageError _) = "Database connection error."
message (SQL.SessionUsageError (SQL.QueryError _ _ e)) = message e
message SQL.AcquisitionTimeoutUsageError = "Timed out acquiring connection from connection pool."
+125 -1
View File
@@ -1,3 +1,4 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE RecordWildCards #-}
{-|
Module : PostgREST.Logger
@@ -35,7 +36,14 @@ import PostgREST.Config (LogLevel (..))
import PostgREST.Observation
import PostgREST.Query (MainQuery (..))
import Protolude
import qualified Data.ByteString.Lazy as LBS
import qualified Data.Text as T
import qualified Hasql.Connection as SQL
import qualified Hasql.Pool.Observation as SQL
import Numeric (showFFloat)
import PostgREST.Config.PgVersion (pgvName)
import qualified PostgREST.Error as Error
import Protolude
data LoggerState = LoggerState
{ stateGetZTime :: IO ZonedTime -- ^ Time with time zone used for logs
@@ -146,3 +154,119 @@ renderSnippet snippet =
prepared = False -- unused
in
sql
observationMessage :: Observation -> Text
observationMessage = \case
AdminStartObs address ->
"Admin server listening on " <> address
AdminServerCrashedObs ex ->
"Admin server crashed unexpectedly: " <> (showOnSingleLine '\t' . show) ex
AppStartObs ver ->
"Starting PostgREST " <> T.decodeUtf8 ver <> "..."
AppServerAddressObs address ->
"API server listening on " <> address
DBConnectedObs ver ->
"Successfully connected to " <> ver
ExitUnsupportedPgVersion pgVer minPgVer ->
"Cannot run in this PostgreSQL version (" <> pgvName pgVer <> "), PostgREST needs at least " <> pgvName minPgVer
ExitDBNoRecoveryObs ->
"Automatic recovery disabled, exiting."
ExitDBFatalError ServerAuthError usageErr ->
"Failed to establish a connection. " <> jsonMessage usageErr
ExitDBFatalError ServerPgrstBug usageErr ->
"This is probably a bug in PostgREST, please report it at https://github.com/PostgREST/postgrest/issues. " <> jsonMessage usageErr
ExitDBFatalError ServerError42P05 usageErr ->
"If you are using connection poolers in transaction mode, try setting db-prepared-statements to false. " <> jsonMessage usageErr
ExitDBFatalError ServerError08P01 usageErr ->
"Connection poolers in statement mode are not supported." <> jsonMessage usageErr
SchemaCacheEmptyObs ->
T.decodeUtf8 . LBS.toStrict . Error.errorPayload $ Error.NoSchemaCacheError
SchemaCacheErrorObs dbSchemas extraPaths usageErr ->
"Failed to load the schema cache using "
<> "db-schemas=" <> T.intercalate "," (toList dbSchemas)
<> " and "
<> "db-extra-search-path=" <> T.intercalate "," extraPaths <> ". " <> jsonMessage usageErr
SchemaCacheQueriedObs resultTime ->
"Schema cache queried in " <> showMillis resultTime <> " milliseconds"
SchemaCacheSummaryObs summary ->
"Schema cache loaded " <> summary
SchemaCacheLoadedObs resultTime ->
"Schema cache loaded in " <> showMillis resultTime <> " milliseconds"
ConnectionRetryObs delay ->
"Attempting to reconnect to the database in " <> (show delay::Text) <> " seconds..."
QueryPgVersionError usageErr ->
"Failed to query the PostgreSQL version. " <> jsonMessage usageErr
DBListenStart host port fullName channel -> do
"Listener connected to " <> fullName <> " on " <> show (fold $ host <> fmap (":" <>) port) <> " and listening for database notifications on the " <> show channel <> " channel"
DBListenFail channel listenErr ->
"Failed listening for database notifications on the " <> show channel <> " channel. " <>
either showListenerConnError showListenerException listenErr
DBListenRetry delay ->
"Retrying listening for database notifications in " <> (show delay::Text) <> " seconds..."
DBListenBugCallQueryFix ->
"This is likely a PostgreSQL bug in the notification queue, executing the following to try to solve it: SELECT pg_notification_queue_usage();"
DBListenerGotSCacheMsg channel ->
"Received a schema cache reload message on the " <> show channel <> " channel"
DBListenerGotConfigMsg channel ->
"Received a config reload message on the " <> show channel <> " channel"
DBListenerConnectionCleanupFail ex ->
"Failed during listener connection cleanup: " <> showOnSingleLine '\t' (show ex)
QueryObs{} ->
mempty -- TODO pending refactor: The logic for printing the query cannot be done here. Join the observationMessage function into observationLogger to avoid this mempty.
ConfigReadErrorObs usageErr ->
"Failed to query database settings for the config parameters." <> jsonMessage usageErr
QueryRoleSettingsErrorObs usageErr ->
"Failed to query the role settings. " <> jsonMessage usageErr
QueryErrorCodeHighObs usageErr ->
jsonMessage usageErr
ConfigInvalidObs err ->
"Failed reloading config: " <> err
ConfigSucceededObs ->
"Config reloaded"
PoolInit poolSize ->
"Connection Pool initialized with a maximum size of " <> show poolSize <> " connections"
PoolAcqTimeoutObs usageErr ->
jsonMessage usageErr
HasqlPoolObs (SQL.ConnectionObservation uuid status) ->
"Connection " <> show uuid <> (
case status of
SQL.ConnectingConnectionStatus -> " is being established"
SQL.ReadyForUseConnectionStatus -> " is available"
SQL.InUseConnectionStatus -> " is used"
SQL.TerminatedConnectionStatus reason -> " is terminated due to " <> case reason of
SQL.AgingConnectionTerminationReason -> "max lifetime"
SQL.IdlenessConnectionTerminationReason -> "max idletime"
SQL.ReleaseConnectionTerminationReason -> "release"
SQL.NetworkErrorConnectionTerminationReason _ -> "network error" -- usage error is already logged, no need to repeat the same message.
)
PoolRequest ->
"Trying to borrow a connection from pool"
PoolRequestFullfilled ->
"Borrowed a connection from the pool"
PoolFlushed ->
"Database connection pool flushed"
JwtCacheLookup _ ->
"Looked up a JWT in JWT cache"
JwtCacheEviction ->
"Evicted entry from JWT cache"
TerminationUnixSignalObs signal ->
"Received termination unix signal " <> signal
WarpServerObs txt ->
"Warp server: " <> txt
where
showMillis :: Double -> Text
showMillis x = toS $ showFFloat (Just 1) x ""
jsonMessage err = T.decodeUtf8 . LBS.toStrict . Error.errorPayload $ Error.PgError False err
showListenerConnError :: SQL.ConnectionError -> Text
showListenerConnError = maybe "Connection error" (showOnSingleLine '\t' . T.decodeUtf8)
showListenerException :: SomeException -> Text
showListenerException = showOnSingleLine '\t' . show
showOnSingleLine :: Char -> Text -> Text
showOnSingleLine split txt = T.intercalate " " $ T.filter (/= split) <$> T.lines txt -- the errors from hasql-notifications come intercalated with "\t\n"
+53 -11
View File
@@ -5,7 +5,10 @@ Description : Metrics based on the Observation module. See Observation.hs.
-}
module PostgREST.Metrics
( init
, ConnTrack
, ConnStats (..)
, MetricsState (..)
, connectionCounts
, observationMetrics
, metricsToText
) where
@@ -17,12 +20,18 @@ import Prometheus
import PostgREST.Observation
import Protolude
import Control.Arrow ((&&&))
import Data.Bitraversable (bisequenceA)
import Data.Tuple.Extra (both)
import Data.UUID (UUID)
import qualified Focus
import Protolude
import qualified StmHamt.SizedHamt as SH
data MetricsState =
MetricsState {
poolTimeouts :: Counter,
poolAvailable :: Gauge,
connTrack :: ConnTrack,
poolWaiting :: Gauge,
poolMaxSize :: Gauge,
schemaCacheLoads :: Vector Label1 Counter,
@@ -36,7 +45,7 @@ init :: Int -> IO MetricsState
init configDbPoolSize = do
metricState <- MetricsState <$>
register (counter (Info "pgrst_db_pool_timeouts_total" "The total number of pool connection timeouts")) <*>
register (gauge (Info "pgrst_db_pool_available" "Available connections in the pool")) <*>
register (Metric ((identity &&& dbPoolAvailable) <$> connectionTracker)) <*>
register (gauge (Info "pgrst_db_pool_waiting" "Requests waiting to acquire a pool connection")) <*>
register (gauge (Info "pgrst_db_pool_max" "Max pool connections")) <*>
register (vector "status" $ counter (Info "pgrst_schema_cache_loads_total" "The total number of times the schema cache was loaded")) <*>
@@ -46,20 +55,28 @@ init configDbPoolSize = do
register (counter (Info "pgrst_jwt_cache_evictions_total" "The total number of JWT cache evictions"))
setGauge (poolMaxSize metricState) (fromIntegral configDbPoolSize)
pure metricState
where
dbPoolAvailable = (pure . noLabelsGroup (Info "pgrst_db_pool_available" "Available connections in the pool") GaugeType . calcAvailable <$>) . connectionCounts
where
calcAvailable = liftA2 (-) connected inUse
toSample name labels = Sample name labels . encodeUtf8 . show
noLabelsGroup info sampleType = SampleGroup info sampleType . pure . toSample (metricName info) mempty
-- Only some observations are used as metrics
observationMetrics :: MetricsState -> ObservationHandler
observationMetrics MetricsState{..} obs = case obs of
(PoolAcqTimeoutObs _) -> do
incCounter poolTimeouts
(HasqlPoolObs (SQL.ConnectionObservation _ status)) -> case status of
SQL.ReadyForUseConnectionStatus -> do
incGauge poolAvailable
SQL.InUseConnectionStatus -> do
decGauge poolAvailable
SQL.TerminatedConnectionStatus _ -> do
decGauge poolAvailable
SQL.ConnectingConnectionStatus -> pure ()
-- Handle pool observations with connection tracking
-- this is necessary because it is not possible
-- to accurately maintain open/in use conneciton counts
-- statelessly based only on pool observation events.
-- The reason is that hasql-pool emits TerminatedConnectionStatus
-- both for connections successfully established and failed when connecting.
-- When receiving TerminatedConnectionStatus we have to find out
-- if we can decrement established connection count. To do that we have to track
-- established connections.
(HasqlPoolObs sqlObs) -> trackConnections connTrack sqlObs
PoolRequest ->
incGauge poolWaiting
PoolRequestFullfilled ->
@@ -77,3 +94,28 @@ observationMetrics MetricsState{..} obs = case obs of
metricsToText :: IO LBS.ByteString
metricsToText = exportMetricsAsText
data ConnStats = ConnStats {
connected :: Int,
inUse :: Int
} deriving (Eq, Show)
data ConnTrack = ConnTrack { connTrackConnected :: SH.SizedHamt UUID, connTrackInUse :: SH.SizedHamt UUID }
connectionTracker :: IO ConnTrack
connectionTracker = ConnTrack <$> SH.newIO <*> SH.newIO
trackConnections :: ConnTrack -> SQL.Observation -> IO ()
trackConnections ConnTrack{..} (SQL.ConnectionObservation uuid status) = case status of
SQL.ReadyForUseConnectionStatus -> atomically $
SH.insert identity uuid connTrackConnected *>
SH.focus Focus.delete identity uuid connTrackInUse
SQL.TerminatedConnectionStatus _ -> atomically $
SH.focus Focus.delete identity uuid connTrackConnected *>
SH.focus Focus.delete identity uuid connTrackInUse
SQL.InUseConnectionStatus -> atomically $
SH.insert identity uuid connTrackInUse
_ -> mempty
connectionCounts :: ConnTrack -> IO ConnStats
connectionCounts = atomically . fmap (uncurry ConnStats) . bisequenceA . both SH.size . (connTrackConnected &&& connTrackInUse)
+1 -123
View File
@@ -1,5 +1,4 @@
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE LambdaCase #-}
{-|
Module : PostgREST.Observation
Description : This module holds an Observation type which is the core of Observability for PostgREST.
@@ -10,28 +9,21 @@ Description : This module holds an Observation type which is the core of Observa
module PostgREST.Observation
( Observation(..)
, ObsFatalError(..)
, observationMessage
, ObservationHandler
, showOnSingleLine
) where
import qualified Data.ByteString.Lazy as LBS
import Data.List.NonEmpty (toList)
import qualified Data.Text as T
import qualified Data.Text.Encoding as T
import qualified Hasql.Connection as SQL
import qualified Hasql.Pool as SQL
import qualified Hasql.Pool.Observation as SQL
import Network.HTTP.Types.Status (Status)
import Numeric (showFFloat)
import PostgREST.Config.PgVersion
import qualified PostgREST.Error as Error
import PostgREST.Query (MainQuery)
import Protolude hiding (toList)
data Observation
= AdminStartObs Text
| AdminServerCrashedObs SomeException
| AppStartObs ByteString
| AppServerAddressObs Text
| ExitUnsupportedPgVersion PgVersion PgVersion
@@ -73,117 +65,3 @@ data Observation
data ObsFatalError = ServerAuthError | ServerPgrstBug | ServerError42P05 | ServerError08P01
type ObservationHandler = Observation -> IO ()
observationMessage :: Observation -> Text
observationMessage = \case
AdminStartObs address ->
"Admin server listening on " <> address
AppStartObs ver ->
"Starting PostgREST " <> T.decodeUtf8 ver <> "..."
AppServerAddressObs address ->
"API server listening on " <> address
DBConnectedObs ver ->
"Successfully connected to " <> ver
ExitUnsupportedPgVersion pgVer minPgVer ->
"Cannot run in this PostgreSQL version (" <> pgvName pgVer <> "), PostgREST needs at least " <> pgvName minPgVer
ExitDBNoRecoveryObs ->
"Automatic recovery disabled, exiting."
ExitDBFatalError ServerAuthError usageErr ->
"Failed to establish a connection. " <> jsonMessage usageErr
ExitDBFatalError ServerPgrstBug usageErr ->
"This is probably a bug in PostgREST, please report it at https://github.com/PostgREST/postgrest/issues. " <> jsonMessage usageErr
ExitDBFatalError ServerError42P05 usageErr ->
"If you are using connection poolers in transaction mode, try setting db-prepared-statements to false. " <> jsonMessage usageErr
ExitDBFatalError ServerError08P01 usageErr ->
"Connection poolers in statement mode are not supported." <> jsonMessage usageErr
SchemaCacheEmptyObs ->
T.decodeUtf8 . LBS.toStrict . Error.errorPayload $ Error.NoSchemaCacheError
SchemaCacheErrorObs dbSchemas extraPaths usageErr ->
"Failed to load the schema cache using "
<> "db-schemas=" <> T.intercalate "," (toList dbSchemas)
<> " and "
<> "db-extra-search-path=" <> T.intercalate "," extraPaths
<> ". " <> jsonMessage usageErr
SchemaCacheQueriedObs resultTime ->
"Schema cache queried in " <> showMillis resultTime <> " milliseconds"
SchemaCacheSummaryObs summary ->
"Schema cache loaded " <> summary
SchemaCacheLoadedObs resultTime ->
"Schema cache loaded in " <> showMillis resultTime <> " milliseconds"
ConnectionRetryObs delay ->
"Attempting to reconnect to the database in " <> (show delay::Text) <> " seconds..."
QueryPgVersionError usageErr ->
"Failed to query the PostgreSQL version. " <> jsonMessage usageErr
DBListenStart host port fullName channel -> do
"Listener connected to " <> fullName <> " on " <> show (fold $ host <> fmap (":" <>) port) <> " and listening for database notifications on the " <> show channel <> " channel"
DBListenFail channel listenErr ->
"Failed listening for database notifications on the " <> show channel <> " channel. " <>
either showListenerConnError showListenerException listenErr
DBListenRetry delay ->
"Retrying listening for database notifications in " <> (show delay::Text) <> " seconds..."
DBListenBugCallQueryFix ->
"This is likely a PostgreSQL bug in the notification queue, executing the following to try to solve it: SELECT pg_notification_queue_usage();"
DBListenerGotSCacheMsg channel ->
"Received a schema cache reload message on the " <> show channel <> " channel"
DBListenerGotConfigMsg channel ->
"Received a config reload message on the " <> show channel <> " channel"
DBListenerConnectionCleanupFail ex ->
"Failed during listener connection cleanup: " <> showOnSingleLine '\t' (show ex)
QueryObs{} ->
mempty -- TODO pending refactor: The logic for printing the query cannot be done here. Join the observationMessage function into observationLogger to avoid this mempty.
ConfigReadErrorObs usageErr ->
"Failed to query database settings for the config parameters." <> jsonMessage usageErr
QueryRoleSettingsErrorObs usageErr ->
"Failed to query the role settings. " <> jsonMessage usageErr
QueryErrorCodeHighObs usageErr ->
jsonMessage usageErr
ConfigInvalidObs err ->
"Failed reloading config: " <> err
ConfigSucceededObs ->
"Config reloaded"
PoolInit poolSize ->
"Connection Pool initialized with a maximum size of " <> show poolSize <> " connections"
PoolAcqTimeoutObs usageErr ->
jsonMessage usageErr
HasqlPoolObs (SQL.ConnectionObservation uuid status) ->
"Connection " <> show uuid <> (
case status of
SQL.ConnectingConnectionStatus -> " is being established"
SQL.ReadyForUseConnectionStatus -> " is available"
SQL.InUseConnectionStatus -> " is used"
SQL.TerminatedConnectionStatus reason -> " is terminated due to " <> case reason of
SQL.AgingConnectionTerminationReason -> "max lifetime"
SQL.IdlenessConnectionTerminationReason -> "max idletime"
SQL.ReleaseConnectionTerminationReason -> "release"
SQL.NetworkErrorConnectionTerminationReason _ -> "network error" -- usage error is already logged, no need to repeat the same message.
)
PoolRequest ->
"Trying to borrow a connection from pool"
PoolRequestFullfilled ->
"Borrowed a connection from the pool"
PoolFlushed ->
"Database connection pool flushed"
JwtCacheLookup _ ->
"Looked up a JWT in JWT cache"
JwtCacheEviction ->
"Evicted entry from JWT cache"
TerminationUnixSignalObs signal ->
"Received termination unix signal " <> signal
WarpServerObs txt ->
"Warp server: " <> txt
where
showMillis :: Double -> Text
showMillis x = toS $ showFFloat (Just 1) x ""
jsonMessage err = T.decodeUtf8 . LBS.toStrict . Error.errorPayload $ Error.PgError False err
showListenerConnError :: SQL.ConnectionError -> Text
showListenerConnError = maybe "Connection error" (showOnSingleLine '\t' . T.decodeUtf8)
showListenerException :: SomeException -> Text
showListenerException = showOnSingleLine '\t' . show
showOnSingleLine :: Char -> Text -> Text
showOnSingleLine split txt = T.intercalate " " $ T.filter (/= split) <$> T.lines txt -- the errors from hasql-notifications come intercalated with "\t\n"
+5 -9
View File
@@ -67,10 +67,9 @@ import PostgREST.SchemaCache.Table (Column (..), ColumnMap,
import qualified PostgREST.MediaType as MediaType
import Control.Arrow ((&&&))
import qualified Data.FuzzySet as Fuzzy
import Control.Arrow ((&&&))
import qualified Data.FuzzySet as Fuzzy
import Protolude
import System.IO.Unsafe (unsafePerformIO)
type TablesFuzzyIndex = HM.HashMap Schema Fuzzy.FuzzySet
@@ -167,11 +166,9 @@ querySchemaCache conf@AppConfig{..} = do
let tabsWViewsPks = addViewPrimaryKeys tabs keyDeps
rels = addInverseRels $ addM2MRels tabsWViewsPks $ addViewM2OAndO2ORels keyDeps m2oRels
-- Add delay in loading schema cache when internal-schema-cache-load-sleep config is set
return $ delayEval configInternalSCLoadSleep $ removeInternal schemas $ SchemaCache {
return $ removeInternal schemas $ SchemaCache {
dbTables = tabsWViewsPks
-- Add delay in loading relationships when internal-schema-cache-relationship-load-sleep config is set
, dbRelationships = delayEval configInternalSCRelLoadSleep $ getOverrideRelationshipsMap rels cRels
, dbRelationships = getOverrideRelationshipsMap rels cRels
, dbRoutines = funcs
, dbRepresentations = reps
, dbMediaHandlers = HM.union mHdlers initialMediaHandlers -- the custom handlers will override the initial ones
@@ -185,7 +182,6 @@ querySchemaCache conf@AppConfig{..} = do
where
schemas = toList configDbSchemas
prepared = configDbPreparedStatements
delayEval confDelay result = maybe result (unsafePerformIO . (($> result) . (threadDelay . (1000 *) . fromIntegral))) confDelay
-- | overrides detected relationships with the computed relationships and gets the RelationshipsMap
getOverrideRelationshipsMap :: [Relationship] -> [Relationship] -> RelationshipsMap
@@ -452,7 +448,7 @@ funcsSqlQuery = encodeUtf8 [trimming|
bt.oid <> bt.base_type as rettype_is_composite_alias,
p.provolatile,
p.provariadic > 0 as hasvariadic,
lower((regexp_split_to_array((regexp_split_to_array(iso_config, '='))[2], ','))[1]) AS transaction_isolation_level,
(regexp_split_to_array((regexp_split_to_array(iso_config, '='))[2], ','))[1] AS transaction_isolation_level,
coalesce(func_settings.kvs, '{}') as kvs
FROM pg_proc p
LEFT JOIN arguments a ON a.oid = p.oid
+7 -1
View File
@@ -9,9 +9,15 @@ nix:
pure: false
extra-deps:
- auto-update-0.2.7
- configurator-pg-0.2.11
- fuzzyset-0.2.4
- hasql-pool-1.0.1
- jose-jwt-0.10.0
- http-semantics-0.4.0
- http2-5.4.0
- jose-jwt-0.9.6
- network-control-0.1.7
- postgresql-libpq-0.10.1.0
- streaming-commons-0.2.3.1
- time-manager-0.3.2
- warp-3.4.14
+46 -4
View File
@@ -4,6 +4,13 @@
# https://docs.haskellstack.org/en/stable/topics/lock_files
packages:
- completed:
hackage: auto-update-0.2.7@sha256:32ca6ce351604a17ee79e4086939f798f958f8407478288e9adb7adeb194c6ea,1670
pantry-tree:
sha256: 641faa7d5ee516195ecd4b538f170722d56f84e3bc5714c6bbe8123849b49983
size: 1105
original:
hackage: auto-update-0.2.7
- completed:
hackage: configurator-pg-0.2.11@sha256:de0c56386591e85159436b0af04a8f15a4f4e156354e99709676c2c2ee959505,2850
pantry-tree:
@@ -26,12 +33,33 @@ packages:
original:
hackage: hasql-pool-1.0.1
- completed:
hackage: jose-jwt-0.10.0@sha256:6ed175a01c721e317ceea15eb251a81de145c03711a977517935633a5cdec1d4,3546
hackage: http-semantics-0.4.0@sha256:da8a98d542b2032cc12590847179577b0208a52bb3b9aa9a07c08d27d2a1714c,1513
pantry-tree:
sha256: 58649e68e2d1adb47d8ed8741bd27ac23a2f19e3ee62bc28a68ac8642b3e0858
size: 1231
sha256: d0e08875907c0fbff71813747fd93ce7bfd4e3d4a6b979df5c137430a9130f1d
size: 1188
original:
hackage: jose-jwt-0.10.0
hackage: http-semantics-0.4.0
- completed:
hackage: http2-5.4.0@sha256:1e9f6f5f32bfb3176136f35e041aa279bc456e81d4674ddaaeaa7c0d091be0c7,10624
pantry-tree:
sha256: 5c89815392d85d854efe75cf2279f58ff9f5e4b8fc1f83c55de93191edd128da
size: 44864
original:
hackage: http2-5.4.0
- completed:
hackage: jose-jwt-0.9.6@sha256:cc234805da58fc75bc4c11af3db2dabf920f2c7d5d8b9a2b73bdb7c024b8d087,3557
pantry-tree:
sha256: 0bcaa403f0d6f3f7ad993d4d9e5d8f3dfd7e88c4e5e64b0219d7a1a03cb31e31
size: 1288
original:
hackage: jose-jwt-0.9.6
- completed:
hackage: network-control-0.1.7@sha256:bfe3318c5cf6573dd585b126ec1197eae291a9a6df178006bf19da5c8aa14d68,1255
pantry-tree:
sha256: 6deb24c404f6e592be1f391f04c97f66169f97cacb0dd37403a6aa1e781846ef
size: 619
original:
hackage: network-control-0.1.7
- completed:
hackage: postgresql-libpq-0.10.1.0@sha256:6b580c9d5068e78eecc13e655b2885c8e79cdacfca513c5d1e5a6b9dc61d9758,3166
pantry-tree:
@@ -46,6 +74,20 @@ packages:
size: 2374
original:
hackage: streaming-commons-0.2.3.1
- completed:
hackage: time-manager-0.3.2@sha256:74c16026c8592802d8a1cd9510c0223dad247b2a9ca791fa0153d243b56cc09e,1290
pantry-tree:
sha256: f282e2630df833732bde29944d4b0ac0d86399bcce80c1a3d523c2a5651b9e49
size: 461
original:
hackage: time-manager-0.3.2
- completed:
hackage: warp-3.4.14@sha256:66b82af637f79ae4d39f7373b39491067f6d059b2581cb4caf18760f1d82e686,10066
pantry-tree:
sha256: 6db2e6d37acebd24d4e73d506481a4a2e0c53b35099756131a3852c8c8b84b45
size: 4175
original:
hackage: warp-3.4.14
snapshots:
- completed:
sha256: 238fa745b64f91184f9aa518fe04bdde6552533d169b0da5256670df83a0f1a9
@@ -565,6 +565,23 @@
pdSchema: public
pdVolatility: Volatile
- - qiName: get_work_mem
qiSchema: public
- - pdDescription: null
pdFuncSettings: []
pdHasVariadic: false
pdName: get_work_mem
pdParams: []
pdReturnType:
contents:
contents:
qiName: text
qiSchema: pg_catalog
tag: Scalar
tag: Single
pdSchema: public
pdVolatility: Volatile
- - qiName: notify_do_nothing
qiSchema: public
- - pdDescription: null
+1
View File
@@ -10,6 +10,7 @@ CONFIGSDIR = BASEDIR / "configs"
FIXTURES = yaml.load(
(BASEDIR / "fixtures/fixtures.yaml").read_text(), Loader=yaml.Loader
)
NGINX_BIN = shutil.which("nginx")
POSTGREST_BIN = shutil.which("postgrest")
SECRET = "reallyreallyreallyreallyverysafe"
+3
View File
@@ -0,0 +1,3 @@
SET client_min_messages = WARNING; -- suppress "NOTICE: ..." messages which pollute the log
SET check_function_bodies = false; -- to allow conditionals based on the pg version
SET search_path = public;
+8 -299
View File
@@ -1,300 +1,9 @@
-- Load all IO tests fixtures for PostgREST
\set ON_ERROR_STOP on
\ir database.sql
\ir db_config.sql
set check_function_bodies = false; -- to allow conditionals based on the pg version
set search_path to public;
CREATE ROLE postgrest_test_anonymous;
ALTER ROLE :"PGUSER" SET pgrst.db_anon_role = 'postgrest_test_anonymous';
CREATE ROLE postgrest_test_author;
CREATE ROLE postgrest_test_serializable;
alter role postgrest_test_serializable set default_transaction_isolation = 'serializable';
CREATE ROLE postgrest_test_repeatable_read;
alter role postgrest_test_repeatable_read set default_transaction_isolation = 'REPEATABLE READ';
CREATE ROLE postgrest_test_w_superuser_settings;
alter role postgrest_test_w_superuser_settings set log_min_duration_statement = 1;
alter role postgrest_test_w_superuser_settings set log_min_messages = 'fatal';
DO $do$BEGIN
IF (SELECT current_setting('server_version_num')::INT >= 150000) THEN
ALTER ROLE postgrest_test_w_superuser_settings SET log_min_duration_sample = 12345;
GRANT SET ON PARAMETER log_min_duration_sample to "Postgrest_Test_Authenticator";
END IF;
END$do$;
GRANT
postgrest_test_anonymous, postgrest_test_author,
postgrest_test_serializable, postgrest_test_repeatable_read,
postgrest_test_w_superuser_settings TO :"PGUSER";
CREATE SCHEMA v1;
GRANT USAGE ON SCHEMA v1 TO postgrest_test_anonymous;
CREATE SCHEMA test;
GRANT USAGE ON SCHEMA test TO postgrest_test_anonymous;
CREATE TABLE authors_only ();
GRANT SELECT ON authors_only TO postgrest_test_author;
CREATE TABLE projects AS SELECT FROM generate_series(1,5);
GRANT SELECT ON projects TO postgrest_test_anonymous, postgrest_test_w_superuser_settings;
create function get_guc_value(name text) returns text as $$
select nullif(current_setting(name), '')::text;
$$ language sql;
create function v1.get_guc_value(name text) returns text as $$
select nullif(current_setting(name), '')::text;
$$ language sql;
create function uses_prepared_statements() returns bool as $$
select count(name) > 0 from pg_catalog.pg_prepared_statements
$$ language sql;
create function change_max_rows_config(val int, notify bool default false) returns void as $_$
begin
execute format($$
alter role "Postgrest_Test_Authenticator" set pgrst.db_max_rows = %L;
$$, val);
if notify then
perform pg_notify('pgrst', 'reload config');
end if;
end $_$ volatile security definer language plpgsql ;
create function reset_max_rows_config() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" reset pgrst.db_max_rows;
end $_$ volatile security definer language plpgsql ;
create function change_db_schema_and_full_reload(schemas text) returns void as $_$
begin
execute format($$
alter role "Postgrest_Test_Authenticator" set pgrst.db_schemas = %L;
$$, schemas);
perform pg_notify('pgrst', 'reload config');
perform pg_notify('pgrst', 'reload schema');
end $_$ volatile security definer language plpgsql ;
create function v1.reset_db_schema_config() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" reset pgrst.db_schemas;
perform pg_notify('pgrst', 'reload config');
perform pg_notify('pgrst', 'reload schema');
end $_$ volatile security definer language plpgsql ;
create function invalid_role_claim_key_reload() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" set pgrst.jwt_role_claim_key = 'test';
perform pg_notify('pgrst', 'reload config');
end $_$ volatile security definer language plpgsql ;
create function notify_do_nothing() returns void as $_$
notify pgrst, 'nothing';
$_$ language sql;
create function do_nothing() returns void as $_$
$_$ language sql;
create function reset_invalid_role_claim_key() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" reset pgrst.jwt_role_claim_key;
perform pg_notify('pgrst', 'reload config');
end $_$ volatile security definer language plpgsql ;
create function reload_pgrst_config() returns void as $_$
begin
perform pg_notify('pgrst', 'reload config');
end $_$ language plpgsql ;
create or replace function sleep(seconds double precision) returns void as $$
select pg_sleep(seconds);
$$ language sql;
create or replace function hello() returns text as $$
select 'hello'::text;
$$ language sql;
create table cats(id uuid primary key, name text);
grant all on cats to postgrest_test_anonymous;
create function drop_change_cats() returns void
language sql security definer
as $$
drop table cats;
create table cats(id bigint primary key, name text);
grant all on table cats to postgrest_test_anonymous;
notify pgrst, 'reload schema';
$$;
alter role postgrest_test_anonymous set statement_timeout to '2s';
alter role postgrest_test_author set statement_timeout to '10s';
create function change_role_statement_timeout(timeout text) returns void as $_$
begin
execute format($$
alter role current_user set statement_timeout = %L;
$$, timeout);
end $_$ volatile language plpgsql ;
create table items as select x as id from generate_series(1,5) x;
create view items_w_isolation_level as
select
id,
current_setting('transaction_isolation', true) as isolation_level
from items;
grant all on items_w_isolation_level to postgrest_test_anonymous, postgrest_test_repeatable_read, postgrest_test_serializable;
create function default_isolation_level()
returns text as $$
select current_setting('transaction_isolation', true);
$$
language sql;
create function serializable_isolation_level()
returns text as $$
select current_setting('transaction_isolation', true);
$$
language sql set default_transaction_isolation = 'serializable';
create function repeatable_read_isolation_level()
returns text as $$
select current_setting('transaction_isolation', true);
$$
language sql set default_transaction_isolation = 'REPEATABLE READ';
create or replace function create_function() returns void as $_$
drop function if exists mult_them(int, int);
create or replace function mult_them(a int, b int) returns int as $$
select a*b;
$$ language sql;
notify pgrst, 'reload schema';
$_$ language sql security definer;
create or replace function migrate_function() returns void as $_$
drop function if exists mult_them(int, int);
create or replace function mult_them(c int, d int) returns int as $$
select c*d;
$$ language sql;
notify pgrst, 'reload schema';
$_$ language sql security definer;
create or replace function get_pgrst_version() returns text
language sql
as $$
select application_name
from pg_stat_activity
where application_name ilike 'postgrest%'
limit 1;
$$;
create function terminate_pgrst(appname text) returns setof record as $$
select pg_terminate_backend(pid) from pg_stat_activity where application_name iLIKE '%' || appname || '%';
$$ language sql security definer;
create or replace function one_sec_timeout() returns void as $$
select pg_sleep(3);
$$ language sql set statement_timeout = '1s';
create or replace function four_sec_timeout() returns void as $$
select pg_sleep(3);
$$ language sql set statement_timeout = '4s';
create function get_postgres_version() returns int as $$
select current_setting('server_version_num')::int;
$$ language sql;
create or replace function rpc_work_mem() returns items as $$
select 1
$$ language sql
set work_mem = '6000';
create or replace function rpc_with_one_hoisted() returns items as $$
select 1
$$ language sql
set work_mem = '3000'
set statement_timeout = '7s';
create or replace function rpc_with_two_hoisted() returns items as $$
select 1
$$ language sql
set work_mem = '5000'
set statement_timeout = '10s';
create function get_work_mem(items) returns text as $$
select current_setting('work_mem', true) as work_mem
$$ language sql;
create function get_statement_timeout(items) returns text as $$
select current_setting('statement_timeout', true) as statement_timeout
$$ language sql;
create function change_db_schemas_config() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" set pgrst.db_schemas = 'test';
end $_$ volatile security definer language plpgsql;
create function reset_db_schemas_config() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" reset pgrst.db_schemas;
end $_$ volatile security definer language plpgsql ;
create function test.get_current_schema() returns text as $$
select current_schema()::text;
$$ language sql;
create or replace function root() returns json as $_$
select '{"swagger": "2.0"}'::json;
$_$ language sql;
create view infinite_recursion as
select * from projects;
create or replace view infinite_recursion as
select * from infinite_recursion;
create or replace function "true"() returns boolean as $_$
select true;
$_$ language sql;
create or replace function notify_pgrst() returns void as $$
notify pgrst;
$$ language sql;
-- directors and films table can be used for resource embedding tests
create table directors (
id int primary key,
name text
);
create table films (
id int primary key,
title text,
director_id int,
constraint fk_director
foreign key (director_id) references directors (id)
on update cascade
on delete cascade
);
-- data to test resource embedding
truncate table directors cascade;
insert into directors
values (1, 'quentin tarantino'),
(2, 'christopher nolan'),
(3, 'yorgos lathinmos');
truncate table films cascade;
insert into films
values (1, 'pulp fiction', 1),
(2, 'intersteller',2),
(3, 'dogtooth',3),
(4, 'reservoir dogs', 1);
GRANT SELECT ON directors, films TO postgrest_test_anonymous, postgrest_test_w_superuser_settings;
\ir roles.sql
\ir schema.sql
\ir privileges.sql
+9
View File
@@ -0,0 +1,9 @@
GRANT USAGE ON SCHEMA v1 TO postgrest_test_anonymous;
GRANT USAGE ON SCHEMA test TO postgrest_test_anonymous;
GRANT SELECT ON authors_only TO postgrest_test_author;
GRANT SELECT ON projects TO postgrest_test_anonymous, postgrest_test_w_superuser_settings;
GRANT SELECT ON directors, films TO postgrest_test_anonymous, postgrest_test_w_superuser_settings;
GRANT ALL ON cats TO postgrest_test_anonymous;
GRANT ALL ON items_w_isolation_level TO postgrest_test_anonymous, postgrest_test_repeatable_read, postgrest_test_serializable;
+30
View File
@@ -0,0 +1,30 @@
DROP ROLE IF EXISTS
postgrest_test_anonymous, postgrest_test_author,
postgrest_test_serializable, postgrest_test_repeatable_read,
postgrest_test_w_superuser_settings;
CREATE ROLE postgrest_test_anonymous;
CREATE ROLE postgrest_test_author;
CREATE ROLE postgrest_test_serializable;
CREATE ROLE postgrest_test_repeatable_read;
CREATE ROLE postgrest_test_w_superuser_settings;
CREATE ROLE postgrest_test_work_mem;
GRANT
postgrest_test_anonymous, postgrest_test_author,
postgrest_test_serializable, postgrest_test_repeatable_read,
postgrest_test_w_superuser_settings, postgrest_test_work_mem TO :"PGUSER";
GRANT postgrest_test_anonymous TO timeout_authenticator;
ALTER ROLE :"PGUSER" SET pgrst.db_anon_role = 'postgrest_test_anonymous';
ALTER ROLE postgrest_test_serializable SET default_transaction_isolation = 'serializable';
ALTER ROLE postgrest_test_repeatable_read SET default_transaction_isolation = 'REPEATABLE READ';
ALTER ROLE postgrest_test_w_superuser_settings SET log_min_duration_statement = 1;
ALTER ROLE postgrest_test_w_superuser_settings SET log_min_messages = 'fatal';
ALTER ROLE postgrest_test_anonymous SET statement_timeout TO '2s';
ALTER ROLE postgrest_test_author SET statement_timeout TO '10s';
ALTER ROLE postgrest_test_work_mem SET work_mem TO '3MB';
+264
View File
@@ -0,0 +1,264 @@
DROP SCHEMA IF EXISTS v1, test;
CREATE SCHEMA v1;
CREATE SCHEMA test;
CREATE TABLE authors_only ();
CREATE TABLE projects AS SELECT FROM generate_series(1,5);
CREATE TABLE cats(id uuid primary key, name text);
CREATE TABLE items AS SELECT x AS id FROM generate_series(1,5) x;
-- directors and films table can be used for resource embedding tests
CREATE TABLE directors (
id int primary key,
name text
);
CREATE TABLE films (
id int primary key,
title text,
director_id int,
constraint fk_director
foreign key (director_id) references directors (id)
on update cascade
on delete cascade
);
-- data to test resource embedding
TRUNCATE TABLE directors CASCADE;
INSERT INTO directors
VALUES (1, 'quentin tarantino'),
(2, 'christopher nolan'),
(3, 'yorgos lathinmos');
TRUNCATE TABLE films CASCADE;
INSERT INTO films
VALUES (1, 'pulp fiction', 1),
(2, 'intersteller',2),
(3, 'dogtooth',3),
(4, 'reservoir dogs', 1);
DO $do$BEGIN
IF (SELECT current_setting('server_version_num')::INT >= 150000) THEN
ALTER ROLE postgrest_test_w_superuser_settings SET log_min_duration_sample = 12345;
GRANT SET ON PARAMETER log_min_duration_sample to "Postgrest_Test_Authenticator";
END IF;
END$do$;
create function get_guc_value(name text) returns text as $$
select nullif(current_setting(name), '')::text;
$$ language sql;
create function v1.get_guc_value(name text) returns text as $$
select nullif(current_setting(name), '')::text;
$$ language sql;
create function uses_prepared_statements() returns bool as $$
select count(name) > 0 from pg_catalog.pg_prepared_statements
$$ language sql;
create function change_max_rows_config(val int, notify bool default false) returns void as $_$
begin
execute format($$
alter role "Postgrest_Test_Authenticator" set pgrst.db_max_rows = %L;
$$, val);
if notify then
perform pg_notify('pgrst', 'reload config');
end if;
end $_$ volatile security definer language plpgsql ;
create function reset_max_rows_config() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" reset pgrst.db_max_rows;
end $_$ volatile security definer language plpgsql ;
create function change_db_schema_and_full_reload(schemas text) returns void as $_$
begin
execute format($$
alter role "Postgrest_Test_Authenticator" set pgrst.db_schemas = %L;
$$, schemas);
perform pg_notify('pgrst', 'reload config');
perform pg_notify('pgrst', 'reload schema');
end $_$ volatile security definer language plpgsql ;
create function v1.reset_db_schema_config() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" reset pgrst.db_schemas;
perform pg_notify('pgrst', 'reload config');
perform pg_notify('pgrst', 'reload schema');
end $_$ volatile security definer language plpgsql ;
create function invalid_role_claim_key_reload() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" set pgrst.jwt_role_claim_key = 'test';
perform pg_notify('pgrst', 'reload config');
end $_$ volatile security definer language plpgsql ;
create function notify_do_nothing() returns void as $_$
notify pgrst, 'nothing';
$_$ language sql;
create function do_nothing() returns void as $_$
$_$ language sql;
create function reset_invalid_role_claim_key() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" reset pgrst.jwt_role_claim_key;
perform pg_notify('pgrst', 'reload config');
end $_$ volatile security definer language plpgsql ;
create function reload_pgrst_config() returns void as $_$
begin
perform pg_notify('pgrst', 'reload config');
end $_$ language plpgsql ;
create or replace function sleep(seconds double precision) returns void as $$
select pg_sleep(seconds);
$$ language sql;
create or replace function hello() returns text as $$
select 'hello'::text;
$$ language sql;
create function drop_change_cats() returns void
language sql security definer
as $$
drop table cats;
create table cats(id bigint primary key, name text);
grant all on table cats to postgrest_test_anonymous;
notify pgrst, 'reload schema';
$$;
create function change_role_statement_timeout(timeout text) returns void as $_$
begin
execute format($$
alter role current_user set statement_timeout = %L;
$$, timeout);
end $_$ volatile language plpgsql ;
create view items_w_isolation_level as
select
id,
current_setting('transaction_isolation', true) as isolation_level
from items;
create function default_isolation_level()
returns text as $$
select current_setting('transaction_isolation', true);
$$
language sql;
create function serializable_isolation_level()
returns text as $$
select current_setting('transaction_isolation', true);
$$
language sql set default_transaction_isolation = 'serializable';
create function repeatable_read_isolation_level()
returns text as $$
select current_setting('transaction_isolation', true);
$$
language sql set default_transaction_isolation = 'REPEATABLE READ';
create or replace function create_function() returns void as $_$
drop function if exists mult_them(int, int);
create or replace function mult_them(a int, b int) returns int as $$
select a*b;
$$ language sql;
notify pgrst, 'reload schema';
$_$ language sql security definer;
create or replace function migrate_function() returns void as $_$
drop function if exists mult_them(int, int);
create or replace function mult_them(c int, d int) returns int as $$
select c*d;
$$ language sql;
notify pgrst, 'reload schema';
$_$ language sql security definer;
create or replace function get_pgrst_version() returns text
language sql
as $$
select application_name
from pg_stat_activity
where application_name ilike 'postgrest%'
limit 1;
$$;
create function terminate_pgrst(appname text) returns setof record as $$
select pg_terminate_backend(pid) from pg_stat_activity where application_name iLIKE '%' || appname || '%';
$$ language sql security definer;
create or replace function one_sec_timeout() returns void as $$
select pg_sleep(3);
$$ language sql set statement_timeout = '1s';
create or replace function four_sec_timeout() returns void as $$
select pg_sleep(3);
$$ language sql set statement_timeout = '4s';
create function get_postgres_version() returns int as $$
select current_setting('server_version_num')::int;
$$ language sql;
create or replace function rpc_work_mem() returns items as $$
select 1
$$ language sql
set work_mem = '6000';
create or replace function rpc_with_one_hoisted() returns items as $$
select 1
$$ language sql
set work_mem = '3000'
set statement_timeout = '7s';
create or replace function rpc_with_two_hoisted() returns items as $$
select 1
$$ language sql
set work_mem = '5000'
set statement_timeout = '10s';
create function get_work_mem(items) returns text as $$
select current_setting('work_mem', true) as work_mem
$$ language sql;
create function get_statement_timeout(items) returns text as $$
select current_setting('statement_timeout', true) as statement_timeout
$$ language sql;
create function change_db_schemas_config() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" set pgrst.db_schemas = 'test';
end $_$ volatile security definer language plpgsql;
create function reset_db_schemas_config() returns void as $_$
begin
alter role "Postgrest_Test_Authenticator" reset pgrst.db_schemas;
end $_$ volatile security definer language plpgsql ;
create function test.get_current_schema() returns text as $$
select current_schema()::text;
$$ language sql;
create or replace function root() returns json as $_$
select '{"swagger": "2.0"}'::json;
$_$ language sql;
create view infinite_recursion as
select * from projects;
create or replace view infinite_recursion as
select * from infinite_recursion;
create or replace function "true"() returns boolean as $_$
select true;
$_$ language sql;
create or replace function notify_pgrst() returns void as $$
notify pgrst;
$$ language sql;
create or replace function get_work_mem() returns text as $$
select current_setting('work_mem', true);
$$ language sql;
+13
View File
@@ -0,0 +1,13 @@
# the PG* variables are replaced by preprocessing, not done by nginx itself
daemon off;
pid ./nginx.pid;
events {}
stream {
server {
listen unix:$PGPROXYHOST/.s.PGSQL.5432;
proxy_timeout $PGPROXY_TIMEOUT;
proxy_pass unix:$PGHOST/.s.PGSQL.5432;
}
}
+60 -3
View File
@@ -2,18 +2,20 @@
import contextlib
import dataclasses
import enum
import os
import pathlib
import socket
import subprocess
import tempfile
import time
import string
import urllib.parse
import requests
import requests_unixsocket
from config import POSTGREST_BIN, hpctixfile
from config import POSTGREST_BIN, NGINX_BIN, hpctixfile
def sleep_until_postgrest_scache_reload():
@@ -35,6 +37,13 @@ class PostgrestTimedOut(Exception):
"Connecting to PostgREST endpoint timed out."
class Admin(str, enum.Enum):
"Admin endpoint to wait for before yielding a PostgREST process."
live = "live"
ready = "ready"
class PostgrestSession(requests_unixsocket.Session):
"HTTP client session directed at a PostgREST endpoint."
@@ -86,7 +95,7 @@ def run(
env=None,
port=None,
host=None,
wait_for_readiness=True,
wait_for=Admin.ready,
wait_max_seconds=1,
no_pool_connection_available=False,
no_startup_stdout=True,
@@ -138,8 +147,10 @@ def run(
process.stdin.write(stdin or b"")
process.stdin.close()
if wait_for_readiness:
if wait_for == Admin.ready:
wait_until_status_code(adminurl + "/ready", wait_max_seconds, 200)
elif wait_for == Admin.live:
wait_until_status_code(adminurl + "/live", wait_max_seconds, 200)
if no_startup_stdout:
process.stdout.read()
@@ -165,6 +176,52 @@ def run(
process.wait()
@contextlib.contextmanager
def run_pgproxy(env=None, proxy_timeout="1s"):
"Run nginx as a unix socket proxy for PostgreSQL and expose PGPROXYHOST."
env = dict(os.environ if env is None else env)
with tempfile.TemporaryDirectory() as tmpdir:
# build a <tmpdir>/conf/ so `nginx -p` picks the config automatically
tmpdir = pathlib.Path(tmpdir)
conf_dir = tmpdir / "conf"
conf_dir.mkdir(parents=True)
nginx_env = dict(env)
nginx_env["PGPROXYHOST"] = str(tmpdir)
nginx_env["PGPROXY_TIMEOUT"] = proxy_timeout
source_conf = pathlib.Path("test/io/nginx/nginx.conf")
out_conf = conf_dir / "nginx.conf"
out_conf.write_text(
string.Template(source_conf.read_text()).substitute(nginx_env)
)
process = subprocess.Popen(
[NGINX_BIN, "-p", str(tmpdir), "-e", "stderr"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
env=nginx_env,
)
if process.poll() is not None:
(_, stderr_output) = process.communicate(timeout=1)
raise RuntimeError(
f"{NGINX_BIN} exited with {process.returncode}: {stderr_output}"
)
try:
yield str(tmpdir)
finally:
process.terminate()
try:
process.wait(timeout=1)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
def freeport(used_ports=None):
"Find an unused free port on localhost."
while True:
+1 -1
View File
@@ -167,7 +167,7 @@ def test_fail_with_invalid_password(defaultenv):
"Connecting with an invalid password should fail without retries."
uri = f'postgresql://?dbname={defaultenv["PGDATABASE"]}&host={defaultenv["PGHOST"]}&user=some_protected_user&password=invalid_pass'
env = {**defaultenv, "PGRST_DB_URI": uri}
with run(env=env, wait_for_readiness=False) as postgrest:
with run(env=env, wait_for=None) as postgrest:
exitCode = wait_until_exit(postgrest)
assert exitCode == 1
+127 -94
View File
@@ -8,12 +8,19 @@ import time
import pytest
from config import CONFIGSDIR, FIXTURES, SECRET
from util import Thread, jwtauthheader, parse_server_timings_header
from util import (
Thread,
jwtauthheader,
parse_server_timings_header,
match_log,
)
from postgrest import (
Admin,
freeport,
is_ipv6,
reset_statement_timeout,
run,
run_pgproxy,
set_statement_timeout,
sleep_until_postgrest_config_reload,
sleep_until_postgrest_full_reload,
@@ -22,17 +29,18 @@ from postgrest import (
)
def match_log(output, matchers):
ito = iter(output)
itm = iter(matchers)
nextMatcher = next(itm, None)
while nextMatcher is not None and (line := next(ito, None)) is not None:
if re.match(nextMatcher, line) is not None:
nextMatcher = next(itm, None)
if nextMatcher is not None:
raise AssertionError(
f"Expected log line matching {nextMatcher} not found in output"
)
def psql_as_superuser(query):
subprocess.check_call(
[
"psql",
"--username",
"postgres",
"--set",
"ON_ERROR_STOP=1",
"-c",
query,
]
)
def test_connect_with_dburi(dburi, defaultenv):
@@ -597,7 +605,7 @@ def test_admin_ready_includes_schema_cache_state(defaultenv, metapostgrest):
env = {
**defaultenv,
"PGUSER": role,
"PGRST_DB_ANON_ROLE": role,
"PGRST_DB_ANON_ROLE": "postgrest_test_anonymous",
"PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP": "500",
}
@@ -964,6 +972,61 @@ def test_notify_reloading_catalog_cache(defaultenv):
assert response.status_code == 200
def test_stale_schema_cache_dropped_table_returns_database_error(defaultenv):
"dropped table should return a database error while schema cache is stale"
internal_sleep = 2
env = {
**defaultenv,
"PGRST_DB_POOL": "2",
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP": str(internal_sleep * 1000),
}
try:
psql_as_superuser(
"""
drop table if exists stale_schema_cache_items;
create table stale_schema_cache_items(id int primary key);
insert into stale_schema_cache_items values (1);
grant select on stale_schema_cache_items to postgrest_test_anonymous;
"""
)
with run(env=env, wait_max_seconds=10) as postgrest:
response = postgrest.session.get("/stale_schema_cache_items")
assert response.status_code == 200
psql_as_superuser(
"""
drop table stale_schema_cache_items;
notify pgrst, 'reload schema';
"""
)
response = postgrest.session.get("/stale_schema_cache_items")
payload = response.json()
assert response.status_code == 404
assert payload["code"] == "42P01"
assert (
payload["message"]
== 'relation "public.stale_schema_cache_items" does not exist'
)
time.sleep(internal_sleep + 0.3)
response = postgrest.session.get("/stale_schema_cache_items")
payload = response.json()
assert response.status_code == 404
assert payload["code"] == "PGRST205"
assert (
payload["message"]
== "Could not find the table 'public.stale_schema_cache_items' in the schema cache"
)
finally:
psql_as_superuser("drop table if exists stale_schema_cache_items;")
def test_role_settings(defaultenv):
"statement_timeout should be set per role"
@@ -1086,7 +1149,7 @@ def test_schema_cache_concurrent_notifications(slow_schema_cache_env):
int(slow_schema_cache_env["PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP"]) / 1000
)
with run(env=slow_schema_cache_env, wait_for_readiness=False) as postgrest:
with run(env=slow_schema_cache_env, wait_for=None) as postgrest:
time.sleep(2 * internal_sleep + 0.1) # wait for readiness manually
# first request, create a function and set a schema cache reload in progress
@@ -1138,31 +1201,6 @@ def test_schema_cache_query_sleep_logs(defaultenv):
assert 1000 < observed_ms < 2000
def test_schema_cache_load_sleep_logs(defaultenv):
"""Schema cache load sleep should be reflected in the logged load duration."""
env = {
**defaultenv,
"PGRST_INTERNAL_SCHEMA_CACHE_LOAD_SLEEP": "1000",
}
log_pattern = re.compile(r"Schema cache loaded in ([\d.]+) milliseconds")
with run(env=env, wait_max_seconds=3, no_startup_stdout=False) as postgrest:
observed_ms = None
collected = []
lines = postgrest.read_stdout(nlines=10)
collected.extend(lines)
for line in lines:
match = log_pattern.search(line)
if match:
observed_ms = float(match.group(1))
break
assert observed_ms is not None
assert 1000 < observed_ms < 2000
@pytest.mark.parametrize("dburi_type", ["no_params", "no_params_qmark", "with_params"])
def test_get_pgrst_version_with_uri_connection_string(dburi_type, dburi, defaultenv):
"The fallback_application_name should be added to the db-uri if it has a URI format"
@@ -1277,7 +1315,7 @@ def test_fail_with_invalid_dbname_and_automatic_recovery_disabled(defaultenv):
"PGRST_DB_POOL_AUTOMATIC_RECOVERY": "false",
}
with run(env=env, wait_for_readiness=False) as postgrest:
with run(env=env, wait_for=None) as postgrest:
exitCode = wait_until_exit(postgrest)
assert exitCode == 1
@@ -1566,7 +1604,7 @@ def test_log_error_when_empty_schema_cache_on_startup_to_stderr(defaultenv):
"PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP": "300",
}
with run(env=env, wait_for_readiness=False) as postgrest:
with run(env=env, wait_for=None) as postgrest:
postgrest.wait_until_scache_starts_loading()
response = postgrest.session.get("/projects")
@@ -1587,7 +1625,7 @@ def test_no_double_schema_cache_reload_on_empty_schema(defaultenv):
"PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP": "300",
}
with run(env=env, port=freeport(), wait_for_readiness=False) as postgrest:
with run(env=env, port=freeport(), wait_for=None) as postgrest:
postgrest.wait_until_scache_starts_loading()
response = postgrest.session.get("/projects")
@@ -1669,7 +1707,7 @@ def test_schema_cache_error_observation(defaultenv):
"PGRST_DB_EXTRA_SEARCH_PATH": "x",
}
with run(env=env, no_startup_stdout=False, wait_for_readiness=False) as postgrest:
with run(env=env, no_startup_stdout=False, wait_for=None) as postgrest:
# TODO: postgrest should exit here, instead it keeps retrying
# exitCode = wait_until_exit(postgrest)
# assert exitCode == 1
@@ -1690,7 +1728,7 @@ def test_log_listener_connection_errors(defaultenv):
"PGRST_DB_CHANNEL_ENABLED": "true",
}
with run(env=env, no_startup_stdout=False, wait_for_readiness=False) as postgrest:
with run(env=env, no_startup_stdout=False, wait_for=None) as postgrest:
output = postgrest.read_stdout(nlines=5)
assert any(
'Failed listening for database notifications on the "pgrst" channel. could not translate host name "no_host" to address:'
@@ -1707,7 +1745,7 @@ def test_log_listener_connection_start(defaultenv):
"PGRST_DB_CHANNEL_ENABLED": "true",
}
with run(env=env, no_startup_stdout=False, wait_for_readiness=True) as postgrest:
with run(env=env, no_startup_stdout=False, wait_for=Admin.ready) as postgrest:
output = postgrest.read_stdout(nlines=10)
# Check for the listener start message containing host and port
# Do not check if pg version is displayed properly as it is tricky to test it
@@ -1718,6 +1756,15 @@ def test_log_listener_connection_start(defaultenv):
)
def test_connection_error_message_does_not_claim_retry(defaultenv):
"The connection error message should not claim retrying, since PostgREST stops on fatal errors."
uri = f'postgresql://?dbname={defaultenv["PGDATABASE"]}&host={defaultenv["PGHOST"]}&user=some_protected_user&password=invalid_pass'
env = {**defaultenv, "PGRST_DB_URI": uri}
with run(env=env, no_startup_stdout=False, wait_for=None) as postgrest:
output = postgrest.read_stdout(nlines=8)
assert any('"message":"Database connection error."' in line for line in output)
def test_db_pre_config_with_pg_reserved_words(defaultenv):
"The db-pre-config should not fail unexpectedly when function name is a postgres reserved word"
@@ -1735,7 +1782,7 @@ def test_db_pre_config_with_pg_reserved_words(defaultenv):
"PGRST_DB_PRE_CONFIG": "select", # no "select" function in our fixtures, fail gracefully at startup
}
with run(env=env, no_startup_stdout=False, wait_for_readiness=False) as postgrest:
with run(env=env, no_startup_stdout=False, wait_for=None) as postgrest:
output = postgrest.read_stdout(nlines=8)
assert any(
'Failed to query database settings for the config parameters.{"code":"42883","details":null,"hint":"No function matches the given name and argument types. You might need to add explicit type casts.","message":"function select() does not exist"}'
@@ -1744,54 +1791,6 @@ def test_db_pre_config_with_pg_reserved_words(defaultenv):
)
def test_requests_with_resource_embedding_wait_for_schema_cache_reload(defaultenv):
"requests that use the schema cache with resource embedding wait long for the schema cache to reload"
env = {
**defaultenv,
"PGRST_DB_POOL": "2",
"PGRST_INTERNAL_SCHEMA_CACHE_RELATIONSHIP_LOAD_SLEEP": "5100",
}
with run(env=env, wait_max_seconds=30) as postgrest:
# reload the schema cache
response = postgrest.session.get("/rpc/notify_pgrst")
assert response.status_code == 204
postgrest.wait_until_scache_starts_loading()
response = postgrest.session.get("/directors?select=id,name,films(title)")
assert response.status_code == 200
assert response.elapsed.total_seconds() > 5
def test_requests_without_resource_embedding_wait_for_schema_cache_reload(defaultenv):
"requests that use the schema cache without resource embedding wait less for the schema cache to reload"
env = {
**defaultenv,
"PGRST_DB_POOL": "2",
"PGRST_INTERNAL_SCHEMA_CACHE_LOAD_SLEEP": "1100",
"PGRST_INTERNAL_SCHEMA_CACHE_RELATIONSHIP_LOAD_SLEEP": "5000",
}
with run(env=env, wait_max_seconds=30) as postgrest:
# reload the schema cache
response = postgrest.session.get("/rpc/notify_pgrst")
assert response.status_code == 204
postgrest.wait_until_scache_starts_loading()
response = postgrest.session.get("/films")
assert response.status_code == 200
assert (
response.elapsed.total_seconds() > 1
and response.elapsed.total_seconds() < 5
)
def test_server_timing_transaction_duration(defaultenv, metapostgrest):
"server-timing transaction duration should be accurate"
@@ -1816,3 +1815,37 @@ def test_server_timing_transaction_duration(defaultenv, metapostgrest):
]
assert 2000 <= response_dur < 3000
def test_positive_pool_metric(defaultenv):
"When a network failure is caused on the pg connection, pgrst_db_pool_available stays positive"
with run_pgproxy(defaultenv, proxy_timeout="1ms") as pgproxyhost:
env = {**defaultenv, "PGHOST": pgproxyhost}
with run(env=env, wait_for=Admin.live) as postgrest:
response = postgrest.admin.get("/metrics", timeout=1)
assert response.status_code == 200
metrics = float(
re.search(
r"pgrst_db_pool_available (-?\d+(?:\.\d+)?)", response.text
).group(1)
)
assert metrics >= 0
def test_work_mem_in_role_settings(defaultenv):
"Should work when setting work_mem on a role. See https://github.com/PostgREST/postgrest/issues/4955"
env = {
**defaultenv,
"PGRST_JWT_SECRET": SECRET,
}
headers = jwtauthheader({"role": "postgrest_test_work_mem"}, SECRET)
with run(env=env) as postgrest:
response = postgrest.session.post("/rpc/get_work_mem", headers=headers)
assert response.status_code == 200
assert response.text == '"3MB"'
+14
View File
@@ -1,3 +1,4 @@
import re
import threading
import jwt
@@ -21,6 +22,19 @@ class Thread(threading.Thread):
raise self._exception
def match_log(output, matchers):
ito = iter(output)
itm = iter(matchers)
nextMatcher = next(itm, None)
while nextMatcher is not None and (line := next(ito, None)) is not None:
if re.match(nextMatcher, line) is not None:
nextMatcher = next(itm, None)
if nextMatcher is not None:
raise AssertionError(
f"Expected log line matching {nextMatcher} not found in output"
)
def authheader(token):
"Bearer token HTTP authorization header."
return {"Authorization": f"Bearer {token}"}
+3 -3
View File
@@ -110,9 +110,9 @@ jsonKeyTest "10M" "POST" "/rpc/leak?columns=blob" "32M"
jsonKeyTest "10M" "POST" "/leak?columns=blob" "32M"
jsonKeyTest "10M" "PATCH" "/leak?id=eq.1&columns=blob" "50M"
jsonKeyTest "50M" "POST" "/rpc/leak?columns=blob" "73M"
jsonKeyTest "50M" "POST" "/leak?columns=blob" "73M"
jsonKeyTest "50M" "PATCH" "/leak?id=eq.1&columns=blob" "73M"
jsonKeyTest "50M" "POST" "/rpc/leak?columns=blob" "77M"
jsonKeyTest "50M" "POST" "/leak?columns=blob" "77M"
jsonKeyTest "50M" "PATCH" "/leak?id=eq.1&columns=blob" "77M"
postJsonArrayTest "1000" "/perf_articles?columns=id,body" "21M"
postJsonArrayTest "10000" "/perf_articles?columns=id,body" "22M"
-2
View File
@@ -114,8 +114,6 @@ baseCfg = let secret = encodeUtf8 "reallyreallyreallyreallyverysafe" in
, configRoleSettings = mempty
, configRoleIsoLvl = mempty
, configInternalSCQuerySleep = Nothing
, configInternalSCLoadSleep = Nothing
, configInternalSCRelLoadSleep = Nothing
, configServerTimingEnabled = True
}
+44 -9
View File
@@ -6,17 +6,21 @@
module Observation.MetricsSpec where
import Data.List (lookup)
import Network.Wai (Application)
import Data.List (lookup)
import qualified Hasql.Pool.Observation as SQL
import Network.Wai (Application)
import ObsHelper
import qualified PostgREST.AppState as AppState
import PostgREST.Config (AppConfig (configDbSchemas))
import qualified PostgREST.Metrics as Metrics
import qualified PostgREST.AppState as AppState
import PostgREST.Config (AppConfig (configDbSchemas))
import PostgREST.Metrics (ConnStats (..),
MetricsState (..),
connectionCounts)
import PostgREST.Observation
import Prometheus (getCounter, getVectorWith)
import Protolude
import Test.Hspec (SpecWith, describe, it)
import Test.Hspec.Wai (getState)
import Prometheus (getCounter, getVectorWith)
import Test.Hspec (SpecWith, describe, it)
import Test.Hspec.Wai (getState)
import Protolude
spec :: SpecWith (SpecState, Application)
spec = describe "Server started with metrics enabled" $ do
@@ -71,9 +75,40 @@ spec = describe "Server started with metrics enabled" $ do
-- (there should be none but we need to verify that)
threadDelay $ 1 * sec
-- The test verifies we properly count in use connections
-- The idea is to fork a worker thread that
-- borrows connection from the pool and waits for a signal to release it
-- Main thread checks that
-- in use connections counter is incremented by worker
-- then it signals the worker to release the connection
-- and finally verifies that in use connection counter is back to original value
it "Should track in use connections" $ do
SpecState{specAppState = appState, specMetrics = metrics, specObsChan} <- getState
let waitFor = waitForObs specObsChan
liftIO $ checkState' metrics [
-- we expect in use connections to be the same once finished
inUseConnections (+ 0)
] $ do
signal <- newEmptyMVar
-- make sure waiting thread is signaled
(`finally` tryPutMVar signal ()) $
-- expecting one more connection in use
checkState' metrics [
inUseConnections (+ 1)
] $ do
-- start a thread hanging on a single connection until signaled
void $ forkIO $ void $ AppState.usePool appState $ liftIO (readMVar signal)
-- main thread waits for ConnectionObservation with InUseConnectionStatus
-- after which used connections count should be incremented
waitFor (1 * sec) "InUseConnectionStatus" $ \x -> [ o | o@(HasqlPoolObs (SQL.ConnectionObservation _ SQL.InUseConnectionStatus)) <- pure x]
-- hanging thread was signaled and should return the connection
waitFor (1 * sec) "ReadyForUseConnectionStatus" $ \x -> [ o | o@(HasqlPoolObs (SQL.ConnectionObservation _ SQL.ReadyForUseConnectionStatus)) <- pure x]
where
-- prometheus-client api to handle vectors is convoluted
schemaCacheLoads label = expectField @"schemaCacheLoads" $
fmap (maybe (0::Int) round . lookup label) . (`getVectorWith` getCounter)
inUseConnections = expectField @"connTrack" ((inUse <$>) . connectionCounts)
sec = 1000000
-2
View File
@@ -157,8 +157,6 @@ baseCfg = let secret = encodeUtf8 "reallyreallyreallyreallyverysafe" in
, configRoleSettings = mempty
, configRoleIsoLvl = mempty
, configInternalSCQuerySleep = Nothing
, configInternalSCLoadSleep = Nothing
, configInternalSCRelLoadSleep = Nothing
, configServerTimingEnabled = True
}