Compare commits

..
14 Commits
Author SHA1 Message Date
Taimoor Zaeem 02d83d1c01 test(io): move remaining tests in test_io.py to their modules
We had just 3 tests remaining in test_io.py. This commit moves them to
their modules. So we have:

* test_graceful_shutdown.py

* test_zero_downtime.py

* test_pg_internal.py

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-08-13 03:42:29 +05:00
Taimoor Zaeem e89e0bc255 test(io): move config related behavior tests to test_config.py
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-08-13 03:42:29 +05:00
Taimoor Zaeem 9e20e5df90 test(io): move logs and observations tests to test_log.py
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-08-13 03:42:29 +05:00
Taimoor Zaeem 2122dcef97 test(io): move connection related tests to test_connection.py
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-08-13 03:42:29 +05:00
Taimoor Zaeem d84d00be8e test(io): move reloading related tests to test_reloading.py
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-08-13 03:42:29 +05:00
Taimoor ZaeemandSteve Chavez 3a034fcc2a chore: add changelog entry for v16.1
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-08-12 10:14:34 -05:00
Michał KłeczekandSteve Chavez a2a592a617 fix: JWT validation uses wrong current time due to a bug in auto-update
Upgrade auto-update to 0.2.7 which contains a fix to a bug causing some threads not seeing updates to the cached values.
2026-08-10 17:20:56 -05:00
Taimoor ZaeemandWolfgang Walther 89fe63fce0 nix(release): automate updating source file links in the docs
Closes #5133.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-08-10 19:14:04 +00:00
steve-chavez f21949a517 chore(changelog): missed entry for ghcr 2026-08-07 19:15:50 -05:00
steve-chavez 6ead34e70b chore(changelog): add integrations section
Also clarify some parts.
2026-08-07 16:17:56 -05:00
steve-chavezandWolfgang Walther e3206262d2 chore(changelog): make it in postgres style
This is to make it easier for users to understand the changes,
much like PostgreSQL does for its changelog: https://www.postgresql.org/docs/19/release-19.html#RELEASE-19-SERVER
(see how they classify "Optimizer", "General Performance", etc).

We classify features, fixes and deprecation in subsections.
Breaking changes are put inside a "Migration to v16" guide.
2026-08-07 19:59:15 +00:00
Taimoor Zaeem a3f36d6184 test: fix failing test of legacy target names warning
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
2026-08-07 22:47:09 +05:00
renovate[bot]andWolfgang Walther 6cc1698c05 chore(deps): update vmactions/freebsd-vm action to v1.5.3 2026-08-07 07:43:06 +00:00
Wolfgang Walther 77d6055683 chore: bump version to 17 2026-08-07 09:33:46 +02:00
17 changed files with 1870 additions and 1787 deletions
+20 -5
View File
@@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file. From versio
## Unreleased ## Unreleased
## [16.1] - 2026-08-10
### Fixed
- JWT validation uses wrong current time due to a bug in auto-update by @mkleczek in #5159
## [16.0] - 2026-08-07 ## [16.0] - 2026-08-07
### Changes ### Changes
@@ -23,6 +29,11 @@ All notable changes to this project will be documented in this file. From versio
- [Prefer: timezone](https://docs.postgrest.org/en/v16/references/api/preferences.html#prefer-timezone) no longer requires the schema cache by @steve-chavez in #5100 - [Prefer: timezone](https://docs.postgrest.org/en/v16/references/api/preferences.html#prefer-timezone) no longer requires the schema cache by @steve-chavez in #5100
+ Previously this required caching [pg_timezone_names](https://www.postgresql.org/docs/current/view-pg-timezone-names.html) which was slow in some systems + Previously this required caching [pg_timezone_names](https://www.postgresql.org/docs/current/view-pg-timezone-names.html) which was slow in some systems
#### Integrations
- PostgREST is now tested to work with [OrioleDB](https://github.com/orioledb/orioledb/) in #4845 by @wolfgangwalther
+ See [our guide for running OrioleDB on NixOS](https://docs.postgrest.org/en/v16/integrations/nixos.html)
#### JWT #### JWT
- [JWT Role Extraction](https://docs.postgrest.org/en/v16/references/auth.html#jwt-role-extract) is now more flexible, supporting the standard JSON Path defined in RFC 9535 by @taimoorzaeem in #4984 - [JWT Role Extraction](https://docs.postgrest.org/en/v16/references/auth.html#jwt-role-extract) is now more flexible, supporting the standard JSON Path defined in RFC 9535 by @taimoorzaeem in #4984
@@ -60,10 +71,11 @@ All notable changes to this project will be documented in this file. From versio
- Make executable for aarch64-linux static instead of Ubuntu-based by @wolfgangwalther in #4193 - Make executable for aarch64-linux static instead of Ubuntu-based by @wolfgangwalther in #4193
- Docker image for aarch64-linux is now built from scratch instead of being Ubuntu-based by @wolfgangwalther in #4193 - Docker image for aarch64-linux is now built from scratch instead of being Ubuntu-based by @wolfgangwalther in #4193
- Besides Docker Hub, docker images are now published to Github Container Registry by @wolfgangwalther in #2836
#### Schema Cache #### Schema Cache
- Fix requests failing when the schema cache fails to reload, when this happens PostgREST will continue serving requests as "best effort" by @mkleczek in #4873 #4869 - Fix requests failing when the schema cache fails to reload, when this happens PostgREST will continue serving requests in "best effort" by @mkleczek in #4873 #4869
- Fix reporting 503s errors unnecessarily while the schema cache is loading at startup by @mkleczek in #4880 - Fix reporting 503s errors unnecessarily while the schema cache is loading at startup by @mkleczek in #4880
- Fix schema cache dump missing RPC transaction isolation level by @taimoorzaeem in #5079 - Fix schema cache dump missing RPC transaction isolation level by @taimoorzaeem in #5079
@@ -74,7 +86,7 @@ All notable changes to this project will be documented in this file. From versio
### Migration to v16 ### Migration to v16
- Drop support for PostgreSQL EOL version 13 by @wolfgangwalther in #4193 - Drop support for PostgreSQL EOL version 13 by @wolfgangwalther in #4193
+ PostgreSQL 13 end of life was on 2025 [ref](https://www.postgresql.org/support/versioning/) + PostgreSQL 13 end of life was on 2025 ([ref](https://www.postgresql.org/support/versioning/))
+ Upgrade your PostgreSQL version to at least 14 to use this new PostgREST version. + Upgrade your PostgreSQL version to at least 14 to use this new PostgREST version.
- Fail at startup when `db-schemas` contains schema `pg_catalog` or `information_schema` by @taimoorzaeem in #4359 - Fail at startup when `db-schemas` contains schema `pg_catalog` or `information_schema` by @taimoorzaeem in #4359
@@ -86,9 +98,12 @@ All notable changes to this project will be documented in this file. From versio
+ Ensure your requests always have a valid timezone. + Ensure your requests always have a valid timezone.
- `jwt-role-claim-key` no longer uses the JSPath DSL and instead uses JSON Path by @taimoorzaeem in #4984 - `jwt-role-claim-key` no longer uses the JSPath DSL and instead uses JSON Path by @taimoorzaeem in #4984
+ Now all config values must start with `$` character. Example: `.roles.read` -> `$.roles.read` + Now all config values must start with `$` character.
+ Keys with special characters, with the exception of `_` char must be quoted. Example: `.roles.write-role` -> `$.roles["write-role"]` Example: `.roles.read` -> `$.roles.read`
+ String comparison operators (`^==`, `==^` and `*==`) are replaced with regular expression search. Example: `.roles[?(@ ^== "postgrest_test_")]` -> `$.roles[?search(@, "^postgrest_test_")]` + Keys with special characters, with the exception of `_` char must be quoted.
Example: `.roles.write-role` -> `$.roles["write-role"]`
+ String comparison operators (`^==`, `==^` and `*==`) are replaced with regular expression search.
Example: `.roles[?(@ ^== "postgrest_test_")]` -> `$.roles[?search(@, "^postgrest_test_")]`
+ Update the `jwt-role-claim-key` value accoring to the above rules. Also see the syntax reference: [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535.html#name-jsonpath-syntax-and-semanti). + Update the `jwt-role-claim-key` value accoring to the above rules. Also see the syntax reference: [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535.html#name-jsonpath-syntax-and-semanti).
## [14.16] - 2026-07-27 ## [14.16] - 2026-07-27
+1 -1
View File
@@ -1 +1 @@
index-state: hackage.haskell.org 2026-07-11T17:34:10Z index-state: hackage.haskell.org 2026-08-10T16:58:32Z
+1 -1
View File
@@ -49,7 +49,7 @@ source_suffix = ".rst"
master_doc = "index" master_doc = "index"
# This is overridden by readthedocs with the version tag anyway # This is overridden by readthedocs with the version tag anyway
version = "16" version = "devel"
# To avoid repetition in <title> we set this to an empty string. # To avoid repetition in <title> we set this to an empty string.
release = "" release = ""
+10
View File
@@ -49,6 +49,16 @@ let
# Before upgrading fuzzyset to 0.3, check: https://github.com/PostgREST/postgrest/issues/3329 # Before upgrading fuzzyset to 0.3, check: https://github.com/PostgREST/postgrest/issues/3329
fuzzyset = prev.fuzzyset_0_2_4; fuzzyset = prev.fuzzyset_0_2_4;
# TODO: Remove once available in nixpkgs
auto-update =
prev.callHackageDirect
{
pkg = "auto-update";
ver = "0.2.7";
sha256 = "sha256-fHX/OqF/cB9rbpGpLUtA29bcEJS43HUWHcK55yUxKoo=";
}
{ };
# TODO: Remove once available in nixpkgs # TODO: Remove once available in nixpkgs
aeson-jsonpath = aeson-jsonpath =
prev.callHackageDirect prev.callHackageDirect
+4 -1
View File
@@ -45,7 +45,10 @@ let
echo "Updating docs/conf.py ..." echo "Updating docs/conf.py ..."
sed -i -E "s/^(version = ).*$/\1\"$new_docs_version\"/" docs/conf.py > /dev/null sed -i -E "s/^(version = ).*$/\1\"$new_docs_version\"/" docs/conf.py > /dev/null
git add postgrest.cabal docs/conf.py > /dev/null echo "Updating Haskell source file links ..."
sed -i -E "s#(github\.com/PostgREST/postgrest/blob)/main/#\1/$new_version/#g" docs/explanations/architecture.rst
git add postgrest.cabal docs/conf.py docs/explanations/architecture.rst > /dev/null
} }
today_date_for_changelog="$(date '+%Y-%m-%d')" today_date_for_changelog="$(date '+%Y-%m-%d')"
+3 -3
View File
@@ -1,6 +1,6 @@
cabal-version: 3.0 cabal-version: 3.0
name: postgrest name: postgrest
version: 16.0 version: 17
synopsis: REST API for any Postgres database synopsis: REST API for any Postgres database
description: Reads the schema of a PostgreSQL database and creates RESTful routes description: Reads the schema of a PostgreSQL database and creates RESTful routes
for tables, views, and functions, supporting all HTTP methods that security for tables, views, and functions, supporting all HTTP methods that security
@@ -106,7 +106,7 @@ library
, Ranged-sets >= 0.3 && < 0.6 , Ranged-sets >= 0.3 && < 0.6
, aeson >= 2.0.3 && < 2.3 , aeson >= 2.0.3 && < 2.3
, aeson-jsonpath >= 0.4.2 && < 0.5 , aeson-jsonpath >= 0.4.2 && < 0.5
, auto-update >= 0.1.4 && < 0.3 , auto-update >= 0.2.7 && < 0.3
, base64-bytestring >= 1 && < 1.3 , base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.13 , bytestring >= 0.10.8 && < 0.13
, case-insensitive >= 1.2 && < 1.3 , case-insensitive >= 1.2 && < 1.3
@@ -158,7 +158,7 @@ library
, wai-extra >= 3.1.8 && < 3.2 , wai-extra >= 3.1.8 && < 3.2
-- We already depend on wai-logger >= 2.3.7 indirectly via wai-extra, -- We already depend on wai-logger >= 2.3.7 indirectly via wai-extra,
-- but we want to depend on 2.4.0 which fixes 'unknownSocket' log output -- but we want to depend on 2.4.0 which fixes 'unknownSocket' log output
-- for unix sockets; this is tested in test/io/test_io.py. See -- for unix sockets; this is tested in test/io/test_log.py. See
-- https://github.com/kazu-yamamoto/logger/commit/3a71ca70afdbb93d4ecf0083eeba1fbbbcab3fc3 -- https://github.com/kazu-yamamoto/logger/commit/3a71ca70afdbb93d4ecf0083eeba1fbbbcab3fc3
, wai-logger >= 2.4.0 , wai-logger >= 2.4.0
, warp >= 3.4.14 && < 3.5 , warp >= 3.4.14 && < 3.5
+1
View File
@@ -10,6 +10,7 @@ nix:
extra-deps: extra-deps:
- aeson-jsonpath-0.4.2.0 - aeson-jsonpath-0.4.2.0
- auto-update-0.2.7
- configurator-pg-0.2.11 - configurator-pg-0.2.11
- fuzzyset-0.2.4 - fuzzyset-0.2.4
- hasql-notifications-0.2.4.0 - hasql-notifications-0.2.4.0
+7
View File
@@ -11,6 +11,13 @@ packages:
size: 2160 size: 2160
original: original:
hackage: aeson-jsonpath-0.4.2.0 hackage: aeson-jsonpath-0.4.2.0
- completed:
hackage: auto-update-0.2.7@sha256:32ca6ce351604a17ee79e4086939f798f958f8407478288e9adb7adeb194c6ea,1670
pantry-tree:
sha256: 641faa7d5ee516195ecd4b538f170722d56f84e3bc5714c6bbe8123849b49983
size: 1105
original:
hackage: auto-update-0.2.7
- completed: - completed:
hackage: configurator-pg-0.2.11@sha256:de0c56386591e85159436b0af04a8f15a4f4e156354e99709676c2c2ee959505,2850 hackage: configurator-pg-0.2.11@sha256:de0c56386591e85159436b0af04a8f15a4f4e156354e99709676c2c2ee959505,2850
pantry-tree: pantry-tree:
-221
View File
@@ -1,221 +0,0 @@
"Auth related IO tests for PostgREST"
from operator import attrgetter
import signal
import pytest
from config import BASEDIR, CONFIGSDIR, FIXTURES, SECRET
from util import authheader, jwtauthheader
from postgrest import (
run,
sleep_until_postgrest_config_reload,
sleep_until_postgrest_scache_reload,
wait_until_exit,
)
@pytest.mark.parametrize(
"secretpath",
[path for path in (BASEDIR / "secrets").iterdir() if path.suffix != ".jwt"],
ids=attrgetter("name"),
)
def test_read_secret_from_file(secretpath, defaultenv):
"Authorization should succeed when the secret is read from a file."
env = {**defaultenv, "PGRST_JWT_SECRET": f"@{secretpath}"}
if secretpath.suffix == ".b64":
env["PGRST_JWT_SECRET_IS_BASE64"] = "true"
secret = secretpath.read_bytes()
headers = authheader(secretpath.with_suffix(".jwt").read_text())
with run(stdin=secret, env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
print(response.text)
assert response.status_code == 200
def test_read_secret_from_stdin(defaultenv):
"Authorization should succeed when the secret is read from stdin."
env = {**defaultenv, "PGRST_DB_CONFIG": "false", "PGRST_JWT_SECRET": "@/dev/stdin"}
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
with run(stdin=SECRET.encode(), env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
print(response.text)
assert response.status_code == 200
# TODO: This test would fail right now, because of
# https://github.com/PostgREST/postgrest/issues/2126
@pytest.mark.skip
def test_read_secret_from_stdin_dbconfig(defaultenv):
"Authorization should succeed when the secret is read from stdin with db-config=true."
env = {**defaultenv, "PGRST_DB_CONFIG": "true", "PGRST_JWT_SECRET": "@/dev/stdin"}
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
with run(stdin=SECRET.encode(), env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
print(response.text)
assert response.status_code == 200
def test_fail_with_invalid_password(defaultenv):
"Connecting with an invalid password should fail without retries."
uri = f'postgresql://?dbname={defaultenv["PGDATABASE"]}&host={defaultenv["PGHOST"]}&user=some_protected_user&password=invalid_pass'
env = {**defaultenv, "PGRST_DB_URI": uri}
with run(env=env, wait_for=None) as postgrest:
exitCode = wait_until_exit(postgrest)
assert exitCode == 1
@pytest.mark.parametrize(
"roleclaim", FIXTURES["roleclaims"], ids=lambda claim: claim["key"]
)
def test_role_claim_key(roleclaim, defaultenv):
"Authorization should depend on a correct role-claim-key and JWT claim."
env = {
**defaultenv,
"PGRST_JWT_ROLE_CLAIM_KEY": roleclaim["key"],
"PGRST_JWT_SECRET": SECRET,
}
headers = jwtauthheader(roleclaim["data"], SECRET)
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == roleclaim["expected_status"]
@pytest.mark.parametrize(
"jwtaudroleclaim",
FIXTURES["jwtaudroleclaims"],
ids=lambda claim: claim["key"] + "_" + str(claim["expected_status"]),
)
def test_jwt_aud_in_role_claim_key(jwtaudroleclaim, defaultenv):
"Allows authorization with JWT aud claim in role-claim-key"
env = {
**defaultenv,
"PGRST_JWT_AUD": "postgrest_test_author",
"PGRST_JWT_ROLE_CLAIM_KEY": jwtaudroleclaim["key"],
"PGRST_JWT_SECRET": SECRET,
}
headers = jwtauthheader(jwtaudroleclaim["data"], SECRET)
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == jwtaudroleclaim["expected_status"]
def test_jwt_secret_reload(tmp_path, defaultenv):
"JWT secret should be reloaded from file when PostgREST is sent SIGUSR2."
config = (CONFIGSDIR / "sigusr2-settings.config").read_text()
configfile = tmp_path / "test.config"
configfile.write_text(config)
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
with run(configfile, env=defaultenv) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
# change setting
configfile.write_text(config.replace("invalid" * 5, SECRET))
# reload config
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 200
def test_jwt_secret_external_file_reload(tmp_path, defaultenv):
"JWT secret external file should be reloaded when PostgREST is sent a SIGUSR2 or a NOTIFY."
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
external_secret_file = tmp_path / "jwt-secret-config"
external_secret_file.write_text("invalid" * 5)
env = {
**defaultenv,
"PGRST_JWT_SECRET": f"@{external_secret_file}",
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGRST_DB_CONFIG": "false",
"PGRST_DB_ANON_ROLE": "postgrest_test_anonymous", # required for NOTIFY
}
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
# change external file
external_secret_file.write_text(SECRET)
# SIGUSR1 doesn't reload external files, at least when db-config=false
postgrest.process.send_signal(signal.SIGUSR1)
sleep_until_postgrest_scache_reload()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
# reload config and external file with SIGUSR2
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 200
# change external file to wrong value again
external_secret_file.write_text("invalid" * 5)
# reload config and external file with NOTIFY
response = postgrest.session.post("/rpc/reload_pgrst_config")
assert response.text == ""
assert response.status_code == 204
sleep_until_postgrest_config_reload()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
def test_invalidate_jwt_cache_when_secret_changes(tmp_path, defaultenv):
"JWT cache should be emptied after jwt-secret is changed in a config reload"
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
external_secret_file = tmp_path / "jwt-secret-config"
external_secret_file.write_text(SECRET)
env = {
**defaultenv,
"PGRST_JWT_SECRET": f"@{external_secret_file}",
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGRST_JWT_CACHE_MAX_ENTRIES": "86400", # enable cache
"PGRST_DB_ANON_ROLE": "postgrest_test_anonymous", # required for NOTIFY
}
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 200 # jwt gets cached
# change external file
external_secret_file.write_text("invalid" * 5)
# reload config and external file with NOTIFY
# jwt-cache should get empty
response = postgrest.session.post("/rpc/reload_pgrst_config")
assert response.text == ""
assert response.status_code == 204
sleep_until_postgrest_config_reload()
# now the request should fail because the cached token is removed
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
+251
View File
@@ -0,0 +1,251 @@
"Test PostgREST configuration related behavior"
import time
import pytest
from operator import attrgetter
from config import BASEDIR, FIXTURES, SECRET
from util import (
Thread,
authheader,
jwtauthheader,
)
from postgrest import (
PostgrestTimedOut,
freeport,
run,
)
def test_pool_size(defaultenv, metapostgrest):
"Verify that PGRST_DB_POOL setting allows the correct number of parallel requests"
env = {
**defaultenv,
"PGRST_DB_POOL": "2",
}
with run(env=env) as postgrest:
start = time.time()
threads = []
for i in range(4):
def sleep(i=i):
response = postgrest.session.get("/rpc/sleep?seconds=0.5")
assert response.text == ""
assert response.status_code == 204, "thread {}".format(i)
t = Thread(target=sleep)
t.start()
threads.append(t)
for t in threads:
t.join()
end = time.time()
delta = end - start
# sleep 4 times for 0.5s each, with 2 requests in parallel
# => total time roughly 1s
assert delta > 1 and delta < 1.5
@pytest.mark.parametrize(
"secretpath",
[path for path in (BASEDIR / "secrets").iterdir() if path.suffix != ".jwt"],
ids=attrgetter("name"),
)
def test_read_secret_from_file(secretpath, defaultenv):
"Authorization should succeed when the secret is read from a file."
env = {**defaultenv, "PGRST_JWT_SECRET": f"@{secretpath}"}
if secretpath.suffix == ".b64":
env["PGRST_JWT_SECRET_IS_BASE64"] = "true"
secret = secretpath.read_bytes()
headers = authheader(secretpath.with_suffix(".jwt").read_text())
with run(stdin=secret, env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
print(response.text)
assert response.status_code == 200
def test_read_secret_from_stdin(defaultenv):
"Authorization should succeed when the secret is read from stdin."
env = {**defaultenv, "PGRST_DB_CONFIG": "false", "PGRST_JWT_SECRET": "@/dev/stdin"}
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
with run(stdin=SECRET.encode(), env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
print(response.text)
assert response.status_code == 200
# TODO: This test would fail right now, because of
# https://github.com/PostgREST/postgrest/issues/2126
@pytest.mark.skip
def test_read_secret_from_stdin_dbconfig(defaultenv):
"Authorization should succeed when the secret is read from stdin with db-config=true."
env = {**defaultenv, "PGRST_DB_CONFIG": "true", "PGRST_JWT_SECRET": "@/dev/stdin"}
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
with run(stdin=SECRET.encode(), env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
print(response.text)
assert response.status_code == 200
@pytest.mark.parametrize(
"roleclaim", FIXTURES["roleclaims"], ids=lambda claim: claim["key"]
)
def test_role_claim_key(roleclaim, defaultenv):
"Authorization should depend on a correct role-claim-key and JWT claim."
env = {
**defaultenv,
"PGRST_JWT_ROLE_CLAIM_KEY": roleclaim["key"],
"PGRST_JWT_SECRET": SECRET,
}
headers = jwtauthheader(roleclaim["data"], SECRET)
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == roleclaim["expected_status"]
@pytest.mark.parametrize(
"jwtaudroleclaim",
FIXTURES["jwtaudroleclaims"],
ids=lambda claim: claim["key"] + "_" + str(claim["expected_status"]),
)
def test_jwt_aud_in_role_claim_key(jwtaudroleclaim, defaultenv):
"Allows authorization with JWT aud claim in role-claim-key"
env = {
**defaultenv,
"PGRST_JWT_AUD": "postgrest_test_author",
"PGRST_JWT_ROLE_CLAIM_KEY": jwtaudroleclaim["key"],
"PGRST_JWT_SECRET": SECRET,
}
headers = jwtauthheader(jwtaudroleclaim["data"], SECRET)
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == jwtaudroleclaim["expected_status"]
def test_random_port_bound(defaultenv):
"PostgREST should bind to a random port when PGRST_SERVER_PORT is 0."
with run(env=defaultenv, port="0"):
assert True # liveness check is done by run(), so we just need to check that it doesn't fail
def test_so_reuseport_defaults_to_false(defaultenv):
"A second PostgREST instance should not bind to the same port by default."
host = "0.0.0.0"
port = freeport()
admin_port = freeport(used_ports=[port])
with run(
env={**defaultenv},
port=port,
host=host,
admin_port=admin_port,
):
with pytest.raises(PostgrestTimedOut):
with run(
env={**defaultenv},
port=port,
host=host,
admin_port=freeport(used_ports=[port, admin_port]),
wait_max_seconds=1,
):
pass
def test_schema_cache_startup_load_with_in_db_config(defaultenv, metapostgrest):
"verify that the Schema Cache loads correctly at startup, using the in-db `pgrst.db_schemas` config"
response = metapostgrest.session.post("/rpc/change_db_schemas_config")
assert response.text == ""
assert response.status_code == 204
with run(env=defaultenv) as postgrest:
response = postgrest.session.get("/rpc/get_current_schema")
assert response.text == '"test"'
assert response.status_code == 200
response = metapostgrest.session.post("/rpc/reset_db_schemas_config")
assert response.text == ""
assert response.status_code == 204
def test_allow_configs_to_be_set_to_empty(defaultenv):
'configs that are explicitly set to empty (= "<empty>") should not throw parse error'
env = {
**defaultenv,
"PGRST_DB_EXTRA_SEARCH_PATH": "",
}
with run(env=env) as postgrest:
response = postgrest.session.get("/projects")
assert response.status_code == 200
def test_connection_error_message_does_not_claim_retry(defaultenv):
"The connection error message should not claim retrying, since PostgREST stops on fatal errors."
uri = f'postgresql://?dbname={defaultenv["PGDATABASE"]}&host={defaultenv["PGHOST"]}&user=some_protected_user&password=invalid_pass'
env = {**defaultenv, "PGRST_DB_URI": uri}
with run(env=env, no_startup_stdout=False, wait_for=None) as postgrest:
output = postgrest.read_stdout(nlines=8)
assert any('"message":"Database connection error."' in line for line in output)
def test_db_pre_config_with_non_existent_function(defaultenv):
"Log error when db-pre-config is set to non-existent function"
env = {
**defaultenv,
"PGRST_DB_PRE_CONFIG": "select", # no "select" function in our fixtures, fail gracefully at startup
}
with run(env=env, no_startup_stdout=False, wait_for=None) as postgrest:
output = postgrest.read_stdout(nlines=8)
assert any("function select() does not exist" in line for line in output)
@pytest.mark.parametrize("enabled", ["true", "false"])
def test_use_legacy_target_names(enabled, defaultenv):
"Show a warning when a target name is used instead of an alias, only when config is enabled"
env = {
**defaultenv,
"PGRST_URL_USE_LEGACY_TARGET_NAMES": enabled,
}
with run(env=env) as postgrest:
response = postgrest.session.get(
"/directors?select=name,all_films:films(title),awards_2026:awards(name)&films.order=title&awards.year=eq.2026"
)
output = postgrest.read_stdout(nlines=10)
log_err_warning = "WARNING: Embedded resource was referenced by relation name even though it has an alias. This is deprecated and will stop working in a future release."
log_err_hint = "Update filters, orders or limits that use `films` to `all_films`, `awards` to `awards_2026` in `GET /directors?select=name,all_films:films(title),awards_2026:awards(name)&films.order=title&awards.year=eq.2026`"
has_warning_log = any(log_err_warning in line for line in output)
has_hint_log = any(log_err_hint in line for line in output)
if enabled == "true":
assert response.status_code == 200
assert has_warning_log and has_hint_log
else:
assert response.status_code == 400
assert not has_warning_log and not has_hint_log
+317
View File
@@ -0,0 +1,317 @@
"Tests related to PostgREST connection and connections pools"
import os
import re
import signal
import time
import pytest
from config import SECRET
from util import (
Thread,
jwtauthheader,
drain_stdout,
)
from postgrest import (
Admin,
run,
run_pgproxy,
wait_until_exit,
)
def test_fail_with_invalid_password(defaultenv):
"Connecting with an invalid password should fail without retries."
uri = f'postgresql://?dbname={defaultenv["PGDATABASE"]}&host={defaultenv["PGHOST"]}&user=some_protected_user&password=invalid_pass'
env = {**defaultenv, "PGRST_DB_URI": uri}
with run(env=env, wait_for=None) as postgrest:
exitCode = wait_until_exit(postgrest)
assert exitCode == 1
def test_connect_with_dburi(dburi, defaultenv):
"Connecting with db-uri instead of LIPQ* environment variables should work."
defaultenv_without_libpq = {
key: value
for key, value in defaultenv.items()
if key not in ["PGDATABASE", "PGHOST", "PGUSER"]
}
env = {**defaultenv_without_libpq, "PGRST_DB_URI": dburi.decode()}
with run(env=env):
pass
@pytest.mark.parametrize("dburi_type", ["no_params", "no_params_qmark", "with_params"])
def test_get_pgrst_version_with_uri_connection_string(dburi_type, dburi, defaultenv):
"The fallback_application_name should be added to the db-uri if it has a URI format"
defaultenv_without_libpq = {
key: value
for key, value in defaultenv.items()
if key not in ["PGDATABASE", "PGHOST", "PGUSER"]
}
env = {
"no_params": {**defaultenv, "PGRST_DB_URI": "postgresql://"},
"no_params_qmark": {**defaultenv, "PGRST_DB_URI": "postgresql://?"},
"with_params": {**defaultenv_without_libpq, "PGRST_DB_URI": dburi.decode()},
}
with run(env=env[dburi_type]) as postgrest:
response = postgrest.session.post("/rpc/get_pgrst_version")
version = '"%s"' % response.headers["Server"].replace(
"postgrest/", "PostgREST "
)
assert response.text == version
def test_get_pgrst_version_with_keyval_connection_string(defaultenv):
"The fallback_application_name should be added to the db-uri if it has a keyword/value format"
uri = f'dbname={defaultenv["PGDATABASE"]} host={defaultenv["PGHOST"]} user={defaultenv["PGUSER"]}'
defaultenv_without_libpq = {
key: value
for key, value in defaultenv.items()
if key not in ["PGDATABASE", "PGHOST", "PGUSER"]
}
env = {**defaultenv_without_libpq, "PGRST_DB_URI": uri}
with run(env=env) as postgrest:
response = postgrest.session.post("/rpc/get_pgrst_version")
version = '"%s"' % response.headers["Server"].replace(
"postgrest/", "PostgREST "
)
assert response.text == version
def test_fail_with_invalid_dbname_and_automatic_recovery_disabled(defaultenv):
"Should fail without retries when automatic recovery is disabled and dbname is invalid"
dbname = "INVALID"
uri = f'postgresql://?dbname={dbname}&host={defaultenv["PGHOST"]}&user={defaultenv["PGUSER"]}'
env = {
**defaultenv,
"PGRST_DB_URI": uri,
"PGRST_DB_POOL_AUTOMATIC_RECOVERY": "false",
}
with run(env=env, wait_for=None) as postgrest:
exitCode = wait_until_exit(postgrest)
assert exitCode == 1
def test_fail_with_automatic_recovery_disabled_and_terminated_using_query(defaultenv):
"Should fail without retries when automatic recovery is disabled and pg_terminate_backend(pid) is called"
env = {
**defaultenv,
"PGRST_DB_POOL_AUTOMATIC_RECOVERY": "false",
"PGAPPNAME": "target",
}
app_name = "'{}'".format(env["PGAPPNAME"])
with run(env=env) as postgrest:
os.system(
f'psql -d {env["PGDATABASE"]} -U {env["PGUSER"]} -h {env["PGHOST"]} --set ON_ERROR_STOP=1 -a -c "SELECT terminate_pgrst({app_name})"'
)
exitCode = wait_until_exit(postgrest)
assert exitCode == 1
def test_read_dburi_from_stdin_without_eol(dburi, defaultenv):
"Reading the dburi from stdin with a single line should work."
defaultenv_without_libpq = {
key: value
for key, value in defaultenv.items()
if key not in ["PGDATABASE", "PGHOST", "PGUSER"]
}
env = {**defaultenv_without_libpq, "PGRST_DB_URI": "@/dev/stdin"}
with run(env=env, stdin=dburi):
pass
def test_read_dburi_from_stdin_with_eol(dburi, defaultenv):
"Reading the dburi from stdin containing a newline should work."
defaultenv_without_libpq = {
key: value
for key, value in defaultenv.items()
if key not in ["PGDATABASE", "PGHOST", "PGUSER"]
}
env = {**defaultenv_without_libpq, "PGRST_DB_URI": "@/dev/stdin"}
with run(env=env, stdin=dburi + b"\n"):
pass
def test_flush_pool_no_interrupt(defaultenv):
"Flushing the pool via SIGUSR1 doesn't interrupt ongoing requests"
with run(env=defaultenv) as postgrest:
def sleep():
response = postgrest.session.get("/rpc/sleep?seconds=0.5")
assert response.text == ""
assert response.status_code == 204
t = Thread(target=sleep)
t.start()
# make sure the request has started
time.sleep(0.1)
# SIGUSR1 causes the postgres connection pool to be flushed
postgrest.process.send_signal(signal.SIGUSR1)
t.join()
def test_no_pool_connection_required_on_bad_http_logic(defaultenv):
"no pool connection should be consumed for failing on invalid http logic"
with run(env=defaultenv, no_pool_connection_available=True) as postgrest:
# not found nested route shouldn't require opening a connection
response = postgrest.session.head("/path/notfound")
assert response.status_code == 404
# an invalid http method on a resource shouldn't require opening a connection
response = postgrest.session.request("TRACE", "/projects")
assert response.status_code == 405
response = postgrest.session.patch("/rpc/hello")
assert response.status_code == 405
def test_no_pool_connection_required_on_options(defaultenv):
"no pool connection should be consumed for OPTIONS requests"
with run(env=defaultenv, no_pool_connection_available=True) as postgrest:
# OPTIONS on a table shouldn't require opening a connection
response = postgrest.session.options("/projects")
assert response.status_code == 200
# OPTIONS on RPC shouldn't require opening a connection
response = postgrest.session.options("/rpc/hello")
assert response.status_code == 200
# OPTIONS on root shouldn't require opening a connection
response = postgrest.session.options("/")
assert response.status_code == 200
def test_no_pool_connection_required_on_bad_jwt_claim(defaultenv):
"no pool connection should be consumed for failing on invalid jwt"
env = {**defaultenv, "PGRST_JWT_SECRET": SECRET}
with run(env=env, no_pool_connection_available=True) as postgrest:
# A JWT with an invalid signature shouldn't open a connection
wrong_secret = "This is the most wrong secret of all secrets"
headers = jwtauthheader({"role": "postgrest_test_author"}, wrong_secret)
response = postgrest.session.get("/projects", headers=headers)
assert response.status_code == 401
def test_no_pool_connection_required_on_bad_embedding(defaultenv):
"no pool connection should be consumed for failing to embed"
with run(env=defaultenv, no_pool_connection_available=True) as postgrest:
# OPTIONS on a table shouldn't require opening a connection
response = postgrest.session.get("/projects?select=*,unexistent(*)")
assert response.status_code == 400
@pytest.mark.parametrize("level", ["crit", "error", "warn", "info", "debug"])
def test_pool_acquisition_timeout(level, defaultenv, metapostgrest):
"Verify that PGRST_DB_POOL_ACQUISITION_TIMEOUT times out when the pool is empty"
env = {
**defaultenv,
"PGRST_DB_POOL": "1",
"PGRST_DB_POOL_ACQUISITION_TIMEOUT": "1", # 1 second
"PGRST_LOG_LEVEL": level,
}
with run(
env=env, no_pool_connection_available=True, wait_max_seconds=3
) as postgrest:
response = postgrest.session.get("/projects")
assert response.status_code == 504
data = response.json()
assert data["message"] == "Timed out acquiring connection from connection pool."
# ensure the message appears on the logs as well
output = sorted(postgrest.read_stdout(nlines=10))
if level == "crit":
assert len(output) == 0
else:
assert any(" 504 " in line for line in output)
assert any(
"Timed out acquiring connection from connection pool." in line
for line in output
)
def test_pool_acquisition_timeout_logs_are_debounced(defaultenv):
"Pool acquisition timeout diagnostic logs should be debounced over a burst of failures"
env = {
**defaultenv,
"PGRST_DB_POOL": "1",
"PGRST_DB_POOL_ACQUISITION_TIMEOUT": "1",
"PGRST_LOG_LEVEL": "error",
}
total_requests = 6
with run(
env=env, no_pool_connection_available=True, wait_max_seconds=3
) as postgrest:
def request_timeout():
response = postgrest.session.get("/projects")
assert response.status_code == 504
assert (
response.json()["message"]
== "Timed out acquiring connection from connection pool."
)
return response
request_timeout()
threads = [Thread(target=request_timeout) for _ in range(total_requests - 1)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
# Logger debouncing logs the first timeout immediately and, if more
# timeouts happen during the cooldown, logs one more time afterwards.
time.sleep(6)
output = drain_stdout(postgrest)
access_logs = [line for line in output if ' "GET /projects HTTP/1.1" 504 ' in line]
timeout_logs = [
line
for line in output
if "Timed out acquiring connection from connection pool." in line
]
assert len(access_logs) == total_requests
assert len(timeout_logs) == 2
def test_positive_pool_metric(defaultenv):
"When a network failure is caused on the pg connection, pgrst_db_pool_available stays positive"
with run_pgproxy(defaultenv, proxy_timeout="1ms") as pgproxyhost:
env = {**defaultenv, "PGHOST": pgproxyhost}
with run(env=env, wait_for=Admin.live) as postgrest:
response = postgrest.admin.get("/metrics", timeout=1)
assert response.status_code == 200
metrics = float(
re.search(
r"pgrst_db_pool_available (-?\d+(?:\.\d+)?)", response.text
).group(1)
)
assert metrics >= 0
+25
View File
@@ -0,0 +1,25 @@
import time
from util import Thread
from postgrest import run
def test_graceful_shutdown_waits_for_in_flight_request(defaultenv):
"SIGTERM should allow in-flight requests to finish before exiting"
with run(env=defaultenv, wait_max_seconds=5) as postgrest:
def sleep():
response = postgrest.session.get("/rpc/sleep?seconds=3", timeout=10)
assert response.text == ""
assert response.status_code == 204
t = Thread(target=sleep)
t.start()
# Wait for the request to be in-flight before shutting down.
time.sleep(1)
postgrest.process.terminate()
t.join()
-1555
View File
File diff suppressed because it is too large Load Diff
+621
View File
@@ -0,0 +1,621 @@
"Test PostgREST logs and observations"
import re
import signal
import time
import pytest
import requests
from config import SECRET
from util import (
jwtauthheader,
relativeSeconds,
drain_stdout,
match_log,
)
from postgrest import (
Admin,
freeport,
is_ipv6,
reset_statement_timeout,
run,
set_statement_timeout,
wait_until_exit,
)
@pytest.mark.parametrize("level", ["crit", "error", "warn", "info", "debug"])
def test_log_level(level, defaultenv):
"log_level should filter request logging"
env = {**defaultenv, "PGRST_LOG_LEVEL": level}
# any token to test 500 response for "Server lacks JWT secret"
claim = {"role": "postgrest_test_author"}
headers = jwtauthheader(claim, SECRET)
with run(env=env) as postgrest:
response = postgrest.session.get("/", headers=headers)
assert response.status_code == 500
response = postgrest.session.get("/unknown")
assert response.status_code == 404
response = postgrest.session.get("/")
assert response.status_code == 200
output = postgrest.read_stdout(nlines=9)
if level == "crit":
assert len(output) == 0
elif level == "error":
match_log(
output,
[r'- - - \[.+\] "GET / HTTP/1.1" 500 \d+ "" "python-requests/.+"'],
)
assert len(output) == 1
elif level == "warn":
match_log(
output,
[
r'- - - \[.+\] "GET / HTTP/1.1" 500 \d+ "" "python-requests/.+"',
r'- - postgrest_test_anonymous \[.+\] "GET /unknown HTTP/1.1" 404 \d+ "" "python-requests/.+"',
],
)
assert len(output) == 2
elif level == "info":
match_log(
output,
[
r'- - - \[.+\] "GET / HTTP/1.1" 500 \d+ "" "python-requests/.+"',
r'- - postgrest_test_anonymous \[.+\] "GET /unknown HTTP/1.1" 404 \d+ "" "python-requests/.+"',
r'- - postgrest_test_anonymous \[.+\] "GET / HTTP/1.1" 200 \d+ "" "python-requests/.+"',
],
)
assert len(output) == 3
elif level == "debug":
match_log(
output,
[
r'- - - \[.+\] "GET / HTTP/1.1" 500 \d+ "" "python-requests/.+"',
r'- - postgrest_test_anonymous \[.+\] "GET /unknown HTTP/1.1" 404 \d+ "" "python-requests/.+"',
r'- - postgrest_test_anonymous \[.+\] "GET / HTTP/1.1" 200 \d+ "" "python-requests/.+"',
],
)
assert len(output) == 9
assert any("Connection" and "is available" in line for line in output)
assert any("Connection" and "is used" in line for line in output)
@pytest.mark.parametrize("level", ["crit", "error", "warn", "info", "debug"])
def test_log_query(level, defaultenv):
"log_query=true should log the SQL query according to the log_level"
env = {
**defaultenv,
"PGRST_LOG_LEVEL": level,
"PGRST_LOG_QUERY": "true",
}
with run(env=env) as postgrest:
response = postgrest.session.get("/")
assert response.status_code == 200
response = postgrest.session.get("/projects")
assert response.status_code == 200
response = postgrest.session.get(
"/projects", headers={"Prefer": "count=estimated"}
)
assert response.status_code == 200
response = postgrest.session.get(
"/projects", headers={"Prefer": "count=planned"}
)
assert response.status_code == 200
response = postgrest.session.get("/infinite_recursion")
assert response.status_code == 500
get_2xx_regx = r'.+: WITH pgrst_source AS.+SELECT "public"\."projects"\.\* FROM "public"\."projects".+_postgrest_t'
get_2xx_count_regx = (
r'.+: EXPLAIN \(FORMAT JSON\) SELECT 1 FROM "public"."projects"'
)
infinite_recursion_5xx_regx = r'.+: WITH pgrst_source AS.+SELECT "public"\."infinite_recursion"\.\* FROM "public"\."infinite_recursion".+_postgrest_t'
root_tables_regx = r".+: SELECT n.nspname AS table_schema, .+ FROM pg_class c .+ ORDER BY table_schema, table_name"
root_procs_regx = r".+: WITH.+base_types AS.+pn\.nspname AS proc_schema.+FROM pg_proc p.+p\.pronamespace = \$1::regnamespace"
root_descr_regx = r".+: SELECT pg_catalog\.obj_description\(\$1::regnamespace, 'pg_namespace'\)"
set_config_regx = (
r".+: select set_config\('search_path', \$1, true\), set_config\("
)
output = drain_stdout(postgrest)
project_queries = [line for line in output if re.match(get_2xx_regx, line)]
project_counts = [line for line in output if re.match(get_2xx_count_regx, line)]
infinite_queries = [
line for line in output if re.match(infinite_recursion_5xx_regx, line)
]
root_tables = [line for line in output if re.match(root_tables_regx, line)]
root_procs = [line for line in output if re.match(root_procs_regx, line)]
root_descr = [line for line in output if re.match(root_descr_regx, line)]
set_configs = [line for line in output if re.match(set_config_regx, line)]
if level == "crit":
assert not set_configs
assert not project_queries
assert not project_counts
assert not infinite_queries
assert not root_tables
assert not root_procs
assert not root_descr
elif level in {"error", "warn"}:
assert len(set_configs) == 1
assert len(infinite_queries) == 1
assert not project_queries
assert not project_counts
assert not root_tables
assert not root_procs
assert not root_descr
elif level == "info":
assert len(set_configs) == 5
assert len(project_queries) == 3
assert len(project_counts) == 2
assert len(infinite_queries) == 1
assert len(root_tables) == 1
assert len(root_procs) == 1
assert len(root_descr) == 1
elif level == "debug":
assert len(set_configs) == 5
assert len(project_queries) == 3
assert len(project_counts) == 2
assert len(infinite_queries) == 1
assert len(root_tables) == 1
assert len(root_procs) == 1
assert len(root_descr) == 1
pre_req_env = {
**env,
"PGRST_DB_PRE_REQUEST": "do_nothing",
}
with run(env=pre_req_env) as postgrest:
response = postgrest.session.get("/projects")
assert response.status_code == 200
output = drain_stdout(postgrest)
pre_request_regx = r'.+: select "do_nothing"()'
pre_reqs = [line for line in output if re.match(pre_request_regx, line)]
if level == "crit":
assert not pre_reqs
elif level in {"error", "warn"}:
assert not pre_reqs
elif level == "info":
assert len(pre_reqs) == 1
elif level == "debug":
assert len(pre_reqs) == 1
def test_log_lacks_role_with_empty_anon_role(defaultenv):
"Requests are logged without a role when db-anon-role is empty."
env = {
**defaultenv,
"PGRST_DB_CONFIG": "false",
"PGRST_DB_ANON_ROLE": "",
}
with run(env=env) as postgrest:
response = postgrest.session.get("/projects")
assert response.status_code == 401
output = postgrest.read_stdout(nlines=1)
assert len(output) == 1
assert re.match(
r'- - - \[.+\] "GET /projects HTTP/1.1" 401 \d+ "" "python-requests/.+"',
output[0],
)
def test_log_postgrest_version(defaultenv):
"Should show the PostgREST version in the logs"
with run(env=defaultenv, no_startup_stdout=False) as postgrest:
version = postgrest.session.head("/").headers["Server"].split("/")[1]
output = postgrest.read_stdout(nlines=1)
assert "Starting PostgREST %s..." % version in output[0]
@pytest.mark.parametrize(
"host", ["127.0.0.1", "::1", None], ids=["IPv4", "IPv6", "Unix"]
)
def test_log_postgrest_host_and_port(host, defaultenv):
"PostgREST should output the host and port it is bound to."
# We run postgrest on unix socket when host and port are set to None
is_unix = host is None
port = None if is_unix else freeport()
with run(
env=defaultenv, host=host, port=port, no_startup_stdout=False
) as postgrest:
output = postgrest.read_stdout(nlines=11)
# Cannot assume a particular log entry order
# Listening on a socket happens after schema querying
# but is concurrent to the schema loading process
# and migh happen before or after writing of the
# "Schema cache loaded" log entry
if is_unix:
match_log(output, [r".*API server listening on .*/tmp/.*\.sock"])
elif is_ipv6(host):
match_log(output, [r".*API server listening on \[.+]:\d+"])
else: # IPv4
match_log(output, [r".*API server listening on .+:\d+"])
@pytest.mark.parametrize(
"host", ["127.0.0.1", "::1", None], ids=["IPv4", "IPv6", "Unix"]
)
def test_log_postgrest_admin_server_host_and_port(host, defaultenv):
"PostgREST should log the admin server host and port"
# We run admin server on unix socket when host and admin_port are set to None
is_unix = host is None
port = None if is_unix else freeport()
admin_port = None if is_unix else freeport(used_ports=[port])
with run(
env=defaultenv,
host=host,
port=port,
admin_port=admin_port,
no_startup_stdout=False,
wait_for=Admin.ready,
) as postgrest:
output = postgrest.read_stdout(nlines=11)
# Cannot assume a particular log entry order
# Listening on a socket happens after schema querying
# but is concurrent to the schema loading process
# and migh happen before or after writing of the
# "Schema cache loaded" log entry
if is_unix:
match_log(output, [r".*Admin server listening on .*/tmp/.*\.sock"])
elif is_ipv6(host):
match_log(output, [r".*Admin server listening on \[.+]:\d+"])
else: # IPv4
match_log(output, [r".*Admin server listening on .+:\d+"])
def test_log_error_when_schema_cache_load_error_on_startup_to_stderr(defaultenv):
"Should log the 503 error message when there is an error loading schema cache on startup"
env = {
**defaultenv,
"PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP_BEFORE_QUERIES": "1000",
"PGRST_DB_SCHEMAS": "non_existent_schema_aaaa",
}
with run(env=env, wait_for=None) as postgrest:
postgrest.wait_until_scache_starts_loading()
# First call should fail with connection refused
with pytest.raises(requests.ConnectionError):
postgrest.session.get("/projects")
# Next call should return 503
time.sleep(1)
response = postgrest.session.get("/projects")
assert response.status_code == 503
output_start = postgrest.read_stdout(nlines=10)
log_err_message = '{"code":"PGRST002","details":null,"hint":null,"message":"Could not query the database for the schema cache. Retrying."}'
assert any(log_err_message in line for line in output_start)
@pytest.mark.parametrize("level", ["crit", "error", "warn", "info", "debug"])
def test_log_pool_req_observation(level, defaultenv):
"PostgREST should log PoolRequest and PoolRequestFullfilled observation when log-level=debug"
env = {**defaultenv, "PGRST_LOG_LEVEL": level, "PGRST_JWT_SECRET": SECRET}
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
pool_req = r".*Trying to borrow a connection from pool.*"
pool_req_fullfill = r".*Borrowed a connection from the pool.*"
with run(env=env) as postgrest:
postgrest.session.get("/authors_only", headers=headers)
if level == "debug":
output = postgrest.read_stdout(nlines=7)
assert len(output) == 7
match_log(output, [pool_req, pool_req_fullfill])
elif level == "info":
output = postgrest.read_stdout(nlines=4)
assert len(output) == 1
else:
output = postgrest.read_stdout(nlines=4)
assert len(output) == 0
def test_log_listener_connection_errors(defaultenv):
"The logs should show the listener connection error message in a single line"
env = {
**defaultenv,
"PGHOST": "no_host",
"PGRST_DB_CHANNEL_ENABLED": "true",
}
with run(env=env, no_startup_stdout=False, wait_for=None) as postgrest:
output = postgrest.read_stdout(nlines=5)
assert any(
'Failed listening for database notifications on the "pgrst" channel. could not translate host name "no_host" to address:'
in line
for line in output
)
def test_log_listener_connection_start(defaultenv):
"The logs should show the listener connection start message in a single line"
env = {
**defaultenv,
"PGRST_DB_CHANNEL_ENABLED": "true",
}
with run(env=env, no_startup_stdout=False, wait_for=Admin.ready) as postgrest:
output = postgrest.read_stdout(nlines=10)
# Check for the listener start message containing host and port
# Do not check if pg version is displayed properly as it is tricky to test it
assert any(
f'"{defaultenv["PGHOST"]}:5432" and listening for database notifications on the "pgrst" channel'
in line
for line in output
)
@pytest.mark.parametrize("level", ["crit", "error", "warn", "info", "debug"])
def test_db_error_logging_to_stderr(level, defaultenv, metapostgrest):
"verify that DB errors are logged to stderr"
role = "timeout_authenticator"
set_statement_timeout(metapostgrest, role, 500)
env = {
**defaultenv,
"PGUSER": role,
"PGRST_DB_ANON_ROLE": role,
"PGRST_LOG_LEVEL": level,
}
with run(env=env) as postgrest:
response = postgrest.session.get("/rpc/sleep?seconds=1")
assert response.status_code == 500
# ensure the message appears on the logs
output = postgrest.read_stdout(nlines=8)
if level == "crit":
assert len(output) == 0
elif level == "debug":
match_log(
output,
[
r".*canceling statement due to statement timeout.*",
r".*500.*",
],
)
else:
assert " 500 " in output[1]
assert "canceling statement due to statement timeout" in output[0]
reset_statement_timeout(metapostgrest, role)
def test_schema_cache_query_sleep_logs(defaultenv):
"""Schema cache sleep should be reflected in the logged query duration."""
env = {
**defaultenv,
"PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP": "1000",
}
log_pattern = re.compile(r"Schema cache queried in ([\d.]+) milliseconds")
with run(env=env, wait_max_seconds=3, no_startup_stdout=False) as postgrest:
observed_ms = None
collected = []
lines = postgrest.read_stdout(nlines=10)
collected.extend(lines)
for line in lines:
match = log_pattern.search(line)
if match:
observed_ms = float(match.group(1))
break
assert observed_ms is not None
assert 1000 < observed_ms < 2000
@pytest.mark.parametrize("level", ["crit", "error", "warn", "info", "debug"])
def test_schema_cache_query_timings_log(level, defaultenv):
"Schema cache query timings should be logged on log-level=debug."
env = {
**defaultenv,
"PGRST_LOG_LEVEL": level,
}
log_pattern = re.compile(
r".+: tables: [\d.]+ ms, keydeps: [\d.]+ ms, rels: [\d.]+ ms, funcs: [\d.]+ ms, comprels: [\d.]+ ms, dreps: [\d.]+ ms, mhandlers: [\d.]+ ms"
)
with run(env=env, no_startup_stdout=False) as postgrest:
output = drain_stdout(postgrest)
timing_matches = [
match for line in output if (match := log_pattern.match(line))
]
if level == "debug":
assert len(timing_matches) == 1
else:
assert not timing_matches
def test_empty_schema_cache_log_contains_jwt_role(defaultenv):
"Requests are logged with the role when the schema cache is empty on startup"
env = {
**defaultenv,
"PGRST_DB_SCHEMAS": "non_existent_schema_aaaa",
"PGRST_JWT_SECRET": SECRET,
}
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
with run(env=env, wait_for=None) as postgrest:
postgrest.wait_until_scache_starts_loading()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 503
output = drain_stdout(postgrest)
assert any(
re.match(
r'- - postgrest_test_author \[.+\] "GET /authors_only HTTP/1.1" 503 \d+ "" "python-requests/.+"',
line,
)
for line in output
)
def test_expired_jwt_log_lacks_role(defaultenv):
"Expired JWT requests are logged without a role."
env = {**defaultenv, "PGRST_JWT_SECRET": SECRET}
headers = jwtauthheader({"exp": relativeSeconds(-35)}, SECRET)
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
output = postgrest.read_stdout(nlines=1)
assert len(output) == 1
assert re.match(
r'- - - \[.+\] "GET /authors_only HTTP/1.1" 401 \d+ "" "python-requests/.+"',
output[0],
)
def test_schema_cache_error_observation(defaultenv):
"schema cache error observation should be logged with invalid db-schemas or db-extra-search-path"
env = {
**defaultenv,
"PGRST_DB_EXTRA_SEARCH_PATH": "x",
}
with run(env=env, no_startup_stdout=False, wait_for=None) as postgrest:
# TODO: postgrest should exit here, instead it keeps retrying
# exitCode = wait_until_exit(postgrest)
# assert exitCode == 1
output = postgrest.read_stdout(nlines=9)
assert (
"Failed to load the schema cache using db-schemas=public and db-extra-search-path=x"
in output[6]
)
def test_invalid_rpc_method_log_contains_role(defaultenv):
"Invalid RPC method requests are logged with the anonymous role."
with run(env=defaultenv) as postgrest:
response = postgrest.session.put("/rpc/sleep")
assert response.status_code == 405
output = postgrest.read_stdout(nlines=1)
assert len(output) == 1
assert re.match(
r'- - postgrest_test_anonymous \[.+\] "PUT /rpc/sleep HTTP/1.1" 405 \d+ "" "python-requests/.+"',
output[0],
)
def test_pgrst_log_503_client_error_to_stderr(defaultenv):
"PostgREST should log 503 errors to stderr"
env = {
**defaultenv,
"PGAPPNAME": "test-io",
}
with run(env=env) as postgrest:
postgrest.session.get("/rpc/terminate_pgrst?appname=test-io")
output = postgrest.read_stdout(nlines=6)
log_message = '{"code":"PGRST001","details":"no connection to the server\\n","hint":null,"message":"Database client error. Retrying the connection."}\n'
assert any(log_message in line for line in output)
def test_termination_unix_signal_logging(defaultenv):
"Server logs when handling termination unix signals."
with run(env=defaultenv) as postgrest:
postgrest.process.send_signal(signal.SIGTERM)
lines = postgrest.read_stdout(nlines=1)
wait_until_exit(postgrest)
assert any("SIGTERM" in line for line in lines)
with run(env=defaultenv) as postgrest:
postgrest.process.send_signal(signal.SIGINT)
lines = postgrest.read_stdout(nlines=1)
wait_until_exit(postgrest)
assert any("SIGINT" in line for line in lines)
def test_options_request_logs_but_cors_preflight_does_not(defaultenv):
"Plain OPTIONS requests should be logged, but CORS preflight requests should not."
env = {
**defaultenv,
"PGRST_LOG_LEVEL": "info",
"PGRST_SERVER_CORS_ALLOWED_ORIGINS": "http://example.com",
}
preflight_headers = {
"Origin": "http://example.com",
"Access-Control-Request-Method": "POST",
"Access-Control-Request-Headers": "Content-Type",
}
with run(env=env) as postgrest:
response = postgrest.session.options("/projects")
assert response.status_code == 200
response = postgrest.session.options("/projects", headers=preflight_headers)
assert response.status_code == 200
assert response.headers["Access-Control-Allow-Origin"] == "http://example.com"
output = drain_stdout(postgrest)
assert len(output) == 1
assert re.match(
r'- - postgrest_test_anonymous \[.+\] "OPTIONS /projects HTTP/1.1" 200 \d+ "" "python-requests/.+"',
output[0],
)
+26
View File
@@ -0,0 +1,26 @@
from util import psql_as_superuser
from postgrest import run
def test_listener_query_is_visible_in_pg_stat_activity(defaultenv):
"The listener connection should show the LISTEN pgrst statement in pg_stat_activity"
env = {
**defaultenv,
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGAPPNAME": "listener-query-test",
}
with run(env=env):
output = psql_as_superuser(
"""
select query
from pg_stat_activity
where application_name = 'listener-query-test'
and query = 'LISTEN "pgrst"'
limit 1;
""",
capture_output=True,
).strip()
assert output == 'LISTEN "pgrst"'
+508
View File
@@ -0,0 +1,508 @@
"Test reloading behaviors in PostgREST"
import signal
import time
import pytest
import requests
from config import CONFIGSDIR, SECRET
from util import (
jwtauthheader,
psql_as_superuser,
)
from postgrest import (
run,
sleep_until_postgrest_config_reload,
sleep_until_postgrest_full_reload,
sleep_until_postgrest_scache_reload,
)
def test_db_schema_notify_reload(defaultenv):
"DB schema and config should be reloaded when PostgREST is sent a NOTIFY"
env = {**defaultenv, "PGRST_DB_CONFIG": "true", "PGRST_DB_CHANNEL_ENABLED": "true"}
with run(env=env) as postgrest:
response = postgrest.session.get("/rpc/get_guc_value?name=search_path")
assert response.text == '"\\"public\\", \\"public\\""'
# change db-schemas config on the db and reload config and cache with notify
postgrest.session.post(
"/rpc/change_db_schema_and_full_reload", data={"schemas": "v1"}
)
sleep_until_postgrest_full_reload()
response = postgrest.session.get("/rpc/get_guc_value?name=search_path")
assert response.text == '"\\"v1\\", \\"public\\""'
# reset db-schemas config on the db
response = postgrest.session.post("/rpc/reset_db_schema_config")
assert response.text == ""
assert response.status_code == 204
def test_db_schema_reload(tmp_path, defaultenv):
"DB schema should be reloaded from file when PostgREST is sent SIGUSR2."
config = (CONFIGSDIR / "sigusr2-settings.config").read_text()
configfile = tmp_path / "test.config"
configfile.write_text(config)
with run(configfile, env=defaultenv) as postgrest:
response = postgrest.session.get("/rpc/get_guc_value?name=search_path")
assert response.text == '"\\"public\\", \\"public\\""'
# change setting
configfile.write_text(
config.replace('db-schemas = "public"', 'db-schemas = "v1"')
)
# reload config
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
# reload schema cache to verify that the config reload actually happened
postgrest.process.send_signal(signal.SIGUSR1)
sleep_until_postgrest_scache_reload()
response = postgrest.session.get("/rpc/get_guc_value?name=search_path")
assert response.text == '"\\"v1\\", \\"public\\""'
def test_invalid_role_claim_key_notify_reload(defaultenv):
"NOTIFY reload config should show an error if role-claim-key is invalid"
env = {
**defaultenv,
"PGRST_DB_CONFIG": "true",
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGRST_LOG_LEVEL": "crit",
}
with run(env=env) as postgrest:
postgrest.session.post("/rpc/invalid_role_claim_key_reload")
output = postgrest.read_stdout()
assert 'Received a config reload message on the "pgrst" channel' in output[0]
output = postgrest.read_stdout()
assert "failed to parse role-claim-key value" in output[0]
response = postgrest.session.post("/rpc/reset_invalid_role_claim_key")
assert response.text == ""
assert response.status_code == 204
def test_max_rows_reload(defaultenv):
"max-rows should be reloaded from role settings when PostgREST receives a SIGUSR2."
env = {
**defaultenv,
"PGRST_DB_CONFIG": "true",
}
with run(env=env) as postgrest:
response = postgrest.session.head("/projects")
assert response.status_code == 200
assert response.headers["Content-Range"] == "0-4/*"
# change max-rows config on the db
postgrest.session.post("/rpc/change_max_rows_config", data={"val": 1})
# reload config
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
response = postgrest.session.head("/projects")
assert response.status_code == 200
assert response.headers["Content-Range"] == "0-0/*"
# reset max-rows config on the db
response = postgrest.session.post("/rpc/reset_max_rows_config")
assert response.text == ""
assert response.status_code == 204
def test_max_rows_notify_reload(defaultenv):
"max-rows should be reloaded from role settings when PostgREST receives a NOTIFY"
env = {
**defaultenv,
"PGRST_DB_CONFIG": "true",
"PGRST_DB_CHANNEL_ENABLED": "true",
}
with run(env=env) as postgrest:
response = postgrest.session.head("/projects")
assert response.status_code == 200
assert response.headers["Content-Range"] == "0-4/*"
# change max-rows config on the db and reload with notify
postgrest.session.post(
"/rpc/change_max_rows_config", data={"val": 1, "notify": True}
)
sleep_until_postgrest_config_reload()
response = postgrest.session.head("/projects")
assert response.status_code == 200
assert response.headers["Content-Range"] == "0-0/*"
# reset max-rows config on the db
response = postgrest.session.post("/rpc/reset_max_rows_config")
assert response.text == ""
assert response.status_code == 204
def test_no_double_schema_cache_reload_on_empty_schema(defaultenv):
"Should only load the schema cache once when there's an empty schema cache on startup"
env = {
**defaultenv,
"PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP": "300",
}
with run(env=env, wait_for=None) as postgrest:
postgrest.wait_until_scache_starts_loading()
with pytest.raises(requests.ConnectionError):
postgrest.session.get("/projects")
# Should wait enough time to load the schema cache twice to guarantee that the test is valid
time.sleep(1)
response = postgrest.session.get("/projects")
assert response.status_code == 200
response = postgrest.admin.get("/metrics")
assert response.status_code == 200
assert 'pgrst_schema_cache_loads_total{status="SUCCESS"} 1.0' in response.text
# https://github.com/PostgREST/postgrest/issues/2620
def test_notify_reloading_catalog_cache(defaultenv):
"notify should reload the connection catalog cache"
with run(env=defaultenv) as postgrest:
# first the id col is an uuid
response = postgrest.session.get(
"/cats?id=eq.dea27321-f988-4a57-93e4-8eeb38f3cf1e"
)
assert response.status_code == 200
# change it to a bigint
response = postgrest.session.post("/rpc/drop_change_cats")
assert response.text == ""
assert response.status_code == 204
sleep_until_postgrest_scache_reload()
# next request should succeed with a bigint value
response = postgrest.session.get("/cats?id=eq.1")
assert response.status_code == 200
def test_notify_do_nothing(defaultenv):
"NOTIFY with unknown message should do nothing"
env = {
**defaultenv,
"PGRST_DB_CONFIG": "true",
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGRST_LOG_LEVEL": "crit",
}
with run(env=env) as postgrest:
response = postgrest.session.post("/rpc/notify_do_nothing")
assert response.text == ""
assert response.status_code == 204
output = postgrest.read_stdout()
assert output == []
def test_schema_cache_concurrent_notifications(slow_schema_cache_env):
"schema cache should be up-to-date whenever a notification is sent while another reload is in progress, see https://github.com/PostgREST/postgrest/issues/2791"
internal_sleep = (
int(slow_schema_cache_env["PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP"]) / 1000
)
with run(env=slow_schema_cache_env, wait_for=None) as postgrest:
time.sleep(2 * internal_sleep + 0.1) # wait for readiness manually
# first request, create a function and set a schema cache reload in progress
response = postgrest.session.post("/rpc/create_function")
assert response.text == ""
assert response.status_code == 204
time.sleep(
internal_sleep / 2
) # wait to be inside the schema cache reload process
# second request, change the same function and do another schema cache reload
response = postgrest.session.post("/rpc/migrate_function")
assert response.text == ""
assert response.status_code == 204
time.sleep(
2 * internal_sleep
) # wait enough time to get the final schema cache state
# confirm the schema cache is up-to-date and the 2nd reload wasn't lost
response = postgrest.session.get("/rpc/mult_them?c=3&d=4")
assert response.text == "12"
assert response.status_code == 200
def test_jwt_secret_reload(tmp_path, defaultenv):
"JWT secret should be reloaded from file when PostgREST is sent SIGUSR2."
config = (CONFIGSDIR / "sigusr2-settings.config").read_text()
configfile = tmp_path / "test.config"
configfile.write_text(config)
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
with run(configfile, env=defaultenv) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
# change setting
configfile.write_text(config.replace("invalid" * 5, SECRET))
# reload config
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 200
def test_jwt_secret_external_file_reload(tmp_path, defaultenv):
"JWT secret external file should be reloaded when PostgREST is sent a SIGUSR2 or a NOTIFY."
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
external_secret_file = tmp_path / "jwt-secret-config"
external_secret_file.write_text("invalid" * 5)
env = {
**defaultenv,
"PGRST_JWT_SECRET": f"@{external_secret_file}",
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGRST_DB_CONFIG": "false",
"PGRST_DB_ANON_ROLE": "postgrest_test_anonymous", # required for NOTIFY
}
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
# change external file
external_secret_file.write_text(SECRET)
# SIGUSR1 doesn't reload external files, at least when db-config=false
postgrest.process.send_signal(signal.SIGUSR1)
sleep_until_postgrest_scache_reload()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
# reload config and external file with SIGUSR2
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 200
# change external file to wrong value again
external_secret_file.write_text("invalid" * 5)
# reload config and external file with NOTIFY
response = postgrest.session.post("/rpc/reload_pgrst_config")
assert response.text == ""
assert response.status_code == 204
sleep_until_postgrest_config_reload()
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
def test_invalidate_jwt_cache_when_secret_changes(tmp_path, defaultenv):
"JWT cache should be emptied after jwt-secret is changed in a config reload"
headers = jwtauthheader({"role": "postgrest_test_author"}, SECRET)
external_secret_file = tmp_path / "jwt-secret-config"
external_secret_file.write_text(SECRET)
env = {
**defaultenv,
"PGRST_JWT_SECRET": f"@{external_secret_file}",
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGRST_JWT_CACHE_MAX_ENTRIES": "86400", # enable cache
"PGRST_DB_ANON_ROLE": "postgrest_test_anonymous", # required for NOTIFY
}
with run(env=env) as postgrest:
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 200 # jwt gets cached
# change external file
external_secret_file.write_text("invalid" * 5)
# reload config and external file with NOTIFY
# jwt-cache should get empty
response = postgrest.session.post("/rpc/reload_pgrst_config")
assert response.text == ""
assert response.status_code == 204
sleep_until_postgrest_config_reload()
# now the request should fail because the cached token is removed
response = postgrest.session.get("/authors_only", headers=headers)
assert response.status_code == 401
def test_stale_schema_cache_dropped_table_returns_database_error(defaultenv):
"dropped table should return a database error while schema cache is stale"
internal_sleep = 2
env = {
**defaultenv,
"PGRST_DB_POOL": "2",
"PGRST_DB_CHANNEL_ENABLED": "true",
"PGRST_INTERNAL_SCHEMA_CACHE_QUERY_SLEEP": str(internal_sleep * 1000),
}
try:
psql_as_superuser("""
drop table if exists stale_schema_cache_items;
create table stale_schema_cache_items(id int primary key);
insert into stale_schema_cache_items values (1);
grant select on stale_schema_cache_items to postgrest_test_anonymous;
""")
with run(env=env, wait_max_seconds=10) as postgrest:
response = postgrest.session.get("/stale_schema_cache_items")
assert response.status_code == 200
psql_as_superuser("""
drop table stale_schema_cache_items;
notify pgrst, 'reload schema';
""")
response = postgrest.session.get("/stale_schema_cache_items")
payload = response.json()
assert response.status_code == 404
assert payload["code"] == "42P01"
assert (
payload["message"]
== 'relation "public.stale_schema_cache_items" does not exist'
)
time.sleep(internal_sleep + 0.3)
response = postgrest.session.get("/stale_schema_cache_items")
payload = response.json()
assert response.status_code == 404
assert payload["code"] == "PGRST205"
assert (
payload["message"]
== "Could not find the table 'public.stale_schema_cache_items' in the schema cache"
)
finally:
psql_as_superuser("drop table if exists stale_schema_cache_items;")
def test_config_log_level_is_reloadable(tmp_path, defaultenv):
"Config log-level should be reloadable on SIGUSR2"
config = (CONFIGSDIR / "sigusr2-settings.config").read_text()
configfile = tmp_path / "test.config"
configfile.write_text(config)
# Delete the env variable for "log-level" so the config file value isn't overridden
del defaultenv["PGRST_LOG_LEVEL"]
with run(configfile, env=defaultenv) as postgrest:
response = postgrest.session.get("/projects")
assert response.status_code == 200
output = postgrest.read_stdout(nlines=5)
# log-level = error, so this log line shouldn't be logged
assert not any(
"Trying to borrow a connection from pool" in line for line in output
)
# change setting
configfile.write_text(
config.replace('log-level = "error"', 'log-level = "debug"')
)
# reload
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
response = postgrest.session.get("/projects")
assert response.status_code == 200
output = postgrest.read_stdout(nlines=5)
# log-level = debug now, so this log line must be logged
assert any("Trying to borrow a connection from pool" in line for line in output)
def test_config_db_channel_enabled_is_reloadable(tmp_path, defaultenv):
"Config db-channel-enabled should be reloadable on SIGUSR2"
config = (CONFIGSDIR / "sigusr2-settings.config").read_text()
configfile = tmp_path / "test.config"
configfile.write_text(config)
with run(configfile, env=defaultenv, no_startup_stdout=False) as postgrest:
output = postgrest.read_stdout(nlines=7)
# db-channel-enabled = false, so this shouldn't be logged
assert not any(
f'"{defaultenv["PGHOST"]}:5432" and listening for database notifications on the "pgrst" channel'
in line
for line in output
)
# change setting
configfile.write_text(
config.replace(
'db-channel-enabled = "false"', 'db-channel-enabled = "true"'
)
)
# reload
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
output = postgrest.read_stdout(nlines=7)
# db-channel-enabled = true, so this logged
assert any(
f'"{defaultenv["PGHOST"]}:5432" and listening for database notifications on the "pgrst" channel'
in line
for line in output
)
# change setting back to false
configfile.write_text(
configfile.read_text().replace(
'db-channel-enabled = "true"', 'db-channel-enabled = "false"'
)
)
# reload
postgrest.process.send_signal(signal.SIGUSR2)
sleep_until_postgrest_config_reload()
output = postgrest.read_stdout(nlines=7)
# db-channel-enabled = false, so this shouldn't be logged
assert not any(
f'"{defaultenv["PGHOST"]}:5432" and listening for database notifications on the "pgrst" channel'
in line
for line in output
)
+75
View File
@@ -0,0 +1,75 @@
import time
from util import Thread
from postgrest import (
freeport,
run,
wait_until_exit,
)
def test_so_reuseport_zero_downtime_handover(defaultenv):
"A second PostgREST instance should take over on the same main/admin ports without request failures."
# set host to _all_ addresses to force port conflict without SO_REUSEPORT
# setting to localhost (which is the default)
# might allow running multiple instances on the same port
# as the name might be resolved to many IP addresses
host = "0.0.0.0"
port = freeport()
admin_port = freeport(used_ports=[port])
failures = []
# mutable location shared between threads
keep_running = {"value": True}
# 1. Start first PostgREST instance
# 2. Start a "client" thread issuing requests in a loop
# remembering all received errors
# 3. Start second PostgREST instance on the same port as the first one
# 4. Wait a little and terminate the first instance
#
# We expect the client does not get any errors after stopping the first instance
# and seamlessly migrate to the second instance.
#
# 5. Stop client thread
# 6. Stop second PostgREST instance
# 7. Verify client did not get any errors
with run(
env={**defaultenv, "PGRST_SERVER_REUSEPORT": "true"},
port=port,
host=host,
admin_port=admin_port,
) as first:
def continuously_request():
while keep_running["value"]:
try:
response = first.session.get("/projects", timeout=1)
assert response.status_code == 200
except Exception as exc:
failures.append(exc)
break
time.sleep(0.2)
requester = Thread(target=continuously_request)
requester.start()
try:
time.sleep(1)
with run(
env={**defaultenv, "PGRST_SERVER_REUSEPORT": "true"},
port=port,
host=host,
# we do not set SO_REUSEPORT on admin socket
admin_port=freeport(used_ports=[port, admin_port]),
):
time.sleep(1)
first.process.terminate()
wait_until_exit(first, 2)
time.sleep(1)
finally:
keep_running["value"] = False
requester.join()
assert failures == []