Compare commits

...
38 Commits
Author SHA1 Message Date
Laurence Isla 519615d5fb bump version to 12.2.3 2024-08-01 11:46:52 -05:00
Laurence Isla de5a8954b2 changelog: add missing entry for 3670 2024-08-01 11:18:15 -05:00
64b5a59410 fix: schema cache loading before the in-db config (#3670)
Fixes #3660. Load the config after getting the pg version but before loading the schema.

The regression happened on f09655b.

Also remove schema cache load wrapper and separate db queries in different functions.

Co-authored-by: Laurence Isla <lau.isla.c@gmail.com>
2024-08-01 17:41:07 +02:00
Dan KurinandWolfgang Walther 417a96d137 fix: Remove OpenAPI format for rowFilter params (#3661) 2024-08-01 17:41:07 +02:00
Wolfgang Walther 53593824a8 fix: Embed One-to-One relationship with different column order properly 2024-07-13 22:27:04 +02:00
Wolfgang Walther e642086d27 chore: Fix changelog after 13868983 2024-07-13 17:16:06 +02:00
steve-chavezandWolfgang Walther e451700d5d chore: remove links to gitter
We'll now use github discussions for support.
2024-07-13 17:11:06 +02:00
Salim BandWolfgang Walther 13868983af fix: update OpenAPI externalDocs URL
fixes https://github.com/PostgREST/postgrest/issues/3091
2024-07-13 17:11:06 +02:00
steve-chavez db9da0b1ca bump version to 12.2.2 2024-07-10 23:07:23 -05:00
steve-chavez f09655b7a6 fix: schema cache retrying without backoff
Fixes https://github.com/PostgREST/postgrest/issues/3523.

Now if there's a failure when obtaining the pg version OR schema cache,
we do the same retrying process. This way we don't add two retries.

Refactors and renames the "connectionWorker" to "schemaCacheLoader".
This makes more sense since what we really want is the schema cache,
the version is the pre-requisite for ensuring our
schema cache queries work.

Additionally, we no longer log ` Attempting to connect to the database...`
at startup unnecessarily. This is only logged whenever there's a retry attempt.
2024-07-10 21:11:20 -05:00
steve-chavezandWolfgang Walther 8715e426c0 chore: remove paypal links
It was tied to a personal account and donations there have been too rare.
2024-07-10 21:13:23 +02:00
Wolfgang WaltherandWolfgang Walther 9a40bc266e fix: List correct enum options when multiple types with same name are present
The schema cache and OpenAPI output would currently list the first found
enum with the same name instead of the correct type. One other case
where this comes up is when a regular type and an enum type have the
same name. For example in the spec fixtures, we have an enum called
"bit". Every "bit" type, no matter whether it's that enum or the
built-in bit type, will show those enum options in the OpenApi output.

Not adding a test, because OpenAPI is supposed to go away in the future
anyway.
2024-07-09 09:09:22 +02:00
Wolfgang WaltherandWolfgang Walther e67461b991 fix: Show number of loaded timezones in log output
There is no reason to hide those, right?
2024-07-09 09:09:22 +02:00
Wolfgang WaltherandWolfgang Walther 420933ceba fix: Make --dump-schema work with in-database pgrst.db_schemas setting
This needs to be loaded from in-database configuration first, otherwise
the dump-schema output will be for the default (public) schema.
2024-07-09 09:09:22 +02:00
Laurence IslaandWolfgang Walther 8f8dbd3425 fix: nested empty embeds no longer return empty values and are correctly omitted 2024-07-09 09:09:22 +02:00
renovate[bot]andWolfgang Walther 8d7d7e4112 chore(deps): update all dependencies 2024-07-06 11:35:11 +02:00
Sandro BauerandWolfgang Walther 521acf61cb docs: fix rendering for inline code block in operator list 2024-07-02 12:28:24 +02:00
steve-chavez d3f7cba92e bump version to 12.2.1 2024-06-27 14:59:35 -05:00
Laurence IslaandSteve Chavez a71d9ea50f changelog: add missing entries for #3592 and #3616 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 5a1e9b01e4 nix: remove texlive dependencies from postgrest-docs-render 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez d3292d42de docs: use PlantUML instead of Latex to generate Schema Isolation image 2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 572581cbb6 chore: organize diagrams in different folders 2024-06-27 14:08:16 -05:00
steve-chavez 9daddb549c docs: add listener recovery 2024-06-27 14:08:16 -05:00
steve-chavez 716fcbf147 fix: don't reload cache on every listener fail
Revert "prevent GSSAPI error between Listener and pool"

This reverts commit 4beac10d3d.
2024-06-27 14:08:16 -05:00
Laurence IslaandSteve Chavez 5b62501ce7 docs: improve architecture diagram
- SVG format instead of PNG
- The components now have links to their reference in the Docs
- Supports dark mode
2024-06-27 14:08:16 -05:00
Wolfgang WaltherandSteve Chavez 7860e7287f docs: Fix punctuation in install.rst 2024-06-27 14:08:16 -05:00
Michal KleczekandSteve Chavez c958f94753 docs: add pg-notify-stdout to ecosystem 2024-06-27 14:08:16 -05:00
renovate[bot]andWolfgang Walther 94f0548344 chore(deps): update ubuntu:noble docker digest to 2e863c4 2024-06-19 19:22:47 +02:00
Wolfgang Walther 16d0e1ca8c fix: Treat pre-release and docs versions correctly for new release workflow
Since we changed our release workflow, we have adjusted:
- the docs to use postgrest.org/en/v12/ -style URLs, i.e. only using the
major component.
- the pre-release / devel versions to contain only two instead of four
version parts, i.e. currently 12.3.
2024-06-19 08:42:34 +02:00
Wolfgang Walther 9657d46c42 docs: Fix linkcheck
Some URLs are still forbidden for our linkcheck tool, so disabling them
again.

Others are permanently redirected, so adjusting them.
2024-06-19 08:40:58 +02:00
renovate[bot]andWolfgang Walther 70e07ccbc6 chore(deps): update dependency urllib3 to v2.2.2 2024-06-17 21:04:03 +02:00
renovate[bot]andWolfgang Walther cc5c4ae5ef chore(deps): update codecov/codecov-action action to v4.5.0 2024-06-13 19:51:15 +02:00
renovate[bot]andWolfgang Walther af8b62ba10 chore(deps): update all dependencies 2024-06-13 08:46:23 +02:00
Laurence Isla d4e7d23268 docs: fix example of listener failure on read replicas 2024-06-12 19:46:23 -05:00
Laurence Isla 4e084b664c docs: clarify what is logged when "log-level=debug" 2024-06-12 18:30:19 -05:00
Laurence Isla 91024b6eab docs: add missing logs to stderr
- Schema cache stats are now logged to stderr
- Log when the LISTEN channel gets a notification
2024-06-12 18:30:19 -05:00
Laurence IslaandSteve Chavez 0e4fcd7387 docs: add "Listener" page
Co-authored-by: Steve Chavez <stevechavezast@gmail.com>
2024-06-12 18:30:19 -05:00
Wolfgang Walther 49a7aa9933 bump docs version to 12.2 2024-06-12 07:47:17 +02:00
62 changed files with 484 additions and 448 deletions
@@ -112,7 +112,7 @@ runs:
echo "artifacts=${artifacts}" >> "$GITHUB_OUTPUT"
- name: Save artifact to GitHub Actions
if: steps.find-task.outputs.task_found
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ inputs.upload }}
path: ${{ steps.download.outputs.artifacts }}
+1 -1
View File
@@ -1,6 +1,6 @@
# PostgREST docker hub image
FROM ubuntu:noble@sha256:3f85b7caad41a95462cf5b787d8a04604c8262cdcdf9a472b8c52ef83375fe15 AS postgrest
FROM ubuntu:noble@sha256:2e863c44b718727c860746568e1d54afd13b2fa71b160f5cd9058fc436217b30 AS postgrest
RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
+10 -10
View File
@@ -33,7 +33,7 @@ jobs:
name: Nix - Linux static
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -42,7 +42,7 @@ jobs:
- name: Build static executable
run: nix-build -A postgrestStatic
- name: Save built executable as artifact
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-linux-static-x64
path: result/bin/postgrest
@@ -51,7 +51,7 @@ jobs:
- name: Build Docker image
run: nix-build -A docker.image --out-link postgrest-docker.tar.gz
- name: Save built Docker image as artifact
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-docker-x64
path: postgrest-docker.tar.gz
@@ -62,7 +62,7 @@ jobs:
name: Nix - MacOS
runs-on: macos-12
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -118,8 +118,8 @@ jobs:
name: Stack - ${{ matrix.name }}
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: haskell-actions/setup@33585e1a16afa5875e124b0ebc89dd0c2f872c21 # v2.7.3
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: haskell-actions/setup@ec49483bfc012387b227434aba94f59a6ecd0900 # v2.7.5
with:
# This must match the version in stack.yaml's resolver
ghc-version: 9.6.5
@@ -148,7 +148,7 @@ jobs:
run: strip result/postgrest*
- name: Save built executable as artifact
if: matrix.artifact
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: ${{ matrix.artifact }}
path: |
@@ -161,7 +161,7 @@ jobs:
name: Stack - FreeBSD from CirrusCI
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: ./.github/actions/artifact-from-cirrus
with:
token: ${{ github.token }}
@@ -178,8 +178,8 @@ jobs:
name: Cabal - Linux GHC ${{ matrix.ghc }}
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: haskell-actions/setup@33585e1a16afa5875e124b0ebc89dd0c2f872c21 # v2.7.3
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: haskell-actions/setup@ec49483bfc012387b227434aba94f59a6ecd0900 # v2.7.5
with:
ghc-version: ${{ matrix.ghc }}
- name: Cache .cabal
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
name: Lint & Style
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
+13 -13
View File
@@ -59,7 +59,7 @@ jobs:
GITHUB_COMMIT: ${{ github.sha }}
GHC_VERSION: '9.4.8'
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- id: Remote-Dir
name: Unique directory name for the remote build
run: echo "remotepath=postgrest-build-$(uuidgen)" >> "$GITHUB_OUTPUT"
@@ -99,7 +99,7 @@ jobs:
- name: Extract downloaded binaries
run: tar -xvf result.tar.xz && rm result.tar.xz
- name: Save aarch64 executable as artifact
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: postgrest-ubuntu-aarch64
path: result/postgrest
@@ -129,7 +129,7 @@ jobs:
- build
- arm
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ssh-key: ${{ secrets.POSTGREST_SSH_KEY }}
- name: Tag latest commit
@@ -160,7 +160,7 @@ jobs:
- build
- arm
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
@@ -183,7 +183,7 @@ jobs:
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Save CHANGES.md as artifact
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: release-changes
path: CHANGES.md
@@ -199,9 +199,9 @@ jobs:
- prepare
if: success() || needs.prepare.result == 'success'
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Download all artifacts
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: artifacts
- name: Create release bundle with archives for all builds
@@ -226,7 +226,7 @@ jobs:
artifacts/postgrest-windows-x64/postgrest.exe
- name: Save release bundle
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: release-bundle
path: release-bundle
@@ -273,9 +273,9 @@ jobs:
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Download Docker image
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: postgrest-docker-x64
- name: Publish images on Docker Hub
@@ -303,7 +303,7 @@ jobs:
vars.DOCKER_REPO && vars.DOCKER_USER &&
github.ref == 'refs/tags/devel'
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: peter-evans/dockerhub-description@e98e4d1628a5f3be2be7c231e50981aee98723ae # v4.0.0
with:
username: ${{ vars.DOCKER_USER }}
@@ -325,7 +325,7 @@ jobs:
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Publish images for ARM builds on Docker Hub
uses: appleboy/ssh-action@master
env:
@@ -350,7 +350,7 @@ jobs:
env:
REMOTE_DIR: ${{ needs.arm.outputs.remotepath }}
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Remove uploaded files from server
uses: appleboy/ssh-action@master
with:
+3 -3
View File
@@ -27,7 +27,7 @@ jobs:
name: Build
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -41,7 +41,7 @@ jobs:
name: Spellcheck
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -58,7 +58,7 @@ jobs:
if: github.base_ref == 'main'
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
if: github.event.workflow_run.conclusion == 'success' || github.event.workflow_run.conclusion == 'failure'
steps:
- name: Download from Artifacts
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
github-token: ${{ github.token }}
run-id: ${{ github.event.workflow_run.id }}
+6 -6
View File
@@ -39,7 +39,7 @@ jobs:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -49,7 +49,7 @@ jobs:
- name: Run coverage (IO tests and Spec tests against PostgreSQL 15)
run: postgrest-coverage
- name: Upload coverage to codecov
uses: codecov/codecov-action@125fc84a9a348dbcf27191600683ec096ec9021c # v4.4.1
uses: codecov/codecov-action@e28ff129e5465c2c0dcc6f003fc735cb6ae0c673 # v4.5.0
with:
files: ./coverage/codecov.json
token: ${{ secrets.CODECOV_TOKEN }}
@@ -76,7 +76,7 @@ jobs:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -100,7 +100,7 @@ jobs:
name: Memory
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
@@ -114,7 +114,7 @@ jobs:
name: Loadtest
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
fetch-depth: 0
- name: Setup Nix Environment
@@ -131,7 +131,7 @@ jobs:
postgrest-loadtest-against main ${{ steps.get-latest-tag.outputs.tag }}
postgrest-loadtest-report > loadtest/loadtest.md
- name: Upload report
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
with:
name: loadtest.md
path: loadtest/loadtest.md
+30
View File
@@ -5,6 +5,36 @@ This project adheres to [Semantic Versioning](http://semver.org/).
## Unreleased
## [12.2.3] - 2024-08-01
### Fixed
- #3091, Broken link in OpenAPI description `externalDocs` - @salim-b
- #3659, Embed One-to-One relationship with different column order properly - @wolfgangwalther
- #3504, Remove `format` from `rowFilter` parameters in OpenAPI - @dantheman2865
- #3660, Fix regression that loaded the schema cache before the in-database configuration - @steve-chavez, @laurenceisla
## [12.2.2] - 2024-07-10
### Fixed
- #3093, Nested empty embeds no longer show empty values and are correctly omitted - @laurenceisla
- #3644, Make --dump-schema work with in-database pgrst.db_schemas setting - @wolfgangwalther
- #3644, Show number of timezones in schema cache load report - @wolfgangwalther
- #3644, List correct enum options in OpenApi output when multiple types with same name are present - @wolfgangwalther
- #3523, Fix schema cache loading retry without backoff - @steve-chavez
## [12.2.1] - 2024-06-27
### Fixed
- #3147, Don't reload schema cache on every listener failure - @steve-chavez
### Documentation
- #3592, Architecture diagram now supports dark mode and has links - @laurenceisla
- #3616, The schema isolation diagram now supports dark mode and uses well-known schemas - @laurenceisla
## [12.2.0] - 2024-06-11
### Added
+1 -7
View File
@@ -1,8 +1,6 @@
![Logo](static/postgrest.png "Logo")
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854)](https://www.patreon.com/postgrest)
[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/postgrest)
[![Join the chat at https://gitter.im/begriffs/postgrest](https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/begriffs/postgrest)
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://postgrest.org)
[![Docker Stars](https://img.shields.io/docker/pulls/postgrest/postgrest.svg)](https://hub.docker.com/r/postgrest/postgrest/)
[![Build Status](https://github.com/postgrest/postgrest/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/PostgREST/postgrest/actions?query=branch%3Amain)
@@ -145,11 +143,7 @@ and the [API guide](http://postgrest.org/en/stable/api.html).
## Supporting development
You can help PostgREST ongoing maintenance and development by:
- Making a regular donation through Patreon https://www.patreon.com/postgrest
- Alternatively, you can make a one-time donation via Paypal https://www.paypal.me/postgrest
You can help PostgREST ongoing maintenance and development by making a regular donation through Patreon https://www.patreon.com/postgrest
Every donation will be spent on making PostgREST better for the whole community.
-2
View File
@@ -1,8 +1,6 @@
# PostgREST
[![Join the chat at https://gitter.im/begriffs/postgrest](https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/begriffs/postgrest)
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854)](https://www.patreon.com/postgrest)
[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/postgrest)
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://postgrest.org)
[![Build Status](https://github.com/postgrest/postgrest/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/PostgREST/postgrest/actions?query=branch%3Amain)
+8 -23
View File
@@ -5,35 +5,20 @@ The ER diagrams were created with https://github.com/BurntSushi/erd/.
You can go download erd from https://github.com/BurntSushi/erd/releases and then do:
```bash
./erd_static-x86-64 -i film.er -o ../_static/film.png
./erd_static-x86-64 -i ./er/film.er -o ../_static/film.png
```
The fonts used belong to the GNU FreeFont family. You can download them here: http://ftp.gnu.org/gnu/freefont/
## LaTeX
## UML
The schema structure diagram is done with LaTeX. You can use a GUI like https://www.mathcha.io/editor to create the .tex file.
The UML diagrams are created with https://plantuml.com/.
Then use this command to generate the png file.
PlantUML only creates one diagram per file.
That's why we need to create another one for dark mode.
For example, for the file [uml/arch.uml](uml/arch.uml) there's [uml/dark/arch-dark.uml](uml/dark/arch-dark.uml) which includes the first one:
```bash
postgrest-docs-render
```
LaTeX is used because it's a tweakable plain text format.
You can install the full latex suite with `nix`:
```
nix-env -iA texlive.combined.scheme-full
```
To tweak the file with a live reload environment use:
```bash
# open the pdf(zathura used as an example)
zathura db.pdf &
# live reload with entr
echo db.tex | entr pdflatex --shell-escape -halt-on-error db.tex
plantuml -tsvg uml/arch.uml -o ../../_static
plantuml -tsvg -darkmode uml/dark/arch-dark.uml -o ../../../_static
```
-71
View File
@@ -1,71 +0,0 @@
\documentclass[convert]{standalone}
\usepackage{amsmath}
\usepackage{tikz}
\usepackage{mathdots}
\usepackage{yhmath}
\usepackage{cancel}
\usepackage{color}
\usepackage{siunitx}
\usepackage{array}
\usepackage{multirow}
\usepackage{amssymb}
\usepackage{gensymb}
\usepackage{tabularx}
\usepackage{booktabs}
\usetikzlibrary{fadings}
\usetikzlibrary{patterns}
\usetikzlibrary{shadows.blur}
\usetikzlibrary{shapes}
\begin{document}
\newcommand\customScale{0.35}
\begin{tikzpicture}[x=0.75pt,y=0.75pt,yscale=-1,xscale=1, scale=\customScale, every node/.style={scale=\customScale}]
%Shape: Can [id:dp7234864758664346]
\draw [fill={rgb, 255:red, 47; green, 97; blue, 144 } ,fill opacity=1 ] (497.5,51.5) -- (497.5,255.5) .. controls (497.5,275.66) and (423.18,292) .. (331.5,292) .. controls (239.82,292) and (165.5,275.66) .. (165.5,255.5) -- (165.5,51.5) .. controls (165.5,31.34) and (239.82,15) .. (331.5,15) .. controls (423.18,15) and (497.5,31.34) .. (497.5,51.5) .. controls (497.5,71.66) and (423.18,88) .. (331.5,88) .. controls (239.82,88) and (165.5,71.66) .. (165.5,51.5) ;
%Shape: Rectangle [id:dp7384065579958246]
\draw [fill={rgb, 255:red, 236; green, 227; blue, 227 } ,fill opacity=1 ] (189,115) -- (252.5,115) -- (252.5,155) -- (189,155) -- cycle ;
%Shape: Rectangle [id:dp24763906430298177]
\draw [fill={rgb, 255:red, 236; green, 227; blue, 227 } ,fill opacity=1 ] (292,118) -- (362,118) -- (362,158) -- (292,158) -- cycle ;
%Shape: Rectangle [id:dp3775601612537265]
\draw [fill={rgb, 255:red, 236; green, 227; blue, 227 } ,fill opacity=1 ] (397,114) -- (467,114) -- (467,154) -- (397,154) -- cycle ;
%Shape: Rectangle [id:dp7071457022893852]
\draw [fill={rgb, 255:red, 248; green, 231; blue, 28 } ,fill opacity=1 ] (269,199) -- (397.5,199) -- (397.5,273) -- (269,273) -- cycle ;
%Straight Lines [id:da8846759047437789]
\draw (268,234) -- (226.44,155.77) ;
\draw [shift={(225.5,154)}, rotate = 422.02] [color={rgb, 255:red, 0; green, 0; blue, 0 } ][line width=0.75] (10.93,-3.29) .. controls (6.95,-1.4) and (3.31,-0.3) .. (0,0) .. controls (3.31,0.3) and (6.95,1.4) .. (10.93,3.29) ;
%Straight Lines [id:da6908444738113828]
\draw (309.5,198) -- (307.6,161) ;
\draw [shift={(307.5,159)}, rotate = 447.06] [color={rgb, 255:red, 0; green, 0; blue, 0 } ][line width=0.75] (10.93,-3.29) .. controls (6.95,-1.4) and (3.31,-0.3) .. (0,0) .. controls (3.31,0.3) and (6.95,1.4) .. (10.93,3.29) ;
%Straight Lines [id:da7168757864413169]
\draw (398.5,233) -- (431.72,154.84) ;
\draw [shift={(432.5,153)}, rotate = 473.03] [color={rgb, 255:red, 0; green, 0; blue, 0 } ][line width=0.75] (10.93,-3.29) .. controls (6.95,-1.4) and (3.31,-0.3) .. (0,0) .. controls (3.31,0.3) and (6.95,1.4) .. (10.93,3.29) ;
%Up Down Arrow [id:dp14059754167108496]
\draw [fill={rgb, 255:red, 126; green, 211; blue, 33 } ,fill opacity=1 ] (312.5,288.5) -- (330,273) -- (347.5,288.5) -- (338.75,288.5) -- (338.75,319.5) -- (347.5,319.5) -- (330,335) -- (312.5,319.5) -- (321.25,319.5) -- (321.25,288.5) -- cycle ;
% Text Node
\draw (201,129) node [anchor=north west][inner sep=0.75pt] [align=left] {tables};
% Text Node
\draw (307,130) node [anchor=north west][inner sep=0.75pt] [align=left ] {tables};
% Text Node
\draw (414,127) node [anchor=north west][inner sep=0.75pt] [align=left] {tables};
% Text Node
\draw (300,203) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 0; green, 0; blue, 0 } ,opacity=1 ] [align=center] { \\ views \\ + \\ \ \ functions};
% Text Node
\draw (322,178) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 255; green, 255; blue, 255 } ,opacity=1 ] [align=left] {\large\textbf{api}};
% Text Node
\draw (190,97) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 255; green, 255; blue, 255 } ,opacity=1 ] [align=left] {\large\textbf{internal}};
% Text Node
\draw (300,99) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 255; green, 255; blue, 255 } ,opacity=1 ] [align=left] {\large\textbf{private}};
% Text Node
\draw (417,101) node [anchor=north west][inner sep=0.75pt] [color={rgb, 255:red, 255; green, 255; blue, 255 } ,opacity=1 ] [align=left] {\large\textbf{core}};
% Text Node
\draw (358,306) node [anchor=north west][inner sep=0.75pt] [align=left] {REST};
\end{tikzpicture}
\end{document}
@@ -1,5 +1,7 @@
@startuml
skinparam backgroundColor transparent
package "PostgREST" {
() HTTP as HTTPAPI
HTTPAPI - [Auth]
@@ -14,11 +16,9 @@ package "PostgREST" {
[Admin] -r- () HTTPADMIN
[Config] -l- () CLI
[Config] <-l~ Listener
[Config] <-r~ Listener
HTTPADMIN -[hidden]r- CLI
Listener -[hidden]r- [Schema Cache]
Listener -[hidden]r- [Config]
[Schema Cache] -l[hidden]- [Config]
[Schema Cache] -l[hidden]- [Admin]
[Schema Cache] -l[hidden]- CLI
@@ -26,16 +26,16 @@ package "PostgREST" {
database "PostgreSQL" {
node "API schema" as API {
rectangle "Functions, Views"
}
node Authorization {
rectangle "Roles, GRANT, RLS"
}
node "API schema" as API {
rectangle "Functions, Views"
}
rectangle "Tables, extensions" as tbs
API -d- tbs
API -[hidden]l- Authorization
API -l[hidden]- Authorization
}
:user:
@@ -67,4 +67,18 @@ end note
note top of Listener
LISTEN session
end note
url of Admin is [[../references/admin_server.html#admin-server]]
url of API is [[../explanations/schema_isolation.html]]
url of Auth is [[../references/auth.html#authn]]
url of Authorization is [[../explanations/db_authz.html]]
url of CLI is [[../references/cli.html#cli]]
url of "Connection Pool" is [[../references/connection_pool.html]]
url of Config is [[../references/configuration.html#configuration]]
url of HTTPADMIN is [[https://aosabook.org/en/posa/warp.html]]
url of HTTPAPI is [[https://aosabook.org/en/posa/warp.html]]
url of Listener is [[../references/listener.html#listener]]
url of Proxy is [[../explanations/nginx.html]]
url of "Schema Cache" is [[../references/schema_cache.html#schema-cache]]
@enduml
+3
View File
@@ -0,0 +1,3 @@
@startuml
!include ../arch.uml
@enduml
+3
View File
@@ -0,0 +1,3 @@
@startuml
!include ../sch-iso.uml
@enduml
+29
View File
@@ -0,0 +1,29 @@
@startuml
skinparam backgroundColor transparent
skinparam linetype ortho
skinparam node {
backgroundColor transparent
borderThickness 1
}
database "PostgreSQL" {
node public {
rectangle tables_public as "tables"
}
node extensions as "**extensions**" {
}
node API as "<size:20>api" {
rectangle vf_api as "views + functions"
}
tables_public <-- vf_api
extensions <-- vf_api
}
vf_api <-[thickness=3]-> () PostgREST
@enduml
+1
View File
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 23 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

+1
View File
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 24 KiB

+4
View File
@@ -149,3 +149,7 @@ html[data-theme="dark"] .img-translucent img {
transition: background-color 0.3s;
margin-bottom: 24px;
}
.svg-container-md {
max-width: 400px;
}
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

+1
View File
@@ -0,0 +1 @@
<?xml version="1.0" encoding="us-ascii" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentStyleType="text/css" height="411px" preserveAspectRatio="none" style="width:315px;height:411px;" version="1.1" viewBox="0 0 315 411" width="315px" zoomAndPan="magnify"><defs/><g><!--cluster PostgreSQL--><g id="cluster_PostgreSQL"><path d="M6,16 C6,6 152,6 152,6 C152,6 298,6 298,16 L298,309.14 C298,319.14 152,319.14 152,319.14 C152,319.14 6,319.14 6,309.14 L6,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><path d="M6,16 C6,26 152,26 152,26 C152,26 298,26 298,16 " fill="none" style="stroke:#E7E7E7;stroke-width:1.0;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="81.375" x="111.3125" y="42.9659">PostgreSQL</text></g><!--cluster public--><g id="cluster_public"><polygon fill="none" points="30,77,40,67,150,67,150,155.07,140,165.07,30,165.07,30,77" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="140" x2="150" y1="77" y2="67"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="30" x2="140" y1="77" y2="77"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="140" x2="140" y1="77" y2="165.07"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="44.625" x="63.6875" y="94.9659">public</text></g><!--cluster API--><g id="cluster_API"><polygon fill="none" points="68,199.07,78,189.07,236,189.07,236,285.14,226,295.14,68,295.14,68,199.07" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="226" x2="236" y1="199.07" y2="189.07"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="68" x2="226" y1="199.07" y2="199.07"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="226" x2="226" y1="199.07" y2="295.14"/><text fill="#FFFFFF" font-family="sans-serif" font-size="20" font-weight="bold" lengthAdjust="spacing" textLength="29.875" x="133.0625" y="223.45">api</text></g><!--entity tables_public--><g id="elem_tables_public"><rect fill="#313139" height="39.0679" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="61" x="69.5" y="110"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="41" x="79.5" y="134.9659">tables</text></g><!--entity extensions--><g id="elem_extensions"><polygon fill="none" points="165.81,115,175.81,105,282.185,105,282.185,144.0679,272.185,154.0679,165.81,154.0679,165.81,115" style="stroke:#E7E7E7;stroke-width:1.0;"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="272.185" x2="282.185" y1="115" y2="105"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="165.81" x2="272.185" y1="115" y2="115"/><line style="stroke:#E7E7E7;stroke-width:1.0;" x1="272.185" x2="272.185" y1="115" y2="154.0679"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" font-weight="bold" lengthAdjust="spacing" textLength="76.375" x="180.81" y="139.9659">extensions</text></g><!--entity vf_api--><g id="elem_vf_api"><rect fill="#313139" height="39.0679" rx="2.5" ry="2.5" style="stroke:#E7E7E7;stroke-width:0.5;" width="136" x="84" y="240.07"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="116" x="94" y="265.0359">views + functions</text></g><!--entity PostgREST--><g id="elem_PostgREST"><ellipse cx="152" cy="371.14" fill="#313139" rx="8" ry="8" style="stroke:#E7E7E7;stroke-width:0.5;"/><text fill="#FFFFFF" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="70" x="117" y="403.1059">PostgREST</text></g><!--reverse link tables_public to vf_api--><g id="link_tables_public_vf_api"><path d="M107.25,155.29 C107.25,179.74 107.25,215.13 107.25,239.67 " fill="none" id="tables_public-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="107.25,149.29,103.25,158.29,107.25,154.29,111.25,158.29,107.25,149.29" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--reverse link extensions to vf_api--><g id="link_extensions_vf_api"><path d="M192.91,160.31 C192.91,185.07 192.91,216.68 192.91,239.61 " fill="none" id="extensions-backto-vf_api" style="stroke:#E7E7E7;stroke-width:1.0;"/><polygon fill="#E7E7E7" points="192.91,154.31,188.91,163.31,192.91,159.31,196.91,163.31,192.91,154.31" style="stroke:#E7E7E7;stroke-width:1.0;"/></g><!--link vf_api to PostgREST--><g id="link_vf_api_PostgREST"><path d="M152,285.55 C152,309.98 152,338.34 152,356.18 " fill="none" id="vf_api-PostgREST" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="152,279.55,148,288.55,152,284.55,156,288.55,152,279.55" style="stroke:#E7E7E7;stroke-width:3.0;"/><polygon fill="#E7E7E7" points="152,362.18,156,353.18,152,357.18,148,353.18,152,362.18" style="stroke:#E7E7E7;stroke-width:3.0;"/></g><!--SRC=[KypCIyufJKbLqDFJBqxEqCqipjShpSq10000]--></g></svg>

After

Width:  |  Height:  |  Size: 4.8 KiB

+1
View File
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 5.1 KiB

+1 -4
View File
@@ -50,7 +50,7 @@ source_suffix = ".rst"
master_doc = "index"
# This is overriden by readthedocs with the version tag anyway
version = "devel"
version = "12.2"
# To avoid repetition in <title> we set this to an empty string.
release = ""
@@ -295,10 +295,7 @@ def setup(app):
# taken from https://github.com/sphinx-doc/sphinx/blob/82dad44e5bd3776ecb6fd8ded656bc8151d0e63d/sphinx/util/requests.py#L42
user_agent = "Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0"
# TODO: these pages are returning 403 right now. Verify if they can be fixed by addressing the issue:
# https://github.com/PostgREST/postgrest/issues/3544
linkcheck_ignore = [
r"https://stackoverflow.com/",
r"https://www.patreon.com/postgrest",
r"https://blog.frankel.ch/poor-man-api",
]
+1
View File
@@ -62,6 +62,7 @@ External Notification
These are PostgreSQL bridges that propagate LISTEN/NOTIFY to external queues for further processing. This allows functions to initiate actions outside the database such as sending emails.
* `pg-notify-stdout <https://github.com/mkleczek/pg-notify-stdout>`_ - writes notifications to standard output (use in shell scripts etc.)
* `pg-notify-webhook <https://github.com/vbalasu/pg-notify-webhook>`_ - trigger webhooks from PostgreSQL's LISTEN/NOTIFY
* `pgsql-listen-exchange <https://github.com/gmr/pgsql-listen-exchange>`_ - RabbitMQ
* `postgres-websockets <https://github.com/diogob/postgres-websockets>`_ - expose web sockets for PostgreSQL's LISTEN/NOTIFY
+17 -3
View File
@@ -6,7 +6,22 @@ This page describes the architecture of PostgREST.
Bird's Eye View
===============
.. image:: ../_static/arch.png
You can click on the components to navigate to their respective documentation.
.. container:: img-dark
.. See https://github.com/sphinx-doc/sphinx/issues/2240#issuecomment-187366626
.. raw:: html
<object width="100%" data="../_static/arch-dark.svg" type="image/svg+xml"></object>
.. container:: img-light
.. raw:: html
<object width="100%" data="../_static/arch.svg" type="image/svg+xml"></object>
Code Map
========
@@ -77,5 +92,4 @@ The HTTP server is provided by `Warp <https://aosabook.org/en/posa/warp.html>`_.
Listener
--------
`Listener.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Listener.hs>`_ is in charge of maintaining a `LISTEN session <https://www.postgresql.org/docs/current/sql-listen.html>`_
that keeps the :ref:`schema_cache` and the :ref:`in_db_config` up to date.
`Listener.hs <https://github.com/PostgREST/postgrest/blob/main/src/PostgREST/Listener.hs>`_ is in charge of the :ref:`listener`.
+1 -1
View File
@@ -190,7 +190,7 @@ When a pre-built binary does not exist for your system you can build the project
You can build PostgREST from source with `Stack <https://github.com/commercialhaskell/stack>`_. It will install any necessary Haskell dependencies on your system.
* `Install Stack <https://docs.haskellstack.org/en/stable/README/#how-to-install-stack>`_ for your platform
* `Install Stack <https://docs.haskellstack.org/en/stable/#how-to-install-stack>`_ for your platform
* Install Library Dependencies
===================== =======================================
+14 -6
View File
@@ -1,7 +1,3 @@
.. note::
This page is a work in progress.
.. _schema_isolation:
Schema Isolation
@@ -12,6 +8,18 @@ A PostgREST instance exposes all the tables, views, and functions of a single `P
It is recommended that you don't expose tables on your API schema. Instead expose views and functions which insulate the internal details from the outside world.
This allows you to change the internals of your schema and maintain backwards compatibility. It also keeps your code easier to refactor, and provides a natural way to do API versioning.
.. container:: img-translucent
.. container:: svg-container-md
.. image:: ../_static/db.png
.. container:: img-dark
.. See https://github.com/sphinx-doc/sphinx/issues/2240#issuecomment-187366626
.. raw:: html
<object width="100%" data="../_static/sch-iso-dark.svg" type="image/svg+xml"></object>
.. container:: img-light
.. raw:: html
<object width="100%" data="../_static/sch-iso.svg" type="image/svg+xml"></object>
+2 -8
View File
@@ -16,15 +16,9 @@ PostgREST Documentation
.. image:: https://img.shields.io/docker/pulls/postgrest/postgrest.svg
:target: https://hub.docker.com/r/postgrest/postgrest/
.. image:: https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg
:target: https://gitter.im/begriffs/postgrest
.. image:: https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854
:target: https://www.patreon.com/postgrest
.. image:: https://img.shields.io/badge/Donate-PayPal-green.svg
:target: https://www.paypal.com/paypalme/postgrest
|
PostgREST is a standalone web server that turns your PostgreSQL database directly into a RESTful API. The structural constraints and permissions in the database determine the API endpoints and operations.
@@ -113,7 +107,7 @@ PostgREST has a focused scope. It works well with other tools like Nginx. This f
Getting Support
----------------
The project has a friendly and growing community. For discussions, use the Github `discussions page <https://github.com/PostgREST/postgrest/discussions>`_ or join our `chat room <https://gitter.im/begriffs/postgrest>`_. You can also report or search for bugs/features on the Github `issues <https://github.com/PostgREST/postgrest/issues>`_ page.
The project has a friendly and growing community. For discussions, use the Github `discussions page <https://github.com/PostgREST/postgrest/discussions>`_. You can also report or search for bugs/features on the Github `issues <https://github.com/PostgREST/postgrest/issues>`_ page.
Release Notes
-------------
@@ -218,7 +212,7 @@ Here are some companies that use PostgREST in production.
* `Datrium <https://www.datrium.com>`_
* `Drip Depot <https://www.dripdepot.com>`_
* `Image-charts <https://www.image-charts.com>`_
* `Moat <https://www.oracle.com/advertising/measurement/>`_
* `Moat <https://www.oracle.com/advertising/>`_
* `Netwo <https://www.netwo.io>`_
* `Nimbus <https://www.nimbusfacility.com/sg/home>`_
- See how Nimbus uses PostgREST in `Paul Copplestone's blog post <https://paul.copplest.one/blog/nimbus-tech-2019-04.html>`_.
+1 -1
View File
@@ -3,7 +3,7 @@
Functions as RPC
================
*"A single resource can be the equivalent of a database function, with the power to abstract state changes over any number of storage items"* -- `Roy T. Fielding <http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven#comment-743>`_
*"A single resource can be the equivalent of a database function, with the power to abstract state changes over any number of storage items"* -- `Roy T. Fielding <https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven#comment-743>`_
Functions can perform any operation allowed by PostgreSQL (read data, modify data, :ref:`raise errors <raise_error>`, and even DDL operations). Every function in the :ref:`exposed schema <schemas>` and accessible by the :ref:`active database role <roles>` is executable under the :code:`/rpc` prefix.
+2 -3
View File
@@ -135,14 +135,12 @@ To reload the configuration via signal, send a SIGUSR2 signal to the server proc
Configuration Reload with NOTIFY
--------------------------------
To reload the configuration from within the database, you can use a NOTIFY command.
To reload the configuration from within the database, you can use the ``NOTIFY`` command. See :ref:`listener`.
.. code:: postgresql
NOTIFY pgrst, 'reload config'
The ``"pgrst"`` notification channel is enabled by default. You can name the channel with :ref:`db-channel` and enable or disable it with :ref:`db-channel-enabled`.
.. _config_full_list:
List of parameters
@@ -696,6 +694,7 @@ log-level
log-level = "info"
# All the above plus events for development purposes are logged
# Logs connection pool events and the schema cache parsing time
log-level = "debug"
Because currently there's no buffering for logging, the levels with minimal logging(``crit/error``) will increase throughput.
+60
View File
@@ -0,0 +1,60 @@
.. _listener:
Listener
########
PostgREST uses `LISTEN <https://www.postgresql.org/docs/current/sql-listen.html>`_ to reload its :ref:`Schema Cache <schema_reloading_notify>` and :ref:`Configuration <config_reloading_notify>` via `NOTIFY <https://www.postgresql.org/docs/current/sql-notify.html>`_.
This is useful in environments where you cant send SIGUSR1 or SIGUSR2 Unix Signals.
Like on cloud managed containers or on Windows systems.
.. code:: postgresql
NOTIFY pgrst, 'reload schema'; -- reload schema cache
NOTIFY pgrst, 'reload config'; -- reload config
NOTIFY pgrst; -- reload both
By default, the LISTEN channel is enabled (:ref:`db-channel-enabled`) and named ``pgrst`` (:ref:`db-channel`).
Listener on Read Replicas
=========================
The ``LISTEN`` and ``NOTIFY`` commands do not work on PostgreSQL read replicas.
Thus, if you connect PostgREST to a read replica the Listener will fail to start.
.. code:: psql
-- check if the instance is a replica
postgres=# select pg_is_in_recovery();
pg_is_in_recovery
-------------------
t
(1 row)
postgres=# LISTEN pgrst;
ERROR: cannot execute LISTEN during recovery
To work around this, you can connect the Listener to the primary while still using the :ref:`connection_pool` on the replica.
This can be done by using the standard `libpq multiple hosts <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-MULTIPLE-HOSTS>`_ and `target_session_attrs <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS>`_ in your :ref:`connection string <db-uri>`.
.. code:: bash
db-uri = "postgres://read_replica.host,primary.host/mydb?target_session_attrs=read-only"
This will cause the :ref:`connection_pool` to connect to the read replica host and ``LISTEN`` on the fallback primary host.
.. note::
Under the hood, PostgREST forces `target_session_attrs=read-write <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS>`_ for the ``LISTEN`` session.
.. _listener_automatic_recovery:
Automatic Recovery
==================
The listener will retry reconnecting to the database if connection loss happens.
- It will retry forever with exponential backoff, with a maximum backoff time of 32 seconds between retries. Each of these attempts are :ref:`logged <pgrst_logging>`.
- Automatic recovery can be disabled by setting :ref:`db-pool-automatic-recovery` to ``false``.
- To ensure a valid state, the listener reloads the :ref:`schema_cache` and :ref:`configuration` when recovering.
+11 -1
View File
@@ -22,7 +22,11 @@ PostgREST logs basic request information to ``stdout``, including the authentica
127.0.0.1 - user [26/Jul/2021:01:56:38 -0500] "GET /clients HTTP/1.1" 200 - "" "curl/7.64.0"
127.0.0.1 - anonymous [26/Jul/2021:01:56:48 -0500] "GET /unexistent HTTP/1.1" 404 - "" "curl/7.64.0"
For diagnostic information about the server itself, PostgREST logs to ``stderr``. It includes the server version and also the version of the connected PostgreSQL.
For diagnostic information about the server itself, PostgREST logs to ``stderr``:
- The full version of the connected PostgreSQL database.
- :ref:`schema_cache` statistics.
- The messages received by the :ref:`listener`.
.. code::
@@ -30,6 +34,12 @@ For diagnostic information about the server itself, PostgREST logs to ``stderr``
06/May/2024:08:16:11 -0500: Attempting to connect to the database...
06/May/2024:08:16:11 -0500: Successfully connected to PostgreSQL 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit
06/May/2024:08:16:11 -0500: Listening on port 3000
06/May/2024:08:16:11 -0500: Listening for notifications on the "pgrst" channel
06/May/2024:08:16:11 -0500: Config reloaded
06/May/2024:08:16:11 -0500: Schema cache queried in 3.8 milliseconds
06/May/2024:08:16:11 -0500: Schema cache loaded 15 Relations, 8 Relationships, 8 Functions, 0 Domain Representations, 4 Media Type Handlers
06/May/2024:14:11:27 -0500: Received a config reload message on the "pgrst" channel
06/May/2024:14:11:27 -0500: Config reloaded
Database Logs
-------------
+1 -5
View File
@@ -47,16 +47,12 @@ For docker you can do:
Schema Cache Reloading with NOTIFY
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PostgREST also allows you to reload its schema cache through PostgreSQL `NOTIFY <https://www.postgresql.org/docs/current/sql-notify.html>`_.
To reload the schema cache from within the database, you can use the ``NOTIFY`` command. See :ref:`listener`.
.. code-block:: postgres
NOTIFY pgrst, 'reload schema'
This is useful in environments where you cant send the SIGUSR1 Unix Signal. Like on cloud managed containers or on Windows systems.
The ``pgrst`` notification channel is enabled by default. For configuring the channel, see :ref:`db-channel` and :ref:`db-channel-enabled`.
.. _auto_schema_reloading:
Automatic Schema Cache Reloading
+1 -1
View File
@@ -6,4 +6,4 @@ sphinx-rtd-dark-mode>=1.3.0
sphinx-tabs>=3.2.0
sphinx>=5.0.2
sphinxext-opengraph==0.9.1
urllib3==2.2.1
urllib3==2.2.2
+4 -9
View File
@@ -17,12 +17,7 @@ To make an API we'll simply be building a database. All the endpoints and permis
By the end of this tutorial you'll have a working database, PostgREST server, and a simple single-user todo list API.
Step 1. Relax, we'll help
-------------------------
As you begin the tutorial, pop open the project `chat room <https://gitter.im/begriffs/postgrest>`_ in another tab. There are a nice group of people active in the project and we'll help you out if you get stuck.
Step 2. Install PostgreSQL
Step 1. Install PostgreSQL
--------------------------
If you're already familiar with using PostgreSQL and have it installed on your system you can use the existing installation (see :ref:`pg-dependency` for minimum requirements). For this tutorial we'll describe how to use the database in Docker because database configuration is otherwise too complicated for a simple tutorial.
@@ -48,7 +43,7 @@ This will run the Docker instance as a daemon and expose port 5432 to the host s
In this case, you will need to change the **first** of the two 5432 to something else, for example to :code:`5433:5432`. Remember to also adjust the port in your config file in Step 5!
Step 3. Install PostgREST
Step 2. Install PostgREST
-------------------------
Using a Package Manager
@@ -122,7 +117,7 @@ If everything is working correctly it will print out its version and the availab
</details>
</p>
Step 4. Create Database for API
Step 3. Create Database for API
-------------------------------
Connect to the SQL console (psql) inside the container. To do so, run this from your command line:
@@ -185,7 +180,7 @@ Now quit out of psql; it's time to start the API!
\q
Step 5. Run PostgREST
Step 4. Run PostgREST
---------------------
PostgREST can use a configuration file to tell it how to connect to the database. Create a file :code:`tutorial.conf` with this inside:
+2 -25
View File
@@ -2,10 +2,8 @@
, aspellDicts
, buildToolbox
, checkedShellScript
, imagemagick
, python3
, python3Packages
, texlive
, writers
, plantuml
}:
@@ -61,36 +59,15 @@ let
'';
render =
let
pdflatex = texlive.combine {
inherit (texlive)
amsmath
booktabs
cancel
gensymb
mathdots
multirow
pgf
pgf-blur
scheme-basic
siunitx
standalone
yhmath
;
};
in
checkedShellScript
{
name = "postgrest-docs-render";
docs = "Render the diagrams.";
workingDir = "/docs/_diagrams";
withTmpDir = true;
}
''
${pdflatex}/bin/pdflatex -halt-on-error -output-directory="$tmpdir" db.tex
${imagemagick}/bin/convert -density 300 "$tmpdir/db.pdf" ../_static/db.png
${plantuml}/bin/plantuml arch.uml -o ../_static
${plantuml}/bin/plantuml -tsvg uml/*.uml -o ../../_static
${plantuml}/bin/plantuml -tsvg -darkmode uml/dark/*.uml -o ../../../_static
'';
server =
+1 -1
View File
@@ -1,5 +1,5 @@
name: postgrest
version: 12.2.0
version: 12.2.3
synopsis: REST API for any Postgres database
description: Reads the schema of a PostgreSQL database and creates RESTful routes
for tables, views, and functions, supporting all HTTP methods that security
+3 -3
View File
@@ -68,14 +68,14 @@ run appState = do
observer $ AppStartObs prettyVersion
AppState.connectionWorker appState
Unix.installSignalHandlers (AppState.getMainThreadId appState) (AppState.connectionWorker appState) (AppState.reReadConfig False appState)
AppState.schemaCacheLoader appState -- Loads the initial SchemaCache
Unix.installSignalHandlers (AppState.getMainThreadId appState) (AppState.schemaCacheLoader appState) (AppState.readInDbConfig False appState)
Listener.runListener appState
Admin.runAdmin appState (serverSettings conf)
let app = postgrest configLogLevel appState (AppState.connectionWorker appState)
let app = postgrest configLogLevel appState (AppState.schemaCacheLoader appState)
case configServerUnixSocket of
Just path -> do
+76 -163
View File
@@ -24,12 +24,11 @@ module PostgREST.AppState
, putPgVersion
, putIsListenerOn
, usePool
, reReadConfig
, connectionWorker
, readInDbConfig
, schemaCacheLoader
, getObserver
, isLoaded
, isPending
, waitForListenerCanStart
) where
import qualified Data.Aeson as JSON
@@ -87,20 +86,16 @@ data AuthResult = AuthResult
data AppState = AppState
-- | Database connection pool
{ statePool :: SQL.Pool
-- | Database server version, will be updated by the connectionWorker
-- | Database server version
, statePgVersion :: IORef PgVersion
-- | No schema cache at the start. Will be filled in by the connectionWorker
-- | Schema cache
, stateSchemaCache :: IORef (Maybe SchemaCache)
-- | The schema cache status
, stateSCacheStatus :: IORef SchemaCacheStatus
-- | The connection status
, stateConnStatus :: IORef ConnectionStatus
-- | State of the LISTEN channel
, stateIsListenerOn :: IORef Bool
-- | starts the connection worker with a debounce
, debouncedConnectionWorker :: IO ()
-- | Binary semaphore used to sync the listener with the connectionWorker.
, stateListenerCanStart :: MVar ()
, debouncedSCacheLoader :: IO ()
-- | Config that can change at runtime
, stateConf :: IORef AppConfig
-- | Time used for verifying JWT expiration
@@ -129,15 +124,8 @@ data SchemaCacheStatus
| SCPending
deriving Eq
-- | Current database connection status
data ConnectionStatus
= ConnEstablished
| ConnPending
deriving Eq
type AppSockets = (NS.Socket, Maybe NS.Socket)
init :: AppConfig -> IO AppState
init conf@AppConfig{configLogLevel, configDbPoolSize} = do
loggerState <- Logger.init
@@ -156,10 +144,8 @@ initWithPool (sock, adminSock) pool conf loggerState metricsState observer = do
<$> newIORef minimumPgVersion -- assume we're in a supported version when starting, this will be corrected on a later step
<*> newIORef Nothing
<*> newIORef SCPending
<*> newIORef ConnPending
<*> newIORef False
<*> pure (pure ())
<*> newEmptyMVar
<*> newIORef conf
<*> mkAutoUpdate defaultUpdateSettings { updateAction = getCurrentTime }
<*> myThreadId
@@ -172,15 +158,15 @@ initWithPool (sock, adminSock) pool conf loggerState metricsState observer = do
<*> pure loggerState
<*> pure metricsState
debWorker <-
deb <-
let decisecond = 100000 in
mkDebounce defaultDebounceSettings
{ debounceAction = internalConnectionWorker appState
{ debounceAction = retryingSchemaCacheLoad appState
, debounceFreq = decisecond
, debounceEdge = leadingEdge -- runs the worker at the start and the end
}
return appState { debouncedConnectionWorker = debWorker}
return appState { debouncedSCacheLoader = deb}
destroy :: AppState -> IO ()
destroy = destroyPool
@@ -306,41 +292,18 @@ getSchemaCache = readIORef . stateSchemaCache
putSchemaCache :: AppState -> Maybe SchemaCache -> IO ()
putSchemaCache appState = atomicWriteIORef (stateSchemaCache appState)
connectionWorker :: AppState -> IO ()
connectionWorker = debouncedConnectionWorker
schemaCacheLoader :: AppState -> IO ()
schemaCacheLoader = debouncedSCacheLoader
getNextDelay :: AppState -> IO Int
getNextDelay = readIORef . stateNextDelay
putNextDelay :: AppState -> Int -> IO ()
putNextDelay = atomicWriteIORef . stateNextDelay
getNextListenerDelay :: AppState -> IO Int
getNextListenerDelay = readIORef . stateNextListenerDelay
putNextListenerDelay :: AppState -> Int -> IO ()
putNextListenerDelay = atomicWriteIORef . stateNextListenerDelay
--------------------------------------------------------------------------------------
-------------------------------------------IMPORTANT----------------------------------
--------------------------------------------------------------------------------------
-- Both of these function ensure there's no parallel connection attempts between the listener and the connection pool.
-- Doing that raised an error with GSSAPI as discussed on https://github.com/PostgREST/postgrest/issues/3569.
-- Until the root cause is found and solved, we need to prevent parallel connection attempts.
-- tryPutMVar doesn't lock the thread. It should always succeed since
-- the connectionWorker is the only mvar producer.
setListenerCanStart :: AppState -> IO ()
setListenerCanStart appState = void $ tryPutMVar (stateListenerCanStart appState) ()
-- | As this IO action uses `takeMVar` internally, it will only return once
-- `stateListenerCanStart` has been set using `setListenerCanStart`.
waitForListenerCanStart :: AppState -> IO ()
waitForListenerCanStart = takeMVar . stateListenerCanStart
--------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------
getConfig :: AppState -> IO AppConfig
getConfig = readIORef . stateConf
@@ -362,53 +325,81 @@ getSocketAdmin = stateSocketAdmin
getMainThreadId :: AppState -> ThreadId
getMainThreadId = stateMainThreadId
getIsListenerOn :: AppState -> IO Bool
getIsListenerOn appState = do
isConnEstablished :: AppState -> IO Bool
isConnEstablished appState = do
AppConfig{..} <- getConfig appState
if configDbChannelEnabled then
if configDbChannelEnabled then -- if the listener is enabled, we can be sure the connection is up
readIORef $ stateIsListenerOn appState
else
pure True
else -- otherwise the only way to check the connection is to make a query
isRight <$> usePool appState (SQL.sql "SELECT 1")
putIsListenerOn :: AppState -> Bool -> IO ()
putIsListenerOn = atomicWriteIORef . stateIsListenerOn
isConnEstablished :: AppState -> IO Bool
isConnEstablished x = do
conf <- getConfig x
if configDbChannelEnabled conf
then do -- if the listener is enabled, we can be sure the connection status is always up to date
st <- readIORef $ stateConnStatus x
return $ st == ConnEstablished
else -- otherwise the only way to check the connection is to make a query
isRight <$> usePool x (SQL.sql "SELECT 1")
isLoaded :: AppState -> IO Bool
isLoaded x = do
scacheStatus <- readIORef $ stateSCacheStatus x
connEstablished <- isConnEstablished x
listenerOn <- getIsListenerOn x
return $ scacheStatus == SCLoaded && connEstablished && listenerOn
return $ scacheStatus == SCLoaded && connEstablished
isPending :: AppState -> IO Bool
isPending x = do
scacheStatus <- readIORef $ stateSCacheStatus x
connStatus <- readIORef $ stateConnStatus x
listenerOn <- getIsListenerOn x
return $ scacheStatus == SCPending || connStatus == ConnPending || not listenerOn
connEstablished <- isConnEstablished x
return $ scacheStatus == SCPending || not connEstablished
putSCacheStatus :: AppState -> SchemaCacheStatus -> IO ()
putSCacheStatus = atomicWriteIORef . stateSCacheStatus
putConnStatus :: AppState -> ConnectionStatus -> IO ()
putConnStatus = atomicWriteIORef . stateConnStatus
getObserver :: AppState -> ObservationHandler
getObserver = stateObserver
-- | Load the SchemaCache by using a connection from the pool.
loadSchemaCache :: AppState -> IO SchemaCacheStatus
loadSchemaCache appState@AppState{stateObserver=observer} = do
-- | Try to load the schema cache and retry if it fails.
--
-- This is done by repeatedly: 1) flushing the pool, 2) querying the version and validating that the postgres version is supported by us, and 3) loading the schema cache.
-- It's necessary to flush the pool:
--
-- + Because connections cache the pg catalog(see #2620)
-- + For rapid recovery. Otherwise, the pool idle or lifetime timeout would have to be reached for new healthy connections to be acquired.
retryingSchemaCacheLoad :: AppState -> IO ()
retryingSchemaCacheLoad appState@AppState{stateObserver=observer, stateMainThreadId=mainThreadId} =
void $ retrying retryPolicy shouldRetry (\RetryStatus{rsIterNumber, rsPreviousDelay} -> do
when (rsIterNumber > 0) $ do
let delay = fromMaybe 0 rsPreviousDelay `div` oneSecondInUs
observer $ ConnectionRetryObs delay
putNextListenerDelay appState delay
flushPool appState
(,) <$> qPgVersion <*> (qInDbConfig *> qSchemaCache)
)
where
qPgVersion :: IO (Maybe PgVersion)
qPgVersion = do
AppConfig{..} <- getConfig appState
pgVersion <- usePool appState (queryPgVersion False) -- No need to prepare the query here, as the connection might not be established
case pgVersion of
Left e -> do
observer $ QueryPgVersionError e
unless configDbPoolAutomaticRecovery $ do
observer ExitDBNoRecoveryObs
killThread mainThreadId
return Nothing
Right actualPgVersion -> do
when (actualPgVersion < minimumPgVersion) $ do
observer $ ExitUnsupportedPgVersion actualPgVersion minimumPgVersion
killThread mainThreadId
observer $ DBConnectedObs $ pgvFullName actualPgVersion
putPgVersion appState actualPgVersion
return $ Just actualPgVersion
qInDbConfig :: IO ()
qInDbConfig = do
AppConfig{..} <- getConfig appState
when configDbConfig $ readInDbConfig False appState
qSchemaCache :: IO (Maybe SchemaCache)
qSchemaCache = do
conf@AppConfig{..} <- getConfig appState
(resultTime, result) <-
let transaction = if configDbPreparedStatements then SQL.transaction else SQL.unpreparedTransaction in
@@ -418,7 +409,7 @@ loadSchemaCache appState@AppState{stateObserver=observer} = do
putSCacheStatus appState SCPending
putSchemaCache appState Nothing
observer $ SchemaCacheErrorObs e
return SCPending
return Nothing
Right sCache -> do
-- IMPORTANT: While the pending schema cache state starts from running the above querySchemaCache, only at this stage we block API requests due to the usage of an
@@ -429,103 +420,25 @@ loadSchemaCache appState@AppState{stateObserver=observer} = do
(t, _) <- timeItT $ observer $ SchemaCacheSummaryObs $ showSummary sCache
observer $ SchemaCacheLoadedObs t
putSCacheStatus appState SCLoaded
return SCLoaded
return $ Just sCache
-- | The purpose of this worker is to obtain a healthy connection to pg and an
-- up-to-date schema cache(SchemaCache). This method is meant to be called
-- multiple times by the same thread, but does nothing if the previous
-- invocation has not terminated. In all cases this method does not halt the
-- calling thread, the work is performed in a separate thread.
--
-- Background thread that does the following :
-- 1. Tries to connect to pg server and will keep trying until success.
-- 2. Checks if the pg version is supported and if it's not it kills the main
-- program.
-- 3. Obtains the sCache. If this fails, it goes back to 1.
internalConnectionWorker :: AppState -> IO ()
internalConnectionWorker appState@AppState{stateObserver=observer, stateMainThreadId=mainThreadId} = work
where
work = do
shouldRetry :: RetryStatus -> (Maybe PgVersion, Maybe SchemaCache) -> IO Bool
shouldRetry _ (pgVer, sCache) = do
AppConfig{..} <- getConfig appState
observer DBConnectAttemptObs
connStatus <- establishConnection appState
case connStatus of
ConnPending ->
unless configDbPoolAutomaticRecovery $ do
observer ExitDBNoRecoveryObs
killThread mainThreadId
ConnEstablished -> do
actualPgVersion <- getPgVersion appState
when (actualPgVersion < minimumPgVersion) $ do
observer $ ExitUnsupportedPgVersion actualPgVersion minimumPgVersion
killThread mainThreadId
observer (DBConnectedObs $ pgvFullName actualPgVersion)
-- Wake up the Listener
when configDbChannelEnabled $
setListenerCanStart appState
-- this could be fail because the connection drops, but the loadSchemaCache will pick the error and retry again
-- We cannot retry after it fails immediately, because db-pre-config could have user errors. We just log the error and continue.
when configDbConfig $ reReadConfig False appState
scStatus <- loadSchemaCache appState
case scStatus of
SCLoaded ->
-- do nothing and proceed if the load was successful
return ()
SCPending ->
-- retry reloading the schema cache
work
-- | Repeatedly flush the pool, and check if a connection from the
-- pool allows access to the PostgreSQL database.
--
-- Releasing the pool is key for rapid recovery. Otherwise, the pool
-- timeout would have to be reached for new healthy connections to be acquired.
-- Which might not happen if the server is busy with requests. No idle
-- connection, no pool timeout.
--
-- It's also necessary to release the pool connections because they cache the pg catalog(see #2620)
--
-- The connection tries are capped, but if the connection times out no error is
-- thrown, just 'False' is returned.
establishConnection :: AppState -> IO ConnectionStatus
establishConnection appState@AppState{stateObserver=observer} =
retrying retryPolicy shouldRetry $
const $ flushPool appState >> getConnectionStatus
where
getConnectionStatus :: IO ConnectionStatus
getConnectionStatus = do
pgVersion <- usePool appState (queryPgVersion False) -- No need to prepare the query here, as the connection might not be established
case pgVersion of
Left e -> do
observer $ ConnectionPgVersionErrorObs e
putConnStatus appState ConnPending
return ConnPending
Right version -> do
putConnStatus appState ConnEstablished
putPgVersion appState version
return ConnEstablished
shouldRetry :: RetryStatus -> ConnectionStatus -> IO Bool
shouldRetry rs isConnSucc = do
AppConfig{..} <- getConfig appState
let
delay = fromMaybe 0 (rsPreviousDelay rs) `div` oneSecondInUs
itShould = ConnPending == isConnSucc && configDbPoolAutomaticRecovery
when itShould $ observer $ ConnectionRetryObs delay
when itShould $ putNextDelay appState delay
let itShould = configDbPoolAutomaticRecovery && (isNothing pgVer || isNothing sCache)
return itShould
retryPolicy :: RetryPolicy
retryPolicy =
let
delayMicroseconds = 32000000 -- 32 seconds
in
let delayMicroseconds = 32*oneSecondInUs {-32 seconds-} in
capDelay delayMicroseconds $ exponentialBackoff oneSecondInUs
oneSecondInUs = 1000000 -- | One second in microseconds
-- | Re-reads the config plus config options from the db
reReadConfig :: Bool -> AppState -> IO ()
reReadConfig startingUp appState@AppState{stateObserver=observer} = do
oneSecondInUs = 1000000 -- one second in microseconds
-- | Reads the in-db config and reads the config file again
-- | We don't retry reading the in-db config after it fails immediately, because it could have user errors. We just report the error and continue.
readInDbConfig :: Bool -> AppState -> IO ()
readInDbConfig startingUp appState@AppState{stateObserver=observer} = do
AppConfig{..} <- getConfig appState
pgVer <- getPgVersion appState
dbSettings <-
+4 -2
View File
@@ -42,9 +42,11 @@ main CLI{cliCommand, cliPath} = do
AppState.destroy
(\appState -> case cliCommand of
CmdDumpConfig -> do
when configDbConfig $ AppState.reReadConfig True appState
when configDbConfig $ AppState.readInDbConfig True appState
putStr . Config.toText =<< AppState.getConfig appState
CmdDumpSchema -> putStrLn =<< dumpSchema appState
CmdDumpSchema -> do
when configDbConfig $ AppState.readInDbConfig True appState
putStrLn =<< dumpSchema appState
CmdRun -> App.run appState)
-- | Dump SchemaCache schema to JSON
+4 -8
View File
@@ -27,16 +27,10 @@ runListener appState = do
-- | Starts a LISTEN connection and handles notifications. It recovers with exponential backoff with a cap of 32 seconds, if the LISTEN connection is lost.
retryingListen :: AppState -> IO ()
retryingListen appState = do
AppState.waitForListenerCanStart appState
AppConfig{..} <- AppState.getConfig appState
let
dbChannel = toS configDbChannel
handleFinally err = do
-- assume we lost notifications, call the connection worker which will also reload the schema cache
-- and will setListenerCanStart again
-- TODO: When the connection error is only on the Listener, it's wasteful to call the connectionWorker everytime.
AppState.connectionWorker appState
AppState.putIsListenerOn appState False
observer $ DBListenFail dbChannel (Right err)
unless configDbPoolAutomaticRecovery $
@@ -60,6 +54,8 @@ retryingListen appState = do
delay <- AppState.getNextListenerDelay appState
when (delay > 1) $ do -- if we did a retry
-- assume we lost notifications, refresh the schema cache
AppState.schemaCacheLoader appState
-- reset the delay
AppState.putNextListenerDelay appState 1
@@ -78,8 +74,8 @@ retryingListen appState = do
handleNotification channel msg =
if | BS.null msg -> observer (DBListenerGotSCacheMsg channel) >> cacheReloader
| msg == "reload schema" -> observer (DBListenerGotSCacheMsg channel) >> cacheReloader
| msg == "reload config" -> observer (DBListenerGotConfigMsg channel) >> AppState.reReadConfig False appState
| msg == "reload config" -> observer (DBListenerGotConfigMsg channel) >> AppState.readInDbConfig False appState
| otherwise -> pure () -- Do nothing if anything else than an empty message is sent
cacheReloader =
AppState.connectionWorker appState
AppState.schemaCacheLoader appState
+4 -7
View File
@@ -29,7 +29,6 @@ data Observation
| AppStartObs ByteString
| AppServerPortObs NS.PortNumber
| AppServerUnixObs FilePath
| DBConnectAttemptObs
| ExitUnsupportedPgVersion PgVersion PgVersion
| ExitDBNoRecoveryObs
| ExitDBFatalError ObsFatalError SQL.UsageError
@@ -39,7 +38,6 @@ data Observation
| SchemaCacheSummaryObs Text
| SchemaCacheLoadedObs Double
| ConnectionRetryObs Int
| ConnectionPgVersionErrorObs SQL.UsageError
| DBListenStart Text
| DBListenFail Text (Either SQL.ConnectionError (Either SomeException ()))
| DBListenRetry Int
@@ -50,6 +48,7 @@ data Observation
| ConfigSucceededObs
| QueryRoleSettingsErrorObs SQL.UsageError
| QueryErrorCodeHighObs SQL.UsageError
| QueryPgVersionError SQL.UsageError
| PoolAcqTimeoutObs SQL.UsageError
| HasqlPoolObs SQL.Observation
| PoolRequest
@@ -69,8 +68,6 @@ observationMessage = \case
"Listening on port " <> show port
AppServerUnixObs sock ->
"Listening on unix socket " <> show sock
DBConnectAttemptObs ->
"Attempting to connect to the database..."
DBConnectedObs ver ->
"Successfully connected to " <> ver
ExitUnsupportedPgVersion pgVer minPgVer ->
@@ -78,7 +75,7 @@ observationMessage = \case
ExitDBNoRecoveryObs ->
"Automatic recovery disabled, exiting."
ExitDBFatalError ServerAuthError usageErr ->
jsonMessage usageErr
"Failed to establish a connection. " <> jsonMessage usageErr
ExitDBFatalError ServerPgrstBug usageErr ->
"This is probably a bug in PostgREST, please report it at https://github.com/PostgREST/postgrest/issues. " <> jsonMessage usageErr
ExitDBFatalError ServerError42P05 usageErr ->
@@ -95,8 +92,8 @@ observationMessage = \case
"Schema cache loaded in " <> showMillis resultTime <> " milliseconds"
ConnectionRetryObs delay ->
"Attempting to reconnect to the database in " <> (show delay::Text) <> " seconds..."
ConnectionPgVersionErrorObs usageErr ->
jsonMessage usageErr
QueryPgVersionError usageErr ->
"Failed to query the PostgreSQL version. " <> jsonMessage usageErr
DBListenStart channel -> do
"Listening for notifications on the " <> show channel <> " channel"
DBListenFail channel listenErr ->
+5 -1
View File
@@ -622,7 +622,11 @@ generateRelSelectField (Node ReadPlan{relToParent=Just rel, select, relName, rel
where
rsSelName = fromMaybe relName relAlias
rsEmbedMode = if relIsToOne rel then JsonObject else JsonArray
rsEmptyEmbed = null select && null forest
rsEmptyEmbed = hasOnlyNullEmbed (null select) forest
hasOnlyNullEmbed = foldr checkIfNullEmbed
checkIfNullEmbed :: ReadPlanTree -> Bool -> Bool
checkIfNullEmbed (Node ReadPlan{select=s} f) isNullEmbed =
isNullEmbed && hasOnlyNullEmbed (null s) f
generateRelSelectField _ = Nothing
generateSpreadSelectFields :: ReadPlan -> [SpreadSelectField]
+2 -3
View File
@@ -296,8 +296,7 @@ makeRowFilter tn c =
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamQuery
& type_ ?~ SwaggerString
& format ?~ colType c))
& type_ ?~ SwaggerString))
makeRowFilters :: Text -> [Column] -> [(Text, Param)]
makeRowFilters tn = fmap (makeRowFilter tn)
@@ -402,7 +401,7 @@ postgrestSpec (prettyVersion, docsVersion) rels pds ti (s, h, p, b) sd allowSecu
& description ?~ fromMaybe "This is a dynamic API generated by PostgREST" dDesc)
& externalDocs ?~ ((mempty :: ExternalDocs)
& description ?~ "PostgREST Documentation"
& url .~ URL ("https://postgrest.org/en/" <> docsVersion <> "/api.html"))
& url .~ URL ("https://postgrest.org/en/" <> docsVersion <> "/references/api.html"))
& host .~ h'
& definitions .~ fromList (makeTableDef rels <$> ti)
& parameters .~ fromList (makeParamDefs ti)
+7 -7
View File
@@ -95,13 +95,14 @@ instance JSON.ToJSON SchemaCache where
]
showSummary :: SchemaCache -> Text
showSummary (SchemaCache tbls rels routs reps mediaHdlrs _) =
showSummary (SchemaCache tbls rels routs reps mediaHdlrs tzs) =
T.intercalate ", "
[ show (HM.size tbls) <> " Relations"
, show (HM.size rels) <> " Relationships"
, show (HM.size routs) <> " Functions"
, show (HM.size reps) <> " Domain Representations"
, show (HM.size mediaHdlrs) <> " Media Type Handlers"
, show (S.size tzs) <> " Timezones"
]
-- | A view foreign key or primary key dependency detected on its source table
@@ -646,7 +647,7 @@ tablesSqlQuery pgVer =
information_schema._pg_truetypid(a.*, t.*),
information_schema._pg_truetypmod(a.*, t.*)
)::integer AS character_maximum_length,
COALESCE(bt.typname, t.typname)::name AS udt_name,
COALESCE(bt.oid, t.oid) AS base_type,
a.attnum::integer AS position
FROM pg_attribute a
LEFT JOIN pg_description AS d
@@ -690,14 +691,13 @@ tablesSqlQuery pgVer =
FROM columns info
LEFT OUTER JOIN (
SELECT
n.nspname AS s,
t.typname AS n,
e.enumtypid,
array_agg(e.enumlabel ORDER BY e.enumsortorder) AS vals
FROM pg_type t
JOIN pg_enum e ON t.oid = e.enumtypid
JOIN pg_namespace n ON n.oid = t.typnamespace
GROUP BY s,n
) AS enum_info ON info.udt_name = enum_info.n
GROUP BY enumtypid
) AS enum_info ON info.base_type = enum_info.enumtypid
WHERE info.table_schema NOT IN ('pg_catalog', 'information_schema')
GROUP BY info.table_schema, info.table_name
),
@@ -872,7 +872,7 @@ allM2OandO2ORels pgVer =
JOIN LATERAL (
SELECT
array_agg(row(cols.attname, refs.attname) order by ord) AS cols_and_fcols,
jsonb_agg(cols.attname order by ord) AS cols
jsonb_agg(cols.attname order by cols.attnum) AS cols
FROM unnest(traint.conkey, traint.confkey) WITH ORDINALITY AS _(col, ref, ord)
JOIN pg_attribute cols ON cols.attrelid = traint.conrelid AND cols.attnum = col
JOIN pg_attribute refs ON refs.attrelid = traint.confrelid AND refs.attnum = ref
+5 -5
View File
@@ -15,7 +15,7 @@ import Protolude
-- | User friendly version number such as '1.1.1'.
-- Pre-release versions are tagged as such, e.g., '1.1.1.1 (pre-release)'.
-- Pre-release versions are tagged as such, e.g., '1.1 (pre-release)'.
-- If a git hash is available, it's added to the version, e.g., '1.1.1 (abcdef0)'.
prettyVersion :: ByteString
prettyVersion =
@@ -31,14 +31,14 @@ prettyVersion =
-- | Version number used in docs.
-- Pre-release versions link to the latest docs
-- Uses only the two first components of the version. Example: 'v1.1'
-- Uses only the first component of the version. Example: 'v1'
docsVersion :: Text
docsVersion
| isPreRelease = "latest"
| otherwise = "v" <> (T.intercalate "." . map show . take 2 $ versionBranch version)
| otherwise = "v" <> (T.intercalate "." . map show . take 1 $ versionBranch version)
-- | Versions with four components (e.g., '1.1.1.1') are treated as pre-releases.
-- | Versions with two components (e.g., '1.1') are treated as pre-releases.
isPreRelease :: Bool
isPreRelease =
length (versionBranch version) == 4
length (versionBranch version) == 2
@@ -218,6 +218,40 @@
pdSchema: public
pdVolatility: Volatile
- - qiName: reset_db_schemas_config
qiSchema: public
- - pdDescription: null
pdFuncSettings: []
pdHasVariadic: false
pdName: reset_db_schemas_config
pdParams: []
pdReturnType:
contents:
contents:
qiName: void
qiSchema: pg_catalog
tag: Scalar
tag: Single
pdSchema: public
pdVolatility: Volatile
- - qiName: change_db_schemas_config
qiSchema: public
- - pdDescription: null
pdFuncSettings: []
pdHasVariadic: false
pdName: change_db_schemas_config
pdParams: []
pdReturnType:
contents:
contents:
qiName: void
qiSchema: pg_catalog
tag: Scalar
tag: Single
pdSchema: public
pdVolatility: Volatile
- - qiName: change_db_schema_and_full_reload
qiSchema: public
- - pdDescription: null
+17
View File
@@ -33,6 +33,9 @@ GRANT
CREATE SCHEMA v1;
GRANT USAGE ON SCHEMA v1 TO postgrest_test_anonymous;
CREATE SCHEMA test;
GRANT USAGE ON SCHEMA test TO postgrest_test_anonymous;
CREATE TABLE authors_only ();
GRANT SELECT ON authors_only TO postgrest_test_author;
@@ -226,3 +229,17 @@ $$ language sql;
create function get_statement_timeout(items) returns text as $$
select current_setting('statement_timeout', true) as statement_timeout
$$ language sql;
create function change_db_schemas_config() returns void as $_$
begin
alter role postgrest_test_authenticator set pgrst.db_schemas = 'test';
end $_$ volatile security definer language plpgsql;
create function reset_db_schemas_config() returns void as $_$
begin
alter role postgrest_test_authenticator reset pgrst.db_schemas;
end $_$ volatile security definer language plpgsql ;
create function test.get_current_schema() returns text as $$
select current_schema()::text;
$$ language sql;
+18 -1
View File
@@ -778,7 +778,7 @@ def test_metrics_include_schema_cache_fails(defaultenv, metapostgrest):
r'pgrst_schema_cache_loads_total{status="FAIL"} (\d+)', response.text
).group(1)
)
assert metrics > 3.0
assert metrics == 1.0
reset_statement_timeout(metapostgrest, role)
@@ -1579,3 +1579,20 @@ def test_admin_metrics(defaultenv):
assert "pgrst_db_pool_waiting" in response.text
assert "pgrst_db_pool_available" in response.text
assert "pgrst_db_pool_timeouts_total" in response.text
def test_schema_cache_startup_load_with_in_db_config(defaultenv, metapostgrest):
"verify that the Schema Cache loads correctly at startup, using the in-db `pgrst.db_schemas` config"
response = metapostgrest.session.post("/rpc/change_db_schemas_config")
assert response.text == ""
assert response.status_code == 204
with run(env=defaultenv) as postgrest:
response = postgrest.session.get("/rpc/get_current_schema")
assert response.text == '"test"'
assert response.status_code == 200
response = metapostgrest.session.post("/rpc/reset_db_schemas_config")
assert response.text == ""
assert response.status_code == 204
+1 -1
View File
@@ -45,7 +45,7 @@ spec actualPgVersion = describe "OpenAPI" $ do
let docsUrl = r ^? key "externalDocs" . key "url"
liftIO $ docsUrl `shouldBe` Just (String ("https://postgrest.org/en/" <> docsVersion <> "/api.html"))
liftIO $ docsUrl `shouldBe` Just (String ("https://postgrest.org/en/" <> docsVersion <> "/references/api.html"))
describe "schema" $ do
+14 -1
View File
@@ -1218,6 +1218,20 @@ spec actualPgVersion = do
[json|[{"id":1,"name":"Angela Martin"}]|]
{ matchHeaders = [matchContentTypeJson] }
it "works on nested relationships" $ do
get "/users?select=*,users_tasks(tasks(projects()))" `shouldRespondWith`
[json| [{"id":1,"name":"Angela Martin"}, {"id":2,"name":"Michael Scott"}, {"id":3,"name":"Dwight Schrute"}]|]
{ matchHeaders = [matchContentTypeJson] }
get "/users?select=*,users_tasks!inner(tasks!inner(projects()))&users_tasks.tasks.id=eq.3" `shouldRespondWith`
[json| [{"id":1,"name":"Angela Martin"}]|]
{ matchHeaders = [matchContentTypeJson] }
get "/users?select=*,tasks(projects(clients()),users_tasks())" `shouldRespondWith`
[json| [{"id":1,"name":"Angela Martin"}, {"id":2,"name":"Michael Scott"}, {"id":3,"name":"Dwight Schrute"}]|]
{ matchHeaders = [matchContentTypeJson] }
get "/users?select=*,tasks!inner(projects(clients()),users_tasks(),name)&tasks.id=eq.3" `shouldRespondWith`
[json| [{"id":1,"name":"Angela Martin","tasks":[{"name": "Design w10"}]}]|]
{ matchHeaders = [matchContentTypeJson] }
context "empty root select" $
it "gives all columns" $ do
get "/projects?select=" `shouldRespondWith`
@@ -1413,4 +1427,3 @@ spec actualPgVersion = do
get "/infinite_recursion?select=*" `shouldRespondWith`
[json|{"code":"42P17","message":"infinite recursion detected in rules for relation \"infinite_recursion\"","details":null,"hint":null}|]
{ matchStatus = 500 }
+1 -1
View File
@@ -2872,7 +2872,7 @@ CREATE TABLE test.students_info(
, code text
, address text
, primary key(id, code)
, foreign key (id, code) references test.students(id, code) on delete cascade
, foreign key (code, id) references test.students(code, id) on delete cascade
);
CREATE TABLE test.country(