chore(deps): update cachix/install-nix-action digest to 13d8dd5

They're not really a sponsor, so it's not fair to include them.

ARCHITECTURE.md

@@ -55,12 +55,6 @@ It builds the OpenAPI response using the schema cache.
 
 This module provides functions to deal with JWT authorization.
 
-### Workers.hs
-
-This spawns threads which are used to execute concurrent jobs.
-
-Jobs include connection recovery, a listener for the PostgreSQL LISTEN command, and an admin server.
-
 ### SchemaCache.hs
 
 This queries the PostgreSQL system catalogs and caches the metadata into a SchemaCache type,
@@ -68,3 +62,7 @@ This queries the PostgreSQL system catalogs and caches the metadata into a Schem
 ### AppState.hs
 
 The state of the App which is kept across requests.
+
+This spawns threads which are used to execute concurrent jobs.
+
+Jobs include connection recover and a listener for the PostgreSQL LISTEN command.

BACKERS.md

@@ -46,12 +46,14 @@ PostgREST ongoing development is only possible thanks to our Sponsors and Backer
 
 ## Lead Backers
 
+- [Roboflow](https://github.com/roboflow)
 - Evans Fernandes
 - [Jan Sommer](https://github.com/nerfpops)
 - [Franz Gusenbauer](https://www.igutech.at/)
 
 ## Backers
 
+- Zac Miller
 - Tsingson Qin
 - Michel Pelletier
 - Jay Hannah

CHANGELOG.md

@@ -3,6 +3,157 @@
 All notable changes to this project will be documented in this file.
 This project adheres to [Semantic Versioning](http://semver.org/).
 
+## Unreleased
+
+## [11.2.2] - 2023-10-25
+
+### Fixed
+
+ - #2824, Fix regression by reverting fix that returned 206 when first position = length in a `Range` header - @laurenceisla, @strengthless
+
+## [11.2.1] - 2023-10-03
+
+### Fixed
+
+ - #2899, Fix `application/vnd.pgrst.array` not accepted as a valid mediatype - @taimoorzaeem
+ - #2524, Fix schema cache and configuration reloading with `NOTIFY` not working on Windows - @diogob, @laurenceisla
+ - #2915, Fix duplicate headers in response - @taimoorzaeem
+ - #2824, Fix range request with first position same as length return status 206 - @taimoorzaeem
+ - #2939, Fix wrong `Preference-Applied` with `Prefer: tx=commit` when transaction is rollbacked - @steve-chavez
+ - #2939, Fix `count=exact` not being included in `Preference-Applied` - @steve-chavez
+ - #2800, Fix not including to-one embed resources that had a `NULL` value in any of the selected fields when doing null filtering on them - @laurenceisla
+ - #2846, Fix error when requesting `Prefer: count=<type>` and doing null filtering on embedded resources - @laurenceisla
+ - #2959, Fix setting `default_transaction_isolation` unnecessarily - @steve-chavez
+ - #2929, Fix arrow filtering on RPC returning dynamic TABLE with composite type - @steve-chavez
+ - #2963, Fix RPCs not embedding correctly when using overloaded functions for computed relationships - @laurenceisla
+ - #2970, Fix regression that rejects URI connection strings with certain unescaped characters in the password - @laurenceisla, @steve-chavez
+
+## [11.2.0] - 2023-08-10
+
+### Added
+
+ - #2523, Data representations - @aljungberg
+   + Allows for flexible API output formatting and input parsing on a per-column type basis using regular SQL functions configured in the database
+   + Enables greater flexibility in the form and shape of your APIs, both for output and input, making PostgREST a more versatile general-purpose API server
+   + Examples include base64 encode/decode your binary data (like a `bytea` column containing an image), choose whether to present a timestamp column as seconds since the Unix epoch or as an ISO 8601 string, or represent fixed precision decimals as strings, not doubles, to preserve precision
+   + ...and accept the same in `POST/PUT/PATCH` by configuring the reverse transformation(s)
+   + Other use-cases include custom representation of enums, arrays, nested objects, CSS hex colour strings, gzip compressed fields, metric to imperial conversions, and much more
+   + Works when using the `select` parameter to select only a subset of columns, embedding through complex joins, renaming fields, with views and computed columns
+   + Works when filtering on a formatted column without extra indexes by parsing to the canonical representation
+   + Works for data `RETURNING` operations, such as requesting the full body in a POST/PUT/PATCH with `Prefer: return=representation`
+   + Works for batch updates and inserts
+   + Completely optional, define the functions in the database and they will be used automatically everywhere
+   + Data representations preserve the ability to write to the original column and require no extra storage or complex triggers (compared to using `GENERATED ALWAYS` columns)
+   + Note: data representations require Postgres 10 (Postgres 11 if using `IN` predicates); data representations are not implemented for RPC
+ - #2647, Allow to verify the PostgREST version in SQL: `select distinct application_name from pg_stat_activity`. - @laurenceisla
+ - #2856, Add the `--version` CLI option that prints the version information - @laurenceisla
+ - #1655, Improve `details` field of the singular error response - @taimoorzaeem
+ - #740, Add `Preference-Applied` in response for `Prefer: return=representation/headers-only/minimal` - @taimoorzaeem
+ - #1601, Add optional `nulls=stripped` parameter for mediatypes `application/vnd.pgrst.array+json` and `application/vnd.pgrst.object+json` - @taimoorzaeem
+
+### Fixed
+
+ - #2821, Fix OPTIONS not accepting all available media types - @steve-chavez
+ - #2834, Fix compilation on Ubuntu by being compatible with GHC 9.0.2 - @steve-chavez
+ - #2840, Fix `Prefer: missing=default` with DOMAIN default values - @steve-chavez
+ - #2849, Fix HEAD unnecessarily executing aggregates - @steve-chavez
+ - #2594, Fix unused index on jsonb/jsonb arrow filter and order (``/bets?data->>contractId=eq.1`` and ``/bets?order=data->>contractId``) - @steve-chavez
+ - #2861, Fix character and bit columns with fixed length not inserting/updating properly - @laurenceisla
+   + Fixes the error "value too long for type character(1)" when the char length of the column was bigger than one.
+ - #2862, Fix null filtering on embedded resource when using a column name equal to the relation name - @steve-chavez
+ - #1586, Fix function parameters of type character and bit not ignoring length - @laurenceisla
+   + Fixes the error "value too long for type character(1)" when the char length of the parameter was bigger than one.
+ - #2881, Fix error when a function returns `RECORD` or `SET OF RECORD` - @laurenceisla
+ - #2896, Fix applying superuser settings for impersonated role - @steve-chavez
+
+### Deprecated
+
+ - #2863, Deprecate resource embedding target disambiguation - @steve-chavez
+   + The `/table?select=*,other!fk(*)` must be used to disambiguate
+   + The server aids in choosing the `!fk` by sending a `hint` on the error whenever an ambiguous request happens.
+
+## [11.1.0] - 2023-06-07
+
+### Added
+
+ - #2786, Limit idle postgresql connection lifetime - @robx
+   + New option `db-pool-max-idletime` (default 30s).
+   + This is equivalent to the old option `db-pool-timeout` of PostgREST 10.0.0.
+   + A config alias for `db-pool-timeout` is included.
+ - #2703, Add pre-config function - @steve-chavez
+    + New config option `db-pre-config`(empty by default)
+    + Allows using the in-database configuration without SUPERUSER
+ - #2781, When `db-channel-enabled` is false, start automatic connection recovery on a new request when pool connections are closed with `pg_terminate_backend` - @steve-chavez
+    + Mitigates the lack of LISTEN/NOTIFY for schema cache reloading on read replicas.
+
+### Fixed
+
+ - #2791, Fix dropping schema cache reload notifications  - @steve-chavez
+ - #2801, Stop retrying connection when "no password supplied" - @steve-chavez
+
+## [11.0.1] - 2023-04-27
+
+### Fixed
+
+ - #2762, Fixes "permission denied for schema" error during schema cache load - @steve-chavez
+ - #2756, Fix bad error message on generated columns when using `Prefer: missing=default` - @steve-chavez
+ - #1139, Allow a 30 second skew for JWT validation - @steve-chavez
+   + It used to be 1 second, which was too strict
+
+## [11.0.0] - 2023-04-16
+
+### Added
+
+ - #1414, Add related orders - @steve-chavez
+   + On a many-to-one or one-to-one relationship, you can order a parent by a child column `/projects?select=*,clients(*)&order=clients(name).desc.nullsfirst`
+ - #1233, #1907, #2566, Allow spreading embedded resources - @steve-chavez
+   + On a many-to-one or one-to-one relationship, you can unnest a json object with `/projects?select=*,...clients(client_name:name)`
+   + Allows including the join table columns when resource embedding
+   + Allows disambiguating a recursive m2m embed
+   + Allows disambiguating an embed that has a many-to-many relationship using two foreign keys on a junction
+ - #2340, Allow embedding without selecting any column - @steve-chavez
+ - #2563, Allow `is.null` or `not.is.null` on an embedded resource - @steve-chavez
+   + Offers a more flexible replacement for `!inner`, e.g. `/projects?select=*,clients(*)&clients=not.is.null`
+   + Allows doing an anti join, e.g. `/projects?select=*,clients(*)&clients=is.null`
+   + Allows using or across related tables conditions
+ - #1100, Customizable OpenAPI title - @AnthonyFisi
+ - #2506, Add `server-trace-header` for tracing HTTP requests.  - @steve-chavez
+   + When the client sends the request header specified in the config it will be included in the response headers.
+ - #2694, Make `db-root-spec` stable. - @steve-chavez
+   + This can be used to override the OpenAPI spec with a custom database function
+ - #1567, On bulk inserts, missing values can get the column DEFAULT by using the `Prefer: missing=default` header - @steve-chavez
+ - #2501, Allow filtering by`IS DISTINCT FROM` using the `isdistinct` operator, e.g. `/people?alias=isdistinct.foo`
+ - #1569, Allow `any/all` modifiers on the `eq,like,ilike,gt,gte,lt,lte,match,imatch` operators, e.g. `/tbl?id=eq(any).{1,2,3}` - @steve-chavez
+   - This converts the input into an array type
+ - #2561, Configurable role settings - @steve-chavez
+   - Database roles that are members of the connection role get their settings applied, e.g. doing
+     `ALTER ROLE anon SET statement_timeout TO '5s'` will result in that `statement_timeout` getting applied for that role.
+   - Works when switching roles when a JWT is sent
+   - Settings can be reloaded with `NOTIFY pgrst, 'reload config'`.
+ - #2468, Configurable transaction isolation level with `default_transaction_isolation` - @steve-chavez
+   - Can be set per function `create function .. set default_transaction_isolation = 'repeatable read'`
+   - Or per role `alter role .. set default_transaction_isolation = 'serializable'`
+
+### Fixed
+
+ - #2651, Add the missing `get` path item for RPCs to the OpenAPI output - @laurenceisla
+ - #2648, Fix inaccurate error codes with new ones - @laurenceisla
+   + `PGRST204`: Column is not found
+   + `PGRST003`: Timed out when acquiring connection to db
+ - #1652, Fix function call with arguments not inlining - @steve-chavez
+ - #2705, Fix bug when using the `Range` header on `PATCH/DELETE` - @laurenceisla
+   + Fix the`"message": "syntax error at or near \"RETURNING\""` error
+   + Fix doing a limited update/delete when an `order` query parameter was present
+ - #2742, Fix db settings and pg version queries not getting prepared  - @steve-chavez
+ - #2618, Fix `PATCH` requests not recognizing embedded filters and using the top-level resource instead - @steve-chavez
+
+### Changed
+
+ - #2705, The `Range` header is now only considered on `GET` requests and is ignored for any other method - @laurenceisla
+   + Other methods should use the `limit/offset` query parameters for sub-ranges
+   + `PUT` requests no longer return an error when this header is present (using `limit/offset` still triggers the error)
+ - #2733, Remove bulk RPC call with the `Prefer: params=multiple-objects` header. A function with a JSON array or object parameter should be used instead.
+
 ## [10.2.0] - 2023-04-12
 
 ### Added
@@ -36,6 +187,7 @@ This project adheres to [Semantic Versioning](http://semver.org/).
 
  - #2548, Fix regression when embedding views with partial references to multi column FKs - @wolfgangwalther
  - #2558, Fix regression when requesting limit=0 and `db-max-row` is set - @laurenceisla
+ - #2542, Return a clear error without hitting the database when trying to update or insert an unknown column with `?columns` - @aljungberg
 
 ## [10.1.0] - 2022-10-28
 

default.nix

@@ -41,6 +41,7 @@ let
       allOverlays.postgresql-legacy
       allOverlays.postgresql-future
       (allOverlays.haskell-packages { inherit compiler; })
+      allOverlays.slocat
     ];
 
   # Evaluated expression of the Nixpkgs repository.
@@ -49,6 +50,19 @@ let
 
   postgresqlVersions =
     [
+      {
+        name = "postgresql-16";
+        postgresql = pkgs.postgresql_16.withPackages (p: [
+          p.postgis
+          (p.pg_safeupdate.overrideAttrs (old: {
+            installPhase = ''
+              mkdir -p $out/bin
+              cp safeupdate.dylib safeupdate.so || true
+              install -D safeupdate.so -t $out/lib
+            '';
+          }))
+        ]);
+      }
       { name = "postgresql-15"; postgresql = pkgs.postgresql_15.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
       { name = "postgresql-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
       { name = "postgresql-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
@@ -65,11 +79,17 @@ let
   postgrest =
     pkgs.haskell.packages."${compiler}".callCabal2nix name src { };
 
-  # Function that derives a fully static Haskell package based on
+  # Functionality that derives a fully static Haskell package based on
   # nh2/static-haskell-nix
   staticHaskellPackage =
     import nix/static-haskell-package.nix { inherit nixpkgs system compiler patches allOverlays; };
 
+  # Static executable.
+  postgrestStatic =
+    lib.justStaticExecutables (lib.dontCheck (staticHaskellPackage name src).package);
+
+  packagesStatic = (staticHaskellPackage name src).survey;
+
   # Options passed to cabal in dev tools and tests
   devCabalOptions =
     "-f dev --test-show-detail=direct";
@@ -94,10 +114,6 @@ rec {
   postgrestPackage =
     lib.dontCheck postgrest;
 
-  # Static executable.
-  postgrestStatic =
-    lib.justStaticExecutables (lib.dontCheck (staticHaskellPackage name src));
-
   # Profiled dynamic executable.
   postgrestProfiled =
     lib.enableExecutableProfiling (
@@ -119,14 +135,13 @@ rec {
   cabalTools =
     pkgs.callPackage nix/tools/cabalTools.nix { inherit devCabalOptions postgrest; };
 
+  withTools =
+    pkgs.callPackage nix/tools/withTools.nix { inherit cabalTools devCabalOptions postgresqlVersions postgrest; };
+
   # Development tools.
   devTools =
     pkgs.callPackage nix/tools/devTools.nix { inherit tests style devCabalOptions hsie withTools; };
 
-  # Docker images and loading script.
-  docker =
-    pkgs.callPackage nix/tools/docker { postgrest = postgrestStatic; };
-
   # Load testing tools.
   loadtest =
     pkgs.callPackage nix/tools/loadtest.nix { inherit withTools; };
@@ -155,7 +170,12 @@ rec {
       inherit (pkgs.haskell.packages."${compiler}") hpc-codecov;
       inherit (pkgs.haskell.packages."${compiler}") weeder;
     };
+} // pkgs.lib.optionalAttrs pkgs.stdenv.isLinux rec {
+  # Static executable.
+  inherit postgrestStatic;
+  inherit packagesStatic;
 
-  withTools =
-    pkgs.callPackage nix/tools/withTools.nix { inherit devCabalOptions postgresqlVersions postgrest; };
+  # Docker images and loading script.
+  docker =
+    pkgs.callPackage nix/tools/docker { postgrest = postgrestStatic; };
 }

docs/.gitignore

@@ -5,4 +5,4 @@ Pipfile.lock
 _diagrams/db.pdf
 misspellings
 unuseddict
-*.mo
+.history

docs/_diagrams/README.md

@@ -8,6 +8,8 @@ You can go download erd from https://github.com/BurntSushi/erd/releases and then
 ./erd_static-x86-64 -i film.er -o ../_static/film.png
 ```
 
+The fonts used belong to the GNU FreeFont family. You can download them here: http://ftp.gnu.org/gnu/freefont/
+
 ## LaTeX
 
 The schema structure diagram is done with LaTeX. You can use a GUI like https://www.mathcha.io/editor to create the .tex file.

docs/_diagrams/boxoffice.er

@@ -0,0 +1,15 @@
+entity {font: "FreeSans"}
+relationship {font: "FreeMono"}
+
+[Box_Office]
+*bo_date
+*+film_id
+gross_revenue
+
+[Films]
+*id
++director_id
+title
+`...`
+
+Box_Office +--1 Films

docs/_diagrams/employees.er

@@ -0,0 +1,12 @@
+# Build using: -e ortho
+
+entity {font: "FreeSans"}
+relationship {font: "FreeMono"}
+
+[Employees]
+*id
+first_name
+last_name
++supervisor_id
+
+Employees 1--* Employees

docs/_diagrams/film.er

@@ -1,3 +1,6 @@
+entity {font: "FreeSans"}
+relationship {font: "FreeSerif"}
+
 [Films]
 *id
 +director_id
@@ -31,6 +34,12 @@ year
 *+film_id
 rank
 
+[Technical_Specs]
+*+film_id
+runtime
+camera
+sound
+
 Roles *--1 Actors
 Roles *--1 Films
 
@@ -38,3 +47,5 @@ Nominations *--1 Competitions
 Nominations *--1 Films
 
 Films *--1 Directors
+
+Films 1--1 Technical_Specs

docs/_diagrams/orders.er

@@ -1,3 +1,8 @@
+# Build using: -e ortho
+
+entity {font: "FreeSans"}
+relationship {font: "FreeMono"}
+
 [Addresses]
 *id
 name

docs/_diagrams/premieres.er

@@ -0,0 +1,16 @@
+entity {font: "FreeSans"}
+relationship {font: "FreeMono"}
+
+[Premieres]
+*id
+location
+date
++film_id
+
+[Films]
+*id
++director_id
+title
+`...`
+
+Premieres *--1 Films

docs/_diagrams/presidents.er

@@ -0,0 +1,12 @@
+# Build using: -e ortho
+
+entity {font: "FreeSans"}
+relationship {font: "FreeMono"}
+
+[Presidents]
+*id
+first_name
+last_name
++predecessor_id
+
+Presidents 1--? Presidents

docs/_diagrams/users.er

@@ -0,0 +1,18 @@
+# Build using: -e ortho
+
+entity {font: "FreeSans"}
+relationship {font: "FreeMono"}
+
+[Users]
+*id
+first_name
+last_name
+username
+
+[Subscriptions]
+*+subscriber_id
+*+subscribed_id
+type
+
+Users 1--* Subscriptions
+Subscriptions *--1 Users

docs/_static/css/custom.css

@@ -65,3 +65,31 @@ div.line-block {
 .wy-table-responsive {
     overflow: visible !important;
 }
+
+#tutorials span.caption-text {
+    display: none;
+}
+
+#references span.caption-text {
+    display: none;
+}
+
+#explanations span.caption-text {
+    display: none;
+}
+
+#how-tos span.caption-text {
+    display: none;
+}
+
+#ecosystem span.caption-text {
+    display: none;
+}
+
+#integrations span.caption-text {
+    display: none;
+}
+
+#api span.caption-text {
+    display: none;
+}

docs/admin.rst

@@ -1,385 +0,0 @@
-.. _admin:
-
-Hardening PostgREST
-===================
-
-PostgREST is a fast way to construct a RESTful API. Its default behavior is great for scaffolding in development. When it's time to go to production it works great too, as long as you take precautions. PostgREST is a small sharp tool that focuses on performing the API-to-database mapping. We rely on a reverse proxy like Nginx for additional safeguards.
-
-The first step is to create an Nginx configuration file that proxies requests to an underlying PostgREST server.
-
-.. code-block:: nginx
-
-  http {
-    # ...
-    # upstream configuration
-    upstream postgrest {
-      server localhost:3000;
-    }
-    # ...
-    server {
-      # ...
-      # expose to the outside world
-      location /api/ {
-        default_type  application/json;
-        proxy_hide_header Content-Location;
-        add_header Content-Location  /api/$upstream_http_content_location;
-        proxy_set_header  Connection "";
-        proxy_http_version 1.1;
-        proxy_pass http://postgrest/;
-      }
-      # ...
-    }
-  }
-
-.. note::
-
-  For ubuntu, if you already installed nginx through :code:`apt` you can add this to the config file in
-  :code:`/etc/nginx/sites-enabled/default`.
-
-.. _block_fulltable:
-
-Block Full-Table Operations
----------------------------
-
-Each table in the admin-selected schema gets exposed as a top level route. Client requests are executed by certain database roles depending on their authentication. All HTTP verbs are supported that correspond to actions permitted to the role. For instance if the active role can drop rows of the table then the DELETE verb is allowed for clients. Here's an API request to delete old rows from a hypothetical logs table:
-
-.. tabs::
-
-  .. code-tab:: http
-
-    DELETE /logs?time=lt.1991-08-06 HTTP/1.1
-
-  .. code-tab:: bash Curl
-
-    curl "http://localhost:3000/logs?time=lt.1991-08-06" -X DELETE
-
-However it's very easy to delete the **entire table** by omitting the query parameter!
-
-.. tabs::
-
-  .. code-tab:: http
-
-    DELETE /logs HTTP/1.1
-
-  .. code-tab:: bash Curl
-
-    curl "http://localhost:3000/logs" -X DELETE
-
-This can happen accidentally such as by switching a request from a GET to a DELETE. To protect against accidental operations use the `pg-safeupdate <https://github.com/eradman/pg-safeupdate>`_ PostgreSQL extension. It raises an error if UPDATE or DELETE are executed without specifying conditions. To install it you can use the `PGXN <https://pgxn.org/>`_ network:
-
-.. code-block:: bash
-
-  sudo -E pgxn install safeupdate
-
-  # then add this to postgresql.conf:
-  # shared_preload_libraries='safeupdate';
-
-This does not protect against malicious actions, since someone can add a url parameter that does not affect the result set. To prevent this you must turn to database permissions, forbidding the wrong people from deleting rows, and using `row-level security <https://www.postgresql.org/docs/current/ddl-rowsecurity.html>`_ if finer access control is required.
-
-Count-Header DoS
-----------------
-
-For convenience to client-side pagination controls PostgREST supports counting and reporting total table size in its response. As described in :ref:`limits`, responses ordinarily include a range but leave the total unspecified like
-
-.. code-block:: http
-
-  HTTP/1.1 200 OK
-  Range-Unit: items
-  Content-Range: 0-14/*
-
-However including the request header :code:`Prefer: count=exact` calculates and includes the full count:
-
-.. code-block:: http
-
-  HTTP/1.1 206 Partial Content
-  Range-Unit: items
-  Content-Range: 0-14/3573458
-
-This is fine in small tables, but count performance degrades in big tables due to the MVCC architecture of PostgreSQL. For very large tables it can take a very long time to retrieve the results which allows a denial of service attack. The solution is to strip this header from all requests:
-
-.. code-block:: postgres
-
-  -- Pending nginx config: Remove any prefer header which contains the word count
-
-.. _https:
-
-HTTPS
------
-
-PostgREST aims to do one thing well: add an HTTP interface to a PostgreSQL database. To keep the code small and focused we do not implement HTTPS. Use a reverse proxy such as NGINX to add this, `here's how <https://nginx.org/en/docs/http/configuring_https_servers.html>`_. Note that some Platforms as a Service like Heroku also add SSL automatically in their load balancer.
-
-Rate Limiting
--------------
-
-Nginx supports "leaky bucket" rate limiting (see `official docs <https://nginx.org/en/docs/http/ngx_http_limit_req_module.html>`_). Using standard Nginx configuration, routes can be grouped into *request zones* for rate limiting. For instance we can define a zone for login attempts:
-
-.. code-block:: nginx
-
-  limit_req_zone $binary_remote_addr zone=login:10m rate=1r/s;
-
-This creates a shared memory zone called "login" to store a log of IP addresses that access the rate limited urls. The space reserved, 10 MB (:code:`10m`) will give us enough space to store a history of 160k requests. We have chosen to allow only allow one request per second (:code:`1r/s`).
-
-Next we apply the zone to certain routes, like a hypothetical stored procedure called :code:`login`.
-
-.. code-block:: nginx
-
-  location /rpc/login/ {
-    # apply rate limiting
-    limit_req zone=login burst=5;
-  }
-
-The burst argument tells Nginx to start dropping requests if more than five queue up from a specific IP.
-
-Nginx rate limiting is general and indiscriminate. To rate limit each authenticated request individually you will need to add logic in a :ref:`Custom Validation <custom_validation>` function.
-
-.. _external_connection_poolers:
-
-Using External Connection Poolers
----------------------------------
-
-PostgREST manages its :ref:`own pool of connections <db-pool>` and uses prepared statements by default in order to increase performance. However, this setting is incompatible with external connection poolers such as PgBouncer working in transaction pooling mode. In this case, you need to set the :ref:`db-prepared-statements` config option to ``false``. On the other hand, session pooling is fully compatible with PostgREST, while statement pooling is not compatible at all.
-
-.. note::
-
-  If prepared statements are enabled, PostgREST will quit after detecting that transaction or statement pooling is being used.
-
-You should also set the :ref:`db-channel-enabled` config option to ``false``, due to the ``LISTEN`` command not being compatible with transaction pooling, although it should not give any errors if it's left enabled by default.
-
-Debugging
-=========
-
-Server Version
---------------
-
-When debugging a problem it's important to verify the PostgREST version. At any time you can make a request to the running server and determine exactly which version is deployed. Look for the :code:`Server` HTTP response header, which contains the version number.
-
-Errors
-------
-
-See the :doc:`Errors <errors>` reference page for detailed information on the errors that PostgREST returns.
-
-.. _pgrst_logging:
-
-Logging
--------
-
-PostgREST logs basic request information to ``stdout``, including the authenticated user if available, the requesting IP address and user agent, the URL requested, and HTTP response status.
-
-.. code::
-
-   127.0.0.1 - user [26/Jul/2021:01:56:38 -0500] "GET /clients HTTP/1.1" 200 - "" "curl/7.64.0"
-   127.0.0.1 - anonymous [26/Jul/2021:01:56:48 -0500] "GET /unexistent HTTP/1.1" 404 - "" "curl/7.64.0"
-
-For diagnostic information about the server itself, PostgREST logs to ``stderr``.
-
-.. code::
-
-   12/Jun/2021:17:47:39 -0500: Attempting to connect to the database...
-   12/Jun/2021:17:47:39 -0500: Listening on port 3000
-   12/Jun/2021:17:47:39 -0500: Connection successful
-   12/Jun/2021:17:47:39 -0500: Config re-loaded
-   12/Jun/2021:17:47:40 -0500: Schema cache loaded
-
-.. note::
-
-   When running it in an SSH session you must detach it from stdout or it will be terminated when the session closes. The easiest technique is redirecting the output to a log file or to the syslog:
-
-   .. code-block:: bash
-
-     ssh foo@example.com \
-       'postgrest foo.conf </dev/null >/var/log/postgrest.log 2>&1 &'
-
-     # another option is to pipe the output into "logger -t postgrest"
-
-PostgREST logging provides limited information for debugging server errors. It's helpful to get full information about both client requests and the corresponding SQL commands executed against the underlying database.
-
-HTTP Requests
--------------
-
-A great way to inspect incoming HTTP requests including headers and query parameters is to sniff the network traffic on the port where PostgREST is running. For instance on a development server bound to port 3000 on localhost, run this:
-
-.. code:: bash
-
-  # sudo access is necessary for watching the network
-  sudo ngrep -d lo0 port 3000
-
-The options to ngrep vary depending on the address and host on which you've bound the server. The binding is described in the :ref:`configuration` section. The ngrep output isn't particularly pretty, but it's legible.
-
-.. _automatic_recovery:
-
-Automatic Connection Recovery
------------------------------
-
-When PostgREST loses the connection to the database, it retries the connection using capped exponential backoff, with 32 seconds being the maximum backoff time.
-
-This retry behavior is triggered immediately after the connection is lost if :ref:`db-channel-enabled` is set to true(the default), otherwise it will be activated once a request is made.
-
-To notify the client when the next reconnection attempt will be, PostgREST responds with ``503 Service Unavailable`` and the ``Retry-After: x`` header, where ``x`` is the number of seconds programmed for the next retry.
-
-Database Logs
--------------
-
-Once you've verified that requests are as you expect, you can get more information about the server operations by watching the database logs. By default PostgreSQL does not keep these logs, so you'll need to make the configuration changes below. Find :code:`postgresql.conf` inside your PostgreSQL data directory (to find that, issue the command :code:`show data_directory;`). Either find the settings scattered throughout the file and change them to the following values, or append this block of code to the end of the configuration file.
-
-.. code:: sql
-
-  # send logs where the collector can access them
-  log_destination = "stderr"
-
-  # collect stderr output to log files
-  logging_collector = on
-
-  # save logs in pg_log/ under the pg data directory
-  log_directory = "pg_log"
-
-  # (optional) new log file per day
-  log_filename = "postgresql-%Y-%m-%d.log"
-
-  # log every kind of SQL statement
-  log_statement = "all"
-
-Restart the database and watch the log file in real-time to understand how HTTP requests are being translated into SQL commands.
-
-.. note::
-
-  On Docker you can enable the logs by using a custom ``init.sh``:
-
-  .. code:: bash
-
-    #!/bin/sh
-    echo "log_statement = 'all'" >> /var/lib/postgresql/data/postgresql.conf
-
-  After that you can start the container and check the logs with ``docker logs``.
-
-  .. code:: bash
-
-    docker run -v "$(pwd)/init.sh":"/docker-entrypoint-initdb.d/init.sh" -d postgres
-    docker logs -f <container-id>
-
-Schema Reloading
-----------------
-
-Changing the schema while the server is running can lead to errors due to a stale schema cache. To learn how to refresh the cache see :ref:`schema_reloading`.
-
-.. _health_check:
-
-Health Check
-------------
-
-You can enable a minimal health check to verify if PostgREST is available for client requests and to check the status of its internal state.
-
-To do this, set the configuration variable :ref:`admin-server-port` to the port number of your preference. Two endpoints ``live`` and ``ready`` will then be available.
-
-The ``live`` endpoint verifies if PostgREST is running on its configured port. A request will return ``200 OK`` if PostgREST is alive or ``503`` otherwise.
-
-The ``ready`` endpoint also checks the state of both the Database Connection and the :ref:`schema_cache`. A request will return ``200 OK`` if it is ready or ``503`` if not.
-
-For instance, to verify if PostgREST is running at ``localhost:3000`` while the ``admin-server-port`` is set to ``3001``:
-
-.. tabs::
-
-  .. code-tab:: http
-
-    GET localhost:3001/live HTTP/1.1
-
-  .. code-tab:: bash Curl
-
-    curl -I "http://localhost:3001/live"
-
-.. code-block:: http
-
-  HTTP/1.1 200 OK
-
-If you have a machine with multiple network interfaces and multiple PostgREST instances in the same port, you need to specify a unique :ref:`hostname <server-host>` in the configuration of each PostgREST instance for the health check to work correctly. Don't use the special values(``!4``, ``*``, etc) in this case because the health check could report a false positive.
-
-Daemonizing
-===========
-
-For Linux distributions that use **systemd** (Ubuntu, Debian, Archlinux) you can create a daemon in the following way.
-
-First, create postgrest configuration in ``/etc/postgrest/config``
-
-.. code-block:: ini
-
-  db-uri = "postgres://<your_user>:<your_password>@localhost:5432/<your_db>"
-  db-schemas = "<your_exposed_schema>"
-  db-anon-role = "<your_anon_role>"
-  jwt-secret = "<your_secret>"
-
-Then create the systemd service file in ``/etc/systemd/system/postgrest.service``
-
-.. code-block:: ini
-
-  [Unit]
-  Description=REST API for any PostgreSQL database
-  After=postgresql.service
-
-  [Service]
-  ExecStart=/bin/postgrest /etc/postgrest/config
-  ExecReload=/bin/kill -SIGUSR1 $MAINPID
-
-  [Install]
-  WantedBy=multi-user.target
-
-After that, you can enable the service at boot time and start it with:
-
-.. code-block:: bash
-
-  systemctl enable postgrest
-  systemctl start postgrest
-
-  ## For reloading the service
-  ## systemctl restart postgrest
-
-.. _file_descriptors:
-
-File Descriptors
-----------------
-
-File descriptors are kernel resources that are used by HTTP connections (among others). File descriptors are limited per process. The kernel default limit is 1024, which is increased in some Linux distributions.
-When under heavy traffic, PostgREST can reach this limit and start showing ``No file descriptors available`` errors. To clear these errors, you can increase the process' file descriptor limit.
-
-.. code-block:: ini
-
-  [Service]
-  LimitNOFILE=10000
-
-Alternate URL Structure
-=======================
-
-As discussed in :ref:`singular_plural`, there are no special URL forms for singular resources in PostgREST, only operators for filtering. Thus there are no URLs like :code:`/people/1`. It would be specified instead as
-
-.. tabs::
-
-  .. code-tab:: http
-
-    GET /people?id=eq.1 HTTP/1.1
-    Accept: application/vnd.pgrst.object+json
-
-  .. code-tab:: bash Curl
-
-    curl "http://localhost:3000/people?id=eq.1" \
-      -H "Accept: application/vnd.pgrst.object+json"
-
-This allows compound primary keys and makes the intent for singular response independent of a URL convention.
-
-Nginx rewrite rules allow you to simulate the familiar URL convention. The following example adds a rewrite rule for all table endpoints, but you'll want to restrict it to those tables that have a numeric simple primary key named "id."
-
-.. code-block:: nginx
-
-  # support /endpoint/:id url style
-  location ~ ^/([a-z_]+)/([0-9]+) {
-
-    # make the response singular
-    proxy_set_header Accept 'application/vnd.pgrst.object+json';
-
-    # assuming an upstream named "postgrest"
-    proxy_pass http://postgrest/$1?id=eq.$2;
-
-  }
-
-.. TODO
-.. Administration
-..   API Versioning
-..   HTTP Caching
-..   Upgrading

docs/auth.rst

@@ -1,496 +0,0 @@
-.. _roles:
-
-Overview of Role System
-=======================
-
-PostgREST is designed to keep the database at the center of API security. All authorization happens through database roles and permissions. It is PostgREST's job to **authenticate** requests -- i.e. verify that a client is who they say they are -- and then let the database **authorize** client actions.
-
-Authentication Sequence
------------------------
-
-There are three types of roles used by PostgREST, the **authenticator**, **anonymous** and **user** roles. The database administrator creates these roles and configures PostgREST to use them.
-
-.. image:: _static/security-roles.png
-
-The authenticator should be created :code:`NOINHERIT` and configured in the database to have very limited access. It is a chameleon whose job is to "become" other users to service authenticated HTTP requests. The picture below shows how the server handles authentication. If auth succeeds, it switches into the user role specified by the request, otherwise it switches into the anonymous role (if it's set in :ref:`db-anon-role`).
-
-.. image:: _static/security-anon-choice.png
-
-Here are the technical details. We use `JSON Web Tokens <https://jwt.io/>`_ to authenticate API requests. As you'll recall a JWT contains a list of cryptographically signed claims. All claims are allowed but PostgREST cares specifically about a claim called role.
-
-.. code:: json
-
-  {
-    "role": "user123"
-  }
-
-When a request contains a valid JWT with a role claim PostgREST will switch to the database role with that name for the duration of the HTTP request.
-
-.. code:: sql
-
-  SET LOCAL ROLE user123;
-
-Note that the database administrator must allow the authenticator role to switch into this user by previously executing
-
-.. code:: sql
-
-  GRANT user123 TO authenticator;
-
-If the client included no JWT (or one without a role claim) then PostgREST switches into the anonymous role whose actual database-specific name, like that of with the authenticator role, is specified in the PostgREST server configuration file. The database administrator must set anonymous role permissions correctly to prevent anonymous users from seeing or changing things they shouldn't.
-
-Users and Groups
-----------------
-
-PostgreSQL manages database access permissions using the concept of roles. A role can be thought of as either a database user, or a group of database users, depending on how the role is set up.
-
-Roles for Each Web User
-~~~~~~~~~~~~~~~~~~~~~~~
-
-PostgREST can accommodate either viewpoint. If you treat a role as a single user then the JWT-based role switching described above does most of what you need. When an authenticated user makes a request PostgREST will switch into the role for that user, which in addition to restricting queries, is available to SQL through the :code:`current_user` variable.
-
-You can use row-level security to flexibly restrict visibility and access for the current user. Here is an `example <https://www.2ndquadrant.com/en/blog/application-users-vs-row-level-security/>`_ from Tomas Vondra, a chat table storing messages sent between users. Users can insert rows into it to send messages to other users, and query it to see messages sent to them by other users.
-
-.. code-block:: postgres
-
-  CREATE TABLE chat (
-    message_uuid    UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
-    message_time    TIMESTAMP NOT NULL DEFAULT now(),
-    message_from    NAME      NOT NULL DEFAULT current_user,
-    message_to      NAME      NOT NULL,
-    message_subject VARCHAR(64) NOT NULL,
-    message_body    TEXT
-  );
-
-  ALTER TABLE chat ENABLE ROW LEVEL SECURITY;
-
-We want to enforce a policy that ensures a user can see only those messages sent by them or intended for them. Also we want to prevent a user from forging the message_from column with another person's name.
-
-PostgreSQL allows us to set this policy with row-level security:
-
-.. code-block:: postgres
-
-  CREATE POLICY chat_policy ON chat
-    USING ((message_to = current_user) OR (message_from = current_user))
-    WITH CHECK (message_from = current_user)
-
-Anyone accessing the generated API endpoint for the chat table will see exactly the rows they should, without our needing custom imperative server-side coding.
-
-.. warning::
-
-   Roles are namespaced per-cluster rather than per-database so they may be prone to collision.
-
-Web Users Sharing Role
-~~~~~~~~~~~~~~~~~~~~~~
-
-Alternately database roles can represent groups instead of (or in addition to) individual users. You may choose that all signed-in users for a web app share the role webuser. You can distinguish individual users by including extra claims in the JWT such as email.
-
-.. code:: json
-
-  {
-    "role": "webuser",
-    "email": "john@doe.com"
-  }
-
-SQL code can access claims through GUC variables set by PostgREST per request. For instance to get the email claim, call this function:
-
-For PostgreSQL server version >= 14
-
-.. code:: sql
-
-  current_setting('request.jwt.claims', true)::json->>'email';
-
-
-For PostgreSQL server version < 14
-
-.. code:: sql
-
-  current_setting('request.jwt.claim.email', true);
-
-This allows JWT generation services to include extra information and your database code to react to it. For instance the RLS example could be modified to use this current_setting rather than current_user. The second 'true' argument tells current_setting to return NULL if the setting is missing from the current configuration.
-
-Hybrid User-Group Roles
-~~~~~~~~~~~~~~~~~~~~~~~
-
-You can mix the group and individual role policies. For instance we could still have a webuser role and individual users which inherit from it:
-
-.. code-block:: postgres
-
-  CREATE ROLE webuser NOLOGIN;
-  -- grant this role access to certain tables etc
-
-  CREATE ROLE user000 NOLOGIN;
-  GRANT webuser TO user000;
-  -- now user000 can do whatever webuser can
-
-  GRANT user000 TO authenticator;
-  -- allow authenticator to switch into user000 role
-  -- (the role itself has nologin)
-
-.. _custom_validation:
-
-Custom Validation
------------------
-
-PostgREST honors the :code:`exp` claim for token expiration, rejecting expired tokens. However it does not enforce any extra constraints. An example of an extra constraint would be to immediately revoke access for a certain user. The configuration file parameter :code:`db-pre-request` specifies a stored procedure to call immediately after the authenticator switches into a new role and before the main query itself runs.
-
-Here's an example. In the config file specify a stored procedure:
-
-.. code:: ini
-
-  db-pre-request = "public.check_user"
-
-In the function you can run arbitrary code to check the request and raise an exception to block it if desired.
-
-.. code-block:: postgres
-
-  CREATE OR REPLACE FUNCTION check_user() RETURNS void AS $$
-  BEGIN
-    IF current_user = 'evil_user' THEN
-      RAISE EXCEPTION 'No, you are evil'
-        USING HINT = 'Stop being so evil and maybe you can log in';
-    END IF;
-  END
-  $$ LANGUAGE plpgsql;
-
-.. _client_auth:
-
-Client Auth
-===========
-
-To make an authenticated request the client must include an :code:`Authorization` HTTP header with the value :code:`Bearer <jwt>`. For instance:
-
-.. tabs::
-
-  .. code-tab:: http
-
-    GET /foo HTTP/1.1
-    Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiamRvZSIsImV4cCI6MTQ3NTUxNjI1MH0.GYDZV3yM0gqvuEtJmfpplLBXSGYnke_Pvnl0tbKAjB4
-
-  .. code-tab:: bash Curl
-
-    curl "http://localhost:3000/foo" \
-      -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiamRvZSIsImV4cCI6MTQ3NTUxNjI1MH0.GYDZV3yM0gqvuEtJmfpplLBXSGYnke_Pvnl0tbKAjB4"
-
-The ``Bearer`` header value can be used with or without capitalization(``bearer``).
-
-JWT Generation
---------------
-
-You can create a valid JWT either from inside your database or via an external service. Each token is cryptographically signed with a secret key. In the case of symmetric cryptography the signer and verifier share the same secret passphrase. In asymmetric cryptography the signer uses the private key and the verifier the public key. PostgREST supports both symmetric and asymmetric cryptography.
-
-JWT from SQL
-~~~~~~~~~~~~
-
-You can create JWT tokens in SQL using the `pgjwt extension <https://github.com/michelp/pgjwt>`_. It's simple and requires only pgcrypto. If you're on an environment like Amazon RDS which doesn't support installing new extensions, you can still manually run the `SQL inside pgjwt <https://github.com/michelp/pgjwt/blob/master/pgjwt--0.1.1.sql>`_ (you'll need to replace ``@extschema@`` with another schema or just delete it) which creates the functions you will need.
-
-Next write a stored procedure that returns the token. The one below returns a token with a hard-coded role, which expires five minutes after it was issued. Note this function has a hard-coded secret as well.
-
-.. code-block:: postgres
-
-  CREATE TYPE jwt_token AS (
-    token text
-  );
-
-  CREATE FUNCTION jwt_test() RETURNS public.jwt_token AS $$
-    SELECT public.sign(
-      row_to_json(r), 'reallyreallyreallyreallyverysafe'
-    ) AS token
-    FROM (
-      SELECT
-        'my_role'::text as role,
-        extract(epoch from now())::integer + 300 AS exp
-    ) r;
-  $$ LANGUAGE sql;
-
-PostgREST exposes this function to clients via a POST request to ``/rpc/jwt_test``.
-
-.. note::
-
-  To avoid hard-coding the secret in stored procedures, save it as a property of the database.
-
-  .. code-block:: postgres
-
-    -- run this once
-    ALTER DATABASE mydb SET "app.jwt_secret" TO 'reallyreallyreallyreallyverysafe';
-
-    -- then all functions can refer to app.jwt_secret
-    SELECT sign(
-      row_to_json(r), current_setting('app.jwt_secret')
-    ) AS token
-    FROM ...
-
-JWT from Auth0
-~~~~~~~~~~~~~~
-
-An external service like `Auth0 <https://auth0.com/>`_ can do the hard work transforming OAuth from Github, Twitter, Google etc into a JWT suitable for PostgREST. Auth0 can also handle email signup and password reset flows.
-
-To use Auth0, create `an application <https://auth0.com/docs/get-started/applications>`_ for your app and `an API <https://auth0.com/docs/get-started/apis>`_ for your PostgREST server. Auth0 supports both HS256 and RS256 scheme for the issued tokens for APIs. For simplicity, you may first try HS256 scheme while creating your API on Auth0. Your application should use your PostgREST API's `API identifier <https://auth0.com/docs/get-started/apis/api-settings>`_ by setting it with the `audience parameter <https://auth0.com/docs/secure/tokens/access-tokens/get-access-tokens#control-access-token-audience>`_  during the authorization request. This will ensure that Auth0 will issue an access token for your PostgREST API. For PostgREST to verify the access token, you will need to set ``jwt-secret`` on PostgREST config file with your API's signing secret.
-
-.. note::
-
-  Our code requires a database role in the JWT. To add it you need to save the database role in Auth0 `app metadata <https://auth0.com/docs/manage-users/user-accounts/metadata/manage-metadata-rules>`_. Then, you will need to write `a rule <https://auth0.com/docs/customize/rules>`_ that will extract the role from the user's app_metadata and set it as a `custom claim <https://auth0.com/docs/get-started/apis/scopes/sample-use-cases-scopes-and-claims#add-custom-claims-to-a-token>`_ in the access token. Note that, you may use Auth0's `core authorization feature <https://auth0.com/docs/manage-users/access-control/rbac>`_ for more complex use cases. Metadata solution is mentioned here for simplicity.
-
-  .. code:: javascript
-
-    function (user, context, callback) {
-
-      // Follow the documentations at
-      // https://postgrest.org/en/latest/configuration.html#db-role-claim-key
-      // to set a custom role claim on PostgREST
-      // and use it as custom claim attribute in this rule
-      const myRoleClaim = 'https://myapp.com/role';
-
-      user.app_metadata = user.app_metadata || {};
-      context.accessToken[myRoleClaim] = user.app_metadata.role;
-      callback(null, user, context);
-    }
-
-.. _asym_keys:
-
-Asymmetric Keys
-~~~~~~~~~~~~~~~
-
-As described in the :ref:`configuration` section, PostgREST accepts a ``jwt-secret`` config file parameter. If it is set to a simple string value like "reallyreallyreallyreallyverysafe" then PostgREST interprets it as an HMAC-SHA256 passphrase. However you can also specify a literal JSON Web Key (JWK) or set. For example, you can use an RSA-256 public key encoded as a JWK:
-
-.. code-block:: json
-
-  {
-    "alg":"RS256",
-    "e":"AQAB",
-    "key_ops":["verify"],
-    "kty":"RSA",
-    "n":"9zKNYTaYGfGm1tBMpRT6FxOYrM720GhXdettc02uyakYSEHU2IJz90G_MLlEl4-WWWYoS_QKFupw3s7aPYlaAjamG22rAnvWu-rRkP5sSSkKvud_IgKL4iE6Y2WJx2Bkl1XUFkdZ8wlEUR6O1ft3TS4uA-qKifSZ43CahzAJyUezOH9shI--tirC028lNg767ldEki3WnVr3zokSujC9YJ_9XXjw2hFBfmJUrNb0-wldvxQbFU8RPXip-GQ_JPTrCTZhrzGFeWPvhA6Rqmc3b1PhM9jY7Dur1sjYWYVyXlFNCK3c-6feo5WlRfe1aCWmwZQh6O18eTmLeT4nWYkDzQ"
-  }
-
-.. note::
-
-  This could also be a JSON Web Key Set (JWKS) if it was contained within an array assigned to a `keys` member, e.g. ``{ keys: [jwk1, jwk2] }``.
-
-Just pass it in as a single line string, escaping the quotes:
-
-.. code-block:: ini
-
-  jwt-secret = "{ \"alg\":\"RS256\", … }"
-
-To generate such a public/private key pair use a utility like `latchset/jose <https://github.com/latchset/jose>`_.
-
-.. code-block:: bash
-
-  jose jwk gen -i '{"alg": "RS256"}' -o rsa.jwk
-  jose jwk pub -i rsa.jwk -o rsa.jwk.pub
-
-  # now rsa.jwk.pub contains the desired JSON object
-
-You can specify the literal value as we saw earlier, or reference a filename to load the JWK from a file:
-
-.. code-block:: ini
-
-  jwt-secret = "@rsa.jwk.pub"
-
-JWT security
-~~~~~~~~~~~~
-
-There are at least three types of common critiques against using JWT: 1) against the standard itself, 2) against using libraries with known security vulnerabilities, and 3) against using JWT for web sessions. We'll briefly explain each critique, how PostgREST deals with it, and give recommendations for appropriate user action.
-
-The critique against the `JWT standard <https://datatracker.ietf.org/doc/html/rfc7519>`_ is voiced in detail `elsewhere on the web <https://web.archive.org/web/20230123041631/https://paragonie.com/blog/2017/03/jwt-json-web-tokens-is-bad-standard-that-everyone-should-avoid>`_. The most relevant part for PostgREST is the so-called :code:`alg=none` issue. Some servers implementing JWT allow clients to choose the algorithm used to sign the JWT. In this case, an attacker could set the algorithm to :code:`none`, remove the need for any signature at all and gain unauthorized access. The current implementation of PostgREST, however, does not allow clients to set the signature algorithm in the HTTP request, making this attack irrelevant. The critique against the standard is that it requires the implementation of the :code:`alg=none` at all.
-
-Critiques against JWT libraries are only relevant to PostgREST via the library it uses. As mentioned above, not allowing clients to choose the signature algorithm in HTTP requests removes the greatest risk. Another more subtle attack is possible where servers use asymmetric algorithms like RSA for signatures. Once again this is not relevant to PostgREST since it is not supported. Curious readers can find more information in `this article <https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/>`_. Recommendations about high quality libraries for usage in API clients can be found on `jwt.io <https://jwt.io/>`_.
-
-The last type of critique focuses on the misuse of JWT for maintaining web sessions. The basic recommendation is to `stop using JWT for sessions <http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-for-sessions/>`_ because most, if not all, solutions to the problems that arise when you do, `do not work <http://cryto.net/~joepie91/blog/2016/06/19/stop-using-jwt-for-sessions-part-2-why-your-solution-doesnt-work/>`_. The linked articles discuss the problems in depth but the essence of the problem is that JWT is not designed to be secure and stateful units for client-side storage and therefore not suited to session management.
-
-PostgREST uses JWT mainly for authentication and authorization purposes and encourages users to do the same. For web sessions, using cookies over HTTPS is good enough and well catered for by standard web frameworks.
-
-Schema Isolation
-================
-
-You can isolate your api schema from internal implementation details, as explained in :ref:`schema_isolation`. For an example of wrapping a private table with a public view see the :ref:`public_ui` section below.
-
-.. _sql_user_management:
-
-SQL User Management
-===================
-
-Storing Users and Passwords
----------------------------
-
-As mentioned, an external service can provide user management and coordinate with the PostgREST server using JWT. It's also possible to support logins entirely through SQL. It's a fair bit of work, so get ready.
-
-The following table, functions, and triggers will live in a :code:`basic_auth` schema that you shouldn't expose publicly in the API. The public views and functions will live in a different schema which internally references this internal information.
-
-First we'll need a table to keep track of our users:
-
-.. code:: sql
-
-  -- We put things inside the basic_auth schema to hide
-  -- them from public view. Certain public procs/views will
-  -- refer to helpers and tables inside.
-  create schema if not exists basic_auth;
-
-  create table if not exists
-  basic_auth.users (
-    email    text primary key check ( email ~* '^.+@.+\..+$' ),
-    pass     text not null check (length(pass) < 512),
-    role     name not null check (length(role) < 512)
-  );
-
-We would like the role to be a foreign key to actual database roles, however PostgreSQL does not support these constraints against the :code:`pg_roles` table. We'll use a trigger to manually enforce it.
-
-.. code-block:: plpgsql
-
-  create or replace function
-  basic_auth.check_role_exists() returns trigger as $$
-  begin
-    if not exists (select 1 from pg_roles as r where r.rolname = new.role) then
-      raise foreign_key_violation using message =
-        'unknown database role: ' || new.role;
-      return null;
-    end if;
-    return new;
-  end
-  $$ language plpgsql;
-
-  drop trigger if exists ensure_user_role_exists on basic_auth.users;
-  create constraint trigger ensure_user_role_exists
-    after insert or update on basic_auth.users
-    for each row
-    execute procedure basic_auth.check_role_exists();
-
-Next we'll use the pgcrypto extension and a trigger to keep passwords safe in the :code:`users` table.
-
-.. code-block:: plpgsql
-
-  create extension if not exists pgcrypto;
-
-  create or replace function
-  basic_auth.encrypt_pass() returns trigger as $$
-  begin
-    if tg_op = 'INSERT' or new.pass <> old.pass then
-      new.pass = crypt(new.pass, gen_salt('bf'));
-    end if;
-    return new;
-  end
-  $$ language plpgsql;
-
-  drop trigger if exists encrypt_pass on basic_auth.users;
-  create trigger encrypt_pass
-    before insert or update on basic_auth.users
-    for each row
-    execute procedure basic_auth.encrypt_pass();
-
-With the table in place we can make a helper to check a password against the encrypted column. It returns the database role for a user if the email and password are correct.
-
-.. code-block:: plpgsql
-
-  create or replace function
-  basic_auth.user_role(email text, pass text) returns name
-    language plpgsql
-    as $$
-  begin
-    return (
-    select role from basic_auth.users
-     where users.email = user_role.email
-       and users.pass = crypt(user_role.pass, users.pass)
-    );
-  end;
-  $$;
-
-.. _public_ui:
-
-Public User Interface
----------------------
-
-In the previous section we created an internal table to store user information. Here we create a login function which takes an email address and password and returns JWT if the credentials match a user in the internal table.
-
-Permissions
-~~~~~~~~~~~
-
-Your database roles need access to the schema, tables, views and functions in order to service HTTP requests.
-Recall from the `Overview of Role System`_ that PostgREST uses special roles to process requests, namely the authenticator and
-anonymous roles. Below is an example of permissions that allow anonymous users to create accounts and attempt to log in.
-
-.. code-block:: postgres
-
-  -- the names "anon" and "authenticator" are configurable and not
-  -- sacred, we simply choose them for clarity
-  create role anon noinherit;
-  create role authenticator noinherit;
-  grant anon to authenticator;
-
-Then, add ``db-anon-role`` to the configuration file to allow anonymous requests.
-
-.. code:: ini
-
-  db-anon-role = "anon"
-
-Logins
-~~~~~~
-
-As described in `JWT from SQL`_, we'll create a JWT inside our login function. Note that you'll need to adjust the secret key which is hard-coded in this example to a secure (at least thirty-two character) secret of your choosing.
-
-.. code-block:: postgres
-
-  -- add type
-  CREATE TYPE basic_auth.jwt_token AS (
-    token text
-  );
-
-  -- login should be on your exposed schema
-  create or replace function
-  login(email text, pass text) returns basic_auth.jwt_token as $$
-  declare
-    _role name;
-    result basic_auth.jwt_token;
-  begin
-    -- check email and password
-    select basic_auth.user_role(email, pass) into _role;
-    if _role is null then
-      raise invalid_password using message = 'invalid user or password';
-    end if;
-
-    select sign(
-        row_to_json(r), 'reallyreallyreallyreallyverysafe'
-      ) as token
-      from (
-        select _role as role, login.email as email,
-           extract(epoch from now())::integer + 60*60 as exp
-      ) r
-      into result;
-    return result;
-  end;
-  $$ language plpgsql security definer;
-
-  grant execute on function login(text,text) to anon;
-
-Since the above :code:`login` function is defined as `security definer <https://www.postgresql.org/docs/current/sql-createfunction.html#id-1.9.3.67.10.2>`_,
-the anonymous user :code:`anon` doesn't need permission to read the :code:`basic_auth.users` table. It doesn't even need permission to access the :code:`basic_auth` schema.
-:code:`grant execute on function` is included for clarity but it might not be needed, see :ref:`func_privs` for more details.
-
-An API request to call this function would look like:
-
-.. tabs::
-
-  .. code-tab:: http
-
-    POST /rpc/login HTTP/1.1
-
-    { "email": "foo@bar.com", "pass": "foobar" }
-
-  .. code-tab:: bash Curl
-
-    curl "http://localhost:3000/rpc/login" \
-      -X POST -H "Content-Type: application/json" \
-      -d '{ "email": "foo@bar.com", "pass": "foobar" }'
-
-The response would look like the snippet below. Try decoding the token at `jwt.io <https://jwt.io/>`_. (It was encoded with a secret of :code:`reallyreallyreallyreallyverysafe` as specified in the SQL code above. You'll want to change this secret in your app!)
-
-.. code:: json
-
-  {
-    "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImZvb0BiYXIuY29tIiwicGFzcyI6ImZvb2JhciJ9.37066TTRlh-1hXhnA9oO9Pj6lgL6zFuJU0iCHhuCFno"
-  }
-
-
-Alternatives
-~~~~~~~~~~~~
-
-See the how-to :ref:`sql-user-management-using-postgres-users-and-passwords` for a similar way that completely avoids the table :code:`basic_auth.users`.

docs/conf.py

@@ -45,7 +45,7 @@ source_suffix = ".rst"
 master_doc = "index"
 
 # This is overriden by readthedocs with the version tag anyway
-version = "10.2"
+version = "11.2"
 # To avoid repetition in <title> we set this to an empty string.
 release = ""
 

docs/default.nix

@@ -77,7 +77,9 @@ in
          | tail -n+2 \
          | tr '\n' '\0' \
          | xargs -0 -n 1 -i \
-           sh -c "grep \"{}\" $FILES > /dev/null || echo \"{}\""
+           sh -c "grep \"{}\" $FILES > /dev/null || echo \"{}\"" \
+         | tee unuseddict
+        test ! -s unuseddict
       '';
 
   linkcheck =

docs/ecosystem.rst

@@ -16,6 +16,12 @@ Community Tutorials
 
 * `How PostgreSQL triggers work when called with a PostgREST PATCH HTTP request <https://blog.fgribreau.com/2020/11/how-postgresql-triggers-works-when.html>`_ - A tutorial to see how the old and new values are set or not when doing a PATCH request to PostgREST.
 
+* `REST Data Service on YugabyteDB / PostgreSQL <https://dev.to/yugabyte/rest-data-service-on-yugabytedb-postgresql-5f2h>`_
+
+* `Build data-driven applications with Workers and PostgreSQL <https://developers.cloudflare.com/workers/tutorials/postgres/>`_ - A tutorial on how to integrate with PostgREST and PostgreSQL using Cloudflare Workers.
+
+* `A poor man's API <https://blog.frankel.ch/poor-man-api>`_ - Shows how to integrate PostgREST with Apache APISIX as an alternative to Nginx.
+
 .. _templates:
 
 Templates
@@ -29,28 +35,10 @@ Templates
 Example Apps
 ------------
 
-* `chronicle <https://github.com/srid/chronicle>`_ - tracking a tree of personal memories
-* `code-du-travail-backoffice <https://github.com/SocialGouv/code-du-travail-backoffice>`_ - data administration portal for the official French Labor Code and Agreements
 * `delibrium-postgrest <https://gitlab.com/delibrium/delibrium-postgrest/>`_ - example school API and front-end in Vue.js
-* `elm-workshop <https://github.com/diogob/elm-workshop>`_ - building a simple database query UI
-* `ember-postgrest-dynamic-ui <https://github.com/benoror/ember-postgrest-dynamic-ui>`_ - generating Ember forms to edit data
 * `ETH-transactions-storage <https://github.com/Adamant-im/ETH-transactions-storage>`_ - indexer for Ethereum to get transaction list by ETH address
-* `ext-postgrest-crud <https://github.com/timwis/ext-postgrest-crud>`_ - browser-based spreadsheet
 * `general <https://github.com/PierreRochard/general>`_ - example auth back-end
-* `goodfilm <https://github.com/tyrchen/goodfilm>`_ - example film API
 * `guild-operators <https://github.com/cardano-community/koios-artifacts/tree/main/files/grest>`_ - example queries and functions that the Cardano Community uses for their Guild Operators' Repository
-* `handsontable-postgrest <https://github.com/timwis/handsontable-postgrest>`_ - an excel-like database table editor
-* `heritage-near-me <https://github.com/CodeforAustralia/heritage-near-me>`_ - Elm and PostgREST with PostGIS
-* `ng-admin-postgrest <https://github.com/marmelab/ng-admin-postgrest>`_ - automatic database admin panel
-* `pgrst-dev-setup <https://github.com/Qu4tro/pgrst-dev-setup>`_ - docker-compose and tmuxp setup for experimentation.
-* `postgres-postgrest-cloudflared-example <https://github.com/cloudflare/postgres-postgrest-cloudflared-example>`_ - docker-compose setup exposing PostgREST using cloudfared
-* `postgrest-demo <https://github.com/SMRxT/postgrest-demo>`_ - multi-tenant logging system
-* `postgrest-example <https://github.com/begriffs/postgrest-example>`_ - sqitch versioning for API
-* `postgrest-sessions-example <https://github.com/monacoremo/postgrest-sessions-example>`_ - example for cookie-based sessions
-* `postgrest-translation-proxy <https://github.com/NikolayS/postgrest-translation-proxy>`_ - calling to external translation service
-* `postgrest-ui <https://github.com/tatut/postgrest-ui>`_ - ClojureScript UI components for PostgREST
-* `postgrest-vercel <https://github.com/seveibar/postgrest-vercel>`_ - run PostgREST on Vercel (Serverless/AWS Lambda)
-* `PostgrestSkeleton <https://github.com/Recmo/PostgrestSkeleton>`_ - Docker Compose, PostgREST, Nginx and Auth0
 * `PostGUI <https://github.com/priyank-purohit/PostGUI>`_ - React Material UI admin panel
 * `prospector <https://github.com/sfcta/prospector>`_ - data warehouse and visualization platform
 
@@ -63,6 +51,7 @@ DevOps
 * `cloudstark/helm-charts <https://github.com/cloudstark/helm-charts/tree/master/postgrest>`_ - helm chart to deploy PostgREST to a Kubernetes cluster via a Deployment and Service
 * `jbkarle/postgrest <https://github.com/jbkarle/postgrest>`_ - helm chart with a demo database for development and test purposes
 * `Limezest/postgrest-cloud-run <https://github.com/Limezest/postgrest-cloud-run>`_ - expose a PostgreSQL database on Cloud SQL using Cloud Run
+* `eyberg/postgrest <https://repo.ops.city/v2/packages/eyberg/postgrest/10.1.1/x86_64/show>`_ - run PostgREST as a Nanos unikernel
 
 .. _eco_external_notification:
 
@@ -71,14 +60,10 @@ External Notification
 
 These are PostgreSQL bridges that propagate LISTEN/NOTIFY to external queues for further processing. This allows stored procedures to initiate actions outside the database such as sending emails.
 
-* `pg-bridge <https://github.com/matthewmueller/pg-bridge>`_ - Amazon SNS
-* `pg-kinesis-bridge <https://github.com/daurnimator/pg-kinesis-bridge>`_ - Amazon Kinesis
 * `pg-notify-webhook <https://github.com/vbalasu/pg-notify-webhook>`_ - trigger webhooks from PostgreSQL's LISTEN/NOTIFY
 * `pgsql-listen-exchange <https://github.com/gmr/pgsql-listen-exchange>`_ - RabbitMQ
 * `postgres-websockets <https://github.com/diogob/postgres-websockets>`_ - expose web sockets for PostgreSQL's LISTEN/NOTIFY
-* `postgresql-to-amqp <https://github.com/FGRibreau/postgresql-to-amqp>`_ - AMQP
 * `postgresql2websocket <https://github.com/frafra/postgresql2websocket>`_ - Websockets
-* `skeeter <https://github.com/SpiderOak/skeeter>`_ - ZeroMQ
 
 
 .. _eco_extensions:
@@ -88,25 +73,14 @@ Extensions
 
 * `aiodata <https://github.com/Exahilosys/aiodata>`_ - Python, event-based proxy and caching client.
 * `pg-safeupdate <https://github.com/eradman/pg-safeupdate>`_ - prevent full-table updates or deletes
-* `postgrest-auth (criles25) <https://github.com/criles25/postgrest-auth>`_ - email based auth/signup
 * `postgrest-node <https://github.com/seveibar/postgrest-node>`_ - Run a PostgREST server in Node.js via npm module
-* `postgrest-oauth <https://github.com/nblumoe/postgrest-oauth>`_ - OAuth2 WAI middleware
-* `postgrest-oauth/api <https://github.com/postgrest-oauth/api>`_ - OAuth2 server
 * `PostgREST-writeAPI <https://github.com/ppKrauss/PostgREST-writeAPI>`_ - generate Nginx rewrite rules to fit an OpenAPI spec
-* `spas <https://github.com/srid/spas>`_ - allow file uploads and basic auth
 
 .. _clientside_libraries:
 
 Client-Side Libraries
 ---------------------
 
-* `aor-postgrest-client <https://github.com/tomberek/aor-postgrest-client>`_ - JS, admin-on-rest
-* `elm-postgrest <https://github.com/john-kelly/elm-postgrest>`_ - Elm
-* `general-angular <https://github.com/PierreRochard/general-angular>`_ - TypeScript, generate UI from API description
-* `jarvus-postgrest-apikit <https://github.com/JarvusInnovations/jarvus-postgrest-apikit>`_ - JS, Sencha framework
-* `mithril-postgrest <https://github.com/catarse/mithril-postgrest>`_ - JS, Mithril
-* `ng-postgrest <https://github.com/team142/ng-postgrest>`_ - Angular app for browsing, editing data exposed over PostgREST.
-* `postgrest-client <https://github.com/calebmer/postgrest-client>`_ - JS
 * `postgrest-csharp <https://github.com/supabase-community/postgrest-csharp>`_ - C#
 * `postgrest-dart <https://github.com/supabase-community/postgrest-dart>`_ - Dart
 * `postgrest-ex <https://github.com/J0/postgrest-ex>`_ - Elixir
@@ -114,15 +88,8 @@ Client-Side Libraries
 * `postgrest-js <https://github.com/supabase/postgrest-js>`_ - TypeScript/JavaScript
 * `postgrest-kt <https://github.com/supabase-community/postgrest-kt>`_ - Kotlin
 * `postgrest-py <https://github.com/supabase/postgrest-py>`_ - Python
-* `postgrest-request <https://github.com/lewisjared/postgrest-request>`_ - JS, SuperAgent
 * `postgrest-rs <https://github.com/supabase-community/postgrest-rs>`_ - Rust
-* `postgrest-sharp-client <https://github.com/thejettdurham/postgrest-sharp-client>`_ (needs maintainer) - C#, RestSharp
 * `postgrest-swift <https://github.com/supabase-community/postgrest-swift>`_ - Swift
-* `postgrest-url <https://github.com/hugomrdias/postgrest-url>`_ - JS, just for generating query URLs
-* `postgrest_python_requests_client <https://github.com/davidthewatson/postgrest_python_requests_client>`_ - Python
-* `postgrester <https://github.com/ivangabriele/postgrester>`_ - JS + Typescript
-* `postgrestR <https://github.com/clesiemo3/postgrestR>`_ - R
-* `py-postgrest <https://github.com/Kong/py-postgrest>`_ - Python
 * `redux-postgrest <https://github.com/andytango/redux-postgrest>`_ - TypeScript/JS, client integrated with (React) Redux.
 * `vue-postgrest <https://github.com/technowledgy/vue-postgrest>`_ - Vue.js
 

docs/explanations/db_authz.rst

@@ -0,0 +1,207 @@
+.. _db_authz:
+
+Database Authorization
+######################
+
+Database authorization is the process of granting and verifying database access permissions. PostgreSQL manages permissions using the concept of roles.
+
+Users and Groups
+================
+
+A role can be thought of as either a database user, or a group of database users, depending on how the role is set up.
+
+Roles for Each Web User
+-----------------------
+
+PostgREST can accommodate either viewpoint. If you treat a role as a single user then the :ref:`jwt_impersonation` does most of what you need. When an authenticated user makes a request PostgREST will switch into the database role for that user, which in addition to restricting queries, is available to SQL through the :code:`current_user` variable.
+
+You can use row-level security to flexibly restrict visibility and access for the current user. Here is an `example <https://www.enterprisedb.com:443/blog/application-users-vs-row-level-security>`_ from Tomas Vondra, a chat table storing messages sent between users. Users can insert rows into it to send messages to other users, and query it to see messages sent to them by other users.
+
+.. code-block:: postgres
+
+  CREATE TABLE chat (
+    message_uuid    UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+    message_time    TIMESTAMP NOT NULL DEFAULT now(),
+    message_from    NAME      NOT NULL DEFAULT current_user,
+    message_to      NAME      NOT NULL,
+    message_subject VARCHAR(64) NOT NULL,
+    message_body    TEXT
+  );
+
+  ALTER TABLE chat ENABLE ROW LEVEL SECURITY;
+
+We want to enforce a policy that ensures a user can see only those messages sent by them or intended for them. Also we want to prevent a user from forging the ``message_from`` column with another person's name.
+
+PostgreSQL allows us to set this policy with row-level security:
+
+.. code-block:: postgres
+
+  CREATE POLICY chat_policy ON chat
+    USING ((message_to = current_user) OR (message_from = current_user))
+    WITH CHECK (message_from = current_user)
+
+Anyone accessing the generated API endpoint for the chat table will see exactly the rows they should, without our needing custom imperative server-side coding.
+
+.. warning::
+
+   Roles are namespaced per-cluster rather than per-database so they may be prone to collision.
+
+Web Users Sharing Role
+----------------------
+
+Alternately database roles can represent groups instead of (or in addition to) individual users. You may choose that all signed-in users for a web app share the role ``webuser``. You can distinguish individual users by including extra claims in the JWT such as email.
+
+.. code:: json
+
+  {
+    "role": "webuser",
+    "email": "john@doe.com"
+  }
+
+SQL code can access claims through PostgREST :ref:`tx_settings`. For instance to get the email claim, call this function:
+
+.. code:: sql
+
+  current_setting('request.jwt.claims', true)::json->>'email';
+
+.. note::
+
+  For PostgreSQL < 14
+
+  .. code:: sql
+
+    current_setting('request.jwt.claim.email', true);
+
+This allows JWT generation services to include extra information and your database code to react to it. For instance the RLS example could be modified to use this ``current_setting`` rather than ``current_user``. The second ``'true'`` argument tells ``current_setting`` to return NULL if the setting is missing from the current configuration.
+
+Hybrid User-Group Roles
+-----------------------
+
+You can mix the group and individual role policies. For instance we could still have a webuser role and individual users which inherit from it:
+
+.. code-block:: postgres
+
+  CREATE ROLE webuser NOLOGIN;
+  -- grant this role access to certain tables etc
+
+  CREATE ROLE user000 NOLOGIN;
+  GRANT webuser TO user000;
+  -- now user000 can do whatever webuser can
+
+  GRANT user000 TO authenticator;
+  -- allow authenticator to switch into user000 role
+  -- (the role itself has nologin)
+
+.. _schema_isolation:
+
+Schemas
+=======
+
+A PostgREST instance exposes all the tables, views, and stored procedures of the schemas configured in :ref:`db-schemas`. This means private data or implementation details can go inside private schemas and be invisible to HTTP clients.
+
+It is recommended that you don't expose tables on the schemas you expose, instead expose views and stored procedures which insulate the internal details from the outside world.
+This allows you to change the internals of your schema and maintain backwards compatibility. It also keeps your code easier to refactor, and provides a natural way to do API versioning.
+
+.. image:: ../_static/db.png
+
+You must explicitly allow roles to access the exposed schemas:
+
+.. code-block:: postgres
+
+   GRANT USAGE ON SCHEMA api TO webuser;
+
+Tables
+======
+
+To let web users access tables you must grant them privileges for the operations you want them to do.
+
+.. code-block:: postgres
+
+  GRANT
+    SELECT
+  , INSERT
+  , UPDATE(message_body)
+  , DELETE
+  ON chat TO webuser;
+
+You can also choose on which table columns the operation is valid. In the above example, the web user can only update the ``message_body`` column.
+
+.. _func_privs:
+
+Functions
+=========
+
+By default, when a function is created, the privilege to execute it is not restricted by role. The function access is ``PUBLIC`` — executable by all roles (more details at `PostgreSQL Privileges page <https://www.postgresql.org/docs/current/ddl-priv.html>`_). This is not ideal for an API schema. To disable this behavior, you can run the following SQL statement:
+
+.. code-block:: postgres
+
+  ALTER DEFAULT PRIVILEGES REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;
+
+This will change the privileges for all functions created in the future in all schemas. Currently there is no way to limit it to a single schema. In our opinion it's a good practice anyway.
+
+.. note::
+
+    It is however possible to limit the effect of this clause only to functions you define. You can put the above statement at the beginning of the API schema definition, and then at the end reverse it with:
+
+    .. code-block:: postgres
+
+        ALTER DEFAULT PRIVILEGES GRANT EXECUTE ON FUNCTIONS TO PUBLIC;
+
+    This will work because the :code:`alter default privileges` statement has effect on function created *after* it is executed. See `PostgreSQL alter default privileges <https://www.postgresql.org/docs/current/sql-alterdefaultprivileges.html>`_ for more details.
+
+After that, you'll need to grant EXECUTE privileges on functions explicitly:
+
+.. code-block:: postgres
+
+   GRANT EXECUTE ON FUNCTION login TO anonymous;
+   GRANT EXECUTE ON FUNCTION signup TO anonymous;
+
+You can also grant execute on all functions in a schema to a higher privileged role:
+
+.. code-block:: postgres
+
+    GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA api TO web_user;
+
+Security definer
+----------------
+
+A function is executed with the privileges of the user who calls it. This means that the user has to have all permissions to do the operations the procedure performs.
+If the function accesses private database objects, your :ref:`API roles <roles>` won't be able to successfully execute the function.
+
+Another option is to define the function with the :code:`SECURITY DEFINER` option. Then only one permission check will take place, the permission to call the function, and the operations in the function will have the authority of the user who owns the function itself.
+
+.. code-block:: postgres
+
+  -- login as a user wich has privileges on the private schemas
+
+  -- create a sample function
+  create or replace function login(email text, pass text) returns jwt_token as $$
+  begin
+    -- access to a private schema called 'auth'
+    select auth.user_role(email, pass) into _role;
+    -- other operations
+    -- ...
+  end;
+  $$ language plpgsql security definer;
+
+Note the ``SECURITY DEFINER`` keywords at the end of the function. See `PostgreSQL documentation <https://www.postgresql.org/docs/current/sql-createfunction.html#SQL-CREATEFUNCTION-SECURITY>`_ for more details.
+
+Views
+=====
+
+Views are invoked with the privileges of the view owner, much like stored procedures with the ``SECURITY DEFINER`` option. When created by a SUPERUSER role, all `row-level security <https://www.postgresql.org/docs/current/ddl-rowsecurity.html>`_ policies will be bypassed.
+
+If you're on PostgreSQL >= 15, this behavior can be changed by specifying the ``security_invoker`` option.
+
+.. code-block:: postgres
+
+  CREATE VIEW sample_view WITH (security_invoker = true) AS
+  SELECT * FROM sample_table;
+
+On PostgreSQL < 15, you can create a non-SUPERUSER role and make this role the view's owner.
+
+.. code-block:: postgres
+
+  CREATE ROLE api_views_owner NOSUPERUSER NOBYPASSRLS;
+  ALTER VIEW sample_view OWNER TO api_views_owner;
+

docs/explanations/install.rst

@@ -1,7 +1,7 @@
 .. _install:
 
 Installation
-============
+############
 
 The release page has `pre-compiled binaries for Mac OS X, Windows, Linux and FreeBSD <https://github.com/PostgREST/postgrest/releases/latest>`_ .
 The Linux binary is a static executable that can be run on any Linux distribution.
@@ -32,7 +32,7 @@ You can also use your OS package manager.
 
       .. tab:: Arch Linux
 
-        You can install PostgREST from the `community repo <https://archlinux.org/packages/community/x86_64/postgrest>`_.
+        You can install PostgREST from the `community repo <https://archlinux.org/packages/extra/x86_64/postgrest/>`_.
 
         .. code:: bash
 
@@ -55,6 +55,18 @@ You can also use your OS package manager.
       choco install postgrest
       scoop install postgrest
 
+
+.. _pg-dependency:
+
+Supported PostgreSQL versions
+=============================
+
+=============== =================================
+**Supported**   PostgreSQL >= 9.6
+=============== =================================
+
+PostgREST works with all PostgreSQL versions starting from 9.6.
+
 Running PostgREST
 =================
 
@@ -95,19 +107,12 @@ For a complete reference of the configuration file, see :ref:`configuration`.
 
   If you see a dialog box like this on Windows, it may be that the :code:`pg_config` program is not in your system path.
 
-  .. image:: _static/win-err-dialog.png
+  .. image:: ../_static/win-err-dialog.png
 
   It usually lives in :code:`C:\Program Files\PostgreSQL\<version>\bin`. See this `article <https://www.howtogeek.com/118594/how-to-edit-your-system-path-for-easy-command-line-access/>`_ about how to modify the system path.
 
   To test that the system path is set correctly, run ``pg_config`` from the command line. You should see it output a list of paths.
 
-.. _pg-dependency:
-
-PostgreSQL dependency
----------------------
-
-To use PostgREST you will need an underlying database. We require PostgreSQL 9.6 or greater. You can use something like `Amazon RDS <https://aws.amazon.com/rds/>`_ but installing your own locally is cheaper and more convenient for development. You can also run PostgreSQL in a :ref:`docker container<pg-in-docker>`.
-
 Docker
 ======
 
@@ -256,126 +261,3 @@ You can build PostgREST from source with `Stack <https://github.com/commercialha
    - `--install-ghc` flag is only needed for the first build and can be omitted in the subsequent builds.
 
 * Check that the server is installed: :code:`postgrest --help`.
-
-.. _deploy_heroku:
-
-Deploying to Heroku
-===================
-
-1.  Log into Heroku using the `Heroku CLI <https://devcenter.heroku.com/articles/heroku-cli>`_:
-
-  .. code-block:: bash
-
-    # If you have multiple Heroku accounts, use flag '--interactive' to switch between them
-    heroku login --interactive
-
-
-2.  Create a new Heroku app using the PostgREST buildpack:
-
-  .. code-block:: bash
-
-    mkdir ${YOUR_APP_NAME}
-    cd ${YOUR_APP_NAME}
-    git init .
-
-    heroku apps:create ${YOUR_APP_NAME} --buildpack https://github.com/PostgREST/postgrest-heroku.git
-    heroku git:remote -a ${YOUR_APP_NAME}
-
-3.  Create a new Heroku PostgreSQL add-on attached to the app and keep notes of the assigned add-on name (e.g. :code:`postgresql-curly-58902`) referred later as ${HEROKU_PG_DB_NAME}
-
-  .. code-block:: bash
-
-    heroku addons:create heroku-postgresql:standard-0 -a ${YOUR_APP_NAME}
-    # wait until the add-on is available
-    heroku pg:wait -a ${YOUR_APP_NAME}
-
-4.  Create the necessary user roles according to the 
-    `PostgREST documentation <https://postgrest.org/en/stable/auth.html>`_:
-
-  .. code-block:: bash
-
-    heroku pg:credentials:create --name api_user -a ${YOUR_APP_NAME}
-    # use the following command to ensure the new credential state is active before attaching it
-    heroku pg:credentials -a ${YOUR_APP_NAME}
-
-    heroku addons:attach ${HEROKU_PG_DB_NAME} --credential api_user -a ${YOUR_APP_NAME}
-
-5.  Connect to the PostgreSQL database and create some sample data:
-
-  .. code-block:: bash
-
-    heroku psql -a ${YOUR_APP_NAME}
-
-  .. code-block:: postgres
-
-    # from the psql command prompt execute the following commands:
-    create schema api;
-
-    create table api.todos (
-    id serial primary key,
-    done boolean not null default false,
-    task text not null,
-    due timestamptz
-    );
-
-    insert into api.todos (task) values
-    ('finish tutorial 0'), ('pat self on back');
-
-    grant usage on schema api to api_user;
-    grant select on api.todos to api_user;
-
-6.  Create the :code:`Procfile`:
-
-  .. code-block:: bash
-
-    web: PGRST_SERVER_HOST=0.0.0.0 PGRST_SERVER_PORT=${PORT} PGRST_DB_URI=${PGRST_DB_URI:-${DATABASE_URL}} ./postgrest-${POSTGREST_VER}
-  ..
-    
-  Set the following environment variables on Heroku:
-
-  .. code-block:: bash
-
-    heroku config:set POSTGREST_VER=10.0.0
-    heroku config:set PGRST_DB_SCHEMA=api
-    heroku config:set PGRST_DB_ANON_ROLE=api_user
-  ..
-
-  PGRST_DB_URI can be set if an external database is used or if it's different from the default Heroku DATABASE_URL. This latter is used if nothing is provided.  
-  POSTGREST_VER is mandatory to select and build the required PostgREST release.
-
-  See https://postgrest.org/en/stable/configuration.html#environment-variables for the full list of environment variables.
-
-7.  Build and deploy your app:
-
-  .. code-block:: bash
-
-    git add Procfile
-    git commit -m "PostgREST on Heroku"
-    git push heroku master
-  ..
-
-  Your Heroku app should be live at :code:`${YOUR_APP_NAME}.herokuapp.com`
-
-8.  Test your app
-
-    From a terminal display the application logs:
-
-  .. code-block:: bash
-
-    heroku logs -t
-  ..
-
-  From a different terminal retrieve with curl the records previously created:
-
-  .. code-block:: bash
- 
-    curl https://${YOUR_APP_NAME}.herokuapp.com/todos
-  .. 
-
-  and test that any attempt to modify the table via a read-only user is not allowed:
-
-  .. code-block:: bash
-
-    curl https://${YOUR_APP_NAME}.herokuapp.com/todos -X POST \
-     -H "Content-Type: application/json" \
-     -d '{"task": "do bad thing"}'   

docs/explanations/nginx.rst

@@ -0,0 +1,109 @@
+.. _nginx:
+
+Nginx
+=====
+
+PostgREST is a fast way to construct a RESTful API. Its default behavior is great for scaffolding in development. When it's time to go to production it works great too, as long as you take precautions.
+PostgREST is a small sharp tool that focuses on performing the API-to-database mapping. We rely on a reverse proxy like Nginx for additional safeguards.
+
+The first step is to create an Nginx configuration file that proxies requests to an underlying PostgREST server.
+
+.. code-block:: nginx
+
+  http {
+    # ...
+    # upstream configuration
+    upstream postgrest {
+      server localhost:3000;
+    }
+    # ...
+    server {
+      # ...
+      # expose to the outside world
+      location /api/ {
+        default_type  application/json;
+        proxy_hide_header Content-Location;
+        add_header Content-Location  /api/$upstream_http_content_location;
+        proxy_set_header  Connection "";
+        proxy_http_version 1.1;
+        proxy_pass http://postgrest/;
+      }
+      # ...
+    }
+  }
+
+.. note::
+
+  For ubuntu, if you already installed nginx through :code:`apt` you can add this to the config file in
+  :code:`/etc/nginx/sites-enabled/default`.
+
+.. _https:
+
+HTTPS
+-----
+
+PostgREST aims to do one thing well: add an HTTP interface to a PostgreSQL database. To keep the code small and focused we do not implement HTTPS. Use a reverse proxy such as NGINX to add this, `here's how <https://nginx.org/en/docs/http/configuring_https_servers.html>`_. Note that some Platforms as a Service like Heroku also add SSL automatically in their load balancer.
+
+Rate Limiting
+-------------
+
+Nginx supports "leaky bucket" rate limiting (see `official docs <https://nginx.org/en/docs/http/ngx_http_limit_req_module.html>`_). Using standard Nginx configuration, routes can be grouped into *request zones* for rate limiting. For instance we can define a zone for login attempts:
+
+.. code-block:: nginx
+
+  limit_req_zone $binary_remote_addr zone=login:10m rate=1r/s;
+
+This creates a shared memory zone called "login" to store a log of IP addresses that access the rate limited urls. The space reserved, 10 MB (:code:`10m`) will give us enough space to store a history of 160k requests. We have chosen to allow only allow one request per second (:code:`1r/s`).
+
+Next we apply the zone to certain routes, like a hypothetical stored procedure called :code:`login`.
+
+.. code-block:: nginx
+
+  location /rpc/login/ {
+    # apply rate limiting
+    limit_req zone=login burst=5;
+  }
+
+The burst argument tells Nginx to start dropping requests if more than five queue up from a specific IP.
+
+Nginx rate limiting is general and indiscriminate. To rate limit each authenticated request individually you will need to add logic in a :ref:`Custom Validation <custom_validation>` function.
+
+Alternate URL Structure
+-----------------------
+
+As discussed in :ref:`singular_plural`, there are no special URL forms for singular resources in PostgREST, only operators for filtering. Thus there are no URLs like :code:`/people/1`. It would be specified instead as
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /people?id=eq.1 HTTP/1.1
+    Accept: application/vnd.pgrst.object+json
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/people?id=eq.1" \
+      -H "Accept: application/vnd.pgrst.object+json"
+
+This allows compound primary keys and makes the intent for singular response independent of a URL convention.
+
+Nginx rewrite rules allow you to simulate the familiar URL convention. The following example adds a rewrite rule for all table endpoints, but you'll want to restrict it to those tables that have a numeric simple primary key named "id."
+
+.. code-block:: nginx
+
+  # support /endpoint/:id url style
+  location ~ ^/([a-z_]+)/([0-9]+) {
+
+    # make the response singular
+    proxy_set_header Accept 'application/vnd.pgrst.object+json';
+
+    # assuming an upstream named "postgrest"
+    proxy_pass http://postgrest/$1?id=eq.$2;
+
+  }
+
+.. TODO
+.. Administration
+..   API Versioning
+..   HTTP Caching
+..   Upgrading

docs/how-tos/sql-user-management-using-postgres-users-and-passwords.rst

@@ -59,7 +59,7 @@ In order to be able to work with postgres' SCRAM-SHA-256 password hashes, we als
   CREATE FUNCTION basic_auth.pbkdf2(salt bytea, pw text, count integer, desired_length integer, algorithm text) RETURNS bytea
       LANGUAGE plpgsql IMMUTABLE
       AS $$
-  DECLARE 
+  DECLARE
     hash_length integer;
     block_count integer;
     output bytea;
@@ -97,7 +97,7 @@ In order to be able to work with postgres' SCRAM-SHA-256 password hashes, we als
       --
       FOR j IN 2 .. count LOOP
         the_last := ext_pgcrypto.HMAC(the_last, pw::bytea, algorithm);
-        
+
         -- xor the two
         FOR k IN 1 .. length(xorsum) LOOP
           xorsum := set_byte(xorsum, k - 1, get_byte(xorsum, k - 1) # get_byte(the_last, k - 1));
@@ -210,8 +210,6 @@ anonymous roles. Below is an example of permissions that allow anonymous users t
 
 .. code-block:: postgres
 
-  -- the names "anon" and "authenticator" are configurable and not
-  -- sacred, we simply choose them for clarity
   CREATE ROLE anon NOINHERIT;
   CREATE role authenticator NOINHERIT LOGIN PASSWORD 'secret';
   GRANT anon TO authenticator;
@@ -299,7 +297,7 @@ Let's add a table, intended for the :code:`foo` user:
 Now try to get the table's contents with:
 
 .. tabs::
-  
+
   .. code-tab:: http
 
     GET /foobar HTTP/1.1

docs/how-tos/sql-user-management.rst

@@ -0,0 +1,228 @@
+.. _sql_user_management:
+
+SQL User Management
+===================
+
+As mentioned on :ref:`jwt_generation`, an external service can provide user management and coordinate with the PostgREST server using JWT. It’s also possible to support logins entirely through SQL. It’s a fair bit of work, so get ready.
+
+Storing Users and Passwords
+---------------------------
+
+The following table, functions, and triggers will live in a :code:`basic_auth` schema that you shouldn't expose publicly in the API. The public views and functions will live in a different schema which internally references this internal information.
+
+First we'll need a table to keep track of our users:
+
+.. code:: sql
+
+  -- We put things inside the basic_auth schema to hide
+  -- them from public view. Certain public procs/views will
+  -- refer to helpers and tables inside.
+  create schema if not exists basic_auth;
+
+  create table if not exists
+  basic_auth.users (
+    email    text primary key check ( email ~* '^.+@.+\..+$' ),
+    pass     text not null check (length(pass) < 512),
+    role     name not null check (length(role) < 512)
+  );
+
+We would like the role to be a foreign key to actual database roles, however PostgreSQL does not support these constraints against the :code:`pg_roles` table. We'll use a trigger to manually enforce it.
+
+.. code-block:: plpgsql
+
+  create or replace function
+  basic_auth.check_role_exists() returns trigger as $$
+  begin
+    if not exists (select 1 from pg_roles as r where r.rolname = new.role) then
+      raise foreign_key_violation using message =
+        'unknown database role: ' || new.role;
+      return null;
+    end if;
+    return new;
+  end
+  $$ language plpgsql;
+
+  drop trigger if exists ensure_user_role_exists on basic_auth.users;
+  create constraint trigger ensure_user_role_exists
+    after insert or update on basic_auth.users
+    for each row
+    execute procedure basic_auth.check_role_exists();
+
+Next we'll use the pgcrypto extension and a trigger to keep passwords safe in the :code:`users` table.
+
+.. code-block:: plpgsql
+
+  create extension if not exists pgcrypto;
+
+  create or replace function
+  basic_auth.encrypt_pass() returns trigger as $$
+  begin
+    if tg_op = 'INSERT' or new.pass <> old.pass then
+      new.pass = crypt(new.pass, gen_salt('bf'));
+    end if;
+    return new;
+  end
+  $$ language plpgsql;
+
+  drop trigger if exists encrypt_pass on basic_auth.users;
+  create trigger encrypt_pass
+    before insert or update on basic_auth.users
+    for each row
+    execute procedure basic_auth.encrypt_pass();
+
+With the table in place we can make a helper to check a password against the encrypted column. It returns the database role for a user if the email and password are correct.
+
+.. code-block:: plpgsql
+
+  create or replace function
+  basic_auth.user_role(email text, pass text) returns name
+    language plpgsql
+    as $$
+  begin
+    return (
+    select role from basic_auth.users
+     where users.email = user_role.email
+       and users.pass = crypt(user_role.pass, users.pass)
+    );
+  end;
+  $$;
+
+.. _public_ui:
+
+Public User Interface
+---------------------
+
+In the previous section we created an internal table to store user information. Here we create a login function which takes an email address and password and returns JWT if the credentials match a user in the internal table.
+
+Permissions
+~~~~~~~~~~~
+
+Your database roles need access to the schema, tables, views and functions in order to service HTTP requests.
+Recall from the :ref:`roles` that PostgREST uses special roles to process requests, namely the authenticator and
+anonymous roles. Below is an example of permissions that allow anonymous users to create accounts and attempt to log in.
+
+.. code-block:: postgres
+
+  create role anon noinherit;
+  create role authenticator noinherit;
+  grant anon to authenticator;
+
+Then, add ``db-anon-role`` to the configuration file to allow anonymous requests.
+
+.. code:: ini
+
+  db-anon-role = "anon"
+
+JWT from SQL
+~~~~~~~~~~~~
+
+You can create JWT tokens in SQL using the `pgjwt extension <https://github.com/michelp/pgjwt>`_. It's simple and requires only pgcrypto. If you're on an environment like Amazon RDS which doesn't support installing new extensions, you can still manually run the `SQL inside pgjwt <https://github.com/michelp/pgjwt/blob/master/pgjwt--0.1.1.sql>`_ (you'll need to replace ``@extschema@`` with another schema or just delete it) which creates the functions you will need.
+
+Next write a stored procedure that returns the token. The one below returns a token with a hard-coded role, which expires five minutes after it was issued. Note this function has a hard-coded secret as well.
+
+.. code-block:: postgres
+
+  CREATE TYPE jwt_token AS (
+    token text
+  );
+
+  CREATE FUNCTION jwt_test() RETURNS public.jwt_token AS $$
+    SELECT public.sign(
+      row_to_json(r), 'reallyreallyreallyreallyverysafe'
+    ) AS token
+    FROM (
+      SELECT
+        'my_role'::text as role,
+        extract(epoch from now())::integer + 300 AS exp
+    ) r;
+  $$ LANGUAGE sql;
+
+PostgREST exposes this function to clients via a POST request to ``/rpc/jwt_test``.
+
+.. note::
+
+  To avoid hard-coding the secret in stored procedures, save it as a property of the database.
+
+  .. code-block:: postgres
+
+    -- run this once
+    ALTER DATABASE mydb SET "app.jwt_secret" TO 'reallyreallyreallyreallyverysafe';
+
+    -- then all functions can refer to app.jwt_secret
+    SELECT sign(
+      row_to_json(r), current_setting('app.jwt_secret')
+    ) AS token
+    FROM ...
+
+Logins
+~~~~~~
+
+As described in `JWT from SQL`_, we'll create a JWT inside our login function. Note that you'll need to adjust the secret key which is hard-coded in this example to a secure (at least thirty-two character) secret of your choosing.
+
+.. code-block:: postgres
+
+  -- add type
+  CREATE TYPE basic_auth.jwt_token AS (
+    token text
+  );
+
+  -- login should be on your exposed schema
+  create or replace function
+  login(email text, pass text) returns basic_auth.jwt_token as $$
+  declare
+    _role name;
+    result basic_auth.jwt_token;
+  begin
+    -- check email and password
+    select basic_auth.user_role(email, pass) into _role;
+    if _role is null then
+      raise invalid_password using message = 'invalid user or password';
+    end if;
+
+    select sign(
+        row_to_json(r), 'reallyreallyreallyreallyverysafe'
+      ) as token
+      from (
+        select _role as role, login.email as email,
+           extract(epoch from now())::integer + 60*60 as exp
+      ) r
+      into result;
+    return result;
+  end;
+  $$ language plpgsql security definer;
+
+  grant execute on function login(text,text) to anon;
+
+Since the above :code:`login` function is defined as `security definer <https://www.postgresql.org/docs/current/sql-createfunction.html#id-1.9.3.67.10.2>`_,
+the anonymous user :code:`anon` doesn't need permission to read the :code:`basic_auth.users` table. It doesn't even need permission to access the :code:`basic_auth` schema.
+:code:`grant execute on function` is included for clarity but it might not be needed, see :ref:`func_privs` for more details.
+
+An API request to call this function would look like:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/login HTTP/1.1
+
+    { "email": "foo@bar.com", "pass": "foobar" }
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/login" \
+      -X POST -H "Content-Type: application/json" \
+      -d '{ "email": "foo@bar.com", "pass": "foobar" }'
+
+The response would look like the snippet below. Try decoding the token at `jwt.io <https://jwt.io/>`_. (It was encoded with a secret of :code:`reallyreallyreallyreallyverysafe` as specified in the SQL code above. You'll want to change this secret in your app!)
+
+.. code:: json
+
+  {
+    "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImZvb0BiYXIuY29tIiwicGFzcyI6ImZvb2JhciJ9.37066TTRlh-1hXhnA9oO9Pj6lgL6zFuJU0iCHhuCFno"
+  }
+
+
+Alternatives
+~~~~~~~~~~~~
+
+See the how-to :ref:`sql-user-management-using-postgres-users-and-passwords` for a similar way that completely avoids the table :code:`basic_auth.users`.

docs/index.rst

@@ -32,24 +32,18 @@ Sponsors
     :target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
     :width:  13em
 
-  .. image:: _static/2ndquadrant.png
-    :target: https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo
-    :width:  13em
-
-  .. image:: _static/retool.png
-    :target: https://retool.com/?utm_source=sponsor&utm_campaign=postgrest
-    :width:  13em
-
   .. image:: _static/gnuhost.png
     :target: https://euronodes.com/?utm_source=sponsor&utm_campaign=postgrest
     :width:  13em
 
+  |
+
   .. image:: _static/supabase.png
     :target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
     :width:  13em
 
-  .. image:: _static/oblivious.jpg
-    :target: https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest
+  .. image:: _static/neon.jpg
+    :target: https://neon.tech/?utm_source=sponsor&utm_campaign=postgrest
     :width:  13em
 
   .. The static/empty.png(created with `convert -size 320x95 xc:#fcfcfc empty.png`) is an ugly workaround
@@ -61,15 +55,15 @@ Sponsors
 
 |
 
-Motivation
-----------
+Database as Single Source of Truth
+----------------------------------
 
 Using PostgREST is an alternative to manual CRUD programming. Custom API servers suffer problems. Writing business logic often duplicates, ignores or hobbles database structure. Object-relational mapping is a leaky abstraction leading to slow imperative code. The PostgREST philosophy establishes a single declarative source of truth: the data itself.
 
 Declarative Programming
 -----------------------
 
-It's easier to ask PostgreSQL to join data for you and let its query planner figure out the details than to loop through rows yourself. It's easier to assign permissions to db objects than to add guards in controllers. (This is especially true for cascading permissions in data dependencies.) It's easier to set constraints than to litter code with sanity checks.
+It's easier to ask PostgreSQL to join data for you and let its query planner figure out the details than to loop through rows yourself. It's easier to assign permissions to database objects than to add guards in controllers. (This is especially true for cascading permissions in data dependencies.) It's easier to set constraints than to litter code with sanity checks.
 
 Leak-proof Abstraction
 ----------------------
@@ -89,18 +83,10 @@ The project has a friendly and growing community. For discussions, use the Githu
 .. toctree::
    :glob:
    :caption: Release Notes
-   :titlesonly:
-   :hidden:
+   :reversed:
+   :maxdepth: 1
 
-   v10.2.0 <releases/v10.2.0>
-   v10.0.0 <releases/v10.0.0>
-   v9.0.1 <releases/v9.0.1>
-   v9.0.0 <releases/v9.0.0>
-   releases/v8.0.0
-   releases/v7.0.1
-   releases/v7.0.0
-   releases/v6.0.2
-   releases/v5.2.0
+   releases/*
 
 Tutorials
 ---------
@@ -110,104 +96,72 @@ Are you new to PostgREST? This is the place to start!
 .. toctree::
    :glob:
    :caption: Tutorials
-   :hidden:
+   :maxdepth: 1
 
    tutorials/*
 
-- :doc:`tutorials/tut0`
-- :doc:`tutorials/tut1`
+Also have a look at :ref:`install` and :ref:`community_tutorials`.
 
-Also have a look at :doc:`Installation <install>` and :ref:`community_tutorials`.
-
-Reference guides
-----------------
+References
+----------
 
 Technical references for PostgREST's functionality.
 
 .. toctree::
-   :caption: API
-   :hidden:
+   :glob:
+   :caption: References
+   :name: references
+   :maxdepth: 1
 
-   api.rst
+   references/auth.rst
+   references/api.rst
+   references/transactions.rst
+   references/connection_pool.rst
+   references/schema_cache.rst
+   references/errors.rst
+   references/configuration.rst
+   references/*
 
-.. toctree::
-   :caption: Configuration
-   :hidden:
-
-   configuration.rst
-
-.. toctree::
-   :caption: Schema Cache
-   :hidden:
-
-   schema_cache.rst
-
-.. toctree::
-   :caption: Errors
-   :hidden:
-
-   errors.rst
-
-- :doc:`API <api>`
-- :doc:`configuration`
-- :doc:`Schema Cache <schema_cache>`
-- :doc:`Errors <errors>`
-
-Topic guides
+Explanations
 ------------
 
-Explanations of some key concepts in PostgREST.
+Key concepts in PostgREST.
 
 .. toctree::
-   :caption: Authentication
-   :hidden:
+   :glob:
+   :caption: Explanations
+   :name: explanations
+   :maxdepth: 1
 
-   auth.rst
+   explanations/*
 
-.. toctree::
-   :caption: Schema Structure
-   :hidden:
+How-tos
+-------
 
-   schema_structure.rst
-
-.. toctree::
-   :caption: Administration
-   :hidden:
-
-   admin.rst
-
-.. toctree::
-   :caption: Installation
-   :hidden:
-
-   install.rst
-
-- :doc:`Authentication <auth>`
-- :doc:`Schema Structure <schema_structure>`
-- :doc:`Administration <admin>`
-- :doc:`Installation <install>`
-
-.. _how_tos:
-
-How-to guides
--------------
-
-These are recipes that'll help you address specific use-cases.
+Recipes that'll help you address specific use-cases.
 
 .. toctree::
    :glob:
    :caption: How-to guides
-   :hidden:
+   :name: how-tos
+   :maxdepth: 1
 
-   how-tos/working-with-postgresql-data-types
-   how-tos/providing-images-for-img
-   how-tos/create-soap-endpoint
-   how-tos/sql-user-management-using-postgres-users-and-passwords
+   how-tos/sql-user-*
+   how-tos/working-*
+   how-tos/*
 
-- :doc:`how-tos/providing-images-for-img`
-- :doc:`how-tos/working-with-postgresql-data-types`
-- :doc:`how-tos/create-soap-endpoint`
-- :doc:`how-tos/sql-user-management-using-postgres-users-and-passwords`
+.. _intgrs:
+
+Integrations
+------------
+
+.. toctree::
+   :glob:
+   :caption: Integrations
+   :name: integrations
+   :maxdepth: 1
+
+   integrations/*
 
 Ecosystem
 ---------
@@ -216,27 +170,11 @@ PostgREST has a growing ecosystem of examples, libraries, and experiments. Here
 
 .. toctree::
    :caption: Ecosystem
-   :hidden:
+   :name: ecosystem
+   :maxdepth: 1
 
    ecosystem.rst
 
-* :ref:`community_tutorials`
-* :ref:`templates`
-* :ref:`eco_example_apps`
-* :ref:`devops`
-* :ref:`eco_external_notification`
-* :ref:`eco_extensions`
-* :ref:`clientside_libraries`
-
-
-Release Notes
--------------
-
-Changes among versions.
-
-- :doc:`releases/v9.0.0`
-- :doc:`releases/v8.0.0`
-
 In Production
 -------------
 

docs/integrations/heroku.rst

@@ -0,0 +1,122 @@
+.. _deploy_heroku:
+
+Heroku
+======
+
+1.  Log into Heroku using the `Heroku CLI <https://devcenter.heroku.com/articles/heroku-cli>`_:
+
+  .. code-block:: bash
+
+    # If you have multiple Heroku accounts, use flag '--interactive' to switch between them
+    heroku login --interactive
+
+
+2.  Create a new Heroku app using the PostgREST buildpack:
+
+  .. code-block:: bash
+
+    mkdir ${YOUR_APP_NAME}
+    cd ${YOUR_APP_NAME}
+    git init .
+
+    heroku apps:create ${YOUR_APP_NAME} --buildpack https://github.com/PostgREST/postgrest-heroku.git
+    heroku git:remote -a ${YOUR_APP_NAME}
+
+3.  Create a new Heroku PostgreSQL add-on attached to the app and keep notes of the assigned add-on name (e.g. :code:`postgresql-curly-58902`) referred later as ${HEROKU_PG_DB_NAME}
+
+  .. code-block:: bash
+
+    heroku addons:create heroku-postgresql:standard-0 -a ${YOUR_APP_NAME}
+    # wait until the add-on is available
+    heroku pg:wait -a ${YOUR_APP_NAME}
+
+4.  Create the necessary user roles according to the
+    `PostgREST documentation <https://postgrest.org/en/stable/auth.html>`_:
+
+  .. code-block:: bash
+
+    heroku pg:credentials:create --name api_user -a ${YOUR_APP_NAME}
+    # use the following command to ensure the new credential state is active before attaching it
+    heroku pg:credentials -a ${YOUR_APP_NAME}
+
+    heroku addons:attach ${HEROKU_PG_DB_NAME} --credential api_user -a ${YOUR_APP_NAME}
+
+5.  Connect to the PostgreSQL database and create some sample data:
+
+  .. code-block:: bash
+
+    heroku psql -a ${YOUR_APP_NAME}
+
+  .. code-block:: postgres
+
+    # from the psql command prompt execute the following commands:
+    create schema api;
+
+    create table api.todos (
+    id serial primary key,
+    done boolean not null default false,
+    task text not null,
+    due timestamptz
+    );
+
+    insert into api.todos (task) values
+    ('finish tutorial 0'), ('pat self on back');
+
+    grant usage on schema api to api_user;
+    grant select on api.todos to api_user;
+
+6.  Create the :code:`Procfile`:
+
+  .. code-block:: bash
+
+    web: PGRST_SERVER_HOST=0.0.0.0 PGRST_SERVER_PORT=${PORT} PGRST_DB_URI=${PGRST_DB_URI:-${DATABASE_URL}} ./postgrest-${POSTGREST_VER}
+  ..
+
+  Set the following environment variables on Heroku:
+
+  .. code-block:: bash
+
+    heroku config:set POSTGREST_VER=10.0.0
+    heroku config:set PGRST_DB_SCHEMA=api
+    heroku config:set PGRST_DB_ANON_ROLE=api_user
+  ..
+
+  PGRST_DB_URI can be set if an external database is used or if it's different from the default Heroku DATABASE_URL. This latter is used if nothing is provided.
+  POSTGREST_VER is mandatory to select and build the required PostgREST release.
+
+  See https://postgrest.org/en/stable/configuration.html#environment-variables for the full list of environment variables.
+
+7.  Build and deploy your app:
+
+  .. code-block:: bash
+
+    git add Procfile
+    git commit -m "PostgREST on Heroku"
+    git push heroku master
+  ..
+
+  Your Heroku app should be live at :code:`${YOUR_APP_NAME}.herokuapp.com`
+
+8.  Test your app
+
+    From a terminal display the application logs:
+
+  .. code-block:: bash
+
+    heroku logs -t
+  ..
+
+  From a different terminal retrieve with curl the records previously created:
+
+  .. code-block:: bash
+
+    curl https://${YOUR_APP_NAME}.herokuapp.com/todos
+  ..
+
+  and test that any attempt to modify the table via a read-only user is not allowed:
+
+  .. code-block:: bash
+
+    curl https://${YOUR_APP_NAME}.herokuapp.com/todos -X POST \
+     -H "Content-Type: application/json" \
+     -d '{"task": "do bad thing"}'

docs/integrations/jwt_gen.rst

@@ -0,0 +1,31 @@
+.. _external_jwt:
+
+External JWT Generation
+-----------------------
+
+JWT from Auth0
+~~~~~~~~~~~~~~
+
+An external service like `Auth0 <https://auth0.com/>`_ can do the hard work transforming OAuth from Github, Twitter, Google etc into a JWT suitable for PostgREST. Auth0 can also handle email signup and password reset flows.
+
+To use Auth0, create `an application <https://auth0.com/docs/get-started/applications>`_ for your app and `an API <https://auth0.com/docs/get-started/apis>`_ for your PostgREST server. Auth0 supports both HS256 and RS256 scheme for the issued tokens for APIs. For simplicity, you may first try HS256 scheme while creating your API on Auth0. Your application should use your PostgREST API's `API identifier <https://auth0.com/docs/get-started/apis/api-settings>`_ by setting it with the `audience parameter <https://auth0.com/docs/secure/tokens/access-tokens/get-access-tokens#control-access-token-audience>`_  during the authorization request. This will ensure that Auth0 will issue an access token for your PostgREST API. For PostgREST to verify the access token, you will need to set ``jwt-secret`` on PostgREST config file with your API's signing secret.
+
+.. note::
+
+  Our code requires a database role in the JWT. To add it you need to save the database role in Auth0 `app metadata <https://auth0.com/docs/manage-users/user-accounts/metadata/manage-metadata-rules>`_. Then, you will need to write `a rule <https://auth0.com/docs/customize/rules>`_ that will extract the role from the user's app_metadata and set it as a `custom claim <https://auth0.com/docs/get-started/apis/scopes/sample-use-cases-scopes-and-claims#add-custom-claims-to-a-token>`_ in the access token. Note that, you may use Auth0's `core authorization feature <https://auth0.com/docs/manage-users/access-control/rbac>`_ for more complex use cases. Metadata solution is mentioned here for simplicity.
+
+  .. code:: javascript
+
+    function (user, context, callback) {
+
+      // Follow the documentations at
+      // https://postgrest.org/en/latest/configuration.html#db-role-claim-key
+      // to set a custom role claim on PostgREST
+      // and use it as custom claim attribute in this rule
+      const myRoleClaim = 'https://myapp.com/role';
+
+      user.app_metadata = user.app_metadata || {};
+      context.accessToken[myRoleClaim] = user.app_metadata.role;
+      callback(null, user, context);
+    }
+

docs/integrations/pg-safeupdate.rst

@@ -0,0 +1,42 @@
+pg-safeupdate
+#############
+
+.. _block_fulltable:
+
+Block Full-Table Operations
+---------------------------
+
+If the :ref:`active role <user_impersonation>` can delete table rows then the DELETE verb is allowed for clients. Here's an API request to delete old rows from a hypothetical logs table:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    DELETE /logs?time=lt.1991-08-06 HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/logs?time=lt.1991-08-06" -X DELETE
+
+Note that it's very easy to delete the **entire table** by omitting the query parameter!
+
+.. tabs::
+
+  .. code-tab:: http
+
+    DELETE /logs HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/logs" -X DELETE
+
+This can happen accidentally such as by switching a request from a GET to a DELETE. To protect against accidental operations use the `pg-safeupdate <https://github.com/eradman/pg-safeupdate>`_ PostgreSQL extension. It raises an error if UPDATE or DELETE are executed without specifying conditions. To install it you can use the `PGXN <https://pgxn.org/>`_ network:
+
+.. code-block:: bash
+
+  sudo -E pgxn install safeupdate
+
+  # then add this to postgresql.conf:
+  # shared_preload_libraries='safeupdate';
+
+This does not protect against malicious actions, since someone can add a url parameter that does not affect the result set. To prevent this you must turn to database permissions, forbidding the wrong people from deleting rows, and using `row-level security <https://www.postgresql.org/docs/current/ddl-rowsecurity.html>`_ if finer access control is required.

docs/integrations/systemd.rst

@@ -0,0 +1,51 @@
+systemd
+=======
+
+For Linux distributions that use **systemd** (Ubuntu, Debian, Archlinux) you can create a daemon in the following way.
+
+First, create postgrest configuration in ``/etc/postgrest/config``
+
+.. code-block:: ini
+
+  db-uri = "postgres://<your_user>:<your_password>@localhost:5432/<your_db>"
+  db-schemas = "<your_exposed_schema>"
+  db-anon-role = "<your_anon_role>"
+  jwt-secret = "<your_secret>"
+
+Then create the systemd service file in ``/etc/systemd/system/postgrest.service``
+
+.. code-block:: ini
+
+  [Unit]
+  Description=REST API for any PostgreSQL database
+  After=postgresql.service
+
+  [Service]
+  ExecStart=/bin/postgrest /etc/postgrest/config
+  ExecReload=/bin/kill -SIGUSR1 $MAINPID
+
+  [Install]
+  WantedBy=multi-user.target
+
+After that, you can enable the service at boot time and start it with:
+
+.. code-block:: bash
+
+  systemctl enable postgrest
+  systemctl start postgrest
+
+  ## For reloading the service
+  ## systemctl restart postgrest
+
+.. _file_descriptors:
+
+File Descriptors
+----------------
+
+File descriptors are kernel resources that are used by HTTP connections (among others). File descriptors are limited per process. The kernel default limit is 1024, which is increased in some Linux distributions.
+When under heavy traffic, PostgREST can reach this limit and start showing ``No file descriptors available`` errors. To clear these errors, you can increase the process' file descriptor limit.
+
+.. code-block:: ini
+
+  [Service]
+  LimitNOFILE=10000

docs/postgrest.dict

@@ -1,8 +1,7 @@
 personal_ws-1.1 en 0 utf-8
-Adossi
-AMQP
 api
 API's
+APISIX
 Archlinux
 aud
 Auth
@@ -10,7 +9,6 @@ auth
 authenticator
 backoff
 balancer
-Beles
 booleans
 Bouscal
 buildpack
@@ -19,19 +17,18 @@ Cardano
 cd
 centric
 changelog
-ClojureScript
-cloudfared
+coercible
+conf
+Cloudflare
 config
 CORS
 CPUs
 cryptographically
 CSV
-Daemonizing
 DDL
 DevOps
 DiBiase
 dockerize
-DoS
 eq
 ETH
 Ethereum
@@ -52,7 +49,6 @@ gte
 GUC
 GUCs
 gucs
-Gumbs
 Haskell
 Heroku
 HMAC
@@ -62,43 +58,44 @@ HTTP
 HTTPS
 HV
 Ibarluzea
+Inlining
+inlined
+Integrations
+idletime
+IDLETIME
 ilike
 imatch
 io
 IP
+isdistinct
 JS
 js
 JSON
 JWK
 JWT
 jwt
-JWTs
-Kinesis
-Kofi
 Kubernetes
 localhost
 login
+lookups
 Logins
+LIBPQ
 logins
 lon
 lt
 lte
-middleware
 misprediction
-Mithril
 multi
-MVCC
 namespace
 namespaced
+Nanos
 neq
 nginx
-ngrep
 nixpkgs
 npm
 nxl
 nxr
 OAuth
-onwards
 OpenAPI
 openapi
 ORM
@@ -107,7 +104,6 @@ passphrase
 Pawel
 PBKDF
 Pelletier
-Petr
 PgBouncer
 pgcrypto
 pgjwt
@@ -131,6 +127,7 @@ postgrest
 PostgREST's
 pre
 preflight
+plpgsql
 psql
 Qin
 RabbitMQ
@@ -138,29 +135,27 @@ Rafaj
 RDS
 reallyreallyreallyreallyverysafe
 Rechkemmer
-reconnection
 Redux
 refactor
+reloadable
 Reloadable
 Remo
 requester's
 RESTful
-RestSharp
 RLS
 RPC
 RSA
 Saleeba
+safeupdate
 savepoint
 schemas
-Sencha
-Serverless
+schema's
 Severin
 SHA
+Sommer
 signup
 SIGUSR
 sl
-SNS
-sqitch
 SQL
 sql
 sr
@@ -168,16 +163,14 @@ SSL
 stateful
 stdout
 Stolarz
-subselect
-SuperAgent
+supervisees
 SvelteKit
 SwaggerUI
 syslog
 systemd
-Tcl
-tmuxp
 todo
 todos
+tos
 Tsingson
 tsquery
 tx
@@ -186,24 +179,26 @@ TypeScript
 UI
 ui
 unicode
+unikernel
 unix
 updatable
+Untyped
 UPSERT
 Upsert
 upsert
 uri
 url
+urlencoded
 urls
 variadic
-Vercel
 verifier
 versioning
 Vondra
 Vue
-WAI
 webhooks
 websearch
 Websockets
 webuser
 wfts
-ZeroMQ
+www
+Zac

docs/references/admin.rst

@@ -0,0 +1,280 @@
+.. _admin:
+
+Admin
+#####
+
+.. _pgrst_logging:
+
+Logging
+-------
+
+PostgREST logs basic request information to ``stdout``, including the authenticated user if available, the requesting IP address and user agent, the URL requested, and HTTP response status.
+
+.. code::
+
+   127.0.0.1 - user [26/Jul/2021:01:56:38 -0500] "GET /clients HTTP/1.1" 200 - "" "curl/7.64.0"
+   127.0.0.1 - anonymous [26/Jul/2021:01:56:48 -0500] "GET /unexistent HTTP/1.1" 404 - "" "curl/7.64.0"
+
+For diagnostic information about the server itself, PostgREST logs to ``stderr``.
+
+.. code::
+
+   12/Jun/2021:17:47:39 -0500: Starting PostgREST 11.1.0...
+   12/Jun/2021:17:47:39 -0500: Attempting to connect to the database...
+   12/Jun/2021:17:47:39 -0500: Listening on port 3000
+   12/Jun/2021:17:47:39 -0500: Connection successful
+   12/Jun/2021:17:47:39 -0500: Config re-loaded
+   12/Jun/2021:17:47:40 -0500: Schema cache loaded
+
+.. note::
+
+   When running it in an SSH session you must detach it from stdout or it will be terminated when the session closes. The easiest technique is redirecting the output to a log file or to the syslog:
+
+   .. code-block:: bash
+
+     ssh foo@example.com \
+       'postgrest foo.conf </dev/null >/var/log/postgrest.log 2>&1 &'
+
+     # another option is to pipe the output into "logger -t postgrest"
+
+Currently PostgREST doesn't log the SQL commands executed against the underlying database.
+
+Database Logs
+~~~~~~~~~~~~~
+
+To find the SQL operations, you can watch the database logs. By default PostgreSQL does not keep these logs, so you'll need to make the configuration changes below.
+
+Find :code:`postgresql.conf` inside your PostgreSQL data directory (to find that, issue the command :code:`show data_directory;`). Either find the settings scattered throughout the file and change them to the following values, or append this block of code to the end of the configuration file.
+
+.. code:: sql
+
+  # send logs where the collector can access them
+  log_destination = "stderr"
+
+  # collect stderr output to log files
+  logging_collector = on
+
+  # save logs in pg_log/ under the pg data directory
+  log_directory = "pg_log"
+
+  # (optional) new log file per day
+  log_filename = "postgresql-%Y-%m-%d.log"
+
+  # log every kind of SQL statement
+  log_statement = "all"
+
+Restart the database and watch the log file in real-time to understand how HTTP requests are being translated into SQL commands.
+
+.. note::
+
+  On Docker you can enable the logs by using a custom ``init.sh``:
+
+  .. code:: bash
+
+    #!/bin/sh
+    echo "log_statement = 'all'" >> /var/lib/postgresql/data/postgresql.conf
+
+  After that you can start the container and check the logs with ``docker logs``.
+
+  .. code:: bash
+
+    docker run -v "$(pwd)/init.sh":"/docker-entrypoint-initdb.d/init.sh" -d postgres
+    docker logs -f <container-id>
+
+Server Version
+--------------
+
+When debugging a problem it's important to verify the running PostgREST version. There are three ways to do this:
+
+- Look for the :code:`Server` HTTP response header that is returned on every request.
+
+.. code::
+
+  HEAD /users HTTP/1.1
+
+  Server: postgrest/11.0.1
+
+- Query ``application_name`` on `pg_stat_activity <https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-ACTIVITY-VIEW>`_.
+
+.. code-block:: psql
+
+  select distinct application_name
+  from pg_stat_activity
+  where application_name ilike '%postgrest%';
+
+        application_name
+  ------------------------------
+  PostgREST 11.1.0
+
+.. note::
+
+  The server sets the `fallback_application_name <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-FALLBACK-APPLICATION-NAME>`_ for this query to work. To override the value set ``application_name`` on the connection string.
+
+- The ``stderr`` logs also contain the version, as noted on :ref:`pgrst_logging`.
+
+.. _trace_header:
+
+Trace Header
+------------
+
+You can enable tracing HTTP requests by setting :ref:`server-trace-header`. Specify the set header in the request, and the server will include it in the response.
+
+.. code:: bash
+
+  server-trace-header = "X-Request-Id"
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /users HTTP/1.1
+
+    X-Request-Id: 123
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/users" \
+      -H "X-Request-Id: 123"
+
+.. code::
+
+  HTTP/1.1 200 OK
+  X-Request-Id: 123
+
+.. _explain_plan:
+
+Execution plan
+--------------
+
+You can get the `EXPLAIN execution plan <https://www.postgresql.org/docs/current/sql-explain.html>`_ of a request by adding the ``Accept: application/vnd.pgrst.plan`` header.
+This is enabled by :ref:`db-plan-enabled` (false by default).
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /users?select=name&order=id HTTP/1.1
+    Accept: application/vnd.pgrst.plan
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/users?select=name&order=id" \
+      -H "Accept: application/vnd.pgrst.plan"
+
+.. code-block:: psql
+
+  Aggregate  (cost=73.65..73.68 rows=1 width=112)
+    ->  Index Scan using users_pkey on users  (cost=0.15..60.90 rows=850 width=36)
+
+The output of the plan is generated in ``text`` format by default but you can change it to JSON by using the ``+json`` suffix.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /users?select=name&order=id HTTP/1.1
+    Accept: application/vnd.pgrst.plan+json
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/users?select=name&order=id" \
+      -H "Accept: application/vnd.pgrst.plan+json"
+
+.. code-block:: json
+
+  [
+    {
+      "Plan": {
+        "Node Type": "Aggregate",
+        "Strategy": "Plain",
+        "Partial Mode": "Simple",
+        "Parallel Aware": false,
+        "Async Capable": false,
+        "Startup Cost": 73.65,
+        "Total Cost": 73.68,
+        "Plan Rows": 1,
+        "Plan Width": 112,
+        "Plans": [
+          {
+            "Node Type": "Index Scan",
+            "Parent Relationship": "Outer",
+            "Parallel Aware": false,
+            "Async Capable": false,
+            "Scan Direction": "Forward",
+            "Index Name": "users_pkey",
+            "Relation Name": "users",
+            "Alias": "users",
+            "Startup Cost": 0.15,
+            "Total Cost": 60.90,
+            "Plan Rows": 850,
+            "Plan Width": 36
+          }
+        ]
+      }
+    }
+  ]
+
+By default the plan is assumed to generate the JSON representation of a resource(``application/json``), but you can obtain the plan for the :ref:`different representations that PostgREST supports <res_format>` by adding them to the ``for`` parameter. For instance, to obtain the plan for a ``text/xml``, you would use ``Accept: application/vnd.pgrst.plan; for="text/xml``.
+
+The other available parameters are ``analyze``, ``verbose``, ``settings``, ``buffers`` and ``wal``, which correspond to the `EXPLAIN command options <https://www.postgresql.org/docs/current/sql-explain.html>`_. To use the ``analyze`` and ``wal`` parameters for example, you would add them like ``Accept: application/vnd.pgrst.plan; options=analyze|wal``.
+
+Note that akin to the EXPLAIN command, the changes will be committed when using the ``analyze`` option. To avoid this, you can use the :ref:`db-tx-end` and the ``Prefer: tx=rollback`` header.
+
+Securing the Execution Plan
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's recommended to only activate :ref:`db-plan-enabled` on testing environments since it reveals internal database details.
+However, if you choose to use it in production you can add a :ref:`db-pre-request` to filter the requests that can use this feature.
+
+For example, to only allow requests from an IP address to get the execution plans:
+
+.. code-block:: postgresql
+
+ -- Assuming a proxy(Nginx, Cloudflare, etc) passes an "X-Forwarded-For" header(https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For)
+ create or replace function filter_plan_requests()
+ returns void as $$
+ declare
+   headers   json := current_setting('request.headers', true)::json;
+   client_ip text := coalesce(headers->>'x-forwarded-for', '');
+   accept    text := coalesce(headers->>'accept', '');
+ begin
+   if accept like 'application/vnd.pgrst.plan%' and client_ip != '144.96.121.73' then
+     raise insufficient_privilege using
+       message = 'Not allowed to use application/vnd.pgrst.plan';
+   end if;
+ end; $$ language plpgsql;
+
+ -- set this function on your postgrest.conf
+ -- db-pre-request = filter_plan_requests
+
+
+.. _health_check:
+
+Health Check
+------------
+
+You can enable a health check to verify if PostgREST is available for client requests. Also to check the status of its internal state.
+
+To do this, set the configuration variable :ref:`admin-server-port` to the port number of your preference. Two endpoints ``live`` and ``ready`` will then be available.
+
+The ``live`` endpoint verifies if PostgREST is running on its configured port. A request will return ``200 OK`` if PostgREST is alive or ``503`` otherwise.
+
+The ``ready`` endpoint also checks the state of both the Database Connection and the :ref:`schema_cache`. A request will return ``200 OK`` if it is ready or ``503`` if not.
+
+For instance, to verify if PostgREST is running at ``localhost:3000`` while the ``admin-server-port`` is set to ``3001``:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET localhost:3001/live HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl -I "http://localhost:3001/live"
+
+.. code-block:: http
+
+  HTTP/1.1 200 OK
+
+If you have a machine with multiple network interfaces and multiple PostgREST instances in the same port, you need to specify a unique :ref:`hostname <server-host>` in the configuration of each PostgREST instance for the health check to work correctly. Don't use the special values(``!4``, ``*``, etc) in this case because the health check could report a false positive.

docs/references/api.rst

@@ -0,0 +1,120 @@
+.. _api:
+
+API
+###
+
+PostgREST exposes three database objects of a schema as resources: tables, views and stored procedures.
+
+.. toctree::
+   :glob:
+   :maxdepth: 1
+
+   api/tables_views.rst
+   api/stored_procedures.rst
+   api/schemas.rst
+   api/computed_fields.rst
+   api/domain_representations.rst
+   api/resource_embedding.rst
+   api/resource_representation.rst
+   api/openapi.rst
+   api/*
+
+.. raw:: html
+
+  <script type="text/javascript">
+    let hash = window.location.hash;
+
+    const redirects = {
+      // Tables and Views
+      '#horizontal-filtering-rows': 'api/tables_views.html#horizontal-filtering-rows',
+      '#operators': 'api/tables_views.html#operators',
+      '#logical-operators': 'api/tables_views.html#logical-operators',
+      '#pattern-matching': 'api/tables_views.html#pattern-matching',
+      '#full-text-search': 'api/tables_views.html#full-text-search',
+      '#vertical-filtering-columns': 'api/tables_views.html#vertical-filtering-columns',
+      '#renaming-columns': 'api/tables_views.html#renaming-columns',
+      '#casting-columns': 'api/tables_views.html#casting-columns',
+      '#json-columns': 'api/tables_views.html#json-columns',
+      '#composite-array-columns': 'api/tables_views.html#composite-array-columns',
+      '#computed-virtual-columns': 'api/computed_fields.html#computed-fields',
+      '#ordering': 'api/tables_views.html#ordering',
+      '#limits-and-pagination': 'api/tables_views.html#limits-and-pagination',
+      '#exact-count': 'api/tables_views.html#exact-count',
+      '#planned-count': 'api/tables_views.html#planned-count',
+      '#estimated-count': 'api/tables_views.html#estimated-count',
+      '#updates': 'api/tables_views.html#update',
+      '#insertions': 'api/tables_views.html#insert',
+      '#bulk-insert': 'api/tables_views.html#bulk-insert',
+      '#specifying-columns': 'api/tables_views.html#specifying-columns',
+      '#upsert': 'api/tables_views.html#upsert',
+      '#on-conflict': 'api/tables_views.html#on-conflict',
+      '#put': 'api/tables_views.html#put',
+      '#deletions': 'api/tables_views.html#delete',
+      '#limited-updates-deletions': 'api/tables_views.html#limited-update-delete',
+      // Stored procedures
+      '#stored-procedures': 'api/stored_procedures.html#stored-procedures',
+      '#calling-functions-with-a-single-json-parameter': 'api/stored_procedures.html#functions-with-a-single-json-parameter',
+      '#calling-functions-with-a-single-unnamed-parameter': 'api/stored_procedures.html#functions-with-a-single-unnamed-parameter',
+      '#calling-functions-with-array-parameters': 'api/stored_procedures.html#functions-with-array-parameters',
+      '#calling-variadic-functions': 'api/stored_procedures.html#variadic-functions',
+      '#scalar-functions': 'api/stored_procedures.html#scalar-functions',
+      '#function-filters': 'api/stored_procedures.html#table-valued-functions',
+      '#overloaded-functions': 'api/stored_procedures.html#overloaded-functions',
+      // Schemas
+      '#switching-schemas': 'api/schemas.html',
+      // Resource Embedding
+      '#resource-embedding': 'api/resource_embedding.html#resource-embedding',
+      '#many-to-one-relationships': 'api/resource_embedding.html#many-to-one-relationships',
+      '#one-to-many-relationships': 'api/resource_embedding.html#one-to-many-relationships',
+      '#many-to-many-relationships': 'api/resource_embedding.html#many-to-many-relationships',
+      '#one-to-one-relationships': 'api/resource_embedding.html#one-to-one-relationships',
+      '#computed-relationships': 'api/resource_embedding.html#computed-relationships',
+      '#nested-embedding': 'api/resource_embedding.html#nested-embedding',
+      '#embedded-filters': 'api/resource_embedding.html#embedded-filters',
+      '#embedding-with-top-level-filtering': 'api/resource_embedding.html#top-level-filtering',
+      '#embedding-partitioned-tables': 'api/resource_embedding.html#embedding-partitioned-tables',
+      '#embedding-views': 'api/resource_embedding.html#embedding-views',
+      '#embedding-chains-of-views': 'api/resource_embedding.html#embedding-chains-of-views',
+      '#embedding-on-stored-procedures': 'api/resource_embedding.html#embedding-on-stored-procedures',
+      '#embedding-after-insertions-updates-deletions': 'api/resource_embedding.html#embedding-after-insertions-updates-deletions',
+      '#embedding-disambiguation': 'api/resource_embedding.html#embedding-disambiguation',
+      '#target-disambiguation': 'api/resource_embedding.html#target-disambiguation',
+      '#hint-disambiguation': 'api/resource_embedding.html#hint-disambiguation',
+      "#embedding-through-join-tables": "api/resource_embedding.html#many-to-many-relationships",
+      // OpenAPI
+      '#openapi-support': 'api/openapi.html',
+      // Resource Representation
+      '#response-format': 'api/resource_representation.html#response-format',
+      '#singular-or-plural': 'api/resource_representation.html#singular-or-plural',
+      '#response-formats-for-scalar-responses': 'api/resource_representation.html#scalar-function-response-format',
+      // CORS
+      '#cors': 'api/cors.html',
+      // OPTIONS
+      '#options': 'api/options.html',
+      // URL Grammar
+      '#custom-queries': 'api/url_grammar.html#custom-queries',
+      '#unicode-support': 'api/url_grammar.html#unicode-support',
+      '#table-columns-with-spaces': 'api/url_grammar.html#table-columns-with-spaces',
+      '#reserved-characters': 'api/url_grammar.html#reserved-characters',
+      // Transactions
+      '#immutable-and-stable-functions': 'transactions.html#access-mode',
+      '#http-context': 'transactions.html#transaction-scoped-settings',
+      '#accessing-request-headers-cookies-and-jwt-claims': 'transactions.html#request-headers-cookies-and-jwt-claims',
+      '#legacy-guc-variable-names': 'transactions.html#legacy-settings',
+      '#accessing-request-path-and-method': 'transactions.html#request-path-and-method',
+      '#setting-response-headers': 'transactions.html#response-headers',
+      '#setting-headers-via-pre-request': 'transactions.html#setting-headers-via-pre-request',
+      '#setting-response-status-code': 'transactions.html#response-status-code',
+      '#raise-errors-with-http-status-codes': 'transactions.html#raise-errors-with-http-status-codes',
+      // Admin
+      '#execution-plan': 'admin.html#execution-plan',
+      // Deprecated
+      '#bulk-call': '../releases/v11.0.1.html#breaking-changes',
+    };
+
+    let willRedirectTo = redirects[hash];
+
+    if (willRedirectTo) {
+      window.location.href = willRedirectTo;
+    }
+  </script>

docs/references/api/computed_fields.rst

@@ -0,0 +1,93 @@
+.. _computed_cols:
+
+Computed Fields
+###############
+
+Computed fields are virtual columns that are not stored in a table. PostgreSQL makes it possible to implement them using functions on table types.
+
+.. code-block:: postgres
+
+  CREATE TABLE people (
+    first_name text
+  , last_name  text
+  , job        text
+  );
+
+  -- a computed field that combines data from two columns
+  CREATE FUNCTION full_name(people)
+  RETURNS text AS $$
+    SELECT $1.first_name || ' ' || $1.last_name;
+  $$ LANGUAGE SQL;
+
+Horizontal Filtering on Computed Fields
+=======================================
+
+:ref:`h_filter` can be applied to computed fields. For example, we can do a :ref:`fts` on :code:`full_name`:
+
+.. code-block:: postgres
+
+  -- (optional) you can add an index on the computed field to speed up the query
+  CREATE INDEX people_full_name_idx ON people
+    USING GIN (to_tsvector('english', full_name(people)));
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /people?full_name=fts.Beckett HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/people?full_name=fts.Beckett"
+
+.. code-block:: json
+
+  [
+    {"first_name": "Samuel", "last_name": "Beckett", "job": "novelist"}
+  ]
+
+Vertical Filtering on Computed Fields
+=====================================
+
+Computed fields won't appear on the response by default but you can use :ref:`v_filter` to include them:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /people?select=full_name,job HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/people?select=full_name,job"
+
+.. code-block:: json
+
+  [
+    {"full_name": "Samuel Beckett", "job": "novelist"}
+  ]
+
+Ordering on Computed Fields
+===========================
+
+:ref:`ordering` on computed fields is also possible:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /people?order=full_name.desc HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/people?order=full_name.desc"
+
+.. important::
+
+  Computed columns must be created in the :ref:`exposed schema <db-schemas>` or in a schema in the :ref:`extra search path <db-extra-search-path>` to be used in this way. When placing the computed column in the :ref:`exposed schema <db-schemas>` you can use an **unnamed** parameter, as in the example above, to prevent it from being exposed as an :ref:`RPC <s_procs>` under ``/rpc``.
+
+.. note::
+
+   - PostgreSQL 12 introduced `generated columns <https://www.postgresql.org/docs/12/ddl-generated-columns.html>`_, which can also compute a value based on other columns. However they're stored, not virtual.
+   - "computed fields" are documented on https://www.postgresql.org/docs/current/rowtypes.html#ROWTYPES-USAGE (search for "computed fields")
+   - On previous PostgREST versions this feature was documented with the name of "computed columns".

docs/references/api/cors.rst

@@ -0,0 +1,34 @@
+CORS
+====
+
+PostgREST sets highly permissive cross origin resource sharing, that is why it accepts Ajax requests from any domain.
+
+It also handles `preflight requests <https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request>`_ done by the browser, which are cached using the returned ``Access-Control-Max-Age: 86400`` header (86400 seconds = 24 hours). This is useful to reduce the latency of the subsequent requests.
+
+A ``POST`` preflight request would look like this:
+
+.. tabs::
+
+  .. code-tab:: http
+
+     OPTIONS /items HTTP/1.1
+     Origin: http://example.com
+     Access-Control-Allow-Method: POST
+     Access-Control-Allow-Headers: Content-Type
+
+  .. code-tab:: bash Curl
+
+     curl -i "http://localhost:3000/items" \
+       -X OPTIONS \
+       -H "Origin: http://example.com" \
+       -H "Access-Control-Request-Method: POST" \
+       -H "Access-Control-Request-Headers: Content-Type"
+
+.. code-block:: http
+
+  HTTP/1.1 200 OK
+  Access-Control-Allow-Origin: http://example.com
+  Access-Control-Allow-Credentials: true
+  Access-Control-Allow-Methods: GET, POST, PATCH, PUT, DELETE, OPTIONS, HEAD
+  Access-Control-Allow-Headers: Authorization, Content-Type, Accept, Accept-Language, Content-Language
+  Access-Control-Max-Age: 86400

docs/references/api/domain_representations.rst

@@ -0,0 +1,200 @@
+.. _domain_reps:
+
+Domain Representations
+######################
+
+Domain Representations separates "how the data is presented" from "how the data is stored". It works by creating `domains <https://www.postgresql.org/docs/current/sql-createdomain.html>`_ and `casts <https://www.postgresql.org/docs/current/sql-createcast.html>`_, the latter act on the former to present and receive the data in different formats.
+
+.. contents::
+   :depth: 1
+   :local:
+   :backlinks: none
+
+Custom Domain
+=============
+
+Suppose you want to use a ``uuid`` type for a primary key and want to present it shortened to web users.
+
+For this, let's create a domain based on ``uuid``.
+
+.. code-block:: postgres
+
+  create domain app_uuid as uuid;
+
+  -- and use it as our table PK.
+  create table profiles(
+    id   app_uuid
+  , name text
+  );
+
+  -- some data for the example
+  insert into profiles values ('846c4ffd-92ce-4de7-8d11-8e29929f4ec4', 'John Doe');
+
+Domain Response Format
+======================
+
+We can shorten the ``uuid`` with ``base64`` encoding. Let's use JSON as our response format for this example.
+
+To change the domain format for JSON, create a function that converts ``app_uuid`` to ``json``.
+
+.. code-block:: postgres
+
+  -- the name of the function is arbitrary
+  CREATE OR REPLACE FUNCTION json(app_uuid) RETURNS json AS $$
+    select to_json(encode(uuid_send($1),'base64'));
+  $$ LANGUAGE SQL IMMUTABLE;
+
+  -- check it works
+  select json('846c4ffd-92ce-4de7-8d11-8e29929f4ec4'::app_uuid);
+              json
+  ----------------------------
+   "hGxP/ZLOTeeNEY4pkp9OxA=="
+
+Then create a CAST to tell PostgREST to convert it automatically whenever a JSON response is requested.
+
+.. code-block:: postgres
+
+  CREATE CAST (app_uuid AS json) WITH FUNCTION json(app_uuid) AS IMPLICIT;
+
+With this you can obtain the data in the shortened format.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /profiles HTTP/1.1
+    Accept: application/json
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/profiles" \
+      -H "Accept: application/json"
+
+.. code-block:: json
+
+  [{"id":"hGxP/ZLOTeeNEY4pkp9OxA==","name":"John Doe"}]
+
+.. note::
+
+  - Casts on domains are ignored by PostgreSQL, their interpretation is left to the application. We're discussing the possibility of including the Domain Representations behavior on `pgsql-hackers <https://www.postgresql.org/message-id/flat/CAGRrpzZKa%2BGu91j1SOvN3tM1f-7Gh_w441c5nAX1QqdH3Q31Lg%40mail.gmail.com>`_.
+  - It would make more sense to use ``base58`` encoding as it's URL friendly but for simplicity we use ``base64`` (supported natively in PostgreSQL).
+
+.. important::
+
+  After creating a cast over a domain, you must refresh PostgREST schema cache. See :ref:`schema_reloading`.
+
+Domain Filter Format
+====================
+
+For :ref:`h_filter` to work with the shortened format, you need a different conversion.
+
+PostgREST considers the URL query string to be, in the most generic sense, ``text``. So let's create a function that converts ``text`` to ``app_uuid``.
+
+.. code-block:: postgres
+
+  -- the name of the function is arbitrary
+  CREATE OR REPLACE FUNCTION app_uuid(text) RETURNS app_uuid AS $$
+    select substring(decode($1,'base64')::text from 3)::uuid;
+  $$ LANGUAGE SQL IMMUTABLE;
+
+  -- plus a CAST to tell PostgREST to use this function
+  CREATE CAST (text AS app_uuid) WITH FUNCTION app_uuid(text) AS IMPLICIT;
+
+Now you can filter as usual.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /profiles?id=eq.hGxP/ZLOTeeNEY4pkp9OxA== HTTP/1.1
+    Accept: application/json
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/profiles?id=eq.ZLOTeeNEY4pkp9OxA==" \
+      -H "Accept: application/json"
+
+.. code-block:: json
+
+  [{"id":"hGxP/ZLOTeeNEY4pkp9OxA==","name":"John Doe"}]
+
+.. note::
+
+  If there's no CAST from ``text`` to ``app_uuid`` defined, the filter will still work with the native uuid format (``846c4ffd-92ce-4de7-8d11-8e29929f4ec4``).
+
+Domain Request Body Format
+==========================
+
+To accept the shortened format in a JSON request body, for example when creating a new record, define a ``json`` to ``app_uuid`` conversion.
+
+.. code-block:: postgres
+
+  -- the name of the function is arbitrary
+  CREATE OR REPLACE FUNCTION app_uuid(json) RETURNS public.app_uuid AS $$
+    -- here we reuse the previous app_uuid(text) function
+    select app_uuid($1 #>> '{}');
+  $$ LANGUAGE SQL IMMUTABLE;
+
+  CREATE CAST (json AS public.app_uuid) WITH FUNCTION app_uuid(json) AS IMPLICIT;
+
+Now we can :ref:`insert` (or :ref:`update`) as usual.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /profiles HTTP/1.1
+    Content-Type: application/json
+    Prefer: return=representation
+
+    {"id":"zH7HbFJUTfy/GZpwuirpuQ==","name":"Jane Doe"}
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/profiles" \
+      -H "Prefer: return=representation" \
+      -H "Content-Type: application/json" \
+      -d @- <<JSON
+
+    {"id":"zH7HbFJUTfy/GZpwuirpuQ==","name":"Jane Doe"}
+
+    JSON
+
+The response:
+
+.. code-block:: json
+
+  [{"id":"zH7HbFJUTfy/GZpwuirpuQ==","name":"Jane Doe"}]
+
+Note that on the database side we have our regular ``uuid`` format.
+
+.. code-block:: postgres
+
+  select * from profiles;
+
+                    id                  |   name
+  --------------------------------------+----------
+   846c4ffd-92ce-4de7-8d11-8e29929f4ec4 | John Doe
+   cc7ec76c-5254-4dfc-bf19-9a70ba2ae9b9 | Jane Doe
+  (2 rows)
+
+.. note::
+
+  If there's no CAST from ``json`` to ``app_uuid`` defined, the request body will still work with the native uuid format (``cc7ec76c-5254-4dfc-bf19-9a70ba2ae9b9``).
+
+Advantages over Views
+=====================
+
+`Views <https://www.postgresql.org/docs/current/sql-createview.html>`_ also allow us to change the format of the underlying type. However they come with drawbacks that increase complexity.
+
+1) Formatting the column in the view makes it `non-updatable <https://www.postgresql.org/docs/current/sql-createview.html#SQL-CREATEVIEW-UPDATABLE-VIEWS>`_ since Postgres doesn't know how to reverse the transform. This can be worked around using INSTEAD OF triggers.
+2) When filtering by this column, we get full table scans for the same reason (also applies to :ref:`computed_cols`) . The performance loss here can be avoided with a computed index, or using a materialized generated column.
+3) If the formatted column is used as a foreign key, PostgREST can no longer detect that relationship and :ref:`resource_embedding` breaks. This can be worked around with :ref:`computed_relationships`.
+
+Domain Representations avoid all the above drawbacks. Their only drawback is that for existing tables, you have to change the column types. But this should be a fast operation since domains are binary coercible with their underlying types. A table rewrite won't be required.
+
+.. note::
+
+  Why not create a `base type <https://www.postgresql.org/docs/current/sql-createtype.html#id-1.9.3.94.5.8>`_ instead? ``CREATE TYPE app_uuid (INTERNALLENGTH = 22, INPUT = app_uuid_parser, OUTPUT = app_uuid_formatter)``.
+
+  Creating base types need superuser, which is restricted on cloud hosted databases. Additionally this way lets “how the data is presented” dictate “how the data is stored” which would be backwards.

docs/references/api/openapi.rst

@@ -0,0 +1,104 @@
+.. _open-api:
+
+OpenAPI
+=======
+
+PostgREST automatically serves a full `OpenAPI <https://www.openapis.org/>`_ description on the root path. This provides a list of all endpoints (tables, foreign tables, views, functions), along with supported HTTP verbs and example payloads.
+
+.. note::
+
+  By default, this output depends on the permissions of the role that is contained in the JWT role claim (or the :ref:`db-anon-role` if no JWT is sent). If you need to show all the endpoints disregarding the role's permissions, set the :ref:`openapi-mode` config to :code:`ignore-privileges`.
+
+For extra customization, the OpenAPI output contains a "description" field for every `SQL comment <https://www.postgresql.org/docs/current/sql-comment.html>`_ on any database object. For instance,
+
+.. code-block:: sql
+
+  COMMENT ON SCHEMA mammals IS
+    'A warm-blooded vertebrate animal of a class that is distinguished by the secretion of milk by females for the nourishment of the young';
+
+  COMMENT ON TABLE monotremes IS
+    'Freakish mammals lay the best eggs for breakfast';
+
+  COMMENT ON COLUMN monotremes.has_venomous_claw IS
+    'Sometimes breakfast is not worth it';
+
+These unsavory comments will appear in the generated JSON as the fields, ``info.description``, ``definitions.monotremes.description`` and ``definitions.monotremes.properties.has_venomous_claw.description``.
+
+Also if you wish to generate a ``summary`` field you can do it by having a multiple line comment, the ``summary`` will be the first line and the ``description`` the lines that follow it:
+
+.. code-block:: plpgsql
+
+  COMMENT ON TABLE entities IS
+    $$Entities summary
+
+    Entities description that
+    spans
+    multiple lines$$;
+
+Similarly, you can override the API title by commenting the schema.
+
+.. code-block:: plpgsql
+
+  COMMENT ON SCHEMA api IS
+  $$FooBar API
+
+  A RESTful API that serves FooBar data.$$;
+
+If you need to include the ``security`` and ``securityDefinitions`` options, set the :ref:`openapi-security-active` configuration to ``true``.
+
+You can use a tool like `Swagger UI <https://swagger.io/tools/swagger-ui/>`_ to create beautiful documentation from the description and to host an interactive web-based dashboard. The dashboard allows developers to make requests against a live PostgREST server, and provides guidance with request headers and example request bodies.
+
+.. important::
+
+  The OpenAPI information can go out of date as the schema changes under a running server. See :ref:`schema_reloading`.
+
+.. _override_openapi:
+
+Overriding Full OpenAPI Response
+--------------------------------
+
+You can override the whole default response with a function result. To do this, set the function on :ref:`db-root-spec`.
+
+.. code:: bash
+
+   db-root-spec = "root"
+
+.. code:: postgres
+
+  create or replace function root() returns json as $_$
+  declare
+  openapi json = $$
+    {
+      "swagger": "2.0",
+      "info":{
+        "title":"Overridden",
+        "description":"This is a my own API"
+      }
+    }
+  $$;
+  begin
+    return openapi;
+  end
+  $_$ language plpgsql;
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET / HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl http://localhost:3000
+
+.. code-block:: http
+
+  HTTP/1.1 200 OK
+
+  {
+    "swagger": "2.0",
+    "info":{
+      "title":"Overridden",
+      "description":"This is a my own API"
+    }
+  }

docs/references/api/options.rst

@@ -0,0 +1,52 @@
+.. _options_requests:
+
+OPTIONS method
+==============
+
+You can verify which HTTP methods are allowed on endpoints for tables and views by using an OPTIONS request. These methods are allowed depending on what operations *can* be done on the table or view, not on the database permissions assigned to them.
+
+For a table named ``people``, OPTIONS would show:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    OPTIONS /people HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/people" -X OPTIONS -i
+
+.. code-block:: http
+
+  HTTP/1.1 200 OK
+  Allow: OPTIONS,GET,HEAD,POST,PUT,PATCH,DELETE
+
+For a view, the methods are determined by the presence of INSTEAD OF TRIGGERS.
+
+.. table::
+   :widths: auto
+
+   +--------------------+-------------------------------------------------------------------------------------------------+
+   | Method allowed     | View's requirements                                                                             |
+   +====================+=================================================================================================+
+   | OPTIONS, GET, HEAD | None (Always allowed)                                                                           |
+   +--------------------+-------------------------------------------------------------------------------------------------+
+   | POST               | INSTEAD OF INSERT TRIGGER                                                                       |
+   +--------------------+-------------------------------------------------------------------------------------------------+
+   | PUT                | INSTEAD OF INSERT TRIGGER, INSTEAD OF UPDATE TRIGGER, also requires the presence of a           |
+   |                    | primary key                                                                                     |
+   +--------------------+-------------------------------------------------------------------------------------------------+
+   | PATCH              | INSTEAD OF UPDATE TRIGGER                                                                       |
+   +--------------------+-------------------------------------------------------------------------------------------------+
+   | DELETE             | INSTEAD OF DELETE TRIGGER                                                                       |
+   +--------------------+-------------------------------------------------------------------------------------------------+
+   | All the above methods are allowed for                                                                                |
+   | `auto-updatable views <https://www.postgresql.org/docs/current/sql-createview.html#SQL-CREATEVIEW-UPDATABLE-VIEWS>`_ |
+   +--------------------+-------------------------------------------------------------------------------------------------+
+
+For functions, the methods depend on their volatility. ``VOLATILE`` functions allow only ``OPTIONS,POST``, whereas the rest also permit ``GET,HEAD``.
+
+.. important::
+
+  Whenever you add or remove tables or views, or modify a view's INSTEAD OF TRIGGERS on the database, you must refresh PostgREST's schema cache for OPTIONS requests to work properly. See the section :ref:`schema_reloading`.

docs/references/api/resource_representation.rst

@@ -0,0 +1,210 @@
+Resource Representation
+#######################
+
+PostgREST uses proper HTTP content negotiation (`RFC7231 <https://datatracker.ietf.org/doc/html/rfc7231#section-5.3>`_) to deliver a resource representation.
+That is to say the same API endpoint can respond in different formats like JSON or CSV depending on the request.
+
+.. _res_format:
+
+Response Format
+===============
+
+Use the Accept request header to specify the acceptable format (or formats) for the response:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /people HTTP/1.1
+    Accept: application/json
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/people" \
+      -H "Accept: application/json"
+
+For tables and views the current possibilities are:
+
+* ``*/*``
+* ``text/csv``
+* ``application/json``
+* ``application/openapi+json``
+* ``application/geo+json``
+
+The server will default to JSON for API endpoints and OpenAPI on the root.
+
+.. _singular_plural:
+
+Singular or Plural
+------------------
+
+By default PostgREST returns all JSON results in an array, even when there is only one item. For example, requesting :code:`/items?id=eq.1` returns
+
+.. code:: json
+
+  [
+    { "id": 1 }
+  ]
+
+This can be inconvenient for client code. To return the first result as an object unenclosed by an array, specify :code:`vnd.pgrst.object` as part of the :code:`Accept` header
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /items?id=eq.1 HTTP/1.1
+    Accept: application/vnd.pgrst.object+json
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/items?id=eq.1" \
+      -H "Accept: application/vnd.pgrst.object+json"
+
+This returns
+
+.. code:: json
+
+  { "id": 1 }
+
+with a :code:`Content-Type: application/vnd.pgrst.object+json`.
+
+When a singular response is requested but no entries are found, the server responds with an error message and 406 Not Acceptable status code rather than the usual empty array and 200 status:
+
+.. code-block:: json
+
+  {
+    "message": "JSON object requested, multiple (or no) rows returned",
+    "details": "Results contain 0 rows, application/vnd.pgrst.object+json requires 1 row",
+    "hint": null,
+    "code": "PGRST505"
+  }
+
+.. note::
+
+  Many APIs distinguish plural and singular resources using a special nested URL convention e.g. `/stories` vs `/stories/1`. Why do we use `/stories?id=eq.1`? The answer is because a singular resource is (for us) a row determined by a primary key, and primary keys can be compound (meaning defined across more than one column). The more familiar nested urls consider only a degenerate case of simple and overwhelmingly numeric primary keys. These so-called artificial keys are often introduced automatically by Object Relational Mapping libraries.
+
+  Admittedly PostgREST could detect when there is an equality condition holding on all columns constituting the primary key and automatically convert to singular. However this could lead to a surprising change of format that breaks unwary client code just by filtering on an extra column. Instead we allow manually specifying singular vs plural to decouple that choice from the URL format.
+
+Stripped Nulls
+--------------
+
+By default PostgREST returns all JSON null values. For example, requesting ``/projects?id=gt.10`` returns
+
+.. code:: json
+
+  [
+    { "id": 11, "name": "OSX",      "client_id": 1,    "another_col": "val" },
+    { "id": 12, "name": "ProjectX", "client_id": null, "another_col": null },
+    { "id": 13, "name": "Y",        "client_id": null, "another_col": null }
+  ]
+
+On large result sets, the unused keys with ``null`` values can waste bandwith unnecessarily. To remove them, specify ``nulls=stripped`` as a parameter of ``application/vnd.pgrst.array``:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /projects?id=gt.10 HTTP/1.1
+    Accept: application/vnd.pgrst.array+json;nulls=stripped
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/projects?id=gt.10" \
+      -H "Accept: application/vnd.pgrst.array+json;nulls=stripped"
+
+This returns
+
+.. code:: json
+
+  [
+    { "id": 11, "name": "OSX", "client_id": 1, "another_col": "val" },
+    { "id": 12, "name": "ProjectX" },
+    { "id": 13, "name": "Y"}
+  ]
+
+.. _scalar_return_formats:
+
+Scalar Function Response Format
+-------------------------------
+
+In the special case of a :ref:`scalar_functions` there are three additional formats:
+
+* ``application/octet-stream``
+* ``text/plain``
+* ``text/xml``
+
+Example 1: If you want to return raw binary data from a :code:`bytea` column, you must specify :code:`application/octet-stream` as part of the :code:`Accept` header
+and select a single column :code:`?select=bin_data`.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /items?select=bin_data&id=eq.1 HTTP/1.1
+    Accept: application/octet-stream
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/items?select=bin_data&id=eq.1" \
+      -H "Accept: application/octet-stream"
+
+Example 2: You can request XML output when having a scalar function that returns a type of ``text/xml``. You are not forced to use select for this case.
+
+.. code-block:: postgres
+
+  CREATE FUNCTION generate_xml_content(..) RETURNS xml ..
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/generate_xml_content HTTP/1.1
+    Accept: text/xml
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/generate_xml_content" \
+      -X POST -H "Accept: text/xml"
+
+Example 3: If the stored procedure returns non-scalar values, you need to do a :code:`select` in the same way as for GET binary output.
+
+.. code-block:: sql
+
+  CREATE FUNCTION get_descriptions(..) RETURNS SETOF TABLE(id int, description text) ..
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/get_descriptions?select=description HTTP/1.1
+    Accept: text/plain
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/get_descriptions?select=description" \
+      -X POST -H "Accept: text/plain"
+
+.. note::
+
+  If more than one row would be returned the binary/plain-text/xml results will be concatenated with no delimiter.
+
+.. _req_body:
+
+Request Body
+============
+
+The server handles the following request body media types:
+
+* ``application/json``
+* ``application/x-www-form-urlencoded``
+* ``text/csv``
+
+For :ref:`tables_views` this works on ``POST``, ``PATCH`` and ``PUT`` methods. For :ref:`s_procs`, it works on ``POST`` methods.
+
+For stored procedures there are three additional types:
+
+* ``application/octet-stream``
+* ``text/plain``
+* ``text/xml``
+
+See :ref:`s_proc_single_unnamed`.

docs/references/api/schemas.rst

@@ -0,0 +1,156 @@
+.. _schemas:
+
+Schemas
+=======
+
+PostgREST can expose a single or multiple schema's tables, views and functions. The :ref:`active database role <roles>` must have the usage privilege on the schemas to access them.
+
+Single schema
+-------------
+
+To expose a single schema, specify a single value in :ref:`db-schemas`.
+
+.. code:: bash
+
+   db-schemas = "api"
+
+This schema is added to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request using :ref:`tx_settings`.
+
+.. _multiple-schemas:
+
+Multiple schemas
+----------------
+
+To expose multiple schemas, specify a comma-separated list on :ref:`db-schemas`:
+
+.. code:: bash
+
+   db-schemas = "tenant1, tenant2"
+
+To switch schemas, use the ``Accept-Profile`` and ``Content-Profile`` headers.
+
+If you don't specify a Profile header, the first schema in the list(``tenant1`` here) is selected as the default schema.
+
+Only the selected schema gets added to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request.
+
+.. note::
+
+   These headers are based on the "Content Negotiation by Profile" spec: https://www.w3.org/TR/dx-prof-conneg
+
+GET/HEAD
+~~~~~~~~
+
+For GET or HEAD, select the schema with ``Accept-Profile``.
+
+.. tabs::
+
+  .. code-tab:: http
+
+     GET /items HTTP/1.1
+     Accept-Profile: tenant2
+
+  .. code-tab:: bash Curl
+
+     curl "http://localhost:3000/items" \
+       -H "Accept-Profile: tenant2"
+
+Other methods
+~~~~~~~~~~~~~
+
+For POST, PATCH, PUT and DELETE, select the schema with ``Content-Profile``.
+
+.. tabs::
+
+  .. code-tab:: http
+
+     POST /items HTTP/1.1
+     Content-Profile: tenant2
+
+     {...}
+
+  .. code-tab:: bash Curl
+
+     curl "http://localhost:3000/items" \
+       -X POST -H "Content-Type: application/json" \
+       -H "Content-Profile: tenant2" \
+       -d '{...}'
+
+You can also select the schema for :ref:`s_procs` and :ref:`open-api`.
+
+Restricted schemas
+~~~~~~~~~~~~~~~~~~
+
+You can only switch to a schema included in :ref:`db-schemas`. Using another schema will result in an error:
+
+.. tabs::
+
+  .. code-tab:: http
+
+     GET /items HTTP/1.1
+     Accept-Profile: tenant3
+
+     {...}
+
+  .. code-tab:: bash Curl
+
+     curl "http://localhost:3000/items" \
+       -H "Accept-Profile: tenant3"
+
+.. code-block::
+
+  {
+    "code":"PGRST106",
+    "details":null,
+    "hint":null,
+    "message":"The schema must be one of the following: tenant1, tenant2"
+  }
+
+
+Dynamic schemas
+~~~~~~~~~~~~~~~
+
+To add schemas dynamically, you can use :ref:`in_db_config` plus :ref:`config reloading <config_reloading_notify>` and :ref:`schema cache reloading <schema_reloading_notify>`. Here are some options for how to do this:
+
+- If the schemas' names have a pattern, like a ``tenant_`` prefix, do:
+
+.. code-block:: postgresql
+
+  create or replace function postgrest.pre_config()
+  returns void as $$
+    select
+      set_config('pgrst.db_schemas', string_agg(nspname, ','), true)
+    from pg_namespace
+    where nspname like 'tenant_%';
+  $$ language sql;
+
+- If there's no name pattern but they're created with a particular role (``CREATE SCHEMA mine AUTHORIZATION joe``), do:
+
+.. code-block:: postgresql
+
+  create or replace function postgrest.pre_config()
+  returns void as $$
+    select
+      set_config('pgrst.db_schemas', string_agg(nspname, ','), true)
+    from pg_namespace
+    where nspowner = 'joe'::regrole;
+  $$ language sql;
+
+- Otherwise, you might need to create a table that stores the allowed schemas.
+
+.. code-block:: postgresql
+
+  create table postgrest.config (schemas text);
+
+  create or replace function postgrest.pre_config()
+  returns void as $$
+    select
+      set_config('pgrst.db_schemas', schemas, true)
+    from postgrest.config;
+  $$ language sql;
+
+Then each time you add an schema, do:
+
+.. code-block:: postgresql
+
+   NOTIFY pgrst, 'reload config';
+   NOTIFY pgrst, 'reload schema';

docs/references/api/stored_procedures.rst

@@ -0,0 +1,489 @@
+.. _s_procs:
+
+Stored Procedures
+=================
+
+*"A single resource can be the equivalent of a database stored procedure, with the power to abstract state changes over any number of storage items"* -- `Roy T. Fielding <http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven#comment-743>`_
+
+Procedures can perform any operations allowed by PostgreSQL (read data, modify data, :ref:`raise errors <raise_error>`, and even DDL operations). Every stored procedure in the :ref:`exposed schema <schemas>` and accessible by the :ref:`active database role <roles>` is executable under the :code:`/rpc` prefix.
+
+If they return table types, Stored Procedures can:
+
+- Use all the same :ref:`read filters as Tables and Views <read>` (horizontal/vertical filtering, counts, limits, etc.).
+- Use :ref:`Resource Embedding <s_proc_embed>`, if the returned table type has relationships to other tables.
+
+.. note::
+
+  Why the ``/rpc`` prefix? PostgreSQL allows a table or view to have the same name as a function. The prefix allows us to avoid routes collisions.
+
+Calling with POST
+-----------------
+
+To supply arguments in an API call, include a JSON object in the request payload. Each key/value of the object will become an argument.
+
+For instance, assume we have created this function in the database.
+
+.. code-block:: plpgsql
+
+  CREATE FUNCTION add_them(a integer, b integer)
+  RETURNS integer AS $$
+   SELECT a + b;
+  $$ LANGUAGE SQL IMMUTABLE;
+
+.. important::
+
+  Whenever you create or change a function you must refresh PostgREST's schema cache. See the section :ref:`schema_reloading`.
+
+The client can call it by posting an object like
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/add_them HTTP/1.1
+
+    { "a": 1, "b": 2 }
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/add_them" \
+      -X POST -H "Content-Type: application/json" \
+      -d '{ "a": 1, "b": 2 }'
+
+.. code-block:: json
+
+  3
+
+.. note::
+
+  PostgreSQL converts identifier names to lowercase unless you quote them like:
+
+  .. code-block:: postgres
+
+    CREATE FUNCTION "someFunc"("someParam" text) ...
+
+Calling with GET
+----------------
+
+If the function doesn't modify the database, it will also run under the GET method (see :ref:`access_mode`).
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/add_them?a=1&b=2 HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/add_them?a=1&b=2"
+
+The function parameter names match the JSON object keys in the POST case, for the GET case they match the query parameters ``?a=1&b=2``.
+
+.. _s_proc_single_json:
+
+Functions with a single JSON parameter
+--------------------------------------
+
+You can also call a function that takes a single parameter of type JSON by sending the header :code:`Prefer: params=single-object` with your request. That way the JSON request body will be used as the single argument.
+
+.. code-block:: plpgsql
+
+  CREATE FUNCTION mult_them(param json) RETURNS int AS $$
+    SELECT (param->>'x')::int * (param->>'y')::int
+  $$ LANGUAGE SQL;
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/mult_them HTTP/1.1
+    Prefer: params=single-object
+
+    { "x": 4, "y": 2 }
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/mult_them" \
+      -X POST -H "Content-Type: application/json" \
+      -H "Prefer: params=single-object" \
+      -d '{ "x": 4, "y": 2 }'
+
+.. code-block:: json
+
+  8
+
+.. _s_proc_single_unnamed:
+
+Functions with a single unnamed parameter
+-----------------------------------------
+
+You can make a POST request to a function with a single unnamed parameter to send raw ``json/jsonb``, ``bytea``, ``text`` or ``xml`` data.
+
+To send raw JSON, the function must have a single unnamed ``json`` or ``jsonb`` parameter and the header ``Content-Type: application/json`` must be included in the request.
+
+.. code-block:: plpgsql
+
+  CREATE FUNCTION mult_them(json) RETURNS int AS $$
+    SELECT ($1->>'x')::int * ($1->>'y')::int
+  $$ LANGUAGE SQL;
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/mult_them HTTP/1.1
+    Content-Type: application/json
+
+    { "x": 4, "y": 2 }
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/mult_them" \
+      -X POST -H "Content-Type: application/json" \
+      -d '{ "x": 4, "y": 2 }'
+
+.. code-block:: json
+
+  8
+
+.. note::
+
+  If an overloaded function has a single ``json`` or ``jsonb`` unnamed parameter, PostgREST will call this function as a fallback provided that no other overloaded function is found with the parameters sent in the POST request.
+
+To send raw XML, the parameter type must be ``xml`` and the header ``Content-Type: text/xml`` must be included in the request.
+
+To send raw binary, the parameter type must be ``bytea`` and the header ``Content-Type: application/octet-stream`` must be included in the request.
+
+.. code-block:: plpgsql
+
+  CREATE TABLE files(blob bytea);
+
+  CREATE FUNCTION upload_binary(bytea) RETURNS void AS $$
+    INSERT INTO files(blob) VALUES ($1);
+  $$ LANGUAGE SQL;
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/upload_binary HTTP/1.1
+    Content-Type: application/octet-stream
+
+    file_name.ext
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/upload_binary" \
+      -X POST -H "Content-Type: application/octet-stream" \
+      --data-binary "@file_name.ext"
+
+.. code-block:: http
+
+  HTTP/1.1 200 OK
+
+  [ ... ]
+
+To send raw text, the parameter type must be ``text`` and the header ``Content-Type: text/plain`` must be included in the request.
+
+.. _s_procs_array:
+
+Functions with array parameters
+-------------------------------
+
+You can call a function that takes an array parameter:
+
+.. code-block:: postgres
+
+   create function plus_one(arr int[]) returns int[] as $$
+      SELECT array_agg(n + 1) FROM unnest($1) AS n;
+   $$ language sql;
+
+.. tabs::
+
+  .. code-tab:: http
+
+     POST /rpc/plus_one HTTP/1.1
+     Content-Type: application/json
+
+     {"arr": [1,2,3,4]}
+
+  .. code-tab:: bash Curl
+
+     curl "http://localhost:3000/rpc/plus_one" \
+       -X POST -H "Content-Type: application/json" \
+       -d '{"arr": [1,2,3,4]}'
+
+.. code-block:: json
+
+   [2,3,4,5]
+
+For calling the function with GET, you can pass the array as an `array literal <https://www.postgresql.org/docs/current/arrays.html#ARRAYS-INPUT>`_,
+as in ``{1,2,3,4}``. Note that the curly brackets have to be urlencoded(``{`` is ``%7B`` and ``}`` is ``%7D``).
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/plus_one?arr=%7B1,2,3,4%7D' HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/plus_one?arr=%7B1,2,3,4%7D'"
+
+.. note::
+
+   For versions prior to PostgreSQL 10, to pass a PostgreSQL native array on a POST payload, you need to quote it and use an array literal:
+
+   .. tabs::
+
+     .. code-tab:: http
+
+       POST /rpc/plus_one HTTP/1.1
+
+       { "arr": "{1,2,3,4}" }
+
+     .. code-tab:: bash Curl
+
+       curl "http://localhost:3000/rpc/plus_one" \
+         -X POST -H "Content-Type: application/json" \
+         -d '{ "arr": "{1,2,3,4}" }'
+
+   In these versions we recommend using function parameters of type JSON to accept arrays from the client.
+
+.. _s_procs_variadic:
+
+Variadic functions
+------------------
+
+You can call a variadic function by passing a JSON array in a POST request:
+
+.. code-block:: postgres
+
+   create function plus_one(variadic v int[]) returns int[] as $$
+      SELECT array_agg(n + 1) FROM unnest($1) AS n;
+   $$ language sql;
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/plus_one HTTP/1.1
+    Content-Type: application/json
+
+    {"v": [1,2,3,4]}
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/plus_one" \
+      -X POST -H "Content-Type: application/json" \
+      -d '{"v": [1,2,3,4]}'
+
+.. code-block:: json
+
+   [2,3,4,5]
+
+In a GET request, you can repeat the same parameter name:
+
+.. tabs::
+
+  .. code-tab:: http
+
+     GET /rpc/plus_one?v=1&v=2&v=3&v=4 HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+     curl "http://localhost:3000/rpc/plus_one?v=1&v=2&v=3&v=4"
+
+Repeating also works in POST requests with ``Content-Type: application/x-www-form-urlencoded``:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    POST /rpc/plus_one HTTP/1.1
+    Content-Type: application/x-www-form-urlencoded
+
+    v=1&v=2&v=3&v=4
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/plus_one" \
+      -X POST -H "Content-Type: application/x-www-form-urlencoded" \
+      -d 'v=1&v=2&v=3&v=4'
+
+Table-Valued functions
+----------------------
+
+A function that returns a table type can be filtered using the same filters as :ref:`tables and views <tables_views>`. They can also use :ref:`Resource Embedding <s_proc_embed>`.
+
+.. code-block:: postgres
+
+  CREATE FUNCTION best_films_2017() RETURNS SETOF films ..
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/best_films_2017?select=title,director:directors(*) HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/best_films_2017?select=title,director:directors(*)"
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/best_films_2017?rating=gt.8&order=title.desc HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/best_films_2017?rating=gt.8&order=title.desc"
+
+.. _function_inlining:
+
+Function Inlining
+~~~~~~~~~~~~~~~~~
+
+A function that follows the `rules for inlining <https://wiki.postgresql.org/wiki/Inlining_of_SQL_functions#Inlining_conditions_for_table_functions>`_ will also inline :ref:`filters <h_filter>`, :ref:`order <ordering>` and :ref:`limits <limits>`.
+
+For example, for the following function:
+
+.. code-block:: postgres
+
+  create function getallprojects() returns setof projects
+  language sql stable
+  as $$
+    select * from projects;
+  $$;
+
+Let's get its :ref:`explain_plan` when calling it with filters applied:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/getallprojects?id=eq.1 HTTP/1.1
+    Accept: application/vnd.pgrst.plan
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/getallprojects?id=eq.1" \
+      -H "Accept: application/vnd.pgrst.plan"
+
+.. code-block:: psql
+
+  Aggregate  (cost=8.18..8.20 rows=1 width=112)
+    ->  Index Scan using projects_pkey on projects  (cost=0.15..8.17 rows=1 width=40)
+          Index Cond: (id = 1)
+
+Notice there's no "Function Scan" node in the plan, which tells us it has been inlined.
+
+.. _scalar_functions:
+
+Scalar functions
+----------------
+
+PostgREST will detect if the function is scalar or table-valued and will shape the response format accordingly:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/add_them?a=1&b=2 HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/add_them?a=1&b=2"
+
+.. code-block:: json
+
+  3
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/best_films_2017 HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/best_films_2017"
+
+.. code-block:: json
+
+  [
+    { "title": "Okja", "rating": 7.4},
+    { "title": "Call me by your name", "rating": 8},
+    { "title": "Blade Runner 2049", "rating": 8.1}
+  ]
+
+To manually choose a return format such as binary, plain text or XML, see the section :ref:`scalar_return_formats`.
+
+.. _untyped_functions:
+
+Untyped functions
+-----------------
+
+Functions that return ``record`` or ``SETOF record`` are supported:
+
+.. code-block:: postgres
+
+  create function projects_setof_record() returns setof record as $$
+    select * from projects;
+  $$ language sql;
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/projects_setof_record HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/projects_setof_record"
+
+.. code-block:: json
+
+  [{"id":1,"name":"Windows 7","client_id":1},
+   {"id":2,"name":"Windows 10","client_id":1},
+   {"id":3,"name":"IOS","client_id":2}]
+
+However note that they will fail when trying to use :ref:`v_filter` and :ref:`h_filter` on them.
+
+So while they can be used for quick tests, it's recommended to always choose a strict return type for the function.
+
+Overloaded functions
+--------------------
+
+You can call overloaded functions with different number of arguments.
+
+.. code-block:: postgres
+
+  CREATE FUNCTION rental_duration(customer_id integer) ..
+
+  CREATE FUNCTION rental_duration(customer_id integer, from_date date) ..
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/rental_duration?customer_id=232 HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/rental_duration?customer_id=232"
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/rental_duration?customer_id=232&from_date=2018-07-01 HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/rental_duration?customer_id=232&from_date=2018-07-01"
+
+.. important::
+
+  Overloaded functions with the same argument names but different types are not supported.

docs/references/api/url_grammar.rst

@@ -0,0 +1,108 @@
+.. note::
+
+  This page is a work in progress.
+
+URL Grammar
+===========
+
+.. _custom_queries:
+
+Custom Queries
+--------------
+
+The PostgREST URL grammar limits the kinds of queries clients can perform. It prevents arbitrary, potentially poorly constructed and slow client queries. It's good for quality of service, but means database administrators must create custom views and stored procedures to provide richer endpoints. The most common causes for custom endpoints are
+
+* Table unions
+* More complicated joins than those provided by :ref:`resource_embedding`.
+* Geo-spatial queries that require an argument, like "points near (lat,lon)"
+
+Unicode support
+---------------
+
+PostgREST supports unicode in schemas, tables, columns and values. To access a table with unicode name, use percent encoding.
+
+To request this:
+
+.. code-block:: http
+
+  GET /موارد HTTP/1.1
+
+Do this:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /%D9%85%D9%88%D8%A7%D8%B1%D8%AF HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/%D9%85%D9%88%D8%A7%D8%B1%D8%AF"
+
+.. _tabs-cols-w-spaces:
+
+Table / Columns with spaces
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can request table/columns with spaces in them by percent encoding the spaces with ``%20``:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /Order%20Items?Unit%20Price=lt.200 HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/Order%20Items?Unit%20Price=lt.200"
+
+.. _reserved-chars:
+
+Reserved characters
+~~~~~~~~~~~~~~~~~~~
+
+If filters include PostgREST reserved characters(``,``, ``.``, ``:``, ``()``) you'll have to surround them in percent encoded double quotes ``%22`` for correct processing.
+
+Here ``Hebdon,John`` and ``Williams,Mary`` are values.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /employees?name=in.(%22Hebdon,John%22,%22Williams,Mary%22) HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/employees?name=in.(%22Hebdon,John%22,%22Williams,Mary%22)"
+
+Here ``information.cpe`` is a column name.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /vulnerabilities?%22information.cpe%22=like.*MS* HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/vulnerabilities?%22information.cpe%22=like.*MS*"
+
+If the value filtered by the ``in`` operator has a double quote (``"``), you can escape it using a backslash ``"\""``. A backslash itself can be used with a double backslash ``"\\"``.
+
+Here ``Quote:"`` and ``Backslash:\`` are percent-encoded values. Note that ``%5C`` is the percent-encoded backslash.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /marks?name=in.(%22Quote:%5C%22%22,%22Backslash:%5C%5C%22) HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/marks?name=in.(%22Quote:%5C%22%22,%22Backslash:%5C%5C%22)"
+
+.. note::
+
+   Some HTTP libraries might encode URLs automatically(e.g. :code:`axios`). In these cases you should use double quotes
+   :code:`""` directly instead of :code:`%22`.
+

docs/references/auth.rst

@@ -0,0 +1,216 @@
+.. _authn:
+
+Authentication
+==============
+
+PostgREST is designed to keep the database at the center of API security. All :ref:`authorization happens in the database <db_authz>` . It is PostgREST's job to **authenticate** requests -- i.e. verify that a client is who they say they are -- and then let the database **authorize** client actions.
+
+.. _roles:
+
+Overview of role system
+-----------------------
+
+There are three types of roles used by PostgREST, the **authenticator**, **anonymous** and **user** roles. The database administrator creates these roles and configures PostgREST to use them.
+
+.. image:: ../_static/security-roles.png
+
+The authenticator role is used for connecting to the database and should be configured to have very limited access. It is a chameleon whose job is to "become" other users to service authenticated HTTP requests.
+
+
+.. code:: sql
+
+
+  CREATE ROLE authenticator LOGIN NOINHERIT NOCREATEDB NOCREATEROLE NOSUPERUSER;
+
+.. note::
+
+  The names "authenticator" and "anon" names are configurable and not sacred, we simply choose them for clarity. See :ref:`db-uri` and :ref:`db-anon-role`.
+
+.. _user_impersonation:
+
+User Impersonation
+------------------
+
+The picture below shows how the server handles authentication. If auth succeeds, it switches into the user role specified by the request, otherwise it switches into the anonymous role (if it's set in :ref:`db-anon-role`).
+
+.. image:: ../_static/security-anon-choice.png
+
+This role switching mechanism is called **user impersonation**. In PostgreSQL it's done with the ``SET ROLE`` statement.
+
+.. _impersonated_settings:
+
+Impersonated Role Settings
+--------------------------
+
+The impersonated role has its settings applied. For example, if you do:
+
+.. code-block:: postgresql
+
+  ALTER ROLE webuser SET statement_timeout TO '5s';
+
+Every ``webuser`` :ref:`transaction <transactions>` gets its queries executed with a ``statement_timeout`` of 5 seconds.
+
+.. note::
+
+   Settings that have a high privilege context (like ``superuser``) won't be applied, only settings that have a ``user`` context will be. This is so we don't cause permission errors.
+   For more details see `Understanding Postgres Parameter Context <https://www.enterprisedb.com/blog/understanding-postgres-parameter-context>`_.
+
+.. _jwt_impersonation:
+
+JWT-Based User Impersonation
+----------------------------
+
+We use `JSON Web Tokens <https://jwt.io/>`_ to authenticate API requests, this allows us to be stateless and not require database lookups for verification. As you'll recall a JWT contains a list of cryptographically signed claims. All claims are allowed but PostgREST cares specifically about a claim called role.
+
+.. code:: json
+
+  {
+    "role": "user123"
+  }
+
+When a request contains a valid JWT with a role claim PostgREST will switch to the database role with that name for the duration of the HTTP request.
+
+.. code:: sql
+
+  SET LOCAL ROLE user123;
+
+Note that the database administrator must allow the authenticator role to switch into this user by previously executing
+
+.. code:: sql
+
+  GRANT user123 TO authenticator;
+  -- similarly for the anonymous role
+  -- GRANT anonymous TO authenticator;
+
+If the client included no JWT (or one without a role claim) then PostgREST switches into the anonymous role. The database administrator must set the anonymous role permissions correctly to prevent anonymous users from seeing or changing things they shouldn't.
+
+.. _jwt_generation:
+
+JWT Generation
+~~~~~~~~~~~~~~
+
+You can create a valid JWT either from inside your database (see :ref:`sql_user_management`) or via an external service (see :ref:`external_jwt`).
+
+.. _client_auth:
+
+Client Auth
+~~~~~~~~~~~
+
+To make an authenticated request the client must include an :code:`Authorization` HTTP header with the value :code:`Bearer <jwt>`. For instance:
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /foo HTTP/1.1
+    Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiamRvZSIsImV4cCI6MTQ3NTUxNjI1MH0.GYDZV3yM0gqvuEtJmfpplLBXSGYnke_Pvnl0tbKAjB4
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/foo" \
+      -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiamRvZSIsImV4cCI6MTQ3NTUxNjI1MH0.GYDZV3yM0gqvuEtJmfpplLBXSGYnke_Pvnl0tbKAjB4"
+
+The ``Bearer`` header value can be used with or without capitalization(``bearer``).
+
+Symmetric Keys
+~~~~~~~~~~~~~~
+
+Each token is cryptographically signed with a secret key. In the case of symmetric cryptography the signer and verifier share the same secret passphrase, which can be configured with :ref:`jwt-secret`.
+If it is set to a simple string value like “reallyreallyreallyreallyverysafe” then PostgREST interprets it as an HMAC-SHA256 passphrase.
+
+.. _asym_keys:
+
+Asymmetric Keys
+~~~~~~~~~~~~~~~
+
+In asymmetric cryptography the signer uses the private key and the verifier the public key.
+
+As described in the :ref:`configuration` section, PostgREST accepts a ``jwt-secret`` config file parameter. However you can also specify a literal JSON Web Key (JWK) or set. For example, you can use an RSA-256 public key encoded as a JWK:
+
+.. code-block:: json
+
+  {
+    "alg":"RS256",
+    "e":"AQAB",
+    "key_ops":["verify"],
+    "kty":"RSA",
+    "n":"9zKNYTaYGfGm1tBMpRT6FxOYrM720GhXdettc02uyakYSEHU2IJz90G_MLlEl4-WWWYoS_QKFupw3s7aPYlaAjamG22rAnvWu-rRkP5sSSkKvud_IgKL4iE6Y2WJx2Bkl1XUFkdZ8wlEUR6O1ft3TS4uA-qKifSZ43CahzAJyUezOH9shI--tirC028lNg767ldEki3WnVr3zokSujC9YJ_9XXjw2hFBfmJUrNb0-wldvxQbFU8RPXip-GQ_JPTrCTZhrzGFeWPvhA6Rqmc3b1PhM9jY7Dur1sjYWYVyXlFNCK3c-6feo5WlRfe1aCWmwZQh6O18eTmLeT4nWYkDzQ"
+  }
+
+.. note::
+
+  This could also be a JSON Web Key Set (JWKS) if it was contained within an array assigned to a `keys` member, e.g. ``{ keys: [jwk1, jwk2] }``.
+
+Just pass it in as a single line string, escaping the quotes:
+
+.. code-block:: ini
+
+  jwt-secret = "{ \"alg\":\"RS256\", … }"
+
+To generate such a public/private key pair use a utility like `latchset/jose <https://github.com/latchset/jose>`_.
+
+.. code-block:: bash
+
+  jose jwk gen -i '{"alg": "RS256"}' -o rsa.jwk
+  jose jwk pub -i rsa.jwk -o rsa.jwk.pub
+
+  # now rsa.jwk.pub contains the desired JSON object
+
+You can specify the literal value as we saw earlier, or reference a filename to load the JWK from a file:
+
+.. code-block:: ini
+
+  jwt-secret = "@rsa.jwk.pub"
+
+JWT Claims Validation
+~~~~~~~~~~~~~~~~~~~~~
+
+PostgREST honors the following `JWT claims <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.4>`_:
+
+- ``exp`` Expiration Time
+- ``iat`` Issued At
+- ``nbf`` Not Before
+- ``aud`` Audience, see :ref:`jwt-aud`
+
+.. note::
+  PostgREST allows for a 30-second clock skew when validating the ``exp`` and ``iat`` claims. In other words, it gives an extra 30 seconds before the token is rejected if there is a slight discrepancy in the timestamps.
+
+JWT Security
+~~~~~~~~~~~~
+
+There are at least three types of common critiques against using JWT: 1) against the standard itself, 2) against using libraries with known security vulnerabilities, and 3) against using JWT for web sessions. We'll briefly explain each critique, how PostgREST deals with it, and give recommendations for appropriate user action.
+
+The critique against the `JWT standard <https://datatracker.ietf.org/doc/html/rfc7519>`_ is voiced in detail `elsewhere on the web <https://web.archive.org/web/20230123041631/https://paragonie.com/blog/2017/03/jwt-json-web-tokens-is-bad-standard-that-everyone-should-avoid>`_. The most relevant part for PostgREST is the so-called :code:`alg=none` issue. Some servers implementing JWT allow clients to choose the algorithm used to sign the JWT. In this case, an attacker could set the algorithm to :code:`none`, remove the need for any signature at all and gain unauthorized access. The current implementation of PostgREST, however, does not allow clients to set the signature algorithm in the HTTP request, making this attack irrelevant. The critique against the standard is that it requires the implementation of the :code:`alg=none` at all.
+
+Critiques against JWT libraries are only relevant to PostgREST via the library it uses. As mentioned above, not allowing clients to choose the signature algorithm in HTTP requests removes the greatest risk. Another more subtle attack is possible where servers use asymmetric algorithms like RSA for signatures. Once again this is not relevant to PostgREST since it is not supported. Curious readers can find more information in `this article <https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/>`_. Recommendations about high quality libraries for usage in API clients can be found on `jwt.io <https://jwt.io/>`_.
+
+The last type of critique focuses on the misuse of JWT for maintaining web sessions. The basic recommendation is to `stop using JWT for sessions <http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-for-sessions/>`_ because most, if not all, solutions to the problems that arise when you do, `do not work <http://cryto.net/~joepie91/blog/2016/06/19/stop-using-jwt-for-sessions-part-2-why-your-solution-doesnt-work/>`_. The linked articles discuss the problems in depth but the essence of the problem is that JWT is not designed to be secure and stateful units for client-side storage and therefore not suited to session management.
+
+PostgREST uses JWT mainly for authentication and authorization purposes and encourages users to do the same. For web sessions, using cookies over HTTPS is good enough and well catered for by standard web frameworks.
+
+.. _custom_validation:
+
+Custom Validation
+-----------------
+
+PostgREST does not enforce any extra constraints besides JWT validation. An example of an extra constraint would be to immediately revoke access for a certain user. Using :ref:`db-pre-request` you can specify a stored procedure to call immediately after :ref:`user_impersonation` and before the main query itself runs.
+
+.. code:: ini
+
+  db-pre-request = "public.check_user"
+
+In the function you can run arbitrary code to check the request and raise an exception(see :ref:`raise_error`) to block it if desired. Here you can take advantage of :ref:`guc_req_headers_cookies_claims` for
+doing custom logic based on the web user info.
+
+.. code-block:: postgres
+
+  CREATE OR REPLACE FUNCTION check_user() RETURNS void AS $$
+  DECLARE
+    email text := current_setting('request.jwt.claims', true)::json->>'email';
+  BEGIN
+    IF email = 'evil.user@malicious.com' THEN
+      RAISE EXCEPTION 'No, you are evil'
+        USING HINT = 'Stop being so evil and maybe you can log in';
+    END IF;
+  END
+  $$ LANGUAGE plpgsql;

docs/references/configuration.rst

@@ -1,35 +1,33 @@
 .. _configuration:
 
 Configuration
-=============
+#############
 
-Without configuration, PostgREST won't be able to serve requests. At the minimum it needs either :ref:`a role to serve anonymous requests with <db-anon-role>` - or :ref:`a secret to use for JWT authentication <jwt-secret>`. Config parameters can be provided via :ref:`file_config`, via :ref:`env_variables_config` or through :ref:`in_db_config`.
+Configuration parameters can be provided via:
 
-To connect to a database it uses a `libpq connection string <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_. The connection string can be set in the configuration file or via environment variable or can be read from an external file. See :ref:`db-uri` for details. Any parameter that is not set in the connection string is read from `libpq environment variables <https://www.postgresql.org/docs/current/libpq-envars.html>`_. The default connection string is ``postgresql://``, which reads **all** parameters from the environment.
+- :ref:`file_config`.
+- :ref:`env_variables_config`, overriding values from the config file.
+- :ref:`in_db_config`, overriding values from both the config file and environment variables.
 
-The user with whom PostgREST connects to the database is also known as the authenticator role. For more information about the anonymous vs authenticator roles see :ref:`roles`.
+Using :ref:`config_reloading` you can modify the parameters without restarting the server.
 
-Config parameters are read in the following order:
 
-1. From the config file.
-2. From environment variables, overriding values from the config file.
-3. From the database, overriding values from both the config file and environment variables.
+Minimum parameters
+==================
+
+The server is able to start without any config parameters, but it won't be able to serve requests unless it has :ref:`a role to serve anonymous requests with <db-anon-role>` - or :ref:`a secret to use for JWT authentication <jwt-secret>`.
 
 .. _file_config:
 
 Config File
------------
+===========
 
-PostgREST can read a config file. There is no predefined location for this file, you must specify the file path as the one and only argument to the server:
+There is no predefined location for the config file, you must specify the file path as the one and only argument to the server:
 
 .. code:: bash
 
   ./postgrest /path/to/postgrest.conf
 
-.. note::
-
-   Configuration can be reloaded without restarting the server. See :ref:`config_reloading`.
-
 The configuration file must contain a set of key value pairs:
 
 .. code::
@@ -47,50 +45,67 @@ The configuration file must contain a set of key value pairs:
   # The secret to verify the JWT for authenticated requests with.
   # Needs to be 32 characters minimum.
   jwt-secret           = "reallyreallyreallyreallyverysafe"
-  jwt-secret-is-base64 = False
+  jwt-secret-is-base64 = false
 
   # Port the postgrest process is listening on for http requests
-  server-port = 80
+  server-port = 3000
 
 You can run ``postgrest --example`` to display all possible configuration parameters and how to use them in a configuration file.
 
 .. _env_variables_config:
 
 Environment Variables
----------------------
+=====================
 
-You can also set these :ref:`configuration parameters <config_full_list>` using environment variables. They are capitalized, have a ``PGRST_`` prefix, and use underscores. For example: ``PGRST_DB_URI`` corresponds to ``db-uri`` and ``PGRST_APP_SETTINGS_*`` to ``app.settings.*``.
+Environment variables are capitalized, have a ``PGRST_`` prefix, and use underscores. For example: ``PGRST_DB_URI`` corresponds to ``db-uri`` and ``PGRST_APP_SETTINGS_*`` to ``app.settings.*``.
+
+`libpq environment variables <https://www.postgresql.org/docs/current/libpq-envars.html>`_ are also supported for constructing the connection string, see :ref:`db-uri`.
+
+See the full list of environment variable names on :ref:`config_full_list`.
 
 .. _in_db_config:
 
 In-Database Configuration
--------------------------
+=========================
 
-By adding settings to the **authenticator** role (see :ref:`roles`), you can make the database the single source of truth for PostgREST's configuration.
-This is enabled by :ref:`db-config`.
+You can also configure the server with database settings by using a :ref:`pre-config <db-pre-config>` function. For example, you can configure :ref:`db-schemas` and :ref:`jwt-secret` like this:
 
-For example, you can configure :ref:`db-schemas` and :ref:`jwt-secret` like this:
+.. code-block::
 
-.. code:: postgresql
+  # postgrest.conf
 
-   ALTER ROLE authenticator SET pgrst.db_schemas = "tenant1, tenant2, tenant3"
-   ALTER ROLE authenticator IN DATABASE <your_database_name> SET pgrst.jwt_secret = "REALLYREALLYREALLYREALLYVERYSAFE"
+  db-pre-config  = "postgrest.pre_config"
 
-You can use both database-specific settings with `IN DATABASE` and cluster-wide settings without it. Database-specific settings will override cluster-wide settings if both are used for the same parameter.
+  # or env vars
 
-Note that underscores(``_``) need to be used instead of dashes(``-``) for the in-database config parameters.
+  PGRST_DB_PRE_CONFIG = "postgrest.pre_config"
 
-.. important::
+.. code-block:: postgresql
 
-   For altering a role in this way, you need a SUPERUSER. You might not be able to use this configuration mode on cloud-hosted databases.
+  -- create a dedicated schema, hidden from the API
+  create schema postgrest;
+  -- grant usage on this schema to the authenticator
+  grant usage on schema postgrest to authenticator;
 
-When using both the configuration file and the in-database configuration, the latter takes precedence.
+  -- the function can configure postgREST by using set_config
+  create or replace function postgrest.pre_config()
+  returns void as $$
+    select
+        set_config('pgrst.db_schemas', 'schema1, schema2', true)
+      , set_config('pgrst.jwt_secret', 'REALLYREALLYREALLYREALLYVERYSAFE', true);
+  $$ language sql;
 
-.. danger::
+Note that underscores(``_``) need to be used instead of dashes(``-``) for the in-database config parameters. See the full list of in-database names on :ref:`config_full_list`.
 
-  If direct connections to the database are allowed, then it's not safe to use the in-db configuration for storing the :ref:`jwt-secret`.
-  The settings of every role are PUBLIC - they can be viewed by any user that queries the ``pg_catalog.pg_db_role_setting`` table.
-  In this case you should keep the :ref:`jwt-secret` in the configuration file or as environment variables.
+You can disable the in-database configuration by setting :ref:`db-config` to ``false``.
+
+.. note::
+  For backwards compatibility, you can do in-db config by modifying the :ref:`authenticator role <roles>`. This is no longer recommended as it requires SUPERUSER.
+
+  .. code:: postgresql
+
+     ALTER ROLE authenticator SET pgrst.db_schemas = "tenant1, tenant2, tenant3"
+     ALTER ROLE authenticator IN DATABASE <your_database_name> SET pgrst.db_schemas = "tenant4, tenant5" -- database-specific setting, overrides the previous setting
 
 .. _config_reloading:
 
@@ -99,21 +114,10 @@ Configuration Reloading
 
 It's possible to reload PostgREST's configuration without restarting the server. You can do this :ref:`via signal <config_reloading_signal>` or :ref:`via notification <config_reloading_notify>`.
 
-It's not possible to change :ref:`env_variables_config` for a running process and reloading a Docker container configuration will not work. In these cases, you need to restart the PostgREST server or use :ref:`in_db_config` as an alternative.
-
-.. important::
-
-  The following settings will not be reloaded. You will need to restart PostgREST to change those.
-
-    * :ref:`admin-server-port`
-    * :ref:`db-uri`
-    * :ref:`db-pool`
-    * :ref:`db-pool-acquisition-timeout`
-    * :ref:`db-pool-max-lifetime`
-    * :ref:`server-host`
-    * :ref:`server-port`
-    * :ref:`server-unix-socket`
-    * :ref:`server-unix-socket-mode`
+- Any modification to the :ref:`file_config` will be applied during reload.
+- Any modification to the :ref:`in_db_config` will be applied during reload.
+- Not all settings are reloadable, see the reloadable list on :ref:`config_full_list`.
+- It's not possible to change :ref:`env_variables_config` for a running process, hence reloading a Docker container configuration will not work. In these cases, you can restart the process or use :ref:`in_db_config`.
 
 .. _config_reloading_signal:
 
@@ -137,70 +141,40 @@ To reload the configuration from within the database, you can use a NOTIFY comma
 
    NOTIFY pgrst, 'reload config'
 
-The ``"pgrst"`` notification channel is enabled by default. For configuring the channel, see :ref:`db-channel` and :ref:`db-channel-enabled`.
+The ``"pgrst"`` notification channel is enabled by default. You can name the channel with :ref:`db-channel` and enable or disable it with :ref:`db-channel-enabled`.
 
 .. _config_full_list:
 
 List of parameters
 ==================
 
-=========================== ======= ================= ==========
-Name                        Type    Default           Reloadable
-=========================== ======= ================= ==========
-admin-server-port           Int
-app.settings.*              String                    Y
-db-anon-role                String                    Y
-db-channel                  String  pgrst             Y
-db-channel-enabled          Boolean True              Y
-db-config                   Boolean True              Y
-db-extra-search-path        String  public            Y
-db-max-rows                 Int     ∞                 Y
-db-plan-enabled             Boolean False             Y
-db-pool                     Int     10
-db-pool-acquisition-timeout Int     10
-db-pool-max-lifetime        Int     1800
-db-pre-request              String                    Y
-db-prepared-statements      Boolean True              Y
-db-schemas                  String  public            Y
-db-tx-end                   String  commit
-db-uri                      String  postgresql://
-db-use-legacy-gucs          Boolean True              Y
-jwt-aud                     String                    Y
-jwt-role-claim-key          String  .role             Y
-jwt-secret                  String                    Y
-jwt-secret-is-base64        Boolean False             Y
-log-level                   String  error             Y
-openapi-mode                String  follow-privileges Y
-openapi-security-active     Boolean False             Y
-openapi-server-proxy-uri    String                    Y
-raw-media-types             String                    Y
-server-host                 String  !4
-server-port                 Int     3000
-server-unix-socket          String
-server-unix-socket-mode     String  660
-=========================== ======= ================= ==========
-
 .. _admin-server-port:
 
 admin-server-port
 -----------------
 
   =============== =======================
+  **Type**        Int
+  **Default**     `n/a`
+  **Reloadable**  N
   **Environment** PGRST_ADMIN_SERVER_PORT
   **In-Database** `n/a`
   =============== =======================
 
-Specifies the port for the :ref:`health_check` endpoints.
+  Specifies the port for the :ref:`health_check` endpoints.
 
 .. _app.settings.*:
 
 app.settings.*
 --------------
 
-  =============== ====================
+  =============== =======================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  &
   **Environment** PGRST_APP_SETTINGS_*
-  **In-Database** pgrst.app_settings_*
-  =============== ====================
+  **In-Database** `n/a`
+  =============== =======================
 
   Arbitrary settings that can be used to pass in secret keys directly as strings, or via OS environment variables. For instance: :code:`app.settings.jwt_secret = "$(MYAPP_JWT_SECRET)"` will take :code:`MYAPP_JWT_SECRET` from the environment and make it available to postgresql functions as :code:`current_setting('app.settings.jwt_secret')`.
 
@@ -209,10 +183,13 @@ app.settings.*
 db-anon-role
 ------------
 
-  =============== ==================
+  =============== =======================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  Y
   **Environment** PGRST_DB_ANON_ROLE
-  **In-Database** `n/a`
-  =============== ==================
+  **In-Database** pgrst.db_anon_role
+  =============== =======================
 
   The database role to use when executing commands on behalf of unauthenticated clients. For more information, see :ref:`roles`.
 
@@ -223,10 +200,13 @@ db-anon-role
 db-channel
 ----------
 
-  =============== ================
+  =============== =======================
+  **Type**        String
+  **Default**     pgrst
+  **Reloadable**  Y
   **Environment** PGRST_DB_CHANNEL
   **In-Database** `n/a`
-  =============== ================
+  =============== =======================
 
   The name of the notification channel that PostgREST uses for :ref:`schema_reloading` and configuration reloading.
 
@@ -235,10 +215,13 @@ db-channel
 db-channel-enabled
 ------------------
 
-  =============== ========================
+  =============== =======================
+  **Type**        Boolean
+  **Default**     True
+  **Reloadable**  Y
   **Environment** PGRST_DB_CHANNEL_ENABLED
   **In-Database** `n/a`
-  =============== ========================
+  =============== =======================
 
   When this is set to :code:`true`, the notification channel specified in :ref:`db-channel` is enabled.
 
@@ -249,19 +232,40 @@ db-channel-enabled
 db-config
 ---------
 
-  =============== ===============
+  =============== =======================
+  **Type**        Boolean
+  **Default**     True
+  **Reloadable**  Y
   **Environment** PGRST_DB_CONFIG
   **In-Database** `n/a`
-  =============== ===============
+  =============== =======================
 
    Enables the in-database configuration.
 
+.. _db-pre-config:
+
+db-pre-config
+-------------
+
+  =============== =======================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  Y
+  **Environment** PGRST_DB_PRE_CONFIG
+  **In-Database** pgrst.db_pre_config
+  =============== =======================
+
+   Name of the function that does :ref:`in_db_config`.
+
 .. _db-extra-search-path:
 
 db-extra-search-path
 --------------------
 
   =============== ==========================
+  **Type**        String
+  **Default**     public
+  **Reloadable**  Y
   **Environment** PGRST_DB_EXTRA_SEARCH_PATH
   **In-Database** pgrst.db_extra_search_path
   =============== ==========================
@@ -277,12 +281,15 @@ db-extra-search-path
 db-max-rows
 -----------
 
-  *For backwards compatibility, this config parameter is also available without prefix as "max-rows".*
-
-  =============== =================
+  =============== ==========================
+  **Type**        Int
+  **Default**     ∞
+  **Reloadable**  Y
   **Environment** PGRST_DB_MAX_ROWS
   **In-Database** pgrst.db_max_rows
-  =============== =================
+  =============== ==========================
+
+  *For backwards compatibility, this config parameter is also available without prefix as "max-rows".*
 
   A hard limit to the number of rows PostgREST will fetch from a view, table, or stored procedure. Limits payload size for accidental or malicious requests.
 
@@ -291,96 +298,107 @@ db-max-rows
 db-plan-enabled
 ---------------
 
-  =============== =====================
+  =============== ==========================
+  **Type**        Boolean
+  **Default**     False
+  **Reloadable**  Y
   **Environment** PGRST_DB_PLAN_ENABLED
   **In-Database** pgrst.db_plan_enabled
-  =============== =====================
+  =============== ==========================
 
   When this is set to :code:`true`, the execution plan of a request can be retrieved by using the :code:`Accept: application/vnd.pgrst.plan` header. See :ref:`explain_plan`.
 
-  It's recommended to use this in testing environments only since it reveals internal database details.
-  However, if you choose to use it in production you can add a :ref:`db-pre-request` to filter the requests that can use this feature.
-
-  For example, to only allow requests from an IP address to get the execution plans:
-
-  .. code-block:: postgresql
-
-   -- Assuming a proxy(Nginx, Cloudflare, etc) passes an "X-Forwarded-For" header(https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For)
-   create or replace function filter_plan_requests()
-   returns void as $$
-   declare
-     headers   json := current_setting('request.headers', true)::json;
-     client_ip text := coalesce(headers->>'x-forwarded-for', '');
-     accept    text := coalesce(headers->>'accept', '');
-   begin
-     if accept like 'application/vnd.pgrst.plan%' and client_ip != '144.96.121.73' then
-       raise insufficient_privilege using
-         message = 'Not allowed to use application/vnd.pgrst.plan';
-     end if;
-   end; $$ language plpgsql;
-
-   -- set this function on your postgrest.conf
-   -- db-pre-request = filter_plan_requests
-
 .. _db-pool:
 
 db-pool
 -------
 
-  =============== =================
+  =============== ==========================
+  **Type**        Int
+  **Default**     10
+  **Reloadable**  N
   **Environment** PGRST_DB_POOL
-  **In-Database** `n/a`
-  =============== =================
+  **In-Database** n/a
+  =============== ==========================
 
-  Number of connections to keep open in PostgREST's database pool. Having enough here for the maximum expected simultaneous client connections can improve performance. Note it's pointless to set this higher than the :code:`max_connections` GUC in your database.
+  Number of maximum connections to keep open in PostgREST's database pool.
 
 .. _db-pool-acquisition-timeout:
 
 db-pool-acquisition-timeout
 ---------------------------
 
-  =============== =================
+  =============== =================================
+  **Type**        Int
+  **Default**     10
+  **Reloadable**  N
   **Environment** PGRST_DB_POOL_ACQUISITION_TIMEOUT
   **In-Database** `n/a`
-  =============== =================
+  =============== =================================
 
-  Specifies the maximum time in seconds that the request will wait for the pool to free up a connection slot to the database. If it times out without acquiring a connection, then the request is aborted and a ``504`` error is returned.
+  Specifies the maximum time in seconds that the request will wait for the pool to free up a connection slot to the database.
+
+.. _db-pool-max-idletime:
+
+db-pool-max-idletime
+--------------------
+
+  =============== =================================
+  **Type**        Int
+  **Default**     30
+  **Reloadable**  N
+  **Environment** PGRST_DB_POOL_MAX_IDLETIME
+  **In-Database** `n/a`
+  =============== =================================
+
+   *For backwards compatibility, this config parameter is also available as “db-pool-timeout”.*
+
+   Time in seconds to close idle pool connections.
 
 .. _db-pool-max-lifetime:
 
 db-pool-max-lifetime
 --------------------
 
-  =============== =================
+  =============== =================================
+  **Type**        Int
+  **Default**     1800
+  **Reloadable**  N
   **Environment** PGRST_DB_POOL_MAX_LIFETIME
   **In-Database** `n/a`
-  =============== =================
+  =============== =================================
 
-  Specifies the maximum time in seconds of an existing connection in the pool. When this lifetime is reached, then the connection will be closed and returned to the pool.
+  Specifies the maximum time in seconds of an existing connection in the pool.
 
 .. _db-pre-request:
 
 db-pre-request
 --------------
 
-  *For backwards compatibility, this config parameter is also available without prefix as "pre-request".*
-
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  Y
   **Environment** PGRST_DB_PRE_REQUEST
   **In-Database** pgrst.db_pre_request
-  =============== =================
+  =============== =================================
 
-  A schema-qualified stored procedure name to call right after switching roles for a client request. This provides an opportunity to modify SQL variables or raise an exception to prevent the request from completing.
+  *For backwards compatibility, this config parameter is also available without prefix as "pre-request".*
+
+  A schema-qualified stored procedure name to call right after the :ref:`tx_settings` are set. See :ref:`pre-request`.
 
 .. _db-prepared-statements:
 
 db-prepared-statements
 ----------------------
 
-  =============== =================
+  =============== =================================
+  **Type**        Boolean
+  **Default**     True
+  **Reloadable**  Y
   **Environment** PGRST_DB_PREPARED_STATEMENTS
   **In-Database** pgrst.db_prepared_statements
-  =============== =================
+  =============== =================================
 
   Enables or disables prepared statements.
 
@@ -388,52 +406,50 @@ db-prepared-statements
 
   You should only set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
 
+.. _db-root-spec:
+
+db-root-spec
+------------
+
+  =============== =================================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  Y
+  **Environment** PGRST_DB_ROOT_SPEC
+  **In-Database** pgrst.db_root_spec
+  =============== =================================
+
+  Function to override the OpenAPI response. See :ref:`override_openapi`.
+
 .. _db-schemas:
 
 db-schemas
 ----------
 
-  *For backwards compatibility, this config parameter is also available in singular as "db-schema".*
-
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     public
+  **Reloadable**  Y
   **Environment** PGRST_DB_SCHEMAS
   **In-Database** pgrst.db_schemas
-  =============== =================
+  =============== =================================
 
-  The database schema to expose to REST clients. Tables, views and stored procedures in this schema will get API endpoints.
+  *For backwards compatibility, this config parameter is also available in singular as "db-schema".*
 
-  .. code:: bash
-
-     db-schemas = "api"
-
-  This schema gets added to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request.
-
-List of schemas
-~~~~~~~~~~~~~~~
-
-  You can also specify a list of schemas that can be used for **schema-based multitenancy** and **api versioning** by :ref:`multiple-schemas`. Example:
-
-  .. code:: bash
-
-     db-schemas = "tenant1, tenant2"
-
-  If you don't :ref:`Switch Schemas <multiple-schemas>`, the first schema in the list(``tenant1`` in this case) is chosen as the default schema.
-
-  *Only the chosen schema* gets added to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request.
-
-  .. warning::
-
-     Never expose private schemas in this way. See :ref:`schema_isolation`.
+  The list of database schemas to expose to clients. See :ref:`schemas`.
 
 .. _db-tx-end:
 
 db-tx-end
 ---------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     commit
+  **Reloadable**  N
   **Environment** PGRST_DB_TX_END
-  **In-Database** pgrst.db_tx_end
-  =============== =================
+  **In-Database** `n/a`
+  =============== =================================
 
   Specifies how to terminate the database transactions.
 
@@ -456,26 +472,62 @@ db-tx-end
 db-uri
 ------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     postgresql://
+  **Reloadable**  N
   **Environment** PGRST_DB_URI
   **In-Database** `n/a`
-  =============== =================
+  =============== =================================
 
-  The standard connection PostgreSQL `URI format <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_. Symbols and unusual characters in the password or other fields should be percent encoded to avoid a parse error. If enforcing an SSL connection to the database is required you can use `sslmode <https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS>`_ in the URI, for example ``postgres://user:pass@host:5432/dbname?sslmode=require``.
+  The standard `PostgreSQL connection string <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_, there are different ways to specify it:
 
-  When running PostgREST on the same machine as PostgreSQL, it is also possible to connect to the database using a `Unix socket <https://en.wikipedia.org/wiki/Unix_domain_socket>`_ and the `Peer Authentication method <https://www.postgresql.org/docs/current/auth-peer.html>`_ as an alternative to TCP/IP communication and authentication with a password, this also grants higher performance.  To do this you can omit the host and the password, e.g. ``postgres://user@/dbname``, see the `libpq connection string <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_ documentation for more details.
+URI Format
+~~~~~~~~~~
+
+  .. code::
+
+    "postgres://authenticator:mysecretpassword@localhost:5433/postgres?parameters=val"
+
+  - Under this format symbols and unusual characters in the password or other fields should be percent encoded to avoid a parse error.
+  - If enforcing an SSL connection to the database is required you can use `sslmode <https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS>`_ in the URI, for example ``postgres://user:pass@host:5432/dbname?sslmode=require``.
+  - The user with whom PostgREST connects to the database is also known as the ``authenticator`` role. For more information see :ref:`roles`.
+  - When running PostgREST on the same machine as PostgreSQL, it is also possible to connect to the database using a `Unix socket <https://en.wikipedia.org/wiki/Unix_domain_socket>`_ and the `Peer Authentication method <https://www.postgresql.org/docs/current/auth-peer.html>`_ as an alternative to TCP/IP communication and authentication with a password, this also grants higher performance.  To do this you can omit the host and the password, e.g. ``postgres://user@/dbname``, see the `libpq connection string <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_ documentation for more details.
+
+Keyword/Value Format
+~~~~~~~~~~~~~~~~~~~~
+
+  .. code::
+
+    "host=localhost port=5433 user=authenticator password=mysecretpassword dbname=postgres"
+
+LIBPQ Environment Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+  .. code::
+
+    PGHOST=localhost PGPORT=5433 PGUSER=authenticator PGDATABASE=postgres
+
+  Any parameter that is not set in the above formats is read from `libpq environment variables <https://www.postgresql.org/docs/current/libpq-envars.html>`_. The default connection string is ``postgresql://``, which reads **all** parameters from the environment.
+
+External config file
+~~~~~~~~~~~~~~~~~~~~
 
   Choosing a value for this parameter beginning with the at sign such as ``@filename`` (e.g. ``@./configs/my-config``) loads the connection string out of an external file.
 
+
 .. _db-use-legacy-gucs:
 
 db-use-legacy-gucs
 ------------------
 
-  =============== =================
+  =============== =================================
+  **Type**        Boolean
+  **Default**     True
+  **Reloadable**  Y
   **Environment** PGRST_DB_USE_LEGACY_GUCS
   **In-Database** pgrst.db_use_legacy_gucs
-  =============== =================
+  =============== =================================
 
   Determine if GUC request settings for headers, cookies and jwt claims use the :ref:`legacy names <guc_legacy_names>` (string with dashes, invalid starting from PostgreSQL v14) with text values instead of the :ref:`new names <guc_req_headers_cookies_claims>` (string without dashes, valid on all PostgreSQL versions) with json values.
 
@@ -486,10 +538,13 @@ db-use-legacy-gucs
 jwt-aud
 -------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  Y
   **Environment** PGRST_JWT_AUD
   **In-Database** pgrst.jwt_aud
-  =============== =================
+  =============== =================================
 
   Specifies the `JWT audience claim <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.3>`_. If this claim is present in the client provided JWT then you must set this to the same value as in the JWT, otherwise verifying the JWT will fail.
 
@@ -498,12 +553,15 @@ jwt-aud
 jwt-role-claim-key
 ------------------
 
-  *For backwards compatibility, this config parameter is also available without prefix as "role-claim-key".*
-
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     .role
+  **Reloadable**  Y
   **Environment** PGRST_JWT_ROLE_CLAIM_KEY
   **In-Database** pgrst.jwt_role_claim_key
-  =============== =================
+  =============== =================================
+
+  *For backwards compatibility, this config parameter is also available without prefix as "role-claim-key".*
 
   A JSPath DSL that specifies the location of the :code:`role` key in the JWT claims. This can be used to consume a JWT provided by a third party service like Auth0, Okta or Keycloak. Usage examples:
 
@@ -522,10 +580,13 @@ jwt-role-claim-key
 jwt-secret
 ----------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  Y
   **Environment** PGRST_JWT_SECRET
   **In-Database** pgrst.jwt_secret
-  =============== =================
+  =============== =================================
 
   The secret or `JSON Web Key (JWK) (or set) <https://datatracker.ietf.org/doc/html/rfc7517>`_ used to decode JWT tokens clients provide for authentication. For security the key must be **at least 32 characters long**. If this parameter is not specified then PostgREST refuses authentication requests. Choosing a value for this parameter beginning with the at sign such as :code:`@filename` loads the secret out of an external file. This is useful for automating deployments. Note that any binary secrets must be base64 encoded. Both symmetric and asymmetric cryptography are supported. For more info see :ref:`asym_keys`.
 
@@ -540,10 +601,13 @@ jwt-secret
 jwt-secret-is-base64
 --------------------
 
-  =============== =================
+  =============== =================================
+  **Type**        Boolean
+  **Default**     False
+  **Reloadable**  Y
   **Environment** PGRST_JWT_SECRET_IS_BASE64
   **In-Database** pgrst.jwt_secret_is_base64
-  =============== =================
+  =============== =================================
 
   When this is set to :code:`true`, the value derived from :code:`jwt-secret` will be treated as a base64 encoded secret.
 
@@ -552,10 +616,13 @@ jwt-secret-is-base64
 log-level
 ---------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     error
+  **Reloadable**  N
   **Environment** PGRST_LOG_LEVEL
   **In-Database** `n/a`
-  =============== =================
+  =============== =================================
 
   Specifies the level of information to be logged while running PostgREST.
 
@@ -581,10 +648,13 @@ log-level
 openapi-mode
 ------------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     follow-privileges
+  **Reloadable**  Y
   **Environment** PGRST_OPENAPI_MODE
   **In-Database** pgrst.openapi_mode
-  =============== =================
+  =============== =================================
 
   Specifies how the OpenAPI output should be displayed.
 
@@ -607,10 +677,13 @@ openapi-mode
 openapi-security-active
 -----------------------
 
-  =============== =============================
+  =============== =================================
+  **Type**        Boolean
+  **Default**     False
+  **Reloadable**  Y
   **Environment** PGRST_OPENAPI_SECURITY_ACTIVE
   **In-Database** pgrst.openapi_security_active
-  =============== =============================
+  =============== =================================
 
 When this is set to :code:`true`, security options are included in the :ref:`OpenAPI output <open-api>`.
 
@@ -619,10 +692,13 @@ When this is set to :code:`true`, security options are included in the :ref:`Ope
 openapi-server-proxy-uri
 ------------------------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  N
   **Environment** PGRST_OPENAPI_SERVER_PROXY_URI
   **In-Database** pgrst.openapi_server_proxy_uri
-  =============== =================
+  =============== =================================
 
   Overrides the base URL used within the OpenAPI self-documentation hosted at the API root path. Use a complete URI syntax :code:`scheme:[//[user:password@]host[:port]][/]path[?query][#fragment]`. Ex. :code:`https://postgrest.com`
 
@@ -647,10 +723,13 @@ openapi-server-proxy-uri
 raw-media-types
 ---------------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  Y
   **Environment** PGRST_RAW_MEDIA_TYPES
   **In-Database** pgrst.raw_media_types
-  =============== =================
+  =============== =================================
 
  This serves to extend the `Media Types <https://en.wikipedia.org/wiki/Media_type>`_ that PostgREST currently accepts through an ``Accept`` header.
 
@@ -668,10 +747,13 @@ raw-media-types
 server-host
 -----------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     !4
+  **Reloadable**  N
   **Environment** PGRST_SERVER_HOST
   **In-Database** `n/a`
-  =============== =================
+  =============== =================================
 
   Where to bind the PostgREST web server. In addition to the usual address options, PostgREST interprets these reserved addresses with special meanings:
 
@@ -686,22 +768,43 @@ server-host
 server-port
 -----------
 
-  =============== =================
+  =============== =================================
+  **Type**        Int
+  **Default**     3000
+  **Reloadable**  N
   **Environment** PGRST_SERVER_PORT
   **In-Database** `n/a`
-  =============== =================
+  =============== =================================
 
   The TCP port to bind the web server.
 
+.. _server-trace-header:
+
+server-trace-header
+-------------------
+
+  =============== =================================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  Y
+  **Environment** PGRST_SERVER_TRACE_HEADER
+  **In-Database** pgrst.server_trace_header
+  =============== =================================
+
+  The header name used to trace HTTP requests. See :ref:`trace_header`.
+
 .. _server-unix-socket:
 
 server-unix-socket
 ------------------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     `n/a`
+  **Reloadable**  N
   **Environment** PGRST_SERVER_UNIX_SOCKET
   **In-Database** `n/a`
-  =============== =================
+  =============== =================================
 
   `Unix domain socket <https://en.wikipedia.org/wiki/Unix_domain_socket>`_ where to bind the PostgREST web server.
   If specified, this takes precedence over :ref:`server-port`. Example:
@@ -715,10 +818,13 @@ server-unix-socket
 server-unix-socket-mode
 -----------------------
 
-  =============== =================
+  =============== =================================
+  **Type**        String
+  **Default**     660
+  **Reloadable**  N
   **Environment** PGRST_SERVER_UNIX_SOCKET_MODE
   **In-Database** `n/a`
-  =============== =================
+  =============== =================================
 
   `Unix file mode <https://en.wikipedia.org/wiki/File_system_permissions>`_ to be set for the socket specified in :ref:`server-unix-socket`
   Needs to be a valid octal between 600 and 777.

docs/references/connection_pool.rst

@@ -0,0 +1,93 @@
+.. _connection_pool:
+
+Connection Pool
+===============
+
+A connection pool is a cache of reusable database connections. It allows serving many HTTP requests using few database connections. Every request to an :doc:`API resource <api>` borrows a connection from the pool to start a :doc:`transaction <transactions>`.
+
+Minimizing connections is paramount to performance. Each PostgreSQL connection creates a process, having too many can exhaust available resources.
+
+Connection String
+-----------------
+
+For connecting to the database, the pool requires a connection string. You can can configure it using :ref:`db-uri`.
+
+.. _pool_growth_limit:
+.. _dyn_conn_pool:
+
+Dynamic Connection Pool
+-----------------------
+
+To conserve system resources, PostgREST uses a dynamic connection pool. This enables the number of connections in the pool to increase and decrease depending on request traffic.
+
+- If all the connections are being used, a new connection is added. The pool can grow until it reaches the :ref:`db-pool` size. Note that it’s pointless to set this higher than the ``max_connections`` setting in your database.
+- If a connection is unused for a period of time (:ref:`db-pool-max-idletime`), it will be released.
+
+Connection lifetime
+-------------------
+
+Long-lived PostgreSQL connections can consume considerable memory (see `here <https://www.postgresql.org/message-id/CAFj8pRCQN2B2vrVMH1-bd-8xtzjytWR%2BAjZ%2BMCj9J2wPxKPa9Q%40mail.gmail.com>`_ for more details).
+Under a busy system, the :ref:`db-pool-max-idletime` won't be reached and the connection pool can be full of long-lived connections.
+
+To avoid this problem and save resources, a connection max lifetime (:ref:`db-pool-max-lifetime`) is enforced.
+After the max lifetime is reached, connections from the pool will be released and new ones will be created. This doesn't affect running requests, only unused connections will be released.
+
+Acquisition Timeout
+-------------------
+
+If all the available connections in the pool are busy, an HTTP request will wait until reaching a timeout (:ref:`db-pool-acquisition-timeout`).
+
+If the request reaches the timeout, it will be aborted with the following response:
+
+.. code-block:: http
+
+  HTTP/1.1 504 Gateway Timeout
+
+  {"code":"PGRST003",
+   "details":null,
+   "hint":null,
+   "message":"Timed out acquiring connection from connection pool."}
+
+.. important::
+
+  Getting this error message is an indicator of a performance issue. To solve it, you can:
+
+  - Reduce your queries execution time.
+
+    - Check the request :ref:`explain_plan` to tune your query, this usually means adding indexes.
+
+  - Reduce the amount of requests.
+
+    - Reduce write requests. Do :ref:`bulk_insert` (or :ref:`upsert`) instead of inserting rows one by one.
+    - Reduce read requests. Use :ref:`resource_embedding`. Combine unrelated data into a single request using custom database views or functions.
+    - Use :ref:`s_procs` for combining read and write logic into a single request.
+
+  - Increase the :ref:`db-pool` size.
+
+    - Not a panacea since connections can't grow infinitely. Try the previous recommendations before this.
+
+.. _automatic_recovery:
+
+Automatic Recovery
+------------------
+
+The server will retry reconnecting to the database if connection loss happens.
+
+- It will retry forever with exponential backoff, with a maximum backoff time of 32 seconds between retries. Each of these attempts are :ref:`logged <pgrst_logging>`.
+- It will only stop retrying if the server deems the error to be fatal. This can be a password authentication failure or an internal error.
+- The retries happen immediately after a connection loss, if :ref:`db-channel-enabled` is set to true (the default). Otherwise they'll happen once a request arrives.
+- To ensure a valid state, the server reloads the :ref:`schema_cache` and :ref:`configuration` when recovering.
+- To notify the client of the next retry, the server sends a ``503 Service Unavailable`` status with the ``Retry-After: x`` header. Where ``x`` is the number of seconds programmed for the next retry.
+
+.. _external_connection_poolers:
+
+Using External Connection Poolers
+---------------------------------
+
+It's possible to use external connection poolers, such as PgBouncer. Session pooling is compatible, while transaction pooling requires :ref:`db-prepared-statements` set to ``false``. Statement pooling is not compatible with PostgREST.
+
+Also set :ref:`db-channel-enabled` to ``false`` since ``LISTEN`` is not compatible with transaction pooling. Although it should not give any errors if left enabled.
+
+.. note::
+
+  It’s not recommended to use an external connection pooler. `Our benchmarks <https://github.com/PostgREST/postgrest/issues/2294#issuecomment-1139148672>`_ indicate it provides much lower performance than PostgREST built-in pool.

docs/references/errors.rst

@@ -1,9 +1,14 @@
 .. _error_source:
 
-Error Source
-============
+Errors
+######
 
-For the most part, error messages will come directly from the database with the same `structure that PostgreSQL uses <https://www.postgresql.org/docs/current/error-style-guide.html>`_. PostgREST will convert the ``MESSAGE``, ``DETAIL``, ``HINT`` and ``ERRCODE`` from the PostgreSQL error to JSON format and add an HTTP status code to the response (see :ref:`status_codes`). For instance, this is the error you will get when querying a nonexistent table:
+PostgREST error messages follow the PostgreSQL error structure. It includes ``MESSAGE``, ``DETAIL``, ``HINT``, ``ERRCODE`` and will add an HTTP status code to the response.
+
+Errors from PostgreSQL
+======================
+
+PostgREST will forward errors coming from PostgreSQL. For instance, when querying a nonexistent table:
 
 .. code-block:: http
 
@@ -23,30 +28,10 @@ For the most part, error messages will come directly from the database with the
     "message": "relation \"api.nonexistent_table\" does not exist"
   }
 
-However, some errors do come from PostgREST itself (such as those related to the :ref:`schema_cache`). These have the same structure as the PostgreSQL errors but are differentiated by the ``PGRST`` prefix in the ``code`` field (see :ref:`pgrst_errors`). For instance, when querying a function that does not exist, the error will be:
-
-.. code-block:: http
-
-  POST /rpc/nonexistent_function HTTP/1.1
-
-.. code-block:: http
-
-  HTTP/1.1 404 Not Found
-  Content-Type: application/json; charset=utf-8
-
-.. code-block:: json
-
-  {
-    "hint": "If a new function was created in the database with this name and parameters, try reloading the schema cache.",
-    "details": null
-    "code": "PGRST202",
-    "message": "Could not find the api.nonexistent_function() function in the schema cache"
-  }
-
 .. _status_codes:
 
 HTTP Status Codes
-=================
+-----------------
 
 PostgREST translates `PostgreSQL error codes <https://www.postgresql.org/docs/current/errcodes-appendix.html>`_ into HTTP status as follows:
 
@@ -111,17 +96,101 @@ PostgREST translates `PostgreSQL error codes <https://www.postgresql.org/docs/cu
 | other                    | 400                     |                                 |
 +--------------------------+-------------------------+---------------------------------+
 
+.. _raise_error:
+
+RAISE errors with HTTP Status Codes
+-----------------------------------
+
+You can return custom HTTP status codes by raising SQL exceptions inside :ref:`functions <s_procs>`. For instance, here's a saucy function that always responds with an error:
+
+.. code-block:: postgresql
+
+  CREATE OR REPLACE FUNCTION just_fail() RETURNS void
+    LANGUAGE plpgsql
+    AS $$
+  BEGIN
+    RAISE EXCEPTION 'I refuse!'
+      USING DETAIL = 'Pretty simple',
+            HINT = 'There is nothing you can do.';
+  END
+  $$;
+
+Calling the function returns HTTP 400 with the body
+
+.. code-block:: json
+
+  {
+    "message":"I refuse!",
+    "details":"Pretty simple",
+    "hint":"There is nothing you can do.",
+    "code":"P0001"
+  }
+
+One way to customize the HTTP status code is by raising particular exceptions according to the PostgREST :ref:`error to status code mapping <status_codes>`. For example, :code:`RAISE insufficient_privilege` will respond with HTTP 401/403 as appropriate.
+
+For even greater control of the HTTP status code, raise an exception of the ``PTxyz`` type. For instance to respond with HTTP 402, raise ``PT402``:
+
+.. code-block:: sql
+
+  RAISE sqlstate 'PT402' using
+    message = 'Payment Required',
+    detail = 'Quota exceeded',
+    hint = 'Upgrade your plan';
+
+Returns:
+
+.. code-block:: http
+
+  HTTP/1.1 402 Payment Required
+  Content-Type: application/json; charset=utf-8
+
+  {
+    "message": "Payment Required",
+    "details": "Quota exceeded",
+    "hint": "Upgrade your plan",
+    "code": "PT402"
+  }
+
+
+Errors from PostgREST
+=====================
+
+Errors that come from PostgREST itself maintain the same structure. But differ in the ``PGRST`` prefix in the ``code`` field. For instance, when querying a function that does not exist in the :doc:`schema cache <schema_cache>`:
+
+.. code-block:: http
+
+  POST /rpc/nonexistent_function HTTP/1.1
+
+.. code-block:: http
+
+  HTTP/1.1 404 Not Found
+  Content-Type: application/json; charset=utf-8
+
+.. code-block:: json
+
+  {
+    "hint": "...",
+    "details": null
+    "code": "PGRST202",
+    "message": "Could not find the api.nonexistent_function() function in the schema cache"
+  }
+
+
 .. _pgrst_errors:
 
 PostgREST Error Codes
-=====================
+---------------------
 
-PostgREST error codes have the form ``PGRSTgxx``, where ``PGRST`` is the prefix that differentiates the error from a PostgreSQL error, ``g`` is the group where the error belongs and ``xx`` is the number that identifies the error in the group.
+PostgREST error codes have the form ``PGRSTgxx``
+
+- ``PGRST`` is the prefix that differentiates the error from a PostgreSQL error.
+- ``g`` is the error group
+- ``xx`` is the error identifier in the group.
 
 .. _pgrst0**:
 
 Group 0 - Connection
---------------------
+~~~~~~~~~~~~~~~~~~~~
 
 Related to the connection with the database.
 
@@ -137,8 +206,8 @@ Related to the connection with the database.
 | PGRST001      |             |                                                             |
 +---------------+-------------+-------------------------------------------------------------+
 | .. _pgrst002: | 503         | Could not connect with the database when building the       |
-|               |             | :ref:`schema_cache` due to the PostgreSQL service not       |
-| PGRST002      |             | running.                                                    |
+|               |             | :doc:`Schema Cache <schema_cache>`                          |
+| PGRST002      |             | due to the PostgreSQL service not running.                  |
 +---------------+-------------+-------------------------------------------------------------+
 | .. _pgrst003: | 504         | The request timed out waiting for a pool connection         |
 |               |             | to be available. See :ref:`db-pool-acquisition-timeout`.    |
@@ -148,7 +217,7 @@ Related to the connection with the database.
 .. _pgrst1**:
 
 Group 1 - Api Request
----------------------
+~~~~~~~~~~~~~~~~~~~~~
 
 Related to the HTTP request elements.
 
@@ -223,26 +292,38 @@ Related to the HTTP request elements.
 |               |             |                                                             |
 | PGRST117      |             |                                                             |
 +---------------+-------------+-------------------------------------------------------------+
+| .. _pgrst118: | 400         | Could not order the result using the related table because  |
+|               |             | there is no many-to-one or one-to-one relationship between  |
+| PGRST118      |             | them.                                                       |
++---------------+-------------+-------------------------------------------------------------+
+| .. _pgrst119: | 400         | Could not use the spread operator on the related table      |
+|               |             | because there is no many-to-one or one-to-one relationship  |
+| PGRST119      |             | between them.                                               |
++---------------+-------------+-------------------------------------------------------------+
+| .. _pgrst120: | 400         | An embedded resource can only be filtered using the         |
+|               |             | ``is.null`` or ``not.is.null`` :ref:`operators <operators>`.|
+| PGRST120      |             |                                                             |
++---------------+-------------+-------------------------------------------------------------+
 
 .. _pgrst2**:
 
 Group 2 - Schema Cache
-----------------------
+~~~~~~~~~~~~~~~~~~~~~~
 
 Related to a :ref:`stale schema cache <stale_schema>`. Most of the time, these errors are solved by :ref:`reloading the schema cache <schema_reloading>`.
 
 +---------------+-------------+-------------------------------------------------------------+
 | Code          | HTTP status | Description                                                 |
 +===============+=============+=============================================================+
-| .. _pgrst200: | 400         | Caused by :ref:`stale_fk_relationships`, otherwise any of   |
+| .. _pgrst200: | 400         | Caused by stale foreign key relationships, otherwise any of |
 |               |             | the embedding resources or the relationship itself may not  |
 | PGRST200      |             | exist in the database.                                      |
 +---------------+-------------+-------------------------------------------------------------+
 | .. _pgrst201: | 300         | An ambiguous embedding request was made.                    |
-|               |             | See :ref:`embed_disamb`.                                    |
+|               |             | See :ref:`complex_rels`.                                    |
 | PGRST201      |             |                                                             |
 +---------------+-------------+-------------------------------------------------------------+
-| .. _pgrst202: | 404         | Caused by a :ref:`stale_function_signature`, otherwise      |
+| .. _pgrst202: | 404         | Caused by a stale function signature, otherwise             |
 |               |             | the function may not exist in the database.                 |
 | PGRST202      |             |                                                             |
 +---------------+-------------+-------------------------------------------------------------+
@@ -260,7 +341,7 @@ Related to a :ref:`stale schema cache <stale_schema>`. Most of the time, these e
 .. _pgrst3**:
 
 Group 3 - JWT
--------------
+~~~~~~~~~~~~~
 
 Related to the authentication process using JWT. You can follow the :ref:`tut1` for an example on how to implement authentication and the :doc:`Authentication page <auth>` for more information on this process.
 
@@ -285,7 +366,7 @@ Related to the authentication process using JWT. You can follow the :ref:`tut1`
 .. _pgrst_X**:
 
 Group X - Internal
-------------------
+~~~~~~~~~~~~~~~~~~
 
 Internal errors. If you encounter any of these, you may have stumbled on a PostgREST bug, please `open an issue <https://github.com/PostgREST/postgrest/issues>`_ and we'll be glad to fix it.
 

docs/references/schema_cache.rst

@@ -3,8 +3,7 @@
 Schema Cache
 ============
 
-Certain PostgREST features require metadata from the database schema. Getting this metadata requires executing expensive queries, so
-in order to avoid repeating this work, PostgREST uses a schema cache.
+Some PostgREST features need metadata from the database schema. Getting this metadata requires expensive queries. To avoid repeating this work, PostgREST uses a schema cache.
 
 +--------------------------------------------+-------------------------------------------------------------------------------+
 | Feature                                    | Required Metadata                                                             |
@@ -29,84 +28,19 @@ in order to avoid repeating this work, PostgREST uses a schema cache.
 
 .. _stale_schema:
 
-The Stale Schema Cache
-----------------------
+Stale Schema Cache
+------------------
 
-When you make changes on the metadata mentioned above, the schema cache will turn stale on a running PostgREST. Future requests that use the above features will need the :ref:`schema cache to be reloaded <schema_reloading>`; otherwise, you'll get an error instead of the expected result.
+One operational problem that comes a cache is that it can go stale. This can happen for PostgREST when you make changes to the metadata before mentioned. Requests that depend on the metadata will fail.
 
-For instance, let's see what would happen if you have a stale schema cache for foreign key relationships and function signatures.
-
-.. _stale_fk_relationships:
-
-Stale Foreign Key Relationships
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Suppose you add a ``cities`` table to your database and define a foreign key that references an existing ``countries`` table. Then, you make a request to get the ``cities`` and their belonging ``countries``.
-
-.. tabs::
-
-  .. code-tab:: http
-
-    GET /cities?select=name,country:countries(id,name) HTTP/1.1
-
-  .. code-tab:: bash Curl
-
-    curl "http://localhost:3000/cities?select=name,country:countries(id,name)"
-
-The result will be an error:
-
-.. code-block:: json
-
-  {
-    "hint": "Verify that 'cities' and 'countries' exist in the schema 'api' and that there is a foreign key relationship between them. If a new relationship was created, try reloading the schema cache.",
-    "details": null,
-    "code": "PGRST200",
-    "message": "Could not find a relationship between 'cities' and 'countries' in the schema cache"
-  }
-
-As you can see, PostgREST couldn't find the newly created foreign key in the schema cache. See :ref:`schema_reloading` and :ref:`auto_schema_reloading` to solve this issue.
-
-.. _stale_function_signature:
-
-Stale Function Signature
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-The same issue will occur on newly created functions on a running PostgREST.
-
-.. code-block:: plpgsql
-
-  CREATE FUNCTION plus_one(num integer)
-  RETURNS integer AS $$
-   SELECT num + 1;
-  $$ LANGUAGE SQL IMMUTABLE;
-
-.. tabs::
-
-  .. code-tab:: http
-
-    GET /rpc/plus_one?num=1 HTTP/1.1
-
-  .. code-tab:: bash Curl
-
-    curl "http://localhost:3000/rpc/plus_one?num=1"
-
-.. code-block:: json
-
-  {
-    "hint": "If a new function was created in the database with this name and parameters, try reloading the schema cache.",
-    "details": null,
-    "code": "PGRST202",
-    "message": "Could not find the api.plus_one(num) function in the schema cache"
-  }
-
-Here, PostgREST tries to find the function on the stale schema to no avail. See :ref:`schema_reloading` and :ref:`auto_schema_reloading` to solve this issue.
+You can solve this by reloading the cache manually or automatically.
 
 .. _schema_reloading:
 
 Schema Cache Reloading
 ----------------------
 
-To reload the cache without restarting the PostgREST server, send a SIGUSR1 signal to the server process.
+To manually reload the cache without restarting the PostgREST server, send a SIGUSR1 signal to the server process.
 
 .. code:: bash
 
@@ -122,27 +56,29 @@ For docker you can do:
   # or in docker-compose
   docker-compose kill -s SIGUSR1 <service>
 
-There's no downtime when reloading the schema cache. The reloading will happen on a background thread while requests keep being served.
+There’s no downtime when reloading the schema cache. The reloading will happen on a background thread while serving requests.
 
 .. _schema_reloading_notify:
 
 Reloading with NOTIFY
 ~~~~~~~~~~~~~~~~~~~~~
 
-There are environments where you can't send the SIGUSR1 Unix Signal (like on managed containers in cloud services or on Windows systems). For this reason, PostgREST also allows you to reload its schema cache through PostgreSQL `NOTIFY <https://www.postgresql.org/docs/current/sql-notify.html>`_ as follows:
+PostgREST also allows you to reload its schema cache through PostgreSQL `NOTIFY <https://www.postgresql.org/docs/current/sql-notify.html>`_.
 
 .. code-block:: postgresql
 
   NOTIFY pgrst, 'reload schema'
 
-The ``"pgrst"`` notification channel is enabled by default. For configuring the channel, see :ref:`db-channel` and :ref:`db-channel-enabled`.
+This is useful in environments where you can’t send the SIGUSR1 Unix Signal. Like on cloud managed containers or on Windows systems.
+
+The ``pgrst`` notification channel is enabled by default. For configuring the channel, see :ref:`db-channel` and :ref:`db-channel-enabled`.
 
 .. _auto_schema_reloading:
 
 Automatic Schema Cache Reloading
 --------------------------------
 
-You can do automatic schema cache reloading in a pure SQL way and forget about stale schema cache errors with an `event trigger <https://www.postgresql.org/docs/current/event-trigger-definition.html>`_ and ``NOTIFY``.
+You can do automatic schema cache reloading in a pure SQL way and forget about stale schema cache errors. For this use an `event trigger <https://www.postgresql.org/docs/current/event-trigger-definition.html>`_ and ``NOTIFY``.
 
 .. code-block:: postgresql
 
@@ -160,9 +96,9 @@ You can do automatic schema cache reloading in a pure SQL way and forget about s
     ON ddl_command_end
     EXECUTE PROCEDURE pgrst_watch();
 
-Now, whenever the ``pgrst_watch`` trigger is fired in the database, PostgREST will automatically reload the schema cache.
+Now, whenever the ``pgrst_watch`` trigger fires, PostgREST will auto-reload the schema cache.
 
-To disable auto reloading, drop the trigger:
+To disable auto reloading, drop the trigger.
 
 .. code-block:: postgresql
 
@@ -171,12 +107,12 @@ To disable auto reloading, drop the trigger:
 Finer-Grained Event Trigger
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-You can refine the previous event trigger and only react to the events relevant to the schema cache. This also prevents unnecessary
-reloading when creating temporary tables(``CREATE TEMP TABLE``) inside functions.
+You can refine the previous event trigger to only react to the events relevant to the schema cache. This also prevents unnecessary
+reloading when creating temporary tables inside functions.
 
 .. code-block:: postgresql
 
-  -- watch create and alter
+  -- watch CREATE and ALTER
   CREATE OR REPLACE FUNCTION pgrst_ddl_watch() RETURNS event_trigger AS $$
   DECLARE
     cmd record;
@@ -203,7 +139,7 @@ reloading when creating temporary tables(``CREATE TEMP TABLE``) inside functions
     END LOOP;
   END; $$ LANGUAGE plpgsql;
 
-  -- watch drop
+  -- watch DROP
   CREATE OR REPLACE FUNCTION pgrst_drop_watch() RETURNS event_trigger AS $$
   DECLARE
     obj record;

docs/references/transactions.rst

@@ -0,0 +1,327 @@
+.. _transactions:
+
+Transactions
+============
+
+After :ref:`user_impersonation`, every request to an :doc:`API resource <api>` runs inside a transaction. The sequence of the transaction is as follows:
+
+.. code-block:: postgresql
+
+  BEGIN; -- <Access Mode> <Isolation Level>
+  -- <Transaction-scoped settings>
+  -- <Main Query>;
+  END;
+
+.. _access_mode:
+
+Access Mode
+-----------
+
+The access mode on :ref:`tables_views` is determined by the HTTP method.
+
+.. list-table::
+   :header-rows: 1
+
+   * - HTTP Method
+     - Access Mode
+   * - GET, HEAD
+     - READ ONLY
+   * - POST, PATCH, PUT, DELETE
+     - READ WRITE
+
+:ref:`s_procs` additionally depend on the function `volatility <https://www.postgresql.org/docs/current/xfunc-volatility.html>`_.
+
+.. list-table::
+   :header-rows: 2
+
+   * -
+     - Access Mode
+     -
+     -
+   * - HTTP Method
+     - VOLATILE
+     - STABLE
+     - IMMUTABLE
+   * - GET, HEAD
+     - READ ONLY
+     - READ ONLY
+     - READ ONLY
+   * - POST
+     - READ WRITE
+     - READ ONLY
+     - READ ONLY
+
+Modifying the database inside READ ONLY transactions is not possible. PostgREST uses this fact to enforce HTTP semantics in GET and HEAD requests.
+
+.. note::
+
+  The volatility marker is a promise about the behavior of the function.  PostgreSQL will let you mark a function that modifies the database as ``IMMUTABLE`` or ``STABLE`` without failure.  But, because of the READ ONLY transaction the function will fail under PostgREST.
+
+The :ref:`options_requests` method doesn't start a transaction, so it's not relevant here.
+
+.. _isolation_lvl:
+
+Isolation Level
+---------------
+
+Every transaction uses the PostgreSQL default isolation level: READ COMMITTED. Unless you modify `default_transaction_isolation <https://www.postgresql.org/docs/15/runtime-config-client.html#GUC-DEFAULT-TRANSACTION-ISOLATION>`_  for an impersonated role or function.
+
+Using :ref:`impersonated_settings`, change the isolation level for all the role's requests with:
+
+.. code-block:: postgresql
+
+  ALTER ROLE webuser SET default_transaction_isolation TO 'repeatable read';
+
+Or to change the isolation level per function call.
+
+.. code-block:: postgresql
+
+  CREATE OR REPLACE FUNCTION myfunc()
+  RETURNS text as $$
+    SELECT 'hello';
+  $$
+  LANGUAGE SQL
+  SET default_transaction_isolation TO 'serializable';
+
+.. _tx_settings:
+
+Transaction-Scoped Settings
+---------------------------
+
+PostgREST uses settings tied to the transaction lifetime. These can be used to get data about the HTTP request. Or to modify the HTTP response.
+
+You can get these with ``current_setting``
+
+.. code-block:: postgresql
+
+  -- request settings use the ``request.`` prefix.
+  SELECT
+    current_setting('request.<setting>', true);
+
+And you can set them with ``set_config``
+
+.. code-block:: postgresql
+
+  -- response settings use the ``response.`` prefix.
+  SELECT
+    set_config('response.<setting>', 'value1' ,true);
+
+.. _guc_req_headers_cookies_claims:
+
+Request Headers, Cookies and JWT claims
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+PostgREST stores the headers, cookies and headers as JSON. To get them:
+
+.. important::
+
+  The headers names are lowercased. e.g. If the request sends ``User-Agent: x`` this will be obtainable as ``current_setting('request.headers', true)::json->>'user-agent'``.
+
+.. code-block:: postgresql
+
+  -- To get all the headers sent in the request
+  SELECT current_setting('request.headers', true)::json;
+
+  -- To get a single header, you can use JSON arrow operators
+  SELECT current_setting('request.headers', true)::json->>'user-agent';
+
+  -- value of sessionId in a cookie
+  SELECT current_setting('request.cookies', true)::json->>'sessionId';
+
+  -- value of the email claim in a jwt
+  SELECT current_setting('request.jwt.claims', true)::json->>'email';
+
+.. note::
+
+  The ``role`` in ``request.jwt.claims`` defaults to the value of :ref:`db-anon-role`.
+
+.. _guc_legacy_names:
+
+Legacy settings
+^^^^^^^^^^^^^^^
+
+For PostgreSQL versions below 14, PostgREST will take into consideration the :ref:`db-use-legacy-gucs` config, which is set to true by default.
+This means that the interface for accessing these GUCs is the same as in older versions (see below).
+You can opt in to use the JSON GUCs mentioned above by setting the ``db-use-legacy-gucs`` to false.
+
+.. raw:: html
+
+  <p>
+    <details>
+      <summary>Old GUCs</summary>
+
+.. code-block:: postgresql
+
+  -- To read the value of the User-Agent request header:
+  SELECT current_setting('request.header.user-agent', true);
+
+  -- To read the value of sessionId in a cookie:
+  SELECT current_setting('request.cookie.sessionId', true);
+
+  -- To read the value of the email claim in a jwt:
+  SELECT current_setting('request.jwt.claim.email', true);
+
+.. note::
+
+  ``request.jwt.claim.role`` defaults to the value of :ref:`db-anon-role`.
+
+.. raw:: html
+
+    </details>
+  </p>
+
+
+.. _guc_req_path_method:
+
+Request Path and Method
+~~~~~~~~~~~~~~~~~~~~~~~
+
+The path and method are stored as ``text``.
+
+.. code-block:: postgresql
+
+  SELECT current_setting('request.path', true);
+
+  SELECT current_setting('request.method', true);
+
+Request Role and Search Path
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Because of :ref:`user_impersonation`, PostgREST sets the standard ``role``. You can get this in different ways:
+
+.. code-block:: postgresql
+
+  SELECT current_role;
+
+  SELECT current_user;
+
+  SELECT current_setting('role', true);
+
+Additionally it also sets the ``search_path`` based on :ref:`db-schemas` and :ref:`db-extra-search-path`.
+
+.. _guc_resp_hdrs:
+
+Response Headers
+~~~~~~~~~~~~~~~~
+
+You can set ``response.headers`` to add headers to the HTTP response. For instance, this statement would add caching headers to the response:
+
+.. code-block:: sql
+
+  -- tell client to cache response for two days
+
+  SELECT set_config('response.headers',
+    '[{"Cache-Control": "public"}, {"Cache-Control": "max-age=259200"}]', true);
+
+.. code-block:: http
+
+  HTTP/1.1 200 OK
+  Content-Type: application/json; charset=utf-8
+  Cache-Control: no-cache, no-store, must-revalidate
+
+Notice that the ``response.headers`` should be set to an *array* of single-key objects rather than a single multiple-key object. This is because headers such as ``Cache-Control`` or ``Set-Cookie`` need repeating when setting many values. An object would not allow the repeated key.
+
+.. note::
+
+  PostgREST provided headers such as ``Content-Type``, ``Location``, etc. can be overriden this way. Note that irrespective of overridden ``Content-Type`` response header, the content will still be converted to JSON, unless you also set :ref:`raw-media-types` to something like ``text/html``.
+
+.. _guc_resp_status:
+
+Response Status Code
+~~~~~~~~~~~~~~~~~~~~
+
+You can set the ``response.status`` to override the default status code PostgREST provides. For instance, the following function would replace the default ``200`` status code.
+
+.. code-block:: postgres
+
+   create or replace function teapot() returns json as $$
+   begin
+     perform set_config('response.status', '418', true);
+     return json_build_object('message', 'The requested entity body is short and stout.',
+                              'hint', 'Tip it over and pour it out.');
+   end;
+   $$ language plpgsql;
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /rpc/teapot HTTP/1.1
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/rpc/teapot" -i
+
+.. code-block:: http
+
+  HTTP/1.1 418 I'm a teapot
+
+  {
+    "message" : "The requested entity body is short and stout.",
+    "hint" : "Tip it over and pour it out."
+  }
+
+If the status code is standard, PostgREST will complete the status message(**I'm a teapot** in this example).
+
+.. _main_query:
+
+Main query
+----------
+
+The main query is generated by requesting :ref:`tables_views` or :ref:`s_procs`. All generated queries use prepared statements(:ref:`db-prepared-statements`).
+
+Transaction End
+---------------
+
+If the transaction doesn't fail, it will always end in a COMMIT. Unless :ref:`db-tx-end` is configured to ROLLBACK in any case or conditionally with ``Prefer: tx=rollback``. This can be used for testing purposes.
+
+Aborting transactions
+---------------------
+
+Any database failure(like a failed constraint) will result in a rollback of the transaction. You can also :ref:`RAISE an error inside a function <raise_error>` to cause a rollback.
+
+.. _pre-request:
+
+Pre-Request
+-----------
+
+The pre-request is a function that can run after the :ref:`tx_settings` are set and before the :ref:`main_query`. It's enabled with :ref:`db-pre-request`.
+
+This provides an opportunity to modify settings or raise an exception to prevent the request from completing.
+
+.. _pre_req_headers:
+
+Setting headers via pre-request
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As an example, let's add some cache headers for all requests that come from an Internet Explorer(6 or 7) browser.
+
+.. code-block:: postgresql
+
+   create or replace function custom_headers()
+   returns void as $$
+   declare
+     user_agent text := current_setting('request.headers', true)::json->>'user-agent';
+   begin
+     if user_agent similar to '%MSIE (6.0|7.0)%' then
+       perform set_config('response.headers',
+         '[{"Cache-Control": "no-cache, no-store, must-revalidate"}]', false);
+     end if;
+   end; $$ language plpgsql;
+
+   -- set this function on postgrest.conf
+   -- db-pre-request = custom_headers
+
+Now when you make a GET request to a table or view, you'll get the cache headers.
+
+.. tabs::
+
+  .. code-tab:: http
+
+    GET /people HTTP/1.1
+    User-Agent: Mozilla/4.01 (compatible; MSIE 6.0; Windows NT 5.1)
+
+  .. code-tab:: bash Curl
+
+    curl "http://localhost:3000/people" -i \
+     -H "User-Agent: Mozilla/4.01 (compatible; MSIE 6.0; Windows NT 5.1)"

docs/releases/v09.0.0.rst

@@ -1,6 +1,6 @@
 
-PostgREST 9.0.0
-===============
+9.0.0
+=====
 
 This major version is released with PostgreSQL 14 compatibility and is accompanied with new features and bug fixes. You can look at the detailed changelog and download the pre-compiled binaries on the `GitHub release page <https://github.com/PostgREST/postgrest/releases/tag/v9.0.0>`_.
 
@@ -12,7 +12,7 @@ PostgreSQL 14 compatibility
 
 PostgreSQL 14 Beta 1 tightened its GUC naming scheme making it impossible to use multiple dots (``.``) and dashes (``-``) on custom GUC parameters, this caused our :ref:`old HTTP Context <guc_legacy_names>` to fail across all requests. Thankfully, `@robertsosinski <https://github.com/robertsosinski>`_ got the PostgreSQL team to reconsider allowing multiple dots in the GUC name, allowing us to avoid a major breaking change. You can see the full discussion `here <https://www.postgresql.org/message-id/17045-6a4a9f0d1513f72b%40postgresql.org>`_.
 
-Still, dashes cannot be used on PostgreSQL 14 custom GUC parameters, so we changed our HTTP Context :ref:`to namespace using a mix of dots and JSON <guc_req_headers_cookies_claims>`. On older PostgreSQL versions we still use the :ref:`guc_legacy_names`. If you wish to use the new JSON GUCs on these versions, set the :ref:`db-use-legacy-gucs` config option to false.
+Still, dashes cannot be used on PostgreSQL 14 custom GUC parameters, so we changed our HTTP Context :ref:`to namespace using a mix of dots and JSON <guc_req_headers_cookies_claims>`. On older PostgreSQL versions we still use the settings legacy names. If you wish to use the new JSON GUCs on these versions, set the :ref:`db-use-legacy-gucs` config option to false.
 
 Resource Embedding with Top-level Filtering
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -50,7 +50,7 @@ Error messages
 Documentation improvements
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-* Added ``curl`` snippets to the :doc:`API <../api>` page.
+* Added ``curl`` snippets to the :ref:`api` page.
 
 * Added the :ref:`automatic_recovery` section.
 
@@ -74,7 +74,7 @@ Breaking changes
 
 * Dropped support for PostgreSQL 9.5 as it already reached its end-of-life according to `PostgreSQL versioning policy <https://www.postgresql.org/support/versioning/>`_.
 
-* Partitions of a `partitioned table <https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE>`_ are no longer included in the :doc:`../schema_cache`. This is so errors are not generated when doing resource embedding on partitioned tables.
+* Partitions of a `partitioned table <https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE>`_ are no longer included in the :ref:`schema_cache`. This is so errors are not generated when doing resource embedding on partitioned tables.
 
 * Dropped support for doing :ref:`hint_disamb` using dots instead of exclamation marks, e.g. doing ``select=*,projects.client_id(*)`` instead of ``select=*,projects!client_id(*)``). Using dots was undocumented and deprecated back in `v6.0.2 <https://github.com/PostgREST/postgrest/releases/tag/v6.0.2>`_.
 
@@ -110,7 +110,7 @@ Big thanks from the `PostgREST team <https://github.com/orgs/PostgREST/people>`_
     :width:  13em
 
 * Evans Fernandes
-* `Jan Sommer <https://github.com/nerfpops>`_
+* Jan Sommer
 * `Franz Gusenbauer <https://www.igutech.at/>`_
 * `Daniel Babiak <https://github.com/dbabiak>`_
 * Tsingson Qin

docs/releases/v09.0.1.rst

@@ -1,6 +1,6 @@
 
-PostgREST 9.0.1
-===============
+9.0.1
+=====
 
 This version includes important fixes for production environments and other miscellaneous fixes. You can download the pre-compiled binaries on the `GitHub release page <https://github.com/PostgREST/postgrest/releases/tag/v9.0.1>`_.
 
@@ -69,7 +69,7 @@ Big thanks from the `PostgREST team <https://github.com/orgs/PostgREST/people>`_
     :width:  13em
 
 * Evans Fernandes
-* `Jan Sommer <https://github.com/nerfpops>`_
+* Jan Sommer
 * `Franz Gusenbauer <https://www.igutech.at/>`_
 * `Daniel Babiak <https://github.com/dbabiak>`_
 * Tsingson Qin

docs/releases/v10.0.0.rst

@@ -1,6 +1,6 @@
 
-PostgREST 10.0.0
-================
+10.0.0
+======
 
 Features
 --------
@@ -73,9 +73,9 @@ Documentation improvements
 
 * Added the :ref:`file_descriptors` subsection.
 
-* Added a reference page for :doc:`Error documentation </errors>`.
+* Added a reference page for :ref:`error_source`.
 
-* Moved the :ref:`error_source` and the :ref:`status_codes` sections to the :doc:`errors reference page </errors>`.
+* Moved the :ref:`error_source` and the :ref:`status_codes` sections to the :ref:`errors reference page <error_source>`.
 
 * Moved the *Casting type to custom JSON* how-to to the :ref:`casting_range_to_json` subsection.
 
@@ -207,7 +207,7 @@ Big thanks from the `PostgREST team <https://github.com/orgs/PostgREST/people>`_
     :width:  13em
 
 * Evans Fernandes
-* `Jan Sommer <https://github.com/nerfpops>`_
+* Jan Sommer
 * `Franz Gusenbauer <https://www.igutech.at/>`_
 * `Daniel Babiak <https://github.com/dbabiak>`_
 * Tsingson Qin

docs/releases/v10.2.0.rst

@@ -1,6 +1,6 @@
 
-PostgREST 10.2.0
-================
+10.2.0
+======
 
 This minor version adds bug fixes and some features that provide stability to v10.0.0. These release notes include the changes added in versions `10.1.0 <https://github.com/PostgREST/postgrest/releases/tag/v10.1.0>`_, `10.1.1 <https://github.com/PostgREST/postgrest/releases/tag/v10.1.1>`_ and `10.1.2 <https://github.com/PostgREST/postgrest/releases/tag/v10.1.2>`_. You can look at the detailed changelog and download the pre-compiled binaries on the `GitHub release page <https://github.com/PostgREST/postgrest/releases/tag/v10.2.0>`_.
 
@@ -133,7 +133,7 @@ Big thanks from the `PostgREST team <https://github.com/orgs/PostgREST/people>`_
     :width:  13em
 
 * Evans Fernandes
-* `Jan Sommer <https://github.com/nerfpops>`_
+* Jan Sommer
 * `Franz Gusenbauer <https://www.igutech.at/>`_
 * `Daniel Babiak <https://github.com/dbabiak>`_
 * Tsingson Qin

docs/releases/v11.0.1.rst

@@ -0,0 +1,123 @@
+11.0.1
+======
+
+These release notes include the changes in versions `11.0.0 <https://github.com/PostgREST/postgrest/releases/tag/v11.0.0>`_ and `11.0.1 <https://github.com/PostgREST/postgrest/releases/tag/v11.0.1>`_. The detailed changelog and pre-compiled binaries are available on the `GitHub release pages <https://github.com/PostgREST/postgrest/releases>`_.
+
+Features
+--------
+
+Transactions
+~~~~~~~~~~~~
+
+- New configurable :ref:`isolation_lvl`.
+- New :ref:`impersonated_settings`.
+
+Inserts
+~~~~~~~
+
+- New :ref:`bulk_insert_default`.
+
+Functions
+~~~~~~~~~
+
+- Filters on Table-Valued Functions are now guaranteed to be inlined. See :ref:`function_inlining`.
+
+Horizontal Filtering
+~~~~~~~~~~~~~~~~~~~~
+
+- New ``isdistinct`` :ref:`operator <operators>`. It is a direct translation of `IS DINTINCT FROM <https://www.postgresql.org/docs/current/functions-comparison.html#FUNCTIONS-COMPARISON-PRED-TABLE>`_.
+- New ``and/all`` :ref:`modifiers`.
+
+Resource Embedding
+~~~~~~~~~~~~~~~~~~
+
+- New :ref:`top_level_order`.
+- New :ref:`spread_embed`.
+- New :ref:`null_embed`.
+- New :ref:`empty_embed`.
+
+OpenAPI
+~~~~~~~
+
+- Allow :ref:`overriding the OpenAPI title <open-api>`.
+- Allow :ref:`override_openapi`.
+
+Admin
+~~~~~
+
+- New :ref:`trace_header`.
+
+Documentation improvements
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- New :ref:`transactions` reference.
+- New :ref:`connection_pool` reference.
+- Split :ref:`api` into other sub-references.
+- Split :ref:`authn` into :ref:`db_authz` and :ref:`sql_user_management`.
+- Split :ref:`admin` into :ref:`intgrs`, :ref:`nginx` and :ref:`connection_pool`.
+
+Bug fixes
+---------
+
+- Please check the "Fixed" section on https://github.com/PostgREST/postgrest/releases/tag/v11.0.0 and https://github.com/PostgREST/postgrest/releases/tag/v11.0.1.
+
+Breaking changes
+----------------
+
+- Removed Bulk Call with ``Prefer: params=multiple-objects`` on Stored Procedures.
+- To comply with `RFC 9110 <https://www.rfc-editor.org/rfc/rfc9110.html#name-range>`_, the Range header is now only considered on GET.
+
+  + Other methods will ignore it and instead should use the ``limit/offset``.
+  + PUT requests no longer return an error when this header is present (using limit/offset still triggers the error)
+
+Thanks
+------
+
+Big thanks from the `PostgREST team <https://github.com/orgs/PostgREST/people>`_ to our sponsors!
+
+.. container:: image-container
+
+  .. image:: ../_static/cybertec-new.png
+    :target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
+    :width:  13em
+
+  .. image:: ../_static/2ndquadrant.png
+    :target: https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo
+    :width:  13em
+
+  .. image:: ../_static/retool.png
+    :target: https://retool.com/?utm_source=sponsor&utm_campaign=postgrest
+    :width:  13em
+
+  .. image:: ../_static/gnuhost.png
+    :target: https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest
+    :width:  13em
+
+  .. image:: ../_static/supabase.png
+    :target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
+    :width:  13em
+
+  .. image:: ../_static/oblivious.jpg
+    :target: https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest
+    :width:  13em
+
+* `Roboflow <https://github.com/roboflow>`_
+* Evans Fernandes
+* Jan Sommer
+* `Franz Gusenbauer <https://www.igutech.at/>`_
+* Zac Miller
+* Tsingson Qin
+* Michel Pelletier
+* Jay Hannah
+* Robert Stolarz
+* Nicholas DiBiase
+* Christopher Reid
+* Nathan Bouscal
+* Daniel Rafaj
+* David Fenko
+* Remo Rechkemmer
+* Severin Ibarluzea
+* Tom Saleeba
+* Pawel Tyll
+
+If you like to join them please consider `supporting PostgREST development <https://github.com/PostgREST/postgrest#user-content-supporting-development>`_.

docs/releases/v11.1.0.rst

@@ -0,0 +1,73 @@
+11.1.0
+======
+
+Features
+--------
+
+Connection Pool
+~~~~~~~~~~~~~~~
+
+- New :ref:`db-pool-max-idletime`. It enables a :ref:`dyn_conn_pool`.
+
+Configuration
+~~~~~~~~~~~~~
+
+- New :ref:`in_db_config`. It no longer requires high privileges and can be used on cloud-hosted databases.
+
+Bug fixes
+---------
+
+- Fix dropping schema cache reload notifications.
+- Stop automatic recovery when the error is "no password supplied".
+
+Thanks
+------
+
+Big thanks from the `PostgREST team <https://github.com/orgs/PostgREST/people>`_ to our sponsors!
+
+.. container:: image-container
+
+  .. image:: ../_static/cybertec-new.png
+    :target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
+    :width:  13em
+
+  .. image:: ../_static/2ndquadrant.png
+    :target: https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo
+    :width:  13em
+
+  .. image:: ../_static/retool.png
+    :target: https://retool.com/?utm_source=sponsor&utm_campaign=postgrest
+    :width:  13em
+
+  .. image:: ../_static/gnuhost.png
+    :target: https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest
+    :width:  13em
+
+  .. image:: ../_static/supabase.png
+    :target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
+    :width:  13em
+
+  .. image:: ../_static/oblivious.jpg
+    :target: https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest
+    :width:  13em
+
+* `Roboflow <https://github.com/roboflow>`_
+* Evans Fernandes
+* Jan Sommer
+* `Franz Gusenbauer <https://www.igutech.at/>`_
+* Zac Miller
+* Tsingson Qin
+* Michel Pelletier
+* Jay Hannah
+* Robert Stolarz
+* Nicholas DiBiase
+* Christopher Reid
+* Nathan Bouscal
+* Daniel Rafaj
+* David Fenko
+* Remo Rechkemmer
+* Severin Ibarluzea
+* Tom Saleeba
+* Pawel Tyll
+
+If you like to join them please consider `supporting PostgREST development <https://github.com/PostgREST/postgrest#user-content-supporting-development>`_.

docs/releases/v11.2.0.rst

@@ -0,0 +1,4 @@
+11.2.0
+======
+
+Starting from this version, the release notes will be posted on PostgREST's GitHub release page. Please see https://github.com/PostgREST/postgrest/releases/tag/v11.2.0.

docs/releases/v11.2.1.rst

@@ -0,0 +1,4 @@
+11.2.1
+======
+
+v11.2.1 is a maintenance release that addresses bugs introduced in v11.2.0. Also PostgreSQL 16 is now tested and confirmed to work. For more details, see https://github.com/PostgREST/postgrest/releases/tag/v11.2.1.

docs/releases/v5.2.0.rst

@@ -1,26 +0,0 @@
-v5.2.0
-======
-
-* Explicit qualification introduced in ``v5.0`` is no longer necessary, this section will not be included from this version onwards.  A :ref:`db-extra-search-path` configuration parameter was introduced to avoid the need to explictly qualify database objects. If you install PostgreSQL extensions on the ``public`` schema, they'll work normally from now on.
-
-* Now you can filter :ref:`tabs-cols-w-spaces`.
-
-* Included the ability to quote columns that have :ref:`reserved-chars`.
-
-* Thanks to `Zhou Feng <https://github.com/zhoufeng1989>`_, now is possible to reference an external file in :ref:`db-uri`.
-
-* Thanks to `Russell Davies <https://github.com/russelldavies>`_, Json Web Key Sets are now accepted by :ref:`jwt-secret`.
-
-Thanks
-------
-
-This release was made possible thanks to:
-
-* `Daniel Babiak <https://github.com/dbabiak>`_
-* `Michel Pelletier <https://github.com/michelp>`_
-* Tsingson Qin
-* Jay Hannah
-* Victor Adossi
-* Petr Beles
-
-If you like to join them please consider `supporting PostgREST development <https://github.com/PostgREST/postgrest#user-content-supporting-development>`_.

docs/releases/v6.0.2.rst

@@ -1,79 +0,0 @@
-.. |br| raw:: html
-
-   <br />
-
-v6.0.2
-======
-
-Full changelog is available at `PostgREST releases page <https://github.com/PostgREST/postgrest/releases>`_.
-
-Added
------
-
-* Ignoring payload keys for insert/update can be now done with the ``?columns`` query parameter. See :ref:`specify_columns`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* `websearch_to_tsquery <https://www.postgresql.org/docs/current/functions-textsearch.html#id-1.5.8.19.7.2.2.7.1.1.1>`_ can now be used
-  through the ``wfts`` operator. See :ref:`fts`.
-  |br| -- `@herulume <https://github.com/herulume>`_
-
-* Resource Embedding on materialized views is now possible. See :ref:`embedding_views`.
-  |br| -- `@vitorbaptista <https://github.com/vitorbaptista>`_
-
-* Bulk calling an RPC is now allowed. See :ref:`bulk_call`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* It's now possible to request a ``text/plain`` output. See :ref:`scalar_return_formats`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Config option for specifying PostgREST database pool timeout ``db-pool-timeout``.
-  |br| -- `@Qu4tro <https://github.com/Qu4tro>`_
-
-* Config option for binding the PostgREST web server to an unix socket. See :ref:`server-unix-socket`.
-  |br| -- `@Dansvidania <https://github.com/Dansvidania>`_
-
-* Config option for extending the supported media types. See :ref:`raw-media-types`.
-  |br| -- `@Dansvidania <https://github.com/Dansvidania>`_
-
-* We now offer an statically linked binary for Linux. Look for **postgrest-<version>-linux-x64-static.tar.xz** on the
-  `releases page <https://github.com/PostgREST/postgrest/releases>`_.
-  |br| -- `@clojurians-org <https://github.com/clojurians-org>`_
-
-* A :ref:`how_tos` section was added to the documentation.
-
-Changed
--------
-
-* ``SIGHUP`` support was removed. You should use ``SIGUSR1`` instead. See :ref:`schema_reloading`.
-
-* server-host default of ``127.0.0.1`` was changed to ``!4``. See :ref:`server-host`.
-
-Thanks
-------
-
-This release is sponsored by:
-
-.. image:: ../_static/cybertec.png
-  :target: https://www.cybertec-postgresql.com/en/
-  :width:  13em
-
-.. image:: ../_static/2ndquadrant.png
-  :target: https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo
-  :width:  13em
-
-.. image:: ../_static/retool.png
-  :target: https://retool.com/?utm_source=sponsor&utm_campaign=postgrest
-  :width:  13em
-
-* `Daniel Babiak <https://github.com/dbabiak>`_
-* Evans Fernandes
-* Tsingson Qin
-* Michel Pelletier
-* Jay Hannah
-* Robert Stolarz
-* Kofi Gumbs
-* Nicholas DiBiase
-* Christopher Reid
-* Nathan Bouscal
-
-If you like to join them please consider `supporting PostgREST development <https://github.com/PostgREST/postgrest#user-content-supporting-development>`_.

docs/releases/v7.0.0.rst

@@ -1,106 +0,0 @@
-.. |br| raw:: html
-
-   <br />
-
-v7.0.0
-======
-
-You can download this release at the `PostgREST v7.0.0 release page <https://github.com/PostgREST/postgrest/releases/tag/v7.0.0>`_.
-
-Added
------
-
-* Support for :ref:`Switching to a schema <multiple-schemas>` defined in :ref:`db-schemas`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_, `@mahmoudkassem <https://github.com/mahmoudkassem>`_
-
-* Support for :ref:`planned_count` and :ref:`estimated_count`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_, `@LorenzHenk <https://github.com/LorenzHenk>`_
-
-* Support for the :ref:`on_conflict <on_conflict>` query parameter to UPSERT based on a unique constraint.
-  |br| -- `@ykst <https://github.com/ykst>`_
-
-* Support for :ref:`Resource Embedding Disambiguation <embed_disamb>`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Support for user defined socket permission via :ref:`server-unix-socket-mode` config option
-  |br| -- `@Dansvidania <https://github.com/Dansvidania>`_
-
-* HTTP logic improvements -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-   + Support for HTTP HEAD requests.
-   + GUCs for :ref:`guc_req_path_method`.
-   + Support for :ref:`pre_req_headers`.
-   + Allow overriding provided headers(Content-Type, Location, etc) by :ref:`guc_resp_hdrs`
-   + Access to the ``Authorization`` header value through ``request.header.authorization``
-
-* Documentation improvements
-
-  + Explanation for :doc:`Schema Structure <../schema_structure>`.
-  + Reference for :ref:`s_proc_embed`.
-  + Reference for :ref:`mutation_embed`.
-  + Reference for filters on :ref:`json_columns`.
-  + How-to for :ref:`providing_img`.
-  + Added :ref:`community_tutorials` section.
-
-Fixed
------
-
-* Allow embedding a view when its source table foreign key is UNIQUE
-  |br| -- `@bwbroersma <https://github.com/bwbroersma>`_
-
-* ``Accept: application/vnd.pgrst.object+json`` behavior is now enforced for POST/PATCH/DELETE regardless of ``Prefer: return=minimal``
-  |br| -- `@dwagin <https://github.com/dwagin>`_
-
-* Fix self join resource embedding on PATCH
-  |br| -- `@herulume <https://github.com/herulume>`_, `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Allow PATCH/DELETE without ``Prefer: return=minimal`` on tables with no SELECT privileges
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Fix many to many resource embedding for RPC/PATCH
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-Changed
--------
-
-* :ref:`bulk_call` should now be done by specifying a ``Prefer: params=multiple-objects`` header. This fixes a performance regression when calling stored procedures.
-
-* Resource Embedding now outputs an error when multiple relationships between two tables are found, see :ref:`embed_disamb`.
-
-* ``server-proxy-uri`` config option has been renamed to :ref:`openapi-server-proxy-uri`.
-
-* Default Unix Socket file mode from 755 to 660
-
-Thanks
-------
-
-This release was made possible thanks to:
-
-.. image:: ../_static/cybertec.png
-  :target: https://www.cybertec-postgresql.com/en/
-  :width:  13em
-
-.. image:: ../_static/2ndquadrant.png
-  :target: https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo
-  :width:  13em
-
-.. image:: ../_static/retool.png
-  :target: https://retool.com/?utm_source=sponsor&utm_campaign=postgrest
-  :width:  13em
-
-* `Daniel Babiak <https://github.com/dbabiak>`_
-* Evans Fernandes
-* `Jan Sommer <https://github.com/nerfpops>`_
-* Tsingson Qin
-* Michel Pelletier
-* Jay Hannah
-* Robert Stolarz
-* Kofi Gumbs
-* Nicholas DiBiase
-* Christopher Reid
-* Nathan Bouscal
-* Daniel Rafaj
-* David Fenko
-
-
-If you like to join them please consider `supporting PostgREST development <https://github.com/PostgREST/postgrest#user-content-supporting-development>`_.

docs/releases/v7.0.1.rst

@@ -1,69 +0,0 @@
-.. |br| raw:: html
-
-   <br />
-
-v7.0.1
-======
-
-You can see the full changelog at `PostgREST v7.0.1 release page <https://github.com/PostgREST/postgrest/releases/tag/v7.0.1>`_.
-
-Fixed
------
-
-* Fix overloaded computed columns on RPC
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix POST, PATCH, DELETE with ``?select=`` and ``Prefer: return=minimal`` and PATCH with empty body
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix missing ``openapi-server-proxy-uri`` config option
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Fix ``Content-Profile`` not working for POST RPC
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Fix PUT restriction for including all columns in payload
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Documentation improvements
-
-  + Added package managers to :ref:`install`.
-
-Changed
--------
-
-* From this version onwards, the release page will include a single Linux static executable that can be run on any Linux distribution.
-
-Thanks
-------
-
-This release was made possible thanks to:
-
-.. image:: ../_static/cybertec.png
-  :target: https://www.cybertec-postgresql.com/en/
-  :width:  13em
-
-.. image:: ../_static/2ndquadrant.png
-  :target: https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo
-  :width:  13em
-
-.. image:: ../_static/retool.png
-  :target: https://retool.com/?utm_source=sponsor&utm_campaign=postgrest
-  :width:  13em
-
-* `Daniel Babiak <https://github.com/dbabiak>`_
-* Evans Fernandes
-* `Jan Sommer <https://github.com/nerfpops>`_
-* Tsingson Qin
-* Michel Pelletier
-* Jay Hannah
-* Robert Stolarz
-* Kofi Gumbs
-* Nicholas DiBiase
-* Christopher Reid
-* Nathan Bouscal
-* Daniel Rafaj
-* David Fenko
-
-
-If you'd like to join them, consider `supporting PostgREST development <https://github.com/PostgREST/postgrest#user-content-supporting-development>`_.

docs/releases/v8.0.0.rst

@@ -1,191 +0,0 @@
-.. |br| raw:: html
-
-   <br />
-
-v8.0.0
-======
-
-You can download this release at the `PostgREST v8.0.0 release page <https://github.com/PostgREST/postgrest/releases/tag/v8.0.0>`_.
-
-Added
------
-
-* Allow HTTP status override through the :ref:`response.status <guc_resp_status>` GUC.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Allow :ref:`s_procs_variadic`.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Allow :ref:`embedding_view_chains` recursively to any depth.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* No downtime when reloading the schema cache. See :ref:`schema_reloading`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Allow schema cache reloading using PostgreSQL :ref:`NOTIFY <schema_reloading_notify>` command. This enables :ref:`auto_schema_reloading`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Allow sending the header ``Prefer: headers-only`` to get a response with a ``Location`` header. See :ref:`insert`.
-  |br| -- `@laurenceisla <https://github.com/laurenceisla>`_
-
-* Allow :ref:`external_connection_poolers` such as PgBouncer in transaction pooling mode.
-  |br| -- `@laurenceisla <https://github.com/laurenceisla>`_
-
-* Allow :ref:`config_reloading` by sending a SIGUSR2 signal.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Allow ``Bearer`` with and without capitalization as authentication schema. See :ref:`client_auth`.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* :ref:`in_db_config` that can be :ref:`reloaded with NOTIFY <config_reloading_notify>`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Allow OPTIONS to generate HTTP methods based on views triggers. See :ref:`OPTIONS requests <options_requests>`.
-  |br| -- `@laurenceisla <https://github.com/laurenceisla>`_
-
-* Show timestamps for server diagnostic information. See :ref:`pgrst_logging`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Config options for showing a full OpenAPI output regardless of the JWT role privileges and for disabling it altogether. See :ref:`openapi-mode`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Config option for logging level. See :ref:`log-level`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Config option for enabling or disabling prepared statements. See :ref:`db-prepared-statements`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Config option for specifying how to terminate the transactions (allowing rollbacks, useful for testing). See :ref:`db-tx-end`.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Documentation improvements
-
-  + Added the :doc:`../schema_cache` page.
-  + Moved the :ref:`schema_reloading` reference from :doc:`../admin` to :doc:`../schema_cache`
-
-Changed
--------
-
-* Docker images are now optimized to be built from the scratch image. This reduces the compressed image size from over 30 MB to about 4 MB.
-  For more details, see `Docker image built with Nix <https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker#user-content-docker-image-built-with-nix>`_.
-  |br| -- `@monacoremo <https://github.com/monacoremo>`_
-
-* The Docker image no longer has an internal ``/etc/postgrest.conf`` file, you must use :ref:`env_variables_config` to configure it.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* The ``pg_listen`` `utility <https://github.com/begriffs/pg_listen>`_ is no longer needed to automatically reload the schema cache
-  and it's replaced entirely by database notifications. See :ref:`auto_schema_reloading`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* POST requests for insertions no longer include a ``Location`` header in the response by default and behave the same way as having a
-  ``Prefer: return=minimal`` header in the request. This prevents permissions errors when having a write-only table. See :ref:`insert`.
-  |br| -- `@laurenceisla <https://github.com/laurenceisla>`_
-
-* Modified the default logging level from ``info`` to ``error``. See :ref:`log-level`.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Changed the error message for a not found RPC on a stale schema (see :ref:`stale_function_signature`) and for the unsupported case of
-  overloaded functions with the same argument names but different types.
-  |br| -- `@laurenceisla <https://github.com/laurenceisla>`_
-
-* Changed the error message for the no relationship found error. See :ref:`stale_fk_relationships`.
-  |br| -- `@laurenceisla <https://github.com/laurenceisla>`_
-
-Fixed
------
-
-* Fix showing UNKNOWN on ``postgrest --help`` invocation.
-  |br| -- `@monacoremo <https://github.com/monacoremo>`_
-
-* Removed single column restriction to allow composite foreign keys in join tables.
-  |br| -- `@goteguru <https://github.com/goteguru>`_
-
-* Fix expired JWTs starting an empty transaction on the db.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Fix location header for POST request with ``select=`` without PK.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix error messages on connection failure for localized PostgreSQL on Windows.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix ``application/octet-stream`` appending ``charset=utf-8``.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Fix overloading of functions with unnamed arguments.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Return ``405 Method not Allowed`` for GET of volatile RPC instead of 500.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix RPC return type handling and embedding for domains with composite base type.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix embedding through views that have COALESCE with subselect.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix parsing of boolean config values for Docker environment variables, now it accepts double quoted truth values ``("true", "false")`` and numbers ``("1", "0")``.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix using ``app.settings.xxx`` config options in Docker, now they can be used as ``PGRST_APP_SETTINGS_xxx``.
-  |br| -- `@wolfgangwalther <https://github.com/wolfgangwalther>`_
-
-* Fix panic when attempting to run with unix socket on non-unix host and properly close unix domain socket on exit.
-  |br| -- `@monacoremo <https://github.com/monacoremo>`_
-
-* Disregard internal junction (in non-exposed schema) when embedding.
-  |br| -- `@steve-chavez <https://github.com/steve-chavez>`_
-
-* Fix requests for overloaded functions from HTML forms to no longer hang.
-  |br| -- `@laurenceisla <https://github.com/laurenceisla>`_
-
-Thanks
-------
-
-Big thanks from the `PostgREST team <https://github.com/orgs/PostgREST/people>`_ to our sponsors!
-
-.. container:: image-container
-
-  .. image:: ../_static/cybertec-new.png
-    :target: https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest
-    :width:  13em
-
-  .. image:: ../_static/2ndquadrant.png
-    :target: https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo
-    :width:  13em
-
-  .. image:: ../_static/retool.png
-    :target: https://retool.com/?utm_source=sponsor&utm_campaign=postgrest
-    :width:  13em
-
-  .. image:: ../_static/gnuhost.png
-    :target: https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest
-    :width:  13em
-
-  .. image:: ../_static/supabase.png
-    :target: https://supabase.com/?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage
-    :width:  13em
-
-  .. image:: ../_static/oblivious.jpg
-    :target: https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest
-    :width:  13em
-
-* Evans Fernandes
-* `Jan Sommer <https://github.com/nerfpops>`_
-* `Franz Gusenbauer <https://www.igutech.at/>`_
-* `Daniel Babiak <https://github.com/dbabiak>`_
-* Tsingson Qin
-* Michel Pelletier
-* Jay Hannah
-* Robert Stolarz
-* Nicholas DiBiase
-* Christopher Reid
-* Nathan Bouscal
-* Daniel Rafaj
-* David Fenko
-* Remo Rechkemmer
-* Severin Ibarluzea
-* Tom Saleeba
-* Pawel Tyll
-
-If you like to join them please consider `supporting PostgREST development <https://github.com/PostgREST/postgrest#user-content-supporting-development>`_.

docs/schema_structure.rst

@@ -1,96 +0,0 @@
-
-.. note::
-
-  This page is a work in progress.
-
-.. _schema_isolation:
-
-Schema Isolation
-================
-
-A PostgREST instance exposes all the tables, views, and stored procedures of a single `PostgreSQL schema <https://www.postgresql.org/docs/current/ddl-schemas.html>`_ (a namespace of database objects). This means private data or implementation details can go inside different private schemas and be invisible to HTTP clients.
-
-It is recommended that you don't expose tables on your API schema. Instead expose views and stored procedures which insulate the internal details from the outside world.
-This allows you to change the internals of your schema and maintain backwards compatibility. It also keeps your code easier to refactor, and provides a natural way to do API versioning.
-
-.. image:: _static/db.png
-
-.. _func_privs:
-
-Functions
-=========
-
-By default, when a function is created, the privilege to execute it is not restricted by role. The function access is ``PUBLIC`` — executable by all roles (more details at `PostgreSQL Privileges page <https://www.postgresql.org/docs/current/ddl-priv.html>`_). This is not ideal for an API schema. To disable this behavior, you can run the following SQL statement:
-
-.. code-block:: postgres
-
-  ALTER DEFAULT PRIVILEGES REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;
-
-This will change the privileges for all functions created in the future in all schemas. Currently there is no way to limit it to a single schema. In our opinion it's a good practice anyway.
-
-.. note::
-
-    It is however possible to limit the effect of this clause only to functions you define. You can put the above statement at the beginning of the API schema definition, and then at the end reverse it with:
-
-    .. code-block:: postgres
-
-        ALTER DEFAULT PRIVILEGES GRANT EXECUTE ON FUNCTIONS TO PUBLIC;
-
-    This will work because the :code:`alter default privileges` statement has effect on function created *after* it is executed. See `PostgreSQL alter default privileges <https://www.postgresql.org/docs/current/sql-alterdefaultprivileges.html>`_ for more details.
-
-After that, you'll need to grant EXECUTE privileges on functions explicitly:
-
-.. code-block:: postgres
-
-   GRANT EXECUTE ON FUNCTION login TO anonymous;
-   GRANT EXECUTE ON FUNCTION signup TO anonymous;
-
-You can also grant execute on all functions in a schema to a higher privileged role:
-
-.. code-block:: postgres
-
-    GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA api TO web_user;
-
-Security definer
-----------------
-
-A function is executed with the privileges of the user who calls it. This means that the user has to have all permissions to do the operations the procedure performs.
-If the function accesses private database objects, your :ref:`API roles <roles>` won't be able to successfully execute the function.
-
-Another option is to define the function with the :code:`SECURITY DEFINER` option. Then only one permission check will take place, the permission to call the function, and the operations in the function will have the authority of the user who owns the function itself.
-
-.. code-block:: postgres
-
-  -- login as a user wich has privileges on the private schemas
-
-  -- create a sample function
-  create or replace function login(email text, pass text) returns jwt_token as $$
-  begin
-    -- access to a private schema called 'auth'
-    select auth.user_role(email, pass) into _role;
-    -- other operations
-    -- ...
-  end;
-  $$ language plpgsql security definer;
-
-Note the ``SECURITY DEFINER`` keywords at the end of the function. See `PostgreSQL documentation <https://www.postgresql.org/docs/current/sql-createfunction.html#SQL-CREATEFUNCTION-SECURITY>`_ for more details.
-
-Views
-=====
-
-Views are invoked with the privileges of the view owner, much like stored procedures with the ``SECURITY DEFINER`` option. When created by a SUPERUSER role, all `row-level security <https://www.postgresql.org/docs/current/ddl-rowsecurity.html>`_ will be bypassed unless a different, non-SUPERUSER owner is specified.
-
-For changing this, we can create a non-SUPERUSER role and make this role the view's owner.
-
-.. code-block:: postgres
-
-  CREATE ROLE api_views_owner NOINHERIT;
-  ALTER VIEW sample_view OWNER TO api_views_owner;
-
-Rules
------
-
-Insertion on views with complex `rules <https://www.postgresql.org/docs/current/sql-createrule.html>`_ might not work out of the box with PostgREST.
-It's recommended that you `use triggers instead of rules <https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_use_rules>`_.
-If you want to keep using rules, a workaround is to wrap the view insertion in a stored procedure and call it through the :ref:`s_procs` interface.
-For more details, see this `github issue <https://github.com/PostgREST/postgrest/issues/1283>`_.

docs/shell.nix

@@ -14,4 +14,8 @@ pkgs.mkShell {
     docs.dictcheck
     docs.linkcheck
   ];
+
+  shellHook = ''
+    export HISTFILE=.history
+  '';
 }

docs/tutorials/tut0.rst

@@ -160,7 +160,7 @@ PostgREST can use a configuration file to tell it how to connect to the database
   db-schemas = "api"
   db-anon-role = "web_anon"
 
-The configuration file has other :doc:`options <../configuration>`, but this is all we need. 
+The configuration file has other :ref:`options <configuration>`, but this is all we need.
 If you are not using Docker, make sure that your port number is correct and replace `postgres` with the name of the database where you added the todos table.
 
 Now run the server:

nix/README.md

@@ -5,24 +5,14 @@ for developing, testing and building PostgREST.
 
 ## Getting started with Nix
 
-You'll need to [get Nix](https://nixos.org/download.html). The installer will
-create your Nix store in the `/nix/` directory, where all build artifacts and
-their dependencies will be stored. It will also link the Nix executables like
-`nix-env`, `nix-build` and `nix-shell` into your PATH. Nix will manage all
-other PostgREST dependencies from here on out. To clean up older build
-artifacts from the `/nix/store`, you can run `nix-collect-garbage`.
-
-If you are on a system that does not support nix, for example Windows, you can
-run the nix development environment in a docker container. Inside the `nix/`
-directory run `docker-compose run --rm nix` to start the docker container. This
-will set up the binary cache and launch `nix-shell` automatically.
+You'll need to [get Nix](https://nixos.org/download.html). Follow the recommended installation for your operating system from the official download website.
 
 ## Building PostgREST
 
 To build PostgREST from your local checkout of the repository, run:
 
 ```bash
-nix-build --attr postgrestPackage
+$ nix-build --attr postgrestPackage
 
 ```
 
@@ -39,10 +29,10 @@ We recommend that you use the PostgREST binary cache on
 
 ```bash
 # Install cachix:
-nix-env -iA cachix -f https://cachix.org/api/v1/install
+$ nix-env -iA cachix -f https://cachix.org/api/v1/install
 
 # Set cachix up to use the PostgREST binary cache:
-cachix use postgrest
+$ cachix use postgrest
 
 ```
 
@@ -56,7 +46,7 @@ following command will put you into a new shell that has GHC and Cabal on the
 PATH:
 
 ```bash
-nix-shell
+$ nix-shell
 
 ```
 
@@ -146,10 +136,10 @@ Note: Once inside nix-shell, the utilities work from any directory inside
 the PostgREST repo. Paths are resolved relative to the repo root:
 
 ```bash
-$ cd src
+[nix-shell]$ cd src
 # Even though the current directory is ./src, the config path must still start
 # from the repo root:
-$ postgrest-run test/io/configs/simple.conf
+[nix-shell]$ postgrest-run test/io/configs/simple.conf
 ```
 
 ## Testing
@@ -177,21 +167,21 @@ run with `postgrest-test-io`. The test runner under the hood is
 
 ```bash
 # Filter the tests to run by name, including all that contain 'config':
-postgrest-test-io -k config
+[nix-shell]$ postgrest-test-io -k config
 
 # Run tests in parallel using xdist, specifying the number of processes:
-postgrest-test-io -n auto
-postgrest-test-io -n 8
+[nix-shell]$ postgrest-test-io -n auto
+[nix-shell]$ postgrest-test-io -n 8
 ```
 
 The memory tests check that we don't surpass a memory threshold for big request bodies.
 
 ```bash
 # Build the dependencies needed for the memory test
-nix-shell --arg memory true
+$ nix-shell --arg memory true
 
 # Run the memory test
-postgrest-test-memory
+[nix-shell]$ postgrest-test-memory
 ```
 
 The loadtests ensure that performance doesn't drop on a change. Underlyingly they use
@@ -199,19 +189,25 @@ The loadtests ensure that performance doesn't drop on a change. Underlyingly the
 
 ```bash
 # Run the loadtests on the latest commit(HEAD)
-postgrest-loadtest
+[nix-shell]$ postgrest-loadtest
 
 # You can loadtest comparing to a different branch
-postgrest-loadtest-against master
+[nix-shell]$ postgrest-loadtest-against master
+
+# You can simulate latency client/postgrest and postgrest/database
+[nix-shell]$ PGRST_DELAY=5ms PGDELAY=5ms postgrest-loadtest
+
+# You can build postgrest directly with cabal for faster iteration
+[nix-shell]$ PGRST_BUILD_CABAL=1 postgrest-loadtest
 
 # Produce a markdown report to be used on CI
-postgrest-loadtest-report
+[nix-shell]$ postgrest-loadtest-report
 ```
 
 doctests for some of our modules are also available:
 
 ```bash
-postgrest-test-doctest
+[nix-shell]$ postgrest-test-doctest
 ```
 
 ## Code coverage
@@ -220,11 +216,11 @@ Code coverage is available under the `postgrest-coverage` command. This will pro
 
 ```bash
 # Will run all the tests and produce a coverage dir
-postgrest-coverage
+[nix-shell]$ postgrest-coverage
 
 # Visualize the output
-cd coverage
-python -mSimpleHTTPServer 8080
+[nix-shell]$ cd coverage
+[nix-shell]$ python -mSimpleHTTPServer 8080
 ```
 
 ## Linting and styling code

nix/nixpkgs-version.nix

@@ -1,6 +1,6 @@
 # Pinned version of Nixpkgs, generated with postgrest-nixpkgs-upgrade.
 {
-  date = "2023-01-12";
-  rev = "92f9580a4c369b4b51a7b6a5e77da43720134c9f";
-  tarballHash = "0w9bz4f2bmkj4a59n4z279zcgs9clyc40a4ny312rafyaknzghvw";
+  date = "2023-03-25";
+  rev = "dbf5322e93bcc6cfc52268367a8ad21c09d76fea";
+  tarballHash = "0lwk4v9dkvd28xpqch0b0jrac4xl9lwm6snrnzx8k5lby72kmkng";
 }

nix/overlays/default.nix

@@ -7,4 +7,5 @@
   postgresql-default = import ./postgresql-default.nix;
   postgresql-legacy = import ./postgresql-legacy.nix;
   postgresql-future = import ./postgresql-future.nix;
+  slocat = import ./slocat.nix;
 }

nix/overlays/haskell-packages.nix

@@ -43,8 +43,8 @@ let
         (prev.callHackageDirect
           {
             pkg = "hasql-notifications";
-            ver = "0.2.0.4";
-            sha256 = "sha256-fm1xiDyvDkb5WLOJ73/s8wrWEW23XFS7luAv2brfr8I=";
+            ver = "0.2.0.6";
+            sha256 = "sha256-7PyFlB2B70njudOjaX6tk1m77ol9vnF5fI0LF86kVAI=";
           }
           { });
 
@@ -52,8 +52,8 @@ let
         (prev.callHackageDirect
           {
             pkg = "hasql-pool";
-            ver = "0.9";
-            sha256 = "sha256-5UshbbaBVY8eJ/9VagNVVxonRwMcd7UmGqDc35pJNFY=";
+            ver = "0.10";
+            sha256 = "sha256-kHzoqtNV9BFWnn1h560JRqMooQRwxokVKgDRBexamNI=";
           }
           { });
     } // extraOverrides final prev;

nix/overlays/postgresql-future.nix

@@ -4,16 +4,16 @@ self: super:
 {
   ## Example for including a postgresql version from a specific nixpks commit:
   ##
-  #  postgresql_14 =
-  #    let
-  #      rev = "76b1e16c6659ccef7187ca69b287525fea133244";
-  #      tarballHash = "1vsahpcx80k2bgslspb0sa6j4bmhdx77sw6la455drqcrqhdqj6a";
-  #
-  #      pinnedPkgs =
-  #        builtins.fetchTarball {
-  #          url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
-  #          sha256 = tarballHash;
-  #        };
-  #    in
-  #    (import pinnedPkgs { }).pkgs.postgresql_14;
+  postgresql_16 =
+    let
+      rev = "5148520bfab61f99fd25fb9ff7bfbb50dad3c9db";
+      tarballHash = "1dfjmz65h8z4lk845724vypzmf3dbgsdndjpj8ydlhx6c7rpcq3p";
+
+      pinnedPkgs =
+        builtins.fetchTarball {
+          url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
+          sha256 = tarballHash;
+        };
+    in
+    (import pinnedPkgs { }).pkgs.postgresql_16;
 }

nix/overlays/slocat.nix

@@ -0,0 +1,13 @@
+final: prev:
+{
+  slocat = prev.buildGoModule {
+    name = "slocat";
+    src = prev.fetchFromGitHub {
+      owner = "robx";
+      repo = "slocat";
+      rev = "52e7512c6029fd00483e41ccce260a3b4b9b3b64";
+      sha256 = "sha256-qn6luuh5wqREu3s8RfuMCP5PKdS2WdwPrujRYTpfzQ8=";
+    };
+    vendorSha256 = "sha256-pQpattmS9VmO3ZIQUFn66az8GSmB4IvYhTTCFn6SUmo=";
+  };
+}

nix/static-haskell-package.nix

@@ -59,4 +59,7 @@ let
   survey =
     import "${patched-static-haskell-nix}/survey" { inherit normalPkgs compiler defaultCabalPackageVersionComingWithGhc; };
 in
-survey.haskellPackages."${name}"
+{
+  inherit survey;
+  package = survey.haskellPackages."${name}";
+}

nix/tools/cabalTools.nix

@@ -37,12 +37,22 @@ let
     checkedShellScript
       {
         name = "postgrest-run";
-        docs = "Run PostgREST after buidling it interactively with cabal-install";
-        args = [ "ARG_LEFTOVERS([PostgREST arguments])" ];
+        docs = "Run PostgREST after building it interactively with cabal-install";
+        args =
+          [
+            "ARG_USE_ENV([PGRST_DB_ANON_ROLE], [postgrest_test_anonymous], [PostgREST anonymous role])"
+            "ARG_USE_ENV([PGRST_DB_POOL], [1], [PostgREST pool size])"
+            "ARG_USE_ENV([PGRST_DB_POOL_ACQUISITION_TIMEOUT], [1], [PostgREST pool size])"
+            "ARG_LEFTOVERS([PostgREST arguments])"
+          ];
         inRootDir = true;
         withEnv = postgrest.env;
       }
       ''
+        export PGRST_DB_ANON_ROLE
+        export PGRST_DB_POOL
+        export PGRST_DB_POOL_ACQUISITION_TIMEOUT
+
         exec ${cabal-install}/bin/cabal v2-run ${devCabalOptions} --verbose=0 -- \
           postgrest "''${_arg_leftovers[@]}"
       '';

nix/tools/devTools.nix

@@ -304,4 +304,5 @@ buildToolbox
     hsieGraphModules
     hsieGraphSymbols
   ];
+  extra = { inherit pushCachix; };
 }

nix/tools/loadtest.nix

@@ -56,11 +56,14 @@ let
         export PGRST_LOG_LEVEL="crit"
 
         mkdir -p "$(dirname "$_arg_output")"
+        abs_output="$(realpath "$_arg_output")"
 
         # shellcheck disable=SC2145
         ${withTools.withPg} --fixtures "$_arg_testdir"/fixtures.sql \
+        ${withTools.withSlowPg} \
         ${withTools.withPgrst} \
-        sh -c "cd \"$_arg_testdir\" && ${runner} -targets targets.http -output \"$_arg_output\" \"''${_arg_leftovers[@]}\""
+        ${withTools.withSlowPgrst} \
+        sh -c "cd \"$_arg_testdir\" && ${runner} -targets targets.http -output \"$abs_output\" \"''${_arg_leftovers[@]}\""
         ${vegeta}/bin/vegeta report -type=text "$_arg_output"
       '';
 

nix/tools/release/default.nix

@@ -72,14 +72,15 @@ let
         today_date="$(date '+%Y%m%d')"
         today_date_for_changelog="$(date '+%Y-%m-%d')"
         bump_pre="$major.$minor.$patch.$today_date"
+        bump_pre_minor="$major.$((minor+1)).0.$today_date"
         bump_patch="$major.$minor.$((patch+1))"
         bump_minor="$major.$((minor+1)).0"
         bump_major="$((major+1)).0.0"
 
         PS3="Please select the new version: "
-        select new_version in "$bump_pre" "$bump_patch" "$bump_minor" "$bump_major"; do
+        select new_version in "$bump_pre" "$bump_pre_minor" "$bump_patch" "$bump_minor" "$bump_major"; do
           case "$REPLY" in
-            1|2|3|4)
+            1|2|3|4|5)
               echo "Selected $new_version"
               break
               ;;
@@ -95,7 +96,7 @@ let
         echo "Committing ..."
         git add postgrest.cabal > /dev/null
 
-        if [[ "$new_version" != "$bump_pre" ]]; then
+        if [[ "$new_version" != "$bump_pre" && "$new_version" != "$bump_pre_minor" ]]; then
           echo "Updating CHANGELOG.md ..."
           sed -i -E "s/Unreleased/&\n\n## [$new_version] - $today_date_for_changelog/" CHANGELOG.md > /dev/null
           git add CHANGELOG.md > /dev/null

nix/tools/withTools.nix

@@ -1,6 +1,7 @@
 { bash-completion
 , buildToolbox
 , cabal-install
+, cabalTools
 , checkedShellScript
 , curl
 , devCabalOptions
@@ -8,15 +9,20 @@
 , lib
 , postgresqlVersions
 , postgrest
+, slocat
 , writeText
 }:
 let
   withTmpDb =
     { name, postgresql }:
+    let
+      commandName = "postgrest-with-${name}";
+      superuserRole = "postgres";
+    in
     checkedShellScript
       {
-        name = "postgrest-with-${name}";
-        docs = "Run the given command in a temporary database with ${name}";
+        name = commandName;
+        docs = "Run the given command in a temporary database with ${name}. If you wish to mutate the database, login with the '${superuserRole}' role.";
         args =
           [
             "ARG_OPTIONAL_SINGLE([fixtures], [f], [SQL file to load fixtures from], [test/spec/fixtures/load.sql])"
@@ -25,6 +31,8 @@ let
             "ARG_USE_ENV([PGUSER], [postgrest_test_authenticator], [Authenticator PG role])"
             "ARG_USE_ENV([PGDATABASE], [postgres], [PG database name])"
             "ARG_USE_ENV([PGRST_DB_SCHEMAS], [test], [Schema to expose])"
+            "ARG_USE_ENV([PGTZ], [utc], [Timezone to use])"
+            "ARG_USE_ENV([PGOPTIONS], [-c search_path=public,test], [PG options to use])"
           ];
         positionalCompletion = "_command";
         inRootDir = true;
@@ -53,6 +61,8 @@ let
         export PGUSER
         export PGDATABASE
         export PGRST_DB_SCHEMAS
+        export PGTZ
+        export PGOPTIONS
 
         HBA_FILE="$tmpdir/pg_hba.conf"
         echo "local $PGDATABASE some_protected_user password" > "$HBA_FILE"
@@ -61,12 +71,13 @@ let
         log "Initializing database cluster..."
         # We try to make the database cluster as independent as possible from the host
         # by specifying the timezone, locale and encoding.
-        PGTZ=UTC initdb --no-locale --encoding=UTF8 --nosync -U "$PGUSER" --auth=trust \
+        # initdb -U creates a superuser(man initdb)
+        PGTZ=UTC initdb --no-locale --encoding=UTF8 --nosync -U "${superuserRole}" --auth=trust \
           >> "$setuplog"
 
         log "Starting the database cluster..."
         # Instead of listening on a local port, we will listen on a unix domain socket.
-        pg_ctl -l "$tmpdir/db.log" -w start -o "-F -c listen_addresses=\"\" -c hba_file=$HBA_FILE -k $PGHOST -c log_statement=\"all\"" \
+        pg_ctl -l "$tmpdir/db.log" -w start -o "-F -c listen_addresses=\"\" -c hba_file=$HBA_FILE -k $PGHOST -c log_statement=\"all\" " \
           >> "$setuplog"
 
         # shellcheck disable=SC2317
@@ -77,10 +88,17 @@ let
         }
         trap stop EXIT
 
-        log "Loading fixtures..."
-        psql -v ON_ERROR_STOP=1 -f "$_arg_fixtures" >> "$setuplog"
+        log "Creating a minimally privileged $PGUSER connection role..."
+        createuser "$PGUSER" -U "${superuserRole}" --host="$tmpdir/socket" --no-createdb --no-inherit --no-superuser --no-createrole --no-replication --login
+
+        log "Loading fixtures under the ${superuserRole} role..."
+        psql -U "${superuserRole}" -v PGUSER="$PGUSER" -v ON_ERROR_STOP=1 -f "$_arg_fixtures" >> "$setuplog"
 
         log "Done. Running command..."
+
+        echo "${commandName}: You can connect with: psql 'postgres:///$PGDATABASE?host=$tmpdir/socket' -U ${superuserRole}"
+        echo "${commandName}: You can tail the logs with: tail -f $tmpdir/db.log"
+
         ("$_arg_command" "''${_arg_leftovers[@]}")
       '';
 
@@ -130,6 +148,81 @@ let
 
   withPg = builtins.head withPgVersions;
 
+  withSlowPg =
+    checkedShellScript
+      {
+        name = "postgrest-with-slow-pg";
+        docs = "Run the given command with simulated high latency postgresql";
+        args =
+          [
+            "ARG_POSITIONAL_SINGLE([command], [Command to run])"
+            "ARG_LEFTOVERS([command arguments])"
+            "ARG_USE_ENV([PGHOST], [], [PG host (socket name)])"
+            "ARG_USE_ENV([PGDELAY], [0ms], [extra PG latency (duration)])"
+          ];
+        positionalCompletion = "_command";
+        inRootDir = true;
+        redirectTixFiles = false;
+        withTmpDir = true;
+      }
+      ''
+        delay="''${PGDELAY:-0ms}"
+        echo "delaying data to/from postgres by $delay"
+
+        REALPGHOST="$PGHOST"
+        export PGHOST="$tmpdir/socket"
+        mkdir -p "$PGHOST"
+
+        ${slocat}/bin/slocat -delay "$delay" -src "$PGHOST/.s.PGSQL.5432" -dst "$REALPGHOST/.s.PGSQL.5432" &
+        SLOCAT_PID=$!
+        # shellcheck disable=SC2317
+        stop_slocat() {
+          kill "$SLOCAT_PID" || true
+          wait "$SLOCAT_PID" || true
+        }
+        trap stop_slocat EXIT
+        sleep 1 # should wait for socket file to appear instead
+
+        ("$_arg_command" "''${_arg_leftovers[@]}")
+      '';
+
+  withSlowPgrst =
+    checkedShellScript
+      {
+        name = "postgrest-with-slow-postgrest";
+        docs = "Run the given command with simulated high latency postgrest";
+        args =
+          [
+            "ARG_POSITIONAL_SINGLE([command], [Command to run])"
+            "ARG_LEFTOVERS([command arguments])"
+            "ARG_USE_ENV([PGRST_SERVER_UNIX_SOCKET], [], [PostgREST host (socket name)])"
+            "ARG_USE_ENV([PGRST_DELAY], [0ms], [extra PostgREST latency (duration)])"
+          ];
+        positionalCompletion = "_command";
+        inRootDir = true;
+        redirectTixFiles = false;
+        withTmpDir = true;
+      }
+      ''
+        delay="''${PGRST_DELAY:-0ms}"
+        echo "delaying data to/from PostgREST by $delay"
+
+        REAL_PGRST_SERVER_UNIX_SOCKET="$PGRST_SERVER_UNIX_SOCKET"
+        export PGRST_SERVER_UNIX_SOCKET="$tmpdir/postgrest.socket"
+
+        ${slocat}/bin/slocat -delay "$delay" -src "$PGRST_SERVER_UNIX_SOCKET" -dst "$REAL_PGRST_SERVER_UNIX_SOCKET" &
+        SLOCAT_PID=$!
+        # shellcheck disable=SC2317
+        stop_slocat() {
+          kill "$SLOCAT_PID" || true
+          wait "$SLOCAT_PID" || true
+        }
+        trap stop_slocat EXIT
+        sleep 1 # should wait for socket file to appear instead
+
+        ("$_arg_command" "''${_arg_leftovers[@]}")
+      '';
+
   withGit =
     let
       name = "postgrest-with-git";
@@ -250,16 +343,23 @@ let
         export PGRST_SERVER_UNIX_SOCKET="$tmpdir"/postgrest.socket
 
         rm -f result
-        echo -n "Building postgrest... "
-        nix-build -A postgrestPackage > "$tmpdir"/build.log 2>&1 || {
-          echo "failed, output:"
-          cat "$tmpdir"/build.log
-          exit 1
-        }
+        if [ -z "''${PGRST_BUILD_CABAL:-}" ]; then
+          echo -n "Building postgrest (nix)... "
+          nix-build -A postgrestPackage > "$tmpdir"/build.log 2>&1 || {
+            echo "failed, output:"
+            cat "$tmpdir"/build.log
+            exit 1
+          }
+          PGRST_CMD=./result/bin/postgrest
+        else
+          echo -n "Building postgrest (cabal)... "
+          postgrest-build
+          PGRST_CMD=postgrest-run
+        fi
         echo "done."
 
         echo -n "Starting postgrest... "
-        ./result/bin/postgrest ${legacyConfig} > "$tmpdir"/run.log 2>&1 &
+        $PGRST_CMD ${legacyConfig} > "$tmpdir"/run.log 2>&1 &
         pid=$!
         # shellcheck disable=SC2317
         cleanup() {
@@ -281,7 +381,7 @@ in
 buildToolbox
 {
   name = "postgrest-with";
-  tools = [ withPgAll withGit withPgrst ] ++ withPgVersions;
+  tools = [ withPgAll withGit withPgrst withSlowPg withSlowPgrst ] ++ withPgVersions;
   # make withTools available for other nix files
-  extra = { inherit withGit withPg withPgAll withPgrst; };
+  extra = { inherit withGit withPg withPgAll withPgrst withSlowPg withSlowPgrst; };
 }

postgrest.cabal

@@ -1,5 +1,5 @@
 name:               postgrest
-version:            10.2.0
+version:            11.2.2
 synopsis:           REST API for any Postgres database
 description:        Reads the schema of a PostgreSQL database and creates RESTful routes
                     for tables, views, and functions, supporting all HTTP methods that security
@@ -34,7 +34,8 @@ library
   default-extensions: OverloadedStrings
                       NoImplicitPrelude
   hs-source-dirs:     src
-  exposed-modules:    PostgREST.App
+  exposed-modules:    PostgREST.Admin
+                      PostgREST.App
                       PostgREST.AppState
                       PostgREST.Auth
                       PostgREST.CLI
@@ -46,8 +47,9 @@ library
                       PostgREST.Cors
                       PostgREST.SchemaCache
                       PostgREST.SchemaCache.Identifiers
-                      PostgREST.SchemaCache.Proc
+                      PostgREST.SchemaCache.Routine
                       PostgREST.SchemaCache.Relationship
+                      PostgREST.SchemaCache.Representations
                       PostgREST.SchemaCache.Table
                       PostgREST.Error
                       PostgREST.Logger
@@ -60,6 +62,7 @@ library
                       PostgREST.Plan.CallPlan
                       PostgREST.Plan.MutatePlan
                       PostgREST.Plan.ReadPlan
+                      PostgREST.Plan.Types
                       PostgREST.RangeQuery
                       PostgREST.ApiRequest
                       PostgREST.ApiRequest.Preferences
@@ -69,7 +72,6 @@ library
                       PostgREST.Response.OpenAPI
                       PostgREST.Response.GucHeader
                       PostgREST.Version
-                      PostgREST.Workers
   other-modules:      Paths_postgrest
   build-depends:      base                      >= 4.9 && < 4.17
                     , HTTP                      >= 4000.3.7 && < 4000.5
@@ -85,12 +87,13 @@ library
                     , contravariant-extras      >= 0.3.3 && < 0.4
                     , cookie                    >= 0.4.2 && < 0.5
                     , either                    >= 4.4.1 && < 5.1
+                    , extra                     >= 1.7.0 && < 2.0
                     , fuzzyset                  >= 0.2.3
                     , gitrev                    >= 1.2 && < 1.4
                     , hasql                     >= 1.6.1.1 && < 1.7
                     , hasql-dynamic-statements  >= 0.3.1 && < 0.4
-                    , hasql-notifications       >= 0.1 && < 0.3
-                    , hasql-pool                >= 0.9 && < 0.10
+                    , hasql-notifications       >= 0.2.0.6 && < 0.3
+                    , hasql-pool                >= 0.10 && < 0.11
                     , hasql-transaction         >= 1.0.1 && < 1.1
                     , heredoc                   >= 0.2 && < 0.3
                     , http-types                >= 0.12.2 && < 0.13
@@ -186,6 +189,8 @@ test-suite spec
                       Feature.CorsSpec
                       Feature.ExtraSearchPathSpec
                       Feature.LegacyGucsSpec
+                      Feature.NoSuperuserSpec
+                      Feature.ObservabilitySpec
                       Feature.OpenApi.DisabledOpenApiSpec
                       Feature.OpenApi.IgnorePrivOpenApiSpec
                       Feature.OpenApi.OpenApiSpec
@@ -210,15 +215,17 @@ test-suite spec
                       Feature.Query.QuerySpec
                       Feature.Query.RangeSpec
                       Feature.Query.RawOutputTypesSpec
+                      Feature.Query.RelatedQueriesSpec
                       Feature.Query.RpcSpec
                       Feature.Query.SingularSpec
+                      Feature.Query.NullsStrip
+                      Feature.Query.SpreadQueriesSpec
                       Feature.Query.UnicodeSpec
                       Feature.Query.UpdateSpec
                       Feature.Query.UpsertSpec
                       Feature.RollbackSpec
                       Feature.RpcPreRequestGucsSpec
                       SpecHelper
-                      TestTypes
   build-depends:      base              >= 4.9 && < 4.17
                     , aeson             >= 2.0.3 && < 2.2
                     , aeson-qq          >= 0.8.1 && < 0.9
@@ -228,7 +235,7 @@ test-suite spec
                     , bytestring        >= 0.10.8 && < 0.12
                     , case-insensitive  >= 1.2 && < 1.3
                     , containers        >= 0.5.7 && < 0.7
-                    , hasql-pool        >= 0.9 && < 0.10
+                    , hasql-pool        >= 0.10 && < 0.11
                     , hasql-transaction >= 1.0.1 && < 1.1
                     , heredoc           >= 0.2 && < 0.3
                     , hspec             >= 2.3 && < 2.10

src/PostgREST/Admin.hs

@@ -0,0 +1,87 @@
+{-# LANGUAGE NamedFieldPuns  #-}
+{-# LANGUAGE RecordWildCards #-}
+
+module PostgREST.Admin
+  ( runAdmin
+  ) where
+
+import qualified Data.Text                 as T
+import qualified Hasql.Session             as SQL
+import qualified Network.HTTP.Types.Status as HTTP
+import qualified Network.Wai               as Wai
+import qualified Network.Wai.Handler.Warp  as Warp
+
+import Control.Monad.Extra (whenJust)
+
+import Network.Socket
+import Network.Socket.ByteString
+
+import PostgREST.AppState (AppState)
+import PostgREST.Config   (AppConfig (..))
+
+import qualified PostgREST.AppState as AppState
+
+import Protolude
+
+runAdmin :: AppConfig -> AppState -> Warp.Settings -> IO ()
+runAdmin conf@AppConfig{configAdminServerPort} appState settings =
+  whenJust configAdminServerPort $ \adminPort -> do
+    AppState.logWithZTime appState $ "Admin server listening on port " <> show adminPort
+    void . forkIO $ Warp.runSettings (settings & Warp.setPort adminPort) adminApp
+  where
+    adminApp = admin appState conf
+
+-- | PostgREST admin application
+admin :: AppState.AppState -> AppConfig -> Wai.Application
+admin appState appConfig req respond  = do
+  isMainAppReachable  <- any isRight <$> reachMainApp appConfig
+  isSchemaCacheLoaded <- isJust <$> AppState.getSchemaCache appState
+  isConnectionUp      <-
+    if configDbChannelEnabled appConfig
+      then AppState.getIsListenerOn appState
+      else isRight <$> AppState.usePool appState (SQL.sql "SELECT 1")
+
+  case Wai.pathInfo req of
+    ["ready"] ->
+      respond $ Wai.responseLBS (if isMainAppReachable && isConnectionUp && isSchemaCacheLoaded then HTTP.status200 else HTTP.status503) [] mempty
+    ["live"] ->
+      respond $ Wai.responseLBS (if isMainAppReachable then HTTP.status200 else HTTP.status503) [] mempty
+    _ ->
+      respond $ Wai.responseLBS HTTP.status404 [] mempty
+
+-- Try to connect to the main app socket
+-- Note that it doesn't even send a valid HTTP request, we just want to check that the main app is accepting connections
+-- The code for resolving the "*4", "!4", "*6", "!6", "*" special values is taken from
+-- https://hackage.haskell.org/package/streaming-commons-0.2.2.4/docs/src/Data.Streaming.Network.html#bindPortGenEx
+reachMainApp :: AppConfig -> IO [Either IOException ()]
+reachMainApp AppConfig{..} =
+  case configServerUnixSocket of
+    Just path ->  do
+      sock <- socket AF_UNIX Stream 0
+      (:[]) <$> try (do
+        connect sock $ SockAddrUnix path
+        withSocketsDo $ bracket (pure sock) close sendEmpty)
+    Nothing -> do
+      let
+        host | configServerHost `elem` ["*4", "!4", "*6", "!6", "*"] = Nothing
+             | otherwise                                             = Just configServerHost
+        filterAddrs xs =
+          case configServerHost of
+              "*4" -> ipv4Addrs xs ++ ipv6Addrs xs
+              "!4" -> ipv4Addrs xs
+              "*6" -> ipv6Addrs xs ++ ipv4Addrs xs
+              "!6" -> ipv6Addrs xs
+              _    -> xs
+        ipv4Addrs = filter ((/=) AF_INET6 . addrFamily)
+        ipv6Addrs = filter ((==) AF_INET6 . addrFamily)
+
+      addrs <- getAddrInfo (Just $ defaultHints { addrSocketType = Stream }) (T.unpack <$> host) (Just . show $ configServerPort)
+      tryAddr `traverse` filterAddrs addrs
+  where
+    sendEmpty sock = void $ send sock mempty
+    tryAddr :: AddrInfo -> IO (Either IOException ())
+    tryAddr addr = do
+      sock <- socket (addrFamily addr) (addrSocketType addr) (addrProtocol addr)
+      try $ do
+        connect sock $ addrAddress addr
+        withSocketsDo $ bracket (pure sock) close sendEmpty

src/PostgREST/ApiRequest.hs

@@ -3,6 +3,7 @@ Module      : PostgREST.Request.ApiRequest
 Description : PostgREST functions to translate HTTP request to a domain type called ApiRequest.
 -}
 {-# LANGUAGE LambdaCase      #-}
+{-# LANGUAGE MultiWayIf      #-}
 {-# LANGUAGE NamedFieldPuns  #-}
 {-# LANGUAGE RecordWildCards #-}
 
@@ -32,43 +33,33 @@ import qualified Data.Set              as S
 import qualified Data.Text.Encoding    as T
 import qualified Data.Vector           as V
 
+import Data.Either.Combinators (mapBoth)
+
 import Control.Arrow             ((***))
 import Data.Aeson.Types          (emptyArray, emptyObject)
 import Data.List                 (lookup, union)
 import Data.Ranged.Ranges        (emptyRange, rangeIntersection,
                                   rangeIsEmpty)
-import Data.Tree                 (Tree (..))
 import Network.HTTP.Types.Header (RequestHeaders, hCookie)
 import Network.HTTP.Types.URI    (parseSimpleQuery)
 import Network.Wai               (Request (..))
 import Network.Wai.Parse         (parseHttpAccept)
 import Web.Cookie                (parseCookies)
 
-import PostgREST.ApiRequest.Preferences  (PreferCount (..),
-                                          PreferParameters (..),
-                                          PreferRepresentation (..),
-                                          PreferResolution (..),
-                                          PreferTransaction (..))
 import PostgREST.ApiRequest.QueryParams  (QueryParams (..))
 import PostgREST.ApiRequest.Types        (ApiRequestError (..),
-                                          RangeError (..),
-                                          SelectItem (..))
+                                          RangeError (..))
 import PostgREST.Config                  (AppConfig (..),
                                           OpenAPIMode (..))
-import PostgREST.MediaType               (MTPlanAttrs (..),
-                                          MTPlanFormat (..),
+import PostgREST.MediaType               (MTPlanFormat (..),
                                           MediaType (..))
 import PostgREST.RangeQuery              (NonnegRange, allRange,
                                           convertToLimitZeroRange,
                                           hasLimitZero,
                                           rangeRequested)
-import PostgREST.SchemaCache             (SchemaCache (..))
 import PostgREST.SchemaCache.Identifiers (FieldName,
                                           QualifiedIdentifier (..),
                                           Schema)
-import PostgREST.SchemaCache.Proc        (ProcDescription (..),
-                                          ProcParam (..), ProcsMap,
-                                          procReturnsScalar)
 
 import qualified PostgREST.ApiRequest.Preferences as Preferences
 import qualified PostgREST.ApiRequest.QueryParams as QueryParams
@@ -90,6 +81,7 @@ data Payload
       -- ^ Keys of the object or if it's an array these keys are guaranteed to
       -- be the same across all its objects
       }
+  | ProcessedUrlEncoded { payArray  :: [(Text, Text)], payKeys :: S.Set Text }
   | RawJSON { payRaw  :: LBS.ByteString }
   | RawPay  { payRaw  :: LBS.ByteString }
 
@@ -114,41 +106,9 @@ data PathInfo
       }
 -- | The target db object of a user action
 data Target = TargetIdent QualifiedIdentifier
-            | TargetProc{tProc :: ProcDescription, tpIsRootSpec :: Bool}
+            | TargetProc{tProc :: QualifiedIdentifier, tpIsRootSpec :: Bool}
             | TargetDefaultSpec{tdsSchema :: Schema} -- The default spec offered at root "/"
 
--- | RPC query param value `/rpc/func?v=<value>`, used for VARIADIC functions on form-urlencoded POST and GETs
--- | It can be fixed `?v=1` or repeated `?v=1&v=2&v=3.
-data RpcParamValue = Fixed Text | Variadic [Text]
-instance JSON.ToJSON RpcParamValue where
-  toJSON (Fixed    v) = JSON.toJSON v
-  toJSON (Variadic v) = JSON.toJSON v
-
-toRpcParamValue :: ProcDescription -> (Text, Text) -> (Text, RpcParamValue)
-toRpcParamValue proc (k, v) | prmIsVariadic k = (k, Variadic [v])
-                            | otherwise       = (k, Fixed v)
-  where
-    prmIsVariadic prm = isJust $ find (\ProcParam{ppName, ppVar} -> ppName == prm && ppVar) $ pdParams proc
-
--- | Convert rpc params `/rpc/func?a=val1&b=val2` to json `{"a": "val1", "b": "val2"}
-jsonRpcParams :: ProcDescription -> [(Text, Text)] -> Payload
-jsonRpcParams proc prms =
-  if not $ pdHasVariadic proc then -- if proc has no variadic param, save steps and directly convert to json
-    ProcessedJSON (JSON.encode $ HM.fromList $ second JSON.toJSON <$> prms) (S.fromList $ fst <$> prms)
-  else
-    let paramsMap = HM.fromListWith mergeParams $ toRpcParamValue proc <$> prms in
-    ProcessedJSON (JSON.encode paramsMap) (S.fromList $ HM.keys paramsMap)
-  where
-    mergeParams :: RpcParamValue -> RpcParamValue -> RpcParamValue
-    mergeParams (Variadic a) (Variadic b) = Variadic $ b ++ a
-    mergeParams v _                       = v -- repeated params for non-variadic parameters are not merged
-
-targetToJsonRpcParams :: Maybe Target -> [(Text, Text)] -> Maybe Payload
-targetToJsonRpcParams target params =
-  case target of
-    Just TargetProc{tProc} -> Just $ jsonRpcParams tProc params
-    _                      -> Nothing
-
 {-|
   Describes what the user wants to do. This data type is a
   translation of the raw elements of an HTTP request into domain
@@ -157,37 +117,60 @@ targetToJsonRpcParams target params =
   if it is an action we are able to perform.
 -}
 data ApiRequest = ApiRequest {
-    iAction               :: Action                           -- ^ Similar but not identical to HTTP method, e.g. Create/Invoke both POST
-  , iRange                :: HM.HashMap Text NonnegRange      -- ^ Requested range of rows within response
-  , iTopLevelRange        :: NonnegRange                      -- ^ Requested range of rows from the top level
-  , iTarget               :: Target                           -- ^ The target, be it calling a proc or accessing a table
-  , iPayload              :: Maybe Payload                    -- ^ Data sent by client and used for mutation actions
-  , iPreferRepresentation :: PreferRepresentation             -- ^ If client wants created items echoed back
-  , iPreferParameters     :: Maybe PreferParameters           -- ^ How to pass parameters to a stored procedure
-  , iPreferCount          :: Maybe PreferCount                -- ^ Whether the client wants a result count
-  , iPreferResolution     :: Maybe PreferResolution           -- ^ Whether the client wants to UPSERT or ignore records on PK conflict
-  , iPreferTransaction    :: Maybe PreferTransaction          -- ^ Whether the clients wants to commit or rollback the transaction
-  , iQueryParams          :: QueryParams.QueryParams
-  , iColumns              :: S.Set FieldName                  -- ^ parsed colums from &columns parameter and payload
-  , iHeaders              :: [(ByteString, ByteString)]       -- ^ HTTP request headers
-  , iCookies              :: [(ByteString, ByteString)]       -- ^ Request Cookies
-  , iPath                 :: ByteString                       -- ^ Raw request path
-  , iMethod               :: ByteString                       -- ^ Raw request method
-  , iSchema               :: Schema                           -- ^ The request schema. Can vary depending on profile headers.
-  , iNegotiatedByProfile  :: Bool                             -- ^ If schema was was chosen according to the profile spec https://www.w3.org/TR/dx-prof-conneg/
-  , iAcceptMediaType      :: MediaType                        -- ^ The media type in the Accept header
-  , iBinaryField          :: Maybe FieldName                  -- ^ field used for raw output
+    iAction              :: Action                           -- ^ Similar but not identical to HTTP method, e.g. Create/Invoke both POST
+  , iRange               :: HM.HashMap Text NonnegRange      -- ^ Requested range of rows within response
+  , iTopLevelRange       :: NonnegRange                      -- ^ Requested range of rows from the top level
+  , iTarget              :: Target                           -- ^ The target, be it calling a proc or accessing a table
+  , iPayload             :: Maybe Payload                    -- ^ Data sent by client and used for mutation actions
+  , iPreferences         :: Preferences.Preferences          -- ^ Prefer header values
+  , iQueryParams         :: QueryParams.QueryParams
+  , iColumns             :: S.Set FieldName                  -- ^ parsed colums from &columns parameter and payload
+  , iHeaders             :: [(ByteString, ByteString)]       -- ^ HTTP request headers
+  , iCookies             :: [(ByteString, ByteString)]       -- ^ Request Cookies
+  , iPath                :: ByteString                       -- ^ Raw request path
+  , iMethod              :: ByteString                       -- ^ Raw request method
+  , iSchema              :: Schema                           -- ^ The request schema. Can vary depending on profile headers.
+  , iNegotiatedByProfile :: Bool                             -- ^ If schema was was chosen according to the profile spec https://www.w3.org/TR/dx-prof-conneg/
+  , iAcceptMediaType     :: MediaType                        -- ^ The media type in the Accept header
+  , iContentMediaType    :: MediaType                        -- ^ The media type in the Content-Type header
   }
 
 -- | Examines HTTP request and translates it into user intent.
-userApiRequest :: AppConfig -> SchemaCache -> Request -> RequestBody -> Either ApiRequestError ApiRequest
-userApiRequest conf sCache req reqBody = do
-  qPrms <- first QueryParamError $ QueryParams.parse $ rawQueryString req
-  pInfo <- getPathInfo conf $ pathInfo req
-  act <- getAction pInfo $ requestMethod req
-  mediaTypes <- getMediaTypes conf (requestHeaders req) act pInfo
-  negotiatedSchema <- getSchema conf (requestHeaders req) (requestMethod req)
-  apiRequest conf sCache req reqBody qPrms pInfo act mediaTypes negotiatedSchema
+userApiRequest :: AppConfig -> Request -> RequestBody -> Either ApiRequestError ApiRequest
+userApiRequest conf req reqBody = do
+  pInfo@PathInfo{..} <- getPathInfo conf $ pathInfo req
+  act <- getAction pInfo method
+  qPrms <- first QueryParamError $ QueryParams.parse (pathIsProc && act `elem` [ActionInvoke InvGet, ActionInvoke InvHead]) $ rawQueryString req
+  (acceptMediaType, contentMediaType) <- getMediaTypes conf hdrs act pInfo
+  (schema, negotiatedByProfile) <- getSchema conf hdrs method
+  (topLevelRange, ranges) <- getRanges method qPrms hdrs
+  (payload, columns) <- getPayload reqBody contentMediaType qPrms act pInfo
+  return $ ApiRequest {
+    iAction = act
+  , iTarget = if | pathIsProc    -> TargetProc (QualifiedIdentifier schema pathName) pathIsRootSpec
+                 | pathIsDefSpec -> TargetDefaultSpec schema
+                 | otherwise     -> TargetIdent $ QualifiedIdentifier schema pathName
+  , iRange = ranges
+  , iTopLevelRange = topLevelRange
+  , iPayload = payload
+  , iPreferences = Preferences.fromHeaders (configDbTxAllowOverride conf) hdrs
+  , iQueryParams = qPrms
+  , iColumns = columns
+  , iHeaders = iHdrs
+  , iCookies = iCkies
+  , iPath = rawPathInfo req
+  , iMethod = method
+  , iSchema = schema
+  , iNegotiatedByProfile = negotiatedByProfile
+  , iAcceptMediaType = acceptMediaType
+  , iContentMediaType = contentMediaType
+  }
+  where
+    method = requestMethod req
+    hdrs = requestHeaders req
+    lookupHeader    = flip lookup hdrs
+    iHdrs = [ (CI.foldedCase k, v) | (k,v) <- hdrs, k /= hCookie]
+    iCkies = maybe [] parseCookies $ lookupHeader "Cookie"
 
 getPathInfo :: AppConfig -> [Text] -> Either ApiRequestError PathInfo
 getPathInfo AppConfig{configOpenApiMode, configDbRootSpec} path =
@@ -223,7 +206,7 @@ getAction PathInfo{pathIsProc, pathIsDefSpec} method =
 
 getMediaTypes :: AppConfig -> RequestHeaders -> Action -> PathInfo -> Either ApiRequestError (MediaType, MediaType)
 getMediaTypes conf hdrs action path = do
-   acceptMediaType <- findAcceptMediaType conf action path accepts
+   acceptMediaType <- negotiateContent conf action path accepts
    pure (acceptMediaType, contentMediaType)
   where
     accepts = maybe [MTAny] (map MediaType.decodeMediaType . parseHttpAccept) $ lookupHeader "accept"
@@ -249,127 +232,71 @@ getSchema AppConfig{configDbSchemas} hdrs method = do
     acceptProfile = T.decodeUtf8 <$> lookupHeader "Accept-Profile"
     lookupHeader    = flip lookup hdrs
 
-apiRequest :: AppConfig -> SchemaCache -> Request -> RequestBody -> QueryParams.QueryParams -> PathInfo -> Action -> (MediaType, MediaType) -> (Schema, Bool) -> Either ApiRequestError ApiRequest
-apiRequest conf sCache req reqBody queryparams@QueryParams{..} PathInfo{pathName, pathIsProc, pathIsRootSpec, pathIsDefSpec} action (acceptMediaType, contentMediaType) (schema, negotiatedByProfile)
+getRanges :: ByteString -> QueryParams -> RequestHeaders -> Either ApiRequestError (NonnegRange, HM.HashMap Text NonnegRange)
+getRanges method QueryParams{qsOrder,qsRanges} hdrs
   | isInvalidRange = Left $ InvalidRange (if rangeIsEmpty headerRange then LowerGTUpper else NegativeLimit)
-  | shouldParsePayload && isLeft payload = either (Left . InvalidBody) witness payload
-  | not expectParams && not (L.null qsParams) = Left $ ParseRequestError "Unexpected param or filter missing operator" ("Failed to parse " <> show qsParams)
   | method `elem` ["PATCH", "DELETE"] && not (null qsRanges) && null qsOrder = Left LimitNoOrderError
-  | method == "PUT" && topLevelRange /= allRange = Left PutRangeNotAllowedError
-  | otherwise = do
-     checkedTarget <- target
-     bField <- binaryField conf acceptMediaType checkedTarget queryparams
-     return ApiRequest {
-      iAction = action
-      , iTarget = checkedTarget
-      , iRange = ranges
-      , iTopLevelRange = topLevelRange
-      , iPayload = relevantPayload
-      , iPreferRepresentation = fromMaybe None preferRepresentation
-      , iPreferParameters = preferParameters
-      , iPreferCount = preferCount
-      , iPreferResolution = preferResolution
-      , iPreferTransaction = preferTransaction
-      , iQueryParams = queryparams
-      , iColumns = payloadColumns
-      , iHeaders = [ (CI.foldedCase k, v) | (k,v) <- hdrs, k /= hCookie]
-      , iCookies = maybe [] parseCookies $ lookupHeader "Cookie"
-      , iPath = rawPathInfo req
-      , iMethod = method
-      , iSchema = schema
-      , iNegotiatedByProfile = negotiatedByProfile
-      , iAcceptMediaType = acceptMediaType
-      , iBinaryField = bField
-      }
- where
-  expectParams = pathIsProc && method /= "POST"
+  | method == "PUT" && topLevelRange /= allRange = Left PutLimitNotAllowedError
+  | otherwise = Right (topLevelRange, ranges)
+  where
+    -- According to the RFC (https://www.rfc-editor.org/rfc/rfc9110.html#name-range),
+    -- the Range header must be ignored for all methods other than GET
+    headerRange = if method == "GET" then rangeRequested hdrs else allRange
+    limitRange = fromMaybe allRange (HM.lookup "limit" qsRanges)
+    headerAndLimitRange = rangeIntersection headerRange limitRange
+    -- Bypass all the ranges and send only the limit zero range (0 <= x <= -1) if
+    -- limit=0 is present in the query params (not allowed for the Range header)
+    ranges = HM.insert "limit" (convertToLimitZeroRange limitRange headerAndLimitRange) qsRanges
+    -- The only emptyRange allowed is the limit zero range
+    isInvalidRange = topLevelRange == emptyRange && not (hasLimitZero limitRange)
+    topLevelRange = fromMaybe allRange $ HM.lookup "limit" ranges -- if no limit is specified, get all the request rows
 
-  columns = case action of
-    ActionMutate MutationCreate -> qsColumns
-    ActionMutate MutationUpdate -> qsColumns
-    ActionInvoke InvPost        -> qsColumns
-    _                           -> Nothing
+getPayload :: RequestBody -> MediaType -> QueryParams.QueryParams -> Action -> PathInfo -> Either ApiRequestError (Maybe Payload, S.Set FieldName)
+getPayload reqBody contentMediaType QueryParams{qsColumns} action PathInfo{pathIsProc}= do
+  checkedPayload <- if shouldParsePayload then payload else Right Nothing
+  let cols = case (checkedPayload, columns) of
+        (Just ProcessedJSON{payKeys}, _)       -> payKeys
+        (Just ProcessedUrlEncoded{payKeys}, _) -> payKeys
+        (Just RawJSON{}, Just cls)             -> cls
+        _                                      -> S.empty
+  return (checkedPayload, cols)
+  where
+    payload :: Either ApiRequestError (Maybe Payload)
+    payload = mapBoth InvalidBody Just $ case (contentMediaType, pathIsProc) of
+      (MTApplicationJSON, _) ->
+        if isJust columns
+          then Right $ RawJSON reqBody
+          else note "All object keys must match" . payloadAttributes reqBody
+                 =<< if LBS.null reqBody && pathIsProc
+                       then Right emptyObject
+                       else first BS.pack $ JSON.eitherDecode reqBody
+      (MTTextCSV, _) -> do
+        json <- csvToJson <$> first BS.pack (CSV.decodeByName reqBody)
+        note "All lines must have same number of fields" $ payloadAttributes (JSON.encode json) json
+      (MTUrlEncoded, isProc) -> do
+        let params = (T.decodeUtf8 *** T.decodeUtf8) <$> parseSimpleQuery (LBS.toStrict reqBody)
+        if isProc
+          then Right $ ProcessedUrlEncoded params (S.fromList $ fst <$> params)
+          else
+            let paramsMap = HM.fromList $ (identity *** JSON.String) <$> params in
+            Right $ ProcessedJSON (JSON.encode paramsMap) $ S.fromList (HM.keys paramsMap)
+      (MTTextPlain, True) -> Right $ RawPay reqBody
+      (MTTextXML, True) -> Right $ RawPay reqBody
+      (MTOctetStream, True) -> Right $ RawPay reqBody
+      (ct, _) -> Left $ "Content-Type not acceptable: " <> MediaType.toMime ct
 
-  payloadColumns =
-    case (contentMediaType, action) of
-      (_, ActionInvoke InvGet)  -> S.fromList $ fst <$> qsParams
-      (_, ActionInvoke InvHead) -> S.fromList $ fst <$> qsParams
-      (MTUrlEncoded, _)         -> S.fromList $ map (T.decodeUtf8 . fst) $ parseSimpleQuery $ LBS.toStrict reqBody
-      _ -> case (relevantPayload, columns) of
-        (Just ProcessedJSON{payKeys}, _) -> payKeys
-        (Just RawJSON{}, Just cls)       -> cls
-        _                                -> S.empty
-  payload :: Either ByteString Payload
-  payload = case (contentMediaType, pathIsProc) of
-    (MTApplicationJSON, _) ->
-      if isJust columns
-        then Right $ RawJSON reqBody
-        else note "All object keys must match" . payloadAttributes reqBody
-               =<< if LBS.null reqBody && pathIsProc
-                     then Right emptyObject
-                     else first BS.pack $ JSON.eitherDecode reqBody
-    (MTTextCSV, _) -> do
-      json <- csvToJson <$> first BS.pack (CSV.decodeByName reqBody)
-      note "All lines must have same number of fields" $ payloadAttributes (JSON.encode json) json
-    (MTUrlEncoded, _) ->
-      let paramsMap = HM.fromList $ (T.decodeUtf8 *** JSON.String . T.decodeUtf8) <$> parseSimpleQuery (LBS.toStrict reqBody) in
-      Right $ ProcessedJSON (JSON.encode paramsMap) $ S.fromList (HM.keys paramsMap)
-    (MTTextPlain, True) -> Right $ RawPay reqBody
-    (MTTextXML, True) -> Right $ RawPay reqBody
-    (MTOctetStream, True) -> Right $ RawPay reqBody
-    (ct, _) -> Left $ "Content-Type not acceptable: " <> MediaType.toMime ct
-  topLevelRange = fromMaybe allRange $ HM.lookup "limit" ranges -- if no limit is specified, get all the request rows
+    shouldParsePayload = case (action, contentMediaType) of
+      (ActionMutate MutationCreate, _)       -> True
+      (ActionInvoke InvPost, _)              -> True
+      (ActionMutate MutationSingleUpsert, _) -> True
+      (ActionMutate MutationUpdate, _)       -> True
+      _                                      -> False
 
-  target
-    | pathIsProc    = (`TargetProc` pathIsRootSpec) <$> callFindProc schema pathName
-    | pathIsDefSpec = Right $ TargetDefaultSpec schema
-    | otherwise     = Right $ TargetIdent $ QualifiedIdentifier schema pathName
-    where
-      callFindProc procSch procNam = findProc
-        (QualifiedIdentifier procSch procNam) payloadColumns (preferParameters == Just SingleObject) (dbProcs sCache)
-        contentMediaType (action == ActionInvoke InvPost)
-
-  shouldParsePayload = case (action, contentMediaType) of
-    (ActionMutate MutationCreate, _)       -> True
-    (ActionInvoke InvPost, MTUrlEncoded)   -> False
-    (ActionInvoke InvPost, _)              -> True
-    (ActionMutate MutationSingleUpsert, _) -> True
-    (ActionMutate MutationUpdate, _)       -> True
-    _                                      -> False
-  relevantPayload = case (contentMediaType, action) of
-    -- Though ActionInvoke GET/HEAD doesn't really have a payload, we use the payload variable as a way
-    -- to store the query string arguments to the function.
-    (_, ActionInvoke InvGet)             -> targetToJsonRpcParams (rightToMaybe target) qsParams
-    (_, ActionInvoke InvHead)            -> targetToJsonRpcParams (rightToMaybe target) qsParams
-    (MTUrlEncoded, ActionInvoke InvPost) -> targetToJsonRpcParams (rightToMaybe target) $ (T.decodeUtf8 *** T.decodeUtf8) <$> parseSimpleQuery (LBS.toStrict reqBody)
-    _ | shouldParsePayload               -> rightToMaybe payload
-      | otherwise                        -> Nothing
-  method          = requestMethod req
-  hdrs            = requestHeaders req
-  lookupHeader    = flip lookup hdrs
-  Preferences.Preferences{..} = Preferences.fromHeaders hdrs
-  headerRange = rangeRequested hdrs
-  limitRange = fromMaybe allRange (HM.lookup "limit" qsRanges)
-  headerAndLimitRange = rangeIntersection headerRange limitRange
-
-  -- Bypass all the ranges and send only the limit zero range (0 <= x <= -1) if
-  -- limit=0 is present in the query params (not allowed for the Range header)
-  ranges = HM.insert "limit" (convertToLimitZeroRange limitRange headerAndLimitRange) qsRanges
-  -- The only emptyRange allowed is the limit zero range
-  isInvalidRange = topLevelRange == emptyRange && not (hasLimitZero limitRange)
-
-{-|
-  Find the best match from a list of media types accepted by the
-  client in order of decreasing preference and a list of types
-  producible by the server.  If there is no match but the client
-  accepts */* then return the top server pick.
--}
-mutuallyAgreeable :: [MediaType] -> [MediaType] -> Maybe MediaType
-mutuallyAgreeable sProduces cAccepts =
-  let exact = listToMaybe $ L.intersect cAccepts sProduces in
-  if isNothing exact && MTAny `elem` cAccepts
-     then listToMaybe sProduces
-     else exact
+    columns = case action of
+      ActionMutate MutationCreate -> qsColumns
+      ActionMutate MutationUpdate -> qsColumns
+      ActionInvoke InvPost        -> qsColumns
+      _                           -> Nothing
 
 type CsvData = V.Vector (M.Map Text LBS.ByteString)
 
@@ -420,117 +347,33 @@ payloadAttributes raw json =
   where
     emptyPJArray = ProcessedJSON (JSON.encode emptyArray) S.empty
 
-findAcceptMediaType :: AppConfig -> Action -> PathInfo -> [MediaType] -> Either ApiRequestError MediaType
-findAcceptMediaType conf action path accepts =
-  case mutuallyAgreeable (requestMediaTypes conf action path) accepts of
-    Just ct ->
-      Right ct
-    Nothing ->
-      Left . MediaTypeError $ map MediaType.toMime accepts
 
-requestMediaTypes :: AppConfig -> Action -> PathInfo -> [MediaType]
-requestMediaTypes conf action path =
+-- | Do content negotiation. i.e. choose a media type based on the intersection of accepted/produced media types.
+negotiateContent :: AppConfig -> Action -> PathInfo -> [MediaType] -> Either ApiRequestError MediaType
+negotiateContent conf action path accepts =
+  case firstAcceptedPick of
+    Just MTAny -> Right MTApplicationJSON -- by default(for */*) we respond with json
+    Just mt    -> Right mt
+    Nothing    -> Left . MediaTypeError $ map MediaType.toMime accepts
+  where
+    -- if there are multiple accepted media types, pick the first
+    firstAcceptedPick = listToMaybe $ L.intersect accepts $ producedMediaTypes conf action path
+
+producedMediaTypes :: AppConfig -> Action -> PathInfo -> [MediaType]
+producedMediaTypes conf action path =
   case action of
     ActionRead _    -> defaultMediaTypes ++ rawMediaTypes
     ActionInvoke _  -> invokeMediaTypes
-    ActionInspect _ -> [MTOpenAPI, MTApplicationJSON]
-    ActionInfo      -> [MTTextCSV]
-    _               -> defaultMediaTypes
+    ActionInfo      -> defaultMediaTypes
+    ActionMutate _  -> defaultMediaTypes
+    ActionInspect _ -> inspectMediaTypes
   where
+    inspectMediaTypes = [MTOpenAPI, MTApplicationJSON, MTArrayJSONStrip, MTAny]
     invokeMediaTypes =
       defaultMediaTypes
         ++ rawMediaTypes
         ++ [MTOpenAPI | pathIsRootSpec path]
     defaultMediaTypes =
-      [MTApplicationJSON, MTSingularJSON, MTGeoJSON, MTTextCSV] ++
-      [MTPlan $ MTPlanAttrs Nothing PlanJSON mempty | configDbPlanEnabled conf]
+      [MTApplicationJSON, MTArrayJSONStrip, MTSingularJSON True, MTSingularJSON False, MTGeoJSON, MTTextCSV] ++
+      [MTPlan MTApplicationJSON PlanText mempty | configDbPlanEnabled conf] ++ [MTAny]
     rawMediaTypes = configRawMediaTypes conf `union` [MTOctetStream, MTTextPlain, MTTextXML]
-
-{-|
-  Search a pg proc by matching name and arguments keys to parameters. Since a function can be overloaded,
-  the name is not enough to find it. An overloaded function can have a different volatility or even a different return type.
--}
-findProc :: QualifiedIdentifier -> S.Set Text -> Bool -> ProcsMap -> MediaType -> Bool -> Either ApiRequestError ProcDescription
-findProc qi argumentsKeys paramsAsSingleObject allProcs contentMediaType isInvPost =
-  case matchProc of
-    ([], [])     -> Left $ NoRpc (qiSchema qi) (qiName qi) (S.toList argumentsKeys) paramsAsSingleObject contentMediaType isInvPost (HM.keys allProcs) lookupProcName
-    -- If there are no functions with named arguments, fallback to the single unnamed argument function
-    ([], [proc]) -> Right proc
-    ([], procs)  -> Left $ AmbiguousRpc (toList procs)
-    -- Matches the functions with named arguments
-    ([proc], _)  -> Right proc
-    (procs, _)   -> Left $ AmbiguousRpc (toList procs)
-  where
-    matchProc = overloadedProcPartition lookupProcName
-    -- First find the proc by name
-    lookupProcName = HM.lookupDefault mempty qi allProcs
-    -- The partition obtained has the form (overloadedProcs,fallbackProcs)
-    -- where fallbackProcs are functions with a single unnamed parameter
-    overloadedProcPartition = foldr select ([],[])
-    select proc ~(ts,fs)
-      | matchesParams proc         = (proc:ts,fs)
-      | hasSingleUnnamedParam proc = (ts,proc:fs)
-      | otherwise                  = (ts,fs)
-    -- If the function is called with post and has a single unnamed parameter
-    -- it can be called depending on content type and the parameter type
-    hasSingleUnnamedParam ProcDescription{pdParams=[ProcParam{ppType}]} = isInvPost && case (contentMediaType, ppType) of
-      (MTApplicationJSON, "json")  -> True
-      (MTApplicationJSON, "jsonb") -> True
-      (MTTextPlain, "text")        -> True
-      (MTTextXML, "xml")           -> True
-      (MTOctetStream, "bytea")     -> True
-      _                            -> False
-    hasSingleUnnamedParam _ = False
-    matchesParams proc =
-      let
-        params = pdParams proc
-        firstType = (ppType <$> headMay params)
-      in
-      -- exceptional case for Prefer: params=single-object
-      if paramsAsSingleObject
-        then length params == 1 && (firstType == Just "json" || firstType == Just "jsonb")
-      -- If the function has no parameters, the arguments keys must be empty as well
-      else if null params
-        then null argumentsKeys && not (isInvPost && contentMediaType `elem` [MTOctetStream, MTTextPlain, MTTextXML])
-      -- A function has optional and required parameters. Optional parameters have a default value and
-      -- don't require arguments for the function to be executed, required parameters must have an argument present.
-      else case L.partition ppReq params of
-      -- If the function only has required parameters, the arguments keys must match those parameters
-        (reqParams, [])        -> argumentsKeys == S.fromList (ppName <$> reqParams)
-      -- If the function only has optional parameters, the arguments keys can match none or any of them(a subset)
-        ([], optParams)        -> argumentsKeys `S.isSubsetOf` S.fromList (ppName <$> optParams)
-      -- If the function has required and optional parameters, the arguments keys have to match the required parameters
-      -- and can match any or none of the default parameters.
-        (reqParams, optParams) -> argumentsKeys `S.difference` S.fromList (ppName <$> optParams) == S.fromList (ppName <$> reqParams)
-
--- | If raw(binary) output is requested, check that MediaType is one of the
--- admitted rawMediaTypes and that`?select=...` contains only one field other
--- than `*`
-binaryField :: AppConfig -> MediaType -> Target -> QueryParams -> Either ApiRequestError (Maybe FieldName)
-binaryField AppConfig{configRawMediaTypes} acceptMediaType target QueryParams{qsSelect}
-  | returnsScalar target && isRawMediaType =
-      Right $ Just "pgrst_scalar"
-  | isRawMediaType =
-      let
-        fieldName = fstFieldName qsSelect
-      in
-      case fieldName of
-        Just fld -> Right $ Just fld
-        Nothing  -> Left $ BinaryFieldError acceptMediaType
-  | otherwise =
-      Right Nothing
-  where
-    isRawMediaType = acceptMediaType `elem` configRawMediaTypes `union` [MTOctetStream, MTTextPlain, MTTextXML] || isRawPlan acceptMediaType
-    isRawPlan mt = case mt of
-      MTPlan (MTPlanAttrs (Just MTOctetStream) _ _) -> True
-      MTPlan (MTPlanAttrs (Just MTTextPlain) _ _)   -> True
-      MTPlan (MTPlanAttrs (Just MTTextXML) _ _)     -> True
-      _                                             -> False
-    returnsScalar :: Target -> Bool
-    returnsScalar (TargetProc proc _) = procReturnsScalar proc
-    returnsScalar _                   = False
-
-    fstFieldName :: [Tree SelectItem] -> Maybe FieldName
-    fstFieldName [Node SelectField{selField=("*", _)} []] = Nothing
-    fstFieldName [Node SelectField{selField=(fld, _)} []] = Just fld
-    fstFieldName _                                        = Nothing

src/PostgREST/ApiRequest/Preferences.hs

@@ -6,16 +6,18 @@
 --
 -- [1] https://datatracker.ietf.org/doc/html/rfc7240
 --
+{-# LANGUAGE NamedFieldPuns #-}
 module PostgREST.ApiRequest.Preferences
   ( Preferences(..)
   , PreferCount(..)
+  , PreferMissing(..)
   , PreferParameters(..)
   , PreferRepresentation(..)
   , PreferResolution(..)
   , PreferTransaction(..)
   , fromHeaders
-  , ToAppliedHeader(..)
   , shouldCount
+  , prefAppliedHeader
   ) where
 
 import qualified Data.ByteString.Char8     as BS
@@ -33,6 +35,7 @@ import Protolude
 -- >>> deriving instance Show PreferParameters
 -- >>> deriving instance Show PreferCount
 -- >>> deriving instance Show PreferTransaction
+-- >>> deriving instance Show PreferMissing
 -- >>> deriving instance Show Preferences
 
 -- | Preferences recognized by the application.
@@ -43,6 +46,7 @@ data Preferences
     , preferParameters     :: Maybe PreferParameters
     , preferCount          :: Maybe PreferCount
     , preferTransaction    :: Maybe PreferTransaction
+    , preferMissing        :: Maybe PreferMissing
     }
 
 -- |
@@ -50,35 +54,37 @@ data Preferences
 --
 -- One header with comma-separated values can be used to set multiple preferences:
 --
--- >>> pPrint $ fromHeaders [("Prefer", "resolution=ignore-duplicates, count=exact")]
+-- >>> pPrint $ fromHeaders True [("Prefer", "resolution=ignore-duplicates, count=exact")]
 -- Preferences
 --     { preferResolution = Just IgnoreDuplicates
 --     , preferRepresentation = Nothing
 --     , preferParameters = Nothing
 --     , preferCount = Just ExactCount
 --     , preferTransaction = Nothing
+--     , preferMissing = Nothing
 --     }
 --
 -- Multiple headers can also be used:
 --
--- >>> pPrint $ fromHeaders [("Prefer", "resolution=ignore-duplicates"), ("Prefer", "count=exact")]
+-- >>> pPrint $ fromHeaders True [("Prefer", "resolution=ignore-duplicates"), ("Prefer", "count=exact"), ("Prefer", "missing=null")]
 -- Preferences
 --     { preferResolution = Just IgnoreDuplicates
 --     , preferRepresentation = Nothing
 --     , preferParameters = Nothing
 --     , preferCount = Just ExactCount
 --     , preferTransaction = Nothing
+--     , preferMissing = Just ApplyNulls
 --     }
 --
 -- If a preference is set more than once, only the first is used:
 --
--- >>> preferTransaction $ fromHeaders [("Prefer", "tx=commit, tx=rollback")]
+-- >>> preferTransaction $ fromHeaders True [("Prefer", "tx=commit, tx=rollback")]
 -- Just Commit
 --
 -- This is also the case across multiple headers:
 --
 -- >>> :{
---   preferResolution . fromHeaders $
+--   preferResolution . fromHeaders True $
 --     [ ("Prefer", "resolution=ignore-duplicates")
 --     , ("Prefer", "resolution=merge-duplicates")
 --     ]
@@ -87,28 +93,30 @@ data Preferences
 --
 -- Preferences not recognized by the application are ignored:
 --
--- >>> preferResolution $ fromHeaders [("Prefer", "resolution=foo")]
+-- >>> preferResolution $ fromHeaders True [("Prefer", "resolution=foo")]
 -- Nothing
 --
 -- Preferences can be separated by arbitrary amounts of space, lower-case header is also recognized:
 --
--- >>> pPrint $ fromHeaders [("prefer", "count=exact,    tx=commit   ,return=minimal")]
+-- >>> pPrint $ fromHeaders True [("prefer", "count=exact,    tx=commit   ,return=representation , missing=default")]
 -- Preferences
 --     { preferResolution = Nothing
---     , preferRepresentation = Just None
+--     , preferRepresentation = Just Full
 --     , preferParameters = Nothing
 --     , preferCount = Just ExactCount
 --     , preferTransaction = Just Commit
+--     , preferMissing = Just ApplyDefaults
 --     }
 --
-fromHeaders :: [HTTP.Header] -> Preferences
-fromHeaders headers =
+fromHeaders :: Bool -> [HTTP.Header] -> Preferences
+fromHeaders allowTxEndOverride headers =
   Preferences
-    { preferResolution = parsePrefs [MergeDuplicates, IgnoreDuplicates]
+    { preferResolution     = parsePrefs [MergeDuplicates, IgnoreDuplicates]
     , preferRepresentation = parsePrefs [Full, None, HeadersOnly]
-    , preferParameters = parsePrefs [SingleObject, MultipleObjects]
-    , preferCount = parsePrefs [ExactCount, PlannedCount, EstimatedCount]
-    , preferTransaction = parsePrefs [Commit, Rollback]
+    , preferParameters     = parsePrefs [SingleObject]
+    , preferCount          = parsePrefs [ExactCount, PlannedCount, EstimatedCount]
+    , preferTransaction    = if allowTxEndOverride then parsePrefs [Commit, Rollback] else Nothing
+    , preferMissing        = parsePrefs [ApplyDefaults, ApplyNulls]
     }
   where
     prefHeaders = filter ((==) HTTP.hPrefer . fst) headers
@@ -121,6 +129,22 @@ fromHeaders headers =
     prefMap :: ToHeaderValue a => [a] -> Map.Map ByteString a
     prefMap = Map.fromList . fmap (\pref -> (toHeaderValue pref, pref))
 
+prefAppliedHeader :: Preferences -> Maybe HTTP.Header
+prefAppliedHeader Preferences {preferResolution, preferRepresentation, preferParameters, preferCount, preferTransaction, preferMissing } =
+  if null prefsVals
+    then Nothing
+    else Just (HTTP.hPreferenceApplied, combined)
+  where
+    combined = BS.intercalate ", " prefsVals
+    prefsVals = catMaybes [
+        toHeaderValue <$> preferResolution
+      , toHeaderValue <$> preferMissing
+      , toHeaderValue <$> preferRepresentation
+      , toHeaderValue <$> preferParameters
+      , toHeaderValue <$> preferCount
+      , toHeaderValue <$> preferTransaction
+      ]
+
 -- |
 -- Convert a preference into the value that we look for in the 'Prefer' headers.
 --
@@ -130,16 +154,6 @@ fromHeaders headers =
 class ToHeaderValue a where
   toHeaderValue :: a -> ByteString
 
--- |
--- Header to indicate that a preference has been applied.
---
--- >>> toAppliedHeader MergeDuplicates
--- ("Preference-Applied","resolution=merge-duplicates")
---
-class ToHeaderValue a => ToAppliedHeader a where
-  toAppliedHeader :: a -> HTTP.Header
-  toAppliedHeader x = (HTTP.hPreferenceApplied, toHeaderValue x)
-
 -- | How to handle duplicate values.
 data PreferResolution
   = MergeDuplicates
@@ -149,8 +163,6 @@ instance ToHeaderValue PreferResolution where
   toHeaderValue MergeDuplicates  = "resolution=merge-duplicates"
   toHeaderValue IgnoreDuplicates = "resolution=ignore-duplicates"
 
-instance ToAppliedHeader PreferResolution
-
 -- |
 -- How to return the mutated data.
 --
@@ -169,13 +181,10 @@ instance ToHeaderValue PreferRepresentation where
 -- | How to pass parameters to stored procedures.
 data PreferParameters
   = SingleObject    -- ^ Pass all parameters as a single json object to a stored procedure.
-  | MultipleObjects -- ^ Pass an array of json objects as params to a stored procedure.
   deriving Eq
 
--- TODO: Deprecate params=multiple-objects in next major version
 instance ToHeaderValue PreferParameters where
   toHeaderValue SingleObject    = "params=single-object"
-  toHeaderValue MultipleObjects = "params=multiple-objects"
 
 -- | How to determine the count of (expected) results
 data PreferCount
@@ -203,4 +212,14 @@ instance ToHeaderValue PreferTransaction where
   toHeaderValue Commit   = "tx=commit"
   toHeaderValue Rollback = "tx=rollback"
 
-instance ToAppliedHeader PreferTransaction
+-- |
+-- How to handle the insertion/update when the keys specified in ?columns are not present
+-- in the json body.
+data PreferMissing
+  = ApplyDefaults  -- ^ Use the default column value for missing values.
+  | ApplyNulls     -- ^ Use the null value for missing values.
+  deriving Eq
+
+instance ToHeaderValue PreferMissing where
+  toHeaderValue ApplyDefaults = "missing=default"
+  toHeaderValue ApplyNulls    = "missing=null"

src/PostgREST/ApiRequest/QueryParams.hs

@@ -30,14 +30,13 @@ import Data.Ranged.Ranges            (Range (..))
 import Data.Tree                     (Tree (..))
 import Text.Parsec.Error             (errorMessages,
                                       showErrorMessages)
-import Text.Parsec.Prim              (parserFail)
 import Text.ParserCombinators.Parsec (GenParser, ParseError, Parser,
                                       anyChar, between, char, digit,
                                       eof, errorPos, letter,
                                       lookAhead, many1, noneOf,
                                       notFollowedBy, oneOf,
-                                      optionMaybe, sepBy1, string,
-                                      try, (<?>))
+                                      optionMaybe, sepBy, sepBy1,
+                                      string, try, (<?>))
 
 import PostgREST.RangeQuery              (NonnegRange, allRange,
                                           rangeGeq, rangeLimit,
@@ -46,35 +45,21 @@ import PostgREST.SchemaCache.Identifiers (FieldName)
 
 import PostgREST.ApiRequest.Types (EmbedParam (..), EmbedPath, Field,
                                    Filter (..), FtsOperator (..),
-                                   JoinType (..), JsonOperand (..),
+                                   Hint, JoinType (..),
+                                   JsonOperand (..),
                                    JsonOperation (..), JsonPath,
                                    ListVal, LogicOperator (..),
                                    LogicTree (..), OpExpr (..),
-                                   Operation (..),
+                                   OpQuantifier (..), Operation (..),
                                    OrderDirection (..),
                                    OrderNulls (..), OrderTerm (..),
-                                   QPError (..), SelectItem (..),
+                                   QPError (..), QuantOperator (..),
+                                   SelectItem (..),
                                    SimpleOperator (..), SingleVal,
                                    TrileanVal (..))
 
 import Protolude hiding (try)
 
-
--- $setup
--- Setup for doctests
--- >>> import Text.Pretty.Simple (pPrint)
--- >>> deriving instance Show QPError
--- >>> deriving instance Show TrileanVal
--- >>> deriving instance Show FtsOperator
--- >>> deriving instance Show SimpleOperator
--- >>> deriving instance Show Operation
--- >>> deriving instance Show OpExpr
--- >>> deriving instance Show JsonOperand
--- >>> deriving instance Show JsonOperation
--- >>> deriving instance Show Filter
--- >>> deriving instance Show JoinType
--- >>> deriving instance Show SelectItem
-
 data QueryParams =
   QueryParams
     { qsCanonical      :: ByteString
@@ -108,39 +93,45 @@ data QueryParams =
 --
 -- The canonical representation of the query string has parameters sorted alphabetically:
 --
--- >>> qsCanonical <$> parse "a=1&c=3&b=2&d"
+-- >>> qsCanonical <$> parse True "a=1&c=3&b=2&d"
 -- Right "a=1&b=2&c=3&d="
 --
 -- 'select' is a reserved parameter that selects the fields to be returned:
 --
--- >>> qsSelect <$> parse "select=name,location"
+-- >>> qsSelect <$> parse False "select=name,location"
 -- Right [Node {rootLabel = SelectField {selField = ("name",[]), selCast = Nothing, selAlias = Nothing}, subForest = []},Node {rootLabel = SelectField {selField = ("location",[]), selCast = Nothing, selAlias = Nothing}, subForest = []}]
 --
 -- Filters are parameters whose value contains an operator, separated by a '.' from its value:
 --
--- >>> qsFilters <$> parse "a.b=eq.0"
--- Right [(["a"],Filter {field = ("b",[]), opExpr = OpExpr False (Op OpEqual "0")})]
+-- >>> qsFilters <$> parse False "a.b=eq.0"
+-- Right [(["a"],Filter {field = ("b",[]), opExpr = OpExpr False (OpQuant OpEqual Nothing "0")})]
 --
 -- If the operator specified in a filter does not exist, parsing the query string fails:
 --
--- >>> qsFilters <$> parse "a.b=noop.0"
--- Left (QPError "\"failed to parse filter (noop.0)\" (line 1, column 6)" "unknown single value operator noop")
-parse :: ByteString -> Either QPError QueryParams
-parse qs =
-  QueryParams
-    canonical
-    params
-    ranges
-    <$> pRequestOrder `traverse` order
-    <*> pRequestLogicTree `traverse` logic
-    <*> pRequestColumns columns
-    <*> pRequestSelect select
-    <*> pRequestFilter `traverse` filters
-    <*> (fmap snd <$> (pRequestFilter `traverse` filtersRoot))
-    <*> pRequestFilter `traverse` filtersNotRoot
-    <*> pure (S.fromList (fst <$> filters))
-    <*> pRequestOnConflict `traverse` onConflict
+-- >>> qsFilters <$> parse False "a.b=noop.0"
+-- Left (QPError "\"failed to parse filter (noop.0)\" (line 1, column 1)" "unexpected \"o\" expecting \"not\" or operator (eq, gt, ...)")
+parse :: Bool -> ByteString -> Either QPError QueryParams
+parse isRpcGet qs = do
+  rOrd                      <- pRequestOrder `traverse` order
+  rLogic                    <- pRequestLogicTree `traverse` logic
+  rCols                     <- pRequestColumns columns
+  rSel                      <- pRequestSelect select
+  (rFlts, params)           <- L.partition hasOp <$> pRequestFilter isRpcGet `traverse` filters
+  (rFltsRoot, rFltsNotRoot) <- pure $ L.partition hasRootFilter rFlts
+  rOnConflict               <- pRequestOnConflict `traverse` onConflict
+
+  let rFltsFields           = S.fromList (fst <$> filters)
+      params'               = mapMaybe (\case {(_, Filter (fld, _) (NoOpExpr v)) -> Just (fld,v); _ -> Nothing}) params
+      rFltsRoot'            = snd <$> rFltsRoot
+
+  return $ QueryParams canonical params' ranges rOrd rLogic rCols rSel rFlts rFltsRoot' rFltsNotRoot rFltsFields rOnConflict
   where
+    hasRootFilter, hasOp :: (EmbedPath, Filter) -> Bool
+    hasRootFilter ([], _) = True
+    hasRootFilter _       = False
+    hasOp (_, Filter (_, _) (NoOpExpr _)) = False
+    hasOp _                               = True
+
     logic = filter (endingIn ["and", "or"] . fst) nonemptyParams
     select = fromMaybe "*" $ lookupParam "select"
     onConflict = lookupParam "on_conflict"
@@ -167,32 +158,11 @@ parse qs =
     endingIn xx key = lastWord `elem` xx
       where lastWord = L.last $ T.split (== '.') key
 
-    (filters, params) = L.partition isParam filtersAndParams
-    isParam (k, v) = isEmbedPath k || hasOperator v || hasFtsOperator v
-
-    filtersAndParams = filter (isFilterOrParam . fst)  nonemptyParams
-    isFilterOrParam k = not (endingIn reservedEmbeddable k) && notElem k reserved
+    filters = filter (isFilter . fst) nonemptyParams
+    isFilter k = not (endingIn reservedEmbeddable k) && notElem k reserved
     reserved = ["select", "columns", "on_conflict"]
     reservedEmbeddable = ["order", "limit", "offset", "and", "or"]
 
-    (filtersNotRoot, filtersRoot) = L.partition isNotRoot filters
-    isNotRoot = flip T.isInfixOf "." . fst
-
-    -- TODO: These checks are redundant to the parsers, should use parsers to differentiate params
-    hasOperator val =
-      case T.splitOn "." val of
-        "not" : _ : _ -> True
-        "is" : _      -> True
-        "in" : _      -> True
-        x : _         -> isJust (operator x) || isJust (ftsOperator x)
-        _             -> False
-
-    hasFtsOperator val =
-      case T.splitOn "(" val of
-        x : _ : _ -> isJust $ ftsOperator x
-        _         -> False
-
-    isEmbedPath = T.isInfixOf "."
     replaceLast x s = T.intercalate "." $ L.init (T.split (=='.') s) <> [x]
 
     ranges :: HM.HashMap Text (Range Integer)
@@ -209,39 +179,31 @@ parse qs =
         offsetParams =
           HM.fromList [(k, maybe allRange rangeGeq (readMaybe v)) | (k,v) <- offsets]
 
-operator :: Text -> Maybe SimpleOperator
-operator = \case
-  "eq"     -> Just OpEqual
-  "gte"    -> Just OpGreaterThanEqual
-  "gt"     -> Just OpGreaterThan
-  "lte"    -> Just OpLessThanEqual
-  "lt"     -> Just OpLessThan
-  "neq"    -> Just OpNotEqual
-  "like"   -> Just OpLike
-  "ilike"  -> Just OpILike
-  "cs"     -> Just OpContains
-  "cd"     -> Just OpContained
-  "ov"     -> Just OpOverlap
-  "sl"     -> Just OpStrictlyLeft
-  "sr"     -> Just OpStrictlyRight
-  "nxr"    -> Just OpNotExtendsRight
-  "nxl"    -> Just OpNotExtendsLeft
-  "adj"    -> Just OpAdjacent
-  "match"  -> Just OpMatch
-  "imatch" -> Just OpIMatch
-  _        -> Nothing
-
-ftsOperator :: Text -> Maybe FtsOperator
-ftsOperator = \case
-  "fts"   -> Just FilterFts
-  "plfts" -> Just FilterFtsPlain
-  "phfts" -> Just FilterFtsPhrase
-  "wfts"  -> Just FilterFtsWebsearch
-  _       -> Nothing
-
-
--- PARSERS
+simpleOperator :: Parser SimpleOperator
+simpleOperator =
+  try (string "neq" $> OpNotEqual) <|>
+  try (string "cs" $> OpContains) <|>
+  try (string "cd" $> OpContained) <|>
+  try (string "ov" $> OpOverlap) <|>
+  try (string "sl" $> OpStrictlyLeft) <|>
+  try (string "sr" $> OpStrictlyRight) <|>
+  try (string "nxr" $> OpNotExtendsRight) <|>
+  try (string "nxl" $> OpNotExtendsLeft) <|>
+  try (string "adj" $> OpAdjacent) <?>
+  "unknown single value operator"
 
+quantOperator :: Parser QuantOperator
+quantOperator =
+  try (string "eq" $> OpEqual) <|>
+  try (string "gte" $> OpGreaterThanEqual) <|>
+  try (string "gt" $> OpGreaterThan) <|>
+  try (string "lte" $> OpLessThanEqual) <|>
+  try (string "lt" $> OpLessThan) <|>
+  try (string "like" $> OpLike) <|>
+  try (string "ilike" $> OpILike) <|>
+  try (string "match" $> OpMatch) <|>
+  try (string "imatch" $> OpIMatch) <?>
+  "unknown single value operator"
 
 pRequestSelect :: Text -> Either QPError [Tree SelectItem]
 pRequestSelect selStr =
@@ -251,11 +213,25 @@ pRequestOnConflict :: Text -> Either QPError [FieldName]
 pRequestOnConflict oncStr =
   mapError $ P.parse pColumns ("failed to parse on_conflict parameter (" <> toS oncStr <> ")") (toS oncStr)
 
-pRequestFilter :: (Text, Text) -> Either QPError (EmbedPath, Filter)
-pRequestFilter (k, v) = mapError $ (,) <$> path <*> (Filter <$> fld <*> oper)
+-- |
+-- Parse `id=eq.1`(id, eq.1) into (EmbedPath, Filter)
+--
+-- >>> pRequestFilter False ("id", "eq.1")
+-- Right ([],Filter {field = ("id",[]), opExpr = OpExpr False (OpQuant OpEqual Nothing "1")})
+--
+-- >>> pRequestFilter False ("id", "val")
+-- Left (QPError "\"failed to parse filter (val)\" (line 1, column 1)" "unexpected \"v\" expecting \"not\" or operator (eq, gt, ...)")
+--
+-- >>> pRequestFilter True ("id", "val")
+-- Right ([],Filter {field = ("id",[]), opExpr = NoOpExpr "val"})
+pRequestFilter :: Bool -> (Text, Text) -> Either QPError (EmbedPath, Filter)
+pRequestFilter isRpcGet (k, v) = mapError $ (,) <$> path <*> (Filter <$> fld <*> oper)
   where
     treePath = P.parse pTreePath ("failed to parse tree path (" ++ toS k ++ ")") $ toS k
-    oper = P.parse (pOpExpr pSingleVal) ("failed to parse filter (" ++ toS v ++ ")") $ toS v
+    oper = P.parse parseFlt ("failed to parse filter (" ++ toS v ++ ")") $ toS v
+    parseFlt = if isRpcGet
+      then pOpExpr pSingleVal <|> pure (NoOpExpr v)
+      else pOpExpr pSingleVal
     path = fst <$> treePath
     fld = snd <$> treePath
 
@@ -314,20 +290,28 @@ pTreePath = do
 -- >>> P.parse pFieldForest "" "*,client(*,nested(*))"
 -- Right [Node {rootLabel = SelectField {selField = ("*",[]), selCast = Nothing, selAlias = Nothing}, subForest = []},Node {rootLabel = SelectRelation {selRelation = "client", selAlias = Nothing, selHint = Nothing, selJoinType = Nothing}, subForest = [Node {rootLabel = SelectField {selField = ("*",[]), selCast = Nothing, selAlias = Nothing}, subForest = []},Node {rootLabel = SelectRelation {selRelation = "nested", selAlias = Nothing, selHint = Nothing, selJoinType = Nothing}, subForest = [Node {rootLabel = SelectField {selField = ("*",[]), selCast = Nothing, selAlias = Nothing}, subForest = []}]}]}]
 --
+-- >>> P.parse pFieldForest "" "*,...client(*),other(*)"
+-- Right [Node {rootLabel = SelectField {selField = ("*",[]), selCast = Nothing, selAlias = Nothing}, subForest = []},Node {rootLabel = SpreadRelation {selRelation = "client", selHint = Nothing, selJoinType = Nothing}, subForest = [Node {rootLabel = SelectField {selField = ("*",[]), selCast = Nothing, selAlias = Nothing}, subForest = []}]},Node {rootLabel = SelectRelation {selRelation = "other", selAlias = Nothing, selHint = Nothing, selJoinType = Nothing}, subForest = [Node {rootLabel = SelectField {selField = ("*",[]), selCast = Nothing, selAlias = Nothing}, subForest = []}]}]
+--
+-- >>> P.parse pFieldForest "" ""
+-- Right []
+--
 -- >>> P.parse pFieldForest "" "id,clients(name[])"
 -- Left (line 1, column 16):
 -- unexpected '['
--- expecting letter, digit, "-", "!", "(", "->>", "->", "::", ")", "," or end of input
+-- expecting letter, digit, "-", "->>", "->", "::", ")", "," or end of input
+--
+-- >>> P.parse pFieldForest "" "data->>-78xy"
+-- Left (line 1, column 11):
+-- unexpected 'x'
+-- expecting digit, "->", "::", ".", "," or end of input
 pFieldForest :: Parser [Tree SelectItem]
-pFieldForest = pFieldTree `sepBy1` lexeme (char ',')
+pFieldForest = pFieldTree `sepBy` lexeme (char ',')
   where
-    pFieldTree :: Parser (Tree SelectItem)
-    pFieldTree =  try (Node <$> pRelationSelect <*> between (char '(') (char ')') pFieldForest) <|>
+    pFieldTree =  Node <$> try pSpreadRelationSelect <*> between (char '(') (char ')') pFieldForest <|>
+                  Node <$> try pRelationSelect       <*> between (char '(') (char ')') pFieldForest <|>
                   Node <$> pFieldSelect <*> pure []
 
-pStar :: Parser Text
-pStar = string "*" $> "*"
-
 -- |
 -- Parse field names
 --
@@ -393,6 +377,23 @@ pFieldName =
 --
 -- >>> P.parse pJsonPath "" "->0.desc"
 -- Right [JArrow {jOp = JIdx {jVal = "+0"}}]
+--
+-- Fails on badly formed negatives
+--
+-- >>> P.parse pJsonPath "" "->>-78xy"
+-- Left (line 1, column 7):
+-- unexpected 'x'
+-- expecting digit, "->", "::", ".", "," or end of input
+--
+-- >>> P.parse pJsonPath "" "->>--34"
+-- Left (line 1, column 5):
+-- unexpected "-"
+-- expecting digit
+--
+-- >>> P.parse pJsonPath "" "->>-xy-4"
+-- Left (line 1, column 5):
+-- unexpected "x"
+-- expecting digit
 pJsonPath :: Parser JsonPath
 pJsonPath = many pJsonOperation
   where
@@ -448,27 +449,12 @@ aliasSeparator = char ':' >> notFollowedBy (char ':')
 -- Left (line 1, column 6):
 -- unexpected '>'
 pRelationSelect :: Parser SelectItem
-pRelationSelect = lexeme $ try ( do
+pRelationSelect = lexeme $ do
     alias <- optionMaybe ( try(pFieldName <* aliasSeparator) )
     name <- pFieldName
-    prm1 <- optionMaybe pEmbedParam
-    prm2 <- optionMaybe pEmbedParam
+    (hint, jType) <- pEmbedParams
     try (void $ lookAhead (string "("))
-    return $ SelectRelation name alias (embedParamHint prm1 <|> embedParamHint prm2) (embedParamJoin prm1 <|> embedParamJoin prm2)
-  )
-  where
-    pEmbedParam :: Parser EmbedParam
-    pEmbedParam =
-      char '!' *> (
-        try (string "left"  $> EPJoinType JTLeft)  <|>
-        try (string "inner" $> EPJoinType JTInner) <|>
-        try (EPHint <$> pFieldName))
-    embedParamHint prm = case prm of
-      Just (EPHint hint) -> Just hint
-      _                  -> Nothing
-    embedParamJoin prm = case prm of
-      Just (EPJoinType jt) -> Just jt
-      _                    -> Nothing
+    return $ SelectRelation name alias hint jType
 
 -- |
 -- Parse regular fields in select
@@ -506,43 +492,123 @@ pRelationSelect = lexeme $ try ( do
 -- unexpected end of input
 -- expecting letter or digit
 pFieldSelect :: Parser SelectItem
-pFieldSelect = lexeme $
-  try (
-    do
-      alias <- optionMaybe ( try(pFieldName <* aliasSeparator) )
-      fld <- pField
-      cast' <- optionMaybe (string "::" *> pIdentifier)
-      pEnd
-      return $ SelectField fld (toS <$> cast') alias
-  )
-  <|> do
+pFieldSelect = lexeme $ try (do
     s <- pStar
     pEnd
-    return $ SelectField (s, []) Nothing Nothing
+    return $ SelectField (s, []) Nothing Nothing)
+  <|> do
+    alias <- optionMaybe ( try(pFieldName <* aliasSeparator) )
+    fld <- pField
+    cast' <- optionMaybe (string "::" *> pIdentifier)
+    pEnd
+    return $ SelectField fld (toS <$> cast') alias
   where
     pEnd = try (void $ lookAhead (string ")")) <|>
            try (void $ lookAhead (string ",")) <|>
            try eof
+    pStar = string "*" $> "*"
+
+
+-- |
+-- Parse spread relations in select
+--
+-- >>> P.parse pSpreadRelationSelect "" "...rel(*)"
+-- Right (SpreadRelation {selRelation = "rel", selHint = Nothing, selJoinType = Nothing})
+--
+-- >>> P.parse pSpreadRelationSelect "" "...rel!hint!inner(*)"
+-- Right (SpreadRelation {selRelation = "rel", selHint = Just "hint", selJoinType = Just JTInner})
+--
+-- >>> P.parse pSpreadRelationSelect "" "rel(*)"
+-- Left (line 1, column 1):
+-- unexpected "r"
+-- expecting "..."
+--
+-- >>> P.parse pSpreadRelationSelect "" "alias:...rel(*)"
+-- Left (line 1, column 1):
+-- unexpected "a"
+-- expecting "..."
+--
+-- >>> P.parse pSpreadRelationSelect "" "...rel->jsonpath(*)"
+-- Left (line 1, column 9):
+-- unexpected '>'
+pSpreadRelationSelect :: Parser SelectItem
+pSpreadRelationSelect = lexeme $ do
+    name <- string "..." >> pFieldName
+    (hint, jType) <- pEmbedParams
+    try (void $ lookAhead (string "("))
+    return $ SpreadRelation name hint jType
+
+pEmbedParams :: Parser (Maybe Hint, Maybe JoinType)
+pEmbedParams = do
+  prm1 <- optionMaybe pEmbedParam
+  prm2 <- optionMaybe pEmbedParam
+  return (embedParamHint prm1 <|> embedParamHint prm2, embedParamJoin prm1 <|> embedParamJoin prm2)
+  where
+    pEmbedParam :: Parser EmbedParam
+    pEmbedParam =
+      char '!' *> (
+        try (string "left"  $> EPJoinType JTLeft)  <|>
+        try (string "inner" $> EPJoinType JTInner) <|>
+        try (EPHint <$> pFieldName))
+    embedParamHint prm = case prm of
+      Just (EPHint hint) -> Just hint
+      _                  -> Nothing
+    embedParamJoin prm = case prm of
+      Just (EPJoinType jt) -> Just jt
+      _                    -> Nothing
 
 -- |
 -- Parse operator expression used in horizontal filtering
 --
 -- >>> P.parse (pOpExpr pSingleVal) "" "fts().value"
--- Left (line 1, column 7):
--- unknown single value operator fts()
+-- Left (line 1, column 5):
+-- unexpected ")"
+-- expecting operator (eq, gt, ...)
+--
+-- >>> P.parse (pOpExpr pSingleVal) "" "eq(any).value"
+-- Right (OpExpr False (OpQuant OpEqual (Just QuantAny) "value"))
+--
+-- >>> P.parse (pOpExpr pSingleVal) "" "eq(all).value"
+-- Right (OpExpr False (OpQuant OpEqual (Just QuantAll) "value"))
+--
+-- >>> P.parse (pOpExpr pSingleVal) "" "not.eq(all).value"
+-- Right (OpExpr True (OpQuant OpEqual (Just QuantAll) "value"))
+--
+-- >>> P.parse (pOpExpr pSingleVal) "" "eq().value"
+-- Left (line 1, column 4):
+-- unexpected ")"
+-- expecting operator (eq, gt, ...)
+--
+-- >>> P.parse (pOpExpr pSingleVal) "" "is().value"
+-- Left (line 1, column 3):
+-- unexpected "("
+-- expecting operator (eq, gt, ...)
+--
+-- >>> P.parse (pOpExpr pSingleVal) "" "in().value"
+-- Left (line 1, column 3):
+-- unexpected "("
+-- expecting operator (eq, gt, ...)
 pOpExpr :: Parser SingleVal -> Parser OpExpr
-pOpExpr pSVal = try ( string "not" *> pDelimiter *> (OpExpr True <$> pOperation)) <|> OpExpr False <$> pOperation
+pOpExpr pSVal = do
+  boolExpr <- try (string "not" *> pDelimiter $> True) <|> pure False
+  OpExpr boolExpr <$> pOperation
   where
     pOperation :: Parser Operation
-    pOperation = pIn <|> pIs <|> try pFts <|> pOp <?> "operator (eq, gt, ...)"
+    pOperation = pIn <|> pIs <|> pIsDist <|> try pFts <|> try pSimpleOp <|> try pQuantOp <?> "operator (eq, gt, ...)"
 
     pIn = In <$> (try (string "in" *> pDelimiter) *> pListVal)
     pIs = Is <$> (try (string "is" *> pDelimiter) *> pTriVal)
 
-    pOp = do
-      opStr <- try (P.manyTill anyChar (try pDelimiter))
-      op <- parseMaybe ("unknown single value operator " <> opStr) . operator $ toS opStr
-      Op op <$> pSVal
+    pIsDist = IsDistinctFrom <$> (try (string "isdistinct" *> pDelimiter) *> pSVal)
+
+    pSimpleOp = do
+      op <- simpleOperator
+      pDelimiter *> (Op op <$> pSVal)
+
+    pQuantOp = do
+      op <- quantOperator
+      quant <- optionMaybe $ try (between (char '(') (char ')') (try (string "any" $> QuantAny) <|> string "all" $> QuantAll))
+      pDelimiter *> (OpQuant op quant <$> pSVal)
 
     pTriVal = try (ciString "null"    $> TriNull)
           <|> try (ciString "unknown" $> TriUnknown)
@@ -551,15 +617,14 @@ pOpExpr pSVal = try ( string "not" *> pDelimiter *> (OpExpr True <$> pOperation)
           <?> "null or trilean value (unknown, true, false)"
 
     pFts = do
-      opStr <- try (P.many (noneOf ".("))
-      op <- parseMaybe ("unknown fts operator " <> opStr) . ftsOperator $ toS opStr
+      op <-  try (string "fts"   $> FilterFts)
+         <|> try (string "plfts" $> FilterFtsPlain)
+         <|> try (string "phfts" $> FilterFtsPhrase)
+         <|> try (string "wfts"  $> FilterFtsWebsearch)
+
       lang <- optionMaybe $ try (between (char '(') (char ')') pIdentifier)
       pDelimiter >> Fts op (toS <$> lang) <$> pSVal
 
-    parseMaybe :: [Char] -> Maybe a -> Parser a
-    parseMaybe err Nothing = parserFail err
-    parseMaybe _ (Just x)  = pure x
-
     -- case insensitive char and string
     ciChar :: Char -> GenParser Char state Char
     ciChar c = char c <|> char (toUpper c)
@@ -583,24 +648,119 @@ pQuotedValue = toS <$> (char '"' *> many pCharsOrSlashed <* char '"')
 pDelimiter :: Parser Char
 pDelimiter = char '.' <?> "delimiter (.)"
 
+-- |
+-- Parses the elements in the order query parameter
+--
+-- >>> P.parse pOrder "" "name.desc.nullsfirst"
+-- Right [OrderTerm {otTerm = ("name",[]), otDirection = Just OrderDesc, otNullOrder = Just OrderNullsFirst}]
+--
+-- >>> P.parse pOrder "" "json_col->key.asc.nullslast"
+-- Right [OrderTerm {otTerm = ("json_col",[JArrow {jOp = JKey {jVal = "key"}}]), otDirection = Just OrderAsc, otNullOrder = Just OrderNullsLast}]
+--
+-- >>> P.parse pOrder "" "clients(json_col->key).desc.nullsfirst"
+-- Right [OrderRelationTerm {otRelation = "clients", otRelTerm = ("json_col",[JArrow {jOp = JKey {jVal = "key"}}]), otDirection = Just OrderDesc, otNullOrder = Just OrderNullsFirst}]
+--
+-- >>> P.parse pOrder "" "clients(name,id)"
+-- Left (line 1, column 8):
+-- unexpected '('
+-- expecting letter, digit, "-", "->>", "->", delimiter (.), "," or end of input
+--
+-- >>> P.parse pOrder "" "name,clients(name),id"
+-- Right [OrderTerm {otTerm = ("name",[]), otDirection = Nothing, otNullOrder = Nothing},OrderRelationTerm {otRelation = "clients", otRelTerm = ("name",[]), otDirection = Nothing, otNullOrder = Nothing},OrderTerm {otTerm = ("id",[]), otDirection = Nothing, otNullOrder = Nothing}]
+--
+-- >>> P.parse pOrder "" "id.ac"
+-- Left (line 1, column 4):
+-- unexpected "c"
+-- expecting "asc", "desc", "nullsfirst" or "nullslast"
+--
+-- >>> P.parse pOrder "" "id.descc"
+-- Left (line 1, column 8):
+-- unexpected 'c'
+-- expecting delimiter (.), "," or end of input
+--
+-- >>> P.parse pOrder "" "id.nulsfist"
+-- Left (line 1, column 4):
+-- unexpected "n"
+-- expecting "asc", "desc", "nullsfirst" or "nullslast"
+--
+-- >>> P.parse pOrder "" "id.nullslasttt"
+-- Left (line 1, column 13):
+-- unexpected 't'
+-- expecting "," or end of input
+--
+-- >>> P.parse pOrder "" "id.smth34"
+-- Left (line 1, column 4):
+-- unexpected "s"
+-- expecting "asc", "desc", "nullsfirst" or "nullslast"
+--
+-- >>> P.parse pOrder "" "id.asc.nlsfst"
+-- Left (line 1, column 8):
+-- unexpected "l"
+-- expecting "nullsfirst" or "nullslast"
+--
+-- >>> P.parse pOrder "" "id.asc.nullslasttt"
+-- Left (line 1, column 17):
+-- unexpected 't'
+-- expecting "," or end of input
+--
+-- >>> P.parse pOrder "" "id.asc.smth34"
+-- Left (line 1, column 8):
+-- unexpected "s"
+-- expecting "nullsfirst" or "nullslast"
 pOrder :: Parser [OrderTerm]
-pOrder = lexeme pOrderTerm `sepBy1` char ','
-
-pOrderTerm :: Parser OrderTerm
-pOrderTerm = do
-  fld <- pField
-  dir <- optionMaybe $
-         try (pDelimiter *> string "asc" $> OrderAsc) <|>
-         try (pDelimiter *> string "desc" $> OrderDesc)
-  nls <- optionMaybe pNulls <* pEnd <|>
-         pEnd $> Nothing
-  return $ OrderTerm fld dir nls
+pOrder = lexeme (try pOrderRelationTerm <|> pOrderTerm) `sepBy1` char ','
   where
+    pOrderTerm = do
+      fld <- pField
+      dir <- optionMaybe pOrdDir
+      nls <- optionMaybe pNulls <* pEnd <|>
+             pEnd $> Nothing
+      return $ OrderTerm fld dir nls
+
+    pOrderRelationTerm = do
+      nam <- pFieldName
+      fld <- between (char '(') (char ')') pField
+      dir <- optionMaybe pOrdDir
+      nls <- optionMaybe pNulls <* pEnd <|> pEnd $> Nothing
+      return $ OrderRelationTerm nam fld dir nls
+
+    pNulls :: Parser OrderNulls
     pNulls = try (pDelimiter *> string "nullsfirst" $> OrderNullsFirst) <|>
              try (pDelimiter *> string "nullslast"  $> OrderNullsLast)
-    pEnd = try (void $ lookAhead (char ',')) <|>
-           try eof
 
+    pOrdDir :: Parser OrderDirection
+    pOrdDir = try (pDelimiter *> string "asc" $> OrderAsc) <|>
+              try (pDelimiter *> string "desc" $> OrderDesc)
+
+    pEnd = try (void $ lookAhead (char ',')) <|> try eof
+
+-- |
+-- Parses the elements inside or/and
+--
+-- >>> P.parse pLogicTree "" "or()"
+-- Left (line 1, column 4):
+-- unexpected ")"
+-- expecting field name (* or [a..z0..9_$]), negation operator (not) or logic operator (and, or)
+--
+-- >>> P.parse pLogicTree "" "or(id.in.1,2,id.eq.3)"
+-- Left (line 1, column 10):
+-- unexpected "1"
+-- expecting "("
+--
+-- >>> P.parse pLogicTree "" "or)("
+-- Left (line 1, column 3):
+-- unexpected ")"
+-- expecting "("
+--
+-- >>> P.parse pLogicTree "" "and(ord(id.eq.1,id.eq.1),id.eq.2)"
+-- Left (line 1, column 7):
+-- unexpected "d"
+-- expecting "("
+--
+-- >>> P.parse pLogicTree "" "or(id.eq.1,not.xor(id.eq.2,id.eq.3))"
+-- Left (line 1, column 16):
+-- unexpected "x"
+-- expecting logic operator (and, or)
 pLogicTree :: Parser LogicTree
 pLogicTree = Stmnt <$> try pLogicFilter
              <|> Expr <$> pNot <*> pLogicOp <*> (lexeme (char '(') *> pLogicTree `sepBy1` lexeme (char ',') <* lexeme (char ')'))

src/PostgREST/ApiRequest/Types.hs

@@ -19,6 +19,7 @@ module PostgREST.ApiRequest.Types
   , NodeName
   , OpExpr(..)
   , Operation (..)
+  , OpQuantifier(..)
   , OrderDirection(..)
   , OrderNulls(..)
   , OrderTerm(..)
@@ -27,6 +28,7 @@ module PostgREST.ApiRequest.Types
   , SingleVal
   , TrileanVal(..)
   , SimpleOperator(..)
+  , QuantOperator(..)
   , FtsOperator(..)
   , SelectItem(..)
   ) where
@@ -34,30 +36,37 @@ module PostgREST.ApiRequest.Types
 import PostgREST.MediaType                (MediaType (..))
 import PostgREST.SchemaCache.Identifiers  (FieldName,
                                            QualifiedIdentifier)
-import PostgREST.SchemaCache.Proc         (ProcDescription (..))
 import PostgREST.SchemaCache.Relationship (Relationship,
                                            RelationshipsMap)
+import PostgREST.SchemaCache.Routine      (Routine (..))
 
 import Protolude
 
--- | The select value in `/tbl?select=alias:field::cast`
+-- | The value in `/tbl?select=alias:field::cast`
 data SelectItem
   = SelectField
     { selField :: Field
     , selCast  :: Maybe Cast
     , selAlias :: Maybe Alias
     }
+-- | The value in `/tbl?select=alias:another_tbl(*)`
   | SelectRelation
     { selRelation :: FieldName
     , selAlias    :: Maybe Alias
     , selHint     :: Maybe Hint
     , selJoinType :: Maybe JoinType
     }
-  deriving (Eq)
+-- | The value in `/tbl?select=...another_tbl(*)`
+  | SpreadRelation
+    { selRelation :: FieldName
+    , selHint     :: Maybe Hint
+    , selJoinType :: Maybe JoinType
+    }
+  deriving (Eq, Show)
 
 data ApiRequestError
   = AmbiguousRelBetween Text Text [Relationship]
-  | AmbiguousRpc [ProcDescription]
+  | AmbiguousRpc [Routine]
   | BinaryFieldError MediaType
   | MediaTypeError [ByteString]
   | InvalidBody ByteString
@@ -67,39 +76,52 @@ data ApiRequestError
   | LimitNoOrderError
   | NotFound
   | NoRelBetween Text Text (Maybe Text) Text RelationshipsMap
-  | NoRpc Text Text [Text] Bool MediaType Bool [QualifiedIdentifier] [ProcDescription]
+  | NoRpc Text Text [Text] Bool MediaType Bool [QualifiedIdentifier] [Routine]
   | NotEmbedded Text
-  | ParseRequestError Text Text
-  | PutRangeNotAllowedError
+  | PutLimitNotAllowedError
   | QueryParamError QPError
+  | RelatedOrderNotToOne Text Text
+  | SpreadNotToOne Text Text
+  | UnacceptableFilter Text
   | UnacceptableSchema [Text]
   | UnsupportedMethod ByteString
+  | ColumnNotFound Text Text
+  deriving Show
 
 data QPError = QPError Text Text
+  deriving Show
 data RangeError
   = NegativeLimit
   | LowerGTUpper
   | OutOfBounds Text Text
+  deriving Show
 
 type NodeName = Text
 type Depth = Integer
 
-data OrderTerm = OrderTerm
-  { otTerm      :: Field
-  , otDirection :: Maybe OrderDirection
-  , otNullOrder :: Maybe OrderNulls
-  }
-  deriving (Eq)
+data OrderTerm
+  = OrderTerm
+    { otTerm      :: Field
+    , otDirection :: Maybe OrderDirection
+    , otNullOrder :: Maybe OrderNulls
+    }
+  | OrderRelationTerm
+    { otRelation  :: FieldName
+    , otRelTerm   :: Field
+    , otDirection :: Maybe OrderDirection
+    , otNullOrder :: Maybe OrderNulls
+    }
+  deriving (Eq, Show)
 
 data OrderDirection
   = OrderAsc
   | OrderDesc
-  deriving (Eq)
+  deriving (Eq, Show)
 
 data OrderNulls
   = OrderNullsFirst
   | OrderNullsLast
-  deriving (Eq)
+  deriving (Eq, Show)
 
 type Field = (FieldName, JsonPath)
 type Cast = Text
@@ -116,7 +138,7 @@ data EmbedParam
 data JoinType
   = JTInner
   | JTLeft
-  deriving Eq
+  deriving (Eq, Show)
 
 -- | Path of the embedded levels, e.g "clients.projects.name=eq.." gives Path
 -- ["clients", "projects"]
@@ -130,7 +152,7 @@ type JsonPath = [JsonOperation]
 data JsonOperation
   = JArrow { jOp :: JsonOperand }
   | J2Arrow { jOp :: JsonOperand }
-  deriving (Eq)
+  deriving (Eq, Show, Ord)
 
 -- | Represents the key(`->'key'`) or index(`->'1`::int`), the index is Text
 -- because we reuse our escaping functons and let pg do the casting with
@@ -138,7 +160,7 @@ data JsonOperation
 data JsonOperand
   = JKey { jVal :: Text }
   | JIdx { jVal :: Text }
-  deriving (Eq)
+  deriving (Eq, Show, Ord)
 
 -- | Boolean logic expression tree e.g. "and(name.eq.N,or(id.eq.1,id.eq.2))" is:
 --
@@ -150,29 +172,36 @@ data JsonOperand
 data LogicTree
   = Expr Bool LogicOperator [LogicTree]
   | Stmnt Filter
-  deriving (Eq)
+  deriving (Eq, Show)
 
 data LogicOperator
   = And
   | Or
-  deriving Eq
+  deriving (Eq, Show)
 
-data Filter = Filter
+data Filter
+  = Filter
   { field  :: Field
   , opExpr :: OpExpr
   }
-  deriving (Eq)
+  deriving (Eq, Show)
 
-data OpExpr =
-  OpExpr Bool Operation
-  deriving (Eq)
+data OpExpr
+  = OpExpr Bool Operation
+  | NoOpExpr Text
+  deriving (Eq, Show)
+
+data OpQuantifier = QuantAny | QuantAll
+  deriving (Eq, Show)
 
 data Operation
   = Op SimpleOperator SingleVal
+  | OpQuant QuantOperator (Maybe OpQuantifier) SingleVal
   | In ListVal
   | Is TrileanVal
+  | IsDistinctFrom SingleVal
   | Fts FtsOperator (Maybe Language) SingleVal
-  deriving (Eq)
+  deriving (Eq, Show)
 
 type Language = Text
 
@@ -188,17 +217,23 @@ data TrileanVal
   | TriFalse
   | TriNull
   | TriUnknown
-  deriving Eq
+  deriving (Eq, Show)
 
-data SimpleOperator
+-- Operators that are quantifiable, i.e. they can be used with the any/all modifiers
+data QuantOperator
   = OpEqual
   | OpGreaterThanEqual
   | OpGreaterThan
   | OpLessThanEqual
   | OpLessThan
-  | OpNotEqual
   | OpLike
   | OpILike
+  | OpMatch
+  | OpIMatch
+  deriving (Eq, Show)
+
+data SimpleOperator
+  = OpNotEqual
   | OpContains
   | OpContained
   | OpOverlap
@@ -207,14 +242,13 @@ data SimpleOperator
   | OpNotExtendsRight
   | OpNotExtendsLeft
   | OpAdjacent
-  | OpMatch
-  | OpIMatch
-  deriving Eq
+  deriving (Eq, Show)
 
+--
 -- | Operators for full text search operators
 data FtsOperator
   = FilterFts
   | FilterFtsPlain
   | FilterFtsPhrase
   | FilterFtsWebsearch
-  deriving Eq
+  deriving (Eq, Show)

src/PostgREST/App.hs

@@ -9,7 +9,6 @@ Some of its functionality includes:
 - Producing HTTP Headers according to RFCs.
 - Content Negotiation
 -}
-{-# LANGUAGE LambdaCase      #-}
 {-# LANGUAGE RecordWildCards #-}
 module PostgREST.App
   ( SignalHandlerInstaller
@@ -20,18 +19,20 @@ module PostgREST.App
 
 
 import Control.Monad.Except     (liftEither)
-import Data.Either.Combinators  (mapLeft, whenLeft)
+import Data.Either.Combinators  (mapLeft)
 import Data.Maybe               (fromJust)
 import Data.String              (IsString (..))
 import Network.Wai.Handler.Warp (defaultSettings, setHost, setPort,
                                  setServerName)
 import System.Posix.Types       (FileMode)
 
-import qualified Hasql.Pool                 as SQL
+import qualified Data.HashMap.Strict        as HM
+import qualified Data.Text.Encoding         as T
 import qualified Hasql.Transaction.Sessions as SQL
 import qualified Network.Wai                as Wai
 import qualified Network.Wai.Handler.Warp   as Warp
 
+import qualified PostgREST.Admin            as Admin
 import qualified PostgREST.ApiRequest       as ApiRequest
 import qualified PostgREST.ApiRequest.Types as ApiRequestTypes
 import qualified PostgREST.AppState         as AppState
@@ -42,18 +43,18 @@ import qualified PostgREST.Logger           as Logger
 import qualified PostgREST.Plan             as Plan
 import qualified PostgREST.Query            as Query
 import qualified PostgREST.Response         as Response
-import qualified PostgREST.Workers          as Workers
 
-import PostgREST.ApiRequest       (Action (..), ApiRequest (..),
-                                   Mutation (..), Target (..))
-import PostgREST.AppState         (AppState)
-import PostgREST.Auth             (AuthResult (..))
-import PostgREST.Config           (AppConfig (..), LogLevel (..))
-import PostgREST.Config.PgVersion (PgVersion (..))
-import PostgREST.Error            (Error)
-import PostgREST.Query            (DbHandler)
-import PostgREST.SchemaCache      (SchemaCache (..))
-import PostgREST.Version          (prettyVersion)
+import PostgREST.ApiRequest          (Action (..), ApiRequest (..),
+                                      Mutation (..), Target (..))
+import PostgREST.AppState            (AppState)
+import PostgREST.Auth                (AuthResult (..))
+import PostgREST.Config              (AppConfig (..))
+import PostgREST.Config.PgVersion    (PgVersion (..))
+import PostgREST.Error               (Error)
+import PostgREST.Query               (DbHandler)
+import PostgREST.SchemaCache         (SchemaCache (..))
+import PostgREST.SchemaCache.Routine (Routine (..))
+import PostgREST.Version             (docsVersion, prettyVersion)
 
 import Protolude hiding (Handler)
 
@@ -66,14 +67,14 @@ type SocketRunner = Warp.Settings -> Wai.Application -> FileMode -> FilePath ->
 run :: SignalHandlerInstaller -> Maybe SocketRunner -> AppState -> IO ()
 run installHandlers maybeRunWithSocket appState = do
   conf@AppConfig{..} <- AppState.getConfig appState
-  Workers.connectionWorker appState -- Loads the initial SchemaCache
+  AppState.connectionWorker appState -- Loads the initial SchemaCache
   installHandlers appState
   -- reload schema cache + config on NOTIFY
-  Workers.runListener conf appState
+  AppState.runListener conf appState
 
-  Workers.runAdmin conf appState $ serverSettings conf
+  Admin.runAdmin conf appState $ serverSettings conf
 
-  let app = postgrest configLogLevel appState (Workers.connectionWorker appState)
+  let app = postgrest conf appState (AppState.connectionWorker appState)
 
   case configServerUnixSocket of
     Just socket ->
@@ -97,25 +98,25 @@ serverSettings AppConfig{..} =
     & setServerName ("postgrest/" <> prettyVersion)
 
 -- | PostgREST application
-postgrest :: LogLevel -> AppState.AppState -> IO () -> Wai.Application
-postgrest logLevel appState connWorker =
+postgrest :: AppConfig -> AppState.AppState -> IO () -> Wai.Application
+postgrest conf appState connWorker =
+  Response.traceHeaderMiddleware conf .
   Cors.middleware .
   Auth.middleware appState .
-  Logger.middleware logLevel $
+  Logger.middleware (configLogLevel conf) $
     -- fromJust can be used, because the auth middleware will **always** add
     -- some AuthResult to the vault.
     \req respond -> case fromJust $ Auth.getResult req of
       Left err -> respond $ Error.errorResponseFor err
       Right authResult -> do
-        conf <- AppState.getConfig appState
+        appConf <- AppState.getConfig appState -- the config must be read again because it can reload
         maybeSchemaCache <- AppState.getSchemaCache appState
         pgVer <- AppState.getPgVersion appState
-        jsonDbS <- AppState.getJsonDbS appState
 
         let
           eitherResponse :: IO (Either Error Wai.Response)
           eitherResponse =
-            runExceptT $ postgrestResponse appState conf maybeSchemaCache jsonDbS pgVer authResult req
+            runExceptT $ postgrestResponse appState appConf maybeSchemaCache pgVer authResult req
 
         response <- either Error.errorResponseFor identity <$> eitherResponse
         -- Launch the connWorker when the connection is down.  The postgrest
@@ -131,12 +132,11 @@ postgrestResponse
   :: AppState.AppState
   -> AppConfig
   -> Maybe SchemaCache
-  -> ByteString
   -> PgVersion
   -> AuthResult
   -> Wai.Request
   -> Handler IO Wai.Response
-postgrestResponse appState conf@AppConfig{..} maybeSchemaCache jsonDbS pgVer authResult@AuthResult{..} req = do
+postgrestResponse appState conf@AppConfig{..} maybeSchemaCache pgVer authResult@AuthResult{..} req = do
   sCache <-
     case maybeSchemaCache of
       Just sCache ->
@@ -148,20 +148,15 @@ postgrestResponse appState conf@AppConfig{..} maybeSchemaCache jsonDbS pgVer aut
 
   apiRequest <-
     liftEither . mapLeft Error.ApiRequestError $
-      ApiRequest.userApiRequest conf sCache req body
+      ApiRequest.userApiRequest conf req body
 
-  Response.optionalRollback conf apiRequest $
-    handleRequest authResult conf appState (Query.txMode apiRequest) (Just authRole /= configDbAnonRole) configDbPreparedStatements jsonDbS pgVer apiRequest sCache
+  handleRequest authResult conf appState (Just authRole /= configDbAnonRole) configDbPreparedStatements pgVer apiRequest sCache
 
-runDbHandler :: AppState.AppState -> SQL.Mode -> Bool -> Bool -> DbHandler b -> Handler IO b
-runDbHandler appState mode authenticated prepared handler = do
+runDbHandler :: AppState.AppState -> SQL.IsolationLevel -> SQL.Mode -> Bool -> Bool -> DbHandler b -> Handler IO b
+runDbHandler appState isoLvl mode authenticated prepared handler = do
   dbResp <- lift $ do
     let transaction = if prepared then SQL.transaction else SQL.unpreparedTransaction
-    res <- AppState.usePool appState . transaction SQL.ReadCommitted mode $ runExceptT handler
-    whenLeft res (\case
-      SQL.AcquisitionTimeoutUsageError -> AppState.debounceLogAcquisitionTimeout appState -- this can happen rapidly for many requests, so we debounce
-      _ -> pure ())
-    return res
+    AppState.usePool appState . transaction isoLvl mode $ runExceptT handler
 
   resp <-
     liftEither . mapLeft Error.PgErr $
@@ -169,52 +164,62 @@ runDbHandler appState mode authenticated prepared handler = do
 
   liftEither resp
 
-handleRequest :: AuthResult -> AppConfig -> AppState.AppState -> SQL.Mode -> Bool -> Bool -> ByteString -> PgVersion -> ApiRequest -> SchemaCache -> Handler IO Wai.Response
-handleRequest AuthResult{..} conf appState mode authenticated prepared jsonDbS pgVer apiReq@ApiRequest{..} sCache =
+handleRequest :: AuthResult -> AppConfig -> AppState.AppState -> Bool -> Bool -> PgVersion -> ApiRequest -> SchemaCache -> Handler IO Wai.Response
+handleRequest AuthResult{..} conf appState authenticated prepared pgVer apiReq@ApiRequest{..} sCache =
   case (iAction, iTarget) of
     (ActionRead headersOnly, TargetIdent identifier) -> do
-      rPlan <- liftEither $ Plan.readPlan identifier conf sCache apiReq
-      resultSet <- runQuery $ Query.readQuery rPlan conf apiReq
+      wrPlan <- liftEither $ Plan.wrappedReadPlan identifier conf sCache apiReq
+      resultSet <- runQuery roleIsoLvl (Plan.wrTxMode wrPlan) $ Query.readQuery wrPlan conf apiReq
       return $ Response.readResponse headersOnly identifier apiReq resultSet
 
     (ActionMutate MutationCreate, TargetIdent identifier) -> do
       mrPlan <- liftEither $ Plan.mutateReadPlan MutationCreate apiReq identifier conf sCache
-      resultSet <- runQuery $ Query.createQuery mrPlan apiReq conf
+      resultSet <- runQuery roleIsoLvl (Plan.mrTxMode mrPlan) $ Query.createQuery mrPlan apiReq conf
       return $ Response.createResponse identifier mrPlan apiReq resultSet
 
     (ActionMutate MutationUpdate, TargetIdent identifier) -> do
       mrPlan <- liftEither $ Plan.mutateReadPlan MutationUpdate apiReq identifier conf sCache
-      resultSet <- runQuery $ Query.updateQuery mrPlan apiReq conf
+      resultSet <- runQuery roleIsoLvl (Plan.mrTxMode mrPlan) $ Query.updateQuery mrPlan apiReq conf
       return $ Response.updateResponse apiReq resultSet
 
     (ActionMutate MutationSingleUpsert, TargetIdent identifier) -> do
       mrPlan <- liftEither $ Plan.mutateReadPlan MutationSingleUpsert apiReq identifier conf sCache
-      resultSet <- runQuery $ Query.singleUpsertQuery mrPlan apiReq conf
+      resultSet <- runQuery roleIsoLvl (Plan.mrTxMode mrPlan) $ Query.singleUpsertQuery mrPlan apiReq conf
       return $ Response.singleUpsertResponse apiReq resultSet
 
     (ActionMutate MutationDelete, TargetIdent identifier) -> do
       mrPlan <- liftEither $ Plan.mutateReadPlan MutationDelete apiReq identifier conf sCache
-      resultSet <- runQuery $ Query.deleteQuery mrPlan apiReq conf
+      resultSet <- runQuery roleIsoLvl (Plan.mrTxMode mrPlan) $ Query.deleteQuery mrPlan apiReq conf
       return $ Response.deleteResponse apiReq resultSet
 
-    (ActionInvoke invMethod, TargetProc proc _) -> do
-      cPlan <- liftEither $ Plan.callReadPlan proc conf sCache apiReq
-      resultSet <- runQuery $ Query.invokeQuery proc cPlan apiReq conf
-      return $ Response.invokeResponse invMethod proc apiReq resultSet
+    (ActionInvoke invMethod, TargetProc identifier _) -> do
+      cPlan <- liftEither $ Plan.callReadPlan identifier conf sCache apiReq invMethod
+      resultSet <- runQuery (fromMaybe roleIsoLvl $ pdIsoLvl (Plan.crProc cPlan))(Plan.crTxMode cPlan) $ Query.invokeQuery (Plan.crProc cPlan) cPlan apiReq conf pgVer
+      return $ Response.invokeResponse invMethod (Plan.crProc cPlan) apiReq resultSet
 
     (ActionInspect headersOnly, TargetDefaultSpec tSchema) -> do
-      oaiResult <- runQuery $ Query.openApiQuery sCache pgVer conf tSchema
-      return $ Response.openApiResponse headersOnly oaiResult conf sCache iSchema iNegotiatedByProfile
+      oaiResult <- runQuery roleIsoLvl Plan.inspectPlanTxMode $ Query.openApiQuery sCache pgVer conf tSchema
+      return $ Response.openApiResponse (T.decodeUtf8 prettyVersion, docsVersion) headersOnly oaiResult conf sCache iSchema iNegotiatedByProfile
 
-    (ActionInfo, _) ->
-      return $ Response.infoResponse iTarget sCache
+    (ActionInfo, TargetIdent identifier) ->
+      return $ Response.infoIdentResponse identifier sCache
+
+    (ActionInfo, TargetProc identifier _) -> do
+      cPlan <- liftEither $ Plan.callReadPlan identifier conf sCache apiReq ApiRequest.InvHead
+      return $ Response.infoProcResponse (Plan.crProc cPlan)
+
+    (ActionInfo, TargetDefaultSpec _) ->
+      return Response.infoRootResponse
 
     _ ->
       -- This is unreachable as the ApiRequest.hs rejects it before
       -- TODO Refactor the Action/Target types to remove this line
       throwError $ Error.ApiRequestError ApiRequestTypes.NotFound
   where
-    runQuery query =
-      runDbHandler appState mode authenticated prepared $ do
-        Query.setPgLocals conf authClaims authRole apiReq jsonDbS pgVer
+    roleSettings = fromMaybe mempty (HM.lookup authRole $ configRoleSettings conf)
+    roleIsoLvl = HM.findWithDefault SQL.ReadCommitted authRole $ configRoleIsoLvl conf
+    runQuery isoLvl mode query =
+      runDbHandler appState isoLvl mode authenticated prepared $ do
+        Query.setPgLocals conf authClaims authRole (HM.toList roleSettings) apiReq pgVer
+        Query.runPreReq conf
         query

src/PostgREST/AppState.hs

@@ -1,52 +1,63 @@
+{-# LANGUAGE LambdaCase      #-}
+{-# LANGUAGE NamedFieldPuns  #-}
 {-# LANGUAGE RecordWildCards #-}
 
 module PostgREST.AppState
   ( AppState
   , destroy
-  , flushPool
   , getConfig
   , getSchemaCache
   , getIsListenerOn
-  , getJsonDbS
   , getMainThreadId
   , getPgVersion
   , getRetryNextIn
   , getTime
-  , getWorkerSem
   , init
   , initWithPool
   , logWithZTime
-  , logPgrstError
-  , putConfig
   , putSchemaCache
-  , putIsListenerOn
-  , putJsonDbS
   , putPgVersion
-  , putRetryNextIn
-  , signalListener
   , usePool
-  , waitListener
-  , debounceLogAcquisitionTimeout
+  , loadSchemaCache
+  , reReadConfig
+  , connectionWorker
+  , runListener
   ) where
 
-import qualified Data.ByteString.Lazy as LBS
-import qualified Data.Text.Encoding   as T
-import qualified Hasql.Pool           as SQL
-import qualified Hasql.Session        as SQL
-import qualified PostgREST.Error      as Error
+import qualified Data.ByteString.Char8      as BS
+import qualified Data.ByteString.Lazy       as LBS
+import           Data.Either.Combinators    (whenLeft)
+import qualified Data.Text.Encoding         as T
+import           Hasql.Connection           (acquire)
+import qualified Hasql.Notifications        as SQL
+import qualified Hasql.Pool                 as SQL
+import qualified Hasql.Session              as SQL
+import qualified Hasql.Transaction.Sessions as SQL
+import qualified PostgREST.Error            as Error
+import           PostgREST.Version          (prettyVersion)
 
 import Control.AutoUpdate (defaultUpdateSettings, mkAutoUpdate,
                            updateAction)
 import Control.Debounce
+import Control.Retry      (RetryStatus, capDelay, exponentialBackoff,
+                           retrying, rsPreviousDelay)
 import Data.IORef         (IORef, atomicWriteIORef, newIORef,
                            readIORef)
 import Data.Time          (ZonedTime, defaultTimeLocale, formatTime,
                            getZonedTime)
 import Data.Time.Clock    (UTCTime, getCurrentTime)
 
-import PostgREST.Config           (AppConfig (..))
-import PostgREST.Config.PgVersion (PgVersion (..), minimumPgVersion)
-import PostgREST.SchemaCache      (SchemaCache)
+import PostgREST.Config                  (AppConfig (..),
+                                          addFallbackAppName,
+                                          readAppConfig)
+import PostgREST.Config.Database         (queryDbSettings,
+                                          queryPgVersion,
+                                          queryRoleSettings)
+import PostgREST.Config.PgVersion        (PgVersion (..),
+                                          minimumPgVersion)
+import PostgREST.SchemaCache             (SchemaCache,
+                                          querySchemaCache)
+import PostgREST.SchemaCache.Identifiers (dumpQi)
 
 import Protolude
 
@@ -58,10 +69,8 @@ data AppState = AppState
   , statePgVersion                :: IORef PgVersion
   -- | No schema cache at the start. Will be filled in by the connectionWorker
   , stateSchemaCache              :: IORef (Maybe SchemaCache)
-  -- | Cached SchemaCache in json
-  , stateJsonDbS                  :: IORef ByteString
-  -- | Binary semaphore to make sure just one connectionWorker can run at a time
-  , stateWorkerSem                :: MVar ()
+  -- | starts the connection worker with a debounce
+  , debouncedConnectionWorker     :: IO ()
   -- | Binary semaphore used to sync the listener(NOTIFY reload) with the connectionWorker.
   , stateListener                 :: MVar ()
   -- | State of the LISTEN channel, used for the admin server checks
@@ -90,8 +99,7 @@ initWithPool pool conf = do
   appState <- AppState pool
     <$> newIORef minimumPgVersion -- assume we're in a supported version when starting, this will be corrected on a later step
     <*> newIORef Nothing
-    <*> newIORef mempty
-    <*> newEmptyMVar
+    <*> pure (pure ())
     <*> newEmptyMVar
     <*> newIORef False
     <*> newIORef conf
@@ -101,7 +109,8 @@ initWithPool pool conf = do
     <*> newIORef 0
     <*> pure (pure ())
 
-  deb <-
+
+  debLogTimeout <-
     let oneSecond = 1000000 in
     mkDebounce defaultDebounceSettings
        { debounceAction = logPgrstError appState SQL.AcquisitionTimeoutUsageError
@@ -109,7 +118,15 @@ initWithPool pool conf = do
        , debounceEdge = leadingEdge -- logs at the start and the end
        }
 
-  return appState { debounceLogAcquisitionTimeout = deb }
+  debWorker <-
+    let decisecond = 100000 in
+    mkDebounce defaultDebounceSettings
+       { debounceAction = internalConnectionWorker appState
+       , debounceFreq = decisecond
+       , debounceEdge = leadingEdge -- runs the worker at the start and the end
+       }
+
+  return appState { debounceLogAcquisitionTimeout = debLogTimeout, debouncedConnectionWorker = debWorker }
 
 destroy :: AppState -> IO ()
 destroy = destroyPool
@@ -120,11 +137,17 @@ initPool AppConfig{..} =
     configDbPoolSize
     (fromIntegral configDbPoolAcquisitionTimeout)
     (fromIntegral configDbPoolMaxLifetime)
-    (toUtf8 configDbUri)
+    (fromIntegral configDbPoolMaxIdletime)
+    (toUtf8 $ addFallbackAppName prettyVersion configDbUri)
 
 -- | Run an action with a database connection.
 usePool :: AppState -> SQL.Session a -> IO (Either SQL.UsageError a)
-usePool AppState{..} = SQL.use statePool
+usePool AppState{..} x = do
+  res <- SQL.use statePool x
+  whenLeft res (\case
+    SQL.AcquisitionTimeoutUsageError -> debounceLogAcquisitionTimeout -- this can happen rapidly for many requests, so we debounce
+    _                                -> pure ())
+  return res
 
 -- | Flush the connection pool so that any future use of the pool will
 -- use connections freshly established after this call.
@@ -147,14 +170,8 @@ getSchemaCache = readIORef . stateSchemaCache
 putSchemaCache :: AppState -> Maybe SchemaCache -> IO ()
 putSchemaCache appState = atomicWriteIORef (stateSchemaCache appState)
 
-getJsonDbS :: AppState -> IO ByteString
-getJsonDbS = readIORef . stateJsonDbS
-
-putJsonDbS :: AppState -> ByteString -> IO ()
-putJsonDbS appState = atomicWriteIORef (stateJsonDbS appState)
-
-getWorkerSem :: AppState -> MVar ()
-getWorkerSem = stateWorkerSem
+connectionWorker :: AppState -> IO ()
+connectionWorker = debouncedConnectionWorker
 
 getRetryNextIn :: AppState -> IO Int
 getRetryNextIn = readIORef . stateRetryNextIn
@@ -199,3 +216,263 @@ getIsListenerOn = readIORef . stateIsListenerOn
 
 putIsListenerOn :: AppState -> Bool -> IO ()
 putIsListenerOn = atomicWriteIORef . stateIsListenerOn
+
+-- | Schema cache status
+data SCacheStatus
+  = SCLoaded
+  | SCOnRetry
+  | SCFatalFail
+
+-- | Load the SchemaCache by using a connection from the pool.
+loadSchemaCache :: AppState -> IO SCacheStatus
+loadSchemaCache appState = do
+  conf@AppConfig{..} <- getConfig appState
+  result <-
+    let transaction = if configDbPreparedStatements then SQL.transaction else SQL.unpreparedTransaction in
+    usePool appState . transaction SQL.ReadCommitted SQL.Read $
+      querySchemaCache conf
+  case result of
+    Left e -> do
+      case checkIsFatal e of
+        Just hint -> do
+          logWithZTime appState "A fatal error ocurred when loading the schema cache"
+          logPgrstError appState e
+          logWithZTime appState hint
+          return SCFatalFail
+        Nothing -> do
+          putSchemaCache appState Nothing
+          logWithZTime appState "An error ocurred when loading the schema cache"
+          logPgrstError appState e
+          return SCOnRetry
+
+    Right sCache -> do
+      putSchemaCache appState (Just sCache)
+      logWithZTime appState "Schema cache loaded"
+      return SCLoaded
+
+-- | Current database connection status data ConnectionStatus
+data ConnectionStatus
+  = NotConnected
+  | Connected PgVersion
+  | FatalConnectionError Text
+  deriving (Eq)
+
+-- | The purpose of this worker is to obtain a healthy connection to pg and an
+-- up-to-date schema cache(SchemaCache).  This method is meant to be called
+-- multiple times by the same thread, but does nothing if the previous
+-- invocation has not terminated. In all cases this method does not halt the
+-- calling thread, the work is performed in a separate thread.
+--
+-- Background thread that does the following :
+--  1. Tries to connect to pg server and will keep trying until success.
+--  2. Checks if the pg version is supported and if it's not it kills the main
+--     program.
+--  3. Obtains the sCache. If this fails, it goes back to 1.
+internalConnectionWorker :: AppState -> IO ()
+internalConnectionWorker appState = work
+  where
+    work = do
+      AppConfig{..} <- getConfig appState
+      logWithZTime appState $ "Starting PostgREST " <> T.decodeUtf8 prettyVersion <> "..."
+      logWithZTime appState "Attempting to connect to the database..."
+      connected <- establishConnection appState
+      case connected of
+        FatalConnectionError reason ->
+          -- Fatal error when connecting
+          logWithZTime appState reason >> killThread (getMainThreadId appState)
+        NotConnected ->
+          -- Unreachable because establishConnection will keep trying to connect
+          return ()
+        Connected actualPgVersion -> do
+          -- Procede with initialization
+          putPgVersion appState actualPgVersion
+          when configDbChannelEnabled $
+            signalListener appState
+          logWithZTime appState "Connection successful"
+          -- this could be fail because the connection drops, but the loadSchemaCache will pick the error and retry again
+          -- We cannot retry after it fails immediately, because db-pre-config could have user errors. We just log the error and continue.
+          when configDbConfig $ reReadConfig False appState
+          scStatus <- loadSchemaCache appState
+          case scStatus of
+            SCLoaded ->
+              -- do nothing and proceed if the load was successful
+              return ()
+            SCOnRetry ->
+              -- retry reloading the schema cache
+              work
+            SCFatalFail ->
+              -- die if our schema cache query has an error
+              killThread $ getMainThreadId appState
+
+-- | Repeatedly flush the pool, and check if a connection from the
+-- pool allows access to the PostgreSQL database.
+--
+-- Releasing the pool is key for rapid recovery. Otherwise, the pool
+-- timeout would have to be reached for new healthy connections to be acquired.
+-- Which might not happen if the server is busy with requests. No idle
+-- connection, no pool timeout.
+--
+-- The connection tries are capped, but if the connection times out no error is
+-- thrown, just 'False' is returned.
+establishConnection :: AppState -> IO ConnectionStatus
+establishConnection appState =
+  retrying retrySettings shouldRetry $
+    const $ flushPool appState >> getConnectionStatus
+  where
+    retrySettings = capDelay delayMicroseconds $ exponentialBackoff backoffMicroseconds
+    delayMicroseconds = 32000000 -- 32 seconds
+    backoffMicroseconds = 1000000 -- 1 second
+
+    getConnectionStatus :: IO ConnectionStatus
+    getConnectionStatus = do
+      pgVersion <- usePool appState $ queryPgVersion False -- No need to prepare the query here, as the connection might not be established
+      case pgVersion of
+        Left e -> do
+          logPgrstError appState e
+          case checkIsFatal e of
+            Just reason ->
+              return $ FatalConnectionError reason
+            Nothing ->
+              return NotConnected
+        Right version ->
+          if version < minimumPgVersion then
+            return . FatalConnectionError $
+              "Cannot run in this PostgreSQL version, PostgREST needs at least "
+              <> pgvName minimumPgVersion
+          else
+            return . Connected  $ version
+
+    shouldRetry :: RetryStatus -> ConnectionStatus -> IO Bool
+    shouldRetry rs isConnSucc = do
+      let
+        delay = fromMaybe 0 (rsPreviousDelay rs) `div` backoffMicroseconds
+        itShould = NotConnected == isConnSucc
+      when itShould . logWithZTime appState $
+        "Attempting to reconnect to the database in "
+        <> (show delay::Text)
+        <> " seconds..."
+      when itShould $ putRetryNextIn appState delay
+      return itShould
+
+-- | Re-reads the config plus config options from the db
+reReadConfig :: Bool -> AppState -> IO ()
+reReadConfig startingUp appState = do
+  AppConfig{..} <- getConfig appState
+  dbSettings <-
+    if configDbConfig then do
+      qDbSettings <- usePool appState $ queryDbSettings (dumpQi <$> configDbPreConfig) configDbPreparedStatements
+      case qDbSettings of
+        Left e -> do
+          logWithZTime appState
+            "An error ocurred when trying to query database settings for the config parameters"
+          case checkIsFatal e of
+            Just hint -> do
+              logPgrstError appState e
+              logWithZTime appState hint
+              killThread (getMainThreadId appState)
+            Nothing -> do
+              logPgrstError appState e
+          pure mempty
+        Right x -> pure x
+    else
+      pure mempty
+  (roleSettings, roleIsolationLvl) <-
+    if configDbConfig then do
+      rSettings <- usePool appState $ queryRoleSettings configDbPreparedStatements
+      case rSettings of
+        Left e -> do
+          logWithZTime appState "An error ocurred when trying to query the role settings"
+          logPgrstError appState e
+          pure (mempty, mempty)
+        Right x -> pure x
+    else
+      pure mempty
+  readAppConfig dbSettings configFilePath (Just configDbUri) roleSettings roleIsolationLvl >>= \case
+    Left err   ->
+      if startingUp then
+        panic err -- die on invalid config if the program is starting up
+      else
+        logWithZTime appState $ "Failed reloading config: " <> err
+    Right newConf -> do
+      putConfig appState newConf
+      if startingUp then
+        pass
+      else
+        logWithZTime appState "Config reloaded"
+
+
+runListener :: AppConfig -> AppState -> IO ()
+runListener AppConfig{configDbChannelEnabled} appState =
+  when configDbChannelEnabled $ listener appState
+
+-- | Starts a dedicated pg connection to LISTEN for notifications.  When a
+-- NOTIFY <db-channel> - with an empty payload - is done, it refills the schema
+-- cache.  It uses the connectionWorker in case the LISTEN connection dies.
+listener :: AppState -> IO ()
+listener appState = do
+  AppConfig{..} <- getConfig appState
+  let dbChannel = toS configDbChannel
+
+  -- The listener has to wait for a signal from the connectionWorker.
+  -- This is because when the connection to the db is lost, the listener also
+  -- tries to recover the connection, but not with the same pace as the connectionWorker.
+  -- Not waiting makes stderr quickly fill with connection retries messages from the listener.
+  waitListener appState
+
+  -- forkFinally allows to detect if the thread dies
+  void . flip forkFinally (handleFinally dbChannel) $ do
+    dbOrError <- acquire $ toUtf8 (addFallbackAppName prettyVersion configDbUri)
+    case dbOrError of
+      Right db -> do
+        logWithZTime appState $ "Listening for notifications on the " <> dbChannel <> " channel"
+        putIsListenerOn appState True
+        SQL.listen db $ SQL.toPgIdentifier dbChannel
+        SQL.waitForNotifications handleNotification db
+      _ ->
+        die $ "Could not listen for notifications on the " <> dbChannel <> " channel"
+  where
+    handleFinally dbChannel _ = do
+      -- if the thread dies, we try to recover
+      logWithZTime appState $ "Retrying listening for notifications on the " <> dbChannel <> " channel.."
+      putIsListenerOn appState False
+      -- assume the pool connection was also lost, call the connection worker
+      connectionWorker appState
+      -- retry the listener
+      listener appState
+
+    handleNotification _ msg
+      | BS.null msg            = cacheReloader
+      | msg == "reload schema" = cacheReloader
+      | msg == "reload config" = reReadConfig False appState
+      | otherwise              = pure () -- Do nothing if anything else than an empty message is sent
+
+    cacheReloader =
+      -- reloads the schema cache + restarts pool connections
+      -- it's necessary to restart the pg connections because they cache the pg catalog(see #2620)
+      connectionWorker appState
+
+checkIsFatal :: SQL.UsageError -> Maybe Text
+checkIsFatal (SQL.ConnectionUsageError e)
+  | isAuthFailureMessage = Just $ toS failureMessage
+  | otherwise = Nothing
+  where isAuthFailureMessage =
+          ("FATAL:  password authentication failed" `isInfixOf` failureMessage) ||
+          ("no password supplied" `isInfixOf` failureMessage)
+        failureMessage = BS.unpack $ fromMaybe mempty e
+checkIsFatal(SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ResultError serverError)))
+  = case serverError of
+      -- Check for a syntax error (42601 is the pg code). This would mean the error is on our part somehow, so we treat it as fatal.
+      SQL.ServerError "42601" _ _ _ _
+        -> Just "Hint: This is probably a bug in PostgREST, please report it at https://github.com/PostgREST/postgrest/issues"
+      -- Check for a "prepared statement <name> already exists" error (Code 42P05: duplicate_prepared_statement).
+      -- This would mean that a connection pooler in transaction mode is being used
+      -- while prepared statements are enabled in the PostgREST configuration,
+      -- both of which are incompatible with each other.
+      SQL.ServerError "42P05" _ _ _ _
+        -> Just "Hint: If you are using connection poolers in transaction mode, try setting db-prepared-statements to false."
+      -- Check for a "transaction blocks not allowed in statement pooling mode" error (Code 08P01: protocol_violation).
+      -- This would mean that a connection pooler in statement mode is being used which is not supported in PostgREST.
+      SQL.ServerError "08P01" "transaction blocks not allowed in statement pooling mode" _ _ _
+        -> Just "Hint: Connection poolers in statement mode are not supported."
+      _ -> Nothing
+checkIsFatal _ = Nothing

src/PostgREST/Auth.hs

@@ -23,8 +23,8 @@ import qualified Data.Aeson                      as JSON
 import qualified Data.Aeson.Key                  as K
 import qualified Data.Aeson.KeyMap               as KM
 import qualified Data.Aeson.Types                as JSON
+import qualified Data.ByteString                 as BS
 import qualified Data.ByteString.Lazy.Char8      as LBS
-import qualified Data.Text.Encoding              as T
 import qualified Data.Vault.Lazy                 as Vault
 import qualified Data.Vector                     as V
 import qualified Network.HTTP.Types.Header       as HTTP
@@ -47,7 +47,7 @@ import Protolude
 
 data AuthResult = AuthResult
   { authClaims :: KM.KeyMap JSON.Value
-  , authRole   :: Text
+  , authRole   :: BS.ByteString
   }
 
 -- | Receives the JWT secret and audience (from config) and a JWT and returns a
@@ -63,7 +63,7 @@ parseToken AppConfig{..} token time = do
   liftEither . mapLeft jwtClaimsError $ JSON.toJSON <$> eitherClaims
   where
     validation =
-      JWT.defaultJWTValidationSettings audienceCheck & set JWT.allowedSkew 1
+      JWT.defaultJWTValidationSettings audienceCheck & set JWT.allowedSkew 30
 
     audienceCheck :: JWT.StringOrURI -> Bool
     audienceCheck = maybe (const True) (==) configJwtAudience
@@ -79,7 +79,7 @@ parseClaims AppConfig{..} jclaims@(JSON.Object mclaims) = do
   role <- liftEither . maybeToRight JwtTokenRequired $
     unquoted <$> walkJSPath (Just jclaims) configJwtRoleClaimKey <|> configDbAnonRole
   return AuthResult
-           { authClaims = mclaims & KM.insert "role" (JSON.toJSON role)
+           { authClaims = mclaims & KM.insert "role" (JSON.toJSON $ decodeUtf8 role)
            , authRole = role
            }
   where
@@ -89,9 +89,9 @@ parseClaims AppConfig{..} jclaims@(JSON.Object mclaims) = do
     walkJSPath (Just (JSON.Array ar)) (JSPIdx idx:rest) = walkJSPath (ar V.!? idx) rest
     walkJSPath _                      _                 = Nothing
 
-    unquoted :: JSON.Value -> Text
-    unquoted (JSON.String t) = t
-    unquoted v = T.decodeUtf8 . LBS.toStrict $ JSON.encode v
+    unquoted :: JSON.Value -> BS.ByteString
+    unquoted (JSON.String t) = encodeUtf8 t
+    unquoted v               = LBS.toStrict $ JSON.encode v
 -- impossible case - just added to please -Wincomplete-patterns
 parseClaims _ _ = return AuthResult { authClaims = KM.empty, authRole = mempty }
 
@@ -117,5 +117,5 @@ authResultKey = unsafePerformIO Vault.newKey
 getResult :: Wai.Request -> Maybe (Either Error AuthResult)
 getResult = Vault.lookup authResultKey . Wai.vault
 
-getRole :: Wai.Request -> Maybe Text
+getRole :: Wai.Request -> Maybe BS.ByteString
 getRole req = authRole <$> (rightToMaybe =<< getResult req)

src/PostgREST/CLI.hs

@@ -21,7 +21,6 @@ import PostgREST.AppState    (AppState)
 import PostgREST.Config      (AppConfig (..))
 import PostgREST.SchemaCache (querySchemaCache)
 import PostgREST.Version     (prettyVersion)
-import PostgREST.Workers     (reReadConfig)
 
 import qualified PostgREST.App      as App
 import qualified PostgREST.AppState as AppState
@@ -33,7 +32,7 @@ import Protolude hiding (hPutStrLn)
 main :: App.SignalHandlerInstaller -> Maybe App.SocketRunner -> CLI -> IO ()
 main installSignalHandlers runAppWithSocket CLI{cliCommand, cliPath} = do
   conf@AppConfig{..} <-
-    either panic identity <$> Config.readAppConfig mempty cliPath Nothing
+    either panic identity <$> Config.readAppConfig mempty cliPath Nothing mempty mempty
 
   -- Per https://github.com/PostgREST/postgrest/issues/268, we want to
   -- explicitly close the connections to PostgreSQL on shutdown.
@@ -43,7 +42,7 @@ main installSignalHandlers runAppWithSocket CLI{cliCommand, cliPath} = do
     AppState.destroy
     (\appState -> case cliCommand of
       CmdDumpConfig -> do
-        when configDbConfig $ reReadConfig True appState
+        when configDbConfig $ AppState.reReadConfig True appState
         putStr . Config.toText =<< AppState.getConfig appState
       CmdDumpSchema -> putStrLn =<< dumpSchema appState
       CmdRun -> App.run installSignalHandlers runAppWithSocket appState)
@@ -51,15 +50,12 @@ main installSignalHandlers runAppWithSocket CLI{cliCommand, cliPath} = do
 -- | Dump SchemaCache schema to JSON
 dumpSchema :: AppState -> IO LBS.ByteString
 dumpSchema appState = do
-  AppConfig{..} <- AppState.getConfig appState
+  conf@AppConfig{..} <- AppState.getConfig appState
   result <-
     let transaction = if configDbPreparedStatements then SQL.transaction else SQL.unpreparedTransaction in
     AppState.usePool appState $
       transaction SQL.ReadCommitted SQL.Read $
-        querySchemaCache
-          (toList configDbSchemas)
-          configDbExtraSearchPath
-          configDbPreparedStatements
+        querySchemaCache conf
   case result of
     Left e -> do
       hPutStrLn stderr $ "An error ocurred when loading the schema cache:\n" <> show e
@@ -84,7 +80,7 @@ readCLIShowHelp =
   where
     prefs = O.prefs $ O.showHelpOnError <> O.showHelpOnEmpty
     opts = O.info parser $ O.fullDesc <> progDesc
-    parser = O.helper <*> exampleParser <*> cliParser
+    parser = O.helper <*> versionFlag <*> exampleParser <*> cliParser
 
     progDesc =
       O.progDesc $
@@ -92,6 +88,12 @@ readCLIShowHelp =
         <> BS.unpack prettyVersion
         <> " / create a REST API to an existing Postgres database"
 
+    versionFlag =
+      O.infoOption ("PostgREST " <> BS.unpack prettyVersion) $
+        O.long "version"
+        <> O.short 'v'
+        <> O.help "Show the version information"
+
     exampleParser =
       O.infoOption exampleConfigFile $
         O.long "example"
@@ -136,6 +138,9 @@ exampleConfigFile =
       |## Enable in-database configuration
       |db-config = true
       |
+      |## Function for in-database configuration
+      |## db-pre-config = "postgrest.pre_config"
+      |
       |## Extra schemas to add to the search_path of every request
       |db-extra-search-path = "public"
       |
@@ -154,6 +159,9 @@ exampleConfigFile =
       |## Time in seconds after which to recycle pool connections
       |# db-pool-max-lifetime = 1800
       |
+      |## Time in seconds after which to recycle unused pool connections
+      |# db-pool-max-idletime = 30
+      |
       |## Stored proc to exec immediately after auth
       |# db-pre-request = "stored_proc_name"
       |

src/PostgREST/Config.hs

@@ -24,6 +24,7 @@ module PostgREST.Config
   , readPGRSTEnvironment
   , toURI
   , parseSecret
+  , addFallbackAppName
   ) where
 
 import qualified Crypto.JOSE.Types      as JOSE
@@ -32,6 +33,7 @@ import qualified Data.Aeson             as JSON
 import qualified Data.ByteString        as BS
 import qualified Data.ByteString.Base64 as B64
 import qualified Data.ByteString.Lazy   as LBS
+import qualified Data.CaseInsensitive   as CI
 import qualified Data.Configurator      as C
 import qualified Data.Map.Strict        as M
 import qualified Data.Text              as T
@@ -46,10 +48,15 @@ import Data.List               (lookup)
 import Data.List.NonEmpty      (fromList, toList)
 import Data.Maybe              (fromJust)
 import Data.Scientific         (floatingOrInteger)
+import Network.URI             (escapeURIString,
+                                isUnescapedInURIComponent, parseURI,
+                                uriQuery)
 import Numeric                 (readOct, showOct)
 import System.Environment      (getEnvironment)
 import System.Posix.Types      (FileMode)
 
+import PostgREST.Config.Database         (RoleIsolationLvl,
+                                          RoleSettings)
 import PostgREST.Config.JSPath           (JSPath, JSPathExp (..),
                                           dumpJSPath, pRoleClaimKey)
 import PostgREST.Config.Proxy            (Proxy (..),
@@ -63,7 +70,7 @@ import Protolude hiding (Proxy, toList)
 
 data AppConfig = AppConfig
   { configAppSettings              :: [(Text, Text)]
-  , configDbAnonRole               :: Maybe Text
+  , configDbAnonRole               :: Maybe BS.ByteString
   , configDbChannel                :: Text
   , configDbChannelEnabled         :: Bool
   , configDbExtraSearchPath        :: [Text]
@@ -72,11 +79,13 @@ data AppConfig = AppConfig
   , configDbPoolSize               :: Int
   , configDbPoolAcquisitionTimeout :: Int
   , configDbPoolMaxLifetime        :: Int
+  , configDbPoolMaxIdletime        :: Int
   , configDbPreRequest             :: Maybe QualifiedIdentifier
   , configDbPreparedStatements     :: Bool
   , configDbRootSpec               :: Maybe QualifiedIdentifier
   , configDbSchemas                :: NonEmpty Text
   , configDbConfig                 :: Bool
+  , configDbPreConfig              :: Maybe QualifiedIdentifier
   , configDbTxAllowOverride        :: Bool
   , configDbTxRollbackAll          :: Bool
   , configDbUri                    :: Text
@@ -94,9 +103,13 @@ data AppConfig = AppConfig
   , configRawMediaTypes            :: [MediaType]
   , configServerHost               :: Text
   , configServerPort               :: Int
+  , configServerTraceHeader        :: Maybe (CI.CI BS.ByteString)
   , configServerUnixSocket         :: Maybe FilePath
   , configServerUnixSocketMode     :: FileMode
   , configAdminServerPort          :: Maybe Int
+  , configRoleSettings             :: RoleSettings
+  , configRoleIsoLvl               :: RoleIsolationLvl
+  , configInternalSCSleep          :: Maybe Int32
   }
 
 data LogLevel = LogCrit | LogError | LogWarn | LogInfo
@@ -124,7 +137,7 @@ toText conf =
   where
     -- apply conf to all pgrst settings
     pgrstSettings = (\(k, v) -> (k, v conf)) <$>
-      [("db-anon-role",              q . fromMaybe "" . configDbAnonRole)
+      [("db-anon-role",              q . T.decodeUtf8 . fromMaybe "" . configDbAnonRole)
       ,("db-channel",                q . configDbChannel)
       ,("db-channel-enabled",            T.toLower . show . configDbChannelEnabled)
       ,("db-extra-search-path",      q . T.intercalate "," . configDbExtraSearchPath)
@@ -133,11 +146,13 @@ toText conf =
       ,("db-pool",                       show . configDbPoolSize)
       ,("db-pool-acquisition-timeout",   show . configDbPoolAcquisitionTimeout)
       ,("db-pool-max-lifetime",          show . configDbPoolMaxLifetime)
+      ,("db-pool-max-idletime",          show . configDbPoolMaxIdletime)
       ,("db-pre-request",            q . maybe mempty dumpQi . configDbPreRequest)
       ,("db-prepared-statements",        T.toLower . show . configDbPreparedStatements)
       ,("db-root-spec",              q . maybe mempty dumpQi . configDbRootSpec)
       ,("db-schemas",                q . T.intercalate "," . toList . configDbSchemas)
       ,("db-config",                     T.toLower . show . configDbConfig)
+      ,("db-pre-config",             q . maybe mempty dumpQi . configDbPreConfig)
       ,("db-tx-end",                 q . showTxEnd)
       ,("db-uri",                    q . configDbUri)
       ,("db-use-legacy-gucs",            T.toLower . show . configDbUseLegacyGucs)
@@ -152,6 +167,7 @@ toText conf =
       ,("raw-media-types",           q . T.decodeUtf8 . BS.intercalate "," . fmap toMime . configRawMediaTypes)
       ,("server-host",               q . configServerHost)
       ,("server-port",                   show . configServerPort)
+      ,("server-trace-header",       q . T.decodeUtf8 . maybe mempty CI.original . configServerTraceHeader)
       ,("server-unix-socket",        q . maybe mempty T.pack . configServerUnixSocket)
       ,("server-unix-socket-mode",   q . T.pack . showSocketMode)
       ,("admin-server-port",             maybe "\"\"" show . configAdminServerPort)
@@ -188,13 +204,13 @@ instance JustIfMaybe a (Maybe a) where
 
 -- | Reads and parses the config and overrides its parameters from env vars,
 -- files or db settings.
-readAppConfig :: [(Text, Text)] -> Maybe FilePath -> Maybe Text -> IO (Either Text AppConfig)
-readAppConfig dbSettings optPath prevDbUri = do
+readAppConfig :: [(Text, Text)] -> Maybe FilePath -> Maybe Text -> RoleSettings -> RoleIsolationLvl -> IO (Either Text AppConfig)
+readAppConfig dbSettings optPath prevDbUri roleSettings roleIsolationLvl = do
   env <- readPGRSTEnvironment
   -- if no filename provided, start with an empty map to read config from environment
   conf <- maybe (return $ Right M.empty) loadConfig optPath
 
-  case C.runParser (parser optPath env dbSettings) =<< mapLeft show conf of
+  case C.runParser (parser optPath env dbSettings roleSettings roleIsolationLvl) =<< mapLeft show conf of
     Left err ->
       return . Left $ "Error in config " <> err
     Right parsedConfig ->
@@ -209,11 +225,11 @@ readAppConfig dbSettings optPath prevDbUri = do
       decodeJWKS <$>
         (decodeSecret =<< readSecretFile =<< readDbUriFile prevDbUri parsedConfig)
 
-parser :: Maybe FilePath -> Environment -> [(Text, Text)] -> C.Parser C.Config AppConfig
-parser optPath env dbSettings =
+parser :: Maybe FilePath -> Environment -> [(Text, Text)] -> RoleSettings -> RoleIsolationLvl -> C.Parser C.Config AppConfig
+parser optPath env dbSettings roleSettings roleIsolationLvl =
   AppConfig
     <$> parseAppSettings "app.settings"
-    <*> optString "db-anon-role"
+    <*> (fmap encodeUtf8 <$> optString "db-anon-role")
     <*> (fromMaybe "pgrst" <$> optString "db-channel")
     <*> (fromMaybe True <$> optBool "db-channel-enabled")
     <*> (maybe ["public"] splitOnCommas <$> optValue "db-extra-search-path")
@@ -223,6 +239,8 @@ parser optPath env dbSettings =
     <*> (fromMaybe 10 <$> optInt "db-pool")
     <*> (fromMaybe 10 <$> optInt "db-pool-acquisition-timeout")
     <*> (fromMaybe 1800 <$> optInt "db-pool-max-lifetime")
+    <*> (fromMaybe 30 <$> optWithAlias (optInt "db-pool-timeout")
+                                       (optInt "db-pool-max-idletime"))
     <*> (fmap toQi <$> optWithAlias (optString "db-pre-request")
                                     (optString "pre-request"))
     <*> (fromMaybe True <$> optBool "db-prepared-statements")
@@ -231,6 +249,7 @@ parser optPath env dbSettings =
     <*> (fromList . maybe ["public"] splitOnCommas <$> optWithAlias (optValue "db-schemas")
                                                                     (optValue "db-schema"))
     <*> (fromMaybe True <$> optBool "db-config")
+    <*> (fmap toQi <$> optString "db-pre-config")
     <*> parseTxEnd "db-tx-end" snd
     <*> parseTxEnd "db-tx-end" fst
     <*> (fromMaybe "postgresql://" <$> optString "db-uri")
@@ -250,9 +269,13 @@ parser optPath env dbSettings =
     <*> (maybe [] (fmap (MTOther . encodeUtf8) . splitOnCommas) <$> optValue "raw-media-types")
     <*> (fromMaybe "!4" <$> optString "server-host")
     <*> (fromMaybe 3000 <$> optInt "server-port")
+    <*> (fmap (CI.mk . encodeUtf8) <$> optString "server-trace-header")
     <*> (fmap T.unpack <$> optString "server-unix-socket")
     <*> parseSocketFileMode "server-unix-socket-mode"
     <*> optInt "admin-server-port"
+    <*> pure roleSettings
+    <*> pure roleIsolationLvl
+    <*> optInt "internal-schema-cache-sleep"
   where
     parseAppSettings :: C.Key -> C.Parser C.Config [(Text, Text)]
     parseAppSettings key = addFromEnv . fmap (fmap coerceText) <$> C.subassocs key C.value
@@ -347,21 +370,14 @@ parser optPath env dbSettings =
                                (C.Key -> C.Parser C.Value a -> C.Parser C.Config b) ->
                                C.Key -> (C.Value -> a) -> C.Parser C.Config b
     overrideFromDbOrEnvironment necessity key coercion =
-      case reloadableDbSetting <|> M.lookup envVarName env of
+      case dbConf <|> M.lookup envVarName env of
         Just dbOrEnvVal -> pure $ justIfMaybe $ coercion $ C.String dbOrEnvVal
-        Nothing  -> necessity key (coercion <$> C.value)
+        Nothing         -> necessity key (coercion <$> C.value)
       where
         dashToUnderscore '-' = '_'
         dashToUnderscore c   = c
         envVarName = "PGRST_" <> (toUpper . dashToUnderscore <$> toS key)
-        reloadableDbSetting =
-          let dbSettingName = T.pack $ dashToUnderscore <$> toS key in
-          if dbSettingName `notElem` [
-            "server_host", "server_port", "server_unix_socket", "server_unix_socket_mode", "admin_server_port", "log_level",
-            "db_uri", "db_channel_enabled", "db_channel", "db_pool", "db_pool_acquisition_timeout",
-            "db_pool_max_lifetime", "db_config"]
-          then lookup dbSettingName dbSettings
-          else Nothing
+        dbConf = lookup (T.pack $ dashToUnderscore <$> toS key) dbSettings
 
     coerceText :: C.Value -> Text
     coerceText (C.String s) = s
@@ -448,3 +464,41 @@ type Environment = M.Map [Char] Text
 readPGRSTEnvironment :: IO Environment
 readPGRSTEnvironment =
   M.map T.pack . M.fromList . filter (isPrefixOf "PGRST_" . fst) <$> getEnvironment
+
+-- | Adds a `fallback_application_name` value to the connection string. This allows querying the PostgREST version on pg_stat_activity.
+--
+-- >>> let ver = "11.1.0 (5a04ec7)"::ByteString
+-- >>> let strangeVer = "11'1&0@#$%,.:\"[]{}?+^()=asdfqwer"::ByteString
+--
+-- >>> addFallbackAppName ver "postgres://user:pass@host:5432/postgres"
+-- "postgres://user:pass@host:5432/postgres?fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29"
+--
+-- >>> addFallbackAppName ver "postgres://user:pass@host:5432/postgres?"
+-- "postgres://user:pass@host:5432/postgres?fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29"
+--
+-- >>> addFallbackAppName ver "postgres:///postgres?host=server&port=5432"
+-- "postgres:///postgres?host=server&port=5432&fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29"
+--
+-- >>> addFallbackAppName ver "postgresql://"
+-- "postgresql://?fallback_application_name=PostgREST%2011.1.0%20%285a04ec7%29"
+--
+-- >>> addFallbackAppName strangeVer "postgres:///postgres?host=server&port=5432"
+-- "postgres:///postgres?host=server&port=5432&fallback_application_name=PostgREST%2011%271%260%40%23%24%25%2C.%3A%22%5B%5D%7B%7D%3F%2B%5E%28%29%3Dasdfqwer"
+--
+-- >>> addFallbackAppName ver "postgres://user:invalid_chars[]#@host:5432/postgres"
+-- "postgres://user:invalid_chars[]#@host:5432/postgres"
+--
+-- >>> addFallbackAppName ver "invalid_uri1=val1 invalid_uri2=val2"
+-- "invalid_uri1=val1 invalid_uri2=val2"
+addFallbackAppName :: ByteString -> Text -> Text
+addFallbackAppName version dbUri = dbUri <>
+  case uriQuery <$> parseURI (toS dbUri) of
+    -- Does not add the application name to key=val connection strings or invalid URIs
+    Nothing  -> mempty
+    Just ""  -> "?" <> uriFmt
+    Just "?" -> uriFmt
+    _        -> "&" <> uriFmt
+  where
+    uriFmt = pKeyWord <> toS (escapeURIString isUnescapedInURIComponent $ toS pgrstVer)
+    pKeyWord = "fallback_application_name="
+    pgrstVer = "PostgREST " <> T.decodeUtf8 version

src/PostgREST/Config/Database.hs

@@ -3,11 +3,19 @@
 module PostgREST.Config.Database
   ( pgVersionStatement
   , queryDbSettings
+  , queryRoleSettings
   , queryPgVersion
+  , RoleSettings
+  , RoleIsolationLvl
+  , toIsolationLevel
   ) where
 
+import Control.Arrow ((***))
+
 import PostgREST.Config.PgVersion (PgVersion (..))
 
+import qualified Data.HashMap.Strict as HM
+
 import qualified Hasql.Decoders             as HD
 import qualified Hasql.Encoders             as HE
 import           Hasql.Session              (Session, statement)
@@ -15,51 +23,174 @@ import qualified Hasql.Statement            as SQL
 import qualified Hasql.Transaction          as SQL
 import qualified Hasql.Transaction.Sessions as SQL
 
-import Text.InterpolatedString.Perl6 (q)
+import Text.InterpolatedString.Perl6 (q, qc)
 
 import Protolude
 
-queryPgVersion :: Session PgVersion
-queryPgVersion = statement mempty pgVersionStatement
+type RoleSettings     = (HM.HashMap ByteString (HM.HashMap ByteString ByteString))
+type RoleIsolationLvl = HM.HashMap ByteString SQL.IsolationLevel
 
-pgVersionStatement :: SQL.Statement () PgVersion
-pgVersionStatement = SQL.Statement sql HE.noParams versionRow False
+toIsolationLevel :: (Eq a, IsString a) => a -> SQL.IsolationLevel
+toIsolationLevel a = case a of
+  "repeatable read" -> SQL.RepeatableRead
+  "serializable"    -> SQL.Serializable
+  _                 -> SQL.ReadCommitted
+
+prefix :: Text
+prefix = "pgrst."
+
+-- | In-db settings names
+dbSettingsNames :: [Text]
+dbSettingsNames =
+  (prefix <>) <$>
+  ["db_anon_role"
+  ,"db_pre_config"
+  ,"db_extra_search_path"
+  ,"db_max_rows"
+  ,"db_plan_enabled"
+  ,"db_pre_request"
+  ,"db_prepared_statements"
+  ,"db_root_spec"
+  ,"db_schemas"
+  ,"db_tx_end"
+  ,"db_use_legacy_gucs"
+  ,"jwt_aud"
+  ,"jwt_role_claim_key"
+  ,"jwt_secret"
+  ,"jwt_secret_is_base64"
+  ,"openapi_mode"
+  ,"openapi_security_active"
+  ,"openapi_server_proxy_uri"
+  ,"raw_media_types"
+  ,"server_trace_header"
+  ]
+
+queryPgVersion :: Bool -> Session PgVersion
+queryPgVersion prepared = statement mempty $ pgVersionStatement prepared
+
+pgVersionStatement :: Bool -> SQL.Statement () PgVersion
+pgVersionStatement = SQL.Statement sql HE.noParams versionRow
   where
     sql = "SELECT current_setting('server_version_num')::integer, current_setting('server_version')"
     versionRow = HD.singleRow $ PgVersion <$> column HD.int4 <*> column HD.text
 
-queryDbSettings :: Bool -> Session [(Text, Text)]
-queryDbSettings prepared =
+-- | Query the in-database configuration. The settings have the following priorities:
+--
+-- 1. Role + with database-specific settings:
+--    ALTER ROLE authenticator IN DATABASE postgres SET <prefix>jwt_aud = 'val';
+-- 2. Role + with settings:
+--    ALTER ROLE authenticator SET <prefix>jwt_aud = 'overridden';
+-- 3. pre-config function:
+--    CREATE FUNCTION pre_config() .. PERFORM set_config(<prefix>jwt_aud, 'pre_config_aud'..)
+--
+-- The example above will result in <prefix>jwt_aud = 'val'
+-- A setting on the database only will have no effect: ALTER DATABASE postgres SET <prefix>jwt_aud = 'xx'
+queryDbSettings :: Maybe Text -> Bool -> Session [(Text, Text)]
+queryDbSettings preConfFunc prepared =
   let transaction = if prepared then SQL.transaction else SQL.unpreparedTransaction in
-  transaction SQL.ReadCommitted SQL.Read $ SQL.statement mempty dbSettingsStatement
-
--- | Get db settings from the connection role. Global settings will be overridden by database specific settings.
-dbSettingsStatement :: SQL.Statement () [(Text, Text)]
-dbSettingsStatement = SQL.Statement sql HE.noParams decodeSettings False
+  transaction SQL.ReadCommitted SQL.Read $ SQL.statement dbSettingsNames $ SQL.Statement sql (arrayParam HE.text) decodeSettings prepared
   where
-    sql = [q|
+    sql = [qc|
       WITH
-      role_setting (database, setting) AS (
-        SELECT setdatabase,
-               unnest(setconfig)
-          FROM pg_catalog.pg_db_role_setting
-         WHERE setrole = CURRENT_USER::regrole::oid
-           AND setdatabase IN (0, (SELECT oid FROM pg_catalog.pg_database WHERE datname = CURRENT_CATALOG))
+      role_setting AS (
+        SELECT setdatabase as database,
+               unnest(setconfig) as setting
+        FROM pg_catalog.pg_db_role_setting
+        WHERE setrole = CURRENT_USER::regrole::oid
+          AND setdatabase IN (0, (SELECT oid FROM pg_catalog.pg_database WHERE datname = CURRENT_CATALOG))
       ),
-      kv_settings (database, k, v) AS (
+      kv_settings AS (
         SELECT database,
-               substr(setting, 1, strpos(setting, '=') - 1),
-               substr(setting, strpos(setting, '=') + 1)
-          FROM role_setting
-         WHERE setting LIKE 'pgrst.%'
+               substr(setting, 1, strpos(setting, '=') - 1) as k,
+               substr(setting, strpos(setting, '=') + 1) as v
+        FROM role_setting
+        {preConfigF}
       )
       SELECT DISTINCT ON (key)
-             replace(k, 'pgrst.', '') AS key,
+             replace(k, '{prefix}', '') AS key,
              v AS value
-        FROM kv_settings
-       ORDER BY key, database DESC;
+      FROM kv_settings
+      WHERE k = ANY($1) AND v IS NOT NULL
+      ORDER BY key, database DESC NULLS LAST;
     |]
+    preConfigF = case preConfFunc of
+      Nothing   -> mempty
+      Just func -> [qc|
+          UNION
+          SELECT
+            null as database,
+            x as k,
+            current_setting(x, true) as v
+          FROM unnest($1) x
+          JOIN {func}() _ ON TRUE
+      |]::Text
     decodeSettings = HD.rowList $ (,) <$> column HD.text <*> column HD.text
 
+queryRoleSettings :: Bool -> Session (RoleSettings, RoleIsolationLvl)
+queryRoleSettings prepared =
+  let transaction = if prepared then SQL.transaction else SQL.unpreparedTransaction in
+  transaction SQL.ReadCommitted SQL.Read $ SQL.statement mempty $ SQL.Statement sql HE.noParams (processRows <$> rows) prepared
+  where
+    sql = [q|
+      with
+      role_setting as (
+        select r.rolname, unnest(r.rolconfig) as setting
+        from pg_auth_members m
+        join pg_roles r on r.oid = m.roleid
+        where member = current_user::regrole::oid
+      ),
+      kv_settings AS (
+        SELECT
+          rolname,
+          substr(setting, 1, strpos(setting, '=') - 1) as key,
+          lower(substr(setting, strpos(setting, '=') + 1)) as value
+        FROM role_setting
+      ),
+      iso_setting AS (
+        SELECT rolname, value
+        FROM kv_settings
+        WHERE key = 'default_transaction_isolation'
+      )
+      select
+        kv.rolname,
+        i.value as iso_lvl,
+        coalesce(array_agg(row(kv.key, kv.value)) filter (where key <> 'default_transaction_isolation'), '{}') as role_settings
+      from kv_settings kv
+      join pg_settings ps on ps.name = kv.key and ps.context = 'user'
+      left join iso_setting i on i.rolname = kv.rolname
+      group by kv.rolname, i.value;
+    |]
+
+    processRows :: [(Text, Maybe Text, [(Text, Text)])] -> (RoleSettings, RoleIsolationLvl)
+    processRows rs =
+      let
+        rowsWRoleSettings = [ (x, z) | (x, _, z) <- rs ]
+        rowsWIsolation    = [ (x, y) | (x, Just y, _) <- rs ]
+      in
+      ( HM.fromList $ bimap encodeUtf8 (HM.fromList . ((encodeUtf8 *** encodeUtf8) <$>)) <$> rowsWRoleSettings
+      , HM.fromList $ (encodeUtf8 *** toIsolationLevel) <$> rowsWIsolation
+      )
+
+    rows :: HD.Result [(Text, Maybe Text, [(Text, Text)])]
+    rows = HD.rowList $ (,,) <$> column HD.text <*> nullableColumn HD.text <*> compositeArrayColumn ((,) <$> compositeField HD.text <*> compositeField HD.text)
+
 column :: HD.Value a -> HD.Row a
 column = HD.column . HD.nonNullable
+
+nullableColumn :: HD.Value a -> HD.Row (Maybe a)
+nullableColumn = HD.column . HD.nullable
+
+compositeField :: HD.Value a -> HD.Composite a
+compositeField = HD.field . HD.nonNullable
+
+compositeArrayColumn :: HD.Composite a -> HD.Row [a]
+compositeArrayColumn = arrayColumn . HD.composite
+
+arrayColumn :: HD.Value a -> HD.Row [a]
+arrayColumn = column . HD.listArray . HD.nonNullable
+
+param :: HE.Value a -> HE.Params a
+param = HE.param . HE.nonNullable
+
+arrayParam :: HE.Value a -> HE.Params [a]
+arrayParam = param . HE.foldableArray . HE.nonNullable

src/PostgREST/Error.hs

@@ -11,7 +11,6 @@ module PostgREST.Error
   , PgError(..)
   , Error(..)
   , errorPayload
-  , checkIsFatal
   , singularityError
   ) where
 
@@ -39,12 +38,12 @@ import qualified PostgREST.MediaType        as MediaType
 
 import PostgREST.SchemaCache.Identifiers  (QualifiedIdentifier (..),
                                            Schema)
-import PostgREST.SchemaCache.Proc         (ProcDescription (..),
-                                           ProcParam (..))
 import PostgREST.SchemaCache.Relationship (Cardinality (..),
                                            Junction (..),
                                            Relationship (..),
                                            RelationshipsMap)
+import PostgREST.SchemaCache.Routine      (Routine (..),
+                                           RoutineParam (..))
 import Protolude
 
 
@@ -68,15 +67,19 @@ instance PgrstError ApiRequestError where
   status InvalidRpcMethod{}      = HTTP.status405
   status InvalidRange{}          = HTTP.status416
   status NotFound                = HTTP.status404
+
   status NoRelBetween{}          = HTTP.status400
   status NoRpc{}                 = HTTP.status404
   status NotEmbedded{}           = HTTP.status400
-  status ParseRequestError{}     = HTTP.status400
-  status PutRangeNotAllowedError = HTTP.status400
+  status PutLimitNotAllowedError = HTTP.status400
   status QueryParamError{}       = HTTP.status400
+  status RelatedOrderNotToOne{}  = HTTP.status400
+  status SpreadNotToOne{}        = HTTP.status400
+  status UnacceptableFilter{}    = HTTP.status400
   status UnacceptableSchema{}    = HTTP.status406
   status UnsupportedMethod{}     = HTTP.status405
   status LimitNoOrderError       = HTTP.status400
+  status ColumnNotFound{}        = HTTP.status400
 
   headers _ = [MediaType.toContentType MTApplicationJSON]
 
@@ -104,11 +107,6 @@ instance JSON.ToJSON ApiRequestError where
                    LowerGTUpper            -> "The lower boundary must be lower than or equal to the upper boundary in the Range header."
                    OutOfBounds lower total -> "An offset of " <> lower <> " was requested, but there are only " <> total <> " rows."),
     "hint"    .= JSON.Null]
-  toJSON (ParseRequestError message details) = JSON.object [
-    "code"    .= ApiRequestErrorCode04,
-    "message" .= message,
-    "details" .= details,
-    "hint"    .= JSON.Null]
   toJSON InvalidFilters = JSON.object [
     "code"    .= ApiRequestErrorCode05,
     "message" .= ("Filters must include all and only primary key columns with 'eq' operators" :: Text),
@@ -127,7 +125,7 @@ instance JSON.ToJSON ApiRequestError where
   toJSON NotFound = JSON.object []
   toJSON (NotEmbedded resource) = JSON.object [
     "code"    .= ApiRequestErrorCode08,
-    "message" .= ("Cannot apply filter because '" <> resource <> "' is not an embedded resource in this request" :: Text),
+    "message" .= ("'" <> resource <> "' is not an embedded resource in this request" :: Text),
     "details" .= JSON.Null,
     "hint"    .= ("Verify that '" <> resource <> "' is included in the 'select' query parameter." :: Text)]
 
@@ -143,9 +141,9 @@ instance JSON.ToJSON ApiRequestError where
     "details" .= JSON.Null,
     "hint"    .= JSON.Null]
 
-  toJSON PutRangeNotAllowedError = JSON.object [
+  toJSON PutLimitNotAllowedError = JSON.object [
     "code"    .= ApiRequestErrorCode14,
-    "message" .= ("Range header and limit/offset querystring parameters are not allowed for PUT" :: Text),
+    "message" .= ("limit/offset querystring parameters are not allowed for PUT" :: Text),
     "details" .= JSON.Null,
     "hint"    .= JSON.Null]
 
@@ -155,6 +153,24 @@ instance JSON.ToJSON ApiRequestError where
     "details" .= JSON.Null,
     "hint"    .= JSON.Null]
 
+  toJSON (RelatedOrderNotToOne origin target) = JSON.object [
+    "code"    .= ApiRequestErrorCode18,
+    "message" .= ("A related order on '" <> target <> "' is not possible" :: Text),
+    "details" .= ("'" <> origin <> "' and '" <> target <> "' do not form a many-to-one or one-to-one relationship" :: Text),
+    "hint"    .= JSON.Null]
+
+  toJSON (SpreadNotToOne origin target) = JSON.object [
+    "code"    .= ApiRequestErrorCode19,
+    "message" .= ("A spread operation on '" <> target <> "' is not possible" :: Text),
+    "details" .= ("'" <> origin <> "' and '" <> target <> "' do not form a many-to-one or one-to-one relationship" :: Text),
+    "hint"    .= JSON.Null]
+
+  toJSON (UnacceptableFilter target) = JSON.object [
+    "code"    .= ApiRequestErrorCode20,
+    "message" .= ("Bad operator on the '" <> target <> "' embedded resource":: Text),
+    "details" .= ("Only is null or not is null filters are allowed on embedded resources":: Text),
+    "hint"    .= JSON.Null]
+
   toJSON (NoRelBetween parent child embedHint schema allRels) = JSON.object [
     "code"    .= SchemaCacheErrorCode00,
     "message" .= ("Could not find a relationship between '" <> parent <> "' and '" <> child <> "' in the schema cache" :: Text),
@@ -194,6 +210,11 @@ instance JSON.ToJSON ApiRequestError where
     "message" .= ("Could not choose the best candidate function between: " <> T.intercalate ", " [pdSchema p <> "." <> pdName p <> "(" <> T.intercalate ", " [ppName a <> " => " <> ppType a | a <- pdParams p] <> ")" | p <- procs]),
     "details" .= JSON.Null,
     "hint"    .= ("Try renaming the parameters or the function itself in the database so function overloading can be resolved" :: Text)]
+  toJSON (ColumnNotFound relName colName) = JSON.object [
+    "code"    .= SchemaCacheErrorCode04,
+    "message" .= ("Column '" <> colName <> "' of relation '" <> relName <> "' does not exist" :: Text),
+    "details" .= JSON.Null,
+    "hint"    .= JSON.Null]
 
 -- |
 -- If no relationship is found then:
@@ -261,7 +282,7 @@ noRelBetweenHint parent child schema allRels = ("Perhaps you meant '" <>) <$>
 -- to all the overloaded functions' params using the form "param1, param2, param3, ..."
 -- and shows the best match as hint.
 --
--- >>> let procsDesc = [ProcDescription {pdParams = [ProcParam {ppName="val"}, ProcParam {ppName="param"}, ProcParam {ppName="name"}]}, ProcDescription {pdParams = [ProcParam {ppName="id"}, ProcParam {ppName="attr"}]}]
+-- >>> let procsDesc = [Function {pdParams = [RoutineParam {ppName="val"}, RoutineParam {ppName="param"}, RoutineParam {ppName="name"}]}, Function {pdParams = [RoutineParam {ppName="id"}, RoutineParam {ppName="attr"}]}]
 --
 -- >>> noRpcHint "api" "test" ["vall", "pqaram", "nam"] procs procsDesc
 -- Just "Perhaps you meant to call the function api.test(name, param, val)"
@@ -278,7 +299,7 @@ noRelBetweenHint parent child schema allRels = ("Perhaps you meant '" <>) <$>
 -- >>> noRpcHint "api" "test" ["noclosealternative"] procs procsDesc
 -- Nothing
 --
-noRpcHint :: Text -> Text -> [Text] -> [QualifiedIdentifier] -> [ProcDescription] -> Maybe Text
+noRpcHint :: Text -> Text -> [Text] -> [QualifiedIdentifier] -> [Routine] -> Maybe Text
 noRpcHint schema procName params allProcs overloadedProcs =
   fmap (("Perhaps you meant to call the function " <> schema <> ".") <>) possibleProcs
   where
@@ -354,7 +375,7 @@ instance JSON.ToJSON SQL.UsageError where
     "hint"    .= JSON.Null]
   toJSON (SQL.SessionUsageError e) = JSON.toJSON e -- SQL.Error
   toJSON SQL.AcquisitionTimeoutUsageError = JSON.object [
-    "code"    .= ConnectionErrorCode00,
+    "code"    .= ConnectionErrorCode03,
     "message" .= ("Timed out acquiring connection from connection pool." :: Text),
     "details" .= JSON.Null,
     "hint"    .= JSON.Null]
@@ -407,6 +428,7 @@ pgErrorStatus authed (SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ResultError
         '5':'3':_ -> HTTP.status503 -- insufficient resources
         '5':'4':_ -> HTTP.status413 -- too complex
         '5':'5':_ -> HTTP.status500 -- obj not on prereq state
+        '5':'7':'P':'0':'1':_ -> HTTP.status503 -- terminating connection due to administrator command
         '5':'7':_ -> HTTP.status500 -- operator intervention
         '5':'8':_ -> HTTP.status500 -- system error
         'F':'0':_ -> HTTP.status500 -- conf file error
@@ -424,29 +446,6 @@ pgErrorStatus authed (SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ResultError
 
     _                       -> HTTP.status500
 
-checkIsFatal :: SQL.UsageError -> Maybe Text
-checkIsFatal (SQL.ConnectionUsageError e)
-  | isAuthFailureMessage = Just $ toS failureMessage
-  | otherwise = Nothing
-  where isAuthFailureMessage = "FATAL:  password authentication failed" `isInfixOf` failureMessage
-        failureMessage = BS.unpack $ fromMaybe mempty e
-checkIsFatal(SQL.SessionUsageError (SQL.QueryError _ _ (SQL.ResultError serverError)))
-  = case serverError of
-      -- Check for a syntax error (42601 is the pg code). This would mean the error is on our part somehow, so we treat it as fatal.
-      SQL.ServerError "42601" _ _ _ _
-        -> Just "Hint: This is probably a bug in PostgREST, please report it at https://github.com/PostgREST/postgrest/issues"
-      -- Check for a "prepared statement <name> already exists" error (Code 42P05: duplicate_prepared_statement).
-      -- This would mean that a connection pooler in transaction mode is being used
-      -- while prepared statements are enabled in the PostgREST configuration,
-      -- both of which are incompatible with each other.
-      SQL.ServerError "42P05" _ _ _ _
-        -> Just "Hint: If you are using connection poolers in transaction mode, try setting db-prepared-statements to false."
-      -- Check for a "transaction blocks not allowed in statement pooling mode" error (Code 08P01: protocol_violation).
-      -- This would mean that a connection pooler in statement mode is being used which is not supported in PostgREST.
-      SQL.ServerError "08P01" "transaction blocks not allowed in statement pooling mode" _ _ _
-        -> Just "Hint: Connection poolers in statement mode are not supported."
-      _ -> Nothing
-checkIsFatal _ = Nothing
 
 
 data Error
@@ -479,7 +478,7 @@ instance PgrstError Error where
   headers (JwtTokenInvalid m)    = [MediaType.toContentType MTApplicationJSON, invalidTokenHeader m]
   headers JwtTokenRequired       = [MediaType.toContentType MTApplicationJSON, requiredTokenHeader]
   headers (PgErr err)            = headers err
-  headers SingularityError{}     = [MediaType.toContentType MTSingularJSON]
+  headers SingularityError{}     = [MediaType.toContentType (MTSingularJSON False)]
   headers _                      = [MediaType.toContentType MTApplicationJSON]
 
 instance JSON.ToJSON Error where
@@ -531,7 +530,7 @@ instance JSON.ToJSON Error where
   toJSON (SingularityError n) = JSON.object [
     "code"    .= ApiRequestErrorCode16,
     "message" .= ("JSON object requested, multiple (or no) rows returned" :: Text),
-    "details" .= T.unwords ["Results contain", show n, "rows,", T.decodeUtf8 (MediaType.toMime MTSingularJSON), "requires 1 row"],
+    "details" .= T.unwords ["The result contains", show n, "rows"],
     "hint"    .= JSON.Null]
 
   toJSON (PgErr err) = JSON.toJSON err
@@ -553,12 +552,13 @@ data ErrorCode
   = ConnectionErrorCode00
   | ConnectionErrorCode01
   | ConnectionErrorCode02
+  | ConnectionErrorCode03
   -- API Request errors
   | ApiRequestErrorCode00
   | ApiRequestErrorCode01
   | ApiRequestErrorCode02
   | ApiRequestErrorCode03
-  | ApiRequestErrorCode04
+  | ApiRequestErrorCode04 -- no longer used (used to be mapped to ParseRequestError)
   | ApiRequestErrorCode05
   | ApiRequestErrorCode06
   | ApiRequestErrorCode07
@@ -572,11 +572,15 @@ data ErrorCode
   | ApiRequestErrorCode15
   | ApiRequestErrorCode16
   | ApiRequestErrorCode17
+  | ApiRequestErrorCode18
+  | ApiRequestErrorCode19
+  | ApiRequestErrorCode20
   -- Schema Cache errors
   | SchemaCacheErrorCode00
   | SchemaCacheErrorCode01
   | SchemaCacheErrorCode02
   | SchemaCacheErrorCode03
+  | SchemaCacheErrorCode04
   -- JWT authentication errors
   | JWTErrorCode00
   | JWTErrorCode01
@@ -594,6 +598,7 @@ buildErrorCode code = "PGRST" <> case code of
   ConnectionErrorCode00  -> "000"
   ConnectionErrorCode01  -> "001"
   ConnectionErrorCode02  -> "002"
+  ConnectionErrorCode03  -> "003"
 
   ApiRequestErrorCode00  -> "100"
   ApiRequestErrorCode01  -> "101"
@@ -613,11 +618,15 @@ buildErrorCode code = "PGRST" <> case code of
   ApiRequestErrorCode15  -> "115"
   ApiRequestErrorCode16  -> "116"
   ApiRequestErrorCode17  -> "117"
+  ApiRequestErrorCode18  -> "118"
+  ApiRequestErrorCode19  -> "119"
+  ApiRequestErrorCode20  -> "120"
 
   SchemaCacheErrorCode00 -> "200"
   SchemaCacheErrorCode01 -> "201"
   SchemaCacheErrorCode02 -> "202"
   SchemaCacheErrorCode03 -> "203"
+  SchemaCacheErrorCode04 -> "204"
 
   JWTErrorCode00         -> "300"
   JWTErrorCode01         -> "301"

src/PostgREST/Logger.hs

@@ -26,5 +26,5 @@ middleware logLevel = case logLevel of
       { Wai.outputFormat = Wai.ApacheWithSettings $
           Wai.defaultApacheSettings
             & Wai.setApacheRequestFilter (\_ res -> filterStatus $ Wai.responseStatus res)
-            & Wai.setApacheUserGetter (fmap encodeUtf8 . Auth.getRole)
+            & Wai.setApacheUserGetter Auth.getRole
       }

src/PostgREST/MediaType.hs

@@ -4,16 +4,13 @@ module PostgREST.MediaType
   ( MediaType(..)
   , MTPlanOption (..)
   , MTPlanFormat (..)
-  , MTPlanAttrs(..)
   , toContentType
   , toMime
   , decodeMediaType
-  , getMediaType
   ) where
 
 import qualified Data.ByteString          as BS
 import qualified Data.ByteString.Internal as BS (c2w)
-import           Data.Maybe               (fromJust)
 
 import Network.HTTP.Types.Header (Header, hContentType)
 
@@ -22,7 +19,8 @@ import Protolude
 -- | Enumeration of currently supported media types
 data MediaType
   = MTApplicationJSON
-  | MTSingularJSON
+  | MTArrayJSONStrip
+  | MTSingularJSON Bool
   | MTGeoJSON
   | MTTextCSV
   | MTTextPlain
@@ -32,18 +30,32 @@ data MediaType
   | MTOctetStream
   | MTAny
   | MTOther ByteString
-  | MTPlan MTPlanAttrs
-  deriving Eq
-
-data MTPlanAttrs = MTPlanAttrs (Maybe MediaType) MTPlanFormat [MTPlanOption]
-instance Eq MTPlanAttrs where
-  MTPlanAttrs {} == MTPlanAttrs {} = True -- we don't care about the attributes when comparing two MTPlan media types
+  -- TODO MTPlan should only have its options as [Text]. Its ResultAggregate should have the typed attributes.
+  | MTPlan MediaType MTPlanFormat [MTPlanOption]
+  deriving Show
+instance Eq MediaType where
+  MTApplicationJSON    == MTApplicationJSON = True
+  MTArrayJSONStrip     == MTArrayJSONStrip  = True
+  MTSingularJSON x     == MTSingularJSON y  = x == y
+  MTGeoJSON            == MTGeoJSON         = True
+  MTTextCSV            == MTTextCSV         = True
+  MTTextPlain          == MTTextPlain       = True
+  MTTextXML            == MTTextXML         = True
+  MTOpenAPI            == MTOpenAPI         = True
+  MTUrlEncoded         == MTUrlEncoded      = True
+  MTOctetStream        == MTOctetStream     = True
+  MTAny                == MTAny             = True
+  MTOther x            == MTOther y         = x == y
+  MTPlan{}             == MTPlan{}          = True
+  _                    == _                 = False
 
 data MTPlanOption
   = PlanAnalyze | PlanVerbose | PlanSettings | PlanBuffers | PlanWAL
+  deriving (Eq, Show)
 
 data MTPlanFormat
   = PlanJSON | PlanText
+  deriving (Eq, Show)
 
 -- | Convert MediaType to a Content-Type HTTP Header
 toContentType :: MediaType -> Header
@@ -56,20 +68,22 @@ toContentType ct = (hContentType, toMime ct <> charset)
 
 -- | Convert from MediaType to a ByteString representing the mime type
 toMime :: MediaType -> ByteString
-toMime MTApplicationJSON = "application/json"
-toMime MTGeoJSON         = "application/geo+json"
-toMime MTTextCSV         = "text/csv"
-toMime MTTextPlain       = "text/plain"
-toMime MTTextXML         = "text/xml"
-toMime MTOpenAPI         = "application/openapi+json"
-toMime MTSingularJSON    = "application/vnd.pgrst.object+json"
-toMime MTUrlEncoded      = "application/x-www-form-urlencoded"
-toMime MTOctetStream     = "application/octet-stream"
-toMime MTAny             = "*/*"
-toMime (MTOther ct)      = ct
-toMime (MTPlan (MTPlanAttrs mt fmt opts)) =
+toMime MTApplicationJSON      = "application/json"
+toMime MTArrayJSONStrip       = "application/vnd.pgrst.array+json;nulls=stripped"
+toMime MTGeoJSON              = "application/geo+json"
+toMime MTTextCSV              = "text/csv"
+toMime MTTextPlain            = "text/plain"
+toMime MTTextXML              = "text/xml"
+toMime MTOpenAPI              = "application/openapi+json"
+toMime (MTSingularJSON True)  = "application/vnd.pgrst.object+json;nulls=stripped"
+toMime (MTSingularJSON False) = "application/vnd.pgrst.object+json"
+toMime MTUrlEncoded           = "application/x-www-form-urlencoded"
+toMime MTOctetStream          = "application/octet-stream"
+toMime MTAny                  = "*/*"
+toMime (MTOther ct)           = ct
+toMime (MTPlan mt fmt opts)   =
   "application/vnd.pgrst.plan+" <> toMimePlanFormat fmt <>
-  (if isNothing mt then mempty else "; for=\"" <> toMime (fromJust mt) <> "\"") <>
+  ("; for=\"" <> toMime mt <> "\"") <>
   (if null opts then mempty else "; options=" <> BS.intercalate "|" (toMimePlanOption <$> opts))
 
 toMimePlanOption :: MTPlanOption -> ByteString
@@ -83,42 +97,73 @@ toMimePlanFormat :: MTPlanFormat -> ByteString
 toMimePlanFormat PlanJSON = "json"
 toMimePlanFormat PlanText = "text"
 
--- | Convert from ByteString to MediaType. Warning: discards MIME parameters
+-- | Convert from ByteString to MediaType.
+--
+-- >>> decodeMediaType "application/json"
+-- MTApplicationJSON
+--
+-- >>> decodeMediaType "application/vnd.pgrst.plan;"
+-- MTPlan MTApplicationJSON PlanText []
+--
+-- >>> decodeMediaType "application/vnd.pgrst.plan;for=\"application/json\""
+-- MTPlan MTApplicationJSON PlanText []
+--
+-- >>> decodeMediaType "application/vnd.pgrst.plan+json;for=\"text/csv\""
+-- MTPlan MTTextCSV PlanJSON []
+--
+-- >>> decodeMediaType "application/vnd.pgrst.array+json;nulls=stripped"
+-- MTArrayJSONStrip
+--
+-- >>> decodeMediaType "application/vnd.pgrst.array+json"
+-- MTApplicationJSON
+--
+-- >>> decodeMediaType "application/vnd.pgrst.object+json;nulls=stripped"
+-- MTSingularJSON True
+--
+-- >>> decodeMediaType "application/vnd.pgrst.object+json"
+-- MTSingularJSON False
+
 decodeMediaType :: BS.ByteString -> MediaType
 decodeMediaType mt =
   case BS.split (BS.c2w ';') mt of
-    "application/json":_                   -> MTApplicationJSON
-    "application/geo+json":_               -> MTGeoJSON
-    "text/csv":_                           -> MTTextCSV
-    "text/plain":_                         -> MTTextPlain
-    "text/xml":_                           -> MTTextXML
-    "application/openapi+json":_           -> MTOpenAPI
-    "application/vnd.pgrst.object+json":_  -> MTSingularJSON
-    "application/vnd.pgrst.object":_       -> MTSingularJSON
-    "application/x-www-form-urlencoded":_  -> MTUrlEncoded
-    "application/octet-stream":_           -> MTOctetStream
-    "application/vnd.pgrst.plan":rest      -> getPlan PlanText rest
-    "application/vnd.pgrst.plan+text":rest -> getPlan PlanText rest
-    "application/vnd.pgrst.plan+json":rest -> getPlan PlanJSON rest
-    "*/*":_                                -> MTAny
-    other:_                                -> MTOther other
-    _                                      -> MTAny
+    "application/json":_                     -> MTApplicationJSON
+    "application/geo+json":_                 -> MTGeoJSON
+    "text/csv":_                             -> MTTextCSV
+    "text/plain":_                           -> MTTextPlain
+    "text/xml":_                             -> MTTextXML
+    "application/openapi+json":_             -> MTOpenAPI
+    "application/x-www-form-urlencoded":_    -> MTUrlEncoded
+    "application/octet-stream":_             -> MTOctetStream
+    "application/vnd.pgrst.plan":rest        -> getPlan PlanText rest
+    "application/vnd.pgrst.plan+text":rest   -> getPlan PlanText rest
+    "application/vnd.pgrst.plan+json":rest   -> getPlan PlanJSON rest
+    "application/vnd.pgrst.object+json":rest -> checkSingularNullStrip rest
+    "application/vnd.pgrst.object":rest      -> checkSingularNullStrip rest
+    "application/vnd.pgrst.array+json":rest  -> checkArrayNullStrip rest
+    "application/vnd.pgrst.array":rest       -> checkArrayNullStrip rest
+    "*/*":_                                  -> MTAny
+    other:_                                  -> MTOther other
+    _                                        -> MTAny
   where
-    getPlan fmt rest =
-     let
-       opts         = BS.split (BS.c2w '|') $ fromMaybe mempty (BS.stripPrefix "options=" =<< find (BS.isPrefixOf "options=") rest)
-       inOpts str   = str `elem` opts
-       mtFor        = decodeMediaType . dropAround (== BS.c2w '"') <$> (BS.stripPrefix "for=" =<< find (BS.isPrefixOf "for=") rest)
-       dropAround p = BS.dropWhile p . BS.dropWhileEnd p in
-     MTPlan $ MTPlanAttrs mtFor fmt $
-      [PlanAnalyze  | inOpts "analyze" ] ++
-      [PlanVerbose  | inOpts "verbose" ] ++
-      [PlanSettings | inOpts "settings"] ++
-      [PlanBuffers  | inOpts "buffers" ] ++
-      [PlanWAL      | inOpts "wal"     ]
+    checkArrayNullStrip ["nulls=stripped"] = MTArrayJSONStrip
+    checkArrayNullStrip  _                 = MTApplicationJSON
 
-getMediaType :: MediaType -> MediaType
-getMediaType mt = case mt of
-  MTPlan (MTPlanAttrs (Just mType) _ _) -> mType
-  MTPlan (MTPlanAttrs Nothing _ _)      -> MTApplicationJSON
-  other                                 -> other
+    checkSingularNullStrip ["nulls=stripped"] = MTSingularJSON True
+    checkSingularNullStrip  _                 = MTSingularJSON False
+
+    getPlan fmt rest =
+      let
+        opts         = BS.split (BS.c2w '|') $ fromMaybe mempty (BS.stripPrefix "options=" =<< find (BS.isPrefixOf "options=") rest)
+        inOpts str   = str `elem` opts
+        dropAround p = BS.dropWhile p . BS.dropWhileEnd p
+        mtFor        = fromMaybe MTApplicationJSON $ do
+          foundFor    <- find (BS.isPrefixOf "for=") rest
+          strippedFor <- BS.stripPrefix "for=" foundFor
+          pure . decodeMediaType $ dropAround (== BS.c2w '"') strippedFor
+      in
+      MTPlan mtFor fmt $
+        [PlanAnalyze  | inOpts "analyze" ] ++
+        [PlanVerbose  | inOpts "verbose" ] ++
+        [PlanSettings | inOpts "settings"] ++
+        [PlanBuffers  | inOpts "buffers" ] ++
+        [PlanWAL      | inOpts "wal"     ]