jimmy/postgrest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

remove manual inclusion of pages on index

Browse Source 
* drop unavailable doc pages from release notes
* rename releases pages so they order in TOC
This commit is contained in:
steve-chavez
2023-05-08 08:58:09 -03:00
committed by Steve Chavez
parent 36300da16d
commit 79620396ed
23 changed files with 112 additions and 656 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/references/api.rst
+2863
View File
File diff suppressed because it is too large Load Diff
docs/references/auth.rst
+188
View File
@@ -0,0 +1,188 @@
Authentication
==============
PostgREST is designed to keep the database at the center of API security. All :ref:`authorization happens in the database <db_authz>` . It is PostgREST's job to **authenticate** requests -- i.e. verify that a client is who they say they are -- and then let the database **authorize** client actions.
.. _roles:
Overview of role system
-----------------------
There are three types of roles used by PostgREST, the **authenticator**, **anonymous** and **user** roles. The database administrator creates these roles and configures PostgREST to use them.
.. image:: ../_static/security-roles.png
The authenticator role is used for connecting to the database and should be configured to have very limited access. It is a chameleon whose job is to "become" other users to service authenticated HTTP requests.
.. code:: sql
CREATE ROLE authenticator LOGIN NOINHERIT NOCREATEDB NOCREATEROLE NOSUPERUSER;
.. note::
The names "authenticator" and "anon" names are configurable and not sacred, we simply choose them for clarity. See :ref:`db-uri` and :ref:`db-anon-role`.
.. _user_impersonation:
User Impersonation
------------------
The picture below shows how the server handles authentication. If auth succeeds, it switches into the user role specified by the request, otherwise it switches into the anonymous role (if it's set in :ref:`db-anon-role`).
.. image:: ../_static/security-anon-choice.png
This role switching mechanism is called **user impersonation**. In PostgreSQL it's done with the ``SET ROLE`` statement.
.. _jwt_impersonation:
JWT-Based User Impersonation
----------------------------
We use `JSON Web Tokens <https://jwt.io/>`_ to authenticate API requests. As you'll recall a JWT contains a list of cryptographically signed claims. All claims are allowed but PostgREST cares specifically about a claim called role.
.. code:: json
{
"role": "user123"
}
When a request contains a valid JWT with a role claim PostgREST will switch to the database role with that name for the duration of the HTTP request.
.. code:: sql
SET LOCAL ROLE user123;
Note that the database administrator must allow the authenticator role to switch into this user by previously executing
.. code:: sql
GRANT user123 TO authenticator;
-- similarly for the anonymous role
-- GRANT anonymous TO authenticator;
If the client included no JWT (or one without a role claim) then PostgREST switches into the anonymous role. The database administrator must set the anonymous role permissions correctly to prevent anonymous users from seeing or changing things they shouldn't.
.. _jwt_generation:
JWT Generation
~~~~~~~~~~~~~~
You can create a valid JWT either from inside your database(see :ref:`sql_user_management`) or via an external service(see :ref:`external_jwt`).
.. _client_auth:
Client Auth
~~~~~~~~~~~
To make an authenticated request the client must include an :code:`Authorization` HTTP header with the value :code:`Bearer <jwt>`. For instance:
.. tabs::
.. code-tab:: http
GET /foo HTTP/1.1
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiamRvZSIsImV4cCI6MTQ3NTUxNjI1MH0.GYDZV3yM0gqvuEtJmfpplLBXSGYnke_Pvnl0tbKAjB4
.. code-tab:: bash Curl
curl "http://localhost:3000/foo" \
-H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiamRvZSIsImV4cCI6MTQ3NTUxNjI1MH0.GYDZV3yM0gqvuEtJmfpplLBXSGYnke_Pvnl0tbKAjB4"
The ``Bearer`` header value can be used with or without capitalization(``bearer``).
Symmetric Keys
~~~~~~~~~~~~~~
Each token is cryptographically signed with a secret key. In the case of symmetric cryptography the signer and verifier share the same secret passphrase, which can be configured with :ref:`jwt-secret`.
If it is set to a simple string value like “reallyreallyreallyreallyverysafe” then PostgREST interprets it as an HMAC-SHA256 passphrase.
.. _asym_keys:
Asymmetric Keys
~~~~~~~~~~~~~~~
In asymmetric cryptography the signer uses the private key and the verifier the public key.
As described in the :ref:`configuration` section, PostgREST accepts a ``jwt-secret`` config file parameter. However you can also specify a literal JSON Web Key (JWK) or set. For example, you can use an RSA-256 public key encoded as a JWK:
.. code-block:: json
{
"alg":"RS256",
"e":"AQAB",
"key_ops":["verify"],
"kty":"RSA",
"n":"9zKNYTaYGfGm1tBMpRT6FxOYrM720GhXdettc02uyakYSEHU2IJz90G_MLlEl4-WWWYoS_QKFupw3s7aPYlaAjamG22rAnvWu-rRkP5sSSkKvud_IgKL4iE6Y2WJx2Bkl1XUFkdZ8wlEUR6O1ft3TS4uA-qKifSZ43CahzAJyUezOH9shI--tirC028lNg767ldEki3WnVr3zokSujC9YJ_9XXjw2hFBfmJUrNb0-wldvxQbFU8RPXip-GQ_JPTrCTZhrzGFeWPvhA6Rqmc3b1PhM9jY7Dur1sjYWYVyXlFNCK3c-6feo5WlRfe1aCWmwZQh6O18eTmLeT4nWYkDzQ"
}
.. note::
This could also be a JSON Web Key Set (JWKS) if it was contained within an array assigned to a `keys` member, e.g. ``{ keys: [jwk1, jwk2] }``.
Just pass it in as a single line string, escaping the quotes:
.. code-block:: ini
jwt-secret = "{ \"alg\":\"RS256\", … }"
To generate such a public/private key pair use a utility like `latchset/jose <https://github.com/latchset/jose>`_.
.. code-block:: bash
jose jwk gen -i '{"alg": "RS256"}' -o rsa.jwk
jose jwk pub -i rsa.jwk -o rsa.jwk.pub
# now rsa.jwk.pub contains the desired JSON object
You can specify the literal value as we saw earlier, or reference a filename to load the JWK from a file:
.. code-block:: ini
jwt-secret = "@rsa.jwk.pub"
JWT Claims Validation
~~~~~~~~~~~~~~~~~~~~~
PostgREST honors the :code:`exp` claim for token expiration, rejecting expired tokens.
JWT Security
~~~~~~~~~~~~
There are at least three types of common critiques against using JWT: 1) against the standard itself, 2) against using libraries with known security vulnerabilities, and 3) against using JWT for web sessions. We'll briefly explain each critique, how PostgREST deals with it, and give recommendations for appropriate user action.
The critique against the `JWT standard <https://datatracker.ietf.org/doc/html/rfc7519>`_ is voiced in detail `elsewhere on the web <https://web.archive.org/web/20230123041631/https://paragonie.com/blog/2017/03/jwt-json-web-tokens-is-bad-standard-that-everyone-should-avoid>`_. The most relevant part for PostgREST is the so-called :code:`alg=none` issue. Some servers implementing JWT allow clients to choose the algorithm used to sign the JWT. In this case, an attacker could set the algorithm to :code:`none`, remove the need for any signature at all and gain unauthorized access. The current implementation of PostgREST, however, does not allow clients to set the signature algorithm in the HTTP request, making this attack irrelevant. The critique against the standard is that it requires the implementation of the :code:`alg=none` at all.
Critiques against JWT libraries are only relevant to PostgREST via the library it uses. As mentioned above, not allowing clients to choose the signature algorithm in HTTP requests removes the greatest risk. Another more subtle attack is possible where servers use asymmetric algorithms like RSA for signatures. Once again this is not relevant to PostgREST since it is not supported. Curious readers can find more information in `this article <https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/>`_. Recommendations about high quality libraries for usage in API clients can be found on `jwt.io <https://jwt.io/>`_.
The last type of critique focuses on the misuse of JWT for maintaining web sessions. The basic recommendation is to `stop using JWT for sessions <http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-for-sessions/>`_ because most, if not all, solutions to the problems that arise when you do, `do not work <http://cryto.net/~joepie91/blog/2016/06/19/stop-using-jwt-for-sessions-part-2-why-your-solution-doesnt-work/>`_. The linked articles discuss the problems in depth but the essence of the problem is that JWT is not designed to be secure and stateful units for client-side storage and therefore not suited to session management.
PostgREST uses JWT mainly for authentication and authorization purposes and encourages users to do the same. For web sessions, using cookies over HTTPS is good enough and well catered for by standard web frameworks.
.. _custom_validation:
Custom Validation
-----------------
PostgREST does not enforce any extra constraints besides JWT validation. An example of an extra constraint would be to immediately revoke access for a certain user. Using :ref:`db-pre-request` you can specify a stored procedure to call immediately after :ref:`user_impersonation` and before the main query itself runs.
.. code:: ini
db-pre-request = "public.check_user"
In the function you can run arbitrary code to check the request and raise an exception(see :ref:`raise_error`) to block it if desired. Here you can take advantage of :ref:`guc_req_headers_cookies_claims` for
doing custom logic based on the web user info.
.. code-block:: postgres
CREATE OR REPLACE FUNCTION check_user() RETURNS void AS $$
DECLARE
email text := current_setting('request.jwt.claims', true)::json->>'email';
BEGIN
IF email = 'evil.user@malicious.com' THEN
RAISE EXCEPTION 'No, you are evil'
USING HINT = 'Stop being so evil and maybe you can log in';
END IF;
END
$$ LANGUAGE plpgsql;
docs/references/configuration.rst
+728
View File
@@ -0,0 +1,728 @@
.. _configuration:
Configuration
#############
Without configuration, PostgREST won't be able to serve requests. At the minimum it needs either :ref:`a role to serve anonymous requests with <db-anon-role>` - or :ref:`a secret to use for JWT authentication <jwt-secret>`. Config parameters can be provided via :ref:`file_config`, via :ref:`env_variables_config` or through :ref:`in_db_config`.
To connect to a database it uses a `libpq connection string <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_. The connection string can be set in the configuration file or via environment variable or can be read from an external file. See :ref:`db-uri` for details. Any parameter that is not set in the connection string is read from `libpq environment variables <https://www.postgresql.org/docs/current/libpq-envars.html>`_. The default connection string is ``postgresql://``, which reads **all** parameters from the environment.
Config parameters are read in the following order:
1. From the config file.
2. From environment variables, overriding values from the config file.
3. From the database, overriding values from both the config file and environment variables.
.. _file_config:
Config File
===========
PostgREST can read a config file. There is no predefined location for this file, you must specify the file path as the one and only argument to the server:
.. code:: bash
./postgrest /path/to/postgrest.conf
.. note::
Configuration can be reloaded without restarting the server. See :ref:`config_reloading`.
The configuration file must contain a set of key value pairs:
.. code::
# postgrest.conf
# The standard connection URI format, documented at
# https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING
db-uri = "postgres://user:pass@host:5432/dbname"
# The database role to use when no client authentication is provided.
# Should differ from authenticator
db-anon-role = "anon"
# The secret to verify the JWT for authenticated requests with.
# Needs to be 32 characters minimum.
jwt-secret = "reallyreallyreallyreallyverysafe"
jwt-secret-is-base64 = False
# Port the postgrest process is listening on for http requests
server-port = 80
You can run ``postgrest --example`` to display all possible configuration parameters and how to use them in a configuration file.
.. _env_variables_config:
Environment Variables
=====================
You can also set these :ref:`configuration parameters <config_full_list>` using environment variables. They are capitalized, have a ``PGRST_`` prefix, and use underscores. For example: ``PGRST_DB_URI`` corresponds to ``db-uri`` and ``PGRST_APP_SETTINGS_*`` to ``app.settings.*``.
.. _in_db_config:
In-Database Configuration
=========================
By adding settings to the **authenticator** role (see :ref:`roles`), you can make the database the single source of truth for PostgREST's configuration.
This is enabled by :ref:`db-config`.
For example, you can configure :ref:`db-schemas` and :ref:`jwt-secret` like this:
.. code:: postgresql
ALTER ROLE authenticator SET pgrst.db_schemas = "tenant1, tenant2, tenant3"
ALTER ROLE authenticator IN DATABASE <your_database_name> SET pgrst.jwt_secret = "REALLYREALLYREALLYREALLYVERYSAFE"
You can use both database-specific settings with `IN DATABASE` and cluster-wide settings without it. Database-specific settings will override cluster-wide settings if both are used for the same parameter.
Note that underscores(``_``) need to be used instead of dashes(``-``) for the in-database config parameters.
.. important::
For altering a role in this way, you need a SUPERUSER. You might not be able to use this configuration mode on cloud-hosted databases.
When using both the configuration file and the in-database configuration, the latter takes precedence.
.. danger::
If direct connections to the database are allowed, then it's not safe to use the in-db configuration for storing the :ref:`jwt-secret`.
The settings of every role are PUBLIC - they can be viewed by any user that queries the ``pg_catalog.pg_db_role_setting`` table.
In this case you should keep the :ref:`jwt-secret` in the configuration file or as environment variables.
.. _config_reloading:
Configuration Reloading
=======================
It's possible to reload PostgREST's configuration without restarting the server. You can do this :ref:`via signal <config_reloading_signal>` or :ref:`via notification <config_reloading_notify>`.
It's not possible to change :ref:`env_variables_config` for a running process and reloading a Docker container configuration will not work. In these cases, you need to restart the PostgREST server or use :ref:`in_db_config` as an alternative.
.. important::
The following settings will not be reloaded. You will need to restart PostgREST to change those.
* :ref:`admin-server-port`
* :ref:`db-uri`
* :ref:`db-pool`
* :ref:`db-pool-acquisition-timeout`
* :ref:`db-pool-max-lifetime`
* :ref:`server-host`
* :ref:`server-port`
* :ref:`server-unix-socket`
* :ref:`server-unix-socket-mode`
.. _config_reloading_signal:
Reload with signal
------------------
To reload the configuration via signal, send a SIGUSR2 signal to the server process.
.. code:: bash
killall -SIGUSR2 postgrest
.. _config_reloading_notify:
Reload with NOTIFY
------------------
To reload the configuration from within the database, you can use a NOTIFY command.
.. code:: postgresql
NOTIFY pgrst, 'reload config'
The ``"pgrst"`` notification channel is enabled by default. For configuring the channel, see :ref:`db-channel` and :ref:`db-channel-enabled`.
.. _config_full_list:
List of parameters
==================
=========================== ======= ================= ==========
Name Type Default Reloadable
=========================== ======= ================= ==========
admin-server-port Int
app.settings.* String Y
db-anon-role String Y
db-channel String pgrst Y
db-channel-enabled Boolean True Y
db-config Boolean True Y
db-extra-search-path String public Y
db-max-rows Int ∞ Y
db-plan-enabled Boolean False Y
db-pool Int 10
db-pool-acquisition-timeout Int 10
db-pool-max-lifetime Int 1800
db-pre-request String Y
db-prepared-statements Boolean True Y
db-schemas String public Y
db-tx-end String commit
db-uri String postgresql://
db-use-legacy-gucs Boolean True Y
jwt-aud String Y
jwt-role-claim-key String .role Y
jwt-secret String Y
jwt-secret-is-base64 Boolean False Y
log-level String error Y
openapi-mode String follow-privileges Y
openapi-security-active Boolean False Y
openapi-server-proxy-uri String Y
raw-media-types String Y
server-host String !4
server-port Int 3000
server-unix-socket String
server-unix-socket-mode String 660
=========================== ======= ================= ==========
.. _admin-server-port:
admin-server-port
-----------------
=============== =======================
**Environment** PGRST_ADMIN_SERVER_PORT
**In-Database** `n/a`
=============== =======================
Specifies the port for the :ref:`health_check` endpoints.
.. _app.settings.*:
app.settings.*
--------------
=============== ====================
**Environment** PGRST_APP_SETTINGS_*
**In-Database** pgrst.app_settings_*
=============== ====================
Arbitrary settings that can be used to pass in secret keys directly as strings, or via OS environment variables. For instance: :code:`app.settings.jwt_secret = "$(MYAPP_JWT_SECRET)"` will take :code:`MYAPP_JWT_SECRET` from the environment and make it available to postgresql functions as :code:`current_setting('app.settings.jwt_secret')`.
.. _db-anon-role:
db-anon-role
------------
=============== ==================
**Environment** PGRST_DB_ANON_ROLE
**In-Database** `n/a`
=============== ==================
The database role to use when executing commands on behalf of unauthenticated clients. For more information, see :ref:`roles`.
When unset anonymous access will be blocked.
.. _db-channel:
db-channel
----------
=============== ================
**Environment** PGRST_DB_CHANNEL
**In-Database** `n/a`
=============== ================
The name of the notification channel that PostgREST uses for :ref:`schema_reloading` and configuration reloading.
.. _db-channel-enabled:
db-channel-enabled
------------------
=============== ========================
**Environment** PGRST_DB_CHANNEL_ENABLED
**In-Database** `n/a`
=============== ========================
When this is set to :code:`true`, the notification channel specified in :ref:`db-channel` is enabled.
You should set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
.. _db-config:
db-config
---------
=============== ===============
**Environment** PGRST_DB_CONFIG
**In-Database** `n/a`
=============== ===============
Enables the in-database configuration.
.. _db-extra-search-path:
db-extra-search-path
--------------------
=============== ==========================
**Environment** PGRST_DB_EXTRA_SEARCH_PATH
**In-Database** pgrst.db_extra_search_path
=============== ==========================
Extra schemas to add to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request. These schemas tables, views and stored procedures **don't get API endpoints**, they can only be referred from the database objects inside your :ref:`db-schemas`.
This parameter was meant to make it easier to use **PostgreSQL extensions** (like PostGIS) that are outside of the :ref:`db-schemas`.
Multiple schemas can be added in a comma-separated string, e.g. ``public, extensions``.
.. _db-max-rows:
db-max-rows
-----------
*For backwards compatibility, this config parameter is also available without prefix as "max-rows".*
=============== =================
**Environment** PGRST_DB_MAX_ROWS
**In-Database** pgrst.db_max_rows
=============== =================
A hard limit to the number of rows PostgREST will fetch from a view, table, or stored procedure. Limits payload size for accidental or malicious requests.
.. _db-plan-enabled:
db-plan-enabled
---------------
=============== =====================
**Environment** PGRST_DB_PLAN_ENABLED
**In-Database** pgrst.db_plan_enabled
=============== =====================
When this is set to :code:`true`, the execution plan of a request can be retrieved by using the :code:`Accept: application/vnd.pgrst.plan` header. See :ref:`explain_plan`.
It's recommended to use this in testing environments only since it reveals internal database details.
However, if you choose to use it in production you can add a :ref:`db-pre-request` to filter the requests that can use this feature.
For example, to only allow requests from an IP address to get the execution plans:
.. code-block:: postgresql
-- Assuming a proxy(Nginx, Cloudflare, etc) passes an "X-Forwarded-For" header(https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For)
create or replace function filter_plan_requests()
returns void as $$
declare
headers json := current_setting('request.headers', true)::json;
client_ip text := coalesce(headers->>'x-forwarded-for', '');
accept text := coalesce(headers->>'accept', '');
begin
if accept like 'application/vnd.pgrst.plan%' and client_ip != '144.96.121.73' then
raise insufficient_privilege using
message = 'Not allowed to use application/vnd.pgrst.plan';
end if;
end; $$ language plpgsql;
-- set this function on your postgrest.conf
-- db-pre-request = filter_plan_requests
.. _db-pool:
db-pool
-------
=============== =================
**Environment** PGRST_DB_POOL
**In-Database** `n/a`
=============== =================
Number of maximum connections to keep open in PostgREST's database pool.
.. _db-pool-acquisition-timeout:
db-pool-acquisition-timeout
---------------------------
=============== =================
**Environment** PGRST_DB_POOL_ACQUISITION_TIMEOUT
**In-Database** `n/a`
=============== =================
Specifies the maximum time in seconds that the request will wait for the pool to free up a connection slot to the database.
.. _db-pool-max-lifetime:
db-pool-max-lifetime
--------------------
=============== =================
**Environment** PGRST_DB_POOL_MAX_LIFETIME
**In-Database** `n/a`
=============== =================
Specifies the maximum time in seconds of an existing connection in the pool.
.. _db-pre-request:
db-pre-request
--------------
*For backwards compatibility, this config parameter is also available without prefix as "pre-request".*
=============== =================
**Environment** PGRST_DB_PRE_REQUEST
**In-Database** pgrst.db_pre_request
=============== =================
A schema-qualified stored procedure name to call right after the :ref:`tx_settings` are set. See :ref:`pre-request`.
.. _db-prepared-statements:
db-prepared-statements
----------------------
=============== =================
**Environment** PGRST_DB_PREPARED_STATEMENTS
**In-Database** pgrst.db_prepared_statements
=============== =================
Enables or disables prepared statements.
When disabled, the generated queries will be parameterized (invulnerable to SQL injection) but they will not be prepared (cached in the database session). Not using prepared statements will noticeably decrease performance, so it's recommended to always have this setting enabled.
You should only set this to ``false`` when using PostgresSQL behind an external connection pooler such as PgBouncer working in transaction pooling mode. See :ref:`this section <external_connection_poolers>` for more information.
.. _db-schemas:
db-schemas
----------
*For backwards compatibility, this config parameter is also available in singular as "db-schema".*
=============== =================
**Environment** PGRST_DB_SCHEMAS
**In-Database** pgrst.db_schemas
=============== =================
The database schema to expose to REST clients. Tables, views and stored procedures in this schema will get API endpoints.
.. code:: bash
db-schemas = "api"
This schema gets added to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request.
List of schemas
~~~~~~~~~~~~~~~
You can also specify a list of schemas that can be used for **schema-based multitenancy** and **api versioning** by :ref:`multiple-schemas`. Example:
.. code:: bash
db-schemas = "tenant1, tenant2"
If you don't :ref:`Switch Schemas <multiple-schemas>`, the first schema in the list(``tenant1`` in this case) is chosen as the default schema.
*Only the chosen schema* gets added to the `search_path <https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATH>`_ of every request.
.. warning::
Never expose private schemas in this way. See :ref:`schema_isolation`.
.. _db-tx-end:
db-tx-end
---------
=============== =================
**Environment** PGRST_DB_TX_END
**In-Database** pgrst.db_tx_end
=============== =================
Specifies how to terminate the database transactions.
.. code:: bash
# The transaction is always committed
db-tx-end = "commit"
# The transaction is committed unless a "Prefer: tx=rollback" header is sent
db-tx-end = "commit-allow-override"
# The transaction is always rolled back
db-tx-end = "rollback"
# The transaction is rolled back unless a "Prefer: tx=commit" header is sent
db-tx-end = "rollback-allow-override"
.. _db-uri:
db-uri
------
=============== =================
**Environment** PGRST_DB_URI
**In-Database** `n/a`
=============== =================
The standard connection PostgreSQL `URI format <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_. Symbols and unusual characters in the password or other fields should be percent encoded to avoid a parse error. If enforcing an SSL connection to the database is required you can use `sslmode <https://www.postgresql.org/docs/current/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS>`_ in the URI, for example ``postgres://user:pass@host:5432/dbname?sslmode=require``.
The user with whom PostgREST connects to the database is also known as the ``authenticator`` role. For more information see :ref:`roles`.
When running PostgREST on the same machine as PostgreSQL, it is also possible to connect to the database using a `Unix socket <https://en.wikipedia.org/wiki/Unix_domain_socket>`_ and the `Peer Authentication method <https://www.postgresql.org/docs/current/auth-peer.html>`_ as an alternative to TCP/IP communication and authentication with a password, this also grants higher performance. To do this you can omit the host and the password, e.g. ``postgres://user@/dbname``, see the `libpq connection string <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>`_ documentation for more details.
Choosing a value for this parameter beginning with the at sign such as ``@filename`` (e.g. ``@./configs/my-config``) loads the connection string out of an external file.
.. _db-use-legacy-gucs:
db-use-legacy-gucs
------------------
=============== =================
**Environment** PGRST_DB_USE_LEGACY_GUCS
**In-Database** pgrst.db_use_legacy_gucs
=============== =================
Determine if GUC request settings for headers, cookies and jwt claims use the `legacy names <https://postgrest.org/en/v8.0/api.html#accessing-request-headers-cookies-and-jwt-claims>`_ (string with dashes, invalid starting from PostgreSQL v14) with text values instead of the :ref:`new names <guc_req_headers_cookies_claims>` (string without dashes, valid on all PostgreSQL versions) with json values.
On PostgreSQL versions 14 and above, this parameter is ignored.
.. _jwt-aud:
jwt-aud
-------
=============== =================
**Environment** PGRST_JWT_AUD
**In-Database** pgrst.jwt_aud
=============== =================
Specifies the `JWT audience claim <https://datatracker.ietf.org/doc/html/rfc7519#section-4.1.3>`_. If this claim is present in the client provided JWT then you must set this to the same value as in the JWT, otherwise verifying the JWT will fail.
.. _jwt-role-claim-key:
jwt-role-claim-key
------------------
*For backwards compatibility, this config parameter is also available without prefix as "role-claim-key".*
=============== =================
**Environment** PGRST_JWT_ROLE_CLAIM_KEY
**In-Database** pgrst.jwt_role_claim_key
=============== =================
A JSPath DSL that specifies the location of the :code:`role` key in the JWT claims. This can be used to consume a JWT provided by a third party service like Auth0, Okta or Keycloak. Usage examples:
.. code:: bash
# {"postgrest":{"roles": ["other", "author"]}}
# the DSL accepts characters that are alphanumerical or one of "_$@" as keys
jwt-role-claim-key = ".postgrest.roles[1]"
# {"https://www.example.com/role": { "key": "author }}
# non-alphanumerical characters can go inside quotes(escaped in the config value)
jwt-role-claim-key = ".\"https://www.example.com/role\".key"
.. _jwt-secret:
jwt-secret
----------
=============== =================
**Environment** PGRST_JWT_SECRET
**In-Database** pgrst.jwt_secret
=============== =================
The secret or `JSON Web Key (JWK) (or set) <https://datatracker.ietf.org/doc/html/rfc7517>`_ used to decode JWT tokens clients provide for authentication. For security the key must be **at least 32 characters long**. If this parameter is not specified then PostgREST refuses authentication requests. Choosing a value for this parameter beginning with the at sign such as :code:`@filename` loads the secret out of an external file. This is useful for automating deployments. Note that any binary secrets must be base64 encoded. Both symmetric and asymmetric cryptography are supported. For more info see :ref:`asym_keys`.
Choosing a value for this parameter beginning with the at sign such as ``@filename`` (e.g. ``@./configs/my-config``) loads the secret out of an external file.
.. warning::
Only when using the :ref:`file_config`, if the ``jwt-secret`` contains a ``$`` character by itself it will give errors. In this case, use ``$$`` and PostgREST will interpret it as a single ``$`` character.
.. _jwt-secret-is-base64:
jwt-secret-is-base64
--------------------
=============== =================
**Environment** PGRST_JWT_SECRET_IS_BASE64
**In-Database** pgrst.jwt_secret_is_base64
=============== =================
When this is set to :code:`true`, the value derived from :code:`jwt-secret` will be treated as a base64 encoded secret.
.. _log-level:
log-level
---------
=============== =================
**Environment** PGRST_LOG_LEVEL
**In-Database** `n/a`
=============== =================
Specifies the level of information to be logged while running PostgREST.
.. code:: bash
# Only startup and db connection recovery messages are logged
log-level = "crit"
# All the "crit" level events plus server errors (status 5xx) are logged
log-level = "error"
# All the "error" level events plus request errors (status 4xx) are logged
log-level = "warn"
# All the "warn" level events plus all requests (every status code) are logged
log-level = "info"
Because currently there's no buffering for logging, the levels with minimal logging(``crit/error``) will increase throughput.
.. _openapi-mode:
openapi-mode
------------
=============== =================
**Environment** PGRST_OPENAPI_MODE
**In-Database** pgrst.openapi_mode
=============== =================
Specifies how the OpenAPI output should be displayed.
.. code:: bash
# Follows the privileges of the JWT role claim (or from db-anon-role if the JWT is not sent)
# Shows information depending on the permissions that the role making the request has
openapi-mode = "follow-privileges"
# Ignores the privileges of the JWT role claim (or from db-anon-role if the JWT is not sent)
# Shows all the exposed information, regardless of the permissions that the role making the request has
openapi-mode = "ignore-privileges"
# Disables the OpenApi output altogether.
# Throws a `404 Not Found` error when accessing the API root path
openapi-mode = "disabled"
.. _openapi-security-active:
openapi-security-active
-----------------------
=============== =============================
**Environment** PGRST_OPENAPI_SECURITY_ACTIVE
**In-Database** pgrst.openapi_security_active
=============== =============================
When this is set to :code:`true`, security options are included in the :ref:`OpenAPI output <open-api>`.
.. _openapi-server-proxy-uri:
openapi-server-proxy-uri
------------------------
=============== =================
**Environment** PGRST_OPENAPI_SERVER_PROXY_URI
**In-Database** pgrst.openapi_server_proxy_uri
=============== =================
Overrides the base URL used within the OpenAPI self-documentation hosted at the API root path. Use a complete URI syntax :code:`scheme:[//[user:password@]host[:port]][/]path[?query][#fragment]`. Ex. :code:`https://postgrest.com`
.. code:: json
{
"swagger": "2.0",
"info": {
"version": "0.4.3.0",
"title": "PostgREST API",
"description": "This is a dynamic API generated by PostgREST"
},
"host": "postgrest.com:443",
"basePath": "/",
"schemes": [
"https"
]
}
.. _raw-media-types:
raw-media-types
---------------
=============== =================
**Environment** PGRST_RAW_MEDIA_TYPES
**In-Database** pgrst.raw_media_types
=============== =================
This serves to extend the `Media Types <https://en.wikipedia.org/wiki/Media_type>`_ that PostgREST currently accepts through an ``Accept`` header.
These media types can be requested by following the same rules as the ones defined in :ref:`scalar_return_formats`.
As an example, the below config would allow you to request an **image** and a **XML** file by doing a request with ``Accept: image/png``
or ``Accept: font/woff2``, respectively.
.. code:: bash
raw-media-types="image/png, font/woff2"
.. _server-host:
server-host
-----------
=============== =================
**Environment** PGRST_SERVER_HOST
**In-Database** `n/a`
=============== =================
Where to bind the PostgREST web server. In addition to the usual address options, PostgREST interprets these reserved addresses with special meanings:
* :code:`*` - any IPv4 or IPv6 hostname
* :code:`*4` - any IPv4 or IPv6 hostname, IPv4 preferred
* :code:`!4` - any IPv4 hostname
* :code:`*6` - any IPv4 or IPv6 hostname, IPv6 preferred
* :code:`!6` - any IPv6 hostname
.. _server-port:
server-port
-----------
=============== =================
**Environment** PGRST_SERVER_PORT
**In-Database** `n/a`
=============== =================
The TCP port to bind the web server.
.. _server-unix-socket:
server-unix-socket
------------------
=============== =================
**Environment** PGRST_SERVER_UNIX_SOCKET
**In-Database** `n/a`
=============== =================
`Unix domain socket <https://en.wikipedia.org/wiki/Unix_domain_socket>`_ where to bind the PostgREST web server.
If specified, this takes precedence over :ref:`server-port`. Example:
.. code:: bash
server-unix-socket = "/tmp/pgrst.sock"
.. _server-unix-socket-mode:
server-unix-socket-mode
-----------------------
=============== =================
**Environment** PGRST_SERVER_UNIX_SOCKET_MODE
**In-Database** `n/a`
=============== =================
`Unix file mode <https://en.wikipedia.org/wiki/File_system_permissions>`_ to be set for the socket specified in :ref:`server-unix-socket`
Needs to be a valid octal between 600 and 777.
.. code:: bash
server-unix-socket-mode = "660"
docs/references/connection_pool.rst
+83
View File
@@ -0,0 +1,83 @@
Connection Pool
===============
Every request to an :doc:`API resource <api>` borrows a connection from the connection pool to start a :doc:`transaction <transactions>`.
A connection pool is a cache of reusable database connections. It allows serving many HTTP requests using few database connections.
Minimizing connections it’s paramount to performance. Each PostgreSQL connection creates a process, having too many can exhaust available resources.
.. _pool_growth_limit:
Growth Limit
------------
If all the connections are being used, a new connection is added to the pool. The pool can grow until it reaches the :ref:`db-pool` size.
Note it’s pointless to set this higher than the ``max_connections`` setting in your database.
Connection lifetime
-------------------
After a period of time, connections from the pool will be released and news ones will be created. This time is specified by :ref:`db-pool-max-lifetime`.
The lifetime doesn't affect running requests. Only unused connections will be released.
For knowing why a connection lifetime is necessary, see the following discussion:
https://www.postgresql.org/message-id/flat/CA%2Bmi_8bnvpxHZtb6EgHSHY-xn29W8VJMzjPU3fiCOv1bfjrNuA%40mail.gmail.com.
Acquisition Timeout
-------------------
If all the available connections in the pool are busy, an HTTP request will wait until reaching a timeout. You can configure this timeout with :ref:`db-pool-acquisition-timeout`.
If the request reaches the timeout, it will be aborted with the following response:
.. code-block:: http
HTTP/1.1 504 Gateway Timeout
{"code":"PGRST003",
"details":null,
"hint":null,
"message":"Timed out acquiring connection from connection pool."}
Getting this error message is an indicator of a performance issue. To solve it, you can:
- Reduce your queries execution time.
- Check the request :ref:`explain_plan` to tune your query, this usually means adding indexes.
- Reduce the amount of requests.
- Reduce write requests. Do :ref:`bulk_insert` (or :ref:`upsert`) instead of inserting rows one by one.
- Reduce read requests. Use :ref:`resource_embedding`. Combine unrelated data into a single request using custom database views or functions.
- Use :ref:`s_procs` for combining read and write logic into a single request.
- Increase the :ref:`pool growth limit <pool_growth_limit>`.
- Not a panacea since connections can't grow infinitely. Try the previous recommendations before this.
.. _automatic_recovery:
Automatic Recovery
------------------
If the pool loses the connection to the database, it will retry reconnecting using exponential backoff. With 32 seconds being the maximum backoff time between retries.
The retries happen immediately after a connection loss, if :ref:`db-channel-enabled` is set to true(the default). Otherwise they'll happen once a request arrives.
To notify the client of the next retry, the server sends a ``503 Service Unavailable`` status with the ``Retry-After: x`` header. Where ``x`` is the number of seconds programmed for the next retry.
.. _external_connection_poolers:
Using External Connection Poolers
---------------------------------
It's possible to use external connection poolers, such as PgBouncer. Session pooling is compatible, while transaction pooling requires :ref:`db-prepared-statements` set to ``false``. Statement pooling is not compatible with PostgREST.
Also set :ref:`db-channel-enabled` to ``false`` since ``LISTEN`` is not compatible with transaction pooling. Although it should not give any errors if left enabled.
.. note::
It’s not recommended to use an external connection pooler. `Our benchmarks <https://github.com/PostgREST/postgrest/issues/2294#issuecomment-1139148672>`_ indicate it provides much lower performance than PostgREST built-in pool.
docs/references/errors.rst
+323
View File
@@ -0,0 +1,323 @@
.. _error_source:
Errors
######
PostgREST error messages follow the PostgreSQL error structure. It includes ``MESSAGE``, ``DETAIL``, ``HINT``, ``ERRCODE`` and will add an HTTP status code to the response.
Errors from PostgreSQL
======================
PostgREST will forward errors coming from PostgreSQL. For instance, when querying a nonexistent table:
.. code-block:: http
GET /nonexistent_table?id=eq.1 HTTP/1.1
.. code-block:: http
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
.. code-block:: json
{
"hint": null,
"details": null,
"code": "42P01",
"message": "relation \"api.nonexistent_table\" does not exist"
}
.. _status_codes:
HTTP Status Codes
-----------------
PostgREST translates `PostgreSQL error codes <https://www.postgresql.org/docs/current/errcodes-appendix.html>`_ into HTTP status as follows:
+--------------------------+-------------------------+---------------------------------+
| PostgreSQL error code(s) | HTTP status | Error description |
+==========================+=========================+=================================+
| 08* | 503 | pg connection err |
+--------------------------+-------------------------+---------------------------------+
| 09* | 500 | triggered action exception |
+--------------------------+-------------------------+---------------------------------+
| 0L* | 403 | invalid grantor |
+--------------------------+-------------------------+---------------------------------+
| 0P* | 403 | invalid role specification |
+--------------------------+-------------------------+---------------------------------+
| 23503 | 409 | foreign key violation |
+--------------------------+-------------------------+---------------------------------+
| 23505 | 409 | uniqueness violation |
+--------------------------+-------------------------+---------------------------------+
| 25006 | 405 | read only sql transaction |
+--------------------------+-------------------------+---------------------------------+
| 25* | 500 | invalid transaction state |
+--------------------------+-------------------------+---------------------------------+
| 28* | 403 | invalid auth specification |
+--------------------------+-------------------------+---------------------------------+
| 2D* | 500 | invalid transaction termination |
+--------------------------+-------------------------+---------------------------------+
| 38* | 500 | external routine exception |
+--------------------------+-------------------------+---------------------------------+
| 39* | 500 | external routine invocation |
+--------------------------+-------------------------+---------------------------------+
| 3B* | 500 | savepoint exception |
+--------------------------+-------------------------+---------------------------------+
| 40* | 500 | transaction rollback |
+--------------------------+-------------------------+---------------------------------+
| 53* | 503 | insufficient resources |
+--------------------------+-------------------------+---------------------------------+
| 54* | 413 | too complex |
+--------------------------+-------------------------+---------------------------------+
| 55* | 500 | obj not in prerequisite state |
+--------------------------+-------------------------+---------------------------------+
| 57* | 500 | operator intervention |
+--------------------------+-------------------------+---------------------------------+
| 58* | 500 | system error |
+--------------------------+-------------------------+---------------------------------+
| F0* | 500 | config file error |
+--------------------------+-------------------------+---------------------------------+
| HV* | 500 | foreign data wrapper error |
+--------------------------+-------------------------+---------------------------------+
| P0001 | 400 | default code for "raise" |
+--------------------------+-------------------------+---------------------------------+
| P0* | 500 | PL/pgSQL error |
+--------------------------+-------------------------+---------------------------------+
| XX* | 500 | internal error |
+--------------------------+-------------------------+---------------------------------+
| 42883 | 404 | undefined function |
+--------------------------+-------------------------+---------------------------------+
| 42P01 | 404 | undefined table |
+--------------------------+-------------------------+---------------------------------+
| 42501 | | if authenticated 403, | insufficient privileges |
| | | else 401 | |
+--------------------------+-------------------------+---------------------------------+
| other | 400 | |
+--------------------------+-------------------------+---------------------------------+
Errors from PostgREST
=====================
Errors that come from PostgREST itself maintain the same structure. But differ in the ``PGRST`` prefix in the ``code`` field. For instance, when querying a function that does not exist in the :doc:`schema cache <schema_cache>`:
.. code-block:: http
POST /rpc/nonexistent_function HTTP/1.1
.. code-block:: http
HTTP/1.1 404 Not Found
Content-Type: application/json; charset=utf-8
.. code-block:: json
{
"hint": "...",
"details": null
"code": "PGRST202",
"message": "Could not find the api.nonexistent_function() function in the schema cache"
}
.. _pgrst_errors:
PostgREST Error Codes
---------------------
PostgREST error codes have the form ``PGRSTgxx``
- ``PGRST`` is the prefix that differentiates the error from a PostgreSQL error.
- ``g`` is the error group
- ``xx`` is the error identifier in the group.
.. _pgrst0**:
Group 0 - Connection
~~~~~~~~~~~~~~~~~~~~
Related to the connection with the database.
+---------------+-------------+-------------------------------------------------------------+
| Code | HTTP status | Description |
+===============+=============+=============================================================+
| .. _pgrst000: | 503 | Could not connect with the database due to an incorrect |
| | | :ref:`db-uri` or due to the PostgreSQL service not running. |
| PGRST000 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst001: | 503 | Could not connect with the database due to an internal |
| | | error. |
| PGRST001 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst002: | 503 | Could not connect with the database when building the |
| | | :doc:`Schema Cache <schema_cache>` |
| PGRST002 | | due to the PostgreSQL service not running. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst003: | 504 | The request timed out waiting for a pool connection |
| | | to be available. See :ref:`db-pool-acquisition-timeout`. |
| PGRST003 | | |
+---------------+-------------+-------------------------------------------------------------+
.. _pgrst1**:
Group 1 - Api Request
~~~~~~~~~~~~~~~~~~~~~
Related to the HTTP request elements.
+---------------+-------------+-------------------------------------------------------------+
| Code | HTTP status | Description |
+===============+=============+=============================================================+
| .. _pgrst100: | 400 | Parsing error in the query string parameter. |
| | | See :ref:`h_filter`, :ref:`operators` and :ref:`ordering`. |
| PGRST100 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst101: | 405 | For :ref:`functions <s_procs>`, only ``GET`` and ``POST`` |
| | | verbs are allowed. Any other verb will throw this error. |
| PGRST101 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst102: | 400 | An invalid request body was sent(e.g. an empty body or |
| | | malformed JSON). |
| PGRST102 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst103: | 416 | An invalid range was specified for :ref:`limits`. |
| | | |
| PGRST103 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst105: | 405 | An invalid :ref:`PUT <upsert_put>` request was done |
| | | |
| PGRST105 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst106: | 406 | The schema specified when |
| | | :ref:`switching schemas <multiple-schemas>` is not present |
| PGRST106 | | in the :ref:`db-schemas` configuration variable. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst107: | 415 | The ``Content-Type`` sent in the request is invalid. |
| | | |
| PGRST107 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst108: | 400 | The filter is applied to a embedded resource that is not |
| | | specified in the ``select`` part of the query string. |
| PGRST108 | | See :ref:`embed_filters`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst109: | 400 | Restricting a Deletion or an Update using limits must |
| | | include the ordering of a unique column. |
| PGRST109 | | See :ref:`limited_update_delete`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst110: | 400 | When restricting a Deletion or an Update using limits |
| | | modifies more rows than the maximum specified in the limit. |
| PGRST110 | | See :ref:`limited_update_delete`. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst111: | 500 | An invalid ``response.headers`` was set. |
| | | See :ref:`guc_resp_hdrs`. |
| PGRST111 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst112: | 500 | The status code must be a positive integer. |
| | | See :ref:`guc_resp_status`. |
| PGRST112 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst113: | 406 | More than one column was returned for a scalar result. |
| | | See :ref:`scalar_return_formats`. |
| PGRST113 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst114: | 400 | For an :ref:`UPSERT using PUT <upsert_put>`, when |
| | | :ref:`limits and offsets <limits>` are used. |
| PGRST114 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst115: | 400 | For an :ref:`UPSERT using PUT <upsert_put>`, when the |
| | | primary key in the query string and the body are different. |
| PGRST115 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst116: | 406 | More than 1 or no items where returned when requesting |
| | | a singular response. See :ref:`singular_plural`. |
| PGRST116 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst117: | 405 | The HTTP verb used in the request in not supported. |
| | | |
| PGRST117 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst118: | 400 | Could not order the result using the related table because |
| | | there is no many-to-one or one-to-one relationship between |
| PGRST118 | | them. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst119: | 400 | Could not use the spread operator on the related table |
| | | because there is no many-to-one or one-to-one relationship |
| PGRST119 | | between them. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst120: | 400 | An embedded resource can only be filtered using the |
| | | ``is.null`` or ``not.is.null`` :ref:`operators <operators>`.|
| PGRST120 | | |
+---------------+-------------+-------------------------------------------------------------+
.. _pgrst2**:
Group 2 - Schema Cache
~~~~~~~~~~~~~~~~~~~~~~
Related to a :ref:`stale schema cache <stale_schema>`. Most of the time, these errors are solved by :ref:`reloading the schema cache <schema_reloading>`.
+---------------+-------------+-------------------------------------------------------------+
| Code | HTTP status | Description |
+===============+=============+=============================================================+
| .. _pgrst200: | 400 | Caused by stale foreign key relationships, otherwise any of |
| | | the embedding resources or the relationship itself may not |
| PGRST200 | | exist in the database. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst201: | 300 | An ambiguous embedding request was made. |
| | | See :ref:`embed_disamb`. |
| PGRST201 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst202: | 404 | Caused by a stale function signature, otherwise |
| | | the function may not exist in the database. |
| PGRST202 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst203: | 300 | Caused by requesting overloaded functions with the same |
| | | argument names but different types, or by using a ``POST`` |
| PGRST203 | | verb to request overloaded functions with a ``JSON`` or |
| | | ``JSONB`` type unnamed parameter. The solution is to rename |
| | | the function or add/modify the names of the arguments. |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst204: | 400 | Caused when the :ref:`column specified <specify_columns>` |
| | | in the ``columns`` query parameter is not found. |
| PGRST204 | | |
+---------------+-------------+-------------------------------------------------------------+
.. _pgrst3**:
Group 3 - JWT
~~~~~~~~~~~~~
Related to the authentication process using JWT. You can follow the :ref:`tut1` for an example on how to implement authentication and the :doc:`Authentication page <auth>` for more information on this process.
+---------------+-------------+-------------------------------------------------------------+
| Code | HTTP status | Description |
+===============+=============+=============================================================+
| .. _pgrst300: | 500 | A :ref:`JWT secret <jwt-secret>` is missing from the |
| | | configuration. |
| PGRST300 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst301: | 401 | Any error related to the verification of the JWT, |
| | | which means that the JWT provided is invalid in some way. |
| PGRST301 | | |
+---------------+-------------+-------------------------------------------------------------+
| .. _pgrst302: | 401 | Attempted to do a request without |
| | | :ref:`authentication <client_auth>` when the anonymous role |
| PGRST302 | | is disabled by not setting it in :ref:`db-anon-role`. |
+---------------+-------------+-------------------------------------------------------------+
.. The Internal Errors Group X** is always at the end
.. _pgrst_X**:
Group X - Internal
~~~~~~~~~~~~~~~~~~
Internal errors. If you encounter any of these, you may have stumbled on a PostgREST bug, please `open an issue <https://github.com/PostgREST/postgrest/issues>`_ and we'll be glad to fix it.
+---------------+-------------+-------------------------------------------------------------+
| Code | HTTP status | Description |
+===============+=============+=============================================================+
| .. _pgrstX00: | 500 | Internal errors related to the library used for connecting |
| | | to the database. |
| PGRSTX00 | | |
+---------------+-------------+-------------------------------------------------------------+
docs/references/schema_cache.rst
+173
View File
@@ -0,0 +1,173 @@
.. _schema_cache:
Schema Cache
============
Some PostgREST features need metadata from the database schema. Getting this metadata requires expensive queries. To avoid repeating this work, PostgREST uses a schema cache.
+--------------------------------------------+-------------------------------------------------------------------------------+
| Feature | Required Metadata |
+============================================+===============================================================================+
| :ref:`resource_embedding` | Foreign key constraints |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`Stored Functions <s_procs>` | Function signature (parameters, return type, volatility and |
| | `overloading <https://www.postgresql.org/docs/current/xfunc-overload.html>`_) |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`Upserts <upsert>` | Primary keys |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`Insertions <insert>` | Primary keys (optional: only if the Location header is requested) |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`OPTIONS requests <options_requests>` | View INSTEAD OF TRIGGERS and primary keys |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`open-api` | Table columns, primary keys and foreign keys |
+ +-------------------------------------------------------------------------------+
| | View columns and INSTEAD OF TRIGGERS |
+ +-------------------------------------------------------------------------------+
| | Function signature |
+--------------------------------------------+-------------------------------------------------------------------------------+
.. _stale_schema:
Stale Schema Cache
------------------
One operational problem that comes a cache is that it can go stale. This can happen for PostgREST when you make changes to the metadata before mentioned. Requests that depend on the metadata will fail.
You can solve this by reloading the cache manually or automatically.
.. _schema_reloading:
Schema Cache Reloading
----------------------
To manually reload the cache without restarting the PostgREST server, send a SIGUSR1 signal to the server process.
.. code:: bash
killall -SIGUSR1 postgrest
For docker you can do:
.. code:: bash
docker kill -s SIGUSR1 <container>
# or in docker-compose
docker-compose kill -s SIGUSR1 <service>
There’s no downtime when reloading the schema cache. The reloading will happen on a background thread while serving requests.
.. _schema_reloading_notify:
Reloading with NOTIFY
~~~~~~~~~~~~~~~~~~~~~
PostgREST also allows you to reload its schema cache through PostgreSQL `NOTIFY <https://www.postgresql.org/docs/current/sql-notify.html>`_.
.. code-block:: postgresql
NOTIFY pgrst, 'reload schema'
This is useful in environments where you can’t send the SIGUSR1 Unix Signal. Like on cloud managed containers or on Windows systems.
The ``pgrst`` notification channel is enabled by default. For configuring the channel, see :ref:`db-channel` and :ref:`db-channel-enabled`.
.. _auto_schema_reloading:
Automatic Schema Cache Reloading
--------------------------------
You can do automatic schema cache reloading in a pure SQL way and forget about stale schema cache errors. For this use an `event trigger <https://www.postgresql.org/docs/current/event-trigger-definition.html>`_ and ``NOTIFY``.
.. code-block:: postgresql
-- Create an event trigger function
CREATE OR REPLACE FUNCTION pgrst_watch() RETURNS event_trigger
LANGUAGE plpgsql
AS $$
BEGIN
NOTIFY pgrst, 'reload schema';
END;
$$;
-- This event trigger will fire after every ddl_command_end event
CREATE EVENT TRIGGER pgrst_watch
ON ddl_command_end
EXECUTE PROCEDURE pgrst_watch();
Now, whenever the ``pgrst_watch`` trigger fires, PostgREST will auto-reload the schema cache.
To disable auto reloading, drop the trigger.
.. code-block:: postgresql
DROP EVENT TRIGGER pgrst_watch
Finer-Grained Event Trigger
~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can refine the previous event trigger to only react to the events relevant to the schema cache. This also prevents unnecessary
reloading when creating temporary tables inside functions.
.. code-block:: postgresql
-- watch CREATE and ALTER
CREATE OR REPLACE FUNCTION pgrst_ddl_watch() RETURNS event_trigger AS $$
DECLARE
cmd record;
BEGIN
FOR cmd IN SELECT * FROM pg_event_trigger_ddl_commands()
LOOP
IF cmd.command_tag IN (
'CREATE SCHEMA', 'ALTER SCHEMA'
, 'CREATE TABLE', 'CREATE TABLE AS', 'SELECT INTO', 'ALTER TABLE'
, 'CREATE FOREIGN TABLE', 'ALTER FOREIGN TABLE'
, 'CREATE VIEW', 'ALTER VIEW'
, 'CREATE MATERIALIZED VIEW', 'ALTER MATERIALIZED VIEW'
, 'CREATE FUNCTION', 'ALTER FUNCTION'
, 'CREATE TRIGGER'
, 'CREATE TYPE', 'ALTER TYPE'
, 'CREATE RULE'
, 'COMMENT'
)
-- don't notify in case of CREATE TEMP table or other objects created on pg_temp
AND cmd.schema_name is distinct from 'pg_temp'
THEN
NOTIFY pgrst, 'reload schema';
END IF;
END LOOP;
END; $$ LANGUAGE plpgsql;
-- watch DROP
CREATE OR REPLACE FUNCTION pgrst_drop_watch() RETURNS event_trigger AS $$
DECLARE
obj record;
BEGIN
FOR obj IN SELECT * FROM pg_event_trigger_dropped_objects()
LOOP
IF obj.object_type IN (
'schema'
, 'table'
, 'foreign table'
, 'view'
, 'materialized view'
, 'function'
, 'trigger'
, 'type'
, 'rule'
)
AND obj.is_temporary IS false -- no pg_temp objects
THEN
NOTIFY pgrst, 'reload schema';
END IF;
END LOOP;
END; $$ LANGUAGE plpgsql;
CREATE EVENT TRIGGER pgrst_ddl_watch
ON ddl_command_end
EXECUTE PROCEDURE pgrst_ddl_watch();
CREATE EVENT TRIGGER pgrst_drop_watch
ON sql_drop
EXECUTE PROCEDURE pgrst_drop_watch();
docs/references/transactions.rst
+322
View File
@@ -0,0 +1,322 @@
Transactions
============
After :ref:`user_impersonation`, every request to an :doc:`API resource <api>` runs inside a transaction. The sequence of the transaction is as follows:
.. code-block:: postgresql
BEGIN; -- <Access Mode> <Isolation Level>
-- <Transaction-scoped settings>
-- <Main Query>;
END;
.. _access_mode:
Access Mode
-----------
The access mode on :ref:`tables_views` is determined by the HTTP method.
.. list-table::
:header-rows: 1
* - HTTP Method
- Access Method
* - GET, HEAD
- READ ONLY
* - POST, PATCH, PUT, DELETE
- READ WRITE
:ref:`s_procs` additionally depend on the function `volatility <https://www.postgresql.org/docs/current/xfunc-volatility.html>`_.
.. list-table::
:header-rows: 2
* -
- Access Method
-
-
* - HTTP Method
- VOLATILE
- STABLE
- IMMUTABLE
* - GET, HEAD
- READ ONLY
- READ ONLY
- READ ONLY
* - POST
- READ WRITE
- READ ONLY
- READ ONLY
Modifying the database inside READ ONLY transactions is not possible. PostgREST uses this fact to enforce HTTP semantics in GET and HEAD requests.
.. note::
The volatility marker is a promise about the behavior of the function. PostgreSQL will let you mark a function that modifies the database as ``IMMUTABLE`` or ``STABLE`` without failure. But, because of the READ ONLY transaction the function will fail under PostgREST.
The :ref:`options_requests` method doesn't start a transaction, so it's not relevant here.
Isolation Level
---------------
Every transaction uses the PostgreSQL default isolation level: READ COMMITTED.
.. _tx_settings:
Transaction-Scoped Settings
---------------------------
PostgREST uses settings tied to the transaction lifetime. These can be used to get data about the HTTP request. Or to modify the HTTP response.
You can get these with ``current_setting``
.. code-block:: postgresql
-- request settings use the ``request.`` prefix.
SELECT
current_setting('request.<setting>', true);
And you can set them with ``set_config``
.. code-block:: postgresql
-- response settings use the ``response.`` prefix.
SELECT
set_config('response.<setting>', 'value1' ,true);
Request Role and Search Path
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Because of :ref:`user_impersonation`, PostgREST sets the standard ``role``. You can get this in different ways:
.. code-block:: postgresql
SELECT current_role;
SELECT current_user;
SELECT current_setting('role', true);
Additionally it also sets the ``search_path`` based on :ref:`db-schemas` and :ref:`db-extra-search-path`.
.. _guc_req_headers_cookies_claims:
Request Headers, Cookies and JWT claims
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PostgREST stores the headers, cookies and headers as JSON. To get them:
.. code-block:: postgresql
-- To get all the headers sent in the request
SELECT current_setting('request.headers', true)::json;
-- To get a single header, you can use JSON arrow operators
SELECT current_setting('request.headers', true)::json->>'user-agent';
-- value of sessionId in a cookie
SELECT current_setting('request.cookies', true)::json->>'sessionId';
-- value of the email claim in a jwt
SELECT current_setting('request.jwt.claims', true)::json->>'email';
.. note::
The ``role`` in ``request.jwt.claims`` defaults to the value of :ref:`db-anon-role`.
.. _guc_req_path_method:
Request Path and Method
~~~~~~~~~~~~~~~~~~~~~~~
The path and method are stored as ``text``.
.. code-block:: postgresql
SELECT current_setting('request.path', true);
SELECT current_setting('request.method', true);
.. _guc_resp_hdrs:
Response Headers
~~~~~~~~~~~~~~~~
You can set ``response.headers`` to add headers to the HTTP response. For instance, this statement would add caching headers to the response:
.. code-block:: sql
-- tell client to cache response for two days
SELECT set_config('response.headers',
'[{"Cache-Control": "public"}, {"Cache-Control": "max-age=259200"}]', true);
.. code-block:: http
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Cache-Control: no-cache, no-store, must-revalidate
Notice that the ``response.headers`` should be set to an *array* of single-key objects rather than a single multiple-key object. This is because headers such as ``Cache-Control`` or ``Set-Cookie`` need repeating when setting many values. An object would not allow the repeated key.
.. note::
PostgREST provided headers such as ``Content-Type``, ``Location``, etc. can be overriden this way. Note that irrespective of overridden ``Content-Type`` response header, the content will still be converted to JSON, unless you also set :ref:`raw-media-types` to something like ``text/html``.
.. _guc_resp_status:
Response Status Code
~~~~~~~~~~~~~~~~~~~~
You can set the ``response.status`` to override the default status code PostgREST provides. For instance, the following function would replace the default ``200`` status code.
.. code-block:: postgres
create or replace function teapot() returns json as $$
begin
perform set_config('response.status', '418', true);
return json_build_object('message', 'The requested entity body is short and stout.',
'hint', 'Tip it over and pour it out.');
end;
$$ language plpgsql;
.. tabs::
.. code-tab:: http
GET /rpc/teapot HTTP/1.1
.. code-tab:: bash Curl
curl "http://localhost:3000/rpc/teapot" -i
.. code-block:: http
HTTP/1.1 418 I'm a teapot
{
"message" : "The requested entity body is short and stout.",
"hint" : "Tip it over and pour it out."
}
If the status code is standard, PostgREST will complete the status message(**I'm a teapot** in this example).
.. _main_query:
Main query
----------
The main query is produced by requesting the :doc:`API resources <api>`.
Transaction End
---------------
If the transaction doesn't fail, it will always end in a COMMIT. Unless :ref:`db-tx-end` is configured to ROLLBACK in any case or conditionally with ``Prefer: tx=rollback``. This can be used for testing purposes.
Aborting transactions
---------------------
Any database failure(like a failed constraint) will result in a rollback of the transaction. You can also do a RAISE inside a function to cause a rollback.
.. _raise_error:
Raise errors with HTTP Status Codes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can return non-200 HTTP status codes by raising SQL exceptions. For instance, here's a saucy function that always responds with an error:
.. code-block:: postgresql
CREATE OR REPLACE FUNCTION just_fail() RETURNS void
LANGUAGE plpgsql
AS $$
BEGIN
RAISE EXCEPTION 'I refuse!'
USING DETAIL = 'Pretty simple',
HINT = 'There is nothing you can do.';
END
$$;
Calling the function returns HTTP 400 with the body
.. code-block:: json
{
"message":"I refuse!",
"details":"Pretty simple",
"hint":"There is nothing you can do.",
"code":"P0001"
}
One way to customize the HTTP status code is by raising particular exceptions according to the PostgREST :ref:`error to status code mapping <status_codes>`. For example, :code:`RAISE insufficient_privilege` will respond with HTTP 401/403 as appropriate.
For even greater control of the HTTP status code, raise an exception of the ``PTxyz`` type. For instance to respond with HTTP 402, raise 'PT402':
.. code-block:: sql
RAISE sqlstate 'PT402' using
message = 'Payment Required',
detail = 'Quota exceeded',
hint = 'Upgrade your plan';
Returns:
.. code-block:: http
HTTP/1.1 402 Payment Required
Content-Type: application/json; charset=utf-8
{
"message": "Payment Required",
"details": "Quota exceeded",
"hint": "Upgrade your plan",
"code": "PT402"
}
.. _pre-request:
Pre-Request
-----------
The pre-request is a function that can run after the :ref:`tx_settings` are set and before the :ref:`main_query`. It's enabled with :ref:`db-pre-request`.
This provides an opportunity to modify settings or raise an exception to prevent the request from completing.
.. _pre_req_headers:
Setting headers via pre-request
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As an example, let's add some cache headers for all requests that come from an Internet Explorer(6 or 7) browser.
.. code-block:: postgresql
create or replace function custom_headers()
returns void as $$
declare
user_agent text := current_setting('request.headers', true)::json->>'user-agent';
begin
if user_agent similar to '%MSIE (6.0|7.0)%' then
perform set_config('response.headers',
'[{"Cache-Control": "no-cache, no-store, must-revalidate"}]', false);
end if;
end; $$ language plpgsql;
-- set this function on postgrest.conf
-- db-pre-request = custom_headers
Now when you make a GET request to a table or view, you'll get the cache headers.
.. tabs::
.. code-tab:: http
GET /people HTTP/1.1
User-Agent: Mozilla/4.01 (compatible; MSIE 6.0; Windows NT 5.1)
.. code-tab:: bash Curl
curl "http://localhost:3000/people" -i \
-H "User-Agent: Mozilla/4.01 (compatible; MSIE 6.0; Windows NT 5.1)"