jimmy/postgrest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reorganize the Schema Cache information into a separate reference page (#404)

Browse Source
This commit is contained in:
laurenceisla
2021-06-18 12:10:31 -05:00
committed by GitHub
parent 326019cca2
commit f7f3aadab8
5 changed files with 143 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
admin.rst
+1 -49
View File
@@ -196,58 +196,10 @@ Restart the database and watch the log file in real-time to understand how HTTP
docker run -v "$(pwd)/init.sh":"/docker-entrypoint-initdb.d/init.sh" -d postgres
docker logs -f <container-id>
.. _schema_reloading:
Schema Reloading
----------------
Users are often confused by PostgREST's database schema cache. It is present because detecting foreign key relationships between tables (including how those relationships pass through views) is necessary, but costly. API requests consult the schema cache as part of :ref:`resource_embedding`. However if the schema changes while the server is running it results in a stale cache and leads to errors claiming that no relations are detected between tables.
.. important::
Since v5.0, PostgREST also makes use of the schema cache for stored functions metadata: parameters, return type, volatility.
It also uses the schema cache for resolving overloaded functions. You should refresh the cache if a change in any of the prior is done.
To refresh the cache without restarting the PostgREST server, send the server process a SIGUSR1 signal:
.. code:: bash
killall -SIGUSR1 postgrest
.. note::
To refresh the cache in docker:
.. code:: bash
docker kill -s SIGUSR1 <container>
# or in docker-compose
docker-compose kill -s SIGUSR1 <service>
The above is the manual way to do it. To automate the schema reloads, use a database trigger like this:
.. code-block:: postgresql
CREATE OR REPLACE FUNCTION public.notify_ddl_postgrest()
RETURNS event_trigger
LANGUAGE plpgsql
AS $$
BEGIN
NOTIFY ddl_command_end;
END;
$$;
CREATE EVENT TRIGGER ddl_postgrest ON ddl_command_end
EXECUTE PROCEDURE public.notify_ddl_postgrest();
Then run the `pg_listen <https://github.com/begriffs/pg_listen>`_ utility to monitor for that event and send a SIGUSR1 when it occurs:
.. code-block:: bash
pg_listen <db-uri> ddl_command_end $(which killall) -SIGUSR1 postgrest
Now, whenever the structure of the database schema changes, PostgreSQL will notify the ``ddl_command_end`` channel, which will cause ``pg_listen`` to send PostgREST the signal to reload its cache. Note that pg_listen requires full path to the executable in the example above.
Changing the schema while the server is running can lead to errors due to a stale schema cache. To learn how to refresh the cache see :ref:`schema_reloading`.
Daemonizing
===========
api.rst
+2 -12
View File
@@ -990,6 +990,8 @@ In this case, only **source**, **publication_date** and **figure** will be inser
Using this also has the side-effect of being more efficient for :ref:`bulk_insert` since PostgREST will not process the JSON and
it'll send it directly to PostgreSQL.
.. _upsert:
UPSERT
------
@@ -1109,18 +1111,6 @@ For instance, assume we have created this function in the database.
Whenever you create or change a function you must refresh PostgREST's schema cache. See the section :ref:`schema_reloading`.
If the schema cache is not refreshed, PostgREST will assume :code:`text` as the default type for function arguments. This could
lead to getting error responses like:
.. code-block:: json
{
"hint":"No function matches the given name and argument types. You might need to add explicit type casts.",
"details":null,
"code":"42883",
"message":"function test.add_them(a => text, b => text) does not exist"
}
The client can call it by posting an object like
.. code-block:: http
index.rst
+7
View File
@@ -127,8 +127,15 @@ Technical references for PostgREST's functionality.
configuration.rst
.. toctree::
:caption: Schema Cache
:hidden:
schema_cache.rst
- :doc:`API <api>`
- :doc:`configuration`
- :doc:`Schema Cache <schema_cache>`
Topic guides
------------
releases/upcoming.rst
+2
View File
@@ -19,6 +19,8 @@ Added
* Documentation improvements
+ Added the :ref:`OPTIONS requests <options_requests>` section.
+ Added the :ref:`schema_cache` section.
+ Moved the :ref:`schema_reloading` reference from :ref:`admin` to :ref:`schema_cache`
Fixed
-----
schema_cache.rst
+131
View File
@@ -0,0 +1,131 @@
.. _schema_cache:
Schema Cache
============
PostgREST caches metadata from the database schema to avoid repeating expensive queries. This metadata is not required by all of the PostgREST features, only the following:
+--------------------------------------------+-------------------------------------------------------------------------------+
| Feature | Required Metadata |
+============================================+===============================================================================+
| :ref:`resource_embedding` | Foreign key constraints |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`Stored Functions <s_procs>` | Function signature (parameters, return type, volatility and |
| | `overloading <https://www.postgresql.org/docs/current/xfunc-overload.html>`_) |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`Upserts <upsert>` | Primary keys |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`Insertions <insert_update>` | Primary keys (optional: only if the Location header is requested) |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`OPTIONS requests <options_requests>` | View INSTEAD OF TRIGGERS and primary keys |
+--------------------------------------------+-------------------------------------------------------------------------------+
| :ref:`open-api` | Table columns, primary keys and foreign keys |
+ +-------------------------------------------------------------------------------+
| | View columns and INSTEAD OF TRIGGERS |
+ +-------------------------------------------------------------------------------+
| | Function signature |
+--------------------------------------------+-------------------------------------------------------------------------------+
The Stale Schema Cache
----------------------
When you make changes on the metadata mentioned above, the schema cache will turn stale on a running PostgREST. Future requests that use the above features will need the :ref:`schema cache to be reloaded <schema_reloading>`; otherwise, you'll get an error instead of the expected result.
For instance, let's see what would happen if you have a stale schema for foreign key relationships and function signature:
Stale Foreign Key Relationships
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Suppose you add a ``cities`` table to your database. This table has a foreign key referencing an existing ``countries`` table. Then, you make a request to get the ``cities`` and their belonging ``countries``:
.. code-block:: http
GET /cities?select=name,country:countries(id,name) HTTP/1.1
But instead, you get an error message that looks like this:
.. code-block:: json
{
"hint": "If a new foreign key between these entities was created in the database, try reloading the schema cache.",
"message": "Could not find a relationship between cities and countries in the schema cache"
}
As you can see, PostgREST couldn't find the newly created foreign key in the schema cache. See the section :ref:`schema_reloading` to solve this issue.
Stale Function Signature
~~~~~~~~~~~~~~~~~~~~~~~~
Suppose you create the following function while PostgREST is running:
.. code-block:: plpgsql
CREATE FUNCTION plus_one(num integer)
RETURNS integer AS $$
SELECT num + 1;
$$ LANGUAGE SQL IMMUTABLE;
Then, you make this request:
.. code-block:: http
GET /rpc/plus_one?num=1 HTTP/1.1
On a stale schema, PostgREST will assume :code:`text` as the default type for the function argument ``num``. Thus, the response you get is:
.. code-block:: json
{
"hint":"No function matches the given name and argument types. You might need to add explicit type casts.",
"details":null,
"code":"42883",
"message":"function test.plus_one(num => text) does not exist"
}
See the section :ref:`schema_reloading` to solve this issue.
.. _schema_reloading:
Schema Cache Reloading
----------------------
To refresh the cache without restarting the PostgREST server, send the server process a SIGUSR1 signal:
.. code:: bash
killall -SIGUSR1 postgrest
.. note::
To refresh the cache in docker:
.. code:: bash
docker kill -s SIGUSR1 <container>
# or in docker-compose
docker-compose kill -s SIGUSR1 <service>
The above is the manual way to do it. To automate cache reloads, use a database trigger like this:
.. code-block:: postgresql
CREATE OR REPLACE FUNCTION public.notify_ddl_postgrest()
RETURNS event_trigger
LANGUAGE plpgsql
AS $$
BEGIN
NOTIFY ddl_command_end;
END;
$$;
CREATE EVENT TRIGGER ddl_postgrest ON ddl_command_end
EXECUTE PROCEDURE public.notify_ddl_postgrest();
Then run the `pg_listen <https://github.com/begriffs/pg_listen>`_ utility to monitor for that event and send a SIGUSR1 when it occurs:
.. code-block:: bash
pg_listen <db-uri> ddl_command_end $(which killall) -SIGUSR1 postgrest
Now, whenever the structure of the database changes, PostgreSQL will notify the ``ddl_command_end`` channel, which will cause ``pg_listen`` to send PostgREST the signal to reload its cache. Note that pg_listen requires full path to the executable in the example above.