From 96c169e593b5ef75a461da5a15a0e6e3fac68602 Mon Sep 17 00:00:00 2001 From: Laurence Isla Date: Fri, 27 May 2022 17:12:47 -0500 Subject: [PATCH] Add bytea to working with types and improve img example (#542) --- docs/how-tos/providing-images-for-img.rst | 78 ++++++------------- .../working-with-postgresql-data-types.rst | 78 +++++++++++++++++++ docs/index.rst | 3 +- postgrest.dict | 1 + 4 files changed, 104 insertions(+), 56 deletions(-) diff --git a/docs/how-tos/providing-images-for-img.rst b/docs/how-tos/providing-images-for-img.rst index 989e19749..f1d892e0d 100644 --- a/docs/how-tos/providing-images-for-img.rst +++ b/docs/how-tos/providing-images-for-img.rst @@ -5,25 +5,18 @@ Providing images for ```` :author: `pkel `_ -In this how-to, you will learn how to create an endpoint for providing images to HTML :code:`` tags without client side JavaScript. -The resulting HTML might look like this: - -.. code-block:: html - - Cute Kittens - -In fact, the presented technique is suitable for providing not only images, but arbitrary files. +In this how-to, you will learn how to create an endpoint for providing images to HTML :code:`` tags without client side JavaScript. In fact, the presented technique is suitable for providing not only images, but arbitrary files. We will start with a minimal example that highlights the general concept. -Afterwards we present are more detailed solution that fixes a few shortcomings of the first approach. +Afterwards we present a more detailed solution that fixes a few shortcomings of the first approach. + +.. warning:: + + Be careful when saving binaries in the database, having a separate storage service for these is preferable in most cases. See `Storing Binary files in the Database `_. Minimal Example --------------- -PostgREST returns binary data on requests that set the :code:`Accept: application/octet-stream` header. -The general idea is to configure the reverse proxy in front of the API to set this header for all requests to :code:`/files/`. -We will show how to achieve this using Nginx. - First, we need a public table for storing the files. .. code-block:: postgres @@ -36,64 +29,40 @@ First, we need a public table for storing the files. Let's assume this table contains an image of two cute kittens with id 42. We can retrieve this image in binary format from our PostgREST API by requesting :code:`/files?select=blob&id=eq.42` with the :code:`Accept: application/octet-stream` header. Unfortunately, putting the URL into the :code:`src` of an :code:`` tag will not work. -That's because browsers do not send the required header. +That's because browsers do not send the required :code:`Accept: application/octet-stream` header. -Luckily, we can configure our :doc:`Nginx reverse proxy <../admin>` to fix this problem for us. -We assume that PostgREST is running on port 3000. -We provide a new location :code:`/files/` that redirects requests to our endpoint with the :code:`Accept` header set to :code:`application/octet-stream`. +Luckily we can specify the accepted media types in the :ref:`raw-media-types` configuration variable. +In this case, the :code:`Accept: image/webp` header is sent by many web browsers by default, so let's add it to the configuration variable, like this: :code:`raw-media-types="image/webp"`. +Now, the image will be displayed in the HTML page: -.. code-block:: nginx +.. code-block:: html - server { - # rest of reverse proxy and web server configuration - ... - - location /files/ { - # /files//* ---> /files?select=blob&id=eq. - rewrite /files/([^/]+).* /files?select=blob&id=eq.$1 break; - # if id is missing - return 404; - # request binary output - proxy_set_header Accept application/octet-stream; - # usual proxy setup - proxy_hide_header Content-Location; - add_header Content-Location /api/$upstream_http_content_location; - proxy_set_header Connection ""; - proxy_http_version 1.1; - proxy_pass http://localhost:3000/; - } - -With this setup, we can request the cat image at :code:`localhost/files/42/cats.jpeg` without setting any headers. -In fact, you can replace :code:`cats.jpeg` with any other filename or simply omit it. -Putting the URL into the :code:`src` of an :code:`` tag should now work as expected. + Cute Kittens Improved Version ---------------- The basic solution has some shortcomings: -1. The response :code:`Content-Type` header is set to :code:`application/octet-stream`. - This might confuse clients and users. -2. Download requests (e.g. Right Click -> Save Image As) to :code:`files/42` will propose :code:`42` as filename. +1. The response :code:`Content-Type` header is set to :code:`image/webp`. + This might be a problem if you want to specify a different format for the file. +2. Download requests (e.g. Right Click -> Save Image As) to :code:`/files?select=blob&id=eq.42` will propose :code:`files` as filename. This might confuse users. 3. Requests to the binary endpoint are not cached. This will cause unnecessary load on the database. The following improved version addresses these problems. -First, we store the media types and names of our files in the database. +First, in addition to the minimal example, we need to store the media types and names of our files in the database. .. code-block:: postgres - create table files( - id int primary key - , type text - , name text - , blob bytea - ); + alter table files + add column type text, + add column name text; Next, we set up an RPC endpoint that sets the content type and filename. We use this opportunity to configure some basic, client-side caching. -For production, you probably want to configure additional caches, e.g. on the reverse proxy. +For production, you probably want to configure additional caches, e.g. on the :ref:`reverse proxy `. .. code-block:: postgres @@ -120,9 +89,8 @@ For production, you probably want to configure additional caches, e.g. on the re end $$ language plpgsql; -With this, we can obtain the cat image from :code:`/rpc/file?id=42`. -Consequently, we have to replace our previous rewrite rule in the Nginx recipe with the following. +With this, we can obtain the cat image from :code:`/rpc/file?id=42`. Thus, the resulting HTML will be: -.. code-block:: nginx +.. code-block:: html - rewrite /files/([^/]+).* /rpc/file?id=$1 break; + Cute Kittens diff --git a/docs/how-tos/working-with-postgresql-data-types.rst b/docs/how-tos/working-with-postgresql-data-types.rst index c439301d0..9cfcf5279 100644 --- a/docs/how-tos/working-with-postgresql-data-types.rst +++ b/docs/how-tos/working-with-postgresql-data-types.rst @@ -5,6 +5,10 @@ Working with PostgreSQL data types PostgREST makes use of PostgreSQL string representations to work with data types. Thanks to this, you can use special values, such as ``now`` for timestamps, ``yes`` for booleans or time values including the time zones. This page describes how you can take advantage of these string representations to perform operations on different PostgreSQL data types. +.. contents:: + :local: + :depth: 1 + Timestamps ---------- @@ -461,6 +465,80 @@ Finally, do the request :ref:`casting the range column `: create cast (mytsrange as json) with function mytsrange_to_json(mytsrange) as assignment; +Bytea +----- + +To send raw binary to PostgREST you need a function with a single unnamed parameter of `bytea type `_. For example, let's create a table that will save some files and a function that inserts data to that table: + +.. code-block:: postgres + + create table files ( + id int primary key generated always as identity, + file bytea + ); + + create function upload_binary(bytea) returns void as $$ + insert into files (file) values ($1); + $$ language sql; + +Next, let's use the PostgREST logo for our test. + +.. code-block:: bash + + curl "https://postgrest.org/en/latest/_images/logo.png" -o postgrest-logo.png + +Now, to send the file ``postgrest-logo.png`` we need to set the ``Content-Type: application/octet-stream`` header in the request: + +.. tabs:: + + .. code-tab:: http + + POST /rpc/upload_binary HTTP/1.1 + Content-Type: application/octet-stream + + postgrest-logo.png + + .. code-tab:: bash Curl + + curl "http://localhost:3000/rpc/upload_binary" \ + -X POST -H "Content-Type: application/octet-stream" \ + --data-binary "@postgrest-logo.png" + +To get the image from the database, you will need to set the ``Accept: application/octet-stream`` header in the request and select only the +``bytea`` column. + +.. tabs:: + + .. code-tab:: http + + GET /files?select=file&id=eq.1 HTTP/1.1 + Accept: application/octet-stream + + .. code-tab:: bash Curl + + curl "http://localhost:3000/files?select=file&id=eq.1" \ + -H "Accept: application/octet-stream" + +You can also use more accurate headers depending on the type of the files by using the :ref:`raw-media-types` configuration. For example, adding the ``raw-media-types="image/png"`` setting to the configuration file will allow you to use the ``Accept: image/png`` header: + +.. tabs:: + + .. code-tab:: http + + GET /files?select=file&id=eq.1 HTTP/1.1 + Accept: image/png + + .. code-tab:: bash Curl + + curl "http://localhost:3000/files?select=file&id=eq.1" \ + -H "Accept: image/png" + +See :ref:`providing_img` for a step-by-step example on how to handle images in HTML. + +.. warning:: + + Be careful when saving binaries in the database, having a separate storage service for these is preferable in most cases. See `Storing Binary files in the Database `_. + hstore ------ diff --git a/docs/index.rst b/docs/index.rst index e2caa8096..c26e2477a 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -203,7 +203,8 @@ These are recipes that'll help you address specific use-cases. :caption: How-to guides :hidden: - how-tos/* + how-tos/working-with-postgresql-data-types + how-tos/providing-images-for-img - :doc:`how-tos/providing-images-for-img` - :doc:`how-tos/working-with-postgresql-data-types` diff --git a/postgrest.dict b/postgrest.dict index 5a3f17271..fe4bf205a 100644 --- a/postgrest.dict +++ b/postgrest.dict @@ -15,6 +15,7 @@ booleans Bouscal buildpack bugfixes +Bytea Cardano cd centric