Remove integrations section
This commit is contained in:
steve-chavez
committed by
Steve Chávez
Steve Chávez
parent
bed1013ecd
commit
abdc2ac2a9
@@ -112,12 +112,6 @@ Translations
|
||||
how-tos/embedding-table-from-another-schema.rst
|
||||
how-tos/casting-type-to-custom-json.rst
|
||||
|
||||
.. toctree::
|
||||
:caption: Integrations
|
||||
:titlesonly:
|
||||
|
||||
integrations/timescaledb.rst
|
||||
|
||||
.. toctree::
|
||||
:caption: Installation
|
||||
:titlesonly:
|
||||
|
||||
@@ -1,327 +0,0 @@
|
||||
TimescaleDB for Time-Series Data
|
||||
================================
|
||||
|
||||
`TimescaleDB <https://www.timescale.com?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=tutorial>`_ is an open-source database designed to make SQL scalable for time-series data. It is engineered up from PostgreSQL, providing automatic partitioning across time and space, while retaining the standard PostgreSQL interface.
|
||||
|
||||
PostgREST turns your PostgreSQL database directly into a RESTful API, since TimescaleDB is packaged as a PostgreSQL extension it works with PostgREST as well.
|
||||
|
||||
In this tutorial we'll explore some of TimescaleDB features through PostgREST.
|
||||
|
||||
Install Docker
|
||||
--------------
|
||||
|
||||
For an easier setup we're going to use `Docker <https://www.docker.com/get-started>`_, make sure you have it installed.
|
||||
|
||||
Run TimescaleDB
|
||||
---------------
|
||||
|
||||
First, let’s pull and start the `TimescaleDB container image <http://bit.ly/2SpxiYJ>`_:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run --name tsdb_tut \
|
||||
-e POSTGRES_PASSWORD=mysecretpassword \
|
||||
-p 5433:5432 \
|
||||
-d timescale/timescaledb:latest-pg11
|
||||
|
||||
This will run the container as a daemon and expose port ``5433`` to the host system so that it doesn't conflict with another PostgreSQL installation.
|
||||
|
||||
Set up TimescaleDB
|
||||
------------------
|
||||
|
||||
Now, we'll create the ``timescaledb`` extension in our database.
|
||||
|
||||
Run ``psql`` in the container we created in the previous step.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker exec -it tsdb_tut psql -U postgres
|
||||
## Run all the following commands inside psql
|
||||
|
||||
And create the extension:
|
||||
|
||||
.. code-block:: postgres
|
||||
|
||||
create extension if not exists timescaledb cascade;
|
||||
|
||||
Create an Hypertable
|
||||
--------------------
|
||||
|
||||
`Hypertables <https://docs.timescale.com/latest/using-timescaledb/hypertables?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=hypertables>`_ are the core abstraction TimescaleDB offers for dealing with time-series data.
|
||||
|
||||
To create an ``hypertable``, first we need to create standard PostgreSQL tables:
|
||||
|
||||
.. code-block:: postgres
|
||||
|
||||
create table if not exists locations(
|
||||
device_id text primary key
|
||||
, location text
|
||||
, environment text
|
||||
);
|
||||
|
||||
create table if not exists conditions(
|
||||
time timestamp with time zone not null
|
||||
, device_id text references locations(device_id)
|
||||
, temperature numeric
|
||||
, humidity numeric
|
||||
);
|
||||
|
||||
Now, we'll convert ``conditions`` into an hypertable with `create_hypertable <http://docs.timescale.com/latest/api?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=create-hypertable#create_hypertable>`_:
|
||||
|
||||
.. code-block:: postgres
|
||||
|
||||
SELECT create_hypertable('conditions', 'time', chunk_time_interval => interval '1 day');
|
||||
-- This also implicitly creates an index: CREATE INDEX ON "conditions"(time DESC);
|
||||
|
||||
-- Exit psql
|
||||
exit
|
||||
|
||||
|
||||
Load sample data
|
||||
----------------
|
||||
|
||||
To have some data to play with, we'll download the ``weather_small`` data set from `TimescaleDB's sample datasets <https://docs.timescale.com/latest/tutorials/other-sample-datasets?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=datasets>`_.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
## Run bash inside the database container
|
||||
docker exec -it tsdb_tut bash
|
||||
|
||||
## Download and uncompress the data
|
||||
wget -qO- https://timescaledata.blob.core.windows.net/datasets/weather_small.tar.gz | tar xvz
|
||||
|
||||
## Copy data into the database
|
||||
psql -U postgres <<EOF
|
||||
\COPY locations FROM weather_small_locations.csv CSV
|
||||
\COPY conditions FROM weather_small_conditions.csv CSV
|
||||
EOF
|
||||
|
||||
## Exit bash
|
||||
exit
|
||||
|
||||
Run PostgREST
|
||||
-------------
|
||||
|
||||
For the last step in the setup, pull and start the official `PostgREST image <https://hub.docker.com/r/postgrest/postgrest/>`_:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run --rm -p 3000:3000 \
|
||||
--name tsdb_pgrst \
|
||||
--link tsdb_tut \
|
||||
-e PGRST_DB_URI="postgres://postgres:mysecretpassword@tsdb_tut/postgres" \
|
||||
-e PGRST_DB_ANON_ROLE="postgres" \
|
||||
-d postgrest/postgrest:latest
|
||||
|
||||
PostgREST on Hypertables
|
||||
------------------------
|
||||
|
||||
We'll now see how to read data from hypertables through PostgREST.
|
||||
|
||||
Since hypertables can be queried using standard `SELECT statements <https://docs.timescale.com/v1.2/using-timescaledb/reading-data?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=reading-data>`_, we can query them through PostgREST normally.
|
||||
|
||||
Suppose we want to run this query on ``conditions``:
|
||||
|
||||
.. code-block:: postgres
|
||||
|
||||
select
|
||||
time,
|
||||
device_id,
|
||||
humidity
|
||||
from conditions
|
||||
where
|
||||
humidity > 90 and
|
||||
time < '2016-11-16'
|
||||
order by time desc
|
||||
limit 10;
|
||||
|
||||
Using PostgREST :ref:`horizontal <h_filter>`/:ref:`vertical <v_filter>` filtering, this query can be expressed as:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
curl -G "localhost:3000/conditions" \
|
||||
-d select=time,device_id,humidity \
|
||||
-d humidity=gt.90 \
|
||||
-d time=lt.2016-11-16 \
|
||||
-d order=time.desc \
|
||||
-d limit=10
|
||||
## This command is equivalent to:
|
||||
## curl "localhost:3000/conditions?select=time,device_id,humidity&humidity=gt.90&time=lt.2016-11-16&order=time.desc&limit=10"
|
||||
## Here we used -G and -d to make the command more readable
|
||||
|
||||
The response will be:
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
[{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000982","humidity":90.90000000000006},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000968","humidity":92.3},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000963","humidity":96.29999999999993},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000951","humidity":94.39999999999998},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000950","humidity":93.69999999999982},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000915","humidity":94.69999999999997},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000911","humidity":93.2000000000001},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000910","humidity":91.30000000000017},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000901","humidity":92.30000000000005},
|
||||
{"time":"2016-11-15T23:58:00+00:00","device_id":"weather-pro-000895","humidity":91.00000000000014}]
|
||||
|
||||
JOINs with relational tables
|
||||
----------------------------
|
||||
|
||||
Hypertables support all standard `PostgreSQL constraints <https://docs.timescale.com/latest/using-timescaledb/schema-management?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=constraints#constraints>`_ . We can make use of the foreign key defined on ``locations`` to make a JOIN through PostgREST. A query such as:
|
||||
|
||||
.. code-block:: postgres
|
||||
|
||||
select
|
||||
c.time,
|
||||
c.temperature,
|
||||
l.location,
|
||||
l.environment
|
||||
from conditions c
|
||||
left join locations l on
|
||||
c.device_id = l.device_id
|
||||
order by time desc
|
||||
limit 10;
|
||||
|
||||
Can be expressed in PostgREST by using :ref:`resource_embedding`.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
curl -G localhost:3000/conditions \
|
||||
-d select="time,temperature,device:locations(location,environment)" \
|
||||
-d order=time.desc \
|
||||
-d limit=10
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
[{"time":"2016-11-16T21:18:00+00:00","temperature":69.49999999999991,"device":{"location":"office-000202","environment":"inside"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":90,"device":{"location":"field-000205","environment":"outside"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":60.499999999999986,"device":{"location":"door-00085","environment":"doorway"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":91,"device":{"location":"swamp-000188","environment":"outside"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":42,"device":{"location":"arctic-000219","environment":"outside"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":70.80000000000003,"device":{"location":"office-000201","environment":"inside"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":62.699999999999974,"device":{"location":"door-00084","environment":"doorway"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":85.49999999999918,"device":{"location":"field-000204","environment":"outside"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":42,"device":{"location":"arctic-000218","environment":"outside"}},
|
||||
{"time":"2016-11-16T21:18:00+00:00","temperature":42,"device":{"location":"arctic-000217","environment":"outside"}}]
|
||||
|
||||
Time-Oriented Analytics
|
||||
-----------------------
|
||||
|
||||
TimescaleDB includes new aggregate functions for time-oriented `analytics <https://docs.timescale.com/latest/api?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=analytics#analytics>`_.
|
||||
|
||||
For using aggregate queries with PostgREST you must create VIEWs or :ref:`s_procs`. Here's an example for using `time_bucket <https://docs.timescale.com/latest/api?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=time-bucket#time_bucket>`_:
|
||||
|
||||
.. code-block:: postgres
|
||||
|
||||
-- Run psql in the database container
|
||||
docker exec -it tsdb_tut psql -U postgres
|
||||
|
||||
-- Create the function
|
||||
create or replace function temperature_summaries(gap interval default '1 hour', prefix text default 'field')
|
||||
returns table(hour text, avg_temp numeric, min_temp numeric, max_temp numeric) as $$
|
||||
select
|
||||
time_bucket(gap, time)::text as hour,
|
||||
trunc(avg(temperature), 2),
|
||||
trunc(min(temperature), 2),
|
||||
trunc(max(temperature), 2)
|
||||
from conditions c
|
||||
where c.device_id in (
|
||||
select device_id from locations
|
||||
where location like prefix || '-%')
|
||||
group by hour
|
||||
$$ language sql stable;
|
||||
|
||||
-- Exit psql
|
||||
exit
|
||||
|
||||
Every time the schema is changed you must reload PostgREST :ref:`schema cache <schema_reloading>` so it can pick up the function parameters correctly. To reload, run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker kill --signal=USR1 tsdb_pgrst
|
||||
|
||||
|
||||
Now, since the function is ``stable``, we can call it with ``GET`` as:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
curl -G "localhost:3000/rpc/temperature_summaries" \
|
||||
-d gap=2minutes \
|
||||
-d order=hour.asc \
|
||||
-d limit=10 \
|
||||
-H "Accept: text/csv"
|
||||
## time_bucket accepts an interval type as it's argument
|
||||
## so you can pass gap=5minutes or gap=5hours
|
||||
|
||||
.. code-block:: sql
|
||||
|
||||
hour,avg_temp,min_temp,max_temp
|
||||
"2016-11-15 12:00:00+00",72.97,68.00,78.00
|
||||
"2016-11-15 12:02:00+00",73.01,68.00,78.00
|
||||
"2016-11-15 12:04:00+00",73.05,68.00,78.10
|
||||
"2016-11-15 12:06:00+00",73.07,68.00,78.10
|
||||
"2016-11-15 12:08:00+00",73.11,68.00,78.10
|
||||
"2016-11-15 12:10:00+00",73.14,68.00,78.10
|
||||
"2016-11-15 12:12:00+00",73.17,68.00,78.19
|
||||
"2016-11-15 12:14:00+00",73.21,68.10,78.19
|
||||
"2016-11-15 12:16:00+00",73.24,68.10,78.29
|
||||
"2016-11-15 12:18:00+00",73.27,68.10,78.39
|
||||
|
||||
Note you can use PostgREST standard filtering on function results. Here we also changed the :ref:`res_format` to CSV.
|
||||
|
||||
Fast Ingestion with Bulk Insert
|
||||
-------------------------------
|
||||
|
||||
You can use PostgREST :ref:`bulk_insert` to leverage TimescaleDB `fast ingestion <https://docs.timescale.com/latest/introduction/timescaledb-vs-postgres?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=fast-ingest>`_.
|
||||
|
||||
Let's do an insert of three rows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
curl "localhost:3000/conditions" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Prefer: return=representation" \
|
||||
-d @- << EOF
|
||||
[
|
||||
{"time": "2019-02-21 01:00:01-05", "device_id": "weather-pro-000000", "temperature": 40.0, "humidity": 59.9},
|
||||
{"time": "2019-02-21 01:00:02-05", "device_id": "weather-pro-000000", "temperature": 42.0, "humidity": 69.9},
|
||||
{"time": "2019-02-21 01:00:03-05", "device_id": "weather-pro-000000", "temperature": 44.0, "humidity": 79.9}
|
||||
]
|
||||
EOF
|
||||
|
||||
By using the ``Prefer: return=representation`` header we can see the successfully inserted rows:
|
||||
|
||||
.. code-block:: json
|
||||
|
||||
[{"time":"2019-02-21T06:00:01+00:00","device_id":"weather-pro-000000","temperature":40.0,"humidity":59.9},
|
||||
{"time":"2019-02-21T06:00:02+00:00","device_id":"weather-pro-000000","temperature":42.0,"humidity":69.9},
|
||||
{"time":"2019-02-21T06:00:03+00:00","device_id":"weather-pro-000000","temperature":44.0,"humidity":79.9}]
|
||||
|
||||
Let's now insert a thousand rows, we'll use `jq <https://stedolan.github.io/jq/>`_ for constructing the array.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
yes "{\"time\": \"$(date +'%F %T')\", \"device_id\": \"weather-pro-000001\", \"temperature\": 50, \"humidity\": 60}" | \
|
||||
head -n 1000 | jq -s '.' | \
|
||||
curl -i -d @- "http://localhost:3000/conditions" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Prefer: count=exact"
|
||||
|
||||
With ``Prefer: count=exact`` we can know how many rows were inserted. Check out the response:
|
||||
|
||||
.. code-block:: haskell
|
||||
|
||||
HTTP/1.1 201 Created
|
||||
Transfer-Encoding: chunked
|
||||
Date: Fri, 22 Feb 2019 16:47:05 GMT
|
||||
Server: postgrest/5.2.0 (9969262)
|
||||
Content-Range: */1000
|
||||
|
||||
You can see in ``Content-Range`` that the total number of inserted rows is ``1000``.
|
||||
|
||||
Summing it up
|
||||
-------------
|
||||
|
||||
There you have it, with PostgREST you can get an instant and performant RESTful API for a TimescaleDB database.
|
||||
|
||||
For a more in depth exploration of TimescaleDB capabilities, check their `docs <https://docs.timescale.com?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=docs-tutorial>`_.
|
||||
Reference in New Issue
Block a user