Documentation outline

2015-10-19 16:45:52 -07:00
parent d3eca26393
commit 20198367da
17 changed files with 632 additions and 0 deletions
@@ -8,3 +8,4 @@ codex.tags
 .anvil
 .stack-work
 tags
+site
@@ -0,0 +1 @@
+postgrest.com
@@ -0,0 +1,9 @@
+## Deployment
+
+### Heroku
+
+#### Getting Started
+
+#### Using Amazon RDS
+
+### Debian
@@ -0,0 +1,9 @@
+## Data Migration
+
+### Sqitch
+
+### Test-Driven Migrations
+
+#### Structural Tests
+
+#### Value Tests with pgTAP
@@ -0,0 +1,9 @@
+## Performance
+
+### Benchmarks
+
+### Caching
+
+### Quality of Service
+
+### Tips
@@ -0,0 +1,21 @@
+## Security
+
+### SSL
+
+### Database Roles
+
+### JSON Web Tokens
+
+#### Issuing via sql procedures
+
+### Row-Level Security
+
+#### Simulated - PostgreSQL <9.5
+
+#### Real - PostgreSQL >=9.5
+
+### Building Auth on top of JWT
+
+#### Basic Auth
+
+#### Github Sign-in
@@ -0,0 +1,9 @@
+## API Versioning
+
+### Schema Search Path
+
+### Changing a Resource
+
+### Removing a Resource
+
+### Avoiding DB and Client Coupling
@@ -0,0 +1,29 @@
+## Requesting Information
+
+### Tables and Views
+
+### Stored Procedures
+
+### Filtering
+
+#### Computed Columns
+
+#### Inside JSONB
+
+### Ordering
+
+### Limiting and Pagination
+
+#### Pagination by Limit-Offset
+
+#### Suppressing Counts
+
+### Embedding Foreign Keys
+
+### Response Format
+
+### Singular vs Plural
+
+### Data Schema
+
+### CORS
@@ -0,0 +1,13 @@
+## Updating Data
+
+### Record Creation
+
+### Bulk Insertion
+
+### Upsertion
+
+### Bulk Updates
+
+### Deletion
+
+### Protecting Dangerous Actions
@@ -0,0 +1,346 @@
+## Getting Started
+
+### Your First (simple) API
+
+Let's start with the simplest thing possible. We will expose some tables directly for reading and writing by anyone.
+
+Start by making a database
+
+```sh
+createdb demo1
+```
+
+We'll set it up with a film example (courtesy of [Jonathan Harrington](http://blog.jonharrington.org/postgrest-introduction/)). Copy the following into your clipboard:
+
+```sql
+BEGIN;
+
+CREATE TABLE director
+(
+  name text NOT NULL PRIMARY KEY
+);
+
+CREATE TABLE film
+(
+  id serial PRIMARY KEY,
+  title text NOT NULL,
+  year date NOT NULL,
+  director text,
+  rating real NOT NULL DEFAULT 0,
+  language text NOT NULL,
+  CONSTRAINT film_director_fkey FOREIGN KEY (director)
+      REFERENCES director (name) MATCH SIMPLE
+      ON UPDATE CASCADE ON DELETE CASCADE
+);
+
+CREATE TABLE festival
+(
+  name text NOT NULL PRIMARY KEY
+);
+
+CREATE TABLE competition
+(
+  id serial PRIMARY KEY,
+  name text NOT NULL,
+  festival text NOT NULL,
+  year date NOT NULL,
+
+  CONSTRAINT comp_festival_fkey FOREIGN KEY (festival)
+      REFERENCES festival (name) MATCH SIMPLE
+      ON UPDATE CASCADE ON DELETE CASCADE
+);
+
+CREATE TABLE film_nomination
+(
+  id serial PRIMARY KEY,
+  competition integer NOT NULL,
+  film integer NOT NULL,
+  won boolean NOT NULL DEFAULT true,
+
+  CONSTRAINT nomination_competition_fkey FOREIGN KEY (competition)
+     REFERENCES competition (id) MATCH SIMPLE
+     ON UPDATE NO ACTION ON DELETE NO ACTION,
+  CONSTRAINT nomination_film_fkey FOREIGN KEY (film)
+     REFERENCES film (id) MATCH SIMPLE
+     ON UPDATE CASCADE ON DELETE CASCADE
+);
+
+COMMIT;
+```
+
+Apply it to your new database by running
+
+```sh
+# On OS X
+pbpaste | psql demo1
+
+# Or Linux
+# xclip -selection clipboard -o | psql demo1
+```
+
+Start the PostgREST server and point it at the new database.
+
+```sh
+postgrest -d demo1 -U postgres -a postgres --v1schema public
+```
+
+<div class="admonition note">
+    <p class="admonition-title">Note about database users</p>
+
+    <p>If you installed PostgreSQL with Homebrew on Mac then the
+    database username may be your own login rather than
+    <code>postgres</code>.</p>
+</div>
+
+Let's use PostgREST to populate the database. Install a REST client such as [Postman](https://chrome.google.com/webstore/detail/postman/fhbjgbiflinjbdggehcddcbncdddomop?hl=en). Now let's insert some data as a bulk post in CSV format:
+
+```HTTP
+POST http://localhost:3000/festival
+Content-Type: text/csv
+
+name
+Venice Film Festival
+Cannes Film Festival
+```
+
+In Postman it will look like this
+
+![Festival bulk insert in postman](/img/post-festivals.png)
+
+Notice that the post type is `raw` and that `Content-Type: text/csv` set in the Headers tab. 
+
+Note that the server returns a multipart response with URL of each created resource.
+
+```HTTP
+Content-Type: application/json
+Location: /festival?name=eq.Venice%20Film%20Festival
+
+
+--postgrest_boundary
+Content-Type: application/json
+Location: /festival?name=eq.Cannes%20Film%20Festival
+```
+
+If you send a GET request to `/festival` it should return
+
+```json
+[
+  {
+    "name": "Venice Film Festival"
+  },
+  {
+    "name": "Cannes Film Festival"
+  }
+]
+```
+
+Now that you've seen how to do a bulk insert, let's do some more and fully populate the database.
+
+Post the following to `/competition`:
+
+```csv
+name,festival,year
+Golden Lion,Venice Film Festival,2014-01-01
+Palme d'Or,Cannes Film Festival,2014-01-01
+```
+
+Now `/director`:
+
+```csv
+name
+Bertrand Bonello
+Atom Egoyan
+David Gordon Green
+Andrey Konchalovskiy
+Mario Martone
+Mike Leigh
+Roy Andersson
+Saverio Costanzo
+Alix Delaporte
+Jean-Pierre Dardenne
+Xiaoshuai Wang
+Kaan Müjdeci
+Tommy Lee Jones
+Nuri Bilge Ceylan
+Michel Hazanavicius
+Xavier Dolan
+Ramin Bahrani
+Alice Rohrwacher
+Andrew Niccol
+Rakhshan Bani-Etemad
+David Oelhoffen
+Bennett Miller
+David Cronenberg
+Shin'ya Tsukamoto
+Joshua Oppenheimer
+Olivier Assayas
+Jean-Luc Godard
+Alejandro González Iñárritu
+Benoît Jacquot
+Fatih Akin
+Francesco Munzi
+Ken Loach
+Abel Ferrara
+Xavier Beauvois
+Naomi Kawase
+```
+
+And `/film`:
+
+```csv
+title,year,director,rating,language
+Chuang ru zhe,2014-01-01,Xiaoshuai Wang,6.19999981,english
+The Look of Silence,2014-01-01,Joshua Oppenheimer,8.30000019,Indonesian
+Fires on the Plain,2014-01-01,Shin'ya Tsukamoto,5.80000019,Japanese
+Far from Men,2014-01-01,David Oelhoffen,7.5,english
+Good Kill,2014-01-01,Andrew Niccol,6.0999999,english
+Leopardi,2014-01-01,Mario Martone,6.9000001,english
+Sivas,2014-01-01,Kaan Müjdeci,7.69999981,english
+Black Souls,2014-01-01,Francesco Munzi,7.0999999,english
+Three Hearts,2014-01-01,Benoît Jacquot,5.80000019,French
+Pasolini,2014-01-01,Abel Ferrara,5.80000019,english
+Le dernier coup de marteau,2014-01-01,Alix Delaporte,6.5,english
+Manglehorn,2014-01-01,David Gordon Green,7.0999999,english
+Hungry Hearts,2014-01-01,Saverio Costanzo,6.4000001,English
+Belye nochi pochtalona Alekseya Tryapitsyna,2014-01-01,Andrey Konchalovskiy,6.9000001,Russian
+99 Homes,2014-01-01,Ramin Bahrani,7.30000019,english
+The Cut,2014-01-01,Fatih Akin,6,Armenian
+Birdman: Or (The Unexpected Virtue of Ignorance),2014-01-01,Alejandro González Iñárritu,8,English
+La rançon de la gloire,2014-01-01,Xavier Beauvois,5.69999981,French
+A Pigeon Sat on a Branch Reflecting on Existence,2014-01-01,Roy Andersson,7.19999981,english
+Tales,2014-01-01,Rakhshan Bani-Etemad,6.80000019,english
+The Wonders,2014-01-01,Alice Rohrwacher,6.80000019,Italian
+Foxcatcher,2014-01-01,Bennett Miller,7.19999981,English
+Mr. Turner,2014-01-01,Mike Leigh,7,English
+Jimmy's Hall,2014-01-01,Ken Loach,6.69999981,English
+The Homesman,2014-01-01,Tommy Lee Jones,6.5999999,English
+The Captive,2014-01-01,Atom Egoyan,5.9000001,english
+Goodbye to Language,2014-01-01,Jean-Luc Godard,6.19999981,French
+The Search,2014-01-01,Michel Hazanavicius,6.9000001,French
+Still the Water,2014-01-01,Naomi Kawase,6.9000001,Japanese
+Mommy,2014-01-01,Xavier Dolan,8.30000019,French
+"Two Days, One Night",2014-01-01,Jean-Pierre Dardenne,7.4000001,French
+Maps to the Stars,2014-01-01,David Cronenberg,6.4000001,English
+Saint Laurent,2014-01-01,Bertrand Bonello,6.5,French
+Clouds of Sils Maria,2014-01-01,Olivier Assayas,6.9000001,english
+Winter Sleep,2014-01-01,Nuri Bilge Ceylan,8.5,Turkish
+```
+
+Finally `/film_nomination`:
+
+```csv
+competition,film,won
+1,1,f
+1,2,f
+1,3,f
+1,4,f
+1,5,f
+1,6,f
+1,7,f
+1,8,f
+1,9,f
+1,10,f
+1,11,f
+1,12,f
+1,13,f
+1,14,f
+1,15,f
+1,16,f
+1,17,f
+1,18,f
+1,19,f
+1,20,f
+2,21,f
+2,22,f
+2,23,f
+2,24,f
+2,25,f
+2,26,f
+2,27,f
+2,28,f
+2,29,f
+2,30,f
+2,31,f
+2,32,f
+2,33,f
+2,34,f
+2,35,f
+```
+
+At this point nominations are fully specified but it's not a convenient interface for a rest client. Let's make a view they can use. Paste this into `psql demo1`.
+
+```sql
+create or replace view nomination as
+select comp.festival,  
+       comp.name as competition,
+       comp.year,
+       film.title,
+       film.director,
+       film.rating
+ from film_nomination as nom
+ left join film on nom.film = film.id
+ left join competition as comp on nom.competition = comp.id
+ order by comp.year desc, comp.festival, competition;
+```
+
+Time to try it out. Let's get the contents of the new view, ordered by film rating
+
+```
+GET http://localhost:3000/nomination?order=rating.desc
+```
+
+If you find it more human readable, add an `Accept: text/csv` header.
+
+### Releasing a New Version
+
+Suppose we want this endpoint to cater to those moviegoers with attention deficit disorder. In today's busy world we don't have time to read an extra couple words or compare nuanced reviews. In API version two we will truncate the names and round the ratings!
+
+Each version lives in a numbered schema, so let's make a schema for version two.
+
+```sql
+CREATE SCHEMA "2";
+GRANT USAGE ON SCHEMA "2" TO PUBLIC;
+ALTER DATABASE demo1 SET search_path = "2", "public";
+```
+
+To override the `films` endpoint create a view in the "2" schema with that name:
+
+```sql
+create or replace view "2".film as
+select id, substring(f.title from 1 for 10) as title,
+       year, director, round(f.rating) as rating, language
+from "public".film as f;
+```
+
+We select the desired version as part of content negotiation. Try this get request:
+
+```HTTP
+GET http://localhost:3000/film
+Accept: text/csv; version=2
+```
+
+Then try toggling the version string in the Accept header and watch the results change. Pretty good, now how about writing values? PostgreSQL's nice feature called auto-updatable views allows writes to pass through views. Sadly this view is not eligible because truncation and rounding cannot be uniquely reversed. If we attempt to post a new result it complains:
+
+```json
+{
+  "hint": null,
+  "details": "View columns that are not columns of their base relation are not updatable.",
+  "code": "0A000",
+  "message": "cannot insert into column \"title\" of view \"film\""
+}
+```
+
+This is a case where we need explicit triggers
+
+```sql
+-- TODO - FIX THIS
+
+-- CREATE OR REPLACE RULE insert_v2_films AS
+--   ON INSERT TO "2".film
+--   DO INSTEAD
+--      INSERT INTO public.film (id, title, year, director, rating, language)
+--      VALUES (NEW.id,     NEW.title,
+--              NEW.year,   NEW.director,
+--              NEW.rating, NEW.language)
+--      RETURNING public.film.*;
+```
@@ -0,0 +1,68 @@
+![PostgREST logo](img/logo.png)
+
+## Introduction
+
+PostgREST is a standalone web server that turns your database directly into a RESTful API. The structural constraints and permissions in the database determine the API endpoints and operations.
+
+This guide explains how to install the software and provides practical examples of its use. You'll learn how to build a fast, versioned, secure API and how to deploy it to production.
+
+The project has a friendly and growing community. Here are some ways to get help or get involved:
+
+* The project [chat room](https://gitter.im/begriffs/postgrest)
+* Report or search [issues](https://github.com/begriffs/postgrest/issues)
+
+### Motivation
+
+Using PostgREST is an alternative to manual CRUD programming. Custom API servers suffer problems. Writing business logic often duplicates, ignores or hobbles database structure. Object-relational mapping is a leaky abstraction leading to slow imperative code. The PostgREST philosophy establishes a single declarative source of truth: the data itself.
+
+#### Declarative Programming
+
+It's easier to ask Postgres to join data for you and let its query planner figure out the details than to loop through rows yourself. It's easier to assign permissions to db objects than to add guards in controllers. (This is especially true for cascading permissions in data dependencies.) It's easier set constraints than to litter code with sanity checks.
+
+#### Leakproof Abstraction
+
+There is no ORM involved. Creating new views happens in SQL with known performance implications. A database administrator can now create an API from scratch with no custom programming. 
+
+#### Embracing the Relational Model
+
+In 1970 E. F. Codd criticized the then-dominant hierarchical model of databases in his article <a href="https://www.seas.upenn.edu/~zives/03f/cis550/codd.pdf">A Relational Model of Data for Large Shared Data Banks</a>. Reading the article reveals a striking similarity between hierarchical databases and nested http routes. With PostgREST we attempt to use flexible filtering and embedding rather than nested routes.
+
+#### One Thing Well
+
+PostgREST has a focused scope. It works well with other tools like Nginx. This forces you to cleanly separate the data-centric CRUD operations from other concerns. Use a collection of sharp tools rather than building a big ball of mud.
+
+#### Shared Improvements
+
+As with any open source project, we all gain from features and fixes in the tool. It's more beneficial than improvements locked inextricably within custom codebases.
+
+### Myths
+
+#### You have to make tons of stored procs and triggers
+
+Modern PostgreSQL features like auto-updatable views and computed columns make this mostly unnecessary. Triggers do play a part, but generally not for irksome boilerplate. When they are required triggers are preferable to ad-hoc app code anyway, since the former work reliably for any codepath.
+
+#### Exposing the database destroys encapsulation
+
+PostgREST does versioning through database schemas. This allows you to expose tables and views without making the app brittle. Underlying tables can be superseded and hidden behind public facing views. The chapter about versioning shows how to do this.
+
+### Conventions
+
+This guide contains highlighted notes and tangential information interspersed with the text.
+
+<div class="admonition note">
+    <p class="admonition-title">Design Consideration</p>
+
+    <p>Contains history which informed the current design. Sometimes it discusses unavoidable tradeoffs or a point of theory.</p>
+</div>
+
+<div class="admonition warning">
+    <p class="admonition-title">Invitation to Contribute</p>
+
+    <p>Points out things we know we want to add or improve. They might give you ideas for ways to contribute to the project.</p>
+</div>
+
+<div class="admonition danger">
+    <p class="admonition-title">Deprecation Warning</p>
+
+    <p>Alerts you to features which will be removed in the next major (breaking) release.</p>
+</div>
@@ -0,0 +1,23 @@
+## Ecosystem
+
+### Client-Side Libraries
+
+* [mithril.postgrest](https://github.com/catarse/mithril.postgrest) - Mithril plugin to create and authenticate requests
+* [lewisjared/postgrest-request](https://github.com/lewisjared/postgrest-request) - node interface to postgrest instances
+* [JarvusInnovations/jarvus-postgrest-apikit](https://github.com/JarvusInnovations/jarvus-postgrest-apikit) - Sencha framework package for binding models/stores/proxies to PostgREST tables
+
+### Extensions
+
+* [srid/spas](https://github.com/srid/spas) - allow file uploads and basic auth
+
+### Example Apps
+
+* [timwis/ext-postgrest-crud](https://github.com/timwis/ext-postgrest-crud) - browser-based spreadsheet
+* [srid/chronicle](https://github.com/srid/chronicle#deploying-to-heroku) - tracking a tree of personal memories
+* [begriffs/postgrest-example](https://github.com/begriffs/postgrest-example) - how to configure a db for use as an API
+* [marmelab/ng-admin-postgrest](https://github.com/marmelab/ng-admin-postgrest) - automatic database admin panel
+* [tyrchen/goodfilm](https://github.com/tyrchen/goodfilm) - example film api
+
+### In Production
+
+* [Catarse](https://www.catarse.me/)
@@ -0,0 +1,70 @@
+## Installation
+
+### Installing from Pre-Built Release
+
+The [release page](https://github.com/begriffs/postgrest/releases/latest) has precompiled binaries for Mac OS X and 64-bit Ubuntu. Next extract the tarball and run the binary inside with no arguments to see usage instructions:
+
+```sh
+# Untar the release (available at https://github.com/begriffs/postgrest/releases/latest)
+
+$ tar zxf postgrest-0.2.11.1-osx.tar.xz
+
+# Try running it
+$ ./postgrest-0.2.11.1
+
+# You should see a usage help message
+```
+
+<div class="admonition danger">
+    <p class="admonition-title">Deprecation Warning</p>
+
+    <p>The filename inside the tarball currently includes a version
+    number, but this will be removed in the next version for cleaner
+    post-extraction scripting.</p>
+</div>
+
+<div class="admonition warning">
+    <p class="admonition-title">Invitation to Contribute</p>
+
+    <p>I currently build the binaries manually for each version. We need to set up an automated build matrix for various architectures. It should support 32- and 64-bit versions of
+
+    <ul><li>Scientific Linux 6</li><li>CentOS</li><li>RHEL 6</li></ul>
+
+    Also it would be good to create packages for Homebrew, and apt.</p>
+</div>
+
+We'll learn the meaning of the command line flags later, but here is a minimal example of running the app. It does all operations as user `postgres`, including for unauthenticated requests.
+
+```sh
+$ ./postgrest-0.2.11.1 -d dbname -U postgres --a postgres --v1schema public
+```
+
+### Building from Source
+
+When a prebuilt binary does not exist for your system you can build the project from source. You'll also need to do this if you want to help with development. [Stack](https://github.com/commercialhaskell/stack) makes it easy. It will install any necessary Haskell dependencies on your system.
+
+* [Install Stack](https://github.com/commercialhaskell/stack#how-to-install) for your platform
+* Build the project
+
+```bash
+git clone https://github.com/begriffs/postgrest.git
+cd postgrest
+stack build
+```
+
+* Run the server
+
+```bash
+stack exec postgrest -- arg1 arg2
+# ... your arguments after the double dashes
+```
+
+If you want to run the test suite, stack can do that too: `stack test`.
+
+### Installing PostgreSQL
+
+To use PostgREST you will need an underlying database. You can use something like Amazon [RDS](https://aws.amazon.com/rds/) but installing your own locally is cheaper and more convenient for development.
+
+* [Instructions for OS X](http://exponential.io/blog/2015/02/21/install-postgresql-on-mac-os-x-via-brew/)
+* [Instructions for Ubuntu 14.04](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-postgresql-on-ubuntu-14-04)
+
@@ -0,0 +1,24 @@
+site_name: PostgREST
+site_url: http://postgrest.com
+site_description: Building declarative APIs
+site_author: Joe Nelson
+site_favicon: favicon.ico
+
+repo_url: https://github.com/begriffs/postgrest
+
+pages:
+- Home: index.md
+- Install:
+  - The Server: install/server.md
+  - Ecosystem: install/ecosystem.md
+- API:
+  - Reading: api/reading.md
+  - Writing: api/writing.md
+- Admin:
+  - Security: admin/security.md
+  - Versioning: admin/versioning.md
+  - Migration: admin/migration.md
+  - Deployment: admin/deployment.md
+  - Performance: admin/performance.md
+- Examples:
+  - Getting Started: examples/start.md