Compare commits

..
798 Commits
Author SHA1 Message Date
steve-chavez a41380b965 bump version to 10.0.0 2022-08-18 23:35:41 -05:00
steve-chavez b2a2504201 changelog: clarify and stamp date for v10 2022-08-18 23:07:24 -05:00
steve-chavez c45e85c5a6 feat: add one to one relationship for embedding
BREAKING CHANGE: For the cases where one to one relationships are
detected, json objects will be returned instead of json arrays of length
1.

If you wish to override this behavior, you can use computed
relationships to return arrays again.
2022-08-18 22:36:47 -05:00
Laurence IslaandSteve Chavez 2afe13fa89 Move feature to fix in the changelog 2022-08-18 20:15:57 -05:00
steve-chavez 36c24d9cbe refactor: clean allM2ORels query
* remove unnecessary lateral usage
* use alias for pg_constraint
2022-08-18 15:08:14 -05:00
steve-chavez d6ec171bcb feat: add computed relationships
* work for select, mutations, rpc
* overrides detected relationships
2022-08-17 22:50:06 -05:00
steve-chavez 06c9e246f4 refactor: simplify addJoinConditions
* remove unnecessary Either
* remove implicitJoins from ReadQuery
* remove unnecessary foldr in addJoinConditions
* move aliasing to addRels
2022-08-17 22:50:06 -05:00
Laurence IslaandGitHub b2b92e75cc Make text the default format for the execution plan 2022-08-17 17:58:02 -05:00
GabrielandGitHub 506929c586 fix: Make views pk and fk schema query compatible with postgres v15 2022-08-17 14:20:34 -05:00
Matt VotavaandGitHub f75e77cf60 Nix: allow building from pure-eval mode (#2427)
* default.nix: allow import with pure-eval enabled

* Update hercules-ci/gitignore

* shell.nix: fix import of default.nix
2022-08-15 20:44:02 -05:00
Laurence IslaandGitHub ba3ba9fd5b refactor: simplify tests where mutations need to be verified in the db 2022-08-15 12:06:54 -05:00
Laurence IslaandGitHub 7de8d5446a Revert bulk update patch
- Revert patch #2311
- Keep the refactor done to qsFiltersRoot
- Keep the refactor done to the items tables
- Add tests that now work with pg-safeupdate as a result
2022-08-12 10:00:33 -05:00
Laurence IslaandGitHub 4ac47df528 tests: Add tests for the pg-safeupdate library 2022-08-10 11:50:57 -05:00
Robert Vollmert 57bc3d805c nix: bump nixpkgs version
- move from GHC 9.2.3 to GHC 9.2.4
- drop upstreamed openssl runtime dependency patch
2022-08-10 06:46:43 +02:00
steve-chavez 3648986aa8 refactor: correlated subquery for o2m query
Improves the query costs a bit as shown in the tests and reduces code.
2022-08-06 21:42:53 -05:00
steve-chavez c050b61db8 tests: add basic resource embedding costs
Adds costs thresholds on the PlanSpec for the o2m,m2o and m2m queries.
2022-08-06 21:42:53 -05:00
Robert Vollmert 92d00749a3 refactor: move pool release on shutdown out of signal handler
This changes behaviour somewhat in that:
- We now consistently release the pool on shutdown, even on non-Unix
  platforms, and including for CmdDumpConfig.
- We release the pool *after* interrupting `App.run`, which will
  rather cause more than fewer connection to be closed properly.
  (Previously any in-use connections would not have been caught by
  `releasePool`, though *maybe* the `UserInterrupt` handling in
  the web handler ends up closing the connections properly already
  anyway).

(The main aim of the change is to make it clearer when and why the
pool is released.)
2022-08-03 19:02:33 +02:00
Robert Vollmert c5849ecbe0 misc: comment typo fix 2022-08-03 19:02:33 +02:00
Robert Vollmert fea38a4caf misc: style fix 2022-08-03 19:02:33 +02:00
steve-chavez d24f0c8216 bump version to 9.0.1.20220802 2022-08-02 21:13:35 -05:00
Robert Vollmert 35a114a81b tests: add failing test showing reconfiguration doesn't take effect (#2401) 2022-08-02 19:03:45 +02:00
Robert Vollmert ed988100c6 tests: add a test to verify pool size, and parallel request handling
The main point for this test is to verify the test setup for follow-up
tests of pool behaviour.
2022-08-02 19:03:45 +02:00
Robert Vollmert b6a93aae1f tests: add a Thread class that catches exceptions 2022-08-02 19:03:45 +02:00
Robert Vollmert 70cae5beed tests: move pool settings to defaultenv from run
This allows overriding them if necessary.
2022-08-02 19:03:45 +02:00
Robert Vollmert 0a24a83fa9 tests: decouple config tests from test-specific postgrest settings 2022-08-02 19:03:45 +02:00
Robert Vollmert c8739aa8a3 fix: fix some race conditions running connection worker
Previously, it was quite possible to have two connection workers
running, or to get into a state where a failed connection worker
is still considered running preventing new connection workers
from starting.
2022-08-02 09:51:19 +02:00
Robert Vollmert d4950c6460 refactor: drop getPool 2022-08-02 09:24:14 +02:00
Robert Vollmert 1d15060c63 refactor: AppState.usePool for App
This is a bit ugly, it would be a bit nicer to just pass
'AppState.usePool appState' but then the types get messy.
Or we could introduce our own 'Pool' wrapper type.
2022-08-02 09:24:14 +02:00
Robert Vollmert 401b98c5b4 refactor: introduce AppState.usePool 2022-08-02 09:24:14 +02:00
Robert Vollmert d53949e4a4 refactor: queryDbSettings doesn't need the pool 2022-08-02 09:24:14 +02:00
Robert Vollmert 421163dc75 refactor: use AppState.releasePool everywhere
This prepares for wrapping the pool in a reference.
2022-08-02 09:24:14 +02:00
Robert Vollmert eae169ca67 refactor: move interrupt out of releasePool helper 2022-08-02 09:24:14 +02:00
Robert Vollmert ec23067e3e refactor: simplify main, address reReadConfig TODO 2022-08-02 09:24:14 +02:00
Robert Vollmert bfacec3a67 ci: check python file formatting
Previously, postgrest-style would reformat python files, but their
formatting wasn't enforced.
2022-08-02 09:24:14 +02:00
Robert Vollmert e7b69cb99c tests: fix python formatting 2022-08-02 09:24:14 +02:00
steve-chavez fd7c23f7a1 Add "for" param for the vnd.pgrst.plan media type 2022-08-01 17:41:23 -05:00
steve-chavez 84e03a16dd fix: columns w/ dollar sign($) don't need quoting 2022-07-30 19:50:09 -05:00
Robert Vollmert d556cea8ce tests: add tests for statement_timeout
The new tests verify that:
- statement_timeout on the authenticator role works to cancel slow statements
- changes to statement_timeout take effect on SIGUSR1

This reuses the old "limited_authenticator" role and adds some plumbing to
allow reliably changing the statement timeout even if the current role is
not functional due to a low statement timeout, and to make tests that modify
the role independent from each other.

- introduce module-wide metapostgrest fixture to have an out-of-band way to
  manipulate the database, without having to spin up extra postgrest instances
  per test
- reset statement_timeout at the start of the respective tests
2022-07-29 21:46:57 +02:00
Steve ChavezandGitHub 8911afd079 feat: Allow getting the EXPLAIN plan of a request 2022-07-27 19:33:34 -05:00
steve-chavez d2df289696 bump version to 9.0.1.20220717 2022-07-17 18:44:13 -05:00
steve-chavez 2eb7c803e3 feat: support OPTIONS on RPC and root path 2022-07-17 18:40:14 -05:00
steve-chavez 71a5748718 refactor: DRY pytest no pool connection logic 2022-07-17 14:41:51 -05:00
steve-chavez 4ab5e63e58 fix: no empty transaction on OPTIONS request 2022-07-17 14:41:51 -05:00
steve-chavez c445105d55 correct query on rpc count=exact
corrects bad logic on e4006da(perf: remove toSourceRel)
2022-07-16 17:18:25 -05:00
steve-chavez 2905753194 test: no empty tx for invalid JWT 2022-07-15 22:23:33 -05:00
steve-chavez df49f482dc test: no empty tx started for invalid path/method 2022-07-15 22:23:33 -05:00
Robert Vollmert 23e4c7fe6e nix: bump nixpkgs version 2022-07-16 00:14:25 +02:00
Laurence Isla 5ac4c29a9e bump version to 9.0.1.20220714 2022-07-14 15:44:31 -05:00
Laurence IslaandGitHub e0ba6b6d1c Add security definitions to the OpenAPI output 2022-07-13 22:47:09 -05:00
Laurence IslaandGitHub bdf1cbe111 fix: inaccurate result count when an inner embed is selected after a normal embed in the query string 2022-07-12 22:29:42 -05:00
Robert Vollmert 9252ec2509 ci: verify that static executable is static 2022-07-12 23:25:06 +02:00
steve-chavez 8230128ff6 fix: no empty tx on bad HTTP method 2022-07-11 18:21:36 -05:00
steve-chavez 6d7bf9faa9 fix: no empty tx on Not Found error 2022-07-11 18:21:36 -05:00
steve-chavez 28183a667c refactor: rm raw media types unnecessary decoding
Just convert them to MTOther instead
2022-07-11 18:21:36 -05:00
Robert Vollmert 1eda7db0fb stack: build with latest LTS release, GHC 9.0
Also bump FreeBSD release.
2022-07-09 02:30:53 +02:00
Steve ChavezandGitHub 3cdc6d15d7 test: add bulk insert to loadtest (#2358) 2022-07-06 13:50:58 -05:00
steve-chavez 0708f46df3 refactor: pass media type to Statements module 2022-07-06 13:47:59 -05:00
steve-chavez 7a7ceaf39a refactor: rename ContentType to MediaType
The core type was wrongly named as the header
2022-07-06 13:47:59 -05:00
steve-chavez 3e83bef9c4 bump version to 9.0.1.20220630 2022-06-30 13:53:33 -05:00
Laurence IslaandGitHub 169547aee6 Return Location header only when using Prefer: return=header-only 2022-06-24 12:53:27 -05:00
Laurence IslaandGitHub f7745e1569 fix: Allow schema names with special characters in the search path
Fixes regression where the search path did not recognize schemas with uppercase, spaces and other special characters in their names.
2022-06-23 18:54:43 -05:00
steve-chavez 870f7a39b0 Add PostGIS GeoJSON support
Works for GET, POST, PATCH, PUT, DELETE, RPC.
2022-06-22 22:16:44 -05:00
steve-chavez 86c40e8df9 test: don't run spec tests with stack
We cannot handle postgis and other postgresql dependencies with
stack.

Also delete test/with_tmp_db since it's no longer used.
2022-06-22 22:16:44 -05:00
steve-chavez d87b622d64 nix: update tests README 2022-06-20 15:55:22 -05:00
Laurence IslaandGitHub 7088e9d510 nix: Add command to remove old hie files in postgrest-clean 2022-06-16 19:16:15 -05:00
RobertandGitHub fe8b32d077 Merge pull request #2324 from robx/ghc92
Update GHC to 9.2.2 (fixes #2288)
2022-06-16 18:45:10 +02:00
Robert Vollmert 478c48cc84 nix: define GHC 9.2.2's Cabal version for static build 2022-06-16 10:39:03 +02:00
Robert Vollmert c9c64fd71f nix: update hsie for GHC 9.2 2022-06-16 10:39:03 +02:00
Robert Vollmert 93210f9380 nix: upgrade weeder to 2.4.0 (GHC 9.2 compatible) 2022-06-16 10:39:02 +02:00
Robert Vollmert c25473e001 nix: upgrade ptr to 0.16.8.2 (GHC 9.2 compatible) 2022-06-16 10:39:02 +02:00
Robert Vollmert f3cbab6f82 nix: use GHC 9.2.2 2022-06-16 10:39:02 +02:00
Robert Vollmert 3d65d66b1b tests: fix incomplete pattern warnings (GHC 9.2) 2022-06-16 10:39:02 +02:00
Robert Vollmert f582798276 ReadQuery: split out to fix record field warning
The use of `where_` in DbRequestBuilder issues a warning since
GHC 9.2 as per
https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0366-no-ambiguous-field-access.rst

To disambiguate that, move the type to a separate module and use
the record field names qualified. For symmetry, MutateQuery also
gets its own module.
2022-06-16 10:38:59 +02:00
Robert Vollmert ba87a60a47 rangeParse: fix incomplete pattern match warning
The warning is new in GHC 9.2 compared to 8.10.
2022-06-16 10:37:05 +02:00
Robert Vollmert 356bff9cbd cabal: relax upper bounds to allow GHC 9.2 2022-06-16 10:37:05 +02:00
Steve ChavezandGitHub 86574c89a8 Bulk update (#2311)
* refactor: remove EmbedPath type from qsFiltersRoot

* Rename reset items function for reusability
2022-06-15 20:30:31 -05:00
Robert Vollmert 734f8e6df2 nix: fix static build
- the isExecutable patch was broken, remove it
- the ncurses fix was broken, we still need `enableStatic`
- the original error was caused by the new ghc-bignum
  package, which isn't a "proper" Haskell package; we filter
  this out explicitly now
2022-06-16 01:07:28 +02:00
steve-chavez df7d71db32 fix: Increase the db-pool-timeout to 1 hour 2022-06-15 13:19:44 -05:00
Robert Vollmert 0c5d2e553e nix: build postgrest with nix in loadtest (fixes #2308)
This allows running the load test against main even if the
dependencies change.
2022-06-15 00:41:38 +02:00
Robert Vollmert 36ed7e2fb7 src, nix: hlint update
This pulls in an updated hlint version from nixpkgs, and fixes some
new eta-reduction warnings. (I don't particularly think this is more
readable, but I suppose being consistently hlint-clean is worth it.)
2022-06-14 19:13:55 +02:00
RobertandGitHub bac63f339d Merge pull request #2292 from robx/aeson2
Upgrade aeson to 2.0.*
2022-06-14 07:43:08 +02:00
Robert Vollmert 8d8074d500 nix: keep old hlint
This keeps the pre-nixpkgs update hlint version, so we can defer
addressing the warnings to a follow-up PR.
2022-06-13 13:25:54 +02:00
Robert Vollmert 63ebf6e92b nix: hack static-haskell-nix for updated nixpkgs
This patches static-haskell-nix to work for building postgrest
with updated nixpkgs (from ~202203):

- The ncurses 'enableStatic' argument doesn't exist anymore.
  We use the vanilla package instead, which seems to work fine.
- The 'isExecutable' check fails with a strange error related
  to trying to override 'mkDerivation'.
  We patch 'isExecutable' to check explicitly whether we're building
  postgrest. ('isExecutable' is used to determine whether to build
  a package statically.)
2022-06-13 13:25:54 +02:00
Robert Vollmert d7f54e76f4 nix: haskell overlay update 2022-06-13 13:25:54 +02:00
Robert Vollmert 34d0f34620 nix: bashCompletion -> bash-completion 2022-06-13 13:25:54 +02:00
Robert Vollmert 14c882a566 nix: fetch PostgreSQL 9.6 via postgresql legacy overlay 2022-06-13 13:25:54 +02:00
Wolfgang WaltherandRobert Vollmert ff56c3adf4 nix: port openssl patch to updated nixpkgs 2022-06-13 13:25:54 +02:00
Robert Vollmert 12967f8643 nix: nix version bump 2022-06-13 13:25:54 +02:00
Robert Vollmert b056735535 stack: update stack.yaml 2022-06-13 13:25:54 +02:00
Robert Vollmert f5afa419f1 cabal, tests: require wai-logger 2.4.0, adapt tests
wai-logger version 2.4.0 fixes log output to not say 'unknownSocket'
for unix sockets.
2022-06-13 13:25:54 +02:00
Robert Vollmert 7531c5490d cabal: allow hasql-1.5, hasql-dynamic-statements-0.3.* 2022-06-13 13:25:54 +02:00
Robert Vollmert bb62e559d6 cabal: require jose >= 0.8.5.1 (aeson-2 compat)
jose version before 0.8.5.1 lacked an upper bound on
aeson-2, causing build failures with aeson-2 present.
2022-06-13 13:25:54 +02:00
Robert Vollmert e42bf30aeb cabal: allow swagger 2.8 2022-06-13 13:25:54 +02:00
Robert Vollmert 2b8ffc8e61 src: update for changed map type in aeson-2
This means that we're now using Data.Map.Strict instead of
Data.HashMap.Strict for JSON objects in general, and specifically
for claims maps and CSV rows.

This addresses certain hash flooding vulnerabilities, but may
have performance downsides.

Compare e.g. https://frasertweedale.github.io/blog-fp/posts/2021-10-12-aeson-hash-flooding-protection.html
2022-06-13 13:25:54 +02:00
Robert Vollmert 48a3d1d516 cabal: update aeson to 2.0.3 2022-06-13 13:16:56 +02:00
Robert Vollmert 5e6987b1d8 src: consistently import HashMap as HM, Map as M
With both HashMap and Map imported as M in different modules,
linter rules prevented ever importing both modules in one place.
2022-06-13 13:16:50 +02:00
Robert Vollmert 4dfcb59f73 nix: more explicit logging in load test
* Write vegeta output to file explicitly.
  The previous version using `tee` captured the standard
  output of the `withPgrst` helper, too, causing weird
  errors when those had unexpected output.

* Log and fail the postgrest build explicitly.
  In situations such as changing nix dependencies, the
  `cabal build` run could fail, but this wasn't visible
  in CI.

With these changes, the load test now fails with a nice
error message when the build fails.
2022-06-09 14:56:50 +02:00
Laurence IslaandGitHub 620da2e776 Update CHANGELOG to include release v9.0.1 (#2307) 2022-06-08 23:00:19 -05:00
Laurence IslaandGitHub f9688c3553 Upgrade protolude to 0.3.1 2022-06-08 11:41:39 -05:00
Robert Vollmert 4ff2469f57 nix: don't write to /nix/store from hsie build (fixes #2304)
This is a kind of hacky fix to the kind of hacky hsie derivation.
2022-06-07 21:36:38 +02:00
Robert Vollmert e3252a6e9e nix: test script without glibc locales
glibcLocales is not available e.g. on darwin. This change allows
running various dev tools on darwin, and other theoretical
non-glibc systems.
2022-06-07 21:36:38 +02:00
Robert Vollmert a9035bba2f nix: use the matching weeder
weeder needs to be compiled with the same GHC that we build postgrest
with.

Before this change, we were relying on the fact that we happened
to set ${compiler} to the same ghc8107 as used by nixpkgs for
haskellPackages, now we explicitly take it from the correct
haskell package set.
2022-06-07 21:36:38 +02:00
Robert Vollmert 4c8c3ccfb2 nix: pass through hspec arguments to postgrest-test-spec
This allows e.g. running

  postgrest-test-spec --match "some failing test"
2022-06-07 21:36:38 +02:00
Robert Vollmert 09f3e09c96 tests: don't specify openapi root result
Whereas the test used to specify a body of

  [{"qiName":"referrals","qiSchema":"test"},"private"]

we see both

  [{"qiName":"authors_w_entities","qiSchema":"test"},"test"]

and

  [{"qiName":"organizations","qiSchema":"test"},"test"]

on the GHC 9.2 upgrade PR.

This changes the test to no longer expect a particular body.

Compare:
- https://github.com/PostgREST/postgrest/pull/2292#issuecomment-1146160792
- https://github.com/PostgREST/postgrest/issues/1698
2022-06-07 16:56:22 +02:00
Franz-Josef FärberandSteve Chavez 807364bd7b RPC POST for function w/single unnamed XML param 2022-06-03 11:52:57 -05:00
steve-chavez d14f745406 bump version to 9.0.0.20220531 2022-05-31 18:19:02 -05:00
steve-chavez 1cdc9169c7 fix: using CPU while idle 2022-05-31 12:17:36 -05:00
steve-chavez 9aa400c8f3 fix: disable parallel GC for perf on high-core CPU 2022-05-31 12:17:36 -05:00
Laurence IslaandGitHub e4cdb8c491 ci: run on branches starting with "rel-" 2022-05-30 22:38:05 -05:00
Laurence IslaandSteve Chavez c812a3fe3e ci: Run ARM builds on releases 2022-05-16 18:10:36 -05:00
steve-chavez 1ab11de96c bump version to 9.0.0.20220516 2022-05-16 14:36:04 -05:00
steve-chavez f7b173163c fix: ignore views on col/fk as target
Also refactor self relationship findRel logic
2022-05-16 12:22:24 -05:00
steve-chavez 2be63b36d6 refactor: self relationship
* Add test for self relationship in view
2022-05-09 21:31:19 -05:00
steve-chavez c60380b5fc perf: RelationshipsMap with foreign schema as key 2022-05-09 21:31:19 -05:00
steve-chavez d88b16e5ab fix: allow cast on types with underscores/numbers
e.g. select=oid_array::_int4
2022-05-09 13:02:33 -05:00
Laurence IslaandGitHub 867f2569a0 ci: Fix Windows build using Stack
Added PostgreSQL binaries to the PATH and removed the use of msys2-keyring
2022-05-05 00:06:00 -05:00
Franz-Josef FärberandSteve Chavez 5a3fbd7111 Allow returning XML from single-column queries 2022-05-04 10:27:28 -05:00
Laurence IslaandGitHub a2349d90c0 Reorganize error groups
* Grouped Hasql errors into one error and renamed it to InternalError
* Move all the General errors to ApiRequest errors
2022-05-02 21:16:31 -05:00
steve-chavez 980f27a6d1 bump version to 9.0.0.20220502 2022-05-02 16:06:51 -05:00
steve-chavez ce96c88ba0 test: increase memory tests memory by 1M 2022-05-02 11:12:02 -05:00
steve-chavez 50b275d3e2 refactor: move PUT checks to ApiRequest 2022-05-02 11:12:02 -05:00
steve-chavez e90391b7ac limited update/delete requires explicit order
* limited update/delete now works on views with explicit order
* no default order, enforce order presence
* apply row count to ensure limited mutations
* move requiring order to ApiRequest
2022-05-02 11:12:02 -05:00
Laurence IslaandGitHub 5ad8800773 feat: Allow limit=0 in query params to return an empty array 2022-04-28 17:14:29 -05:00
Franz-Josef FärberandSteve Chavez d2aa50be52 Allow returning XML from RPCs 2022-04-26 17:37:59 -05:00
Laurence IslaandGitHub a8477b7822 Add tests for the ~ ltree operator (#2267) 2022-04-26 22:48:15 +02:00
steve-chavez 115dae7484 perf: Relationship list to a hash map 2022-04-25 21:41:51 -05:00
steve-chavez e4006da9bc perf: remove toSourceRel
toSourceRel is an expensive operation since it iterates over all the
relationships detected. We now use an alias on the CTE so embedding can
work without toSourceRel.
2022-04-25 21:41:51 -05:00
steve-chavez a47c59baca fix: restrict generated m2m relationships
Only add m2m relationship based on real junctions:
a table has FKs to two other tables and the FK columns
are part of the PK columns.

Also add some stats to big_schema.sql
2022-04-25 12:35:16 -05:00
dependabot[bot]andSteve Chavez b191431530 build(deps): bump codecov/codecov-action from 3.0.0 to 3.1.0
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3.0.0 to 3.1.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v3.0.0...v3.1.0)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-25 12:24:38 -05:00
Laurence IslaandGitHub 9bdb4e5c4f ci: remove armv7 architectures for ARM builds
Now only builds aarch64 on the remote server without Docker
2022-04-23 20:30:07 +02:00
steve-chavez 07d619981e perf: don't lookup Table pkCols unnecessarily
handleCreate only needs it on some headers, udpate and delete don't need
pkcols.
2022-04-22 19:04:16 -05:00
steve-chavez 778b8d074d refactor: merge Column list into Table
* Remove Table from Column
* Correct OpenAPI test where the schema was appearing for the enum unnecessarily
* Increase memory-tests consumption by 2M

Seems the Hasql decoder consumes more memory now that the allTables
results also have [Column] inside.
2022-04-22 19:04:16 -05:00
steve-chavez 902e4e8921 refactor: clean allColumns query
* Add OpenAPI test to ensure enum labels are present
* Remove unnecessary fk/pk columns cte
* Remove unnecessary step for parsing enum labels
2022-04-22 11:46:45 -05:00
steve-chavez d2719420f4 refactor: PKcols in table, ViewKeyDependency type
* Get PKcols inside tables - done with SQL for tables and
  with an additional step in Haskell for views.

  This fixes an fk column being considered as a pk column on views
  and corrects the test added on
  https://github.com/PostgREST/postgrest/pull/1875/files/1d549768580310e18aac4ffa6dbd01c5b77934a7#r853674126

* classify view key dependencies in SQL

* remove Column from Relationship

* Merge cols/fcols in Relationship and
  ensure allM2ORels and allViewsKeyDependencies fk columns
  are ordered - done by attnum in SQL

* Cardinality now contains relColumns instead of Relationship -
  this simplifies getJoinConditions.
2022-04-21 14:44:45 -05:00
steve-chavez 057e8d63bf test: add big schema openapi test
The io test is marked as pending for now, because the stack size it
requires is too big.
2022-04-20 20:05:11 -05:00
steve-chavez f4e171aa2b refactor: ignore partitions in allM2ORels
Also clarify foreign key filter
2022-04-19 17:23:12 -05:00
steve-chavez 7f1507b9fb refactor: remove Table from Relationship
Just having the QualifiedIdentifier gets us closer to having
Relationship as a Table attribute since it avoids a cyclic dependency

* remove unnecessary findTable
* modify RootSpec test
2022-04-18 15:48:03 -05:00
steve-chavez cdcc175abf refactor: table list to table map 2022-04-18 15:48:03 -05:00
steve-chavez f995799e8f refactor: remove fkey from allPrimaryKeys query
The query had the unnecessary 'f' in `c.contype in ('p', 'u', 'f')`

Also comment and clarify the query by using explicit JOINs
2022-04-18 15:48:03 -05:00
steve-chavez 44e2b2fbb4 refactor: DRY allTables/accessibleTables query 2022-04-18 15:48:03 -05:00
steve-chavez 2eaf81334e nix: local history for nix-shell 2022-04-18 15:48:03 -05:00
steve-chavez c399b1f5f8 nix: log all statements in temp db 2022-04-18 15:48:03 -05:00
Laurence IslaandGitHub bc7d54c7d2 Add CI for ARM architectures (#2127) 2022-04-18 17:40:32 +02:00
Laurence IslaandGitHub 7589102c6c Fix misleading disambiguation error where relationship looks like valid syntax
* Add columns for the m2m relationship
2022-04-12 00:02:19 +02:00
Michael Kane JunckerandWolfgang Walther 007f49a8bc feat(query): add basic regular expression operator support - refs #2236 2022-04-11 08:33:16 +02:00
dependabot[bot]andWolfgang Walther a768bcbf20 build(deps): bump codecov/codecov-action from 2.1.0 to 3.0.0
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2.1.0 to 3.0.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v2.1.0...v3.0.0)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-11 08:23:26 +02:00
dependabot[bot]andWolfgang Walther cb145271ac build(deps): bump actions/cache from 2.1.7 to 3
Bumps [actions/cache](https://github.com/actions/cache) from 2.1.7 to 3.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.7...v3)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-28 07:45:07 +02:00
Laurence IslaandGitHub 9124dfd427 Bypass FreeBSD builds due to timeouts 2022-03-26 15:14:54 -05:00
Laurence Isla 8e86177486 bump version to 9.0.0.20220326 2022-03-26 12:26:47 -05:00
Laurence IslaandGitHub fe898c27ae Fix incorrectly placed feature in CHANGELOG
Move incorrectly placed feature for PostgREST errors from v9.0.0 to Upcoming
2022-03-26 11:08:37 -05:00
steve-chavez f3f9030c48 disallow limit/offset on views 2022-03-26 15:29:13 +01:00
steve-chavez 73dc2692b1 refactor: add findTable function 2022-03-26 15:29:13 +01:00
steve-chavez 87ffe39746 fix: ignore max-rows on POST/PATCH/PUT/DELETE 2022-03-26 15:29:13 +01:00
steve-chavez 331e88ea39 feat: add limited delete 2022-03-26 15:29:13 +01:00
steve-chavez f4becf99ad feat: add limited update 2022-03-26 15:29:13 +01:00
Ezequiel AlvarezandGitHub 799daa7556 fix: json/jsonb columns should not have type "string" in OpenAPI spec (#2203)
* Switch to no type for json/jsonb
2022-03-26 15:16:35 +01:00
Steve ChavezandGitHub a3c1d9977f Correct live/ready checks to consider special host values (#2182) 2022-03-11 15:16:45 +01:00
dependabot[bot]andGitHub c803c4d0b6 build(deps): bump actions/checkout from 2.4.0 to 3 (#2186) 2022-03-08 01:56:58 +01:00
dependabot[bot]andGitHub 8aef09574c build(deps): bump actions/upload-artifact from 2.3.1 to 3 (#2187) 2022-03-08 01:46:30 +01:00
dependabot[bot]andGitHub efcc93ad89 build(deps): bump actions/download-artifact from 2.1.0 to 3 (#2185) 2022-03-07 22:54:47 +01:00
Laurence IslaandGitHub d6834e8bf8 Add 'PGRST' error code to differentiate from PostgreSQL errors 2022-03-04 15:57:38 -05:00
Laurence IslaandGitHub 99d0b805df fix: Using GET with certain Content-Type headers now correctly calls the no parameter function if it exists
Using GET with text/plain or application/octet-stream as Content-Type headers no longer returns 404 Not Found when a function with no parameters exists
2022-02-28 13:23:55 -05:00
Laurence IslaandGitHub 69070f341a fix: Remove admin-server-port from in-db configuration 2022-02-23 17:26:01 -05:00
steve-chavez dcf7ade5bc bump version to 9.0.0.20220211 2022-02-11 14:47:33 -05:00
Steve ChavezandGitHub 9021c96baa fix: Keep working when EMFILE is reached (#2158)
Done by upgrading to warp 3.3.19
2022-02-09 17:27:29 -05:00
Wolfgang WaltherandWolfgang Walther 41a4396147 fix: Remove uncallable functions from schema cache and OpenAPI output
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-02-08 09:08:27 +01:00
Wolfgang WaltherandWolfgang Walther d9f7f6f509 refactor: Remove pg_catalog prefix from schema cache queries
Since the search path is voided at the beginning, there is no need to qualify those.

Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-02-08 09:08:27 +01:00
Wolfgang WaltherandWolfgang Walther bd62008e1c fix: Remove aggregates, procedures and window functions from schema cache and OpenAPI output
Aggregates and Window functions can't be called as RPCs in a useful way.

Procedures are not supported right now, but might be added later.

Resolves #2101

Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-02-05 17:48:22 +01:00
Wolfgang WaltherandWolfgang Walther ded8981368 fix: Take PG version into account in --dump-schema
The PG version is only read by the Connection Worker, which is not used in the case dump-schema. Now, the pg version is read in the schema cache queries directly, avoiding this problem in all cases.

Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-02-05 17:48:22 +01:00
Wolfgang WaltherandWolfgang Walther 52d628f1ed fix: Return 204 No Content without Content-Type for RPCs returning VOID
Resolves #2001

BREAKING CHANGE: Previously, those RPCs would return "null" as a body with Content-Type: application/json.
2022-02-05 09:28:28 +01:00
Wolfgang Walther 9ca9a54d21 bump version to 9.0.0.20220204 2022-02-04 08:34:51 +01:00
Wolfgang Walther bf689565a5 fix broken link in docker hub description, resolves #2148
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-02-04 08:34:35 +01:00
Wolfgang WaltherandWolfgang Walther 9f1b5c0a81 feat: Support accessing array items and fields of composite types through json operators
This is supported in select=, in filters and in order=.

Resolves #1543

Resolves #2075

Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-02-03 08:13:38 +01:00
Wolfgang WaltherandWolfgang Walther 87bd8e72cf fix changelog
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-02-03 08:13:38 +01:00
Wolfgang WaltherandWolfgang Walther 8934190d43 nix: Add postgrest-release tool
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-30 19:52:35 +01:00
Wolfgang Walther c894b543b0 bump version to 9.0.0.20220130 2022-01-30 16:15:27 +01:00
Wolfgang WaltherandWolfgang Walther 58f76f3d6d fix: Return 405 Method Not Allowed for unsupported verbs
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-28 19:26:33 +01:00
Wolfgang WaltherandWolfgang Walther 8980b09419 refactor: Sort error types
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-28 19:26:33 +01:00
Wolfgang WaltherandWolfgang Walther 927ff6f1e5 test: Split spec tests into multiple subfolders
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-28 19:26:33 +01:00
Wolfgang WaltherandWolfgang Walther 8a722e2cfe refactor: Replace elem with pattern matching to highlight missing code coverage
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-28 19:26:33 +01:00
Wolfgang WaltherandWolfgang Walther 271c329677 nix: Fix postgrest-git-hooks causing merge conflicts when staging files partially
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-28 19:26:33 +01:00
Wolfgang WaltherandWolfgang Walther c63420f0e7 nix: Fix postgrest-git-hooks disable not working
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-28 19:26:33 +01:00
Wolfgang WaltherandWolfgang Walther 2fe2c9104b ci: Fix weeder not stopping CI on error
Removes dead code that was unnoticed

Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-28 17:12:32 +01:00
Wolfgang WaltherandWolfgang Walther 0ea36fc039 ci: Fix windows build failing because of outdated keyring
Signed-off-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-28 17:12:32 +01:00
Wolfgang WaltherandWolfgang Walther fc7113e52e fix: Remove trigger functions from schema cache and OpenAPI output
Trigger functions can't be called directly from SQL and can't be called
via the /rpc prefix either - it makes no sense to expose them in the
OpenAPI output. And we don't need to cache them in the schema cache
either.

Best practice would be to keep the trigger functions in a non-exposed
schema anyway.
2022-01-28 16:45:36 +01:00
Steve ChavezandGitHub c7cadcd776 Fix embedded filter silently getting ignored (#2133)
When the embedded resource is not included in the request
2022-01-24 17:52:27 -05:00
Wolfgang WaltherandWolfgang Walther fae1e168a7 Fix memory tests timing out after c3ade07 2022-01-24 14:56:06 +01:00
Wolfgang WaltherandWolfgang Walther d2791ea0e5 Sort example config alphabetically 2022-01-24 14:56:06 +01:00
Wolfgang WaltherandWolfgang Walther f4637c64c9 Remove db-root-spec from example config file
We are not going to support this officially right now.
2022-01-24 14:56:06 +01:00
Wolfgang WaltherandWolfgang Walther 1b9a30cff4 Make db-uri, db-schemas and db-anon-role optional in example config file 2022-01-24 14:56:06 +01:00
Wolfgang WaltherandWolfgang Walther 5a627e76b2 format comments in example config consistently 2022-01-24 14:56:06 +01:00
Wolfgang Walther d9e016be8b feat: Allow running postgrest without configuration
Resolves #1823
2022-01-22 15:59:26 +01:00
Wolfgang Walther c3ade07ad6 feat: Make db-anon-role optional
Without db-anon-role, PostgREST will block any anonymous access without hitting the database.

Resolves #1689, Ref #1823
2022-01-22 15:59:26 +01:00
Wolfgang Walther 05b5ecd23b Reformatting of some test-cases and removal of superfluous comments. 2022-01-22 15:13:21 +01:00
Wolfgang Walther dbbde6aae1 feat: Make db-schemas optional
Resolves #1769, Ref #1823
2022-01-22 15:13:21 +01:00
Wolfgang Walther 9ed30c0ba4 feat: Make db-uri optional
The default is now "postgresql://" which falls back to LIBPQ environment variables.

Resolves #1991, Ref #1823
2022-01-22 15:13:19 +01:00
Wolfgang Walther ec09b87940 test: Make io tests use environment variables instead of config files where possible.
Load jwt secret from file instead of stdin in tests, because stdin is not supported when reloading the config. Resolves #2126.
2022-01-22 15:10:25 +01:00
Wolfgang WaltherandWolfgang Walther 6b5370f145 ci: Add -with-rtsopts=-KxK option to prevent introducing space leaks
Resolves #387
2022-01-10 22:09:32 +01:00
Wolfgang WaltherandWolfgang Walther a25ad42644 Align columns in postgrest.cabal 2022-01-10 22:09:32 +01:00
Wolfgang WaltherandWolfgang Walther 995215c6a5 fix: Read database configuration properly when = is present in value
Resolves #2120
2022-01-10 09:24:49 +01:00
Wolfgang WaltherandWolfgang Walther 4b841dba61 refactor: Reformat db-config query 2022-01-10 09:24:49 +01:00
Wolfgang WaltherandWolfgang Walther 9639cd5751 feat: Add current user to apache-style request logging. 2022-01-07 20:26:33 +01:00
3c17f97c87 refactor: Make JWT authorization a middleware
This follows the style of wai-middleware-auth package and
makes the JWT parsing a middleware.

Co-authored-by: Wolfgang Walther <walther@technowledgy.de>
2022-01-07 20:26:33 +01:00
Wolfgang WaltherandWolfgang Walther d97bf3b864 refactor: Move each Wai Middleware to a separate file 2022-01-07 20:26:33 +01:00
Steve ChavezandGitHub 65f27653dc cov: empty coverage overlay (#2119)
The overlay depended on the version number, which caused errors when upgrading.

Also update cabal description.
2022-01-07 13:06:03 -05:00
steve-chavez a134d8a493 pre-release: Bump to 9.0.0.20220107
Previous pre-release had a wrong date
2022-01-07 11:06:03 -05:00
steve-chavez 3ded567f6d pre-release: Bump to 9.0.0.20210107 2022-01-07 11:01:59 -05:00
Wolfgang WaltherandWolfgang Walther 8b63d928ae test: Reorganize test/ folder into one subdirectory for each test type 2022-01-07 13:10:36 +01:00
Wolfgang WaltherandWolfgang Walther 8060fe3559 Remove unused dependencies from postgrest.cabal 2022-01-07 12:07:48 +01:00
Wolfgang WaltherandWolfgang Walther df5d9400fe test: Make io tests use admin/ready to wait for startup 2022-01-07 12:07:09 +01:00
Wolfgang Walther 6bc965e2a0 run postgrest-style 2022-01-07 11:39:41 +01:00
monacoremoandWolfgang Walther bfbec8fa36 refactor: Carve out QueryParams module 2022-01-07 09:32:33 +01:00
steve-chavezandWolfgang Walther 51ee072f84 fix: clarify error for failed schema cache load 2022-01-07 09:31:30 +01:00
steve-chavezandWolfgang Walther aa82d2e277 Consider schema cache state on the ready check
* empty schema cache when it fails loading so the ready check can pick
  its new state
2022-01-07 09:31:30 +01:00
Wolfgang WaltherandWolfgang Walther e13d912a79 cov: Add coverage overlay for impossible case in corsPolicy 2022-01-05 11:01:54 +01:00
Wolfgang WaltherandWolfgang Walther 826ae7459a refactor: simplify logger middleware 2022-01-05 11:01:54 +01:00
Wolfgang WaltherandWolfgang Walther cde9cd4ab7 cov: Remove unreachable static icon 2022-01-05 11:01:54 +01:00
Wolfgang WaltherandWolfgang Walther b0a0c4768d cov: Add tests for request logging 2022-01-05 11:01:54 +01:00
steve-chavez 4a9ca9bdae Add live endpoint to admin server
* refactor io tests
* add network package to windows
2022-01-04 20:12:52 -05:00
steve-chavez 65a3ae08f0 Change admin health endpoint to ready 2022-01-04 20:12:52 -05:00
steve-chavez bba6e96fd3 Connect to main app socket on health check 2022-01-04 20:12:52 -05:00
Wolfgang WaltherandWolfgang Walther cc900787de test: Remove unused unix-socket.config file
Unix sockets are the default in io tests and we are testing ip connections explicitly.
2022-01-04 09:36:22 +01:00
Wolfgang WaltherandWolfgang Walther 01e65f5221 test: Improve io test perfomance
Local test improves performance by 15%.
2022-01-04 09:36:22 +01:00
Wolfgang WaltherandWolfgang Walther 1e9031573c test: Move io-tests SQL fixtures to a separate file 2022-01-04 09:36:22 +01:00
Wolfgang WaltherandWolfgang Walther 81ebdff977 test: Improve log output for failing io-tests 2022-01-04 09:36:22 +01:00
Wolfgang WaltherandWolfgang Walther 9baad293e8 test: Refactor invalid_role_claim_key_notify_reload test to properly read all available input 2022-01-04 09:36:22 +01:00
Wolfgang WaltherandWolfgang Walther efc7a21427 test: Refactor test-cases for admin/health endpoint 2022-01-04 09:36:22 +01:00
Wolfgang WaltherandWolfgang Walther b9c497a3ab test: Use .encode() end .decode default values in io-tests
Those default to "utf-8" anyway.
2022-01-04 09:36:22 +01:00
Wolfgang WaltherandWolfgang Walther 5f43235370 nix: Fix nested postgrest-with-postgresql-x calls when -f is used 2022-01-04 09:36:22 +01:00
Wolfgang WaltherandWolfgang Walther 57ff89be64 Remove outdated heroku manifest 2022-01-01 15:57:58 +01:00
Wolfgang WaltherandWolfgang Walther 4f7ee6f36a cov: Improve jwt claims test-cases 2021-12-25 13:27:53 +01:00
Wolfgang WaltherandWolfgang Walther 14f3b25022 fix: Fix wrong CORS header Authentication -> Authorization
Also refactors defaultCorsPolicy and corsPolicy and cleans up CORS tests
2021-12-25 13:27:53 +01:00
Wolfgang WaltherandWolfgang Walther b99c8c897c fix: Make recursive view parsing work with XMLTABLE + DEFAULT 2021-12-24 19:01:20 +01:00
Steve ChavezandGitHub ac3655df1d feat: minimal health check (#2092) 2021-12-23 08:00:10 -05:00
Laurence IslaandGitHub c858d158aa Add hint to verify table and foreign key existence on a relationship error 2021-12-20 12:11:07 -05:00
dependabot[bot]andGitHub cc8a7ef0c2 build(deps): bump actions/upload-artifact from 2.3.0 to 2.3.1 (#2094) 2021-12-20 10:54:13 -05:00
Steve ChavezandGitHub c86e254a24 fix: is not working with upper/mixed case values (#2081) 2021-12-14 19:13:38 -05:00
dependabot[bot]andSteve Chavez 1c7ca51308 build(deps): bump actions/upload-artifact from 2.2.4 to 2.3.0
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2.2.4 to 2.3.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2.2.4...v2.3.0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-13 12:43:16 -05:00
dependabot[bot]andSteve Chavez 6a3f8ca828 build(deps): bump actions/download-artifact from 2.0.10 to 2.1.0
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2.0.10 to 2.1.0.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2.0.10...v2.1.0)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-13 12:42:53 -05:00
Wolfgang WaltherandWolfgang Walther 031de4d35e fix: Dump db-config option without quotes as it's a boolean 2021-12-11 08:12:59 +01:00
Wolfgang WaltherandWolfgang Walther 595c2a5257 nix(feat): Add postgrest-git-hooks to enable pre-commit and pre-push hooks 2021-12-06 10:40:23 +01:00
Wolfgang WaltherandRemo 74f6e5225c nix: Add weeder to postgrest-coverage
Removes a bit of dead code from SpecHelper.hs
2021-12-04 11:39:56 +01:00
Wolfgang WaltherandWolfgang Walther c581148f1b refactor: Make asJsonSingleF "safe", by accessing only the first element of a json_agg result 2021-12-01 09:19:47 +01:00
Wolfgang WaltherandWolfgang Walther 0e20b47875 fix: Return 204 No Content without Content-Type for PUT 2021-12-01 09:19:47 +01:00
Wolfgang WaltherandWolfgang Walther 1cb00d3c62 fix: Execute deferred constraint triggers when using Prefer: tx=rollback
Resolves #2020
2021-11-30 15:54:12 +01:00
Wolfgang WaltherandWolfgang Walther bbc07d3f40 fix: Link to latest docs in pre-release openapi output
Resolves #2018
2021-11-29 08:40:38 +01:00
Wolfgang WaltherandSteve Chavez 9c6597c351 ci: Use matrix strategy to test against postgres versions 2021-11-27 13:15:42 -05:00
Wolfgang WaltherandSteve Chavez defdcb85ef ci: Run IO tests against all postgres versions
Resolves #1820.
2021-11-27 13:15:42 -05:00
Wolfgang WaltherandSteve Chavez 94115ff134 ci: Disable update of dockerhub description on release
This is not allowed via personal access token right now and could be re-enabled in the future, once solved by docker hub.
2021-11-27 13:11:30 -05:00
Wolfgang WaltherandWolfgang Walther 5a8d9b1246 nix(feat): Add bash completion for git references 2021-11-27 10:57:29 +01:00
Wolfgang WaltherandWolfgang Walther 32ff9dfb48 nix(feat): Add postgrest-loadtest 2021-11-27 10:57:29 +01:00
Wolfgang WaltherandWolfgang Walther ab90bd24e0 nix(feat): Add postgrest-with-git and postgrest-with-pgrst tools 2021-11-27 10:57:29 +01:00
Steve ChavezandGitHub c9f017c632 Bump to v9 (#2045)
* Reorganize the CHANGELOG
* Add missing change for dropping support for embedding hints used with '.'
* Move the pg 14 compatibility entry to Added
2021-11-26 14:10:57 -05:00
steve-chavez 132ecd8d74 Correct count query when doing inner joins 2021-11-26 10:39:19 -05:00
Wolfgang Walther 19c3ee2143 ci: Fix target commit for report workflow 2021-11-26 08:01:00 +01:00
4fb0d1279c test: Add test to confirm ASSERT will return 500 Internal Server Error
Co-authored-by: Dmitry Wagin <dmitry.wagin@ya.ru>
2021-11-25 22:56:49 +01:00
Wolfgang Walther 81f9bf290c ci: Fix permissions for report workflow 2021-11-24 23:01:33 +01:00
Wolfgang WaltherandWolfgang Walther dbef11869c ci: Add report workflow ahead of #1812 2021-11-24 21:23:57 +01:00
Wolfgang WaltherandWolfgang Walther b38564e623 nix(refactor): Make checking for required environment variables a no-op 2021-11-24 21:23:57 +01:00
Wolfgang WaltherandWolfgang Walther 2a2a4bdeac nix: remove unused ghr import 2021-11-24 21:23:57 +01:00
Wolfgang WaltherandWolfgang Walther a65ac9b0f6 feat: drop support for pg 9.5 2021-11-24 19:42:53 +01:00
Wolfgang WaltherandWolfgang Walther 7ed8b10776 test: Remove conditional for pgVersion95 in JsonOperatorSpec
pgVersion95 is the minimum already.
2021-11-24 19:42:53 +01:00
dependabot[bot]andWolfgang Walther 2f239b9048 build(deps): bump actions/cache from 2.1.6 to 2.1.7
Bumps [actions/cache](https://github.com/actions/cache) from 2.1.6 to 2.1.7.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.6...v2.1.7)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-24 18:55:38 +01:00
Wolfgang WaltherandWolfgang Walther c3854e7c74 ci: Add dependabot config for auto updates of github actions 2021-11-24 18:41:32 +01:00
Wolfgang WaltherandRemo b7dcc63e37 nix: Simplify with_tmp_db 2021-11-24 18:06:56 +01:00
Laurence IslaandGitHub 62243852b6 feat: Improve error message for ambiguous embedding
- Adds a relevant hint that includes unambiguous embedding suggestions.
- Joins origin and target into one single embedding key
2021-11-23 22:23:29 -05:00
steve-chavez 1b48531369 remove the db-embed-default-join config 2021-11-23 12:32:51 -05:00
Wolfgang WaltherandWolfgang Walther 1981160536 ci: Use include in stack build matrix 2021-11-21 21:53:24 +01:00
Wolfgang WaltherandWolfgang Walther f3eb8a317d ci: Disable fail-fast in stack matrix to keep other builds alive on failure 2021-11-21 21:53:24 +01:00
Wolfgang WaltherandWolfgang Walther 7aedebecee ci: Visually separate jobs in workflow file 2021-11-21 18:40:42 +01:00
Wolfgang WaltherandWolfgang Walther d5afdc7c53 ci: Use matrix strategy to build with stack 2021-11-21 18:40:42 +01:00
Wolfgang WaltherandWolfgang Walther b7c261f520 ci: Update actions 2021-11-21 18:40:42 +01:00
Wolfgang WaltherandWolfgang Walther 942391c508 ci: Add .github scripts to shellcheck 2021-11-21 18:40:42 +01:00
Wolfgang WaltherandWolfgang Walther f333305491 ci: Add setup-nix action to reuse nix-related code 2021-11-21 18:40:42 +01:00
Wolfgang WaltherandWolfgang Walther 88283801cb ci: Limit release job permissions to minimum required 2021-11-21 18:40:42 +01:00
Laurence IslaandGitHub 4e07846c5d fix: Add tests for embedding on overloaded functions with a default param
Resolves #1672

Was actually fixed in #1841 / 67c2ed7, this just adds the tests to confirm it.
2021-11-19 09:00:16 +01:00
steve-chavez bf58a3600a refactor: remove pgFmtLit 2021-11-18 17:01:58 -05:00
steve-chavez bae5afbad6 feat: allow unknown on the is operator
Also make the IS values strict at the Parser level since IS cannot be
parametrized.
2021-11-18 17:01:58 -05:00
Laurence IslaandGitHub 91689e28f0 Remove partitions from the schema cache
breaking: Embedding, OpenApi and other features related to the schema cache are no longer allowed on partitions.
2021-11-18 14:12:14 -05:00
Wolfgang WaltherandRemo 041f73cae8 nix(feat): Add actionlint to postgrest-lint 2021-11-17 18:58:34 +01:00
monacoremo 43c8bde8ce refactor: Add doctests 2021-11-13 04:07:46 +01:00
monacoremo 82dd1b732d ci: disable cirrusci stack work cache which seems to create issues 2021-11-12 20:25:22 +01:00
monacoremo 41ac0d5d93 refactor: Remove Protolude.Conv from Middleware 2021-11-12 20:25:22 +01:00
monacoremo 60db026dcc refactor: Remove Protolude.Conv from RangeQuery 2021-11-12 20:25:22 +01:00
monacoremo 47b6f04a66 refactor: Remove Protolude.Conv from AppState 2021-11-12 20:25:22 +01:00
monacoremo f228cccbae refactor: Remove Protolude.Conv from Proxy 2021-11-12 20:25:22 +01:00
monacoremo 3820590649 refactor: Remove Protolude.Conv from SqlFragment 2021-11-12 20:25:22 +01:00
monacoremo 8cc8531e49 refactor: Remove Protolude.Conv from Workers 2021-11-12 20:25:22 +01:00
monacoremo 65062364c4 refactor: Remove Protolude.Conv from Error 2021-11-12 20:25:22 +01:00
monacoremo ebe66cb985 refactor: Remove Protolude.Conv from Statements 2021-11-12 20:25:22 +01:00
monacoremo 9c172db534 refactor: Remove Protolude.Conv from CLI 2021-11-12 20:25:22 +01:00
monacoremo de531b6b41 refactor: Remove Protolude.Conv from OpenAPIw 2021-11-12 20:25:22 +01:00
monacoremo 71cb8afc0d refactor: Remove Protolude.Conv from JSPath 2021-11-12 20:25:22 +01:00
monacoremo 36dbd09459 refactor: Remove Protolude.Conv from ApiRequest 2021-11-12 20:25:22 +01:00
monacoremo fa4df95f55 refactor: Remove Protolude.Conv from Parsers 2021-11-12 20:25:22 +01:00
monacoremo 4104f5fb75 refactor: Remove Protolude.Conv from App 2021-11-12 20:25:22 +01:00
monacoremo abb6167caa refactor: Remove Protolude.Conv from DbStructure 2021-11-12 20:25:22 +01:00
monacoremo 15ea7dfe09 refactor: Remove Protolude.Conv from GucHeader 2021-11-12 20:25:22 +01:00
monacoremo a2c2544271 refactor: Remove Protolude.Conv from Config 2021-11-12 20:25:22 +01:00
monacoremo 2b22f889bd refactor: Remove GHC.Show instances from JSPath module 2021-11-09 19:13:52 +01:00
monacoremo cd3013569e refactor: Remove GHC.Show instances from QualifiedIdentifier and Config 2021-11-09 19:13:52 +01:00
monacoremo 5dc37fc8e8 refactor: Remove GHC.Show instances from Request.Types module 2021-11-09 19:13:52 +01:00
monacoremo 038d84b62d refactor: Remove GHC.Show instances in Preferences module 2021-11-09 19:13:52 +01:00
Laurence IslaandGitHub 40f9a6068a Allow overloaded functions if one has a single unnamed JSON param
Avoids the breaking change in #1927: If there's a function "my_func" having a single unnamed json param and other overloaded pairs(with any number of params), PostgREST won't be able to resolve a POST request to "my_func".
2021-11-08 18:26:33 -05:00
monacoremo 5cd7d35966 refactor: Make import aliases consistent across the codebase 2021-11-05 08:08:00 +01:00
monacoremo 71262fbc5c nix: Hook hsie check-aliases into postgrest-lint 2021-11-05 08:08:00 +01:00
monacoremo d16a4a9a2b nix: fix statix hints 2021-11-02 23:10:23 +01:00
monacoremo 2357961f48 nix: add statix to postgrest-style 2021-11-02 23:10:23 +01:00
monacoremo 162c4e38af Update stackage to LTS 18.14 (GHC 8.10.7) 2021-11-01 23:23:19 +01:00
monacoremo 0f95900822 nix: Upgrade hsie tool to GHC 8.10.7 2021-11-01 23:23:19 +01:00
monacoremo 8ac14447ef nix: Update nixpkgs to latest unstable version 2021-11-01 23:23:19 +01:00
Wolfgang WaltherandWolfgang Walther a1b3bca7fc fix: Make pfkSourceColumns query work with standard_conforming_strings = off 2021-10-28 08:21:11 +02:00
Wolfgang WaltherandWolfgang Walther 8cde90b7fe refactor: Use include_triggers argument to pg_relation_is_updatable instead of manual check 2021-10-26 19:15:02 +02:00
3b83f536ca Correct inner join to use proper aliases (#1978)
"_" was used for all the internal json_agg aliases. Now an alias
with "_" plus a local table name is used.

Co-authored-by: Cole Arendt <cole.arendt@outlook.com>
2021-10-18 17:34:31 -05:00
laurenceislaandGitHub 42d7cf8ae5 Add db-embed-default-join config parameter to the configuration example 2021-10-15 19:09:40 -05:00
laurenceislaandGitHub 627c3c34b7 feat: Make GUC names for headers, cookies and jwt claims compatible with PostgreSQL v14
Getting the value for a header GUC on PostgreSQL v14 is done using `current_setting('request.headers')::json->>'name-of-header'` and in a similar way for `request.cookies` and `request.jwt.claims`

PostgreSQL versions below 14 can opt in to the new JSON GUCs by setting the `db-use-legacy-gucs` config option to false (true by default)
2021-10-12 12:58:52 -05:00
steve-chavez ee56dd5db1 feat: filter top-level resource with embed filter
This is enabled by adding `!inner` to the embedded resource

/projects?select=*,clients!inner(*)&clients.id=eq.12

This behaviour can be enabled by default with the config option

db-embed-default-join='inner'

Which saves the need for specifying `!inner` on every request.
If this is enabled, the previous behavior can be restored
per request by specifying `!left`  on the embedded resource.

/projects?select=*,clients!left(*)&clients.id=eq.12`

Tested on M20/02M/M2M relationships, views, RPC.
2021-10-04 13:46:32 -05:00
steve-chavez bf91187e63 Drop support for embed hints used as '.'
Hints used like `select=projects.client_id(*)` were already
deprecated.

'!' should be used from now on `select=projects!client_id(*)`
2021-10-04 13:46:32 -05:00
monacoremo 2982c5de2e nix: Fix postgrest-check 2021-09-23 22:36:58 +02:00
monacoremo add95c32ca ci: Fix triggering events and release actions
* Run CI only on pull requests, push to main and push
  to tags matching v*. This avoids running the CI more than
  once for a single user action
* Change pre-releases, which are now identified by a version
  number with 4 components (e.g., 1.1.1.1)
* Change how release artifacts are packaged, ensuring they
  are executable and compressed with xz
2021-09-23 22:36:58 +02:00
laurenceislaandGitHub 243e4f0408 test: correct limit proc results tests (#1951)
Query params where being appended using `?` instead of `&`
2021-09-16 22:03:35 -05:00
steve-chavez 885ff60138 pre-release: Bump to v9.0.0-a1 2021-09-14 15:51:32 -05:00
steve-chavez 72a28a145d feat: Allow backslash escaping inside double quotes
Examples:

- ?col=in.("Double\"Quote")
- ?col=in.("Back\\slash")

Backslashed chars get passed as is

- ?col=in.("\a\b\c") = ?col=in.(abc)
2021-09-13 19:06:30 -05:00
steve-chavez 4a594d5e95 fix: using " and \ as values on the in filter
Properly escape the array literal built at the QueryBuilder
stage.
2021-09-13 19:06:30 -05:00
Remo 0ba133a0bd ci: Fix FreeBSD build 2021-09-08 19:11:00 +02:00
Wolfgang WaltherandSteve Chavez b05898d17f fix: Return array of objects for RPCs with single-column TABLE return type
This is a regression after changing responses for RPCs with INOUT or OUT arguments in #1615.

Resolves #1930.
2021-09-02 10:13:03 -05:00
steve-chavez d4c6abbaec feat: RPC POST for function w/single unnamed param
For POST on RPC, allows:

* passing a json object without using `Prefer: params=single-object`
  The function must be defined with a single unnamed json param and
  `Content-Type: application/json` must be specified.

* uploading binary to a function
  The function must be defined with a single unnamed bytea param and
  `Content-Type: application/octet-stream` must be specified.

* uploading raw text to a function
  The function must be defined with a single unnamed text param and
  `Content-Type: text/plain` must be specified.

BREAKING CHANGE If there's a function "my_func" having a single
unnamed json param and other overloaded pairs(with any number of
params), PostgREST won't be able to resolve a POST request to
"my_func". For solving this, you can name the unnamed json param.

my_func(json) -> my_func(prm json)
2021-08-30 18:17:59 -05:00
steve-chavez caaa34b5de refactor: add CallQuery type
Shortens requestToCallProcQuery, which should generate SQL in a more
direct way.
2021-08-30 18:17:59 -05:00
steve-chavez c9a60373f6 refactor: PgArg to ProcParam
Clarify the difference between arguments and parameters.
Parameters are part of the function definition, arguments are the values
passed to the function.

Also clarify the findProc function comments and error message.
2021-08-30 18:17:59 -05:00
laurenceislaandGitHub ed5072f4b1 fix: Detect default values for String types in OpenAPI output (#1928) 2021-08-27 21:59:54 -05:00
Felix YanandRemo ed98ac9d7e Allow newer base, hspec, and retry
Builds fine and all tests are passing here, with the new versions.
2021-08-19 18:44:30 +02:00
Gaurav GautamandGitHub d9f1ae6a46 feat: Add Retry-After hint (#1916)
Add Retry-After header when response status is 503.
Its value is the connection worker delay(seconds) when
it's recovering.

This closes issue #1817.
2021-08-18 10:14:29 -05:00
Wolfgang WaltherandRemo 5874482f03 nix(fix): Database cluster not cleaned up before artifact upload 2021-08-15 12:35:52 +02:00
Wolfgang WaltherandRemo 0511e99dd4 nix(refactor): Add withPath to checkedShellScript 2021-08-15 12:35:52 +02:00
laurenceislaandGitHub 4e4548d702 feat: Include partitioned tables into the schema cache
Allows embedding, UPSERT, INSERT with Location response, OPTIONS request and OpenAPI support for partitioned tables
2021-08-13 15:06:56 -05:00
Remo b3899e7aa4 ci: Set Docker Hub username 2021-08-08 06:42:59 +02:00
monacoremo 6d9fcfa09f ci: Remove unused create_test_db script 2021-08-08 06:42:59 +02:00
monacoremo 38e5b76996 ci: Remove appveyor.yml 2021-08-08 06:42:59 +02:00
monacoremo ca4539e78d ci: Remove .travis.yml 2021-08-08 06:42:59 +02:00
monacoremo a54a61d00e ci: Remove .circleci/config.yml 2021-08-08 06:42:59 +02:00
monacoremo 142fc06b15 ci: Add .github/release script that tags and pushes a new release 2021-08-08 06:42:59 +02:00
monacoremo c7b1cda3ed ci: Move codecov.yml and add thresholds as intermittent fix to random coverage changes 2021-08-08 06:42:59 +02:00
monacoremo fb2adea5e9 ci: Adjust build status badge in READMEs 2021-08-08 06:42:59 +02:00
monacoremo d9b15adb35 ci: Use GitHub Actions for CI/CD 2021-08-08 06:42:59 +02:00
steve-chavez 0bc7c034cf Bump to version v8.0.0 2021-07-25 23:12:51 -05:00
steve-chavez cbe0e5254e Reorganize the CHANGELOG
* Removed some entries that were fixes for the nigthly versions, not
stable versions.
* Reference the actual issue number(not the PR number) on some entries
* Remove duplicate entries
* Clarify some entries
* Grouped CLI entries
2021-07-25 23:12:51 -05:00
Remo RechkemmerandGitHub 68bf17118d Upgrade Nix dependencies including nixpkgs and static-haskell-nix (#1899) 2021-07-23 19:54:29 +02:00
Remo RechkemmerandGitHub fdf6c510ec Update changelog (#1897) 2021-07-17 00:14:37 +02:00
laurenceislaandGitHub db95bd1c37 feat: Add compatibility with connection poolers on transaction mode
Update the hasql-transaction library to version 1.0.1

Add hints and kill thread at configuration read when using incompatible pooling modes: statement pooling and transaction pooling with prepared statements enabled.
2021-07-15 13:39:55 -05:00
monacoremo 83cc358e1b nix: Update static-haskell-nix and fix stripped executable 2021-07-15 09:41:52 +02:00
monacoremo 11374c2fae ci: upgrade stackage to LTS 18.2 / GHC 8.10.4 2021-07-15 09:41:52 +02:00
monacoremo 27c8dabd8d nix: Use GHC 8.10.4 in Nix build environment 2021-07-15 09:41:52 +02:00
monacoremo 9568c4605a nix: Update nixpkgs and static-haskell-nix 2021-07-15 09:41:52 +02:00
Laurence Isla 6fe88ef53c Add Oblivious as a sponsor and a Sponsors section to the docker hub description 2021-07-12 19:30:35 -05:00
steve-chavez 0961a587c0 Correct openapi ignore-privs table comments 2021-07-06 11:19:54 -05:00
laurenceislaandGitHub 67c2ed7c62 feat: Show comprehensive error when an RPC is not found in a stale schema cache (#1841)
* Add 300 response for overloaded functions with same argument names but different types
* Add Path type to handle validations and errors before defining the Target
2021-07-06 09:19:26 -05:00
steve-chavez f3a184af01 Correct openapi ignore mode to filter schemas
Also rename openapi-mode options

* follow-acl -> follow-privileges
* ignore-acl -> ignore-privileges
2021-06-27 22:48:49 -05:00
Steve ChavezandGitHub 41d119b19f feat: add openapi-mode config (#1881)
* openapi-mode="follow-acl"(default): follows access control for the
JWT role.
* openapi-mode="ignore-acl": ignores access control for the JWT role.
* openapi-mode="disabled": disables OpenAPI output, the root endpoint
replies with 404 Not Found.
2021-06-25 19:20:04 -05:00
laurenceislaandGitHub 214a92f207 Change the default db-channel-enabled config value from False to True (#1880) 2021-06-25 11:12:28 -05:00
steve-chavez 1759c1c75b Update CirrusCI encrypted var 2021-06-22 20:10:43 -05:00
Steve ChavezandGitHub 2944cf94c7 feat: Add time to startup/worker logs (#1872)
BREAKING CHANGE

Sends startup/worker logs to stderr to differentiate them from
access logs, which go to stdout
2021-06-19 15:55:18 -05:00
laurenceislaandGitHub 59e0b006a5 fix: Location header with a null PK on a view (#1875)
Regression after #1475
2021-06-18 18:09:19 -05:00
laurenceislaandGitHub 1b12b112a1 fix: Add a hint and clarification to the no relationship found error (#1858) 2021-05-31 12:15:31 -05:00
Alexander LjungbergandGitHub 88a481da8a fix: drop gzip support, improve performance. (#1854)
BREAKING CHANGE

Not gzipping the output more than doubles the throughput for a simple "do nothing" RPC function.

Postgrest's gzip support was surprisingly slow, lacked in configurability, was undocumented and untested.

It was possible to achieve this before by not sending `Accept-Encoding: gzip`, but that turned into a bit of an undocumented gotcha. Most browsers do send it and by default a proxy pass would forward this header. Better to not support gzip at all, further focusing on ensuring Postgrest is a minimal layer over Postgres.

Nginx, or whatever proxy, provides critical options such as min length, vary headers and so forth, and will likely have much better performance as well.
2021-05-30 17:33:44 -05:00
steve-chavez d99909c403 feat: request.spec GUC for db-root-spec
The request.spec GUC contains the schema cache structure in json.

It's only available when the root endpoint(/) is requested and when
db-root-spec is not empty.

Also correct db-root-spec to accept a schema.
2021-05-30 16:48:08 -05:00
steve-chavez f169661ce6 refactor: move DbStructure.PgVersion to Config
Now that PgVersion is not part of DbStructure, Config is a more apt
module for it.

Also rename getDbStructure to queryDbStructure. AppState also had a
getDbStructure function for a record field.
2021-05-30 16:48:08 -05:00
steve-chavez 823348a72a refactor: configDbPreRequest to QualifiedIdentifier 2021-05-30 16:48:08 -05:00
steve-chavez 5c75f0dcc2 Update BACKERS.md
Correct wrong link
2021-05-16 14:32:51 -05:00
steve-chavez 601c802f23 Update BACKERS.md 2021-05-16 14:23:10 -05:00
Laurence IslaandWolfgang Walther b4ca70708a fix: Fix requests for overloaded functions from html forms to no longer hang
Fixes #1846
2021-05-13 10:07:49 +02:00
steve-chavez 082c91c855 refactor: remove pgVersion from DbStructure
Also use a dedicated MVar for listener
2021-04-30 10:06:07 -05:00
steve-chavez 0f6a13191c refactor: remove colFK from Column 2021-04-30 10:06:07 -05:00
Remo RechkemmerandGitHub 6670a3214b refactor: Move accept content type code from App into ApiRequest (#1830) 2021-04-25 21:58:03 +02:00
monacoremo 63e0292e23 fix: Panic when attempting to run with unix socket on non-unix host 2021-04-24 19:42:58 +02:00
monacoremo 59f0fe5b32 fix: Properly close unix domain socket on exit 2021-04-24 19:42:58 +02:00
monacoremo 387c387073 fix: Improve log message on config load 2021-04-24 19:42:58 +02:00
monacoremo acd787a5af refactor: Split main/Main.hs into library modules 2021-04-24 19:42:58 +02:00
Wolfgang WaltherandWolfgang Walther 4ded01b104 nix(fix): Fix missed devtools->devTools renaming 2021-04-24 15:25:10 +02:00
Wolfgang WaltherandWolfgang Walther c691b37f76 nix(feat): Create all temporary directories with script name template
Temporary directories are now created in $TMPDIR/postgrest/script-name-XXX. This allows recognizing the origin of a single temporary directory and uploading the whole $TMPDIR/postgrest folder as an artifact in CI.
2021-04-24 15:07:13 +02:00
Wolfgang WaltherandWolfgang Walther 14ddecc81a nix(fix): Avoid rebuilding during postgrest-coverage 2021-04-24 15:07:13 +02:00
Wolfgang WaltherandWolfgang Walther 3611eab39c nix(fix): Make postgrest-dump-schema work in nix-shell --pure 2021-04-24 15:07:13 +02:00
Wolfgang WaltherandWolfgang Walther e5350d2d50 nix(refactor): Split cabalTools from devtools
This allows to use `postgrest-build` in CI without pulling in a lot of dependencies.
2021-04-24 15:07:13 +02:00
Wolfgang WaltherandWolfgang Walther 249d4117e0 nix(style): reorder tools in default.nix 2021-04-24 15:07:13 +02:00
Wolfgang WaltherandWolfgang Walther ba47a54293 nix(refactor): Make with_tmp_db true checkedShellScript
This extends the interface of withTmpDb to allow loading different sets
of database fixtures via `postgrest-with-postgresql-xx --fixtures
<path>`.
2021-04-24 15:07:13 +02:00
laurenceislaandGitHub f6b3a5cc22 fix: Make OPTIONS consider view instead of triggers (#1824) 2021-04-23 18:12:43 -05:00
steve-chavez 6e0ef95320 Add GNUHost as a sponsor 2021-04-23 14:29:33 -05:00
steve-chavez 082da78f4c fix: disregard internal junction when embedding
Added test cases for a lone internal junction and an internal junction
exposed homonym
2021-04-21 10:28:20 -05:00
steve-chavez 21d280497b refactor: Relation to Relationship
Also remove the unused UnknownRelationship error.
2021-04-21 10:28:20 -05:00
Wolfgang WaltherandWolfgang Walther 7ca0d46936 nix(fix): Fix postgrest-push-cachix in nix-shell --pure 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther b566481477 nix(refactor): Move shell scripts to subdirectory nix/tools 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther fe497fc438 nix(refactor): Add buildToolbox to streamline module interface towards shell.nix 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther de73ced34a nix(feat): Change memoryTests shell argument to memory
Avoids confusion with `memoryTest`.
2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther bc879507ae nix(fix): Make naming of tools in postgrest-release- namespace consistent 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther 280b89d87c lint: Apply shellcheck suggestions for bash scripts in test/* 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther eee5d5d482 nix(feat): Add shellcheck to postgrest-lint for bash scripts in test/* 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther 2e73886adc nix(refactor): Split memoryTest from tests.nix into memory.nix 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther e199337a53 nix(feat): Check required environment variables in postgrest-release tools 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther 2417cfb02f nix(feat): Force optional arguments to go before positional arguments in argbash scripts
This improves passing subcommands to postgrest-with- tools.
2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther 2559d32912 nix(feat): Add argbash bash completion support 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther c2df7d8198 nix(fix): Avoid losing exit trap through subcommand in postgrest-with-all 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther c8edfac39e nix(feat): Add argbash support to checked-shell-script
Each postgrest- script now has a -h/--help option showing the
checked-shell-script.docs argument.

Additional CLI arguments can be defined through the
checked-shell-script.args argument using the argbash template syntax:
https://argbash.readthedocs.io/en/stable/guide.html
2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther 39d4646a4c nix(refactor): Add withEnv to checked-shell-script 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther 40b1dccc54 nix(fix): Load withTmpDb into nix to have shellcheck run on it 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther 32bac81d91 nix(refactor): Move withPostgresqlVersions and withAllVersions to new withTools 2021-04-18 13:51:39 +02:00
Wolfgang WaltherandWolfgang Walther dccce946e3 nix(refactor): Move name arguments into scripts and use global postgresql in tests.nix 2021-04-18 13:51:39 +02:00
Remo RechkemmerandGitHub 915d568257 ci: Use cachix authtoken in CI (#1821) 2021-04-17 16:25:28 +02:00
Steve ChavezandGitHub 698bfe6e7b refactor: make Junction non-recursive (#1818)
Also Cardinality now includes a Junction or ConstraintName
2021-04-16 13:12:58 -05:00
Wolfgang WaltherandWolfgang Walther 1f206a560b ci: check sha512 of downloaded codecov script 2021-04-16 14:10:49 +02:00
Wolfgang WaltherandWolfgang Walther 42f8f4fdcb ci: remove stack-test-memory 2021-04-16 14:10:49 +02:00
Wolfgang WaltherandWolfgang Walther c67cd5e6fc ci: reorder tests in nix-test to improve performance and give early coverage feedback 2021-04-16 14:10:49 +02:00
Wolfgang WaltherandWolfgang Walther 67e3886547 ci: split nix-build-test into nix-build nix-test and introduce circleci contexts 2021-04-16 14:10:49 +02:00
laurenceislaandGitHub 4b46c4eff3 feat: Add explicit headers-only POST request using Prefer header
Allows using header "Prefer=headers-only" explicitly to get a response with only a Location header.

BREAKING CHANGE: Change default for POST request from headers-only to minimal

Resolves #1656
2021-04-14 00:14:12 +02:00
Wolfgang WaltherandWolfgang Walther cd09cc6c52 ci: fix cirrus build timeout 2021-04-14 00:13:08 +02:00
Remo RechkemmerandGitHub c2c7bbe9dd refactor: remove obsoleted Private module (#1811) 2021-04-12 20:47:22 +02:00
Wolfgang WaltherandWolfgang Walther d40d704043 remove unused static assets 2021-04-12 20:31:32 +02:00
Wolfgang WaltherandWolfgang Walther 91d0c732f0 add comment to Setup.hs 2021-04-12 20:31:32 +02:00
Remo RechkemmerandGitHub 801e229c59 refactor: Carve CLI and Version modules out of Config (#1806) 2021-04-12 19:17:47 +02:00
Wolfgang WaltherandWolfgang Walther 67ca814d0b nix: properly throw away tix files for postgrest-watch, resolves #1807 2021-04-12 19:03:16 +02:00
Wolfgang WaltherandWolfgang Walther fc791a6320 nix: add withTmpDir to checked-shell-script 2021-04-12 19:03:16 +02:00
579a626de2 nix: move withtmpdb to separate file
Co-authored-by: Remo Rechkemmer <59358383+monacoremo@users.noreply.github.com>
2021-04-12 19:03:16 +02:00
Remo RechkemmerandGitHub f99fd6cbad refactor: Split up Types.hs and logically organize modules (#1793) 2021-04-11 18:28:01 +02:00
Remo RechkemmerandGitHub 8c44410ce0 remove misleadingly named emptyOnFalse and similar (#1803) 2021-04-11 10:38:35 +02:00
Wolfgang WaltherandWolfgang Walther 9118a4a780 fix random codecov/project drops in CI, resolves #1800
Those are caused by running the spec tests with `-threaded`, which results in a race to write to spec.tix.
2021-04-10 20:38:51 +02:00
Wolfgang WaltherandGitHub a5372e4713 nix: show better error messages for missing arguments in hsie scripts (#1798) 2021-04-09 18:48:00 +02:00
Jérémy PagéandGitHub 67f555d24e nix: Fix incorrect use of bash variable (#1797) 2021-04-08 14:33:18 +02:00
7f11c1a991 fix: Void functions return null instead of empty body (#1795)
* Change empty body reponse to null for scalar functions returning void or null values

* Add test for functions returning an integer with null value

Co-authored-by: Laurence Isla <lau.isla.c@gmail.com>
2021-04-08 01:45:13 +02:00
steve-chavez 9c79a4174c Add Contributor Covenant Code of Conduct v2 2021-04-06 22:22:13 -05:00
Wolfgang WaltherandWolfgang Walther 376beac22f update contributing guidelines 2021-04-06 17:07:14 +02:00
Remo RechkemmerandGitHub 65e7f9e846 nix: Add a tool for checking Haskell imports and exports (#1768) 2021-04-06 11:46:12 +02:00
Boris KorzunandSteve Chavez 9f074cecce cirrusci: update FreeBSD pkg 2021-03-26 10:13:38 -05:00
steve-chavez 71f6061d30 Add Supabase as a sponsor 2021-03-25 20:42:43 -05:00
Robert VollmertandSteve Chavez 3a466aea9a tests: don't match JSON bodies literally
The change of hash function with hashable-1.3.1.0 changes object
ordering in JSON output, causing some test failures:
https://app.circleci.com/pipelines/github/PostgREST/postgrest/805/workflows/067844c9-9ce4-49e8-8790-315625ab309b/jobs/8309
e.g.:

> expected: "[{\"b\":\"baz\",\"a\":\"bar\"}]"
>  but got: "[{\"a\":\"bar\",\"b\":\"baz\"}]"

This changes the tests to not compare against the body literally,
and instead use the ResponseMatcher instance from Test.Hspec.Wai.JSON.
2021-03-14 12:45:02 -05:00
Robert VollmertandSteve Chavez 5baec4819f circleci: update base image
Switches to the non-legacy cimg/base: https://circleci.com/developer/images/image/cimg/base

In particular, this updates the gcc version to fix the build of
cryptonite-0.27: https://github.com/haskell-crypto/cryptonite/issues/326
2021-03-14 12:45:02 -05:00
steve-chavez d3a8b5f6e1 Change db-load-guc-config to db-config
Make clear that in-db config is being read
2021-03-05 19:03:15 -05:00
steve-chavez 498e77215a Reread in-db config when recoverying connection
* Separate reading files from whole config re-read
* Only reload external file on SIGUSR2/NOTIFY
2021-03-05 19:03:15 -05:00
steve-chavez 6750a5c44d Restrict db settings to current db and global
Global settings are overriden by database specific settings if they
share common ones.
2021-03-05 19:03:15 -05:00
steve-chavez e4516ab606 Correct db settings to use "_" instead of "-"
GUC settings with dashes cannot be shown with show or current_setting.

https://www.postgresql.org/message-id/flat/20210209144059.GA21360%40depesz.com
2021-03-05 19:03:15 -05:00
Robert VollmertandSteve Chavez c3ccaf1a08 Allow lens-5.0 2021-02-28 13:02:37 -05:00
Robert VollmertandSteve Chavez a7dab6d95a cirrusci: update to FreeBSD 12.2
This fixes CI failure because FreeBSD 12.1 became EOL in Jan 2021
(3 months after release of 12.2): https://www.freebsd.org/security/#sup
2021-02-27 13:26:02 -05:00
steve-chavez b50b67491d Update CYBERTEC logo and BACKERS 2021-02-25 20:03:38 -05:00
Remo RechkemmerandGitHub e6973f966b refactor: App.hs and related changes (#1725)
* Use ExceptT to avoid 'staircasing' case analysis in App.hs
* Split large function in App.hs into individual handler functions
* Adapt API of Auth.hs, OpenApi.hs etc. to simplify the use of those modules in App.hs
* Split optional rollback functionality into Middleware
* Unify SimpleError and ApiRequestError into one Error type, so it can be used across modules
2021-02-23 22:41:48 +01:00
Remo RechkemmerandGitHub 0ddd676ef0 nix: Add a script to generate a graph of Haskell imports (#1751) 2021-02-07 22:15:44 +01:00
Steve ChavezandGitHub c93e8f9e0c Correct hardcoded postgrest_test_authenticator (#1743)
Also remove panic from Config and correct io test not running
2021-01-25 18:38:46 -05:00
steve-chavez 6557f1f9c0 correct NOTIFY config reload dying on error 2021-01-22 15:56:08 -05:00
steve-chavez 17af56adb1 refactor: config validation inside readAppConfig
Remove Either from configJwtRoleClaimKey/configServerUnixSocketMode
and remove whenLefts.
2021-01-22 15:56:08 -05:00
steve-chavez 4344cc9202 refactor: use optString and move overrideFrom 2021-01-22 15:56:08 -05:00
steve-chavez 125ea8f6d9 feat: allow reloading config with NOTIFY
Enables reloading the config by doing:
NOTIFY pgrst, 'reload config'

Adds an alias for reloading the schema cache:
NOTIFY pgrst, 'reload schema'
2021-01-19 13:49:40 -05:00
steve-chavez 9c005fc683 feat: get configuration parameters from the db
Allows configuring postgrest from the db by setting config parameters
on the connection role. For example:

ALTER ROLE postgrest_test_authenticator
SET pgrst.jwt-secret = "REALLYREALLYREALLYREALLYVERYSAFE"

The above wWill set the `jwt-secret` config option accordingly.

SUPERUSER privileges are required for ALTERing role settings,
so this might not work on some cloud-managed databases.

This feature is enabled by default, for disabling it you can add the
following to the config file:

db-load-guc-config = false
2021-01-19 13:49:40 -05:00
steve-chavez 674615041a Correct --dump-schema swallowing error
When there's no connection to pg, the following error happens
when running --dump-schema:

postgrest: user error (Pattern match failure in do expression at
main/Main.hs:361:5-14)

Now it shows a regular "could not connect to server.." error.
2021-01-19 13:49:40 -05:00
steve-chavez 15039553db Correct openapi preparing statements by default 2021-01-19 13:49:40 -05:00
Wolfgang WaltherandWolfgang Walther ba86b479f8 cov: refactor ApiRequest profile header detection 2021-01-14 16:40:29 +01:00
Wolfgang WaltherandWolfgang Walther 6dd126461e cov: Remove unused code 2021-01-14 16:40:29 +01:00
Wolfgang WaltherandWolfgang Walther 97d8456382 cov: Add io tests for basic cli commands and invalid config options
Full code coverage for Config.hs except expected overlays.
2021-01-14 16:40:29 +01:00
Wolfgang WaltherandRemo Rechkemmer ab6f90aa78 nix: Add postgrest-with-* tools to run with temporary databases of different versions 2021-01-04 23:14:44 +01:00
Wolfgang WaltherandWolfgang Walther 522308217a refactor: Combine relConstraint and relJunction in relLink 2021-01-03 17:54:28 +01:00
Wolfgang WaltherandWolfgang Walther d173b7d8d6 cov: remove unused fields from type Column 2021-01-03 17:54:28 +01:00
Wolfgang WaltherandWolfgang Walther b8e6450af1 refactor: simplify addXRels 2021-01-03 17:54:28 +01:00
Wolfgang WaltherandWolfgang Walther 45ebaf0ad8 cov: Add comment on SCHEMA in test fixtures 2021-01-03 17:54:28 +01:00
Wolfgang WaltherandWolfgang Walther 1edbef0538 Add STABLE RPCs in test fixtures to increase coverage 2021-01-03 17:54:28 +01:00
Wolfgang WaltherandWolfgang Walther 6efa304142 refactor: Move parseArg to SQL 2021-01-03 17:54:28 +01:00
Wolfgang WaltherandWolfgang Walther eed6015e70 refactor: Move addHasVariadic to SQL 2021-01-03 17:54:28 +01:00
Wolfgang WaltherandWolfgang Walther 94b516d074 Add flag to postgrest.cabal to disable collecting coverage data 2021-01-02 13:51:00 +01:00
Wolfgang WaltherandWolfgang Walther 56a7bc5030 nix: Fix build errors with stale .tix files present 2021-01-02 13:51:00 +01:00
Wolfgang WaltherandWolfgang Walther af2788468e Add basic error handling to --dump-schema 2021-01-02 13:50:19 +01:00
Wolfgang Walther cc54135bf0 Rename master branch to main; replace whiteList with isAllowed 2020-12-31 14:37:52 +01:00
Wolfgang WaltherandGitHub 519550e9af Add codecov to README.md 2020-12-30 17:52:15 +01:00
Wolfgang WaltherandRemo Rechkemmer 93541035f6 Fix postgrest-coverage throwing error when installed with nix-env 2020-12-30 16:38:13 +01:00
Wolfgang WaltherandRemo Rechkemmer 568477b6af Fix io tests not collecting coverage data when using run()
Before, postgrest was killed after each test and no coverage data was saved.
Now the process is terminated gracefully via SIGTERM first to allow writing the .tix file.
2020-12-30 14:51:58 +01:00
Wolfgang WaltherandGitHub c7f0d42323 Add postgrest-coverage to show and upload hpc reports to codecov 2020-12-30 13:22:25 +01:00
Wolfgang WaltherandGitHub 2015688312 Fix spec test concurrency issue related to tx=commit on simple_pk 2020-12-29 23:58:37 +01:00
Wolfgang WaltherandWolfgang Walther 8e02551956 Fix "unused-imports" warning for windows build 2020-12-29 16:07:07 +01:00
Remo RechkemmerandGitHub 96478ed016 Add docs to all checked shell scripts in nix 2020-12-28 13:23:12 +01:00
Remo RechkemmerandGitHub 2425cddaed improve io-tests
Tests are now run through unix socket by default allowing
parallel execution. Test setup configuration is done via
environment variables instead of config files.
2020-12-24 16:09:25 +01:00
Wolfgang WaltherandWolfgang Walther b5185de706 nix: add "cd $rootdir" to all shell scripts via checked-shell-script 2020-12-24 14:44:01 +01:00
Wolfgang WaltherandWolfgang Walther 56557c9c40 ci: remove broken travis coverage job 2020-12-23 19:39:40 +01:00
Wolfgang WaltherandWolfgang Walther 2cbe1ba903 feat: Add --example cli option to show example config file 2020-12-23 19:39:40 +01:00
Wolfgang WaltherandWolfgang Walther 69b459e7b6 Use environment variables for configuration in developer tooling
Avoid the need for configuration files for temporary database connections.
2020-12-23 19:39:40 +01:00
Wolfgang WaltherandWolfgang Walther add326a79d ci: skip postgrest-release-dockerhubdescription for nightly builds 2020-12-23 19:39:40 +01:00
Wolfgang WaltherandWolfgang Walther b7fc393e49 feat: Read config directly from environment variables
resolves #1624
2020-12-23 19:39:40 +01:00
monacoremoandSteve Chavez f2f639e484 update stack to lts-16.26 2020-12-23 09:39:36 -05:00
monacoremoandSteve Chavez 9cfc66a6a4 upgrade nixpkgs and apply related fixes 2020-12-23 09:39:36 -05:00
Remo RechkemmerandGitHub bf141ca13f feat: Added --dump-schema CLI option to dump JSON of dbStructure schema cache.
Added postgrest-dump-schema to nix-shell returning a YAML dump of the test fixtures.

Authored-by: monacoremo <monacoremo>
2020-12-20 10:22:52 +01:00
monacoremoandSteve Chavez fe09637711 catch ReadTimeout in IO tests 2020-12-17 17:05:03 -05:00
Wolfgang WaltherandWolfgang Walther dbe3c163bd run spec tests in parallel
reduces time to run postgrest-test-spec-all by 20-25%
2020-12-14 00:42:03 +01:00
monacoremoandWolfgang Walther a52c114ff4 add test for spec idempotence 2020-12-13 22:13:05 +01:00
Remo RechkemmerandGitHub 71fa87937b Migrate io-tests from bash to pytest
Authored-by: monacoremo <monacoremo>
2020-12-13 16:19:26 +01:00
Wolfgang WaltherandWolfgang Walther 0c25f12825 fix: fix output of RPCs returning scalar values with multiple rows
resolves #1584

BREAKING CHANGE:
Changed output of RPCs to match return type better:
* single scalar return: value (unchanged)
* setof scalar return: array of values (new; was array of objects)
* single composite return: object (new; was array of objects)
* setof composite return: array of objects (unchanged)
* void return: nothing (new; was "null")

A single OUT column is now treated as "composite" instead of "scalar",
i.e. consistent with multiple OUT columns.
2020-12-10 19:55:42 +01:00
Wolfgang Walther 093fd3c8f6 fix: fix embedding through views with subqueries inside function calls
resolves #1608

Refactors the pfkSourceColumns query to use an intermediate JSON format
for parsing. This allows much more robust extraction of source columns.
2020-12-10 19:48:05 +01:00
Steve ChavezandGitHub ebd474a7e6 fix: retry connection on failed schema cache load (#1685)
Retry the connection when the
"Failed to load the schema cache" error happens.

Also die if the schema cache query has a syntax error.
2020-12-09 14:20:00 -05:00
steve-chavez 11d62a8010 Prepared statement for set_config 2020-12-07 20:09:18 -05:00
steve-chavez 7069bb3c01 refactor: Change SET LOCAL gucs to set_config 2020-12-07 20:09:18 -05:00
Wolfgang WaltherandWolfgang Walther 9254f119f6 fix: implement robust parsing of boolean config values
resolves #1572
2020-12-06 21:59:32 +01:00
Wolfgang WaltherandWolfgang Walther ed58511de3 feat: renamed config options with prefixes; added aliases for old names
* secret-is-base64 -> jwt-secret-is-base64
* role-claim-key -> jwt-role-claim-key
* max-rows -> db-max-rows
* pre-request -> db-pre-request
* root-spec -> db-root-spec
* db-schema -> db-schemas

This is not a breaking change, because aliases are added as well.

refactor: sorted all config keys alphabetically where applicable
2020-12-06 21:59:03 +01:00
Wolfgang Walther ab3375998d ci: fix cirrus low memory error 2020-12-06 20:49:48 +01:00
Wolfgang WaltherandWolfgang Walther 8979442e27 dev: add colors to io tests 2020-12-06 13:33:25 +01:00
Wolfgang WaltherandWolfgang Walther eb46cf2662 added: cli option --dump-config prints loaded config and exits
This is most useful for automated tests for upcoming configuration
features. Can also be used for debugging.
2020-12-06 13:33:25 +01:00
Wolfgang WaltherandWolfgang Walther d218a9eaff dev: improve postgrest-watch experience
* speed up io tests
* avoid infinite loop while watching io tests
* improve auto-complete with postgrest-watch postgrest-run
* detect when files are added or deleted
2020-12-05 16:47:50 +01:00
Wolfgang WaltherandWolfgang Walther 7d6d015822 improve error output for io-tests and with_tmp_db 2020-12-03 22:32:07 +01:00
Wolfgang WaltherandGitHub 609c9aead8 Make transaction-rollback=true the default for test-suite (#1663)
Change test-suite to use db-tx-rollback-all = true by default
2020-12-03 18:54:33 +01:00
monacoremoandWolfgang Walther 10a70d4e52 add autocompletion for postgrest-watch 2020-12-01 22:13:41 +01:00
Wolfgang WaltherandWolfgang Walther de5742fc7d Make nix the one and only dev environment; remove obsolete tooling 2020-12-01 00:42:21 +01:00
Wolfgang WaltherandWolfgang Walther 1a5f6cce46 Add docker files to run nix dev environment on Windows 2020-12-01 00:42:21 +01:00
Wolfgang WaltherandGitHub f2cb91740e Use cabal v2-exec to run io tests, include io tests in nix-shell by default; remove obsolete ncat dependency (#1673)
Use cabal v2-exec to run io tests, include io tests in nix-shell by default; remove obsolete ncat dependency
2020-12-01 00:37:45 +01:00
Remo RechkemmerGitHubmonacoremo <monacoremo>
07cb47e6ac Improve dev tools in the Nix environment (#1666)
* Improve dev tools in the nix environment

Added postgrest-build, postgrest-run, postgrest-clean, postgrest-check
and postgrest-watch tools. Changed order of postgrest-test-spec-all to
run backwards from pg13 to pg9.5. Added shellcheck to lint shell scripts
in nix environment.

Co-authored-by: monacoremo <monacoremo>
2020-11-30 22:21:19 +01:00
steve-chavez 9adec12a67 cirrusci: add build timeout 2020-11-23 19:00:50 -05:00
steve-chavez 787973f323 Add db-prepared-statements config 2020-11-23 19:00:50 -05:00
steve-chavez 4bd5e6bd82 perf: enable prepared statements for GET
- Parametrize filters
- Parametrize LIMIT/OFFSET
- Parametrize JSON path(select=col->$1)
- Single parameter for IN(use ANY)
- Also enable prepared statement for the EXPLAIN
  used on the estimated count.
2020-11-23 19:00:50 -05:00
steve-chavez 8e58b56d5c refactor: Use hasql-dynamic on RPC/POST/PUT/PATCH
Change callProc/createWriteStatement to H.Snippet.

Parametrize the inputs on the same SQLFragments by taking advantage
of hasql-dynamic-statements. It's not necessary to parametrize
every input, inlining with pgFmtLit can still be used for queries
that can't be parametrized(like SET LOCALs).

Also adds hasql-dynamic-statements to Nix and stack.
2020-11-23 19:00:50 -05:00
Wolfgang WaltherandWolfgang Walther b13e95aefb Refactor tx-... config options to single tx-end option 2020-11-23 22:43:49 +01:00
Wolfgang WaltherandWolfgang Walther 3425352035 Clean up postgrest --help output 2020-11-23 17:40:04 +01:00
Wolfgang WaltherandSteve Chavez dbf99c6ac1 Add support for Prefer tx=rollback 2020-11-22 18:21:06 -05:00
Wolfgang WaltherandSteve Chavez 698fac8ff2 Add test to ignore functions unnamed arguments for rpc disambiguation, resolves #1638 2020-11-22 14:48:48 -05:00
YvanandSteve Chavez 0d5520d91d Update README.md
Added "Since" for the security row level PostgreSQL has it since 9.5
2020-11-21 15:50:27 -05:00
Wolfgang WaltherandSteve Chavez 944efb3a6a Split StructureSpec into OpenApiSpec and OptionsSpec 2020-11-20 11:51:49 -05:00
Wolfgang WaltherandSteve Chavez d91f47e582 Remove executable bit on .nix and .sql file 2020-11-20 10:48:47 -05:00
Wolfgang WaltherandSteve Chavez 2c52b96e04 Refactor tests: replace str with json QuasiQuoter where appropriate 2020-11-20 09:24:32 -05:00
Wolfgang WaltherandSteve Chavez e08bb3a197 Fix overloading of functions with unnamed arguments (specifically for prefer params=single-object) 2020-11-20 09:24:32 -05:00
Wolfgang WaltherandSteve Chavez b091586394 Refactor tests: renaming json table to prevent hiding of native json type 2020-11-20 09:24:32 -05:00
Wolfgang WaltherandSteve Chavez ca7ffd0a70 Fix RPC return type handling for domains with composite base type 2020-11-20 09:24:32 -05:00
Gergely MészárosandGitHub 3f690ec78f Removing single column limit from join table M2M mapping detection. (#1593) 2020-11-17 18:49:02 -05:00
Wolfgang WaltherandGitHub 6e04fe7454 Enable embedding through multiple layers of views recursively (#1625)
* Include hidden views from the search path. Hidden views are views in unexposed schemas that are part of a view dependency chain.

* Change allSourceColumns to only return pk and fk columns
2020-11-15 14:58:57 -05:00
steve-chavez 65968b5320 circleci: fix nightly release
Also delete unused variable in release/default.nix
2020-11-13 14:12:28 -05:00
steve-chavez ed8bf8272f Update BACKERS.md 2020-11-13 14:01:58 -05:00
steve-chavez 3830887577 ci: add nightly version to cabal file 2020-11-10 22:47:58 -05:00
steve-chavez b6d8d89b32 ci: add git sha with hours/minutes to nightly 2020-11-10 22:47:58 -05:00
steve-chavez 814da058ec cirrusci: add nightly release 2020-11-10 22:47:58 -05:00
steve-chavez 5d140f6fa0 circleci: add nightly release
For github and dockerhub
2020-11-10 22:47:58 -05:00
steve-chavez 51b43f5605 nix: change release scripts to use a version arg 2020-11-10 22:47:58 -05:00
steve-chavez 8fd9a9ab5d travisci: add nightly release 2020-11-10 22:47:58 -05:00
steve-chavez 5f3b581562 appveyor: add nightly release 2020-11-10 22:47:58 -05:00
steve-chavez 139f9b987a nix: fix postgrest-release-dockerhubdescription 2020-11-10 22:47:58 -05:00
steve-chavez 63849f1275 circleci: make release job a machine
setup_remote_docker is somehow not working and gives:

docker: command not found
2020-11-10 22:47:58 -05:00
steve-chavez 9f8d1af2cd nix: add postgrest-docker-login wrapper func 2020-11-10 22:47:58 -05:00
Wolfgang WaltherandSteve Chavez 122fea1507 Add test to insert into a view and expect location header return 2020-11-03 10:43:06 -05:00
Wolfgang WaltherandSteve Chavez a7403fecc2 refactor: split InsertSpec into InsertSpec and UpdateSpec 2020-11-03 10:43:06 -05:00
Wolfgang WaltherandSteve Chavez efc725fb24 run stylish-haskell v0.12.1.0 2020-11-02 13:22:01 -05:00
Wolfgang WaltherandSteve Chavez 4b4a622a17 fix non-variadic repeated param in variadic function 2020-10-30 10:06:53 -05:00
steve-chavez 8618ffa5fc perf: shortcut for proc with no variadic arg
Also refactor how rpc params are converted to json
2020-10-29 18:53:24 -05:00
steve-chavez 2798ced9b9 Fix charset=utf-8 appending to binary output 2020-10-29 18:53:24 -05:00
steve-chavez a6696f3ba1 refactor: add Hasql arrayColumn decoder 2020-10-29 18:53:24 -05:00
steve-chavez 04eaeec7fc refactor: reorder PgVersion(95/96)Spec tests
* move json ops tests to JsonOperatorSpec
* move phfts operator tests to QuerySpec and RpcSpec
* add custom spec for pre-request header guc tests
2020-10-29 18:53:24 -05:00
Wolfgang WaltherandSteve Chavez 302d4e15ad Allow calling variadic functions with repeated query params or JSON array in body 2020-10-26 17:49:51 -05:00
Wolfgang WaltherandSteve Chavez 18cc214c04 refactor moving findProc into ApiRequest to allow parsing parameters differently by proc 2020-10-26 17:49:51 -05:00
Wolfgang WaltherandSteve Chavez 06e85357c7 add tests to pass regular arrays properly to RPCs 2020-10-26 17:49:51 -05:00
Wolfgang WaltherandSteve Chavez eed0d3a66c ci: add coveralls.io integration 2020-10-22 12:05:25 -05:00
steve-chavez d1d0c6772a perf: change Text queries to ByteString
Improves performance by not utf8 encoding the whole query with
encodeUtf8. Only certain parts.

It's also a gradual step needed to use the Snippet type
from hasql-dynamic-statements.
2020-10-17 15:01:06 -05:00
steve-chavez 7e3e19acbb refactor: untangle addFiltersOrdersRanges
Remove convoluted function composition
2020-10-17 15:01:06 -05:00
Wolfgang WaltherandGitHub b7b66b600d ci: remove io tests from stack-test because of random segmentation faults (#1627)
io tests already run on nix-build-test
2020-10-16 13:53:43 -05:00
Wolfgang WaltherandGitHub 780970885e ci: run spec tests against all pg versions, even if one of them fails (#1628)
- when the build fails no tests are run(circleci-agent step halt)
- when the build succeeds all tests are run, even if some of them fail(when: always)
2020-10-16 11:15:20 -05:00
Wolfgang WaltherandSteve Chavez 5f33f01094 correct issue numbers in changelog 2020-10-16 10:24:41 -05:00
Wolfgang WaltherandSteve Chavez 719c4abba2 Drop support for postgres 9.4 2020-10-15 08:43:28 -05:00
Wolfgang WaltherandSteve Chavez 29858b8d0d improve cirrus build time by properly using cabal cache 2020-10-14 22:25:42 -05:00
Wolfgang WaltherandSteve Chavez b21a0f6192 fix travis build timeout swallowing error code 2020-10-14 10:49:47 -05:00
Wolfgang WaltherandSteve Chavez f5331d1a77 fix build error on windows 2020-10-14 09:26:35 -05:00
Wolfgang WaltherandSteve Chavez 5bb670b736 fix appveyour build script 2020-10-14 09:26:35 -05:00
Remo RechkemmerandSteve Chavez 2eb8083869 Simplify Nix CI jobs 2020-10-14 09:10:47 -05:00
Wolfgang WaltherandGitHub 99ecc7af70 Update stack.yaml.lock (#1616) 2020-10-13 10:50:19 -05:00
steve-chavez ccad9eb9bc Change default log-level, from info to error 2020-10-06 14:21:46 -05:00
steve-chavez d9a608d9b0 Add warn log level 2020-10-06 14:21:46 -05:00
steve-chavez f6b6abe734 Add error log level 2020-10-06 14:21:46 -05:00
steve-chavez e9efcc70a5 Add log-level config 2020-10-06 14:21:46 -05:00
Robert VollmertandSteve Chavez 60398ad538 Allow wai-middleware-static 0.9
- Only serve static files on HEAD or GET requests.
2020-10-06 12:45:40 -05:00
Remo RechkemmerandSteve Chavez 741f017a17 Use Postgres 13 in tests/docker-compose.yml 2020-10-05 13:47:53 -05:00
monacoremoandSteve Chavez e4e84e5714 Upgrade nixpkgs and add Postgres 13 support 2020-10-05 13:47:53 -05:00
Wolfgang WaltherandSteve Chavez d395bb6052 Fix error messages on connection failure for postgres on localized Windows 2020-10-03 10:11:41 -05:00
Felix YanandGitHub 17f04a886e Allow hspec-wai/hspec-wai-json 0.11 (#1595) 2020-09-27 16:04:47 -05:00
Robert VollmertandSteve Chavez 807dae1768 Allow wai-extra 3.1 2020-09-22 11:45:58 -05:00
Remo RechkemmerGitHubmonacoremo <monacoremo>
7af54c5813 Upgrade nixpkgs to current master and add an upgrade checklist (#1579)
* simplify haskell packages overrides based on new nixpkgs
* update Nix gitignore
* simplify Haskell packages overlay

Co-authored-by: monacoremo <monacoremo>
2020-09-09 19:08:16 -05:00
Robert VollmertandSteve Chavez bd2160db26 Make cabals bound more consistent
Specifically, allow point-updates to hasql-notifications, and
give a reasonable lower bound for network (the version that comess
with GHC 8.0 on stackage).
2020-08-14 21:12:58 -05:00
Robert VollmertandSteve Chavez a3f4548a81 Allow base64-bytestring 1.2 and optparse-applicative 0.16 2020-08-14 21:12:58 -05:00
Wolfgang WaltherandGitHub 8e4687fb53 Return 405 Method not Allowed for GET of volatile RPC instead of 500 (#1560) 2020-07-16 16:05:57 -05:00
Steve ChavezandGitHub 189847927e Schema cache reload with zero downtime (#1559)
* Improve error messages and comments

* Reorder Main.hs functions
2020-07-16 13:39:25 -05:00
Wolfgang WaltherandGitHub 6b2767d35c Additionally allow "bearer" instead of only "Bearer" in Authorization… (#1558) 2020-07-16 12:09:40 -05:00
RemoandSteve Chavez 1f6a824dfb Bump nixpkgs pin to unstable as of 2020-07-13 2020-07-13 18:47:45 -05:00
steve-chavez e8b4e3771c Allow config file reloading with SIGUSR2
* move config validation to Config.hs
* Add tests for jwt-secret/app.settings.*/db-schema reload
2020-07-13 11:30:16 -05:00
steve-chavez e272ea47be refactor: move logStdout/corsPolicy to Middleware
* remove configQuiet from Config

configQuiet was not an end user setting.
The logging setup is now an internal parameter.

* move proxy uri validation to Private dir
2020-07-13 11:30:16 -05:00
steve-chavez 0ff05edd16 refactor: move parseSecret out of App.postgrest
parseSecret only needs to be computed once, after the config is read.
2020-07-13 11:30:16 -05:00
steve-chavez 96a16a377f refactor: move loadDbUriFile/SecretFile to Config
* make schema cacher filler use Appconfig
* change configRoleClaimKey to Either Text JSPath
2020-07-13 11:30:16 -05:00
steve-chavez 896b79f05b refactor: separate reading file from parsing it 2020-07-13 11:30:16 -05:00
Wolfgang WaltherandGitHub 343e41c51d Location header improvements(#1475)
* Create location header only on primary key columns, closes https://github.com/PostgREST/postgrest/issues/1461

* Return location header when pk columns not selected, fixes https://github.com/PostgREST/postgrest/issues/1162
2020-07-08 14:36:40 -05:00
steve-chavez 55b4f4fbe7 Fix expired JWTs starting an empty transaction
Fixes https://github.com/PostgREST/postgrest/issues/1094.

Expired JWTs were doing an empty BEGIN/COMMIT in the db.
2020-07-03 17:23:10 -05:00
Wolfgang WaltherandSteve Chavez a5bc293372 add test cases for singular minimal 2020-07-03 13:54:39 -05:00
Steve ChavezandGitHub 43d71e95ac Allow schema cache reloading with NOTIFY (#1542)
Fixes https://github.com/PostgREST/postgrest/issues/1512

Helps on environments where you can't send unix signals(Windows, managed
containers). Also provides better UX for schema reloads - NOTIFY
can be sent from pg clients(psql, pgadmin).

`NOTIFY pgrst` - with no payload - should be done to reload the schema cache.
Notifications with a payload will be ignored.

The channel can be enabled with `db-channel-enabled`(false by default)
and its name can be configured with `db-channel`.

The LISTEN thread uses a dedicated pg connection.
This connection is recovered if it fails.
A debounce of 1ms is done in case too many NOTIFYs arrive.
2020-06-24 19:00:02 -05:00
steve-chavez 24064f8626 github: CHANGELOG reminder in PR template 2020-06-05 13:57:47 -05:00
steve-chavez 8588a42aa9 Fix broken links in .github dir
Also add a PR template
2020-06-05 13:40:03 -05:00
Steve ChavezandGitHub 0f0d617951 Allow http status override through response.status guc (#1541)
Fixes https://github.com/PostgREST/postgrest/issues/1525
2020-06-05 13:01:36 -05:00
Remo RechkemmerandGitHub 69b09e312a Nixify io and memory tests (#1538)
* Include tests for io and memory in the Nix environment.
* include spec tests in nix-shell by default
* install the io and memory tests in CI
2020-05-28 12:45:14 -05:00
Felix YanandGitHub 784ebe57d7 Allow aeson 1.5 (#1537) 2020-05-26 10:43:14 -05:00
Remo RechkemmerandGitHub 08186ea51c Nixify CircleCI setup (#1535)
* package ghr
* add nix-based release scripts
* refactor CI based on nix tests and release scripts
* update stack version in circleci
* update makefile
* remove docker directory
* add Docker Hub description
* add README on docker
2020-05-25 11:27:35 -05:00
steve-chavez d4aba5cb08 Add notice about single Linux static executable 2020-05-22 14:30:23 -05:00
Remo RechkemmerandGitHub 3a1844ec8e Fix the static build (#1534)
* openssl linking fix is required after all

* remove Cabal override (obsoleted on new nixpkgs)
2020-05-21 14:47:07 -05:00
Remo RechkemmerandGitHub 48c9ac36b1 Bump nixpkgs and simplify/clean up nix setup (#1529)
* Refactor tests and fix README

* fix linting hints from new hlint version

* postgrest-style with new nixpkgs-fmt
2020-05-20 21:30:20 -05:00
Remo RechkemmerandGitHub e6874c866d Fix UNKNOWN being shown as git commit hash (#1533)
Fixes https://github.com/PostgREST/postgrest/issues/1530
2020-05-20 14:10:53 -05:00
Remo RechkemmerandGitHub 289bb66f56 Reduce size of the static docker image by patching the Nix openssl package (#1528)
* hook docker images into the default builds
2020-05-20 11:49:11 -05:00
Steve ChavezandGitHub 67344c8e0a cabal: fix upload to hackage (#1532)
When doing:

```
cabal upload dist-newstyle/sdist/postgrest-7.0.1.tar.gz
```

The following error is shown:

```
Error: Invalid package

'ghc-options: -Wall -Werror' makes the package very easy to break with
future GHC versions because new GHC versions often add new warnings.
Use just 'ghc-options: -Wall' instead. Alternatively, if you want to
use this, make it conditional based on a Cabal configuration flag
(with 'manual: True' and 'default: False') and enable that flag
during development.
```

Put -Werror in a cabal flag to work around this restriction.
2020-05-19 13:59:33 -05:00
steve-chavez 222a53015e Fix Dockerfile
Remove libpq5 dependency since static bin already contains it.
2020-05-18 16:50:53 -05:00
steve-chavez d6050c8615 Bump to version v7.0.1 2020-05-18 13:05:09 -05:00
steve-chavez 7ffac522e3 circleci: remove linux distro releases
Prefer the linux static binary for x64
2020-05-18 13:05:09 -05:00
steve-chavez 6c4208d9e7 cirrusci: add binary artifact 2020-05-18 13:05:09 -05:00
steve-chavez 98bf4d861d circleci: add static bin to release 2020-05-18 13:05:09 -05:00
Remo RechkemmerandGitHub bfcd289855 Build Docker image from Nix (#1522)
* Static docker image

* added tags for latest and with version

* add postgrest repository to docker image name

* set creation date on nix docker images
2020-05-16 11:35:32 -05:00
Remo RechkemmerandGitHub 7c0fbf9b3f Add static build with Nix (fixes #1295) (#1494)
* Include modifications to nixpkgs and static-haskell-nix as patches.

* use the patched static-haskell-nix version

* Add cachix postgrest
2020-05-12 13:31:37 -05:00
steve-chavez 6fae07241f Add FreeBSD CI 2020-05-08 15:11:44 -05:00
steve-chavez a4f687fdfd appveyor: specify windows image 2020-05-08 15:11:44 -05:00
steve-chavez b1a101c253 Update BACKERS.md 2020-05-04 12:17:52 -05:00
steve-chavez 10c363b588 Remove PUT restriction for all columns/single row
Fixes https://github.com/PostgREST/postgrest/issues/1452.

The single row restriction can be lifted because the
PUT will consider only the first object of the array.
2020-05-02 11:51:32 -05:00
steve-chavez 9a52632024 Fix Content-Profile not working for POST RPC
Accept-Profile was applied instead.

Fixes https://github.com/PostgREST/postgrest/issues/1508.
2020-05-02 11:51:32 -05:00
steve-chavez f57caf0987 Readd openapi-server-proxy-uri config option
This option accidentaly got lost during a merge.

Fixes https://github.com/PostgREST/postgrest/issues/1500
2020-05-02 11:51:32 -05:00
RemoandGitHub f02904a959 Simplify the DbStructure queries (#1478)
* remove unused parts of the DbStructure queries

* explain magic numbers in DbStructure queries

* simplify primary keys query
2020-04-29 13:01:46 -05:00
RemoandGitHub 8b41b71db7 More consistent loading of the test fixtures (#1506)
Remove db setup from the Haskell test suite and include it in the test bash scripts.
2020-04-29 12:22:15 -05:00
RemoandGitHub d4cf8e7abb Hook the Nix styling scripts into CI (#1504) 2020-04-29 10:44:11 -05:00
RemoandGitHub 9134171b95 Fix the cachix push logic in CircleCI
Before pushing to cachix, check if the CACHIX_SIGNING_KEY env var is available.
2020-04-28 14:04:06 -05:00
RemoandGitHub 328c3453f8 Add tips and tricks to using nix-shell (#1505) 2020-04-28 12:45:29 -05:00
Robert VollmertandSteve Chavez bb27eb57a9 Allow base64-bytestring 1.2. 2020-04-28 11:06:05 -05:00
RemoandGitHub dbc3aa28c4 Add scripts for linting and styling with all dependencies managed by Nix (#1501)
* Integrate the style and lint dependencies with Nix

* Add documentation on the nix styling scripts.
2020-04-27 15:04:50 -05:00
RemoandGitHub ad92c207f2 Run the spec test suite from the nix build job (#1503)
* Add caching to the tests run in CI with Nix
2020-04-27 13:39:57 -05:00
RemoandSteve Chavez dea6c5eb92 Remove ghc options from stack.yaml
They are redundant after being added to postgrest.cabal in #1489
2020-04-27 12:41:55 -05:00
RobertandGitHub 3baefa1d96 Relax dependencies to build with GHC 8.10 (base, swagger2) (#1502)
* Allow swagger 2.7.
* Allow base-4.14 (GHC 8.10).
2020-04-25 13:35:01 -05:00
RemoandGitHub c0546e0e46 Basic CI integration for Nix builds (#1495)
* Skip pushing to cachix for pull requests from forks.
* Build PostgREST first in the nix CI job in order to fail fast
2020-04-25 13:08:30 -05:00
RemoandGitHub d5f1d1b1ad Pin version 0.3 of protolude in nix (#1499) 2020-04-24 12:09:56 -05:00
RemoandGitHub 1c19bbde93 Run tests against several PostgreSQL versions using Nix (#1492)
* Show test output with postgrest-test-spec* in nix-shell

* Test output in color
2020-04-24 11:05:37 -05:00
Robert VollmertandSteve Chavez 3da5a2875e Update to protolude 0.3.0
The good part is that protolude 0.3.0 builds with GHC 8.10.

The bad part is that this change is a bit painful:
- the default `toS` has changed to no longer convert to and from ByteString
- similarly, `show` no longer outputs ByteString

The changes here are pretty much minimal to keep things compiling; I didn't
see a nice way to work with the new ConvertText class, even though `toUtf8`
seems like it might help if used besides `toS` at just the right spots.
2020-04-23 14:14:21 -05:00
RemoandSteve Chavez 2e6a094d48 Disable nix pure by default in stack.yaml 2020-04-23 12:48:04 -05:00
RemoandSteve Chavez 178c5d54d5 Integrate stack in the nix-shell environment 2020-04-22 14:30:02 -05:00
RemoandGitHub a7c396e464 Fix and improvements to with_tmp_db (#1488)
* Load fixtures in one psql session

* Add an option to keet the tmp directory and logs.
2020-04-22 11:57:40 -05:00
RemoandGitHub 24db4a1e25 Basics for building PostgREST with Nix (#1489)
* Add GHC options used with stack to postgrest.cabal

* Add a Nix derivation for just the executable.
2020-04-21 11:59:00 -05:00
RobertandGitHub ae77cf9a08 circleci: Build linters with memory-saving ghc-options. (#1490)
Previously, the linters would use the cached build artefacts from
building postgrest with -fno-spec-constr via stack.yaml. Potentially
passing it to the linter build explicitly will fix recent out-of-memory
issues.
2020-04-19 14:15:12 -05:00
Robert VollmertandSteve Chavez 4a0a37588f Fix hlint 2.2.11 bracketing suggestions 2020-04-16 17:47:52 -05:00
Robert VollmertandSteve Chavez 5838214910 Fix tests for explicit errors in aeson-1.4.7, and raise lower bound 2020-04-16 17:47:52 -05:00
Robert VollmertandSteve Chavez b609d8491e Update stack.yaml to lts-15.8 (GHC 8.8.3) 2020-04-16 17:47:52 -05:00
Robert VollmertandSteve Chavez 18538707ab Relax upper bounds for lens, network-uri, time.
This allows lens-4.19, network-uri-2.7 and time-0.10. According to
the respective changelogs, none of these should cause trouble.

lens-4.19 is required to build with GHC 8.10.
2020-04-16 13:36:26 -05:00
Robert VollmertandSteve Chavez e59c72cff3 circleci: Break out a plain build and cache job 2020-04-16 13:10:21 -05:00
Robert VollmertandSteve Chavez da573a1805 circleci: Dedicated cache key for style checks 2020-04-16 13:10:21 -05:00
Robert VollmertandSteve Chavez e24a7d005a circleci: Split style and hlint out into separate job 2020-04-16 13:10:21 -05:00
Robert VollmertandSteve Chavez 79399686db Remove unused imports (with newer protolude)
The changes here are from fixing stack build with LTS 15.8 (GHC 8.8.3)
to be warning-free. Most if not all come from the newer protolude
version, which adds some things like `unwords`.

So to make sure that we don't get unused import warnings with newer
stackage, this both raises the protolude lower bound and removes
these imports.
2020-04-16 11:10:24 -05:00
RemoandGitHub f6ce93f2c8 Add a script for running the test suite in a local temporary database. (#1476) 2020-04-14 12:34:50 -05:00
RemoandGitHub 8c35c9d711 Fix CI tests for Postgres 9.4 (#1482) 2020-04-14 11:00:43 -05:00
Wolfgang WaltherandGitHub be674eb41d Fix POST, PATCH, DELETE with ?select= and empty body or return=minimal (#1471)
* add tests for POST, PATCH, DELETE when using ?select= with empty bodies or ret=min

* fix select with empty bodies and ret=min

* improved tests for default cases, fixed computed overloaded columns on empty-body?select= PATCH
2020-04-13 12:32:35 -05:00
steve-chavez 052843ac9b Update BACKERS.md 2020-04-13 12:13:35 -05:00
Wolfgang WaltherandGitHub c524531784 Fix overloaded computed columns on RPC (#1473)
* fix some typos and spelling

* fix pg_source CTE name should be prefixed with pgrst_

* added tests for overloaded computed columns on patch calls
2020-04-06 14:04:55 -05:00
Wolfgang WaltherandGitHub 82fa1d8812 add postgres-12 tests to circleci (#1472)
* circleci: new postgres docker containers need host auth trust
2020-04-06 11:00:26 -05:00
steve-chavez 2b61a63686 Bump to version v7.0.0 2020-04-03 11:22:11 -05:00
steve-chavez 18e45659ea Fix compiling on windows(unix socket issue)
Group unix socket functionality into its own module
2020-04-03 11:22:11 -05:00
steve-chavez 426637a47c Fix circleci golang docker image
Upgrade golang otherwise ghr throws:
unrecognized import path "math/bits"
2020-04-03 11:22:11 -05:00
Steve ChavezandGitHub ababf7d4fa Change docker image to support scram-sha-256 auth (#1467)
Debian stretch has a libpq < 10: https://packages.debian.org/stretch/libpq5.
But buster has libpq >= 10 https://packages.debian.org/buster/libpq5.

Fixes https://github.com/PostgREST/postgrest/issues/1443.
2020-04-01 10:14:00 -05:00
691bb5640d Allow multiple schemas to be exposed in one instance (#1450)
The schema to use can be selected through the headers `Accept-Profile` for GET/HEAD and `Content-Profile` for POST/PATCH/PUT/DELETE.

This is based on the https://www.w3.org/TR/dx-prof-conneg/ttps://www.w3.org/TR/dx-prof-conneg/ spec.

Also increase all memory tests by 1M(otherwise CI fails).

Co-authored-by: Mahmoud Kassem <MKassem@gk-software.com>
Co-authored-by: Mahmoud Kassem <mahmoud_k@mail.com>
2020-03-30 14:04:20 -05:00
Wolfgang WaltherandGitHub a80eb2ff0e Add docker-compose file to run tests (#1463) 2020-03-24 09:50:06 -05:00
Duncan OgilvieandGitHub fe59f9bedf Improve command line help (#1465)
Make jwt-secret explicit about 32 characters minimum.
2020-03-21 18:44:08 -05:00
Steve ChavezandGitHub 0f8838623b Fix empty headers being added on POST/PATCH/DELETE (#1458) 2020-03-13 11:02:38 -05:00
RobertandGitHub 5b5945e427 Update configurator-pg to 0.2.0 (better error messages) (#1457)
* Don't indent first line of configuration parser error messages

configurator-pg returns multi-line errors now. Alternatively,
we could indent all the lines.
2020-03-12 10:04:44 -05:00
Robert VollmertandSteve Chavez dea57bd1be Raise lower bound for configurator-pg.
Building on GHC 8.8.3 requires the MonadFail instance that 0.1.0.6
provides.
2020-03-09 13:06:27 -05:00
Robert VollmertandSteve Chavez 3e81a38438 Relax upper bounds for GHC 8.8.3 / stackage lts-15.3.
Specifically, allow base 4.13 which ships with GHC 8.8, as well
as several dependency updates including hasql-transaction 1.0,
swagger 2.5 and warp 3.3.
2020-03-09 13:06:27 -05:00
Robert VollmertandSteve Chavez dfdf3d30b3 Fix test typo. 2020-03-07 13:04:17 -05:00
Robert VollmertandSteve Chavez 60b64d3e81 Upgrade hspec-wai dependency to 0.10. 2020-03-07 13:04:17 -05:00
Robert VollmertandSteve Chavez 962fba4d16 Update stack.yaml.lock for HSYaml extra-deps. 2020-03-07 13:04:17 -05:00
steve-chavez de218e900b Fix for sylish-haskell 2020-02-21 13:30:30 -05:00
steve-chavez b75e7cef90 Change server-proxy-uri 2 openapi-server-proxy-uri
The server-proxy-uri config option mislead into thinking it was a
web server global setting when it's only related to OpenAPI.
2020-02-02 15:23:32 -05:00
steve-chavez 9b1224827a Allow overriding headers Location,Content-Type,etc 2020-01-21 12:28:05 -05:00
steve-chavez c7f78fa7fc Add response.headers on GET/POST/PATCH/PUT/DELETE 2020-01-21 12:28:05 -05:00
steve-chavez 7dade7f466 Add request method and path GUCs 2020-01-21 12:28:05 -05:00
steve-chavez b20e1150a5 Add GUC for accessing the Authorization header 2020-01-21 12:28:05 -05:00
steve-chavez aa0d6a6831 Update BACKERS.md 2020-01-21 12:28:05 -05:00
Steve ChavezandGitHub 663faa1f82 Allow specifying the constraint name to disambiguate an embedding (#1430)
Makes previous duck typing regex unnecessary since the FK can be renamed
to a singular name or to any other format.

* Remove embedding with duck typed column names
* Allow embedding by foreign key name
* Add junction disambiguation tests
2020-01-06 09:42:33 -05:00
YUKISHITA YohsukeandSteve Chavez 99b13fa25f Upsert with "Prefer: resolution=merge-duplicates" on specified columns (#1327) (#1432)
Allow query param 'on_conflict=key1,key2,...' to upsert with explicit columns with unique
constraint.
2019-12-29 11:52:58 -05:00
Dmitry WaginandSteve Chavez e12c1319b6 Change default server-unix-socket-mode 755 -> 660 (#1423) 2019-12-16 08:37:00 -05:00
Dmitry WaginandSteve Chavez 7f365bf60b Enforce singular behavior despite Prefer: return (#1417) 2019-12-08 01:49:20 -05:00
Dan AmorosoandSteve Chavez 2e6c78d723 add config option server-unix-socket-mode (#1415)
* added config option server-unix-socket-mode to enable custom socket permissions

* added server-unix-socket-mode input validation
2019-11-23 10:52:33 -05:00
steve-chavez f9c64d9f65 refactor: improve disambiguation error message
* reverse backwards relationships
* remove redundancy from getJoinSelects
* properly name Cardinality constructors
2019-11-17 13:28:21 -05:00
steve-chavez 4ef6926791 Update authorship 2019-11-17 13:28:21 -05:00
Brian WignallandSteve Chávez 9645f1011c Fix double-and typo 2019-11-12 10:00:23 -05:00
steve-chavez 9847e60dca Err embedding when multiple relationships found
When having one-to-many relationships like:

person        -< message[sender]
person        -< message[recipient]
person_detail -< message[sender]
person_detail -< message[recipient]

Where person_detail is a view of person.

This request:

GET "/message?select=*,sender(*)"

Is ambiguous. Both person or person_detail could be embedded.

Until now we have returned the first detected relationship but
now we return a 300 Multiple Choices error with a
descriptive error message asking the user to disambiguate.

This is more helpful for the user and also aids in cases of more
complex relationships.
2019-11-05 12:55:48 -05:00
steve-chavez cb3d9ab625 refactor: rename RelationType to Cardinality
Remove Root type constructor
2019-11-05 12:55:48 -05:00
steve-chavez db41fb454e refactor: Rename synonyms to source columns 2019-11-05 12:55:48 -05:00
steve-chavez 3b133d5554 Fix #1397, correct compression to XZ for osx bin 2019-10-24 13:12:39 -05:00
Steve ChávezandGitHub 80f763448f Fix unique foreign key in view (#1395) 2019-10-16 12:45:38 -05:00
steve-chavez a3701f5de8 Update BACKERS.md 2019-10-14 14:31:29 -05:00
steve-chavez ed2bfc09a6 Allow PATCH/DELETE w/o Prefer when no SELECT privs
PATCH/DELETE can now be done without adding Prefer return=minimal when
the user doesn't have SELECT privileges.

* Also fix PATCH wrong HTTP status code
2019-10-08 12:41:39 -05:00
steve-chavez 337f821e00 refactor: remove parent embed workaround(in #647)
This workaround is no more necessary since the addition of #978.

* Also add a test for proving parent embeds offset is consistent
  with other types of embeds.
2019-10-08 12:41:39 -05:00
steve-chavez f2b126f147 refactor: Move hasql helpers to Common module 2019-10-08 12:41:39 -05:00
steve-chavez 1173bc277b refactor: rm configSchema ref in postgrest func
* make TargetDefaultSpec take a schema attribute

* remove schema param from addJoinConditions
2019-10-08 12:41:39 -05:00
steve-chavez 50f2cc16ab refactor: rm schema arg from QueryBuilder funcs
* Change TableName to QualifiedIdentifier in ReadQuery
  and MutateQuery.

* Move removeSourceCTESchema to DbRequestBuilder.
2019-10-08 12:41:39 -05:00
steve-chavez eebe319bfd Recommend doing make check pre-commit 2019-09-30 08:46:31 -05:00
steve-chavez 75a42b77ea Fix M2M resource embedding on RPC and mutations 2019-09-28 13:45:18 -05:00
steve-chavez d71d3450af Fix PATCH embed when not having the id in ?select 2019-09-28 13:45:18 -05:00
steve-chavez f080159268 refactor: add the returningCols function
* separate fieldNames from getting fkCols

* Put binaryField inside readSqlParts

* Move scalar proc logic to binaryField

* Move logic for the "SELECT *" default to DbRequestBuilder
2019-09-28 13:45:18 -05:00
steve-chavez 0183d32c7f refactor: Remove DbRequest type
This type only adds an extra constructor.
2019-09-28 13:45:18 -05:00
steve-chavez 94f5894d7f Fix self join resource embedding on PATCH 2019-09-28 13:45:18 -05:00
steve-chavez 81e5a62f25 refactor: remove fromJust in DbRequestBuilder
* Also comment and reorganize DbRequestBuilder
2019-09-28 13:45:18 -05:00
Steve ChávezandGitHub 186381bab2 Add support for Prefer: count=planned/estimated on GET /table (#1386) 2019-09-18 10:13:48 -05:00
steve-chavez e044488f73 Make costly bulk call query optional
Bulk Call should be used by specifying
the `Prefer: params=multiple-objects` header.
2019-09-11 12:01:41 -05:00
steve-chavez b077974ebc Add tests for call proc queries EXPLAIN costs
* circleci: add run query costs tests
2019-09-11 12:01:41 -05:00
steve-chavez 200540dfc3 refactor: put callProc core query to QueryBuilder
* Move set local queries to QueryBuilder

* Move unquoted to Middleware
2019-09-11 12:01:41 -05:00
steve-chavez 3c00f46e36 refactor: Move statements logic to own module
* Move gucHeader decoding to Statements
2019-09-11 12:01:41 -05:00
steve-chavez 620721dea7 Add support for HEAD request 2019-09-03 13:38:53 -05:00
steve-chavez 68cbe34c11 refactor: only run db reset on certain specs
This speeds up running the whole test suite
2019-09-03 13:38:53 -05:00
steve-chavez e21b010c6e refactor: reorganize modules
* Move range logic from App to RangeQuery

* Move decoder logic to Statement modules

* Move location headers logic to functions

* Add a comment for App.hs module
2019-09-03 13:38:53 -05:00
Robert VollmertandSteve Chávez 0846d4d7b2 stack.yaml: Document use of obscure ghc options. 2019-08-27 14:49:07 -05:00
Robert VollmertandSteve Chávez 2183a2a1ae stack.yaml: Remove no-warn-redundant-constraints.
The warning appears to not be part of -Wall anymore in recent
GHC versions.

Also move the warning-related options next to each other.
2019-08-27 14:49:07 -05:00
Robert VollmertandSteve Chávez e8475b18d3 Update stack.yaml.lock for lts-14.3 2019-08-27 09:35:27 -05:00
Robert VollmertandSteve Chávez cdc1177762 stylish-haskell: check Haskell source files only
The style check was failing due to a changed stack.yaml.lock file.

(That file still should be updated, but this is not where it should
cause a failure.)
2019-08-27 09:35:27 -05:00
Robert VollmertandSteve Chávez 97035e0b8b circleci: make sure hlint is in the $PATH 2019-08-27 09:35:27 -05:00
Robert VollmertandSteve Chávez aaf62c1c96 Build with -fno-spec-constr.
Compare https://gitlab.haskell.org/ghc/ghc/issues/16017#note_219304
2019-08-27 09:35:27 -05:00
Robert VollmertandSteve Chávez 4d0661fd9b Upgrade swagger to 2.4.
Two changes:
- tags are now an insert-ordered hashset
- type_ .~ -> type_ ?~ with some magic type inference

This also updates stack.yaml to the newest GHC 8.6 LTS release.
2019-08-27 09:35:27 -05:00
Steve ChávezandGitHub 713b214c9a Bump to version v6.0.2
* Add missing CHANGELOG entry for #1368
2019-08-22 11:12:01 -05:00
steve-chavez 681388631b Add raw-media-types to Dockerfile 2019-08-21 10:34:28 -05:00
steve-chavez ae9e27a0c7 Fix #1348, go back to converting plus "+" to space
Not doing this conflicts with some http clients and proxies.

Use the alternative url-safe character '!' instead for
disambiguating resource embedding.
2019-08-21 10:34:28 -05:00
Steve ChávezandGitHub e83144ce7f Add documentation links 2019-08-16 12:41:24 -05:00
Joe NelsonandGitHub 1c54c7130a Another funding link 2019-08-14 14:05:12 -05:00
Steve ChávezandGitHub 1a8d5fed8a Update ISSUE_TEMPLATE.md 2019-08-13 08:15:11 -05:00
Nathan BouscalandSteve Chávez 57ebf43e85 Allow select in OpenAPI for POST requests (#1375)
If you use `return=representation`, `select` is useful in `POST` requests.
It already works correctly, but is missing from the OpenAPI spec.
2019-08-09 14:10:20 -05:00
steve-chavez b87734343e Update BACKERS.md 2019-08-08 18:50:34 -05:00
Dan AmorosoandSteve Chávez c80c9ef726 Reword error message for BinaryFieldError (#1372) 2019-08-08 14:20:06 -05:00
Dan AmorosoandSteve Chávez d5758523f3 Change raw-media-types to accept a string of comma separated MIME types (#1370) 2019-08-06 13:23:52 -05:00
Amédée d'AbovilleandSteve Chávez ee40e7e0d7 Fix long column descriptions being truncated at 63 characters in pg12 (#1368)
* Lengthen child_entities name comment to make tests check it does not get truncated at 63 chars.

* Don't cast column description to sql_identifier type (maxlen 63 chars) when introspecting schema in DbStructure.hs
2019-08-02 16:13:01 +00:00
steve-chavez 53b606e1c1 Remove raw-media-types config from Docker
Passing a list through an env var is not working
in configurator-pg
2019-07-31 12:06:19 -05:00
steve-chavez 47c0141c49 Bump to v6.0.1 2019-07-30 22:34:28 +00:00
steve-chavez 5b8a17e366 Correct appveyor conditional deploy 2019-07-30 22:34:28 +00:00
steve-chavez 64a86b899f Remove centos6 release
Centos 6 is missing ghc 8.4.4 support, see
https://github.com/commercialhaskell/stack/issues/4161

The static binary release for linux can be used instead.
2019-07-30 22:34:28 +00:00
steve-chavez 312e295a47 Add raw-media-types config option to Docker 2019-07-30 22:34:28 +00:00
steve-chavez e7544687d1 Fix/organize latest entries in CHANGELOG 2019-07-30 22:34:28 +00:00
Diogo BiazusandSteve Chávez 291de5bc1c LTS 13.29 (#1364)
* Update resolver to lts-13.29 and add lock file to repository

* Upgrade stack version

* Save cache after building dependencies only to have faster feedback loop when tests fail

* Move private functions from QueryBuilder to a separate Private module

* Move more functions over to private trying to make compilation consume less memory

* Split private in 4 modules

* Remove unused LambdaCase pragma

* Add profile to memory-tests.sh so it can find postgrest executable

* Move save dependencies before building and running tests for faster feedback loop
2019-07-29 17:14:06 +00:00
Steve ChávezandGitHub c37a9f5ec3 Fix windows build (#1354)
* Fix windows build by pinning network and text-printer versions
* Enable travis and appveyor builds on PRs
2019-07-23 22:27:35 +00:00
Dan AmorosoandSteve Chávez f5cef205f1 Introduced raw-media-types config option (#1349)
* extracted rawOutputTypes to config variable raw-output-media-types

* removed CTTextHtml from Types.hs
2019-07-22 23:37:53 +00:00
RobertandSteve Chávez afb7266f17 Update jose to 0.8.1 to fix Windows build (and adapt error message test) (#1351) 2019-07-15 15:14:22 +00:00
Eduardo JorgeandSteve Chávez e639c77aa2 Add websearch_to_tsquery support (#1339) 2019-07-08 13:25:58 -05:00
Robert VollmertandSteve Chávez ea97055449 appveyor: cache stack appdata directory
This is where stack installs GHC versions.
2019-07-07 11:40:41 -05:00
RobertandSteve Chávez 64dc6ab9ac Fix json test versioning for postgresql 10.9 bugfix (#1340)
This removes the test for buggy behaviour, it seems not useful
anymore.
2019-07-05 12:54:44 -05:00
Eduardo JorgeandSteve Chávez 25dedd1098 Add tests for overlap operator in array types (#1337)
* Add basic tests for some array operators
  - eq
  - neq
  - tl
  - gt
  - lte
  - gte
  - cs
  - cd
  - ov
2019-07-04 11:42:19 -05:00
RobertandSteve Chávez 617bf7b6a3 Update jose to 0.8 (and adapt error message test) (#1324)
The message changes between jose versions 0.7 and 0.8.
2019-07-04 10:56:45 -05:00
Robert VollmertandSteve Chávez e3a53de8a6 circleci: bump postgres to 11.4 to test json bugfix 2019-06-28 11:45:27 -05:00
Robert VollmertandSteve Chávez 4cc91fd5b1 Update json rpc argument tests for bugfix in PostgreSQL 11.4
See https://www.postgresql.org/docs/release/11.4/.
2019-06-28 11:45:27 -05:00
RobertandSteve Chávez 296a12e394 Work around macos case sensitivity issue (#1332)
See also https://github.com/haskell/cabal/issues/4739.

/Users/travis/build/PostgREST/postgrest/<built-in>:15:10: error:
     error: non-portable path to file '".stack-work/dist/x86_64-osx/Cabal-2.2.0.1/build/PostgREST/autogen/cabal_macros.h"'; specified path differs in case from file name on disk [-Werror,-Wnonportable-include-path]
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         ".stack-work/dist/x86_64-osx/Cabal-2.2.0.1/build/PostgREST/autogen/cabal_macros.h"
2019-06-27 10:33:38 -05:00
steve-chavez da7aa1d72f Add root-spec to official Dockerfile
* Delete unused config file
2019-06-26 18:14:36 -05:00
steve-chavez 367ad8ea43 Update pgdg url for centos dockerfiles
* Also update ubuntu i386 image repo
2019-06-26 18:14:36 -05:00
steve-chavez 2c3bc2d75e AppVeyor workaround for TMP issue, also add timeout
See https://github.com/quasicomputational/hpack/commit/ce868d17e53289eab3703261a5a3dc8d48364533,
and https://github.com/commercialhaskell/stack/issues/3944
2019-06-26 18:14:36 -05:00
steve-chavez 3cce6ca02b Change docker image to postgres 11.3
PostgreSQL 11.4 patched an error for wich we had a test.
See https://www.postgresql.org/docs/release/11.4/ json_to_record.
2019-06-26 18:14:36 -05:00
Steve ChávezandGitHub dd86fe372c Bump to v6.0.0 (#1329)
* Also update BACKERS.md
2019-06-21 13:11:48 -05:00
Steve ChávezandGitHub ea7d747107 Accept text/plain and text/html for raw output (#1330) 2019-06-21 11:51:54 -05:00
steve-chavez 40ae7ce2b1 Add CYBERTEC as Sponsor 2019-06-19 23:48:02 -05:00
RobertandSteve Chávez 9bcf39f41f Upgrade hasql to 1.4 (#1325) 2019-06-17 16:54:45 -05:00
steve-chavez 51d3a7864a Rename logo to invalidate cache 2019-06-16 15:20:22 -05:00
steve-chavez e2dc432385 Correct url and adjust logos sizes 2019-06-16 15:03:52 -05:00
steve-chavez b101d5f0f9 Add 2ndQuadrant as sponsor 2019-06-15 16:09:40 -05:00
RobertandSteve Chávez 23ca27d27e Update stackage GHC to 8.4.4 (#1322)
* Update stack.yaml to lts-12.26 (GHC 8.4.4) with simplified nix config

(On GHC 8.6.5, profiled build was exiting with:

--  While building package postgrest-5.2.0 using:
      /home/circleci/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.4.0.1 build lib:postgrest exe:postgrest --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure (-9) (THIS MAY INDICATE OUT OF MEMORY)
Exited with code 1)

* stack.yaml: Add postgresql to nix packages
2019-06-11 13:20:04 -05:00
Steve ChávezandGitHub 1df749a7a8 Add option for overriding root spec (#1317)
* Only for pg >= 9.6

* Disallow specifying schema on root-spec

* Increase memory test upper bound
2019-06-10 13:45:50 -05:00
RobertandSteve Chávez ea82b9f820 Update cabal bounds (#1319)
* Relax some cabal upper bounds

Primarily based on building against GHC 8.6.5 with stackage lts-13.23.

* Add swagger2 upper bound

With swagger2 2.4, we get a number of errors like the following:

src/PostgREST/OpenAPI.hs:58:11: error:
    • Couldn't match type ‘Maybe
                             (SwaggerType 'Data.Swagger.Internal.SwaggerKindSchema)’
                     with ‘SwaggerType 'Data.Swagger.Internal.SwaggerKindSchema’
        arising from a functional dependency between:
          constraint ‘HasType
                        Schema (SwaggerType 'Data.Swagger.Internal.SwaggerKindSchema)’
            arising from a use of ‘type_’
          instance ‘HasType
                      Schema
                      (Maybe (SwaggerType 'Data.Swagger.Internal.SwaggerKindSchema))’
            at <no location info>
    • In the first argument of ‘(.~)’, namely ‘type_’
      In the second argument of ‘(&)’, namely ‘type_ .~ SwaggerObject’
      In the first argument of ‘(&)’, namely
        ‘(mempty :: Schema) & description .~ tableDescription t
           & type_ .~ SwaggerObject’
   |
58 |         & type_ .~ SwaggerObject
   |           ^^^^^

* Require base 4.9 / GHC 8.0.1

The DuplicateRecordFields extension, which is used here,
was introduced with GHC 8.0.

* Set cabal lower bounds

These are based on building against stackage lts-7.24 / GHC 8.0.1.

* Set cabal upper bounds, and truncate lower bounds

These are primarily based on the output of `cabal bounds`, with a good
build against GHC 8.6.5. Some test dependencies were missing; these
were filled out based on a build against stackage nightly.

Also I truncated lower bounds to the third level, since that is what
`cabal bounds` seems to suggest anyway.

* Format postgrest.cabal using stylish-cabal

* Sort build-depends list
2019-06-08 21:42:28 -05:00
RobertandSteve Chávez 7356327e5b Tweaks to circle-ci profiling build (#1321)
* circleci: Update stack versions from 1.1.2 to 1.9.3.

* circleci: Don't try to install unused packdeps and cabal-install.

* memory-tests: Rely on `stack build` to have been called already.

This happens in the previous circleci step.
2019-06-08 21:12:55 -05:00
RobertandSteve Chávez 33532cfbb6 Fix hlint warnings for hlint v2.1.20 (#1320)
* Apply some hlint suggestions.

* Simplify config parser (to avoid hlint error)

* Reorganize for clarity (and fix hlint error)

* Remove redundant language extension

* Reformat slice more conventionally to avoid hlint bug

* Refactor for clarity (and to avoid hlint error)

* Simplify (and avoid hlint error)

* Fix hlint complaint
2019-06-08 20:52:48 -05:00
Dan AmorosoandSteve Chávez 78e5677fbe Add server-unix-socket config option (#1303)
Add ncat workaround to test socket connection on CircleCI
2019-06-05 12:57:47 -05:00
RobertandSteve Chávez e292fb5eb9 Replace configurator-ng by configurator-pg (#1312)
The change should not affect parse results for valid
configuration files. Error messages should be somewhat
improved.
2019-06-04 15:49:37 -05:00
Robert VollmertandSteve Chávez 8fe9e94e24 io-tests: more consistent output
This brings the messages for passing/non-passing of individual
tests closer together, making it easier to compare successful
and failing test runs.
2019-06-03 12:07:05 -05:00
Robert VollmertandSteve Chávez 30d5a81156 Test json rpc arguments
In particular, check buggy behavior around embedded quoted JSON
and Postgres versions, compare

https://www.postgresql.org/message-id/D6921B37-BD8E-4664-8D5F-DB3525765DCD%40vllmrt.net

This adds version-bounded tests for the handling of quoted JSON,
and a pending test that documents the assumption that Postgres >=10
intends to parse quoted JSON as a string (similar to how jsonb works
now).
2019-06-02 01:26:52 -05:00
Robert VollmertandSteve Chávez 1f69822fa3 Fail tests when sql fixtures fail to load 2019-06-02 01:26:52 -05:00
Robert VollmertandSteve Chávez 65fc672417 Change tests to use postgres-json-schema instead of hjsonschema.
Embeds https://github.com/gavinwahl/postgres-json-schema.
2019-06-01 23:26:33 -05:00
Robert VollmertandSteve Chávez e34669b137 More helpful usage messages for test/create_test_db 2019-05-31 18:27:11 -05:00
Robert VollmertandSteve Chávez ea2f89e234 Fix non-exhaustive pattern match warning by duplicating a line. 2019-05-29 11:26:58 -05:00
Robert VollmertandSteve Chávez 97ea99402d Add lower bound for http-types
parseQueryReplacePlus was introduced with version 0.12.2.
2019-05-29 10:19:50 -05:00
Xavier FranciscoandSteve Chávez 35cef22254 Add makefile with common commands (#1300) 2019-05-24 17:40:39 -05:00
Xavier FranciscoandSteve Chávez 28b3d6cafd Update stylish haskell config; apply all; add CI config (#1299)
* Update config default; Copy non-defaults

* Update .stylish-haskell config version to match pgrst

* Apply stylish haskell to all files

* CircleCI config

* Remove redundant import.

What is used from Network.HTTP.Types.Headers is also exported by Network.HTTP.Types.

* Grouped imports

* Show un-styled files on CircleCI failure

* Fix styling imports

* Apply adhoc standard correctly
2019-05-23 10:44:34 -05:00
Steve ChávezandGitHub 16af470a99 Add shell.nix (#1298) 2019-05-22 10:26:36 -05:00
Xavier FranciscoandSteve Chávez 1cf54e6575 Small refactor around error responses and more tests (#1282)
* Fix #880, Clean and consolidate error responses
* Fix #1285, Abort on fatal errors
* Add / Detail tests
2019-05-22 09:30:33 -05:00
steve-chavez e2d917f7b9 Update BACKERS.md 2019-05-21 10:39:54 -05:00
Russell DaviesandSteve Chávez 8d8374cef0 Update Dockerfile base image to debian:stretch-slim 2019-05-19 15:22:19 -05:00
steve-chavez 3078a11144 Remove circleci extra cabal checks
Currently these extra checks are giving an out of memory error on CircleCI:
Hint: Max memory usage of this container is 4231065600
2019-05-19 13:18:03 -05:00
steve-chavez 3c7738a8c7 Deprecate '.' for disambiguating resource embedding
'+' should be used instead.
2019-05-19 13:18:03 -05:00
steve-chavez 181b608c04 Change server-host default of 127.0.0.1 to !4
This way the server works in LAN settings by default.
2019-05-19 13:18:03 -05:00
steve-chavez 37de12d376 Remove support for schema reloading with SIGHUP
SIGUSR1 should be used instead
2019-05-19 13:18:03 -05:00
David ErnstandSteve Chávez bcc317db81 Fix reference to deprecated SIGUP call (#1284)
This updates the instructions about sending a `SIGUP` call (deprecated) to `SIGUSR1`, and links to relevant docs: http://postgrest.org/en/v5.2/admin.html#schema-reloading

Per the docs:

> As of PostgREST v5.1 reloading with SIGHUP is deprecated, it’s still supported but will be removed in v6.0. SIGUSR1 should be used instead.

See [5.1.0 CHANGELOG](https://github.com/PostgREST/postgrest/blob/d32f373e1ee95ba68141a3f3a48ab8c94ea2453a/CHANGELOG.md#deprecated) or https://github.com/PostgREST/postgrest/issues/724
2019-04-26 19:46:20 -05:00
Xavier FranciscoandSteve Chávez d32f373e1e Add config option for pool timeout (#1280) 2019-04-25 12:23:20 -05:00
steve-chavez 2044f77d49 Fix #1273, don't ignore RPC arguments by default
* Add different error message for pg 9.4
2019-04-19 16:03:42 -05:00
steve-chavez 033ee5a06e Refactor: remove pjKeys from RawJSON 2019-04-19 16:03:42 -05:00
Xavier FranciscoandSteve Chávez 553531711b Fix wrong status 404 when PATCH request didn't change anything (#1272) 2019-04-19 13:50:19 -05:00
Xavier FranciscoandSteve Chávez 87f7e86aa7 Set fts tests language instead of depending on system default (#1276) 2019-04-13 19:50:45 -05:00
Xavier FranciscoandSteve Chávez 74e38a1d80 Fix invalid SQL generated for bulk upsert with empty array (#1270) 2019-04-12 23:26:36 -05:00
Xavier FranciscoandSteve Chávez 63826e9509 Fix tests with postgres instances with a different locale (#1269)
* Fix LC_MONETARY

* Add other local parameters

* Change syntax to support 9.4
2019-04-08 10:34:59 -05:00
Xavier FranciscoandSteve Chávez 32725f2f35 Fix tests expected responses due to changes in pg11 (#1268) 2019-04-07 19:03:02 -05:00
steve-chavez b53e8932e5 Add CircleCI test suite for pg11 2019-04-07 18:39:12 -05:00
steve-chavez bdfb11001e Fix #1264, allow bulk RPC call 2019-04-05 19:10:00 -05:00
steve-chavez 40b004c9f7 Comment OpenAPI tests that time out
These should be run locally for now.
2019-04-05 19:10:00 -05:00
Tobias FlühmannandSteve Chávez fe56029f61 The process should not run as root in docker(#1267) 2019-04-04 10:38:11 -05:00
fpuschandSteve Chávez cefbe8f07f Add numeric and character types to OpenAPI spec (#1259)
* Add numeric and character types

* Remove decimal type mapping

It is treated as numeric by PostgreSQL so the mapping was redundant.
2019-03-29 11:09:42 -05:00
Vitor BaptistaandSteve Chávez 9387e70b66 Fixes #1239, support resource embedding on materialized views 2019-03-14 16:59:55 -05:00
steve-chavez 1f513f24a5 Fix #1242, embed a view having a select in a where 2019-03-09 16:03:26 -05:00
steve-chavez 7c376d6e84 Delete unused files 2019-03-09 16:03:26 -05:00
steve-chavez 39adbefb9d Add TimescaleDB sponsorship
* Add Sponsors to BACKERS.md
2019-03-01 13:56:24 -05:00
steve-chavez c9b2830e52 Fix #690, add columns query arg for RPC/POST/PATCH
* Refactor normalizing json CTE
* Refactor CTE to use CASE instead of UNION
2019-02-26 11:38:21 -05:00
steve-chavez 3946dfbc64 Remove pjIsEmpty and rm pjType from ActionInvoke
* Remove PayloadJSON from Insert/Update
2019-02-26 11:38:21 -05:00
steve-chavez 50509b52b8 Fix POST/PATCH error on a [{}] json 2019-02-26 11:38:21 -05:00
steve-chavez 16059ad470 Remove pjType from ActionCreate
* Remove isSingle from ActionCreate
* Remove nRows from ActionCreate
2019-02-26 11:38:21 -05:00
steve-chavez 00a0d8b9b7 Remove pjType from QueryBuilder Insert/Update
* Refactor Insert onConflict
2019-02-26 11:38:21 -05:00
steve-chavez 36e9d779fc Update BACKERS.md, resize logos 2019-02-19 17:04:09 -05:00
steve-chavez 3fc8a105ec Add Retool sponsorship 2019-02-19 12:50:30 -05:00
steve-chavez 673aa25082 Fix #1221, embedding when having a self join 2019-02-04 10:22:02 -05:00
steve-chavez 1037313e77 Add comments to modules 2019-02-04 10:22:02 -05:00
steve-chavez 86e460c5c4 Add implicitJoins to ReadQuery type 2019-02-04 10:22:02 -05:00
steve-chavez fb5adce5ce Reformat and reorder Types.hs 2019-02-04 10:22:02 -05:00
Steve ChávezandGitHub 04ab0ea753 Fix #1223, incorrect OpenAPI externalDocs url (#1232) 2019-01-25 12:04:36 -05:00
Vassilis PapakonstantinouandSteve Chávez 3900baa6ce Add db-extra-search-path config param to dockerized postgREST (#1229)
* add db-extra-search-path config param

* add default value for PGRST_DB_EXTRA_SEARCH_PATH env variable
2019-01-16 10:55:16 -05:00
271 changed files with 39881 additions and 10347 deletions
-327
View File
@@ -1,327 +0,0 @@
version: 2
build-distro-bin: &build-distro-bin
machine: true
steps:
- checkout
# cannot interpolate env var and use as a cache key so just copy the Dockerfile to another filename
- run: cp docker/distro_release/Dockerfile.$CIRCLE_JOB docker/distro_release/Dockerfile
- restore_cache:
keys:
- v1-{{ .Environment.CIRCLE_JOB }}-image-{{ checksum "docker/distro_release/Dockerfile" }}
- restore_cache:
keys:
- v1-{{ .Environment.CIRCLE_JOB }}-dependencies-{{ checksum "postgrest.cabal" }}-{{ checksum "stack.yaml" }}
- run:
name: load or build docker image
command: |
if [[ -e ~/image.tar ]]; then
docker load -i ~/image.tar
else
docker build --rm=false -t $CIRCLE_JOB -f docker/distro_release/Dockerfile.$CIRCLE_JOB docker/distro_release/
docker save $CIRCLE_JOB > ~/image.tar
fi
- run:
name: build binary
command: |
docker run -it \
-v $HOME/.stack:/root/.stack \
-v $(pwd):/source \
-v $HOME/bin/:/root/.local/bin/ \
$CIRCLE_JOB build --allow-different-user --install-ghc --copy-bins
# volumes owned by root if chown is not done the save_cache step fails silently
sudo chown -R circleci:circleci ~/.stack .stack-work
- run:
name: compress binary
command: |
mkdir -p /tmp/workspace/bin
cd /tmp/workspace/bin
tar cvJf postgrest-$CIRCLE_TAG-$CIRCLE_JOB.tar.xz -C ~/bin postgrest
- persist_to_workspace:
root: /tmp/workspace
paths:
- bin/*
- save_cache:
paths:
- ~/image.tar
key: v1-{{ .Environment.CIRCLE_JOB }}-image-{{ checksum "docker/distro_release/Dockerfile" }}
- save_cache:
paths:
- "~/.stack"
- ".stack-work"
key: v1-{{ .Environment.CIRCLE_JOB }}-dependencies-{{ checksum "postgrest.cabal" }}-{{ checksum "stack.yaml" }}
jobs:
build-test-9.4:
docker:
- image: circleci/buildpack-deps:trusty
environment:
- PGHOST=localhost
- image: circleci/postgres:9.4.14
environment:
- POSTGRES_USER=circleci
- POSTGRES_DB=circleci
steps:
- checkout
- restore_cache:
keys:
- v1-stack-dependencies-{{ checksum "postgrest.cabal" }}-{{ checksum "stack.yaml" }}
- run:
name: install stack & dependencies
command: |
curl -L https://github.com/commercialhaskell/stack/releases/download/v1.1.2/stack-1.1.2-linux-x86_64.tar.gz | tar zx -C /tmp
sudo mv /tmp/stack-1.1.2-linux-x86_64/stack /usr/bin
sudo apt-get update
sudo apt-get install -y libgmp-dev
sudo apt-get install -y --only-upgrade binutils
sudo apt-get install -y postgresql-client
stack setup
rm -rf $(stack path --dist-dir) $(stack path --local-install-root)
stack install hlint packdeps cabal-install
- run:
name: build src and tests
command: |
stack build --fast -j1
stack build --fast --test --no-run-tests
- run:
name: run tests
command: |
POSTGREST_TEST_CONNECTION=$(test/create_test_db "postgres://circleci@localhost" postgrest_test) stack test
test/io-tests.sh
- run:
name: run linter
command: git ls-files | grep '\.l\?hs$' | xargs stack exec -- hlint -X QuasiQuotes -X NoPatternSynonyms "$@"
- run:
name: extra checks
command: |
stack exec -- cabal update
stack exec --no-ghc-package-path -- cabal install --only-d --dry-run
stack exec -- packdeps *.cabal || true
stack exec -- cabal check
stack haddock --no-haddock-deps
stack sdist
- save_cache:
paths:
- "~/.stack"
- ".stack-work"
key: v1-stack-dependencies-{{ checksum "postgrest.cabal" }}-{{ checksum "stack.yaml" }}
build-test-9.6:
docker:
- image: circleci/buildpack-deps:trusty
environment:
- PGHOST=localhost
- image: circleci/postgres:9.6.2
environment:
- POSTGRES_USER=circleci
- POSTGRES_DB=circleci
steps:
- checkout
- restore_cache:
keys:
- v1-stack-dependencies-{{ checksum "postgrest.cabal" }}-{{ checksum "stack.yaml" }}
- run:
name: install stack & dependencies
command: |
curl -L https://github.com/commercialhaskell/stack/releases/download/v1.1.2/stack-1.1.2-linux-x86_64.tar.gz | tar zx -C /tmp
sudo mv /tmp/stack-1.1.2-linux-x86_64/stack /usr/bin
sudo apt-get update
sudo apt-get install -y libgmp-dev
sudo apt-get install -y postgresql-client
stack setup
- run:
name: build src and tests
command: |
stack build --fast -j1
stack build --fast --test --no-run-tests
- run:
name: run tests
command: POSTGREST_TEST_CONNECTION=$(test/create_test_db "postgres://circleci@localhost" postgrest_test) stack test
build-test-10:
docker:
- image: circleci/buildpack-deps:trusty
environment:
- PGHOST=localhost
- image: circleci/postgres:10.5
environment:
- POSTGRES_USER=circleci
- POSTGRES_DB=circleci
steps:
- checkout
- restore_cache:
keys:
- v1-stack-dependencies-{{ checksum "postgrest.cabal" }}-{{ checksum "stack.yaml" }}
- run:
name: install stack & dependencies
command: |
curl -L https://github.com/commercialhaskell/stack/releases/download/v1.1.2/stack-1.1.2-linux-x86_64.tar.gz | tar zx -C /tmp
sudo mv /tmp/stack-1.1.2-linux-x86_64/stack /usr/bin
sudo apt-get update
sudo apt-get install -y libgmp-dev
sudo apt-get install -y postgresql-client
stack setup
- run:
name: build src and tests
command: |
stack build --fast -j1
stack build --fast --test --no-run-tests
- run:
name: run tests
command: POSTGREST_TEST_CONNECTION=$(test/create_test_db "postgres://circleci@localhost" postgrest_test) stack test
build-prof-test:
docker:
- image: circleci/buildpack-deps:trusty
environment:
- PGHOST=localhost
- TERM=xterm
- image: circleci/postgres:9.6.2
environment:
- POSTGRES_USER=circleci
- POSTGRES_DB=circleci
steps:
- checkout
- restore_cache:
keys:
- v1-stack-prof-dependencies-{{ checksum "postgrest.cabal" }}-{{ checksum "stack.yaml" }}
- run:
name: install stack & dependencies
command: |
curl -L https://github.com/commercialhaskell/stack/releases/download/v1.1.2/stack-1.1.2-linux-x86_64.tar.gz | tar zx -C /tmp
sudo mv /tmp/stack-1.1.2-linux-x86_64/stack /usr/bin
sudo apt-get update
sudo apt-get install -y libgmp-dev
sudo apt-get install -y postgresql-client
stack setup
- run:
name: build with profiling enabled
command: |
stack build --profile -j1
- run:
name: run memory usage tests
command: |
test/create_test_db "postgres://circleci@localhost" postgrest_test
psql "postgres:///postgrest_test" -f test/fixtures/database.sql
psql "postgres:///postgrest_test" -f test/fixtures/roles.sql
psql "postgres:///postgrest_test" -f test/fixtures/schema.sql
psql "postgres:///postgrest_test" -f test/fixtures/jwt.sql
psql "postgres:///postgrest_test" -f test/fixtures/privileges.sql
test/memory-tests.sh
- save_cache:
paths:
- "~/.stack"
- ".stack-work"
key: v1-stack-prof-dependencies-{{ checksum "postgrest.cabal" }}-{{ checksum "stack.yaml" }}
centos6:
<<: *build-distro-bin
centos7:
<<: *build-distro-bin
ubuntu:
<<: *build-distro-bin
ubuntui386:
<<: *build-distro-bin
release:
docker:
- image: circleci/golang:1.8
steps:
- attach_workspace:
at: /tmp/workspace
- checkout
- run:
name: add body and tars to github release
command: |
go get -u github.com/tcnksm/ghr
START=$(echo $CIRCLE_TAG | cut -c2-)
END='## \['
BODY=$(sed -n "1,/$START/d;/$END/q;p" CHANGELOG.md)
ghr -t $GITHUB_TOKEN -u $CIRCLE_PROJECT_USERNAME -r $CIRCLE_PROJECT_REPONAME -b "$BODY" --replace $CIRCLE_TAG /tmp/workspace/bin
- setup_remote_docker
- run:
name: publish docker image
command: |
docker build --build-arg POSTGREST_VERSION=$CIRCLE_TAG -t postgrest ./docker/
docker login -u $DOCKER_USER -p $DOCKER_PASS
docker tag postgrest postgrest/postgrest:$CIRCLE_TAG
docker push postgrest/postgrest:$CIRCLE_TAG
docker tag postgrest postgrest/postgrest:latest
docker push postgrest/postgrest:latest
workflows:
version: 2
build-test-release:
jobs:
- build-test-9.4:
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
- build-test-9.6:
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
- build-test-10:
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
- build-prof-test:
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
- centos6:
requires:
- build-test-9.4
- build-test-9.6
- build-test-10
- build-prof-test
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
branches:
ignore: /.*/
- centos7:
requires:
- build-test-9.4
- build-test-9.6
- build-test-10
- build-prof-test
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
branches:
ignore: /.*/
- ubuntu:
requires:
- build-test-9.4
- build-test-9.6
- build-test-10
- build-prof-test
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
branches:
ignore: /.*/
- ubuntui386:
requires:
- build-test-9.4
- build-test-9.6
- build-test-10
- build-prof-test
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
branches:
ignore: /.*/
- release:
requires:
- centos6
- centos7
- ubuntu
- ubuntui386
filters:
tags:
only: /v[0-9]+(\.[0-9]+)*/
+15
View File
@@ -0,0 +1,15 @@
freebsd_instance:
image_family: freebsd-13-1
build_task:
name: Build FreeBSD (Stack)
install_script: pkg install -y postgresql13-client hs-stack
stack_cache:
folders: /.stack
fingerprint_script: cat postgrest.cabal stack.yaml.lock
reupload_on_changes: false
build_script: stack build -j 1 --local-bin-path . --copy-bins
bin_artifacts:
path: postgrest
+15 -18
View File
@@ -15,14 +15,14 @@ your contributions.
## Issues
For questions on how to use PostgREST, please use
[GitHub discussions](https://github.com/PostgREST/postgrest/discussions).
### Reporting an Issue
* Make sure you test against the latest released version. It is possible
we already fixed the bug you're experiencing.
* Also check the `CHANGELOG.md` to see if any unreleased changes affect
the issue. The very newest changes can take a little while to be released
as a new official version.
* Make sure you test against the latest [stable release](https://github.com/PostgREST/postgrest/releases/latest)
and also against the latest [nightly release](https://github.com/PostgREST/postgrest/releases/tag/nightly).
It is possible we already fixed the bug you're experiencing.
* Provide steps to reproduce the issue, including your OS version and
the specific database schema that you are using.
@@ -32,27 +32,24 @@ your contributions.
then [find your logs](http://blog.endpoint.com/2014/11/dear-postgresql-where-are-my-logs.html).
* If your database schema has changed while the PostgREST server is running,
send the server a `SIGHUP` signal or restart it to ensure the schema cache
[send the server a `SIGUSR1` signal](http://postgrest.org/en/latest/admin.html#schema-reloading) or restart it to ensure the schema cache
is not stale. This sometimes fixes apparent bugs.
## Code
We have a fully nix-based development environment with many tools for a smooth development workflow available.
Check the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md) on how to set it up and use it.
### Haskell Conventions
* All contributions must pass the tests before being merged. When
you create a pull request your code will automatically be tested.
* All code must also pass [hlint](http://community.haskell.org/~ndm/hlint/)
with no warnings. This helps enforce a uniform style for all
committers. Continuous integration will check this as well on every
pull request.
* For help building the Haskell code on your computer check out the [building from
source](https://postgrest.com/en/stable/install.html#build-from-source)
wiki page.
* All code must also pass [hlint](http://community.haskell.org/~ndm/hlint/) and [stylish-haskell](https://github.com/jaspervdj/stylish-haskell)
with no warnings. This helps enforce a uniform style for all committers. Continuous integration will check this as well on every
pull request. There are useful tools in the nix-shell that help with checking this locally. You can run `postgrest-check` to do this manually but
we recommend adding it to `.git/hooks/pre-commit` as `nix-shell --run postgrest-check` to automatically check this before doing a commit.
### Running Tests
For instructions on running tests, see the official docs hosted here:
https://postgrest.com/en/stable/install.html#postgrest-test-suite
For instructions on running tests, see the [development docs](https://github.com/PostgREST/postgrest/blob/main/nix/README.md#testing).
+3
View File
@@ -0,0 +1,3 @@
# These are supported funding model platforms
patreon: postgrest
+6
View File
@@ -1,3 +1,9 @@
<!--
Before reporting a bug:
If your database schema has changed while the PostgREST server is running,
send the server a SIGUSR1 signal or restart it(http://postgrest.org/en/stable/admin.html#schema-reloading)
to ensure the schema cache is not stale. This sometimes fixes apparent bugs.
-->
### Environment
* PostgreSQL version: (if using docker, specify the image)
+6
View File
@@ -0,0 +1,6 @@
<!--
When submitting a new feature or fix:
- Add a new entry to the CHANGELOG - https://github.com/PostgREST/postgrest/blob/main/CHANGELOG.md#unreleased
- If relevant, update the docs - https://github.com/PostgREST/postgrest-docs
-->
+21
View File
@@ -0,0 +1,21 @@
name: Setup Nix
description: Installs nix, sets up cachix and installs a subset of tooling.
inputs:
authToken:
description: Token to pass to cachix
tools:
description: Tools to install with nix-env -iA <tools>
runs:
using: composite
steps:
- uses: cachix/install-nix-action@v16
- uses: cachix/cachix-action@v10
with:
name: postgrest
authToken: ${{ inputs.authToken }}
- if: ${{ inputs.tools }}
run: nix-env -f default.nix -iA ${{ inputs.tools }}
shell: bash
+18
View File
@@ -0,0 +1,18 @@
codecov:
branch: main
require_ci_to_pass: false
comment: false
coverage:
status:
project:
default:
target: auto
threshold: 1%
only_pulls: false
patch:
default:
target: auto
threshold: 1%
only_pulls: true
+6
View File
@@ -0,0 +1,6 @@
version: 2
updates:
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
set -euo pipefail
# Get the FreeBSD PostgREST executable built by CirrusCI for the given GITHUB_COMMIT in GITHUB_REPOSITORY
# We use the GitHub API for 'check suites' to find the corresponding CirrusCI job, see:
# https://docs.github.com/en/rest/reference/checks#list-check-suites-for-a-git-reference
cirrus_artifact_name=bin
gh_accept_header="Accept: application/vnd.github.v3+json"
get_gh_check_runs_url() {
gh_checks_list_url="https://api.github.com/repos/$GITHUB_REPOSITORY/commits/$GITHUB_COMMIT/check-suites"
>&2 echo "Getting list of check-suites from $gh_checks_list_url ..."
curl --fail -H "$gh_accept_header" "$gh_checks_list_url" \
| jq -r '.check_suites[] | select(.app.slug == "cirrus-ci") | .check_runs_url'
}
wait_for_cirrusci() {
gh_check_runs_url="$(get_gh_check_runs_url)"
>&2 echo "Waiting to CirrusCI run to complete (two hours maximum)..."
for _ in $(seq 1 120); do
echo "Checking for CirrusCI task status at $gh_check_runs_url ..."
status=$(curl --fail "$gh_check_runs_url" | jq -r '.check_runs[] | .status')
if [ "$status" == "completed" ]; then
break
else
echo "CirrusCI task is still $status, waiting..."
sleep 60
fi
done
}
# The CirrusCI taskid can change if a new check run is started for the same commit,
# e.g. when pushing both a branch and tag. We make sure that we have the very
# latest taskid by re-loading the 'gh_check_runs_url' and the 'check run' itself.
get_cirrus_taskid() {
gh_check_runs_url="$(get_gh_check_runs_url)"
>&2 echo "Getting the CirrusCI task id from $gh_check_runs_url ..."
curl --fail -H "$gh_accept_header" "$gh_check_runs_url" \
| jq -r '.check_runs[] | .external_id'
}
download_artifact() {
cirrus_task_id="$(get_cirrus_taskid)"
cirrus_artifact_url="https://api.cirrus-ci.com/v1/artifact/task/$cirrus_task_id/$cirrus_artifact_name.zip"
>&2 echo "Attemping to download the CirrusCI artifact from $cirrus_artifact_url ..."
curl --fail "$cirrus_artifact_url" -o freebsd.zip
}
wait_for_cirrusci
download_artifact
echo "Unpacking executable..."
unzip freebsd.zip -d .
rm -rf freebsd.zip
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Tag a release that will be built with Github Actions. The version is
# read from 'postgrest.cabal'.
version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
echo "Tagging version v$version"
git tag -f "v$version"
echo "Pushing tag..."
git push -f origin "refs/tags/v$version"
+63
View File
@@ -0,0 +1,63 @@
#!/bin/bash
# This script builds PostgREST in a remote ARM server
[ -z "$1" ] && { echo "Missing 1st argument: PostgREST github commit SHA"; exit 1; }
[ -z "$2" ] && { echo "Missing 2nd argument: Build environment directory name"; exit 1; }
PGRST_GITHUB_COMMIT="$1"
SCRIPT_DIR="$2"
DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"
install_packages() {
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get install -y git build-essential curl libffi-dev libffi7 libgmp-dev libgmp10 libncurses-dev libncurses5 libtinfo5 llvm libnuma-dev zlib1g-dev libpq-dev jq gcc
sudo apt-get clean
}
install_ghcup() {
export BOOTSTRAP_HASKELL_NONINTERACTIVE=1
export BOOTSTRAP_HASKELL_MINIMAL=1
curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh
source ~/.ghcup/env
}
install_cabal() {
ghcup upgrade
ghcup install cabal 3.6.0.0
ghcup set cabal 3.6.0.0
}
install_ghc() {
ghcup install ghc 8.10.7
ghcup set ghc 8.10.7
}
install_packages
# Add ghcup to the PATH for this session
[ -f ~/.ghcup/env ] && source ~/.ghcup/env
ghcup --version || install_ghcup
cabal --version || install_cabal
ghc --version || install_ghc
cd ~/$SCRIPT_DIR
# Clone the repository and build the project
git clone https://github.com/PostgREST/postgrest.git
cd postgrest
git checkout $PGRST_GITHUB_COMMIT
cabal v2-update && cabal v2-build
# Copy the built binary to the Dockerfile directory
PGRST_BIN=$(cabal exec which postgrest | tail -1)
cp $PGRST_BIN ~/$DOCKER_BUILD_DIR
# Move and compress the built binary
mkdir -p ~/$SCRIPT_DIR/result
mv $PGRST_BIN ~/$SCRIPT_DIR/result
cd ~/$SCRIPT_DIR
tar -cJf result.tar.xz result
+16
View File
@@ -0,0 +1,16 @@
# PostgREST docker hub image
FROM ubuntu:focal AS postgrest
RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY postgrest /usr/bin/postgrest
EXPOSE 3000
USER 1000
CMD postgrest
+50
View File
@@ -0,0 +1,50 @@
#!/bin/bash
# This script publishes the Docker ARM images to Docker Hub.
[ -z "$1" ] && { echo "Missing 1st argument: PostgREST github commit SHA"; exit 1; }
[ -z "$2" ] && { echo "Missing 2nd argument: Docker repo"; exit 1; }
[ -z "$3" ] && { echo "Missing 3rd argument: Docker username"; exit 1; }
[ -z "$4" ] && { echo "Missing 4th argument: Docker password"; exit 1; }
[ -z "$5" ] && { echo "Missing 5th argument: Build environment directory name"; exit 1; }
[ -z "$6" ] && { echo "Missing 6th argument: PostgREST version"; exit 1; }
PGRST_GITHUB_COMMIT="$1"
DOCKER_REPO="$2"
DOCKER_USER="$3"
DOCKER_PASS="$4"
SCRIPT_DIR="$5"
PGRST_VERSION="v$6"
IS_PRERELEASE="$7"
DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"
clean_env()
{
sudo docker logout
}
# Login to Docker
sudo docker logout
{ echo $DOCKER_PASS | sudo docker login -u $DOCKER_USER --password-stdin; } || { echo "Couldn't login to docker"; exit 1; }
trap clean_env sigint sigterm exit
# Move to the docker build environment
cd ~/$DOCKER_BUILD_DIR
# Push final images to Docker hub
# NOTE: This command publishes a separate ARM image because the builds cannot
# be added to the manifest if they are not in the registry beforehand.
# This image must be manually deleted from Docker Hub at the end of the process.
sudo docker buildx build --build-arg PGRST_GITHUB_COMMIT=$PGRST_GITHUB_COMMIT \
-t $DOCKER_REPO/postgrest:$PGRST_VERSION-arm \
--push .
# Add the arm images to the manifest
# NOTE: This assumes that there already is a `postgrest:<version>` image
# for the amd64 architecture pushed to Docker Hub
sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:$PGRST_VERSION $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
[ -z $IS_PRERELEASE ] && sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:latest $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
sudo docker logout
+476
View File
@@ -0,0 +1,476 @@
name: CI
on:
push:
branches:
- main
- rel-*
tags:
- v*
pull_request:
branches:
- main
- rel-*
jobs:
Lint-Style:
name: Lint & check code style
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
tools: style
- name: Run linter (check locally with `nix-shell --run postgrest-lint`)
run: postgrest-lint
- name: Run style check (auto-format with `nix-shell --run postgrest-style`)
run: postgrest-style-check
Test-Nix:
name: Test (Nix)
runs-on: ubuntu-latest
defaults:
run:
# Hack for enabling color output, see:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@v3
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
tools: tests
- name: Run coverage (IO tests and Spec tests against PostgreSQL 14)
run: postgrest-coverage
- name: Upload coverage to codecov
uses: codecov/codecov-action@v3.1.0
with:
files: ./coverage/codecov.json
- name: Run doctests
if: always()
run: nix-shell --run postgrest-test-doctests
- name: Check the spec tests for idempotence
if: always()
run: postgrest-test-spec-idempotence
Test-Pg-Nix:
strategy:
fail-fast: false
matrix:
pgVersion: [9.6, 10, 11, 12, 13, 14]
name: Test PG ${{ matrix.pgVersion }} (Nix)
runs-on: ubuntu-latest
defaults:
run:
# Hack for enabling color output, see:
# https://github.com/actions/runner/issues/241#issuecomment-842566950
shell: script -qec "bash --noprofile --norc -eo pipefail {0}"
steps:
- uses: actions/checkout@v3
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
tools: tests withTools
- name: Run spec tests
if: always()
run: postgrest-with-postgresql-${{ matrix.pgVersion }} postgrest-test-spec
- name: Run IO tests
if: always()
run: postgrest-with-postgresql-${{ matrix.pgVersion }} -f test/io/fixtures.sql postgrest-test-io
- name: Run query cost tests
if: always()
run: postgrest-with-postgresql-${{ matrix.pgVersion }} postgrest-test-querycost
Test-Memory-Nix:
name: Test memory (Nix)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
tools: memory
- name: Run memory tests
run: postgrest-test-memory
Build-Static-Nix:
name: Build Linux static (Nix)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: tests
- name: Build static executable
run: nix-build -A postgrestStatic
- name: Check static executable
run: postgrest-check-static result/bin/postgrest
- name: Save built executable as artifact
uses: actions/upload-artifact@v3
with:
name: postgrest-linux-static-x64
path: result/bin/postgrest
if-no-files-found: error
- name: Build Docker image
run: nix-build -A docker.image --out-link postgrest-docker.tar.gz
- name: Save built Docker image as artifact
uses: actions/upload-artifact@v3
with:
name: postgrest-docker-x64
path: postgrest-docker.tar.gz
if-no-files-found: error
- name: Build and push everything to Cachix (main branch only)
if: ${{ github.ref == 'refs/heads/main' }}
run: |
nix-build
nix-env -f default.nix -iA devTools
postgrest-push-cachix
Build-Stack:
strategy:
fail-fast: false
matrix:
include:
- name: Linux & test
runs-on: ubuntu-latest
cache: |
~/.stack
.stack-work
artifact: postgrest-ubuntu-x64
- name: MacOS & test
runs-on: macos-latest
cache: |
~/.stack
.stack-work
artifact: postgrest-macos-x64
- name: Windows
runs-on: windows-latest
cache: |
~\AppData\Roaming\stack
~\AppData\Local\Programs\stack
.stack-work
deps: Add-Content $env:GITHUB_PATH $env:PGBIN
artifact: postgrest-windows-x64
name: Build ${{ matrix.name }} (Stack)
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@v3
- name: Stack working files cache
uses: actions/cache@v3
with:
path: ${{ matrix.cache }}
key: ${{ runner.os }}-${{ hashFiles('stack.yaml.lock') }}
- name: Install dependencies
if: ${{ matrix.deps }}
run: ${{ matrix.deps }}
- name: Build with Stack
run: stack build --local-bin-path result --copy-bins
- name: Save built executable as artifact
uses: actions/upload-artifact@v3
with:
name: ${{ matrix.artifact }}
path: |
result/postgrest
result/postgrest.exe
if-no-files-found: error
Get-FreeBSD-CirrusCI:
name: Get FreeBSD build from CirrusCI
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Get FreeBSD executable from CirrusCI
env:
# GITHUB_SHA does weird things for pull request, so we roll our own:
GITHUB_COMMIT: ${{github.event.pull_request.head.sha || github.sha}}
run: .github/get_cirrusci_freebsd
- name: Save executable as artifact
uses: actions/upload-artifact@v3
with:
name: postgrest-freebsd-x64
path: postgrest
if-no-files-found: error
Build-Cabal-Arm:
name: Build aarch64 (Cabal)
if: ${{ github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v') || startsWith(github.ref, 'refs/heads/rel-') }}
runs-on: ubuntu-latest
outputs:
remotepath: ${{ steps.Remote-Dir.outputs.remotepath }}
env:
GITHUB_COMMIT: ${{ github.sha }}
steps:
- uses: actions/checkout@v2.4.0
- id: Remote-Dir
name: Unique directory name for the remote build
run: echo "::set-output name=remotepath::postgrest-build-$(uuidgen)"
- name: Copy script files to the remote server
uses: appleboy/scp-action@master
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
source: ".github/scripts/arm/*"
target: ${{ steps.Remote-Dir.outputs.remotepath }}
strip_components: 3
- name: Build ARM
uses: appleboy/ssh-action@master
env:
REMOTE_DIR: ${{ steps.Remote-Dir.outputs.remotepath }}
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
command_timeout: 120m
script_stop: true
envs: GITHUB_COMMIT,REMOTE_DIR
script: bash ~/$REMOTE_DIR/build.sh "$GITHUB_COMMIT" "$REMOTE_DIR"
- name: Download binaries from remote server
uses: nicklasfrahm/scp-action@main
with:
direction: download
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
source: "${{ steps.Remote-Dir.outputs.remotepath }}/result.tar.xz"
target: "result.tar.xz"
- name: Extract downloaded binaries
run: tar -xvf result.tar.xz && rm result.tar.xz
- name: Save aarch64 executable as artifact
uses: actions/upload-artifact@v2.3.1
with:
name: postgrest-ubuntu-aarch64
path: result/postgrest
if-no-files-found: error
Prepare-Release:
name: Prepare release
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
needs:
- Lint-Style
- Test-Nix
- Test-Pg-Nix
- Test-Memory-Nix
- Build-Static-Nix
- Build-Stack
#- Get-FreeBSD-CirrusCI
- Build-Cabal-Arm
outputs:
version: ${{ steps.Identify-Version.outputs.version }}
isprerelease: ${{ steps.Identify-Version.outputs.isprerelease }}
steps:
- uses: actions/checkout@v3
- id: Identify-Version
name: Identify the version to be released
run: |
tag_version="${GITHUB_REF##*/}"
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
if [ "$tag_version" != "v$cabal_version" ]; then
echo "Tagged version ($tag_version) does not match the one in postgrest.cabal (v$cabal_version). Aborting release..."
exit 1
else
echo "Version to be released is $cabal_version"
echo "::set-output name=version::$cabal_version"
fi
if [[ "$cabal_version" != *.*.*.* ]]; then
echo "Version is for a full release (version does not have four components)"
else
echo "Version is for a pre-release (version has four components, e.g., 1.1.1.1)"
echo "::set-output name=isprerelease::1"
fi
- name: Identify changes from CHANGELOG.md
run: |
version="${{ steps.Identify-Version.outputs.version }}"
isprerelease="${{ steps.Identify-Version.outputs.isprerelease }}"
if [ -n "$isprerelease" ]; then
echo "Getting unreleased changes..."
sed -n "1,/## Unreleased/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
else
echo "Full release (no '-' in version), getting changes for version $version ..."
sed -n "1,/## \[$version\]/d;/## \[/q;p" CHANGELOG.md > CHANGES.md
fi
echo "Relevant extract from CHANGELOG.md:"
cat CHANGES.md
- name: Save CHANGES.md as artifact
uses: actions/upload-artifact@v3
with:
name: release-changes
path: CHANGES.md
if-no-files-found: error
Release-GitHub:
name: Release on GitHub
permissions:
contents: write
runs-on: ubuntu-latest
needs: Prepare-Release
env:
VERSION: ${{ needs.Prepare-Release.outputs.version }}
steps:
- uses: actions/checkout@v3
- name: Download all artifacts
uses: actions/download-artifact@v3
with:
path: artifacts
- name: Create release bundle with archives for all builds
run: |
find artifacts -type f -iname postgrest -exec chmod +x {} \;
mkdir -p release-bundle
tar cJvf "release-bundle/postgrest-v$VERSION-linux-static-x64.tar.xz" \
-C artifacts/postgrest-linux-static-x64 postgrest
# No need to release Ubuntu, as the static Linux binary built with Nix
# covers all Linux use-cases
#tar cfJv "release-bundle/postgrest-v$VERSION-ubuntu-x64.tar.xz" \
# -C artifacts/postgrest-ubuntu-x64 postgrest
tar cJvf "release-bundle/postgrest-v$VERSION-macos-x64.tar.xz" \
-C artifacts/postgrest-macos-x64 postgrest
# TODO: Fix timeouts for FreeBSD builds in Cirrus
#tar cJvf "release-bundle/postgrest-v$VERSION-freebsd-x64.tar.xz" \
# -C artifacts/postgrest-freebsd-x64 postgrest
tar cJvf "release-bundle/postgrest-v$VERSION-ubuntu-aarch64.tar.xz" \
-C artifacts/postgrest-ubuntu-aarch64 postgrest
zip "release-bundle/postgrest-v$VERSION-windows-x64.zip" \
artifacts/postgrest-windows-x64/postgrest.exe
- name: Save release bundle
uses: actions/upload-artifact@v3
with:
name: release-bundle
path: release-bundle
if-no-files-found: error
- name: Publish release on GitHub
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
isprerelease="${{ needs.Prepare-Release.outputs.isprerelease }}"
echo "Releasing version v$VERSION on GitHub (isprerelease=$isprerelease)..."
gh release delete "v$VERSION" || true
gh release create "v$VERSION" \
-F artifacts/release-changes/CHANGES.md \
${isprerelease:+"--prerelease"} \
release-bundle/*
Release-Docker:
name: Release on Docker Hub
runs-on: ubuntu-latest
needs:
- Build-Cabal-Arm
- Prepare-Release
env:
GITHUB_COMMIT: ${{ github.sha }}
DOCKER_REPO: postgrest
DOCKER_USER: stevechavez
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
VERSION: ${{ needs.Prepare-Release.outputs.version }}
ISPRERELEASE: ${{ needs.Prepare-Release.outputs.isprerelease }}
steps:
- uses: actions/checkout@v3
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
tools: release
- name: Download Docker image
uses: actions/download-artifact@v3
with:
name: postgrest-docker-x64
- name: Publish images on Docker Hub
run: |
docker login -u "$DOCKER_USER" -p "$DOCKER_PASS"
docker load -i postgrest-docker.tar.gz
docker tag postgrest:latest "$DOCKER_REPO/postgrest:v$VERSION"
docker push "$DOCKER_REPO/postgrest:v$VERSION"
# Only tag 'latest' for full releases
if [[ -z "$ISPRERELEASE" ]]; then
echo "Pushing to 'latest' tag for full release of v$VERSION ..."
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
docker push "$DOCKER_REPO"/postgrest:latest
else
echo "Skipping pushing to 'latest' tag for v$VERSION pre-release..."
fi
- name: Publish images for ARM builds on Docker Hub
uses: appleboy/ssh-action@master
env:
REMOTE_DIR: ${{ needs.Build-Cabal-Arm.outputs.remotepath }}
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
script_stop: true
envs: GITHUB_COMMIT,DOCKER_REPO,DOCKER_USER,DOCKER_PASS,REMOTE_DIR,VERSION,ISPRERELEASE
script: bash ~/$REMOTE_DIR/docker-publish.sh "$GITHUB_COMMIT" "$DOCKER_REPO" "$DOCKER_USER" "$DOCKER_PASS" "$REMOTE_DIR" "$VERSION" "$ISPRERELEASE"
# TODO: Enable dockerhub description update again, once a solution for the permission problem is found:
# https://github.com/docker/hub-feedback/issues/1927
# - name: Update descriptions on Docker Hub
# env:
# DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
# run: |
# if [[ -z "$ISPRERELEASE" ]]; then
# echo "Updating description on Docker Hub..."
# postgrest-release-dockerhub-description
# else
# echo "Skipping updating description for pre-release..."
# fi
Clean-Arm-Server:
name: Remove copied files from server
needs:
- Build-Cabal-Arm
- Release-Docker
if: ${{ always() && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v') || startsWith(github.ref, 'refs/heads/rel-')) }}
runs-on: ubuntu-latest
env:
REMOTE_DIR: ${{ needs.Build-Cabal-Arm.outputs.remotepath }}
steps:
- uses: actions/checkout@v2.4.0
- name: Remove uploaded files from server
uses: appleboy/ssh-action@master
with:
host: ${{ secrets.SSH_ARM_HOST }}
username: ubuntu
key: ${{ secrets.SSH_ARM_PRIVATE_KEY }}
fingerprint: ${{ secrets.SSH_ARM_FINGERPRINT }}
envs: REMOTE_DIR
script: rm -rf $REMOTE_DIR
+34
View File
@@ -0,0 +1,34 @@
name: Loadtest
on:
push:
branches:
- main
tags:
- v*
pull_request:
branches:
- main
jobs:
Loadtest-Nix:
name: Loadtest (Nix)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
tools: loadtest
- name: Run loadtest
run: |
postgrest-loadtest-against main
postgrest-loadtest-report > loadtest/loadtest.md
- name: Upload report
uses: actions/upload-artifact@v3
with:
name: loadtest.md
path: loadtest/loadtest.md
if-no-files-found: error
+33
View File
@@ -0,0 +1,33 @@
name: Upload Reports
on:
workflow_run:
workflows: ["Loadtest"]
types:
- completed
jobs:
upload:
name: Loadtest
permissions:
checks: write
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
steps:
- name: Download from Artifacts
uses: dawidd6/action-download-artifact@v2
with:
workflow: ${{ github.event.workflow.name }}
run_id: ${{github.event.workflow_run.id }}
name: loadtest.md
path: artifacts
- name: Upload to GitHub Checks
uses: LouisBrunner/checks-action@v1.2.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
sha: ${{ github.event.workflow_run.head_sha }}
name: Loadtest Results
conclusion: neutral
output: |
{"summary":""}
output_text_description_file: artifacts/loadtest.md
+11
View File
@@ -12,3 +12,14 @@ site
*~
*#*
.#*
*.swp
result*
dist-newstyle
postgrest.hp
postgrest.prof
__pycache__
*.tix
coverage
.hpc
loadtest
.history
+40 -5
View File
@@ -39,9 +39,9 @@ steps:
# - none: Do not perform any alignment.
#
# Default: global.
align: global
align: group
# Folowing options affect only import list alignment.
# The following options affect only import list alignment.
#
# List align has following options:
#
@@ -64,6 +64,25 @@ steps:
# Default: after_alias
list_align: after_alias
# Right-pad the module names to align imports in a group:
#
# - true: a little more readable
#
# > import qualified Data.List as List (concat, foldl, foldr,
# > init, last, length)
# > import qualified Data.List.Extra as List (concat, foldl, foldr,
# > init, last, length)
#
# - false: diff-safe
#
# > import qualified Data.List as List (concat, foldl, foldr, init,
# > last, length)
# > import qualified Data.List.Extra as List (concat, foldl, foldr,
# > init, last, length)
#
# Default: true
pad_module_names: true
# Long list align style takes effect when import is too long. This is
# determined by 'columns' setting.
#
@@ -75,7 +94,7 @@ steps:
# short enough to fit to single line. Otherwise it'll be multiline.
#
# - multiline: One line per import list entry.
# Type with contructor list acts like single import.
# Type with constructor list acts like single import.
#
# > import qualified Data.Map as M
# > ( empty
@@ -109,7 +128,7 @@ steps:
# Useful for 'file' and 'group' align settings.
list_padding: 4
# Separate lists option affects formating of import list for type
# Separate lists option affects formatting of import list for type
# or class. The only difference is single space between type and list
# of constructors, selectors and class functions.
#
@@ -126,6 +145,22 @@ steps:
# Default: true
separate_lists: true
# Space surround option affects formatting of import lists on a single
# line. The only difference is single space after the initial
# parenthesis and a single space before the terminal parenthesis.
#
# - true: There is single space associated with the enclosing
# parenthesis.
#
# > import Data.Foo ( foo )
#
# - false: There is no space associated with the enclosing parenthesis
#
# > import Data.Foo (foo)
#
# Default: false
space_surround: false
# Language pragmas
- language_pragmas:
# We can generate different styles of language pragma lists.
@@ -142,7 +177,7 @@ steps:
# Align affects alignment of closing pragma brackets.
#
# - true: Brackets are aligned in same collumn.
# - true: Brackets are aligned in same column.
#
# - false: Brackets are not aligned together. There is only one space
# between actual import and closing bracket.
-64
View File
@@ -1,64 +0,0 @@
language: generic
sudo: false
os:
- osx
cache:
timeout: 1000
directories:
- $HOME/.stack
- $HOME/.local/bin
before_install:
- mkdir -p "$HOME/.local/bin"
- export PATH="$PATH:$HOME/.local/bin"
install:
- |
if test -f "$HOME/.local/bin/stack"
then
echo 'Stack is already installed.'
else
echo "Installing Stack..."
travis_retry curl -L https://www.stackage.org/stack/osx-x86_64 > stack.tar.gz
gunzip stack.tar.gz
tar -x -f stack.tar --strip-components 1
mv stack "$HOME/.local/bin/"
rm stack.tar
fi
- |
if test -f "$HOME/.local/bin/ghr"
then
echo 'ghr is already installed.'
else
echo "Installing ghr..."
travis_retry curl -L https://github.com/tcnksm/ghr/releases/download/v0.5.4/ghr_v0.5.4_darwin_386.zip > ghr.zip
unzip ghr.zip -d "$HOME/.local/bin"
rm ghr.zip
fi
script:
- gtimeout 1800 stack build --no-terminal --only-snapshot --install-ghc || true
- |
if test ! "$TRAVIS_TAG"
then
echo 'No tag pushed. Skipping build.'
else
stack build --no-terminal --copy-bins --local-bin-path .
fi
- |
if test ! "$TRAVIS_TAG"
then
echo 'No tag pushed. Skipping release.'
else
OWNER="$(echo "$TRAVIS_REPO_SLUG" | cut -f1 -d/)"
REPO="$(echo "$TRAVIS_REPO_SLUG" | cut -f2 -d/)"
START=$(echo $TRAVIS_TAG | cut -c2-)
END='## \['
BODY=$(sed -n "1,/$START/d;/$END/q;p" CHANGELOG.md)
strip postgrest
tar cjf postgrest-$TRAVIS_TAG-osx.tar.xz postgrest
ghr -t $GITHUB_TOKEN -u $OWNER -r $REPO -b "$BODY"--replace $TRAVIS_TAG postgrest-$TRAVIS_TAG-osx.tar.xz
fi
+74 -3
View File
@@ -2,13 +2,84 @@
PostgREST ongoing development is only possible thanks to our Sponsors and Backers, listed below. If you'd like to join them, you can do so by supporting the PostgREST organization on [Patreon](https://www.patreon.com/postgrest).
## Sponsors
<table>
<tbody>
<tr>
<td align="center" valign="middle">
<a href="https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest" target="_blank">
<img width="222px" src="static/cybertec-new.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo" target="_blank">
<img width="296px" src="static/2ndquadrant.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tryretool.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/retool.png">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank">
<img width="296px" src="static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/oblivious.jpg">
</a>
</td>
</tr>
</tbody>
</table>
## Lead Backers
- [Christiaan Westerbeek](https://devotis.nl)
- [Daniel Babiak](https://github.com/d-babiak)
- [Michel Pelletier](https://github.com/michelp/)
- Evans Fernandes
- [Jan Sommer](https://github.com/nerfpops)
- [Franz Gusenbauer](https://www.igutech.at/)
## Backers
- Tsingson Qin
- Michel Pelletier
- Jay Hannah
- Robert Stolarz
- Nicholas DiBiase
- Christopher Reid
- Nathan Bouscal
- Daniel Rafaj
- David Fenko
- Remo Rechkemmer
- Severin Ibarluzea
- Tom Saleeba
- Pawel Tyll
## Former Backers
<table>
<tbody>
<tr>
<td align="center" valign="middle">
<a href="https://www.timescale.com?utm_campaign=postgrest&utm_source=sponsor&utm_medium=referral&utm_content=github" target="_blank">
<img width="222px" src="static/timescaledb.png">
</a>
</td>
</tr>
</tbody>
</table>
- [Christiaan Westerbeek](https://devotis.nl)
- [Daniel Babiak](https://github.com/dbabiak)
- Kofi Gumbs
+296
View File
@@ -5,10 +5,306 @@ This project adheres to [Semantic Versioning](http://semver.org/).
## Unreleased
## [10.0.0] - 2022-08-18
### Added
- #1933, #2109, Add a minimal health check endpoint - @steve-chavez
+ For enabling this, the `admin-server-port` config must be set explictly
+ A `<host>:<admin_server_port>/live` endpoint is available for checking if postgrest is running on its port/socket. 200 OK = alive, 503 = dead.
+ A `<host>:<admin_server_port>/ready` endpoint is available for checking a correct internal state(the database connection plus the schema cache). 200 OK = ready, 503 = not ready.
- #1988, Add the current user to the request log on stdout - @DavidLindbom, @wolfgangwalther
- #1823, Add the ability to run postgrest without any configuration. - @wolfgangwalther
+ #1991, Add the ability to run without `db-uri` using libpq's PG environment variables to connect. - @wolfgangwalther
+ #1769, Add the ability to run without `db-schemas`, defaulting to `db-schemas=public`. - @wolfgangwalther
+ #1689, Add the ability to run without `db-anon-role` disabling anonymous access. - @wolfgangwalther
- #1543, Allow access to fields of composite types in select=, order= and filters through JSON operators -> and ->>. - @wolfgangwalther
- #2075, Allow access to array items in ?select=, ?order= and filters through JSON operators -> and ->>. - @wolfgangwalther
- #2156, #2211, Allow applying `limit/offset` to UPDATE/DELETE to only affect a subset of rows - @steve-chavez
+ It requires an explicit `order` on a unique column(s)
- #1917, Add error codes with the `"PGRST"` prefix to the error response body to differentiate PostgREST errors from PostgreSQL errors - @laurenceisla
- #1917, Normalize the error response body by always having the `detail` and `hint` error fields with a `null` value if they are empty - @laurenceisla
- #2176, Errors raised with `SQLSTATE` now include the message and the code in the response body - @laurenceisla
- #2236, Support POSIX regular expression operators for row filtering - @enote-kane
- #2202, Allow returning XML from RPCs - @fjf2002
- #2268, Allow returning XML from single-column queries - @fjf2002
- #2300, RPC POST for function w/single unnamed XML param #2300 - @fjf2002
- #1564, Allow geojson output by specifying the `Accept: application/geo+json` media type - @steve-chavez
+ Requires postgis >= 3.0
+ Works for GET, RPC, POST/PATCH/DELETE with `Prefer: return=representation`.
+ Resource embedding works and the embedded rows will go into the `properties` key
+ In case of multiple geometries in the same table, you can choose which one will go into the `geometry` key with the usual `?select` query parameter.
- #1082, Add security definitions to the OpenAPI output - @laurenceisla
- #2378, Support http OPTIONS method on RPC and root path - @steve-chavez
- #2354, Allow getting the EXPLAIN plan of a request by using the `Accept: application/vnd.pgrst.plan` header - @steve-chavez
+ Only allowed if the `db-plan-enabled` config is set to true
+ Can generate the plan for different media types using the `for` parameter: `Accept: application/vnd.pgrst.plan; for="application/vnd.pgrst.object"`
+ Different options for the plan can be used with the `options` parameter: `Accept: application/vnd.pgrst.plan; options=analyze|verbose|settings|buffers|wal`
+ The plan can be obtained in text or json by using different media type suffixes: `Accept: application/vnd.pgrst.plan+text` and `Accept: application/vnd.pgrst.plan+json`.
- #2144, Support computed relationships which allow extending and overriding relationships for resource embedding - @steve-chavez, @wolfgangwalther
- #1984, Detect one-to-one relationships for resource embedding - @steve-chavez
+ Detected when there's a foreign key with a unique constraint or when a foreign key is also a primary key
### Fixed
- #2058, Return 204 No Content without Content-Type for PUT - @wolfgangwalther
- #2107, Clarify error for failed schema cache load. - @steve-chavez
+ From `Database connection lost. Retrying the connection` to `Could not query the database for the schema cache. Retrying.`
- #1771, Fix silently ignoring filter on a non-existent embedded resource - @steve-chavez
- #2152, Remove functions, which are uncallable because of unnamend arguments from schema cache and OpenAPI output. - @wolfgangwalther
- #2145, Fix accessing json array fields with -> and ->> in ?select= and ?order=. - @wolfgangwalther
- #2155, Ignore `max-rows` on POST, PATCH, PUT and DELETE - @steve-chavez
- #2254, Fix inferring a foreign key column as a primary key column on views - @steve-chavez
- #2070, Restrict generated many-to-many relationships - @steve-chavez
+ Only adds many-to-many relationships when: a table has FKs to two other tables and these FK columns are part of the table's PK columns.
- #2278, Allow casting to types with underscores and numbers(e.g. `select=oid_array::_int4`) - @steve-chavez
- #2277, #2238, #1643, Prevent views from breaking one-to-many/many-to-one embeds when using column or FK as target - @steve-chavez
+ When using a column or FK as target for embedding(`/tbl?select=*,col-or-fk(*)`), only tables are now detected and views are not.
+ You can still use a column or an inferred FK on a view to embed a table(`/view?select=*,col-or-fk(*)`)
- #2317, Increase the `db-pool-timeout` to 1 hour to prevent frequent high connection latency - @steve-chavez
- #2341, The search path now correctly identifies schemas with uppercase and special characters in their names (regression) - @laurenceisla
- #2364, "404 Not Found" on nested routes and "405 Method Not Allowed" errors no longer start an empty database transaction - @steve-chavez
- #2342, Fix inaccurate result count when an inner embed was selected after a normal embed in the query string - @laurenceisla
- #2376, OPTIONS requests no longer start an empty database transaction - @steve-chavez
- #2395, Allow using columns with dollar sign($) without double quoting in filters and `select` - @steve-chavez
- #2410, Fix loop crash error on startup in Postgres 15 beta 3. Log: "UNION types \"char\" and text cannot be matched". - @yevon
- #2397, Fix race conditions managing database connection helper - @robx
- #2269, Allow `limit=0` in the request query to return an empty array - @gautam1168, @laurenceisla
### Changed
- #2001, Return 204 No Content without Content-Type for RPCs returning VOID - @wolfgangwalther
+ Previously, those RPCs would return "null" as a body with Content-Type: application/json.
- #2156, `limit/offset` now limits the affected rows on UPDATE/DELETE - @steve-chavez
+ Previously, `limit/offset` only limited the returned rows but not the actual updated rows
- #2155, `max-rows` is no longer applied on POST/PATCH/PUT/DELETE returned rows - @steve-chavez
+ This was misleading because the affected rows were not really affected by `max-rows`, only the returned rows were limited
- #2070, Restrict generated many-to-many relationships - @steve-chavez
+ A primary key that contains the foreign key columns is now needed for generating many-to-many relationships.
- #2277, Views now are not detected when embedding using the column or FK as target (`/view?select=*,column(*)`) - @steve-chavez
+ This embedding form was easily made ambiguous whenever a new view was added.
+ You can use computed relationships to keep this embedding form working
- #2312, Using `Prefer: return=representation` no longer returns a `Location` header - @laurenceisla
- #1984, For the cases where one to one relationships are detected, json objects will be returned instead of json arrays of length 1
+ If you wish to override this behavior, you can use computed relationships to return arrays again
## [9.0.1] - 2022-06-03
### Fixed
- #2165, Fix json/jsonb columns should not have type in OpenAPI spec - @clrnd
- #2020, Execute deferred constraint triggers when using `Prefer: tx=rollback` - @wolfgangwalther
- #2077, Fix `is` not working with upper or mixed case values like `NULL, TrUe, FaLsE` - @steve-chavez
- #2024, Fix schema cache loading when views with XMLTABLE and DEFAULT are present - @wolfgangwalther
- #1724, Fix wrong CORS header Authentication -> Authorization - @wolfgangwalther
- #2120, Fix reading database configuration properly when `=` is present in value - @wolfgangwalther
- #2135, Remove trigger functions from schema cache and OpenAPI output, because they can't be called directly anyway. - @wolfgangwalther
- #2101, Remove aggregates, procedures and window functions from the schema cache and OpenAPI output. - @wolfgangwalther
- #2153, Fix --dump-schema running with a wrong PG version. - @wolfgangwalther
- #2042, Keep working when EMFILE(Too many open files) is reached. - @steve-chavez
- #2147, Ignore `Content-Type` headers for `GET` requests when calling RPCs. - @laurenceisla
+ Previously, `GET` without parameters, but with `Content-Type: text/plain` or `Content-Type: application/octet-stream` would fail with `404 Not Found`, even if a function without arguments was available.
- #2239, Fix misleading disambiguation error where the content of the `relationship` key looks like valid syntax - @laurenceisla
- #2294, Disable parallel GC for better performance on higher core CPUs - @steve-chavez
- #1076, Fix using CPU while idle - @steve-chavez
## [9.0.0] - 2021-11-25
### Added
- #1783, Include partitioned tables into the schema cache. Allows embedding, UPSERT, INSERT with Location response, OPTIONS request and OpenAPI support for partitioned tables - @laurenceisla
- #1878, Add Retry-After hint header when in recovery mode - @gautam1168
- #1735, Allow calling function with single unnamed param through RPC POST. - @steve-chavez
+ Enables calling a function with a single json parameter without using `Prefer: params=single-object`
+ Enables uploading bytea to a function with `Content-Type: application/octet-stream`
+ Enables uploading raw text to a function with `Content-Type: text/plain`
- #1938, Allow escaping inside double quotes with a backslash, e.g. `?col=in.("Double\"Quote")`, `?col=in.("Back\\slash")` - @steve-chavez
- #1075, Allow filtering top-level resource based on embedded resources filters. This is enabled by adding `!inner` to the embedded resource, e.g. `/projects?select=*,clients!inner(*)&clients.id=eq.12`- @steve-chavez, @Iced-Sun
- #1857, Make GUC names for headers, cookies and jwt claims compatible with PostgreSQL v14 - @laurenceisla, @robertsosinski
+ Getting the value for a header GUC on PostgreSQL 14 is done using `current_setting('request.headers')::json->>'name-of-header'` and in a similar way for `request.cookies` and `request.jwt.claims`
+ PostgreSQL versions below 14 can opt in to the new JSON GUCs by setting the `db-use-legacy-gucs` config option to false (true by default)
- #1988, Allow specifying `unknown` for the `is` operator - @steve-chavez
- #2031, Improve error message for ambiguous embedding and add a relevant hint that includes unambiguous embedding suggestions - @laurenceisla
### Fixed
- #1871, Fix OpenAPI missing default values for String types and identify Array types as "array" instead of "string" - @laurenceisla
- #1930, Fix RPC return type handling for `RETURNS TABLE` with a single column. Regression of #1615. - @wolfgangwalther
- #1938, Fix using single double quotes(`"`) and backslashes(`/`) as values on the "in" operator - @steve-chavez
- #1992, Fix schema cache query failing with standard_conforming_strings = off - @wolfgangwalther
### Changed
- #1949, Drop support for embedding hints used with '.'(`select=projects.client_id(*)`), '!' should be used instead(`select=projects!client_id(*)`) - @steve-chavez
- #1783, Partitions (created using `PARTITION OF`) are no longer included in the schema cache. - @laurenceisla
- #2038, Dropped support for PostgreSQL 9.5 - @wolfgangwalther
## [8.0.0] - 2021-07-25
### Added
- #1525, Allow http status override through response.status guc - @steve-chavez
- #1512, Allow schema cache reloading with NOTIFY - @steve-chavez
- #1119, Allow config file reloading with SIGUSR2 - @steve-chavez
- #1558, Allow 'Bearer' with and without capitalization as authentication schema - @wolfgangwalther
- #1470, Allow calling RPC with variadic argument by passing repeated params - @wolfgangwalther
- #1559, No downtime when reloading the schema cache with SIGUSR1 - @steve-chavez
- #504, Add `log-level` config option. The admitted levels are: crit, error, warn and info - @steve-chavez
- #1607, Enable embedding through multiple views recursively - @wolfgangwalther
- #1598, Allow rollback of the transaction with Prefer tx=rollback - @wolfgangwalther
- #1633, Enable prepared statements for GET filters. When behind a connection pooler, you can disable preparing with `db-prepared-statements=false`
+ This increases throughput by around 30% for simple GET queries(no embedding, with filters applied).
- #1729, #1760, Get configuration parameters from the db and allow reloading config with NOTIFY - @steve-chavez
- #1824, Allow OPTIONS to generate certain HTTP methods for a DB view - @laurenceisla
- #1872, Show timestamps in startup/worker logs - @steve-chavez
- #1881, Add `openapi-mode` config option that allows ignoring roles privileges when showing the OpenAPI output - @steve-chavez
- CLI options(for debugging):
+ #1678, Add --dump-config CLI option that prints loaded config and exits - @wolfgangwalther
+ #1691, Add --example CLI option to show example config file - @wolfgangwalther
+ #1697, #1723, Add --dump-schema CLI option for debugging purposes - @monacoremo, @wolfgangwalther
- #1794, (Experimental) Add `request.spec` GUC for db-root-spec - @steve-chavez
### Fixed
- #1592, Removed single column restriction to allow composite foreign keys in join tables - @goteguru
- #1530, Fix how the PostgREST version is shown in the help text when the `.git` directory is not available - @monacoremo
- #1094, Fix expired JWTs starting an empty transaction on the db - @steve-chavez
- #1162, Fix location header for POST request with select= without PK - @wolfgangwalther
- #1585, Fix error messages on connection failure for localized postgres on Windows - @wolfgangwalther
- #1636, Fix `application/octet-stream` appending `charset=utf-8` - @steve-chavez
- #1469, #1638 Fix overloading of functions with unnamed arguments - @wolfgangwalther
- #1560, Return 405 Method not Allowed for GET of volatile RPC instead of 500 - @wolfgangwalther
- #1584, Fix RPC return type handling and embedding for domains with composite base type (#1615) - @wolfgangwalther
- #1608, #1635, Fix embedding through views that have COALESCE with subselect - @wolfgangwalther
- #1572, Fix parsing of boolean config values for Docker environment variables, now it accepts double quoted truth values ("true", "false") and numbers("1", "0") - @wolfgangwalther
- #1624, Fix using `app.settings.xxx` config options in Docker, now they can be used as `PGRST_APP_SETTINGS_xxx` - @wolfgangwalther
- #1814, Fix panic when attempting to run with unix socket on non-unix host and properly close unix domain socket on exit - @monacoremo
- #1825, Disregard internal junction(in non-exposed schema) when embedding - @steve-chavez
- #1846, Fix requests for overloaded functions from html forms to no longer hang (#1848) - @laurenceisla
- #1858, Add a hint and clarification to the no relationship found error - @laurenceisla
- #1841, Show comprehensive error when an RPC is not found in a stale schema cache - @laurenceisla
- #1875, Fix Location headers in headers only representation for null PK inserts on views - @laurenceisla
### Changed
- #1522, #1528, #1535, Docker images are now built from scratch based on a the static PostgREST executable (#1494) and with Nix instead of a `Dockerfile`. This reduces the compressed image size from over 30mb to about 4mb - @monacoremo
- #1461, Location header for POST request is only included when PK is available on the table - @wolfgangwalther
- #1560, Volatile RPC called with GET now returns 405 Method not Allowed instead of 500 - @wolfgangwalther
- #1584, #1849 Functions that declare `returns composite_type` no longer return a single object array by default, only functions with `returns setof composite_type` return an array of objects - @wolfgangwalther
- #1604, Change the default logging level to `log-level=error`. Only requests with a status greater or equal than 500 will be logged. If you wish to go back to the previous behaviour and log all the requests, use `log-level=info` - @steve-chavez
+ Because currently there's no buffering for logging, defaulting to the `error` level(minimum logging) increases throughput by around 15% for simple GET queries(no embedding, with filters applied).
- #1617, Dropped support for PostgreSQL 9.4 - @wolfgangwalther
- #1679, Renamed config settings with fallback aliases. e.g. `max-rows` is now `db-max-rows`, but `max-rows` is still accepted - @wolfgangwalther
- #1656, Allow `Prefer=headers-only` on POST requests and change default to `minimal` (#1813) - @laurenceisla
- #1854, Dropped undocumented support for gzip compression (which was surprisingly slow and easily enabled by mistake). In some use-cases this makes Postgres up to 3x faster - @aljungberg
- #1872, Send startup/worker logs to stderr to differentiate from access logs on stdout - @steve-chavez
## [7.0.1] - 2020-05-18
### Fixed
- #1473, Fix overloaded computed columns on RPC - @wolfgangwalther
- #1471, Fix POST, PATCH, DELETE with ?select= and return=minimal and PATCH with empty body - @wolfgangwalther
- #1500, Fix missing `openapi-server-proxy-uri` config option - @steve-chavez
- #1508, Fix `Content-Profile` not working for POST RPC - @steve-chavez
- #1452, Fix PUT restriction for all columns - @steve-chavez
### Changed
- From this version onwards, the release page will only include a single Linux static executable that can be run on any Linux distribution.
## [7.0.0] - 2020-04-03
### Added
- #1417, `Accept: application/vnd.pgrst.object+json` behavior is now enforced for POST/PATCH/DELETE regardless of `Prefer: return=representation/minimal` - @dwagin
- #1415, Add support for user defined socket permission via `server-unix-socket-mode` config option - @Dansvidania
- #1383, Add support for HEAD request - @steve-chavez
- #1378, Add support for `Prefer: count=planned` and `Prefer: count=estimated` on GET /table - @steve-chavez, @LorenzHenk
- #1327, Add support for optional query parameter `on_conflict` to upsert with specified keys for POST - @ykst
- #1430, Allow specifying the foreign key constraint name(`/source?select=fk_constraint(*)`) to disambiguate an embedding - @steve-chavez
- #1168, Allow access to the `Authorization` header through the `request.header.authorization` GUC - @steve-chavez
- #1435, Add `request.method` and `request.path` GUCs - @steve-chavez
- #1088, Allow adding headers to GET/POST/PATCH/PUT/DELETE responses through the `response.headers` GUC - @steve-chavez
- #1427, Allow overriding provided headers(Location, Content-Type, etc) through the `response.headers` GUC - @steve-chavez
- #1450, Allow multiple schemas to be exposed in one instance. The schema to use can be selected through the headers `Accept-Profile` for GET/HEAD and `Content-Profile` for POST/PATCH/PUT/DELETE - @steve-chavez, @mahmoudkassem
### Fixed
- #1301, Fix self join resource embedding on PATCH - @herulume, @steve-chavez
- #1389, Fix many to many resource embedding on RPC/PATCH - @steve-chavez
- #1355, Allow PATCH/DELETE without `return=minimal` on tables with no select privileges - @steve-chavez
- #1361, Fix embedding a VIEW when its source foreign key is UNIQUE - @bwbroersma
### Changed
- #1385, bulk RPC call now should be done by specifying a `Prefer: params=multiple-objects` header - @steve-chavez
- #1401, resource embedding now outputs an error when multiple relationships between two tables are found - @steve-chavez
- #1423, default Unix Socket file mode from 755 to 660 - @dwagin
- #1430, Remove embedding with duck typed column names `GET /projects?select=client(*)`- @steve-chavez
+ You can rename the foreign key to `client` to make this request work in the new version: `alter table projects rename constraint projects_client_id_fkey to client`
- #1413, Change `server-proxy-uri` config option to `openapi-server-proxy-uri` - @steve-chavez
## [6.0.2] - 2019-08-22
### Fixed
- #1369, Change `raw-media-types` to accept a string of comma separated MIME types - @Dansvidania
- #1368, Fix long column descriptions being truncated at 63 characters in PostgreSQL 12 - @amedeedaboville
- #1348, Go back to converting plus "+" to space " " in querystrings by default - @steve-chavez
### Deprecated
- #1348, Deprecate `.` symbol for disambiguating resource embedding(added in #918). The url-safe '!' should be used instead. We refrained from using `+` as part of our syntax because it conflicts with some http clients and proxies.
## [6.0.1] - 2019-07-30
### Added
- #1349, Add user defined raw output media types via `raw-media-types` config option - @Dansvidania
- #1243, Add websearch_to_tsquery support - @herulume
### Fixed
- #1336, Error when testing on Chrome/Firefox: text/html requested but a single column was not selected - @Dansvidania
- #1334, Unable to compile v6.0.0 on windows - @steve-chavez
## [6.0.0] - 2019-06-21
### Added
- #1186, Add support for user defined unix socket via `server-unix-socket` config option - @Dansvidania
- #690, Add `?columns` query parameter for faster bulk inserts, also ignores unspecified json keys in a payload - @steve-chavez
- #1239, Add support for resource embedding on materialized views - @vitorbaptista
- #1264, Add support for bulk RPC call - @steve-chavez
- #1278, Add db-pool-timeout config option - @qu4tro
- #1285, Abort on wrong database password - @qu4tro
- #790, Allow override of OpenAPI spec through `root-spec` config option - @steve-chavez
- #1308, Accept `text/plain` and `text/html` for raw output - @steve-chavez
### Fixed
- #1223, Fix incorrect OpenAPI externalDocs url - @steve-chavez
- #1221, Fix embedding other resources when having a self join - @steve-chavez
- #1242, Fix embedding a view having a select in a where - @steve-chavez
- #1238, Fix PostgreSQL to OpenAPI type mappings for numeric and character types - @fpusch
- #1265, Fix query generated on bulk upsert with an empty array - @qu4tro
- #1273, Fix RPC ignoring unknown arguments by default - @steve-chavez
- #1257, Fix incorrect status when a PATCH request doesn't find rows to change - @qu4tro
### Changed
- #1288, Change server-host default of 127.0.0.1 to !4
### Deprecated
- #1288, Deprecate `.` symbol for disambiguating resource embedding(added in #918). '+' should be used instead. Though '+' is url safe, certain clients might need to encode it to '%2B'.
### Removed
- #1288, Removed support for schema reloading with SIGHUP, SIGUSR1 should be used instead - @steve-chavez
## [5.2.0] - 2018-12-12
### Added
+132
View File
@@ -0,0 +1,132 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, caste, color, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at support@postgrest.org.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
Community Impact Guidelines were inspired by
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
For answers to common questions about this code of conduct, see the FAQ at
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available
at [https://www.contributor-covenant.org/translations][translations].
[homepage]: https://www.contributor-covenant.org
[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations
+1
View File
@@ -1,4 +1,5 @@
Copyright (c) 2014 Joe Nelson
Copyright (c) 2019 Steve Chavez
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
+54 -19
View File
@@ -1,20 +1,62 @@
![Logo](static/logo.png "Logo")
![Logo](static/bigger-logo.png "Logo")
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854)](https://www.patreon.com/postgrest)
[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/postgrest)
<a href="https://heroku.com/deploy?template=https://github.com/PostgREST/postgrest">
<img src="https://img.shields.io/badge/%E2%86%91_Deploy_to-Heroku-7056bf.svg" alt="Deploy">
</a>
[![Join the chat at https://gitter.im/begriffs/postgrest](https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/begriffs/postgrest)
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://postgrest.org)
[![Docker Stars](https://img.shields.io/docker/pulls/postgrest/postgrest.svg)](https://hub.docker.com/r/postgrest/postgrest/)
[![Build Status](https://circleci.com/gh/PostgREST/postgrest/tree/master.svg?style=shield)](https://circleci.com/gh/PostgREST/postgrest/tree/master)
[![Build Status](https://github.com/postgrest/postgrest/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/PostgREST/postgrest/actions?query=branch%3Amain)
[![Coverage Status](https://img.shields.io/codecov/c/github/postgrest/postgrest/main)](https://app.codecov.io/gh/PostgREST/postgrest)
[![Hackage docs](https://img.shields.io/hackage/v/postgrest.svg?label=hackage)](http://hackage.haskell.org/package/postgrest)
PostgREST serves a fully RESTful API from any existing PostgreSQL
database. It provides a cleaner, more standards-compliant, faster
API than you are likely to write from scratch.
## Sponsors
<table>
<tbody>
<tr>
<td align="center" valign="middle">
<a href="https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest" target="_blank">
<img width="222px" src="static/cybertec-new.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo" target="_blank">
<img width="296px" src="static/2ndquadrant.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tryretool.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/retool.png">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank">
<img width="296px" src="static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="static/oblivious.jpg">
</a>
</td>
</tr>
</tbody>
</table>
Big thanks to our sponsors! You can join them by supporting PostgREST on [Patreon](https://www.patreon.com/postgrest).
## Usage
1. Download the binary ([latest release](https://github.com/PostgREST/postgrest/releases/latest))
@@ -24,13 +66,15 @@ API than you are likely to write from scratch.
```bash
postgrest --help
```
## [Documentation](http://postgrest.org)
Latest documentation is at [postgrest.org](http://postgrest.org). You can contribute to the docs in [PostgREST/postgrest-docs](https://github.com/PostgREST/postgrest-docs).
## Performance
TLDR; subsecond response times for up to 2000 requests/sec on Heroku
free tier. If you're used to servers written in interpreted languages
(or named after precious gems), prepare to be pleasantly surprised by
PostgREST performance.
free tier. If you're used to servers written in interpreted languages,
prepare to be pleasantly surprised by PostgREST performance.
Three factors contribute to the speed. First the server is written
in [Haskell](https://www.haskell.org/) using the
@@ -65,15 +109,6 @@ the connection cannot do anything the user themselves couldn't. Other
forms of authentication can be built on top of the JWT primitive. See
the docs for more information.
PostgreSQL 9.5 supports true [row-level
security](http://www.postgresql.org/docs/9.5/static/ddl-rowsecurity.html).
In previous versions it can be simulated with triggers and
security-barrier views. Because the possible queries to the database
are limited to certain templates using
[leakproof](http://blog.2ndquadrant.com/how-do-postgresql-security_barrier-views-work/)
functions, the trigger workaround does not compromise row-level
security.
## Versioning
A robust long-lived API needs the freedom to exist in multiple
@@ -122,8 +157,8 @@ Every donation will be spent on making PostgREST better for the whole community.
The PostgREST organization is grateful to:
- The project [sponsors and backers](https://github.com/PostgREST/postgrest/blob/master/BACKERS.md) who support PostgREST's development.
- The project [sponsors and backers](https://github.com/PostgREST/postgrest/blob/main/BACKERS.md) who support PostgREST's development.
- The project [contributors](https://github.com/PostgREST/postgrest/graphs/contributors) who have improved PostgREST immensely with their code
and good judgement. See more details in the [changelog](https://github.com/PostgREST/postgrest/blob/master/CHANGELOG.md).
and good judgement. See more details in the [changelog](https://github.com/PostgREST/postgrest/blob/main/CHANGELOG.md).
The cool logo came from [Mikey Casalaina](https://github.com/casalaina).
+1
View File
@@ -1,2 +1,3 @@
-- This file is required by Hackage.
import Distribution.Simple
main = defaultMain
-59
View File
@@ -1,59 +0,0 @@
{
"name": "PostgREST",
"description": "RESTful API for any PostgreSQL database.",
"logo": "https://avatars2.githubusercontent.com/u/15115011",
"repository": "https://github.com/PostgREST/postgrest",
"env": {
"BUILDPACK_URL": {
"description": "Heroku buildpack for deploying Haskell applications",
"value": "https://github.com/PostgREST/postgrest-heroku"
},
"POSTGREST_VER": {
"description": "Version of PostgREST to deploy",
"value": "5.2.0"
},
"DB_URI": {
"description": "Database connection string, e.g. postgres://user:pass@xxxxxxx.rds.amazonaws.com/mydb",
"required": true
},
"DB_SCHEMA": {
"description": "The database schema to expose to REST clients. Tables, views and stored procedures in this schema will get API endpoints",
"required": true,
"value": "public"
},
"DB_ANON_ROLE": {
"description": "The database role to use when executing commands on behalf of unauthenticated clients",
"required": true
},
"DB_POOL": {
"description": "Number of connections to keep open in PostgRESTs database pool",
"required": false,
"value": "10"
},
"SERVER_PROXY_URI": {
"description": "Overrides the base URL used within the OpenAPI self-documentation hosted at the API root path",
"required": false
},
"JWT_SECRET": {
"description": "The secret used to decode JWT tokens clients provide for authentication",
"required": false
},
"SECRET_IS_BASE64": {
"description": "When this is set to true, the value derived from jwt-secret will be treated as a base64 encoded secret",
"required": false,
"value": "false"
},
"JWT_AUD": {
"description": "The audience that should be validated if the JWT token contains an aud claim",
"required": false
},
"MAX_ROWS": {
"description": "A hard limit to the number of rows PostgREST will fetch from a view, table, or stored procedure",
"required": false
},
"PRE_REQUEST": {
"description": "A schema-qualified stored procedure name to call right after switching roles for a client request",
"required": false
}
}
}
-38
View File
@@ -1,38 +0,0 @@
platform: x64
cache:
- "c:\\sr"
- .stack-work
environment:
global:
STACK_ROOT: "c:\\sr"
GOPATH: c:\gopath
test: off
skip_non_tags: true
skip_branch_with_pr: true
branches:
only:
- master
install:
- set PATH=C:\Program Files\PostgreSQL\9.6\bin\;%PATH%
- curl -sS -ostack.zip -L --insecure http://www.stackage.org/stack/windows-x86_64
- 7z x stack.zip stack.exe
- set PATH=%GOPATH%\bin;c:\go\bin;%PATH%
- go get -u github.com/tcnksm/ghr
build_script:
- stack setup --no-terminal > nul
- stack build -j1 --copy-bins --local-bin-path .
artifacts:
- path: postgrest.exe
deploy_script:
- 7z a -tzip postgrest-%APPVEYOR_REPO_TAG_NAME%-windows-x64.zip postgrest.exe
- bash -lc "exec 0</dev/null && cd $APPVEYOR_BUILD_FOLDER && ghr -t $GITHUB_TOKEN -u $APPVEYOR_ACCOUNT_NAME -r $APPVEYOR_PROJECT_NAME -b \"$(sed -n \"1,/$(echo $APPVEYOR_REPO_TAG_NAME | cut -c2-)/d;/## \[/q;p\" CHANGELOG.md)\" --replace $APPVEYOR_REPO_TAG_NAME postgrest-$APPVEYOR_REPO_TAG_NAME-windows-x64.zip"
+159
View File
@@ -0,0 +1,159 @@
{ system ? builtins.currentSystem }:
let
name =
"postgrest";
compiler =
"ghc924";
# PostgREST source files, filtered based on the rules in the .gitignore files
# and file extensions. We want to include as litte as possible, as the files
# added here will increase the space used in the Nix store and trigger the
# build of new Nix derivations when changed.
src =
pkgs.lib.sourceFilesBySuffices
(pkgs.gitignoreSource ./.)
[ ".cabal" ".hs" ".lhs" "LICENSE" ];
# Commit of the Nixpkgs repository that we want to use.
nixpkgsVersion =
import nix/nixpkgs-version.nix;
# Nix files that describe the Nixpkgs repository. We evaluate the expression
# using `import` below.
nixpkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${nixpkgsVersion.rev}.tar.gz";
sha256 = nixpkgsVersion.tarballHash;
};
allOverlays =
import nix/overlays;
overlays =
[
allOverlays.build-toolbox
allOverlays.checked-shell-script
allOverlays.gitignore
allOverlays.postgresql-default
allOverlays.postgresql-legacy
allOverlays.postgresql-future
(allOverlays.haskell-packages { inherit compiler; })
];
# Evaluated expression of the Nixpkgs repository.
pkgs =
import nixpkgs { inherit overlays system; };
postgresqlVersions =
[
{ name = "postgresql-14"; postgresql = pkgs.postgresql_14.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-13"; postgresql = pkgs.postgresql_13.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-12"; postgresql = pkgs.postgresql_12.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-11"; postgresql = pkgs.postgresql_11.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-10"; postgresql = pkgs.postgresql_10.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
{ name = "postgresql-9.6"; postgresql = pkgs.postgresql_9_6.withPackages (p: [ p.postgis p.pg_safeupdate ]); }
];
patches =
pkgs.callPackage nix/patches { };
# Dynamic derivation for PostgREST
postgrest =
pkgs.haskell.packages."${compiler}".callCabal2nix name src { };
# Function that derives a fully static Haskell package based on
# nh2/static-haskell-nix
staticHaskellPackage =
import nix/static-haskell-package.nix { inherit nixpkgs system compiler patches allOverlays; };
# Options passed to cabal in dev tools and tests
devCabalOptions =
"-f dev --test-show-detail=direct";
profiledHaskellPackages =
pkgs.haskell.packages."${compiler}".extend (self: super:
{
mkDerivation =
args:
super.mkDerivation (args // { enableLibraryProfiling = true; });
}
);
inherit (pkgs.haskell) lib;
in
rec {
inherit nixpkgs pkgs;
# Derivation for the PostgREST Haskell package, including the executable,
# libraries and documentation. We disable running the test suite on Nix
# builds, as they require a database to be set up.
postgrestPackage =
lib.dontCheck postgrest;
# Static executable.
postgrestStatic =
lib.justStaticExecutables (lib.dontCheck (staticHaskellPackage name src));
# Profiled dynamic executable.
postgrestProfiled =
lib.enableExecutableProfiling (
lib.dontHaddock (
lib.dontCheck (profiledHaskellPackages.callCabal2nix name src { })
)
);
inherit (postgrest) env;
# Tooling for analyzing Haskell imports and exports.
hsie =
pkgs.callPackage nix/hsie {
inherit (pkgs.haskell.packages."${compiler}") ghcWithPackages;
};
### Tools
cabalTools =
pkgs.callPackage nix/tools/cabalTools.nix { inherit devCabalOptions postgrest; };
# Development tools.
devTools =
pkgs.callPackage nix/tools/devTools.nix { inherit tests style devCabalOptions hsie withTools; };
# Docker images and loading script.
docker =
pkgs.callPackage nix/tools/docker { postgrest = postgrestStatic; };
# Load testing tools.
loadtest =
pkgs.callPackage nix/tools/loadtest.nix { inherit withTools; };
# Script for running memory tests.
memory =
pkgs.callPackage nix/tools/memory.nix { inherit postgrestProfiled withTools; };
# Utility for updating the pinned version of Nixpkgs.
nixpkgsTools =
pkgs.callPackage nix/tools/nixpkgsTools.nix { };
# Scripts for publishing new releases.
release =
pkgs.callPackage nix/tools/release { };
# Linting and styling tools.
style =
pkgs.callPackage nix/tools/style.nix { inherit hsie; };
# Scripts for running tests.
tests =
pkgs.callPackage nix/tools/tests.nix {
inherit postgrest devCabalOptions withTools;
ghc = pkgs.haskell.compiler."${compiler}";
inherit (pkgs.haskell.packages."${compiler}") hpc-codecov;
inherit (pkgs.haskell.packages."${compiler}") weeder;
};
withTools =
pkgs.callPackage nix/tools/withTools.nix { inherit devCabalOptions postgresqlVersions postgrest; };
}
-45
View File
@@ -1,45 +0,0 @@
FROM debian:jessie
ARG POSTGREST_VERSION
# Install libpq5
RUN apt-get -qq update && \
apt-get -qq install -y --no-install-recommends libpq5 && \
apt-get -qq clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Install postgrest
RUN BUILD_DEPS="curl ca-certificates xz-utils" && \
apt-get -qq update && \
apt-get -qq install -y --no-install-recommends $BUILD_DEPS && \
cd /tmp && \
curl -SLO https://github.com/PostgREST/postgrest/releases/download/${POSTGREST_VERSION}/postgrest-${POSTGREST_VERSION}-ubuntu.tar.xz && \
tar -xJvf postgrest-${POSTGREST_VERSION}-ubuntu.tar.xz && \
mv postgrest /usr/local/bin/postgrest && \
cd / && \
apt-get -qq purge --auto-remove -y $BUILD_DEPS && \
apt-get -qq clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
COPY postgrest.conf /etc/postgrest.conf
ENV PGRST_DB_URI= \
PGRST_DB_SCHEMA=public \
PGRST_DB_ANON_ROLE= \
PGRST_DB_POOL=100 \
PGRST_SERVER_HOST=*4 \
PGRST_SERVER_PORT=3000 \
PGRST_SERVER_PROXY_URI= \
PGRST_JWT_SECRET= \
PGRST_SECRET_IS_BASE64=false \
PGRST_JWT_AUD= \
PGRST_MAX_ROWS= \
PGRST_PRE_REQUEST= \
PGRST_ROLE_CLAIM_KEY=".role"
# PostgREST reads /etc/postgrest.conf so map the configuration
# file in when you run this container
CMD exec postgrest /etc/postgrest.conf
EXPOSE 3000
-3
View File
@@ -1,3 +0,0 @@
db-uri = "postgres://app_user:password@postgres:5432/app_db"
db-schema = "public"
db-anon-role = "app_user"
-18
View File
@@ -1,18 +0,0 @@
FROM centos:centos6
RUN yum -y update
RUN yum -y install perl make automake gcc gmp-devel libffi zlib zlib-devel xz tar
RUN yum -y install https://download.postgresql.org/pub/repos/yum/9.3/redhat/rhel-6-x86_64/pgdg-centos93-9.3-2.noarch.rpm
RUN yum -y install postgresql93-devel
RUN yum clean all
RUN curl -sSL https://get.haskellstack.org/ | sh
ENV PATH $PATH:/usr/pgsql-9.3/bin
# To disable warning when building
ENV PATH $PATH:/root/.local/bin
RUN mkdir /source
WORKDIR /source
ENTRYPOINT ["stack"]
-18
View File
@@ -1,18 +0,0 @@
FROM centos:centos7
RUN yum -y update
RUN yum -y install perl make automake gcc gmp-devel libffi zlib zlib-devel xz tar
RUN yum -y install yum install https://download.postgresql.org/pub/repos/yum/9.3/redhat/rhel-7-x86_64/pgdg-centos93-9.3-2.noarch.rpm
RUN yum -y install postgresql93-devel
RUN yum clean all
RUN curl -sSL https://get.haskellstack.org/ | sh
ENV PATH $PATH:/usr/pgsql-9.3/bin
# To disable warning when building
ENV PATH $PATH:/root/.local/bin
RUN mkdir /source
WORKDIR /source
ENTRYPOINT ["stack"]
-18
View File
@@ -1,18 +0,0 @@
FROM ubuntu:16.04
RUN BUILD_DEPS="curl ca-certificates build-essential" && \
apt-get -qq update && \
apt-get -qqy --no-install-recommends install \
$BUILD_DEPS \
libpq-dev && \
curl -sSL https://get.haskellstack.org/ | sh && \
apt-get -qq clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# To disable warning when building
ENV PATH $PATH:/root/.local/bin
RUN mkdir /source
WORKDIR /source
ENTRYPOINT ["stack"]
@@ -1,18 +0,0 @@
FROM 32bit/ubuntu:16.04
RUN BUILD_DEPS="curl ca-certificates build-essential" && \
apt-get -qq update && \
apt-get -qqy --no-install-recommends install \
$BUILD_DEPS \
libpq-dev && \
curl -sSL https://get.haskellstack.org/ | sh && \
apt-get -qq clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# To disable warning when building
ENV PATH $PATH:/root/.local/bin
RUN mkdir /source
WORKDIR /source
ENTRYPOINT ["stack"]
-19
View File
@@ -1,19 +0,0 @@
stgrest:
image: pg_local
ports:
- "3000:3000"
links:
- postgres:postgres
environment:
PGRST_DB_URI: postgres://app_user:password@postgres:5432/app_db
PGRST_DB_SCHEMA: public
PGRST_DB_ANON_ROLE: app_user
postgres:
image: postgres
ports:
- "5432:5432"
environment:
POSTGRES_DB: app_db
POSTGRES_USER: app_user
POSTGRES_PASSWORD: password
-16
View File
@@ -1,16 +0,0 @@
db-uri = "$(PGRST_DB_URI)"
db-schema = "$(PGRST_DB_SCHEMA)"
db-anon-role = "$(PGRST_DB_ANON_ROLE)"
db-pool = "$(PGRST_DB_POOL)"
server-host = "$(PGRST_SERVER_HOST)"
server-port = "$(PGRST_SERVER_PORT)"
server-proxy-uri = "$(PGRST_SERVER_PROXY_URI)"
jwt-secret = "$(PGRST_JWT_SECRET)"
secret-is-base64 = "$(PGRST_SECRET_IS_BASE64)"
jwt-aud = "$(PGRST_JWT_AUD)"
role-claim-key = "$(PGRST_ROLE_CLAIM_KEY)"
max-rows = "$(PGRST_MAX_ROWS)"
pre-request = "$(PGRST_PRE_REQUEST)"
+26 -279
View File
@@ -1,295 +1,42 @@
{-# LANGUAGE CPP #-}
module Main where
module Main (main) where
import System.IO (BufferMode (..), hSetBuffering)
import PostgREST.App (postgrest)
import PostgREST.Config (AppConfig (..),
prettyVersion, readOptions)
import PostgREST.DbStructure (getDbStructure, getPgVersion)
import PostgREST.Error (encodeError)
import PostgREST.OpenAPI (isMalformedProxyUri)
import PostgREST.Types (DbStructure, Schema, PgVersion(..), minimumPgVersion)
import Protolude hiding (hPutStrLn, replace)
import qualified PostgREST.App as App
import qualified PostgREST.CLI as CLI
import Control.AutoUpdate (defaultUpdateSettings,
mkAutoUpdate, updateAction)
import Control.Retry (RetryStatus, capDelay,
exponentialBackoff,
retrying, rsPreviousDelay)
import qualified Data.ByteString as BS
import qualified Data.ByteString.Base64 as B64
import Data.IORef (IORef, atomicWriteIORef,
newIORef, readIORef)
import Data.String (IsString (..))
import Data.Text (pack, replace, stripPrefix, strip)
import Data.Text.Encoding (decodeUtf8, encodeUtf8)
import Data.Text.IO (hPutStrLn, readFile)
import Data.Time.Clock (getCurrentTime)
import qualified Hasql.Pool as P
import qualified Hasql.Session as H
import qualified Hasql.Transaction.Sessions as HT
import Network.Wai.Handler.Warp (defaultSettings,
runSettings, setHost,
setPort, setServerName)
import System.IO (BufferMode (..),
hSetBuffering)
import Protolude
#ifndef mingw32_HOST_OS
import System.Posix.Signals
import qualified PostgREST.Unix as Unix
#endif
{-|
The purpose of this worker is to fill the refDbStructure created in 'main'
with the 'DbStructure' returned from calling 'getDbStructure'. This method
is meant to be called by multiple times by the same thread, but does nothing if
the previous invocation has not terminated. In all cases this method does not
halt the calling thread, the work is preformed in a separate thread.
Note: 'atomicWriteIORef' is essentially a lazy semaphore that prevents two
threads from running 'connectionWorker' at the same time.
Background thread that does the following :
1. Tries to connect to pg server and will keep trying until success.
2. Checks if the pg version is supported and if it's not it kills the main
program.
3. Obtains the dbStructure.
4. If 2 or 3 fail to give their result it means the connection is down so it
goes back to 1, otherwise it finishes his work successfully.
-}
connectionWorker
:: ThreadId -- ^ This thread is killed if pg version is unsupported
-> P.Pool -- ^ The PostgreSQL connection pool
-> Schema -- ^ Schema PostgREST is serving up
-> IORef (Maybe DbStructure) -- ^ mutable reference to 'DbStructure'
-> IORef Bool -- ^ Used as a binary Semaphore
-> IO ()
connectionWorker mainTid pool schema refDbStructure refIsWorkerOn = do
isWorkerOn <- readIORef refIsWorkerOn
unless isWorkerOn $ do
atomicWriteIORef refIsWorkerOn True
void $ forkIO work
where
work = do
atomicWriteIORef refDbStructure Nothing
putStrLn ("Attempting to connect to the database..." :: Text)
connected <- connectingSucceeded pool
when connected $ do
result <- P.use pool $ do
actualPgVersion <- getPgVersion
unless (actualPgVersion >= minimumPgVersion) $ liftIO $ do
hPutStrLn stderr
("Cannot run in this PostgreSQL version, PostgREST needs at least "
<> pgvName minimumPgVersion)
killThread mainTid
dbStructure <- HT.transaction HT.ReadCommitted HT.Read $ getDbStructure schema actualPgVersion
liftIO $ atomicWriteIORef refDbStructure $ Just dbStructure
case result of
Left e -> do
putStrLn ("Failed to query the database. Retrying." :: Text)
hPutStrLn stderr (toS $ encodeError e)
work
Right _ -> do
atomicWriteIORef refIsWorkerOn False
putStrLn ("Connection successful" :: Text)
{-|
Used by 'connectionWorker' to check if the provided db-uri lets
the application access the PostgreSQL database. This method is used
the first time the connection is tested, but only to test before
calling 'getDbStructure' inside the 'connectionWorker' method.
The connection tries are capped, but if the connection times out no error is
thrown, just 'False' is returned.
-}
connectingSucceeded :: P.Pool -> IO Bool
connectingSucceeded pool =
retrying (capDelay 32000000 $ exponentialBackoff 1000000)
shouldRetry
(const $ P.release pool >> isConnectionSuccessful)
where
isConnectionSuccessful :: IO Bool
isConnectionSuccessful = do
testConn <- P.use pool $ H.sql "SELECT 1"
case testConn of
Left e -> hPutStrLn stderr (toS $ encodeError e) >> pure False
_ -> pure True
shouldRetry :: RetryStatus -> Bool -> IO Bool
shouldRetry rs isConnSucc = do
delay <- pure $ fromMaybe 0 (rsPreviousDelay rs) `div` 1000000
itShould <- pure $ not isConnSucc
when itShould $
putStrLn $ "Attempting to reconnect to the database in " <> (show delay::Text) <> " seconds..."
return itShould
{-|
This is where everything starts.
-}
main :: IO ()
main = do
--
-- LineBuffering: the entire output buffer is flushed whenever a newline is
-- output, the buffer overflows, a hFlush is issued or the handle is closed
--
-- NoBuffering: output is written immediately and never stored in the buffer
hSetBuffering stdout LineBuffering
hSetBuffering stdin LineBuffering
hSetBuffering stderr NoBuffering
--
-- readOptions builds the 'AppConfig' from the config file specified on the
-- command line
conf <- loadDbUriFile =<< loadSecretFile =<< readOptions
let host = configHost conf
port = configPort conf
proxy = configProxyUri conf
pgSettings = toS (configDatabase conf) -- is the db-uri
roleClaimKey = configRoleClaimKey conf
appSettings =
setHost ((fromString . toS) host) -- Warp settings
. setPort port
. setServerName (toS $ "postgrest/" <> prettyVersion) $
defaultSettings
setBuffering
opts <- CLI.readCLIShowHelp
CLI.main installSignalHandlers runAppInSocket opts
-- Checks that the provided proxy uri is formated correctly
when (isMalformedProxyUri $ toS <$> proxy) $
panic
"Malformed proxy uri, a correct example: https://example.com:8443/basePath"
-- Checks that the provided jspath is valid
when (isLeft roleClaimKey) $
panic $ show roleClaimKey
putStrLn $ ("Listening on port " :: Text) <> show (configPort conf)
--
-- create connection pool with the provided settings, returns either
-- a 'Connection' or a 'ConnectionError'. Does not throw.
pool <- P.acquire (configPool conf, 10, pgSettings)
--
-- To be filled in by connectionWorker
refDbStructure <- newIORef Nothing
--
-- Helper ref to make sure just one connectionWorker can run at a time
refIsWorkerOn <- newIORef False
--
-- This is passed to the connectionWorker method so it can kill the main
-- thread if the PostgreSQL's version is not supported.
mainTid <- myThreadId
--
-- Sets the refDbStructure
connectionWorker
mainTid
pool
(configSchema conf)
refDbStructure
refIsWorkerOn
--
-- Only for systems with signals:
--
-- releases the connection pool whenever the program is terminated,
-- see issue #268
--
-- Plus the SIGHUP signal updates the internal 'DbStructure' by running
-- 'connectionWorker' exactly as before.
installSignalHandlers :: App.SignalHandlerInstaller
#ifndef mingw32_HOST_OS
forM_ [sigINT, sigTERM] $ \sig ->
void $ installHandler sig (Catch $ do
P.release pool
throwTo mainTid UserInterrupt
) Nothing
forM_ [sigHUP, sigUSR1] $ \sig ->
void $ installHandler sig (
Catch $ connectionWorker
mainTid
pool
(configSchema conf)
refDbStructure
refIsWorkerOn
) Nothing
installSignalHandlers = Unix.installSignalHandlers
#else
installSignalHandlers _ = pass
#endif
runAppInSocket :: Maybe App.SocketRunner
#ifndef mingw32_HOST_OS
runAppInSocket = Just Unix.runAppWithSocket
#else
runAppInSocket = Nothing
#endif
-- ask for the OS time at most once per second
getTime <- mkAutoUpdate defaultUpdateSettings {updateAction = getCurrentTime}
-- run the postgrest application
runSettings appSettings $
postgrest
conf
refDbStructure
pool
getTime
(connectionWorker
mainTid
pool
(configSchema conf)
refDbStructure
refIsWorkerOn)
{-|
The purpose of this function is to load the JWT secret from a file if
configJwtSecret is actually a filepath and replaces some characters if the JWT
is base64 encoded.
The reason some characters need to be replaced is because JWT is actually
base64url encoded which must be turned into just base64 before decoding.
To check if the JWT secret is provided is in fact a file path, it must be
decoded as 'Text' to be processed.
decodeUtf8: Decode a ByteString containing UTF-8 encoded text that is known to
be valid.
-}
loadSecretFile :: AppConfig -> IO AppConfig
loadSecretFile conf = extractAndTransform mSecret
where
mSecret = decodeUtf8 <$> configJwtSecret conf
isB64 = configJwtSecretIsBase64 conf
--
-- The Text (variable name secret) here is mSecret from above which is the JWT
-- decoded as Utf8
--
-- stripPrefix: Return the suffix of the second string if its prefix matches
-- the entire first string.
--
-- The configJwtSecret is a filepath instead of the JWT secret itself if the
-- secret has @ as its prefix.
extractAndTransform :: Maybe Text -> IO AppConfig
extractAndTransform Nothing = return conf
extractAndTransform (Just secret) =
fmap setSecret $
transformString isB64 =<<
case stripPrefix "@" secret of
Nothing -> return . encodeUtf8 $ secret
Just filename -> chomp <$> BS.readFile (toS filename)
where
chomp bs = fromMaybe bs (BS.stripSuffix "\n" bs)
--
-- Turns the Base64url encoded JWT into Base64
transformString :: Bool -> ByteString -> IO ByteString
transformString False t = return t
transformString True t =
case B64.decode $ encodeUtf8 $ strip $ replaceUrlChars $ decodeUtf8 t of
Left errMsg -> panic $ pack errMsg
Right bs -> return bs
setSecret bs = conf {configJwtSecret = Just bs}
--
-- replace: Replace every occurrence of one substring with another
replaceUrlChars =
replace "_" "/" . replace "-" "+" . replace "." "="
{-
Load database uri from a separate file if `db-uri` is a filepath.
-}
loadDbUriFile :: AppConfig -> IO AppConfig
loadDbUriFile conf = extractDbUri mDbUri
where
mDbUri = configDatabase conf
extractDbUri :: Text -> IO AppConfig
extractDbUri dbUri =
fmap setDbUri $
case stripPrefix "@" dbUri of
Nothing -> return dbUri
Just filename -> strip <$> readFile (toS filename)
setDbUri dbUri = conf {configDatabase = dbUri}
setBuffering :: IO ()
setBuffering = do
-- LineBuffering: the entire output buffer is flushed whenever a newline is
-- output, the buffer overflows, a hFlush is issued or the handle is closed
hSetBuffering stdout LineBuffering
hSetBuffering stdin LineBuffering
hSetBuffering stderr LineBuffering
+22
View File
@@ -0,0 +1,22 @@
# This Dockerfile is only used as a development environment for
# non-nix systems, i.e. Windows.
FROM nixos/nix:latest
RUN apk --no-cache add \
wget
RUN nix-env -iA cachix -f https://cachix.org/api/v1/install \
&& cachix use postgrest
# We need an unprivileged user here, to make PG run at all.
RUN adduser --disabled-password --ingroup root nix \
&& chown -R nix:root /nix
USER nix:root
ENV USER=nix
VOLUME /nix
VOLUME /postgrest
WORKDIR /postgrest
CMD nix-shell
+308
View File
@@ -0,0 +1,308 @@
# Nix development and build environment
With Nix it's possible to quickly and reliably recreate the full environments
for developing, testing and building PostgREST.
## Getting started with Nix
You'll need to [get Nix](https://nixos.org/download.html). The installer will
create your Nix store in the `/nix/` directory, where all build artifacts and
their dependencies will be stored. It will also link the Nix executables like
`nix-env`, `nix-build` and `nix-shell` into your PATH. Nix will manage all
other PostgREST dependencies from here on out. To clean up older build
artifacts from the `/nix/store`, you can run `nix-collect-garbage`.
If you are on a system that does not support nix, for example Windows, you can
run the nix development environment in a docker container. Inside the `nix/`
directory run `docker-compose run --rm nix` to start the docker container. This
will set up the binary cache and launch `nix-shell` automatically.
## Building PostgREST
To build PostgREST from your local checkout of the repository, run:
```bash
nix-build --attr postgrestPackage
```
This will create a `result` directory that contains the PostgREST binary at
`result/bin/postgrest`. The `--attr` parameter (or short: `-A`) tells Nix to
build the `postgrestPackage` attribute from the Nix expression it finds in our
`default.nix` (see below for details). Nix will take care of getting the right
GHC version and all the build dependencies.
## Binary cache
We recommend that you use the PostgREST binary cache on
[cachix](https://cachix.org/):
```bash
# Install cachix:
nix-env -iA cachix -f https://cachix.org/api/v1/install
# Set cachix up to use the PostgREST binary cache:
cachix use postgrest
```
Without cachix, your machine will have to rebuild all the dependencies that are
derived on top of `Musl` for the static builds, which can take a very long time.
## Developing
A development environment for PostgREST is available with `nix-shell`. The
following command will put you into a new shell that has GHC and Cabal on the
PATH:
```bash
nix-shell
```
Within `nix-shell`, you can run Cabal commands as usual. You can also run
stack with the `--nix` option, which causes stack to pick up the non-Haskell
dependencies from the same pinned Nixpkgs version that the Nix builds use.
## Working with `nix-shell` and the PostgREST utility scripts
The PostgREST utilities available in `nix-shell` all have names that begin with
`postgrest-`, so you can use tab completion (typing `postgrest-` and pressing
`<tab>`) in `nix-shell` to see all that are available:
```bash
# Note: The utilities listed here might not be up to date.
[nix-shell]$ postgrest-<tab>
postgrest-build postgrest-test-spec
postgrest-check postgrest-watch
postgrest-clean postgrest-with-all
postgrest-coverage postgrest-with-postgresql-10
postgrest-lint postgrest-with-postgresql-11
postgrest-run postgrest-with-postgresql-12
postgrest-style postgrest-with-postgresql-13
postgrest-style-check postgrest-with-postgresql-9.6
postgrest-test-io
...
[nix-shell]$
```
Some additional modules like `memory`, `docker` and `release`
have large dependencies that would need to be built before the shell becomes
available, which could take an especially long time if the cachix binary cache
is not used. You can activate those by passing a flag to `nix-shell` with
`nix-shell --arg <module> true`. This will make the respective utilites available:
```bash
$ nix-shell --arg memory true
[nix-shell]$ postgrest-<tab>
postgrest-build postgrest-test-spec
postgrest-check postgrest-watch
postgrest-clean postgrest-with-all
postgrest-coverage postgrest-with-postgresql-10
postgrest-lint postgrest-with-postgresql-11
postgrest-run postgrest-with-postgresql-12
postgrest-style postgrest-with-postgresql-13
postgrest-style-check postgrest-with-postgresql-9.6
postgrest-test-io
postgrest-test-memory
...
```
Note that `postgrest-test-memory` is now also available.
To run one-off commands, you can also use `nix-shell --run <command>`, which
will lauch the Nix shell, run that one command and exit. Note that the tab
completion will not work with `nix-shell --run`, as Nix has yet to evaluate
our Nix expressions to see which utilities are available.
```bash
$ nix-shell --run postgrest-style
# Note that you need to quote any arguments that you would like to pass to
# the command to be run in nix-shell:
$ nix-shell --run "postgrest-foo --bar"
```
A third option is to install utilities that you use very often locally:
```bash
$ nix-env -f default.nix -iA devTools
# `postgrest-style` can now be run directly:
$ postgrest-style
```
If you use `nix-shell` very often, you might like to use
https://github.com/xzfc/cached-nix-shell, which skips evaluating all our Nix
expressions if nothing changed, reducing startup time for the shell
considerably.
Note: Once inside nix-shell, the utilities work from any directory inside
the PostgREST repo. Paths are resolved relative to the repo root:
```bash
$ cd src
# Even though the current directory is ./src, the config path must still start
# from the repo root:
$ postgrest-run test/io/configs/simple.conf
```
## Testing
In nix-shell, you'll find utility scripts that make it very easy to run our
test suite, including setting up all required dependencies and
temporary test databases:
```bash
# Run the tests against the most recent version of PostgreSQL:
$ nix-shell --run postgrest-test-spec
# Run the tests against all supported versions of PostgreSQL:
$ nix-shell --run "postgrest-with-all postgrest-test-spec"
# Run the tests against a specific version of PostgreSQL (use tab-completion in
# nix-shell to see all available versions):
$ nix-shell --run "postgrest-with-postgresql-13 postgrest-test-spec"
```
The io-test that test PostgREST as a black box with inputs and outputs can be
run with `postgrest-test-io`. The test runner under the hood is
[pytest](https://docs.pytest.org/) and you can pass it the usual options:
```bash
# Filter the tests to run by name, including all that contain 'config':
postgrest-test-io -k config
# Run tests in parallel using xdist, specifying the number of processes:
postgrest-test-io -n auto
postgrest-test-io -n 8
```
The memory tests check that we don't surpass a memory threshold for big request bodies.
```bash
# Build the dependencies needed for the memory test
nix-shell --arg memory true
# Run the memory test
postgrest-test-memory
```
The loadtests ensure that performance doesn't drop on a change. Underlyingly they use
[vegeta](https://github.com/tsenart/vegeta).
```bash
# Run the loadtests on the latest commit(HEAD)
postgrest-loadtest
# You can loadtest comparing to a different branch
postgrest-loadtest-against master
# Produce a markdown report to be used on CI
postgrest-loadtest-report
```
Our query cost tests ensure that our generated queries don't surpass a threshold EXPLAIN cost.
```bash
postgrest-test-querycost
```
doctests for some of our modules are also available:
```bash
postgrest-test-doctest
```
## Code coverage
Code coverage is available under the `postgrest-coverage` command. This will produce a `./coverage` directory that can be visualized with a simple http server.
```bash
# Will run all the tests and produce a coverage dir
postgrest-coverage
# Visualize the output
cd coverage
python -mSimpleHTTPServer 8080
```
## Linting and styling code
The nix-shell also contains scripts for linting and styling the PostgREST
source code:
```bash
# Linting
$ nix-shell --run postgrest-lint
# Styling / auto-formatting code
$ nix-shell --run postgrest-style
```
There is also `postgrest-style-check` that exits with a non-zero exit code if
the check resulted in any uncommited changes. It's mostly useful for CI.
## General development tools
Tools like `postgrest-build`, `postgrest-run` etc. are simple wrappers around
`cabal` and should do what you expect. `postgrest-check` runs most checks that will
also run in CI, with the exception of the IO and Memory checks that need to be run
separately.
`postgrest-with-postgresql-*` take a command as an argument and will run it
with a temporary database. `postgrest-with-all` will run the command against
all supported PostgreSQL versions. Tests run without `postgrest-with-*` are
run against the latest PostgreSQL version by default.
`postgrest-watch` takes a command as an argument that it will re-run if any source
file is changed. For example, `postgrest-watch postgrest-with-all postgrest-test-spec`
will re-run the full spec test suite against all PostgreSQL versions on every change.
## Tour
The following is not required for working on PostgREST with Nix, but it will
give you some more background and details on how it works.
### `default.nix`
[`default.nix`](../default.nix) is our 'repository expression' that pulls all
the pieces that we define with Nix together. It returns a set (like a dict in
other programming languages), where each attribute is a derivation that Nix
knows how to build, like the `postgrest` attribute from earlier.
Internally, our `default.nix` uses the `pkgs.callPackage` function to import
the modules that we defined in the `nix` directory. It automatically passes the
arguments those modules require if they are available in `pkgs` (this means
that `pkgs` is defined in terms of itself, better not to think too much about
that).
We also use `default.nix` to load our pinned version of the `nixpkgs`
repository. This set of packages will always be the same, independently from
where or when you use it. The pinned version can be upgraded with the small
`nixpkgs-upgrade` utility. Running `nixpkgs-upgrade > nix/nixpkgs-version.nix`
in `nix-shell` will upgrade the pinned version to the latest `nixpkgs-unstable`
version.
### `shell.nix`
[`shell.nix`](../shell.nix) defines an environment in which PostgREST can be
built and developed. It extends the build enviroment from our `postgrest`
attribute with useful utilities that will be put on the PATH in `nix-shell`.
### `nix/overlays`
Our overlays to the Nix package set are defined here. They allow us to tweak our
`pkgs` in `default.nix` by adding new packages or overriding existing ones.
## Upgrading dependencies
See the [upgrading checklist](UPGRADE.md) for how to upgrade the PostgREST
dependencies.
+91
View File
@@ -0,0 +1,91 @@
# Checklist for upgrading Nix dependencies
The Nix dependencies of PostgREST should be updated regularly, in most cases it
should be a very simple operation.
```bash
# Update pinned version of Nixpkgs
nix-shell --run postgrest-nixpkgs-upgrade
# Verify that everything builds
nix-build
```
The following checklist guides you through the complete process in more detail.
## Upgrade the pinned version of `nixpkgs`
The pinned version of [`nixpkgs`](https://github.com/NixOS/nixpkgs) is defined
in [`nix/nixpkgs-version.nix`](nixpkgs-version.nix). The pin refers directly to
a GitHub tarball for the given revision, which is more efficient than pulling
the complete Git repository. To upgrade it to the current `main` of
`nixpkgs`, you can use a small utility script defined in
[`nix/nixpkgs-update.nix`](nixpkgs-update.nix):
```bash
# From the root of the repository, enter nix-shell
nix-shell
# Run the utility script to pin the latest revision in main
postgrest-nixpkgs-upgrade
# Exit the nix-shell with Ctrl-d
```
## Update pinned version of `static-haskell-nix`
We pin [`static-haskell-nix`](https://github.com/nh2/static-haskell-nix) in
[`nix/static-haskell-package.nix`](static-haskell-package.nix). Upgrade the
pinned revision and the tarball hash if necessary. See
[`nix/nixpkgs-upgrade.nix`](nixpkgs-upgrade.nix) for how to get the correct
tarball hash, or just change the hash to an arbitrary value of correct length,
run `nix-build` and use the expected value from the resulting error message.
## Review overlays
Check whether the individual [overlays](overlays) are still required.
## Check if patches are still required and update them as needed
We track a number of PostgREST-specific patches in [`nix/patches`](patches).
Check whether the pull-requests/issues linked in the
[`default.nix`](patches/default.nix) have progressed and remove/modify the
patches if they did. If conflicting changes occurred, you might have to rebase
the respective patches.
## Build everything
Using the PostgREST binary Nix cache is recommended. Install
[Cachix](https://cachix.org/) and run `cachix use postgrest`.
Run `nix-build` in the root directory of the project to build all PostgREST
artifacts. This might take a long time, e.g. when our static GHC version needs
to be rebuilt due to changes to some underlying package. If there are any
errors, this is probably due to one of our patches. Try to fix them and re-run
`nix-build` until everything builds.
## Update the PostgREST binary cache
If you have access to the PostgREST cachix signing key, you can push the
artifacts that you built locally to the binary cache. This will accelerate the
CI builds and tests, sometimes dramatically. This might sometimes even be
required to avoid build timeouts in CI.
You'll need to set the `CACHIX_SIGNING_KEY` before proceeding, e.g. by creating
a file containing `export CACHIX_SIGNING_KEY=...` and sourcing that file, which
avoids having the secret in you shell history.
To push all new artifacts to Cachix, run:
```
nix-store -qR --include-outputs $$(nix-instantiate) | cachix push postgrest
# Or, equivalently
nix-shell --run postgrest-push-cachix
```
The `nix-store` command will query the nix-store to list all dependencies and
build artifacts of PostgREST. The `cachix` command will efficiently push
everything that is not yet cached to the binary cache.
+12
View File
@@ -0,0 +1,12 @@
version: '3'
services:
nix:
container_name: postgrest-nix
build: .
volumes:
- ../:/postgrest
- nix:/nix
volumes:
nix:
+361
View File
@@ -0,0 +1,361 @@
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE TupleSections #-}
{-# LANGUAGE TypeFamilies #-}
-- | Haskell Imports and Exports tool
--
-- This tool parses imports and exports from Haskell source files and provides
-- analysis on these imports. For example, you can check whether consistent
-- import aliases are used across your codebase.
module Main (main) where
import qualified Data.Aeson as JSON
import qualified Data.ByteString.Lazy.Char8 as LBS8
import qualified Data.Csv as Csv
import qualified Data.List as List
import qualified Data.Map as Map
import qualified Data.Set as Set
import qualified Data.Text as T
import qualified Data.Text.IO as T
import qualified Dot
import qualified GHC
import qualified GHC.Paths
import qualified Language.Haskell.GHC.ExactPrint.Parsers as ExactPrint
import qualified Options.Applicative as O
import qualified System.FilePath as FP
import Data.Aeson.Encode.Pretty (encodePretty)
import Data.Function ((&))
import Data.List (intercalate)
import Data.Maybe (catMaybes, mapMaybe)
import Data.Text (Text)
import GHC.Data.Bag (bagToList)
import GHC.Generics (Generic)
import GHC.Hs.Extension (GhcPs)
import GHC.Types.Name.Occurrence (occNameString)
import GHC.Types.Name.Reader (rdrNameOcc)
import GHC.Unit.Module.Name (moduleNameString)
import System.Directory.Recursive (getFilesRecursive)
import System.Exit (exitFailure)
-- TYPES
data Options =
Options
{ command :: Command
, sources :: [FilePath]
}
data Command
= Dump OutputFormat
| GraphSymbols
| GraphModules
| CheckAliases
| CheckWildcards [Text]
data OutputFormat = OutputCsv | OutputJson
data ImportedSymbol =
ImportedSymbol
{ impFromModule :: Text
, impModule :: Text
, impQualified :: ImportQualified
, impAlias :: Maybe Text
, impType :: ImportType
, impSymbol :: Maybe Text
, impInternal :: ModuleInternal
, impSource :: FilePath
, impFile :: FilePath
}
deriving (Generic, Csv.ToNamedRecord, Csv.DefaultOrdered, JSON.ToJSON)
data ImportQualified
= Qualified
| NotQualified
deriving (Eq, Generic, JSON.ToJSON)
instance Csv.ToField ImportQualified where
toField Qualified = "qualified"
toField NotQualified = "not qualified"
data ModuleInternal
= Internal
| External
deriving (Eq, Generic, JSON.ToJSON)
instance Csv.ToField ModuleInternal where
toField Internal = "internal"
toField External = "external"
data ImportType
= Wildcard
| Hiding
| Explicit
deriving (Eq, Generic, JSON.ToJSON)
instance Csv.ToField ImportType where
toField Wildcard = "wildcard"
toField Hiding = "hiding"
toField Explicit = "explicit"
-- | Mapping of modules to their aliases and to the files they are found in
type ModuleAliases = [(Text, [(Text, [FilePath])])]
-- | Mapping of modules to files
type WildcardImports = [(FilePath, [Text])]
-- MAIN
main :: IO ()
main =
run =<< O.customExecParser prefs infoOpts
where
prefs = O.prefs $ O.subparserInline <> O.showHelpOnEmpty
infoOpts =
O.info (O.helper <*> opts) $
O.fullDesc
<> O.header "hsie - Swiss army knife for HaSkell Imports and Exports"
<> O.progDesc "Parse Haskell code to analyze imports and exports"
opts =
Options <$> commandOption <*> O.some srcOption
srcOption =
O.argument O.str $
O.metavar "SRCDIR"
<> O.help "Haskell source directory"
<> O.action "directory"
commandOption =
O.subparser $
command "dump-imports" "Dump imported symbols as CSV or JSON"
(Dump <$> jsonOutputFlag)
<> command "graph-modules" "Print dot graph of module imports"
(pure GraphModules)
<> command "graph-symbols" "Print dot graph of symbol imports"
(pure GraphSymbols)
<> command "check-aliases"
"Check that aliases of imported modules are consistent"
(pure CheckAliases)
<> command "check-wildcards"
"Check that no modules are imported as unqualified wildcards"
(CheckWildcards <$> O.many okModuleOption)
command name desc options =
O.command name . O.info (O.helper <*> options) $ O.progDesc desc
jsonOutputFlag =
O.flag OutputCsv OutputJson $
O.long "json" <> O.short 'j' <> O.help "Output JSON"
okModuleOption =
O.strOption $
O.long "ok"
<> O.short 'o'
<> O.metavar "OKMODULE"
<> O.help "Module that is ok to import as unqualified wildcard"
run :: Options -> IO ()
run Options{command, sources} =
runCommand command . markInternal . concat =<< mapM sourceSymbols sources
where
runCommand :: Command -> [ImportedSymbol] -> IO ()
runCommand (Dump format) = LBS8.putStr . dump format
runCommand GraphSymbols = T.putStr . symbolsGraph
runCommand GraphModules = T.putStr . Dot.encode . modulesGraph
runCommand CheckAliases = runInconsistentAliases . inconsistentAliases
runCommand (CheckWildcards okModules) = runWildcards . wildcards okModules
runInconsistentAliases :: ModuleAliases -> IO ()
runInconsistentAliases [] = T.putStrLn "No inconsistent module aliases found."
runInconsistentAliases xs = T.putStr (formatInconsistentAliases xs) >> exitFailure
runWildcards :: WildcardImports -> IO ()
runWildcards [] = T.putStrLn "No unwanted wildcard imports found."
runWildcards xs = T.putStr (formatWildcards xs) >> exitFailure
-- | Mark imports from modules that are among the analyzed ones as internal.
markInternal :: [ImportedSymbol] -> [ImportedSymbol]
markInternal symbols =
fmap mark symbols
where
mark s = s { impInternal = if isInternal s then Internal else External }
isInternal = flip Set.member internalModules . impModule
internalModules = Set.fromList $ fmap impFromModule symbols
-- SYMBOLS
-- | Parse all imported symbols from a source of Haskell source files
sourceSymbols :: FilePath -> IO [ImportedSymbol]
sourceSymbols source = do
files <- filterExts [".hs", ".imports"] <$> getFilesRecursive source
concat <$> mapM moduleSymbols files
where
filterExts exts = filter $ flip elem exts . FP.takeExtension
moduleSymbols filepath = do
GHC.HsModule{..} <- parseModule filepath
return $ concatMap (importSymbols source filepath . GHC.unLoc) hsmodImports
-- | Parse a Haskell module
parseModule :: FilePath -> IO GHC.HsModule
parseModule filepath = do
result <- ExactPrint.parseModule GHC.Paths.libdir filepath
case result of
Right hsmod ->
return $ GHC.unLoc hsmod
Left errs ->
fail $ "Errors with " <> show filepath <> ":\n "
<> List.intercalate "\n " (show <$> bagToList errs)
-- | Symbols imported in an import declaration.
--
-- If the import is a wildcard, i.e. no symbols are selected for import, then
-- only one item is returned.
importSymbols :: FilePath -> FilePath -> GHC.ImportDecl GhcPs -> [ImportedSymbol]
importSymbols source filepath GHC.ImportDecl{..} =
case ideclHiding of
Just (hiding, syms) ->
symbol (if hiding then Hiding else Explicit) . Just . GHC.unLoc <$> GHC.unLoc syms
Nothing ->
[ symbol Wildcard Nothing ]
where
symbol hiding sym =
ImportedSymbol
{ impFile = relativePath filepath
, impSource = source
, impFromModule = T.pack $ moduleFromPath filepath
, impModule = T.pack . moduleNameString . GHC.unLoc $ ideclName
, impQualified = if ideclQualified /= GHC.NotQualified then Qualified else NotQualified
, impAlias = T.pack . moduleNameString . GHC.unLoc <$> ideclAs
, impInternal = External
, impType = hiding
, impSymbol = T.pack . occNameString . rdrNameOcc . GHC.ieName <$> sym
}
moduleFromPath =
intercalate "." . FP.splitDirectories . FP.dropExtension . relativePath
relativePath = FP.makeRelative source
-- DUMP
-- | Dump list of symbols as CSV or JSON
dump :: OutputFormat -> [ImportedSymbol] -> LBS8.ByteString
dump OutputCsv = Csv.encodeDefaultOrderedByName
dump OutputJson = encodePretty
-- ALIASES
-- | Find modules that are imported under different aliases
inconsistentAliases :: [ImportedSymbol] -> ModuleAliases
inconsistentAliases symbols =
fmap moduleAlias symbols
& foldr insertSetMapMap Map.empty
& Map.map (aliases . Map.toList)
& Map.filter ((<) 1 . length)
& Map.toList
where
moduleAlias ImportedSymbol{..} =
(impModule, impAlias, FP.joinPath [impSource, impFile])
insertSetMapMap (k1, k2, v) =
Map.insertWith (Map.unionWith Set.union) k1
(Map.singleton k2 $ Set.singleton v)
aliases :: [(Maybe Text, Set.Set FilePath)] -> [(Text, [FilePath])]
aliases = mapMaybe (\(k, v) -> fmap (, Set.toList v) k)
formatInconsistentAliases :: ModuleAliases -> Text
formatInconsistentAliases modules =
"The following imports have inconsistent aliases:\n\n"
<> T.concat (fmap formatModule modules)
where
formatModule (modName, aliases) =
"Module '"
<> modName
<> "' has the aliases:\n"
<> T.concat (fmap formatAlias aliases)
<> "\n"
formatAlias (alias, sourceFiles) =
" '"
<> alias
<> "' in file"
<> (if length sourceFiles > 2 then "s" else "")
<> ":\n"
<> T.concat (fmap formatFile sourceFiles)
formatFile sourceFile =
" " <> T.pack sourceFile <> "\n"
-- WILDCARDS
-- | Find modules that are imported as wildcards, excluding whitelisted modules.
--
-- Wildcard imports are ones that are not qualified and do not specify which
-- symbols should be imported.
wildcards :: [Text] -> [ImportedSymbol] -> WildcardImports
wildcards okModules =
groupByFile . filter isWildcard . filter (not . isOkModule)
where
isWildcard ImportedSymbol{..} =
impQualified == NotQualified && impType /= Explicit
isOkModule = flip Set.member (Set.fromList okModules) . impModule
groupByFile = Map.toList . fmap Set.toList . foldr insertMap Map.empty
insertMap ImportedSymbol{..} =
Map.insertWith Set.union impFile (Set.singleton impModule)
formatWildcards :: WildcardImports -> Text
formatWildcards files =
"Modules in the following files were imported as wildcards:\n\n"
<> T.concat (fmap formatFile files)
where
formatFile (filepath, modules) =
"In " <> T.pack filepath <> ":\n" <> T.concat (fmap formatModule modules) <> "\n"
formatModule moduleName = " " <> moduleName <> "\n"
-- GRAPHS
modulesGraph :: [ImportedSymbol] -> Dot.DotGraph
modulesGraph symbols =
Dot.DotGraph Dot.Strict Dot.Directed (Just "Modules") $ fmap edge edges
where
edge (from, to) =
Dot.StatementEdge $ Dot.EdgeStatement
(Dot.ListTwo (edgeNode from) (edgeNode to) mempty) mempty
edgeNode t = Dot.EdgeNode $ Dot.NodeId (Dot.Id t) Nothing
edges = unique . fmap edgeTuple . filter ((==) Internal . impInternal) $ symbols
edgeTuple ImportedSymbol{..} = (impFromModule, impModule)
unique = Set.toList . Set.fromList
-- Building Text directly as the Dot package currently doesn't support subgraphs.
symbolsGraph :: [ImportedSymbol] -> Text
symbolsGraph symbols =
"digraph Symbols {\n"
<> " rankdir=LR\n"
<> " ranksep=5\n"
<> T.concat (fmap edge edges)
<> T.concat (fmap cluster symbolsByModule)
<> "}\n"
where
edge (from, to, symbol) =
" "
<> quoted from
<> " -> "
<> quoted (to <> maybe "" ("." <>) symbol)
<> "\n"
cluster (moduleName, clusterSymbols) =
" subgraph "
<> quoted ("cluster_" <> moduleName)
<> " {\n"
<> " " <> quoted moduleName <> "\n"
<> T.concat (fmap (clusterNode moduleName) clusterSymbols)
<> " }\n"
clusterNode moduleName symbol =
" " <> quoted (moduleName <> "." <> symbol) <> "\n"
quoted t = "\"" <> t <> "\""
edges = unique . fmap edgeTuple . filter ((==) Internal . impInternal) $ symbols
edgeTuple ImportedSymbol{..} = (impFromModule, impModule, impSymbol)
unique = Set.toList . Set.fromList
symbolsByModule =
Map.toList . Map.map (catMaybes . Set.toList) . foldr insertMap Map.empty $ edges
insertMap (_, to, symbol) = Map.insertWith Set.union to $ Set.singleton symbol
+67
View File
@@ -0,0 +1,67 @@
# hsie - Swiss army knife for HaSkell Imports and Exports
This tool parses Haskell source code to analyse the imports and exports in a
project. It's available in PostgREST's `nix-shell` by default.
## Dumping imports
Given source code in the directories `src` and `main`, for example, you can run:
```
hsie dump-imports src main
```
This dumps all imports of the modules in the given directory to a CSV file,
printed on `stdout`.
To dump to a JSON file (e.g., to further process with `jq`), add the `--json`
flag:
```
hsie dump-imports --json src main
```
## Graphing imports
The tool can generate `graphviz` graphs of module and symbol imports by printing
a file to `stdout` that can directly be rendered with `dot`:
```
hsie graph-modules src main | dot -Tpng -o modules.png
```
The command `graph-modules` prints a graph of which modules insert which other
modules. `graph-symbols` shows which symbols are imported from which modules.
## Checking imports
To check whether modules are imported under consistent aliases in your project,
run:
```
hsie check-aliases main src
```
This will exit with a non-zero exit code if any inconsistent aliases are found.
The following command checks whether any modules are imported as wildcards, i.e.
not qualified and without specifying symbols.
```
hsie check-wildcards main src
```
To whitelist certain modules to be imported as wildcards, use `--ok`:
```
hsie check-wildcards main src --ok Protolude --ok Test.Module
```
## Current limitations
This tool uses the GHC parser to parse Haskell source code. Language extensions
required to parse each file are detected based on the `{-# LANGUAGE ... #-}`
pragmas. If they are not available (e.g., as they are listed as default
extensions in the `.cabal` file), parses may fail. We can fix this by using
an extended set of non-conflicting extensions by default, as `hlint` does for
example.
+35
View File
@@ -0,0 +1,35 @@
{ ghcWithPackages
, runCommand
}:
let
name = "hsie";
src = ./Main.hs;
modules = ps: [
ps.aeson
ps.aeson-pretty
ps.cassava
ps.dir-traverse
ps.dot
ps.ghc-exactprint
ps.optparse-applicative
];
ghc = ghcWithPackages modules;
hsie =
runCommand "haskellimports" { inherit name src; }
''
cd $TMP
cp $src $TMP/Main.hs
${ghc}/bin/ghc -O -Werror -Wall -package ghc Main.hs -o Main
cp Main $out
'';
bin =
runCommand name { inherit hsie name; }
''
mkdir -p $out/bin
ln -s $hsie $out/bin/$name
'';
bash-completion =
runCommand "${name}-bash-completion" { inherit bin name; }
"$bin/bin/$name --bash-completion-script $bin/bin/$name > $out";
in
hsie // { inherit bash-completion bin; }
+6
View File
@@ -0,0 +1,6 @@
# Pinned version of Nixpkgs, generated with postgrest-nixpkgs-upgrade.
{
date = "2022-08-09";
rev = "9f15d6c3a74d2778c6e1af67947c95f100dc6fd2";
tarballHash = "14axdmi3kb6rlib39ik42yq907bm66x6vzswm5w1rsnw9vzgm31a";
}
@@ -0,0 +1,16 @@
# Creates an environment that exposes bash-completion arguments from all checkedShellScripts
{ buildEnv }:
{ name
, tools
, extra ? { }
}:
let
bash-completion = builtins.map (tool: tool.bash-completion) tools;
env = buildEnv {
inherit name;
paths = builtins.map (tool: tool.bin) tools;
};
in
env // { inherit bash-completion; } // extra
+5
View File
@@ -0,0 +1,5 @@
self: super:
# Overlay that adds `buildToolbox`, an enhanced version of `buildEnv`
{
buildToolbox = super.callPackage ./build-toolbox.nix { };
}
@@ -0,0 +1,141 @@
# Create a bash script that is checked with shellcheck. You can either use it
# directly, or use the .bin attribute to get the script in a bin/ directory,
# to be used in a path for example.
{ argbash
, bash_5
, coreutils
, git
, lib
, runCommand
, shellcheck
, stdenv
, writeTextFile
}:
{ name
, docs
, args ? [ ]
, positionalCompletion ? ""
, inRootDir ? false
, redirectTixFiles ? true
, withEnv ? null
, withPath ? [ ]
, withTmpDir ? false
}: text:
let
# square brackets are a pain to escape - if even possible. just don't use them...
escape = builtins.replaceStrings [ "\n" ] [ " \\n" ];
argsTemplate =
writeTextFile {
inherit name;
destination = "/${name}.m4"; # destination is needed to have the proper basename for completion
text =
''
# BASH_ARGV0 sets $0 - which is used in parser.sh for usage information
# stripping the /nix/store/... path for nicer display
BASH_ARGV0="$(basename "$0")"
# ARG_HELP([${name}], [${escape docs}])
${lib.strings.concatMapStrings (arg: "# " + arg) args}
# ARG_POSITIONAL_DOUBLEDASH()
# ARG_DEFAULTS_POS()
# ARGBASH_GO
'';
};
argsParser =
runCommand "${name}-parser" { }
''
${argbash}/bin/argbash ${argsTemplate}/${name}.m4 > $out
# This forces optional arguments to go *before* positional arguments,
# which allows leftovers to pass optional arguments to sub-commands.
# Example: This way `postgrest-watch -h` will return the help output for watch, while
# `postgrest-watch postgrest-test-spec -h` will return the help output for test-spec.
# Taken from: https://github.com/matejak/argbash/issues/114#issuecomment-557108274
sed '/_positionals_count + 1/a\\t\t\t\tset -- "''${@:1:1}" "--" "''${@:2}"' -i $out
'';
bash-completion =
runCommand "${name}-completion" { } (
''
${argbash}/bin/argbash --type completion --strip all ${argsTemplate}/${name}.m4 > $out
''
+ lib.optionalString (positionalCompletion != "") ''
sed 's#COMPREPLY.*compgen -o bashdefault .*$#${escape positionalCompletion}#' -i $out
''
);
bin =
writeTextFile {
inherit name;
executable = true;
destination = "/bin/${name}";
text =
''
#!${bash_5}/bin/bash
source ${argsParser}
set -euo pipefail
''
+ lib.optionalString redirectTixFiles ''
# storing tix files in a temporary throw away directory avoids mix/tix conflicts after changes
hpctixdir=$(${coreutils}/bin/mktemp -d)
export HPCTIXFILE="$hpctixdir"/postgrest.tix
trap 'rm -rf $hpctixdir' EXIT
''
+ lib.optionalString inRootDir ''
cd "$(${git}/bin/git rev-parse --show-toplevel)"
if test ! -f postgrest.cabal; then
>&2 echo "Couldn't find postgrest.cabal. Please make sure to" \
"run this command somewhere in the PostgREST repo."
exit 1
fi
''
+ lib.optionalString withTmpDir ''
mkdir -p "''${TMPDIR:-/tmp}/postgrest"
tmpdir="$(${coreutils}/bin/mktemp -d --tmpdir postgrest/${name}-XXX)"
# we keep the tmpdir when an error occurs for debugging
trap 'echo Temporary directory kept at: $tmpdir' ERR
# remove the tmpdir when cancelled (postgrest-watch)
trap 'rm -rf "$tmpdir"' SIGINT SIGTERM
''
+ lib.optionalString (withEnv != null) ''
env="$(cat ${withEnv})"
export PATH="$env/bin:$PATH"
''
+ lib.optionalString (lib.length withPath > 0) ''
export PATH="${lib.concatMapStrings (p: p + "/bin:") withPath}$PATH"
''
+ "(${text})"
+ lib.optionalString withTmpDir ''
rm -rf "$tmpdir"
'';
checkPhase =
''
# check syntax
${stdenv.shell} -n $out/bin/${name}
# check for shellcheck recommendations
${shellcheck}/bin/shellcheck -x $out/bin/${name}
'';
};
script =
runCommand name { inherit bin name; } "ln -s $bin/bin/$name $out";
in
script // { inherit bin bash-completion; }
@@ -0,0 +1,6 @@
self: super:
# Overlay that adds `checkedShellScript`, an enhanced version of
# writeShellScript and writeShellScriptBin
{
checkedShellScript = super.callPackage ./checked-shell-script.nix { };
}
+9
View File
@@ -0,0 +1,9 @@
{
build-toolbox = import ./build-toolbox;
checked-shell-script = import ./checked-shell-script;
gitignore = import ./gitignore.nix;
haskell-packages = import ./haskell-packages.nix;
postgresql-default = import ./postgresql-default.nix;
postgresql-legacy = import ./postgresql-legacy.nix;
postgresql-future = import ./postgresql-future.nix;
}
+20
View File
@@ -0,0 +1,20 @@
self: super:
# Overlay that adds the `gitignoreSource` function from Hercules-CI.
# This function is useful for filtering which files are added to the Nix store.
# See: https://github.com/hercules-ci/gitignore.nix
# To update to a newer revision, the simplest way is to add a new commit hash
# from GitHub under `rev` and to then add the hash that Nix suggests on first
# use.
{
gitignoreSource =
let
gitignoreSrc = super.fetchFromGitHub {
owner = "hercules-ci";
repo = "gitignore";
rev = "a20de23b925fd8264fd7fad6454652e142fd7f73";
sha256 = "sha256-8DFJjXG8zqoONA1vXtgeKXy68KdJL5UaXR8NtVMUbx8=";
};
in
(super.callPackage gitignoreSrc { }).gitignoreSource;
}
+44
View File
@@ -0,0 +1,44 @@
{ compiler, extraOverrides ? (final: prev: { }) }:
self: super:
let
inherit (self.haskell) lib;
overrides =
final: prev:
rec {
# To pin custom versions of Haskell packages:
# protolude =
# prev.callHackageDirect
# {
# pkg = "protolude";
# ver = "0.3.0";
# sha256 = "0iwh4wsjhb7pms88lw1afhdal9f86nrrkkvv65f9wxbd1b159n72";
# }
# { };
#
# To get the sha256:
# nix-prefetch-url --unpack https://hackage.haskell.org/package/protolude-0.3.0/protolude-0.3.0.tar.gz
# To temporarily pin unreleased versions from GitHub:
# <name> =
# prev.callCabal2nixWithOptions "<name>" (super.fetchFromGitHub {
# owner = "<owner>";
# repo = "<repo>";
# rev = "<commit>";
# sha256 = "<sha256>";
# }) "--subpath=<subpath>" {};
#
# To get the sha256:
# nix-prefetch-url --unpack https://github.com/<owner>/<repo>/archive/<commit>.tar.gz
} // extraOverrides final prev;
in
{
haskell =
super.haskell // {
packages = super.haskell.packages // {
"${compiler}" =
super.haskell.packages."${compiler}".override { inherit overrides; };
};
};
}
+5
View File
@@ -0,0 +1,5 @@
self: super:
# Overlay that sets the default version of PostgreSQL.
{
postgresql = super.postgresql_14;
}
+19
View File
@@ -0,0 +1,19 @@
self: super:
# Overlay that adds future versions of PostgreSQL that are supported by
# PostgREST.
{
## Example for including a postgresql version from a specific nixpks commit:
##
# postgresql_14 =
# let
# rev = "76b1e16c6659ccef7187ca69b287525fea133244";
# tarballHash = "1vsahpcx80k2bgslspb0sa6j4bmhdx77sw6la455drqcrqhdqj6a";
#
# pinnedPkgs =
# builtins.fetchTarball {
# url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
# sha256 = tarballHash;
# };
# in
# (import pinnedPkgs { }).pkgs.postgresql_14;
}
+19
View File
@@ -0,0 +1,19 @@
self: super:
# Overlay that adds legacy versions of PostgreSQL that are supported by
# PostgREST.
{
# PostgreSQL 9.6 was removed from Nixpkgs with
# https://github.com/NixOS/nixpkgs/commit/757dd008b2f2926fc0f7688fa8189f930ea47521
# We pin its parent commit to get the last version that was available.
postgresql_9_6 =
let
rev = "571cbf3d1db477058303cef8754fb85a14e90eb7";
tarballHash = "0q74wn418i1bn5sssacmw8ykpmqvzr0s93sj6pbs3rf6bf134fkz";
pinnedPkgs =
builtins.fetchTarball {
url = "https://github.com/nixos/nixpkgs/archive/${rev}.tar.gz";
sha256 = tarballHash;
};
in
(import pinnedPkgs { }).pkgs.postgresql_9_6;
}
+25
View File
@@ -0,0 +1,25 @@
{ runCommand }:
{
applyPatches =
name: src: patches:
runCommand
name
{ inherit src patches; }
''
set -eou pipefail
cp -r $src $out
chmod -R u+w $out
for patch in $patches; do
echo "Applying patch $patch"
patch -d "$out" -p1 < "$patch"
done
'';
static-haskell-nix-ncurses =
./static-haskell-nix-ncurses.patch;
static-haskell-nix-ghc-bignum =
./static-haskell-nix-ghc-bignum.patch;
}
@@ -0,0 +1,12 @@
diff --git a/survey/default.nix b/survey/default.nix
index 70afbbc..28cb0e9 100644
--- a/survey/default.nix
+++ b/survey/default.nix
@@ -81,6 +81,7 @@ let
# `.override` and the likes).
isProperHaskellPackage = val:
lib.isDerivation val && # must pass lib.isDerivation
+ val.pname != "ghc-bignum" &&
val ? env; # must have an .env key
# Function that tells us if a given Haskell package has an executable.
@@ -0,0 +1,13 @@
diff --git a/survey/default.nix b/survey/default.nix
index 46d8066..a47f214 100644
--- a/survey/default.nix
+++ b/survey/default.nix
@@ -1519,7 +1519,7 @@ let
[
"--enable-executable-static" # requires `useFixedCabal`
# `enableShared` seems to be required to avoid `recompile with -fPIC` errors on some packages.
- "--extra-lib-dirs=${final.ncurses.override { enableStatic = true; enableShared = true; }}/lib"
+ "--extra-lib-dirs=${final.ncurses.override { enableStatic = true; }}/lib"
]
# TODO Figure out why this and the below libffi are necessary.
# `working` and `workingStackageExecutables` don't seem to need that,
+61
View File
@@ -0,0 +1,61 @@
# Derive a fully static Haskell package based on musl instead of glibc.
{ nixpkgs, system, compiler, patches, allOverlays }:
name: src:
let
# The nh2/static-haskell-nix project does all the hard work for us.
static-haskell-nix =
let
rev = "bd66b86b72cff4479e1c76d5916a853c38d09837";
in
builtins.fetchTarball {
url = "https://github.com/nh2/static-haskell-nix/archive/${rev}.tar.gz";
sha256 = "0rnsxaw7v27znsg9lgqk1i4007ydqrc8gfgimrmhf24lv6galbjh";
};
patched-static-haskell-nix =
patches.applyPatches "patched-static-haskell-nix"
static-haskell-nix
[
patches.static-haskell-nix-ncurses
patches.static-haskell-nix-ghc-bignum
];
extraOverrides =
final: prev:
rec {
# We need to add our package needs to the package set that we pass to
# static-haskell-nix. Using callCabal2nix on the haskellPackages that
# it returns would result in a dynamic build based on musl, and not the
# fully static build that we want.
"${name}" = prev.callCabal2nix name src { };
};
overlays =
[
allOverlays.postgresql-future
allOverlays.postgresql-default
(allOverlays.haskell-packages { inherit compiler extraOverrides; })
# Disable failing tests for postgresql on musl that should have no impact
# on the libpq that we need (collate.icu.utf8 and foreign regression
# tests)
(self: super:
{ postgresql = super.postgresql.overrideAttrs (_: { doCheck = false; }); }
)
];
# Apply our overlay to nixpkgs.
normalPkgs =
import nixpkgs { inherit overlays system; };
defaultCabalPackageVersionComingWithGhc =
{
ghc924 = "Cabal_3_6_3_0";
}."${compiler}";
# The static-haskell-nix 'survey' derives a full static set of Haskell
# packages, applying fixes where necessary.
survey =
import "${patched-static-haskell-nix}/survey" { inherit normalPkgs compiler defaultCabalPackageVersionComingWithGhc; };
in
survey.haskellPackages."${name}"
+59
View File
@@ -0,0 +1,59 @@
{ buildToolbox
, cabal-install
, checkedShellScript
, devCabalOptions
, postgrest
}:
let
build =
checkedShellScript
{
name = "postgrest-build";
docs = "Build PostgREST interactively using cabal-install.";
args = [ "ARG_LEFTOVERS([Cabal arguments])" ];
inRootDir = true;
withEnv = postgrest.env;
}
''
exec ${cabal-install}/bin/cabal v2-build ${devCabalOptions} "''${_arg_leftovers[@]}"
'';
clean =
checkedShellScript
{
name = "postgrest-clean";
docs = "Clean the PostgREST project, including all cabal-install artifacts.";
inRootDir = true;
}
''
# clean old coverage data, too
rm -rf .hpc coverage
# clean old hie files
find . -name "*.hie" -type f -delete
exec ${cabal-install}/bin/cabal v2-clean
'';
run =
checkedShellScript
{
name = "postgrest-run";
docs = "Run PostgREST after buidling it interactively with cabal-install";
args = [ "ARG_LEFTOVERS([PostgREST arguments])" ];
inRootDir = true;
withEnv = postgrest.env;
}
''
exec ${cabal-install}/bin/cabal v2-run ${devCabalOptions} --verbose=0 -- \
postgrest "''${_arg_leftovers[@]}"
'';
in
buildToolbox
{
name = "postgrest-cabal";
tools = [
build
clean
run
];
}
+307
View File
@@ -0,0 +1,307 @@
{ buildToolbox
, cabal-install
, cachix
, checkedShellScript
, devCabalOptions
, entr
, git
, gnugrep
, graphviz
, haskellPackages
, hsie
, nix
, silver-searcher
, style
, tests
, withTools
}:
let
watch =
checkedShellScript
{
name = "postgrest-watch";
docs =
''
Watch the project for changes and reinvoke the given command.
Example:
postgrest-watch postgrest-test-io
'';
args =
[
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
];
positionalCompletion = "_command";
redirectTixFiles = false; # will be done by sub-command
inRootDir = true;
}
''
while true; do
(! ${silver-searcher}/bin/ag -l . | ${entr}/bin/entr -dr "$_arg_command" "''${_arg_leftovers[@]}")
done
'';
pushCachix =
checkedShellScript
{
name = "postgrest-push-cachix";
docs = ''
Push all build artifacts to cachix.
Requires authentication with `cachix authtoken ...`.
'';
inRootDir = true;
}
''
${nix}/bin/nix-instantiate \
| while read -r drv; do
${nix}/bin/nix-store -qR --include-outputs "$drv"
done \
| ${cachix}/bin/cachix push postgrest
'';
check =
checkedShellScript
{
name = "postgrest-check";
docs =
''
Run most checks that will also run on CI, but only against the
latest PostgreSQL version.
This currently excludes the memory and spec-idempotence tests,
as those are particularly expensive.
'';
inRootDir = true;
}
''
${tests}/bin/postgrest-test-spec
${tests}/bin/postgrest-test-querycost
${tests}/bin/postgrest-test-doctests
${tests}/bin/postgrest-test-io
${style}/bin/postgrest-lint
${style}/bin/postgrest-style-check
'';
gitHooks =
let
name = "postgrest-git-hooks";
in
checkedShellScript
{
inherit name;
docs =
''
Enable or disable git pre-commit and pre-push hooks.
Basic is faster and will only run:
- pre-commit: postgrest-style
- pre-push: postgrest-lint
Full takes a lot more time and will run:
- pre-commit: postgrest-style && postgrest-lint
- pre-push: postgrest-check
Changes made by postgrest-style will be staged automatically.
Example usage:
postgrest-git-hooks disable
postgrest-git-hooks enable basic
postgrest-git-hooks enable full
The "run" operation and "--hook" argument are only used internally.
'';
args =
[
"ARG_POSITIONAL_SINGLE([operation], [Operation])"
"ARG_TYPE_GROUP_SET([OPERATION], [OPERATION], [operation], [disable,enable,run])"
"ARG_POSITIONAL_SINGLE([mode], [Mode], [basic])"
"ARG_TYPE_GROUP_SET([MODE], [MODE], [mode], [basic,full])"
"ARG_OPTIONAL_SINGLE([hook], , [Hook], [pre-commit])"
"ARG_TYPE_GROUP_SET([HOOK], [HOOK], [hook], [pre-commit,pre-push])"
];
positionalCompletion =
''
if test "$prev" == "${name}"; then
COMPREPLY=( $(compgen -W "enable disable" -- "$cur") )
elif test "$prev" == "enable" || test "$prev" == "disable"; then
COMPREPLY=( $(compgen -W "basic full" -- "$cur") )
fi
'';
inRootDir = true;
}
''
if [ run != "$_arg_operation" ]; then
# Remove all hooks first and ignore failures because the file might be missing.
# This assumes that we're only adding lines that include "postgrest-git-hooks"
# to the hook file.
sed -i -e '/postgrest-git-hooks/d' .git/hooks/pre-{commit,push} 2> /dev/null || true
if [ disable != "$_arg_operation" ]; then
# The nix-shell && + nix-shell || pattern makes sure we can run the hook
# in a pure nix-shell, where nix-shell itself is not available, too.
# The $(nix-shell --run "command -v ...") pattern ensures we only need to enable
# the hooks once and still run the latest of our hook scripts, even when we
# update them in the repo.
echo 'command -v nix-shell > /dev/null || postgrest-git-hooks --hook=pre-commit run' "$_arg_mode" \
>> .git/hooks/pre-commit
# shellcheck disable=SC2016
echo 'command -v nix-shell > /dev/null && $(nix-shell --quiet -Q --run "command -v postgrest-git-hooks") --hook=pre-commit run' "$_arg_mode" \
>> .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
echo 'command -v nix-shell > /dev/null || postgrest-git-hooks --hook=pre-push run' "$_arg_mode" \
>> .git/hooks/pre-push
# shellcheck disable=SC2016
echo 'command -v nix-shell > /dev/null && $(nix-shell --quiet -Q --run "command -v postgrest-git-hooks") --hook=pre-push run' "$_arg_mode" \
>> .git/hooks/pre-push
chmod +x .git/hooks/pre-push
fi
else
# When run from a git hook, the GIT_ environment variables conflict with our withGit helper.
# The following unsets all GIT_ variables.
unset "''${!GIT_@}"
function restore () {
ref="$(git stash list --format=format:%gD --grep "$1" -n1)"
# this will avoid merge conflicts when applying the stash
${git}/bin/git restore --source="$ref" .
# restore untracked files, too. could fail with no files
if [ "$(git show --numstat --format=oneline "$ref^3" | wc -l)" -gt 1 ]; then
${git}/bin/git restore --overlay --source="$ref^3" .
fi
${git}/bin/git stash drop "$ref"
}
case "$_arg_mode" in
basic)
case "$_arg_hook" in
pre-commit)
# To be able to automatically add only changes from postgrest-style to the staging area,
# we need to run postgrest-style twice. Otherwise we'd risk merge conflicts when popping
# the stash afterwards.
${style}/bin/postgrest-style
stash="postgrest-git-hooks-$RANDOM"
${git}/bin/git stash push --include-untracked --keep-index -m "$stash"
if [ "$(git stash list --grep $stash)" ]; then
# Only create the stash pop trap, if we actually created a stash.
# Otherwise stash pop will cause havoc.
trap 'restore "$stash"' EXIT
fi
${style}/bin/postgrest-style
${git}/bin/git add .
;;
pre-push)
# Create a clean working tree without any uncomitted changes.
${withTools.withGit} HEAD ${style}/bin/postgrest-lint
;;
esac
;;
full)
case "$_arg_hook" in
pre-commit)
# To be able to automatically add only changes from postgrest-style to the staging area,
# we need to run postgrest-style twice. Otherwise we'd risk merge conflicts when popping
# the stash afterwards.
${style}/bin/postgrest-style
stash="postgrest-git-hooks-$RANDOM"
${git}/bin/git stash push --include-untracked --keep-index -m "$stash"
if [ "$(git stash list --grep $stash)" ]; then
# Only create the stash pop trap, if we actually created a stash.
# Otherwise stash pop will cause havoc.
trap 'restore "$stash"' EXIT
fi
${style}/bin/postgrest-style
${git}/bin/git add .
${style}/bin/postgrest-lint
;;
pre-push)
# Create a clean working tree without any uncomitted changes.
${withTools.withGit} HEAD ${check}
;;
esac
;;
esac
fi
'';
dumpMinimalImports =
checkedShellScript
{
name = "postgrest-dump-minimal-imports";
docs = "Dump minimal imports into given directory.";
args = [ "ARG_POSITIONAL_SINGLE([dumpdir], [Output directory])" ];
inRootDir = true;
withTmpDir = true;
}
''
mkdir -p "$_arg_dumpdir"
${cabal-install}/bin/cabal v2-build ${devCabalOptions} \
--builddir="$tmpdir" \
--ghc-option=-ddump-minimal-imports \
--ghc-option=-dumpdir="$_arg_dumpdir" \
1>&2
# Fix OverloadedRecordFields imports
# shellcheck disable=SC2016
sed -E 's/\$sel:.*://g' -i "$_arg_dumpdir"/*
'';
hsieMinimalImports =
checkedShellScript
{
name = "postgrest-hsie-minimal-imports";
docs = "Run hsie with a provided dump of minimal imports.";
args = [ "ARG_LEFTOVERS([hsie arguments])" ];
withTmpDir = true;
}
''
${dumpMinimalImports} "$tmpdir"
${hsie} "$tmpdir" "''${_arg_leftovers[@]}"
'';
hsieGraphModules =
checkedShellScript
{
name = "postgrest-hsie-graph-modules";
docs = "Create a PNG graph of modules imported within the codebase.";
args = [ "ARG_POSITIONAL_SINGLE([outfile], [Output filename])" ];
}
''
${hsie} graph-modules main src | ${graphviz}/bin/dot -Tpng -o "$_arg_outfile"
'';
hsieGraphSymbols =
checkedShellScript
{
name = "postgrest-hsie-graph-symbols";
docs = "Create a PNG graph of symbols imported within the codebase.";
args = [ "ARG_POSITIONAL_SINGLE([outfile], [Output filename])" ];
}
''
${hsieMinimalImports} graph-symbols | ${graphviz}/bin/dot -Tpng -o "$_arg_outfile"
'';
in
buildToolbox
{
name = "postgrest-dev";
tools = [
watch
pushCachix
check
gitHooks
dumpMinimalImports
hsieMinimalImports
hsieGraphModules
hsieGraphSymbols
];
}
+95
View File
@@ -0,0 +1,95 @@
# Docker image built with Nix
In order to build an optimal PostgREST Docker image, we create the image from
scratch (i.e., without a parent image like `debian` or `alpine`), and only
include the file that is essential for running PostgREST: the static
PostgREST binary.
This is similar to what you would get with the following `Dockerfile`:
```Dockerfile
# `scratch` is a minimal, reserved image in Docker, see
# https://docs.docker.com/develop/develop-images/baseimages/ . It essentially
# means "don't use a parent image and start with an empty one".
FROM scratch
# The static PostgREST executable has no runtime dependencies, so it's all we
# need to include for running the application.
ADD /absolute/path/to/postgrest /bin/postgrest
EXPOSE 3000
# This is the user id that Docker will run our image under by default. Note
# that we don't actually add the user to `/etc/passwd` or `/etc/shadow`. This
# means that tools like whoami would not work properly, but we don't include
# those in the image anyway. Not adding the user has the benefit that the image
# can be run under any user you specify.
USER 1000
CMD [ "/bin/postgrest" ]
```
# Building the Docker image with Nix
As we are building the static PostgREST executable with Nix and that's the main
input to the Docker file, we can also create the Docker image directly with Nix
using the [`dockerTools`
utilities](https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools). Those
utilities don't actually use `Dockerfiles` or Docker to build Docker images,
but create them directly by putting together the required `json` and `tar`
files that make up an image. This is more efficient, does not rely on Docker or
root permissions and results in fully reproducible builds. See
[`nix/docker/default.nix`](./default.nix) for details how the image is built.
# Building and loading the image
The Nix expression provides a helper script `postgrest-docker-load` that loads
the optimized image into your local Docker instance (using `docker load -i
<image file>` under the hood). You can use it by running:
```
# Running from the root directory of the repository:
# Build the `docker` attribute from `default.nix`, the result will be symlinked
# to `result`:
nix-build -A docker
# Run the loading script:
result/bin/postgrest-docker-load
```
The Docker image built with Nix always has the name "postgrest:latest" when
loaded.
# Inspecting the optimized image
The image does not come with the usual utilities like `bash` and `ls`.
You can, however, explore the `tar` file of the image by saving it with `docker
save postgrest:latest > image.tar`.
[Dive](https://github.com/wagoodman/dive) is also useful for looking at the
contents of the image:
```
┃ ● Layers ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ Current Layer Contents ├────────────────────────────────────────────────────────────────────────────────
Cmp Size Command Permission UID:GID Size Filetree
14 MB FROM 20ee65c811575d2 dr-xr-xr-x 0:0 14 MB ├── bin
-r-xr-xr-x 0:0 14 MB │ └── postgrest
│ Layer Details ├───────────────────────────────────────────────────────────────────────────────────────── drwxr-xr-x 0:0 783 B ├── etc
-r--r--r-- 0:0 783 B │ └── postgrest.conf
Tags: (unavailable) dr-xr-xr-x 0:0 23 kB └── nix
Id: 20ee65c811575d206eb673e1887e7f7e6b7ccde902a63ccb924c5faa50b32cee dr-xr-xr-x 0:0 23 kB └── store
Digest: sha256:ece77302b83fd38fb54395dabc10c2eba06fc1d1933801d36cc2c4732d9c8f38 dr-xr-xr-x 0:0 23 kB └── s440jbrn94wmpzy7f8yfsp6jr2shllw5-openssl-1.1.1g-etc
Command: dr-xr-xr-x 0:0 23 kB └── etc
dr-xr-xr-x 0:0 23 kB └── ssl
-r--r--r-- 0:0 412 B ├── ct_log_list.cnf
│ Image Details ├───────────────────────────────────────────────────────────────────────────────────────── -r--r--r-- 0:0 412 B ├── ct_log_list.cnf.dist
dr-xr-xr-x 0:0 0 B ├── engines-1.1
-r--r--r-- 0:0 11 kB ├── openssl.cnf
Total Image size: 14 MB -r--r--r-- 0:0 11 kB └── openssl.cnf.dist
Potential wasted space: 0 B
Image efficiency score: 100 %
Count Total Space Path
```
+49
View File
@@ -0,0 +1,49 @@
{ buildToolbox
, postgrest
, dockerTools
, checkedShellScript
}:
let
image =
dockerTools.buildImage {
name = "postgrest";
tag = "latest";
contents = postgrest;
# Set the current time as the image creation date. This makes the build
# non-reproducible, but that should not be an issue for us.
created = "now";
extraCommands =
''
rmdir share
'';
config = {
Cmd = [ "/bin/postgrest" ];
User = "1000";
ExposedPorts = {
"3000/tcp" = { };
};
};
};
load =
checkedShellScript
{
name = "postgrest-docker-load";
docs = "Load the PostgREST image into Docker.";
}
''
docker load -i ${image}
'';
in
buildToolbox
{
name = "postgrest-docker";
tools = [ load ];
extra = {
inherit image;
};
}
+178
View File
@@ -0,0 +1,178 @@
{ buildToolbox
, checkedShellScript
, jq
, python3Packages
, vegeta
, withTools
, writers
}:
let
runner =
checkedShellScript
{
name = "postgrest-loadtest-runner";
docs = "Run vegeta. Assume PostgREST to be running.";
args = [
"ARG_LEFTOVERS([additional vegeta arguments])"
"ARG_USE_ENV([PGRST_SERVER_UNIX_SOCKET], [], [Unix socket to connect to running PostgREST instance])"
];
}
''
# ARG_USE_ENV only adds defaults or docs for environment variables
# We manually implement a required check here
# See also: https://github.com/matejak/argbash/issues/80
: "''${PGRST_SERVER_UNIX_SOCKET:?PGRST_SERVER_UNIX_SOCKET is required}"
${vegeta}/bin/vegeta -cpus 1 attack \
-unix-socket "$PGRST_SERVER_UNIX_SOCKET" \
-max-workers 1 \
-workers 1 \
-rate 0 \
-duration 60s \
"''${_arg_leftovers[@]}"
'';
loadtest =
checkedShellScript
{
name = "postgrest-loadtest";
docs = "Run the vegeta loadtests with PostgREST.";
args = [
"ARG_OPTIONAL_SINGLE([output], [o], [Filename to dump json output to], [./loadtest/result.bin])"
"ARG_OPTIONAL_SINGLE([testdir], [t], [Directory to load tests and fixtures from], [./test/load])"
"ARG_LEFTOVERS([additional vegeta arguments])"
];
inRootDir = true;
}
''
# previously required settings to make this work with older branches
export PGRST_DB_ANON_ROLE="postgrest_test_anonymous"
export PGRST_DB_URI="postgresql://"
export PGRST_DB_SCHEMAS="test"
export PGRST_DB_CONFIG="false"
export PGRST_DB_POOL="1"
export PGRST_DB_TX_END="rollback-allow-override"
export PGRST_LOG_LEVEL="crit"
mkdir -p "$(dirname "$_arg_output")"
# shellcheck disable=SC2145
${withTools.withPg} --fixtures "$_arg_testdir"/fixtures.sql \
${withTools.withPgrst} \
sh -c "cd \"$_arg_testdir\" && ${runner} -targets targets.http -output \"$_arg_output\" \"''${_arg_leftovers[@]}\""
${vegeta}/bin/vegeta report -type=text "$_arg_output"
'';
loadtestAgainst =
let
name = "postgrest-loadtest-against";
in
checkedShellScript
{
inherit name;
docs =
''
Run the vegeta loadtest twice:
- once on the <target> branch
- once in the current worktree
'';
args = [
"ARG_POSITIONAL_SINGLE([target], [Commit-ish reference to compare with])"
"ARG_LEFTOVERS([additional vegeta arguments])"
];
positionalCompletion =
''
if test "$prev" == "${name}"; then
__gitcomp_nl "$(__git_refs)"
fi
'';
inRootDir = true;
}
''
cat << EOF
Running loadtest on "$_arg_target"...
EOF
# Runs the test files from the current working tree
# to make sure both tests are run with the same files.
# Save the results in the current working tree, too,
# otherwise they'd be lost in the temporary working tree
# created by withTools.withGit.
${withTools.withGit} "$_arg_target" ${loadtest} --output "$PWD/loadtest/$_arg_target.bin" --testdir "$PWD/test/load" "''${_arg_leftovers[@]}"
cat << EOF
Done running on "$_arg_target".
EOF
cat << EOF
Running loadtest on HEAD...
EOF
${loadtest} --output "$PWD/loadtest/head.bin" --testdir "$PWD/test/load" "''${_arg_leftovers[@]}"
cat << EOF
Done running on HEAD.
EOF
'';
reporter =
checkedShellScript
{
name = "postgrest-loadtest-reporter";
docs = "Create a named json report for a single result file.";
args = [
"ARG_POSITIONAL_SINGLE([file], [Filename of result to create report for])"
"ARG_LEFTOVERS([additional vegeta arguments])"
];
inRootDir = true;
}
''
${vegeta}/bin/vegeta report -type=json "$_arg_file" \
| ${jq}/bin/jq --arg branch "$(basename "$_arg_file" .bin)" '. + {branch: $branch}'
'';
toMarkdown =
writers.writePython3 "postgrest-loadtest-to-markdown"
{
libraries = [ python3Packages.pandas python3Packages.tabulate ];
}
''
import sys
import pandas as pd
pd.read_json(sys.stdin) \
.set_index('param') \
.drop(['branch', 'earliest', 'end', 'latest']) \
.convert_dtypes() \
.to_markdown(sys.stdout, floatfmt='.0f')
'';
report =
checkedShellScript
{
name = "postgrest-loadtest-report";
docs = "Create a report of all loadtest reports as markdown.";
inRootDir = true;
}
''
find loadtest -type f -iname '*.bin' -exec ${reporter} {} \; \
| ${jq}/bin/jq '[leaf_paths as $path | {param: $path | join("."), (.branch): getpath($path)}]' \
| ${jq}/bin/jq --slurp 'flatten | group_by(.param) | map(add)' \
| ${toMarkdown}
'';
in
buildToolbox {
name = "postgrest-loadtest";
tools = [ loadtest loadtestAgainst report ];
}
+28
View File
@@ -0,0 +1,28 @@
# The memory tests have large dependencies (a profiled build of PostgREST)
# and are run less often than the spec tests, so we don't include them in
# the default test environment. We make them available through a separate module.
{ buildToolbox
, checkedShellScript
, curl
, postgrestProfiled
, withTools
}:
let
test =
checkedShellScript
{
name = "postgrest-test-memory";
docs = "Run the memory tests.";
inRootDir = true;
withPath = [ postgrestProfiled curl ];
}
''
${withTools.withPg} test/memory/memory-tests.sh
'';
in
buildToolbox
{
name = "postgrest-memory";
tools = [ test ];
}
+53
View File
@@ -0,0 +1,53 @@
{ buildToolbox
, checkedShellScript
, curl
, jq
, nix
}:
# Utility script for pinning the latest unstable version of Nixpkgs.
# Instead of pinning Nixpkgs based on the huge Git repository, we reference a
# specific tarball that only contains the source of the revision that we want
# to pin.
let
name =
"postgrest-nixpkgs-upgrade";
refUrl =
"https://api.github.com/repos/nixos/nixpkgs/git/ref/heads/nixpkgs-unstable";
githubV3Header =
"Accept: application/vnd.github.v3+json";
tarballUrlBase =
"https://github.com/nixos/nixpkgs/archive/";
upgrade =
checkedShellScript
{
inherit name;
docs = "Pin the newest unstable version of Nixpkgs.";
inRootDir = true;
}
''
commitHash="$(${curl}/bin/curl "${refUrl}" -H "${githubV3Header}" | ${jq}/bin/jq -r .object.sha)"
tarballUrl="${tarballUrlBase}$commitHash.tar.gz"
tarballHash="$(${nix}/bin/nix-prefetch-url --unpack "$tarballUrl")"
currentDate="$(date --iso)"
cat > nix/nixpkgs-version.nix << EOF
# Pinned version of Nixpkgs, generated with ${name}.
{
date = "$currentDate";
rev = "$commitHash";
tarballHash = "$tarballHash";
}
EOF
'';
in
buildToolbox
{
name = "postgrest-nixpkgs";
tools = [ upgrade ];
}
+127
View File
@@ -0,0 +1,127 @@
{ buildToolbox
, checkedShellScript
, curl
, jq
}:
let
dockerHubDescription =
let
description =
./docker-hub-description.md;
fullDescription =
./docker-hub-full-description.md;
in
checkedShellScript
{
name = "postgrest-release-dockerhub-description";
docs = "Update the repository description on Docker Hub.";
args = [
"ARG_USE_ENV([DOCKER_USER], [], [DockerHub user name])"
"ARG_USE_ENV([DOCKER_PASS], [], [DockerHub password])"
"ARG_USE_ENV([DOCKER_REPO], [], [DockerHub repository])"
];
}
''
# ARG_USE_ENV only adds defaults or docs for environment variables
# We manually implement a required check here
# See also: https://github.com/matejak/argbash/issues/80
: "''${DOCKER_USER:?DOCKER_USER is required}"
: "''${DOCKER_PASS:?DOCKER_PASS is required}"
: "''${DOCKER_REPO:?DOCKER_REPO is required}"
echo "Logging in to Docker Hub to get an auth token..."
token="$(
${curl}/bin/curl --fail -s \
--data-urlencode "username=$DOCKER_USER" \
--data-urlencode "password=$DOCKER_PASS" \
"https://hub.docker.com/v2/users/login/" \
| ${jq}/bin/jq -r .token
)"
repo_url="https://hub.docker.com/v2/repositories/$DOCKER_REPO/postgrest/"
echo "Patching both descriptions at $repo_url ..."
${curl}/bin/curl --fail -X PATCH "$repo_url" \
-H "Authorization: JWT $token" \
--data-urlencode description@${description} \
--data-urlencode full_description@${fullDescription}
'';
release =
checkedShellScript
{
name = "postgrest-release";
docs = "Patch postgrest.cabal, tag and push all in one go.";
args = [ "ARG_POSITIONAL_SINGLE([version], [Version to release], [pre])" ];
inRootDir = true;
}
''
trap "echo You need to be on the main branch to proceed. Exiting ..." ERR
[ "$(git rev-parse --abbrev-ref HEAD)" == "main" ]
trap "" ERR
trap "echo You have uncommitted changes in postgrest.cabal. Exiting ..." ERR
git diff --exit-code HEAD postgrest.cabal > /dev/null
trap "" ERR
current_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
# shellcheck disable=SC2034
IFS=. read -r major minor patch pre <<< "$current_version"
echo "Current version is $current_version"
bump_pre="$major.$minor.$patch.$(date '+%Y%m%d')"
bump_patch="$major.$minor.$((patch+1))"
bump_minor="$major.$((minor+1)).0"
bump_major="$((major+1)).0.0"
PS3="Please select the new version: "
select new_version in "$bump_pre" "$bump_patch" "$bump_minor" "$bump_major"; do
case "$REPLY" in
1|2|3|4)
echo "Selected $new_version"
break
;;
*)
echo "Invalid option $REPLY"
;;
esac
done
echo "Updating postgrest.cabal ..."
sed -i -E "s/^(version:\s+).*$/\1$new_version/" postgrest.cabal > /dev/null
echo "Committing ..."
git add postgrest.cabal > /dev/null
git commit -m "bump version to $new_version" > /dev/null
echo "Tagging ..."
git tag "v$new_version" > /dev/null
trap "Couldn't find remote. Please push manually ..." ERR
remote="$(git remote -v | grep PostgREST/postgrest | grep push | cut -f1)"
trap "" ERR
push="git push --atomic $remote main v$new_version"
echo "To push both the branch and the new tag, the following will be run:"
echo
echo "$push"
echo
read -r -p 'Proceed? (y/N) ' REPLY
case "$REPLY" in
y|Y)
$push
;;
*)
echo "Aborting ..."
;;
esac
'';
in
buildToolbox
{
name = "postgrest-release";
tools = [ dockerHubDescription release ];
}
@@ -0,0 +1 @@
REST API for any Postgres database
@@ -0,0 +1,70 @@
# PostgREST
[![Join the chat at https://gitter.im/begriffs/postgrest](https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/begriffs/postgrest)
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854)](https://www.patreon.com/postgrest)
[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/postgrest)
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://postgrest.org)
[![Build Status](https://github.com/postgrest/postgrest/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/PostgREST/postgrest/actions?query=branch%3Amain)
PostgREST serves a fully RESTful API from any existing PostgreSQL database. It
provides a cleaner, more standards-compliant, faster API than you are likely to
write from scratch.
## Sponsors
<table>
<tbody>
<tr>
<td align="center" valign="middle">
<a href="https://www.cybertec-postgresql.com/en/?utm_source=postgrest.org&utm_medium=referral&utm_campaign=postgrest" target="_blank">
<img width="222px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/cybertec-new.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://www.2ndquadrant.com/en/?utm_campaign=External%20Websites&utm_source=PostgREST&utm_medium=Logo" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/2ndquadrant.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://tryretool.com/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/retool.png">
</a>
</td>
</tr>
<tr></tr>
<tr>
<td align="center" valign="middle">
<a href="https://gnuhost.eu/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/gnuhost.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://supabase.io?utm_source=postgrest%20backers&utm_medium=open%20source%20partner&utm_campaign=postgrest%20backers%20github&utm_term=homepage" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/supabase.png">
</a>
</td>
<td align="center" valign="middle">
<a href="https://oblivious.ai/?utm_source=sponsor&utm_campaign=postgrest" target="_blank">
<img width="296px" src="https://raw.githubusercontent.com/PostgREST/postgrest/main/static/oblivious.jpg">
</a>
</td>
</tr>
</tbody>
</table>
# Usage
To learn how to use this container, see the [PostgREST Docker
documentation](https://postgrest.org/en/stable/install.html#docker).
You can configure the PostgREST image by setting
[enviroment variables](https://postgrest.org/en/stable/configuration.html).
# How this image is built
The image is built from scratch using
[Nix](https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools) instead of a
`Dockerfile`, which yields a higly secure and optimized image. This is also why
no commands are listed in the image history. See the [PostgREST
respository](https://github.com/PostgREST/postgrest/tree/main/nix/tools/docker) for
details on the build process and how to inspect the image.
+83
View File
@@ -0,0 +1,83 @@
{ actionlint
, black
, buildToolbox
, checkedShellScript
, git
, hlint
, hsie
, nixpkgs-fmt
, shellcheck
, silver-searcher
, statix
, stylish-haskell
}:
let
style =
checkedShellScript
{
name = "postgrest-style";
docs = "Automatically format Haskell, Nix and Python files.";
inRootDir = true;
}
''
# Format Nix files
${statix}/bin/statix fix
${nixpkgs-fmt}/bin/nixpkgs-fmt . > /dev/null 2> /dev/null
# Format Haskell files
# --vimgrep fixes a bug in ag: https://github.com/ggreer/the_silver_searcher/issues/753
${silver-searcher}/bin/ag -l --vimgrep -g '\.l?hs$' . \
| xargs ${stylish-haskell}/bin/stylish-haskell -i
# Format Python files
${black}/bin/black . 2> /dev/null
'';
# Script to check whether any uncommited changes result from postgrest-style
styleCheck =
checkedShellScript
{
name = "postgrest-style-check";
docs = "Check whether postgrest-style results in any uncommited changes.";
inRootDir = true;
}
''
${style}
trap "echo postgrest-style-check failed. Run postgrest-style to fix issues automatically." ERR
${git}/bin/git diff-index --exit-code HEAD -- '*.hs' '*.lhs' '*.nix' '*.py'
'';
lint =
checkedShellScript
{
name = "postgrest-lint";
docs = "Lint all Haskell files, bash scripts and github workflows.";
inRootDir = true;
}
''
echo "Checking consistency of import aliases in Haskell code..."
${hsie} check-aliases main src
echo "Linting Haskell files..."
# --vimgrep fixes a bug in ag: https://github.com/ggreer/the_silver_searcher/issues/753
${silver-searcher}/bin/ag -l --vimgrep -g '\.l?hs$' . \
| xargs ${hlint}/bin/hlint -X QuasiQuotes -X NoPatternSynonyms
echo "Linting bash scripts..."
${shellcheck}/bin/shellcheck \
.github/get_cirrusci_freebsd \
.github/release
echo "Linting workflows..."
${actionlint}/bin/actionlint
'';
in
buildToolbox
{
name = "postgrest-style";
tools = [ style styleCheck lint ];
}
+246
View File
@@ -0,0 +1,246 @@
{ buildToolbox
, cabal-install
, checkedShellScript
, devCabalOptions
, ghc
, glibcLocales ? null
, gnugrep
, hpc-codecov
, hostPlatform
, jq
, lib
, postgrest
, python3
, runtimeShell
, stdenv
, weeder
, withTools
, yq
}:
let
testSpec =
checkedShellScript
{
name = "postgrest-test-spec";
docs = "Run the Haskell test suite. Use --match PATTERN for running individual specs";
args = [ "ARG_LEFTOVERS([hspec arguments])" ];
inRootDir = true;
withEnv = postgrest.env;
}
''
${withTools.withPg} ${cabal-install}/bin/cabal v2-run ${devCabalOptions} \
test:spec -- "''${_arg_leftovers[@]}"
'';
testQuerycost =
checkedShellScript
{
name = "postgrest-test-querycost";
docs = "Run the Haskell test suite for query costs";
inRootDir = true;
withEnv = postgrest.env;
}
''
${withTools.withPg} ${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:querycost
'';
testDoctests =
checkedShellScript
{
name = "postgrest-test-doctests";
docs = "Run the Haskell doctest test suite";
inRootDir = true;
withEnv = postgrest.env;
}
''
# For unknown reasons, doctests uses the wrong GHC package database outside
# nix-shell and fails, so we set the package path explicitly
#ghcWithPackages="$(cat ${postgrest.env})"
#ghcVersion="$(ls "$ghcWithPackages/lib")"
#export GHC_PACKAGE_PATH="$ghcWithPackages/lib/$ghcVersion/package.conf.d/"
${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:doctests
'';
testSpecIdempotence =
checkedShellScript
{
name = "postgrest-test-spec-idempotence";
docs = "Check that the Haskell tests can be run multiple times against the same db.";
inRootDir = true;
withEnv = postgrest.env;
}
''
${withTools.withPg} ${runtimeShell} -c " \
${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:spec && \
${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:spec"
'';
ioTestPython =
python3.withPackages (ps: [
ps.pyjwt
ps.pytest
ps.pytest_xdist
ps.pyyaml
ps.requests
ps.requests-unixsocket
]);
testIO =
checkedShellScript
{
name = "postgrest-test-io";
docs = "Run the pytest-based IO tests. Add -k to run tests that match a given expression.";
args = [ "ARG_LEFTOVERS([pytest arguments])" ];
inRootDir = true;
withEnv = postgrest.env;
}
''
${cabal-install}/bin/cabal v2-build ${devCabalOptions}
${cabal-install}/bin/cabal v2-exec -- ${withTools.withPg} -f test/io/fixtures.sql \
${ioTestPython}/bin/pytest -v test/io "''${_arg_leftovers[@]}"
'';
dumpSchema =
checkedShellScript
{
name = "postgrest-dump-schema";
docs = "Dump the loaded schema's DbStructure as a yaml file.";
inRootDir = true;
withEnv = postgrest.env;
withPath = [ jq ];
}
''
${withTools.withPg} \
${cabal-install}/bin/cabal v2-run ${devCabalOptions} --verbose=0 -- \
postgrest --dump-schema \
| ${yq}/bin/yq -y .
'';
coverage =
checkedShellScript
{
name = "postgrest-coverage";
docs = "Run spec and io tests while collecting hpc coverage data. First runs weeder to detect dead code.";
args = [ "ARG_LEFTOVERS([hpc report arguments])" ];
inRootDir = true;
redirectTixFiles = false;
withEnv = postgrest.env;
withTmpDir = true;
}
(
# required for `hpc markup` in CI; glibcLocales is not available e.g. on Darwin
lib.optionalString (stdenv.isLinux && hostPlatform.libc == "glibc") ''
export LOCALE_ARCHIVE="${glibcLocales}/lib/locale/locale-archive"
'' +
''
# clean up previous coverage reports
mkdir -p coverage
rm -rf coverage/*
# build once before running all the tests
${cabal-install}/bin/cabal v2-build ${devCabalOptions} exe:postgrest lib:postgrest test:spec test:querycost
(
trap 'echo Found dead code: Check file list above.' ERR ;
${weeder}/bin/weeder --config=./test/weeder.dhall
)
# collect all tests
HPCTIXFILE="$tmpdir"/io.tix \
${withTools.withPg} -f test/io/fixtures.sql ${cabal-install}/bin/cabal v2-exec ${devCabalOptions} -- \
${ioTestPython}/bin/pytest -v test/io
HPCTIXFILE="$tmpdir"/spec.tix \
${withTools.withPg} ${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:spec
HPCTIXFILE="$tmpdir"/querycost.tix \
${withTools.withPg} ${cabal-install}/bin/cabal v2-run ${devCabalOptions} test:querycost
# Note: No coverage for doctests, as doctests leverage GHCi and GHCi does not support hpc
# collect all the tix files
${ghc}/bin/hpc sum --union --exclude=Paths_postgrest --output="$tmpdir"/tests.tix \
"$tmpdir"/io*.tix "$tmpdir"/spec.tix "$tmpdir"/querycost.tix
# prepare the overlay
${ghc}/bin/hpc overlay --output="$tmpdir"/overlay.tix test/coverage.overlay
${ghc}/bin/hpc sum --union --output="$tmpdir"/tests-overlay.tix "$tmpdir"/tests.tix "$tmpdir"/overlay.tix
# check nothing in the overlay is actually tested
${ghc}/bin/hpc map --function=inv --output="$tmpdir"/inverted.tix "$tmpdir"/tests.tix
${ghc}/bin/hpc combine --function=sub \
--output="$tmpdir"/check.tix "$tmpdir"/overlay.tix "$tmpdir"/inverted.tix
# returns zero exit code if any count="<non-zero>" lines are found, i.e.
# something is covered by both the overlay and the tests
if ${ghc}/bin/hpc report --xml "$tmpdir"/check.tix | ${gnugrep}/bin/grep -qP 'count="[^0]'
then
${ghc}/bin/hpc markup --highlight-covered --destdir=coverage/overlay "$tmpdir"/overlay.tix || true
${ghc}/bin/hpc markup --highlight-covered --destdir=coverage/check "$tmpdir"/check.tix || true
echo "ERROR: Something is covered by both the tests and the overlay:"
echo "file://$(pwd)/coverage/check/hpc_index.html"
exit 1
else
# copy the result .tix file to the coverage/ dir to make it available to postgrest-coverage-draft-overlay, too
cp "$tmpdir"/tests-overlay.tix coverage/postgrest.tix
# prepare codecov json report
${hpc-codecov}/bin/hpc-codecov --mix=.hpc --out=coverage/codecov.json coverage/postgrest.tix
# create html and stdout reports
${ghc}/bin/hpc markup --destdir=coverage coverage/postgrest.tix
echo "file://$(pwd)/coverage/hpc_index.html"
${ghc}/bin/hpc report coverage/postgrest.tix "''${_arg_leftovers[@]}"
fi
''
);
coverageDraftOverlay =
checkedShellScript
{
name = "postgrest-coverage-draft-overlay";
docs = "Create a draft overlay from current coverage report.";
inRootDir = true;
}
''
${ghc}/bin/hpc draft --output=test/coverage.overlay coverage/postgrest.tix
sed -i 's|^module \(.*\):|module \1/|g' test/coverage.overlay
'';
checkStatic =
checkedShellScript
{
name = "postgrest-check-static";
docs = "Verify that the argument is a static executable.";
args = [ "ARG_POSITIONAL_SINGLE([executable], [Executable])" ];
inRootDir = true;
withEnv = postgrest.env;
}
''
exe="$_arg_executable"
ldd_output=$(ldd "$exe" 2>&1 || true)
if ! grep -q "not a dynamic executable" <<< "$ldd_output"; then
echo "not a static executable, ldd output:"
echo "$ldd_output"
exit 1
fi
"$exe" --help
'';
in
buildToolbox
{
name = "postgrest-tests";
tools =
[
testSpec
testQuerycost
testDoctests
testSpecIdempotence
testIO
dumpSchema
coverage
coverageDraftOverlay
checkStatic
];
}
+281
View File
@@ -0,0 +1,281 @@
{ bash-completion
, buildToolbox
, cabal-install
, checkedShellScript
, curl
, devCabalOptions
, git
, lib
, postgresqlVersions
, postgrest
, writeText
}:
let
withTmpDb =
{ name, postgresql }:
checkedShellScript
{
name = "postgrest-with-${name}";
docs = "Run the given command in a temporary database with ${name}";
args =
[
"ARG_OPTIONAL_SINGLE([fixtures], [f], [SQL file to load fixtures from], [test/spec/fixtures/load.sql])"
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
"ARG_USE_ENV([PGUSER], [postgrest_test_authenticator], [Authenticator PG role])"
"ARG_USE_ENV([PGDATABASE], [postgres], [PG database name])"
"ARG_USE_ENV([PGRST_DB_SCHEMAS], [test], [Schema to expose])"
];
positionalCompletion = "_command";
inRootDir = true;
redirectTixFiles = false;
withPath = [ postgresql ];
withTmpDir = true;
}
''
# avoid starting multiple layers of withTmpDb
if test -v PGHOST; then
exec "$_arg_command" "''${_arg_leftovers[@]}"
fi
setuplog="$tmpdir/setup.log"
log () {
echo "$1" >> "$setuplog"
}
mkdir -p "$tmpdir"/{db,socket}
# remove data dir, even if we keep tmpdir - no need to upload it to artifacts
trap 'rm -rf $tmpdir/db' EXIT
export PGDATA="$tmpdir/db"
export PGHOST="$tmpdir/socket"
export PGUSER
export PGDATABASE
export PGRST_DB_SCHEMAS
log "Initializing database cluster..."
# We try to make the database cluster as independent as possible from the host
# by specifying the timezone, locale and encoding.
PGTZ=UTC initdb --no-locale --encoding=UTF8 --nosync -U "$PGUSER" --auth=trust \
>> "$setuplog"
log "Starting the database cluster..."
# Instead of listening on a local port, we will listen on a unix domain socket.
pg_ctl -l "$tmpdir/db.log" -w start -o "-F -c listen_addresses=\"\" -k $PGHOST -c log_statement=\"all\"" \
>> "$setuplog"
stop () {
log "Stopping the database cluster..."
pg_ctl stop -m i >> "$setuplog"
rm -rf "$tmpdir/db"
}
trap stop EXIT
log "Loading fixtures..."
psql -v ON_ERROR_STOP=1 -f "$_arg_fixtures" >> "$setuplog"
log "Done. Running command..."
("$_arg_command" "''${_arg_leftovers[@]}")
'';
# Helper script for running a command against all PostgreSQL versions.
withPgAll =
let
runners =
builtins.map
(version:
''
cat << EOF
Running against ${version.name}...
EOF
trap 'echo "Failed on ${version.name}"' exit
(${withTmpDb version} "$_arg_command" "''${_arg_leftovers[@]}")
trap "" exit
cat << EOF
Done running against ${version.name}.
EOF
'')
postgresqlVersions;
in
checkedShellScript
{
name = "postgrest-with-all";
docs = "Run command against all supported PostgreSQL versions.";
args =
[
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
];
positionalCompletion = "_command";
inRootDir = true;
}
(lib.concatStringsSep "\n\n" runners);
# Create a `postgrest-with-postgresql-` for each PostgreSQL version
withPgVersions = builtins.map withTmpDb postgresqlVersions;
withPg = builtins.head withPgVersions;
withGit =
let
name = "postgrest-with-git";
in
checkedShellScript
{
inherit name;
docs =
''
Create a new worktree of the postgrest repo in a temporary directory and
check out <commit>, then run <command> with arguments inside the temporary folder.
'';
args =
[
"ARG_POSITIONAL_SINGLE([commit], [Commit-ish reference to run command with])"
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
];
positionalCompletion =
''
if test "$prev" == "${name}"; then
__gitcomp_nl "$(__git_refs)"
else
_command_offset 2
fi
'';
inRootDir = true;
}
''
# not using withTmpDir here, because we don't want to keep the directory on error
tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT
${git}/bin/git worktree add -f "$tmpdir" "$_arg_commit" > /dev/null
cd "$tmpdir"
("$_arg_command" "''${_arg_leftovers[@]}")
${git}/bin/git worktree remove -f "$tmpdir" > /dev/null
'';
legacyConfig =
writeText "legacy.conf"
''
# Using this config file to support older postgrest versions for `postgrest-loadtest-against`
db-uri="$(PGRST_DB_URI)"
db-schema="$(PGRST_DB_SCHEMAS)"
db-anon-role="$(PGRST_DB_ANON_ROLE)"
db-pool="$(PGRST_DB_POOL)"
server-unix-socket="$(PGRST_SERVER_UNIX_SOCKET)"
log-level="$(PGRST_LOG_LEVEL)"
'';
waitForPgrstPid =
checkedShellScript
{
name = "postgrest-wait-for-pgrst-pid";
docs = "Wait for PostgREST to be running. Needs to be a separate command for timeout to work below.";
args = [
"ARG_USE_ENV([PGRST_SERVER_UNIX_SOCKET], [], [Unix socket to check for running PostgREST instance])"
];
}
''
# ARG_USE_ENV only adds defaults or docs for environment variables
# We manually implement a required check here
# See also: https://github.com/matejak/argbash/issues/80
: "''${PGRST_SERVER_UNIX_SOCKET:?PGRST_SERVER_UNIX_SOCKET is required}"
until [ -S "$PGRST_SERVER_UNIX_SOCKET" ]
do
sleep 0.1
done
# return pid of postgrest process
lsof -t -c '/^postgrest$/' "$PGRST_SERVER_UNIX_SOCKET"
'';
waitForPgrstReady =
checkedShellScript
{
name = "postgrest-wait-for-pgrst-ready";
docs = "Wait for PostgREST to be ready to serve requests. Needs to be a separate command for timeout to work below.";
args = [
"ARG_USE_ENV([PGRST_SERVER_UNIX_SOCKET], [], [Unix socket to check for running PostgREST instance])"
];
}
''
# ARG_USE_ENV only adds defaults or docs for environment variables
# We manually implement a required check here
# See also: https://github.com/matejak/argbash/issues/80
: "''${PGRST_SERVER_UNIX_SOCKET:?PGRST_SERVER_UNIX_SOCKET is required}"
function check_status () {
${curl}/bin/curl -s -o /dev/null -w "%{http_code}" --unix-socket "$PGRST_SERVER_UNIX_SOCKET" http://localhost/
}
while [[ "$(check_status)" != "200" ]];
do sleep 0.1;
done
'';
withPgrst =
checkedShellScript
{
name = "postgrest-with-pgrst";
docs = "Build and run PostgREST and run <command> with PGRST_SERVER_UNIX_SOCKET set.";
args =
[
"ARG_POSITIONAL_SINGLE([command], [Command to run])"
"ARG_LEFTOVERS([command arguments])"
];
positionalCompletion = "_command";
inRootDir = true;
withEnv = postgrest.env;
withTmpDir = true;
}
''
export PGRST_SERVER_UNIX_SOCKET="$tmpdir"/postgrest.socket
rm -f result
echo -n "Building postgrest... "
nix-build -A postgrestPackage > "$tmpdir"/build.log 2>&1 || {
echo "failed, output:"
cat "$tmpdir"/build.log
exit 1
}
echo "done."
echo -n "Starting postgrest... "
./result/bin/postgrest ${legacyConfig} > "$tmpdir"/run.log 2>&1 &
pid=$!
cleanup() {
kill "$pid" || true
}
trap cleanup EXIT
timeout -s TERM 5 ${waitForPgrstReady} || {
echo "timed out, output:"
cat "$tmpdir"/run.log
exit 1
}
echo "done."
("$_arg_command" "''${_arg_leftovers[@]}")
'';
in
buildToolbox
{
name = "postgrest-with";
tools = [ withPgAll withGit withPgrst ] ++ withPgVersions;
# make withTools available for other nix files
extra = { inherit withGit withPg withPgAll withPgrst; };
}
+297 -168
View File
@@ -1,176 +1,305 @@
name: postgrest
description: Reads the schema of a PostgreSQL database and creates RESTful routes
for the tables and views, supporting all HTTP verbs that security
permits.
version: 5.2.0
synopsis: REST API for any Postgres database
license: MIT
license-file: LICENSE
author: Joe Nelson, Adam Baker
homepage: https://postgrest.org
maintainer: Steve Chávez <stevechavezast@gmail.com>
bug-reports: https://github.com/PostgREST/postgrest/issues
category: Executable, PostgreSQL, Network APIs
extra-source-files: CHANGELOG.md
build-type: Simple
cabal-version: >=1.10
name: postgrest
version: 10.0.0
synopsis: REST API for any Postgres database
description: Reads the schema of a PostgreSQL database and creates RESTful routes
for tables, views, and functions, supporting all HTTP methods that security
permits.
license: MIT
license-file: LICENSE
author: Joe Nelson, Adam Baker, Steve Chavez
maintainer: Steve Chavez <stevechavezast@gmail.com>
category: Executable, PostgreSQL, Network APIs
homepage: https://postgrest.org
bug-reports: https://github.com/PostgREST/postgrest/issues
build-type: Simple
extra-source-files: CHANGELOG.md
cabal-version: >= 1.10
source-repository head
type: git
type: git
location: git://github.com/PostgREST/postgrest.git
Flag CI
Description: No warnings allowed in continuous integration
Manual: True
Default: False
flag dev
default: False
manual: True
description: Development flags
executable postgrest
main-is: Main.hs
default-extensions: OverloadedStrings, QuasiQuotes, NoImplicitPrelude
ghc-options:
-threaded
-rtsopts
"-with-rtsopts=-N -I2"
default-language: Haskell2010
build-depends: auto-update
, base >= 4.8 && < 4.10
, hasql >= 1.3 && < 1.4
, hasql-pool >= 0.5 && < 0.6
, hasql-transaction >= 0.7 && < 0.8
, postgrest
, protolude == 0.2.2
, text
, time
, warp
, bytestring
, base64-bytestring
, retry
if !os(windows)
build-depends: unix
hs-source-dirs: main
flag hpc
default: True
manual: True
description: Enable HPC (dev only)
library
default-language: Haskell2010
default-extensions: OverloadedStrings, QuasiQuotes, NoImplicitPrelude
build-depends: aeson
, ansi-wl-pprint
, base >= 4.8 && < 4.10
, base64-bytestring
, bytestring
, case-insensitive
, cassava
, configurator-ng == 0.0.0.1
, containers
, contravariant
, contravariant-extras
, either
, gitrev
, hasql >= 1.3 && < 1.4
, hasql-pool >= 0.5 && < 0.6
, hasql-transaction >= 0.7 && < 0.8
, heredoc
, HTTP
, http-types
, insert-ordered-containers
, interpolatedstring-perl6
, jose == 0.7.0.0
, lens
, lens-aeson
, network-uri
, optparse-applicative >= 0.13 && < 0.15
, parsec
, protolude == 0.2.2
, Ranged-sets == 0.3.0
, regex-tdfa
, scientific
, swagger2
, text
, time
, unordered-containers
, vector
, wai
, wai-cors
, wai-extra
, wai-middleware-static
, cookie
default-language: Haskell2010
default-extensions: OverloadedStrings
NoImplicitPrelude
hs-source-dirs: src
exposed-modules: PostgREST.App
PostgREST.Admin
PostgREST.AppState
PostgREST.Auth
PostgREST.CLI
PostgREST.Config
PostgREST.Config.Database
PostgREST.Config.JSPath
PostgREST.Config.PgVersion
PostgREST.Config.Proxy
PostgREST.Cors
PostgREST.DbStructure
PostgREST.DbStructure.Identifiers
PostgREST.DbStructure.Proc
PostgREST.DbStructure.Relationship
PostgREST.DbStructure.Table
PostgREST.Error
PostgREST.GucHeader
PostgREST.Logger
PostgREST.Middleware
PostgREST.MediaType
PostgREST.OpenAPI
PostgREST.Query.QueryBuilder
PostgREST.Query.SqlFragment
PostgREST.Query.Statements
PostgREST.RangeQuery
PostgREST.Request.ApiRequest
PostgREST.Request.DbRequestBuilder
PostgREST.Request.MutateQuery
PostgREST.Request.Preferences
PostgREST.Request.QueryParams
PostgREST.Request.ReadQuery
PostgREST.Request.Types
PostgREST.Version
PostgREST.Workers
other-modules: Paths_postgrest
build-depends: base >= 4.9 && < 4.17
, HTTP >= 4000.3.7 && < 4000.4
, Ranged-sets >= 0.3 && < 0.5
, aeson >= 2.0.3 && < 2.1
, auto-update >= 0.1.4 && < 0.2
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.12
, case-insensitive >= 1.2 && < 1.3
, cassava >= 0.4.5 && < 0.6
, configurator-pg >= 0.2 && < 0.3
, containers >= 0.5.7 && < 0.7
, contravariant-extras >= 0.3.3 && < 0.4
, cookie >= 0.4.2 && < 0.5
, either >= 4.4.1 && < 5.1
, gitrev >= 1.2 && < 1.4
, hasql >= 1.4 && < 1.6
, hasql-dynamic-statements >= 0.3.1 && < 0.4
, hasql-notifications >= 0.1 && < 0.3
, hasql-pool >= 0.5 && < 0.6
, hasql-transaction >= 1.0.1 && < 1.1
, heredoc >= 0.2 && < 0.3
, http-types >= 0.12.2 && < 0.13
, insert-ordered-containers >= 0.2.2 && < 0.3
, interpolatedstring-perl6 >= 1 && < 1.1
, jose >= 0.8.5.1 && < 0.10
, lens >= 4.14 && < 5.2
, lens-aeson >= 1.0.1 && < 1.2
, mtl >= 2.2.2 && < 2.3
, network >= 2.6 && < 3.2
, network-uri >= 2.6.1 && < 2.8
, optparse-applicative >= 0.13 && < 0.17
, parsec >= 3.1.11 && < 3.2
, protolude >= 0.3.1 && < 0.4
, regex-tdfa >= 1.2.2 && < 1.4
, retry >= 0.7.4 && < 0.10
, scientific >= 0.3.4 && < 0.4
, swagger2 >= 2.4 && < 2.9
, text >= 1.2.2 && < 1.3
, time >= 1.6 && < 1.12
, unordered-containers >= 0.2.8 && < 0.3
, vault >= 0.3.1.5 && < 0.4
, vector >= 0.11 && < 0.13
, wai >= 3.2.1 && < 3.3
, wai-cors >= 0.2.5 && < 0.3
, wai-extra >= 3.1.8 && < 3.2
-- We already depend on wai-logger >= 2.3.7 indirectly via wai-extra,
-- but we want to depend on 2.4.0 which fixes 'unknownSocket' log output
-- for unix sockets; this is tested in test/io/test_io.py. See
-- https://github.com/kazu-yamamoto/logger/commit/3a71ca70afdbb93d4ecf0083eeba1fbbbcab3fc3
, wai-logger >= 2.4.0
, warp >= 3.3.19 && < 3.4
-- -fno-spec-constr may help keep compile time memory use in check,
-- see https://gitlab.haskell.org/ghc/ghc/issues/16017#note_219304
-- -optP-Wno-nonportable-include-path
-- prevents build failures on case-insensitive filesystems (macos),
-- see https://github.com/commercialhaskell/stack/issues/3918
ghc-options: -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
Other-Modules: Paths_postgrest
Exposed-Modules: PostgREST.ApiRequest
, PostgREST.App
, PostgREST.Auth
, PostgREST.Config
, PostgREST.DbStructure
, PostgREST.DbRequestBuilder
, PostgREST.Error
, PostgREST.Middleware
, PostgREST.OpenAPI
, PostgREST.Parsers
, PostgREST.QueryBuilder
, PostgREST.RangeQuery
, PostgREST.Types
hs-source-dirs: src
if flag(dev)
ghc-options: -O0 -fwrite-ide-info
if flag(hpc)
ghc-options: -fhpc -hpcdir .hpc
else
ghc-options: -O2
Test-Suite spec
Type: exitcode-stdio-1.0
Default-Language: Haskell2010
default-extensions: OverloadedStrings, QuasiQuotes, NoImplicitPrelude
ghc-options: -threaded -rtsopts -with-rtsopts=-N
Hs-Source-Dirs: test
Main-Is: Main.hs
Other-Modules: Feature.AudienceJwtSecretSpec
, Feature.AuthSpec
, Feature.AsymmetricJwtSpec
, Feature.BinaryJwtSecretSpec
, Feature.ConcurrentSpec
, Feature.CorsSpec
, Feature.DeleteSpec
, Feature.ExtraSearchPathSpec
, Feature.InsertSpec
, Feature.JsonOperatorSpec
, Feature.NoJwtSpec
, Feature.PgVersion95Spec
, Feature.PgVersion96Spec
, Feature.ProxySpec
, Feature.QueryLimitedSpec
, Feature.QuerySpec
, Feature.RangeSpec
, Feature.SingularSpec
, Feature.StructureSpec
, Feature.UnicodeSpec
, Feature.AndOrParamsSpec
, Feature.RpcSpec
, Feature.NonexistentSchemaSpec
, Feature.UpsertSpec
, SpecHelper
, TestTypes
Build-Depends: aeson
, aeson-qq
, async
, auto-update
, base >= 4.8 && < 4.10
, bytestring
, base64-bytestring
, case-insensitive
, cassava
, containers
, contravariant
, hasql >= 1.3 && < 1.4
, hasql-pool >= 0.5 && < 0.6
, hasql-transaction >= 0.7 && < 0.8
, heredoc
, hjsonschema == 1.5.0.1
, hspec
, hspec-wai >= 0.7.0
, hspec-wai-json
, http-types
, lens
, lens-aeson
, monad-control
, postgrest
, process
, protolude == 0.2.2
, regex-tdfa
, time
, transformers-base
, wai
, wai-extra
if !os(windows)
build-depends:
unix
, directory >= 1.2.6 && < 1.4
exposed-modules:
PostgREST.Unix
executable postgrest
default-language: Haskell2010
default-extensions: OverloadedStrings
NoImplicitPrelude
hs-source-dirs: main
main-is: Main.hs
build-depends: base >= 4.9 && < 4.17
, containers >= 0.5.7 && < 0.7
, postgrest
, protolude >= 0.3.1 && < 0.4
ghc-options: -threaded -rtsopts "-with-rtsopts=-N -I0 -qg"
-O2 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
if flag(dev)
ghc-options: -O0 -fwrite-ide-info
-- https://github.com/PostgREST/postgrest/issues/387
-with-rtsopts=-K1K
if flag(hpc)
ghc-options: -fhpc -hpcdir .hpc
else
ghc-options: -O2
test-suite spec
type: exitcode-stdio-1.0
default-language: Haskell2010
default-extensions: OverloadedStrings
QuasiQuotes
NoImplicitPrelude
hs-source-dirs: test/spec
main-is: Main.hs
other-modules: Feature.Auth.AsymmetricJwtSpec
Feature.Auth.AudienceJwtSecretSpec
Feature.Auth.AuthSpec
Feature.Auth.BinaryJwtSecretSpec
Feature.Auth.NoAnonSpec
Feature.Auth.NoJwtSpec
Feature.ConcurrentSpec
Feature.CorsSpec
Feature.ExtraSearchPathSpec
Feature.LegacyGucsSpec
Feature.OpenApi.DisabledOpenApiSpec
Feature.OpenApi.IgnorePrivOpenApiSpec
Feature.OpenApi.OpenApiSpec
Feature.OpenApi.ProxySpec
Feature.OpenApi.RootSpec
Feature.OpenApi.SecurityOpenApiSpec
Feature.OptionsSpec
Feature.Query.AndOrParamsSpec
Feature.Query.ComputedRelsSpec
Feature.Query.DeleteSpec
Feature.Query.EmbedDisambiguationSpec
Feature.Query.EmbedInnerJoinSpec
Feature.Query.PlanSpec
Feature.Query.HtmlRawOutputSpec
Feature.Query.InsertSpec
Feature.Query.JsonOperatorSpec
Feature.Query.MultipleSchemaSpec
Feature.Query.ErrorSpec
Feature.Query.PgSafeUpdateSpec
Feature.Query.PostGISSpec
Feature.Query.QueryLimitedSpec
Feature.Query.QuerySpec
Feature.Query.RangeSpec
Feature.Query.RawOutputTypesSpec
Feature.Query.RpcSpec
Feature.Query.SingularSpec
Feature.Query.UnicodeSpec
Feature.Query.UpdateSpec
Feature.Query.UpsertSpec
Feature.RollbackSpec
Feature.RpcPreRequestGucsSpec
SpecHelper
TestTypes
build-depends: base >= 4.9 && < 4.17
, aeson >= 2.0.3 && < 2.1
, aeson-qq >= 0.8.1 && < 0.9
, async >= 2.1.1 && < 2.3
, auto-update >= 0.1.4 && < 0.2
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.12
, case-insensitive >= 1.2 && < 1.3
, containers >= 0.5.7 && < 0.7
, hasql-pool >= 0.5 && < 0.6
, hasql-transaction >= 1.0.1 && < 1.1
, heredoc >= 0.2 && < 0.3
, hspec >= 2.3 && < 2.9
, hspec-wai >= 0.10 && < 0.12
, hspec-wai-json >= 0.10 && < 0.12
, http-types >= 0.12.3 && < 0.13
, lens >= 4.14 && < 5.2
, lens-aeson >= 1.0.1 && < 1.2
, monad-control >= 1.0.1 && < 1.1
, postgrest
, process >= 1.4.2 && < 1.7
, protolude >= 0.3.1 && < 0.4
, regex-tdfa >= 1.2.2 && < 1.4
, text >= 1.2.2 && < 1.3
, transformers-base >= 0.4.4 && < 0.5
, wai >= 3.2.1 && < 3.3
, wai-extra >= 3.0.19 && < 3.2
ghc-options: -O0 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
-fno-warn-missing-signatures
-fwrite-ide-info
-- https://github.com/PostgREST/postgrest/issues/387
-with-rtsopts=-K33K
test-suite querycost
type: exitcode-stdio-1.0
default-language: Haskell2010
default-extensions: OverloadedStrings
QuasiQuotes
NoImplicitPrelude
hs-source-dirs: test/spec
main-is: QueryCost.hs
other-modules: SpecHelper
build-depends: base >= 4.9 && < 4.17
, aeson >= 2.0.3 && < 2.1
, base64-bytestring >= 1 && < 1.3
, bytestring >= 0.10.8 && < 0.12
, case-insensitive >= 1.2 && < 1.3
, containers >= 0.5.7 && < 0.7
, contravariant >= 1.4 && < 1.6
, hasql >= 1.4 && < 1.6
, hasql-dynamic-statements >= 0.3.1 && < 0.4
, hasql-pool >= 0.5 && < 0.6
, hasql-transaction >= 1.0.1 && < 1.1
, heredoc >= 0.2 && < 0.3
, hspec >= 2.3 && < 2.9
, hspec-wai >= 0.10 && < 0.12
, hspec-wai-json >= 0.10 && < 0.12
, http-types >= 0.12.3 && < 0.13
, lens >= 4.14 && < 5.2
, lens-aeson >= 1.0.1 && < 1.2
, postgrest
, process >= 1.4.2 && < 1.7
, protolude >= 0.3.1 && < 0.4
, regex-tdfa >= 1.2.2 && < 1.4
, wai-extra >= 3.0.19 && < 3.2
ghc-options: -O0 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
-fwrite-ide-info
-- https://github.com/PostgREST/postgrest/issues/387
-with-rtsopts=-K1K
test-suite doctests
type: exitcode-stdio-1.0
default-language: Haskell2010
default-extensions: OverloadedStrings
NoImplicitPrelude
hs-source-dirs: test/doc
main-is: Main.hs
build-depends: base >= 4.9 && < 4.17
, doctest >= 0.8
, postgrest
, pretty-simple
, protolude >= 0.3.1 && < 0.4
ghc-options: -threaded -O0 -Werror -Wall -fwarn-identities
-fno-spec-constr -optP-Wno-nonportable-include-path
-10
View File
@@ -1,10 +0,0 @@
export POSTGREST_VER=`grep ^version /app/postgrest.cabal | sed -En 's/.*\s+([0-9\.]+)/\1/p'`
curl -L http://sourceforge.net/projects/s3tools/files/s3cmd/1.5.0-alpha1/s3cmd-1.5.0-alpha1.tar.gz | tar zx
cp /app/dist/build/postgrest/postgrest postgrest-${POSTGREST_VER}
tar cJf postgrest-${POSTGREST_VER}.tar.xz postgrest-${POSTGREST_VER}
touch ~/.s3cfg
s3cmd-1.5.0-alpha1/s3cmd put --access_key=${S3_ACCESS_KEY} --secret_key=${S3_SECRET_KEY} -P -f postgrest-${POSTGREST_VER}.tar.xz $S3_BUCKET/postgrest-${POSTGREST_VER}.tar.xz
+62
View File
@@ -0,0 +1,62 @@
# The additional modules below have large dependencies and are therefore
# disabled by default. You can activate them by passing arguments to nix-shell,
# e.g.:
#
# nix-shell --arg docker true
#
# We highly recommend that use the PostgREST binary cache by installing cachix
# (https://app.cachix.org/) and running `cachix use postgrest`.
{ docker ? false
, memory ? false
}:
let
postgrest =
import ./default.nix { };
inherit (postgrest) pkgs;
inherit (pkgs) lib;
toolboxes =
[
postgrest.cabalTools
postgrest.devTools
postgrest.loadtest
postgrest.nixpkgsTools
postgrest.style
postgrest.tests
postgrest.withTools
postgrest.release
]
++ lib.optional docker postgrest.docker
++ lib.optional memory postgrest.memory;
in
lib.overrideDerivation postgrest.env (
base: {
buildInputs =
base.buildInputs ++ [
pkgs.cabal-install
pkgs.cabal2nix
pkgs.git
pkgs.postgresql
postgrest.hsie.bin
]
++ toolboxes;
shellHook =
''
export HISTFILE=.history
source ${pkgs.bash-completion}/etc/profile.d/bash_completion.sh
source ${pkgs.git}/share/git/contrib/completion/git-completion.bash
source ${postgrest.hsie.bash-completion}
''
+ builtins.concatStringsSep "\n" (
builtins.map (bash-completion: "source ${bash-completion}") (
builtins.concatLists (builtins.map (toolbox: toolbox.bash-completion) toolboxes)
)
);
}
)
+74
View File
@@ -0,0 +1,74 @@
{-# LANGUAGE RecordWildCards #-}
module PostgREST.Admin
( postgrestAdmin
) where
import qualified Data.Text as T
import Network.Socket
import Network.Socket.ByteString
import qualified Network.HTTP.Types.Status as HTTP
import qualified Network.Wai as Wai
import qualified Hasql.Session as SQL
import qualified PostgREST.AppState as AppState
import PostgREST.Config (AppConfig (..))
import Protolude
-- | PostgREST admin application
postgrestAdmin :: AppState.AppState -> AppConfig -> Wai.Application
postgrestAdmin appState appConfig req respond = do
isMainAppReachable <- any isRight <$> reachMainApp appConfig
isSchemaCacheLoaded <- isJust <$> AppState.getDbStructure appState
isConnectionUp <-
if configDbChannelEnabled appConfig
then AppState.getIsListenerOn appState
else isRight <$> AppState.usePool appState (SQL.sql "SELECT 1")
case Wai.pathInfo req of
["ready"] ->
respond $ Wai.responseLBS (if isMainAppReachable && isConnectionUp && isSchemaCacheLoaded then HTTP.status200 else HTTP.status503) [] mempty
["live"] ->
respond $ Wai.responseLBS (if isMainAppReachable then HTTP.status200 else HTTP.status503) [] mempty
_ ->
respond $ Wai.responseLBS HTTP.status404 [] mempty
-- Try to connect to the main app socket
-- Note that it doesn't even send a valid HTTP request, we just want to check that the main app is accepting connections
-- The code for resolving the "*4", "!4", "*6", "!6", "*" special values is taken from
-- https://hackage.haskell.org/package/streaming-commons-0.2.2.4/docs/src/Data.Streaming.Network.html#bindPortGenEx
reachMainApp :: AppConfig -> IO [Either IOException ()]
reachMainApp AppConfig{..} =
case configServerUnixSocket of
Just path -> do
sock <- socket AF_UNIX Stream 0
(:[]) <$> try (do
connect sock $ SockAddrUnix path
withSocketsDo $ bracket (pure sock) close sendEmpty)
Nothing -> do
let
host | configServerHost `elem` ["*4", "!4", "*6", "!6", "*"] = Nothing
| otherwise = Just configServerHost
filterAddrs xs =
case configServerHost of
"*4" -> ipv4Addrs xs ++ ipv6Addrs xs
"!4" -> ipv4Addrs xs
"*6" -> ipv6Addrs xs ++ ipv4Addrs xs
"!6" -> ipv6Addrs xs
_ -> xs
ipv4Addrs = filter ((/=) AF_INET6 . addrFamily)
ipv6Addrs = filter ((==) AF_INET6 . addrFamily)
addrs <- getAddrInfo (Just $ defaultHints { addrSocketType = Stream }) (T.unpack <$> host) (Just . show $ configServerPort)
tryAddr `traverse` filterAddrs addrs
where
sendEmpty sock = void $ send sock mempty
tryAddr :: AddrInfo -> IO (Either IOException ())
tryAddr addr = do
sock <- socket (addrFamily addr) (addrSocketType addr) (addrProtocol addr)
try $ do
connect sock $ addrAddress addr
withSocketsDo $ bracket (pure sock) close sendEmpty
-304
View File
@@ -1,304 +0,0 @@
{-# LANGUAGE LambdaCase #-}
{-|
Module : PostgREST.ApiRequest
Description : PostgREST functions to translate HTTP request to a domain type called ApiRequest.
-}
module PostgREST.ApiRequest ( ApiRequest(..)
, ContentType(..)
, Action(..)
, Target(..)
, PreferRepresentation (..)
, mutuallyAgreeable
, userApiRequest
) where
import Protolude
import qualified Data.Aeson as JSON
import Data.Aeson.Types (emptyObject, emptyArray)
import qualified Data.ByteString as BS
import qualified Data.ByteString.Internal as BS (c2w)
import qualified Data.ByteString.Lazy as BL
import qualified Data.Csv as CSV
import qualified Data.List as L
import Data.List (lookup, last, partition)
import qualified Data.HashMap.Strict as M
import qualified Data.Set as S
import Data.Maybe (fromJust)
import Control.Arrow ((***))
import qualified Data.Text as T
import qualified Data.Vector as V
import Network.HTTP.Base (urlEncodeVars)
import Network.HTTP.Types.Header (hAuthorization, hCookie)
import Network.HTTP.Types.URI (parseSimpleQuery)
import Network.Wai (Request (..))
import Network.Wai.Parse (parseHttpAccept)
import PostgREST.RangeQuery (NonnegRange, rangeRequested, restrictRange, rangeGeq, allRange, rangeLimit, rangeOffset)
import Data.Ranged.Boundaries
import PostgREST.Types
import Data.Ranged.Ranges (Range(..), rangeIntersection, emptyRange)
import qualified Data.CaseInsensitive as CI
import Web.Cookie (parseCookiesText)
type RequestBody = BL.ByteString
-- | Types of things a user wants to do to tables/views/procs
data Action = ActionCreate | ActionRead
| ActionUpdate | ActionDelete
| ActionInfo | ActionInvoke{isReadOnly :: Bool}
| ActionInspect | ActionSingleUpsert
deriving Eq
-- | The target db object of a user action
data Target = TargetIdent QualifiedIdentifier
| TargetProc QualifiedIdentifier
| TargetRoot
| TargetUnknown [Text]
deriving Eq
-- | How to return the inserted data
data PreferRepresentation = Full | HeadersOnly | None deriving Eq
{-|
Describes what the user wants to do. This data type is a
translation of the raw elements of an HTTP request into domain
specific language. There is no guarantee that the intent is
sensible, it is up to a later stage of processing to determine
if it is an action we are able to perform.
-}
data ApiRequest = ApiRequest {
-- | Similar but not identical to HTTP verb, e.g. Create/Invoke both POST
iAction :: Action
-- | Requested range of rows within response
, iRange :: M.HashMap ByteString NonnegRange
-- | The target, be it calling a proc or accessing a table
, iTarget :: Target
-- | Content types the client will accept, [CTAny] if no Accept header
, iAccepts :: [ContentType]
-- | Data sent by client and used for mutation actions
, iPayload :: Maybe PayloadJSON
-- | If client wants created items echoed back
, iPreferRepresentation :: PreferRepresentation
-- | Pass all parameters as a single json object to a stored procedure
, iPreferSingleObjectParameter :: Bool
-- | Whether the client wants a result count (slower)
, iPreferCount :: Bool
-- | Whether the client wants to UPSERT or ignore records on PK conflict
, iPreferResolution :: Maybe PreferResolution
-- | Filters on the result ("id", "eq.10")
, iFilters :: [(Text, Text)]
-- | &and and &or parameters used for complex boolean logic
, iLogic :: [(Text, Text)]
-- | &select parameter used to shape the response
, iSelect :: Text
-- | &order parameters for each level
, iOrder :: [(Text, Text)]
-- | Alphabetized (canonical) request query string for response URLs
, iCanonicalQS :: ByteString
-- | JSON Web Token
, iJWT :: Text
-- | HTTP request headers
, iHeaders :: [(Text, Text)]
-- | Request Cookies
, iCookies :: [(Text, Text)]
}
-- | Examines HTTP request and translates it into user intent.
userApiRequest :: Schema -> Request -> RequestBody -> Either ApiRequestError ApiRequest
userApiRequest schema req reqBody
| isTargetingProc && method `notElem` ["GET", "POST"] = Left ActionInappropriate
| topLevelRange == emptyRange = Left InvalidRange
| shouldParsePayload && isLeft payload = either (Left . InvalidBody . toS) witness payload
| otherwise = Right ApiRequest {
iAction = action
, iTarget = target
, iRange = ranges
, iAccepts = maybe [CTAny] (map decodeContentType . parseHttpAccept) $ lookupHeader "accept"
, iPayload = relevantPayload
, iPreferRepresentation = representation
, iPreferSingleObjectParameter = singleObject
, iPreferCount = hasPrefer "count=exact"
, iPreferResolution = if hasPrefer (show MergeDuplicates) then Just MergeDuplicates
else if hasPrefer (show IgnoreDuplicates) then Just IgnoreDuplicates
else Nothing
, iFilters = filters
, iLogic = [(toS k, toS $ fromJust v) | (k,v) <- qParams, isJust v, endingIn ["and", "or"] k ]
, iSelect = toS $ fromMaybe "*" $ fromMaybe (Just "*") $ lookup "select" qParams
, iOrder = [(toS k, toS $ fromJust v) | (k,v) <- qParams, isJust v, endingIn ["order"] k ]
, iCanonicalQS = toS $ urlEncodeVars
. L.sortBy (comparing fst)
. map (join (***) toS)
. parseSimpleQuery
$ rawQueryString req
, iJWT = tokenStr
, iHeaders = [ (toS $ CI.foldedCase k, toS v) | (k,v) <- hdrs, k /= hAuthorization, k /= hCookie]
, iCookies = maybe [] parseCookiesText $ lookupHeader "Cookie"
}
where
-- rpcQParams = Rpc query params e.g. /rpc/name?param1=val1, similar to filter but with no operator(eq, lt..)
(filters, rpcQParams) =
case action of
ActionInvoke{isReadOnly=True} -> partition (liftM2 (||) (isEmbedPath . fst) (hasOperator . snd)) flts
_ -> (flts, [])
flts = [ (toS k, toS $ fromJust v) | (k,v) <- qParams, isJust v, k /= "select", not (endingIn ["order", "limit", "offset", "and", "or"] k) ]
hasOperator val = any (`T.isPrefixOf` val) $
((<> ".") <$> "not":M.keys operators) ++
((<> "(") <$> M.keys ftsOperators)
isEmbedPath = T.isInfixOf "."
isTargetingProc = (== Just "rpc") $ listToMaybe path
payload =
case (decodeContentType . fromMaybe "application/json" $ lookupHeader "content-type", action) of
(_, ActionInvoke{isReadOnly=True}) ->
Right $ PayloadJSON (JSON.encode $ M.fromList $ second JSON.toJSON <$> rpcQParams) PJObject (S.fromList $ fst <$> rpcQParams)
(CTApplicationJSON, _) ->
note "All object keys must match" . payloadAttributes reqBody
=<< if BL.null reqBody && isTargetingProc
then Right emptyObject
else JSON.eitherDecode reqBody
(CTTextCSV, _) -> do
json <- csvToJson <$> CSV.decodeByName reqBody
note "All lines must have same number of fields" $ payloadAttributes (JSON.encode json) json
(CTOther "application/x-www-form-urlencoded", _) ->
let json = M.fromList . map (toS *** JSON.String . toS) . parseSimpleQuery $ toS reqBody
keys = S.fromList $ M.keys json in
Right $ PayloadJSON (JSON.encode json) PJObject keys
(ct, _) ->
Left $ toS $ "Content-Type not acceptable: " <> toMime ct
topLevelRange = fromMaybe allRange $ M.lookup "limit" ranges
action =
case method of
"GET" | target == TargetRoot -> ActionInspect
| isTargetingProc -> ActionInvoke{isReadOnly=True}
| otherwise -> ActionRead
"POST" -> if isTargetingProc
then ActionInvoke{isReadOnly=False}
else ActionCreate
"PATCH" -> ActionUpdate
"PUT" -> ActionSingleUpsert
"DELETE" -> ActionDelete
"OPTIONS" -> ActionInfo
_ -> ActionInspect
target = case path of
[] -> TargetRoot
[table] -> TargetIdent
$ QualifiedIdentifier schema table
["rpc", proc] -> TargetProc
$ QualifiedIdentifier schema proc
other -> TargetUnknown other
shouldParsePayload = action `elem` [ActionCreate, ActionUpdate, ActionSingleUpsert, ActionInvoke{isReadOnly=False}, ActionInvoke{isReadOnly=True}]
relevantPayload | shouldParsePayload = rightToMaybe payload
| otherwise = Nothing
path = pathInfo req
method = requestMethod req
hdrs = requestHeaders req
qParams = [(toS k, v)|(k,v) <- queryString req]
lookupHeader = flip lookup hdrs
hasPrefer :: Text -> Bool
hasPrefer val = any (\(h,v) -> h == "Prefer" && val `elem` split v) hdrs
where
split :: BS.ByteString -> [Text]
split = map T.strip . T.split (==',') . toS
singleObject = hasPrefer "params=single-object"
representation
| hasPrefer "return=representation" = Full
| hasPrefer "return=minimal" = None
| otherwise = HeadersOnly
auth = fromMaybe "" $ lookupHeader hAuthorization
tokenStr = case T.split (== ' ') (toS auth) of
("Bearer" : t : _) -> t
_ -> ""
endingIn:: [Text] -> Text -> Bool
endingIn xx key = lastWord `elem` xx
where lastWord = last $ T.split (=='.') key
headerRange = rangeRequested hdrs
replaceLast x s = T.intercalate "." $ L.init (T.split (=='.') s) ++ [x]
limitParams :: M.HashMap ByteString NonnegRange
limitParams = M.fromList [(toS (replaceLast "limit" k), restrictRange (readMaybe =<< (toS <$> v)) allRange) | (k,v) <- qParams, isJust v, endingIn ["limit"] k]
offsetParams :: M.HashMap ByteString NonnegRange
offsetParams = M.fromList [(toS (replaceLast "limit" k), maybe allRange rangeGeq (readMaybe =<< (toS <$> v))) | (k,v) <- qParams, isJust v, endingIn ["offset"] k]
urlRange = M.unionWith f limitParams offsetParams
where
f rl ro = Range (BoundaryBelow o) (BoundaryAbove $ o + l - 1)
where
l = fromMaybe 0 $ rangeLimit rl
o = rangeOffset ro
ranges = M.insert "limit" (rangeIntersection headerRange (fromMaybe allRange (M.lookup "limit" urlRange))) urlRange
{-|
Find the best match from a list of content types accepted by the
client in order of decreasing preference and a list of types
producible by the server. If there is no match but the client
accepts */* then return the top server pick.
-}
mutuallyAgreeable :: [ContentType] -> [ContentType] -> Maybe ContentType
mutuallyAgreeable sProduces cAccepts =
let exact = listToMaybe $ L.intersect cAccepts sProduces in
if isNothing exact && CTAny `elem` cAccepts
then listToMaybe sProduces
else exact
-- PRIVATE ---------------------------------------------------------------
{-|
Warning: discards MIME parameters
-}
decodeContentType :: BS.ByteString -> ContentType
decodeContentType ct =
case BS.takeWhile (/= BS.c2w ';') ct of
"application/json" -> CTApplicationJSON
"text/csv" -> CTTextCSV
"application/openapi+json" -> CTOpenAPI
"application/vnd.pgrst.object+json" -> CTSingularJSON
"application/vnd.pgrst.object" -> CTSingularJSON
"application/octet-stream" -> CTOctetStream
"*/*" -> CTAny
ct' -> CTOther ct'
type CsvData = V.Vector (M.HashMap Text BL.ByteString)
{-|
Converts CSV like
a,b
1,hi
2,bye
into a JSON array like
[ {"a": "1", "b": "hi"}, {"a": 2, "b": "bye"} ]
The reason for its odd signature is so that it can compose
directly with CSV.decodeByName
-}
csvToJson :: (CSV.Header, CsvData) -> JSON.Value
csvToJson (_, vals) =
JSON.Array $ V.map rowToJsonObj vals
where
rowToJsonObj = JSON.Object .
M.map (\str ->
if str == "NULL"
then JSON.Null
else JSON.String $ toS str
)
payloadAttributes :: RequestBody -> JSON.Value -> Maybe PayloadJSON
payloadAttributes raw json =
-- Test that Array contains only Objects having the same keys
case json of
JSON.Array arr ->
case arr V.!? 0 of
Just (JSON.Object o) ->
let canonicalKeys = S.fromList $ M.keys o
areKeysUniform = all (\case
JSON.Object x -> S.fromList (M.keys x) == canonicalKeys
_ -> False) arr in
if areKeysUniform
then Just $ PayloadJSON raw (PJArray $ V.length arr) canonicalKeys
else Nothing
Just _ -> Nothing
Nothing -> Just emptyPJArray
JSON.Object o -> Just $ PayloadJSON raw PJObject (S.fromList $ M.keys o)
-- truncate everything else to an empty array.
_ -> Just emptyPJArray
where
emptyPJArray = PayloadJSON (JSON.encode emptyArray) (PJArray 0) S.empty
+626 -371
View File
File diff suppressed because it is too large Load Diff
+166
View File
@@ -0,0 +1,166 @@
{-# LANGUAGE RecordWildCards #-}
module PostgREST.AppState
( AppState
, destroy
, getConfig
, getDbStructure
, getIsListenerOn
, getJsonDbS
, getMainThreadId
, getPgVersion
, getRetryNextIn
, getTime
, getWorkerSem
, init
, initWithPool
, logWithZTime
, putConfig
, putDbStructure
, putIsListenerOn
, putJsonDbS
, putPgVersion
, putRetryNextIn
, releasePool
, signalListener
, usePool
, waitListener
) where
import qualified Hasql.Pool as SQL
import qualified Hasql.Session as SQL
import Control.AutoUpdate (defaultUpdateSettings, mkAutoUpdate,
updateAction)
import Data.IORef (IORef, atomicWriteIORef, newIORef,
readIORef)
import Data.Time (ZonedTime, defaultTimeLocale, formatTime,
getZonedTime)
import Data.Time.Clock (UTCTime, getCurrentTime)
import PostgREST.Config (AppConfig (..))
import PostgREST.Config.PgVersion (PgVersion (..), minimumPgVersion)
import PostgREST.DbStructure (DbStructure)
import Protolude
data AppState = AppState
{ statePool :: SQL.Pool -- | Connection pool, either a 'Connection' or a 'ConnectionError'
, statePgVersion :: IORef PgVersion
-- | No schema cache at the start. Will be filled in by the connectionWorker
, stateDbStructure :: IORef (Maybe DbStructure)
-- | Cached DbStructure in json
, stateJsonDbS :: IORef ByteString
-- | Binary semaphore to make sure just one connectionWorker can run at a time
, stateWorkerSem :: MVar ()
-- | Binary semaphore used to sync the listener(NOTIFY reload) with the connectionWorker.
, stateListener :: MVar ()
-- | State of the LISTEN channel, used for the admin server checks
, stateIsListenerOn :: IORef Bool
-- | Config that can change at runtime
, stateConf :: IORef AppConfig
-- | Time used for verifying JWT expiration
, stateGetTime :: IO UTCTime
-- | Time with time zone used for worker logs
, stateGetZTime :: IO ZonedTime
-- | Used for killing the main thread in case a subthread fails
, stateMainThreadId :: ThreadId
-- | Keeps track of when the next retry for connecting to database is scheduled
, stateRetryNextIn :: IORef Int
}
init :: AppConfig -> IO AppState
init conf = do
newPool <- initPool conf
initWithPool newPool conf
initWithPool :: SQL.Pool -> AppConfig -> IO AppState
initWithPool newPool conf =
AppState newPool
<$> newIORef minimumPgVersion -- assume we're in a supported version when starting, this will be corrected on a later step
<*> newIORef Nothing
<*> newIORef mempty
<*> newEmptyMVar
<*> newEmptyMVar
<*> newIORef False
<*> newIORef conf
<*> mkAutoUpdate defaultUpdateSettings { updateAction = getCurrentTime }
<*> mkAutoUpdate defaultUpdateSettings { updateAction = getZonedTime }
<*> myThreadId
<*> newIORef 0
destroy :: AppState -> IO ()
destroy = releasePool
initPool :: AppConfig -> IO SQL.Pool
initPool AppConfig{..} =
SQL.acquire (configDbPoolSize, configDbPoolTimeout, toUtf8 configDbUri)
usePool :: AppState -> SQL.Session a -> IO (Either SQL.UsageError a)
usePool AppState{..} = SQL.use statePool
releasePool :: AppState -> IO ()
releasePool AppState{..} = SQL.release statePool
getPgVersion :: AppState -> IO PgVersion
getPgVersion = readIORef . statePgVersion
putPgVersion :: AppState -> PgVersion -> IO ()
putPgVersion = atomicWriteIORef . statePgVersion
getDbStructure :: AppState -> IO (Maybe DbStructure)
getDbStructure = readIORef . stateDbStructure
putDbStructure :: AppState -> Maybe DbStructure -> IO ()
putDbStructure appState = atomicWriteIORef (stateDbStructure appState)
getJsonDbS :: AppState -> IO ByteString
getJsonDbS = readIORef . stateJsonDbS
putJsonDbS :: AppState -> ByteString -> IO ()
putJsonDbS appState = atomicWriteIORef (stateJsonDbS appState)
getWorkerSem :: AppState -> MVar ()
getWorkerSem = stateWorkerSem
getRetryNextIn :: AppState -> IO Int
getRetryNextIn = readIORef . stateRetryNextIn
putRetryNextIn :: AppState -> Int -> IO ()
putRetryNextIn = atomicWriteIORef . stateRetryNextIn
getConfig :: AppState -> IO AppConfig
getConfig = readIORef . stateConf
putConfig :: AppState -> AppConfig -> IO ()
putConfig = atomicWriteIORef . stateConf
getTime :: AppState -> IO UTCTime
getTime = stateGetTime
-- | Log to stderr with local time
logWithZTime :: AppState -> Text -> IO ()
logWithZTime appState txt = do
zTime <- stateGetZTime appState
hPutStrLn stderr $ toS (formatTime defaultTimeLocale "%d/%b/%Y:%T %z: " zTime) <> txt
getMainThreadId :: AppState -> ThreadId
getMainThreadId = stateMainThreadId
-- | As this IO action uses `takeMVar` internally, it will only return once
-- `stateListener` has been set using `signalListener`. This is currently used
-- to syncronize workers.
waitListener :: AppState -> IO ()
waitListener = takeMVar . stateListener
-- tryPutMVar doesn't lock the thread. It should always succeed since
-- the connectionWorker is the only mvar producer.
signalListener :: AppState -> IO ()
signalListener appState = void $ tryPutMVar (stateListener appState) ()
getIsListenerOn :: AppState -> IO Bool
getIsListenerOn = readIORef . stateIsListenerOn
putIsListenerOn :: AppState -> Bool -> IO ()
putIsListenerOn = atomicWriteIORef . stateIsListenerOn
+98 -88
View File
@@ -1,5 +1,3 @@
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE LambdaCase #-}
{-|
Module : PostgREST.Auth
Description : PostgREST authorization functions.
@@ -12,100 +10,112 @@ Authentication should always be implemented in an external service.
In the test suite there is an example of simple login function that can be used for a
very simple authentication system inside the PostgreSQL database.
-}
module PostgREST.Auth (
containsRole
, jwtClaims
, JWTAttempt(..)
, parseSecret
{-# LANGUAGE RecordWildCards #-}
module PostgREST.Auth
( AuthResult (..)
, getResult
, getRole
, middleware
) where
import Control.Lens.Operators
import Control.Lens (set)
import qualified Data.Aeson as JSON
import qualified Data.HashMap.Strict as M
import Data.Time.Clock (UTCTime)
import Data.Vector as V
import PostgREST.Types
import Protolude
import qualified Crypto.JWT as JWT
import qualified Data.Aeson as JSON
import qualified Data.Aeson.Key as K
import qualified Data.Aeson.KeyMap as KM
import qualified Data.Aeson.Types as JSON
import qualified Data.ByteString.Lazy.Char8 as LBS
import qualified Data.Text.Encoding as T
import qualified Data.Vault.Lazy as Vault
import qualified Data.Vector as V
import qualified Network.HTTP.Types.Header as HTTP
import qualified Network.Wai as Wai
import qualified Network.Wai.Middleware.HttpAuth as Wai
import qualified Crypto.JOSE.Types as JOSE.Types
import Crypto.JWT
import Control.Lens (set)
import Control.Monad.Except (liftEither)
import Data.Either.Combinators (mapLeft)
import Data.List (lookup)
import Data.Time.Clock (UTCTime)
import System.IO.Unsafe (unsafePerformIO)
{-|
Possible situations encountered with client JWTs
-}
data JWTAttempt = JWTInvalid JWTError
| JWTMissingSecret
| JWTClaims (M.HashMap Text JSON.Value)
deriving (Eq, Show)
import PostgREST.AppState (AppState, getConfig, getTime)
import PostgREST.Config (AppConfig (..), JSPath, JSPathExp (..))
import PostgREST.Error (Error (..))
{-|
Receives the JWT secret and audience (from config) and a JWT and returns a map
of JWT claims.
-}
jwtClaims :: Maybe JWKSet -> Maybe StringOrURI -> LByteString -> UTCTime -> Maybe JSPath -> IO JWTAttempt
jwtClaims _ _ "" _ _ = return $ JWTClaims M.empty
jwtClaims secret audience payload time jspath =
case secret of
Nothing -> return JWTMissingSecret
Just s -> do
let validation = set allowedSkew 1 $ defaultJWTValidationSettings (maybe (const True) (==) audience)
eJwt <- runExceptT $ do
jwt <- decodeCompact payload
verifyClaimsAt validation s time jwt
return $ case eJwt of
Left e -> JWTInvalid e
Right jwt -> JWTClaims $ claims2map jwt jspath
import Protolude
{-|
Turn JWT ClaimSet into something easier to work with,
also here the jspath is applied to put the "role" in the map
-}
claims2map :: ClaimsSet -> Maybe JSPath -> M.HashMap Text JSON.Value
claims2map claims jspath = (\case
val@(JSON.Object o) ->
let role = maybe M.empty (M.singleton "role") $
walkJSPath (Just val) =<< jspath in
M.delete "role" o `M.union` role -- mutating the map
_ -> M.empty
) $ JSON.toJSON claims
walkJSPath :: Maybe JSON.Value -> JSPath -> Maybe JSON.Value
walkJSPath x [] = x
walkJSPath (Just (JSON.Object o)) (JSPKey key:rest) = walkJSPath (M.lookup key o) rest
walkJSPath (Just (JSON.Array ar)) (JSPIdx idx:rest) = walkJSPath (ar V.!? idx) rest
walkJSPath _ _ = Nothing
data AuthResult = AuthResult
{ authClaims :: KM.KeyMap JSON.Value
, authRole :: Text
}
{-|
Whether a response from jwtClaims contains a role claim
-}
containsRole :: JWTAttempt -> Bool
containsRole (JWTClaims claims) = M.member "role" claims
containsRole _ = False
-- | Receives the JWT secret and audience (from config) and a JWT and returns a
-- JSON object of JWT claims.
parseToken :: Monad m =>
AppConfig -> LByteString -> UTCTime -> ExceptT Error m JSON.Value
parseToken _ "" _ = return JSON.emptyObject
parseToken AppConfig{..} token time = do
secret <- liftEither . maybeToRight JwtTokenMissing $ configJWKS
eitherClaims <-
lift . runExceptT $
JWT.verifyClaimsAt validation secret time =<< JWT.decodeCompact token
liftEither . mapLeft jwtClaimsError $ JSON.toJSON <$> eitherClaims
where
validation =
JWT.defaultJWTValidationSettings audienceCheck & set JWT.allowedSkew 1
{-|
Parse `jwt-secret` configuration option and turn into a JWKSet.
audienceCheck :: JWT.StringOrURI -> Bool
audienceCheck = maybe (const True) (==) configJwtAudience
There are three ways to specify `jwt-secret`: text secret, JSON Web Key
(JWK), or JSON Web Key Set (JWKS). The first two are converted into a JWKSet
with one key and the last is converted as is.
-}
parseSecret :: ByteString -> JWKSet
parseSecret str =
fromMaybe (maybe secret (\jwk' -> JWKSet [jwk']) maybeJWK)
maybeJWKSet
where
maybeJWKSet = JSON.decode (toS str) :: Maybe JWKSet
maybeJWK = JSON.decode (toS str) :: Maybe JWK
secret = JWKSet [jwkFromSecret str]
jwtClaimsError :: JWT.JWTError -> Error
jwtClaimsError JWT.JWTExpired = JwtTokenInvalid "JWT expired"
jwtClaimsError e = JwtTokenInvalid $ show e
{-|
Internal helper to generate a symmetric HMAC-SHA256 JWK from a text secret.
-}
jwkFromSecret :: ByteString -> JWK
jwkFromSecret key =
fromKeyMaterial km
& jwkUse ?~ Sig
& jwkAlg ?~ JWSAlg HS256
where
km = OctKeyMaterial (OctKeyParameters (JOSE.Types.Base64Octets key))
parseClaims :: Monad m =>
AppConfig -> JSON.Value -> ExceptT Error m AuthResult
parseClaims AppConfig{..} jclaims@(JSON.Object mclaims) = do
-- role defaults to anon if not specified in jwt
role <- liftEither . maybeToRight JwtTokenRequired $
unquoted <$> walkJSPath (Just jclaims) configJwtRoleClaimKey <|> configDbAnonRole
return AuthResult
{ authClaims = mclaims & KM.insert "role" (JSON.toJSON role)
, authRole = role
}
where
walkJSPath :: Maybe JSON.Value -> JSPath -> Maybe JSON.Value
walkJSPath x [] = x
walkJSPath (Just (JSON.Object o)) (JSPKey key:rest) = walkJSPath (KM.lookup (K.fromText key) o) rest
walkJSPath (Just (JSON.Array ar)) (JSPIdx idx:rest) = walkJSPath (ar V.!? idx) rest
walkJSPath _ _ = Nothing
unquoted :: JSON.Value -> Text
unquoted (JSON.String t) = t
unquoted v = T.decodeUtf8 . LBS.toStrict $ JSON.encode v
-- impossible case - just added to please -Wincomplete-patterns
parseClaims _ _ = return AuthResult { authClaims = KM.empty, authRole = mempty }
-- | Validate authorization header.
-- Parse and store JWT claims for future use in the request.
middleware :: AppState -> Wai.Middleware
middleware appState app req respond = do
conf <- getConfig appState
time <- getTime appState
let token = fromMaybe "" $ Wai.extractBearerAuth =<< lookup HTTP.hAuthorization (Wai.requestHeaders req)
authResult <- runExceptT $
parseToken conf (LBS.fromStrict token) time >>=
parseClaims conf
let req' = req { Wai.vault = Wai.vault req & Vault.insert authResultKey authResult }
app req' respond
authResultKey :: Vault.Key (Either Error AuthResult)
authResultKey = unsafePerformIO Vault.newKey
{-# NOINLINE authResultKey #-}
getResult :: Wai.Request -> Maybe (Either Error AuthResult)
getResult = Vault.lookup authResultKey . Wai.vault
getRole :: Wai.Request -> Maybe Text
getRole req = authRole <$> (rightToMaybe =<< getResult req)
+217
View File
@@ -0,0 +1,217 @@
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE QuasiQuotes #-}
{-# LANGUAGE RecordWildCards #-}
module PostgREST.CLI
( main
, CLI (..)
, Command (..)
, readCLIShowHelp
) where
import qualified Data.Aeson as JSON
import qualified Data.ByteString.Char8 as BS
import qualified Data.ByteString.Lazy as LBS
import qualified Hasql.Transaction.Sessions as SQL
import qualified Options.Applicative as O
import Data.Text.IO (hPutStrLn)
import Text.Heredoc (str)
import PostgREST.AppState (AppState)
import PostgREST.Config (AppConfig (..))
import PostgREST.DbStructure (queryDbStructure)
import PostgREST.Version (prettyVersion)
import PostgREST.Workers (reReadConfig)
import qualified PostgREST.App as App
import qualified PostgREST.AppState as AppState
import qualified PostgREST.Config as Config
import Protolude hiding (hPutStrLn)
main :: App.SignalHandlerInstaller -> Maybe App.SocketRunner -> CLI -> IO ()
main installSignalHandlers runAppWithSocket CLI{cliCommand, cliPath} = do
conf@AppConfig{..} <-
either panic identity <$> Config.readAppConfig mempty cliPath Nothing
-- Per https://github.com/PostgREST/postgrest/issues/268, we want to
-- explicitly close the connections to PostgreSQL on shutdown.
-- 'AppState.destroy' takes care of that.
bracket
(AppState.init conf)
AppState.destroy
(\appState -> case cliCommand of
CmdDumpConfig -> do
when configDbConfig $ reReadConfig True appState
putStr . Config.toText =<< AppState.getConfig appState
CmdDumpSchema -> putStrLn =<< dumpSchema appState
CmdRun -> App.run installSignalHandlers runAppWithSocket appState)
-- | Dump DbStructure schema to JSON
dumpSchema :: AppState -> IO LBS.ByteString
dumpSchema appState = do
AppConfig{..} <- AppState.getConfig appState
result <-
let transaction = if configDbPreparedStatements then SQL.transaction else SQL.unpreparedTransaction in
AppState.usePool appState $
transaction SQL.ReadCommitted SQL.Read $
queryDbStructure
(toList configDbSchemas)
configDbExtraSearchPath
configDbPreparedStatements
case result of
Left e -> do
hPutStrLn stderr $ "An error ocurred when loading the schema cache:\n" <> show e
exitFailure
Right dbStructure -> return $ JSON.encode dbStructure
-- | Command line interface options
data CLI = CLI
{ cliCommand :: Command
, cliPath :: Maybe FilePath
}
data Command
= CmdRun
| CmdDumpConfig
| CmdDumpSchema
-- | Read command line interface options. Also prints help.
readCLIShowHelp :: IO CLI
readCLIShowHelp =
O.customExecParser prefs opts
where
prefs = O.prefs $ O.showHelpOnError <> O.showHelpOnEmpty
opts = O.info parser $ O.fullDesc <> progDesc
parser = O.helper <*> exampleParser <*> cliParser
progDesc =
O.progDesc $
"PostgREST "
<> BS.unpack prettyVersion
<> " / create a REST API to an existing Postgres database"
exampleParser =
O.infoOption exampleConfigFile $
O.long "example"
<> O.short 'e'
<> O.help "Show an example configuration file"
cliParser :: O.Parser CLI
cliParser =
CLI
<$> (dumpConfigFlag <|> dumpSchemaFlag)
<*> O.optional configFileOption
configFileOption =
O.strArgument $
O.metavar "FILENAME"
<> O.help "Path to configuration file"
dumpConfigFlag =
O.flag CmdRun CmdDumpConfig $
O.long "dump-config"
<> O.help "Dump loaded configuration and exit"
dumpSchemaFlag =
O.flag CmdRun CmdDumpSchema $
O.long "dump-schema"
<> O.help "Dump loaded schema as JSON and exit (for debugging, output structure is unstable)"
exampleConfigFile :: [Char]
exampleConfigFile =
[str|## Admin server used for checks. It's disabled by default unless a port is specified.
|# admin-server-port = 3001
|
|## The database role to use when no client authentication is provided
|# db-anon-role = "anon"
|
|## Notification channel for reloading the schema cache
|db-channel = "pgrst"
|
|## Enable or disable the notification channel
|db-channel-enabled = true
|
|## Enable in-database configuration
|db-config = true
|
|## Extra schemas to add to the search_path of every request
|db-extra-search-path = "public"
|
|## Limit rows in response
|# db-max-rows = 1000
|
|## Allow getting the EXPLAIN plan through the `Accept: application/vnd.pgrst.plan` header
|# db-plan-enabled = false
|
|## Number of open connections in the pool
|db-pool = 10
|
|## Time to live, in seconds, for an idle database pool connection
|db-pool-timeout = 3600
|
|## Stored proc to exec immediately after auth
|# db-pre-request = "stored_proc_name"
|
|## Enable or disable prepared statements. disabling is only necessary when behind a connection pooler.
|## When disabled, statements will be parametrized but won't be prepared.
|db-prepared-statements = true
|
|## The name of which database schema to expose to REST clients
|db-schemas = "public"
|
|## How to terminate database transactions
|## Possible values are:
|## commit (default)
|## Transaction is always committed, this can not be overriden
|## commit-allow-override
|## Transaction is committed, but can be overriden with Prefer tx=rollback header
|## rollback
|## Transaction is always rolled back, this can not be overriden
|## rollback-allow-override
|## Transaction is rolled back, but can be overriden with Prefer tx=commit header
|db-tx-end = "commit"
|
|## The standard connection URI format, documented at
|## https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING
|db-uri = "postgresql://"
|
|## Determine if GUC request settings for headers, cookies and jwt claims use the legacy names (string with dashes, invalid starting from PostgreSQL v14) with text values instead of the new names (string without dashes, valid on all PostgreSQL versions) with json values.
|## For PostgreSQL v14 and up, this setting will be ignored.
|db-use-legacy-gucs = true
|
|# jwt-aud = "your_audience_claim"
|
|## Jspath to the role claim key
|jwt-role-claim-key = ".role"
|
|## Choose a secret, JSON Web Key (or set) to enable JWT auth
|## (use "@filename" to load from separate file)
|# jwt-secret = "secret_with_at_least_32_characters"
|jwt-secret-is-base64 = false
|
|## Logging level, the admitted values are: crit, error, warn and info.
|log-level = "error"
|
|## Determine if the OpenAPI output should follow or ignore role privileges or be disabled entirely.
|## Admitted values: follow-privileges, ignore-privileges, disabled
|openapi-mode = "follow-privileges"
|
|## Base url for the OpenAPI output
|openapi-server-proxy-uri = ""
|
|## Content types to produce raw output
|# raw-media-types="image/png, image/jpg"
|
|server-host = "!4"
|server-port = 3000
|
|## Unix socket location
|## if specified it takes precedence over server-port
|# server-unix-socket = "/tmp/pgrst.sock"
|
|## Unix socket file mode
|## When none is provided, 660 is applied by default
|# server-unix-socket-mode = "660"
|]
+417 -211
View File
@@ -1,241 +1,447 @@
{-# LANGUAGE LambdaCase, TemplateHaskell #-}
{-# OPTIONS_GHC -fno-warn-type-defaults #-}
{-|
Module : PostgREST.Config
Description : Manages PostgREST configuration options.
Description : Manages PostgREST configuration type and parser.
This module provides a helper function to read the command line
arguments using the optparse-applicative and the AppConfig type to store
them. It also can be used to define other middleware configuration that
may be delegated to some sort of external configuration.
It currently includes a hardcoded CORS policy but this could easly be
turned in configurable behaviour if needed.
Other hardcoded options such as the minimum version number also belong here.
-}
module PostgREST.Config ( prettyVersion
, docsVersion
, readOptions
, corsPolicy
, AppConfig (..)
)
where
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE RecordWildCards #-}
{-# OPTIONS_GHC -fno-warn-type-defaults #-}
import Control.Applicative
import Control.Monad (fail)
import Control.Lens (preview)
import Crypto.JWT (StringOrURI,
stringOrUri)
import qualified Data.ByteString as B
import qualified Data.ByteString.Char8 as BS
import qualified Data.CaseInsensitive as CI
import qualified Data.Configurator as C
import qualified Data.Configurator.Parser as C
import Data.Configurator.Types as C
import Data.List (lookup)
import Data.Monoid
import Data.Scientific (floatingOrInteger)
import Data.String (String)
import Data.Text (dropAround,
intercalate, lines,
strip, take, splitOn)
import Data.Text.Encoding (encodeUtf8)
import Data.Text.IO (hPutStrLn)
import Data.Version (versionBranch)
import Development.GitRev (gitHash)
import Network.Wai
import Network.Wai.Middleware.Cors (CorsResourcePolicy (..))
import Options.Applicative hiding (str)
import Paths_postgrest (version)
import PostgREST.Parsers (pRoleClaimKey)
import PostgREST.Types (ApiRequestError(..),
JSPath, JSPathExp(..))
import Protolude hiding (hPutStrLn, take,
intercalate, (<>))
import System.IO (hPrint)
import System.IO.Error (IOError)
import Text.Heredoc
import Text.PrettyPrint.ANSI.Leijen hiding ((<$>), (<>))
import qualified Text.PrettyPrint.ANSI.Leijen as L
module PostgREST.Config
( AppConfig (..)
, Environment
, JSPath
, JSPathExp(..)
, LogLevel(..)
, OpenAPIMode(..)
, Proxy(..)
, toText
, isMalformedProxyUri
, readAppConfig
, readPGRSTEnvironment
, toURI
, parseSecret
) where
-- | Config file settings for the server
data AppConfig = AppConfig {
configDatabase :: Text
, configAnonRole :: Text
, configProxyUri :: Maybe Text
, configSchema :: Text
, configHost :: Text
, configPort :: Int
import qualified Crypto.JOSE.Types as JOSE
import qualified Crypto.JWT as JWT
import qualified Data.Aeson as JSON
import qualified Data.ByteString as BS
import qualified Data.ByteString.Base64 as B64
import qualified Data.ByteString.Lazy as LBS
import qualified Data.Configurator as C
import qualified Data.Map.Strict as M
import qualified Data.Text as T
import qualified Data.Text.Encoding as T
, configJwtSecret :: Maybe B.ByteString
, configJwtSecretIsBase64 :: Bool
, configJwtAudience :: Maybe StringOrURI
import Control.Lens (preview)
import Control.Monad (fail)
import Crypto.JWT (JWK, JWKSet, StringOrURI, stringOrUri)
import Data.Aeson (toJSON)
import Data.Either.Combinators (mapLeft)
import Data.List (lookup)
import Data.List.NonEmpty (fromList, toList)
import Data.Maybe (fromJust)
import Data.Scientific (floatingOrInteger)
import Data.Time.Clock (NominalDiffTime)
import Numeric (readOct, showOct)
import System.Environment (getEnvironment)
import System.Posix.Types (FileMode)
, configPool :: Int
, configMaxRows :: Maybe Integer
, configReqCheck :: Maybe Text
, configQuiet :: Bool
, configSettings :: [(Text, Text)]
, configRoleClaimKey :: Either ApiRequestError JSPath
, configExtraSearchPath :: [Text]
import PostgREST.Config.JSPath (JSPath, JSPathExp (..),
dumpJSPath, pRoleClaimKey)
import PostgREST.Config.Proxy (Proxy (..),
isMalformedProxyUri, toURI)
import PostgREST.DbStructure.Identifiers (QualifiedIdentifier, dumpQi,
toQi)
import PostgREST.MediaType (MediaType (..), toMime)
import Protolude hiding (Proxy, toList)
data AppConfig = AppConfig
{ configAppSettings :: [(Text, Text)]
, configDbAnonRole :: Maybe Text
, configDbChannel :: Text
, configDbChannelEnabled :: Bool
, configDbExtraSearchPath :: [Text]
, configDbMaxRows :: Maybe Integer
, configDbPlanEnabled :: Bool
, configDbPoolSize :: Int
, configDbPoolTimeout :: NominalDiffTime
, configDbPreRequest :: Maybe QualifiedIdentifier
, configDbPreparedStatements :: Bool
, configDbRootSpec :: Maybe QualifiedIdentifier
, configDbSchemas :: NonEmpty Text
, configDbConfig :: Bool
, configDbTxAllowOverride :: Bool
, configDbTxRollbackAll :: Bool
, configDbUri :: Text
, configDbUseLegacyGucs :: Bool
, configFilePath :: Maybe FilePath
, configJWKS :: Maybe JWKSet
, configJwtAudience :: Maybe StringOrURI
, configJwtRoleClaimKey :: JSPath
, configJwtSecret :: Maybe BS.ByteString
, configJwtSecretIsBase64 :: Bool
, configLogLevel :: LogLevel
, configOpenApiMode :: OpenAPIMode
, configOpenApiSecurityActive :: Bool
, configOpenApiServerProxyUri :: Maybe Text
, configRawMediaTypes :: [MediaType]
, configServerHost :: Text
, configServerPort :: Int
, configServerUnixSocket :: Maybe FilePath
, configServerUnixSocketMode :: FileMode
, configAdminServerPort :: Maybe Int
}
defaultCorsPolicy :: CorsResourcePolicy
defaultCorsPolicy = CorsResourcePolicy Nothing
["GET", "POST", "PATCH", "PUT", "DELETE", "OPTIONS"] ["Authorization"] Nothing
(Just $ 60*60*24) False False True
data LogLevel = LogCrit | LogError | LogWarn | LogInfo
-- | CORS policy to be used in by Wai Cors middleware
corsPolicy :: Request -> Maybe CorsResourcePolicy
corsPolicy req = case lookup "origin" headers of
Just origin -> Just defaultCorsPolicy {
corsOrigins = Just ([origin], True)
, corsRequestHeaders = "Authentication":accHeaders
, corsExposedHeaders = Just [
"Content-Encoding", "Content-Location", "Content-Range", "Content-Type"
, "Date", "Location", "Server", "Transfer-Encoding", "Range-Unit"
dumpLogLevel :: LogLevel -> Text
dumpLogLevel = \case
LogCrit -> "crit"
LogError -> "error"
LogWarn -> "warn"
LogInfo -> "info"
data OpenAPIMode = OAFollowPriv | OAIgnorePriv | OADisabled
deriving Eq
dumpOpenApiMode :: OpenAPIMode -> Text
dumpOpenApiMode = \case
OAFollowPriv -> "follow-privileges"
OAIgnorePriv -> "ignore-privileges"
OADisabled -> "disabled"
-- | Dump the config
toText :: AppConfig -> Text
toText conf =
unlines $ (\(k, v) -> k <> " = " <> v) <$> pgrstSettings ++ appSettings
where
-- apply conf to all pgrst settings
pgrstSettings = (\(k, v) -> (k, v conf)) <$>
[("db-anon-role", q . fromMaybe "" . configDbAnonRole)
,("db-channel", q . configDbChannel)
,("db-channel-enabled", T.toLower . show . configDbChannelEnabled)
,("db-extra-search-path", q . T.intercalate "," . configDbExtraSearchPath)
,("db-max-rows", maybe "\"\"" show . configDbMaxRows)
,("db-plan-enabled", T.toLower . show . configDbPlanEnabled)
,("db-pool", show . configDbPoolSize)
,("db-pool-timeout", show . floor . configDbPoolTimeout)
,("db-pre-request", q . maybe mempty dumpQi . configDbPreRequest)
,("db-prepared-statements", T.toLower . show . configDbPreparedStatements)
,("db-root-spec", q . maybe mempty dumpQi . configDbRootSpec)
,("db-schemas", q . T.intercalate "," . toList . configDbSchemas)
,("db-config", T.toLower . show . configDbConfig)
,("db-tx-end", q . showTxEnd)
,("db-uri", q . configDbUri)
,("db-use-legacy-gucs", T.toLower . show . configDbUseLegacyGucs)
,("jwt-aud", T.decodeUtf8 . LBS.toStrict . JSON.encode . maybe "" toJSON . configJwtAudience)
,("jwt-role-claim-key", q . T.intercalate mempty . fmap dumpJSPath . configJwtRoleClaimKey)
,("jwt-secret", q . T.decodeUtf8 . showJwtSecret)
,("jwt-secret-is-base64", T.toLower . show . configJwtSecretIsBase64)
,("log-level", q . dumpLogLevel . configLogLevel)
,("openapi-mode", q . dumpOpenApiMode . configOpenApiMode)
,("openapi-security-active", T.toLower . show . configOpenApiSecurityActive)
,("openapi-server-proxy-uri", q . fromMaybe mempty . configOpenApiServerProxyUri)
,("raw-media-types", q . T.decodeUtf8 . BS.intercalate "," . fmap toMime . configRawMediaTypes)
,("server-host", q . configServerHost)
,("server-port", show . configServerPort)
,("server-unix-socket", q . maybe mempty T.pack . configServerUnixSocket)
,("server-unix-socket-mode", q . T.pack . showSocketMode)
,("admin-server-port", maybe "\"\"" show . configAdminServerPort)
]
}
Nothing -> Nothing
-- quote all app.settings
appSettings = second q <$> configAppSettings conf
-- quote strings and replace " with \"
q s = "\"" <> T.replace "\"" "\\\"" s <> "\""
showTxEnd c = case (configDbTxRollbackAll c, configDbTxAllowOverride c) of
( False, False ) -> "commit"
( False, True ) -> "commit-allow-override"
( True , False ) -> "rollback"
( True , True ) -> "rollback-allow-override"
showJwtSecret c
| configJwtSecretIsBase64 c = B64.encode secret
| otherwise = secret
where
secret = fromMaybe mempty $ configJwtSecret c
showSocketMode c = showOct (configServerUnixSocketMode c) mempty
-- This class is needed for the polymorphism of overrideFromDbOrEnvironment
-- because C.required and C.optional have different signatures
class JustIfMaybe a b where
justIfMaybe :: a -> b
instance JustIfMaybe a a where
justIfMaybe = identity
instance JustIfMaybe a (Maybe a) where
justIfMaybe = Just
-- | Reads and parses the config and overrides its parameters from env vars,
-- files or db settings.
readAppConfig :: [(Text, Text)] -> Maybe FilePath -> Maybe Text -> IO (Either Text AppConfig)
readAppConfig dbSettings optPath prevDbUri = do
env <- readPGRSTEnvironment
-- if no filename provided, start with an empty map to read config from environment
conf <- maybe (return $ Right M.empty) loadConfig optPath
case C.runParser (parser optPath env dbSettings) =<< mapLeft show conf of
Left err ->
return . Left $ "Error in config " <> err
Right parsedConfig ->
Right <$> decodeLoadFiles parsedConfig
where
headers = requestHeaders req
accHeaders = case lookup "access-control-request-headers" headers of
Just hdrs -> map (CI.mk . toS . strip . toS) $ BS.split ',' hdrs
Nothing -> []
-- Both C.ParseError and IOError are shown here
loadConfig :: FilePath -> IO (Either SomeException C.Config)
loadConfig = try . C.load
-- | User friendly version number
prettyVersion :: Text
prettyVersion =
intercalate "." (map show $ versionBranch version)
<> " (" <> take 7 $(gitHash) <> ")"
-- | Version number used in docs
docsVersion :: Text
docsVersion = "v" <> dropAround (== '.') (dropAround (/= '.') prettyVersion)
-- | Function to read and parse options from the command line
readOptions :: IO AppConfig
readOptions = do
-- First read the config file path from command line
cfgPath <- customExecParser parserPrefs opts
-- Now read the actual config file
conf <- catch
(C.readConfig =<< C.load [C.Required cfgPath])
configNotfoundHint
let (mAppConf, errs) = flip C.runParserM conf $
AppConfig
<$> C.key "db-uri"
<*> C.key "db-anon-role"
<*> (mfilter (/= "") <$> C.key "server-proxy-uri")
<*> C.key "db-schema"
<*> (fromMaybe "127.0.0.1" . mfilter (/= "") <$> C.key "server-host")
<*> (fromMaybe 3000 . join . fmap coerceInt <$> C.key "server-port")
<*> (fmap encodeUtf8 . mfilter (/= "") <$> C.key "jwt-secret")
<*> (fromMaybe False . join . fmap coerceBool <$> C.key "secret-is-base64")
<*> parseJwtAudience "jwt-aud"
<*> (fromMaybe 10 . join . fmap coerceInt <$> C.key "db-pool")
<*> (join . fmap coerceInt <$> C.key "max-rows")
<*> (mfilter (/= "") <$> C.key "pre-request")
<*> pure False
<*> (fmap (fmap coerceText) <$> C.subassocs "app.settings")
<*> (maybe (Right [JSPKey "role"]) parseRoleClaimKey <$> C.key "role-claim-key")
<*> (maybe ["public"] splitExtraSearchPath <$> C.key "db-extra-search-path")
case mAppConf of
Nothing -> do
forM_ errs $ hPrint stderr
exitFailure
Just appConf ->
return appConf
decodeLoadFiles :: AppConfig -> IO AppConfig
decodeLoadFiles parsedConfig =
decodeJWKS <$>
(decodeSecret =<< readSecretFile =<< readDbUriFile prevDbUri parsedConfig)
parser :: Maybe FilePath -> Environment -> [(Text, Text)] -> C.Parser C.Config AppConfig
parser optPath env dbSettings =
AppConfig
<$> parseAppSettings "app.settings"
<*> optString "db-anon-role"
<*> (fromMaybe "pgrst" <$> optString "db-channel")
<*> (fromMaybe True <$> optBool "db-channel-enabled")
<*> (maybe ["public"] splitOnCommas <$> optValue "db-extra-search-path")
<*> optWithAlias (optInt "db-max-rows")
(optInt "max-rows")
<*> (fromMaybe False <$> optBool "db-plan-enabled")
<*> (fromMaybe 10 <$> optInt "db-pool")
<*> (fromIntegral . fromMaybe 3600 <$> optInt "db-pool-timeout")
<*> (fmap toQi <$> optWithAlias (optString "db-pre-request")
(optString "pre-request"))
<*> (fromMaybe True <$> optBool "db-prepared-statements")
<*> (fmap toQi <$> optWithAlias (optString "db-root-spec")
(optString "root-spec"))
<*> (fromList . maybe ["public"] splitOnCommas <$> optWithAlias (optValue "db-schemas")
(optValue "db-schema"))
<*> (fromMaybe True <$> optBool "db-config")
<*> parseTxEnd "db-tx-end" snd
<*> parseTxEnd "db-tx-end" fst
<*> (fromMaybe "postgresql://" <$> optString "db-uri")
<*> (fromMaybe True <$> optBool "db-use-legacy-gucs")
<*> pure optPath
<*> pure Nothing
<*> parseJwtAudience "jwt-aud"
<*> parseRoleClaimKey "jwt-role-claim-key" "role-claim-key"
<*> (fmap encodeUtf8 <$> optString "jwt-secret")
<*> (fromMaybe False <$> optWithAlias
(optBool "jwt-secret-is-base64")
(optBool "secret-is-base64"))
<*> parseLogLevel "log-level"
<*> parseOpenAPIMode "openapi-mode"
<*> (fromMaybe False <$> optBool "openapi-security-active")
<*> parseOpenAPIServerProxyURI "openapi-server-proxy-uri"
<*> (maybe [] (fmap (MTOther . encodeUtf8) . splitOnCommas) <$> optValue "raw-media-types")
<*> (fromMaybe "!4" <$> optString "server-host")
<*> (fromMaybe 3000 <$> optInt "server-port")
<*> (fmap T.unpack <$> optString "server-unix-socket")
<*> parseSocketFileMode "server-unix-socket-mode"
<*> optInt "admin-server-port"
where
parseJwtAudience :: Name -> C.ConfigParserM (Maybe StringOrURI)
parseAppSettings :: C.Key -> C.Parser C.Config [(Text, Text)]
parseAppSettings key = addFromEnv . fmap (fmap coerceText) <$> C.subassocs key C.value
where
addFromEnv f = M.toList $ M.union fromEnv $ M.fromList f
fromEnv = M.mapKeys fromJust $ M.filterWithKey (\k _ -> isJust k) $ M.mapKeys normalize env
normalize k = ("app.settings." <>) <$> T.stripPrefix "PGRST_APP_SETTINGS_" (toS k)
parseSocketFileMode :: C.Key -> C.Parser C.Config FileMode
parseSocketFileMode k =
optString k >>= \case
Nothing -> pure 432 -- return default 660 mode if no value was provided
Just fileModeText ->
case readOct $ T.unpack fileModeText of
[] ->
fail "Invalid server-unix-socket-mode: not an octal"
(fileMode, _):_ ->
if fileMode < 384 || fileMode > 511
then fail "Invalid server-unix-socket-mode: needs to be between 600 and 777"
else pure fileMode
parseOpenAPIMode :: C.Key -> C.Parser C.Config OpenAPIMode
parseOpenAPIMode k =
optString k >>= \case
Nothing -> pure OAFollowPriv
Just "follow-privileges" -> pure OAFollowPriv
Just "ignore-privileges" -> pure OAIgnorePriv
Just "disabled" -> pure OADisabled
Just _ -> fail "Invalid openapi-mode. Check your configuration."
parseOpenAPIServerProxyURI :: C.Key -> C.Parser C.Config (Maybe Text)
parseOpenAPIServerProxyURI k =
optString k >>= \case
Nothing -> pure Nothing
Just val | isMalformedProxyUri val -> fail "Malformed proxy uri, a correct example: https://example.com:8443/basePath"
| otherwise -> pure $ Just val
parseJwtAudience :: C.Key -> C.Parser C.Config (Maybe StringOrURI)
parseJwtAudience k =
C.key k >>= \case
optString k >>= \case
Nothing -> pure Nothing -- no audience in config file
Just aud -> case preview stringOrUri (aud :: String) of
Just aud -> case preview stringOrUri (T.unpack aud) of
Nothing -> fail "Invalid Jwt audience. Check your configuration."
(Just "") -> pure Nothing
aud' -> pure aud'
coerceText :: Value -> Text
coerceText (String s) = s
coerceText v = show v
parseLogLevel :: C.Key -> C.Parser C.Config LogLevel
parseLogLevel k =
optString k >>= \case
Nothing -> pure LogError
Just "crit" -> pure LogCrit
Just "error" -> pure LogError
Just "warn" -> pure LogWarn
Just "info" -> pure LogInfo
Just _ -> fail "Invalid logging level. Check your configuration."
coerceInt :: (Read i, Integral i) => Value -> Maybe i
coerceInt (Number x) = rightToMaybe $ floatingOrInteger x
coerceInt (String x) = readMaybe $ toS x
coerceInt _ = Nothing
parseTxEnd :: C.Key -> ((Bool, Bool) -> Bool) -> C.Parser C.Config Bool
parseTxEnd k f =
optString k >>= \case
-- RollbackAll AllowOverride
Nothing -> pure $ f (False, False)
Just "commit" -> pure $ f (False, False)
Just "commit-allow-override" -> pure $ f (False, True)
Just "rollback" -> pure $ f (True, False)
Just "rollback-allow-override" -> pure $ f (True, True)
Just _ -> fail "Invalid transaction termination. Check your configuration."
coerceBool :: Value -> Maybe Bool
coerceBool (Bool b) = Just b
coerceBool (String b) = readMaybe $ toS b
coerceBool _ = Nothing
parseRoleClaimKey :: C.Key -> C.Key -> C.Parser C.Config JSPath
parseRoleClaimKey k al =
optWithAlias (optString k) (optString al) >>= \case
Nothing -> pure [JSPKey "role"]
Just rck -> either (fail . show) pure $ pRoleClaimKey rck
parseRoleClaimKey :: Value -> Either ApiRequestError JSPath
parseRoleClaimKey (String s) = pRoleClaimKey s
parseRoleClaimKey v = pRoleClaimKey $ show v
optWithAlias :: C.Parser C.Config (Maybe a) -> C.Parser C.Config (Maybe a) -> C.Parser C.Config (Maybe a)
optWithAlias orig alias =
orig >>= \case
Just v -> pure $ Just v
Nothing -> alias
splitExtraSearchPath :: Value -> [Text]
splitExtraSearchPath (String s) = strip <$> splitOn "," s
splitExtraSearchPath _ = []
optString :: C.Key -> C.Parser C.Config (Maybe Text)
optString k = mfilter (/= "") <$> overrideFromDbOrEnvironment C.optional k coerceText
opts = info (helper <*> pathParser) $
fullDesc
<> progDesc (
"PostgREST "
<> toS prettyVersion
<> " / create a REST API to an existing Postgres database"
)
<> footerDoc (Just $
text "Example Config File:"
L.<> nest 2 (hardline L.<> exampleCfg)
)
optValue :: C.Key -> C.Parser C.Config (Maybe C.Value)
optValue k = overrideFromDbOrEnvironment C.optional k identity
parserPrefs = prefs showHelpOnError
optInt :: (Read i, Integral i) => C.Key -> C.Parser C.Config (Maybe i)
optInt k = join <$> overrideFromDbOrEnvironment C.optional k coerceInt
configNotfoundHint :: IOError -> IO a
configNotfoundHint e = do
hPutStrLn stderr $
"Cannot open config file:\n\t" <> show e
exitFailure
optBool :: C.Key -> C.Parser C.Config (Maybe Bool)
optBool k = join <$> overrideFromDbOrEnvironment C.optional k coerceBool
exampleCfg :: Doc
exampleCfg = vsep . map (text . toS) . lines $
[str|db-uri = "postgres://user:pass@localhost:5432/dbname"
|db-schema = "public" # this schema gets added to the search_path of every request
|db-anon-role = "postgres"
|db-pool = 10
|
|server-host = "127.0.0.1"
|server-port = 3000
|
|## base url for swagger output
|# server-proxy-uri = ""
|
|## choose a secret, JSON Web Key (or set) to enable JWT auth
|## (use "@filename" to load from separate file)
|# jwt-secret = "foo"
|# secret-is-base64 = false
|# jwt-aud = "your_audience_claim"
|
|## limit rows in response
|# max-rows = 1000
|
|## stored proc to exec immediately after auth
|# pre-request = "stored_proc_name"
|
|## jspath to the role claim key
|# role-claim-key = ".role"
|
|## extra schemas to add to the search_path of every request
|# db-extra-search-path = "extensions, util"
|]
overrideFromDbOrEnvironment :: JustIfMaybe a b =>
(C.Key -> C.Parser C.Value a -> C.Parser C.Config b) ->
C.Key -> (C.Value -> a) -> C.Parser C.Config b
overrideFromDbOrEnvironment necessity key coercion =
case reloadableDbSetting <|> M.lookup envVarName env of
Just dbOrEnvVal -> pure $ justIfMaybe $ coercion $ C.String dbOrEnvVal
Nothing -> necessity key (coercion <$> C.value)
where
dashToUnderscore '-' = '_'
dashToUnderscore c = c
envVarName = "PGRST_" <> (toUpper . dashToUnderscore <$> toS key)
reloadableDbSetting =
let dbSettingName = T.pack $ dashToUnderscore <$> toS key in
if dbSettingName `notElem` [
"server_host", "server_port", "server_unix_socket", "server_unix_socket_mode", "admin_server_port", "log_level",
"db_uri", "db_channel_enabled", "db_channel", "db_pool", "db_pool_timeout", "db_config"]
then lookup dbSettingName dbSettings
else Nothing
pathParser :: Parser FilePath
pathParser =
strArgument $
metavar "FILENAME" <>
help "Path to configuration file"
coerceText :: C.Value -> Text
coerceText (C.String s) = s
coerceText v = show v
coerceInt :: (Read i, Integral i) => C.Value -> Maybe i
coerceInt (C.Number x) = rightToMaybe $ floatingOrInteger x
coerceInt (C.String x) = readMaybe x
coerceInt _ = Nothing
coerceBool :: C.Value -> Maybe Bool
coerceBool (C.Bool b) = Just b
coerceBool (C.String s) =
-- parse all kinds of text: True, true, TRUE, "true", ...
case readMaybe $ T.toTitle $ T.filter isAlpha $ toS s of
Just b -> Just b
-- numeric instead?
Nothing -> (> 0) <$> (readMaybe s :: Maybe Integer)
coerceBool _ = Nothing
splitOnCommas :: C.Value -> [Text]
splitOnCommas (C.String s) = T.strip <$> T.splitOn "," s
splitOnCommas _ = []
-- | Read the JWT secret from a file if configJwtSecret is actually a
-- filepath(has @ as its prefix). To check if the JWT secret is provided is
-- in fact a file path, it must be decoded as 'Text' to be processed.
readSecretFile :: AppConfig -> IO AppConfig
readSecretFile conf =
maybe (return conf) readSecret maybeFilename
where
maybeFilename = T.stripPrefix "@" . decodeUtf8 =<< configJwtSecret conf
readSecret filename = do
jwtSecret <- chomp <$> BS.readFile (toS filename)
return $ conf { configJwtSecret = Just jwtSecret }
chomp bs = fromMaybe bs (BS.stripSuffix "\n" bs)
decodeSecret :: AppConfig -> IO AppConfig
decodeSecret conf@AppConfig{..} =
case (configJwtSecretIsBase64, configJwtSecret) of
(True, Just secret) ->
either fail (return . updateSecret) $ decodeB64 secret
_ -> return conf
where
updateSecret bs = conf { configJwtSecret = Just bs }
decodeB64 = B64.decode . encodeUtf8 . T.strip . replaceUrlChars . decodeUtf8
replaceUrlChars = T.replace "_" "/" . T.replace "-" "+" . T.replace "." "="
-- | Parse `jwt-secret` configuration option and turn into a JWKSet.
--
-- There are three ways to specify `jwt-secret`: text secret, JSON Web Key
-- (JWK), or JSON Web Key Set (JWKS). The first two are converted into a JWKSet
-- with one key and the last is converted as is.
decodeJWKS :: AppConfig -> AppConfig
decodeJWKS conf =
conf { configJWKS = parseSecret <$> configJwtSecret conf }
parseSecret :: ByteString -> JWKSet
parseSecret bytes =
fromMaybe (maybe secret (\jwk' -> JWT.JWKSet [jwk']) maybeJWK)
maybeJWKSet
where
maybeJWKSet = JSON.decodeStrict bytes :: Maybe JWKSet
maybeJWK = JSON.decodeStrict bytes :: Maybe JWK
secret = JWT.JWKSet [JWT.fromKeyMaterial keyMaterial]
keyMaterial = JWT.OctKeyMaterial . JWT.OctKeyParameters $ JOSE.Base64Octets bytes
-- | Read database uri from a separate file if `db-uri` is a filepath.
readDbUriFile :: Maybe Text -> AppConfig -> IO AppConfig
readDbUriFile maybeDbUri conf =
case maybeDbUri of
Just prevDbUri ->
pure $ conf { configDbUri = prevDbUri }
Nothing ->
case T.stripPrefix "@" $ configDbUri conf of
Nothing -> return conf
Just filename -> do
dbUri <- T.strip <$> readFile (toS filename)
return $ conf { configDbUri = dbUri }
type Environment = M.Map [Char] Text
-- | Read environment variables that start with PGRST_
readPGRSTEnvironment :: IO Environment
readPGRSTEnvironment =
M.map T.pack . M.fromList . filter (isPrefixOf "PGRST_" . fst) <$> getEnvironment
+65
View File
@@ -0,0 +1,65 @@
{-# LANGUAGE QuasiQuotes #-}
module PostgREST.Config.Database
( pgVersionStatement
, queryDbSettings
, queryPgVersion
) where
import PostgREST.Config.PgVersion (PgVersion (..))
import qualified Hasql.Decoders as HD
import qualified Hasql.Encoders as HE
import Hasql.Session (Session, statement)
import qualified Hasql.Statement as SQL
import qualified Hasql.Transaction as SQL
import qualified Hasql.Transaction.Sessions as SQL
import Text.InterpolatedString.Perl6 (q)
import Protolude
queryPgVersion :: Session PgVersion
queryPgVersion = statement mempty pgVersionStatement
pgVersionStatement :: SQL.Statement () PgVersion
pgVersionStatement = SQL.Statement sql HE.noParams versionRow False
where
sql = "SELECT current_setting('server_version_num')::integer, current_setting('server_version')"
versionRow = HD.singleRow $ PgVersion <$> column HD.int4 <*> column HD.text
queryDbSettings :: Bool -> Session [(Text, Text)]
queryDbSettings prepared =
let transaction = if prepared then SQL.transaction else SQL.unpreparedTransaction in
transaction SQL.ReadCommitted SQL.Read $ SQL.statement mempty dbSettingsStatement
-- | Get db settings from the connection role. Global settings will be overridden by database specific settings.
dbSettingsStatement :: SQL.Statement () [(Text, Text)]
dbSettingsStatement = SQL.Statement sql HE.noParams decodeSettings False
where
sql = [q|
WITH
role_setting (database, setting) AS (
SELECT setdatabase,
unnest(setconfig)
FROM pg_catalog.pg_db_role_setting
WHERE setrole = CURRENT_USER::regrole::oid
AND setdatabase IN (0, (SELECT oid FROM pg_catalog.pg_database WHERE datname = CURRENT_CATALOG))
),
kv_settings (database, k, v) AS (
SELECT database,
substr(setting, 1, strpos(setting, '=') - 1),
substr(setting, strpos(setting, '=') + 1)
FROM role_setting
WHERE setting LIKE 'pgrst.%'
)
SELECT DISTINCT ON (key)
replace(k, 'pgrst.', '') AS key,
v AS value
FROM kv_settings
ORDER BY key, database DESC;
|]
decodeSettings = HD.rowList $ (,) <$> column HD.text <*> column HD.text
column :: HD.Value a -> HD.Row a
column = HD.column . HD.nonNullable
+51
View File
@@ -0,0 +1,51 @@
module PostgREST.Config.JSPath
( JSPath
, JSPathExp(..)
, dumpJSPath
, pRoleClaimKey
) where
import qualified Text.ParserCombinators.Parsec as P
import Data.Either.Combinators (mapLeft)
import Text.ParserCombinators.Parsec ((<?>))
import Text.Read (read)
import Protolude
-- | full jspath, e.g. .property[0].attr.detail
type JSPath = [JSPathExp]
-- | jspath expression, e.g. .property, .property[0] or ."property-dash"
data JSPathExp
= JSPKey Text
| JSPIdx Int
dumpJSPath :: JSPathExp -> Text
-- TODO: this needs to be quoted properly for special chars
dumpJSPath (JSPKey k) = "." <> show k
dumpJSPath (JSPIdx i) = "[" <> show i <> "]"
-- Used for the config value "role-claim-key"
pRoleClaimKey :: Text -> Either Text JSPath
pRoleClaimKey selStr =
mapLeft show $ P.parse pJSPath ("failed to parse role-claim-key value (" <> toS selStr <> ")") (toS selStr)
pJSPath :: P.Parser JSPath
pJSPath = toJSPath <$> (period *> pPath `P.sepBy` period <* P.eof)
where
toJSPath :: [(Text, Maybe Int)] -> JSPath
toJSPath = concatMap (\(key, idx) -> JSPKey key : maybeToList (JSPIdx <$> idx))
period = P.char '.' <?> "period (.)"
pPath :: P.Parser (Text, Maybe Int)
pPath = (,) <$> pJSPKey <*> P.optionMaybe pJSPIdx
pJSPKey :: P.Parser Text
pJSPKey = toS <$> P.many1 (P.alphaNum <|> P.oneOf "_$@") <|> pQuotedValue <?> "attribute name [a..z0..9_$@])"
pJSPIdx :: P.Parser Int
pJSPIdx = P.char '[' *> (read <$> P.many1 P.digit) <* P.char ']' <?> "array index [0..n]"
pQuotedValue :: P.Parser Text
pQuotedValue = toS <$> (P.char '"' *> P.many (P.noneOf "\"") <* P.char '"')
+64
View File
@@ -0,0 +1,64 @@
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
module PostgREST.Config.PgVersion
( PgVersion(..)
, minimumPgVersion
, pgVersion96
, pgVersion100
, pgVersion109
, pgVersion110
, pgVersion112
, pgVersion114
, pgVersion120
, pgVersion121
, pgVersion130
, pgVersion140
) where
import qualified Data.Aeson as JSON
import Protolude
data PgVersion = PgVersion
{ pgvNum :: Int32
, pgvName :: Text
}
deriving (Eq, Generic, JSON.ToJSON)
instance Ord PgVersion where
(PgVersion v1 _) `compare` (PgVersion v2 _) = v1 `compare` v2
-- | Tells the minimum PostgreSQL version required by this version of PostgREST
minimumPgVersion :: PgVersion
minimumPgVersion = pgVersion96
pgVersion96 :: PgVersion
pgVersion96 = PgVersion 90600 "9.6"
pgVersion100 :: PgVersion
pgVersion100 = PgVersion 100000 "10"
pgVersion109 :: PgVersion
pgVersion109 = PgVersion 100009 "10.9"
pgVersion110 :: PgVersion
pgVersion110 = PgVersion 110000 "11.0"
pgVersion112 :: PgVersion
pgVersion112 = PgVersion 110002 "11.2"
pgVersion114 :: PgVersion
pgVersion114 = PgVersion 110004 "11.4"
pgVersion120 :: PgVersion
pgVersion120 = PgVersion 120000 "12.0"
pgVersion121 :: PgVersion
pgVersion121 = PgVersion 120001 "12.1"
pgVersion130 :: PgVersion
pgVersion130 = PgVersion 130000 "13.0"
pgVersion140 :: PgVersion
pgVersion140 = PgVersion 140000 "14.0"
+77
View File
@@ -0,0 +1,77 @@
{-|
Module : PostgREST.Private.ProxyUri
Description : Proxy Uri validator
-}
module PostgREST.Config.Proxy
( Proxy(..)
, isMalformedProxyUri
, toURI
) where
import qualified Data.Text as T
import Data.Maybe (fromJust)
import Network.URI (URI (..), URIAuth (..), isAbsoluteURI, parseURI)
import Protolude hiding (Proxy)
data Proxy = Proxy
{ proxyScheme :: Text
, proxyHost :: Text
, proxyPort :: Integer
, proxyPath :: Text
}
{-|
Test whether a proxy uri is malformed or not.
A valid proxy uri should be an absolute uri without query and user info,
only http(s) schemes are valid, port number range is 1-65535.
For example
http://postgrest.com/openapi.json
https://postgrest.com:8080/openapi.json
-}
isMalformedProxyUri :: Text -> Bool
isMalformedProxyUri uri
| isAbsoluteURI (toS uri) = not $ isUriValid $ toURI uri
| otherwise = True
toURI :: Text -> URI
toURI uri = fromJust $ parseURI (toS uri)
isUriValid:: URI -> Bool
isUriValid = fAnd [isSchemeValid, isQueryValid, isAuthorityValid]
fAnd :: [a -> Bool] -> a -> Bool
fAnd fs x = all ($ x) fs
isSchemeValid :: URI -> Bool
isSchemeValid URI {uriScheme = s}
| T.toLower (T.pack s) == "https:" = True
| T.toLower (T.pack s) == "http:" = True
| otherwise = False
isQueryValid :: URI -> Bool
isQueryValid URI {uriQuery = ""} = True
isQueryValid _ = False
isAuthorityValid :: URI -> Bool
isAuthorityValid URI {uriAuthority = a}
| isJust a = fAnd [isUserInfoValid, isHostValid, isPortValid] $ fromJust a
| otherwise = False
isUserInfoValid :: URIAuth -> Bool
isUserInfoValid URIAuth {uriUserInfo = ""} = True
isUserInfoValid _ = False
isHostValid :: URIAuth -> Bool
isHostValid URIAuth {uriRegName = ""} = False
isHostValid _ = True
isPortValid :: URIAuth -> Bool
isPortValid URIAuth {uriPort = ""} = True
isPortValid URIAuth {uriPort = (':':p)} =
case readMaybe p of
Just i -> i > (0 :: Integer) && i < 65536
Nothing -> False
isPortValid _ = False
+42
View File
@@ -0,0 +1,42 @@
{-|
Module : PostgREST.Cors
Description : Wai Middleware to set cors policy.
-}
module PostgREST.Cors (middleware) where
import qualified Data.ByteString.Char8 as BS
import qualified Data.CaseInsensitive as CI
import qualified Network.Wai as Wai
import qualified Network.Wai.Middleware.Cors as Wai
import Data.List (lookup)
import Protolude
middleware :: Wai.Middleware
middleware = Wai.cors corsPolicy
-- | CORS policy to be used in by Wai Cors middleware
corsPolicy :: Wai.Request -> Maybe Wai.CorsResourcePolicy
corsPolicy req = case lookup "origin" headers of
Just origin ->
Just Wai.CorsResourcePolicy
{ Wai.corsOrigins = Just ([origin], True)
, Wai.corsMethods = ["GET", "POST", "PATCH", "PUT", "DELETE", "OPTIONS"]
, Wai.corsRequestHeaders = "Authorization" : accHeaders
, Wai.corsExposedHeaders = Just
[ "Content-Encoding", "Content-Location", "Content-Range", "Content-Type"
, "Date", "Location", "Server", "Transfer-Encoding", "Range-Unit"]
, Wai.corsMaxAge = Just $ 60*60*24
, Wai.corsVaryOrigin = False
, Wai.corsRequireOrigin = False
, Wai.corsIgnoreFailures = True
}
Nothing -> Nothing
where
headers = Wai.requestHeaders req
accHeaders = case lookup "access-control-request-headers" headers of
Just hdrs -> map (CI.mk . BS.strip) $ BS.split ',' hdrs
-- Impossible case, Middleware.Cors will not evaluate this when
-- the Access-Control-Request-Headers header is not set.
Nothing -> []
-342
View File
@@ -1,342 +0,0 @@
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE DuplicateRecordFields#-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE MultiWayIf #-}
{-# LANGUAGE NamedFieldPuns #-}
module PostgREST.DbRequestBuilder (
readRequest
, mutateRequest
, fieldNames
) where
import Control.Applicative
import Control.Arrow ((***))
import Control.Lens.Getter (view)
import Control.Lens.Tuple (_1)
import qualified Data.ByteString.Char8 as BS
import Data.List (delete)
import Data.Maybe (fromJust)
import qualified Data.Set as S
import Data.Text (isInfixOf)
import Data.Tree
import Data.Either.Combinators (mapLeft)
import Network.Wai
import Data.Foldable (foldr1)
import qualified Data.HashMap.Strict as M
import PostgREST.ApiRequest ( ApiRequest(..)
, PreferRepresentation(..)
, Action(..), Target(..)
, PreferRepresentation (..)
)
import PostgREST.Error (apiRequestError)
import PostgREST.Parsers
import PostgREST.RangeQuery (NonnegRange, restrictRange, allRange)
import PostgREST.Types
import Protolude hiding (from)
import Text.Regex.TDFA ((=~))
import Unsafe (unsafeHead)
readRequest :: Maybe Integer -> [Relation] -> Maybe ProcDescription -> ApiRequest -> Either Response ReadRequest
readRequest maxRows allRels proc apiRequest =
mapLeft apiRequestError $
treeRestrictRange maxRows =<<
augumentRequestWithJoin schema relations =<<
addFiltersOrdersRanges apiRequest <*>
(buildReadRequest <$> pRequestSelect (iSelect apiRequest))
where
action = iAction apiRequest
(schema, rootTableName) = fromJust $ -- Make it safe
let target = iTarget apiRequest in
case target of
(TargetIdent (QualifiedIdentifier s t) ) -> Just (s, t)
(TargetProc (QualifiedIdentifier s pName) ) -> Just (s, tName)
where
tName = case pdReturnType <$> proc of
Just (SetOf (Composite qi)) -> qiName qi
Just (Single (Composite qi)) -> qiName qi
_ -> pName
_ -> Nothing
-- Build tree with a Depth attribute so when a self join occurs we can differentiate the parent and child tables by having
-- an alias like "table_depth", this is related to issue #987.
buildReadRequest :: [Tree SelectItem] -> ReadRequest
buildReadRequest fieldTree =
let rootDepth = 0
rootNodeName = if action == ActionRead then rootTableName else sourceCTEName in
foldr (treeEntry rootDepth) (Node (Select [] [rootNodeName] [] [] [] allRange, (rootNodeName, Nothing, Nothing, Nothing, rootDepth)) []) fieldTree
where
treeEntry :: Depth -> Tree SelectItem -> ReadRequest -> ReadRequest
treeEntry depth (Node fld@((fn, _),_,alias,relationDetail) fldForest) (Node (q, i) rForest) =
let nxtDepth = succ depth in
case fldForest of
[] -> Node (q {select=fld:select q}, i) rForest
_ -> Node (q, i) $
foldr (treeEntry nxtDepth) (Node (Select [] [fn] [] [] [] allRange, (fn, Nothing, alias, relationDetail, nxtDepth)) []) fldForest:rForest
relations :: [Relation]
relations = case action of
ActionCreate -> fakeSourceRelations ++ allRels
ActionUpdate -> fakeSourceRelations ++ allRels
ActionDelete -> fakeSourceRelations ++ allRels
ActionInvoke _ -> fakeSourceRelations ++ allRels
_ -> allRels
where fakeSourceRelations = mapMaybe (toSourceRelation rootTableName) allRels
-- in a relation where one of the tables matches "TableName"
-- replace the name to that table with pg_source
-- this "fake" relations is needed so that in a mutate query
-- we can look at the "returning *" part which is wrapped with a "with"
-- as just another table that has relations with other tables
toSourceRelation :: TableName -> Relation -> Maybe Relation
toSourceRelation mt r@(Relation t _ ft _ _ rt _ _)
| mt == tableName t = Just $ r {relTable=t {tableName=sourceCTEName}}
| mt == tableName ft = Just $ r {relFTable=t {tableName=sourceCTEName}}
| Just mt == (tableName <$> rt) = Just $ r {relLinkTable=(\tbl -> tbl {tableName=sourceCTEName}) <$> rt}
| otherwise = Nothing
treeRestrictRange :: Maybe Integer -> ReadRequest -> Either ApiRequestError ReadRequest
treeRestrictRange maxRows_ request = pure $ nodeRestrictRange maxRows_ `fmap` request
where
nodeRestrictRange :: Maybe Integer -> ReadNode -> ReadNode
nodeRestrictRange m (q@Select {range_=r}, i) = (q{range_=restrictRange m r }, i)
augumentRequestWithJoin :: Schema -> [Relation] -> ReadRequest -> Either ApiRequestError ReadRequest
augumentRequestWithJoin schema allRels request =
addRelations schema allRels Nothing request
>>= addJoinConditions schema
addRelations :: Schema -> [Relation] -> Maybe ReadRequest -> ReadRequest -> Either ApiRequestError ReadRequest
addRelations schema allRelations parentNode (Node (query, (nodeName, _, alias, relationDetail, depth)) forest) =
case parentNode of
Just (Node (Select{from=[parentNodeTable]}, _) _) ->
let newFrom r = (\tName -> if tName == nodeName then tableName (relTable r) else tName) <$> from query
newReadNode = (\r -> (query{from=newFrom r}, (nodeName, Just r, alias, Nothing, depth))) <$> rel
rel :: Either ApiRequestError Relation
rel = note (NoRelationBetween parentNodeTable nodeName) $
findRelation schema allRelations nodeName parentNodeTable relationDetail in
Node <$> newReadNode <*> (updateForest . hush $ Node <$> newReadNode <*> pure forest)
_ ->
let rn = (query, (nodeName, Just r, alias, Nothing, depth))
r = Relation t [] t [] Root Nothing Nothing Nothing
t = Table schema nodeName Nothing True in -- !!! TODO find another way to get the table from the query
Node rn <$> updateForest (Just $ Node rn forest)
where
updateForest :: Maybe ReadRequest -> Either ApiRequestError [ReadRequest]
updateForest rq = mapM (addRelations schema allRelations rq) forest
findRelation :: Schema -> [Relation] -> NodeName -> TableName -> Maybe RelationDetail -> Maybe Relation
findRelation schema allRelations nodeTableName parentNodeTableName relationDetail =
find (\Relation{relTable, relColumns, relFTable, relFColumns, relType, relLinkTable} ->
-- Both relation ends need to be on the exposed schema
schema == tableSchema relTable && schema == tableSchema relFTable &&
case relationDetail of
Nothing ->
-- (request) => projects { ..., clients{...} }
-- will match
-- (relation type) => parent
-- (entity) => clients {id}
-- (foriegn entity) => projects {client_id}
(
nodeTableName == tableName relTable && -- match relation table name
parentNodeTableName == tableName relFTable -- match relation foreign table name
) ||
-- (request) => projects { ..., client_id{...} }
-- will match
-- (relation type) => parent
-- (entity) => clients {id}
-- (foriegn entity) => projects {client_id}
(
parentNodeTableName == tableName relFTable &&
length relFColumns == 1 &&
-- match common foreign key names(table_name_id, table_name_fk) to table_name
(toS ("^" <> colName (unsafeHead relFColumns) <> "_?(?:|[iI][dD]|[fF][kK])$") :: BS.ByteString) =~ (toS nodeTableName :: BS.ByteString)
)
-- (request) => project_id { ..., client_id{...} }
-- will match
-- (relation type) => parent
-- (entity) => clients {id}
-- (foriegn entity) => projects {client_id}
-- this case works becasue before reaching this place
-- addRelation will turn project_id to project so the above condition will match
Just rd ->
-- (request) => clients { ..., projects.client_id{...} }
-- will match
-- (relation type) => child
-- (entity) => clients {id}
-- (foriegn entity) => projects {client_id}
(
relType == Child &&
nodeTableName == tableName relTable && -- match relation table name
parentNodeTableName == tableName relFTable && -- match relation foreign table name
length relColumns == 1 &&
rd == colName (unsafeHead relColumns)
) ||
-- (request) => message { ..., person_detail.sender{...} }
-- will match
-- (relation type) => parent
-- (entity) => message {sender}
-- (foriegn entity) => person_detail {id}
(
relType == Parent &&
nodeTableName == tableName relTable && -- match relation table name
parentNodeTableName == tableName relFTable && -- match relation foreign table name
length relFColumns == 1 &&
rd == colName (unsafeHead relFColumns)
) ||
-- (request) => tasks { ..., users.tasks_users{...} }
-- will match
-- (relation type) => many
-- (entity) => users
-- (foriegn entity) => tasks
(
relType == Many &&
nodeTableName == tableName relTable && -- match relation table name
parentNodeTableName == tableName relFTable && -- match relation foreign table name
rd == tableName (fromJust relLinkTable)
)
) allRelations
addJoinConditions :: Schema -> ReadRequest -> Either ApiRequestError ReadRequest
addJoinConditions schema (Node node@(query, nodeProps@(_, relation, _, _, _)) forest) =
case relation of
Just Relation{relType=Root} -> Node node <$> updatedForest -- this is the root node
Just rel@Relation{relType=Parent} -> Node (augmentQuery rel, nodeProps) <$> updatedForest
Just rel@Relation{relType=Child} -> Node (augmentQuery rel, nodeProps) <$> updatedForest
Just rel@Relation{relType=Many, relLinkTable=(Just linkTable)} ->
let rq = augmentQuery rel in
Node (rq{from=tableName linkTable:from rq}, nodeProps) <$> updatedForest
_ -> Left UnknownRelation
where
updatedForest = mapM (addJoinConditions schema) forest
augmentQuery rel = foldr addJoinCond query (getJoinConditions rel)
addJoinCond :: JoinCondition -> ReadQuery -> ReadQuery
addJoinCond jc rq@Select{joinConditions=jcs} = rq{joinConditions=jc:jcs}
getJoinConditions :: Relation -> [JoinCondition]
getJoinConditions (Relation Table{tableSchema=tSchema, tableName=tN} cols Table{tableName=ftN} fCols typ lt lc1 lc2) =
if | typ == Child || typ == Parent ->
zipWith (toJoinCondition tN ftN) cols fCols
| typ == Many ->
let ltN = maybe "" tableName lt in
zipWith (toJoinCondition tN ltN) cols (fromMaybe [] lc1) ++ zipWith (toJoinCondition ftN ltN) fCols (fromMaybe [] lc2)
| typ == Root -> witness
where
toJoinCondition :: Text -> Text -> Column -> Column -> JoinCondition
toJoinCondition tb ftb c fc =
JoinCondition (QualifiedIdentifier tSchema tb, Nothing, colName c)
(QualifiedIdentifier tSchema ftb, Nothing, colName fc)
addFiltersOrdersRanges :: ApiRequest -> Either ApiRequestError (ReadRequest -> ReadRequest)
addFiltersOrdersRanges apiRequest = foldr1 (liftA2 (.)) [
flip (foldr addFilter) <$> filters,
flip (foldr addOrder) <$> orders,
flip (foldr addRange) <$> ranges,
flip (foldr addLogicTree) <$> logicForest
]
{-
The esence of what is going on above is that we are composing tree functions
of type (ReadRequest->ReadRequest) that are in (Either ParseError a) context
-}
where
filters :: Either ApiRequestError [(EmbedPath, Filter)]
filters = mapM pRequestFilter flts
logicForest :: Either ApiRequestError [(EmbedPath, LogicTree)]
logicForest = mapM pRequestLogicTree logFrst
action = iAction apiRequest
-- there can be no filters on the root table when we are doing insert/update/delete
(flts, logFrst) =
case action of
ActionInvoke _ -> (iFilters apiRequest, iLogic apiRequest)
ActionRead -> (iFilters apiRequest, iLogic apiRequest)
_ -> join (***) (filter (( "." `isInfixOf` ) . fst)) (iFilters apiRequest, iLogic apiRequest)
orders :: Either ApiRequestError [(EmbedPath, [OrderTerm])]
orders = mapM pRequestOrder $ iOrder apiRequest
ranges :: Either ApiRequestError [(EmbedPath, NonnegRange)]
ranges = mapM pRequestRange $ M.toList $ iRange apiRequest
addFilterToNode :: Filter -> ReadRequest -> ReadRequest
addFilterToNode flt (Node (q@Select {where_=lf}, i) f) = Node (q{where_=addFilterToLogicForest flt lf}::ReadQuery, i) f
addFilter :: (EmbedPath, Filter) -> ReadRequest -> ReadRequest
addFilter = addProperty addFilterToNode
addOrderToNode :: [OrderTerm] -> ReadRequest -> ReadRequest
addOrderToNode o (Node (q,i) f) = Node (q{order=o}, i) f
addOrder :: (EmbedPath, [OrderTerm]) -> ReadRequest -> ReadRequest
addOrder = addProperty addOrderToNode
addRangeToNode :: NonnegRange -> ReadRequest -> ReadRequest
addRangeToNode r (Node (q,i) f) = Node (q{range_=r}, i) f
addRange :: (EmbedPath, NonnegRange) -> ReadRequest -> ReadRequest
addRange = addProperty addRangeToNode
addLogicTreeToNode :: LogicTree -> ReadRequest -> ReadRequest
addLogicTreeToNode t (Node (q@Select{where_=lf},i) f) = Node (q{where_=t:lf}::ReadQuery, i) f
addLogicTree :: (EmbedPath, LogicTree) -> ReadRequest -> ReadRequest
addLogicTree = addProperty addLogicTreeToNode
addProperty :: (a -> ReadRequest -> ReadRequest) -> (EmbedPath, a) -> ReadRequest -> ReadRequest
addProperty f ([], a) rr = f a rr
addProperty f (targetNodeName:remainingPath, a) (Node rn forest) =
case pathNode of
Nothing -> Node rn forest -- the property is silenty dropped in the Request does not contain the required path
Just tn -> Node rn (addProperty f (remainingPath, a) tn:delete tn forest)
where
pathNode = find (\(Node (_,(nodeName,_,alias,_,_)) _) -> nodeName == targetNodeName || alias == Just targetNodeName) forest
mutateRequest :: ApiRequest -> TableName -> [Text] -> [FieldName] -> Either Response MutateRequest
mutateRequest apiRequest tName pkCols fldNames = mapLeft apiRequestError $
case action of
ActionCreate -> Right $ Insert tName pkCols payload (iPreferResolution apiRequest) [] returnings
ActionUpdate -> Update tName payload <$> combinedLogic <*> pure returnings
ActionSingleUpsert ->
(\flts ->
if null (iLogic apiRequest) &&
S.fromList (fst <$> iFilters apiRequest) == S.fromList pkCols &&
not (null (S.fromList pkCols)) &&
all (\case
Filter _ (OpExpr False (Op "eq" _)) -> True
_ -> False) flts
then Insert tName pkCols payload (Just MergeDuplicates) <$> combinedLogic <*> pure returnings
else
Left InvalidFilters) =<< filters
ActionDelete -> Delete tName <$> combinedLogic <*> pure returnings
_ -> Left UnsupportedVerb
where
action = iAction apiRequest
payload = fromJust $ iPayload apiRequest
returnings = if iPreferRepresentation apiRequest == None then [] else fldNames
filters = map snd <$> mapM pRequestFilter mutateFilters
logic = map snd <$> mapM pRequestLogicTree logicFilters
combinedLogic = foldr addFilterToLogicForest <$> logic <*> filters
-- update/delete filters can be only on the root table
(mutateFilters, logicFilters) = join (***) onlyRoot (iFilters apiRequest, iLogic apiRequest)
onlyRoot = filter (not . ( "." `isInfixOf` ) . fst)
fieldNames :: ReadRequest -> [FieldName]
fieldNames (Node (sel, _) forest) =
map (fst . view _1) (select sel) ++ map colName fks
where
fks = concatMap (fromMaybe [] . f) forest
f (Node (_, (_, Just Relation{relFColumns=cols, relType=Parent}, _, _, _)) _) = Just cols
f _ = Nothing
-- Traditional filters(e.g. id=eq.1) are added as root nodes of the LogicTree
-- they are later concatenated with AND in the QueryBuilder
addFilterToLogicForest :: Filter -> [LogicTree] -> [LogicTree]
addFilterToLogicForest flt lf = Stmnt flt : lf
File diff suppressed because it is too large Load Diff
+42
View File
@@ -0,0 +1,42 @@
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
module PostgREST.DbStructure.Identifiers
( QualifiedIdentifier(..)
, Schema
, TableName
, FieldName
, dumpQi
, toQi
) where
import qualified Data.Aeson as JSON
import qualified Data.Text as T
import Protolude
-- | Represents a pg identifier with a prepended schema name "schema.table".
-- When qiSchema is "", the schema is defined by the pg search_path.
data QualifiedIdentifier = QualifiedIdentifier
{ qiSchema :: Schema
, qiName :: TableName
}
deriving (Eq, Ord, Generic, JSON.ToJSON, JSON.ToJSONKey)
instance Hashable QualifiedIdentifier
dumpQi :: QualifiedIdentifier -> Text
dumpQi (QualifiedIdentifier s i) =
(if T.null s then mempty else s <> ".") <> i
-- TODO: Handle a case where the QI comes like this: "my.fav.schema"."my.identifier"
-- Right now it only handles the schema.identifier case
toQi :: Text -> QualifiedIdentifier
toQi txt = case T.drop 1 <$> T.breakOn "." txt of
(i, "") -> QualifiedIdentifier mempty i
(s, i) -> QualifiedIdentifier s i
type Schema = Text
type TableName = Text
type FieldName = Text
+91
View File
@@ -0,0 +1,91 @@
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
module PostgREST.DbStructure.Proc
( PgType(..)
, ProcDescription(..)
, ProcParam(..)
, ProcVolatility(..)
, ProcsMap
, RetType(..)
, procReturnsScalar
, procReturnsSingle
, procReturnsVoid
, procTableName
) where
import qualified Data.Aeson as JSON
import qualified Data.HashMap.Strict as HM
import PostgREST.DbStructure.Identifiers (QualifiedIdentifier (..),
Schema, TableName)
import Protolude
data PgType
= Scalar
| Composite QualifiedIdentifier
deriving (Eq, Ord, Generic, JSON.ToJSON)
data RetType
= Single PgType
| SetOf PgType
deriving (Eq, Ord, Generic, JSON.ToJSON)
data ProcVolatility
= Volatile
| Stable
| Immutable
deriving (Eq, Ord, Generic, JSON.ToJSON)
data ProcDescription = ProcDescription
{ pdSchema :: Schema
, pdName :: Text
, pdDescription :: Maybe Text
, pdParams :: [ProcParam]
, pdReturnType :: Maybe RetType
, pdVolatility :: ProcVolatility
, pdHasVariadic :: Bool
}
deriving (Eq, Generic, JSON.ToJSON)
data ProcParam = ProcParam
{ ppName :: Text
, ppType :: Text
, ppReq :: Bool
, ppVar :: Bool
}
deriving (Eq, Ord, Generic, JSON.ToJSON)
-- Order by least number of params in the case of overloaded functions
instance Ord ProcDescription where
ProcDescription schema1 name1 des1 prms1 rt1 vol1 hasVar1 `compare` ProcDescription schema2 name2 des2 prms2 rt2 vol2 hasVar2
| schema1 == schema2 && name1 == name2 && length prms1 < length prms2 = LT
| schema2 == schema2 && name1 == name2 && length prms1 > length prms2 = GT
| otherwise = (schema1, name1, des1, prms1, rt1, vol1, hasVar1) `compare` (schema2, name2, des2, prms2, rt2, vol2, hasVar2)
-- | A map of all procs, all of which can be overloaded(one entry will have more than one ProcDescription).
-- | It uses a HashMap for a faster lookup.
type ProcsMap = HM.HashMap QualifiedIdentifier [ProcDescription]
procReturnsScalar :: ProcDescription -> Bool
procReturnsScalar proc = case proc of
ProcDescription{pdReturnType = Just (Single Scalar)} -> True
ProcDescription{pdReturnType = Just (SetOf Scalar)} -> True
_ -> False
procReturnsSingle :: ProcDescription -> Bool
procReturnsSingle proc = case proc of
ProcDescription{pdReturnType = Just (Single _)} -> True
_ -> False
procReturnsVoid :: ProcDescription -> Bool
procReturnsVoid proc = case proc of
ProcDescription{pdReturnType = Nothing} -> True
_ -> False
procTableName :: ProcDescription -> Maybe TableName
procTableName proc = case pdReturnType proc of
Just (SetOf (Composite qi)) -> Just $ qiName qi
Just (Single (Composite qi)) -> Just $ qiName qi
_ -> Nothing
+64
View File
@@ -0,0 +1,64 @@
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
module PostgREST.DbStructure.Relationship
( Cardinality(..)
, Relationship(..)
, Junction(..)
, RelationshipsMap
) where
import qualified Data.Aeson as JSON
import qualified Data.HashMap.Strict as HM
import PostgREST.DbStructure.Identifiers (FieldName,
QualifiedIdentifier, Schema)
import Protolude
-- | Relationship between two tables.
data Relationship = Relationship
{ relTable :: QualifiedIdentifier
, relForeignTable :: QualifiedIdentifier
, relIsSelf :: Bool -- ^ Whether is a self relationship
, relCardinality :: Cardinality
, relTableIsView :: Bool
, relFTableIsView :: Bool
}
| ComputedRelationship
{ relFunction :: QualifiedIdentifier
, relTable :: QualifiedIdentifier
, relForeignTable :: QualifiedIdentifier
, relToOne :: Bool
, relIsSelf :: Bool
}
deriving (Eq, Ord, Generic, JSON.ToJSON)
-- | The relationship cardinality
-- | https://en.wikipedia.org/wiki/Cardinality_(data_modeling)
data Cardinality
= O2M {relCons :: FKConstraint, relColumns :: [(FieldName, FieldName)]}
-- ^ one-to-many
| M2O {relCons :: FKConstraint, relColumns :: [(FieldName, FieldName)]}
-- ^ many-to-one
| O2O {relCons :: FKConstraint, relColumns :: [(FieldName, FieldName)]}
-- ^ one-to-one, this is a refinement over M2O so operating on it is pretty much the same as M2O
| M2M Junction
-- ^ many-to-many
deriving (Eq, Ord, Generic, JSON.ToJSON)
type FKConstraint = Text
-- | Junction table on an M2M relationship
data Junction = Junction
{ junTable :: QualifiedIdentifier
, junConstraint1 :: FKConstraint
, junConstraint2 :: FKConstraint
, junColumns1 :: [(FieldName, FieldName)]
, junColumns2 :: [(FieldName, FieldName)]
}
deriving (Eq, Ord, Generic, JSON.ToJSON)
-- | Key based on the source table and the foreign table schema
type RelationshipsMap = HM.HashMap (QualifiedIdentifier, Schema) [Relationship]
+49
View File
@@ -0,0 +1,49 @@
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
module PostgREST.DbStructure.Table
( Column(..)
, Table(..)
, TablesMap
) where
import qualified Data.Aeson as JSON
import qualified Data.HashMap.Strict as HM
import PostgREST.DbStructure.Identifiers (FieldName,
QualifiedIdentifier (..),
Schema, TableName)
import Protolude
data Table = Table
{ tableSchema :: Schema
, tableName :: TableName
, tableDescription :: Maybe Text
-- TODO Find a better way to separate tables and views
, tableIsView :: Bool
-- The following fields identify what can be done on the table/view, they're not related to the privileges granted to it
, tableInsertable :: Bool
, tableUpdatable :: Bool
, tableDeletable :: Bool
, tablePKCols :: [FieldName]
, tableColumns :: [Column]
}
deriving (Show, Ord, Generic, JSON.ToJSON)
instance Eq Table where
Table{tableSchema=s1,tableName=n1} == Table{tableSchema=s2,tableName=n2} = s1 == s2 && n1 == n2
data Column = Column
{ colName :: FieldName
, colDescription :: Maybe Text
, colNullable :: Bool
, colType :: Text
, colMaxLen :: Maybe Int32
, colDefault :: Maybe Text
, colEnum :: [Text]
}
deriving (Eq, Show, Ord, Generic, JSON.ToJSON)
type TablesMap = HM.HashMap QualifiedIdentifier Table
+482 -169
View File
@@ -1,191 +1,504 @@
{-|
Module : PostgREST.Error
Description : PostgREST error HTTP responses
-}
{-# OPTIONS_GHC -fno-warn-orphans #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE TypeSynonymInstances #-}
{-# LANGUAGE RecordWildCards #-}
module PostgREST.Error (
apiRequestError
, pgError
, simpleError
, singularityError
, binaryFieldError
, connectionLostError
, encodeError
, gucHeadersError
) where
module PostgREST.Error
( errorResponseFor
, ApiRequestError(..)
, PgError(..)
, Error(..)
, errorPayload
, checkIsFatal
, singularityError
) where
import Protolude
import Data.Aeson ((.=))
import qualified Data.Aeson as JSON
import Data.Text (unwords)
import qualified Hasql.Pool as P
import qualified Hasql.Session as H
import Network.HTTP.Types.Header
import qualified Network.HTTP.Types.Status as HT
import Network.Wai (Response, responseLBS)
import PostgREST.Types
import Text.Read (readMaybe)
import qualified Data.ByteString.Char8 as BS
import qualified Data.Text as T
import qualified Data.Text.Encoding as T
import qualified Data.Text.Encoding.Error as T
import qualified Hasql.Pool as SQL
import qualified Hasql.Session as SQL
import qualified Network.HTTP.Types.Status as HTTP
apiRequestError :: ApiRequestError -> Response
apiRequestError err =
errorResponse status
[toHeader CTApplicationJSON] err
where
status =
case err of
ActionInappropriate -> HT.status405
UnsupportedVerb -> HT.status405
InvalidBody _ -> HT.status400
ParseRequestError _ _ -> HT.status400
NoRelationBetween _ _ -> HT.status400
InvalidRange -> HT.status416
UnknownRelation -> HT.status404
InvalidFilters -> HT.status405
import Data.Aeson ((.=))
import Network.Wai (Response, responseLBS)
simpleError :: HT.Status -> [Header] -> Text -> Response
simpleError status hdrs message =
errorResponse status (toHeader CTApplicationJSON : hdrs) $
JSON.object ["message" .= message]
import Network.HTTP.Types.Header (Header)
errorResponse :: JSON.ToJSON a => HT.Status -> [Header] -> a -> Response
errorResponse status hdrs e =
responseLBS status hdrs $ encodeError e
import PostgREST.MediaType (MediaType (..))
import qualified PostgREST.MediaType as MediaType
import PostgREST.Request.Types (ApiRequestError (..),
QPError (..))
pgError :: Bool -> P.UsageError -> Response
pgError authed e =
let status = httpStatus authed e
jsonType = toHeader CTApplicationJSON
wwwAuth = ("WWW-Authenticate", "Bearer")
hdrs = if status == HT.status401
then [jsonType, wwwAuth]
else [jsonType] in
responseLBS status hdrs (encodeError e)
singularityError :: Integer -> Response
singularityError numRows =
responseLBS HT.status406
[toHeader CTSingularJSON]
$ toS . formatGeneralError
"JSON object requested, multiple (or no) rows returned"
$ unwords
[ "Results contain", show numRows, "rows,"
, toS (toMime CTSingularJSON), "requires 1 row"
]
where
formatGeneralError :: Text -> Text -> Text
formatGeneralError message details = toS . JSON.encode $
JSON.object ["message" .= message, "details" .= details]
import PostgREST.DbStructure.Identifiers (QualifiedIdentifier (..))
import PostgREST.DbStructure.Proc (ProcDescription (..),
ProcParam (..))
import PostgREST.DbStructure.Relationship (Cardinality (..),
Junction (..),
Relationship (..))
import Protolude
binaryFieldError :: Response
binaryFieldError =
simpleError HT.status406 [] (toS (toMime CTOctetStream) <>
" requested but a single column was not selected")
class (JSON.ToJSON a) => PgrstError a where
status :: a -> HTTP.Status
headers :: a -> [Header]
gucHeadersError :: Response
gucHeadersError =
simpleError HT.status500 []
"response.headers guc must be a JSON array composed of objects with a single key and a string value"
errorPayload :: a -> LByteString
errorPayload = JSON.encode
connectionLostError :: Response
connectionLostError =
simpleError HT.status503 [] "Database connection lost, retrying the connection."
errorResponseFor :: a -> Response
errorResponseFor err = responseLBS (status err) (headers err) $ errorPayload err
encodeError :: JSON.ToJSON a => a -> LByteString
encodeError = JSON.encode
instance PgrstError ApiRequestError where
status AmbiguousRelBetween{} = HTTP.status300
status AmbiguousRpc{} = HTTP.status300
status MediaTypeError{} = HTTP.status415
status InvalidBody{} = HTTP.status400
status InvalidFilters = HTTP.status405
status InvalidRpcMethod{} = HTTP.status405
status InvalidRange = HTTP.status416
status NotFound = HTTP.status404
status NoRelBetween{} = HTTP.status400
status NoRpc{} = HTTP.status404
status NotEmbedded{} = HTTP.status400
status ParseRequestError{} = HTTP.status400
status PutRangeNotAllowedError = HTTP.status400
status QueryParamError{} = HTTP.status400
status UnacceptableSchema{} = HTTP.status406
status UnsupportedMethod{} = HTTP.status405
status LimitNoOrderError = HTTP.status400
headers _ = [MediaType.toContentType MTApplicationJSON]
instance JSON.ToJSON ApiRequestError where
toJSON (ParseRequestError message details) = JSON.object [
"message" .= message, "details" .= details]
toJSON ActionInappropriate = JSON.object [
"message" .= ("Bad Request" :: Text)]
toJSON (QueryParamError (QPError message details)) = JSON.object [
"code" .= ApiRequestErrorCode00,
"message" .= message,
"details" .= details,
"hint" .= JSON.Null]
toJSON (InvalidRpcMethod method) = JSON.object [
"code" .= ApiRequestErrorCode01,
"message" .= ("Cannot use the " <> T.decodeUtf8 method <> " method on RPC"),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (InvalidBody errorMessage) = JSON.object [
"message" .= (toS errorMessage :: Text)]
"code" .= ApiRequestErrorCode02,
"message" .= T.decodeUtf8 errorMessage,
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON InvalidRange = JSON.object [
"message" .= ("HTTP Range error" :: Text)]
toJSON UnknownRelation = JSON.object [
"message" .= ("Unknown relation" :: Text)]
toJSON (NoRelationBetween parent child) = JSON.object [
"message" .= ("Could not find foreign keys between these entities, No relation found between " <> parent <> " and " <> child :: Text)]
toJSON UnsupportedVerb = JSON.object [
"message" .= ("Unsupported HTTP verb" :: Text)]
"code" .= ApiRequestErrorCode03,
"message" .= ("HTTP Range error" :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (ParseRequestError message details) = JSON.object [
"code" .= ApiRequestErrorCode04,
"message" .= message,
"details" .= details,
"hint" .= JSON.Null]
toJSON InvalidFilters = JSON.object [
"message" .= ("Filters must include all and only primary key columns with 'eq' operators" :: Text)]
"code" .= ApiRequestErrorCode05,
"message" .= ("Filters must include all and only primary key columns with 'eq' operators" :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (UnacceptableSchema schemas) = JSON.object [
"code" .= ApiRequestErrorCode06,
"message" .= ("The schema must be one of the following: " <> T.intercalate ", " schemas),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (MediaTypeError cts) = JSON.object [
"code" .= ApiRequestErrorCode07,
"message" .= ("None of these media types are available: " <> T.intercalate ", " (map T.decodeUtf8 cts)),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON NotFound = JSON.object []
toJSON (NotEmbedded resource) = JSON.object [
"code" .= ApiRequestErrorCode08,
"message" .= ("Cannot apply filter because '" <> resource <> "' is not an embedded resource in this request" :: Text),
"details" .= JSON.Null,
"hint" .= ("Verify that '" <> resource <> "' is included in the 'select' query parameter." :: Text)]
instance JSON.ToJSON P.UsageError where
toJSON (P.ConnectionError e) = JSON.object [
"code" .= ("" :: Text),
"message" .= ("Database connection error" :: Text),
"details" .= (toS $ fromMaybe "" e :: Text)]
toJSON (P.SessionError e) = JSON.toJSON e -- H.Error
toJSON LimitNoOrderError = JSON.object [
"code" .= ApiRequestErrorCode09,
"message" .= ("A 'limit' was applied without an explicit 'order'":: Text),
"details" .= JSON.Null,
"hint" .= ("Apply an 'order' using unique column(s)" :: Text)]
instance JSON.ToJSON H.QueryError where
toJSON (H.QueryError _ _ e) = JSON.toJSON e
toJSON PutRangeNotAllowedError = JSON.object [
"code" .= ApiRequestErrorCode14,
"message" .= ("Range header and limit/offset querystring parameters are not allowed for PUT" :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
instance JSON.ToJSON H.CommandError where
toJSON (H.ResultError (H.ServerError c m d h)) = case toS c of
'P':'T':_ ->
JSON.object [
"details" .= (fmap toS d::Maybe Text),
"hint" .= (fmap toS h::Maybe Text)]
_ ->
JSON.object [
"code" .= (toS c::Text),
"message" .= (toS m::Text),
"details" .= (fmap toS d::Maybe Text),
"hint" .= (fmap toS h::Maybe Text)]
toJSON (H.ResultError (H.UnexpectedResult m)) = JSON.object [
"message" .= (m::Text)]
toJSON (H.ResultError (H.RowError i H.EndOfInput)) = JSON.object [
"message" .= ("Row error: end of input"::Text),
"details" .=
("Attempt to parse more columns than there are in the result"::Text),
"details" .= (("Row number " <> show i)::Text)]
toJSON (H.ResultError (H.RowError i H.UnexpectedNull)) = JSON.object [
"message" .= ("Row error: unexpected null"::Text),
"details" .= ("Attempt to parse a NULL as some value."::Text),
"details" .= (("Row number " <> show i)::Text)]
toJSON (H.ResultError (H.RowError i (H.ValueError d))) = JSON.object [
"message" .= ("Row error: Wrong value parser used"::Text),
"details" .= d,
"details" .= (("Row number " <> show i)::Text)]
toJSON (H.ResultError (H.UnexpectedAmountOfRows i)) = JSON.object [
"message" .= ("Unexpected amount of rows"::Text),
"details" .= i]
toJSON (H.ClientError d) = JSON.object [
"message" .= ("Database client error"::Text),
"details" .= (fmap toS d::Maybe Text)]
toJSON (UnsupportedMethod method) = JSON.object [
"code" .= ApiRequestErrorCode17,
"message" .= ("Unsupported HTTP method: " <> T.decodeUtf8 method),
"details" .= JSON.Null,
"hint" .= JSON.Null]
httpStatus :: Bool -> P.UsageError -> HT.Status
httpStatus _ (P.ConnectionError _) = HT.status503
httpStatus authed (P.SessionError (H.QueryError _ _ (H.ResultError (H.ServerError c m _ _)))) =
case toS c of
'0':'8':_ -> HT.status503 -- pg connection err
'0':'9':_ -> HT.status500 -- triggered action exception
'0':'L':_ -> HT.status403 -- invalid grantor
'0':'P':_ -> HT.status403 -- invalid role specification
"23503" -> HT.status409 -- foreign_key_violation
"23505" -> HT.status409 -- unique_violation
'2':'5':_ -> HT.status500 -- invalid tx state
'2':'8':_ -> HT.status403 -- invalid auth specification
'2':'D':_ -> HT.status500 -- invalid tx termination
'3':'8':_ -> HT.status500 -- external routine exception
'3':'9':_ -> HT.status500 -- external routine invocation
'3':'B':_ -> HT.status500 -- savepoint exception
'4':'0':_ -> HT.status500 -- tx rollback
'5':'3':_ -> HT.status503 -- insufficient resources
'5':'4':_ -> HT.status413 -- too complex
'5':'5':_ -> HT.status500 -- obj not on prereq state
'5':'7':_ -> HT.status500 -- operator intervention
'5':'8':_ -> HT.status500 -- system error
'F':'0':_ -> HT.status500 -- conf file error
'H':'V':_ -> HT.status500 -- foreign data wrapper error
"P0001" -> HT.status400 -- default code for "raise"
'P':'0':_ -> HT.status500 -- PL/pgSQL Error
'X':'X':_ -> HT.status500 -- internal Error
"42883" -> HT.status404 -- undefined function
"42P01" -> HT.status404 -- undefined table
"42501" -> if authed then HT.status403 else HT.status401 -- insufficient privilege
'P':'T':n -> fromMaybe HT.status500 (HT.mkStatus <$> readMaybe n <*> pure m)
_ -> HT.status400
httpStatus _ (P.SessionError (H.QueryError _ _ (H.ResultError _))) = HT.status500
httpStatus _ (P.SessionError (H.QueryError _ _ (H.ClientError _))) = HT.status503
toJSON (NoRelBetween parent child schema) = JSON.object [
"code" .= SchemaCacheErrorCode00,
"message" .= ("Could not find a relationship between '" <> parent <> "' and '" <> child <> "' in the schema cache" :: Text),
"details" .= JSON.Null,
"hint" .= ("Verify that '" <> parent <> "' and '" <> child <> "' exist in the schema '" <> schema <> "' and that there is a foreign key relationship between them. If a new relationship was created, try reloading the schema cache." :: Text)]
toJSON (AmbiguousRelBetween parent child rels) = JSON.object [
"code" .= SchemaCacheErrorCode01,
"message" .= ("Could not embed because more than one relationship was found for '" <> parent <> "' and '" <> child <> "'" :: Text),
"details" .= (compressedRel <$> rels),
"hint" .= ("Try changing '" <> child <> "' to one of the following: " <> relHint rels <> ". Find the desired relationship in the 'details' key." :: Text)]
toJSON (NoRpc schema procName argumentKeys hasPreferSingleObject contentType isInvPost) =
let prms = "(" <> T.intercalate ", " argumentKeys <> ")" in JSON.object [
"code" .= SchemaCacheErrorCode02,
"message" .= ("Could not find the " <> schema <> "." <> procName <>
(case (hasPreferSingleObject, isInvPost, contentType) of
(True, _, _) -> " function with a single json or jsonb parameter"
(_, True, MTTextPlain) -> " function with a single unnamed text parameter"
(_, True, MTTextXML) -> " function with a single unnamed xml parameter"
(_, True, MTOctetStream) -> " function with a single unnamed bytea parameter"
(_, True, MTApplicationJSON) -> prms <> " function or the " <> schema <> "." <> procName <>" function with a single unnamed json or jsonb parameter"
_ -> prms <> " function") <>
" in the schema cache"),
"details" .= JSON.Null,
"hint" .= ("If a new function was created in the database with this name and parameters, try reloading the schema cache." :: Text)]
toJSON (AmbiguousRpc procs) = JSON.object [
"code" .= SchemaCacheErrorCode03,
"message" .= ("Could not choose the best candidate function between: " <> T.intercalate ", " [pdSchema p <> "." <> pdName p <> "(" <> T.intercalate ", " [ppName a <> " => " <> ppType a | a <- pdParams p] <> ")" | p <- procs]),
"details" .= JSON.Null,
"hint" .= ("Try renaming the parameters or the function itself in the database so function overloading can be resolved" :: Text)]
compressedRel :: Relationship -> JSON.Value
-- An ambiguousness error cannot happen for computed relationships TODO refactor so this mempty is not needed
compressedRel ComputedRelationship{} = JSON.object mempty
compressedRel Relationship{..} =
let
fmtEls els = "(" <> T.intercalate ", " els <> ")"
in
JSON.object $
("embedding" .= (qiName relTable <> " with " <> qiName relForeignTable :: Text))
: case relCardinality of
M2M Junction{..} -> [
"cardinality" .= ("many-to-many" :: Text)
, "relationship" .= (qiName junTable <> " using " <> junConstraint1 <> fmtEls (snd <$> junColumns1) <> " and " <> junConstraint2 <> fmtEls (snd <$> junColumns2))
]
M2O cons relColumns -> [
"cardinality" .= ("many-to-one" :: Text)
, "relationship" .= (cons <> " using " <> qiName relTable <> fmtEls (fst <$> relColumns) <> " and " <> qiName relForeignTable <> fmtEls (snd <$> relColumns))
]
O2O cons relColumns -> [
"cardinality" .= ("one-to-one" :: Text)
, "relationship" .= (cons <> " using " <> qiName relTable <> fmtEls (fst <$> relColumns) <> " and " <> qiName relForeignTable <> fmtEls (snd <$> relColumns))
]
O2M cons relColumns -> [
"cardinality" .= ("one-to-many" :: Text)
, "relationship" .= (cons <> " using " <> qiName relTable <> fmtEls (fst <$> relColumns) <> " and " <> qiName relForeignTable <> fmtEls (snd <$> relColumns))
]
relHint :: [Relationship] -> Text
relHint rels = T.intercalate ", " (hintList <$> rels)
where
hintList Relationship{..} =
let buildHint rel = "'" <> qiName relForeignTable <> "!" <> rel <> "'" in
case relCardinality of
M2M Junction{..} -> buildHint (qiName junTable)
M2O cons _ -> buildHint cons
O2O cons _ -> buildHint cons
O2M cons _ -> buildHint cons
-- An ambiguousness error cannot happen for computed relationships TODO refactor so this mempty is not needed
hintList ComputedRelationship{} = mempty
data PgError = PgError Authenticated SQL.UsageError
type Authenticated = Bool
instance PgrstError PgError where
status (PgError authed usageError) = pgErrorStatus authed usageError
headers err =
if status err == HTTP.status401
then [MediaType.toContentType MTApplicationJSON, ("WWW-Authenticate", "Bearer") :: Header]
else [MediaType.toContentType MTApplicationJSON]
instance JSON.ToJSON PgError where
toJSON (PgError _ usageError) = JSON.toJSON usageError
instance JSON.ToJSON SQL.UsageError where
toJSON (SQL.ConnectionError e) = JSON.object [
"code" .= ConnectionErrorCode00,
"message" .= ("Database connection error. Retrying the connection." :: Text),
"details" .= (T.decodeUtf8With T.lenientDecode $ fromMaybe "" e :: Text),
"hint" .= JSON.Null]
toJSON (SQL.SessionError e) = JSON.toJSON e -- SQL.Error
instance JSON.ToJSON SQL.QueryError where
toJSON (SQL.QueryError _ _ e) = JSON.toJSON e
instance JSON.ToJSON SQL.CommandError where
toJSON (SQL.ResultError (SQL.ServerError c m d h)) = JSON.object [
"code" .= (T.decodeUtf8 c :: Text),
"message" .= (T.decodeUtf8 m :: Text),
"details" .= (fmap T.decodeUtf8 d :: Maybe Text),
"hint" .= (fmap T.decodeUtf8 h :: Maybe Text)]
toJSON (SQL.ResultError resultError) = JSON.object [
"code" .= InternalErrorCode00,
"message" .= (show resultError :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (SQL.ClientError d) = JSON.object [
"code" .= ConnectionErrorCode01,
"message" .= ("Database client error. Retrying the connection." :: Text),
"details" .= (fmap T.decodeUtf8 d :: Maybe Text),
"hint" .= JSON.Null]
pgErrorStatus :: Bool -> SQL.UsageError -> HTTP.Status
pgErrorStatus _ (SQL.ConnectionError _) = HTTP.status503
pgErrorStatus _ (SQL.SessionError (SQL.QueryError _ _ (SQL.ClientError _))) = HTTP.status503
pgErrorStatus authed (SQL.SessionError (SQL.QueryError _ _ (SQL.ResultError rError))) =
case rError of
(SQL.ServerError c m _ _) ->
case BS.unpack c of
'0':'8':_ -> HTTP.status503 -- pg connection err
'0':'9':_ -> HTTP.status500 -- triggered action exception
'0':'L':_ -> HTTP.status403 -- invalid grantor
'0':'P':_ -> HTTP.status403 -- invalid role specification
"23503" -> HTTP.status409 -- foreign_key_violation
"23505" -> HTTP.status409 -- unique_violation
"25006" -> HTTP.status405 -- read_only_sql_transaction
'2':'5':_ -> HTTP.status500 -- invalid tx state
'2':'8':_ -> HTTP.status403 -- invalid auth specification
'2':'D':_ -> HTTP.status500 -- invalid tx termination
'3':'8':_ -> HTTP.status500 -- external routine exception
'3':'9':_ -> HTTP.status500 -- external routine invocation
'3':'B':_ -> HTTP.status500 -- savepoint exception
'4':'0':_ -> HTTP.status500 -- tx rollback
'5':'3':_ -> HTTP.status503 -- insufficient resources
'5':'4':_ -> HTTP.status413 -- too complex
'5':'5':_ -> HTTP.status500 -- obj not on prereq state
'5':'7':_ -> HTTP.status500 -- operator intervention
'5':'8':_ -> HTTP.status500 -- system error
'F':'0':_ -> HTTP.status500 -- conf file error
'H':'V':_ -> HTTP.status500 -- foreign data wrapper error
"P0001" -> HTTP.status400 -- default code for "raise"
'P':'0':_ -> HTTP.status500 -- PL/pgSQL Error
'X':'X':_ -> HTTP.status500 -- internal Error
"42883"-> if BS.isPrefixOf "function xmlagg(" m
then HTTP.status406
else HTTP.status404 -- undefined function
"42P01" -> HTTP.status404 -- undefined table
"42501" -> if authed then HTTP.status403 else HTTP.status401 -- insufficient privilege
'P':'T':n -> fromMaybe HTTP.status500 (HTTP.mkStatus <$> readMaybe n <*> pure m)
_ -> HTTP.status400
_ -> HTTP.status500
checkIsFatal :: PgError -> Maybe Text
checkIsFatal (PgError _ (SQL.ConnectionError e))
| isAuthFailureMessage = Just $ toS failureMessage
| otherwise = Nothing
where isAuthFailureMessage = "FATAL: password authentication failed" `isPrefixOf` failureMessage
failureMessage = BS.unpack $ fromMaybe mempty e
checkIsFatal (PgError _ (SQL.SessionError (SQL.QueryError _ _ (SQL.ResultError serverError))))
= case serverError of
-- Check for a syntax error (42601 is the pg code). This would mean the error is on our part somehow, so we treat it as fatal.
SQL.ServerError "42601" _ _ _
-> Just "Hint: This is probably a bug in PostgREST, please report it at https://github.com/PostgREST/postgrest/issues"
-- Check for a "prepared statement <name> already exists" error (Code 42P05: duplicate_prepared_statement).
-- This would mean that a connection pooler in transaction mode is being used
-- while prepared statements are enabled in the PostgREST configuration,
-- both of which are incompatible with each other.
SQL.ServerError "42P05" _ _ _
-> Just "Hint: If you are using connection poolers in transaction mode, try setting db-prepared-statements to false."
-- Check for a "transaction blocks not allowed in statement pooling mode" error (Code 08P01: protocol_violation).
-- This would mean that a connection pooler in statement mode is being used which is not supported in PostgREST.
SQL.ServerError "08P01" "transaction blocks not allowed in statement pooling mode" _ _
-> Just "Hint: Connection poolers in statement mode are not supported."
_ -> Nothing
checkIsFatal _ = Nothing
data Error
= ApiRequestError ApiRequestError
| BinaryFieldError MediaType
| GucHeadersError
| GucStatusError
| JwtTokenInvalid Text
| JwtTokenMissing
| JwtTokenRequired
| NoSchemaCacheError
| OffLimitsChangesError Int64 Integer
| PgErr PgError
| PutMatchingPkError
| SingularityError Integer
instance PgrstError Error where
status (ApiRequestError err) = status err
status BinaryFieldError{} = HTTP.status406
status GucHeadersError = HTTP.status500
status GucStatusError = HTTP.status500
status JwtTokenInvalid{} = HTTP.unauthorized401
status JwtTokenMissing = HTTP.status500
status JwtTokenRequired = HTTP.unauthorized401
status NoSchemaCacheError = HTTP.status503
status OffLimitsChangesError{} = HTTP.status400
status (PgErr err) = status err
status PutMatchingPkError = HTTP.status400
status SingularityError{} = HTTP.status406
headers (ApiRequestError err) = headers err
headers (JwtTokenInvalid m) = [MediaType.toContentType MTApplicationJSON, invalidTokenHeader m]
headers JwtTokenRequired = [MediaType.toContentType MTApplicationJSON, requiredTokenHeader]
headers (PgErr err) = headers err
headers SingularityError{} = [MediaType.toContentType MTSingularJSON]
headers _ = [MediaType.toContentType MTApplicationJSON]
instance JSON.ToJSON Error where
toJSON NoSchemaCacheError = JSON.object [
"code" .= ConnectionErrorCode02,
"message" .= ("Could not query the database for the schema cache. Retrying." :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON JwtTokenMissing = JSON.object [
"code" .= JWTErrorCode00,
"message" .= ("Server lacks JWT secret" :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (JwtTokenInvalid message) = JSON.object [
"code" .= JWTErrorCode01,
"message" .= (message :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON JwtTokenRequired = JSON.object [
"code" .= JWTErrorCode02,
"message" .= ("Anonymous access is disabled" :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (OffLimitsChangesError n maxs) = JSON.object [
"code" .= ApiRequestErrorCode10,
"message" .= ("The maximum number of rows allowed to change was surpassed" :: Text),
"details" .= T.unwords ["Results contain", show n, "rows changed but the maximum number allowed is", show maxs],
"hint" .= JSON.Null]
toJSON GucHeadersError = JSON.object [
"code" .= ApiRequestErrorCode11,
"message" .= ("response.headers guc must be a JSON array composed of objects with a single key and a string value" :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON GucStatusError = JSON.object [
"code" .= ApiRequestErrorCode12,
"message" .= ("response.status guc must be a valid status code" :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (BinaryFieldError ct) = JSON.object [
"code" .= ApiRequestErrorCode13,
"message" .= ((T.decodeUtf8 (MediaType.toMime ct) <> " requested but more than one column was selected") :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON PutMatchingPkError = JSON.object [
"code" .= ApiRequestErrorCode15,
"message" .= ("Payload values do not match URL in primary key column(s)" :: Text),
"details" .= JSON.Null,
"hint" .= JSON.Null]
toJSON (SingularityError n) = JSON.object [
"code" .= ApiRequestErrorCode16,
"message" .= ("JSON object requested, multiple (or no) rows returned" :: Text),
"details" .= T.unwords ["Results contain", show n, "rows,", T.decodeUtf8 (MediaType.toMime MTSingularJSON), "requires 1 row"],
"hint" .= JSON.Null]
toJSON (PgErr err) = JSON.toJSON err
toJSON (ApiRequestError err) = JSON.toJSON err
invalidTokenHeader :: Text -> Header
invalidTokenHeader m =
("WWW-Authenticate", "Bearer error=\"invalid_token\", " <> "error_description=" <> encodeUtf8 (show m))
requiredTokenHeader :: Header
requiredTokenHeader = ("WWW-Authenticate", "Bearer")
singularityError :: (Integral a) => a -> Error
singularityError = SingularityError . toInteger
-- Error codes are grouped by common modules or characteristics
data ErrorCode
-- PostgreSQL connection errors
= ConnectionErrorCode00
| ConnectionErrorCode01
| ConnectionErrorCode02
-- API Request errors
| ApiRequestErrorCode00
| ApiRequestErrorCode01
| ApiRequestErrorCode02
| ApiRequestErrorCode03
| ApiRequestErrorCode04
| ApiRequestErrorCode05
| ApiRequestErrorCode06
| ApiRequestErrorCode07
| ApiRequestErrorCode08
| ApiRequestErrorCode09
| ApiRequestErrorCode10
| ApiRequestErrorCode11
| ApiRequestErrorCode12
| ApiRequestErrorCode13
| ApiRequestErrorCode14
| ApiRequestErrorCode15
| ApiRequestErrorCode16
| ApiRequestErrorCode17
-- Schema Cache errors
| SchemaCacheErrorCode00
| SchemaCacheErrorCode01
| SchemaCacheErrorCode02
| SchemaCacheErrorCode03
-- JWT authentication errors
| JWTErrorCode00
| JWTErrorCode01
| JWTErrorCode02
-- Internal errors related to the Hasql library
| InternalErrorCode00
instance JSON.ToJSON ErrorCode where
toJSON e = JSON.toJSON (buildErrorCode e)
-- New group of errors will be added at the end of all the groups and will have the next prefix in the sequence
-- New errors are added at the end of the group they belong to and will have the next code in the sequence
buildErrorCode :: ErrorCode -> Text
buildErrorCode code = "PGRST" <> case code of
ConnectionErrorCode00 -> "000"
ConnectionErrorCode01 -> "001"
ConnectionErrorCode02 -> "002"
ApiRequestErrorCode00 -> "100"
ApiRequestErrorCode01 -> "101"
ApiRequestErrorCode02 -> "102"
ApiRequestErrorCode03 -> "103"
ApiRequestErrorCode04 -> "104"
ApiRequestErrorCode05 -> "105"
ApiRequestErrorCode06 -> "106"
ApiRequestErrorCode07 -> "107"
ApiRequestErrorCode08 -> "108"
ApiRequestErrorCode09 -> "109"
ApiRequestErrorCode10 -> "110"
ApiRequestErrorCode11 -> "111"
ApiRequestErrorCode12 -> "112"
ApiRequestErrorCode13 -> "113"
ApiRequestErrorCode14 -> "114"
ApiRequestErrorCode15 -> "115"
ApiRequestErrorCode16 -> "116"
ApiRequestErrorCode17 -> "117"
SchemaCacheErrorCode00 -> "200"
SchemaCacheErrorCode01 -> "201"
SchemaCacheErrorCode02 -> "202"
SchemaCacheErrorCode03 -> "203"
JWTErrorCode00 -> "300"
JWTErrorCode01 -> "301"
JWTErrorCode02 -> "302"
InternalErrorCode00 -> "X00"
+37
View File
@@ -0,0 +1,37 @@
module PostgREST.GucHeader
( GucHeader
, unwrapGucHeader
, addHeadersIfNotIncluded
) where
import qualified Data.Aeson as JSON
import qualified Data.Aeson.Key as K
import qualified Data.Aeson.KeyMap as KM
import qualified Data.CaseInsensitive as CI
import Network.HTTP.Types.Header (Header)
import Protolude
{-|
Custom guc header, it's obtained by parsing the json in a:
`SET LOCAL "response.headers" = '[{"Set-Cookie": ".."}]'
-}
newtype GucHeader = GucHeader (CI.CI ByteString, ByteString)
instance JSON.FromJSON GucHeader where
parseJSON (JSON.Object o) =
case KM.toList o of
[(k, JSON.String s)] -> pure $ GucHeader (CI.mk $ toUtf8 $ K.toText k, toUtf8 s)
_ -> mzero
parseJSON _ = mzero
unwrapGucHeader :: GucHeader -> Header
unwrapGucHeader (GucHeader (k, v)) = (k, v)
-- | Add headers not already included to allow the user to override them instead of duplicating them
addHeadersIfNotIncluded :: [Header] -> [Header] -> [Header]
addHeadersIfNotIncluded newHeaders initialHeaders =
filter (\(nk, _) -> isNothing $ find (\(ik, _) -> ik == nk) initialHeaders) newHeaders ++
initialHeaders
+30
View File
@@ -0,0 +1,30 @@
{-|
Module : PostgREST.Logger
Description : Wai Middleware to log requests to stdout.
-}
module PostgREST.Logger (middleware) where
import qualified Network.Wai as Wai
import qualified Network.Wai.Middleware.RequestLogger as Wai
import Network.HTTP.Types.Status (status400, status500)
import System.IO.Unsafe (unsafePerformIO)
import qualified PostgREST.Auth as Auth
import PostgREST.Config (LogLevel (..))
import Protolude
middleware :: LogLevel -> Wai.Middleware
middleware logLevel = case logLevel of
LogInfo -> requestLogger (const True)
LogWarn -> requestLogger (>= status400)
LogError -> requestLogger (>= status500)
LogCrit -> requestLogger (const False)
where
requestLogger filterStatus = unsafePerformIO $ Wai.mkRequestLogger Wai.defaultRequestLoggerSettings
{ Wai.outputFormat = Wai.ApacheWithSettings $
Wai.defaultApacheSettings
& Wai.setApacheRequestFilter (\_ res -> filterStatus $ Wai.responseStatus res)
& Wai.setApacheUserGetter (fmap encodeUtf8 . Auth.getRole)
}
+124
View File
@@ -0,0 +1,124 @@
{-# LANGUAGE DuplicateRecordFields #-}
module PostgREST.MediaType
( MediaType(..)
, MTPlanOption (..)
, MTPlanFormat (..)
, MTPlanAttrs(..)
, toContentType
, toMime
, decodeMediaType
, getMediaType
) where
import qualified Data.ByteString as BS
import qualified Data.ByteString.Internal as BS (c2w)
import Data.Maybe (fromJust)
import Network.HTTP.Types.Header (Header, hContentType)
import Protolude
-- | Enumeration of currently supported media types
data MediaType
= MTApplicationJSON
| MTSingularJSON
| MTGeoJSON
| MTTextCSV
| MTTextPlain
| MTTextXML
| MTOpenAPI
| MTUrlEncoded
| MTOctetStream
| MTAny
| MTOther ByteString
| MTPlan MTPlanAttrs
deriving Eq
data MTPlanAttrs = MTPlanAttrs (Maybe MediaType) MTPlanFormat [MTPlanOption]
instance Eq MTPlanAttrs where
MTPlanAttrs {} == MTPlanAttrs {} = True -- we don't care about the attributes when comparing two MTPlan media types
data MTPlanOption
= PlanAnalyze | PlanVerbose | PlanSettings | PlanBuffers | PlanWAL
data MTPlanFormat
= PlanJSON | PlanText
-- | Convert MediaType to a Content-Type HTTP Header
toContentType :: MediaType -> Header
toContentType ct = (hContentType, toMime ct <> charset)
where
charset = case ct of
MTOctetStream -> mempty
MTOther _ -> mempty
_ -> "; charset=utf-8"
-- | Convert from MediaType to a ByteString representing the mime type
toMime :: MediaType -> ByteString
toMime MTApplicationJSON = "application/json"
toMime MTGeoJSON = "application/geo+json"
toMime MTTextCSV = "text/csv"
toMime MTTextPlain = "text/plain"
toMime MTTextXML = "text/xml"
toMime MTOpenAPI = "application/openapi+json"
toMime MTSingularJSON = "application/vnd.pgrst.object+json"
toMime MTUrlEncoded = "application/x-www-form-urlencoded"
toMime MTOctetStream = "application/octet-stream"
toMime MTAny = "*/*"
toMime (MTOther ct) = ct
toMime (MTPlan (MTPlanAttrs mt fmt opts)) =
"application/vnd.pgrst.plan+" <> toMimePlanFormat fmt <>
(if isNothing mt then mempty else "; for=\"" <> toMime (fromJust mt) <> "\"") <>
(if null opts then mempty else "; options=" <> BS.intercalate "|" (toMimePlanOption <$> opts))
toMimePlanOption :: MTPlanOption -> ByteString
toMimePlanOption PlanAnalyze = "analyze"
toMimePlanOption PlanVerbose = "verbose"
toMimePlanOption PlanSettings = "settings"
toMimePlanOption PlanBuffers = "buffers"
toMimePlanOption PlanWAL = "wal"
toMimePlanFormat :: MTPlanFormat -> ByteString
toMimePlanFormat PlanJSON = "json"
toMimePlanFormat PlanText = "text"
-- | Convert from ByteString to MediaType. Warning: discards MIME parameters
decodeMediaType :: BS.ByteString -> MediaType
decodeMediaType mt =
case BS.split (BS.c2w ';') mt of
"application/json":_ -> MTApplicationJSON
"application/geo+json":_ -> MTGeoJSON
"text/csv":_ -> MTTextCSV
"text/plain":_ -> MTTextPlain
"text/xml":_ -> MTTextXML
"application/openapi+json":_ -> MTOpenAPI
"application/vnd.pgrst.object+json":_ -> MTSingularJSON
"application/vnd.pgrst.object":_ -> MTSingularJSON
"application/x-www-form-urlencoded":_ -> MTUrlEncoded
"application/octet-stream":_ -> MTOctetStream
"application/vnd.pgrst.plan":rest -> getPlan PlanText rest
"application/vnd.pgrst.plan+text":rest -> getPlan PlanText rest
"application/vnd.pgrst.plan+json":rest -> getPlan PlanJSON rest
"*/*":_ -> MTAny
other:_ -> MTOther other
_ -> MTAny
where
getPlan fmt rest =
let
opts = BS.split (BS.c2w '|') $ fromMaybe mempty (BS.stripPrefix "options=" =<< find (BS.isPrefixOf "options=") rest)
inOpts str = str `elem` opts
mtFor = decodeMediaType . dropAround (== BS.c2w '"') <$> (BS.stripPrefix "for=" =<< find (BS.isPrefixOf "for=") rest)
dropAround p = BS.dropWhile p . BS.dropWhileEnd p in
MTPlan $ MTPlanAttrs mtFor fmt $
[PlanAnalyze | inOpts "analyze" ] ++
[PlanVerbose | inOpts "verbose" ] ++
[PlanSettings | inOpts "settings"] ++
[PlanBuffers | inOpts "buffers" ] ++
[PlanWAL | inOpts "wal" ]
getMediaType :: MediaType -> MediaType
getMediaType mt = case mt of
MTPlan (MTPlanAttrs (Just mType) _ _) -> mType
MTPlan (MTPlanAttrs Nothing _ _) -> MTApplicationJSON
other -> other
+114 -55
View File
@@ -1,63 +1,122 @@
{-# OPTIONS_GHC -fno-warn-orphans #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE FlexibleContexts #-}
{-|
Module : PostgREST.Middleware
Description : Sets CORS policy. Also the PostgreSQL GUCs, role, search_path and pre-request function.
-}
{-# LANGUAGE BlockArguments #-}
{-# LANGUAGE RecordWildCards #-}
module PostgREST.Middleware
( runPgLocals
, optionalRollback
) where
module PostgREST.Middleware where
import qualified Data.Aeson as JSON
import qualified Data.Aeson.Key as K
import qualified Data.Aeson.KeyMap as KM
import qualified Data.ByteString.Lazy.Char8 as LBS
import qualified Data.HashMap.Strict as HM
import qualified Data.Text.Encoding as T
import qualified Hasql.Decoders as HD
import qualified Hasql.DynamicStatements.Snippet as SQL hiding (sql)
import qualified Hasql.DynamicStatements.Statement as SQL
import qualified Hasql.Transaction as SQL
import qualified Network.Wai as Wai
import Crypto.JWT
import qualified Data.Aeson as JSON
import qualified Data.HashMap.Strict as M
import qualified Hasql.Transaction as H
import Network.HTTP.Types.Status (unauthorized401, status500)
import Network.Wai (Application, Response)
import Network.Wai.Middleware.Cors (cors)
import Network.Wai.Middleware.Gzip (def, gzip)
import Network.Wai.Middleware.Static (only, staticPolicy)
import Control.Arrow ((***))
import PostgREST.ApiRequest (ApiRequest(..))
import PostgREST.Auth (JWTAttempt(..))
import PostgREST.Config (AppConfig (..), corsPolicy)
import PostgREST.Error (simpleError)
import PostgREST.QueryBuilder (unquoted, pgFmtSetLocal, pgFmtSetLocalSearchPath)
import Data.Scientific (FPFormat (..), formatScientific, isInteger)
import Protolude
import PostgREST.Config (AppConfig (..))
import PostgREST.Config.PgVersion (PgVersion (..), pgVersion140)
import PostgREST.Error (Error, errorResponseFor)
import PostgREST.GucHeader (addHeadersIfNotIncluded)
import PostgREST.Query.SqlFragment (fromQi, intercalateSnippet,
pgFmtIdentList, unknownEncoder)
import PostgREST.Request.ApiRequest (ApiRequest (..), Target (..))
runWithClaims :: AppConfig -> JWTAttempt ->
(ApiRequest -> H.Transaction Response) ->
ApiRequest -> H.Transaction Response
runWithClaims conf eClaims app req =
case eClaims of
JWTInvalid JWTExpired -> return $ unauthed "JWT expired"
JWTInvalid e -> return $ unauthed $ show e
JWTMissingSecret -> return $ simpleError status500 [] "Server lacks JWT secret"
JWTClaims claims -> do
H.sql $ toS . mconcat $ setSearchPathSql : setRoleSql ++ claimsSql ++ headersSql ++ cookiesSql ++ appSettingsSql
mapM_ H.sql customReqCheck
app req
where
headersSql = pgFmtSetLocal "request.header." <$> iHeaders req
cookiesSql = pgFmtSetLocal "request.cookie." <$> iCookies req
claimsSql = pgFmtSetLocal "request.jwt.claim." <$> [(c,unquoted v) | (c,v) <- M.toList claimsWithRole]
appSettingsSql = pgFmtSetLocal mempty <$> configSettings conf
setRoleSql = maybeToList $ (\x ->
pgFmtSetLocal mempty ("role", unquoted x)) <$> M.lookup "role" claimsWithRole
setSearchPathSql = pgFmtSetLocalSearchPath $ configSchema conf : configExtraSearchPath conf
-- role claim defaults to anon if not specified in jwt
claimsWithRole = M.union claims (M.singleton "role" anon)
anon = JSON.String . toS $ configAnonRole conf
customReqCheck = (\f -> "select " <> toS f <> "();") <$> configReqCheck conf
import PostgREST.Request.Preferences
import Protolude
-- | Runs local(transaction scoped) GUCs for every request, plus the pre-request function
runPgLocals :: AppConfig -> KM.KeyMap JSON.Value -> Text ->
(ApiRequest -> ExceptT Error SQL.Transaction Wai.Response) ->
ApiRequest -> ByteString -> PgVersion -> ExceptT Error SQL.Transaction Wai.Response
runPgLocals conf claims role app req jsonDbS actualPgVersion = do
lift $ SQL.statement mempty $ SQL.dynamicallyParameterized
("select " <> intercalateSnippet ", " (searchPathSql : roleSql ++ claimsSql ++ [methodSql, pathSql] ++ headersSql ++ cookiesSql ++ appSettingsSql ++ specSql))
HD.noResult (configDbPreparedStatements conf)
lift $ traverse_ SQL.sql preReqSql
app req
where
unauthed message = simpleError
unauthorized401
[( "WWW-Authenticate"
, "Bearer error=\"invalid_token\", " <>
"error_description=" <> show message
)]
message
methodSql = setConfigLocal mempty ("request.method", iMethod req)
pathSql = setConfigLocal mempty ("request.path", iPath req)
headersSql = if usesLegacyGucs
then setConfigLocal "request.header." <$> iHeaders req
else setConfigLocalJson "request.headers" (iHeaders req)
cookiesSql = if usesLegacyGucs
then setConfigLocal "request.cookie." <$> iCookies req
else setConfigLocalJson "request.cookies" (iCookies req)
claimsSql = if usesLegacyGucs
then setConfigLocal "request.jwt.claim." <$> [(toUtf8 $ K.toText c, toUtf8 $ unquoted v) | (c,v) <- KM.toList claims]
else [setConfigLocal mempty ("request.jwt.claims", LBS.toStrict $ JSON.encode claims)]
roleSql = [setConfigLocal mempty ("role", toUtf8 role)]
appSettingsSql = setConfigLocal mempty <$> (join bimap toUtf8 <$> configAppSettings conf)
searchPathSql =
let schemas = pgFmtIdentList (iSchema req : configDbExtraSearchPath conf) in
setConfigLocal mempty ("search_path", schemas)
preReqSql = (\f -> "select " <> fromQi f <> "();") <$> configDbPreRequest conf
specSql = case iTarget req of
TargetProc{tpIsRootSpec=True} -> [setConfigLocal mempty ("request.spec", jsonDbS)]
_ -> mempty
usesLegacyGucs = configDbUseLegacyGucs conf && actualPgVersion < pgVersion140
defaultMiddle :: Application -> Application
defaultMiddle =
gzip def
. cors corsPolicy
. staticPolicy (only [("favicon.ico", "static/favicon.ico")])
unquoted :: JSON.Value -> Text
unquoted (JSON.String t) = t
unquoted (JSON.Number n) =
toS $ formatScientific Fixed (if isInteger n then Just 0 else Nothing) n
unquoted (JSON.Bool b) = show b
unquoted v = T.decodeUtf8 . LBS.toStrict $ JSON.encode v
-- | Set a transaction to eventually roll back if requested and set respective
-- headers on the response.
optionalRollback
:: AppConfig
-> ApiRequest
-> ExceptT Error SQL.Transaction Wai.Response
-> ExceptT Error SQL.Transaction Wai.Response
optionalRollback AppConfig{..} ApiRequest{..} transaction = do
resp <- catchError transaction $ return . errorResponseFor
when (shouldRollback || (configDbTxRollbackAll && not shouldCommit)) $ lift do
SQL.sql "SET CONSTRAINTS ALL IMMEDIATE"
SQL.condemn
return $ Wai.mapResponseHeaders preferenceApplied resp
where
shouldCommit =
configDbTxAllowOverride && iPreferTransaction == Just Commit
shouldRollback =
configDbTxAllowOverride && iPreferTransaction == Just Rollback
preferenceApplied
| shouldCommit =
addHeadersIfNotIncluded
[toAppliedHeader Commit]
| shouldRollback =
addHeadersIfNotIncluded
[toAppliedHeader Rollback]
| otherwise =
identity
-- | Do a pg set_config(setting, value, true) call. This is equivalent to a SET LOCAL.
setConfigLocal :: ByteString -> (ByteString, ByteString) -> SQL.Snippet
setConfigLocal prefix (k, v) =
"set_config(" <> unknownEncoder (prefix <> k) <> ", " <> unknownEncoder v <> ", true)"
-- | Starting from PostgreSQL v14, some characters are not allowed for config names (mostly affecting headers with "-").
-- | A JSON format string is used to avoid this problem. See https://github.com/PostgREST/postgrest/issues/1857
setConfigLocalJson :: ByteString -> [(ByteString, ByteString)] -> [SQL.Snippet]
setConfigLocalJson prefix keyVals = [setConfigLocal mempty (prefix, gucJsonVal keyVals)]
where
gucJsonVal :: [(ByteString, ByteString)] -> ByteString
gucJsonVal = LBS.toStrict . JSON.encode . HM.fromList . arrayByteStringToText
arrayByteStringToText :: [(ByteString, ByteString)] -> [(Text,Text)]
arrayByteStringToText keyVal = (T.decodeUtf8 *** T.decodeUtf8) <$> keyVal
+182 -158
View File
@@ -1,87 +1,143 @@
{-# LANGUAGE OverloadedStrings #-}
{-|
Module : PostgREST.OpenAPI
Description : Generates the OpenAPI output
-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE RecordWildCards #-}
module PostgREST.OpenAPI (encode) where
module PostgREST.OpenAPI (
encodeOpenAPI
, isMalformedProxyUri
, pickProxy
) where
import qualified Data.Aeson as JSON
import qualified Data.ByteString.Char8 as BS
import qualified Data.ByteString.Lazy as LBS
import qualified Data.HashMap.Strict as HM
import qualified Data.HashSet.InsOrd as Set
import qualified Data.Text as T
import qualified Data.Text.Encoding as T
import Control.Arrow ((&&&))
import Control.Lens
import Data.Aeson (decode, encode)
import Data.HashMap.Strict.InsOrd (InsOrdHashMap, fromList)
import Data.Maybe (fromJust)
import qualified Data.Set as Set
import Data.String (IsString (..))
import Data.Text (unpack, pack, init, tail, toLower, intercalate, append, dropWhile, breakOn)
import Network.URI (parseURI, isAbsoluteURI,
URI (..), URIAuth (..))
import Control.Arrow ((&&&))
import Data.HashMap.Strict.InsOrd (InsOrdHashMap, fromList)
import Data.Maybe (fromJust)
import Data.String (IsString (..))
import Network.URI (URI (..), URIAuth (..))
import Protolude hiding ((&), Proxy, get, intercalate, dropWhile)
import Control.Lens (at, (.~), (?~))
import Data.Swagger
import Data.Swagger
import PostgREST.ApiRequest (ContentType(..))
import PostgREST.Config (prettyVersion, docsVersion)
import PostgREST.Types (Table(..), Column(..), PgArg(..), ForeignKey(..),
PrimaryKey(..), Proxy(..), ProcDescription(..), toMime)
import PostgREST.Config (AppConfig (..), Proxy (..),
isMalformedProxyUri, toURI)
import PostgREST.DbStructure (DbStructure (..))
import PostgREST.DbStructure.Identifiers (QualifiedIdentifier (..))
import PostgREST.DbStructure.Proc (ProcDescription (..),
ProcParam (..))
import PostgREST.DbStructure.Relationship (Cardinality (..),
Relationship (..),
RelationshipsMap)
import PostgREST.DbStructure.Table (Column (..), Table (..),
TablesMap)
import PostgREST.Version (docsVersion, prettyVersion)
makeMimeList :: [ContentType] -> MimeList
makeMimeList cs = MimeList $ map (fromString . toS . toMime) cs
import PostgREST.MediaType
toSwaggerType :: Text -> SwaggerType t
toSwaggerType "text" = SwaggerString
toSwaggerType "integer" = SwaggerInteger
toSwaggerType "boolean" = SwaggerBoolean
toSwaggerType "numeric" = SwaggerNumber
toSwaggerType _ = SwaggerString
import Protolude hiding (Proxy, get)
makeTableDef :: [PrimaryKey] -> (Table, [Column], [Text]) -> (Text, Schema)
makeTableDef pks (t, cs, _) =
encode :: AppConfig -> DbStructure -> TablesMap -> HM.HashMap k [ProcDescription] -> Maybe Text -> LBS.ByteString
encode conf dbStructure tables procs schemaDescription =
JSON.encode $
postgrestSpec
(dbRelationships dbStructure)
(concat $ HM.elems procs)
(snd <$> HM.toList tables)
(proxyUri conf)
schemaDescription
(configOpenApiSecurityActive conf)
makeMimeList :: [MediaType] -> MimeList
makeMimeList cs = MimeList $ fmap (fromString . BS.unpack . toMime) cs
toSwaggerType :: Text -> Maybe (SwaggerType t)
toSwaggerType "character varying" = Just SwaggerString
toSwaggerType "character" = Just SwaggerString
toSwaggerType "text" = Just SwaggerString
toSwaggerType "boolean" = Just SwaggerBoolean
toSwaggerType "smallint" = Just SwaggerInteger
toSwaggerType "integer" = Just SwaggerInteger
toSwaggerType "bigint" = Just SwaggerInteger
toSwaggerType "numeric" = Just SwaggerNumber
toSwaggerType "real" = Just SwaggerNumber
toSwaggerType "double precision" = Just SwaggerNumber
toSwaggerType "ARRAY" = Just SwaggerArray
toSwaggerType "json" = Nothing
toSwaggerType "jsonb" = Nothing
toSwaggerType _ = Just SwaggerString
parseDefault :: Text -> Text -> Text
parseDefault colType colDefault =
case toSwaggerType colType of
Just SwaggerString -> wrapInQuotations $ case T.stripSuffix ("::" <> colType) colDefault of
Just def -> T.dropAround (=='\'') def
Nothing -> colDefault
_ -> colDefault
where
wrapInQuotations text = "\"" <> text <> "\""
makeTableDef :: RelationshipsMap -> Table -> (Text, Schema)
makeTableDef rels t =
let tn = tableName t in
(tn, (mempty :: Schema)
& description .~ tableDescription t
& type_ .~ SwaggerObject
& properties .~ fromList (map (makeProperty pks) cs)
& required .~ map colName (filter (not . colNullable) cs))
& type_ ?~ SwaggerObject
& properties .~ fromList (makeProperty t rels <$> tableColumns t)
& required .~ fmap colName (filter (not . colNullable) $ tableColumns t))
makeProperty :: [PrimaryKey] -> Column -> (Text, Referenced Schema)
makeProperty pks c = (colName c, Inline s)
makeProperty :: Table -> RelationshipsMap -> Column -> (Text, Referenced Schema)
makeProperty tbl rels col = (colName col, Inline s)
where
e = if null $ colEnum c then Nothing else decode $ encode $ colEnum c
fk ForeignKey{fkCol=Column{colTable=Table{tableName=a}, colName=b}} =
intercalate "" ["This is a Foreign Key to `", a, ".", b, "`.<fk table='", a, "' column='", b, "'/>"]
e = if null $ colEnum col then Nothing else JSON.decode $ JSON.encode $ colEnum col
fk :: Maybe Text
fk =
let
-- Finds the relationship that has a single column foreign key
rel = find (\case
Relationship{relCardinality=(M2O _ relColumns)} -> [colName col] == (fst <$> relColumns)
_ -> False
) $ fromMaybe mempty $ HM.lookup (QualifiedIdentifier (tableSchema tbl) (tableName tbl), tableSchema tbl) rels
fCol = (headMay . (\r -> snd <$> relColumns (relCardinality r)) =<< rel)
fTbl = qiName . relForeignTable <$> rel
fTblCol = (,) <$> fTbl <*> fCol
in
(\(a, b) -> T.intercalate "" ["This is a Foreign Key to `", a, ".", b, "`.<fk table='", a, "' column='", b, "'/>"]) <$> fTblCol
pk :: Bool
pk = any (\p -> pkTable p == colTable c && pkName p == colName c) pks
pk = colName col `elem` tablePKCols tbl
n = catMaybes
[ Just "Note:"
, if pk then Just "This is a Primary Key.<pk/>" else Nothing
, fk <$> colFK c
, fk
]
d =
if length n > 1 then
Just $ append (maybe "" (`append` "\n\n") $ colDescription c) (intercalate "\n" n)
Just $ T.append (maybe "" (`T.append` "\n\n") $ colDescription col) (T.intercalate "\n" n)
else
colDescription c
colDescription col
s =
(mempty :: Schema)
& default_ .~ (decode . toS =<< colDefault c)
& default_ .~ (JSON.decode . toUtf8Lazy . parseDefault (colType col) =<< colDefault col)
& description .~ d
& enum_ .~ e
& format ?~ colType c
& maxLength .~ (fromIntegral <$> colMaxLen c)
& type_ .~ toSwaggerType (colType c)
& format ?~ colType col
& maxLength .~ (fromIntegral <$> colMaxLen col)
& type_ .~ toSwaggerType (colType col)
makeProcSchema :: ProcDescription -> Schema
makeProcSchema pd =
(mempty :: Schema)
& description .~ pdDescription pd
& type_ .~ SwaggerObject
& properties .~ fromList (map makeProcProperty (pdArgs pd))
& required .~ map pgaName (filter pgaReq (pdArgs pd))
& type_ ?~ SwaggerObject
& properties .~ fromList (fmap makeProcProperty (pdParams pd))
& required .~ fmap ppName (filter ppReq (pdParams pd))
makeProcProperty :: PgArg -> (Text, Referenced Schema)
makeProcProperty (PgArg n t _) = (n, Inline s)
makeProcProperty :: ProcParam -> (Text, Referenced Schema)
makeProcProperty (ProcParam n t _ _) = (n, Inline s)
where
s = (mempty :: Schema)
& type_ .~ toSwaggerType t
@@ -95,19 +151,19 @@ makePreferParam ts =
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamHeader
& type_ .~ SwaggerString
& enum_ .~ decode (encode ts))
& type_ ?~ SwaggerString
& enum_ .~ JSON.decode (JSON.encode ts))
makeProcParam :: ProcDescription -> [Referenced Param]
makeProcParam pd =
[ Inline $ (mempty :: Param)
& name .~ "args"
& required ?~ True
& schema .~ (ParamBody $ Inline $ makeProcSchema pd)
& schema .~ ParamBody (Inline $ makeProcSchema pd)
, Ref $ Reference "preferParams"
]
makeParamDefs :: [(Table, [Column], [Text])] -> [(Text, Param)]
makeParamDefs :: [Table] -> [(Text, Param)]
makeParamDefs ti =
[ ("preferParams", makePreferParam ["params=single-object"])
, ("preferReturn", makePreferParam ["return=representation", "return=minimal", "return=none"])
@@ -118,46 +174,53 @@ makeParamDefs ti =
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamQuery
& type_ .~ SwaggerString))
& type_ ?~ SwaggerString))
, ("on_conflict", (mempty :: Param)
& name .~ "on_conflict"
& description ?~ "On Conflict"
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamQuery
& type_ ?~ SwaggerString))
, ("order", (mempty :: Param)
& name .~ "order"
& description ?~ "Ordering"
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamQuery
& type_ .~ SwaggerString))
& type_ ?~ SwaggerString))
, ("range", (mempty :: Param)
& name .~ "Range"
& description ?~ "Limiting and Pagination"
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamHeader
& type_ .~ SwaggerString))
& type_ ?~ SwaggerString))
, ("rangeUnit", (mempty :: Param)
& name .~ "Range-Unit"
& description ?~ "Limiting and Pagination"
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamHeader
& type_ .~ SwaggerString
& default_ .~ decode "\"items\""))
& type_ ?~ SwaggerString
& default_ .~ JSON.decode "\"items\""))
, ("offset", (mempty :: Param)
& name .~ "offset"
& description ?~ "Limiting and Pagination"
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamQuery
& type_ .~ SwaggerString))
& type_ ?~ SwaggerString))
, ("limit", (mempty :: Param)
& name .~ "limit"
& description ?~ "Limiting and Pagination"
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamQuery
& type_ .~ SwaggerString))
& type_ ?~ SwaggerString))
]
<> concat [ makeObjectBody (tableName t) : makeRowFilters (tableName t) cs
| (t, cs, _) <- ti
<> concat [ makeObjectBody (tableName t) : makeRowFilters (tableName t) (tableColumns t)
| t <- ti
]
makeObjectBody :: Text -> (Text, Param)
@@ -170,54 +233,54 @@ makeObjectBody tn =
makeRowFilter :: Text -> Column -> (Text, Param)
makeRowFilter tn c =
(intercalate "." ["rowFilter", tn, colName c], (mempty :: Param)
(T.intercalate "." ["rowFilter", tn, colName c], (mempty :: Param)
& name .~ colName c
& description .~ colDescription c
& required ?~ False
& schema .~ ParamOther ((mempty :: ParamOtherSchema)
& in_ .~ ParamQuery
& type_ .~ SwaggerString
& type_ ?~ SwaggerString
& format ?~ colType c))
makeRowFilters :: Text -> [Column] -> [(Text, Param)]
makeRowFilters tn = map (makeRowFilter tn)
makeRowFilters tn = fmap (makeRowFilter tn)
makePathItem :: (Table, [Column], [Text]) -> (FilePath, PathItem)
makePathItem (t, cs, _) = ("/" ++ unpack tn, p $ tableInsertable t)
makePathItem :: Table -> (FilePath, PathItem)
makePathItem t = ("/" ++ T.unpack tn, p $ tableInsertable t || tableUpdatable t || tableDeletable t)
where
-- Use first line of table description as summary; rest as description (if present)
-- We strip leading newlines from description so that users can include a blank line between summary and description
(tSum, tDesc) = fmap fst &&& fmap (dropWhile (=='\n') . snd) $
breakOn "\n" <$> tableDescription t
(tSum, tDesc) = fmap fst &&& fmap (T.dropWhile (=='\n') . snd) $
T.breakOn "\n" <$> tableDescription t
tOp = (mempty :: Operation)
& tags .~ Set.fromList [tn]
& summary .~ tSum
& description .~ mfilter (/="") tDesc
getOp = tOp
& parameters .~ map ref (rs <> ["select", "order", "range", "rangeUnit", "offset", "limit", "preferCount"])
& parameters .~ fmap ref (rs <> ["select", "order", "range", "rangeUnit", "offset", "limit", "preferCount"])
& at 206 ?~ "Partial Content"
& at 200 ?~ Inline ((mempty :: Response)
& description .~ "OK"
& schema ?~ Inline (mempty
& type_ .~ SwaggerArray
& items ?~ (SwaggerItemsObject $ Ref $ Reference $ tableName t)
& type_ ?~ SwaggerArray
& items ?~ SwaggerItemsObject (Ref $ Reference $ tableName t)
)
)
postOp = tOp
& parameters .~ map ref ["body." <> tn, "preferReturn"]
& parameters .~ fmap ref ["body." <> tn, "select", "preferReturn"]
& at 201 ?~ "Created"
patchOp = tOp
& parameters .~ map ref (rs <> ["body." <> tn, "preferReturn"])
& parameters .~ fmap ref (rs <> ["body." <> tn, "preferReturn"])
& at 204 ?~ "No Content"
deletOp = tOp
& parameters .~ map ref (rs <> ["preferReturn"])
& parameters .~ fmap ref (rs <> ["preferReturn"])
& at 204 ?~ "No Content"
pr = (mempty :: PathItem) & get ?~ getOp
pw = pr & post ?~ postOp & patch ?~ patchOp & delete ?~ deletOp
p False = pr
p True = pw
tn = tableName t
rs = [ intercalate "." ["rowFilter", tn, colName c ] | c <- cs ]
rs = [ T.intercalate "." ["rowFilter", tn, colName c ] | c <- tableColumns t ]
ref = Ref . Reference
makeProcPathItem :: ProcDescription -> (FilePath, PathItem)
@@ -225,14 +288,14 @@ makeProcPathItem pd = ("/rpc/" ++ toS (pdName pd), pe)
where
-- Use first line of proc description as summary; rest as description (if present)
-- We strip leading newlines from description so that users can include a blank line between summary and description
(pSum, pDesc) = fmap fst &&& fmap (dropWhile (=='\n') . snd) $
breakOn "\n" <$> pdDescription pd
(pSum, pDesc) = fmap fst &&& fmap (T.dropWhile (=='\n') . snd) $
T.breakOn "\n" <$> pdDescription pd
postOp = (mempty :: Operation)
& summary .~ pSum
& description .~ mfilter (/="") pDesc
& parameters .~ makeProcParam pd
& tags .~ Set.fromList ["(rpc) " <> pdName pd]
& produces ?~ makeMimeList [CTApplicationJSON, CTSingularJSON]
& produces ?~ makeMimeList [MTApplicationJSON, MTSingularJSON]
& at 200 ?~ "OK"
pe = (mempty :: PathItem) & post ?~ postOp
@@ -242,14 +305,22 @@ makeRootPathItem = ("/", p)
getOp = (mempty :: Operation)
& tags .~ Set.fromList ["Introspection"]
& summary ?~ "OpenAPI description (this document)"
& produces ?~ makeMimeList [CTOpenAPI, CTApplicationJSON]
& produces ?~ makeMimeList [MTOpenAPI, MTApplicationJSON]
& at 200 ?~ "OK"
pr = (mempty :: PathItem) & get ?~ getOp
p = pr
makePathItems :: [ProcDescription] -> [(Table, [Column], [Text])] -> InsOrdHashMap FilePath PathItem
makePathItems :: [ProcDescription] -> [Table] -> InsOrdHashMap FilePath PathItem
makePathItems pds ti = fromList $ makeRootPathItem :
map makePathItem ti ++ map makeProcPathItem pds
fmap makePathItem ti ++ fmap makeProcPathItem pds
makeSecurityDefinitions :: Text -> Bool -> SecurityDefinitions
makeSecurityDefinitions secName allow
| allow = SecurityDefinitions (fromList [(secName, SecurityScheme secSchType secSchDescription)])
| otherwise = mempty
where
secSchType = SecuritySchemeApiKey (ApiKeyParams "Authorization" ApiKeyHeader)
secSchDescription = Just "Add the token prepending \"Bearer \" (without quotes) to it"
escapeHostName :: Text -> Text
escapeHostName "*" = "0.0.0.0"
@@ -259,48 +330,30 @@ escapeHostName "*6" = "0.0.0.0"
escapeHostName "!6" = "0.0.0.0"
escapeHostName h = h
postgrestSpec :: [ProcDescription] -> [(Table, [Column], [Text])] -> (Text, Text, Integer, Text) -> Maybe Text -> [PrimaryKey] -> Swagger
postgrestSpec pds ti (s, h, p, b) sd pks = (mempty :: Swagger)
& basePath ?~ unpack b
postgrestSpec :: RelationshipsMap -> [ProcDescription] -> [Table] -> (Text, Text, Integer, Text) -> Maybe Text -> Bool -> Swagger
postgrestSpec rels pds ti (s, h, p, b) sd allowSecurityDef = (mempty :: Swagger)
& basePath ?~ T.unpack b
& schemes ?~ [s']
& info .~ ((mempty :: Info)
& version .~ prettyVersion
& version .~ T.decodeUtf8 prettyVersion
& title .~ "PostgREST API"
& description ?~ d)
& externalDocs ?~ ((mempty :: ExternalDocs)
& description ?~ "PostgREST Documentation"
& url .~ URL ("https://postgrest.org/en/" <> docsVersion <> "/api.html"))
& host .~ h'
& definitions .~ fromList (map (makeTableDef pks) ti)
& definitions .~ fromList (makeTableDef rels <$> ti)
& parameters .~ fromList (makeParamDefs ti)
& paths .~ makePathItems pds ti
& produces .~ makeMimeList [CTApplicationJSON, CTSingularJSON, CTTextCSV]
& consumes .~ makeMimeList [CTApplicationJSON, CTSingularJSON, CTTextCSV]
& produces .~ makeMimeList [MTApplicationJSON, MTSingularJSON, MTTextCSV]
& consumes .~ makeMimeList [MTApplicationJSON, MTSingularJSON, MTTextCSV]
& securityDefinitions .~ makeSecurityDefinitions securityDefName allowSecurityDef
& security .~ [SecurityRequirement (fromList [(securityDefName, [])]) | allowSecurityDef]
where
s' = if s == "http" then Http else Https
h' = Just $ Host (unpack $ escapeHostName h) (Just (fromInteger p))
h' = Just $ Host (T.unpack $ escapeHostName h) (Just (fromInteger p))
d = fromMaybe "This is a dynamic API generated by PostgREST" sd
encodeOpenAPI :: [ProcDescription] -> [(Table, [Column], [Text])] -> (Text, Text, Integer, Text) -> Maybe Text -> [PrimaryKey] -> LByteString
encodeOpenAPI pds ti uri sd pks = encode $ postgrestSpec pds ti uri sd pks
{-|
Test whether a proxy uri is malformed or not.
A valid proxy uri should be an absolute uri without query and user info,
only http(s) schemes are valid, port number range is 1-65535.
For example
http://postgrest.com/openapi.json
https://postgrest.com:8080/openapi.json
-}
isMalformedProxyUri :: Maybe Text -> Bool
isMalformedProxyUri Nothing = False
isMalformedProxyUri (Just uri)
| isAbsoluteURI (toS uri) = not $ isUriValid $ toURI uri
| otherwise = True
toURI :: Text -> URI
toURI uri = fromJust $ parseURI (toS uri)
securityDefName = "JWT"
pickProxy :: Maybe Text -> Maybe Proxy
pickProxy proxy
@@ -308,7 +361,7 @@ pickProxy proxy
-- should never happen
-- since the request would have been rejected by the middleware if proxy uri
-- is malformed
| isMalformedProxyUri proxy = Nothing
| isMalformedProxyUri $ fromMaybe mempty proxy = Nothing
| otherwise = Just Proxy {
proxyScheme = scheme
, proxyHost = host'
@@ -317,53 +370,24 @@ pickProxy proxy
}
where
uri = toURI $ fromJust proxy
scheme = init $ toLower $ pack $ uriScheme uri
scheme = T.init $ T.toLower $ T.pack $ uriScheme uri
path URI {uriPath = ""} = "/"
path URI {uriPath = p} = p
path' = pack $ path uri
path URI {uriPath = p} = p
path' = T.pack $ path uri
authority = fromJust $ uriAuthority uri
host' = pack $ uriRegName authority
host' = T.pack $ uriRegName authority
port' = uriPort authority
readPort = fromMaybe 80 . readMaybe
port'' :: Integer
port'' = case (port', scheme) of
("", "http") -> 80
("", "http") -> 80
("", "https") -> 443
_ -> readPort $ unpack $ tail $ pack port'
_ -> readPort $ T.unpack $ T.tail $ T.pack port'
isUriValid:: URI -> Bool
isUriValid = fAnd [isSchemeValid, isQueryValid, isAuthorityValid]
fAnd :: [a -> Bool] -> a -> Bool
fAnd fs x = all ($x) fs
isSchemeValid :: URI -> Bool
isSchemeValid URI {uriScheme = s}
| toLower (pack s) == "https:" = True
| toLower (pack s) == "http:" = True
| otherwise = False
isQueryValid :: URI -> Bool
isQueryValid URI {uriQuery = ""} = True
isQueryValid _ = False
isAuthorityValid :: URI -> Bool
isAuthorityValid URI {uriAuthority = a}
| isJust a = fAnd [isUserInfoValid, isHostValid, isPortValid] $ fromJust a
| otherwise = False
isUserInfoValid :: URIAuth -> Bool
isUserInfoValid URIAuth {uriUserInfo = ""} = True
isUserInfoValid _ = False
isHostValid :: URIAuth -> Bool
isHostValid URIAuth {uriRegName = ""} = False
isHostValid _ = True
isPortValid :: URIAuth -> Bool
isPortValid URIAuth {uriPort = ""} = True
isPortValid URIAuth {uriPort = (':':p)} =
case readMaybe p of
Just i -> i > (0 :: Integer) && i < 65536
Nothing -> False
isPortValid _ = False
proxyUri :: AppConfig -> (Text, Text, Integer, Text)
proxyUri AppConfig{..} =
case pickProxy $ toS <$> configOpenApiServerProxyUri of
Just Proxy{..} ->
(proxyScheme, proxyHost, proxyPort, proxyPath)
Nothing ->
("http", configServerHost, toInteger configServerPort, "/")

Some files were not shown because too many files have changed in this diff Show More