Right now listening sockets initialization, management and usage is split between App, AppState and Admin modules: they are created in AppState.init and remembered in AppState but used only in App and Admin.
It has several negative consequences:
- sockets are initialized even if not needed (eg. command line invocations like dump-config or dump-schema)
- it is impossible to start listening on a socket after initial schema cache load because it requires AppState
This change decouples listen socket management from AppState. Sockets are created only when needed (ie. not in command line tools invocation) and passed to admin application and to Warp by the App module.
- Create separate test suite for observability tests
- Create wrapper script `postgrest-test-observability`
- Add to CI and `postgrest-check`
- Move JWT cache tests under observability tests
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Changes:
1. Refactoring and some cleanup of JWT handling code:
* Instead of caching AuthResult cache decoded claims (which signature was verified). Validating claims and determining role is done after cache lookup
* Cleaned up API so that usage of it is simplified: lookupJwtCache cache key >>= parseClaims configJwtAud time
* Handling of JwtCacheState initialization and updates of configuration is encapsulated in Auth.JwtCache module
2. Generic high performance (hopefully) scalable, dynamically resizeable cache implementation based on stm, stm-hamt and sieve algorithm. It also integrates with PostgREST measurements infrastructure providing usage stats (ie. hit ratio, evictions count)
Previously, we'd silently report "200 OK" on the root endpoint, but
would never return any endpoints from the schema cache.
Now the schema cache query fails because of the ::regnamespace cast.
With this:
- Is no longer necessary to pass observer as an argument
to every function that needs observations.
- We can invoke the observer on every function that uses AppConfig.
However it'd be better to just call the observer in the upper modules
(like on App.hs).
* test text/html and drop HtmlRawOutputSpec.hs
* all tests passing, removed all pendingWith
* make functions compatible with pg <= 12
* move custom media types tests to own spec
* anyelement aggregate
* apply aggregates without a final function
* overriding works
* overriding anyelement with particular agg
* cannot override vendored media types
* plan spec works with custom aggregate
* renamed media types to make clear which ones are overridable
* correct content negotiation with same weight
* text/tab-separated-values media type
* text/csv with BOM plus content-disposition header
See PR #2523. Most notable code changes:
- Load data representation casts into schema cache.
- Data representations for reads, filters, inserts, updates, views, over joins.
- `CoercibleField` represents name references in queries where coercion may be needed.
- `ResolverContext` help facilitate field resolution during planning.
- Planner 'resolves' names in the API query and pairs them with any implicit conversions to be used in the query builder stage.
- Tests for all of the above.
- More consistent naming (TypedX -> CoercibleX).
New: unit tests for more data representation use cases; helpful as examples as well.
New: update CHANGELOG with data representations feature description.
Fixed failing idempotence test.
New: replace date formatter test with one that does something.
Fixup: inadvertent CHANGELOG change after rebase.
Cleanup: `tfName` -> `cfName` and related.
Document what IRType means.
Formatting.
New: use a subquery to interpret `IN` literals requiring data rep transformation.
- With the previous method, very long queries such as `ANY (ARRAY[test.color('000100'), test.color('CAFE12'), test.color('01E240'), ...` could be generated. Consider the case where the parser function name is 45 characters and there's a hundred literals. That's 4.5kB of SQL just for the function name alone!
- New version uses `unnest`: `ANY (SELECT test.color(unnest('{000100,CAFE12,01E240,...}'::text[]))` to produce a much shorter query.
- This is likely to be more performant and either way much more readable and debuggable in the logs.
- new option db-pool-max-idletime limits the maximal idle time of a
postgresql connection, defaults to 30 seconds
- this essentially restores the db-pool-timeout option from versions
<=10, with more specific name
- db-pool-acquisition-timeout is no longer optional, defaults to 10s
- new option db-pool-max-lifetime limits the maximal lifetime of a
postgresql connection, defaults to 30m
- hasql-pool-0.8
* 'release' now flushes the pool, and no longer destroys it;
'PoolIsReleasedUsageError' is gone compared to our fork
* now supports an acquisition timeout, which we don't use
yet
* lower bound on 0.8.0.2 to fix a Windows build issue
- hasql-1.6
introduces a position parameter to ServerError, which we
ignore
This version of hasql-pool is a simplified rewrite that doesn't use
the resource-pool package. The major API changes are that idle
connections are no longer timed out (and the corresponding setting
is gone), and that `release` makes the pool unusable, where it used
to remain usable and only flushed idle connections.
We depend on a PostgREST fork of 0.7.2 that gives us reliable
flushing, compare https://github.com/PostgREST/hasql-pool/pull/1
- hasql-pool 0.7 removes timing out of idle connections, so
this change removes the db-pool-timeout option.
Given that we were typically running with very high
timeout settings, I don't anticipate the lack of timeout
to introduce new issues, though we might want to consider
introducing some retry-logic down the line when we
encounter connection failures.
- See https://github.com/PostgREST/postgrest/issues/2422 for a
discussion on depending on a forked dependency. Besides adding
the dependency to the nix overlay, we're also adding it to
stack.yaml and a new cabal.project to allow stack/cabal users
to build the project.
