Remove internal schema cache load and relationship load sleep settings plus
the delay wrappers they enabled. Drop IO tests that depended on the removed
settings.
- Create separate test suite for observability tests
- Create wrapper script `postgrest-test-observability`
- Add to CI and `postgrest-check`
- Move JWT cache tests under observability tests
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Refactoring: State validation helpers used in JwtCacheSpec moved to SpecHelper
to make them available in other Spec modules.
(cherry picked from commit 85a313a8cc)
To make schema cache loading wait tests robust it is necessary to provide three separate internal config variables:
* "internal-schema-cache-query-sleep" - introduces delay in schema queries execution
* "internal-schema-cache-load-sleep" - introduces delay between schema queries execution and processing their results
* "internal-schema-cache-relationship-load-sleep" - introduces delay in processing relationship query results
Thanks to these changes it is now possible to test various schema loading scenarios with the right granularity robustly (eg. make sure requests wait for schema loading but not for relationship loading).
Changes:
1. Refactoring and some cleanup of JWT handling code:
* Instead of caching AuthResult cache decoded claims (which signature was verified). Validating claims and determining role is done after cache lookup
* Cleaned up API so that usage of it is simplified: lookupJwtCache cache key >>= parseClaims configJwtAud time
* Handling of JwtCacheState initialization and updates of configuration is encapsulated in Auth.JwtCache module
2. Generic high performance (hopefully) scalable, dynamically resizeable cache implementation based on stm, stm-hamt and sieve algorithm. It also integrates with PostgREST measurements infrastructure providing usage stats (ie. hit ratio, evictions count)
Previously, we'd silently report "200 OK" on the root endpoint, but
would never return any endpoints from the schema cache.
Now the schema cache query fails because of the ::regnamespace cast.
With this:
- Is no longer necessary to pass observer as an argument
to every function that needs observations.
- We can invoke the observer on every function that uses AppConfig.
However it'd be better to just call the observer in the upper modules
(like on App.hs).
* test text/html and drop HtmlRawOutputSpec.hs
* all tests passing, removed all pendingWith
* make functions compatible with pg <= 12
* move custom media types tests to own spec
* anyelement aggregate
* apply aggregates without a final function
* overriding works
* overriding anyelement with particular agg
* cannot override vendored media types
* plan spec works with custom aggregate
* renamed media types to make clear which ones are overridable
* correct content negotiation with same weight
* text/tab-separated-values media type
* text/csv with BOM plus content-disposition header
* Don't apply `tx=commit` if the transaction doesn't commit
* Apply `count=exact`
* Also simplifies the Preference-Applied logic, removing the need for
some functions.
- new option db-pool-max-idletime limits the maximal idle time of a
postgresql connection, defaults to 30 seconds
- this essentially restores the db-pool-timeout option from versions
<=10, with more specific name
- db-pool-acquisition-timeout is no longer optional, defaults to 10s
- new option db-pool-max-lifetime limits the maximal lifetime of a
postgresql connection, defaults to 30m
fix: bug when using Range header on PATCH/DELETE
- Fix the "message": "syntax error at or near \"RETURNING\"" error
- Fix doing a limited update/delete when an order query parameter was present
breaking: The Range header is now only considered on GET requests and is ignored for any other method
- Other methods should use the `limit/offset` query parameters for sub-ranges
- PUT requests no longer return an error when this header is present
The configuration option db-pool-acquisition-timeout
specifies the time in seconds to wait for the pool to
free up a connection slot. Otherwise, a 504 error is
returned. By default, there is no timeout.
This version of hasql-pool is a simplified rewrite that doesn't use
the resource-pool package. The major API changes are that idle
connections are no longer timed out (and the corresponding setting
is gone), and that `release` makes the pool unusable, where it used
to remain usable and only flushed idle connections.
We depend on a PostgREST fork of 0.7.2 that gives us reliable
flushing, compare https://github.com/PostgREST/hasql-pool/pull/1
- hasql-pool 0.7 removes timing out of idle connections, so
this change removes the db-pool-timeout option.
Given that we were typically running with very high
timeout settings, I don't anticipate the lack of timeout
to introduce new issues, though we might want to consider
introducing some retry-logic down the line when we
encounter connection failures.
- See https://github.com/PostgREST/postgrest/issues/2422 for a
discussion on depending on a forked dependency. Besides adding
the dependency to the nix overlay, we're also adding it to
stack.yaml and a new cabal.project to allow stack/cabal users
to build the project.
