This change ensures PostgREST starts listening on a server socket only after it loaded the schema cache and is ready to handle requests. It is no longer going to return 503 errors during startup until the schema cache is loaded.
This is useful when multiple instances run on the same machine, for
example behind a proxy. Unix sockets for web and admin servers can then
be put in the same folder for each instance.
Can be helpful when writing tests as well.
The `if response` check a few rows down would not actually work without
initializing the variable first. It'd throw:
```
E UnboundLocalError: cannot access local variable 'response' where
it is not associated with a value
```
retryingSchemaCacheLoad should not clear existing schema cache upon failure - there is no reason to do that. If there is a communication issue with the database server or db is down, clients are going to get 502 anyway. If it was a glitch when loading the schema cache - the clients are going to use old (stale) schema cache for some time until next retry re-loads it successfully.
There was an oversight on d556cea, `timeout_authenticator` doesn't have
any privileges on the tables so it shouldn't be making requests to the
API server or it will end up with 401 responses.
Replaces the `Maybe [Text]` with `[Text]`. The `Maybe` is unnecessary
because we handle `Just []` and `Nothing` the same way.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
This also requires enabling shared_preload_libraries=pg_stat_statements.
Really we only need compute_query_id=on, but pg_stat_statements is
useful for other tests and it already enables compute_query_id (since
it's `auto` by default).
The PGRST000 database connection error message was "Database connection
error. Retrying the connection.", but reconnection attempts are already
logged separately by the reconnection observation, and on fatal errors
(e.g. authentication failure) PostgREST does not retry at all. Drop the
"Retrying the connection." part, leaving "Database connection error.".
The config dump in Config.hs listed db-config, db-pre-config and the
db-pool-* settings out of alphabetical order, while the rest of the dump
was sorted. Now the whole dump is consistently alphabetical and update
the expected IO test configs to match.
This was brought over in the last commit, because the postgis was
re-using the same config as another test. Now it has a separate config,
so we can reduce it a bit.
Previously, information about each test-suite was repeated in 3 separate
places:
- as a label and as implicit knowledge in the test-suite itself,
- as a comment in Main.hs, and
- as a configuration in SpecHelper.hs.
With this change, there will be a single source of truth in the test
suite itself. This will allow a single test-suite to easily test
multiple different configurations.
Because we separate loadtest results per URL now, we can move the error
tests into the regular mixed bag of loadtests - we will be able to tell
from the misspelled URLs when we hit a regression in that area.
We should be able to do similar things for JWT tests, but we'll need
more infrastructure here.
Schema cache query timings are only needed immediately after a schema-cache reload to emit SchemaCacheQueriedObs. Storing them inside SchemaCache makes the cache carry transient observability data that is not part of the cached schema state and is never used by request handling.
This change makes querySchemaCache to return query timings in a tuple in parallel to SchemaCache and removes dbQueryTimings field.
Config variables are tested already via reading the config files in
the `configs/` directory.
If more are to be tested, it should be done via adding a file in
`configs/` and compare it with its associated file in `configs/expected/`.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Remove internal schema cache load and relationship load sleep settings plus
the delay wrappers they enabled. Drop IO tests that depended on the removed
settings.
Add an IO test that drops a table while schema cache reload is delayed. It verifies the stale cache path returns PostgreSQL 42P01 and the refreshed cache returns PGRST205.
Right now metrics observation handler does not track database connections but updates a single Gauge based on HasqlPoolObs events. This is problematic because Hasql pool reports various connection events in multiple phases. The connection state machine is not simple and to precisely report the number of connections in various states, it is necessary to track their lifecycles.
This change adds a ConnTrack data structure and logic to track database connections lifecycles. At the moment it supports "connected" and "inUse" connection counts precisely. The "pgrst_db_pool_available" metric is implemented on top of ConnTrack instead of a simple Gauge.
The functions `drain_stdout` and `match_log` should be in `util.py`
so they can be reused in other modules.
Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Disabling the autovacuum daemon should also help reproducibility in
theory, although I don't know of any cases where we hit a problem with
that.
VACUUM changes the order of rows that PostgreSQL returns for some table
without explicit ordering, thus doing the latter to make it consistently
reproducible.
After ANALYZE estimates are 100% exact for the moment, so some requests
which returned 206 Partial Response now return 200 instead. The fact
that PostgREST returns 206 on an unfiltered endpoint can probably be
considered a bug.
Proves the failure on https://github.com/PostgREST/postgrest/issues/4622.
This doesn't require additional test infra, only nginx. Taking advantage
of the `stream {}` context which is also compatible with unix socket
besides TCP.
These fail with orioledb, because it randomly returns a different order.
Of course, that's allowed - ordering is not guaranteed unless specified.
We should write our tests with more explicit ordering in general, even
if multiple major versions show the same behavior.
This commit removes auth middleware for it hides
side effects and obscures logic. The auth operations
are now done in its own stage in the request-response
cycle.
It also removes the logging middleware because now
we instead use observation module to log the response.
