jimmy/postgrest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,781 Commits 9 Branches 102 Tags
ec83d08be671a9d91283437799ea82f233abc2e1
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
steve-chavez dc96cdb0c3 bump version to 12.2.6 2025-01-29 15:22:22 -05:00
steve-chavezandWolfgang Walther 4e9601d155 fix: always show schema cache load time 
It used to be that this was only enabled with log-level=debug.
But the default log-level is misleading, for example:

```
$ PGRST_DB_SCHEMAS="apflora" postgrest-with-postgresql-16  -f test/io/big_schema.sql postgrest-run

...
13/Nov/2024:22:08:20 -0500: Config reloaded
13/Nov/2024:22:08:20 -0500: Schema cache queried in 36.3 milliseconds
13/Nov/2024:22:08:20 -0500: Schema cache loaded 326 Relations, 305 Relationships, 7 Functions, 0 Domain Representations, 4 Media Type Handlers, 1194 Timezones
```

The "Schema cache loaded" can take a while to appear, yet the 22:08:20
time is the same. If we reveal the load time this is clarified:

```
13/Nov/2024:22:08:37 -0500: Schema cache loaded in 16770.1 milliseconds
```
 2024-11-16 22:41:44 +01:00
Steve ChavezWolfgang WaltherLaurence Isla
 		64b5a59410 fix: schema cache loading before the in-db config (#3670) 
Fixes #3660. Load the config after getting the pg version but before loading the schema.

The regression happened on f09655b.

Also remove schema cache load wrapper and separate db queries in different functions.

Co-authored-by: Laurence Isla <lau.isla.c@gmail.com>
 2024-08-01 17:41:07 +02:00
steve-chavezandWolfgang Walther e451700d5d chore: remove links to gitter 
We'll now use github discussions for support.
 2024-07-13 17:11:06 +02:00
steve-chavez db9da0b1ca bump version to 12.2.2 2024-07-10 23:07:23 -05:00
steve-chavez f09655b7a6 fix: schema cache retrying without backoff 
Fixes https://github.com/PostgREST/postgrest/issues/3523.

Now if there's a failure when obtaining the pg version OR schema cache,
we do the same retrying process. This way we don't add two retries.

Refactors and renames the "connectionWorker" to "schemaCacheLoader".
This makes more sense since what we really want is the schema cache,
the version is the pre-requisite for ensuring our
schema cache queries work.

Additionally, we no longer log ` Attempting to connect to the database...`
at startup unnecessarily. This is only logged whenever there's a retry attempt.
 2024-07-10 21:11:20 -05:00
steve-chavezandWolfgang Walther 8715e426c0 chore: remove paypal links 
It was tied to a personal account and donations there have been too rare.
 2024-07-10 21:13:23 +02:00
steve-chavez d3f7cba92e bump version to 12.2.1 2024-06-27 14:59:35 -05:00
steve-chavez 9daddb549c docs: add listener recovery 2024-06-27 14:08:16 -05:00
steve-chavez 716fcbf147 fix: don't reload cache on every listener fail 
Revert "prevent GSSAPI error between Listener and pool"

This reverts commit 4beac10d3d.
 2024-06-27 14:08:16 -05:00
steve-chavez ec89f6b90c bump version to 12.2.0 2024-06-11 09:57:05 -05:00
steve-chavez 82aa58a08f docs: deprecate EOL pg versions 2024-06-11 09:43:09 -05:00
steve-chavez 4beac10d3d prevent GSSAPI error between Listener and pool 
Brings back the the signaling/waiting between the connection pool and
the Listener.

Prevents the GSSAPI error shown on https://github.com/PostgREST/postgrest/issues/3569
 2024-06-05 13:58:34 -05:00
steve-chavez 1a8b6972a8 correct exponential backoff on Listener 
Clears the limitation mentioned on

https://github.com/PostgREST/postgrest/pull/3536

The Listener no longer uses the https://hackage.haskell.org/package/retry
package and instead uses a much simpler IORef in AppState for the
delays.

Additionally it no longer uses exception throwing/catching, which
is rather messy and brings some
concerns(https://github.com/PostgREST/postgrest/issues/3569#issuecomment-2146013327).
 2024-06-05 08:52:18 -05:00
steve-chavez 82a43c2767 nix: postgrest-gen-ctags use haskdogs 2024-05-25 15:21:44 -05:00
steve-chavez a51a74b3c1 docs: shorten CLI 2024-05-24 18:05:45 -05:00
steve-chavez 1c371d7340 docs: clarify architecture 2024-05-24 18:05:45 -05:00
steve-chavez 30ca64d849 docs: improve observability 2024-05-24 18:05:45 -05:00
steve-chavez da9e497ef1 changelog: add architecture doc 2024-05-23 19:34:42 -05:00
steve-chavez 47e9a2d134 refactor: Listener to own module 2024-05-23 19:34:42 -05:00
steve-chavez 7e61c9deb0 feat: force read-write for listener connection 2024-05-19 23:14:19 -05:00
steve-chavez 3cf565614d fix: listener retries with exponential backoff 
Also corrects the admin ready response which now considers the listener
state.
 2024-05-19 20:48:59 -05:00
steve-chavez bfa4e1bedb test: isolate pg_terminate_backend to appname 
The pg_terminate_backend done in the io test:

`test_fail_with_automatic_recovery_disabled_and_terminated_using_query`

Can affect other connections.
 2024-05-19 19:58:27 -05:00
steve-chavez d5a4c5609e refactor: greppable 57P01 error code 2024-05-19 19:58:27 -05:00
steve-chavez c8ba505b99 docs: linkcheck ignore patreon 2024-05-19 13:44:23 -05:00
steve-chavez ae91fd89b3 test: assert empty response text on 204 2024-05-19 13:44:23 -05:00
steve-chavez 756aad7827 fix: listener silent fail on replica 
Update hasql-notifications to include the fix on
https://github.com/diogob/hasql-notifications/issues/24.

Which now reveals the following error:

```
$ postgrest-with-postgresql-16 --replica -f test/spec/fixtures/load.sql postgrest-run

17/May/2024:18:35:38 -0500: Successfully connected to PostgreSQL 16.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 13.2.0, 64-bit
17/May/2024:18:35:38 -0500: Could not listen for notifications on the "pgrst" channel. ERROR:  cannot execute LISTEN during recovery
17/May/2024:18:35:38 -0500: Retrying listening for notifications...
```

This is still not good because the LISTEN channel will be retried
forever without a backoff.
 2024-05-18 23:33:04 -05:00
steve-chavez aa75412932 nix: PGRST_DB_URI preference for tmp db replica 
When using `postgrest-with-postgresql-* --replica`, the PGRST_DB_URI
will set the replica host as preference. This to be able to run
quick manual tests with postgrest running on a replica.
 2024-05-18 23:33:04 -05:00
steve-chavez 33b6ba8199 refactor: move checkIsFatal logic to usePool 
The fatal logic is now inside `usePool`. It centralizes the
logic which is better for Locality of Behavior.

Removes:

- The need to do checkIsFatal on other parts of the code
- SCFatalFail/ConnFatalFail states which are no longer needed.
 2024-05-16 17:40:25 -05:00
steve-chavez d25df459ea feat: add metric label for scache load 
localhost:3001/metrics now includes:

pgrst_schema_cache_loads_total{status="FAIL"} 352.0
pgrst_schema_cache_loads_total{status="SUCCESS"} 3.0

This allows testing the failure case on:

https://github.com/PostgREST/postgrest/issues/3424#issuecomment-2104904910
 2024-05-14 17:57:21 -05:00
steve-chavez 05447bae33 nix: add postgrest-gen-jwt/secret for manual tests 
```
$ postgrest-gen-secret
uMd97XSQzNkA1CWhMZ7u88Pj0RNyhrpo

$ postgrest-gen-jwt postgrest_test_author
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoicG9zdGdyZXN0X3Rlc3RfYXV0aG9yIn0.Xod-F15qsGL0WhdOCr2j3DdKuTw9QJERVgoFD3vGaWA
```

Also modifies postgrest-run to include a default PGRST_JWT_SECRET for
quicker manual tests.
 2024-05-13 12:37:08 -05:00
steve-chavez 0060abeb01 feat: /live and /ready respond with 500 on failure 
503 is still used by /ready to indicate a transient state
that can be recovered from.
 2024-05-08 17:19:48 -05:00
steve-chavez f9e9740999 nix: add postgrest-ctags command 
Generates ctags for Haskell and Python code.
 2024-05-08 12:27:26 -05:00
steve-chavez 1b584f7e9c refactor: is ready Admin logic to AppState 2024-05-08 11:27:58 -05:00
steve-chavez df9b373465 docs: better place for application_name 2024-05-06 11:12:45 -05:00
steve-chavez b34c00c522 changelog: add missing entry for 3184 2024-05-06 11:12:45 -05:00
steve-chavez 1d4f31514f changelog: move 3340 from fixed to added 
Since it was a feature
 2024-05-06 11:12:45 -05:00
steve-chavez 7e91e5311d fix: not adding application_name on all URIs 2024-05-06 11:12:45 -05:00
steve-chavez d9a51f23ce test: sanity tests for primary and replica 
* new --replica option to `postgrest-with-postgresql-*`
* new command `postgrest-test-replica`
* new sanity tests on test_replica.py
* add postgrest-test-replica to postgrest-check and postgrest-coverage
 2024-04-30 18:35:40 -05:00
steve-chavez cdb877135b test: flush stdout so readline doesn't hang 2024-04-30 13:52:51 -05:00
steve-chavez 3eff4670f0 docs: metrics for schema cache and connection pool 2024-04-23 19:08:37 -05:00
steve-chavez 357400b2b8 feat: schema cache metrics 2024-04-23 19:08:37 -05:00
steve-chavez 653c7955b2 feat: connection pool metrics in admin server 2024-04-23 19:08:37 -05:00
steve-chavez 29cd7d195c nix: add instructions for haskell-packages.nix 2024-04-23 19:08:37 -05:00
steve-chavez 3e615bd0d9 feat: add log-level=debug 2024-04-22 21:36:29 -05:00
steve-chavez 1bf0c54dd6 feat: log connection pool events on log-level=info 2024-04-15 18:31:51 -05:00
Steve ChavezandGitHub 9d1dc783bf test: use postgrest.read_stdout in io tests (#3412) 
It's easier to maintain this way in case there are new log lines.
 2024-04-15 13:51:23 -05:00
steve-chavez 69c6ce9c38 refactor: use LogLevel in Logger 
* remove Logger dependency on Auth.
 2024-04-14 20:10:01 -05:00
steve-chavez c57ec52229 refactor: make stdout explicit on Logger 
Otherwise it's hard to know we're logging to stdout.
 2024-04-14 20:10:01 -05:00
steve-chavez fbc4d565ca refactor: move debounce from AppState to Logger 
Will allow to capture accurate timeout metrics.
 2024-04-12 14:29:39 -05:00
steve-chavez 2de32fc108 refactor: observation handler to AppConfig 
With this:

- Is no longer necessary to pass observer as an argument
  to every function that needs observations.
- We can invoke the observer on every function that uses AppConfig.
  However it'd be better to just call the observer in the upper modules
  (like on App.hs).
 2024-04-12 14:29:39 -05:00
steve-chavez b88191299f docs: remove schema cache dep from Query 2024-04-10 19:35:23 -05:00
steve-chavez c9136816a2 docs: move ARCHITECTURE.md to architecture.rst 2024-04-10 15:51:08 -05:00
steve-chavez 4428253efe docs: move health_check.rst to admin_server.rst 2024-04-10 15:51:08 -05:00
steve-chavez 47a4e2bfd0 docs: add proxy and config reloading to arch 2024-04-10 13:57:49 -05:00
steve-chavez 2772b78013 docs: add config/cli/developer to arch 2024-04-08 15:51:30 -05:00
steve-chavez b91908b749 docs: add architecture description 2024-04-08 13:50:00 -05:00
steve-chavez ab73624366 docs: add architecture diagram 2024-04-08 11:57:03 -05:00
steve-chavez 5ab317caa0 refactor: one entrypoint for Plan/Response/Query 
- deduplicates timing calculation for the different steps
- enabling query logging later on will be simpler
 2024-04-04 08:58:55 -05:00
steve-chavez 3d55f77bae Revert "fix: slow responses on schema cache reload" 
This reverts commit 727ef465c1.

Also documents requests waiting for the schema cache.
 2024-04-02 23:15:09 -05:00
steve-chavez d7c64a93f8 docs: update schema cache 2024-04-01 19:03:14 -05:00
steve-chavez 2543b8d724 fix: clarify PGRST204 error message 2024-04-01 19:03:14 -05:00
steve-chavez 06ff56d323 refactor: move isolation/settings logic to Plan.hs 2024-03-28 18:31:42 -05:00
steve-chavez c33ca4e60c refactor: dry timings calculation for openapi 2024-03-28 18:31:42 -05:00
steve-chavez b75cc853b4 chore: fix compilation 
The App.hs module was missing NamedFieldPuns.
 2024-03-27 19:18:17 -05:00
steve-chavez 745e7868b0 refactor: dry some timings calculation 2024-03-27 18:22:34 -05:00
steve-chavez a5bb20bbf8 refactor: remove unreacheable 404 2024-03-25 18:17:01 +01:00
steve-chavez 941ea0f929 test: notify do nothing 2024-03-19 13:22:51 +03:30
steve-chavez ee8b3ef8fe fix: log on LISTEN notification 2024-03-19 13:22:51 +03:30
steve-chavez 727ef465c1 fix: slow responses on schema cache reload 2024-03-15 11:53:29 -05:00
steve-chavez 11d8da046c fix: incorrect /ready response on slow schema load 2024-03-15 11:03:01 -05:00
steve-chavez 4b289b1c97 test: requests wait for schema cache load 
* nix: add postgrest-test-big-schema command
 2024-03-15 06:18:26 -05:00
steve-chavez 92ac7e574e chore: clarify concurrent notifications test 2024-03-14 23:17:05 -05:00
steve-chavez ec7ab271d6 chore: add postgrest roles to big_schema.sql 2024-03-14 11:21:17 -05:00
steve-chavez 4a2c851ae5 chore: update cabal/stack new hasql-notifications 2024-03-13 20:43:46 -05:00
steve-chavez 86e15dbb77 fix: upgrade hasql-notifications to show error 2024-03-13 11:14:11 -05:00
Steve ChavezandGitHub 00f5780415 fix: don't hide error on LISTEN channel failure (#3323) 2024-03-11 19:56:35 -05:00
Steve ChavezandGitHub 650249ed29 perf: remove json_typeof (#3316) 2024-03-08 16:02:28 -05:00
steve-chavez 9405a62de8 docs: update links to cache/config notify reload 2024-03-08 10:31:55 -05:00
steve-chavez 05cdbb34c2 test: insignificant JSON whitespace on writes 2024-03-07 17:14:16 -05:00
steve-chavez 8eb88ef218 test: reduce threshold on memory tests 2024-03-07 16:01:21 -05:00
Steve ChavezandGitHub 58999f5102 changelog: add how to detect o2o rels on v10.0.0 (#3312) 2024-03-06 20:21:58 -05:00
Steve ChavezandGitHub 2f91853cb1 docs: add index usage section (#3299) 2024-03-02 11:00:31 -05:00
Steve ChavezandGitHub d3f15baa4a nix: add postgrest-profiled-run (#3292) 2024-02-29 16:44:10 -05:00
steve-chavez 3a3601cbeb feat: log schema cache load time 2024-02-21 18:16:37 -05:00
Steve ChavezandGitHub 7c6c056e92 refactor: make observation messages pure (#3250) 
removes the observation messages from the Logger
 2024-02-20 18:42:38 -05:00
Steve ChavezandGitHub 6d506df6f3 refactor: add observation module (#3232) 2024-02-20 12:29:33 -05:00
steve-chavez 21e8ca5051 feat: log full pg version to stderr on connection 2024-02-10 19:21:55 -05:00
steve-chavez 5424be76d9 feat: log schema cache stats to stderr 
adjust the memory tests
 2024-02-09 16:18:04 -05:00
steve-chavez 45cabacdcc fix: wrong subquery error returning as 400 status 2024-02-02 22:08:56 -05:00
steve-chavez 76b1c00935 docs: fix link to op modifiers 2024-01-22 23:49:59 -05:00
Steve ChavezandGitHub 7a387c7a30 nix: add parallel-curl wrapper (#3172) 
stock curl doesn't make it easy to make many parallel requests to a
single endpoint. The endpoint has to be repeated N times.

```
curl --parallel https://example.com https://example.com ..
```

This provides a wrapper for parallel curl. It's useful for testing
scenarios like pool timeouts.

```
parallel-curl N https://example.com
```
 2024-01-18 20:03:01 -05:00
Steve ChavezandGitHub c070cb9502 Revert "build(deps): bump nix-community/cache-nix-action from 4.0.3 to 5.0.1 in /.github/actions/setup-nix" (#3167) 2024-01-17 17:55:38 -05:00
steve-chavez 4bb98225d8 docs: remove Heroku integration 2024-01-10 17:23:13 -05:00
steve-chavez fdaeea8fd2 fix: misleading "Starting.." logs on scache reload 2024-01-09 20:37:25 -05:00
steve-chavez 6c3d7a946d deprecate: params=single-object preference 2024-01-04 17:00:38 -05:00
steve-chavez c29d876f90 docs: add params preference 2024-01-04 16:09:07 -05:00
steve-chavez 1e4efafd9e docs: add tx preference 2024-01-04 16:09:07 -05:00
steve-chavez 50ac91d1ae docs: testing of media handlers as a note 
Otherwise users think they're required steps. See the opening comment on
https://github.com/PostgREST/postgrest/issues/3124.
 2024-01-02 23:44:53 -05:00
steve-chavez 3af63cb94c fix: empty row on handler function 
Closes https://github.com/PostgREST/postgrest/issues/3126
 2023-12-20 16:18:09 -05:00