jimmy/postgrest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,768 Commits 9 Branches 102 Tags
ce7ef3b188f0d517d2ed16f5af5120b7b0d78155
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
steve-chavez ce7ef3b188 chore: remove links to gitter 
We'll now use github discussions for support.
 2024-07-12 14:30:28 -05:00
steve-chavez 6be59066df fix: schema cache retrying without backoff 
Fixes https://github.com/PostgREST/postgrest/issues/3523.

Now if there's a failure when obtaining the pg version OR schema cache,
we do the same retrying process. This way we don't add two retries.

Refactors and renames the "connectionWorker" to "schemaCacheLoader".
This makes more sense since what we really want is the schema cache,
the version is the pre-requisite for ensuring our
schema cache queries work.

Additionally, we no longer log ` Attempting to connect to the database...`
at startup unnecessarily. This is only logged whenever there's a retry attempt.
 2024-07-10 21:14:24 -05:00
steve-chavez 0dc1345eb4 chore: remove paypal links 
It was tied to a personal account and donations there have been too rare.
 2024-07-08 10:19:30 -05:00
steve-chavez ecf9d56c91 docs: add listener recovery 2024-06-25 20:03:22 -05:00
steve-chavez f912c0dd29 fix: don't reload cache on every listener fail 
Revert "prevent GSSAPI error between Listener and pool"

This reverts commit 4beac10d3d.
 2024-06-25 20:03:22 -05:00
steve-chavez 21f15643b4 bump version to 12.3 2024-06-11 09:57:05 -05:00
steve-chavez ec89f6b90c bump version to 12.2.0 2024-06-11 09:57:05 -05:00
steve-chavez 82aa58a08f docs: deprecate EOL pg versions 2024-06-11 09:43:09 -05:00
steve-chavez 4beac10d3d prevent GSSAPI error between Listener and pool 
Brings back the the signaling/waiting between the connection pool and
the Listener.

Prevents the GSSAPI error shown on https://github.com/PostgREST/postgrest/issues/3569
 2024-06-05 13:58:34 -05:00
steve-chavez 1a8b6972a8 correct exponential backoff on Listener 
Clears the limitation mentioned on

https://github.com/PostgREST/postgrest/pull/3536

The Listener no longer uses the https://hackage.haskell.org/package/retry
package and instead uses a much simpler IORef in AppState for the
delays.

Additionally it no longer uses exception throwing/catching, which
is rather messy and brings some
concerns(https://github.com/PostgREST/postgrest/issues/3569#issuecomment-2146013327).
 2024-06-05 08:52:18 -05:00
steve-chavez 82a43c2767 nix: postgrest-gen-ctags use haskdogs 2024-05-25 15:21:44 -05:00
steve-chavez a51a74b3c1 docs: shorten CLI 2024-05-24 18:05:45 -05:00
steve-chavez 1c371d7340 docs: clarify architecture 2024-05-24 18:05:45 -05:00
steve-chavez 30ca64d849 docs: improve observability 2024-05-24 18:05:45 -05:00
steve-chavez da9e497ef1 changelog: add architecture doc 2024-05-23 19:34:42 -05:00
steve-chavez 47e9a2d134 refactor: Listener to own module 2024-05-23 19:34:42 -05:00
steve-chavez 7e61c9deb0 feat: force read-write for listener connection 2024-05-19 23:14:19 -05:00
steve-chavez 3cf565614d fix: listener retries with exponential backoff 
Also corrects the admin ready response which now considers the listener
state.
 2024-05-19 20:48:59 -05:00
steve-chavez bfa4e1bedb test: isolate pg_terminate_backend to appname 
The pg_terminate_backend done in the io test:

`test_fail_with_automatic_recovery_disabled_and_terminated_using_query`

Can affect other connections.
 2024-05-19 19:58:27 -05:00
steve-chavez d5a4c5609e refactor: greppable 57P01 error code 2024-05-19 19:58:27 -05:00
steve-chavez c8ba505b99 docs: linkcheck ignore patreon 2024-05-19 13:44:23 -05:00
steve-chavez ae91fd89b3 test: assert empty response text on 204 2024-05-19 13:44:23 -05:00
steve-chavez 756aad7827 fix: listener silent fail on replica 
Update hasql-notifications to include the fix on
https://github.com/diogob/hasql-notifications/issues/24.

Which now reveals the following error:

```
$ postgrest-with-postgresql-16 --replica -f test/spec/fixtures/load.sql postgrest-run

17/May/2024:18:35:38 -0500: Successfully connected to PostgreSQL 16.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 13.2.0, 64-bit
17/May/2024:18:35:38 -0500: Could not listen for notifications on the "pgrst" channel. ERROR:  cannot execute LISTEN during recovery
17/May/2024:18:35:38 -0500: Retrying listening for notifications...
```

This is still not good because the LISTEN channel will be retried
forever without a backoff.
 2024-05-18 23:33:04 -05:00
steve-chavez aa75412932 nix: PGRST_DB_URI preference for tmp db replica 
When using `postgrest-with-postgresql-* --replica`, the PGRST_DB_URI
will set the replica host as preference. This to be able to run
quick manual tests with postgrest running on a replica.
 2024-05-18 23:33:04 -05:00
steve-chavez 33b6ba8199 refactor: move checkIsFatal logic to usePool 
The fatal logic is now inside `usePool`. It centralizes the
logic which is better for Locality of Behavior.

Removes:

- The need to do checkIsFatal on other parts of the code
- SCFatalFail/ConnFatalFail states which are no longer needed.
 2024-05-16 17:40:25 -05:00
steve-chavez d25df459ea feat: add metric label for scache load 
localhost:3001/metrics now includes:

pgrst_schema_cache_loads_total{status="FAIL"} 352.0
pgrst_schema_cache_loads_total{status="SUCCESS"} 3.0

This allows testing the failure case on:

https://github.com/PostgREST/postgrest/issues/3424#issuecomment-2104904910
 2024-05-14 17:57:21 -05:00
steve-chavez 05447bae33 nix: add postgrest-gen-jwt/secret for manual tests 
```
$ postgrest-gen-secret
uMd97XSQzNkA1CWhMZ7u88Pj0RNyhrpo

$ postgrest-gen-jwt postgrest_test_author
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoicG9zdGdyZXN0X3Rlc3RfYXV0aG9yIn0.Xod-F15qsGL0WhdOCr2j3DdKuTw9QJERVgoFD3vGaWA
```

Also modifies postgrest-run to include a default PGRST_JWT_SECRET for
quicker manual tests.
 2024-05-13 12:37:08 -05:00
steve-chavez 0060abeb01 feat: /live and /ready respond with 500 on failure 
503 is still used by /ready to indicate a transient state
that can be recovered from.
 2024-05-08 17:19:48 -05:00
steve-chavez f9e9740999 nix: add postgrest-ctags command 
Generates ctags for Haskell and Python code.
 2024-05-08 12:27:26 -05:00
steve-chavez 1b584f7e9c refactor: is ready Admin logic to AppState 2024-05-08 11:27:58 -05:00
steve-chavez df9b373465 docs: better place for application_name 2024-05-06 11:12:45 -05:00
steve-chavez b34c00c522 changelog: add missing entry for 3184 2024-05-06 11:12:45 -05:00
steve-chavez 1d4f31514f changelog: move 3340 from fixed to added 
Since it was a feature
 2024-05-06 11:12:45 -05:00
steve-chavez 7e91e5311d fix: not adding application_name on all URIs 2024-05-06 11:12:45 -05:00
steve-chavez d9a51f23ce test: sanity tests for primary and replica 
* new --replica option to `postgrest-with-postgresql-*`
* new command `postgrest-test-replica`
* new sanity tests on test_replica.py
* add postgrest-test-replica to postgrest-check and postgrest-coverage
 2024-04-30 18:35:40 -05:00
steve-chavez cdb877135b test: flush stdout so readline doesn't hang 2024-04-30 13:52:51 -05:00
steve-chavez 3eff4670f0 docs: metrics for schema cache and connection pool 2024-04-23 19:08:37 -05:00
steve-chavez 357400b2b8 feat: schema cache metrics 2024-04-23 19:08:37 -05:00
steve-chavez 653c7955b2 feat: connection pool metrics in admin server 2024-04-23 19:08:37 -05:00
steve-chavez 29cd7d195c nix: add instructions for haskell-packages.nix 2024-04-23 19:08:37 -05:00
steve-chavez 3e615bd0d9 feat: add log-level=debug 2024-04-22 21:36:29 -05:00
steve-chavez 1bf0c54dd6 feat: log connection pool events on log-level=info 2024-04-15 18:31:51 -05:00
Steve ChavezandGitHub 9d1dc783bf test: use postgrest.read_stdout in io tests (#3412) 
It's easier to maintain this way in case there are new log lines.
 2024-04-15 13:51:23 -05:00
steve-chavez 69c6ce9c38 refactor: use LogLevel in Logger 
* remove Logger dependency on Auth.
 2024-04-14 20:10:01 -05:00
steve-chavez c57ec52229 refactor: make stdout explicit on Logger 
Otherwise it's hard to know we're logging to stdout.
 2024-04-14 20:10:01 -05:00
steve-chavez fbc4d565ca refactor: move debounce from AppState to Logger 
Will allow to capture accurate timeout metrics.
 2024-04-12 14:29:39 -05:00
steve-chavez 2de32fc108 refactor: observation handler to AppConfig 
With this:

- Is no longer necessary to pass observer as an argument
  to every function that needs observations.
- We can invoke the observer on every function that uses AppConfig.
  However it'd be better to just call the observer in the upper modules
  (like on App.hs).
 2024-04-12 14:29:39 -05:00
steve-chavez b88191299f docs: remove schema cache dep from Query 2024-04-10 19:35:23 -05:00
steve-chavez c9136816a2 docs: move ARCHITECTURE.md to architecture.rst 2024-04-10 15:51:08 -05:00
steve-chavez 4428253efe docs: move health_check.rst to admin_server.rst 2024-04-10 15:51:08 -05:00
steve-chavez 47a4e2bfd0 docs: add proxy and config reloading to arch 2024-04-10 13:57:49 -05:00
steve-chavez 2772b78013 docs: add config/cli/developer to arch 2024-04-08 15:51:30 -05:00
steve-chavez b91908b749 docs: add architecture description 2024-04-08 13:50:00 -05:00
steve-chavez ab73624366 docs: add architecture diagram 2024-04-08 11:57:03 -05:00
steve-chavez 5ab317caa0 refactor: one entrypoint for Plan/Response/Query 
- deduplicates timing calculation for the different steps
- enabling query logging later on will be simpler
 2024-04-04 08:58:55 -05:00
steve-chavez 3d55f77bae Revert "fix: slow responses on schema cache reload" 
This reverts commit 727ef465c1.

Also documents requests waiting for the schema cache.
 2024-04-02 23:15:09 -05:00
steve-chavez d7c64a93f8 docs: update schema cache 2024-04-01 19:03:14 -05:00
steve-chavez 2543b8d724 fix: clarify PGRST204 error message 2024-04-01 19:03:14 -05:00
steve-chavez 06ff56d323 refactor: move isolation/settings logic to Plan.hs 2024-03-28 18:31:42 -05:00
steve-chavez c33ca4e60c refactor: dry timings calculation for openapi 2024-03-28 18:31:42 -05:00
steve-chavez b75cc853b4 chore: fix compilation 
The App.hs module was missing NamedFieldPuns.
 2024-03-27 19:18:17 -05:00
steve-chavez 745e7868b0 refactor: dry some timings calculation 2024-03-27 18:22:34 -05:00
steve-chavez a5bb20bbf8 refactor: remove unreacheable 404 2024-03-25 18:17:01 +01:00
steve-chavez 941ea0f929 test: notify do nothing 2024-03-19 13:22:51 +03:30
steve-chavez ee8b3ef8fe fix: log on LISTEN notification 2024-03-19 13:22:51 +03:30
steve-chavez 727ef465c1 fix: slow responses on schema cache reload 2024-03-15 11:53:29 -05:00
steve-chavez 11d8da046c fix: incorrect /ready response on slow schema load 2024-03-15 11:03:01 -05:00
steve-chavez 4b289b1c97 test: requests wait for schema cache load 
* nix: add postgrest-test-big-schema command
 2024-03-15 06:18:26 -05:00
steve-chavez 92ac7e574e chore: clarify concurrent notifications test 2024-03-14 23:17:05 -05:00
steve-chavez ec7ab271d6 chore: add postgrest roles to big_schema.sql 2024-03-14 11:21:17 -05:00
steve-chavez 4a2c851ae5 chore: update cabal/stack new hasql-notifications 2024-03-13 20:43:46 -05:00
steve-chavez 86e15dbb77 fix: upgrade hasql-notifications to show error 2024-03-13 11:14:11 -05:00
Steve ChavezandGitHub 00f5780415 fix: don't hide error on LISTEN channel failure (#3323) 2024-03-11 19:56:35 -05:00
Steve ChavezandGitHub 650249ed29 perf: remove json_typeof (#3316) 2024-03-08 16:02:28 -05:00
steve-chavez 9405a62de8 docs: update links to cache/config notify reload 2024-03-08 10:31:55 -05:00
steve-chavez 05cdbb34c2 test: insignificant JSON whitespace on writes 2024-03-07 17:14:16 -05:00
steve-chavez 8eb88ef218 test: reduce threshold on memory tests 2024-03-07 16:01:21 -05:00
Steve ChavezandGitHub 58999f5102 changelog: add how to detect o2o rels on v10.0.0 (#3312) 2024-03-06 20:21:58 -05:00
Steve ChavezandGitHub 2f91853cb1 docs: add index usage section (#3299) 2024-03-02 11:00:31 -05:00
Steve ChavezandGitHub d3f15baa4a nix: add postgrest-profiled-run (#3292) 2024-02-29 16:44:10 -05:00
steve-chavez 3a3601cbeb feat: log schema cache load time 2024-02-21 18:16:37 -05:00
Steve ChavezandGitHub 7c6c056e92 refactor: make observation messages pure (#3250) 
removes the observation messages from the Logger
 2024-02-20 18:42:38 -05:00
Steve ChavezandGitHub 6d506df6f3 refactor: add observation module (#3232) 2024-02-20 12:29:33 -05:00
steve-chavez 21e8ca5051 feat: log full pg version to stderr on connection 2024-02-10 19:21:55 -05:00
steve-chavez 5424be76d9 feat: log schema cache stats to stderr 
adjust the memory tests
 2024-02-09 16:18:04 -05:00
steve-chavez 45cabacdcc fix: wrong subquery error returning as 400 status 2024-02-02 22:08:56 -05:00
steve-chavez 76b1c00935 docs: fix link to op modifiers 2024-01-22 23:49:59 -05:00
Steve ChavezandGitHub 7a387c7a30 nix: add parallel-curl wrapper (#3172) 
stock curl doesn't make it easy to make many parallel requests to a
single endpoint. The endpoint has to be repeated N times.

```
curl --parallel https://example.com https://example.com ..
```

This provides a wrapper for parallel curl. It's useful for testing
scenarios like pool timeouts.

```
parallel-curl N https://example.com
```
 2024-01-18 20:03:01 -05:00
Steve ChavezandGitHub c070cb9502 Revert "build(deps): bump nix-community/cache-nix-action from 4.0.3 to 5.0.1 in /.github/actions/setup-nix" (#3167) 2024-01-17 17:55:38 -05:00
steve-chavez 4bb98225d8 docs: remove Heroku integration 2024-01-10 17:23:13 -05:00
steve-chavez fdaeea8fd2 fix: misleading "Starting.." logs on scache reload 2024-01-09 20:37:25 -05:00
steve-chavez 6c3d7a946d deprecate: params=single-object preference 2024-01-04 17:00:38 -05:00
steve-chavez c29d876f90 docs: add params preference 2024-01-04 16:09:07 -05:00
steve-chavez 1e4efafd9e docs: add tx preference 2024-01-04 16:09:07 -05:00
steve-chavez 50ac91d1ae docs: testing of media handlers as a note 
Otherwise users think they're required steps. See the opening comment on
https://github.com/PostgREST/postgrest/issues/3124.
 2024-01-02 23:44:53 -05:00
steve-chavez 3af63cb94c fix: empty row on handler function 
Closes https://github.com/PostgREST/postgrest/issues/3126
 2023-12-20 16:18:09 -05:00
steve-chavez 1680a6eed3 fix: aggregates not working for all schemas 
Closes https://github.com/PostgREST/postgrest/issues/3124
 2023-12-20 15:20:18 -05:00
steve-chavez 9e04bd7da6 docs: soap how-to is outdated 
`*/*` can now be handled with media type handlers
 2023-12-18 23:58:17 -05:00
steve-chavez a59cfafe10 fix: correct any media type handler 2023-12-12 17:45:14 -05:00
steve-chavez 6b9fde59cd fix: any handler sets a default application/json 
Now it sets application/octet-stream as the generic type.
 2023-12-12 17:44:07 -05:00