Changes:
1. Refactoring and some cleanup of JWT handling code:
* Instead of caching AuthResult cache decoded claims (which signature was verified). Validating claims and determining role is done after cache lookup
* Cleaned up API so that usage of it is simplified: lookupJwtCache cache key >>= parseClaims configJwtAud time
* Handling of JwtCacheState initialization and updates of configuration is encapsulated in Auth.JwtCache module
2. Generic high performance (hopefully) scalable, dynamically resizeable cache implementation based on stm, stm-hamt and sieve algorithm. It also integrates with PostgREST measurements infrastructure providing usage stats (ie. hit ratio, evictions count)
Completes the jwt loadtests, now we have non-cached, cached + worst case
for the cache.
- jwt-hs (edited): now has the cache disabled
- jwt-hs-cache: cache enabled for jwt-hs
- jwt-hs-cache-worst: worst case of the jwt-hs cache
- jwt-rsa (edited): now has the cache disabled
- jwt-rsa-cache: cache enabled for jwt-rsa
- jwt-rsa-cache-worst: worst case of the jwt-rsa cache
Also deletes `nix/tools/generate_targets_rsa.py` and uses a single
python script.
Should prove what's mentioned on
https://github.com/PostgREST/postgrest/pull/4084#issuecomment-2998170423
Splitting the executable in a separate output avoids distributing all of
the libraries and documentation, which are not needed when just running
PostgREST.
Reduces closure size from 4.3G to 73.9M for the flake exported packages.
Resolves#4149
Closes https://github.com/PostgREST/postgrest/issues/4107.
Adds two python scripts:
- monitor_pid.py: monitors the postgrest process each second
until it exits, then outputs a csv with the results. The nix wrappers
use the `loadtest` dir for the output.
- merge_monitor_result.py: receives a list of csvs and merges them into
a single markdown table. The nix wrappers use the `loadtest/*.csv`
files for the input.
The nix `postgrest-with-pgrst` and `postgrest-loadtest-report` commands
use these scripts to add monitoring for `postgrest-loadtest` and
`postgrest-loadtest-against`.
This is so the process monitoring results are more meaningful.
* remove the amount of total targets from the loadtest kind name
since the targets can change.
* clarifies the top comment on the generate_targets python script
* increase TOTAL_TARGETS on jwt-hs loadtest
Until now we had a load test with 50k unique JWTs signed with symmetric key.
This commit adds a new load test with 10k JWTs signed with RSA 4096.
Existing -k jwt parameter was changed to -k jwt-hs-50k.
New test is run with -k jwt-rsa-10k parameter.
Additionally a new parameter --jwtcache=off was added to turn off JWT caching in the above load tests.
This loadtests the jwt decoding logic. For this it adds an optional
`-k`(kind) parameter to `postgrest-loadtest` and
`postgrest-loadtest-against`.
Old kind (default):
```
postgrest-loadtest -k mixed
postgrest-loadtest-against -k mixed
```
New kind:
```
postgrest-loadtest -k jwt
postgrest-loadtest-against -k jwt
```
Internally it uses a dynamically generated targets file using python
which looks like:
```
GET http://postgrest/authors_only
Authorization: Bearer <jwt>
GET http://postgrest/authors_only
Authorization: Bearer <another-jwt>
...
```
Then this is used to run vegeta with the `-lazy` option.
This also moves the pin for nixpkgs into flake.lock instead of our
custom file. Even for the classic interface via default.nix, the pin
will be loaded from flake.lock, thus everything stays in-sync.
Since we're currently on the unstable channel and will likely stay there
for a while, let's encode this in the update script.
Once we switch back to stable, if we do, we can still adjust it again.
Timing dependent tests in the IO tests don't work too well when the next
commit increases the JWT parsing performance.
The remaining IO tests are for coverage and basic breakage. Loadtests
are adapted so that performance regressions for JWT caching would be
detected that way.
When using `postgrest-with-postgresql-* --replica`, the PGRST_DB_URI
will set the replica host as preference. This to be able to run
quick manual tests with postgrest running on a replica.
```
$ postgrest-gen-secret
uMd97XSQzNkA1CWhMZ7u88Pj0RNyhrpo
$ postgrest-gen-jwt postgrest_test_author
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoicG9zdGdyZXN0X3Rlc3RfYXV0aG9yIn0.Xod-F15qsGL0WhdOCr2j3DdKuTw9QJERVgoFD3vGaWA
```
Also modifies postgrest-run to include a default PGRST_JWT_SECRET for
quicker manual tests.
This changes the postgrest-release tool to work with our new workflow.
It can be run on main and the v* release branches. When on a release
branch, it will bump a patch version and push to that branch only.
When on main, it will bump a minor version by default. To bump a major
version, pass --major. This first bump will be force-pushed to the
v<major> branch. A second bump to the current development version will
then be pushed to the main branch.
The tool will not tag commits anymore - this happens in CI
automatically.
References #3113Resolves#3082
