Wolfgang Walther
59dffc258a
We previously used "rate", i.e. number of requests per second, as the primary metric to judge loadtest results. However, this has always been varying from run to run quite a bit, especially in CI where other jobs possibly run on the same VM host. The run-to-run variance has massively increased after splitting the results up per request. Example run in CI with rate on the PR introducing this change (on which we would expect no change at all): | rate [1/s] | main | head | Δ | |:-----------------------------------|-------:|-------:|-----:| | / | 9.4 | 9.5 | 1% | | /actors | 870.4 | 1023.0 | 18% | | /actors?actor=eq.1 | 188.5 | 198.6 | 5% | | /actors?actor=eq.1&columns=name | 197.3 | 167.1 | -15% | | /actors?select=*,roles(*,films(*)) | 153.9 | 144.9 | -6% | | /films?columns=id,title | 157.9 | 182.6 | 16% | | /films?columns=id,title,year,... | 87.0 | 87.1 | 0% | | /roles | 204.5 | 267.3 | 31% | | /rpc/call_me | 231.3 | 208.8 | -10% | | /rpc/call_me?name=John | 212.2 | 201.7 | -5% | From the data we can easily tell that the very reason that rate as a paramter has only worked, so far, because the data was *heavily* dominated by the requests on the root endpoint for OpenAPI. The longer duration makes the request much less vulnerable for concurrent activity. For all other requests its essentially not possible to judge the effect of a PR this way. One way to counter this would be to massively increase the time the loadtest runs. More samples will result in a smoother average. However, that's not practical for usability of CI. In the original PR #1812 I already evaluated using the *minimum latency* as the most reliable criteriumi, but this has never really caught on. The theory behind this is: The variation in timings between requests is happening because of concurrent activity, priority chosen by the scheduler, availability of resources and such - all factors *outside* our control, and *irrelevant* to the Haskell code we're writing. Using the minimum latency is an estimation of how fast the code can run *in the best case*. This might not be a number relevant for production, but it's much more directly related to the code we write. Here's to show how variation becomes *much* smaller with minimum latency as the parameter: | min latency [μs] | main | head | Δ | |:-----------------------------------|---------:|-------:|-----:| | / | 1275.3 | 1263.6 | -1% | | /actors | 10.0 | 9.9 | -1% | | /actors?actor=eq.1 | 50.7 | 48.3 | -5% | | /actors?actor=eq.1&columns=name | 54.1 | 54.0 | 0% | | /actors?select=*,roles(*,films(*)) | 63.2 | 61.9 | -2% | | /films?columns=id,title | 51.1 | 50.7 | -1% | | /films?columns=id,title,year,... | 121.9 | 121.8 | 0% | | /roles | 42.9 | 42.6 | -1% | | /rpc/call_me | 45.6 | 45.4 | 0% | | /rpc/call_me?name=John | 44.4 | 44.2 | 0% | Since we're separating results per request now, we can only sensibly focus on *one* parameter - otherwise this would get really clunky UI-wise. Especially for automated CI failures, minimum latency is the logical choice. This commit starts using minimum latency, i.e. P0, but any percentile should be an improvement over the status quo. A later commit will change to a different P-value.
PostgREST serves a fully RESTful API from any existing PostgreSQL database. It provides a cleaner, more standards-compliant, faster API than you are likely to write from scratch.
Sponsors
|
|
|
|
|
Big thanks to our sponsors! You can join them by supporting PostgREST on Patreon.
Usage
-
See the docs for how to install PostgREST on your platform. You can also use Docker.
-
Invoke for help:
postgrest --help
Documentation
Latest documentation is at postgrest.org. You can contribute to the docs in PostgREST/postgrest/docs.
Performance
TLDR; subsecond response times for up to 2000 requests/sec on Heroku free tier. If you're used to servers written in interpreted languages, prepare to be pleasantly surprised by PostgREST performance.
Three factors contribute to the speed. First the server is written in Haskell using the Warp HTTP server (aka a compiled language with lightweight threads). Next it delegates as much calculation as possible to the database including
- Serializing JSON responses directly in SQL
- Data validation
- Authorization
- Combined row counting and retrieval
- Data post in single command (
returning *)
Finally it uses the database efficiently with the Hasql library by
- Keeping a pool of db connections
- Using the PostgreSQL binary protocol
- Being stateless to allow horizontal scaling
Security
PostgREST handles authentication (via JSON Web Tokens) and delegates authorization to the role information defined in the database. This ensures there is a single declarative source of truth for security. When dealing with the database the server assumes the identity of the currently authenticated user, and for the duration of the connection cannot do anything the user themselves couldn't. Other forms of authentication can be built on top of the JWT primitive. See the docs for more information.
Versioning
A robust long-lived API needs the freedom to exist in multiple versions. PostgREST does versioning through database schemas. This allows you to expose tables and views without making the app brittle. Underlying tables can be superseded and hidden behind public facing views.
Self-documentation
PostgREST uses the OpenAPI standard to generate up-to-date documentation for APIs. You can use a tool like Swagger-UI to render interactive documentation for demo requests against the live API server.
This project uses HTTP to communicate other metadata as well. For instance the number of rows returned by an endpoint is reported by - and limited with - range headers. More about that.
Data Integrity
Rather than relying on an Object Relational Mapper and custom imperative coding, this system requires you to put declarative constraints directly into your database. Hence no application can corrupt your data (including your API server).
The PostgREST exposes HTTP interface with safeguards to prevent surprises, such as enforcing idempotent PUT requests.
See examples of PostgreSQL constraints and the API guide.
Supporting development
You can help PostgREST ongoing maintenance and development by making a regular donation through Patreon https://www.patreon.com/postgrest
Every donation will be spent on making PostgREST better for the whole community.
Contributing
Contributions are always welcome and appreciated. Please see the Contributing guidelines.
Thanks
The PostgREST organization is grateful to:
- The project sponsors and backers who support PostgREST's development.
- The project contributors who have improved PostgREST immensely with their code and good judgement. See more details in the changelog.
The cool logo came from Mikey Casalaina.