75 Commits
Author SHA1 Message Date
Michał KłeczekandSteve Chavez e252a4900c fix: Performance and high memory usage of relation hint calculation
* Calculation of hint message when requested relation is not present in schema cache requires creation of a FuzzySet (to use fuzzy search to find candidate tables). For schemas with many tables it is costly.
This patch introduces dbTablesFuzzyIndex in SchemaCache to memoize the FuzzySet creation.

* Additionally, because of FuzzySet large memory requirements, this patch introduces a limit of 500 relations per schema, above which FuzzySet is not created and hint calculation disabled.

(cherry picked from commit e592d568c6)
2026-01-03 15:18:37 +08:00
Taimoor ZaeemandSteve Chavez 939061baff refactor: move escapeIdent function to Identifiers.hs
Moves the functions `escapeIdent` and `trimNullChars` to
SchemaCache/Identifiers.hs module.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 66f84c5903)
2025-10-30 12:24:49 -05:00
Michal KleczekandGitHub be9a0ab467 perf: Index various lists in SchemaCache to change complexity from O(n*n) to O(n) (#4396) 2025-10-20 21:17:50 -05:00
Michał KłeczekandSteve Chavez c08b87749b test: Separated query and loading internal sleep configs
To make schema cache loading wait tests robust it is necessary to provide three separate internal config variables:
* "internal-schema-cache-query-sleep" - introduces delay in schema queries execution
* "internal-schema-cache-load-sleep" - introduces delay between schema queries execution and processing their results
* "internal-schema-cache-relationship-load-sleep" - introduces delay in processing relationship query results

Thanks to these changes it is now possible to test various schema loading scenarios with the right granularity robustly (eg. make sure requests wait for schema loading but not for relationship loading).
2025-10-17 13:42:35 -05:00
steve-chavez 1dca23e7af refactor: move accessibleFuncs to SqlFragment 2025-09-04 09:35:33 -05:00
steve-chavez 9c07070263 refactor: move accessibleTables to SqlFragment 2025-09-04 09:35:33 -05:00
steve-chavez 1ace298a1a refactor: move schemaCacheDescription to SqlFragment
This doesn't belong in the schema cache module, as
it's unused there. It's only used on Query.hs.
2025-09-04 09:35:33 -05:00
steve-chavez 1dce4a9321 Revert "fix: geojson invalid query on unavailable PostGIS"
This reverts commit 0f1ca8faac.

Reverting for now as it adds one more query to the schema cache and
there's no clear way forward on how to integrate the fix with the
current schema cache queries.

See discussion on
https://github.com/PostgREST/postgrest/pull/4246#pullrequestreview-3093174224.
2025-08-13 14:54:53 -05:00
steve-chavez 0f1ca8faac fix: geojson invalid query on unavailable PostGIS
Closes https://github.com/PostgREST/postgrest/issues/4245.

It adds a query on schema cache construction, but doesn't add a new
attribute to the SchemaCache type. We only need the information to be
passed to the `initialMediaHandlers` function to build the builtin media
handlers.
2025-08-06 10:20:01 -05:00
Laurence Isla 1879c813c8 fix: detect the correct base type of recursive domains in schema cache for tables and views
In OpenAPI it shows the correct base type in properties' definitions (including enums).
2025-06-16 16:53:29 +00:00
Taimoor ZaeemandWolfgang Walther 657cabe757 fix: schema cache loads duplicate objects with different object type but same oid 2025-05-08 19:42:26 +00:00
M. Taimoor ZaeemandSteve Chavez 56c14474da fix: insert with missing=default uses column default before using domain default 2024-12-23 15:27:40 -05:00
Wolfgang WaltherandWolfgang Walther d7da18147b refactor: Simplify pks_uniques_cols 2024-07-13 22:26:36 +02:00
Wolfgang WaltherandWolfgang Walther 50bdb6a3de fix: Embed One-to-One relationship with different column order properly 2024-07-13 22:26:36 +02:00
Wolfgang WaltherandWolfgang Walther b598b594df refactor: Simplify schemaDescription query 2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 86c3257f54 feat: Fail schema cache lookup with invalid db-schemas config
Previously, we'd silently report "200 OK" on the root endpoint, but
would never return any endpoints from the schema cache.

Now the schema cache query fails because of the ::regnamespace cast.
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther f31848f2e5 refactor: Simplify funcsSqlQuery
This allows to re-use ANY($$1) in the next commit.
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 735e1edbf6 refactor: Simplify columns_agg 2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 01a18d8199 fix: List correct enum options when multiple types with same name are present
The schema cache and OpenAPI output would currently list the first found
enum with the same name instead of the correct type. One other case
where this comes up is when a regular type and an enum type have the
same name. For example in the spec fixtures, we have an enum called
"bit". Every "bit" type, no matter whether it's that enum or the
built-in bit type, will show those enum options in the OpenApi output.

Not adding a test, because OpenAPI is supposed to go away in the future
anyway.
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 1747a4fcc4 refactor: Replace pg_namespace joins with ::regnamespace in schema cache
Less joins are much easier to read and understand.
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 5ea83de7ec refactor: Simplify tbl_pk_cols query 2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 3daeec4e76 refactor: Make schema cache dumps more predictable with consistent ORDER
This helps diffing schema cache changes during development.
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 0e2f78d9f2 refactor: Use ::regnamespace casts instead of comparing schemas by name
Casting pg_catalog to regnamespace is slightly more efficient, because
the comparison will be oid-based, not text-based.
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther baae4d715f refactor: Remove redundant conditions in schema cache
Those conditions are covered by the respective nspname = ANY branches.
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 6581663da9 refactor: Remove useless DISTINCT
There is already a GROUP BY in the same SELECT.
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther dfb0be9354 refactor: Remove unused columns from schema cache queries 2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 7e8e9a9529 refactor: Fix some spelling mistakes in comments and whitespace 2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther 0335b465d7 fix: Show number of loaded timezones in log output
There is no reason to hide those, right?
2024-07-09 08:31:31 +02:00
Wolfgang WaltherandWolfgang Walther e5fb1e0ec1 refactor: Replace interpolatedstring-perl6 with neat-interpolation
The former depends on th-orphans which does not cross-compile well,
because of template haskell usage.

neat-interpolation is also much better maintained.

This also potentially helps with packaging for Debian/Ubuntu in #2273.
2024-06-16 14:01:46 +02:00
Wolfgang WaltherandWolfgang Walther 0b25039f0f refactor: Pass params in SchemaCache without contrazip2
Contravariant.Extras uses Template Haskell, which is hard to
cross-compile. Reducing usage of Template Haskell with the ultimate goal
of solving all cross compilation challenges.
2024-06-16 00:59:10 +02:00
Wolfgang WaltherandWolfgang Walther bb96c2dc74 feat: Drop support for pg 11
PostgreSQL 11 is EOL since November 2023.
2024-06-15 17:23:34 +02:00
Wolfgang WaltherandWolfgang Walther 126178642b feat: Drop support for pg 10 2024-06-15 17:23:34 +02:00
Wolfgang WaltherandWolfgang Walther daa77d17aa feat: Drop support for pg 9.6 2024-06-15 17:23:34 +02:00
Taimoor ZaeemandSteve Chavez 8cbcf9867b feat: add config db-hoisted-tx-settings to allow only hoisted function settings 2024-05-21 19:50:03 -05:00
Laurence IslaandGitHub 7671d63f06 refactor: add isParent flag to O2O relationships
It allows to identify the side with the FK when isParent == False
2024-05-21 12:11:27 -05:00
Wolfgang WaltherandWolfgang Walther 8392357863 test: Fix internal_schema_cache_sleep after 747c78f6
The $subject commit broke internal_schema_cache_sleep for other tests.
This reverts the order change, but keeps the scaling by x1000 to ms and
thus changes other users of this setting to the new scale.
2024-05-12 11:44:43 +02:00
Wolfgang WaltherandWolfgang Walther 747c78f6f4 test: Prevent test_admin_ready_includes_schema_cache_state from timing out
By increasing the delays in this test by factor 400x, postgrest will not
swamp pg with connection retries after the failed schema cache anymore.

This would happen because there is no backoff included after fatal
errors. Once it does, the io tests hang indefinitely in CI.
2024-05-09 13:36:03 +02:00
Wolfgang Walther ed407350ad fix: hoist function settings with memory units properly
f9ee1f7e introduced the hoisting of function settings as transaction-scoped
settings. However, this currently doesn't work with memory units, which are
case-sensitive according to the docs [1]. This removes the lowercasing of
values to make them work.

This is not added to the CHANGELOG, because this feature was not released, yet.

[1]: https://www.postgresql.org/docs/current/config-setting.html#CONFIG-SETTING-NAMES-VALUES
2024-04-20 13:58:04 +02:00
Wolfgang WaltherandWolfgang Walther 8a21a9c34e fix: Dump media handlers and timezones with --dump-schema
Those were left out of the schema dump when the features were introduced, probably
because ByteString doesn't have a toJSON instance. Changing the type to Text solves
this easily.

Resolves #3237
2024-02-20 18:44:17 +01:00
Wolfgang Walther 00fbe9ff3e fix: Avoid casting to table type when select= and media type handler are used
Previously using a generic mimetype handler failed when any kind of select= was given, because
we tried to cast the select-result to the original table type. With this change, this cast is
only applied when select=* is given implicitly or explicitly. This is the only case where this
makes sense, because this guarantees that correct columns are selected in the correct order for
this cast to succeed.

Resolves #3160
2024-02-15 19:01:12 +01:00
steve-chavez 5424be76d9 feat: log schema cache stats to stderr
adjust the memory tests
2024-02-09 16:18:04 -05:00
Taimoor ZaeemandSteve Chavez f9ee1f7e73 feat: apply all function settings as transaction-scoped settings 2024-02-09 11:37:01 -05:00
steve-chavez 1680a6eed3 fix: aggregates not working for all schemas
Closes https://github.com/PostgREST/postgrest/issues/3124
2023-12-20 15:20:18 -05:00
steve-chavez 6b9fde59cd fix: any handler sets a default application/json
Now it sets application/octet-stream as the generic type.
2023-12-12 17:44:07 -05:00
steve-chavez 9fe11249f9 feat: custom SQL handler for the "*/*" media type 2023-11-28 23:14:38 -05:00
Taimoor ZaeemandGitHub 125f10a60f feat: add statement_timeout set on functions (#3056) 2023-11-17 12:36:51 -05:00
Taimoor ZaeemandGitHub 3c1a7f2641 feat: add timezone in Prefer header (#3024)
* increase reloading timeout in io-tests
* increase memory test by 1M
2023-11-13 14:16:27 -05:00
steve-chavez 82b38341bf feat: sql handlers for custom media types
* test text/html and drop HtmlRawOutputSpec.hs
* all tests passing, removed all pendingWith
* make functions compatible with pg <= 12
* move custom media types tests to own spec
* anyelement aggregate
* apply aggregates without a final function
* overriding works
* overriding anyelement with particular agg
* cannot override vendored media types
* plan spec works with custom aggregate
* renamed media types to make clear which ones are overridable
* correct content negotiation with same weight
* text/tab-separated-values media type
* text/csv with BOM plus content-disposition header
2023-10-26 09:04:02 -05:00
Laurence IslaandGitHub 910950dbac fix: RPCs not embedding correctly when using overloaded functions for computed relationships 2023-10-02 16:00:33 -05:00
Laurence IslaandGitHub 40c2bcd4a1 fix: paramaters of type character and bit not ignoring length
- Fixes the error "value too long for type character(1)" when the char length of the parameter was bigger than one.
2023-07-31 18:36:37 -05:00