jimmy/postgrest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Nixify CircleCI setup (#1535)

Browse Source 
* package ghr
* add nix-based release scripts
* refactor CI based on nix tests and release scripts
* update stack version in circleci
* update makefile
* remove docker directory
* add Docker Hub description
* add README on docker
This commit is contained in:
Remo Rechkemmer
2020-05-25 11:27:35 -05:00
committed by GitHub
parent d4aba5cb08
commit 08186ea51c
20 changed files with 426 additions and 484 deletions
Download Patch File Download Diff File Expand all files Collapse all files
nix/docker/README.md
+101
View File
@@ -0,0 +1,101 @@
# Docker image built with Nix
In order to build an optimal PostgREST Docker image, we create the image from
scratch (i.e., without a parent image like `debian` or `alpine`), and only
include the files that are essential for running PostgREST (the static
PostgREST binary and a `postgrest.conf`).
This is similar to what you would get with the following `Dockerfile`:
```Dockerfile
# `scratch` is a minimal, reserved image in Docker, see
# https://docs.docker.com/develop/develop-images/baseimages/ . It essentially
# means "don't use a parent image and start with an empty one".
FROM scratch
# The static PostgREST executable has no runtime dependencies, so it's all we
# need to include for running the application.
ADD /absolute/path/to/postgrest /bin/postgrest
# Include a default configuration file.
ADD /absolute/path/to/postgrest.conf /etc/postgrest.conf
ENV PGRST_DB_URI= \
...
EXPOSE 3000
# This is the user id that Docker will run our image under by default. Note
# that we don't actually add the user to `/etc/passwd` or `/etc/shadow`. This
# means that tools like whoami would not work properly, but we don't include
# those in the image anyway. Not adding the user has the benefit that the image
# can be run under any user you specify.
USER 1000
CMD [ "/bin/postgrest", "/etc/postgrest.conf" ]
```
# Building the Docker image with Nix
As we are building the static PostgREST executable with Nix and that's the main
input to the Docker file, we can also create the Docker image directly with Nix
using the [`dockerTools`
utilities](https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools). Those
utilities don't actually use `Dockerfiles` or Docker to build Docker images,
but create them directly by putting together the required `json` and `tar`
files that make up an image. This is more efficient, does not rely on Docker or
root permissions and results in fully reproducible builds. See
[`nix/docker/default.nix`](./default.nix) for details how the image is built.
# Building and loading the image
The Nix expression provides a helper script `postgrest-docker-load` that loads
the optimized image into your local Docker instance (using `docker load -i
<image file>` under the hood). You can use it by running:
```
# Running from the root directory of the repository:
# Build the `docker` attribute from `default.nix`, the result will be symlinked
# to `result`:
nix-build -A docker
# Run the loading script:
result/bin/postgrest-docker-load
```
The Docker image built with Nix always has the name "postgrest:latest" when
loaded.
# Inspecting the optimized image
The image does not come with the usual utilities like `bash` and `ls`.
You can, however, explore the `tar` file of the image by saving it with `docker
save postgrest:latest > image.tar`.
[Dive](https://github.com/wagoodman/dive) is also useful for looking at the
contents of the image:
```
┃ ● Layers ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ Current Layer Contents ├────────────────────────────────────────────────────────────────────────────────
Cmp Size Command Permission UID:GID Size Filetree
14 MB FROM 20ee65c811575d2 dr-xr-xr-x 0:0 14 MB ├── bin
-r-xr-xr-x 0:0 14 MB │ └── postgrest
│ Layer Details ├───────────────────────────────────────────────────────────────────────────────────────── drwxr-xr-x 0:0 783 B ├── etc
-r--r--r-- 0:0 783 B │ └── postgrest.conf
Tags: (unavailable) dr-xr-xr-x 0:0 23 kB └── nix
Id: 20ee65c811575d206eb673e1887e7f7e6b7ccde902a63ccb924c5faa50b32cee dr-xr-xr-x 0:0 23 kB └── store
Digest: sha256:ece77302b83fd38fb54395dabc10c2eba06fc1d1933801d36cc2c4732d9c8f38 dr-xr-xr-x 0:0 23 kB └── s440jbrn94wmpzy7f8yfsp6jr2shllw5-openssl-1.1.1g-etc
Command: dr-xr-xr-x 0:0 23 kB └── etc
dr-xr-xr-x 0:0 23 kB └── ssl
-r--r--r-- 0:0 412 B ├── ct_log_list.cnf
│ Image Details ├───────────────────────────────────────────────────────────────────────────────────────── -r--r--r-- 0:0 412 B ├── ct_log_list.cnf.dist
dr-xr-xr-x 0:0 0 B ├── engines-1.1
-r--r--r-- 0:0 11 kB ├── openssl.cnf
Total Image size: 14 MB -r--r--r-- 0:0 11 kB └── openssl.cnf.dist
Potential wasted space: 0 B
Image efficiency score: 100 %
Count Total Space Path
```
nix/docker/default.nix
+17 -19
View File
@@ -1,11 +1,12 @@
{ postgrest, dockerTools, writeShellScriptBin }:
{ buildEnv, postgrest, dockerTools, writeShellScriptBin }:
let
image =
tag:
dockerTools.buildImage {
inherit tag;
config =
./postgrest.conf;
name = "postgrest/postgrest";
image =
dockerTools.buildImage {
name = "postgrest";
tag = "latest";
contents = postgrest;
# Set the current time as the image creation date. This makes the build
@@ -15,7 +16,8 @@ let
extraCommands =
''
mkdir etc
cp ${./postgrest.conf} etc/postgrest.conf
cp ${config} etc/postgrest.conf
rmdir share
'';
config = {
@@ -25,6 +27,7 @@ let
"PGRST_DB_SCHEMA=public"
"PGRST_DB_ANON_ROLE="
"PGRST_DB_POOL=100"
"PGRST_DB_POOL_TIMEOUT=10"
"PGRST_DB_EXTRA_SEARCH_PATH=public"
"PGRST_SERVER_HOST=*4"
"PGRST_SERVER_PORT=3000"
@@ -44,20 +47,15 @@ let
};
};
};
in
rec {
imageLatest =
image "latest";
imageWithVersion =
image "v${postgrest.version}";
# Helper script for loading the image.
load =
writeShellScriptBin "postgrest-docker-load"
''
set -euo pipefail
docker load -i ${imageLatest}
docker load -i ${imageWithVersion}
docker load -i ${image}
'';
}
in
buildEnv {
name = "postgrest-docker";
paths = [ load ];
} // { inherit image config; }
nix/docker/postgrest.conf
+8
View File
@@ -1,7 +1,15 @@
# See https://postgrest.org/en/stable/configuration.html#configuration
# Required settings
db-uri = "$(PGRST_DB_URI)"
db-schema = "$(PGRST_DB_SCHEMA)"
db-anon-role = "$(PGRST_DB_ANON_ROLE)"
# Optional settings
db-pool = "$(PGRST_DB_POOL)"
db-pool-timeout = "$(PGRST_DB_POOL_TIMEOUT)"
db-extra-search-path = "$(PGRST_DB_EXTRA_SEARCH_PATH)"
server-host = "$(PGRST_SERVER_HOST)"
nix/overlays/default.nix
+1
View File
@@ -1,5 +1,6 @@
{
gitignore = import ./gitignore.nix;
ghr = import ./ghr;
haskell-packages = import ./haskell-packages;
postgresql-default = import ./postgresql-default.nix;
postgresql-legacy = import ./postgresql-legacy.nix;
nix/overlays/ghr/default.nix
+7
View File
@@ -0,0 +1,7 @@
self: super:
# Overlay that adds `ghr`: Upload multiple artifacts to GitHub Release in
# parallel, http://tcnksm.github.io/ghr/
{
ghr = super.callPackage ./ghr.nix { };
}
nix/overlays/ghr/ghr.nix
+15
View File
@@ -0,0 +1,15 @@
{ buildGoModule, fetchFromGitHub }:
buildGoModule rec {
pname = "ghr";
version = "0.13.0";
src = fetchFromGitHub {
rev = "v${version}";
owner = "tcnksm";
repo = "ghr";
sha256 = "1nm5kdjkqayxh06j9nr5daic9sw9nx9w06y9gaqhdrw9byvjpr1a";
};
vendorSha256 = "14avsngzhl1b8a05i43ph6sxh9vj0jls0acxr9j7r0h3f0vpamcj";
}
nix/release/default.nix
+101
View File
@@ -0,0 +1,101 @@
{ buildEnv
, curl
, envsubst
, ghr
, jq
, postgrest
, docker
, runCommand
, writeShellScriptBin
}:
let
# Version from the postgrest.cabal file (gotten with callCabal2nix).
version = postgrest.version;
# Set of files that will be published in the GitHub release.
releaseFiles =
runCommand "postgrest-release-files"
{ inherit postgrest version; }
''
set -euo pipefail
mkdir -p $out
tar cvJf "$out"/postgrest-v"$version"-linux-x64-static.tar.xz \
-C "$postgrest"/bin postgrest
'';
# Script for publishing a new release on GitHub.
github =
writeShellScriptBin "postgrest-release-github"
''
set -euo pipefail
changes="$(sed -n "1,/${version}/d;/## \[/q;p" ${../../CHANGELOG.md})"
${ghr}/bin/ghr \
-t "$GITHUB_TOKEN" \
-u "$GITHUB_USERNAME" \
-r "$GITHUB_REPONAME" \
-b "$changes" \
--replace v${version} \
${releaseFiles}
'';
# Script for publishing a new release on Docker Hub.
dockerHub =
writeShellScriptBin "postgrest-release-dockerhub"
''
set -euo pipefail
docker load -i ${docker.image}
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:latest
docker tag postgrest:latest "$DOCKER_REPO"/postgrest:v${version}
docker push "$DOCKER_REPO"/postgrest:latest
docker push "$DOCKER_REPO"/postgrest:v${version}
'';
# Script for updating the repository description on Docker Hub.
dockerHubDescription =
let
description =
./docker-hub-description.md;
fullDescription =
./docker-hub-full-description.md;
in
writeShellScriptBin "postgrest-release-dockerhubdescription"
''
set -euo pipefail
# Login to Docker Hub and get a token.
token="$(
${curl}/bin/curl -sH "Content-Type: application/json" \
--data-urlencode "username=$DOCKER_USERNAME" \
--data-urlencode "password=$DOCKER_PASSWORD" \
"https://hub.docker.com/v2/users/login/" \
| ${jq}/bin/jq -r .token
)"
# Plug the default config file into the full description.
export DEFAULT_CONFIG="$(cat ${docker.config})"
fullDescription="$(${envsubst}/bin/envsubst < ${fullDescription})"
# Patch the full description.
responseCode="$(
${curl}/bin/curl -s --write-out %{response_code} --output /dev/null \
-H "Authorization: JWT $token" -X PATCH \
--data-urlencode description@${description} \
--data-urlencode "full_description=$fullDescription" \
"https://hub.docker.com/v2/repositories/$DOCKER_REPO/postgrest/"
)"
[ "$responseCode" -eq 200 ]
'';
in
buildEnv {
name = "postgrest-release";
paths = [ github dockerHub dockerHubDescription ];
}
nix/release/docker-hub-description.md
+1
View File
@@ -0,0 +1 @@
REST API for any Postgres database
nix/release/docker-hub-full-description.md
+33
View File
@@ -0,0 +1,33 @@
# PostgREST
[![Join the chat at https://gitter.im/begriffs/postgrest](https://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/begriffs/postgrest)
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg?colorB=F96854)](https://www.patreon.com/postgrest)
[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.me/postgrest)
[![Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://postgrest.org)
[![Build Status](https://circleci.com/gh/PostgREST/postgrest/tree/master.svg?style=shield)](https://circleci.com/gh/PostgREST/postgrest/tree/master)
PostgREST serves a fully RESTful API from any existing PostgreSQL database. It
provides a cleaner, more standards-compliant, faster API than you are likely to
write from scratch.
# Usage
To learn how to use this container, see the [PostgREST Docker
documentation](https://postgrest.com/en/stable/install.html#docker).
You can configure the PostgREST image by setting the enviroment variables used
in the default `/etc/postgrest.conf` file or overriding that file. This is the
default configuration file:
```
$DEFAULT_CONFIG
```
# How this image is built
The image is built from scratch using
[Nix](https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools) instead of a
`Dockerfile`, which yields a higly secure and optimized image. This is also why
no commands are listed in the image history. See the [PostgREST
respository](https://github.com/PostgREST/postgrest/tree/master/nix/docker) for
details on the build process and how to inspect the image.
nix/tests.nix
+2 -2
View File
@@ -33,14 +33,14 @@ let
''
set -euo pipefail
export PATH="$(cat ${postgrestBuildEnv})"/bin:"$PATH"
cat << EOF
Running spec against ${postgresql.name}...
EOF
# TODO: Make this work outside nix-shell when installed with nix-env.
# Probably using postgrestBuildEnv somehow?
${withTmpDb postgresql} ${cabal-install}/bin/cabal v2-test \
--test-show-detail=direct