This reverts commit 840d3f99ed.
This was a nice idea in theory, but in practice this means that the
build jobs do not run at all on the default branch anymore. This means
the caches they push to won't be used by any other jobs, neither on the
tag pipelines, nor on PRs. This in turn makes all of these jobs really
slow.
Not helpful!