139 lines
6.6 KiB
Org Mode
139 lines
6.6 KiB
Org Mode
#+TITLE: Continuous Integration
|
||
|
||
* Table of Contents :TOC_5_gh:noexport:
|
||
- [[#description][Description]]
|
||
- [[#overview][Overview]]
|
||
- [[#tldr][TLDR]]
|
||
- [[#current-stack][Current stack:]]
|
||
- [[#circleci][CircleCI]]
|
||
- [[#github-actions][GitHub Actions]]
|
||
- [[#docker][Docker]]
|
||
- [[#clojure][Clojure]]
|
||
- [[#workflows-groups-of-ci-jobs][Workflows (groups of CI jobs)]]
|
||
- [[#pull-request-jobs][Pull request jobs]]
|
||
- [[#emacs-lisp-tests][Emacs Lisp Tests]]
|
||
- [[#documentation-validation][Documentation validation]]
|
||
- [[#pr-validation][PR validation]]
|
||
- [[#branch-updates-runs-on-merge][Branch updates (runs on merge)]]
|
||
- [[#emacs-lisp-tests-1][Emacs Lisp Tests]]
|
||
- [[#project-files-updates][Project files updates]]
|
||
- [[#general-concepts][General concepts]]
|
||
- [[#built-in-updates][Built-in updates]]
|
||
- [[#documentation-updates][Documentation updates]]
|
||
- [[#web-site-updates][Web site updates]]
|
||
- [[#scheduled-jobs][Scheduled jobs:]]
|
||
- [[#general-concepts-1][General concepts]]
|
||
- [[#built-in-files-updates][Built-in files updates]]
|
||
- [[#potential-improvements-pr-ideas][Potential improvements (PR ideas)]]
|
||
- [[#side-notes][Side notes]]
|
||
- [[#we-used-to-have-travisci-3-ci-providers-at-the-same-time][We used to have TravisCI (3 CI providers at the same time)]]
|
||
- [[#circleci-setup-config-and-cron-jobs][CircleCI setup config and cron jobs]]
|
||
|
||
* Description
|
||
This file explains how our continuous integration operates and what problems
|
||
it solves. =It is works in progress.=
|
||
|
||
* Overview
|
||
** TLDR
|
||
Spacemacs is big - the active maintainers team is small. The more we can
|
||
automate - the better. We use [[https://circleci.com/][CircleCI]], [[https://github.com/features/actions][GitHub Actions]] and [[https://www.docker.com/][Docker]] to test PRs,
|
||
update/fixi documentation and exporting it to [[https://develop.spacemacs.org/]].
|
||
|
||
Most of the code is just a bunch of bash/ELisp scripts and yml files, but
|
||
some of the documentation tools are written in Clojure.
|
||
Check out [[https://github.com/syl20bnr/spacemacs/tree/develop/.circleci][CircleCI]] and [[https://github.com/syl20bnr/spacemacs/tree/develop/.github/workflows][GitHub Actions]] directories, the code is pretty much
|
||
self-explanatory but will be examined in depth below.
|
||
|
||
** Current stack:
|
||
Wait, what? Why Clojure, why 2 CI providers?
|
||
I knew you would ask this question, dear reader, so here's my rationale:
|
||
|
||
*** CircleCI
|
||
It has a cool set of features and a generous quota for open source projects.
|
||
But most importantly, unlike GitHub Actions, there is a straight forward way
|
||
to cache build dependencies between runs and using it in tandem with
|
||
GH Actions provides us with even more concurrency. It means that PR authors
|
||
have to wait less time for feed back. This is crucial since we have a lot of
|
||
test and platforms to cover. Also, CircleCI can run jobs on user provided Docker
|
||
images that it caches, so we do not hit the DockerHub pull quota.
|
||
On the downside, the CircleCI configuration file can be pretty involved,
|
||
has unexpected limitations that can leave you puzzled for quite a while.
|
||
|
||
*** GitHub Actions
|
||
Oh man, that's good. It is clear that GH team had the benefit of hindsight
|
||
when developed their CI platform. And it runs rely fast (at least for now).
|
||
Maybe one day we'll fully switch to Actions. The biggest concern here is
|
||
the vendor lock-in since all the good stuff is highly specific. While CircleCI
|
||
allows you to run a job locally for free. And run whole CI on your own
|
||
hardware with "strings attached".
|
||
|
||
*** Docker
|
||
Having a stable pre-build environment for jobs reduces headaches and
|
||
improves set up time. Duh!
|
||
Also DockerHub used to be a cool place to store and build huge images for
|
||
free, but now it has all sorts of quotas + RAM is pretty limited for memory
|
||
hungry JVM builds (((foreshadowing))).
|
||
|
||
*** Clojure
|
||
Besides the obvious fact that Rich Hickey's talks are the best.
|
||
Before we started with automation, Spacemacs already had a huge set of
|
||
documentation files that couldn't be fixed by a bunch of regular expressions
|
||
wrapped into bash/ELisp code.
|
||
The options were to either fix all README.org files by hand and keep fixing
|
||
them forever, since contributors often forget to format stuff properly and
|
||
nagging them constantly both wastes PR reviewer time and makes the
|
||
contributor less likely to stick. Or go all-in and create a system that
|
||
can extract data out of documentation files and rebuild them from scratch.
|
||
Clojure designed to push data around and it has specs that can be used
|
||
to validate files, generate test data and constructors for org-mode
|
||
elements. The code is compiled to [[https://www.graalvm.org/reference-manual/native-image/][native-image]] so pretty much all of
|
||
the JVM drawbacks are mitigated, for the particular use case anyway.
|
||
|
||
* Workflows (groups of CI jobs)
|
||
** Pull request jobs
|
||
*** TODO Emacs Lisp Tests
|
||
|
||
*** TODO Documentation validation
|
||
|
||
*** TODO PR validation
|
||
|
||
** Branch updates (runs on merge)
|
||
*** TODO Emacs Lisp Tests
|
||
|
||
*** TODO Project files updates
|
||
**** TODO General concepts
|
||
|
||
**** TODO Built-in updates
|
||
|
||
**** TODO Documentation updates
|
||
|
||
**** TODO Web site updates
|
||
|
||
** Scheduled jobs:
|
||
*** TODO General concepts
|
||
|
||
*** TODO Built-in files updates
|
||
|
||
* Potential improvements (PR ideas)
|
||
- CircleCI config generation stage can test if a PR changes any .org file
|
||
and schedule documentation testing job only if it does.
|
||
- PR validation job can be moved to CircleCI config generation stage. If
|
||
it isn't valid all CircleCI jobs can be skipped.
|
||
|
||
* Side notes
|
||
** We used to have TravisCI (3 CI providers at the same time)
|
||
We ran long running jobs there but ended up dropping the CI since TravisCI
|
||
doesn't allow collaborators to read/set environment variables anymore,
|
||
[[https://pbs.twimg.com/media/Eoq3OnWW4AIy7ih?format=jpg&name=large][the could be in some kind of trouble]] or [[https://blog.travis-ci.com/oss-announcement][maybe not]]. Anyway, when TravisCI
|
||
stopped running jobs on their old domain (as a part of the migration from
|
||
[[https://travis-ci.org/]] to [[https://www.travis-ci.com/]]) I decided to use it
|
||
as an opportunity to have fewer kinds of configs. Still, it's good environment
|
||
for building heavy (both in build time and RAM) Docker images.
|
||
|
||
** CircleCI setup config and cron jobs
|
||
- Currently configs with setup step [[https://discuss.circleci.com/t/setup-workflow-and-scheduled-workflow-in-the-same-configuration/39932/6][don't run cron jobs]].
|
||
- We have setup config because environment variables aren't accessible at the
|
||
top level of config files. But we need =IS_BRANCH_UDATE= environment variable
|
||
to figure out if CI runs on PR or branch update. So config generation step
|
||
bakes it into the config that CircleCI will use.
|