diff --git a/doc/CI_PLUMBING.org b/doc/CI_PLUMBING.org index 89228e06c..767e0214b 100644 --- a/doc/CI_PLUMBING.org +++ b/doc/CI_PLUMBING.org @@ -13,7 +13,7 @@ - [[#workflows-groups-of-ci-jobs][Workflows (groups of CI jobs)]] - [[#pull-request-jobs][Pull request jobs]] - [[#emacs-lisp-tests][Emacs Lisp Tests]] - - [[#documentation-validation][Documentation validation]] + - [[#documentation-validation][Documentation validation]] - [[#pr-validation][PR validation]] - [[#branch-updates-runs-on-merge][Branch updates (runs on merge)]] - [[#emacs-lisp-tests-1][Emacs Lisp Tests]] @@ -89,67 +89,65 @@ elements. The code is compiled to [[https://www.graalvm.org/reference-manual/nat the JVM drawbacks are mitigated, for the particular use case anyway. * CI files and directories - - [[https://github.com/syl20bnr/spacemacs/tree/develop/.ci][.ci]] is a shared CI directory that holds two config files: - 1. [[https://github.com/syl20bnr/spacemacs/blob/develop/.ci/built_in_manifest][built_in_manifest]] list of upstream URL and target locations for - built-in files. - 2. [[https://github.com/syl20bnr/spacemacs/blob/develop/.ci/spacedoc-cfg.edn][spacedoc-cfg.edn]] configuration file for Spacemacs documentation tools. - More details in [[#documentation-updates][Documentation updates]]. - - [[https://github.com/syl20bnr/spacemacs/tree/develop/.github/workflows][.github/workflows]] place where GitHub workflow files live: - - =workflows/scripts/test= runner script for EmacsLisp tests. - - =workflows/scripts/dot_lock.el= package lock file that adds local ELPA - mirror. - - =elisp_test.yml= runs EmacsLisp tests on PR and branch updates. - - =rebase.yml= we don't really use it :) It rebases PR onto current HEAD, - it doesn't always work and requires personal token to run automatically. - - =stale.yml= manages stale issues and PR. - - [[https://github.com/syl20bnr/spacemacs/tree/develop/.circleci][.circleci]] everything specific for CircleCI. Documentation related files - stored in the =org= sub folder, =web= is where HTML export stuff hides and - =built_in= is all about updating built-in files. The rest is a bunch of - shared script files. The specific cases are =shared= file that loads before - each script run for every job, =config.yml= - CircleCI bootstrap script that - generates the config that CircleCI will run by populating =config_tmpl.yml= - file - actual config. We need the bootstrap step to inject =IS_BRANCH_UDATE= - environment variable value into generated config file that CircleCI will run - because it used to choose which jobs should be executed but environment - variables aren't loaded soon enough. +- [[https://github.com/syl20bnr/spacemacs/tree/develop/.ci][.ci]] is a shared CI directory that holds two config files: + 1. [[https://github.com/syl20bnr/spacemacs/blob/develop/.ci/built_in_manifest][built_in_manifest]] list of upstream URL and target locations for + built-in files. + 2. [[https://github.com/syl20bnr/spacemacs/blob/develop/.ci/spacedoc-cfg.edn][spacedoc-cfg.edn]] configuration file for Spacemacs documentation tools. + More details in [[#documentation-updates][Documentation updates]]. +- [[https://github.com/syl20bnr/spacemacs/tree/develop/.github/workflows][.github/workflows]] place where GitHub workflow files live: + - =workflows/scripts/test= runner script for EmacsLisp tests. + - =workflows/scripts/dot_lock.el= package lock file that adds local ELPA + mirror. + - =elisp_test.yml= runs EmacsLisp tests on PR and branch updates. + - =rebase.yml= we don't really use it :) It rebases PR onto current HEAD, + it doesn't always work and requires personal token to run automatically. + - =stale.yml= manages stale issues and PR. +- [[https://github.com/syl20bnr/spacemacs/tree/develop/.circleci][.circleci]] everything specific for CircleCI. Documentation related files + stored in the =org= sub folder, =web= is where HTML export stuff hides and + =built_in= is all about updating built-in files. The rest is a bunch of + shared script files. The specific cases are =shared= file that loads before + each script run for every job, =config.yml= - CircleCI bootstrap script that + generates the config that CircleCI will run by populating =config_tmpl.yml= + file - actual config. We need the bootstrap step to inject =IS_BRANCH_UDATE= + environment variable value into generated config file that CircleCI will run + because it used to choose which jobs should be executed but environment + variables aren't loaded soon enough. * Workflows (groups of CI jobs) ** Pull request jobs *** Emacs Lisp Tests - Code tests are handled by GitHub Actions exclusively. - The stages are: - 1. Emacs installation with [[https://github.com/purcell/setup-emacs][purcell/setup-emacs]] - for UNIX and - [[https://github.com/jcs090218/setup-emacs-windows][jcs090218/setup-emacs-windows]] for Windows. The step is configured - by the job matrix. With two keys =os= and =emacs_version=. CI runs test - for every possible combination. The stage ends up seriously bloated with - repetition since the actions sometimes fail (especially for MacOS) - so I added 3 sets of retires for the both actions. Currently GitHub - [[https://github.community/t/how-to-retry-a-failed-step-in-github-actions-workflow/125880][doesn't provide a better way to implement this]]. - 2. Checkout - clones the repo. - 3. Installation of a local ELPA mirror with packages used be the tests. - The archive is build daily in [[https://github.com/JAremko/testelpa-develop][JAremko/testelpa-develop]] repository and - configured by .spacemacs files used in test. The mirror is set as a top - priority package repository via [[https://github.com/syl20bnr/spacemacs/blob/develop/.github/workflows/scripts/dot_lock.el][Spacemacs lock file]] this way we actually - install the packages (it is important to test that the system works) and - if some packages are missing (for example, the mirror can be outdated) - then they will be installed from a remote repository. - 4. Run the tests! CI run core, base and layer tests sequentially because - heaving 20+ CI results in PRs makes people ignore them. And this way - they start faster since we cut on setup time. But the tests have to - =always= clean after themselves to avoid affecting the next ones. - -For more details see the [[https://github.com/syl20bnr/spacemacs/blob/develop/.github/workflows/elisp_test.yml][workflow]] file - +Code tests are handled by GitHub Actions exclusively. +The stages are: +1. Emacs installation with [[https://github.com/purcell/setup-emacs][purcell/setup-emacs]] - for UNIX and + [[https://github.com/jcs090218/setup-emacs-windows][jcs090218/setup-emacs-windows]] for Windows. The step is configured + by the job matrix. With two keys =os= and =emacs_version=. CI runs test + for every possible combination. The stage ends up seriously bloated with + repetition since the actions sometimes fail (especially for MacOS) + so I added 3 sets of retires for the both actions. Currently GitHub + [[https://github.community/t/how-to-retry-a-failed-step-in-github-actions-workflow/125880][doesn't provide a better way to implement this]]. +2. Checkout - clones the repo. +3. Installation of a local ELPA mirror with packages used be the tests. + The archive is build daily in [[https://github.com/JAremko/testelpa-develop][JAremko/testelpa-develop]] repository and + configured by .spacemacs files used in test. The mirror is set as a top + priority package repository via [[https://github.com/syl20bnr/spacemacs/blob/develop/.github/workflows/scripts/dot_lock.el][Spacemacs lock file]] this way we actually + install the packages (it is important to test that the system works) and + if some packages are missing (for example, the mirror can be outdated) + then they will be installed from a remote repository. +4. Run the tests! CI run core, base and layer tests sequentially because + heaving 20+ CI results in PRs makes people ignore them. And this way + they start faster since we cut on setup time. But the tests have to + =always= clean after themselves to avoid affecting the next ones. -*** Documentation validation +For more details see the [[https://github.com/syl20bnr/spacemacs/blob/develop/.github/workflows/elisp_test.yml][workflow]] file + +*** Documentation validation This job uses [[https://github.com/syl20bnr/spacemacs/blob/develop/.circleci/select_pr_changed][.circleci/select_pr_changed]] to find out what files are changed in the tested PR and if any of them are .org files it will check that they can be processed by exporting and validating them. The process will be explored further in the [[#documentation-updates][Documentation updates]] section. - *** PR validation -There are only two jobs here.[[https://github.com/syl20bnr/spacemacs/blob/develop/.circleci/PR_base][ .circleci/PR_base]] makes sure that the PR +There are only two jobs here.[[https://github.com/syl20bnr/spacemacs/blob/develop/.circleci/PR_base][.circleci/PR_base]] makes sure that the PR is against develop branch and [[https://github.com/syl20bnr/spacemacs/blob/develop/.circleci/PR_rebased][.circleci/PR_rebased]] checks if the PR needs a rebase (only when it's updated, so Spacemacs HEAD can actually get, well... Ahead). @@ -187,7 +185,7 @@ built-in files are updated every time Spacemacs develop branch is pushed. - Emacs Install retries can use some delay between the attempts since it is likely that a failed upstream repo will fail again if you don't give it any time to recover/change state. But it shouldn't add delay to runs without - failures since they vastly outnumber failed ones. + failures since they vastly outnumber failed ones. - See if we actually properly clean all they side effects between running EmacsLisp tests. - CircleCI script files can have better names. Also =.circleci= directory gets