guix/HACKING

-*- mode: org; coding: utf-8; -*-

#+TITLE: Hacking GNU Guix and Its Incredible Distro

Copyright © 2012, 2013 Ludovic Courtès <ludo@gnu.org>

  Copying and distribution of this file, with or without modification,
  are permitted in any medium without royalty provided the copyright
  notice and this notice are preserved.


* Running Guix before it is installed

Command-line tools can be used even if you have not run "make install".
To do that, prefix each command with ‘./pre-inst-env’, as in:

  ./pre-inst-env guix build --help

Similarly, for a Guile session using the Guix modules:

  ./pre-inst-env guile -c '(use-modules (guix utils)) (pk (%current-system))'

The ‘pre-inst-env’ script sets up all the environment variables
necessary to support this.

* The Perfect Setup

The Perfect Setup to hack on Guix is basically the perfect setup used
for Guile hacking (info "(guile) Using Guile in Emacs").  First, you
need more than an editor, you need [[http://www.gnu.org/software/emacs][Emacs]], empowered by the wonderful
[[http://nongnu.org/geiser/][Geiser]].

Geiser allows for interactive and incremental development from within
Emacs: code compilation and evaluation from within buffers, access to
on-line documentation (docstrings), context-sensitive completion, M-. to
jump to an object definition, a REPL to try out your code, and more.

To actually edit the code, Emacs already has a neat Scheme mode.  But in
addition to that, you must not miss [[http://www.emacswiki.org/emacs/ParEdit][Paredit]].  It provides facilities to
directly operate on the syntax tree, such as raising an s-expression or
wrapping it, swallowing or rejecting the following s-expression, etc.

* Packaging Guidelines

The GNU distribution is about respecting the freedom of users.  Consequently,
it contains only free software as defined at
http://www.gnu.org/philosophy/free-sw.html .

In addition, we follow the [[http://www.gnu.org/distros/free-system-distribution-guidelines.html][free software distribution guidelines]].  Among other
things, this means that the distribution tries hard not to steer users towards
obtaining information about non-free software.

* Adding new packages

Package recipes in Guix look like this:

#+BEGIN_SRC scheme
  (package
    (name "nettle")
    (version "2.5")
    (source
      (origin
        (method url-fetch)
        (uri (string-append "mirror://gnu/nettle/nettle-"
                            version ".tar.gz"))
        (sha256
          (base32
            "0wicr7amx01l03rm0pzgr1qvw3f9blaw17vjsy1301dh13ll58aa"))))
    (build-system gnu-build-system)
    (inputs `(("m4" ,m4)))
    (propagated-inputs `(("gmp" ,gmp)))
    (home-page
      "http://www.lysator.liu.se/~nisse/nettle/")
    (synopsis "GNU Nettle, a cryptographic library")
    (description
      "Nettle is a cryptographic library...")
    (license gpl2+))
#+END_SRC

Such a recipe can be written by hand, and then tested by running
‘./pre-inst-env guix build nettle’.

When writing the recipe, the base32-encoded SHA256 hash of the source
code tarball, which can be seen in the example above, can be obtained by
running:

  guix download http://ftp.gnu.org/gnu/nettle/nettle-2.5.tar.gz

Alternatively, it is possible to semi-automatically import recipes from
the [[http://nixos.org/nixpkgs/][Nixpkgs]] software distribution using this command:

  guix import /path/to/nixpkgs/checkout nettle

The command automatically fetches and converts to Guix the “Nix
expression” of Nettle.

* Submitting Patches

Development is done using the Git distributed version control system.  Thus,
access to the repository is not strictly necessary.  We welcome contributions
in the form of patches as produced by ‘git format-patch’ sent to
bug-guix@gnu.org.  Please write commit logs in the [[http://www.gnu.org/prep/standards/html_node/Change-Logs.html#Change-Logs][GNU ChangeLog format]].

As you become a regular contributor, you may find it convenient to have write
access to the repository (see below.)

* Commit Access

For frequent contributors, having write access to the repository is
convenient.  When you deem it necessary, feel free to ask for it on the
mailing list.  When you get commit access, please make sure to follow the
policy below (discussions of the policy can take place on bug-guix@gnu.org.)

Non-trivial patches should always be posted to bug-guix@gnu.org (trivial
patches include fixing typos, etc.)

For patches that just add a new package, and a simple one, it’s OK to commit,
if you’re confident (which means you successfully built it in a chroot setup.)
Likewise for package upgrades.  We have a mailing list for commit
notifications (guix-commits@gnu.org), so people can notice.  Before pushing
your changes, make sure to run ‘git pull --rebase’.

For anything else, please post to bug-guix@gnu.org and leave time for a
review, without committing anything.  If you didn’t receive any reply
after two weeks, and if you’re confident, it’s OK to commit.

That last part is subject to being adjusted, allowing individuals to commit
directly on non-controversial changes on parts they’re familiar with.

* Porting the Guix distro on a new platform

** Introduction

Unlike Make or similar build tools, Guix requires absolutely /all/ the
dependencies of a build process to be specified.

For a user-land software distribution, that means that the process that
builds GCC (then used to build all other programs) must itself be
specified; and the process to build the C library to build that GCC; and
the process to build the GCC to build that library; and...  See the
problem?  Chicken-and-egg.

To break that cycle, the distro starts from a set of pre-built
binaries–usually referred to as “bootstrap binaries.”  These include
statically-linked versions of Guile, GCC, Coreutils, Make, Grep, sed,
etc., and the GNU C Library.

This section describes how to build those bootstrap binaries when
porting to a new platform.

** When the platform is supported by Nixpkgs

In that case, the easiest thing is to bootstrap the distro using
binaries from Nixpkgs.

To do that, you need to comment out the definitions of
‘%bootstrap-guile’ and ‘%bootstrap-inputs’ in distro/packages/bootstrap.scm
to force the use of Nixpkgs derivations.  For instance, when porting to
‘i686-linux’, you should redefine these variables along these lines:

#+BEGIN_SRC scheme
  (define %bootstrap-guile
    (nixpkgs-derivation "guile" "i686-linux"))
  
  (define %bootstrap-inputs
    (compile-time-value
     `(("libc" ,(nixpkgs-derivation "glibc" "i686-linux"))
       ,@(map (lambda (name)
                (list name (nixpkgs-derivation name "i686-linux")))
              '("gnutar" "gzip" "bzip2" "xz" "patch"
                "coreutils" "gnused" "gnugrep" "bash"
                "gawk"                                ; used by `config.status'
                "gcc" "binutils")))))
#+END_SRC

That should allow the distro to be bootstrapped.

Then, the tarballs containing the initial binaries of Guile, Coreutils,
GCC, libc, etc. need to be built.  To that end, run the following
commands:

#+BEGIN_SRC sh
  ./pre-inst-env guix build -K                                    \
      -e '(@ (gnu packages make-bootstrap) %bootstrap-tarballs)'  \
      --system=i686-linux

#+END_SRC

These should build tarballs containing statically-linked tools usable on
that system.

In the source tree, you need to install binaries for ‘mkdir’, ‘bash’,
‘tar’, and ‘xz’ under ‘distro/packages/bootstrap/i686-linux’.  These
binaries can be extracted from the static-binaries tarball built above.

A rule for ‘distro/packages/bootstrap/i686-linux/guile-2.0.7.tar.xz’
needs to be added in ‘Makefile.am’, with the appropriate hexadecimal
vrepresentation of its SHA256 hash.

You may then revert your changes to ‘bootstrap.scm’.  For the variables
‘%bootstrap-coreutils&co’, ‘%bootstrap-binutils’, ‘%bootstrap-glibc’,
and ‘%bootstrap-gcc’, the expected SHA256 of the corresponding tarballs
for ‘i686-linux’ (built above) must be added.

This should be enough to bootstrap the distro without resorting to
Nixpkgs.

** When the platform is *not* supported by Nixpkgs

In that case, the bootstrap binaries should be built using whatever
tools are available on the target platform.  That is, the tarballs and
binaries show above must first be built manually, using the available
tools.

They should have the same properties as those built by the Guix recipes
shown above.  For example, all the binaries (except for glibc) must be
statically-linked; the bootstrap Guile must be relocatable (see patch in
the Guix distro); the static-binaries tarball must contain the same
programs (Coreutils, Grep, sed, Awk, etc.); and so on.