Automated via C-c C-u C-a with Emacs. Change-Id: I9e595430529f8f994988c1ac3c919bdaef930dc1
5299 lines
184 KiB
Plaintext
5299 lines
184 KiB
Plaintext
\input texinfo
|
||
@c -*-texinfo-*-
|
||
|
||
@c %**start of header
|
||
@setfilename guix-cookbook.info
|
||
@documentencoding UTF-8
|
||
@settitle GNU Guix Cookbook
|
||
@c %**end of header
|
||
|
||
@c Onion service for ci.guix.gnu.org.
|
||
@set SUBSTITUTE-TOR-URL https://4zwzi66wwdaalbhgnix55ea3ab4pvvw66ll2ow53kjub6se4q2bclcyd.onion
|
||
|
||
@copying
|
||
Copyright @copyright{} 2019, 2022 Ricardo Wurmus@*
|
||
Copyright @copyright{} 2019 Efraim Flashner@*
|
||
Copyright @copyright{} 2019 Pierre Neidhardt@*
|
||
Copyright @copyright{} 2020 Oleg Pykhalov@*
|
||
Copyright @copyright{} 2020 Matthew Brooks@*
|
||
Copyright @copyright{} 2020 Marcin Karpezo@*
|
||
Copyright @copyright{} 2020 Brice Waegeneire@*
|
||
Copyright @copyright{} 2020 André Batista@*
|
||
Copyright @copyright{} 2020 Christine Lemmer-Webber@*
|
||
Copyright @copyright{} 2021 Joshua Branson@*
|
||
Copyright @copyright{} 2022, 2023 Maxim Cournoyer@*
|
||
Copyright @copyright{} 2023 Ludovic Courtès@*
|
||
Copyright @copyright{} 2023 Thomas Ieong
|
||
|
||
Permission is granted to copy, distribute and/or modify this document
|
||
under the terms of the GNU Free Documentation License, Version 1.3 or
|
||
any later version published by the Free Software Foundation; with no
|
||
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
|
||
copy of the license is included in the section entitled ``GNU Free
|
||
Documentation License''.
|
||
@end copying
|
||
|
||
@dircategory System administration
|
||
@direntry
|
||
* Guix cookbook: (guix-cookbook). Tutorials and examples for GNU Guix.
|
||
@end direntry
|
||
|
||
@titlepage
|
||
@title GNU Guix Cookbook
|
||
@subtitle Tutorials and examples for using the GNU Guix Functional Package Manager
|
||
@author The GNU Guix Developers
|
||
|
||
@page
|
||
@vskip 0pt plus 1filll
|
||
|
||
@insertcopying
|
||
@end titlepage
|
||
|
||
@contents
|
||
|
||
@c *********************************************************************
|
||
@node Top
|
||
@top GNU Guix Cookbook
|
||
|
||
This document presents tutorials and detailed examples for GNU@tie{}Guix, a
|
||
functional package management tool written for the GNU system. Please
|
||
@pxref{Top,,, guix, GNU Guix reference manual} for details about the system,
|
||
its API, and related concepts.
|
||
|
||
@c TRANSLATORS: You can replace the following paragraph with information on
|
||
@c how to join your own translation team and how to report issues with the
|
||
@c translation.
|
||
This manual is also available in French (@pxref{Top,,, guix-cookbook.fr,
|
||
Livre de recettes de GNU Guix}), German (@pxref{Top,,,
|
||
guix-cookbook.de, GNU-Guix-Kochbuch}) and Slovak (@pxref{Top,,,
|
||
guix-cookbook.sk, Receptár GNU Guix}). If you would like to translate
|
||
this document in your native language, consider joining
|
||
@uref{https://translate.fedoraproject.org/projects/guix/documentation-cookbook,
|
||
Weblate} (@pxref{Translating Guix,,, guix, GNU Guix reference
|
||
manual}).
|
||
|
||
@menu
|
||
* Scheme tutorials:: Meet your new favorite language!
|
||
* Packaging:: Packaging tutorials
|
||
* System Configuration:: Customizing the GNU System
|
||
* Containers:: Isolated environments and nested systems
|
||
* Advanced package management:: Power to the users!
|
||
* Software Development:: Environments, continuous integration, etc.
|
||
* Environment management:: Control environment
|
||
* Installing Guix on a Cluster:: High-performance computing.
|
||
|
||
* Acknowledgments:: Thanks!
|
||
* GNU Free Documentation License:: The license of this document.
|
||
* Concept Index:: Concepts.
|
||
|
||
@detailmenu
|
||
--- The Detailed Node Listing ---
|
||
|
||
Scheme tutorials
|
||
|
||
* A Scheme Crash Course::
|
||
|
||
Packaging
|
||
|
||
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
|
||
|
||
Packaging Tutorial
|
||
|
||
* A ``Hello World'' package::
|
||
* Setup::
|
||
* Extended example::
|
||
* Other build systems::
|
||
* Programmable and automated package definition::
|
||
* Getting help::
|
||
* Conclusion::
|
||
* References::
|
||
|
||
Setup
|
||
|
||
* Local file::
|
||
* Channels::
|
||
* Direct checkout hacking::
|
||
|
||
Programmable and automated package definition
|
||
|
||
* Recursive importers::
|
||
* Automatic update::
|
||
* Inheritance::
|
||
|
||
System Configuration
|
||
|
||
* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
|
||
* Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
|
||
* Guix System Image API:: Customizing images to target specific platforms.
|
||
* Using security keys:: How to use security keys with Guix System.
|
||
* Dynamic DNS mcron job:: Job to update the IP address behind a DuckDNS host name.
|
||
* Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
|
||
* Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
|
||
* Running Guix on a Linode Server:: Running Guix on a Linode Server.
|
||
* Running Guix on a Kimsufi Server:: Running Guix on a Kimsufi Server.
|
||
* Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
|
||
* Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
|
||
* Setting up NGINX with Lua:: Configuring NGINX web-server to load Lua modules.
|
||
* Music Server with Bluetooth Audio:: Headless music player with Bluetooth output.
|
||
|
||
Customizing a Window Manager
|
||
|
||
* StumpWM::
|
||
* Session lock::
|
||
|
||
Session lock
|
||
|
||
* Xorg::
|
||
|
||
Containers
|
||
|
||
* Guix Containers:: Perfectly isolated environments
|
||
* Guix System Containers:: A system inside your system
|
||
|
||
Guix System Containers
|
||
|
||
* A Database Container::
|
||
* Container Networking::
|
||
|
||
Advanced package management
|
||
|
||
* Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
|
||
|
||
Guix Profiles in Practice
|
||
|
||
* Basic setup with manifests::
|
||
* Required packages::
|
||
* Default profile::
|
||
* The benefits of manifests::
|
||
* Reproducible profiles::
|
||
|
||
Software Development
|
||
|
||
* Getting Started:: Step 0: using `guix shell'.
|
||
* Building with Guix:: Step 1: building your code.
|
||
* The Repository as a Channel:: Step 2: turning the repo in a channel.
|
||
* Package Variants:: Bonus: Defining variants.
|
||
* Setting Up Continuous Integration:: Step 3: continuous integration.
|
||
* Build Manifest:: Bonus: Manifest.
|
||
* Wrapping Up:: Recap.
|
||
|
||
Environment management
|
||
|
||
* Guix environment via direnv:: Setup Guix environment with direnv
|
||
|
||
Installing Guix on a Cluster
|
||
|
||
* Setting Up a Head Node:: The node that runs the daemon.
|
||
* Setting Up Compute Nodes:: Client nodes.
|
||
* Cluster Network Access:: Dealing with network access restrictions.
|
||
* Cluster Disk Usage:: Disk usage considerations.
|
||
* Cluster Security Considerations:: Keeping the cluster secure.
|
||
|
||
@end detailmenu
|
||
@end menu
|
||
|
||
@c *********************************************************************
|
||
@node Scheme tutorials
|
||
@chapter Scheme tutorials
|
||
|
||
GNU@tie{}Guix is written in the general purpose programming language Scheme,
|
||
and many of its features can be accessed and manipulated programmatically.
|
||
You can use Scheme to generate package definitions, to modify them, to build
|
||
them, to deploy whole operating systems, etc.
|
||
|
||
Knowing the basics of how to program in Scheme will unlock many of the
|
||
advanced features Guix provides --- and you don't even need to be an
|
||
experienced programmer to use them!
|
||
|
||
Let's get started!
|
||
|
||
@menu
|
||
* A Scheme Crash Course::
|
||
@end menu
|
||
|
||
@node A Scheme Crash Course
|
||
@section A Scheme Crash Course
|
||
|
||
@cindex Scheme, crash course
|
||
|
||
Guix uses the Guile implementation of Scheme. To start playing with the
|
||
language, install it with @code{guix install guile} and start a
|
||
@dfn{REPL}---short for @uref{https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop,
|
||
@dfn{read-eval-print loop}}---by running @code{guile} from the command line.
|
||
|
||
Alternatively you can also run @code{guix shell guile -- guile}
|
||
if you'd rather not have Guile installed in your user profile.
|
||
|
||
In the following examples, lines show what you would type at the REPL;
|
||
lines starting with ``@result{}'' show evaluation results, while lines
|
||
starting with ``@print{}'' show things that get printed. @xref{Using Guile
|
||
Interactively,,, guile, GNU Guile Reference Manual}, for more details on the
|
||
REPL.
|
||
|
||
@itemize
|
||
@item
|
||
Scheme syntax boils down to a tree of expressions (or @emph{s-expression} in
|
||
Lisp lingo). An expression can be a literal such as numbers and strings, or a
|
||
compound which is a parenthesized list of compounds and literals. @code{#true}
|
||
and @code{#false} (abbreviated @code{#t} and @code{#f}) stand for the
|
||
Booleans ``true'' and ``false'', respectively.
|
||
|
||
Examples of valid expressions:
|
||
|
||
@lisp
|
||
"Hello World!"
|
||
@result{} "Hello World!"
|
||
|
||
17
|
||
@result{} 17
|
||
|
||
(display (string-append "Hello " "Guix" "\n"))
|
||
@print{} Hello Guix!
|
||
@result{} #<unspecified>
|
||
@end lisp
|
||
|
||
@item
|
||
This last example is a function call nested in another function call. When a
|
||
parenthesized expression is evaluated, the first term is the function and the
|
||
rest are the arguments passed to the function. Every function returns the
|
||
last evaluated expression as its return value.
|
||
|
||
@item
|
||
Anonymous functions---@dfn{procedures} in Scheme parlance---are declared
|
||
with the @code{lambda} term:
|
||
|
||
@lisp
|
||
(lambda (x) (* x x))
|
||
@result{} #<procedure 120e348 at <unknown port>:24:0 (x)>
|
||
@end lisp
|
||
|
||
The above procedure returns the square of its argument. Since everything is
|
||
an expression, the @code{lambda} expression returns an anonymous procedure,
|
||
which can in turn be applied to an argument:
|
||
|
||
@lisp
|
||
((lambda (x) (* x x)) 3)
|
||
@result{} 9
|
||
@end lisp
|
||
|
||
Procedures are regular values just like numbers, strings, Booleans, and
|
||
so on.
|
||
|
||
@item
|
||
Anything can be assigned a global name with @code{define}:
|
||
|
||
@lisp
|
||
(define a 3)
|
||
(define square (lambda (x) (* x x)))
|
||
(square a)
|
||
@result{} 9
|
||
@end lisp
|
||
|
||
@item
|
||
Procedures can be defined more concisely with the following syntax:
|
||
|
||
@lisp
|
||
(define (square x) (* x x))
|
||
@end lisp
|
||
|
||
@item
|
||
A list structure can be created with the @code{list} procedure:
|
||
|
||
@lisp
|
||
(list 2 a 5 7)
|
||
@result{} (2 3 5 7)
|
||
@end lisp
|
||
|
||
@item
|
||
Standard procedures are provided by the @code{(srfi srfi-1)} module to
|
||
create and process lists (@pxref{SRFI-1, list processing,, guile, GNU
|
||
Guile Reference Manual}). Here are some of the most useful ones in
|
||
action:
|
||
|
||
@lisp
|
||
(use-modules (srfi srfi-1)) ;import list processing procedures
|
||
|
||
(append (list 1 2) (list 3 4))
|
||
@result{} (1 2 3 4)
|
||
|
||
(map (lambda (x) (* x x)) (list 1 2 3 4))
|
||
@result{} (1 4 9 16)
|
||
|
||
(delete 3 (list 1 2 3 4)) @result{} (1 2 4)
|
||
(filter odd? (list 1 2 3 4)) @result{} (1 3)
|
||
(remove even? (list 1 2 3 4)) @result{} (1 3)
|
||
(find number? (list "a" 42 "b")) @result{} 42
|
||
@end lisp
|
||
|
||
Notice how the first argument to @code{map}, @code{filter},
|
||
@code{remove}, and @code{find} is a procedure!
|
||
|
||
@item
|
||
@cindex S-expression
|
||
The @dfn{quote} disables evaluation of a parenthesized expression, also
|
||
called an S-expression or ``s-exp'': the first term is not called over
|
||
the other terms (@pxref{Expression Syntax, quote,, guile, GNU Guile
|
||
Reference Manual}). Thus it effectively returns a list of terms.
|
||
|
||
@lisp
|
||
'(display (string-append "Hello " "Guix" "\n"))
|
||
@result{} (display (string-append "Hello " "Guix" "\n"))
|
||
|
||
'(2 a 5 7)
|
||
@result{} (2 a 5 7)
|
||
@end lisp
|
||
|
||
@item
|
||
The @code{quasiquote} (@code{`}, a backquote) disables evaluation of a
|
||
parenthesized expression until @code{unquote} (@code{,}, a comma)
|
||
re-enables it. Thus it provides us with fine-grained control over what
|
||
is evaluated and what is not.
|
||
|
||
@lisp
|
||
`(2 a 5 7 (2 ,a 5 ,(+ a 4)))
|
||
@result{} (2 a 5 7 (2 3 5 7))
|
||
@end lisp
|
||
|
||
Note that the above result is a list of mixed elements: numbers, symbols (here
|
||
@code{a}) and the last element is a list itself.
|
||
|
||
@item
|
||
@cindex G-expressions, syntax
|
||
@cindex gexps, syntax
|
||
@findex #~
|
||
@findex #$
|
||
@findex gexp
|
||
@findex ungexp
|
||
Guix defines a variant of S-expressions on steroids called
|
||
@dfn{G-expressions} or ``gexps'', which come with a variant of
|
||
@code{quasiquote} and @code{unquote}: @code{#~} (or @code{gexp}) and
|
||
@code{#$} (or @code{ungexp}). They let you @emph{stage code for later
|
||
execution}.
|
||
|
||
For example, you'll encounter gexps in some package definitions where
|
||
they provide code to be executed during the package build process. They
|
||
look like this:
|
||
|
||
@lisp
|
||
(use-modules (guix gexp) ;so we can write gexps
|
||
(gnu packages base)) ;for 'coreutils'
|
||
|
||
;; Below is a G-expression representing staged code.
|
||
#~(begin
|
||
;; Invoke 'ls' from the package defined by the 'coreutils'
|
||
;; variable.
|
||
(system* #$(file-append coreutils "/bin/ls") "-l")
|
||
|
||
;; Create this package's output directory.
|
||
(mkdir #$output))
|
||
@end lisp
|
||
|
||
@xref{G-Expressions,,, guix, GNU Guix Reference Manual}, for more on
|
||
gexps.
|
||
|
||
@item
|
||
Multiple variables can be named locally with @code{let} (@pxref{Local
|
||
Bindings,,, guile, GNU Guile Reference Manual}):
|
||
|
||
@lisp
|
||
(define x 10)
|
||
(let ((x 2)
|
||
(y 3))
|
||
(list x y))
|
||
@result{} (2 3)
|
||
|
||
x
|
||
@result{} 10
|
||
|
||
y
|
||
@error{} In procedure module-lookup: Unbound variable: y
|
||
@end lisp
|
||
|
||
Use @code{let*} to allow later variable declarations to refer to earlier
|
||
definitions.
|
||
|
||
@lisp
|
||
(let* ((x 2)
|
||
(y (* x 3)))
|
||
(list x y))
|
||
@result{} (2 6)
|
||
@end lisp
|
||
|
||
@item
|
||
@dfn{Keywords} are typically used to identify the named parameters of a
|
||
procedure. They are prefixed by @code{#:} (hash, colon) followed by
|
||
alphanumeric characters: @code{#:like-this}.
|
||
@xref{Keywords,,, guile, GNU Guile Reference Manual}.
|
||
|
||
@item
|
||
The percentage @code{%} is typically used for read-only global variables in
|
||
the build stage. Note that it is merely a convention, like @code{_} in C.
|
||
Scheme treats @code{%} exactly the same as any other letter.
|
||
|
||
@item
|
||
Modules are created with @code{define-module} (@pxref{Creating Guile
|
||
Modules,,, guile, GNU Guile Reference Manual}). For instance
|
||
|
||
@lisp
|
||
(define-module (guix build-system ruby)
|
||
#:use-module (guix store)
|
||
#:export (ruby-build
|
||
ruby-build-system))
|
||
@end lisp
|
||
|
||
defines the module @code{guix build-system ruby} which must be located in
|
||
@file{guix/build-system/ruby.scm} somewhere in the Guile load path. It
|
||
depends on the @code{(guix store)} module and it exports two variables,
|
||
@code{ruby-build} and @code{ruby-build-system}.
|
||
|
||
@xref{Package Modules,,, guix, GNU Guix Reference Manual}, for info on
|
||
modules that define packages.
|
||
@end itemize
|
||
|
||
@quotation Going further
|
||
Scheme is a language that has been widely used to teach programming and
|
||
you'll find plenty of material using it as a vehicle. Here's a
|
||
selection of documents to learn more about Scheme:
|
||
|
||
@itemize
|
||
@item
|
||
@uref{https://spritely.institute/static/papers/scheme-primer.html, @i{A
|
||
Scheme Primer}}, by Christine Lemmer-Webber and the Spritely Institute.
|
||
|
||
@item
|
||
@uref{http://www.troubleshooters.com/codecorn/scheme_guile/hello.htm,
|
||
@i{Scheme at a Glance}}, by Steve Litt.
|
||
|
||
@item
|
||
@c There used to be a copy at mitpress.mit.edu but it vanished.
|
||
@uref{https://sarabander.github.io/sicp/,
|
||
@i{Structure and Interpretation of Computer Programs}}, by Harold
|
||
Abelson and Gerald Jay Sussman, with Julie Sussman. Colloquially known
|
||
as ``SICP'', this book is a reference.
|
||
|
||
You can also install it and read it from your computer:
|
||
|
||
@example
|
||
guix install sicp info-reader
|
||
info sicp
|
||
@end example
|
||
|
||
@end itemize
|
||
|
||
You'll find more books, tutorials and other resources at
|
||
@url{https://schemers.org/}.
|
||
@end quotation
|
||
|
||
|
||
@c *********************************************************************
|
||
@node Packaging
|
||
@chapter Packaging
|
||
|
||
@cindex packaging
|
||
|
||
This chapter is dedicated to teaching you how to add packages to the
|
||
collection of packages that come with GNU Guix. This involves writing package
|
||
definitions in Guile Scheme, organizing them in package modules, and building
|
||
them.
|
||
|
||
@menu
|
||
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
|
||
@end menu
|
||
|
||
@node Packaging Tutorial
|
||
@section Packaging Tutorial
|
||
|
||
GNU Guix stands out as the @emph{hackable} package manager, mostly because it
|
||
uses @uref{https://www.gnu.org/software/guile/, GNU Guile}, a powerful
|
||
high-level programming language, one of the
|
||
@uref{https://en.wikipedia.org/wiki/Scheme_%28programming_language%29, Scheme}
|
||
dialects from the
|
||
@uref{https://en.wikipedia.org/wiki/Lisp_%28programming_language%29, Lisp family}.
|
||
|
||
Package definitions are also written in Scheme, which empowers Guix in some
|
||
very unique ways, unlike most other package managers that use shell scripts or
|
||
simple languages.
|
||
|
||
@itemize
|
||
@item
|
||
Use functions, structures, macros and all of Scheme expressiveness for your
|
||
package definitions.
|
||
|
||
@item
|
||
Inheritance makes it easy to customize a package by inheriting from it and
|
||
modifying only what is needed.
|
||
|
||
@item
|
||
Batch processing: the whole package collection can be parsed, filtered and
|
||
processed. Building a headless server with all graphical interfaces stripped
|
||
out? It's possible. Want to rebuild everything from source using specific
|
||
compiler optimization flags? Pass the @code{#:make-flags "..."} argument to
|
||
the list of packages. It wouldn't be a stretch to think
|
||
@uref{https://wiki.gentoo.org/wiki/USE_flag, Gentoo USE flags} here, but this
|
||
goes even further: the changes don't have to be thought out beforehand by the
|
||
packager, they can be @emph{programmed} by the user!
|
||
@end itemize
|
||
|
||
The following tutorial covers all the basics around package creation with Guix.
|
||
It does not assume much knowledge of the Guix system nor of the Lisp language.
|
||
The reader is only expected to be familiar with the command line and to have some
|
||
basic programming knowledge.
|
||
|
||
@menu
|
||
* A ``Hello World'' package::
|
||
* Setup::
|
||
* Extended example::
|
||
* Other build systems::
|
||
* Programmable and automated package definition::
|
||
* Getting help::
|
||
* Conclusion::
|
||
* References::
|
||
@end menu
|
||
|
||
@node A ``Hello World'' package
|
||
@subsection A ``Hello World'' package
|
||
|
||
The ``Defining Packages'' section of the manual introduces the basics of Guix
|
||
packaging (@pxref{Defining Packages,,, guix, GNU Guix Reference Manual}). In
|
||
the following section, we will partly go over those basics again.
|
||
|
||
GNU@tie{}Hello is a dummy project that serves as an idiomatic example for
|
||
packaging. It uses the GNU build system (@code{./configure && make && make
|
||
install}). Guix already provides a package definition which is a perfect
|
||
example to start with. You can look up its declaration with @code{guix edit
|
||
hello} from the command line. Let's see how it looks:
|
||
|
||
@lisp
|
||
(define-public hello
|
||
(package
|
||
(name "hello")
|
||
(version "2.10")
|
||
(source (origin
|
||
(method url-fetch)
|
||
(uri (string-append "mirror://gnu/hello/hello-" version
|
||
".tar.gz"))
|
||
(sha256
|
||
(base32
|
||
"0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
|
||
(build-system gnu-build-system)
|
||
(synopsis "Hello, GNU world: An example GNU package")
|
||
(description
|
||
"GNU Hello prints the message \"Hello, world!\" and then exits. It
|
||
serves as an example of standard GNU coding practices. As such, it supports
|
||
command-line arguments, multiple languages, and so on.")
|
||
(home-page "https://www.gnu.org/software/hello/")
|
||
(license gpl3+)))
|
||
@end lisp
|
||
|
||
As you can see, most of it is rather straightforward. But let's review the
|
||
fields together:
|
||
|
||
@table @samp
|
||
@item name
|
||
The project name. Using Scheme conventions, we prefer to keep it
|
||
lower case, without underscore and using dash-separated words.
|
||
|
||
@item source
|
||
This field contains a description of the source code origin. The
|
||
@code{origin} record contains these fields:
|
||
|
||
@enumerate
|
||
@item The method, here @code{url-fetch} to download via HTTP/FTP, but other methods
|
||
exist, such as @code{git-fetch} for Git repositories.
|
||
@item The URI, which is typically some @code{https://} location for @code{url-fetch}. Here
|
||
the special `mirror://gnu` refers to a set of well known locations, all of
|
||
which can be used by Guix to fetch the source, should some of them fail.
|
||
@item The @code{sha256} checksum of the requested file. This is essential to ensure
|
||
the source is not corrupted. Note that Guix works with base32 strings,
|
||
hence the call to the @code{base32} function.
|
||
@end enumerate
|
||
|
||
@item build-system
|
||
|
||
This is where the power of abstraction provided by the Scheme language really
|
||
shines: in this case, the @code{gnu-build-system} abstracts away the famous
|
||
@code{./configure && make && make install} shell invocations. Other build
|
||
systems include the @code{trivial-build-system} which does not do anything and
|
||
requires from the packager to program all the build steps, the
|
||
@code{python-build-system}, the @code{emacs-build-system}, and many more
|
||
(@pxref{Build Systems,,, guix, GNU Guix Reference Manual}).
|
||
|
||
@item synopsis
|
||
It should be a concise summary of what the package does. For many packages a
|
||
tagline from the project's home page can be used as the synopsis.
|
||
|
||
@item description
|
||
Same as for the synopsis, it's fine to re-use the project description from the
|
||
homepage. Note that Guix uses Texinfo syntax.
|
||
|
||
@item home-page
|
||
Use HTTPS if available.
|
||
|
||
@item license
|
||
See @code{guix/licenses.scm} in the project source for a full list of
|
||
available licenses.
|
||
@end table
|
||
|
||
Time to build our first package! Nothing fancy here for now: we will stick to a
|
||
dummy @code{my-hello}, a copy of the above declaration.
|
||
|
||
As with the ritualistic ``Hello World'' taught with most programming languages,
|
||
this will possibly be the most ``manual'' approach. We will work out an ideal
|
||
setup later; for now we will go the simplest route.
|
||
|
||
Save the following to a file @file{my-hello.scm}.
|
||
|
||
@lisp
|
||
(use-modules (guix packages)
|
||
(guix download)
|
||
(guix build-system gnu)
|
||
(guix licenses))
|
||
|
||
(package
|
||
(name "my-hello")
|
||
(version "2.10")
|
||
(source (origin
|
||
(method url-fetch)
|
||
(uri (string-append "mirror://gnu/hello/hello-" version
|
||
".tar.gz"))
|
||
(sha256
|
||
(base32
|
||
"0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
|
||
(build-system gnu-build-system)
|
||
(synopsis "Hello, Guix world: An example custom Guix package")
|
||
(description
|
||
"GNU Hello prints the message \"Hello, world!\" and then exits. It
|
||
serves as an example of standard GNU coding practices. As such, it supports
|
||
command-line arguments, multiple languages, and so on.")
|
||
(home-page "https://www.gnu.org/software/hello/")
|
||
(license gpl3+))
|
||
@end lisp
|
||
|
||
We will explain the extra code in a moment.
|
||
|
||
Feel free to play with the different values of the various fields. If you
|
||
change the source, you'll need to update the checksum. Indeed, Guix refuses to
|
||
build anything if the given checksum does not match the computed checksum of the
|
||
source code. To obtain the correct checksum of the package declaration, we
|
||
need to download the source, compute the sha256 checksum and convert it to
|
||
base32.
|
||
|
||
Thankfully, Guix can automate this task for us; all we need is to provide the
|
||
URI:
|
||
|
||
@c TRANSLATORS: This is example shell output.
|
||
@example sh
|
||
$ guix download mirror://gnu/hello/hello-2.10.tar.gz
|
||
|
||
Starting download of /tmp/guix-file.JLYgL7
|
||
From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz...
|
||
following redirection to `https://mirror.ibcp.fr/pub/gnu/hello/hello-2.10.tar.gz'...
|
||
…10.tar.gz 709KiB 2.5MiB/s 00:00 [##################] 100.0%
|
||
/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
|
||
0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i
|
||
@end example
|
||
|
||
In this specific case the output tells us which mirror was chosen.
|
||
If the result of the above command is not the same as in the above snippet,
|
||
update your @code{my-hello} declaration accordingly.
|
||
|
||
Note that GNU package tarballs come with an OpenPGP signature, so you
|
||
should definitely check the signature of this tarball with `gpg` to
|
||
authenticate it before going further:
|
||
|
||
@c TRANSLATORS: This is example shell output.
|
||
@example sh
|
||
$ guix download mirror://gnu/hello/hello-2.10.tar.gz.sig
|
||
|
||
Starting download of /tmp/guix-file.03tFfb
|
||
From https://ftpmirror.gnu.org/gnu/hello/hello-2.10.tar.gz.sig...
|
||
following redirection to `https://ftp.igh.cnrs.fr/pub/gnu/hello/hello-2.10.tar.gz.sig'...
|
||
….tar.gz.sig 819B 1.2MiB/s 00:00 [##################] 100.0%
|
||
/gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig
|
||
0q0v86n3y38z17rl146gdakw9xc4mcscpk8dscs412j22glrv9jf
|
||
$ gpg --verify /gnu/store/rzs8wba9ka7grrmgcpfyxvs58mly0sx6-hello-2.10.tar.gz.sig /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
|
||
gpg: Signature made Sun 16 Nov 2014 01:08:37 PM CET
|
||
gpg: using RSA key A9553245FDE9B739
|
||
gpg: Good signature from "Sami Kerola <kerolasa@@iki.fi>" [unknown]
|
||
gpg: aka "Sami Kerola (http://www.iki.fi/kerolasa/) <kerolasa@@iki.fi>" [unknown]
|
||
gpg: WARNING: This key is not certified with a trusted signature!
|
||
gpg: There is no indication that the signature belongs to the owner.
|
||
Primary key fingerprint: 8ED3 96E3 7E38 D471 A005 30D3 A955 3245 FDE9 B739
|
||
@end example
|
||
|
||
You can then happily run
|
||
|
||
@c TRANSLATORS: Do not translate this command
|
||
@example sh
|
||
$ guix package --install-from-file=my-hello.scm
|
||
@end example
|
||
|
||
You should now have @code{my-hello} in your profile!
|
||
|
||
@c TRANSLATORS: Do not translate this command
|
||
@example sh
|
||
$ guix package --list-installed=my-hello
|
||
my-hello 2.10 out
|
||
/gnu/store/f1db2mfm8syb8qvc357c53slbvf1g9m9-my-hello-2.10
|
||
@end example
|
||
|
||
We've gone as far as we could without any knowledge of Scheme. Before moving
|
||
on to more complex packages, now is the right time to brush up on your Scheme
|
||
knowledge. @pxref{A Scheme Crash Course} to get up to speed.
|
||
|
||
@node Setup
|
||
@subsection Setup
|
||
|
||
In the rest of this chapter we will rely on some basic Scheme
|
||
programming knowledge. Now let's detail the different possible setups
|
||
for working on Guix packages.
|
||
|
||
There are several ways to set up a Guix packaging environment.
|
||
|
||
We recommend you work directly on the Guix source checkout since it makes it
|
||
easier for everyone to contribute to the project.
|
||
|
||
But first, let's look at other possibilities.
|
||
|
||
@menu
|
||
* Local file::
|
||
* Channels::
|
||
* Direct checkout hacking::
|
||
@end menu
|
||
|
||
@node Local file
|
||
@subsubsection Local file
|
||
|
||
This is what we previously did with @samp{my-hello}. With the Scheme basics we've
|
||
covered, we are now able to explain the leading chunks. As stated in @code{guix
|
||
package --help}:
|
||
|
||
@example
|
||
-f, --install-from-file=FILE
|
||
install the package that the code within FILE
|
||
evaluates to
|
||
@end example
|
||
|
||
Thus the last expression @emph{must} return a package, which is the case in our
|
||
earlier example.
|
||
|
||
The @code{use-modules} expression tells which of the modules we need in the file.
|
||
Modules are a collection of values and procedures. They are commonly called
|
||
``libraries'' or ``packages'' in other programming languages.
|
||
|
||
@node Channels
|
||
@subsubsection Channels
|
||
|
||
@cindex channel
|
||
Guix and its package collection can be extended through @dfn{channels}.
|
||
A channel is a Git repository, public or not, containing @file{.scm}
|
||
files that provide packages (@pxref{Defining Packages,,, guix, GNU Guix
|
||
Reference Manual}) or services (@pxref{Defining Services,,, guix, GNU
|
||
Guix Reference Manual}).
|
||
|
||
How would you go about creating a channel? First, create a directory
|
||
that will contain your @file{.scm} files, say @file{~/my-channel}:
|
||
|
||
@example
|
||
mkdir ~/my-channel
|
||
@end example
|
||
|
||
Suppose you want to add the @samp{my-hello} package we saw previously;
|
||
it first needs some adjustments:
|
||
|
||
@lisp
|
||
(define-module (my-hello)
|
||
#:use-module (guix licenses)
|
||
#:use-module (guix packages)
|
||
#:use-module (guix build-system gnu)
|
||
#:use-module (guix download))
|
||
|
||
(define-public my-hello
|
||
(package
|
||
(name "my-hello")
|
||
(version "2.10")
|
||
(source (origin
|
||
(method url-fetch)
|
||
(uri (string-append "mirror://gnu/hello/hello-" version
|
||
".tar.gz"))
|
||
(sha256
|
||
(base32
|
||
"0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
|
||
(build-system gnu-build-system)
|
||
(synopsis "Hello, Guix world: An example custom Guix package")
|
||
(description
|
||
"GNU Hello prints the message \"Hello, world!\" and then exits. It
|
||
serves as an example of standard GNU coding practices. As such, it supports
|
||
command-line arguments, multiple languages, and so on.")
|
||
(home-page "https://www.gnu.org/software/hello/")
|
||
(license gpl3+)))
|
||
@end lisp
|
||
|
||
Note that we have assigned the package value to an exported variable name with
|
||
@code{define-public}. This is effectively assigning the package to the @code{my-hello}
|
||
variable so that it can be referenced, among other as dependency of other
|
||
packages.
|
||
|
||
If you use @code{guix package --install-from-file=my-hello.scm} on the above file, it
|
||
will fail because the last expression, @code{define-public}, does not return a
|
||
package. If you want to use @code{define-public} in this use-case nonetheless, make
|
||
sure the file ends with an evaluation of @code{my-hello}:
|
||
|
||
@lisp
|
||
;; ...
|
||
(define-public my-hello
|
||
;; ...
|
||
)
|
||
|
||
my-hello
|
||
@end lisp
|
||
|
||
This last example is not very typical.
|
||
|
||
Now how do you make that package visible to @command{guix} commands so
|
||
you can test your packages? You need to add the directory to the search
|
||
path using the @option{-L} command-line option, as in these examples:
|
||
|
||
@example
|
||
guix show -L ~/my-channel my-hello
|
||
guix build -L ~/my-channel my-hello
|
||
@end example
|
||
|
||
The final step is to turn @file{~/my-channel} into an actual channel,
|
||
making your package collection seamlessly available @i{via} any
|
||
@command{guix} command. To do that, you first need to make it a Git
|
||
repository:
|
||
|
||
@example
|
||
cd ~/my-channel
|
||
git init
|
||
git add my-hello.scm
|
||
git commit -m "First commit of my channel."
|
||
@end example
|
||
|
||
And that's it, you have a channel! From there on, you can add this
|
||
channel to your channel configuration in
|
||
@file{~/.config/guix/channels.scm} (@pxref{Specifying Additional
|
||
Channels,,, guix, GNU Guix Reference Manual}); assuming you keep your
|
||
channel local for now, the @file{channels.scm} would look something like
|
||
this:
|
||
|
||
@lisp
|
||
(append (list (channel
|
||
(name 'my-channel)
|
||
(url (string-append "file://" (getenv "HOME")
|
||
"/my-channel"))))
|
||
%default-channels)
|
||
@end lisp
|
||
|
||
Next time you run @command{guix pull}, your channel will be picked up
|
||
and the packages it defines will be readily available to all the
|
||
@command{guix} commands, even if you do not pass @option{-L}. The
|
||
@command{guix describe} command will show that Guix is, indeed, using
|
||
both the @code{my-channel} and the @code{guix} channels.
|
||
|
||
@xref{Creating a Channel,,, guix, GNU Guix Reference Manual}, for
|
||
details.
|
||
|
||
@node Direct checkout hacking
|
||
@subsubsection Direct checkout hacking
|
||
|
||
Working directly on the Guix project is recommended: it reduces the friction
|
||
when the time comes to submit your changes upstream to let the community benefit
|
||
from your hard work!
|
||
|
||
Unlike most software distributions, the Guix repository holds in one place both
|
||
the tooling (including the package manager) and the package definitions. This
|
||
choice was made so that it would give developers the flexibility to modify the
|
||
API without breakage by updating all packages at the same time. This reduces
|
||
development inertia.
|
||
|
||
Check out the official @uref{https://git-scm.com/, Git} repository:
|
||
|
||
@example
|
||
$ git clone https://git.savannah.gnu.org/git/guix.git
|
||
@end example
|
||
|
||
In the rest of this article, we use @samp{$GUIX_CHECKOUT} to refer to the location of
|
||
the checkout.
|
||
|
||
|
||
Follow the instructions in the manual (@pxref{Contributing,,, guix, GNU Guix
|
||
Reference Manual}) to set up the repository environment.
|
||
|
||
Once ready, you should be able to use the package definitions from the
|
||
repository environment.
|
||
|
||
Feel free to edit package definitions found in @samp{$GUIX_CHECKOUT/gnu/packages}.
|
||
|
||
The @samp{$GUIX_CHECKOUT/pre-inst-env} script lets you use @samp{guix} over the package
|
||
collection of the repository (@pxref{Running Guix Before It Is
|
||
Installed,,, guix, GNU Guix Reference Manual}).
|
||
|
||
@itemize
|
||
@item
|
||
Search packages, such as Ruby:
|
||
|
||
@example
|
||
$ cd $GUIX_CHECKOUT
|
||
$ ./pre-inst-env guix package --list-available=ruby
|
||
ruby 1.8.7-p374 out gnu/packages/ruby.scm:119:2
|
||
ruby 2.1.6 out gnu/packages/ruby.scm:91:2
|
||
ruby 2.2.2 out gnu/packages/ruby.scm:39:2
|
||
@end example
|
||
|
||
@item
|
||
Build a package, here Ruby version 2.1:
|
||
|
||
@example
|
||
$ ./pre-inst-env guix build --keep-failed ruby@@2.1
|
||
/gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6
|
||
@end example
|
||
|
||
@item
|
||
Install it to your user profile:
|
||
|
||
@example
|
||
$ ./pre-inst-env guix package --install ruby@@2.1
|
||
@end example
|
||
|
||
@item
|
||
Check for common mistakes:
|
||
|
||
@example
|
||
$ ./pre-inst-env guix lint ruby@@2.1
|
||
@end example
|
||
@end itemize
|
||
|
||
Guix strives at maintaining a high packaging standard; when contributing to the
|
||
Guix project, remember to
|
||
|
||
@itemize
|
||
@item
|
||
follow the coding style (@pxref{Coding Style,,, guix, GNU Guix Reference Manual}),
|
||
@item
|
||
and review the check list from the manual (@pxref{Submitting Patches,,, guix, GNU Guix Reference Manual}).
|
||
@end itemize
|
||
|
||
Once you are happy with the result, you are welcome to send your contribution to
|
||
make it part of Guix. This process is also detailed in the manual. (@pxref{Contributing,,, guix, GNU Guix Reference Manual})
|
||
|
||
|
||
It's a community effort so the more join in, the better Guix becomes!
|
||
|
||
@node Extended example
|
||
@subsection Extended example
|
||
|
||
The above ``Hello World'' example is as simple as it goes. Packages can be more
|
||
complex than that and Guix can handle more advanced scenarios. Let's look at
|
||
another, more sophisticated package (slightly modified from the source):
|
||
|
||
@lisp
|
||
(define-module (gnu packages version-control)
|
||
#:use-module ((guix licenses) #:prefix license:)
|
||
#:use-module (guix utils)
|
||
#:use-module (guix packages)
|
||
#:use-module (guix git-download)
|
||
#:use-module (guix build-system cmake)
|
||
#:use-module (gnu packages compression)
|
||
#:use-module (gnu packages pkg-config)
|
||
#:use-module (gnu packages python)
|
||
#:use-module (gnu packages ssh)
|
||
#:use-module (gnu packages tls)
|
||
#:use-module (gnu packages web))
|
||
|
||
(define-public my-libgit2
|
||
(let ((commit "e98d0a37c93574d2c6107bf7f31140b548c6a7bf")
|
||
(revision "1"))
|
||
(package
|
||
(name "my-libgit2")
|
||
(version (git-version "0.26.6" revision commit))
|
||
(source (origin
|
||
(method git-fetch)
|
||
(uri (git-reference
|
||
(url "https://github.com/libgit2/libgit2/")
|
||
(commit commit)))
|
||
(file-name (git-file-name name version))
|
||
(sha256
|
||
(base32
|
||
"17pjvprmdrx4h6bb1hhc98w9qi6ki7yl57f090n9kbhswxqfs7s3"))
|
||
(patches (search-patches "libgit2-mtime-0.patch"))
|
||
(modules '((guix build utils)))
|
||
;; Remove bundled software.
|
||
(snippet '(delete-file-recursively "deps"))))
|
||
(build-system cmake-build-system)
|
||
(outputs '("out" "debug"))
|
||
(arguments
|
||
`(#:tests? #true ; Run the test suite (this is the default)
|
||
#:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
|
||
#:phases
|
||
(modify-phases %standard-phases
|
||
(add-after 'unpack 'fix-hardcoded-paths
|
||
(lambda _
|
||
(substitute* "tests/repo/init.c"
|
||
(("#!/bin/sh") (string-append "#!" (which "sh"))))
|
||
(substitute* "tests/clar/fs.h"
|
||
(("/bin/cp") (which "cp"))
|
||
(("/bin/rm") (which "rm")))))
|
||
;; Run checks more verbosely.
|
||
(replace 'check
|
||
(lambda* (#:key tests? #:allow-other-keys)
|
||
(when tests?
|
||
(invoke "./libgit2_clar" "-v" "-Q"))))
|
||
(add-after 'unpack 'make-files-writable-for-tests
|
||
(lambda _ (for-each make-file-writable (find-files ".")))))))
|
||
(inputs
|
||
(list libssh2 http-parser python-wrapper))
|
||
(native-inputs
|
||
(list pkg-config))
|
||
(propagated-inputs
|
||
;; These two libraries are in 'Requires.private' in libgit2.pc.
|
||
(list openssl zlib))
|
||
(home-page "https://libgit2.github.com/")
|
||
(synopsis "Library providing Git core methods")
|
||
(description
|
||
"Libgit2 is a portable, pure C implementation of the Git core methods
|
||
provided as a re-entrant linkable library with a solid API, allowing you to
|
||
write native speed custom Git applications in any language with bindings.")
|
||
;; GPLv2 with linking exception
|
||
(license license:gpl2))))
|
||
@end lisp
|
||
|
||
(In those cases were you only want to tweak a few fields from a package
|
||
definition, you should rely on inheritance instead of copy-pasting everything.
|
||
See below.)
|
||
|
||
Let's discuss those fields in depth.
|
||
|
||
@subsubsection @code{git-fetch} method
|
||
|
||
Unlike the @code{url-fetch} method, @code{git-fetch} expects a @code{git-reference} which takes
|
||
a Git repository and a commit. The commit can be any Git reference such as
|
||
tags, so if the @code{version} is tagged, then it can be used directly. Sometimes
|
||
the tag is prefixed with a @code{v}, in which case you'd use @code{(commit (string-append
|
||
"v" version))}.
|
||
|
||
To ensure that the source code from the Git repository is stored in a
|
||
directory with a descriptive name, we use @code{(file-name (git-file-name name
|
||
version))}.
|
||
|
||
The @code{git-version} procedure can be used to derive the
|
||
version when packaging programs for a specific commit, following the
|
||
Guix contributor guidelines (@pxref{Version Numbers,,, guix, GNU Guix
|
||
Reference Manual}).
|
||
|
||
How does one obtain the @code{sha256} hash that's in there, you ask? By
|
||
invoking @command{guix hash} on a checkout of the desired commit, along
|
||
these lines:
|
||
|
||
@example
|
||
git clone https://github.com/libgit2/libgit2/
|
||
cd libgit2
|
||
git checkout v0.26.6
|
||
guix hash -rx .
|
||
@end example
|
||
|
||
@command{guix hash -rx} computes a SHA256 hash over the whole directory,
|
||
excluding the @file{.git} sub-directory (@pxref{Invoking guix hash,,,
|
||
guix, GNU Guix Reference Manual}).
|
||
|
||
In the future, @command{guix download} will hopefully be able to do
|
||
these steps for you, just like it does for regular downloads.
|
||
|
||
@subsubsection Snippets
|
||
|
||
Snippets are quoted (i.e. non-evaluated) Scheme code that are a means of patching
|
||
the source. They are a Guix-y alternative to the traditional @file{.patch} files.
|
||
Because of the quote, the code in only evaluated when passed to the Guix daemon
|
||
for building. There can be as many snippets as needed.
|
||
|
||
Snippets might need additional Guile modules which can be imported from the
|
||
@code{modules} field.
|
||
|
||
@subsubsection Inputs
|
||
|
||
There are 3 different input types. In short:
|
||
|
||
@table @asis
|
||
@item native-inputs
|
||
Required for building but not runtime -- installing a package
|
||
through a substitute won't install these inputs.
|
||
@item inputs
|
||
Installed in the store but not in the profile, as well as being
|
||
present at build time.
|
||
@item propagated-inputs
|
||
Installed in the store and in the profile, as well as
|
||
being present at build time.
|
||
@end table
|
||
|
||
@xref{package Reference,,, guix, GNU Guix Reference Manual} for more details.
|
||
|
||
The distinction between the various inputs is important: if a dependency can be
|
||
handled as an @emph{input} instead of a @emph{propagated input}, it should be done so, or
|
||
else it ``pollutes'' the user profile for no good reason.
|
||
|
||
For instance, a user installing a graphical program that depends on a
|
||
command line tool might only be interested in the graphical part, so there is no
|
||
need to force the command line tool into the user profile. The dependency is a
|
||
concern to the package, not to the user. @emph{Inputs} make it possible to handle
|
||
dependencies without bugging the user by adding undesired executable files (or
|
||
libraries) to their profile.
|
||
|
||
Same goes for @emph{native-inputs}: once the program is installed, build-time
|
||
dependencies can be safely garbage-collected.
|
||
It also matters when a substitute is available, in which case only the @emph{inputs}
|
||
and @emph{propagated inputs} will be fetched: the @emph{native inputs} are not required to
|
||
install a package from a substitute.
|
||
|
||
@quotation Note
|
||
You may see here and there snippets where package inputs are written
|
||
quite differently, like so:
|
||
|
||
@lisp
|
||
;; The "old style" for inputs.
|
||
(inputs
|
||
`(("libssh2" ,libssh2)
|
||
("http-parser" ,http-parser)
|
||
("python" ,python-wrapper)))
|
||
@end lisp
|
||
|
||
This is the ``old style'', where each input in the list is explicitly
|
||
given a label (a string). It is still supported but we recommend using
|
||
the style above instead. @xref{package Reference,,, guix, GNU Guix
|
||
Reference Manual}, for more info.
|
||
@end quotation
|
||
|
||
@subsubsection Outputs
|
||
|
||
Just like how a package can have multiple inputs, it can also produce multiple
|
||
outputs.
|
||
|
||
Each output corresponds to a separate directory in the store.
|
||
|
||
The user can choose which output to install; this is useful to save space or
|
||
to avoid polluting the user profile with unwanted executables or libraries.
|
||
|
||
Output separation is optional. When the @code{outputs} field is left out, the
|
||
default and only output (the complete package) is referred to as @code{"out"}.
|
||
|
||
Typical separate output names include @code{debug} and @code{doc}.
|
||
|
||
It's advised to separate outputs only when you've shown it's worth it: if the
|
||
output size is significant (compare with @code{guix size}) or in case the package is
|
||
modular.
|
||
|
||
@subsubsection Build system arguments
|
||
|
||
The @code{arguments} is a keyword-value list used to configure the build process.
|
||
|
||
The simplest argument @code{#:tests?} can be used to disable the test suite when
|
||
building the package. This is mostly useful when the package does not feature
|
||
any test suite. It's strongly recommended to keep the test suite on if there is
|
||
one.
|
||
|
||
Another common argument is @code{:make-flags}, which specifies a list of flags to
|
||
append when running make, as you would from the command line. For instance, the
|
||
following flags
|
||
|
||
@lisp
|
||
#:make-flags (list (string-append "prefix=" (assoc-ref %outputs "out"))
|
||
"CC=gcc")
|
||
@end lisp
|
||
|
||
translate into
|
||
|
||
@example
|
||
$ make CC=gcc prefix=/gnu/store/...-<out>
|
||
@end example
|
||
|
||
This sets the C compiler to @code{gcc} and the @code{prefix} variable (the installation
|
||
directory in Make parlance) to @code{(assoc-ref %outputs "out")}, which is a build-stage
|
||
global variable pointing to the destination directory in the store (something like
|
||
@file{/gnu/store/...-my-libgit2-20180408}).
|
||
|
||
Similarly, it's possible to set the configure flags:
|
||
|
||
@lisp
|
||
#:configure-flags '("-DUSE_SHA1DC=ON")
|
||
@end lisp
|
||
|
||
The @code{%build-inputs} variable is also generated in scope. It's an association
|
||
table that maps the input names to their store directories.
|
||
|
||
The @code{phases} keyword lists the sequential steps of the build system. Typically
|
||
phases include @code{unpack}, @code{configure}, @code{build}, @code{install} and @code{check}. To know
|
||
more about those phases, you need to work out the appropriate build system
|
||
definition in @samp{$GUIX_CHECKOUT/guix/build/gnu-build-system.scm}:
|
||
|
||
@lisp
|
||
(define %standard-phases
|
||
;; Standard build phases, as a list of symbol/procedure pairs.
|
||
(let-syntax ((phases (syntax-rules ()
|
||
((_ p ...) `((p . ,p) ...)))))
|
||
(phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
|
||
bootstrap
|
||
patch-usr-bin-file
|
||
patch-source-shebangs configure patch-generated-file-shebangs
|
||
build check install
|
||
patch-shebangs strip
|
||
validate-runpath
|
||
validate-documentation-location
|
||
delete-info-dir-file
|
||
patch-dot-desktop-files
|
||
install-license-files
|
||
reset-gzip-timestamps
|
||
compress-documentation)))
|
||
@end lisp
|
||
|
||
Or from the REPL:
|
||
|
||
@lisp
|
||
(add-to-load-path "/path/to/guix/checkout")
|
||
,use (guix build gnu-build-system)
|
||
(map first %standard-phases)
|
||
@result{} (set-SOURCE-DATE-EPOCH set-paths install-locale unpack bootstrap patch-usr-bin-file patch-source-shebangs configure patch-generated-file-shebangs build check install patch-shebangs strip validate-runpath validate-documentation-location delete-info-dir-file patch-dot-desktop-files install-license-files reset-gzip-timestamps compress-documentation)
|
||
@end lisp
|
||
|
||
If you want to know more about what happens during those phases, consult the
|
||
associated procedures.
|
||
|
||
For instance, as of this writing the definition of @code{unpack} for the GNU build
|
||
system is:
|
||
|
||
@lisp
|
||
(define* (unpack #:key source #:allow-other-keys)
|
||
"Unpack SOURCE in the working directory, and change directory within the
|
||
source. When SOURCE is a directory, copy it in a sub-directory of the current
|
||
working directory."
|
||
(if (file-is-directory? source)
|
||
(begin
|
||
(mkdir "source")
|
||
(chdir "source")
|
||
|
||
;; Preserve timestamps (set to the Epoch) on the copied tree so that
|
||
;; things work deterministically.
|
||
(copy-recursively source "."
|
||
#:keep-mtime? #true))
|
||
(begin
|
||
(if (string-suffix? ".zip" source)
|
||
(invoke "unzip" source)
|
||
(invoke "tar" "xvf" source))
|
||
(chdir (first-subdirectory "."))))
|
||
#true)
|
||
@end lisp
|
||
|
||
Note the @code{chdir} call: it changes the working directory to where the source was
|
||
unpacked.
|
||
Thus every phase following the @code{unpack} will use the source as a working
|
||
directory, which is why we can directly work on the source files.
|
||
That is to say, unless a later phase changes the working directory to something
|
||
else.
|
||
|
||
We modify the list of @code{%standard-phases} of the build system with the
|
||
@code{modify-phases} macro as per the list of specified modifications, which may have
|
||
the following forms:
|
||
|
||
@itemize
|
||
@item
|
||
@code{(add-before @var{phase} @var{new-phase} @var{procedure})}: Run @var{procedure} named @var{new-phase} before @var{phase}.
|
||
@item
|
||
@code{(add-after @var{phase} @var{new-phase} @var{procedure})}: Same, but afterwards.
|
||
@item
|
||
@code{(replace @var{phase} @var{procedure})}.
|
||
@item
|
||
@code{(delete @var{phase})}.
|
||
@end itemize
|
||
|
||
The @var{procedure} supports the keyword arguments @code{inputs} and @code{outputs}. Each
|
||
input (whether @emph{native}, @emph{propagated} or not) and output directory is referenced
|
||
by their name in those variables. Thus @code{(assoc-ref outputs "out")} is the store
|
||
directory of the main output of the package. A phase procedure may look like
|
||
this:
|
||
|
||
@lisp
|
||
(lambda* (#:key inputs outputs #:allow-other-keys)
|
||
(let ((bash-directory (assoc-ref inputs "bash"))
|
||
(output-directory (assoc-ref outputs "out"))
|
||
(doc-directory (assoc-ref outputs "doc")))
|
||
;; ...
|
||
#true))
|
||
@end lisp
|
||
|
||
The procedure must return @code{#true} on success. It's brittle to rely on the return
|
||
value of the last expression used to tweak the phase because there is no
|
||
guarantee it would be a @code{#true}. Hence the trailing @code{#true} to ensure the right value
|
||
is returned on success.
|
||
|
||
@subsubsection Code staging
|
||
|
||
The astute reader may have noticed the quasi-quote and comma syntax in the
|
||
argument field. Indeed, the build code in the package declaration should not be
|
||
evaluated on the client side, but only when passed to the Guix daemon. This
|
||
mechanism of passing code around two running processes is called @uref{https://arxiv.org/abs/1709.00833, code staging}.
|
||
|
||
@subsubsection Utility functions
|
||
|
||
When customizing @code{phases}, we often need to write code that mimics the
|
||
equivalent system invocations (@code{make}, @code{mkdir}, @code{cp}, etc.)@: commonly used during
|
||
regular ``Unix-style'' installations.
|
||
|
||
Some like @code{chmod} are native to Guile.
|
||
@xref{,,, guile, Guile reference manual} for a complete list.
|
||
|
||
Guix provides additional helper functions which prove especially handy in the
|
||
context of package management.
|
||
|
||
Some of those functions can be found in
|
||
@samp{$GUIX_CHECKOUT/guix/guix/build/utils.scm}. Most of them mirror the behaviour
|
||
of the traditional Unix system commands:
|
||
|
||
@table @code
|
||
@item which
|
||
Like the @samp{which} system command.
|
||
@item find-files
|
||
Akin to the @samp{find} system command.
|
||
@item mkdir-p
|
||
Like @samp{mkdir -p}, which creates all parents as needed.
|
||
@item install-file
|
||
Similar to @samp{install} when installing a file to a (possibly
|
||
non-existing) directory. Guile has @code{copy-file} which works
|
||
like @samp{cp}.
|
||
@item copy-recursively
|
||
Like @samp{cp -r}.
|
||
@item delete-file-recursively
|
||
Like @samp{rm -rf}.
|
||
@item invoke
|
||
Run an executable. This should be used instead of @code{system*}.
|
||
@item with-directory-excursion
|
||
Run the body in a different working directory,
|
||
then restore the previous working directory.
|
||
@item substitute*
|
||
A ``@command{sed}-like'' function.
|
||
@end table
|
||
|
||
@xref{Build Utilities,,, guix, GNU Guix Reference Manual}, for more
|
||
information on these utilities.
|
||
|
||
@subsubsection Module prefix
|
||
|
||
The license in our last example needs a prefix: this is because of how the
|
||
@code{license} module was imported in the package, as @code{#:use-module ((guix licenses)
|
||
#:prefix license:)}. The Guile module import mechanism
|
||
(@pxref{Using Guile Modules,,, guile, Guile reference manual})
|
||
gives the user full control over namespacing: this is needed to avoid
|
||
clashes between, say, the
|
||
@samp{zlib} variable from @samp{licenses.scm} (a @emph{license} value) and the @samp{zlib} variable
|
||
from @samp{compression.scm} (a @emph{package} value).
|
||
|
||
@node Other build systems
|
||
@subsection Other build systems
|
||
|
||
What we've seen so far covers the majority of packages using a build system
|
||
other than the @code{trivial-build-system}. The latter does not automate anything
|
||
and leaves you to build everything manually. This can be more demanding and we
|
||
won't cover it here for now, but thankfully it is rarely necessary to fall back
|
||
on this system.
|
||
|
||
For the other build systems, such as ASDF, Emacs, Perl, Ruby and many more, the
|
||
process is very similar to the GNU build system except for a few specialized
|
||
arguments.
|
||
|
||
@xref{Build Systems,,, guix, GNU Guix Reference Manual}, for more
|
||
information on build systems, or check the source code in the
|
||
@samp{$GUIX_CHECKOUT/guix/build} and
|
||
@samp{$GUIX_CHECKOUT/guix/build-system} directories.
|
||
|
||
@node Programmable and automated package definition
|
||
@subsection Programmable and automated package definition
|
||
|
||
We can't repeat it enough: having a full-fledged programming language at hand
|
||
empowers us in ways that reach far beyond traditional package management.
|
||
|
||
Let's illustrate this with some awesome features of Guix!
|
||
|
||
@menu
|
||
* Recursive importers::
|
||
* Automatic update::
|
||
* Inheritance::
|
||
@end menu
|
||
|
||
@node Recursive importers
|
||
@subsubsection Recursive importers
|
||
|
||
You might find some build systems good enough that there is little to do at all
|
||
to write a package, to the point that it becomes repetitive and tedious after a
|
||
while. A @emph{raison d'être} of computers is to replace human beings at those
|
||
boring tasks. So let's tell Guix to do this for us and create the package
|
||
definition of an R package from CRAN (the output is trimmed for conciseness):
|
||
|
||
@example
|
||
$ guix import cran --recursive walrus
|
||
|
||
(define-public r-mc2d
|
||
; ...
|
||
(license gpl2+)))
|
||
|
||
(define-public r-jmvcore
|
||
; ...
|
||
(license gpl2+)))
|
||
|
||
(define-public r-wrs2
|
||
; ...
|
||
(license gpl3)))
|
||
|
||
(define-public r-walrus
|
||
(package
|
||
(name "r-walrus")
|
||
(version "1.0.3")
|
||
(source
|
||
(origin
|
||
(method url-fetch)
|
||
(uri (cran-uri "walrus" version))
|
||
(sha256
|
||
(base32
|
||
"1nk2glcvy4hyksl5ipq2mz8jy4fss90hx6cq98m3w96kzjni6jjj"))))
|
||
(build-system r-build-system)
|
||
(propagated-inputs
|
||
(list r-ggplot2 r-jmvcore r-r6 r-wrs2))
|
||
(home-page "https://github.com/jamovi/walrus")
|
||
(synopsis "Robust Statistical Methods")
|
||
(description
|
||
"This package provides a toolbox of common robust statistical
|
||
tests, including robust descriptives, robust t-tests, and robust ANOVA.
|
||
It is also available as a module for 'jamovi' (see
|
||
<https://www.jamovi.org> for more information). Walrus is based on the
|
||
WRS2 package by Patrick Mair, which is in turn based on the scripts and
|
||
work of Rand Wilcox. These analyses are described in depth in the book
|
||
'Introduction to Robust Estimation & Hypothesis Testing'.")
|
||
(license gpl3)))
|
||
@end example
|
||
|
||
The recursive importer won't import packages for which Guix already has package
|
||
definitions, except for the very first.
|
||
|
||
Not all applications can be packaged this way, only those relying on a select
|
||
number of supported systems. Read about the full list of importers in
|
||
the guix import section of the manual
|
||
(@pxref{Invoking guix import,,, guix, GNU Guix Reference Manual}).
|
||
|
||
@node Automatic update
|
||
@subsubsection Automatic update
|
||
|
||
Guix can be smart enough to check for updates on systems it knows. It can
|
||
report outdated package definitions with
|
||
|
||
@example
|
||
$ guix refresh hello
|
||
@end example
|
||
|
||
In most cases, updating a package to a newer version requires little more than
|
||
changing the version number and the checksum. Guix can do that automatically as
|
||
well:
|
||
|
||
@example
|
||
$ guix refresh hello --update
|
||
@end example
|
||
|
||
@node Inheritance
|
||
@subsubsection Inheritance
|
||
|
||
If you've started browsing the existing package definitions, you might have
|
||
noticed that a significant number of them have a @code{inherit} field:
|
||
|
||
@lisp
|
||
(define-public adwaita-icon-theme
|
||
(package (inherit gnome-icon-theme)
|
||
(name "adwaita-icon-theme")
|
||
(version "3.26.1")
|
||
(source (origin
|
||
(method url-fetch)
|
||
(uri (string-append "mirror://gnome/sources/" name "/"
|
||
(version-major+minor version) "/"
|
||
name "-" version ".tar.xz"))
|
||
(sha256
|
||
(base32
|
||
"17fpahgh5dyckgz7rwqvzgnhx53cx9kr2xw0szprc6bnqy977fi8"))))
|
||
(native-inputs (list `(,gtk+ "bin")))))
|
||
@end lisp
|
||
|
||
All unspecified fields are inherited from the parent package. This is very
|
||
convenient to create alternative packages, for instance with different source,
|
||
version or compilation options.
|
||
|
||
@node Getting help
|
||
@subsection Getting help
|
||
|
||
Sadly, some applications can be tough to package. Sometimes they need a patch to
|
||
work with the non-standard file system hierarchy enforced by the store.
|
||
Sometimes the tests won't run properly. (They can be skipped but this is not
|
||
recommended.) Other times the resulting package won't be reproducible.
|
||
|
||
Should you be stuck, unable to figure out how to fix any sort of packaging
|
||
issue, don't hesitate to ask the community for help.
|
||
|
||
See the @uref{https://www.gnu.org/software/guix/contact/, Guix homepage} for information on the mailing lists, IRC, etc.
|
||
|
||
@node Conclusion
|
||
@subsection Conclusion
|
||
|
||
This tutorial was a showcase of the sophisticated package management that Guix
|
||
boasts. At this point we have mostly restricted this introduction to the
|
||
@code{gnu-build-system} which is a core abstraction layer on which more advanced
|
||
abstractions are based.
|
||
|
||
Where do we go from here? Next we ought to dissect the innards of the build
|
||
system by removing all abstractions, using the @code{trivial-build-system}: this
|
||
should give us a thorough understanding of the process before investigating some
|
||
more advanced packaging techniques and edge cases.
|
||
|
||
Other features worth exploring are the interactive editing and debugging
|
||
capabilities of Guix provided by the Guile REPL@.
|
||
|
||
Those fancy features are completely optional and can wait; now is a good time
|
||
to take a well-deserved break. With what we've introduced here you should be
|
||
well armed to package lots of programs. You can get started right away and
|
||
hopefully we will see your contributions soon!
|
||
|
||
@node References
|
||
@subsection References
|
||
|
||
@itemize
|
||
@item
|
||
The @uref{https://www.gnu.org/software/guix/manual/en/html_node/Defining-Packages.html, package reference in the manual}
|
||
|
||
@item
|
||
@uref{https://gitlab.com/pjotrp/guix-notes/blob/master/HACKING.org, Pjotr’s hacking guide to GNU Guix}
|
||
|
||
@item
|
||
@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
|
||
@end itemize
|
||
|
||
@c *********************************************************************
|
||
@node System Configuration
|
||
@chapter System Configuration
|
||
|
||
Guix offers a flexible language for declaratively configuring your Guix
|
||
System. This flexibility can at times be overwhelming. The purpose of this
|
||
chapter is to demonstrate some advanced configuration concepts.
|
||
|
||
@pxref{System Configuration,,, guix, GNU Guix Reference Manual} for a complete
|
||
reference.
|
||
|
||
@menu
|
||
* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
|
||
* Customizing the Kernel:: Creating and using a custom Linux kernel on Guix System.
|
||
* Guix System Image API:: Customizing images to target specific platforms.
|
||
* Using security keys:: How to use security keys with Guix System.
|
||
* Dynamic DNS mcron job:: Job to update the IP address behind a DuckDNS host name.
|
||
* Connecting to Wireguard VPN:: Connecting to a Wireguard VPN.
|
||
* Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
|
||
* Running Guix on a Linode Server:: Running Guix on a Linode Server.
|
||
* Running Guix on a Kimsufi Server:: Running Guix on a Kimsufi Server.
|
||
* Setting up a bind mount:: Setting up a bind mount in the file-systems definition.
|
||
* Getting substitutes from Tor:: Configuring Guix daemon to get substitutes through Tor.
|
||
* Setting up NGINX with Lua:: Configuring NGINX web-server to load Lua modules.
|
||
* Music Server with Bluetooth Audio:: Headless music player with Bluetooth output.
|
||
@end menu
|
||
|
||
@node Auto-Login to a Specific TTY
|
||
@section Auto-Login to a Specific TTY
|
||
|
||
While the Guix manual explains auto-login one user to @emph{all} TTYs (
|
||
@pxref{auto-login to TTY,,, guix, GNU Guix Reference Manual}), some
|
||
might prefer a situation, in which one user is logged into one TTY with
|
||
the other TTYs either configured to login different users or no one at
|
||
all. Note that one can auto-login one user to any TTY, but it is
|
||
usually advisable to avoid @code{tty1}, which, by default, is used to
|
||
log warnings and errors.
|
||
|
||
Here is how one might set up auto login for one user to one tty:
|
||
|
||
@lisp
|
||
(define (auto-login-to-tty config tty user)
|
||
(if (string=? tty (mingetty-configuration-tty config))
|
||
(mingetty-configuration
|
||
(inherit config)
|
||
(auto-login user))
|
||
config))
|
||
|
||
(define %my-services
|
||
(modify-services %base-services
|
||
;; @dots{}
|
||
(mingetty-service-type config =>
|
||
(auto-login-to-tty
|
||
config "tty3" "alice"))))
|
||
|
||
(operating-system
|
||
;; @dots{}
|
||
(services %my-services))
|
||
@end lisp
|
||
|
||
One could also @code{compose} (@pxref{Higher-Order Functions,,, guile,
|
||
The Guile Reference Manual}) @code{auto-login-to-tty} to login multiple
|
||
users to multiple ttys.
|
||
|
||
Finally, here is a note of caution. Setting up auto login to a TTY,
|
||
means that anyone can turn on your computer and run commands as your
|
||
regular user.
|
||
However, if you have an encrypted root partition, and thus already need
|
||
to enter a passphrase when the system boots, auto-login might be a
|
||
convenient option.
|
||
|
||
|
||
@node Customizing the Kernel
|
||
@section Customizing the Kernel
|
||
|
||
Guix is, at its core, a source based distribution with substitutes
|
||
(@pxref{Substitutes,,, guix, GNU Guix Reference Manual}), and as such building
|
||
packages from their source code is an expected part of regular package
|
||
installations and upgrades. Given this starting point, it makes sense that
|
||
efforts are made to reduce the amount of time spent compiling packages, and
|
||
recent changes and upgrades to the building and distribution of substitutes
|
||
continues to be a topic of discussion within Guix.
|
||
|
||
The kernel, while not requiring an overabundance of RAM to build, does take a
|
||
rather long time on an average machine. The official kernel configuration, as
|
||
is the case with many GNU/Linux distributions, errs on the side of
|
||
inclusiveness, and this is really what causes the build to take such a long
|
||
time when the kernel is built from source.
|
||
|
||
The Linux kernel, however, can also just be described as a regular old
|
||
package, and as such can be customized just like any other package. The
|
||
procedure is a little bit different, although this is primarily due to the
|
||
nature of how the package definition is written.
|
||
|
||
The @code{linux-libre} kernel package definition is actually a procedure which
|
||
creates a package.
|
||
|
||
@lisp
|
||
(define* (make-linux-libre* version gnu-revision source supported-systems
|
||
#:key
|
||
(extra-version #f)
|
||
;; A function that takes an arch and a variant.
|
||
;; See kernel-config for an example.
|
||
(configuration-file #f)
|
||
(defconfig "defconfig")
|
||
(extra-options %default-extra-linux-options))
|
||
...)
|
||
@end lisp
|
||
|
||
The current @code{linux-libre} package is for the 5.15.x series, and is
|
||
declared like this:
|
||
|
||
@lisp
|
||
(define-public linux-libre-5.15
|
||
(make-linux-libre* linux-libre-5.15-version
|
||
linux-libre-5.15-gnu-revision
|
||
linux-libre-5.15-source
|
||
'("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux" "riscv64-linux")
|
||
#:configuration-file kernel-config))
|
||
@end lisp
|
||
|
||
Any keys which are not assigned values inherit their default value from the
|
||
@code{make-linux-libre} definition. When comparing the two snippets above,
|
||
notice the code comment that refers to @code{#:configuration-file}. Because of
|
||
this, it is not actually easy to include a custom kernel configuration from the
|
||
definition, but don't worry, there are other ways to work with what we do have.
|
||
|
||
There are two ways to create a kernel with a custom kernel configuration. The
|
||
first is to provide a standard @file{.config} file during the build process by
|
||
including an actual @file{.config} file as a native input to our custom
|
||
kernel. The following is a snippet from the custom @code{'configure} phase of
|
||
the @code{make-linux-libre} package definition:
|
||
|
||
@lisp
|
||
(let ((build (assoc-ref %standard-phases 'build))
|
||
(config (assoc-ref (or native-inputs inputs) "kconfig")))
|
||
|
||
;; Use a custom kernel configuration file or a default
|
||
;; configuration file.
|
||
(if config
|
||
(begin
|
||
(copy-file config ".config")
|
||
(chmod ".config" #o666))
|
||
(invoke "make" ,defconfig)))
|
||
@end lisp
|
||
|
||
Below is a sample kernel package. The @code{linux-libre} package is nothing
|
||
special and can be inherited from and have its fields overridden like any
|
||
other package:
|
||
|
||
@lisp
|
||
(define-public linux-libre/E2140
|
||
(package
|
||
(inherit linux-libre)
|
||
(native-inputs
|
||
`(("kconfig" ,(local-file "E2140.config"))
|
||
,@@(alist-delete "kconfig"
|
||
(package-native-inputs linux-libre))))))
|
||
@end lisp
|
||
|
||
In the same directory as the file defining @code{linux-libre-E2140} is a file
|
||
named @file{E2140.config}, which is an actual kernel configuration file. The
|
||
@code{defconfig} keyword of @code{make-linux-libre} is left blank here, so the
|
||
only kernel configuration in the package is the one which was included in the
|
||
@code{native-inputs} field.
|
||
|
||
The second way to create a custom kernel is to pass a new value to the
|
||
@code{extra-options} keyword of the @code{make-linux-libre} procedure. The
|
||
@code{extra-options} keyword works with another function defined right below
|
||
it:
|
||
|
||
@lisp
|
||
(define %default-extra-linux-options
|
||
`(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
|
||
("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #true)
|
||
;; Modules required for initrd:
|
||
("CONFIG_NET_9P" . m)
|
||
("CONFIG_NET_9P_VIRTIO" . m)
|
||
("CONFIG_VIRTIO_BLK" . m)
|
||
("CONFIG_VIRTIO_NET" . m)
|
||
("CONFIG_VIRTIO_PCI" . m)
|
||
("CONFIG_VIRTIO_BALLOON" . m)
|
||
("CONFIG_VIRTIO_MMIO" . m)
|
||
("CONFIG_FUSE_FS" . m)
|
||
("CONFIG_CIFS" . m)
|
||
("CONFIG_9P_FS" . m)))
|
||
|
||
(define (config->string options)
|
||
(string-join (map (match-lambda
|
||
((option . 'm)
|
||
(string-append option "=m"))
|
||
((option . #true)
|
||
(string-append option "=y"))
|
||
((option . #false)
|
||
(string-append option "=n")))
|
||
options)
|
||
"\n"))
|
||
@end lisp
|
||
|
||
And in the custom configure script from the `make-linux-libre` package:
|
||
|
||
@lisp
|
||
;; Appending works even when the option wasn't in the
|
||
;; file. The last one prevails if duplicated.
|
||
(let ((port (open-file ".config" "a"))
|
||
(extra-configuration ,(config->string extra-options)))
|
||
(display extra-configuration port)
|
||
(close-port port))
|
||
|
||
(invoke "make" "oldconfig")
|
||
@end lisp
|
||
|
||
So by not providing a configuration-file the @file{.config} starts blank, and
|
||
then we write into it the collection of flags that we want. Here's another
|
||
custom kernel:
|
||
|
||
@lisp
|
||
(define %macbook41-full-config
|
||
(append %macbook41-config-options
|
||
%file-systems
|
||
%efi-support
|
||
%emulation
|
||
(@@@@ (gnu packages linux) %default-extra-linux-options)))
|
||
|
||
(define-public linux-libre-macbook41
|
||
;; XXX: Access the internal 'make-linux-libre*' procedure, which is
|
||
;; private and unexported, and is liable to change in the future.
|
||
((@@@@ (gnu packages linux) make-linux-libre*)
|
||
(@@@@ (gnu packages linux) linux-libre-version)
|
||
(@@@@ (gnu packages linux) linux-libre-gnu-revision)
|
||
(@@@@ (gnu packages linux) linux-libre-source)
|
||
'("x86_64-linux")
|
||
#:extra-version "macbook41"
|
||
#:extra-options %macbook41-config-options))
|
||
@end lisp
|
||
|
||
In the above example @code{%file-systems} is a collection of flags enabling
|
||
different file system support, @code{%efi-support} enables EFI support and
|
||
@code{%emulation} enables a x86_64-linux machine to act in 32-bit mode also.
|
||
@code{%default-extra-linux-options} are the ones quoted above, which had to be
|
||
added in since they were replaced in the @code{extra-options} keyword.
|
||
|
||
This all sounds like it should be doable, but how does one even know which
|
||
modules are required for a particular system? Two places that can be helpful
|
||
in trying to answer this question is the
|
||
@uref{https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel, Gentoo
|
||
Handbook} and the
|
||
@uref{https://www.kernel.org/doc/html/latest/admin-guide/README.html?highlight=localmodconfig,
|
||
documentation from the kernel itself}. From the kernel documentation, it
|
||
seems that @code{make localmodconfig} is the command we want.
|
||
|
||
In order to actually run @code{make localmodconfig} we first need to get and
|
||
unpack the kernel source code:
|
||
|
||
@example shell
|
||
tar xf $(guix build linux-libre --source)
|
||
@end example
|
||
|
||
Once inside the directory containing the source code run @code{touch .config}
|
||
to create an initial, empty @file{.config} to start with. @code{make
|
||
localmodconfig} works by seeing what you already have in @file{.config} and
|
||
letting you know what you're missing. If the file is blank then you're
|
||
missing everything. The next step is to run:
|
||
|
||
@example shell
|
||
guix shell -D linux-libre -- make localmodconfig
|
||
@end example
|
||
|
||
and note the output. Do note that the @file{.config} file is still empty.
|
||
The output generally contains two types of warnings. The first start with
|
||
"WARNING" and can actually be ignored in our case. The second read:
|
||
|
||
@example shell
|
||
module pcspkr did not have configs CONFIG_INPUT_PCSPKR
|
||
@end example
|
||
|
||
For each of these lines, copy the @code{CONFIG_XXXX_XXXX} portion into the
|
||
@file{.config} in the directory, and append @code{=m}, so in the end it looks
|
||
like this:
|
||
|
||
@example shell
|
||
CONFIG_INPUT_PCSPKR=m
|
||
CONFIG_VIRTIO=m
|
||
@end example
|
||
|
||
After copying all the configuration options, run @code{make localmodconfig}
|
||
again to make sure that you don't have any output starting with ``module''.
|
||
After all of these machine specific modules there are a couple more left that
|
||
are also needed. @code{CONFIG_MODULES} is necessary so that you can build and
|
||
load modules separately and not have everything built into the kernel.
|
||
@code{CONFIG_BLK_DEV_SD} is required for reading from hard drives. It is
|
||
possible that there are other modules which you will need.
|
||
|
||
This post does not aim to be a guide to configuring your own kernel however,
|
||
so if you do decide to build a custom kernel you'll have to seek out other
|
||
guides to create a kernel which is just right for your needs.
|
||
|
||
The second way to setup the kernel configuration makes more use of Guix's
|
||
features and allows you to share configuration segments between different
|
||
kernels. For example, all machines using EFI to boot have a number of EFI
|
||
configuration flags that they need. It is likely that all the kernels will
|
||
share a list of file systems to support. By using variables it is easier to
|
||
see at a glance what features are enabled and to make sure you don't have
|
||
features in one kernel but missing in another.
|
||
|
||
Left undiscussed however, is Guix's initrd and its customization. It is
|
||
likely that you'll need to modify the initrd on a machine using a custom
|
||
kernel, since certain modules which are expected to be built may not be
|
||
available for inclusion into the initrd.
|
||
|
||
@node Guix System Image API
|
||
@section Guix System Image API
|
||
|
||
Historically, Guix System is centered around an @code{operating-system}
|
||
structure. This structure contains various fields ranging from the
|
||
bootloader and kernel declaration to the services to install.
|
||
|
||
Depending on the target machine, that can go from a standard
|
||
@code{x86_64} machine to a small ARM single board computer such as the
|
||
Pine64, the image constraints can vary a lot. The hardware
|
||
manufacturers will impose different image formats with various partition
|
||
sizes and offsets.
|
||
|
||
To create images suitable for all those machines, a new abstraction is
|
||
necessary: that's the goal of the @code{image} record. This record
|
||
contains all the required information to be transformed into a
|
||
standalone image, that can be directly booted on any target machine.
|
||
|
||
@lisp
|
||
(define-record-type* <image>
|
||
image make-image
|
||
image?
|
||
(name image-name ;symbol
|
||
(default #f))
|
||
(format image-format) ;symbol
|
||
(target image-target
|
||
(default #f))
|
||
(size image-size ;size in bytes as integer
|
||
(default 'guess))
|
||
(operating-system image-operating-system ;<operating-system>
|
||
(default #f))
|
||
(partitions image-partitions ;list of <partition>
|
||
(default '()))
|
||
(compression? image-compression? ;boolean
|
||
(default #t))
|
||
(volatile-root? image-volatile-root? ;boolean
|
||
(default #t))
|
||
(substitutable? image-substitutable? ;boolean
|
||
(default #t)))
|
||
@end lisp
|
||
|
||
This record contains the operating-system to instantiate. The
|
||
@code{format} field defines the image type and can be @code{efi-raw},
|
||
@code{qcow2} or @code{iso9660} for instance. In the future, it could be
|
||
extended to @code{docker} or other image types.
|
||
|
||
A new directory in the Guix sources is dedicated to images definition. For now
|
||
there are four files:
|
||
|
||
@itemize @bullet
|
||
@item @file{gnu/system/images/hurd.scm}
|
||
@item @file{gnu/system/images/pine64.scm}
|
||
@item @file{gnu/system/images/novena.scm}
|
||
@item @file{gnu/system/images/pinebook-pro.scm}
|
||
@end itemize
|
||
|
||
Let's have a look to @file{pine64.scm}. It contains the
|
||
@code{pine64-barebones-os} variable which is a minimal definition of an
|
||
operating-system dedicated to the @b{Pine A64 LTS} board.
|
||
|
||
@lisp
|
||
(define pine64-barebones-os
|
||
(operating-system
|
||
(host-name "vignemale")
|
||
(timezone "Europe/Paris")
|
||
(locale "en_US.utf8")
|
||
(bootloader (bootloader-configuration
|
||
(bootloader u-boot-pine64-lts-bootloader)
|
||
(targets '("/dev/vda"))))
|
||
(initrd-modules '())
|
||
(kernel linux-libre-arm64-generic)
|
||
(file-systems (cons (file-system
|
||
(device (file-system-label "my-root"))
|
||
(mount-point "/")
|
||
(type "ext4"))
|
||
%base-file-systems))
|
||
(services (cons (service agetty-service-type
|
||
(agetty-configuration
|
||
(extra-options '("-L")) ; no carrier detect
|
||
(baud-rate "115200")
|
||
(term "vt100")
|
||
(tty "ttyS0")))
|
||
%base-services))))
|
||
@end lisp
|
||
|
||
The @code{kernel} and @code{bootloader} fields are pointing to packages
|
||
dedicated to this board.
|
||
|
||
Right below, the @code{pine64-image-type} variable is also defined.
|
||
|
||
@lisp
|
||
(define pine64-image-type
|
||
(image-type
|
||
(name 'pine64-raw)
|
||
(constructor (cut image-with-os arm64-disk-image <>))))
|
||
@end lisp
|
||
|
||
It's using a record we haven't talked about yet, the @code{image-type} record,
|
||
defined this way:
|
||
|
||
@lisp
|
||
(define-record-type* <image-type>
|
||
image-type make-image-type
|
||
image-type?
|
||
(name image-type-name) ;symbol
|
||
(constructor image-type-constructor)) ;<operating-system> -> <image>
|
||
@end lisp
|
||
|
||
The main purpose of this record is to associate a name to a procedure
|
||
transforming an @code{operating-system} to an image. To understand why
|
||
it is necessary, let's have a look to the command producing an image
|
||
from an @code{operating-system} configuration file:
|
||
|
||
@example
|
||
guix system image my-os.scm
|
||
@end example
|
||
|
||
This command expects an @code{operating-system} configuration but how
|
||
should we indicate that we want an image targeting a Pine64 board? We
|
||
need to provide an extra information, the @code{image-type}, by passing
|
||
the @code{--image-type} or @code{-t} flag, this way:
|
||
|
||
@example
|
||
guix system image --image-type=pine64-raw my-os.scm
|
||
@end example
|
||
|
||
This @code{image-type} parameter points to the @code{pine64-image-type}
|
||
defined above. Hence, the @code{operating-system} declared in
|
||
@code{my-os.scm} will be applied the @code{(cut image-with-os
|
||
arm64-disk-image <>)} procedure to turn it into an image.
|
||
|
||
The resulting image looks like:
|
||
|
||
@lisp
|
||
(image
|
||
(format 'disk-image)
|
||
(target "aarch64-linux-gnu")
|
||
(operating-system my-os)
|
||
(partitions
|
||
(list (partition
|
||
(inherit root-partition)
|
||
(offset root-offset)))))
|
||
@end lisp
|
||
|
||
which is the aggregation of the @code{operating-system} defined in
|
||
@code{my-os.scm} to the @code{arm64-disk-image} record.
|
||
|
||
But enough Scheme madness. What does this image API bring to the Guix user?
|
||
|
||
One can run:
|
||
|
||
@example
|
||
mathieu@@cervin:~$ guix system --list-image-types
|
||
The available image types are:
|
||
|
||
- unmatched-raw
|
||
- rock64-raw
|
||
- pinebook-pro-raw
|
||
- pine64-raw
|
||
- novena-raw
|
||
- hurd-raw
|
||
- hurd-qcow2
|
||
- qcow2
|
||
- iso9660
|
||
- uncompressed-iso9660
|
||
- tarball
|
||
- efi-raw
|
||
- mbr-raw
|
||
- docker
|
||
- wsl2
|
||
- raw-with-offset
|
||
- efi32-raw
|
||
@end example
|
||
|
||
and by writing an @code{operating-system} file based on
|
||
@code{pine64-barebones-os}, you can customize your image to your
|
||
preferences in a file (@file{my-pine-os.scm}) like this:
|
||
|
||
@lisp
|
||
(use-modules (gnu services linux)
|
||
(gnu system images pine64))
|
||
|
||
(let ((base-os pine64-barebones-os))
|
||
(operating-system
|
||
(inherit base-os)
|
||
(timezone "America/Indiana/Indianapolis")
|
||
(services
|
||
(cons
|
||
(service earlyoom-service-type
|
||
(earlyoom-configuration
|
||
(prefer-regexp "icecat|chromium")))
|
||
(operating-system-user-services base-os)))))
|
||
@end lisp
|
||
|
||
run:
|
||
|
||
@example
|
||
guix system image --image-type=pine64-raw my-pine-os.scm
|
||
@end example
|
||
|
||
or,
|
||
|
||
@example
|
||
guix system image --image-type=hurd-raw my-hurd-os.scm
|
||
@end example
|
||
|
||
to get an image that can be written directly to a hard drive and booted
|
||
from.
|
||
|
||
Without changing anything to @code{my-hurd-os.scm}, calling:
|
||
|
||
@example
|
||
guix system image --image-type=hurd-qcow2 my-hurd-os.scm
|
||
@end example
|
||
|
||
will instead produce a Hurd QEMU image.
|
||
|
||
@node Using security keys
|
||
@section Using security keys
|
||
@cindex 2FA, two-factor authentication
|
||
@cindex U2F, Universal 2nd Factor
|
||
@cindex security key, configuration
|
||
|
||
The use of security keys can improve your security by providing a second
|
||
authentication source that cannot be easily stolen or copied, at least
|
||
for a remote adversary (something that you have), to the main secret (a
|
||
passphrase -- something that you know), reducing the risk of
|
||
impersonation.
|
||
|
||
The example configuration detailed below showcases what minimal
|
||
configuration needs to be made on your Guix System to allow the use of a
|
||
Yubico security key. It is hoped the configuration can be useful for
|
||
other security keys as well, with minor adjustments.
|
||
|
||
@subsection Configuration for use as a two-factor authenticator (2FA)
|
||
|
||
To be usable, the udev rules of the system should be extended with
|
||
key-specific rules. The following shows how to extend your udev rules
|
||
with the @file{lib/udev/rules.d/70-u2f.rules} udev rule file provided by
|
||
the @code{libfido2} package from the @code{(gnu packages
|
||
security-token)} module and add your user to the @samp{"plugdev"} group
|
||
it uses:
|
||
|
||
@lisp
|
||
(use-package-modules ... security-token ...)
|
||
...
|
||
(operating-system
|
||
...
|
||
(users (cons* (user-account
|
||
(name "your-user")
|
||
(group "users")
|
||
(supplementary-groups
|
||
'("wheel" "netdev" "audio" "video"
|
||
"plugdev")) ;<- added system group
|
||
(home-directory "/home/your-user"))
|
||
%base-user-accounts))
|
||
...
|
||
(services
|
||
(cons*
|
||
...
|
||
(udev-rules-service 'fido2 libfido2 #:groups '("plugdev")))))
|
||
@end lisp
|
||
|
||
After re-configuring your system and re-logging in your graphical
|
||
session so that the new group is in effect for your user, you can verify
|
||
that your key is usable by launching:
|
||
|
||
@example
|
||
guix shell ungoogled-chromium -- chromium chrome://settings/securityKeys
|
||
@end example
|
||
|
||
and validating that the security key can be reset via the ``Reset your
|
||
security key'' menu. If it works, congratulations, your security key is
|
||
ready to be used with applications supporting two-factor authentication
|
||
(2FA).
|
||
|
||
@subsection Disabling OTP code generation for a Yubikey
|
||
@cindex disabling yubikey OTP
|
||
If you use a Yubikey security key and are irritated by the spurious OTP
|
||
codes it generates when inadvertently touching the key (e.g. causing you
|
||
to become a spammer in the @samp{#guix} channel when discussing from
|
||
your favorite IRC client!), you can disable it via the following
|
||
@command{ykman} command:
|
||
|
||
@example
|
||
guix shell python-yubikey-manager -- ykman config usb --force --disable OTP
|
||
@end example
|
||
|
||
Alternatively, you could use the @command{ykman-gui} command provided by
|
||
the @code{yubikey-manager-qt} package and either wholly disable the
|
||
@samp{OTP} application for the USB interface or, from the
|
||
@samp{Applications -> OTP} view, delete the slot 1 configuration, which
|
||
comes pre-configured with the Yubico OTP application.
|
||
|
||
@subsection Requiring a Yubikey to open a KeePassXC database
|
||
@cindex yubikey, keepassxc integration
|
||
The KeePassXC password manager application has support for Yubikeys, but
|
||
it requires installing a udev rules for your Guix System and some
|
||
configuration of the Yubico OTP application on the key.
|
||
|
||
The necessary udev rules file comes from the
|
||
@code{yubikey-personalization} package, and can be installed like:
|
||
|
||
@lisp
|
||
(use-package-modules ... security-token ...)
|
||
...
|
||
(operating-system
|
||
...
|
||
(services
|
||
(cons*
|
||
...
|
||
(udev-rules-service 'yubikey yubikey-personalization))))
|
||
@end lisp
|
||
|
||
After reconfiguring your system (and reconnecting your Yubikey), you'll
|
||
then want to configure the OTP challenge/response application of your
|
||
Yubikey on its slot 2, which is what KeePassXC uses. It's easy to do so
|
||
via the Yubikey Manager graphical configuration tool, which can be
|
||
invoked with:
|
||
|
||
@example
|
||
guix shell yubikey-manager-qt -- ykman-gui
|
||
@end example
|
||
|
||
First, ensure @samp{OTP} is enabled under the @samp{Interfaces} tab,
|
||
then navigate to @samp{Applications -> OTP}, and click the
|
||
@samp{Configure} button under the @samp{Long Touch (Slot 2)} section.
|
||
Select @samp{Challenge-response}, input or generate a secret key, and
|
||
click the @samp{Finish} button. If you have a second Yubikey you'd like
|
||
to use as a backup, you should configure it the same way, using the
|
||
@emph{same} secret key.
|
||
|
||
Your Yubikey should now be detected by KeePassXC. It can be added to a
|
||
database by navigating to KeePassXC's @samp{Database -> Database
|
||
Security...} menu, then clicking the @samp{Add additional
|
||
protection...} button, then @samp{Add Challenge-Response}, selecting the
|
||
security key from the drop-down menu and clicking the @samp{OK} button
|
||
to complete the setup.
|
||
|
||
@node Dynamic DNS mcron job
|
||
@section Dynamic DNS mcron job
|
||
|
||
@cindex dynamic DNS, DDNS
|
||
If your @acronym{ISP, Internet Service Provider} only provides dynamic
|
||
IP addresses, it can be useful to setup a dynamic @acronym{DNS, Domain
|
||
Name System} (also known as @acronym{DDNS, Dynamic DNS}) service to
|
||
associate a static host name to a public but dynamic (often changing) IP
|
||
address. There are multiple existing services that can be used for
|
||
this; in the following mcron job, @url{https://duckdns.org, DuckDNS} is
|
||
used. It should also work with other dynamic DNS services that offer a
|
||
similar interface to update the IP address, such as
|
||
@url{https://freedns.afraid.org/}, with minor adjustments.
|
||
|
||
The mcron job is provided below, where @var{DOMAIN} should be
|
||
substituted for your own domain prefix, and the DuckDNS provided token
|
||
associated to @var{DOMAIN} added to the
|
||
@file{/etc/duckdns/@var{DOMAIN}.token} file.
|
||
|
||
@lisp
|
||
(define duckdns-job
|
||
;; Update personal domain IP every 5 minutes.
|
||
#~(job '(next-minute (range 0 60 5))
|
||
#$(program-file
|
||
"duckdns-update"
|
||
(with-extensions (list guile-gnutls) ;required by (web client)
|
||
#~(begin
|
||
(use-modules (ice-9 textual-ports)
|
||
(web client))
|
||
(let ((token (string-trim-both
|
||
(call-with-input-file "/etc/duckdns/@var{DOMAIN}.token"
|
||
get-string-all)))
|
||
(query-template (string-append "https://www.duckdns.org/"
|
||
"update?domains=@var{DOMAIN}"
|
||
"&token=~a&ip=")))
|
||
(http-get (format #f query-template token))))))
|
||
"duckdns-update"
|
||
#:user "nobody"))
|
||
@end lisp
|
||
|
||
The job then needs to be added to the list of mcron jobs for your
|
||
system, using something like:
|
||
|
||
@lisp
|
||
(operating-system
|
||
(services
|
||
(cons* (service mcron-service-type
|
||
(mcron-configuration
|
||
(jobs (list duckdns-job ...))))
|
||
...
|
||
%base-services)))
|
||
@end lisp
|
||
|
||
@node Connecting to Wireguard VPN
|
||
@section Connecting to Wireguard VPN
|
||
|
||
To connect to a Wireguard VPN server you need the kernel module to be
|
||
loaded in memory and a package providing networking tools that support
|
||
it (e.g. @code{wireguard-tools} or @code{network-manager}).
|
||
|
||
Here is a configuration example for Linux-Libre < 5.6, where the module
|
||
is out of tree and need to be loaded manually---following revisions of
|
||
the kernel have it built-in and so don't need such configuration:
|
||
|
||
@lisp
|
||
(use-modules (gnu))
|
||
(use-service-modules desktop)
|
||
(use-package-modules vpn)
|
||
|
||
(operating-system
|
||
;; …
|
||
(services (cons (simple-service 'wireguard-module
|
||
kernel-module-loader-service-type
|
||
'("wireguard"))
|
||
%desktop-services))
|
||
(packages (cons wireguard-tools %base-packages))
|
||
(kernel-loadable-modules (list wireguard-linux-compat)))
|
||
@end lisp
|
||
|
||
After reconfiguring and restarting your system you can either use
|
||
Wireguard tools or NetworkManager to connect to a VPN server.
|
||
|
||
@subsection Using Wireguard tools
|
||
|
||
To test your Wireguard setup it is convenient to use @command{wg-quick}.
|
||
Just give it a configuration file @command{wg-quick up ./wg0.conf}; or
|
||
put that file in @file{/etc/wireguard} and run @command{wg-quick up wg0}
|
||
instead.
|
||
|
||
@quotation Note
|
||
Be warned that the author described this command as a: “[…] very quick
|
||
and dirty bash script […]”.
|
||
@end quotation
|
||
|
||
@subsection Using NetworkManager
|
||
|
||
Thanks to NetworkManager support for Wireguard we can connect to our VPN
|
||
using @command{nmcli} command. Up to this point this guide assumes that
|
||
you're using Network Manager service provided by
|
||
@code{%desktop-services}. Ortherwise you need to adjust your services
|
||
list to load @code{network-manager-service-type} and reconfigure your
|
||
Guix system.
|
||
|
||
To import your VPN configuration execute nmcli import command:
|
||
|
||
@example shell
|
||
# nmcli connection import type wireguard file wg0.conf
|
||
Connection 'wg0' (edbee261-aa5a-42db-b032-6c7757c60fde) successfully added
|
||
@end example
|
||
|
||
This will create a configuration file in
|
||
@file{/etc/NetworkManager/wg0.nmconnection}. Next connect to the
|
||
Wireguard server:
|
||
|
||
@example shell
|
||
$ nmcli connection up wg0
|
||
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
|
||
@end example
|
||
|
||
By default NetworkManager will connect automatically on system boot. To
|
||
change that behaviour you need to edit your config:
|
||
|
||
@example shell
|
||
# nmcli connection modify wg0 connection.autoconnect no
|
||
@end example
|
||
|
||
For more specific information about NetworkManager and wireguard
|
||
@uref{https://blogs.gnome.org/thaller/2019/03/15/wireguard-in-networkmanager/,see
|
||
this post by thaller}.
|
||
|
||
@node Customizing a Window Manager
|
||
@section Customizing a Window Manager
|
||
@cindex wm
|
||
|
||
@menu
|
||
* StumpWM::
|
||
* Session lock::
|
||
@end menu
|
||
|
||
@node StumpWM
|
||
@subsection StumpWM
|
||
@cindex stumpwm
|
||
|
||
You could install StumpWM with a Guix system by adding
|
||
@code{stumpwm} and optionally @code{`(,stumpwm "lib")}
|
||
packages to a system configuration file, e.g.@: @file{/etc/config.scm}.
|
||
|
||
An example configuration can look like this:
|
||
|
||
@lisp
|
||
(use-modules (gnu))
|
||
(use-package-modules wm)
|
||
|
||
(operating-system
|
||
;; …
|
||
(packages (append (list sbcl stumpwm `(,stumpwm "lib"))
|
||
%base-packages)))
|
||
@end lisp
|
||
|
||
@cindex stumpwm fonts
|
||
By default StumpWM uses X11 fonts, which could be small or pixelated on
|
||
your system. You could fix this by installing StumpWM contrib Lisp
|
||
module @code{sbcl-ttf-fonts}, adding it to Guix system packages:
|
||
|
||
@lisp
|
||
(use-modules (gnu))
|
||
(use-package-modules fonts wm)
|
||
|
||
(operating-system
|
||
;; …
|
||
(packages (append (list sbcl stumpwm `(,stumpwm "lib"))
|
||
sbcl-ttf-fonts font-dejavu %base-packages)))
|
||
@end lisp
|
||
|
||
Then you need to add the following code to a StumpWM configuration file
|
||
@file{~/.stumpwm.d/init.lisp}:
|
||
|
||
@lisp
|
||
(require :ttf-fonts)
|
||
(setf xft:*font-dirs* '("/run/current-system/profile/share/fonts/"))
|
||
(setf clx-truetype:+font-cache-filename+ (concat (getenv "HOME") "/.fonts/font-cache.sexp"))
|
||
(xft:cache-fonts)
|
||
(set-font (make-instance 'xft:font :family "DejaVu Sans Mono" :subfamily "Book" :size 11))
|
||
@end lisp
|
||
|
||
@node Session lock
|
||
@subsection Session lock
|
||
@cindex sessionlock
|
||
|
||
Depending on your environment, locking the screen of your session might come built in
|
||
or it might be something you have to set up yourself. If you use a desktop environment
|
||
like GNOME or KDE, it's usually built in. If you use a plain window manager like
|
||
StumpWM or EXWM, you might have to set it up yourself.
|
||
|
||
@menu
|
||
* Xorg::
|
||
@end menu
|
||
|
||
@node Xorg
|
||
@subsubsection Xorg
|
||
|
||
If you use Xorg, you can use the utility
|
||
@uref{https://www.mankier.com/1/xss-lock, xss-lock} to lock the screen of your session.
|
||
xss-lock is triggered by DPMS which since Xorg 1.8 is auto-detected and enabled if
|
||
ACPI is also enabled at kernel runtime.
|
||
|
||
To use xss-lock, you can simple execute it and put it into the background before
|
||
you start your window manager from e.g. your @file{~/.xsession}:
|
||
|
||
@example
|
||
xss-lock -- slock &
|
||
exec stumpwm
|
||
@end example
|
||
|
||
In this example, xss-lock uses @code{slock} to do the actual locking of the screen when
|
||
it determines it's appropriate, like when you suspend your device.
|
||
|
||
For slock to be allowed to be a screen locker for the graphical session, it needs to
|
||
be made setuid-root so it can authenticate users, and it needs a PAM service. This
|
||
can be achieved by adding the following service to your @file{config.scm}:
|
||
|
||
@lisp
|
||
(service screen-locker-services-type
|
||
(screen-locker-configuration
|
||
(name "slock")
|
||
(program (file-append slock "/bin/slock"))))
|
||
@end lisp
|
||
|
||
If you manually lock your screen, e.g. by directly calling slock when you want to lock
|
||
your screen but not suspend it, it's a good idea to notify xss-lock about this so no
|
||
confusion occurs. This can be done by executing @code{xset s activate} immediately
|
||
before you execute slock.
|
||
|
||
@node Running Guix on a Linode Server
|
||
@section Running Guix on a Linode Server
|
||
@cindex linode, Linode
|
||
|
||
To run Guix on a server hosted by @uref{https://www.linode.com, Linode},
|
||
start with a recommended Debian server. We recommend using the default
|
||
distro as a way to bootstrap Guix. Create your SSH keys.
|
||
|
||
@example
|
||
ssh-keygen
|
||
@end example
|
||
|
||
Be sure to add your SSH key for easy login to the remote server.
|
||
This is trivially done via Linode's graphical interface for adding
|
||
SSH keys. Go to your profile and click add SSH Key.
|
||
Copy into it the output of:
|
||
|
||
@example
|
||
cat ~/.ssh/<username>_rsa.pub
|
||
@end example
|
||
|
||
Power the Linode down.
|
||
|
||
In the Linode's Storage tab, resize the Debian disk to be smaller.
|
||
30 GB free space is recommended. Then click "Add a disk", and fill
|
||
out the form with the following:
|
||
|
||
@itemize @bullet
|
||
@item
|
||
Label: "Guix"
|
||
|
||
@item
|
||
Filesystem: ext4
|
||
|
||
@item
|
||
Set it to the remaining size
|
||
@end itemize
|
||
|
||
In the Configurations tab, press "Edit" on the default Debian profile.
|
||
Under "Block Device Assignment" click "Add a Device". It should be
|
||
@file{/dev/sdc} and you can select the "Guix" disk. Save Changes.
|
||
|
||
Now "Add a Configuration", with the following:
|
||
@itemize @bullet
|
||
@item
|
||
Label: Guix
|
||
|
||
@item
|
||
Kernel:GRUB 2 (it's at the bottom! This step is @b{IMPORTANT!})
|
||
|
||
@item
|
||
Block device assignment:
|
||
|
||
@item
|
||
@file{/dev/sda}: Guix
|
||
|
||
@item
|
||
@file{/dev/sdb}: swap
|
||
|
||
@item
|
||
Root device: @file{/dev/sda}
|
||
|
||
@item
|
||
Turn off all the filesystem/boot helpers
|
||
@end itemize
|
||
|
||
Now power it back up, booting with the Debian configuration. Once it's
|
||
running, ssh to your server via @code{ssh
|
||
root@@@var{<your-server-IP-here>}}. (You can find your server IP address in
|
||
your Linode Summary section.) Now you can run the "install guix from
|
||
@pxref{Binary Installation,,, guix, GNU Guix}" steps:
|
||
|
||
@example
|
||
sudo apt-get install gpg
|
||
wget https://sv.gnu.org/people/viewgpg.php?user_id=15145 -qO - | gpg --import -
|
||
wget https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
|
||
chmod +x guix-install.sh
|
||
./guix-install.sh
|
||
guix pull
|
||
@end example
|
||
|
||
Now it's time to write out a config for the server. The key information
|
||
is below. Save the resulting file as @file{guix-config.scm}.
|
||
|
||
@lisp
|
||
(use-modules (gnu)
|
||
(guix modules))
|
||
(use-service-modules networking
|
||
ssh)
|
||
(use-package-modules admin
|
||
certs
|
||
package-management
|
||
ssh
|
||
tls)
|
||
|
||
(operating-system
|
||
(host-name "my-server")
|
||
(timezone "America/New_York")
|
||
(locale "en_US.UTF-8")
|
||
;; This goofy code will generate the grub.cfg
|
||
;; without installing the grub bootloader on disk.
|
||
(bootloader (bootloader-configuration
|
||
(bootloader
|
||
(bootloader
|
||
(inherit grub-bootloader)
|
||
(installer #~(const #true))))))
|
||
(file-systems (cons (file-system
|
||
(device "/dev/sda")
|
||
(mount-point "/")
|
||
(type "ext4"))
|
||
%base-file-systems))
|
||
|
||
|
||
(swap-devices (list "/dev/sdb"))
|
||
|
||
|
||
(initrd-modules (cons "virtio_scsi" ; Needed to find the disk
|
||
%base-initrd-modules))
|
||
|
||
(users (cons (user-account
|
||
(name "janedoe")
|
||
(group "users")
|
||
;; Adding the account to the "wheel" group
|
||
;; makes it a sudoer.
|
||
(supplementary-groups '("wheel"))
|
||
(home-directory "/home/janedoe"))
|
||
%base-user-accounts))
|
||
|
||
(packages (cons* nss-certs ;for HTTPS access
|
||
openssh-sans-x
|
||
%base-packages))
|
||
|
||
(services (cons*
|
||
(service dhcp-client-service-type)
|
||
(service openssh-service-type
|
||
(openssh-configuration
|
||
(openssh openssh-sans-x)
|
||
(password-authentication? #false)
|
||
(authorized-keys
|
||
`(("janedoe" ,(local-file "janedoe_rsa.pub"))
|
||
("root" ,(local-file "janedoe_rsa.pub"))))))
|
||
%base-services)))
|
||
@end lisp
|
||
|
||
Replace the following fields in the above configuration:
|
||
@lisp
|
||
(host-name "my-server") ; replace with your server name
|
||
; if you chose a linode server outside the U.S., then
|
||
; use tzselect to find a correct timezone string
|
||
(timezone "America/New_York") ; if needed replace timezone
|
||
(name "janedoe") ; replace with your username
|
||
("janedoe" ,(local-file "janedoe_rsa.pub")) ; replace with your ssh key
|
||
("root" ,(local-file "janedoe_rsa.pub")) ; replace with your ssh key
|
||
@end lisp
|
||
|
||
The last line in the above example lets you log into the server as root
|
||
and set the initial root password (see the note at the end of this
|
||
recipe about root login). After you have done this, you may
|
||
delete that line from your configuration and reconfigure to prevent root
|
||
login.
|
||
|
||
Copy your ssh public key (eg: @file{~/.ssh/id_rsa.pub}) as
|
||
@file{@var{<your-username-here>}_rsa.pub} and put
|
||
@file{guix-config.scm} in the same directory. In a new terminal run
|
||
these commands.
|
||
|
||
@example
|
||
sftp root@@<remote server ip address>
|
||
put /path/to/files/<username>_rsa.pub .
|
||
put /path/to/files/guix-config.scm .
|
||
@end example
|
||
|
||
In your first terminal, mount the guix drive:
|
||
|
||
@example
|
||
mkdir /mnt/guix
|
||
mount /dev/sdc /mnt/guix
|
||
@end example
|
||
|
||
Due to the way we set up the bootloader section of the guix-config.scm,
|
||
only the grub configuration file will be installed. So, we need to copy
|
||
over some of the other GRUB stuff already installed on the Debian system:
|
||
|
||
@example
|
||
mkdir -p /mnt/guix/boot/grub
|
||
cp -r /boot/grub/* /mnt/guix/boot/grub/
|
||
@end example
|
||
|
||
Now initialize the Guix installation:
|
||
|
||
@example
|
||
guix system init guix-config.scm /mnt/guix
|
||
@end example
|
||
|
||
Ok, power it down!
|
||
Now from the Linode console, select boot and select "Guix".
|
||
|
||
Once it boots, you should be able to log in via SSH! (The server config
|
||
will have changed though.) You may encounter an error like:
|
||
|
||
@example
|
||
$ ssh root@@<server ip address>
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
|
||
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
|
||
It is also possible that a host key has just been changed.
|
||
The fingerprint for the ECDSA key sent by the remote host is
|
||
SHA256:0B+wp33w57AnKQuHCvQP0+ZdKaqYrI/kyU7CfVbS7R4.
|
||
Please contact your system administrator.
|
||
Add correct host key in /home/joshua/.ssh/known_hosts to get rid of this message.
|
||
Offending ECDSA key in /home/joshua/.ssh/known_hosts:3
|
||
ECDSA host key for 198.58.98.76 has changed and you have requested strict checking.
|
||
Host key verification failed.
|
||
@end example
|
||
|
||
Either delete @file{~/.ssh/known_hosts} file, or delete the offending line
|
||
starting with your server IP address.
|
||
|
||
Be sure to set your password and root's password.
|
||
|
||
@example
|
||
ssh root@@<remote ip address>
|
||
passwd ; for the root password
|
||
passwd <username> ; for the user password
|
||
@end example
|
||
|
||
You may not be able to run the above commands at this point. If you
|
||
have issues remotely logging into your linode box via SSH, then you may
|
||
still need to set your root and user password initially by clicking on
|
||
the ``Launch Console'' option in your linode. Choose the ``Glish''
|
||
instead of ``Weblish''. Now you should be able to ssh into the machine.
|
||
|
||
Hooray! At this point you can shut down the server, delete the
|
||
Debian disk, and resize the Guix to the rest of the size.
|
||
Congratulations!
|
||
|
||
By the way, if you save it as a disk image right at this point, you'll
|
||
have an easy time spinning up new Guix images! You may need to
|
||
down-size the Guix image to 6144MB, to save it as an image. Then you
|
||
can resize it again to the max size.
|
||
|
||
@node Running Guix on a Kimsufi Server
|
||
@section Running Guix on a Kimsufi Server
|
||
@cindex kimsufi, Kimsufi, OVH
|
||
|
||
To run Guix on a server hosted by @uref{https://www.kimsufi.com/,
|
||
Kimsufi}, click on the netboot tab then select rescue64-pro and restart.
|
||
|
||
OVH will email you the credentials required to ssh into a Debian system.
|
||
|
||
Now you can run the "install guix from @pxref{Binary Installation,,,
|
||
guix, GNU Guix}" steps:
|
||
|
||
@example
|
||
wget https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
|
||
chmod +x guix-install.sh
|
||
./guix-install.sh
|
||
guix pull
|
||
@end example
|
||
|
||
Partition the drives and format them, first stop the raid array:
|
||
|
||
@example
|
||
mdadm --stop /dev/md127
|
||
mdadm --zero-superblock /dev/sda2 /dev/sdb2
|
||
@end example
|
||
|
||
Then wipe the disks and set up the partitions, we will create
|
||
a RAID 1 array.
|
||
|
||
@example
|
||
wipefs -a /dev/sda
|
||
wipefs -a /dev/sdb
|
||
|
||
parted /dev/sda --align=opt -s -m -- mklabel gpt
|
||
parted /dev/sda --align=opt -s -m -- \
|
||
mkpart bios_grub 1049kb 512MiB \
|
||
set 1 bios_grub on
|
||
parted /dev/sda --align=opt -s -m -- \
|
||
mkpart primary 512MiB -512MiB
|
||
set 2 raid on
|
||
parted /dev/sda --align=opt -s -m -- mkpart primary linux-swap 512MiB 100%
|
||
|
||
parted /dev/sdb --align=opt -s -m -- mklabel gpt
|
||
parted /dev/sdb --align=opt -s -m -- \
|
||
mkpart bios_grub 1049kb 512MiB \
|
||
set 1 bios_grub on
|
||
parted /dev/sdb --align=opt -s -m -- \
|
||
mkpart primary 512MiB -512MiB \
|
||
set 2 raid on
|
||
parted /dev/sdb --align=opt -s -m -- mkpart primary linux-swap 512MiB 100%
|
||
@end example
|
||
|
||
Create the array:
|
||
|
||
@example
|
||
mdadm --create /dev/md127 --level=1 --raid-disks=2 \
|
||
--metadata=0.90 /dev/sda2 /dev/sdb2
|
||
@end example
|
||
|
||
Now create file systems on the relevant partitions, first the boot
|
||
partitions:
|
||
|
||
@example
|
||
mkfs.ext4 /dev/sda1
|
||
mkfs.ext4 /dev/sdb1
|
||
@end example
|
||
|
||
Then the root partition:
|
||
|
||
@example
|
||
mkfs.ext4 /dev/md127
|
||
@end example
|
||
|
||
Initialize the swap partitions:
|
||
|
||
@example
|
||
mkswap /dev/sda3
|
||
swapon /dev/sda3
|
||
mkswap /dev/sdb3
|
||
swapon /dev/sdb3
|
||
@end example
|
||
|
||
Mount the guix drive:
|
||
|
||
@example
|
||
mkdir /mnt/guix
|
||
mount /dev/md127 /mnt/guix
|
||
@end example
|
||
|
||
Now is time to write an operating system declaration @file{os.scm} file;
|
||
here is a sample:
|
||
|
||
@lisp
|
||
(use-modules (gnu) (guix))
|
||
(use-service-modules networking ssh vpn virtualization sysctl admin mcron)
|
||
(use-package-modules ssh certs tls tmux vpn virtualization)
|
||
|
||
(operating-system
|
||
(host-name "kimsufi")
|
||
|
||
(bootloader (bootloader-configuration
|
||
(bootloader grub-bootloader)
|
||
(targets (list "/dev/sda" "/dev/sdb"))
|
||
(terminal-outputs '(console))))
|
||
|
||
;; Add a kernel module for RAID-1 (aka. "mirror").
|
||
(initrd-modules (cons* "raid1" %base-initrd-modules))
|
||
|
||
(mapped-devices
|
||
(list (mapped-device
|
||
(source (list "/dev/sda2" "/dev/sdb2"))
|
||
(target "/dev/md127")
|
||
(type raid-device-mapping))))
|
||
|
||
(swap-devices
|
||
(list (swap-space
|
||
(target "/dev/sda3"))
|
||
(swap-space
|
||
(target "/dev/sdb3"))))
|
||
|
||
(issue
|
||
;; Default contents for /etc/issue.
|
||
"\
|
||
This is the GNU system at Kimsufi. Welcome.\n")
|
||
|
||
(file-systems (cons* (file-system
|
||
(mount-point "/")
|
||
(device "/dev/md127")
|
||
(type "ext4")
|
||
(dependencies mapped-devices))
|
||
%base-file-systems))
|
||
|
||
(users (cons (user-account
|
||
(name "guix")
|
||
(comment "guix")
|
||
(group "users")
|
||
(supplementary-groups '("wheel"))
|
||
(home-directory "/home/guix"))
|
||
%base-user-accounts))
|
||
|
||
(sudoers-file
|
||
(plain-file "sudoers" "\
|
||
root ALL=(ALL) ALL
|
||
%wheel ALL=(ALL) ALL
|
||
guix ALL=(ALL) NOPASSWD:ALL\n"))
|
||
|
||
;; Globally-installed packages.
|
||
(packages (cons* tmux nss-certs gnutls wireguard-tools %base-packages))
|
||
(services
|
||
(cons*
|
||
(service static-networking-service-type
|
||
(list (static-networking
|
||
(addresses (list (network-address
|
||
(device "enp3s0")
|
||
(value "@var{server-ip-address}/24"))))
|
||
(routes (list (network-route
|
||
(destination "default")
|
||
(gateway "@var{server-gateway}"))))
|
||
(name-servers '("213.186.33.99")))))
|
||
|
||
(service unattended-upgrade-service-type)
|
||
|
||
(service openssh-service-type
|
||
(openssh-configuration
|
||
(openssh openssh-sans-x)
|
||
(permit-root-login #f)
|
||
(authorized-keys
|
||
`(("guix" ,(plain-file "@var{ssh-key-name.pub}"
|
||
"@var{ssh-public-key-content}"))))))
|
||
(modify-services %base-services
|
||
(sysctl-service-type
|
||
config =>
|
||
(sysctl-configuration
|
||
(settings (append '(("net.ipv6.conf.all.autoconf" . "0")
|
||
("net.ipv6.conf.all.accept_ra" . "0"))
|
||
%default-sysctl-settings))))))))
|
||
@end lisp
|
||
|
||
Don't forget to substitute the @var{server-ip-address},
|
||
@var{server-gateway}, @var{ssh-key-name} and
|
||
@var{ssh-public-key-content} variables with your own values.
|
||
|
||
The gateway is the last usable IP in your block so if you have a server
|
||
with an IP of @samp{37.187.79.10} then its gateway will be
|
||
@samp{37.187.79.254}.
|
||
|
||
Transfer your operating system declaration @file{os.scm} file on the
|
||
server via the @command{scp} or @command{sftp} commands.
|
||
|
||
Now all that is left is to install Guix with a @code{guix system init}
|
||
and restart.
|
||
|
||
However we first need to set up a chroot, because the root partition of
|
||
the rescue system is mounted on an aufs partition and if you try to
|
||
install Guix it will fail at the GRUB install step complaining about the
|
||
canonical path of "aufs".
|
||
|
||
Install packages that will be used in the chroot:
|
||
|
||
@example
|
||
guix install bash-static parted util-linux-with-udev coreutils guix
|
||
@end example
|
||
|
||
Then run the following to create directories needed for the chroot:
|
||
|
||
@example
|
||
cd /mnt && \
|
||
mkdir -p bin etc gnu/store root/.guix-profile/ root/.config/guix/current \
|
||
var/guix proc sys dev
|
||
@end example
|
||
|
||
Copy the host resolv.conf in the chroot:
|
||
|
||
@example
|
||
cp /etc/resolv.conf etc/
|
||
@end example
|
||
|
||
Mount block devices, the store and its database and the current guix config:
|
||
|
||
@example
|
||
mount --rbind /proc /mnt/proc
|
||
mount --rbind /sys /mnt/sys
|
||
mount --rbind /dev /mnt/dev
|
||
mount --rbind /var/guix/ var/guix/
|
||
mount --rbind /gnu/store gnu/store/
|
||
mount --rbind /root/.config/ root/.config/
|
||
mount --rbind /root/.guix-profile/bin/ bin
|
||
mount --rbind /root/.guix-profile root/.guix-profile/
|
||
@end example
|
||
|
||
Chroot in /mnt and install the system:
|
||
|
||
@example
|
||
chroot /mnt/ /bin/bash
|
||
|
||
guix system init /root/os.scm /guix
|
||
@end example
|
||
|
||
Finally, from the web user interface (UI), change @samp{netboot} to
|
||
@samp{boot to disk} and restart (also from the web UI).
|
||
|
||
Wait a few minutes and try to ssh with @code{ssh
|
||
guix@@@var{server-ip-address>} -i @var{path-to-your-ssh-key}}
|
||
|
||
You should have a Guix system up and running on Kimsufi;
|
||
congratulations!
|
||
|
||
@node Setting up a bind mount
|
||
@section Setting up a bind mount
|
||
|
||
To bind mount a file system, one must first set up some definitions
|
||
before the @code{operating-system} section of the system definition. In
|
||
this example we will bind mount a folder from a spinning disk drive to
|
||
@file{/tmp}, to save wear and tear on the primary SSD, without
|
||
dedicating an entire partition to be mounted as @file{/tmp}.
|
||
|
||
First, the source drive that hosts the folder we wish to bind mount
|
||
should be defined, so that the bind mount can depend on it.
|
||
|
||
@lisp
|
||
(define source-drive ;; "source-drive" can be named anything you want.
|
||
(file-system
|
||
(device (uuid "UUID goes here"))
|
||
(mount-point "/path-to-spinning-disk-goes-here")
|
||
(type "ext4"))) ;; Make sure to set this to the appropriate type for your drive.
|
||
@end lisp
|
||
|
||
The source folder must also be defined, so that guix will know it's not
|
||
a regular block device, but a folder.
|
||
@lisp
|
||
(define (%source-directory) "/path-to-spinning-disk-goes-here/tmp") ;; "source-directory" can be named any valid variable name.
|
||
@end lisp
|
||
|
||
Finally, inside the @code{file-systems} definition, we must add the
|
||
mount itself.
|
||
|
||
@lisp
|
||
(file-systems (cons*
|
||
|
||
...<other drives omitted for clarity>...
|
||
|
||
source-drive ;; Must match the name you gave the source drive in the earlier definition.
|
||
|
||
(file-system
|
||
(device (%source-directory)) ;; Make sure "source-directory" matches your earlier definition.
|
||
(mount-point "/tmp")
|
||
(type "none") ;; We are mounting a folder, not a partition, so this type needs to be "none"
|
||
(flags '(bind-mount))
|
||
(dependencies (list source-drive)) ;; Ensure "source-drive" matches what you've named the variable for the drive.
|
||
)
|
||
|
||
...<other drives omitted for clarity>...
|
||
|
||
))
|
||
@end lisp
|
||
|
||
@node Getting substitutes from Tor
|
||
@section Getting substitutes from Tor
|
||
|
||
Guix daemon can use a HTTP proxy to get substitutes, here we are
|
||
configuring it to get them via Tor.
|
||
|
||
@quotation Warning
|
||
@emph{Not all} Guix daemon's traffic will go through Tor! Only
|
||
HTTP/HTTPS will get proxied; FTP, Git protocol, SSH, etc connections
|
||
will still go through the clearnet. Again, this configuration isn't
|
||
foolproof some of your traffic won't get routed by Tor at all. Use it
|
||
at your own risk.
|
||
|
||
Also note that the procedure described here applies only to package
|
||
substitution. When you update your guix distribution with
|
||
@command{guix pull}, you still need to use @command{torsocks} if
|
||
you want to route the connection to guix's git repository servers
|
||
through Tor.
|
||
@end quotation
|
||
|
||
Guix's substitute server is available as a Onion service, if you want
|
||
to use it to get your substitutes through Tor configure your system as
|
||
follow:
|
||
|
||
@lisp
|
||
(use-modules (gnu))
|
||
(use-service-module base networking)
|
||
|
||
(operating-system
|
||
…
|
||
(services
|
||
(cons
|
||
(service tor-service-type
|
||
(tor-configuration
|
||
(config-file (plain-file "tor-config"
|
||
"HTTPTunnelPort 127.0.0.1:9250"))))
|
||
(modify-services %base-services
|
||
(guix-service-type
|
||
config => (guix-configuration
|
||
(inherit config)
|
||
;; ci.guix.gnu.org's Onion service
|
||
(substitute-urls
|
||
"@value{SUBSTITUTE-TOR-URL}")
|
||
(http-proxy "http://localhost:9250")))))))
|
||
@end lisp
|
||
|
||
This will keep a tor process running that provides a HTTP CONNECT tunnel
|
||
which will be used by @command{guix-daemon}. The daemon can use other
|
||
protocols than HTTP(S) to get remote resources, request using those
|
||
protocols won't go through Tor since we are only setting a HTTP tunnel
|
||
here. Note that @code{substitutes-urls} is using HTTPS and not HTTP or
|
||
it won't work, that's a limitation of Tor's tunnel; you may want to use
|
||
@command{privoxy} instead to avoid such limitations.
|
||
|
||
If you don't want to always get substitutes through Tor but using it just
|
||
some of the times, then skip the @code{guix-configuration}. When you
|
||
want to get a substitute from the Tor tunnel run:
|
||
|
||
@example
|
||
sudo herd set-http-proxy guix-daemon http://localhost:9250
|
||
guix build \
|
||
--substitute-urls=@value{SUBSTITUTE-TOR-URL} @dots{}
|
||
@end example
|
||
|
||
@node Setting up NGINX with Lua
|
||
@section Setting up NGINX with Lua
|
||
@cindex nginx, lua, openresty, resty
|
||
|
||
NGINX could be extended with Lua scripts.
|
||
|
||
Guix provides NGINX service with ability to load Lua module and specific
|
||
Lua packages, and reply to requests by evaluating Lua scripts.
|
||
|
||
The following example demonstrates system definition with configuration
|
||
to evaluate @file{index.lua} Lua script on HTTP request to
|
||
@uref{http://localhost/hello} endpoint:
|
||
|
||
@example
|
||
local shell = require "resty.shell"
|
||
|
||
local stdin = ""
|
||
local timeout = 1000 -- ms
|
||
local max_size = 4096 -- byte
|
||
|
||
local ok, stdout, stderr, reason, status =
|
||
shell.run([[/run/current-system/profile/bin/ls /tmp]], stdin, timeout, max_size)
|
||
|
||
ngx.say(stdout)
|
||
@end example
|
||
|
||
@lisp
|
||
(use-modules (gnu))
|
||
(use-service-modules #;… web)
|
||
(use-package-modules #;… lua)
|
||
(operating-system
|
||
;; …
|
||
(services
|
||
;; …
|
||
(service nginx-service-type
|
||
(nginx-configuration
|
||
(modules
|
||
(list
|
||
(file-append nginx-lua-module "/etc/nginx/modules/ngx_http_lua_module.so")))
|
||
(lua-package-path (list lua-resty-core
|
||
lua-resty-lrucache
|
||
lua-resty-signal
|
||
lua-tablepool
|
||
lua-resty-shell))
|
||
(lua-package-cpath (list lua-resty-signal))
|
||
(server-blocks
|
||
(list (nginx-server-configuration
|
||
(server-name '("localhost"))
|
||
(listen '("80"))
|
||
(root "/etc")
|
||
(locations (list
|
||
(nginx-location-configuration
|
||
(uri "/hello")
|
||
(body (list #~(format #f "content_by_lua_file ~s;"
|
||
#$(local-file "index.lua"))))))))))))))
|
||
@end lisp
|
||
|
||
@node Music Server with Bluetooth Audio
|
||
@section Music Server with Bluetooth Audio
|
||
@cindex mpd
|
||
@cindex music server, headless
|
||
@cindex bluetooth, ALSA configuration
|
||
|
||
MPD, the Music Player Daemon, is a flexible server-side application for
|
||
playing music. Client programs on different machines on the network ---
|
||
a mobile phone, a laptop, a desktop workstation --- can connect to it to
|
||
control the playback of audio files from your local music collection.
|
||
MPD decodes the audio files and plays them back on one or many outputs.
|
||
|
||
By default MPD will play to the default audio device. In the example
|
||
below we make things a little more interesting by setting up a headless
|
||
music server. There will be no graphical user interface, no Pulseaudio
|
||
daemon, and no local audio output. Instead we will configure MPD with
|
||
two outputs: a bluetooth speaker and a web server to serve audio streams
|
||
to any streaming media player.
|
||
|
||
Bluetooth is often rather frustrating to set up. You will have to pair
|
||
your Bluetooth device and make sure that the device is automatically
|
||
connected as soon as it powers on. The Bluetooth system service
|
||
returned by the @code{bluetooth-service} procedure provides the
|
||
infrastructure needed to set this up.
|
||
|
||
Reconfigure your system with at least the following services and
|
||
packages:
|
||
|
||
@lisp
|
||
(operating-system
|
||
;; …
|
||
(packages (cons* bluez bluez-alsa
|
||
%base-packages))
|
||
(services
|
||
;; …
|
||
(dbus-service #:services (list bluez-alsa))
|
||
(bluetooth-service #:auto-enable? #t)))
|
||
@end lisp
|
||
|
||
Start the @code{bluetooth} service and then use @command{bluetoothctl}
|
||
to scan for Bluetooth devices. Try to identify your Bluetooth speaker
|
||
and pick out its device ID from the resulting list of devices that is
|
||
indubitably dominated by a baffling smorgasbord of your neighbors' home
|
||
automation gizmos. This only needs to be done once:
|
||
|
||
@example
|
||
$ bluetoothctl
|
||
[NEW] Controller 00:11:22:33:95:7F BlueZ 5.40 [default]
|
||
|
||
[bluetooth]# power on
|
||
[bluetooth]# Changing power on succeeded
|
||
|
||
[bluetooth]# agent on
|
||
[bluetooth]# Agent registered
|
||
|
||
[bluetooth]# default-agent
|
||
[bluetooth]# Default agent request successful
|
||
|
||
[bluetooth]# scan on
|
||
[bluetooth]# Discovery started
|
||
[CHG] Controller 00:11:22:33:95:7F Discovering: yes
|
||
[NEW] Device AA:BB:CC:A4:AA:CD My Bluetooth Speaker
|
||
[NEW] Device 44:44:FF:2A:20:DC My Neighbor's TV
|
||
@dots{}
|
||
|
||
[bluetooth]# pair AA:BB:CC:A4:AA:CD
|
||
Attempting to pair with AA:BB:CC:A4:AA:CD
|
||
[CHG] Device AA:BB:CC:A4:AA:CD Connected: yes
|
||
|
||
[My Bluetooth Speaker]# [CHG] Device AA:BB:CC:A4:AA:CD UUIDs: 0000110b-0000-1000-8000-00xxxxxxxxxx
|
||
[CHG] Device AA:BB:CC:A4:AA:CD UUIDs: 0000110c-0000-1000-8000-00xxxxxxxxxx
|
||
[CHG] Device AA:BB:CC:A4:AA:CD UUIDs: 0000110e-0000-1000-8000-00xxxxxxxxxx
|
||
[CHG] Device AA:BB:CC:A4:AA:CD Paired: yes
|
||
Pairing successful
|
||
|
||
[CHG] Device AA:BB:CC:A4:AA:CD Connected: no
|
||
|
||
[bluetooth]#
|
||
[bluetooth]# trust AA:BB:CC:A4:AA:CD
|
||
[bluetooth]# [CHG] Device AA:BB:CC:A4:AA:CD Trusted: yes
|
||
Changing AA:BB:CC:A4:AA:CD trust succeeded
|
||
|
||
[bluetooth]#
|
||
[bluetooth]# connect AA:BB:CC:A4:AA:CD
|
||
Attempting to connect to AA:BB:CC:A4:AA:CD
|
||
[bluetooth]# [CHG] Device AA:BB:CC:A4:AA:CD RSSI: -63
|
||
[CHG] Device AA:BB:CC:A4:AA:CD Connected: yes
|
||
Connection successful
|
||
|
||
[My Bluetooth Speaker]# scan off
|
||
[CHG] Device AA:BB:CC:A4:AA:CD RSSI is nil
|
||
Discovery stopped
|
||
[CHG] Controller 00:11:22:33:95:7F Discovering: no
|
||
@end example
|
||
|
||
Congratulations, you can now automatically connect to your Bluetooth
|
||
speaker!
|
||
|
||
It is now time to configure ALSA to use the @emph{bluealsa} Bluetooth
|
||
module, so that you can define an ALSA pcm device corresponding to your
|
||
Bluetooth speaker. For a headless server using @emph{bluealsa} with a
|
||
fixed Bluetooth device is likely simpler than configuring Pulseaudio and
|
||
its stream switching behavior. We configure ALSA by crafting a custom
|
||
@code{alsa-configuration} for the @code{alsa-service-type}. The
|
||
configuration will declare a @code{pcm} type @code{bluealsa} from the
|
||
@code{bluealsa} module provided by the @code{bluez-alsa} package, and
|
||
then define a @code{pcm} device of that type for your Bluetooth speaker.
|
||
|
||
All that is left then is to make MPD send audio data to this ALSA
|
||
device. We also add a secondary MPD output that makes the currently
|
||
played audio files available as a stream through a web server on port
|
||
8080. When enabled a device on the network could listen to the audio
|
||
stream by connecting any capable media player to the HTTP server on port
|
||
8080, independent of the status of the Bluetooth speaker.
|
||
|
||
What follows is the outline of an @code{operating-system} declaration
|
||
that should accomplish the above-mentioned tasks:
|
||
|
||
@lisp
|
||
(use-modules (gnu))
|
||
(use-service-modules audio dbus sound #;… etc)
|
||
(use-package-modules audio linux #;… etc)
|
||
(operating-system
|
||
;; …
|
||
(packages (cons* bluez bluez-alsa
|
||
%base-packages))
|
||
(services
|
||
;; …
|
||
(service mpd-service-type
|
||
(mpd-configuration
|
||
(user "your-username")
|
||
(music-dir "/path/to/your/music")
|
||
(address "192.168.178.20")
|
||
(outputs (list (mpd-output
|
||
(type "alsa")
|
||
(name "MPD")
|
||
(extra-options
|
||
;; Use the same name as in the ALSA
|
||
;; configuration below.
|
||
'((device . "pcm.btspeaker"))))
|
||
(mpd-output
|
||
(type "httpd")
|
||
(name "streaming")
|
||
(enabled? #false)
|
||
(always-on? #true)
|
||
(tags? #true)
|
||
(mixer-type 'null)
|
||
(extra-options
|
||
'((encoder . "vorbis")
|
||
(port . "8080")
|
||
(bind-to-address . "192.168.178.20")
|
||
(max-clients . "0") ;no limit
|
||
(quality . "5.0")
|
||
(format . "44100:16:1"))))))))
|
||
(dbus-service #:services (list bluez-alsa))
|
||
(bluetooth-service #:auto-enable? #t)
|
||
(service alsa-service-type
|
||
(alsa-configuration
|
||
(pulseaudio? #false) ;we don't need it
|
||
(extra-options
|
||
#~(string-append "\
|
||
# Declare Bluetooth audio device type \"bluealsa\" from bluealsa module
|
||
pcm_type.bluealsa @{
|
||
lib \"" #$(file-append bluez-alsa "/lib/alsa-lib/libasound_module_pcm_bluealsa.so") "\"
|
||
@}
|
||
|
||
# Declare control device type \"bluealsa\" from the same module
|
||
ctl_type.bluealsa @{
|
||
lib \"" #$(file-append bluez-alsa "/lib/alsa-lib/libasound_module_ctl_bluealsa.so") "\"
|
||
@}
|
||
|
||
# Define the actual Bluetooth audio device.
|
||
pcm.btspeaker @{
|
||
type bluealsa
|
||
device \"AA:BB:CC:A4:AA:CD\" # unique device identifier
|
||
profile \"a2dp\"
|
||
@}
|
||
|
||
# Define an associated controller.
|
||
ctl.btspeaker @{
|
||
type bluealsa
|
||
@}
|
||
"))))))
|
||
@end lisp
|
||
|
||
Enjoy the music with the MPD client of your choice or a media player
|
||
capable of streaming via HTTP!
|
||
|
||
|
||
@c *********************************************************************
|
||
@node Containers
|
||
@chapter Containers
|
||
|
||
The kernel Linux provides a number of shared facilities that are
|
||
available to processes in the system. These facilities include a shared
|
||
view on the file system, other processes, network devices, user and
|
||
group identities, and a few others. Since Linux 3.19 a user can choose
|
||
to @emph{unshare} some of these shared facilities for selected
|
||
processes, providing them (and their child processes) with a different
|
||
view on the system.
|
||
|
||
A process with an unshared @code{mount} namespace, for example, has its
|
||
own view on the file system --- it will only be able to see directories
|
||
that have been explicitly bound in its mount namespace. A process with
|
||
its own @code{proc} namespace will consider itself to be the only
|
||
process running on the system, running as PID 1.
|
||
|
||
Guix uses these kernel features to provide fully isolated environments
|
||
and even complete Guix System containers, lightweight virtual machines
|
||
that share the host system's kernel. This feature comes in especially
|
||
handy when using Guix on a foreign distribution to prevent interference
|
||
from foreign libraries or configuration files that are available
|
||
system-wide.
|
||
|
||
@menu
|
||
* Guix Containers:: Perfectly isolated environments
|
||
* Guix System Containers:: A system inside your system
|
||
@end menu
|
||
|
||
@node Guix Containers
|
||
@section Guix Containers
|
||
|
||
The easiest way to get started is to use @command{guix shell} with the
|
||
@option{--container} option. @xref{Invoking guix shell,,, guix, GNU
|
||
Guix Reference Manual} for a reference of valid options.
|
||
|
||
The following snippet spawns a minimal shell process with most
|
||
namespaces unshared from the system. The current working directory is
|
||
visible to the process, but anything else on the file system is
|
||
unavailable. This extreme isolation can be very useful when you want to
|
||
rule out any sort of interference from environment variables, globally
|
||
installed libraries, or configuration files.
|
||
|
||
@example
|
||
guix shell --container
|
||
@end example
|
||
|
||
It is a bleak environment, barren, desolate. You will find that not
|
||
even the GNU coreutils are available here, so to explore this deserted
|
||
wasteland you need to use built-in shell commands. Even the usually
|
||
gigantic @file{/gnu/store} directory is reduced to a faint shadow of
|
||
itself.
|
||
|
||
@example sh
|
||
$ echo /gnu/store/*
|
||
/gnu/store/@dots{}-gcc-10.3.0-lib
|
||
/gnu/store/@dots{}-glibc-2.33
|
||
/gnu/store/@dots{}-bash-static-5.1.8
|
||
/gnu/store/@dots{}-ncurses-6.2.20210619
|
||
/gnu/store/@dots{}-bash-5.1.8
|
||
/gnu/store/@dots{}-profile
|
||
/gnu/store/@dots{}-readline-8.1.1
|
||
@end example
|
||
|
||
@cindex exiting a container
|
||
There isn't much you can do in an environment like this other than
|
||
exiting it. You can use @key{^D} or @command{exit} to terminate this
|
||
limited shell environment.
|
||
|
||
@cindex exposing directories, container
|
||
@cindex sharing directories, container
|
||
@cindex mapping locations, container
|
||
You can make other directories available inside of the container
|
||
environment; use @option{--expose=DIRECTORY} to bind-mount the given
|
||
directory as a read-only location inside the container, or use
|
||
@option{--share=DIRECTORY} to make the location writable. With an
|
||
additional mapping argument after the directory name you can control the
|
||
name of the directory inside the container. In the following example we
|
||
map @file{/etc} on the host system to @file{/the/host/etc} inside a
|
||
container in which the GNU coreutils are installed.
|
||
|
||
@example sh
|
||
$ guix shell --container --share=/etc=/the/host/etc coreutils
|
||
$ ls /the/host/etc
|
||
@end example
|
||
|
||
Similarly, you can prevent the current working directory from being
|
||
mapped into the container with the @option{--no-cwd} option. Another
|
||
good idea is to create a dedicated directory that will serve as the
|
||
container's home directory, and spawn the container shell from that
|
||
directory.
|
||
|
||
@cindex hide system libraries, container
|
||
@cindex avoid ABI mismatch, container
|
||
On a foreign system a container environment can be used to compile
|
||
software that cannot possibly be linked with system libraries or with
|
||
the system's compiler toolchain. A common use-case in a research
|
||
context is to install packages from within an R session. Outside of a
|
||
container environment there is a good chance that the foreign compiler
|
||
toolchain and incompatible system libraries are found first, resulting
|
||
in incompatible binaries that cannot be used by R. In a container shell
|
||
this problem disappears, as system libraries and executables simply
|
||
aren't available due to the unshared @code{mount} namespace.
|
||
|
||
Let's take a comprehensive manifest providing a comfortable development
|
||
environment for use with R:
|
||
|
||
@lisp
|
||
(specifications->manifest
|
||
(list "r-minimal"
|
||
|
||
;; base packages
|
||
"bash-minimal"
|
||
"glibc-locales"
|
||
"nss-certs"
|
||
|
||
;; Common command line tools lest the container is too empty.
|
||
"coreutils"
|
||
"grep"
|
||
"which"
|
||
"wget"
|
||
"sed"
|
||
|
||
;; R markdown tools
|
||
"pandoc"
|
||
|
||
;; Toolchain and common libraries for "install.packages"
|
||
"gcc-toolchain@@10"
|
||
"gfortran-toolchain"
|
||
"gawk"
|
||
"tar"
|
||
"gzip"
|
||
"unzip"
|
||
"make"
|
||
"cmake"
|
||
"pkg-config"
|
||
"cairo"
|
||
"libxt"
|
||
"openssl"
|
||
"curl"
|
||
"zlib"))
|
||
@end lisp
|
||
|
||
Let's use this to run R inside a container environment. For convenience
|
||
we share the @code{net} namespace to use the host system's network
|
||
interfaces. Now we can build R packages from source the traditional way
|
||
without having to worry about ABI mismatch or incompatibilities.
|
||
|
||
@example sh
|
||
$ guix shell --container --network --manifest=manifest.scm -- R
|
||
|
||
R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
|
||
Copyright (C) 2022 The R Foundation for Statistical Computing
|
||
@dots{}
|
||
> e <- Sys.getenv("GUIX_ENVIRONMENT")
|
||
> Sys.setenv(GIT_SSL_CAINFO=paste0(e, "/etc/ssl/certs/ca-certificates.crt"))
|
||
> Sys.setenv(SSL_CERT_FILE=paste0(e, "/etc/ssl/certs/ca-certificates.crt"))
|
||
> Sys.setenv(SSL_CERT_DIR=paste0(e, "/etc/ssl/certs"))
|
||
> install.packages("Cairo", lib=paste0(getwd()))
|
||
@dots{}
|
||
* installing *source* package 'Cairo' ...
|
||
@dots{}
|
||
* DONE (Cairo)
|
||
|
||
The downloaded source packages are in
|
||
'/tmp/RtmpCuwdwM/downloaded_packages'
|
||
> library("Cairo", lib=getwd())
|
||
> # success!
|
||
@end example
|
||
|
||
Using container shells is fun, but they can become a little cumbersome
|
||
when you want to go beyond just a single interactive process. Some
|
||
tasks become a lot easier when they sit on the rock solid foundation of
|
||
a proper Guix System and its rich set of system services. The next
|
||
section shows you how to launch a complete Guix System inside of a
|
||
container.
|
||
|
||
|
||
@node Guix System Containers
|
||
@section Guix System Containers
|
||
|
||
The Guix System provides a wide array of interconnected system services
|
||
that are configured declaratively to form a dependable stateless GNU
|
||
System foundation for whatever tasks you throw at it. Even when using
|
||
Guix on a foreign distribution you can benefit from the design of Guix
|
||
System by running a system instance as a container. Using the same
|
||
kernel features of unshared namespaces mentioned in the previous
|
||
section, the resulting Guix System instance is isolated from the host
|
||
system and only shares file system locations that you explicitly
|
||
declare.
|
||
|
||
A Guix System container differs from the shell process created by
|
||
@command{guix shell --container} in a number of important ways. While
|
||
in a container shell the containerized process is a Bash shell process,
|
||
a Guix System container runs the Shepherd as PID 1. In a system
|
||
container all system services (@pxref{Services,,, guix, GNU Guix
|
||
Reference Manual}) are set up just as they would be on a Guix System in
|
||
a virtual machine or on bare metal---this includes daemons managed by
|
||
the GNU@tie{}Shepherd (@pxref{Shepherd Services,,, guix, GNU Guix
|
||
Reference Manual}) as well as other kinds of extensions to the operating
|
||
system (@pxref{Service Composition,,, guix, GNU Guix Reference Manual}).
|
||
|
||
The perceived increase in complexity of running a Guix System container
|
||
is easily justified when dealing with more complex applications that
|
||
have higher or just more rigid requirements on their execution
|
||
contexts---configuration files, dedicated user accounts, directories for
|
||
caches or log files, etc. In Guix System the demands of this kind of
|
||
software are satisfied through the deployment of system services.
|
||
|
||
|
||
@menu
|
||
* A Database Container::
|
||
* Container Networking::
|
||
@end menu
|
||
|
||
@node A Database Container
|
||
@subsection A Database Container
|
||
|
||
A good example might be a PostgreSQL database server. Much of the
|
||
complexity of setting up such a database server is encapsulated in this
|
||
deceptively short service declaration:
|
||
|
||
@lisp
|
||
(service postgresql-service-type
|
||
(postgresql-configuration
|
||
(postgresql postgresql-14)))
|
||
@end lisp
|
||
|
||
A complete operating system declaration for use with a Guix System
|
||
container would look something like this:
|
||
|
||
@lisp
|
||
(use-modules (gnu))
|
||
(use-package-modules databases)
|
||
(use-service-modules databases)
|
||
|
||
(operating-system
|
||
(host-name "container")
|
||
(timezone "Europe/Berlin")
|
||
(file-systems (cons (file-system
|
||
(device (file-system-label "does-not-matter"))
|
||
(mount-point "/")
|
||
(type "ext4"))
|
||
%base-file-systems))
|
||
(bootloader (bootloader-configuration
|
||
(bootloader grub-bootloader)
|
||
(targets '("/dev/sdX"))))
|
||
(services
|
||
(cons* (service postgresql-service-type
|
||
(postgresql-configuration
|
||
(postgresql postgresql-14)
|
||
(config-file
|
||
(postgresql-config-file
|
||
(log-destination "stderr")
|
||
(hba-file
|
||
(plain-file "pg_hba.conf"
|
||
"\
|
||
local all all trust
|
||
host all all 10.0.0.1/32 trust"))
|
||
(extra-config
|
||
'(("listen_addresses" "*")
|
||
("log_directory" "/var/log/postgresql")))))))
|
||
(service postgresql-role-service-type
|
||
(postgresql-role-configuration
|
||
(roles
|
||
(list (postgresql-role
|
||
(name "test")
|
||
(create-database? #t))))))
|
||
%base-services)))
|
||
@end lisp
|
||
|
||
With @code{postgresql-role-service-type} we define a role ``test'' and
|
||
create a matching database, so that we can test right away without any
|
||
further manual setup. The @code{postgresql-config-file} settings allow
|
||
a client from IP address 10.0.0.1 to connect without requiring
|
||
authentication---a bad idea in production systems, but convenient for
|
||
this example.
|
||
|
||
Let's build a script that will launch an instance of this Guix System as
|
||
a container. Write the @code{operating-system} declaration above to a
|
||
file @file{os.scm} and then use @command{guix system container} to build
|
||
the launcher. (@pxref{Invoking guix system,,, guix, GNU Guix Reference
|
||
Manual}).
|
||
|
||
@example
|
||
$ guix system container os.scm
|
||
The following derivations will be built:
|
||
/gnu/store/@dots{}-run-container.drv
|
||
@dots{}
|
||
building /gnu/store/@dots{}-run-container.drv...
|
||
/gnu/store/@dots{}-run-container
|
||
@end example
|
||
|
||
Now that we have a launcher script we can run it to spawn the new system
|
||
with a running PostgreSQL service. Note that due to some as yet
|
||
unresolved limitations we need to run the launcher as the root user, for
|
||
example with @command{sudo}.
|
||
|
||
@example
|
||
$ sudo /gnu/store/@dots{}-run-container
|
||
system container is running as PID 5983
|
||
@dots{}
|
||
@end example
|
||
|
||
Background the process with @key{Ctrl-z} followed by @command{bg}. Note
|
||
the process ID in the output; we will need it to connect to the
|
||
container later. You know what? Let's try attaching to the container
|
||
right now. We will use @command{nsenter}, a tool provided by the
|
||
@code{util-linux} package:
|
||
|
||
@example
|
||
$ guix shell util-linux
|
||
$ sudo nsenter -a -t 5983
|
||
root@@container /# pgrep -a postgres
|
||
49 /gnu/store/@dots{}-postgresql-14.4/bin/postgres -D /var/lib/postgresql/data --config-file=/gnu/store/@dots{}-postgresql.conf -p 5432
|
||
51 postgres: checkpointer
|
||
52 postgres: background writer
|
||
53 postgres: walwriter
|
||
54 postgres: autovacuum launcher
|
||
55 postgres: stats collector
|
||
56 postgres: logical replication launcher
|
||
root@@container /# exit
|
||
@end example
|
||
|
||
The PostgreSQL service is running in the container!
|
||
|
||
|
||
@node Container Networking
|
||
@subsection Container Networking
|
||
@cindex container networking
|
||
|
||
What good is a Guix System running a PostgreSQL database service as a
|
||
container when we can only talk to it with processes originating in the
|
||
container? It would be much better if we could talk to the database
|
||
over the network.
|
||
|
||
The easiest way to do this is to create a pair of connected virtual
|
||
Ethernet devices (known as @code{veth}). We move one of the devices
|
||
(@code{ceth-test}) into the @code{net} namespace of the container and
|
||
leave the other end (@code{veth-test}) of the connection on the host
|
||
system.
|
||
|
||
@example
|
||
pid=5983
|
||
ns="guix-test"
|
||
host="veth-test"
|
||
client="ceth-test"
|
||
|
||
# Attach the new net namespace "guix-test" to the container PID.
|
||
sudo ip netns attach $ns $pid
|
||
|
||
# Create the pair of devices
|
||
sudo ip link add $host type veth peer name $client
|
||
|
||
# Move the client device into the container's net namespace
|
||
sudo ip link set $client netns $ns
|
||
@end example
|
||
|
||
Then we configure the host side:
|
||
|
||
@example
|
||
sudo ip link set $host up
|
||
sudo ip addr add 10.0.0.1/24 dev $host
|
||
@end example
|
||
|
||
@dots{}and then we configure the client side:
|
||
|
||
@example
|
||
sudo ip netns exec $ns ip link set lo up
|
||
sudo ip netns exec $ns ip link set $client up
|
||
sudo ip netns exec $ns ip addr add 10.0.0.2/24 dev $client
|
||
@end example
|
||
|
||
At this point the host can reach the container at IP address 10.0.0.2,
|
||
and the container can reach the host at IP 10.0.0.1. This is all we
|
||
need to talk to the database server inside the container from the host
|
||
system on the outside.
|
||
|
||
@example
|
||
$ psql -h 10.0.0.2 -U test
|
||
psql (14.4)
|
||
Type "help" for help.
|
||
|
||
test=> CREATE TABLE hello (who TEXT NOT NULL);
|
||
CREATE TABLE
|
||
test=> INSERT INTO hello (who) VALUES ('world');
|
||
INSERT 0 1
|
||
test=> SELECT * FROM hello;
|
||
who
|
||
-------
|
||
world
|
||
(1 row)
|
||
@end example
|
||
|
||
Now that we're done with this little demonstration let's clean up:
|
||
|
||
@example
|
||
sudo kill $pid
|
||
sudo ip netns del $ns
|
||
sudo ip link del $host
|
||
@end example
|
||
|
||
|
||
@c *********************************************************************
|
||
@node Advanced package management
|
||
@chapter Advanced package management
|
||
|
||
Guix is a functional package manager that offers many features beyond
|
||
what more traditional package managers can do. To the uninitiated,
|
||
those features might not have obvious use cases at first. The purpose
|
||
of this chapter is to demonstrate some advanced package management
|
||
concepts.
|
||
|
||
@pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
|
||
reference.
|
||
|
||
@menu
|
||
* Guix Profiles in Practice:: Strategies for multiple profiles and manifests.
|
||
@end menu
|
||
|
||
@node Guix Profiles in Practice
|
||
@section Guix Profiles in Practice
|
||
|
||
Guix provides a very useful feature that may be quite foreign to newcomers:
|
||
@dfn{profiles}. They are a way to group package installations together and all users
|
||
on the same system are free to use as many profiles as they want.
|
||
|
||
Whether you're a developer or not, you may find that multiple profiles bring you
|
||
great power and flexibility. While they shift the paradigm somewhat compared to
|
||
@emph{traditional package managers}, they are very convenient to use once you've
|
||
understood how to set them up.
|
||
|
||
@quotation Note
|
||
This section is an opinionated guide on the use of multiple profiles.
|
||
It predates @command{guix shell} and its fast profile cache
|
||
(@pxref{Invoking guix shell,,, guix, GNU Guix Reference Manual}).
|
||
|
||
In many cases, you may find that using @command{guix shell} to set up
|
||
the environment you need, when you need it, is less work that
|
||
maintaining a dedicated profile. Your call!
|
||
@end quotation
|
||
|
||
If you are familiar with Python's @samp{virtualenv}, you can think of a profile as a
|
||
kind of universal @samp{virtualenv} that can hold any kind of software whatsoever, not
|
||
just Python software. Furthermore, profiles are self-sufficient: they capture
|
||
all the runtime dependencies which guarantees that all programs within a profile
|
||
will always work at any point in time.
|
||
|
||
Multiple profiles have many benefits:
|
||
|
||
@itemize
|
||
@item
|
||
Clean semantic separation of the various packages a user needs for different contexts.
|
||
|
||
@item
|
||
Multiple profiles can be made available into the environment either on login
|
||
or within a dedicated shell.
|
||
|
||
@item
|
||
Profiles can be loaded on demand. For instance, the user can use multiple
|
||
shells, each of them running different profiles.
|
||
|
||
@item
|
||
Isolation: Programs from one profile will not use programs from the other, and
|
||
the user can even install different versions of the same programs to the two
|
||
profiles without conflict.
|
||
|
||
@item
|
||
Deduplication: Profiles share dependencies that happens to be the exact same.
|
||
This makes multiple profiles storage-efficient.
|
||
|
||
@item
|
||
Reproducible: when used with declarative manifests, a profile can be fully
|
||
specified by the Guix commit that was active when it was set up. This means
|
||
that the exact same profile can be
|
||
@uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/,
|
||
set up anywhere and anytime}, with just the commit information. See the
|
||
section on @ref{Reproducible profiles}.
|
||
|
||
@item
|
||
Easier upgrades and maintenance: Multiple profiles make it easy to keep
|
||
package listings at hand and make upgrades completely frictionless.
|
||
@end itemize
|
||
|
||
Concretely, here follows some typical profiles:
|
||
|
||
@itemize
|
||
@item
|
||
The dependencies of a project you are working on.
|
||
|
||
@item
|
||
Your favourite programming language libraries.
|
||
|
||
@item
|
||
Laptop-specific programs (like @samp{powertop}) that you don't need on a desktop.
|
||
|
||
@item
|
||
@TeX{}live (this one can be really useful when you need to install just one
|
||
package for this one document you've just received over email).
|
||
|
||
@item
|
||
Games.
|
||
@end itemize
|
||
|
||
Let's dive in the set up!
|
||
|
||
@menu
|
||
* Basic setup with manifests::
|
||
* Required packages::
|
||
* Default profile::
|
||
* The benefits of manifests::
|
||
* Reproducible profiles::
|
||
@end menu
|
||
|
||
@node Basic setup with manifests
|
||
@subsection Basic setup with manifests
|
||
|
||
A Guix profile can be set up @i{via} a @dfn{manifest}. A manifest is a
|
||
snippet of Scheme code that specifies the set of packages you want to
|
||
have in your profile; it looks like this:
|
||
|
||
@lisp
|
||
(specifications->manifest
|
||
'("package-1"
|
||
;; Version 1.3 of package-2.
|
||
"package-2@@1.3"
|
||
;; The "lib" output of package-3.
|
||
"package-3:lib"
|
||
; ...
|
||
"package-N"))
|
||
@end lisp
|
||
|
||
@xref{Writing Manifests,,, guix, GNU Guix Reference Manual}, for
|
||
more information about the syntax.
|
||
|
||
We can create a manifest specification per profile and install them this way:
|
||
|
||
@example
|
||
GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
|
||
mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
|
||
guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
|
||
@end example
|
||
|
||
Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the directory
|
||
where we will store our profiles in the rest of this article.
|
||
|
||
Placing all your profiles in a single directory, with each profile getting its
|
||
own sub-directory, is somewhat cleaner. This way, each sub-directory will
|
||
contain all the symlinks for precisely one profile. Besides, ``looping over
|
||
profiles'' becomes obvious from any programming language (e.g.@: a shell script) by
|
||
simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
|
||
|
||
Note that it's also possible to loop over the output of
|
||
|
||
@example
|
||
guix package --list-profiles
|
||
@end example
|
||
|
||
although you'll probably have to filter out @file{~/.config/guix/current}.
|
||
|
||
To enable all profiles on login, add this to your @file{~/.bash_profile} (or similar):
|
||
|
||
@example
|
||
for i in $GUIX_EXTRA_PROFILES/*; do
|
||
profile=$i/$(basename "$i")
|
||
if [ -f "$profile"/etc/profile ]; then
|
||
GUIX_PROFILE="$profile"
|
||
. "$GUIX_PROFILE"/etc/profile
|
||
fi
|
||
unset profile
|
||
done
|
||
@end example
|
||
|
||
Note to Guix System users: the above reflects how your default profile
|
||
@file{~/.guix-profile} is activated from @file{/etc/profile}, that latter being loaded by
|
||
@file{~/.bashrc} by default.
|
||
|
||
You can obviously choose to only enable a subset of them:
|
||
|
||
@example
|
||
for i in "$GUIX_EXTRA_PROFILES"/my-project-1 "$GUIX_EXTRA_PROFILES"/my-project-2; do
|
||
profile=$i/$(basename "$i")
|
||
if [ -f "$profile"/etc/profile ]; then
|
||
GUIX_PROFILE="$profile"
|
||
. "$GUIX_PROFILE"/etc/profile
|
||
fi
|
||
unset profile
|
||
done
|
||
@end example
|
||
|
||
When a profile is off, it's straightforward to enable it for an individual shell
|
||
without "polluting" the rest of the user session:
|
||
|
||
@example
|
||
GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
|
||
@end example
|
||
|
||
The key to enabling a profile is to @emph{source} its @samp{etc/profile} file. This file
|
||
contains shell code that exports the right environment variables necessary to
|
||
activate the software contained in the profile. It is built automatically by
|
||
Guix and meant to be sourced.
|
||
It contains the same variables you would get if you ran:
|
||
|
||
@example
|
||
guix package --search-paths=prefix --profile=$my_profile"
|
||
@end example
|
||
|
||
Once again, see (@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual})
|
||
for the command line options.
|
||
|
||
To upgrade a profile, simply install the manifest again:
|
||
|
||
@example
|
||
guix package -m /path/to/guix-my-project-manifest.scm -p "$GUIX_EXTRA_PROFILES"/my-project/my-project
|
||
@end example
|
||
|
||
To upgrade all profiles, it's easy enough to loop over them. For instance,
|
||
assuming your manifest specifications are stored in
|
||
@file{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} being the name
|
||
of the profile (e.g.@: "project1"), you could do the following in Bourne shell:
|
||
|
||
@example
|
||
for profile in "$GUIX_EXTRA_PROFILES"/*; do
|
||
guix package --profile="$profile" --manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
|
||
done
|
||
@end example
|
||
|
||
Each profile has its own generations:
|
||
|
||
@example
|
||
guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
|
||
@end example
|
||
|
||
You can roll-back to any generation of a given profile:
|
||
|
||
@example
|
||
guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --switch-generations=17
|
||
@end example
|
||
|
||
Finally, if you want to switch to a profile without inheriting from the
|
||
current environment, you can activate it from an empty shell:
|
||
|
||
@example
|
||
env -i $(which bash) --login --noprofile --norc
|
||
. my-project/etc/profile
|
||
@end example
|
||
|
||
@node Required packages
|
||
@subsection Required packages
|
||
|
||
Activating a profile essentially boils down to exporting a bunch of
|
||
environmental variables. This is the role of the @samp{etc/profile} within the
|
||
profile.
|
||
|
||
@emph{Note: Only the environmental variables of the packages that consume them will
|
||
be set.}
|
||
|
||
For instance, @samp{MANPATH} won't be set if there is no consumer application for man
|
||
pages within the profile. So if you need to transparently access man pages once
|
||
the profile is loaded, you've got two options:
|
||
|
||
@itemize
|
||
@item
|
||
Either export the variable manually, e.g.
|
||
@example
|
||
export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH
|
||
@end example
|
||
|
||
@item
|
||
Or include @samp{man-db} to the profile manifest.
|
||
@end itemize
|
||
|
||
The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
|
||
@samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
|
||
|
||
@node Default profile
|
||
@subsection Default profile
|
||
|
||
What about the default profile that Guix keeps in @file{~/.guix-profile}?
|
||
|
||
You can assign it the role you want. Typically you would install the manifest
|
||
of the packages you want to use all the time.
|
||
|
||
Alternatively, you could keep it ``manifest-less'' for throw-away packages
|
||
that you would just use for a couple of days.
|
||
This way makes it convenient to run
|
||
|
||
@example
|
||
guix install package-foo
|
||
guix upgrade package-bar
|
||
@end example
|
||
|
||
without having to specify the path to a profile.
|
||
|
||
@node The benefits of manifests
|
||
@subsection The benefits of manifests
|
||
|
||
Manifests let you @dfn{declare} the set of packages you'd like to have
|
||
in a profile (@pxref{Writing Manifests,,, guix, GNU Guix Reference Manual}).
|
||
They are a convenient way to keep your package lists around and, say,
|
||
to synchronize them across multiple machines using a version control system.
|
||
|
||
A common complaint about manifests is that they can be slow to install when they
|
||
contain large number of packages. This is especially cumbersome when you just
|
||
want get an upgrade for one package within a big manifest.
|
||
|
||
This is one more reason to use multiple profiles, which happen to be just
|
||
perfect to break down manifests into multiple sets of semantically connected
|
||
packages. Using multiple, small profiles provides more flexibility and
|
||
usability.
|
||
|
||
Manifests come with multiple benefits. In particular, they ease maintenance:
|
||
|
||
@itemize
|
||
@item
|
||
When a profile is set up from a manifest, the manifest itself is
|
||
self-sufficient to keep a ``package listing'' around and reinstall the profile
|
||
later or on a different system. For ad-hoc profiles, we would need to
|
||
generate a manifest specification manually and maintain the package versions
|
||
for the packages that don't use the default version.
|
||
|
||
@item
|
||
@code{guix package --upgrade} always tries to update the packages that have
|
||
propagated inputs, even if there is nothing to do. Guix manifests remove this
|
||
problem.
|
||
|
||
@item
|
||
When partially upgrading a profile, conflicts may arise (due to diverging
|
||
dependencies between the updated and the non-updated packages) and they can be
|
||
annoying to resolve manually. Manifests remove this problem altogether since
|
||
all packages are always upgraded at once.
|
||
|
||
@item
|
||
As mentioned above, manifests allow for reproducible profiles, while the
|
||
imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they produce
|
||
different profiles every time even when they hold the same packages. See
|
||
@uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the matter}.
|
||
|
||
@item
|
||
Manifest specifications are usable by other @samp{guix} commands. For example, you
|
||
can run @code{guix weather -m manifest.scm} to see how many substitutes are
|
||
available, which can help you decide whether you want to try upgrading today
|
||
or wait a while. Another example: you can run @code{guix pack -m manifest.scm} to
|
||
create a pack containing all the packages in the manifest (and their
|
||
transitive references).
|
||
|
||
@item
|
||
Finally, manifests have a Scheme representation, the @samp{<manifest>} record type.
|
||
They can be manipulated in Scheme and passed to the various Guix @uref{https://en.wikipedia.org/wiki/Api, APIs}.
|
||
@end itemize
|
||
|
||
It's important to understand that while manifests can be used to declare
|
||
profiles, they are not strictly equivalent: profiles have the side effect that
|
||
they ``pin'' packages in the store, which prevents them from being
|
||
garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
|
||
and ensures that they will still be available at any point in
|
||
the future. The @command{guix shell} command also protects
|
||
recently-used profiles from garbage collection; profiles that have not
|
||
been used for a while may be garbage-collected though, along with the
|
||
packages they refer to.
|
||
|
||
To be 100% sure that a given profile will never be collected,
|
||
install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
|
||
. "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
|
||
hacking environment will be available at all times.
|
||
|
||
@emph{Security warning:} While keeping old profiles around can be convenient, keep in
|
||
mind that outdated packages may not have received the latest security fixes.
|
||
|
||
@node Reproducible profiles
|
||
@subsection Reproducible profiles
|
||
|
||
To reproduce a profile bit-for-bit, we need two pieces of information:
|
||
|
||
@itemize
|
||
@item
|
||
a manifest (@pxref{Writing Manifests,,, guix, GNU Guix Reference Manual});
|
||
@item
|
||
a Guix channel specification (@pxref{Replicating Guix,,, guix, GNU Guix
|
||
Reference Manual}).
|
||
@end itemize
|
||
|
||
Indeed, manifests alone might not be enough: different Guix versions (or
|
||
different channels) can produce different outputs for a given manifest.
|
||
|
||
You can output the Guix channel specification with @samp{guix describe
|
||
--format=channels} (@pxref{Invoking guix describe,,, guix, GNU Guix
|
||
Reference Manual}).
|
||
Save this to a file, say @samp{channel-specs.scm}.
|
||
|
||
On another computer, you can use the channel specification file and the manifest
|
||
to reproduce the exact same profile:
|
||
|
||
@example
|
||
GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
|
||
GUIX_EXTRA=$HOME/.guix-extra
|
||
|
||
mkdir -p "$GUIX_EXTRA"/my-project
|
||
guix pull --channels=channel-specs.scm --profile="$GUIX_EXTRA/my-project/guix"
|
||
|
||
mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
|
||
"$GUIX_EXTRA"/my-project/guix/bin/guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
|
||
@end example
|
||
|
||
It's safe to delete the Guix channel profile you've just installed with the
|
||
channel specification, the project profile does not depend on it.
|
||
|
||
@node Software Development
|
||
@chapter Software Development
|
||
|
||
@cindex development, with Guix
|
||
@cindex software development, with Guix
|
||
Guix is a handy tool for developers; @command{guix shell}, in
|
||
particular, gives a standalone development environment for your package,
|
||
no matter what language(s) it's written in (@pxref{Invoking guix
|
||
shell,,, guix, GNU Guix Reference Manual}). To benefit from it, you
|
||
have to initially write a package definition and have it either in Guix
|
||
proper, or in a channel, or directly in your project's source tree as a
|
||
@file{guix.scm} file. This last option is appealing: all developers
|
||
have to do to get set up is clone the project's repository and run
|
||
@command{guix shell}, with no arguments.
|
||
|
||
Development needs go beyond development environments though. How can
|
||
developers perform continuous integration of their code in Guix build
|
||
environments? How can they deliver their code straight to adventurous
|
||
users? This chapter describes a set of files developers can add to their
|
||
repository to set up Guix-based development environments, continuous
|
||
integration, and continuous delivery---all at once@footnote{This chapter
|
||
is adapted from a
|
||
@uref{https://guix.gnu.org/en/blog/2023/from-development-environments-to-continuous-integrationthe-ultimate-guide-to-software-development-with-guix/,
|
||
blog post} published in June 2023 on the Guix web site.}.
|
||
|
||
@menu
|
||
* Getting Started:: Step 0: using `guix shell'.
|
||
* Building with Guix:: Step 1: building your code.
|
||
* The Repository as a Channel:: Step 2: turning the repo in a channel.
|
||
* Package Variants:: Bonus: Defining variants.
|
||
* Setting Up Continuous Integration:: Step 3: continuous integration.
|
||
* Build Manifest:: Bonus: Manifest.
|
||
* Wrapping Up:: Recap.
|
||
@end menu
|
||
|
||
@node Getting Started
|
||
@section Getting Started
|
||
|
||
How do we go about ``Guixifying'' a repository? The first step, as we've
|
||
seen, will be to add a @file{guix.scm} at the root of the repository in
|
||
question. We'll take @uref{https://www.gnu.org/software/guile,Guile} as
|
||
an example in this chapter: it's written in Scheme (mostly) and C, and
|
||
has a number of dependencies---a C compilation tool chain, C libraries,
|
||
Autoconf and its friends, LaTeX, and so on. The resulting
|
||
@file{guix.scm} looks like the usual package definition (@pxref{Defining
|
||
Packages,,, guix, GNU Guix Reference Manual}), just without the
|
||
@code{define-public} bit:
|
||
|
||
@lisp
|
||
;; The ‘guix.scm’ file for Guile, for use by ‘guix shell’.
|
||
|
||
(use-modules (guix)
|
||
(guix build-system gnu)
|
||
((guix licenses) #:prefix license:)
|
||
(gnu packages autotools)
|
||
(gnu packages base)
|
||
(gnu packages bash)
|
||
(gnu packages bdw-gc)
|
||
(gnu packages compression)
|
||
(gnu packages flex)
|
||
(gnu packages gdb)
|
||
(gnu packages gettext)
|
||
(gnu packages gperf)
|
||
(gnu packages libffi)
|
||
(gnu packages libunistring)
|
||
(gnu packages linux)
|
||
(gnu packages pkg-config)
|
||
(gnu packages readline)
|
||
(gnu packages tex)
|
||
(gnu packages texinfo)
|
||
(gnu packages version-control))
|
||
|
||
(package
|
||
(name "guile")
|
||
(version "3.0.99-git") ;funky version number
|
||
(source #f) ;no source
|
||
(build-system gnu-build-system)
|
||
(native-inputs
|
||
(append (list autoconf
|
||
automake
|
||
libtool
|
||
gnu-gettext
|
||
flex
|
||
texinfo
|
||
texlive-base ;for "make pdf"
|
||
texlive-epsf
|
||
gperf
|
||
git
|
||
gdb
|
||
strace
|
||
readline
|
||
lzip
|
||
pkg-config)
|
||
|
||
;; When cross-compiling, a native version of Guile itself is
|
||
;; needed.
|
||
(if (%current-target-system)
|
||
(list this-package)
|
||
'())))
|
||
(inputs
|
||
(list libffi bash-minimal))
|
||
(propagated-inputs
|
||
(list libunistring libgc))
|
||
|
||
(native-search-paths
|
||
(list (search-path-specification
|
||
(variable "GUILE_LOAD_PATH")
|
||
(files '("share/guile/site/3.0")))
|
||
(search-path-specification
|
||
(variable "GUILE_LOAD_COMPILED_PATH")
|
||
(files '("lib/guile/3.0/site-ccache")))))
|
||
(synopsis "Scheme implementation intended especially for extensions")
|
||
(description
|
||
"Guile is the GNU Ubiquitous Intelligent Language for Extensions,
|
||
and it's actually a full-blown Scheme implementation!")
|
||
(home-page "https://www.gnu.org/software/guile/")
|
||
(license license:lgpl3+))
|
||
@end lisp
|
||
|
||
Quite a bit of boilerplate, but now someone who'd like to hack on Guile
|
||
now only needs to run:
|
||
|
||
@lisp
|
||
guix shell
|
||
@end lisp
|
||
|
||
That gives them a shell containing all the dependencies of Guile: those
|
||
listed above, but also @emph{implicit dependencies} such as the GCC tool
|
||
chain, GNU@ Make, sed, grep, and so on. @xref{Invoking guix shell,,,
|
||
guix, GNU Guix Reference Manual}, for more info on @command{guix shell}.
|
||
|
||
@quotation The chef's recommendation
|
||
Our suggestion is to create development environments like this:
|
||
|
||
@example
|
||
guix shell --container --link-profile
|
||
@end example
|
||
|
||
@noindent
|
||
... or, for short:
|
||
|
||
@example
|
||
guix shell -CP
|
||
@end example
|
||
|
||
That gives a shell in an isolated container, and all the dependencies
|
||
show up in @code{$HOME/.guix-profile}, which plays well with caches such
|
||
as @file{config.cache} (@pxref{Cache Files,,, autoconf, Autoconf}) and
|
||
absolute file names recorded in generated @code{Makefile}s and the
|
||
likes. The fact that the shell runs in a container brings peace of mind:
|
||
nothing but the current directory and Guile's dependencies is visible
|
||
inside the container; nothing from the system can possibly interfere
|
||
with your development.
|
||
@end quotation
|
||
|
||
@node Building with Guix
|
||
@section Level 1: Building with Guix
|
||
|
||
Now that we have a package definition (@pxref{Getting Started}), why not
|
||
also take advantage of it so we can build Guile with Guix? We had left
|
||
the @code{source} field empty, because @command{guix shell} above only
|
||
cares about the @emph{inputs} of our package---so it can set up the
|
||
development environment---not about the package itself.
|
||
|
||
To build the package with Guix, we'll need to fill out the @code{source}
|
||
field, along these lines:
|
||
|
||
@lisp
|
||
(use-modules (guix)
|
||
(guix git-download) ;for ‘git-predicate’
|
||
@dots{})
|
||
|
||
(define vcs-file?
|
||
;; Return true if the given file is under version control.
|
||
(or (git-predicate (current-source-directory))
|
||
(const #t))) ;not in a Git checkout
|
||
|
||
(package
|
||
(name "guile")
|
||
(version "3.0.99-git") ;funky version number
|
||
(source (local-file "." "guile-checkout"
|
||
#:recursive? #t
|
||
#:select? vcs-file?))
|
||
@dots{})
|
||
@end lisp
|
||
|
||
Here's what we changed compared to the previous section:
|
||
|
||
@enumerate
|
||
@item
|
||
We added @code{(guix git-download)} to our set of imported modules, so
|
||
we can use its @code{git-predicate} procedure.
|
||
@item
|
||
We defined @code{vcs-file?} as a procedure that returns true when passed
|
||
a file that is under version control. For good measure, we add a
|
||
fallback case for when we're not in a Git checkout: always return true.
|
||
@item
|
||
We set @code{source} to a
|
||
@uref{https://guix.gnu.org/manual/devel/en/html_node/G_002dExpressions.html#index-local_002dfile,@code{local-file}}---a
|
||
recursive copy of the current directory (@code{"."}), limited to files
|
||
under version control (the @code{#:select?} bit).
|
||
@end enumerate
|
||
|
||
From there on, our @file{guix.scm} file serves a second purpose: it lets
|
||
us build the software with Guix. The whole point of building with Guix
|
||
is that it's a ``clean'' build---you can be sure nothing from your
|
||
working tree or system interferes with the build result---and it lets
|
||
you test a variety of things. First, you can do a plain native build:
|
||
|
||
@example
|
||
guix build -f guix.scm
|
||
@end example
|
||
|
||
But you can also build for another system (possibly after setting up
|
||
@pxref{Daemon Offload Setup, offloading,, guix, GNU Guix Reference Manual}
|
||
or
|
||
@pxref{Virtualization Services, transparent emulation,, guix, GNU Guix
|
||
Reference Manual}):
|
||
|
||
@lisp
|
||
guix build -f guix.scm -s aarch64-linux -s riscv64-linux
|
||
@end lisp
|
||
|
||
@noindent
|
||
@dots{} or cross-compile:
|
||
|
||
@lisp
|
||
guix build -f guix.scm --target=x86_64-w64-mingw32
|
||
@end lisp
|
||
|
||
You can also use @dfn{package transformations} to test package variants
|
||
(@pxref{Package Transformation Options,,, guix, GNU Guix Reference Manual}):
|
||
|
||
@example
|
||
# What if we built with Clang instead of GCC?
|
||
guix build -f guix.scm \
|
||
--with-c-toolchain=guile@@3.0.99-git=clang-toolchain
|
||
|
||
# What about that under-tested configure flag?
|
||
guix build -f guix.scm \
|
||
--with-configure-flag=guile@@3.0.99-git=--disable-networking
|
||
@end example
|
||
|
||
Handy!
|
||
|
||
@node The Repository as a Channel
|
||
@section Level 2: The Repository as a Channel
|
||
|
||
We now have a Git repository containing (among other things) a package
|
||
definition (@pxref{Building with Guix}). Can't we turn it into a
|
||
@dfn{channel} (@pxref{Channels,,, guix, GNU Guix Reference Manual})?
|
||
After all, channels are designed to ship package definitions to users,
|
||
and that's exactly what we're doing with our @file{guix.scm}.
|
||
|
||
Turns out we can indeed turn it into a channel, but with one caveat: we
|
||
must create a separate directory for the @code{.scm} file(s) of our
|
||
channel so that @command{guix pull} doesn't load unrelated @code{.scm}
|
||
files when someone pulls the channel---and in Guile, there are lots of
|
||
them! So we'll start like this, keeping a top-level @file{guix.scm}
|
||
symlink for the sake of @command{guix shell}:
|
||
|
||
@lisp
|
||
mkdir -p .guix/modules
|
||
mv guix.scm .guix/modules/guile-package.scm
|
||
ln -s .guix/modules/guile-package.scm guix.scm
|
||
@end lisp
|
||
|
||
To make it usable as part of a channel, we need to turn our
|
||
@file{guix.scm} file into a @dfn{package module} (@pxref{Package
|
||
Modules,,, guix, GNU Guix Reference Manual}):
|
||
we do that by changing the @code{use-modules} form at the top to a
|
||
@code{define-module} form. We also need to actually @emph{export} a
|
||
package variable, with @code{define-public}, while still returning the
|
||
package value at the end of the file so we can still use
|
||
@command{guix shell} and @command{guix build -f guix.scm}. The end result
|
||
looks like this (not repeating things that haven't changed):
|
||
|
||
@lisp
|
||
(define-module (guile-package)
|
||
#:use-module (guix)
|
||
#:use-module (guix git-download) ;for ‘git-predicate’
|
||
@dots{})
|
||
|
||
(define vcs-file?
|
||
;; Return true if the given file is under version control.
|
||
(or (git-predicate (dirname (dirname (current-source-directory))))
|
||
(const #t))) ;not in a Git checkout
|
||
|
||
(define-public guile
|
||
(package
|
||
(name "guile")
|
||
(version "3.0.99-git") ;funky version number
|
||
(source (local-file "../.." "guile-checkout"
|
||
#:recursive? #t
|
||
#:select? vcs-file?))
|
||
@dots{}))
|
||
|
||
;; Return the package object define above at the end of the module.
|
||
guile
|
||
@end lisp
|
||
|
||
We need one last thing: a
|
||
@uref{https://guix.gnu.org/manual/devel/en/html_node/Package-Modules-in-a-Sub_002ddirectory.html,@code{.guix-channel}
|
||
file} so Guix knows where to look for package modules in our repository:
|
||
|
||
@lisp
|
||
;; This file lets us present this repo as a Guix channel.
|
||
|
||
(channel
|
||
(version 0)
|
||
(directory ".guix/modules")) ;look for package modules under .guix/modules/
|
||
@end lisp
|
||
|
||
To recap, we now have these files:
|
||
|
||
@lisp
|
||
.
|
||
├── .guix-channel
|
||
├── guix.scm → .guix/modules/guile-package.scm
|
||
└── .guix
|
||
└── modules
|
||
└── guile-package.scm
|
||
@end lisp
|
||
|
||
And that's it: we have a channel! (We could do better and support
|
||
@uref{https://guix.gnu.org/manual/devel/en/html_node/Specifying-Channel-Authorizations.html,@emph{channel
|
||
authentication}} so users know they're pulling genuine code. We'll spare
|
||
you the details here but it's worth considering!) Users can pull from
|
||
this channel by
|
||
@uref{https://guix.gnu.org/manual/devel/en/html_node/Specifying-Additional-Channels.html,adding
|
||
it to @code{~/.config/guix/channels.scm}}, along these lines:
|
||
|
||
@lisp
|
||
(append (list (channel
|
||
(name 'guile)
|
||
(url "https://git.savannah.gnu.org/git/guile.git")
|
||
(branch "main")))
|
||
%default-channels)
|
||
@end lisp
|
||
|
||
After running @command{guix pull}, we can see the new package:
|
||
|
||
@example
|
||
$ guix describe
|
||
Generation 264 May 26 2023 16:00:35 (current)
|
||
guile 36fd2b4
|
||
repository URL: https://git.savannah.gnu.org/git/guile.git
|
||
branch: main
|
||
commit: 36fd2b4920ae926c79b936c29e739e71a6dff2bc
|
||
guix c5bc698
|
||
repository URL: https://git.savannah.gnu.org/git/guix.git
|
||
commit: c5bc698e8922d78ed85989985cc2ceb034de2f23
|
||
$ guix package -A ^guile$
|
||
guile 3.0.99-git out,debug guile-package.scm:51:4
|
||
guile 3.0.9 out,debug gnu/packages/guile.scm:317:2
|
||
guile 2.2.7 out,debug gnu/packages/guile.scm:258:2
|
||
guile 2.2.4 out,debug gnu/packages/guile.scm:304:2
|
||
guile 2.0.14 out,debug gnu/packages/guile.scm:148:2
|
||
guile 1.8.8 out gnu/packages/guile.scm:77:2
|
||
$ guix build guile@@3.0.99-git
|
||
[@dots{}]
|
||
/gnu/store/axnzbl89yz7ld78bmx72vpqp802dwsar-guile-3.0.99-git-debug
|
||
/gnu/store/r34gsij7f0glg2fbakcmmk0zn4v62s5w-guile-3.0.99-git
|
||
@end example
|
||
|
||
That's how, as a developer, you get your software delivered directly
|
||
into the hands of users! No intermediaries, yet no loss of transparency
|
||
and provenance tracking.
|
||
|
||
With that in place, it also becomes trivial for anyone to create Docker
|
||
images, Deb/RPM packages, or a plain tarball with @command{guix pack}
|
||
(@pxref{Invoking guix pack,,, guix, GNU Guix Reference Manual}):
|
||
|
||
@example
|
||
# How about a Docker image of our Guile snapshot?
|
||
guix pack -f docker -S /bin=bin guile@@3.0.99-git
|
||
|
||
# And a relocatable RPM?
|
||
guix pack -f rpm -R -S /bin=bin guile@@3.0.99-git
|
||
@end example
|
||
|
||
@node Package Variants
|
||
@section Bonus: Package Variants
|
||
|
||
We now have an actual channel, but it contains only one package
|
||
(@pxref{The Repository as a Channel}). While we're at it, we can define
|
||
@dfn{package variants} (@pxref{Defining Package Variants,,, guix, GNU
|
||
Guix Reference Manual}) in our @file{guile-package.scm} file, variants
|
||
that we want to be able to test as Guile developers---similar to what we
|
||
did above with transformation options. We can add them like so:
|
||
|
||
@lisp
|
||
;; This is the ‘.guix/modules/guile-package.scm’ file.
|
||
|
||
(define-module (guile-package)
|
||
@dots{})
|
||
|
||
(define-public guile
|
||
@dots{})
|
||
|
||
(define (package-with-configure-flags p flags)
|
||
"Return P with FLAGS as additional 'configure' flags."
|
||
(package/inherit p
|
||
(arguments
|
||
(substitute-keyword-arguments (package-arguments p)
|
||
((#:configure-flags original-flags #~(list))
|
||
#~(append #$original-flags #$flags))))))
|
||
|
||
(define-public guile-without-threads
|
||
(package
|
||
(inherit (package-with-configure-flags guile
|
||
#~(list "--without-threads")))
|
||
(name "guile-without-threads")))
|
||
|
||
(define-public guile-without-networking
|
||
(package
|
||
(inherit (package-with-configure-flags guile
|
||
#~(list "--disable-networking")))
|
||
(name "guile-without-networking")))
|
||
|
||
|
||
;; Return the package object defined above at the end of the module.
|
||
guile
|
||
@end lisp
|
||
|
||
We can build these variants as regular packages once we've pulled the
|
||
channel. Alternatively, from a checkout of Guile, we can run a command
|
||
like this one from the top level:
|
||
|
||
@lisp
|
||
guix build -L $PWD/.guix/modules guile-without-threads
|
||
@end lisp
|
||
|
||
@node Setting Up Continuous Integration
|
||
@section Level 3: Setting Up Continuous Integration
|
||
|
||
@cindex continuous integration (CI)
|
||
The channel we defined above (@pxref{The Repository as a Channel})
|
||
becomes even more interesting once we set up
|
||
@uref{https://en.wikipedia.org/wiki/Continuous_integration,
|
||
@dfn{continuous integration}} (CI). There are several ways to do that.
|
||
|
||
You can use one of the mainstream continuous integration tools, such as
|
||
GitLab-CI. To do that, you need to make sure you run jobs in a Docker
|
||
image or virtual machine that has Guix installed. If we were to do that
|
||
in the case of Guile, we'd have a job that runs a shell command like
|
||
this one:
|
||
|
||
@lisp
|
||
guix build -L $PWD/.guix/modules guile@@3.0.99-git
|
||
@end lisp
|
||
|
||
Doing this works great and has the advantage of being easy to achieve on
|
||
your favorite CI platform.
|
||
|
||
That said, you'll really get the most of it by using
|
||
@uref{https://guix.gnu.org/en/cuirass,Cuirass}, a CI tool designed for
|
||
and tightly integrated with Guix. Using it is more work than using a
|
||
hosted CI tool because you first need to set it up, but that setup phase
|
||
is greatly simplified if you use its Guix System service
|
||
(@pxref{Continuous Integration,,, guix, GNU Guix Reference Manual}).
|
||
Going back to our example, we give Cuirass a spec file that goes like
|
||
this:
|
||
|
||
@lisp
|
||
;; Cuirass spec file to build all the packages of the ‘guile’ channel.
|
||
(list (specification
|
||
(name "guile")
|
||
(build '(channels guile))
|
||
(channels
|
||
(append (list (channel
|
||
(name 'guile)
|
||
(url "https://git.savannah.gnu.org/git/guile.git")
|
||
(branch "main")))
|
||
%default-channels))))
|
||
@end lisp
|
||
|
||
It differs from what you'd do with other CI tools in two important ways:
|
||
|
||
@itemize
|
||
@item
|
||
Cuirass knows it's tracking @emph{two} channels, @code{guile} and
|
||
@code{guix}. Indeed, our own @code{guile} package depends on many
|
||
packages provided by the @code{guix} channel---GCC, the GNU libc,
|
||
libffi, and so on. Changes to packages from the @code{guix} channel can
|
||
potentially influence our @code{guile} build and this is something we'd
|
||
like to see as soon as possible as Guile developers.
|
||
@item
|
||
Build results are not thrown away: they can be distributed as
|
||
@dfn{substitutes} so that users of our @code{guile} channel
|
||
transparently get pre-built binaries! (@pxref{Substitutes,,, guix, GNU
|
||
Guix Reference Manual}, for background info on substitutes.)
|
||
@end itemize
|
||
|
||
From a developer's viewpoint, the end result is this
|
||
@uref{https://ci.guix.gnu.org/jobset/guile,status page} listing
|
||
@emph{evaluations}: each evaluation is a combination of commits of the
|
||
@code{guix} and @code{guile} channels providing a number of
|
||
@emph{jobs}---one job per package defined in @file{guile-package.scm}
|
||
times the number of target architectures.
|
||
|
||
As for substitutes, they come for free! As an example, since our
|
||
@code{guile} jobset is built on ci.guix.gnu.org, which runs
|
||
@command{guix publish} (@pxref{Invoking guix publish,,, guix, GNU Guix
|
||
Reference Manual}) in addition to Cuirass, one automatically gets
|
||
substitutes for @code{guile} builds from ci.guix.gnu.org; no additional
|
||
work is needed for that.
|
||
|
||
@node Build Manifest
|
||
@section Bonus: Build manifest
|
||
|
||
The Cuirass spec above is convenient: it builds every package in our
|
||
channel, which includes a few variants (@pxref{Setting Up Continuous
|
||
Integration}). However, this might be insufficiently expressive in some
|
||
cases: one might want specific cross-compilation jobs, transformations,
|
||
Docker images, RPM/Deb packages, or even system tests.
|
||
|
||
To achieve that, you can write a @dfn{manifest} (@pxref{Writing
|
||
Manifests,,, guix, GNU Guix Reference Manual}). The one we have for
|
||
Guile has entries for the package variants we defined above, as well as
|
||
additional variants and cross builds:
|
||
|
||
@lisp
|
||
;; This is ‘.guix/manifest.scm’.
|
||
|
||
(use-modules (guix)
|
||
(guix profiles)
|
||
(guile-package)) ;import our own package module
|
||
|
||
(define* (package->manifest-entry* package system
|
||
#:key target)
|
||
"Return a manifest entry for PACKAGE on SYSTEM, optionally cross-compiled to
|
||
TARGET."
|
||
(manifest-entry
|
||
(inherit (package->manifest-entry package))
|
||
(name (string-append (package-name package) "." system
|
||
(if target
|
||
(string-append "." target)
|
||
"")))
|
||
(item (with-parameters ((%current-system system)
|
||
(%current-target-system target))
|
||
package))))
|
||
|
||
(define native-builds
|
||
(manifest
|
||
(append (map (lambda (system)
|
||
(package->manifest-entry* guile system))
|
||
|
||
'("x86_64-linux" "i686-linux"
|
||
"aarch64-linux" "armhf-linux"
|
||
"powerpc64le-linux"))
|
||
(map (lambda (guile)
|
||
(package->manifest-entry* guile "x86_64-linux"))
|
||
(cons (package
|
||
(inherit (package-with-c-toolchain
|
||
guile
|
||
`(("clang-toolchain"
|
||
,(specification->package
|
||
"clang-toolchain")))))
|
||
(name "guile-clang"))
|
||
(list guile-without-threads
|
||
guile-without-networking
|
||
guile-debug
|
||
guile-strict-typing))))))
|
||
|
||
(define cross-builds
|
||
(manifest
|
||
(map (lambda (target)
|
||
(package->manifest-entry* guile "x86_64-linux"
|
||
#:target target))
|
||
'("i586-pc-gnu"
|
||
"aarch64-linux-gnu"
|
||
"riscv64-linux-gnu"
|
||
"i686-w64-mingw32"
|
||
"x86_64-linux-gnu"))))
|
||
|
||
(concatenate-manifests (list native-builds cross-builds))
|
||
@end lisp
|
||
|
||
We won't go into the details of this manifest; suffice to say that it
|
||
provides additional flexibility. We now need to tell Cuirass to build
|
||
this manifest, which is done with a spec slightly different from the
|
||
previous one:
|
||
|
||
@lisp
|
||
;; Cuirass spec file to build all the packages of the ‘guile’ channel.
|
||
(list (specification
|
||
(name "guile")
|
||
(build '(manifest ".guix/manifest.scm"))
|
||
(channels
|
||
(append (list (channel
|
||
(name 'guile)
|
||
(url "https://git.savannah.gnu.org/git/guile.git")
|
||
(branch "main")))
|
||
%default-channels))))
|
||
@end lisp
|
||
|
||
We changed the @code{(build @dots{})} part of the spec to
|
||
@code{'(manifest ".guix/manifest.scm")} so that it would pick our
|
||
manifest, and that's it!
|
||
|
||
@node Wrapping Up
|
||
@section Wrapping Up
|
||
|
||
We picked Guile as the running example in this chapter and you can see
|
||
the result here:
|
||
|
||
@itemize
|
||
@item
|
||
@uref{https://git.savannah.gnu.org/cgit/guile.git/tree/.guix-channel?id=cd57379b3df636198d8cd8e76c1bfbc523762e79,@code{.guix-channel}};
|
||
@item
|
||
@uref{https://git.savannah.gnu.org/cgit/guile.git/tree/.guix/modules/guile-package.scm?id=cd57379b3df636198d8cd8e76c1bfbc523762e79,@code{.guix/modules/guile-package.scm}}
|
||
with the top-level @file{guix.scm} symlink;
|
||
@item
|
||
@uref{https://git.savannah.gnu.org/cgit/guile.git/tree/.guix/manifest.scm?id=cd57379b3df636198d8cd8e76c1bfbc523762e79,@code{.guix/manifest.scm}}.
|
||
@end itemize
|
||
|
||
These days, repositories are commonly peppered with dot files for
|
||
various tools: @code{.envrc}, @code{.gitlab-ci.yml},
|
||
@code{.github/workflows}, @code{Dockerfile}, @code{.buildpacks},
|
||
@code{Aptfile}, @code{requirements.txt}, and whatnot. It may sound like
|
||
we're proposing a bunch of @emph{additional} files, but in fact those
|
||
files are expressive enough to @emph{supersede} most or all of those
|
||
listed above.
|
||
|
||
With a couple of files, we get support for:
|
||
|
||
@itemize
|
||
@item
|
||
development environments (@command{guix shell});
|
||
@item
|
||
pristine test builds, including for package variants and for
|
||
cross-compilation (@command{guix build});
|
||
@item
|
||
continuous integration (with Cuirass or with some other tool);
|
||
@item
|
||
continuous delivery to users (@emph{via} the channel and with pre-built
|
||
binaries);
|
||
@item
|
||
generation of derivative build artifacts such as Docker images or
|
||
Deb/RPM packages (@command{guix pack}).
|
||
@end itemize
|
||
|
||
This a nice (in our view!) unified tool set for reproducible software
|
||
deployment, and an illustration of how you as a developer can benefit
|
||
from it!
|
||
|
||
|
||
@c *********************************************************************
|
||
@node Environment management
|
||
@chapter Environment management
|
||
|
||
Guix provides multiple tools to manage environment. This chapter
|
||
demonstrate such utilities.
|
||
|
||
@menu
|
||
* Guix environment via direnv:: Setup Guix environment with direnv
|
||
@end menu
|
||
|
||
@node Guix environment via direnv
|
||
@section Guix environment via direnv
|
||
|
||
Guix provides a @samp{direnv} package, which could extend shell after
|
||
directory change. This tool could be used to prepare a pure Guix
|
||
environment.
|
||
|
||
The following example provides a shell function for @file{~/.direnvrc}
|
||
file, which could be used from Guix Git repository in
|
||
@file{~/src/guix/.envrc} file to setup a build environment similar to
|
||
described in @pxref{Building from Git,,, guix, GNU Guix Reference
|
||
Manual}.
|
||
|
||
Create a @file{~/.direnvrc} with a Bash code:
|
||
|
||
@example
|
||
# Thanks <https://github.com/direnv/direnv/issues/73#issuecomment-152284914>
|
||
export_function()
|
||
@{
|
||
local name=$1
|
||
local alias_dir=$PWD/.direnv/aliases
|
||
mkdir -p "$alias_dir"
|
||
PATH_add "$alias_dir"
|
||
local target="$alias_dir/$name"
|
||
if declare -f "$name" >/dev/null; then
|
||
echo "#!$SHELL" > "$target"
|
||
declare -f "$name" >> "$target" 2>/dev/null
|
||
# Notice that we add shell variables to the function trigger.
|
||
echo "$name \$*" >> "$target"
|
||
chmod +x "$target"
|
||
fi
|
||
@}
|
||
|
||
use_guix()
|
||
@{
|
||
# Set GitHub token.
|
||
export GUIX_GITHUB_TOKEN="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
|
||
|
||
# Unset 'GUIX_PACKAGE_PATH'.
|
||
export GUIX_PACKAGE_PATH=""
|
||
|
||
# Recreate a garbage collector root.
|
||
gcroots="$HOME/.config/guix/gcroots"
|
||
mkdir -p "$gcroots"
|
||
gcroot="$gcroots/guix"
|
||
if [ -L "$gcroot" ]
|
||
then
|
||
rm -v "$gcroot"
|
||
fi
|
||
|
||
# Miscellaneous packages.
|
||
PACKAGES_MAINTENANCE=(
|
||
direnv
|
||
git
|
||
git:send-email
|
||
git-cal
|
||
gnupg
|
||
guile-colorized
|
||
guile-readline
|
||
less
|
||
ncurses
|
||
openssh
|
||
xdot
|
||
)
|
||
|
||
# Environment packages.
|
||
PACKAGES=(help2man guile-sqlite3 guile-gcrypt)
|
||
|
||
# Thanks <https://lists.gnu.org/archive/html/guix-devel/2016-09/msg00859.html>
|
||
eval "$(guix environment --search-paths --root="$gcroot" --pure guix --ad-hoc $@{PACKAGES[@@]@} $@{PACKAGES_MAINTENANCE[@@]@} "$@@")"
|
||
|
||
# Predefine configure flags.
|
||
configure()
|
||
@{
|
||
./configure --localstatedir=/var --prefix=
|
||
@}
|
||
export_function configure
|
||
|
||
# Run make and optionally build something.
|
||
build()
|
||
@{
|
||
make -j 2
|
||
if [ $# -gt 0 ]
|
||
then
|
||
./pre-inst-env guix build "$@@"
|
||
fi
|
||
@}
|
||
export_function build
|
||
|
||
# Predefine push Git command.
|
||
push()
|
||
@{
|
||
git push --set-upstream origin
|
||
@}
|
||
export_function push
|
||
|
||
clear # Clean up the screen.
|
||
git-cal --author='Your Name' # Show contributions calendar.
|
||
|
||
# Show commands help.
|
||
echo "
|
||
build build a package or just a project if no argument provided
|
||
configure run ./configure with predefined parameters
|
||
push push to upstream Git repository
|
||
"
|
||
@}
|
||
@end example
|
||
|
||
Every project containing @file{.envrc} with a string @code{use guix}
|
||
will have predefined environment variables and procedures.
|
||
|
||
Run @command{direnv allow} to setup the environment for the first time.
|
||
|
||
|
||
@c *********************************************************************
|
||
@node Installing Guix on a Cluster
|
||
@chapter Installing Guix on a Cluster
|
||
|
||
@cindex cluster installation
|
||
@cindex high-performance computing, HPC
|
||
@cindex HPC, high-performance computing
|
||
Guix is appealing to scientists and @acronym{HPC, high-performance
|
||
computing} practitioners: it makes it easy to deploy potentially complex
|
||
software stacks, and it lets you do so in a reproducible fashion---you
|
||
can redeploy the exact same software on different machines and at
|
||
different points in time.
|
||
|
||
In this chapter we look at how a cluster sysadmin can install Guix for
|
||
system-wide use, such that it can be used on all the cluster nodes, and
|
||
discuss the various tradeoffs@footnote{This chapter is adapted from a
|
||
@uref{https://hpc.guix.info/blog/2017/11/installing-guix-on-a-cluster/,
|
||
blog post published on the Guix-HPC web site in 2017}.}.
|
||
|
||
@quotation Note
|
||
Here we assume that the cluster is running a GNU/Linux distro other than
|
||
Guix System and that we are going to install Guix on top of it.
|
||
@end quotation
|
||
|
||
@menu
|
||
* Setting Up a Head Node:: The node that runs the daemon.
|
||
* Setting Up Compute Nodes:: Client nodes.
|
||
* Cluster Network Access:: Dealing with network access restrictions.
|
||
* Cluster Disk Usage:: Disk usage considerations.
|
||
* Cluster Security Considerations:: Keeping the cluster secure.
|
||
@end menu
|
||
|
||
@node Setting Up a Head Node
|
||
@section Setting Up a Head Node
|
||
|
||
The recommended approach is to set up one @emph{head node} running
|
||
@command{guix-daemon} and exporting @file{/gnu/store} over NFS to
|
||
compute nodes.
|
||
|
||
Remember that @command{guix-daemon} is responsible for spawning build
|
||
processes and downloads on behalf of clients (@pxref{Invoking
|
||
guix-daemon,,, guix, GNU Guix Reference Manual}), and more generally
|
||
accessing @file{/gnu/store}, which contains all the package binaries
|
||
built by all the users (@pxref{The Store,,, guix, GNU Guix Reference
|
||
Manual}). ``Client'' here refers to all the Guix commands that users
|
||
see, such as @code{guix install}. On a cluster, these commands may be
|
||
running on the compute nodes and we'll want them to talk to the head
|
||
node's @code{guix-daemon} instance.
|
||
|
||
To begin with, the head node can be installed following the usual binary
|
||
installation instructions (@pxref{Binary Installation,,, guix, GNU Guix
|
||
Reference Manual}). Thanks to the installation script, this should be
|
||
quick. Once installation is complete, we need to make some adjustments.
|
||
|
||
Since we want @code{guix-daemon} to be reachable not just from the head
|
||
node but also from the compute nodes, we need to arrange so that it
|
||
listens for connections over TCP/IP. To do that, we'll edit the systemd
|
||
startup file for @command{guix-daemon},
|
||
@file{/etc/systemd/system/guix-daemon.service}, and add a
|
||
@code{--listen} argument to the @code{ExecStart} line so that it looks
|
||
something like this:
|
||
|
||
@example
|
||
ExecStart=/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon --build-users-group=guixbuild --listen=/var/guix/daemon-socket/socket --listen=0.0.0.0
|
||
@end example
|
||
|
||
For these changes to take effect, the service needs to be restarted:
|
||
|
||
@example
|
||
systemctl daemon-reload
|
||
systemctl restart guix-daemon
|
||
@end example
|
||
|
||
@quotation Note
|
||
The @code{--listen=0.0.0.0} bit means that @code{guix-daemon} will
|
||
process @emph{all} incoming TCP connections on port 44146
|
||
(@pxref{Invoking guix-daemon,,, guix, GNU Guix Reference Manual}). This
|
||
is usually fine in a cluster setup where the head node is reachable
|
||
exclusively from the cluster's local area network---you don't want that
|
||
to be exposed to the Internet!
|
||
@end quotation
|
||
|
||
The next step is to define our NFS exports in
|
||
@uref{https://linux.die.net/man/5/exports,@file{/etc/exports}} by adding
|
||
something along these lines:
|
||
|
||
@example
|
||
/gnu/store *(ro)
|
||
/var/guix *(rw, async)
|
||
/var/log/guix *(ro)
|
||
@end example
|
||
|
||
The @file{/gnu/store} directory can be exported read-only since only
|
||
@command{guix-daemon} on the master node will ever modify it.
|
||
@file{/var/guix} contains @emph{user profiles} as managed by @code{guix
|
||
package}; thus, to allow users to install packages with @code{guix
|
||
package}, this must be read-write.
|
||
|
||
Users can create as many profiles as they like in addition to the
|
||
default profile, @file{~/.guix-profile}. For instance, @code{guix
|
||
package -p ~/dev/python-dev -i python} installs Python in a profile
|
||
reachable from the @code{~/dev/python-dev} symlink. To make sure that
|
||
this profile is protected from garbage collection---i.e., that Python
|
||
will not be removed from @file{/gnu/store} while this profile exists---,
|
||
@emph{home directories should be mounted on the head node} as well so
|
||
that @code{guix-daemon} knows about these non-standard profiles and
|
||
avoids collecting software they refer to.
|
||
|
||
It may be a good idea to periodically remove unused bits from
|
||
@file{/gnu/store} by running @command{guix gc} (@pxref{Invoking guix
|
||
gc,,, guix, GNU Guix Reference Manual}). This can be done by adding a
|
||
crontab entry on the head node:
|
||
|
||
@example
|
||
root@@master# crontab -e
|
||
@end example
|
||
|
||
@noindent
|
||
... with something like this:
|
||
|
||
@example
|
||
# Every day at 5AM, run the garbage collector to make sure
|
||
# at least 10 GB are free on /gnu/store.
|
||
0 5 * * 1 /usr/local/bin/guix gc -F10G
|
||
@end example
|
||
|
||
We're done with the head node! Let's look at compute nodes now.
|
||
|
||
@node Setting Up Compute Nodes
|
||
@section Setting Up Compute Nodes
|
||
|
||
First of all, we need compute nodes to mount those NFS directories that
|
||
the head node exports. This can be done by adding the following lines
|
||
to @uref{https://linux.die.net/man/5/fstab,@file{/etc/fstab}}:
|
||
|
||
@example
|
||
@var{head-node}:/gnu/store /gnu/store nfs defaults,_netdev,vers=3 0 0
|
||
@var{head-node}:/var/guix /var/guix nfs defaults,_netdev,vers=3 0 0
|
||
@var{head-node}:/var/log/guix /var/log/guix nfs defaults,_netdev,vers=3 0 0
|
||
@end example
|
||
|
||
@noindent
|
||
... where @var{head-node} is the name or IP address of your head node.
|
||
From there on, assuming the mount points exist, you should be able to
|
||
mount each of these on the compute nodes.
|
||
|
||
Next, we need to provide a default @command{guix} command that users can
|
||
run when they first connect to the cluster (eventually they will invoke
|
||
@command{guix pull}, which will provide them with their ``own''
|
||
@command{guix} command). Similar to what the binary installation script
|
||
did on the head node, we'll store that in @file{/usr/local/bin}:
|
||
|
||
@example
|
||
mkdir -p /usr/local/bin
|
||
ln -s /var/guix/profiles/per-user/root/current-guix/bin/guix \
|
||
/usr/local/bin/guix
|
||
@end example
|
||
|
||
We then need to tell @code{guix} to talk to the daemon running on our
|
||
master node, by adding these lines to @code{/etc/profile}:
|
||
|
||
@example
|
||
GUIX_DAEMON_SOCKET="guix://@var{head-node}"
|
||
export GUIX_DAEMON_SOCKET
|
||
@end example
|
||
|
||
To avoid warnings and make sure @code{guix} uses the right locale, we
|
||
need to tell it to use locale data provided by Guix (@pxref{Application
|
||
Setup,,, guix, GNU Guix Reference Manual}):
|
||
|
||
@example
|
||
GUIX_LOCPATH=/var/guix/profiles/per-user/root/guix-profile/lib/locale
|
||
export GUIX_LOCPATH
|
||
|
||
# Here we must use a valid locale name. Try "ls $GUIX_LOCPATH/*"
|
||
# to see what names can be used.
|
||
LC_ALL=fr_FR.utf8
|
||
export LC_ALL
|
||
@end example
|
||
|
||
For convenience, @code{guix package} automatically generates
|
||
@file{~/.guix-profile/etc/profile}, which defines all the environment
|
||
variables necessary to use the packages---@code{PATH},
|
||
@code{C_INCLUDE_PATH}, @code{PYTHONPATH}, etc. Thus it's a good idea to
|
||
source it from @code{/etc/profile}:
|
||
|
||
@example
|
||
GUIX_PROFILE="$HOME/.guix-profile"
|
||
if [ -f "$GUIX_PROFILE/etc/profile" ]; then
|
||
. "$GUIX_PROFILE/etc/profile"
|
||
fi
|
||
@end example
|
||
|
||
Last but not least, Guix provides command-line completion notably for
|
||
Bash and zsh. In @code{/etc/bashrc}, consider adding this line:
|
||
|
||
@verbatim
|
||
. /var/guix/profiles/per-user/root/current-guix/etc/bash_completion.d/guix
|
||
@end verbatim
|
||
|
||
Voilà!
|
||
|
||
You can check that everything's in place by logging in on a compute node
|
||
and running:
|
||
|
||
@example
|
||
guix install hello
|
||
@end example
|
||
|
||
The daemon on the head node should download pre-built binaries on your
|
||
behalf and unpack them in @file{/gnu/store}, and @command{guix install}
|
||
should create @file{~/.guix-profile} containing the
|
||
@file{~/.guix-profile/bin/hello} command.
|
||
|
||
@node Cluster Network Access
|
||
@section Network Access
|
||
|
||
Guix requires network access to download source code and pre-built
|
||
binaries. The good news is that only the head node needs that since
|
||
compute nodes simply delegate to it.
|
||
|
||
It is customary for cluster nodes to have access at best to a
|
||
@emph{white list} of hosts. Our head node needs at least
|
||
@code{ci.guix.gnu.org} in this white list since this is where it gets
|
||
pre-built binaries from by default, for all the packages that are in
|
||
Guix proper.
|
||
|
||
Incidentally, @code{ci.guix.gnu.org} also serves as a
|
||
@emph{content-addressed mirror} of the source code of those packages.
|
||
Consequently, it is sufficient to have @emph{only}
|
||
@code{ci.guix.gnu.org} in that white list.
|
||
|
||
Software packages maintained in a separate repository such as one of the
|
||
various @uref{https://hpc.guix.info/channels, HPC channels} are of
|
||
course unavailable from @code{ci.guix.gnu.org}. For these packages, you
|
||
may want to extend the white list such that source and pre-built
|
||
binaries (assuming this-party servers provide binaries for these
|
||
packages) can be downloaded. As a last resort, users can always
|
||
download source on their workstation and add it to the cluster's
|
||
@file{/gnu/store}, like this:
|
||
|
||
@verbatim
|
||
GUIX_DAEMON_SOCKET=ssh://compute-node.example.org \
|
||
guix download http://starpu.gforge.inria.fr/files/starpu-1.2.3/starpu-1.2.3.tar.gz
|
||
@end verbatim
|
||
|
||
The above command downloads @code{starpu-1.2.3.tar.gz} @emph{and} sends
|
||
it to the cluster's @code{guix-daemon} instance over SSH.
|
||
|
||
Air-gapped clusters require more work. At the moment, our suggestion
|
||
would be to download all the necessary source code on a workstation
|
||
running Guix. For instance, using the @option{--sources} option of
|
||
@command{guix build} (@pxref{Invoking guix build,,, guix, GNU Guix
|
||
Reference Manual}), the example below downloads all the source code the
|
||
@code{openmpi} package depends on:
|
||
|
||
@example
|
||
$ guix build --sources=transitive openmpi
|
||
|
||
@dots{}
|
||
|
||
/gnu/store/xc17sm60fb8nxadc4qy0c7rqph499z8s-openmpi-1.10.7.tar.bz2
|
||
/gnu/store/s67jx92lpipy2nfj5cz818xv430n4b7w-gcc-5.4.0.tar.xz
|
||
/gnu/store/npw9qh8a46lrxiwh9xwk0wpi3jlzmjnh-gmp-6.0.0a.tar.xz
|
||
/gnu/store/hcz0f4wkdbsvsdky3c0vdvcawhdkyldb-mpfr-3.1.5.tar.xz
|
||
/gnu/store/y9akh452n3p4w2v631nj0injx7y0d68x-mpc-1.0.3.tar.gz
|
||
/gnu/store/6g5c35q8avfnzs3v14dzl54cmrvddjm2-glibc-2.25.tar.xz
|
||
/gnu/store/p9k48dk3dvvk7gads7fk30xc2pxsd66z-hwloc-1.11.8.tar.bz2
|
||
/gnu/store/cry9lqidwfrfmgl0x389cs3syr15p13q-gcc-5.4.0.tar.xz
|
||
/gnu/store/7ak0v3rzpqm2c5q1mp3v7cj0rxz0qakf-libfabric-1.4.1.tar.bz2
|
||
/gnu/store/vh8syjrsilnbfcf582qhmvpg1v3rampf-rdma-core-14.tar.gz
|
||
…
|
||
@end example
|
||
|
||
(In case you're wondering, that's more than 320@ MiB of
|
||
@emph{compressed} source code.)
|
||
|
||
We can then make a big archive containing all of this (@pxref{Invoking
|
||
guix archive,,, guix, GNU Guix Reference Manual}):
|
||
|
||
@verbatim
|
||
$ guix archive --export \
|
||
`guix build --sources=transitive openmpi` \
|
||
> openmpi-source-code.nar
|
||
@end verbatim
|
||
|
||
@dots{} and we can eventually transfer that archive to the cluster on
|
||
removable storage and unpack it there:
|
||
|
||
@verbatim
|
||
$ guix archive --import < openmpi-source-code.nar
|
||
@end verbatim
|
||
|
||
This process has to be repeated every time new source code needs to be
|
||
brought to the cluster.
|
||
|
||
As we write this, the research institutes involved in Guix-HPC do not
|
||
have air-gapped clusters though. If you have experience with such
|
||
setups, we would like to hear feedback and suggestions.
|
||
|
||
@node Cluster Disk Usage
|
||
@section Disk Usage
|
||
|
||
@cindex disk usage, on a cluster
|
||
A common concern of sysadmins' is whether this is all going to eat a lot
|
||
of disk space. If anything, if something is going to exhaust disk
|
||
space, it's going to be scientific data sets rather than compiled
|
||
software---that's our experience with almost ten years of Guix usage on
|
||
HPC clusters. Nevertheless, it's worth taking a look at how Guix
|
||
contributes to disk usage.
|
||
|
||
First, having several versions or variants of a given package in
|
||
@file{/gnu/store} does not necessarily cost much, because
|
||
@command{guix-daemon} implements deduplication of identical files, and
|
||
package variants are likely to have a number of common files.
|
||
|
||
As mentioned above, we recommend having a cron job to run @code{guix gc}
|
||
periodically, which removes @emph{unused} software from
|
||
@file{/gnu/store}. However, there's always a possibility that users will
|
||
keep lots of software in their profiles, or lots of old generations of
|
||
their profiles, which is ``live'' and cannot be deleted from the
|
||
viewpoint of @command{guix gc}.
|
||
|
||
The solution to this is for users to regularly remove old generations of
|
||
their profile. For instance, the following command removes generations
|
||
that are more than two-month old:
|
||
|
||
@example
|
||
guix package --delete-generations=2m
|
||
@end example
|
||
|
||
Likewise, it's a good idea to invite users to regularly upgrade their
|
||
profile, which can reduce the number of variants of a given piece of
|
||
software stored in @file{/gnu/store}:
|
||
|
||
@example
|
||
guix pull
|
||
guix upgrade
|
||
@end example
|
||
|
||
As a last resort, it is always possible for sysadmins to do some of this
|
||
on behalf of their users. Nevertheless, one of the strengths of Guix is
|
||
the freedom and control users get on their software environment, so we
|
||
strongly recommend leaving users in control.
|
||
|
||
@node Cluster Security Considerations
|
||
@section Security Considerations
|
||
|
||
@cindex security, on a cluster
|
||
On an HPC cluster, Guix is typically used to manage scientific software.
|
||
Security-critical software such as the operating system kernel and
|
||
system services such as @code{sshd} and the batch scheduler remain under
|
||
control of sysadmins.
|
||
|
||
The Guix project has a good track record delivering security updates in
|
||
a timely fashion (@pxref{Security Updates,,, guix, GNU Guix Reference
|
||
Manual}). To get security updates, users have to run @code{guix pull &&
|
||
guix upgrade}.
|
||
|
||
Because Guix uniquely identifies software variants, it is easy to see if
|
||
a vulnerable piece of software is in use. For instance, to check whether
|
||
the glibc@ 2.25 variant without the mitigation patch against
|
||
``@uref{https://www.qualys.com/2017/06/19/stack-clash/stack-clash.txt,Stack
|
||
Clash}'', one can check whether user profiles refer to it at all:
|
||
|
||
@example
|
||
guix gc --referrers /gnu/store/…-glibc-2.25
|
||
@end example
|
||
|
||
This will report whether profiles exist that refer to this specific
|
||
glibc variant.
|
||
|
||
|
||
@c *********************************************************************
|
||
@node Acknowledgments
|
||
@chapter Acknowledgments
|
||
|
||
Guix is based on the @uref{https://nixos.org/nix/, Nix package manager},
|
||
which was designed and
|
||
implemented by Eelco Dolstra, with contributions from other people (see
|
||
the @file{nix/AUTHORS} file in Guix.) Nix pioneered functional package
|
||
management, and promoted unprecedented features, such as transactional
|
||
package upgrades and rollbacks, per-user profiles, and referentially
|
||
transparent build processes. Without this work, Guix would not exist.
|
||
|
||
The Nix-based software distributions, Nixpkgs and NixOS, have also been
|
||
an inspiration for Guix.
|
||
|
||
GNU@tie{}Guix itself is a collective work with contributions from a
|
||
number of people. See the @file{AUTHORS} file in Guix for more
|
||
information on these fine people. The @file{THANKS} file lists people
|
||
who have helped by reporting bugs, taking care of the infrastructure,
|
||
providing artwork and themes, making suggestions, and more---thank you!
|
||
|
||
This document includes adapted sections from articles that have
|
||
previously been published on the Guix blog at
|
||
@uref{https://guix.gnu.org/blog} and on the Guix-HPC blog at
|
||
@uref{https://hpc.guix.info/blog}.
|
||
|
||
|
||
@c *********************************************************************
|
||
@node GNU Free Documentation License
|
||
@appendix GNU Free Documentation License
|
||
@cindex license, GNU Free Documentation License
|
||
@include fdl-1.3.texi
|
||
|
||
@c *********************************************************************
|
||
@node Concept Index
|
||
@unnumbered Concept Index
|
||
@printindex cp
|
||
|
||
@bye
|
||
|
||
@c Local Variables:
|
||
@c ispell-local-dictionary: "american";
|
||
@c End:
|