Files smaller than 8 KiB typically represent ~70% of the entries in
/gnu/store/.links but only contribute to ~4% of the space savings
afforded by deduplication.
Not considering these files for deduplication speeds up file insertion
in the store and, more importantly, leaves 'removeUnusedLinks' with
fewer entries to traverse, thereby speeding it up proportionally.
Partly fixes <https://issues.guix.gnu.org/24937>.
* config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK
definition.
* guix/store/deduplication.scm (%deduplication-minimum-size): New
variable.
(deduplicate)[loop]: Do not recurse when FILE's size is below
%DEDUPLICATION-MINIMUM-SIZE.
(dump-port): New procedure.
(dump-file/deduplicate)[hash]: Turn into...
[dump-and-compute-hash]: ... this thunk.
Call 'deduplicate' only when SIZE is greater than
%DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'.
* nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where
st.st_size < deduplicationMinSize.
* nix/libstore/local-store.hh (deduplicationMinSize): New declaration.
* nix/libstore/optimise-store.cc (deduplicationMinSize): New variable.
(LocalStore::optimisePath_): Return when PATH is a symlink or smaller
than 'deduplicationMinSize'.
* tests/derivations.scm ("identical files are deduplicated"): Produce
files bigger than %DEDUPLICATION-MINIMUM-SIZE.
* tests/nar.scm ("restore-file-set with directories (signed, valid)"):
Likewise.
* tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"):
New test.
("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than
%DEDUPLICATION-MINIMUM-SIZE.
* tests/store.scm ("substitute, deduplication"): Likewise.
* guix/tests.scm (file=?): Add optional 'stat' parameter. Add fast
patch comparing inode numbers.
* tests/gexp.scm ("imported-files with file-like objects"): Remove
'file=?' procedure and use the one from (guix tests).
Julia's built-in package manager (Pkg) looks for packages in
JULIA_DEPOT_PATH/packages/PACKAGENAME/XXXX, where XXXX is a string
encoding package UUID and SHA1 of files. The link-depot phase creates
a link at the correct location to allow Pkg to find packages that were
already installed by Guix.
* guix/build/julia-build-system.scm (link-depot): New phase.
(%package-path): Modified package path from packages/ to loadpath/.
(project.toml->uuid): New procedure, retrive package uuid from TOML file.
(precompile, check): Adjust to the change in paths.
(%standard-phases): Add link-depot phase.
(julia-build): Add julia-package-uuid keyword.
* guix/build-system/julia.scm (julia-build): Also use the
julia-package-uuid keyword.
* gnu/packages/julia-jll.scm (julia-bzip2-jll, julia-cairo-jll,
julia-compilersupportlibraries-jll, julia-expat-jll, julia-ffmpeg-jll,
julia-fontconfig-jll, julia-freetype2-jll, julia-fribidi-jll,
julia-gettext-jll, julia-glib-jll, julia-gr-jll, julia-gumbo-jll,
julia-imagemagick-jll, julia-jllwrappers-jll, julia-jpegturbo-jll,
julia-lame-jll, julia-libass-jll, julia-libfdk-aac-jll,
julia-libffi-jll, julia-libgcrypt-jll, julia-libglvnd-jll,
julia-libgpg-error-jll, juli-libiconv-jll, julia-libmount-jll,
julia-libpng-jll, julia-libsass-jll, julia-libtiff-jll,
julia-libuuid-jll, julia-libvorbis-jll, julia-lzo-jll,
julia-mbedtls-jll, julia-ogg-jll, julia-openspecfun-jll,
julia-openssl-jll, julia-opus-jll, julia-pcre-jll, julia-pixman-jll,
julia-qt5base-jll, julia-wayland-jll, julia-wayland-protocols-jll,
julia-x264-jll, julia-x265-jll, julia-xkbcommon-jll, julia-xml2-jll,
julia-xorg-libpthread-stubs-jll, julia-xorg-libx11-jll,
julia-xorg-libxau-jll, julia-xorg-libxcb-jll, julia-xorg-libxcursor-jll,
julia-xorg-libxdmcp-jll, julia-xorg-libxext-jll,
julia-xorg-libxfixes-jll, julia-xorg-libxi-jll,
julia-xorg-libxinerama-jll, julia-xorg-libxkbfile-jll,
julia-xorg-libxrandr-jll, julia-xorg-libxrender-jll,
julia-xorg-xcb-util-jll, julia-xorg-xcb-util-keysyms-jll,
julia-xorg-xcb-util-renderutil-jll, julia-xorg-xcb-util-wm-jll,
julia-xorg-xkbcomp-jll, julia-xorg-xkeyboard-config-jll,
julia-xorg-xtrans-jll, julia-xslt-jll, julia-zlib-jll, julia-zstd-jll)
[argument]: Adjust the order of the phases to the change in the
julia-build-system.
* gnu/packages/julia-xyz.scm (julia-bufferedstreams, julia-calculus,
julia-dataframes, julia-datavalues, julia-distances, julia-documenter,
julia-dualnumbers, julia-ellipsisnotation, julia-expronicon,
julia-fileio, julia-fixedpointnumbers, julia-functionwrappers,
julia-fuzzycompletions, julia-genericlinearalgebra, julia-genericschur,
julia-geometrybasics, julia-imagemagick, julia-infinity,
julia-matrixfactorizations, julia-media, julia-missings, julia-nnlib,
julia-optimtestproblems, julia-orderedcollections, julia-prettytables,
julia-pycall, julia-quadmath, julia-safetests, julia-stackviews)
[arguments]: Correct paths to reflect change of %package-path modify
order of phases as link-depot needs to run on an unmodified src
directory add package-name-uuid keyword argument to packages without
Project.toml file.
* gnu/packages/julia.scm (julia)[native-search-paths]: Correct paths to
reflect change of %package-path modify order of phases as link-depot
needs to run on an unmodified src directory.
Signed-off-by: Efraim Flashner <efraim@flashner.co.il>
* guix/packages/packages.scm (<package>): Clarify that the license
field takes a list of licenses rather than a generic list.
Signed-off-by: Liliana Marie Prikler <liliana.prikler@gmail.com>
* guix/import/utils.scm (spdx-string->license): Identify more
licenses (CC-BY-4.0, Freetype, HPND).
* guix/licenses.scm: Add a comment that mentions "MIT" so that people
who grep for MIT end up there.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* guix/import/print.scm (package->code)[variable-reference]
[object->code]: New procedures.
[package-lists->code]: Rewrite in terms of 'object->code'.
Pass the 'arguments' field through 'object->code'.
* tests/print.scm (pkg-with-arguments, pkg-with-arguments-source): New
variables.
("package with arguments"): New test.
* guix/import/print.scm (package->code)[source->code]: Handle patches
that are origins.
* tests/print.scm (pkg-with-origin-input): Add 'patches' field.
(pkg-with-origin-patch, pkg-with-origin-patch-source): New variables.
("package with origin patch"): New test.
* guix/import/print.scm (package->code)[factorized-uri-code]: New
procedure.
[source->code]: Use it, and factorize URI when it's a list.
* tests/print.scm (pkg-with-origin-input): Check origin URI to a list.
* guix/import/print.scm (package->code)[source->code]: Check whether
VERSION is true before calling 'factorize-uri'.
[package-lists->code]: Add clause for inputs that are origins.
* tests/print.scm (pkg-with-origin-input, pkg-with-origin-input-source):
New variables.
("package with origin input"): New test.
* guix/import/cran.scm (download): Handle the case where URL is a list.
(fetch-description-from-tarball): New procedure.
(fetch-description): Add #:version parameter. Honor it when REPOSITORY
is 'cran. Use 'fetch-description-from-tarball' when REPOSITORY is
'bioconductor.
(description->package): SOURCE-URL may now be a list.
(cran->guix-package): Pass VERSION to 'fetch-description'.
(cran-recursive-import): Add #:version parameter.
* guix/scripts/import/cran.scm (guix-import-cran): Expect a spec rather
than a mere package name.
* doc/guix.texi (Invoking guix import): Document it.
Previously, most actions would leave the cache as-is because
'guix-environment*' would indirectly call 'exit', and thus its
continuation was never reached.
* guix/scripts/shell.scm (guix-shell): Move call to
'maybe-remove-expired-cache-entries' to EXIT-HOOK.
Previously, 'read-error' exceptions other than "missing closing paren"
would not be reported; instead, we'd directly call (exit 1) without
printing anything.
Fixes <https://issues.guix.gnu.org/51463>.
Reported by Alice BRENON <alice.brenon@ens-lyon.fr>.
* guix/ui.scm (report-load-error): Report the error without re-throwing
upon 'read-error'.
* tests/guix-build.sh: Add test.
* guix/scripts/environment.scm (launch-environment): Call
'primitive-_exit' upon 'system-error.
(suggest-command-name, validate-exit-status): New procedures.
(launch-environment/fork): Call 'validate-exit-status'.
(launch-environment/container)[exit/status*]: New procedure.
Use it instead of 'exit/status'.
This allows us to skip the expensive man-db profile hook in most cases.
Suggested by Liliana Marie Prikler <liliana.prikler@gmail.com>.
* guix/profiles.scm (manual-database/optional): New procedure.
(%default-profile-hooks): Use it instead of 'manual-database'.
* doc/guix.texi (Documentation): Add footnote about 'man -k' database
creation.
This also makes automated ‘guix --h<Tab>’ completion possible.
* guix/ui.scm (show-guix-help): Document that an OPTION's an option,
and all valid options.
* guix/scripts/home/import.scm (manifest->code): Remove.
(manifest+configuration-files->code): New procedure.
(import-manifest): Use 'manifest+configuration-files->code' instead of
'manifest->code'.
* tests/home-import.scm (eval-test-with-home-environment): Likewise.
(match-home-environment-transformations): New procedure.
("manifest->code: No services, package transformations"): New test.
Suggested by Liliana Marie Prikler <liliana.prikler@gmail.com>.
* guix/scripts/home/import.scm (configurations+modules): Rename
'destination-directory' to 'configuration-directory'. Improve
docstring.
'procedure-name' is a debugging aid and cannot be reliably used to
check for procedure equality.
* guix/scripts/home/import.scm (configurations+modules): Remove use of
'procedure-name'.
Set the name of the file to just the basename of the file passed to
‘local-file’.
* guix/scripts/home/import.scm (basename+remove-dots): New procedure.
(generate-bash-configuration+modules): Use it.
* tests/home-import.scm (match-home-environment-bash-service): Adjust
accordingly.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Two different services might require the same module(s), so delete duplicates
when generating the ‘use-modules’ form.
* import.scm (manifest->code): Delete duplicate modules.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
‘slurp-file-gexp’ is not a bound procedure.
* guix/scripts/home/import.scm (generate-bash-configuration+modules): Don’t
use ‘slurp-file-gexp’.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Previously, only one module could be imported for each service, e.g., only
(gnu home-services shell) could be imported when generating the Bash service
declaration. However, for some services, multiple modules might need to be
imported in order for it to work.
* guix/scripts/home/import.scm (generate-bash-module+configuration): Rename to
...
(generate-bash-configuration+modules): ... this.
(%files-configurations-alist): Rename to ...
(%files+configurations-alist): ... this.
(modules+configurations): Rename to ...
(configurations+modules): ... this.
(manifest->code): Adjust accordingly.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Copy the appropriate the relevant configuration files to the destination
directory, and call ‘local-file’ on them.
Without this, ‘guix home import’ will generate a service declaration like this
(service
home-bash-service-type
(home-bash-configuration
(bashrc
(list (slurp-file-gexp
(local-file "/home/yoctocell/.bashrc"))))))
but when running ‘guix home reconfigure’, the ~/.bashrc file would be moved, so
when running ‘guix home reconfigure’ for the second time, it would read the
~/.bashrc which is itself a symlink to a file the store.
* guix/scripts/home/import.scm (generate-bash-module+configuration): Take
‘destination-directory’ parameter
(modules+configurations): Copy the user’s configuration file to
‘%destination-directory’.
* guix/scripts/home.scm (process-command): Adjust accordingly; create
‘destination’ if it doesn’t exist.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
The previous behavior was confusing: a warning would be printed and
'guix shell' would go on starting an empty environment.
Reported by Tobias Geerinckx-Rice <me@tobias.gr>.
* guix/scripts/shell.scm (auto-detect-manifest): Change "not loading"
case from warning to error.
* tests/guix-shell.sh: Adjust accordingly.
Previously 'guix search' & co. would crash when encountering invalid
Texinfo.
* guix/ui.scm (texi->plain-text*): New procedure.
(package-field-string, package->recutils): Use it.
This is a followup to b19250eec6,
providing a proper fix for <https://issues.guix.gnu.org/46756>.
* guix/remote.scm (remote-eval): Revert b19250eec6.
* guix/store.scm (build-accumulator): Turn into a procedure. Call
CONTINUE when the store is not eq? to the initial store.
(map/accumulate-builds): Adjust accordingly.
* tests/store.scm ("map/accumulate-builds and different store"): New test.
Fixes <https://issues.guix.gnu.org/46756>.
Reported by pkill9 <pkill9@runbox.com>,
Maxim Cournoyer <maxim.cournoyer@gmail.com>, and
Katherine Cox-Buday <cox.katherine.e@gmail.com>.
* guix/remote.scm (remote-eval): When BUILD-LOCALLY? is false, wrap
'build-derivations' call in 'with-build-handler'.
In particular, libutil is not found when running code on a
statically-linked Guile.
Reported by mahmooz on #guix.
* guix/build/syscalls.scm (syscall->procedure): Add #:library parameter
and honor it.
(openpty, login-tty): Use 'syscall->procedure' instead of calling
'dynamic-link' directly.
* guix/scripts/shell.scm (hint-directory, hint-file, record-hint)
(hint-given?): New procedures.
(guix-shell): Record and probe the 'shell-check' hint.
With this change, the wall-clock time of:
./pre-inst-env guile -c '(use-modules (gnu) (guix)(ice-9 time)) (time (pk (fold-packages (lambda (p r)(supported-package? p)(+ 1 r)) 0)))'
goes from 3.2s to 2.0s, a 37% improvement.
* guix/packages.scm (package-transitive-supported-systems): Change
'supported-systems' to 'supported-systems-procedure', returning an
'mlambdaq' instead of the original 'mlambda'. Add 'procs'. Adjust body
accordingly.
shell: Maintain a profile cache.
With this change, running "guix shell" (no arguments) is equivalent to:
guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm
This is the cache miss. On cache hit, it's equivalent to:
guix environment -p ~/.cache/guix/profiles/some-root
... which can run in 0.1s.
* guix/scripts/shell.scm (options-with-caching): New procedure.
(parse-args): Use it.
(%profile-cache-directory): New variable.
(profile-cache-key, profile-cached-gc-root): New procedures.
(show-help, %options): Add '--rebuild-cache'.
(guix-shell)[cache-entries, entry-expiration]: New procedures.
Add call to 'maybe-remove-expired-cache-entries'.
* doc/guix.texi (Invoking guix shell): Document '--rebuild-cache'.
Ludovic Courtès
Jean-Baptiste Volatier
Vagrant Cascadian
Thiago Jung Bauermann
jgart
Attila Lendvai
Ricardo Wurmus
Xinglu Chen
Maxime Devos
Lars-Dominik Braun
Tobias Geerinckx-Rice
Zhu Zihao
Ludovic Courtès