guix/tests
Ludovic Courtès 472a0e82a5
daemon: Do not deduplicate files smaller than 8 KiB.
Files smaller than 8 KiB typically represent ~70% of the entries in
/gnu/store/.links but only contribute to ~4% of the space savings
afforded by deduplication.

Not considering these files for deduplication speeds up file insertion
in the store and, more importantly, leaves 'removeUnusedLinks' with
fewer entries to traverse, thereby speeding it up proportionally.

Partly fixes <https://issues.guix.gnu.org/24937>.

* config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK
definition.
* guix/store/deduplication.scm (%deduplication-minimum-size): New
variable.
(deduplicate)[loop]: Do not recurse when FILE's size is below
%DEDUPLICATION-MINIMUM-SIZE.
(dump-port): New procedure.
(dump-file/deduplicate)[hash]: Turn into...
[dump-and-compute-hash]: ... this thunk.
Call 'deduplicate' only when SIZE is greater than
%DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'.
* nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where
st.st_size < deduplicationMinSize.
* nix/libstore/local-store.hh (deduplicationMinSize): New declaration.
* nix/libstore/optimise-store.cc (deduplicationMinSize): New variable.
(LocalStore::optimisePath_): Return when PATH is a symlink or smaller
than 'deduplicationMinSize'.
* tests/derivations.scm ("identical files are deduplicated"): Produce
files bigger than %DEDUPLICATION-MINIMUM-SIZE.
* tests/nar.scm ("restore-file-set with directories (signed, valid)"):
Likewise.
* tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"):
New test.
("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than
%DEDUPLICATION-MINIMUM-SIZE.
* tests/store.scm ("substitute, deduplication"): Likewise.
2021-11-16 14:34:28 +01:00
..
services Reinstate "services: Add a service for Jami." 2021-08-02 17:23:30 -04:00
accounts.scm
base16.scm
base32.scm build: Remove references to the 'nix-hash' program. 2020-08-29 16:24:17 +02:00
base64.scm
boot-parameters.scm Migrate to the new 'targets' field of bootloader-configuration. 2021-08-29 01:05:26 -04:00
bournish.scm
build-utils.scm Properly deal with build directories containing '~'. 2020-11-16 11:21:42 +01:00
builders.scm
cache.scm Remove workaround for 'time-monotonic' in Guile 2.2.2. 2020-03-22 16:49:47 +01:00
challenge.scm guix: Move narinfo code from substitute script to module. 2021-01-16 11:51:03 +00:00
channels.scm channels: 'channel-news-entry-commit' correctly resolves annotated tags. 2021-09-18 19:37:45 +02:00
civodul.key Add (guix openpgp). 2020-05-04 09:56:12 +02:00
combinators.scm
containers.scm linux-container: Reset jailed root permissions. 2020-10-01 12:51:13 +02:00
cpan.scm
cpio.scm
cran.scm import: cran: Avoid uses of '@@' in the tests. 2020-01-16 23:14:05 +01:00
crate.scm tests: Fix JSON syntax error in 'crate.scm'. 2021-01-13 22:42:49 +01:00
cve-sample.json tests: Fix malformed JSON. 2020-12-21 17:47:35 +01:00
cve.scm
debug-link.scm
derivations.scm daemon: Do not deduplicate files smaller than 8 KiB. 2021-11-16 14:34:28 +01:00
discovery.scm
dsa.key Add (guix openpgp). 2020-05-04 09:56:12 +02:00
ed25519.key Add (guix openpgp). 2020-05-04 09:56:12 +02:00
ed25519.sec Add (guix openpgp). 2020-05-04 09:56:12 +02:00
ed25519bis.key git-authenticate: Add tests. 2020-06-05 22:54:06 +02:00
ed25519bis.sec git-authenticate: Add tests. 2020-06-05 22:54:06 +02:00
egg.scm tests: Adjust tests/egg.scm to latest API changes. 2021-11-14 00:10:58 +01:00
elpa.scm tests: do not hard code HTTP ports 2021-03-06 11:41:48 +01:00
file-systems.scm pack: Streamline how files are included in tarballs. 2021-07-18 02:10:49 -04:00
gem.scm tests: Remove trailing commas in JSON tests. 2020-04-27 23:29:40 +02:00
gexp.scm tests: Factorize 'file=?'. 2021-11-16 14:34:28 +01:00
git-authenticate.scm tests: Allow 'tests/git-authenticate.scm' to run when git/gpg is missing. 2020-06-28 23:26:53 +02:00
git.scm import: Add 'generic-git' updater. 2021-09-18 19:37:45 +02:00
glob.scm guix: Add globstar support. 2020-12-09 22:25:17 +01:00
gnu-maintenance.scm gnu-maintenance: 'release-file?' accepts 'v' prefix as in "PKG-v1.2.tgz". 2021-05-28 23:04:44 +02:00
go.scm tests: go: Fix typo. 2021-09-28 22:00:47 +02:00
grafts.scm grafts: Support rewriting UTF-16 and UTF-32 store references. 2021-04-15 03:22:55 -04:00
graph.scm graph: Add '--max-depth'. 2021-09-21 15:15:52 +02:00
gremlin.scm
guix-archive.sh serialization: 'restore-file' sets canonical timestamp and permissions. 2020-12-15 17:32:09 +01:00
guix-authenticate.sh authenticate: Encode strings as ISO-8859-1. 2020-09-15 14:40:20 +02:00
guix-build-branch.sh tests: Simplify shell exit status negation; 2020-09-28 21:06:36 -05:00
guix-build.sh ui: 'load*' correctly reports 'read-error' in all cases. 2021-11-07 23:10:41 +01:00
guix-daemon.sh tests: Simplify shell exit status negation; 2020-09-28 21:06:36 -05:00
guix-describe.sh
guix-download.sh tests: Simplify shell exit status negation; 2020-09-28 21:06:36 -05:00
guix-environment-container.sh environment: Add tests for '--profile'. 2021-10-25 19:02:32 +02:00
guix-environment.sh environment: Add tests for '--profile'. 2021-10-25 19:02:32 +02:00
guix-gc.sh tests: Simplify shell exit status negation; 2020-09-28 21:06:36 -05:00
guix-git-authenticate.sh tests: Simplify shell exit status negation; 2020-09-28 21:06:36 -05:00
guix-graph.sh tests: Simplify shell exit status negation; 2020-09-28 21:06:36 -05:00
guix-hash.sh guix hash: Honor '-H' when used alongside '-r'. 2021-01-04 11:51:48 +01:00
guix-home.sh home: services: configuration: Support file-like objects. 2021-10-09 19:24:19 +03:00
guix-lint.sh packages: 'package-field-location' handles 'search-path' returning #f. 2021-02-22 12:07:45 +01:00
guix-pack-localstatedir.sh
guix-pack-relocatable.sh tests: pack-relocatable: Ensure commands can run in the current namespace. 2020-12-04 13:12:08 +00:00
guix-pack.sh tests: Simplify shell exit status negation; 2020-09-28 21:06:36 -05:00
guix-package-aliases.sh guix build: Record package transformations in manifest entries. 2020-10-02 23:28:09 +02:00
guix-package-net.sh tests: Simplify shell exit status negation; 2020-09-28 21:06:36 -05:00
guix-package.sh diagnostics, ui: Adjust to 'read-error' and 'syntax-error' in Guile 3.0.6. 2021-04-29 01:22:04 +02:00
guix-repl.sh repl: Look for script files in (getcwd). 2020-09-16 16:04:01 +02:00
guix-shell.sh shell: Error out when an unauthorized guix.scm/manifest.scm is found. 2021-10-30 19:12:40 +02:00
guix-system.sh gnu: bootloader: Support multiple targets. 2021-08-29 01:05:26 -04:00
hackage.scm import: cabal: Treat identifier names correctly. 2021-09-15 14:28:38 +02:00
home-import.scm home: import: Avoid duplication of 'manifest->code'. 2021-10-31 00:51:06 +02:00
import-git.scm import: Add 'generic-git' updater. 2021-09-18 19:37:45 +02:00
import-utils.scm import: utils: Skip not found packages. 2021-09-01 23:41:42 +02:00
inferior.scm inferior: Support querying package replacements. 2021-05-15 12:14:47 +01:00
ipfs.scm Add (guix ipfs). 2021-04-12 18:42:22 +02:00
lint.scm tests: Avoid literal strings for invalid Texinfo markup. 2021-11-06 23:01:21 +01:00
minetest.scm import/minetest: Define an updater for mods on ContentDB. 2021-10-02 17:16:43 +02:00
modules.scm
monads.scm
nar.scm daemon: Do not deduplicate files smaller than 8 KiB. 2021-11-16 14:34:28 +01:00
networking.scm services: openntpd: Remove support for deprecated "-s" option. 2021-01-16 23:30:59 +01:00
offload.scm offload: Modify the build-machine record to accept multiple systems. 2020-08-25 13:45:27 -04:00
opam.scm tests: Allow opam test to run without networking. 2021-09-18 19:37:44 +02:00
openpgp.scm openpgp: Fix argument order of 'fxbit-set?'. 2020-09-29 22:53:36 +02:00
pack.scm pack: Allow embedding custom control files in deb packs. 2021-07-18 02:02:34 -04:00
packages.scm packages: Add 'package-development-inputs'. 2021-10-25 15:29:39 +02:00
pki.scm
print.scm import: print: Replace packages and origins in 'arguments'. 2021-11-11 00:14:31 +01:00
processes.scm
profiles.scm profiles: Add 'package->development-manifest'. 2021-10-25 15:29:42 +02:00
publish.scm publish: Add '--negative-ttl'. 2021-05-16 23:28:11 +02:00
pypi.scm import: pypi: Allow imports of a specific version. 2021-11-11 00:14:30 +01:00
records.scm records: Support field sanitizers. 2021-08-12 12:34:13 +02:00
rsa.key Add (guix openpgp). 2020-05-04 09:56:12 +02:00
scripts.scm
search-paths.scm
services.scm
sets.scm
signing-key.pub
signing-key.sec
size.scm
status.scm
store-database.scm database: Validate #:nar-size and #:time when registering store items. 2021-01-31 21:49:58 +01:00
store-deduplication.scm daemon: Do not deduplicate files smaller than 8 KiB. 2021-11-16 14:34:28 +01:00
store-roots.scm
store.scm daemon: Do not deduplicate files smaller than 8 KiB. 2021-11-16 14:34:28 +01:00
substitute.scm daemon: 'guix substitute' replies on FD 4. 2021-04-09 17:46:38 +02:00
swh.scm swh: Test proper handling of null visit snapshot URL. 2021-01-21 11:22:08 +01:00
syscalls.scm syscalls: Add 'openpty' and 'login-tty'. 2021-10-26 12:46:27 +02:00
system.scm Migrate to the new 'targets' field of bootloader-configuration. 2021-08-29 01:05:26 -04:00
test.drv
texlive.scm tests: do not hard code HTTP ports 2021-03-06 11:41:48 +01:00
transformations.scm transformations: Git tags and 'git describe' style IDs are used as version. 2021-09-08 18:03:50 +02:00
ui.scm tests: Adjust to 'show-manifest-transaction' changes. 2020-03-25 16:02:53 +01:00
union.scm
upstream.scm
utils.scm utils: Define a target-x86-32? and target-x86-64? predicate. 2021-11-07 01:38:23 -04:00
uuid.scm
workers.scm