TakeV
/
guix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

derivations: Add #:substitutable?, distinguished from #:local-build?. 

Fixes <http://bugs.gnu.org/18747>.

* guix/derivations.scm (substitutable-derivation?): Rewrite to check for
  "allowSubstitutes".
  (derivation): Add #:substitutable? parameter.
  [user+system-env-vars]: Honor it.
  (build-expression->derivation): Add #:substitutable? and honor it.
* guix/gexp.scm (gexp->derivation): Likewise.
* tests/derivations.scm ("derivation-prerequisites-to-build and substitutes,
  non-substitutable build"): Use #:substitutable? instead of #:local-build?.
  ("substitutable-derivation?", "derivation-prerequisites-to-build and
  substitutes, local build"): New tests.
* guix/download.scm (url-fetch): Adjust comment.
* guix/git-download.scm (git-fetch): Likewise.
* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): Use #:substitutable?
  instead of #:local-build?.
* doc/guix.texi (Derivations, G-Expressions): Adjust accordingly.
This commit is contained in:
Ludovic Courtès 2015-07-03 00:05:16 +02:00
parent 322eeb87d0
commit 4a6aeb670f
7 changed files with 76 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
doc/guix.texi
View File

@ -2582,7 +2582,8 @@ a derivation is the @code{derivation} procedure:
@var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ @var{args} [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
[#:recursive? #f] [#:inputs '()] [#:env-vars '()] @ [#:recursive? #f] [#:inputs '()] [#:env-vars '()] @
[#:system (%current-system)] [#:references-graphs #f] @ [#:system (%current-system)] [#:references-graphs #f] @
[#:allowed-references #f] [#:leaked-env-vars #f] [#:local-build? #f] [#:allowed-references #f] [#:leaked-env-vars #f] [#:local-build? #f] @
[#:substitutable? #t]
Build a derivation with the given arguments, and return the resulting Build a derivation with the given arguments, and return the resulting
@code{<derivation>} object. @code{<derivation>} object.
@ -2612,6 +2613,11 @@ When @var{local-build?} is true, declare that the derivation is not a
good candidate for offloading and should rather be built locally good candidate for offloading and should rather be built locally
(@pxref{Daemon Offload Setup}). This is the case for small derivations (@pxref{Daemon Offload Setup}). This is the case for small derivations
where the costs of data transfers would outweigh the benefits. where the costs of data transfers would outweigh the benefits.
When @var{substitutable?} is false, declare that substitutes of the
derivation's output should not be used (@pxref{Substitutes}). This is
useful, for instance, when building packages that capture details of the
host CPU instruction set.
@end deffn @end deffn
@noindent @noindent
@ -2651,7 +2657,7 @@ is now deprecated in favor of the much nicer @code{gexp->derivation}.
[#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @ [#:outputs '("out")] [#:hash #f] [#:hash-algo #f] @
[#:recursive? #f] [#:env-vars '()] [#:modules '()] @ [#:recursive? #f] [#:env-vars '()] [#:modules '()] @
[#:references-graphs #f] [#:allowed-references #f] @ [#:references-graphs #f] [#:allowed-references #f] @
[#:local-build? #f] [#:guile-for-build #f] [#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f]
Return a derivation that executes Scheme expression @var{exp} as a Return a derivation that executes Scheme expression @var{exp} as a
builder for derivation @var{name}. @var{inputs} must be a list of builder for derivation @var{name}. @var{inputs} must be a list of
@code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted, @code{(name drv-path sub-drv)} tuples; when @var{sub-drv} is omitted,
@ -2674,7 +2680,8 @@ terminates by passing the result of @var{exp} to @code{exit}; thus, when
@code{%guile-for-build} fluid is used instead. @code{%guile-for-build} fluid is used instead.
See the @code{derivation} procedure for the meaning of See the @code{derivation} procedure for the meaning of
@var{references-graphs}, @var{allowed-references}, and @var{local-build?}. @var{references-graphs}, @var{allowed-references}, @var{local-build?},
and @var{substitutable?}.
@end deffn @end deffn
@noindent @noindent
@ -3163,7 +3170,7 @@ information about monads.)
[#:module-path @var{%load-path}] @ [#:module-path @var{%load-path}] @
[#:references-graphs #f] [#:allowed-references #f] @ [#:references-graphs #f] [#:allowed-references #f] @
[#:leaked-env-vars #f] @ [#:leaked-env-vars #f] @
[#:local-build? #f] [#:guile-for-build #f] [#:local-build? #f] [#:substitutable? #t] [#:guile-for-build #f]
Return a derivation @var{name} that runs @var{exp} (a gexp) with Return a derivation @var{name} that runs @var{exp} (a gexp) with
@var{guile-for-build} (a derivation) on @var{system}. When @var{target} @var{guile-for-build} (a derivation) on @var{system}. When @var{target}
is true, it is used as the cross-compilation target triplet for packages is true, it is used as the cross-compilation target triplet for packages

10
guix/build-system/gnu.scm
View File

@ -367,10 +367,7 @@ are allowed to refer to."
#:inputs input-drvs #:inputs input-drvs
#:outputs outputs #:outputs outputs
#:modules imported-modules #:modules imported-modules
#:substitutable? substitutable?
;; XXX: Update when
;; <http://bugs.gnu.org/18747> is fixed.
#:local-build? (not substitutable?)
#:allowed-references #:allowed-references
(and allowed-references (and allowed-references
@ -513,10 +510,7 @@ platform."
#:inputs (append native-drvs target-drvs) #:inputs (append native-drvs target-drvs)
#:outputs outputs #:outputs outputs
#:modules imported-modules #:modules imported-modules
#:substitutable? substitutable?
;; XXX: Update when
;; <http://bugs.gnu.org/18747> is fixed.
#:local-build? (not substitutable?)
#:allowed-references #:allowed-references
(and allowed-references (and allowed-references

29
guix/derivations.scm
View File

@ -223,10 +223,13 @@ result is the set of prerequisites of DRV not already in valid."
(("preferLocalBuild" . "1") #f) (("preferLocalBuild" . "1") #f)
(_ #t))) (_ #t)))
(define substitutable-derivation? (define (substitutable-derivation? drv)
;; Return #t if the derivation can be substituted. Currently the two are "Return #t if DRV can be substituted."
;; synonymous, see <http://bugs.gnu.org/18747>. (match (assoc "allowSubstitutes"
offloadable-derivation?) (derivation-builder-environment-vars drv))
(("allowSubstitutes" . value)
(string=? value "1"))
(_ #t)))
(define (derivation-output-paths drv sub-drvs) (define (derivation-output-paths drv sub-drvs)
"Return the output paths of outputs SUB-DRVS of DRV." "Return the output paths of outputs SUB-DRVS of DRV."
@ -692,7 +695,8 @@ HASH-ALGO, of the derivation NAME. RECURSIVE? has the same meaning as for
(inputs '()) (outputs '("out")) (inputs '()) (outputs '("out"))
hash hash-algo recursive? hash hash-algo recursive?
references-graphs allowed-references references-graphs allowed-references
leaked-env-vars local-build?) leaked-env-vars local-build?
(substitutable? #t))
"Build a derivation with the given arguments, and return the resulting "Build a derivation with the given arguments, and return the resulting
<derivation> object. When HASH and HASH-ALGO are given, a <derivation> object. When HASH and HASH-ALGO are given, a
fixed-output derivation is created---i.e., one whose result is known in fixed-output derivation is created---i.e., one whose result is known in
@ -715,7 +719,10 @@ as \"http_proxy\" to be passed to derivations that download files.
When LOCAL-BUILD? is true, declare that the derivation is not a good candidate When LOCAL-BUILD? is true, declare that the derivation is not a good candidate
for offloading and should rather be built locally. This is the case for small for offloading and should rather be built locally. This is the case for small
derivations where the costs of data transfers would outweigh the benefits." derivations where the costs of data transfers would outweigh the benefits.
When SUBSTITUTABLE? is false, declare that substitutes of the derivation's
output should not be used."
(define (add-output-paths drv) (define (add-output-paths drv)
;; Return DRV with an actual store path for each of its output and the ;; Return DRV with an actual store path for each of its output and the
;; corresponding environment variable. ;; corresponding environment variable.
@ -753,6 +760,9 @@ derivations where the costs of data transfers would outweigh the benefits."
(let ((env-vars `(,@(if local-build? (let ((env-vars `(,@(if local-build?
`(("preferLocalBuild" . "1")) `(("preferLocalBuild" . "1"))
'()) '())
,@(if (not substitutable?)
`(("allowSubstitutes" . "0"))
'())
,@(if allowed-references ,@(if allowed-references
`(("allowedReferences" `(("allowedReferences"
. ,(string-join allowed-references))) . ,(string-join allowed-references)))
@ -1173,7 +1183,7 @@ applied."
guile-for-build guile-for-build
references-graphs references-graphs
allowed-references allowed-references
local-build?) local-build? (substitutable? #t))
"Return a derivation that executes Scheme expression EXP as a builder "Return a derivation that executes Scheme expression EXP as a builder
for derivation NAME. INPUTS must be a list of (NAME DRV-PATH SUB-DRV) for derivation NAME. INPUTS must be a list of (NAME DRV-PATH SUB-DRV)
tuples; when SUB-DRV is omitted, \"out\" is assumed. MODULES is a list tuples; when SUB-DRV is omitted, \"out\" is assumed. MODULES is a list
@ -1193,7 +1203,7 @@ EXP is built using GUILE-FOR-BUILD (a derivation). When GUILE-FOR-BUILD is
omitted or is #f, the value of the `%guile-for-build' fluid is used instead. omitted or is #f, the value of the `%guile-for-build' fluid is used instead.
See the `derivation' procedure for the meaning of REFERENCES-GRAPHS, See the `derivation' procedure for the meaning of REFERENCES-GRAPHS,
ALLOWED-REFERENCES, and LOCAL-BUILD?." ALLOWED-REFERENCES, LOCAL-BUILD?, and SUBSTITUTABLE?."
(define guile-drv (define guile-drv
(or guile-for-build (%guile-for-build))) (or guile-for-build (%guile-for-build)))
@ -1319,7 +1329,8 @@ ALLOWED-REFERENCES, and LOCAL-BUILD?."
#:outputs outputs #:outputs outputs
#:references-graphs references-graphs #:references-graphs references-graphs
#:allowed-references allowed-references #:allowed-references allowed-references
#:local-build? local-build?))) #:local-build? local-build?
#:substitutable? substitutable?)))
;;; ;;;

4
guix/download.scm
View File

@ -277,8 +277,8 @@ in the store."
;; In general, offloading downloads is not a good idea. ;; In general, offloading downloads is not a good idea.
;;#:local-build? #t ;;#:local-build? #t
;; FIXME: The above would also disable use of ;; FIXME: The above would also disable use of
;; substitutes, so comment it out; see ;; substitutes on old daemons, so comment it out;
;; <https://bugs.gnu.org/18747>. ;; see <https://bugs.gnu.org/18747>.
))))) )))))
(define* (download-to-store store url #:optional (name (basename url)) (define* (download-to-store store url #:optional (name (basename url))

5
guix/gexp.scm
View File

@ -320,7 +320,7 @@ names and file names suitable for the #:allowed-references argument to
references-graphs references-graphs
allowed-references allowed-references
leaked-env-vars leaked-env-vars
local-build?) local-build? (substitutable? #t))
"Return a derivation NAME that runs EXP (a gexp) with GUILE-FOR-BUILD (a "Return a derivation NAME that runs EXP (a gexp) with GUILE-FOR-BUILD (a
derivation) on SYSTEM. When TARGET is true, it is used as the derivation) on SYSTEM. When TARGET is true, it is used as the
cross-compilation target triplet for packages referred to by EXP. cross-compilation target triplet for packages referred to by EXP.
@ -439,7 +439,8 @@ The other arguments are as for 'derivation'."
#:references-graphs (and=> graphs graphs-file-names) #:references-graphs (and=> graphs graphs-file-names)
#:allowed-references allowed #:allowed-references allowed
#:leaked-env-vars leaked-env-vars #:leaked-env-vars leaked-env-vars
#:local-build? local-build?)))) #:local-build? local-build?
#:substitutable? substitutable?))))
(define* (gexp-inputs exp #:key native?) (define* (gexp-inputs exp #:key native?)
"Return the input list for EXP. When NATIVE? is true, return only native "Return the input list for EXP. When NATIVE? is true, return only native

1
guix/git-download.scm
View File

@ -90,6 +90,7 @@ HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f."
(gexp->derivation (or name "git-checkout") build (gexp->derivation (or name "git-checkout") build
#:system system #:system system
;; FIXME: See <https://bugs.gnu.org/18747>. ;; FIXME: See <https://bugs.gnu.org/18747>.
;; Uncomment when fixed daemons are widely deployed.
;;#:local-build? #t ;;#:local-build? #t
#:hash-algo hash-algo #:hash-algo hash-algo
#:hash hash #:hash hash

44
tests/derivations.scm
View File

@ -188,10 +188,22 @@
(test-assert "offloadable-derivation?" (test-assert "offloadable-derivation?"
(and (offloadable-derivation? (derivation %store "foo" %bash '())) (and (offloadable-derivation? (derivation %store "foo" %bash '()))
(offloadable-derivation? ;see <http://bugs.gnu.org/18747>
(derivation %store "foo" %bash '()
#:substitutable? #f))
(not (offloadable-derivation? (not (offloadable-derivation?
(derivation %store "foo" %bash '() (derivation %store "foo" %bash '()
#:local-build? #t))))) #:local-build? #t)))))
(test-assert "substitutable-derivation?"
(and (substitutable-derivation? (derivation %store "foo" %bash '()))
(substitutable-derivation? ;see <http://bugs.gnu.org/18747>
(derivation %store "foo" %bash '()
#:local-build? #f))
(not (substitutable-derivation?
(derivation %store "foo" %bash '()
#:substitutable? #f)))))
(test-assert "fixed-output-derivation?" (test-assert "fixed-output-derivation?"
(let* ((builder (add-text-to-store %store "my-fixed-builder.sh" (let* ((builder (add-text-to-store %store "my-fixed-builder.sh"
"echo -n hello > $out" '())) "echo -n hello > $out" '()))
@ -614,14 +626,11 @@
(null? download*) (null? download*)
(null? build*)))))) (null? build*))))))
(test-assert "derivation-prerequisites-to-build and substitutes, local build" (test-assert "derivation-prerequisites-to-build and substitutes, non-substitutable build"
(let* ((store (open-connection)) (let* ((store (open-connection))
(drv (build-expression->derivation store "prereq-subst-local" (drv (build-expression->derivation store "prereq-no-subst"
(random 1000) (random 1000)
;; XXX: Adjust once #:substitutable? #f))
;; <http://bugs.gnu.org/18747>
;; is fixed.
#:local-build? #t))
(output (derivation->output-path drv))) (output (derivation->output-path drv)))
;; Make sure substitutes are usable. ;; Make sure substitutes are usable.
@ -631,13 +640,34 @@
(let-values (((build download) (let-values (((build download)
(derivation-prerequisites-to-build store drv))) (derivation-prerequisites-to-build store drv)))
;; Despite being available as a substitute, DRV will be built locally ;; Despite being available as a substitute, DRV will be built locally
;; due to #:local-build?. ;; due to #:substitutable? #f.
(and (null? download) (and (null? download)
(match build (match build
(((? derivation-input? input)) (((? derivation-input? input))
(string=? (derivation-input-path input) (string=? (derivation-input-path input)
(derivation-file-name drv))))))))) (derivation-file-name drv)))))))))
(test-assert "derivation-prerequisites-to-build and substitutes, local build"
(with-store store
(let* ((drv (build-expression->derivation store "prereq-subst-local"
(random 1000)
#:local-build? #t))
(output (derivation->output-path drv)))
;; Make sure substitutes are usable.
(set-build-options store #:use-substitutes? #t)
(with-derivation-narinfo drv
(let-values (((build download)
(derivation-prerequisites-to-build store drv)))
;; #:local-build? is not be synonymous with #:substitutable?, so we
;; must be able to substitute DRV's output.
;; See <http://bugs.gnu.org/18747>.
(and (null? build)
(match download
(((? string? item))
(string=? item (derivation->output-path drv))))))))))
(test-assert "build-expression->derivation with expression returning #f" (test-assert "build-expression->derivation with expression returning #f"
(let* ((builder '(begin (let* ((builder '(begin
(mkdir %output) (mkdir %output)