Commit graph

1537 commits

Author SHA1 Message Date
Eelco Dolstra 81a4b4e49b * Implemented tryEval, the last missing primop in the fast-eval
branch.  Also added a test for tryEval.
2010-05-12 11:23:44 +00:00
Eelco Dolstra 1a8eb6e3ec 2010-05-07 15:26:33 +00:00
Eelco Dolstra 83dfa89870 * Sync with the trunk. 2010-05-07 14:46:47 +00:00
Eelco Dolstra 01e58adce0 * Store position info for inherited attributes. 2010-05-07 12:43:57 +00:00
Eelco Dolstra 83d7b89660 * Updated addErrorContext. 2010-05-07 12:33:14 +00:00
Eelco Dolstra e2d5e40f4f * Keep track of the source positions of attributes. 2010-05-07 12:11:05 +00:00
Eelco Dolstra 84ce7ac76f * Store attribute positions in the AST and report duplicate attribute
errors with position info.
* For all positions, use the position of the first character of the
  first token, rather than the last character of the first token plus
  one.
2010-05-06 16:46:48 +00:00
Eelco Dolstra a0e3b84fac * Revert r15436. This was a workaround for a bug in btrfs which seems
to have been fixed now.
2010-05-04 12:42:58 +00:00
Eelco Dolstra f92c9a0ac5 * Allow unprivileged users to do `nix-store --clear-failed-paths' and
`nix-store --query-failed-paths'.
2010-05-04 10:45:10 +00:00
Eelco Dolstra c82782f9a5 2010-04-27 09:05:11 +00:00
Eelco Dolstra ef337f7089 2010-04-26 12:58:12 +00:00
Eelco Dolstra 6199f9b93e * Added a command `nix-store --clear-failed-paths <PATHS>' to clear
the "failed" status of the given store paths.  The special value `*'
  clears all failed paths.
2010-04-26 12:56:42 +00:00
Eelco Dolstra 2398af13c5 * Add an command `nix-store --query-failed-paths' to list the cached
failed paths (when using the `build-cache-failure' option).
2010-04-26 12:43:42 +00:00
Eelco Dolstra 2be6118f4c * Don't need the test program anymore. 2010-04-23 09:09:32 +00:00
Eelco Dolstra 0bc468f195 * Simplify the implementation of `with'. This gives a 7% speedup in
evaluating the NixOS system configuration.
2010-04-22 15:08:09 +00:00
Eelco Dolstra ebade9ff8b * Check for duplicate attribute names / function arguments. `make
check' now succeeds :-)
* An attribute set such as `{ foo = { enable = true; };
  foo.port = 23; }' now parses.  It was previously rejected, but I'm
  too lazy to implement the check.  (The only reason to reject it is
  that the reverse, `{ foo.port = 23; foo = { enable = true; }; }', is
  rejected, which is kind of ugly.)
2010-04-22 11:02:24 +00:00
Eelco Dolstra 2d7636529f * String equality tests should take the context into account. All the
evaluation test cases now succeed.
2010-04-22 09:54:11 +00:00
Eelco Dolstra 6bbfe95e30 * Don't use an ostringstream in unparseDerivation(), because it's
slow.  A `nix-env -qa --drv-path \*' on Nixpkgs was spending 40% of
  its time in unparseDerivation() because of this (now 11%).
2010-04-21 19:25:50 +00:00
Eelco Dolstra 6f0f16497a * Fix the interpretation of ''\<character> in indented strings. 2010-04-21 16:18:27 +00:00
Eelco Dolstra 0777448ca6 * Fixed builtins.genericClosure. 2010-04-21 15:57:11 +00:00
Eelco Dolstra fe2d869e04 * Store user environment manifests as a Nix expression in
$out/manifest.nix rather than as an ATerm.

  (Hm, I thought I committed this two days ago...)
2010-04-21 15:08:58 +00:00
Eelco Dolstra d66ea83a76 * Sync with the trunk. 2010-04-20 09:20:29 +00:00
Eelco Dolstra f3b8833a48 * Drop the dependency on the ATerm library. 2010-04-19 14:51:58 +00:00
Eelco Dolstra efc7a579e8 * Don't use the ATerm library for parsing/printing .drv files. 2010-04-19 13:46:58 +00:00
Eelco Dolstra 55b5ddd3ca * Added parsing of manifests in ATerm format. 2010-04-19 12:10:04 +00:00
Eelco Dolstra b7ff69eb7c * Refactoring: move the user environment stuff into its own module. 2010-04-19 10:47:56 +00:00
Eelco Dolstra 5c31995bb8 * Updated some more primops. 2010-04-16 15:13:47 +00:00
Eelco Dolstra 8bb0210fea * _combineChannels shouldn't be an integer. 2010-04-16 14:07:52 +00:00
Eelco Dolstra 8ca4a001cb * Improve sharing a bit. 2010-04-16 14:03:26 +00:00
Eelco Dolstra 497e4ad126 * Remove some redundant tests. 2010-04-16 13:51:01 +00:00
Eelco Dolstra 02c1dac909 * In an nested with' where the inner with is a variable (with ...;
with someVar; ...'), the contents of the variable would be
  clobbered.  (The attributes in the outer `with' were added to the
  variable.)
2010-04-16 13:44:02 +00:00
Eelco Dolstra 04c4bd3624 * Store lists as lists of pointers to values rather than as lists of
values.  This improves sharing and gives another speed up.
  Evaluation of the NixOS system attribute is now almost 7 times
  faster than the old evaluator.
2010-04-15 00:37:36 +00:00
Eelco Dolstra e41b5828db * Better stats. 2010-04-14 23:48:46 +00:00
Eelco Dolstra d39d3c6264 * Implemented inherit. 2010-04-14 23:25:05 +00:00
Eelco Dolstra 267dc693d2 * Fix builtins. 2010-04-14 22:59:39 +00:00
Eelco Dolstra 81de12bc8f * Refactoring: move variable uses to a separate class. 2010-04-14 15:14:23 +00:00
Eelco Dolstra 110d155778 * Implemented withs. 2010-04-14 15:01:04 +00:00
Eelco Dolstra 9985230c00 * After parsing, compute level/displacement pairs for each variable
use site, allowing environments to be stores as vectors of values
  rather than maps.  This should speed up evaluation and reduce the
  number of allocations.
2010-04-14 14:42:32 +00:00
Eelco Dolstra 816dd3f061 * Remove more obsolete code. 2010-04-14 12:49:05 +00:00
Eelco Dolstra 011b5da0f4 * Get nix-env to compile again. 2010-04-14 09:39:06 +00:00
Eelco Dolstra 85d13c8f93 * Change the semantics of "with" so that inner "withs" take
precedence, i.e. `with {x=1;}; with {x=2;}; x' evaluates to 2'.
  This has a simpler implementation and seems more natural.  There
  doesn't seem to be any code in Nixpkgs or NixOS that relies on the
  old behaviour.
2010-04-14 08:37:08 +00:00
Eelco Dolstra 816f9c0f6f * Use std::tr1::unordered_set instead of std::set for the symbol
table.  This gives a 10% speed increase on `nix-instantiate
  /etc/nixos/nixos -A system --readonly-mode'.
2010-04-13 14:34:11 +00:00
Eelco Dolstra 7d47498b5e * Evaluate lets directly (i.e. without desugaring to `rec { attrs...;
<let-body> = e; }.<let-body>).  This prevents the unnecessary
  allocation of an attribute set.
2010-04-13 13:42:25 +00:00
Eelco Dolstra ac1e8f40d4 * Use a symbol table to represent identifiers and attribute names
efficiently.  The symbol table ensures that there is only one copy
  of each symbol, thus allowing symbols to be compared efficiently
  using a pointer equality test.
2010-04-13 12:25:42 +00:00
Eelco Dolstra 10e8b1fd15 * Finished the ATerm-less parser. 2010-04-12 23:33:23 +00:00
Eelco Dolstra 0d272fca79 * Remove some obsolete functions. 2010-04-12 23:31:47 +00:00
Eelco Dolstra d4f0b0fc6c * Indented strings. 2010-04-12 22:03:27 +00:00
Eelco Dolstra a60317f20f * More missing constructs. 2010-04-12 21:21:24 +00:00
Eelco Dolstra 4d6ad5be17 * Don't use ATerms for the abstract syntax trees anymore. Not
finished yet.
2010-04-12 18:30:11 +00:00
Eelco Dolstra ed711f73bc * Don't use ATerms to represent integers in the lexer. 2010-04-12 10:38:18 +00:00
Eelco Dolstra db90b88e65 * Hack to support builderDefs expressions. 2010-04-12 09:50:20 +00:00
Eelco Dolstra 4e49002576 * Doh. 2010-04-12 09:45:00 +00:00
Eelco Dolstra c3f228f296 2010-04-12 09:14:27 +00:00
Ludovic Courtès aac5fcfbb5 Re-add drvPath' and outPath' attributes to <derivation> XML nodes.
This fixes a regression introduced in r20882 ("Add source location
information to the XML output.").

* src/libexpr/expr-to-xml.cc (nix::printTermAsXML): Dereference the
  attribute RHS from "drvPath" and "outPath".
2010-04-09 21:30:55 +00:00
Eelco Dolstra f3dc7ab877 * Keep more statistics about stack space usage.
* Reduce stack space usage.
2010-04-09 12:00:49 +00:00
Eelco Dolstra b7b3dd55f9 * Remove a lot of dead code. 2010-04-08 11:41:19 +00:00
Eelco Dolstra 7e048eddf5 * Fix blackholing. If evaluation fails due to an assertion failure,
then the blackhole has to be removed to ensure that repeated
  evaluation of the same value gives an assertion failure again rather
  than an "infinite recursion" error.
2010-04-08 11:25:14 +00:00
Eelco Dolstra af2a372bb0 * Update autoCallFunction() and findAlongAttrPath(). 2010-04-07 15:47:06 +00:00
Eelco Dolstra 9a64454faa * expr-to-xml -> value-to-xml. 2010-04-07 13:59:45 +00:00
Eelco Dolstra fc92244ba8 * Implemented the primops necessary for generating the NixOS manual. 2010-04-07 13:55:46 +00:00
Eelco Dolstra a353aef0b1 * In eval(), don't use the target value `v' as a temporary.
Overwriting `v' breaks when the expression evaluation to an
  assertion failure or throw.
2010-04-06 14:15:29 +00:00
Eelco Dolstra a5ece7d016 * Removed the `~' operator. 2010-04-01 16:59:07 +00:00
Eelco Dolstra c172274e17 * Quick hack to make coerceToString work more or less correctly on
nested lists.  `nix-instantiate' can now evaluate the NixOS system
  derivation attribute correctly (in 2.1s on my laptop vs. 6.2s for
  the trunk).
2010-04-01 14:35:03 +00:00
Eelco Dolstra 7b851915bf * Improve sharing. 2010-04-01 12:04:57 +00:00
Eelco Dolstra 95cc417d76 * Functions are incomparable. 2010-04-01 10:55:36 +00:00
Eelco Dolstra 71f026292b * Make `derivation' lazy again for performance. It also turns out
that there are some places in Nixpkgs (php_configurable /
  composableDerivation, it seems) that call `derivation' with
  incorrect arguments (namely, the `name' attribute missing) but get
  away with it because of laziness.
2010-04-01 09:55:57 +00:00
Eelco Dolstra dc31305b38 * Fixed the trace primop and path comparison.
* Removed exprToString and stringToExpr because there is no ATerm
  representation to work on anymore (and exposing the internals of the
  evaluator like this is not a good idea anyway).
2010-03-31 20:09:20 +00:00
Eelco Dolstra 979f163615 * Handle string contexts. `nix-instantiate' can now correctly compute
the `firefoxWrapper' attribute in Nixpkgs, and it's about 3 times
  faster than the trunk :-)
2010-03-31 19:52:29 +00:00
Eelco Dolstra d8cd3115d8 * Get nix-env to compile. 2010-03-31 19:12:08 +00:00
Eelco Dolstra 55e207b2dc * Cache parse trees to prevent repeated parsing of imported Nix
expressions.
2010-03-31 16:14:32 +00:00
Eelco Dolstra 3d94be61ea * Implemented derivations. 2010-03-31 15:38:03 +00:00
Ludovic Courtès 09381cccff Make source location info in the XML output optional.
* src/libexpr/expr-to-xml.cc (nix::showAttrs): Add `location'
  parameter.  Provide location XML attributes when it's true.  Update
  callers.
  (nix::printTermAsXML): Likewise.

* src/libexpr/expr-to-xml.hh (nix::printTermAsXML): Update prototype;
  have `location' default to `false'.

* src/nix-instantiate/nix-instantiate.cc (printResult, processExpr): Add
  `location' parameter; update callers.
  (run): Add support for `--no-location'.

* src/nix-instantiate/help.txt: Update accordingly.

* tests/lang.sh: Invoke `nix-instantiate' with `--no-location' for the
  XML tests.

* tests/lang/eval-okay-toxml.exp, tests/lang/eval-okay-to-xml.nix: New
  files.
2010-03-31 12:38:31 +00:00
Eelco Dolstra 13c2adc897 * Implemented `rec { inherit ...; }'. 2010-03-31 11:05:39 +00:00
Eelco Dolstra 4c53ca2692 * Compare nulls. 2010-03-31 09:54:12 +00:00
Ludovic Courtès 471419d1fa Add source location information to the XML output.
* src/libexpr/expr-to-xml.cc (nix::showAttrs): Dereference the attribute
  RHS.  Add "path", "line", and "column" XML attributes to the node when
  source location information is available.
  (nix::printTermAsXML): Likewise for functions.
2010-03-31 08:29:05 +00:00
Ludovic Courtès eb07a4f1ee Escape `>' signs in the XML output.
* src/libutil/xml-writer.cc (nix::XMLWriter::writeAttrs): Escape `>'.
2010-03-31 08:29:01 +00:00
Eelco Dolstra 7f19e03c65 * More primops. 2010-03-30 22:39:48 +00:00
Eelco Dolstra 47df476daa * More operators / primops. 2010-03-30 18:05:54 +00:00
Eelco Dolstra c9170be2bd * More primops. 2010-03-30 15:18:20 +00:00
Eelco Dolstra c3aa615a5f * More primops. 2010-03-30 14:39:27 +00:00
Eelco Dolstra 5b72d8a749 * Implemented `map'. 2010-03-30 13:47:59 +00:00
Eelco Dolstra d78a05ab40 * Make `import' work. 2010-03-30 09:22:33 +00:00
Eelco Dolstra 31428c3a06 * Started integrating the new evaluator. 2010-03-29 14:37:56 +00:00
Eelco Dolstra 52090d2418 2010-03-29 10:13:51 +00:00
Eelco Dolstra e3f32ac5af 2010-03-29 09:43:55 +00:00
Eelco Dolstra 807a67bc74 2010-03-29 09:43:39 +00:00
Eelco Dolstra 392811eb8f * Strings. 2010-03-28 18:27:07 +00:00
Eelco Dolstra d96cdcea6b 2010-03-28 16:57:16 +00:00
Eelco Dolstra 3d2b835f30 * Implemented multi-argument primops. 2010-03-28 16:37:39 +00:00
Eelco Dolstra 45d822f29c * Primops (not yet finished). 2010-03-26 15:45:53 +00:00
Eelco Dolstra cad8726b2c * Implemented the ==' and !=' operators. These now use a deep
equality test, so they also work for (finite) attribute sets and
  lists.
2010-03-26 13:27:26 +00:00
Eelco Dolstra 8da118e4d0 * Measure stack usage. 2010-03-25 16:35:24 +00:00
Eelco Dolstra c2ba4313fb * Implemented lists. 2010-03-25 15:38:37 +00:00
Eelco Dolstra 25eedf085d * Quick and dirty implementation of with'. with e1; e2' is
basically desugared to `let <with> = e1; e2', and `lookupVar' looks
  in each <with> in the environment chain for an attribute with the
  specified name.
2010-03-25 14:51:04 +00:00
Eelco Dolstra 3c9f8fc9b6 * Don't convert variable names to strings. 2010-03-25 13:10:04 +00:00
Eelco Dolstra f450384ded * Implement blackholing. 2010-03-25 12:51:14 +00:00
Eelco Dolstra ef8bd919fc * Implement `...' and default function arguments. 2010-03-25 12:45:23 +00:00
Eelco Dolstra 8a10360c91 * Simplify @-patterns: only {attrs}@name' or name@{attrs}' are now
allowed.  So `name1@name2', `{attrs1}@{attrs2}' and so on are now no
  longer legal.  This is no big loss because they were not useful
  anyway.

  This also changes the output of builtins.toXML for @-patterns
  slightly.
2010-03-25 12:19:41 +00:00
Eelco Dolstra 7482349fe8 * Implemented attribute set pattern matches. 2010-03-24 23:40:00 +00:00
Eelco Dolstra 0fd3648d34 * Store values in environments. 2010-03-24 12:41:08 +00:00
Eelco Dolstra b70bd8fe56 * Reduce the number of value allocations in eval() by moving
responsibility for allocation of the result to the caller.
2010-03-24 12:11:38 +00:00
Eelco Dolstra d31c59eb17 * Plain lambdas. 2010-03-24 11:06:05 +00:00
Eelco Dolstra e8f7978274 2010-03-23 19:19:52 +00:00
Eelco Dolstra 0910ae9568 * Start of an evaluator that uses call-by-need (with thunk updating)
instead of (memoised) call-by-name.
2010-03-23 17:30:50 +00:00
Eelco Dolstra 141294ff38 * Clean up error messages in killUser(). 2010-03-19 11:36:34 +00:00
Eelco Dolstra f0c473c5f7 * Fix building on Cygwin (http://hydra.nixos.org/build/325071). 2010-03-16 13:01:52 +00:00
Nicolas Pierron 741b7577c1 Merge r20344 & r20346. 2010-03-14 11:58:07 +00:00
Lluís Batlle i Rossell 2fb0df83e9 Uh. somehow a Makefile tab got in as spaces. Fixing. 2010-03-11 21:22:52 +00:00
Lluís Batlle i Rossell 13cce8ec45 Making 'bin2c' to be built with the compiler for the local system.
I copied the configure.ac code about CC_FOR_BUILD from libX11.
2010-03-11 20:56:25 +00:00
Eelco Dolstra e020d80e4e * Sync with the trunk. 2010-03-11 15:45:05 +00:00
Ludovic Courtès 05e15049a5 Show the build user's group in /etc/group in chroots.
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Create
  /etc/group showing the build user's group.
2010-03-11 14:47:04 +00:00
Eelco Dolstra 070057c1b9 * Sync with the trunk. 2010-03-11 10:52:52 +00:00
Ludovic Courtès c752c9f41a Fix thinko in r20547.
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Fix the GID
  of the build user in /etc/passwd.
2010-03-11 10:33:04 +00:00
Ludovic Courtès 2e8eaca573 Clear supplementary groups of `nixbld' in /etc/passwd in chroots.
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Don't
  display any supplementary groups for `nixbld' in /etc/passwd.
2010-03-11 10:21:23 +00:00
Eelco Dolstra 1a65142ec4 * Remove a debug statement. 2010-03-10 12:46:25 +00:00
Eelco Dolstra 4c356acd04 * In `nix-store --export', abort if the contents of a path has
changed.  This prevents corrupt paths from spreading to other
  machines.  Note that checking the hash is cheap because we're
  hashing anyway (because of the --sign feature).
2010-03-09 14:32:03 +00:00
Eelco Dolstra 44f6e6de77 * Set gc-keep-outputs' or gc-keep-derivations' to false with
`--delete --ignore-liveness'.
2010-03-08 21:31:42 +00:00
Eelco Dolstra 2e4ef03aa3 * Increase the sqlite timeout. 2010-03-08 10:35:45 +00:00
Eelco Dolstra 04791840f4 * Emit warning='1' or error='1' attributes for lines marked as
warnings or errors with \e[w or \e[e.
2010-03-05 12:54:58 +00:00
Eelco Dolstra bc6f7fc139 * Remove some unused functions. 2010-03-04 13:03:26 +00:00
Eelco Dolstra fb6e223ddc * Synced with the trunk. 2010-03-04 12:15:00 +00:00
Eelco Dolstra e14e2399ed * Prevent a potential memory corruption problem if an ATerm garbage
collection happens during fixAttrs().
2010-03-02 20:23:42 +00:00
Eelco Dolstra 56af8e86e3 * Protect the true', false' and `__overrides' constants. Without
an ATprotect call, these could be garbage collected, leading to
  weird crashes or wrong results.
2010-03-02 20:09:12 +00:00
Eelco Dolstra 3f9e647ae8 * checkInterrupt() shouldn't be called from a destructor. 2010-03-02 19:04:17 +00:00
Eelco Dolstra 594eaddd11 * When using the included sqlite/aterm libraries, build with
--enable-shared.
* In libutil/libstore/libexpr etc., link against sqlite and aterm.
* Some more header file hygiene.
2010-03-02 15:58:13 +00:00
Eelco Dolstra 24035b98b1 * Implement RemoteStore::queryValidPaths(). 2010-02-26 12:05:01 +00:00
Eelco Dolstra e42401ee7b * Implement RemoteStore::queryDerivationOutputs(). 2010-02-25 15:52:22 +00:00
Eelco Dolstra af565c348a * Support read-only access to the database. 2010-02-24 16:44:43 +00:00
Eelco Dolstra e33f67ff0b * Refactor the upgrade / database initialisation logic a bit. 2010-02-24 16:30:20 +00:00
Eelco Dolstra 84a4dd5ff0 * Don't use fdatasync since it doesn't work on Snow Leopard.
* Don't refer to config.h in util.hh, because config.h is not
  installed (http://hydra.nixos.org/build/303053).
2010-02-24 15:46:06 +00:00
Eelco Dolstra cfe742cfc5 * A function to query just the database id of a valid path. 2010-02-24 15:07:23 +00:00
Eelco Dolstra 6baa2a2f5e 2010-02-24 14:39:52 +00:00
Eelco Dolstra 9fd85c94de * Use `truncate' journal mode, which should be a bit faster. 2010-02-24 14:22:34 +00:00
Eelco Dolstra fefd467539 * `helpText' is now zero-terminated. 2010-02-24 13:24:27 +00:00
Eelco Dolstra a3c63d0d6c * Disable fsync() in SQLite if the fsync-metadata option is set to
false.
* Change the default for `fsync-metadata' to true.
* Disable `fsync-metadata' in `make check'.
2010-02-24 13:12:57 +00:00
Eelco Dolstra 90b6352d0a * Do registerValidPaths() in one transaction, which is much faster.
E.g. it cuts the runtime of the referrers test from 50s to 23s.
2010-02-24 12:48:00 +00:00
Eelco Dolstra fae0427324 * ADDITIONAL_NETWORK_LIBS -> LIBS. 2010-02-24 12:25:48 +00:00
Eelco Dolstra fa6a4fcb11 * Add ${sqlite_lib} everywhere. Just adding it in `libstore' doesn't
work on x86_64 when sqlite is compiled statically.
2010-02-24 12:18:48 +00:00
Eelco Dolstra 5954eadf67 * Remove the fdatasync check since it's no longer needed. 2010-02-24 12:16:50 +00:00
Eelco Dolstra 462bd50aef * Use normal (rather than full) synchronous mode, which I gather from
the description at http://www.sqlite.org/atomiccommit.html should be
  safe enough.
2010-02-24 10:57:57 +00:00
Eelco Dolstra 63b09c5e41 2010-02-23 22:31:38 +00:00
Eelco Dolstra b4e6d98fc3 * configure: flag --with-sqlite. 2010-02-23 22:12:46 +00:00
Eelco Dolstra 2b20318b0e 2010-02-22 14:24:37 +00:00
Eelco Dolstra 9cda616949 * The database needs a trigger to get rid of self-references to
prevent a foreign key constraint violation on the Refs table when
  deleting a path.
2010-02-22 14:18:55 +00:00
Eelco Dolstra c4d388add4 * Get derivation outputs from the database instead of the .drv file,
which requires more I/O.
2010-02-22 12:44:36 +00:00
Eelco Dolstra 103cfee056 * Revert r19650 (implement gc-keep-outputs by looking for derivations
with the same name as the output) and instead use the
  DerivationOutputs table in the database, which is the correct way to
  to do things.
2010-02-22 11:44:17 +00:00
Eelco Dolstra 299ff64812 * Put the derivation outputs in the database. This is useful for the
garbage collector.
2010-02-22 11:15:50 +00:00
Eelco Dolstra 1930570ad9 * Foreign key support in SQLite is not a persistent setting, so enable
it at startup.
* Implement negative caching.  Now `make check' passes.
2010-02-19 17:15:22 +00:00
Eelco Dolstra 9c9a88e9e2 * Implement more stuff. 2010-02-19 16:43:25 +00:00
Eelco Dolstra 762cee72cc * Implement registerValidPath(). 2010-02-19 16:04:51 +00:00
Eelco Dolstra 268f9aaf28 * Implemented queryValidPaths() and verifyStore(). 2010-02-18 16:51:27 +00:00
Eelco Dolstra 836e5b6f57 * Implemented queryReferrers(). 2010-02-18 16:21:59 +00:00
Eelco Dolstra 77cb9e3fb1 * Implement queryPathInfo(). 2010-02-18 15:52:57 +00:00
Eelco Dolstra 885e22b16e * Implement isValidPath(). 2010-02-18 15:11:08 +00:00
Eelco Dolstra cfb09e0fad * Automatically abort transactions if they go out of scope without
committing.
2010-02-18 14:40:07 +00:00
Eelco Dolstra e0305bb7a8 * Some wrapper objects to ensure that SQLite objects are properly
destroyed.
2010-02-18 14:30:42 +00:00
Eelco Dolstra a053d2d8e5 * Add the deriver to the ValidPaths table. In principle we could now
store all the derivers of a path efficiently.  But that opens a big
  can of worms with respect to garbage collector semantics.
2010-02-18 13:48:18 +00:00
Eelco Dolstra dbddac0fe9 * Assign an integer id to every row in the ValidPaths table in order
to make the Refs table more space-efficient.  For instance, this
  reduces the size of the database on my laptop from 93 MiB to 18
  MiB.  (It was 72 MiB with the old schema on an ext3 disk with a 1
  KiB block size.)
2010-02-18 13:40:46 +00:00
Eelco Dolstra c1a07f9445 * Convert the Nix database to SQLite. 2010-02-18 13:16:59 +00:00
Ludovic Courtès 20186a4079 Don't rely on `PATH_MAX' on GNU. 2010-02-10 15:55:50 +00:00
Eelco Dolstra 4e17be7981 * Revert r19797, and use a simpler solution: just don't monitor build
hooks for silence.  It's unnecessary because the remote nix-store
  command is already monitoring the real build.
2010-02-03 21:38:41 +00:00
Eelco Dolstra f859a8d3c3 * While waiting for a lock, print a sign of life every 5 minutes.
This prevents remote builders from being killed by the
  `max-silent-time' inactivity monitor while they are waiting for a
  long garbage collection to finish.  This happens fairly often in the
  Hydra build farm.
2010-02-03 21:22:57 +00:00
Eelco Dolstra 4bbbe25802 * Remove most Cygwin-specific code. Cygwin 1.7 implements advisory
POSIX locks, and simulates Unix-style file deletion semantics
  sufficiently.  Note that this means that Nix won't work on Cygwin
  1.5 anymore.
2010-02-02 15:28:36 +00:00
Eelco Dolstra 2723d9b56e * If fdatasync() isn't available, use fsync(). 2010-02-02 11:57:49 +00:00
Eelco Dolstra 07ffdc2862 * Added an option "fsync-metadata" to fsync() changes to
/nix/var/nix/db.
* Removed the function writeStringToFile since it does (almost) the
  same thing as writeFile.
2010-01-29 12:22:58 +00:00
Eelco Dolstra ad529fb89f * Don't consider a store path valid if its info file exists but is
zero bytes long.  That makes Nix more robust in case of crashes
  (especially on ext4).
2010-01-29 11:53:58 +00:00
Eelco Dolstra fdcaf37361 * Made `nix-store -qR --include-outputs' much faster if there are
multiple paths specified on the command line (from O(n * m) to O(n +
  m), where n is the number of arguments and m is the size of the
  closure).
2010-01-25 17:18:44 +00:00
Eelco Dolstra 5388944e8d * Make the garbage collector do the right thing when `gc-keep-outputs'
is enabled by not depending on the deriver.
2010-01-25 16:04:32 +00:00
Eelco Dolstra f0c0277970 * On startup, set the default SIGCHLD handler. This is so that Nix
works correctly in weird environments where the SIGCHLD handler is
  set to "ignore".
2010-01-12 12:22:38 +00:00
Eelco Dolstra ef92a14bfe * Include config.h before the C library headers, because it defines
_FILE_OFFSET_BITS=64.  Without it, functions like stat() fail on
  large file sizes.  This happened with a Nix store on squashfs:

  $ nix-store --dump /tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds > /dev/null
  error: getting attributes of path `/tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds': Value too large for defined data type

  $ stat /tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds
  File: `/tmp/mnt/46wzqnk4cbdwh1dclhrpqnnz1icak6n7-local-net-cmds'
  Size: 0               Blocks: 36028797018963968 IO Block: 1024   regular empty file

  (This is a bug in squashfs or mksquashfs, but it shouldn't cause Nix
  to fail.)
2009-12-17 14:12:44 +00:00
Eelco Dolstra 945d8218fb * Build correctly against newer ATerm releases. Fixes "error: 'union'
tag used in naming 'struct _ATerm'".
2009-12-16 15:29:50 +00:00
Eelco Dolstra bcd6cdf0d8 * Give a better error message when trying to build something and
readOnlyMode is set.
2009-12-09 17:45:22 +00:00
Eelco Dolstra 13618b191e * Grrr. 2009-11-24 13:28:46 +00:00
Eelco Dolstra aa5a768720 * GCC 4.4 is stricter about the EOF macro
(http://hydra.nixos.org/build/156340).
2009-11-24 12:56:26 +00:00
Eelco Dolstra 9b8fda796b * Templatise getIntArg / string2Int. 2009-11-24 12:26:25 +00:00
Eelco Dolstra 8022015552 * In the garbage collector, don't count files with a link count > 1 in
the "bytes/blocks freed" statistics.
2009-11-24 10:51:52 +00:00
Eelco Dolstra f9e766db98 * Randomise the order in which we delete entries to make the collector
less biased towards deleting paths that come alphabetically first
  (e.g. /nix/store/000...).  This matters when using --max-freed etc.
2009-11-24 09:53:18 +00:00
Eelco Dolstra ca50c83fbb 2009-11-23 21:21:29 +00:00
Eelco Dolstra 3d55f1eb57 * A command `nix-store --query --roots <paths>' to find the garbage
collector roots that point (directly or indirectly) to the given
  paths.
2009-11-23 18:16:25 +00:00
Eelco Dolstra ae6bf87273 * `nix-store --gc --print-roots': also print the path of the actual
root symlink, not just its target.  E.g.:

  /nix/var/nix/profiles/system-99-link -> /nix/store/76kwf88657nq7wgk1hx3l1z5q91zb9wd-system
2009-11-23 17:23:12 +00:00
Eelco Dolstra c364d5d1e3 * Made the garbage collector a lot faster. It no longer computes the
complete set of live and dead paths before starting the actual
  deletion, but determines liveness on demand.  I.e. for any path in
  the store, it first tries to delete all the referrers, and then the
  path itself.  This means that the collector can start deleting paths
  almost immediately.
2009-11-23 16:34:24 +00:00
Eelco Dolstra 8824d60fe5 * Remove the --use-atime / --max-atime garbage collector flags. Many
(Linux) machines no longer maintain the atime because it's too
  expensive, and on the machines where --use-atime is useful (like the
  buildfarm), reading the atimes on the entire Nix store takes way too
  much time to make it practical.
2009-11-20 17:12:38 +00:00
Eelco Dolstra 327a232c85 * Remove support for old (before Nix 0.12pre12020) databases. 2009-11-06 01:15:44 +00:00
Eelco Dolstra 8520542071 * When building in a chroot, make a copy of a file if hard-linking
fails.  This is likely to happen after a `nix-store --optimise',
  because some files may have 32000 links (NIX-111).
2009-10-22 08:28:33 +00:00
Eelco Dolstra 6b9f6b0222 * Remove a prototype for a function that no longer exists. 2009-10-22 08:12:38 +00:00
Eelco Dolstra deb342fb08 * builtins.trace: in the common case that the value is a string, then
show the string, not the ATerm, so we get `trace: bla' instead of
  `trace: Str("bla",[])'.
2009-10-22 08:10:12 +00:00
Eelco Dolstra 437077c39d * Added a primop unsafeDiscardOutputDependency needed by Disnix to
pass derivation paths to a builder without actually building them.
2009-10-21 15:05:30 +00:00
Eelco Dolstra 6f7d7bc1de * Give a useful error message when an evaluation error occurs while
trying to upgrade a package.
2009-10-13 09:30:17 +00:00
Sander van der Burg 53a4981fa2 Added optional parameter which adds -lnsl -lsocket to make the Nix package manager work on OpenSolaris 2009-10-08 14:50:37 +00:00
Eelco Dolstra 96f1517831 * Support platforms that don't have O_ASYNC (e.g. OpenSolaris
apparently).
2009-09-30 11:32:04 +00:00
Eelco Dolstra 1a8f8fd86f * OpenSolaris compatibility. 2009-09-30 09:54:29 +00:00
Eelco Dolstra c7057fc1f2 * And some more. 2009-09-24 07:39:55 +00:00
Eelco Dolstra 193f59e077 * Fix a build failure on Fedora 11. rename() needs <stdio.h>. 2009-09-24 07:21:29 +00:00
Eelco Dolstra 1332dd1ed3 * tryEval shouldn't catch all exceptions of type Error, since not all
of them leave the evaluator in a continuable state.  Also, it should
  be less chatty.
2009-09-23 19:19:26 +00:00
Eelco Dolstra 676e07902e * Darwin hack. 2009-09-23 18:04:55 +00:00
Eelco Dolstra 64e89980e8 * Create some state directories automatically as a convenience. 2009-09-23 17:05:51 +00:00
Rob Vermaas 48b58617e9 * include wait.h for WEXITSTATUS 2009-09-23 12:57:15 +00:00
Eelco Dolstra 0dbd4638e0 * Two primops: builtins.intersectAttrs and builtins.functionArgs.
intersectAttrs returns the (right-biased) intersection between two
  attribute sets, e.g. every attribute from the second set that also
  exists in the first.  functionArgs returns the set of attributes
  expected by a function.

  The main goal of these is to allow the elimination of most of
  all-packages.nix.  Most package instantiations in all-packages.nix
  have this form:

    foo = import ./foo.nix {
      inherit a b c;
    };

  With intersectAttrs and functionArgs, this can be written as:

    foo = callPackage (import ./foo.nix) { };

  where

   callPackage = f: args:
     f ((builtins.intersectAttrs (builtins.functionArgs f) pkgs) // args);

  I.e., foo.nix is called with all attributes from "pkgs" that it
  actually needs (e.g., pkgs.a, pkgs.b and pkgs.c).  (callPackage can
  do any other generic package-level stuff we might want, such as
  applying makeOverridable.)  Of course, the automatically supplied
  arguments can be overriden if needed, e.g.

    foo = callPackage (import ./foo.nix) {
      c = c_version_2;
    };

  but for the vast majority of packages, this won't be needed.

  The advantages are to reduce the amount of typing needed to add a
  dependency (from three sites to two), and to reduce the number of
  trivial commits to all-packages.nix.  For the former, there have
  been two previous attempts:

    - Use "args: with args;" in the package's function definition.
      This however obscures the actual expected arguments of a
      function, which is very bad.

    - Use "{ arg1, arg2, ... }:" in the package's function definition
      (i.e. use the ellipis "..." to allow arbitrary additional
      arguments), and then call the function with all of "pkgs" as an
      argument.  But this inhibits error detection if you call it with
      an misspelled (or obsolete) argument.
2009-09-15 13:01:46 +00:00
Michael Raskin 3bca8931e8 Adding tryEval builtin. It allows to catch presence of errors in an expression. 2009-08-25 16:06:46 +00:00
Eelco Dolstra c6890d6b5c * Replace newlines in table output with spaces. 2009-07-02 08:52:12 +00:00
Eelco Dolstra 749dd97a54 * Support integers and lists of strings in meta fields. This is
useful for fields like meta.maintainers, meta.priority (which can be
  a proper integer now) and even meta.license (if there are multiple
  licenses).
2009-06-30 15:53:39 +00:00
Eelco Dolstra f2c3fc5191 * Don't show trace information by default (`--show-trace' to enable).
NixOS evaluation errors in particular look intimidating and
  generally aren't very useful.  Ideally the builtins.throw messages
  should be self-contained.
2009-06-30 13:28:29 +00:00
Eelco Dolstra 14bc3ce3d6 * Canonicalise timestamps in the Nix store to 1 (1970-01-01 00:00:01
UTC) rather than 0 (00:00:00).  1 is a better choice because some
  programs use 0 as a special value.  For instance, the Template
  Toolkit uses a timestamp of 0 to denote the non-existence of a file,
  so it barfs on files in the Nix store (see
  template-toolkit-nix-store.patch in Nixpkgs).  Similarly, Maya 2008
  fails to load script directories with a timestamp of 0 and can't be
  patched because it's closed source.

  This will also shut up those "implausibly old time stamp" GNU tar
  warnings.
2009-06-13 16:30:58 +00:00
Eelco Dolstra a64bbe049e * Change the scoping of "inherit (e) ..." in recs so that the
attributes of the rec are in scope of `e'.  This is useful in
  expressions such as

    rec {
      lib = import ./lib;
      inherit (lib) concatStrings;
    }

  It does change the semantics of expressions such as

    let x = {y = 1;}; in rec { x = {y = 2;}; inherit (x) y; }.y

  This now returns 2 instead of 1.  However, no code in Nixpkgs or
  NixOS seems to rely on the old behaviour.
2009-05-15 13:46:13 +00:00
Eelco Dolstra d407d572fd * Some syntactic sugar for attribute sets: allow {x.y.z = ...;} as a
shorthand for {x = {y = {z = ...;};};}.  This is especially useful
  for NixOS configuration files, e.g.

    {
      services = {
        sshd = {
          enable = true;
          port = 2022;
        };
      };
    }

  can now be written as

    {
      services.sshd.enable = true;
      services.sshd.port = 2022;
    }

  However, it is currently not permitted to write
  
    {
      services.sshd = {enable = true;};
      services.sshd.port = 2022;
    }

  as this is considered a duplicate definition of `services.sshd'.
2009-05-15 12:35:23 +00:00
Eelco Dolstra e42975490f * Check for duplicate attributes in fixAttrs, rather than doing a
separate traversal after parsing.  Likewise, check for duplicate
  pattern variables right away.
2009-05-14 14:29:45 +00:00
Eelco Dolstra 50d11b90ca * Allow unsafe (unspecified) comparisons between attrsets unless
NIX_NO_UNSAFE_EQ is set, for now.
2009-05-12 11:06:24 +00:00
Eelco Dolstra c34e6d71bc * Disallow equality tests between attribute sets. This was always
broken, but now the evaluator checks for it to prevent Nix
  expressions from relying on undefined behaviour.  Equality tests are
  implemented using a shallow pointer equality test between ATerms.
  However, because attribute sets are lazy and contain position
  information, this can give false positives.  For instance,
  previously

    let y = {x = 1;}; in y == y

  evaluated to true, while the equivalent expression

    {x = 1;} == {x = 1;}

  evaluated to false.  So disallow these tests for now.  (Eventually
  we may want to implement deep equality tests for attribute sets,
  like lib.eqStrict.)
  
* Idem: disallow comparisons between functions.

* Implemented deep comparisons of lists.  This had the same problem as
  attribute sets - the elements in the list weren't evaluated.  For
  instance,

    ["xy"] == [("x" + "y")]

  evaluated to false.  Now it works properly.
2009-05-11 15:50:14 +00:00
Eelco Dolstra 7660e2a068 * Remove a right recursion that causes the parser to barf on very long
lists.  The comment about ATreverse requiring unbounded stack space
  was unfounded anyway.
2009-05-07 11:35:52 +00:00
Michael Raskin 098cb9d233 Add an ftruncate call paired with fallocate to play safe with some FSes (namely, BtrFS fallocate sets file size to allocated size, i.e. multiple of block size) 2009-05-04 08:10:24 +00:00
Eelco Dolstra d5eab2fc82 * Use foreach in a lot of places. 2009-04-21 11:52:16 +00:00
Eelco Dolstra 8f1bf28505 * nix-store --verify: don't bail out if a referenced path is missing.
(It can't fix it though.)
2009-04-21 11:06:27 +00:00
Eelco Dolstra 4e646b0ddb * Fix a few "comparison is always false/true due to limited range of
data type" warnings on 64-bit platforms.  The one in parser.y is
  likely to be a real bug.
2009-04-16 12:03:17 +00:00
Eelco Dolstra dfb863f333 * Don't cache transient build hook problems. 2009-04-15 06:25:02 +00:00
Eelco Dolstra 351bf658f9 * Do a substitution even if --max-jobs == 0. 2009-03-31 21:14:07 +00:00
Eelco Dolstra cff2b2a13a * Doh! 2009-03-31 15:50:03 +00:00
Eelco Dolstra 160a60d663 2009-03-30 19:35:55 +00:00
Eelco Dolstra 805144b705 * Make the poll interval configurable. 2009-03-30 11:34:03 +00:00
Eelco Dolstra 7377195297 * With `--max-jobs 0', print a nicer error message than "Assertion
`!awake.empty()' failed."
2009-03-29 18:06:00 +00:00
Eelco Dolstra 737423a89c * Use polling to wait for a remote build slot when using a build hook
(that is, call the build hook with a certain interval until it
  accepts the build).
* build-remote.pl was totally broken: for all system types other than
  the local system type, it would send all builds to the *first*
  machine of the appropriate type.
2009-03-29 18:00:45 +00:00
Eelco Dolstra 6e946c8e72 * Scan for references and compute the SHA-256 hash of the output in
one pass.  This halves the amount of I/O.
2009-03-28 20:51:33 +00:00
Eelco Dolstra c7152c8f97 * Don't use the non-standard __gnu_cxx::stdio_filebuf class. 2009-03-28 19:41:53 +00:00
Eelco Dolstra 3a2bbe7f8a * Simplify communication with the hook a bit (don't use file
descriptors 3/4, just use stdin/stderr).
2009-03-28 19:29:55 +00:00
Eelco Dolstra f54e800366 * The `fixedOutput' variable didn't get initialised when using a build
hook, causing negative caching to fail randomly.
2009-03-28 16:12:27 +00:00
Eelco Dolstra cb85bc396e * Create /nix/store if it doesn't exist. 2009-03-27 14:19:04 +00:00
Eelco Dolstra 5dd8fb2069 * Don't use ULLONG_MAX in maxFreed - use 0 to mean "no limit".
18446744073709551615ULL breaks on GCC 3.3.6 (`integer constant is
  too large for "long" type').
2009-03-26 11:02:07 +00:00
Eelco Dolstra 92f525ecf4 * Negative caching, i.e. caching of build failures. Disabled by
default.  This is mostly useful for Hydra.
2009-03-25 21:05:42 +00:00
Eelco Dolstra a046858a22 * Doh. 2009-03-24 23:40:24 +00:00
Eelco Dolstra 12c8c64efa 2009-03-24 14:07:37 +00:00
Eelco Dolstra 3a87163b24 * Shut up a GCC warning. 2009-03-23 13:52:52 +00:00
Eelco Dolstra cacff1be88 * No longer block while waiting for a lock on a store path. Instead
poll for it (i.e. if we can't acquire the lock, then let the main
  select() loop wait for at most a few seconds and then try again).
  This improves parallelism: if two nix-store processes are both
  trying to build a path at the same time, the second one shouldn't
  block; it should first see if it can build other goals.  Also, it
  prevents the deadlocks that have been occuring in Hydra lately,
  where a process waits for a lock held by another process that's
  waiting for a lock held by the first.

  The downside is that polling isn't really elegant, but POSIX doesn't
  provide a way to wait for locks in a select() loop.  The only
  solution would be to spawn a thread for each lock to do a blocking
  fcntl() and then signal the main thread, but that would require
  pthreads.
2009-03-23 01:05:54 +00:00
Eelco Dolstra 58969fa2bf * Refactoring. 2009-03-22 23:53:05 +00:00
Eelco Dolstra 7a57b2920b * Better error message when nix-store --import is applied to garbage
(previously it would likely say "implementation cannot deal with >
  32-bit integers").
2009-03-22 17:51:45 +00:00
Eelco Dolstra 77d272623f * NAR archives: handle files larger than 2^32 bytes. Previously it
would just silently store only (fileSize % 2^32) bytes.
* Use posix_fallocate if available when unpacking archives.
* Provide a better error message when trying to unpack something that
  isn't a NAR archive.
2009-03-22 17:36:43 +00:00
Eelco Dolstra 2897286487 * Unify exportReferencesGraph and exportBuildReferencesGraph, and make
sure that it works as expected when you pass it a derivation.  That
  is, we have to make sure that all build-time dependencies are built,
  and that they are all in the input closure (otherwise remote builds
  might fail, for example).  This is ensured at instantiation time by
  adding all derivations and their sources to inputDrvs and inputSrcs.
2009-03-18 17:36:42 +00:00
Eelco Dolstra c183ee5c79 * Acquire the locks on the output paths before trying to run the build
hook.  This fixes a problem with log files being partially or
  completely filled with 0's because another nix-store process
  truncates the log file.  It should also be more efficient.
2009-03-18 14:48:42 +00:00
Eelco Dolstra 33ecb42991 * Cleanup. 2009-03-17 11:42:55 +00:00
Eelco Dolstra d4753c944f 2009-03-06 11:01:45 +00:00
Eelco Dolstra 0e6f604178 * Install some headers in the right location. 2009-03-05 14:57:50 +00:00
Eelco Dolstra 60cb7de336 * Allow options from the Nix config file to be overriden from the
command line (e.g. "--option build-use-chroot true").
2009-02-27 11:04:41 +00:00
Eelco Dolstra 824b154ce8 * Release output locks as soon as possible, not when the destructor of
the DerivationGoal runs.  Otherwise, if a goal is a top-level goal,
  then the lock won't be released until nix-store finishes.  With
  --keep-going and lots of top-level goals, it's possible to run out
  of file descriptors (this happened sometimes in the build farm for
  Nixpkgs).  Also, for failed derivation, it won't be possible to
  build it again  until the lock is released.
  
* Idem for locks on build users: these weren't released in a timely
  manner for failed top-level derivation goals.  So if there were more
  than (say) 10 such failed builds, you would get an error about
  having run out of build users.
2009-02-16 09:24:20 +00:00
Marc Weber 1407a1ec99 added primop functions __isBool, __isString, __isInt 2009-02-05 19:35:40 +00:00
Eelco Dolstra b682fae9d9 * Build hooks: use nix-store --import. This prevents a redundant
scan for runtime dependencies (i.e. the local machine shouldn't do a
  scan that the remote machine has already done).  Also pipe directly
  into `nix-store --import': don't use a temporary file.
2009-02-02 17:24:10 +00:00
Nicolas Pierron 110606d470 Add the "addErrorContext" builtin to add more information in the stack trace. 2009-01-27 14:36:44 +00:00
Eelco Dolstra 061141e632 * Make it compile on Debian 4.0 (which doesn't define PER_LINUX32_3GB
in sys/personality.h).
2009-01-27 13:36:59 +00:00
Eelco Dolstra 019176137f * When using a build hook, distinguish between transient failures
(e.g. an SSH connection problem) and permanent failures (i.e. the
  builder failed).  This matters to Hydra (it wants to know whether it
  makes sense to retry a build).
2009-01-13 11:39:09 +00:00
Eelco Dolstra c504d90c11 * Support i686-linux builds directly on x86_64-linux Nix
installations.  This is implemented using the personality() syscall,
  which causes uname to return "i686" in child processes.
2009-01-12 16:30:32 +00:00
Eelco Dolstra 8e39d9bdb3 * Make Nix build with Bison 2.4. 2009-01-12 12:51:28 +00:00
Eelco Dolstra 8fce03e0ad * nix-store --verify: repair bad hash fields in the metadata file. 2008-12-16 13:28:18 +00:00
Eelco Dolstra 60ec75048a * Pass --use-atime / --max-atime to the daemon. 2008-12-16 12:23:35 +00:00