possible regression: "stack overflow (possible infinite recursion)" #9672

NobbZ · 2023-12-28T12:41:08Z

Describe the bug

When trying to nix build gitlab:nobodyinperson/annextimelog with a recent build from master, an error is printed:

$ nix build gilab:nobodyinperson/annextimelog
copying '/nix/store/p418a043fs5smlx6cgj9z4q57kxlmym8-source' to the storeerror: stack overflow (possible infinite recursion)

2.18.1 worked fine and produces a proper result link. I am currently trying to bisect the issue. Which is not quite easy, as the error only occurs, when parts of the result are not available in the store, and I have not yet figured out what exactly needs to get removed, so I have to do a full GC each cycle (and building nix takes another huge amount of time)

Steps To Reproduce

nix build gilab:nobodyinperson/annextimelog with a nix "nightly"
See error

Expected behavior

A result links is created, like it got created during the 2.18.1

nix-env --version output

$ nix --version
nix (Nix) 2.20.0pre20231227_a21c762

Additional context

So far this one flake is the only that made me have the problem.

Priorities

Add 👍 to issues you find important.

The text was updated successfully, but these errors were encountered:

NobbZ · 2023-12-28T15:16:41Z

bisection ended with ea95327 being the first bad commit.

commit ea95327e72f5781295417b0eae46a5e351bebebd
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Nov 30 16:16:17 2023 +0100

    Move restricted/pure-eval access control out of the evaluator and into the accessor

 src/libcmd/installables.cc             |   7 +-
 src/libexpr/eval.cc                    | 103 ++++++----------------------
 src/libexpr/eval.hh                    |  25 +++----
 src/libexpr/parser.y                   |  19 +++++-
 src/libexpr/primops.cc                 | 119 +++++++++++++++------------------
 src/nix-build/nix-build.cc             |   7 +-
 src/nix-instantiate/nix-instantiate.cc |   2 +-
 tests/functional/restricted.sh         |   4 +-
 8 files changed, 115 insertions(+), 171 deletions(-)

git bisect log
git bisect start
# status: waiting for both good and bad commits
# good: [f5f4de6a550327b4b1a06123c2e450f1b92c73b6] Merge pull request #9086 from NixOS/backport-9081-to-2.18-maintenance
git bisect good f5f4de6a550327b4b1a06123c2e450f1b92c73b6
# status: waiting for bad commit, 1 good commit known
# bad: [a21c762dab365049b77af95355ee4236d173e216] Merge pull request #9666 from unblevable/dervation-typo
git bisect bad a21c762dab365049b77af95355ee4236d173e216
# good: [d40e91440d53a982b8c72b176c14f2a39c3e04d1] Merge pull request #9002 from NixOS/release-notes
git bisect good d40e91440d53a982b8c72b176c14f2a39c3e04d1
# good: [77dceb2844276217bff321d80f601297f3581530] Drop obsolete assert and cast
git bisect good 77dceb2844276217bff321d80f601297f3581530
# good: [5fe2accb754249df6cb8f840330abfcf3bd26695] fix up release note
git bisect good 5fe2accb754249df6cb8f840330abfcf3bd26695
# bad: [ff992f8b4604c28c479a1511cb789dd912494172] Merge remote-tracking branch 'upstream/master' into package-nix
git bisect bad ff992f8b4604c28c479a1511cb789dd912494172
# bad: [d4f6b1d38bfd0d5bb0ae6df9b98e184fe5da58b8] Merge pull request #9497 from edolstra/move-access-control
git bisect bad d4f6b1d38bfd0d5bb0ae6df9b98e184fe5da58b8
# skip: [5334c9c792a208db4d3824e88019a626ded1b65d] HashType: Rename to HashAlgorithm
git bisect skip 5334c9c792a208db4d3824e88019a626ded1b65d
# bad: [504e4fc4576dc6a4cd5c083a3bf7b80dfb0ca220] CanonPath: Support std::hash
git bisect bad 504e4fc4576dc6a4cd5c083a3bf7b80dfb0ca220
# bad: [43d9fb6cf180c421be17b4247f5dd032cf4843f5] Remove InputAccessor::root()
git bisect bad 43d9fb6cf180c421be17b4247f5dd032cf4843f5
# bad: [305939655a6cd680997981ca6077d4ce7f957984] Remove superfluous use of hasAccessControl()
git bisect bad 305939655a6cd680997981ca6077d4ce7f957984
# bad: [ea95327e72f5781295417b0eae46a5e351bebebd] Move restricted/pure-eval access control out of the evaluator and into the accessor
git bisect bad ea95327e72f5781295417b0eae46a5e351bebebd
# first bad commit: [ea95327e72f5781295417b0eae46a5e351bebebd] Move restricted/pure-eval access control out of the evaluator and into the accessor

tomberek · 2024-03-25T13:14:33Z

Successful result in 2.19.2
Failure on 2.21.0

instantiated 'tzdata-2023.4-py2.py3-none-any.whl' -> '/nix/store/4zx8naw73qvj754mklhyb0rl6spq39mh-tzdata-2023.4-py2.py3-none-any.whl.drv'
performing daemon worker op: 7
instantiated 'python3.11-tzdata-2023.4' -> '/nix/store/nm62rq4kw92wriiry3ym2zbjb7cq4ixp-python3.11-tzdata-2023.4.drv'
source path '/nix/store/30744h1jcq0v44i1gwliy0rbv8ibz2im-source' is uncacheable
copying '/nix/store/30744h1jcq0v44i1gwliy0rbv8ibz2im-source' to the store...
performing daemon worker op: 7
evaluating file '/nix/store/fn4i255gjsbqhza45ih8mgxgx47zan8i-source/pkgs/build-support/nix-gitignore/default.nix'
evaluating derivation 'gitlab:nobodyinperson/annextimelog#packages.x86error: stack overflow (possible infinite recursion)

nixos-discourse · 2024-03-25T15:21:20Z

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-03-25-nix-team-meeting-minutes-133/42167/1

sweenu · 2024-03-25T23:05:03Z

Since I updated to the latest nixos-unstable a few days ago, I seem to have this issue with one specific program I wrote: https://github.com/sweenu/ig-story-fetcher
When I try to deploy it now and run a nix flake check, I get:
copying '/nix/store/zw5cyffjf8vi761gf7d3lw9czxjvp6sz-source' to the storeerror: stack overflow (possible infinite recursion)

And if I run nix flake check --no-build:

error:
       … while checking flake output 'nixosConfigurations'

       … while checking the NixOS configuration 'nixosConfigurations.najdorf'
         at /nix/store/94gag39fil45c81x34rragrb3radkhyh-source/lib/mkFlake.nix:132:7:
          131|     {
          132|       ${host.output}.${reverseDomainName} = host.builder ({
             |       ^
          133|         inherit (host) system;

       … while calling the 'head' builtin
         at /nix/store/amxd2p02wx78nyaa4bkb0hjvgwhz1dq7-source/lib/attrsets.nix:1541:11:
         1540|         || pred here (elemAt values 1) (head values) then
         1541|           head values
             |           ^
         1542|         else

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: stack overflow; max-call-depth exceeded
       at /nix/store/kzw8aq83xcx154dbwkld866hc18am5ys-source/lib.nix:61:20:
           60|     in
           61|     lib.optionals (builtins.pathExists path && builtins.toString path != "/" && ! isGitRoot) (findGitIgnores parent) ++ gitIgnores;
             |                    ^
           62|

The error seem to be related to nix-gitignore.

thufschmitt · 2024-04-03T19:22:33Z

Triaged during the maintainers meeting today.
Reproduced by Eelco.
Eelco will look at it.

nixos-discourse · 2024-04-08T15:29:44Z

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-04-03-nix-team-meeting-135/42962/1

Atry · 2024-04-14T19:21:13Z

Changing mkPoetryApplication { projectDir = self; } to mkPoetryApplication { projectDir = ./.; } would suppress the error.

NobbZ · 2024-04-15T12:35:56Z

There are slight different semantics with that.

And in general I'd prefer to keep the former possible to avoid double copy and /nix/store/$newhash-$oldhash-source

AllowListInputAccessor has the invariant that if a path is accessible, its parent directories are also considered accessible (though reading them only yields the allowed subdirectories). As a result `builtins.pathExists "/nix/store"` returns true. However this wasn't the behaviour of previous path access control, where `builtins.pathExists "/nix/store"` returns false even if a subdirectory of the store is accessible. Fixes NixOS#9672.

edolstra · 2024-04-15T14:28:21Z

This is fixed in #10505.

Note: the infinite recursion in findGitIgnores in poetry2nix happens because it has a stop condition builtins.toString path != "/" that never happens, because path will be an uncanonicalized path like /nix/store/<bla>/../../... However, previously that wasn't an issue because pathExists "/nix/store" returned false and caused it to bail out.

AllowListInputAccessor has the invariant that if a path is accessible, its parent directories are also considered accessible (though reading them only yields the allowed subdirectories). As a result `builtins.pathExists "/nix/store"` returns true. However this wasn't the behaviour of previous path access control, where `builtins.pathExists "/nix/store"` returns false even if a subdirectory of the store is accessible. Fixes NixOS#9672.

Ericson2314 · 2024-04-22T14:13:04Z

#10505 (comment) per this comment, I think builtins.pathExists "/nix/store" == true might well be better behavior for pure eval mode. There is definitely a behavior change, but I am apprehensive about calling this a bug to be fixed.

NobbZ added the bug label Dec 28, 2023

roberth added fetching Networking with the outside (non-Nix) world regression Something doesn't work anymore labels Mar 22, 2024

tomberek assigned tomberek and roberth Mar 25, 2024

tomberek assigned edolstra Mar 25, 2024

MaeIsBad mentioned this issue Apr 8, 2024

template fails with stack overflow nix-community/poetry2nix#1577

Open

Atry mentioned this issue Apr 14, 2024

Regression: stackoverflow on nix 2.20 and 2.21 when running poetry2nix with a projectDir from a flake input #10496

Open

edolstra linked a pull request Apr 15, 2024 that will close this issue

pathExists: Return false on "/nix/store" in pure mode #10505

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

possible regression: "stack overflow (possible infinite recursion)" #9672

possible regression: "stack overflow (possible infinite recursion)" #9672

NobbZ commented Dec 28, 2023 •

edited

NobbZ commented Dec 28, 2023

tomberek commented Mar 25, 2024

nixos-discourse commented Mar 25, 2024

sweenu commented Mar 25, 2024

thufschmitt commented Apr 3, 2024

nixos-discourse commented Apr 8, 2024

Atry commented Apr 14, 2024

NobbZ commented Apr 15, 2024

edolstra commented Apr 15, 2024 •

edited

Ericson2314 commented Apr 22, 2024

possible regression: "stack overflow (possible infinite recursion)" #9672

possible regression: "stack overflow (possible infinite recursion)" #9672

Comments

NobbZ commented Dec 28, 2023 • edited

NobbZ commented Dec 28, 2023

tomberek commented Mar 25, 2024

nixos-discourse commented Mar 25, 2024

sweenu commented Mar 25, 2024

thufschmitt commented Apr 3, 2024

nixos-discourse commented Apr 8, 2024

Atry commented Apr 14, 2024

NobbZ commented Apr 15, 2024

edolstra commented Apr 15, 2024 • edited

Ericson2314 commented Apr 22, 2024

NobbZ commented Dec 28, 2023 •

edited

edolstra commented Apr 15, 2024 •

edited