Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] tensorflow: re-enable build from source #63616

Closed
wants to merge 11 commits into from

Commits on Jul 13, 2019

  1. Configuration menu
    Copy the full SHA
    f0747f5 View commit details
    Browse the repository at this point in the history
  2. libtensorflow: 1.9 -> 1.14.0

    This also changes it to a from-source build.
    yorickvP authored and abbradar committed Jul 13, 2019
    Configuration menu
    Copy the full SHA
    952f39b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4172bc0 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2019

  1. Configuration menu
    Copy the full SHA
    83c21e8 View commit details
    Browse the repository at this point in the history
  2. buildBazelPackage: cleanup hacks and enforce them for all builds

    Timestamp verification skip is no longer needed (not sure why). Generally we
    better off always using the environment hack for all packages because that
    ensures all NIX_* flags are correctly applied.
    
    One possible improvement in future is to filter only NIX_* variables to
    passthru in Bazel.
    abbradar committed Jul 14, 2019
    Configuration menu
    Copy the full SHA
    350bba9 View commit details
    Browse the repository at this point in the history
  3. tensorflow: cleanup and merge with libtensorflow

    This merges work done by yorickvP and timokau in NixOS#63208 and NixOS#63616 respectively.
    Now the derivation builds both libtensorflow and the Python package and puts them into
    different outputs.
    
    Quite a bit of improvements were done on the top, including:
    
    * Use official tag revision as source, not a branch;
    * Use all system libraries possible (before only one was actually used);
    * Move various environment variables to the derivation itself from hooks;
    * Use source Python build instead of wheel build to ensure fixup hooks do their important jobs on libraries;
    * And more that I forgot!
    abbradar committed Jul 14, 2019
    Configuration menu
    Copy the full SHA
    7b5a242 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2019

  1. bazel: drop absolute Python path

    Without this Bazel always picks Python 3 which breaks Python 2 packages.
    Strangely enough just dropping this patch works, with all `bazel.tests`
    passing.
    abbradar committed Jul 15, 2019
    Configuration menu
    Copy the full SHA
    9ed16da View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2019

  1. python2: backport fix for pyc race condition

    This is python bug https://bugs.python.org/issue13146. Fixed since
    python 3.4. It makes pyc creation atomic, preventing a race condition.
    The patch has been rebased on our deterministic build patch.
    
    It wasn't backported to python 2.7 because there was a complaint about
    changed semantics. Since files are now created in a temporary directory
    and then moved, symlinks will be overridden. See
    https://bugs.python.org/issue17222.
    
    That is an edge-case however. Ubuntu and debian have backported the fix
    in 2013 already, making it mainstream enough for us to adopt.
    
    (cherry picked from commit 9db3a58)
    timokau committed Jul 18, 2019
    Configuration menu
    Copy the full SHA
    5d52df5 View commit details
    Browse the repository at this point in the history
  2. python2: backport fix for pyc race condition, part 2

    Turns out fixing this only in importlib is not sufficient and we
    need to backport CPython part of the fix too.
    
    This patch is based on https://hg.python.org/cpython/rev/c16063765d3a
    but because the code around is different there are some changes (C-strings
    instead of Python objects etc.)
    
    With this patch Tensorflow builds successfully on many-core machine.
    
    (cherry picked from commit da295a1)
    abbradar authored and timokau committed Jul 18, 2019
    Configuration menu
    Copy the full SHA
    e52d1de View commit details
    Browse the repository at this point in the history
  3. buildBazelPackage: add flags for build and fetch

    They sometimes take separate flags.
    timokau committed Jul 18, 2019
    Configuration menu
    Copy the full SHA
    843c54d View commit details
    Browse the repository at this point in the history
  4. python.pkgs.tensorflow: fix optimization flags

    Now need to be passed in the configure phase.
    timokau committed Jul 18, 2019
    Configuration menu
    Copy the full SHA
    0c3db9a View commit details
    Browse the repository at this point in the history