Fail to Build from Source as Lack of tensorflow_federated/tools/development

[SamuelGong]

Describe the bug
Following the document here, I was installing TFF from source in my CentOS 7 machine using the command

bazel run //tensorflow_federated/tools/development:build_pip_package -- --nightly --output_dir "/data/samuel/tmp/tensorflow_federated"

with Anaconda being 4.12.0, Python being 3.7.6 while bazel being 5.4.0. However, it failed with errors

(base) bash-4.2$ bazel run //tensorflow_federated/tools/development:build_pip_package -- --nightly --output_dir "/data/samuel/tmp/tensorflow_federated"
Extracting Bazel installation...                                                                                                  
Starting local Bazel server and connecting to it...                                                                                                                                                                                                                  
WARNING: The following configs were expanded more than once: [v2]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
INFO: Reading 'startup' options from /data/samuel/federated/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA256
INFO: Options provided by the client:                                                                                             
  Inherited 'common' options: --isatty=1 --terminal_columns=261                                                                                                                                                                                                      
INFO: Reading rc options for 'run' from /data/samuel/federated/.bazelrc:                                                          
  Inherited 'common' options: --announce_rc --enable_platform_specific_config --experimental_repo_remote_exec                                                                                                                                                        
INFO: Reading rc options for 'run' from /data/samuel/federated/.bazelrc:                                                                                                                                                                                             
  Inherited 'build' options: --verbose_failures --spawn_strategy=local --compilation_mode=opt --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --copt=-march=native --host_copt=-march=native --copt=-O3 --define=grpc_no_ares=true --noincompat
ible_remove_legacy_whole_archive --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --config=v2 --cxxopt=-D_G
LIBCXX_USE_CXX11_ABI=1                                                                                                                                                                                                                                               
INFO: Found applicable config definition build:short_logs in file /data/samuel/federated/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /data/samuel/federated/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1                                                                                                                     
INFO: Found applicable config definition build:v2 in file /data/samuel/federated/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1                                                                                                                     
INFO: Found applicable config definition build:linux in file /data/samuel/federated/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cx
xopt=-std=c++17 --host_cxxopt=-std=c++17 --experimental_guard_against_concurrent_changes                                                                                                                                                                             
DEBUG: Rule 'org_tensorflow' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1671056461 +0000"
DEBUG: Repository org_tensorflow instantiated at:      
  /data/samuel/federated/WORKSPACE:39:15: in <toplevel>
Repository rule git_repository defined at:                                                                                                                                                                                                                           
  /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>                                                                                                                
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/bf7308dce2cdf05ba40f13aee258148db4483588.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found                             
DEBUG: Rule 'bazel_skylib' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "6a17363a3c27dde70ab5002ad9f2e29aff1e1f4b", shallow_since = "1662041555 -0400" and dropping ["tag"]                                          
DEBUG: Repository bazel_skylib instantiated at:                                                                                   
  /data/samuel/federated/WORKSPACE:9:15: in <toplevel>                                                                                                                                                                                                               
Repository rule git_repository defined at:                                                                                                                                                                                                                           
  /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>                                                                                                                
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/339a7687e1c036a5f91c9d5391523b93e2e76cd3.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found                              
DEBUG: /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/org_tensorflow/third_party/repo.bzl:132:14:                                                                                                                               
Warning: skipping import of repository 'com_google_absl' because it already exists.                                               
DEBUG: /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/org_tensorflow/third_party/repo.bzl:132:14:                 
Warning: skipping import of repository 'pybind11_abseil' because it already exists.                                               
DEBUG: /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/org_tensorflow/third_party/repo.bzl:132:14:                                                                                                                               
Warning: skipping import of repository 'com_google_protobuf' because it already exists.                                                                                                                                                                              
DEBUG: /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/org_tensorflow/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.                                         
DEBUG: /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/org_tensorflow/third_party/repo.bzl:132:14:                                                                                                                               
Warning: skipping import of repository 'rules_python' because it already exists.                                                  
DEBUG: /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/org_tensorflow/third_party/repo.bzl:132:14:                                                                                                                               
Warning: skipping import of repository 'pybind11' because it already exists.                                                                                                                                                                                         
DEBUG: /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/org_tensorflow/third_party/repo.bzl:132:14:                                                                                                                               
Warning: skipping import of repository 'io_bazel_rules_go' because it already exists.                                                                                                                                                                                
DEBUG: Rule 'io_bazel_rules_go' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "4955534303141eef9570f447328510e5412d0009", shallow_since = "1633554776 -0400" and dropping ["tag"]                                     
DEBUG: Repository io_bazel_rules_go instantiated at:                                                                                                                                                                                                                 
  /data/samuel/federated/WORKSPACE:33:15: in <toplevel>                                                                                                                                                                                                              
Repository rule git_repository defined at:                                                                                                                                                                                                                           
  /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>                                                                                                                
DEBUG: Rule 'com_google_protobuf' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "90b73ac3f0b10320315c2ca0d03a5a9b095d2f66", shallow_since = "1666806648 +0000" and dropping ["tag"]                                   
DEBUG: Repository com_google_protobuf instantiated at:                                                                                                                                                                                                               
  /data/samuel/federated/WORKSPACE:27:15: in <toplevel>
Repository rule git_repository defined at:             
  /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>                                                                                                                
DEBUG: Rule 'bazel_gazelle' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "3ea1d64d6fe943dac06c341f9a265472bb99acd7", shallow_since = "1633971621 -0400" and dropping ["tag"]                                         
DEBUG: Repository bazel_gazelle instantiated at:                                                                                                                                                                                                                     
  /data/samuel/federated/WORKSPACE:132:15: in <toplevel>                                                                                                                                                                                                             
Repository rule git_repository defined at:                                                                                        
  /data/samuel/.cache/bazel/_bazel_zjiangaj/3fe67cea0bf4ee0dd0c8eb14b19e1272/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>                                                                                                                
ERROR: Skipping '//tensorflow_federated/tools/development:build_pip_package': no such package 'tensorflow_federated/tools/development': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.           
 - /data/samuel/federated/tensorflow_federated/tools/development                                                                                                                                                                                                     
WARNING: Target pattern parsing failed.                                                                                                                                                                                                                              
ERROR: no such package 'tensorflow_federated/tools/development': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.                                                                                  
 - /data/samuel/federated/tensorflow_federated/tools/development                                                                  
INFO: Elapsed time: 441.195s                                                                                                                                                                                                                                         
INFO: 0 processes.                                                                                                                
FAILED: Build did NOT complete successfully (0 packages loaded)                                                                                                                                                                                                      
FAILED: Build did NOT complete successfully (0 packages loaded)

The errors complaint that

ERROR: no such package 'tensorflow_federated/tools/development': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.                                                                               -

However, there was no folder named development in tensorflow_federated/tools/development at all. I have already been using the latest commit of the main branch as of today. Did I miss anything?

[huili0140]

Thank you for the question, we're looking into this.

[huili0140]

The same answer as the thread of #3771. This sounds like a question about the python package rather than a "federated learning" question to us. When we had the python executor we supported any version of linux, after we started releasing the C++ executor the linux compatibility was reduced, which we fixed by reducing the linux compatibility.

You can see that at https://github.com/tensorflow/federated/blob/main/tensorflow_federated/tools/python_package/build_python_package.sh#L92

We believe this isn't necessarily a CentOS 7 issue either.