8000 C++ compilation of rule '//tensorflow/python:tf_session_helper' failed · Issue #109 · tensorflow/tensorflow · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

C++ compilation of rule '//tensorflow/python:tf_session_helper' failed #109

New issue

Have a ques 8000 tion about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
markusdr opened this issue Nov 11, 2015 · 17 comments
Closed

C++ compilation of rule '//tensorflow/python:tf_session_helper' failed #109

markusdr opened this issue Nov 11, 2015 · 17 comments
Assignees
@ebrevdo

Comments

@markusdr
Copy link

I am trying to compile build_pip_package on Ubuntu 12.04 with gcc 4.8, but it gives an error during compilation:

$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package --verbose_failures

tensorflow/python/client/tf_session_helper.cc:418:41: error: 'PyArray_SHAPE' was not declared in this scope
   dims.push_back(PyArray_SHAPE(array)[i]);

ERROR: /home/local/ANT/x/tensorflow/tensorflow/python/BUILD:666:1: C++ compilation of rule '//tensorflow/python:tf_session_helper' failed: gcc failed: error executing command 
  (cd /home/local/ANT/x/.cache/bazel/_bazel_x/29843df2d2b24eaae7acd7ee881b7bec/tensorflow && \
  exec env - \
    INTERCEPT_LOCALLY_EXECUTABLE=1 \
    PATH=/home/local/ANT/x/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin \
  /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -iquote . -iquote bazel-out/local_linux-opt/genfiles -isystem google/protobuf/src -isystem bazel-out/local_linux-opt/genfiles/google/protobuf/src -isystem tools/cpp/gcc3 -isystem external/jpeg_archive/jpeg-9a -isystem bazel-out/local_linux-opt/genfiles/external/jpeg_archive/jpeg-9a -isystem external/png_archive/libpng-1.2.53 -isystem bazel-out/local_linux-opt/genfiles/external/png_archive/libpng-1.2.53 -isystem external/re2 -isystem bazel-out/local_linux-opt/genfiles/external/re2 -isystem third_party/gpus/cuda -isystem bazel-out/local_linux-opt/genfiles/third_party/gpus/cuda -isystem third_party/gpus/cuda/include -isystem bazel-out/local_linux-opt/genfiles/third_party/gpus/cuda/include -isystem third_party/eigen3 -isystem bazel-out/local_linux-opt/genfiles/third_party/eigen3 -I/usr/include/python2.7 -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-frandom-seed=bazel-out/local_linux-opt/bin/tensorflow/python/_objs/tf_session_helper/tensorflow/python/client/tf_session_helper.pic.o' -MD -MF bazel-out/local_linux-opt/bin/tensorflow/python/_objs/tf_session_helper/tensorflow/python/client/tf_session_helper.pic.d -fPIC -c tensorflow/python/client/tf_session_helper.cc -o bazel-out/local_linux-opt/bin/tensorflow/python/_objs/tf_session_helper/tensorflow/python/client/tf_session_helper.pic.o): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1: gcc failed: error executing command 
  (cd /home/local/ANT/x/.cache/bazel/_bazel_x/29843df2d2b24eaae7acd7ee881b7bec/tensorflow && \
  exec env - \
    INTERCEPT_LOCALLY_EXECUTABLE=1 \
    PATH=/home/local/ANT/x/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin:/home/local/ANT/x/bin \
  /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -iquote . -iquote bazel-out/local_linux-opt/genfiles -isystem google/protobuf/src -isystem bazel-out/local_linux-opt/genfiles/google/protobuf/src -isystem tools/cpp/gcc3 -isystem external/jpeg_archive/jpeg-9a -isystem bazel-out/local_linux-opt/genfiles/external/jpeg_archive/jpeg-9a -isystem external/png_archive/libpng-1.2.53 -isystem bazel-out/local_linux-opt/genfiles/external/png_archive/libpng-1.2.53 -isystem external/re2 -isystem bazel-out/local_linux-opt/genfiles/external/re2 -isystem third_party/gpus/cuda -isystem bazel-out/local_linux-opt/genfiles/third_party/gpus/cuda -isystem third_party/gpus/cuda/include -isystem bazel-out/local_linux-opt/genfiles/third_party/gpus/cuda/include -isystem third_party/eigen3 -isystem bazel-out/local_linux-opt/genfiles/third_party/eigen3 -I/usr/include/python2.7 -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-frandom-seed=bazel-out/local_linux-opt/bin/tensorflow/python/_objs/tf_session_helper/tensorflow/python/client/tf_session_helper.pic.o' -MD -MF bazel-out/local_linux-opt/bin/tensorflow/python/_objs/tf_session_helper/tensorflow/python/client/tf_session_helper.pic.d -fPIC -c tensorflow/python/client/tf_session_helper.cc -o bazel-out/local_linux-opt/bin/tensorflow/python/_objs/tf_session_helper/tensorflow/python/client/tf_session_helper.pic.o): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 3.223s, Critical Path: 2.46s
@markusdr
Copy link
Author

I can compile that particular source file after adding this to the gcc command:

-I/usr/local/lib/python2.7/dist-packages/numpy/core/include/

But when I rerun the bazel command it tries to compile it again without the needed include path and fails.

@markusdr markusdr reopened this Nov 11, 2015
@vrv
Copy link
vrv commented Nov 11, 2015

@markusdr: what version of numpy do you have installed, by any chance? I have a suspicion that it's probably earlier than numpy 1.8.2 :)

@markusdr
Copy link
Author

I believe it's numpy 1.9.2.

@vrv
Copy link
vrv commented Nov 11, 2015

Gotcha -- that version does have PyArray_SHAPE. Do you have any other numpy directories installed on your machine that might be interfering somehow?

If not, can you try changing tf_session_helper.cc to switch from PyArray_SHAPE to PyArray_DIMS ? (That one is available in earlier versions of numpy and apparently PyArray_SHAPE is an alias)

@markusdr
Copy link
Author

Right, the installation is using /usr/include/python2.7/, which doesn't have PyArray_SHAPE.

But PyArray_SHAPE is defined in /usr/local/lib/python2.7/dist-packages/numpy/core/include.

How can I tell bazel to use the latter python installation path?

@markusdr
Copy link
Author

When I add the correct numpy include path to the tensorflow/python/BUILD file it seems to compile, but at the end it complains:

$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package 
tensorflow/python/BUILD:666:1: in cc_library rule //tensorflow/python:tf_session_helper: 
The include path '/usr/local/lib/python2.7/dist-packages/numpy/core/include' references 
a path outside of the execution root..

@markusdr
Copy link
Author

OK, this workaround gets it to compile:

cd third_party
ln -s /usr/local/lib/python2.7/dist-packages/numpy/core/include/numpy .

Then add -Ithird_party to the includes in tensorflow/python/BUILD and tensorflow/tensorflow.bzl

@ebrevdo
Copy link
Contributor
ebrevdo commented Nov 13, 2015

There may be a way to infer the numpy header location at compile time. I'll take a look.

@ebrevdo ebrevdo self-assigned this Nov 13, 2015
@auroua
Copy link
auroua commented Nov 18, 2015

@markusdr
how to add -Ithird_part to tensorflow/python/BUILD and tensorflow/tensorflow.bzl
thanks

@markusdr
Copy link
Author

@auroua Here is the diff:

diff --git a/tensorflow/python/BUILD b/tensorflow/python/BUILD
-    copts = numpy_macosx_include_dir + ["-I/usr/include/python2.7"],
+    copts = numpy_macosx_include_dir + ["-Ithird_party", "-I/usr/include/python2.7"],

diff --git a/tensorflow/tensorflow.bzl b/tensorflow/tensorflow.bzl
-                    ] + ["-I/usr/include/python2.7"],
+                    ] + ["-Ithird_party", "-I/usr/include/python2.7"],

Alternatively, I could have symlinked /usr/local/lib/python2.7/dist-packages/numpy/core/include/numpy into /usr/include/python2.7, but I didn't have root access.

@auroua
Copy link
auroua commented Nov 19, 2015

@markusdr
I have a problem in #251. Can you help me to solve this problems.
I tried you method but it not working.
thanks

@chenghuige chenghuige mentioned this issue Nov 23, 2015
Closed
@sethbruder sethbruder mentioned this issue Dec 1, 2015
Closed
@noisychannel
Copy link

Do @markusdr 's changes still apply? I don't think that the diff corresponds to the current version of tensorflow/python/BUILD anymore. Suggestions?

@noisychannel
Copy link

Symlinking the numpy headers in util/python worked for me. python_config took care of the rest.

@vrv
Copy link
vrv commented Dec 10, 2015

If you sync to HEAD you should be able to use ./configure to select which python to use.

@noisychannel
Copy link

@vrv Is configure auto detecting the location of numpy headers now (from distconfig possibly?) ?. If not, this issue will still exist.

@vrv
Copy link
vrv commented Dec 10, 2015

I believe so: ./configure will call python_config.sh (https://github.com/tensorflow/tensorflow/blob/master/configure#L25) which gets the numpy headers from https://github.com/tensorflow/tensorflow/blob/master/util/python/python_config.sh#L64

@noisychannel
Copy link

Confirmed. python_config auto detects the location of numpy. None of the changes suggested in this thread are needed anymore as long as python has access to the correct numpy location. Thanks, @vrv.

ilblackdragon pushed a commit to ilblackdragon/tensorflow that referenced this issue Mar 9, 2016
@terrytangyuan
Merge pull request tensorflow#109 from liyongsea/fix_learning_rate_na…
b8d35c6 
…me_in_trainer

Fixed learning_rate in trainer
@lukeiwanski lukeiwanski mentioned this issue Jul 3, 2017
Merged
gunan pushed a commit that referenced this issue Jul 4, 2017
@gunan
Add -Wno-c++11-narrowing to ComputeCpp device compiler flags to avoid…
fc87d91 
… build errors on 32-bit targets. (#109) (#11244)
allenlavoie pushed a commit to allenlavoie/tensorflow that referenced this issue Jul 15, 2017
@gunan
Add -Wno-c++11-narrowing to ComputeCpp device compiler flags to avoid…
2ab4390 
… build errors on 32-bit targets. (tensorflow#109) (tensorflow#11244)
Ark-kun referenced this issue Aug 10, 2017
Merge Rc2 back to master.
3a047ae
lukeiwanski pushed a commit to codeplaysoftware/tensorflow that referenced this issue Oct 26, 2017
@alistair-low
Add -Wno-c++11-narrowing to ComputeCpp device compiler flags to avoid…
e147c94 
… build errors on 32-bit targets. (tensorflow#109)
tensorflow-copybara pushed a commit that referenced this issue Sep 6, 2019
@nmostafa @tensorflower-gardener
Add custom builder for AffineIfOp
977cb12 
Closes #109

COPYBARA_INTEGRATE_REVIEW=tensorflow/mlir#109 from nmostafa:nmostafa/AffineIfOp 7dbf2115f0092ffab26381ea8704aa05a0253971
PiperOrigin-RevId: 267633077
xinan-jiang pushed a commit to xinan-jiang/tensorflow that referenced this issue Oct 4, 2019
@nmostafa @tensorflower-gardener
Add custom builder for AffineIfOp
5009581 
Closes tensorflow#109

COPYBARA_INTEGRATE_REVIEW=tensorflow/mlir#109 from nmostafa:nmostafa/AffineIfOp 7dbf2115f0092ffab26381ea8704aa05a0253971
PiperOrigin-RevId: 267633077
@chengdianxuezi chengdianxuezi mentioned this issue Nov 1, 2019
Closed
@yanceyblog yanceyblog mentioned this issue Nov 28, 2019
Closed
cjolivier01 pushed a commit to Cerebras/tensorflow that referenced this issue Dec 6, 2019
@whchung
Merge pull request tensorflow#109 from ROCmSoftwarePlatform/develop-u…
19c4164 
…pstream-update-cuda-ci-whitelist

adding timeline_test to the CUDA CI whitelist
< 5565 /div>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ebrevdo ebrevdo

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@vrv @martinwicke @noisychannel @teamdandelion @ebrevdo @markusdr @auroua
0