Segmentation fault, import tensorflow, tensorflow 0.9, mac osx #3263

lukecwalters · 2016-07-11T01:45:11Z

I have followed the installation steps for GPU enabled tensorflow 0.9 on OSX, (https://www.tensorflow.org/versions/r0.9/get_started/os_setup.html#installation-for-mac-os-x), within a conda environment. The installation seems to go smoothly, but I get a seg fault error when importing tensorflow. I have tried the fixes from related threads without success.

See below for the segmentation fault error.

Environment info

Operating System: Mac OSX 10.10.5

Installed version of CUDA and cuDNN: CUDA 7.5 and cuDNN 5.1
(please attach the output of ls -l /path/to/cuda/lib/libcud*):
$ ls -l /usr/local/cuda/lib/libcud*
-rwxr-xr-x 1 root wheel 8280 Apr 13 02:02 /usr/local/cuda/lib/libcuda.dylib
lrwxr-xr-x 1 root wheel 45 Apr 13 02:03 /usr/local/cuda/lib/libcudadevrt.a -> /Developer/NVIDIA/CUDA-7.5/lib/libcudadevrt.a
lrwxr-xr-x 1 root wheel 50 Apr 13 02:03 /usr/local/cuda/lib/libcudart.7.5.dylib -> /Developer/NVIDIA/CUDA-7.5/lib/libcudart.7.5.dylib
lrwxr-xr-x 1 root wheel 46 Apr 13 02:03 /usr/local/cuda/lib/libcudart.dylib -> /Developer/NVIDIA/CUDA-7.5/lib/libcudart.dylib
lrwxr-xr-x 1 root wheel 49 Apr 13 02:03 /usr/local/cuda/lib/libcudart_static.a -> /Developer/NVIDIA/CUDA-7.5/lib/libcudart_static.a
-rwxr-xr-x@ 1 lw17567 staff 58975112 Jun 10 04:30 /usr/local/cuda/lib/libcudnn.5.dylib
lrwxr-xr-x@ 1 lw17567 staff 16 Jun 10 04:31 /usr/local/cuda/lib/libcudnn.dylib -> libcudnn.5.dylib
-rw-r--r--@ 1 lw17567 staff 5639232 Jun 10 04:30 /usr/local/cuda/lib/libcudnn_static.a

Installed tensorflow 0.9 within anaconda 2 env,

pip version: pip 8.1.2
Python 2.7.12

The output from python -c "import tensorflow; print(tensorflow.__version__)".

$ python -c "import tensorflow; print(tensorflow.version)"
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.7.5.dylib locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.5.dylib locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.7.5.dylib locally
Segmentation fault: 11

Steps to reproduce

1.) Create a conda environment
2.) Followed https://www.tensorflow.org/versions/r0.9/get_started/os_setup.html#installation-for-mac-os-x
3.) Launch python and import tensorflow

What have you tried?

importing numpy before importing tensorflow
ensuring that I'm outside the tensorflow source directory (well outside)
adding /Developer/NVIDIA/CUDA-7.5/lib to the DYLD_LIBRARY_PATH, per Segmentation fault on tensorflow 0.9.0 #2773
4.) uninstalling and reinstalling everything and trying again.

Logs or other output that would be helpful

Output from bt, after using gdb to run test script with only line, 'import tensorflow'
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.7.5.dylib locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.5.dylib locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.7.5.dylib locally

Program received signal SIGSEGV, Segmentation fault.
0x00007fff8d2b8f92 in strlen () from /usr/lib/system/libsystem_c.dylib

(gdb) bt
#0 0x00007fff8d2b8f92 in strlen () from /usr/lib/system/libsystem_c.dylib
#1 0x0000000104a0b4f0 in perftools::gputools::internal::DsoLoader::GetDsoHandle(tensorflow::StringPiece, void, perftools::gputools::internal::DsoLoader::LoadKind) ()

from /Users/lw17567/anaconda2/envs/tensorflow2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
#2 0x0000000104a0c06c in perftools::gputools::internal::DsoLoader::GetLibcudaDsoHandle(void) ()

from /Users/lw17567/anaconda2/envs/tensorflow2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
#3 0x0000000104a0d0d2 in std::__1::__function::__func, tensorflow::Status (void)>::operator()(void&&) ()

from /Users/lw17567/anaconda2/envs/tensorflow2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
#4 0x0000000104a0c86f in perftools::gputools::internal::CachedDsoLoader::FetchHandleResult(std::__1::function) ()

from /Users/lw17567/anaconda2/envs/tensorflow2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
#5 0x0000000104a0cdcf in perftools::gputools::internal::CachedDsoLoader::GetLibcudaDsoHandle() ()

from /Users/lw17567/anaconda2/envs/tensorflow2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
---Type to continue, or q to quit---
#6 0x0000000104a8ebe5 in perftools::gputools::initialize_cuda_gpu_executor()

()
from /Users/lw17567/anaconda2/envs/tensorflow2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
#7 0x00007fff5fc12d4b in ?? ()
#8 0x000000155fbfc830 in ?? ()
#9 0x0000000000000001 in ?? ()
#10 0x00000001034c02e8 in ?? ()
#11 0x00000001034c0740 in ?? ()
#12 0x00000001034c0420 in ?? ()
#13 0x00007fff5fc38660 in ?? ()
#14 0x00000000000001b8 in ?? ()
#15 0x00000001004613d0 in ?? ()
#16 0x00007fff5fbfc8a0 in ?? ()
#17 0x0000000000000713 in ?? ()
#18 0x00007fff5fc38598 in ?? ()
#19 0x00007fff5fc41a38 in ?? ()
#20 0x00007fff5fbfc830 in ?? ()
#21 0x00007fff5fc12ed8 in ?? ()
#22 0x00000001004613d0 in ?? ()
#23 0x0000b481cbaa22ee in ?? ()
#24 0x00007fff5fbfc890 in ?? ()
#25 0x00007fff5fc0f8d1 in ?? ()

---Type to continue, or q to quit---
#26 0x0000000000000000 in ?? ()

If necessary, please see the full gdb output, and deviceQuery (CUDA sample) output in my related comments in #2773 .

The text was updated successfully, but these errors were encountered:

jmchen-g · 2016-07-11T18:54:57Z

@martinwicke Could you take a look at this please?

Menerve · 2016-07-12T21:17:27Z

Having the same problem, It appears that it fails when loading libcuda.dylib. Unfortunately, since #2878 it's now looking for libcuda.1.dylib instead.

Creating a symlink or changing the name of libcuda.dylib to libcuda.1.dylib in /usr/local/cuda/lib seems to do the job.

lukecwalters · 2016-07-13T01:40:58Z

@Menerve , THANK YOU! It worked, I would have never found that.

ASRagab · 2016-07-16T19:13:16Z

@Menerve three hours of building and pip installing and uninstalling later, I finally find your helpful comment. THANK YOU.

Menerve · 2016-07-17T15:46:23Z

@ASRagab @lw394 Glad to help! But I'm not sure why this issue is closed. Something needs to be done here.

vrv · 2016-07-17T15:56:43Z

@Menerve we've been told that this is a bug in the naming / packaging for OS X cuda installs -- I think we're waiting for the real fix by them, which is to name the library with a .1, and maybe add the symlink.

lukecwalters · 2016-07-18T01:56:11Z

reopened per @Menerve's suggestion, I'll let google close it.

vrv · 2016-07-18T02:35:16Z

:) I wish I could mark this as "still an issue, but not a TensorFlow-specific issue" :(

jrimestad · 2016-09-27T10:49:38Z

You could add the symlink step to you OSX step by step guide?

zhubofei · 2017-02-05T16:13:50Z

@jrimestad Might be too late for your question, but for anyone who doesn't know how to add the symlink:

sudo ln -sf /usr/local/cuda/lib/libcuda.dylib /usr/local/cuda/lib/libcuda.1.dylib

run this in terminal will do the trick.

Xiaolong-Li · 2017-02-21T22:58:12Z

thank you so much!

WolfNiu · 2017-03-11T03:21:44Z

Thank you @Menerve !! Your comment saved me!

semaj7 · 2017-04-04T14:06:25Z

Many thanks @Menerve and @zhubofei
Summary:
After installing tensorflow-gpu and trying to import it in python I also got the Segmentation faults. First had to add the /Developer/NVIDIA/CUDA-8.0/lib as well as /Developer/NVIDIA/cudnn-8.0/lib to DYLD_LIBRARY_PATH in the bash_profile, and then needed to rename libcuda, with "sudo ln -sf /usr/local/cuda/lib/libcuda.dylib /usr/local/cuda/lib/libcuda.1.dylib"

As a newbie in library dependency issues I would like to ask you all: how do naming issues like with the libcuda.1 happen?

kwccoin · 2017-04-16T14:09:01Z

still got problem after done the DYLD_LIBRARY_PATH and sudo. still segmentation fault:11. any further advice.

Dr-Drive · 2017-04-16T22:40:14Z

In my case i was getting seg fault 11 when importing tensorflow to my virtual env python.
Then I suspected I was doing some rookie mistake in my .bash_profile.
I found this link
The tell you to add this to your bash:
export CUDA_HOME="/usr/local/cuda"
export DYLD_LIBRARY_PATH="$CUDA_HOME/lib:$DYLD_LIBRARY_PATH"
export PATH="$CUDA_HOME/bin:$PATH"
I suspect it is crucial to define a CUDA_HOME variable on your bash.
Now my tensorflow is up and running on python.

orgicus · 2017-06-14T09:13:39Z

I needed TensorFlow with GPU support version 0.12.0 specifically (pip install tensorflow-gpu==0.12.0 and ran into similar issues on two different macs: a macbook running 10.11 and an iMac running 10.12.

On 10.11 I could see tensorflow complaining about missing .dylibs, but it didn't crash I manually symlinked libraries from /usr/local/cuda/lib to /usr/local/lib until it was happy:

I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.dylib locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.dylib locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.dylib locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.1.dylib locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.dylib locally

On 10.12 I just got segfault 11 after running import tensorflow in an interpreter.
I got around it in a similar fashion, but perhaps a bit messier: symlinking ALL the libraries :))
for l in /usr/local/cuda/lib;do sudo ln -s $l /usr/local/lib/$(basename $l);done

On both configurations I've used the latest CUDA Driver (8.0.61) and SDK (8.0) with CuDNN 5.1 in this case (which is what the tensorflow-gpu 0.12.0 was compiled against).

jmchen-g added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jul 11, 2016

lukecwalters closed this as completed Jul 13, 2016

xjdeng mentioned this issue Jul 14, 2016

Tensorflow with GPU on Mac OSX: Works in Python but NOT IPython #3321

Closed

lukecwalters reopened this Jul 18, 2016

vrv closed this as completed Jul 18, 2016

cwindolf mentioned this issue Oct 2, 2016

Library not loaded: @rpath/libcudart.7.5.dylib #4187

Closed

geoHeil mentioned this issue Feb 26, 2017

Segmenttation fault tensorflow 1.0 osx #7900

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Segmentation fault, import tensorflow, tensorflow 0.9, mac osx #3263

Segmentation fault, import tensorflow, tensorflow 0.9, mac osx #3263

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Segmentation fault, import tensorflow, tensorflow 0.9, mac osx #3263

Segmentation fault, import tensorflow, tensorflow 0.9, mac osx #3263

Comments

Environment info

Steps to reproduce

What have you tried?

Logs or other output that would be helpful

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!