Ubuntu14.0.4使用anaconda2安装tensorflow后安装caffe小结（GPU版本）
安装配置说明：
- ubuntu14.0.4
- cuda8.0
- cudnn5.5
- Anaconda2
- protobuf-3.5.0（anaconda2安装tensorflow时自动安装）
- protobuf-2.5.0 （针对caffe的环境源码编译安装）
安装完tensorflow后再安装caffe会出现各种各样的问题。主要问题在于tensorflow和caffe在protobuf上的兼容问题。应该注意的是要保持protobuf的编译器protoc和其头文件路径以及动态链接库保持一致。不一致的时候将会出现错误。网上有的解决方案是卸载anaconda中的protobuf以及在系统环境变量PATH中除去anaconda2/bin的路径，python使用系统自带的版本。很显然，由于python中各个库的包都在anaconda2/lib/python2.7/site-packages文件夹中。使用系统自带的python后将会使得python的一个库都使用/usr/lib/python2.7/site-packages中的，这样就会影响tensorflow的使用，每次使用tensorflow又得将anaconda的python激活，比较麻烦。因此，这里主要使用anaconda的python解释器，然后源码编译安装protobuf-2.5.0来安装caffe，使得tensorflow与caffe能够共存。
由于项目需要，要进行TSN网络工程的编译。该工程使用caffe深度学习框架。按照该工程github上的编译过程（cmake）来进行，会出现各种问题，导致失败的核心问题在于protobuf的编译器，动态链接库以及头文件之间不匹配（操作系统含有多个版本的protobuf，caffe对于3.0.0以上的protobuf似乎不兼容）。按照该工程的github上的安装指南，在build文件夹下输入
cmake ..
，此时应该检查各个依赖项的版本以及安装位置。主要关注protobuf是否一致，还有python解释器与包路径，动态链接库之间是否一致。因为cmake中使用了自动搜索paket的方法。当protobuf或者python存在多个版本时，protobuf的编译器与动态链接库的位置很可能就不一致（可能使用了anaconda的protoc，动态链接库却使用了系统的/usr/lib），python也是同样的道理。由于系统环境的原因，我的配置刚开始就是因为protobuf不一致导致的问题。
从/home/wanghao/software/caffe-test/cmake/Modules到/home/wanghao/software/protobuf-2.5.0/bin/protoc是我自己在cmake/ProtoBuf.cmake文件中添加的指令。主要是为了观察工程在编译过程中使用的protobuf的路径。
wanghao@mclab-cv3:~/software/caffe-test/build$ cmake ..
-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Boost version: 1.55.0
-- Found the following Boost libraries:
-- system
-- thread
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found GFlags: /usr/include
-- Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Found Glog: /usr/include
-- Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
--
-- /home/wanghao/software/caffe-test/cmake/Modules
--
-- /home/wanghao/software/protobuf-2.5.0/lib/libprotobuf.so-lpthread
-- /home/wanghao/software/protobuf-2.5.0/include
-- /home/wanghao/software/protobuf-2.5.0/bin/protoc
-- Found PROTOBUF Compiler: /home/wanghao/software/protobuf-2.5.0/bin/protoc
-- Found HDF5: /usr/lib/x86_64-linux-gnu/libhdf5_hl.so;/usr/lib/x86_64-linux-gnu/libhdf5.so
-- Found LMDB: /usr/include
-- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
-- Found LevelDB: /usr/include
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
-- Found Snappy: /usr/include
-- Found Snappy (include: /usr/include, library: /usr/lib/libsnappy.so)
-- CUDA detected: 8.0
-- Found cuDNN (include: /usr/local/cuda-8.0/include, library: /usr/local/cuda-8.0/lib64/libcudnn.so)
-- Added CUDA NVCC flags for: sm_52
-- Cuda + Boost 1.55: Applying noinline work around
-- OpenCV found (/home/wanghao/software/anaconda2/share/OpenCV,/home/wanghao/software/anaconda2/include;/home/wanghao/software/anaconda2/include/opencv)
-- Found Atlas: /usr/include
-- Found Atlas (include: /usr/include, library: /usr/lib/libatlas.so)
-- Found PythonInterp: /home/wanghao/software/anaconda2/bin/python2.7 (found suitable version "2.7.14", minimum required is "2.7")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.6", minimum required is "2.7")
-- Found NumPy: /home/wanghao/software/anaconda2/lib/python2.7/site-packages/numpy/core/include (found suitable version "1.14.1", minimum required is "1.7.1")
-- NumPy ver. 1.14.1 found (include: /home/wanghao/software/anaconda2/lib/python2.7/site-packages/numpy/core/include)
-- Boost version: 1.55.0
-- Found the following Boost libraries:
-- python
-- /home/wanghao/software/anaconda2/include/python2.7
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Found Git: /usr/bin/git (found version "1.9.1")
--
-- ******************* Caffe Configuration Summary *******************
-- General:
-- Version : <TODO> (Caffe doesn't declare its version in headers)
-- Git : v0.9999-1638-g9831a2b-dirty
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- Release CXX flags : -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Debug CXX flags : -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
-- Build type : Release
--
-- BUILD_SHARED_LIBS : ON
-- BUILD_python : ON
-- BUILD_matlab : OFF
-- BUILD_docs : ON
-- CPU_ONLY : OFF
--
-- Dependencies:
-- BLAS : Yes (Atlas)
-- Boost : Yes (ver. 1.55)
-- glog : Yes
-- gflags : Yes
-- protobuf : Yes (ver. 2.5.0)
-- lmdb : Yes (ver. 0.9.16)
-- Snappy : Yes (ver. 1.1.0)
-- LevelDB : Yes (ver. 1.15)
-- OpenCV : Yes (ver. 3.3.1)
-- CUDA : Yes (ver. 8.0)
--
-- NVIDIA CUDA:
-- Target GPU(s) : Auto
-- GPU arch(s) : sm_52
-- cuDNN : Yes
--
-- Python:
-- Interpreter : /home/wanghao/software/anaconda2/bin/python2.7 (ver. 2.7.14)
-- Libraries : /home/wanghao/software/anaconda2/lib/libpython2.7.so (ver 2.7.6)
-- NumPy : /home/wanghao/software/anaconda2/lib/python2.7/site-packages/numpy/core/include (ver 1.14.1)
--
-- Documentaion:
-- Doxygen : No
-- config_file :
--
-- Install:
-- Install path : /home/wanghao/software/caffe-test/build/install
--
-- Configuring done
-- Generating done
-- Build files have been written to: /home/wanghao/software/caffe-test/build
对于protobuf的编译器与动态链接库不一致可能会导致：caffe.pb.h是老版本的protoc生成，不符合当前的protobuf的版本的错误，在生成caffe.pb.o文件时报错。
错误原因在文件build/include/caffe/proto/caffe.pb.h中，GOOGLE_PROTOBUF_MIN_PROTOC_VERSION的定义在文件google/protobuf/stubs/common.h中，而该文件是位于系统的protobuf中还是anaconda的protobuf中由cmake自动搜索而来。
#include <google/protobuf/stubs/common.h>
#if GOOGLE_PROTOBUF_VERSION < 2005000
#error This file was generated by a newer version of protoc which is
#error incompatible with your Protocol Buffer headers. Please update
#error your headers.
#endif
#if 2005000 < GOOGLE_PROTOBUF_MIN_PROTOC_VERSION
#error This file was generated by an older version of protoc which is
#error incompatible with your Protocol Buffer headers. Please
#error regenerate this file with a newer version of protoc.
#endif
解决方法：因此，我们可以通过强制指定protobuf的路径让cmake来构建。
将cmake/ProtoBuf.cmake文件的前面几句修改为如下：
# Finds Google Protocol Buffers library and compilers and extends
# the standard cmake script with version and python generation support
#set(CMAKE_MODULE_PATH "./")
#find_package( Protobuf REQUIRED )
message(STATUS ${SYSTEM})
set(PROTOBUF_PATH "/home/wanghao/software/protobuf-2.5.0")
set(PROTOBUF_INCLUDE_DIR "${PROTOBUF_PATH}/include")
include_directories(SYSTEM ${PROTOBUF_INCLUDE_DIR})
set(PROTOBUF_LIBRARIES "${PROTOBUF_PATH}/lib/libprotobuf.so")
list(APPEND PROTOBUF_LIBRARIES -lpthread)
set(PROTOBUF_PROTOC_EXECUTABLE "${PROTOBUF_PATH}/bin/protoc")
list(APPEND Caffe_LINKER_LIBS ${PROTOBUF_LIBRARIES})
message(STATUS ${CMAKE_MODULE_PATH})
message(STATUS ${PROTOBUF_SRC_ROOT_FOLDER})
message(STATUS ${PROTOBUF_LIBRARIES})
message(STATUS ${PROTOBUF_INCLUDE_DIR})
message(STATUS ${PROTOBUF_PROTOC_EXECUTABLE})
# As of Ubuntu 14.04 protoc is no longer a part of libprotobuf-dev package
# and should be installed separately as in: sudo apt-get install protobuf-compiler
if(EXISTS ${PROTOBUF_PROTOC_EXECUTABLE})
这样，由于protobuf自身的不匹配的问题可以得到解决。
对于opencv引发的问题如下：
原因分析：按照log的信息，应该是anaconda中opencv3.3.1版本的兼容问题，按照以上方法的操作，指定特定的opencv版本给cmake应该可行。
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `VTT for std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)@GLIBCXX_3.4.20'
/home/wanghao/software/anaconda2/lib/libopencv_imgcodecs.so.3.3.1: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::assign(char const*)@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::basic_ostringstream(std::_Ios_Openmode)@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_imgcodecs.so.3.3.1: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream(std::_Ios_Openmode)@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace_aux(unsigned long, unsigned long, unsigned long, char)@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::basic_istream<char, std::char_traits<char> >& std::operator>><char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_stringstream()@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(char const*) const@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `vtable for std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::rfind(char, unsigned long) const@GLIBCXX_3.4.21'
/home/wanghao/software/anaconda2/lib/libopencv_core.so.3.3.1: undefined reference to `__cxa_throw_bad_array_new_length@CXXABI_1.3.8'cmake中修改opencv的路径可以通过修改文件cmake/Dependencies.cmake来完成。
find_package(OpenCV QUIET COMPONENTS core highgui imgproc imgcodecs)
if(NOT OpenCV_FOUND) # if not OpenCV 3.x, then imgcodecs are not found
find_package(OpenCV REQUIRED COMPONENTS core highgui imgproc)
endif()
include_directories(SYSTEM ${OpenCV_INCLUDE_DIRS})
list(APPEND Caffe_LINKER_LIBS ${OpenCV_LIBS})
message(STATUS "OpenCV found (${OpenCV_CONFIG_PATH})")由于对工程修改过多，后来并没有实验该方案。既然我的系统环境需要给cmake指定许多固定的路径，那还不如使用make来进行编译（其实主要是因为本人并不熟悉cmake的操作）。
使用make来编译caffe
因为我们使用了anaconda的python以及一些库，Makefile.config文件中就得设置anaconda的路径，而我们又不能使用anaconda中的protobuf库。因此，INCLUDE_DIRS 以及 LIBRARY_DIRS 两个变量中一定要确保自己安装的protobuf-2.5.0的路径在前方，这样编译器就会先搜索到2.5.0版本的protobuf而不去搜索annaconda中的protobuf。还有，这里的配置中，opencv使用的是系统自带的版本，opencv2.4.13。
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_50,code=compute_50
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /home/wanghao/software/OpenBLAS/include
BLAS_LIB := /home/wanghao/software/OpenBLAS/lib
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := /home/wanghao/software/anaconda2
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python2.7 \
$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib
LIBRARIES += glog gflags protobuf leveldb snappy \
lmdb boost_system hdf5_hl hdf5 m \
opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := /home/wanghao/software/protobuf-2.5.0/include $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := /home/wanghao/software/protobuf-2.5.0/lib $(PYTHON_LIB) /usr/local/lib /usr/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS =/home/wanghao/software/protobuf-2.5.0/include $(INCLUDE_DIRS)
# LIBRARY_DIRS =/home/wanghao/software/protobuf-2.5.0/lib $(LIBRARY_DIRS)
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
USE_PKG_CONFIG := 1
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
#Q ?= @
到此，make all -j16, make test -j16, make runtest -j16, make pycaffe都可以通过。
import caffe引发的问题
import caffe前需要给进行sys.path.insert(0,caffe_root+’python’)的操作。注意在使用anaconda的python时，我的环境出现了mkl相关库的错误，什么libmkl_intel_lp64.so，libmkl_def.so，libmkl_avx.so找不到，可是查看路径anaconda2/lib却可以看到这些库。有的解决方法是conda install nomkl等，总而言之就是不使用mkl进行快速计算。anaconda2中的numpy以及其他的一些库似乎默认使用了mkl来进行优化。因此，我们依然使用mkl来解决此问题。最后，贴上有人给出原因为：The actual issue is that Anaconda linked with mkl, but not with libmkl_core.so, thus it has a missing symbol。解决方法是将一些库预先加载。
在~/.bashrc文件中加入如下语句：
export LD_PRELOAD="/home/wanghao/software/anaconda2/lib/libmkl_core.so:/home/wanghao/software/anaconda2/lib/libmkl_sequential.so"然后source ~/.bashrc
重新import caffe正常启动。