Friday, July 31, 2015

Working with Caffe in Ubuntu 14.04

Data Augumentation
https://groups.google.com/forum/#!topic/caffe-users/ozBkuPnyatc


Faster Caffe Training

Stanford University CS231n Lecture 11 gives useful tips for speeding up your training process:

  1. Use LMDB (seek times and no image decompression ie. jpeg).
  2. Use cuBLAS. This is a CUDA version of BLAS and will be faster than CPU optimized BLAS.
  3. Use cuDNN.
  4. Use 32bit floating point precision (when writing new layers), as they compute faster



Compile custome files
http://stackoverflow.com/questions/32218466/compiling-caffe-c-classification-example

Error: 

https://github.com/BVLC/caffe/issues/2475
got this problem too, I used cuDNN v2 with latest caffe from the master. These are errors:
(If I use cuDNN v1, all will be compiled successfully)

CXX/LD -o .build_release/examples/mnist/convert_mnist_data.bin
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetFilter4dDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnGetConvolutionForwardAlgorithm'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cudnnCreateTensorDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `cudnnDestroyTensorDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnGetConvolutionForwardWorkspaceSize'
.build_release/lib/libcaffe.so: undefined reference to `cudnnAddTensor'
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetPooling2dDescriptor'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cudnnSetConvolution2dDescriptor'
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/convert_imageset.bin] Error 1
make: *** Waiting for unfinished jobs....

Fix:


I solved the problem above by replacing old cuDNN files with v2 files in every paths that contain libcudnn*. In my case (Ubuntu 14.04):

user_Ubuntu:~/caffe-master$ locate libcudnn_static.a 
/home/user/cudnn-6.5-linux-x64-v2/libcudnn_static.a
/lib64/libcudnn_static.a
/usr/lib/libcudnn/libcudnn_static.a
/usr/local/cuda-7.0/targets/x86_64-linux/lib/libcudnn_static.a
/usr/local/lib/libcudnn_static.a


So I did:


cd <cuDNN path>
sudo cp cudnn.h /usr/local/cuda/include
sudo cp libcudnn* /usr/local/cuda/lib64
# Add /usr/local/cuda/lib64 to LD_LIBRARY_PATH in ~/.bashrc
sudo cp libcudnn* /lib64/
sudo cp libcudnn* /usr/lib/libcudnn/
sudo cp libcudnn* /usr/local/cuda-7.0/targets/x86_64-linux/lib/
sudo cp libcudnn* /usr/local/lib 
sudo ldconfig

Then caffe-master is compiled succesfully.

or 

https://groups.google.com/forum/#!topic/caffe-users/QJ7L8c4Tn7c
making a symbolic link in /usr/lib for libcudnn.so.6.5.48 called libcudnn.so, and obviously placed it in that same directory.

sudo cp libcudnn* /usr/lib
sudo ln -s /usr/lib/libcudnn.so.6.5.48 libcudnn.so
ls -l /usr/lib/libcudnn*


Error:
raise ValueError('Mean shape incompatible with input shape.')
Fix:

The fix is in shelhamer@5c84795
Where all you need to do is to change this:

mean=np.load(mean_file)
to this:
mean=np.load(mean_file).mean(1).mean(1)

5 comments:

  1. Tutorial: http://christopher5106.github.io/deep/learning/2015/09/04/Deep-learning-tutorial-on-Caffe-Technology.html

    ReplyDelete
  2. http://www.exaptive.com/blog/how-i-made-a-neural-network-web-application-in-an-hour

    ReplyDelete
  3. ssh to headless server

    https://github.com/BVLC/caffe/issues/861

    ReplyDelete
  4. support 1080P and cudnn v5

    https://github.com/rbgirshick/py-faster-rcnn/issues/237

    -gencode arch=compute_61,code=sm_61 \
    -gencode arch=compute_61,code=compute_61

    ReplyDelete