2016-07-01 19:36:44 wangpengfei163 阅读数 3073
  • 基于Spark的分布式深度学习和认知计算

    你是否曾经面对多个优化算法不知所措?或者无法自由选择学习框架?又或许因为Caffe,Tensorflow, Theano, Torch的诸多参数设置而烦恼?或简单的认为只要有大数据就可以训练计算 机了?如果你不懂复杂的数学、统计学理论,还能做训练吗?...... 带着十万个为什么,让我们与深度学习技术讲师一起,了解基于Spark的分布式数据探索、机器学习/深度学习和认知计算。

    3098 人正在学习 去看看 CSDN讲师

该文章来自Theano官网,http://deeplearning.net/software/theano/install.html

Easy Installation of an Optimized Theano on Current Ubuntu

For NVIDIA Jetson TX1 embedded platform:

sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libblas-dev git
pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git --user  # Need Theano 0.8(not yet released) or more recent

For Ubuntu 16.04 with cuda 7.5

sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev git
sudo pip install Theano

# cuda 7.5 don't support the default g++ version. Install an supported version and make it the default.
sudo apt-get install g++-4.9

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 20
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 10

sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.9 20
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-5 10

sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30
sudo update-alternatives --set cc /usr/bin/gcc

sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30
sudo update-alternatives --set c++ /usr/bin/g++

# Work around a glibc bug
echo -e "\n[nvcc]\nflags=-D_FORCE_INLINES\n" >> ~/.theanorc

For Ubuntu 11.10 through 14.04:

sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev git
sudo pip install Theano

On 14.04, this will install Python 2 by default. If you want to use Python 3:

sudo apt-get install python3-numpy python3-scipy python3-dev python3-pip python3-nose g++ libopenblas-dev git
sudo pip install Theano

For Ubuntu 11.04:

sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ git libatlas3gf-base libatlas-dev
sudo pip install Theano

Note

If you have error that contain “gfortran” in it, like this one:

ImportError: (‘/home/Nick/.theano/compiledir_Linux-2.6.35-31-generic-x86_64-with-Ubuntu-10.10-maverick–2.6.6/tmpIhWJaI/0c99c52c82f7ddc775109a06ca04b360.so: undefined symbol: _gfortran_st_write_done’

The problem is probably that NumPy is linked with a different blasthen then one currently available (probably ATLAS). There is 2possible fixes:

  1. Uninstall ATLAS and install OpenBLAS.
  2. Use the Theano flag “blas.ldflags=-lblas -lgfortran”

1) is better as OpenBLAS is faster then ATLAS and NumPy isprobably already linked with it. So you won’t need any otherchange in Theano files or Theano configuration.

Note

If you are behind a proxy, you must do some extra configuration stepsbefore starting the installation. You must set the environmentvariable http_proxy to the proxy address. Using bash this isaccomplished with the commandexport http_proxy="http://user:pass@my.site:port/"You can also provide the --proxy=[user:pass@]url:port parameterto pip. The [user:pass@] portion is optional.

Note

We use pip for 2 reasons. First, it allows “import module;module.test()” to work correctly. Second, the installation of NumPy1.6 or 1.6.1 with easy_install raises an ImportError at the end ofthe installation. To my knowledge we can ignore this error, butthis is not completely safe. easy_install with NumPy 1.5.1 does notraise this error.

Note

This page describes how to install Theano for Python 2. If you haveinstalled Python 3 on your system, maybe you need to change thecommand pip to pip-2.7 to specify to install it for Python 2, assometimes the pip command refers to the Python 3 version.

The development version of Theano supports Python 3.3 andprobably supports Python 3.2, but we do not test on it.

Bleeding Edge Installs

If you would like, instead, to install the bleeding edge Theano (from github)such that you can edit and contribute to Theano, replace the pip install Theanocommand with:

git clone git://github.com/Theano/Theano.git
cd Theano
python setup.py develop --user
cd ..

VirtualEnv

If you would like to install Theano in a VirtualEnv, you will want to pass the–system-site-packages flag when creating the VirtualEnv so that it will pick upthe system-provided Numpy and SciPy.

virtualenv --system-site-packages -p python2.7 theano-env
source theano-env/bin/activate
pip install Theano

Test the newly installed packages

  1. NumPy (~30s): python -c "import numpy; numpy.test()"
  2. SciPy (~1m): python -c "import scipy; scipy.test()"
  3. Theano (~30m): python -c "import theano; theano.test()"

NumPy 1.6.2, 1.7.0 and 1.7.1, have a bug where it marks some ndarraysas not aligned. Theano does not support unaligned arrays, and raisesan Exception when that happens. This can cause one test to fail withan unaligned error with those versions of NumPy. You can ignore thattest error as at worst, your code will crash. If this happens, you caninstall another NumPy version to fix this problem. NumPy 1.6.2 is usedin Ubuntu 12.10 and NumPy 1.7.1 is used in Ubuntu 13.04.

Speed test Theano/BLAS

It is recommended to test your Theano/BLAS integration. There are many versionsof BLAS that exist and there can be up to 10x speed difference between them.Also, having Theano link directly against BLAS instead of using NumPy/SciPy asan intermediate layer reduces the computational overhead. This isimportant for BLAS calls to ger, gemv and small gemm operations(automatically called when needed when you use dot()). To run theTheano/BLAS speed test:

python `python -c "import os, theano; print(os.path.dirname(theano.__file__))"`/misc/check_blas.py

This will print a table with different versions of BLAS/numbers ofthreads on multiple CPUs and GPUs. It will also print some Theano/NumPyconfiguration information. Then, it will print the running time of the samebenchmarks for your installation. Try to find a CPU similar to yours inthe table, and check that the single-threaded timings are roughly the same.

Theano should link to a parallel version of Blas and use all coreswhen possible. By default it should use all cores. Set the environmentvariable “OMP_NUM_THREADS=N” to specify to use N threads.

Note

It is possible to have a faster installation of Theano than the one theseinstructions provide, but this will make the installation morecomplicated and/or may require that you buy software. This is a simple setof installation instructions that will leave you with a relativelywell-optimized version that uses only free software. With more work or byinvesting money (i.e. buying a license to a proprietary BLASimplementation), it is possible to gain further performance.

Updating Theano

If you followed these installation instructions, you can execute this commandto update only Theano:

sudo pip install --upgrade --no-deps theano

If you want to also installed NumPy/SciPy with pip instead of thesystem package, you can run this:

sudo pip install --upgrade theano

Updating Bleeding Edge Installs

Change to the Theano directory and run:

git pull

Manual Openblas instruction

The openblas included in some older Ubuntu version is limited to 2threads. Ubuntu 14.04 do not have this limit. If you want to use morecores at the same time, you will need to compile it yourself. Here issome code that will help you.

# remove openblas if you installed it
sudo apt-get remove libopenblas-base
# Download the development version of OpenBLAS
git clone git://github.com/xianyi/OpenBLAS
cd OpenBLAS
make FC=gfortran
sudo make PREFIX=/usr/local/ install
# Tell Theano to use OpenBLAS.
# This works only for the current user.
# Each Theano user on that computer should run that line.
echo -e "\n[blas]\nldflags = -lopenblas\n" >> ~/.theanorc

Contributed GPU instruction

Basic configuration for the GPU Using the GPU.

Ubuntu 11.10/12.04 (probably work on 11.04 too):

sudo apt-add-repository ppa:ubuntu-x-swat/x-updates
sudo apt-get update
sudo apt-get install nvidia-current

Then you need to fetch latest CUDA tool kit (download ubuntu 11.04 32/64bit package)from here.

Ubuntu 14.04:

sudo apt-get install nvidia-current
sudo apt-get install nvidia-cuda-toolkit # As of October 31th, 2014, provide cuda 5.5, not the latest cuda 6.5

If you want cuda 6.5, you can download packages from nvidia for Ubuntu 14.04.

If you downloaded the run package (the only one available for CUDA 5.0 and older), you install it like this:

chmod a+x XXX.sh
sudo ./XXX.sh

Since CUDA 5.5, Nvidia provide a DEB package. If you don’t know how tointall it, just double click on it from the graphical interface. Itshould ask if you want to install it. On Ubuntu 14.04, you need to runthis in your terminal:

sudo apt-get update
sudo apt-get install cuda

You must reboot the computer after the driver installation. To testthat it was loaded correctly after the reboot, run the commandnvidia-smi from the command line.

You probably need to change the default version of gcc asexplained by Benjamin J. McCann if the package you downloaded is for another Ubuntu version:

sudo apt-get install nvidia-cuda-toolkit g++-4.4 gcc-4.4
# On Ubuntu 11.10 and 12.04, you probably need to change gcc-4.5 to gcc-4.6 on the next line.
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.5 40 --slave /usr/bin/g++ g++ /usr/bin/g++-4.5
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.4 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.4
sudo update-alternatives --config gcc

Test GPU configuration

THEANO_FLAGS=floatX=float32,device=gpu python /usr/lib/python2.*/site-packages/theano/misc/check_blas.py

Note

Ubuntu 10.04 LTS: default gcc version 4.4.3. gcc 4.1.2, 4.3.4 available.

Ubuntu 11.04: default gcc version 4.5.2. gcc 4.4.5 available.

Ubuntu 11.10: default gcc version 4.6.1. gcc 4.4.6 and 4.5.3 available.

Ubuntu 12.04 LTS: default gcc version 4.6.3. gcc 4.4.7 and 4.5.3 available.

Ubuntu 12.10: default gcc version 4.7.2. gcc 4.4.7, 4.5.4 and 4.6.3 available.

Ubuntu 13.10: default gcc version 4.8.1. gcc 4.4.7, 4.6.4 and 4.7.3 available.

Ubuntu 14.04: default gcc version 4.8.2, gcc 4.4.7,, 4.6.4, and 4.7.3 available.


2016-07-20 14:33:57 yahag 阅读数 5153
  • 基于Spark的分布式深度学习和认知计算

    你是否曾经面对多个优化算法不知所措?或者无法自由选择学习框架?又或许因为Caffe,Tensorflow, Theano, Torch的诸多参数设置而烦恼?或简单的认为只要有大数据就可以训练计算 机了?如果你不懂复杂的数学、统计学理论,还能做训练吗?...... 带着十万个为什么,让我们与深度学习技术讲师一起,了解基于Spark的分布式数据探索、机器学习/深度学习和认知计算。

    3098 人正在学习 去看看 CSDN讲师
因为学习要用到深度学习,因此要配置caffe和theano。记过几日几夜的不懈奋战,最终找到了一种极为简单的安装方法。为将来要用到深度学习的同学节省时间,写下这个博客。由于博主是第一次写博客,若有错误还请大家海涵!

QQ:75690183
email:75690183@qq.com

本教程最大的特点是CUDA和cuDNN的安装简单,这也得益于这个系统版本的更新!

(一)系统安装

先从官网上下载ubuntu16.04系统镜像,再从这里下载Universal-USB-Installer-1.9.6.5软件,最后找一个U盘。准备开始!

(1)制作启动盘

第一个框选择系统——ubuntu;

第二个框选择一下载好的ubuntu16.04系统镜像文件(以.iso结尾);

第三个框选择事先准备好的U盘。

之后点creat,等待….再点close,结束!

(2)安装系统

将我们制作的启动盘插入电脑,选择U盘启动,进入安装系统(作为程序员,这些不应该不会,如果不会自行百度)。等待大约5秒,自动进入ubuntu系统安装界面,之后的不再详述。

(二)安装caffe

由于兼容性问题,下面先安装caffe。在第二节中已经说明了如何安装系统,因此这时我们已经有ubuntu系统,并且已经进入系统!本节对下面没有关联性,不用的同学可跳过此节。

1、先安装各种依赖库,打开终端,依次输入如下命令:

sudo  apt-get  update
sudo  apt-get  upgrade
sudo  apt-get  install  -y  build-essential  cmake  git  pkg-config
sudo  apt-get  install  -y  libprotobuf-dev libleveldb-dev libsnappy-dev libhdf5-serial-dev protobuf-compiler
sudo  apt-get  install  -y  libatlas-base-dev
sudo  apt-get  install  -y  --no-install-recommends  libboost-all-dev
sudo  apt-get  install  -y  libgflags-dev  libgoogle-glog-dev  liblmdb-dev
sudo  apt-get  install  -y  python-pip
sudo  apt-get  install  -y  python-dev
sudo  apt-get  install  -y  python-numpy python-scipy
sudo  apt-get  install  -y  libopencv-dev
sudo  pip  install  protobuf


2、下载caffe源码
git  clone  https://github.com/BVLC/caffe.git


3、配置Makefile.config文件

cd到caffe目录下
cp Makefile.config.example Makefile.config
gedit Makefile.config

4、注意下面的操作很重要

在Makefile.config中找到下面的内容:

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

然后替换为:

INCLUDE_DIRS :=  $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial

将:
PYTHON_LIB := /usr/lib
替换为:

PYTHON_LIB := /usr/lib/python2.7/config-x86_64-linux-gnu 

不确定这一步是否有用,但是在/usr/lib下没有libpythonX.X.so文件


5、配置Makefile文件(注意:它和Makefile.config不是一个文件)

sudo  gedit  Makefile

输入密码,开始编辑。查找NVCCFLAGS,将下面的东西

NVCCFLAGS  +=  -ccbin=(CXX)−Xcompiler−fPIC(COMMON_FLAGS)
替换为:
NVCCFLAGS += -D_FORCE_INLINES -ccbin=(CXX)−Xcompiler−fPIC(COMMON_FLAGS)

修改它的目的是为了防止报错: “string.h ‘memcy’ was not declaredin this scope ”。


6、创建hdf5新的连接

执行一下命令:

find -type f -exec sed -i -e 's^"hdf5.h"^"hdf5/serial/hdf5.h"^g' -e 's^"hdf5_hl.h"^"hdf5/serial/hdf5_hl.h"^g' '{}' \;

再输入:

cd  /usr/lib/x86_64-linux-gnu

接着,执行以下命令:

sudo  ln  -s  libhdf5_serial.so.10.1.0  libhdf5.so
sudo  ln  -s  libhdf5_serial_hl.so.10.0.2  libhdf5_hl.so

这里会提示,文件已存在,没关系,继续。


7、检验

cd到caffe/python目录下

输入一下命令:

for  req  in  $(cat  requirements.txt);  do  pip  install  $req;  done

终端可能会有很多红字,那么在运行下一句:

for  req  in  $(cat  requirements.txt);  do  sudo  -H  pip  install $req --upgrade;  done

8、编译

到这里,我们的准备工作就都完成了,下面开始编译,运行这几行代码:

cd到caffe目录下
make  all  -j8 # j8指计算机的核心数,如8核
make  test  -j8
make  runtest  -j8

如果有错,按照上面的步骤重做一次。先暂时不编译python接口,先给python添加环境变量,打开.bashrc文件:

cd ~
sudo  gedit  ~/.bashrc

输入密码后,进入编辑界面。在最后一行加上:

export PYTHONPATH =/home/xxx /caffe/ python:$PYTHONPATH

注意,路径写自己的。现在编译python接口,输入以下命令:

cd到caffe目录下
make  pycaffe
make  distribute

这一步做完就算大功告成了,输入:python,再输入:import caffe,如果没有报错,说明安装成功。


(三)安装theano

在上一节中不管caffe是否安装成功,对于theano的安装没有影响。如果不只用theano的同学可以跳过此节。下面开始安装Theano:


1、安装各种包

#先update下

sudo  apt-get  update

# 安装gfortran,后面编译过程中会用到

sudo apt-get install gfortran

# 安装blas,Ubuntu下对应的是libopenblas,其它操作系统可能需要安装其它版本的blas——这是个OS相关的

sudo apt-get install libopenblas-dev

# 安装lapack,Ubuntu下对应的是liblapack-dev,和OS相关。

sudo apt-get install liblapack-dev

# 安装atlas,Ubuntu下对应的是libatlas-base-dev,和OS相关。

sudo apt-get install libatlas-base-dev

# 使用pip安装numpy和scipy

sudo apt-get install python-pip

# 安装numpy

sudo pip install numpy

# 测试numpy, 测试通过才能进行下一步~~

python -c "import numpy; numpy.test()"

# 安装scipy

sudo pip install scipy

# 测试scipy

# 测试通过才能进行下一步~~

python -c"import scipy; scipy.test()"
以上顺序不可打乱,否则会报错!

为了安装Theano,最后还需要安装一些库,可以参考官方教程:

sudo apt-get install python-dev
sudo apt-get install python-pip
sudo apt-get install python-nose
sudo apt-get install g++
sudo apt-get install git

2、安装Theano

用pip安装

sudo pip install Theano


# 测试Theano

python -c "import theano; theano.test()"

这里theano.test()报错很正常,我的是errors=80!没关系,不用管它,接着安装cuda。


(四)安装cuda


ubuntu16.04之前的版本作者没有装过,但是看到教程就头大!对于16.04这个版本来说却十分简单(没有sample,也无需添加环境变量)!在这里介绍安装cuda是于大部分教程不同,这是考虑了兼容性问题所做的决定。主要原因在于,caffe的编译需要gcc,g++为5.0以上版本,但是cuda不支持gcc,g++5.0以上版本。但只要caffe编译成功后,它对gcc,g++就没有要求。


1、安装显卡驱动

方法一:在‘Search your computer’中搜索‘Additional Drivers’,打开。选择‘Additional Drivers’选项卡下,选择‘Using NVIDIA binarydriver -…..’,单击‘Apply Changes’,等待完成。重启!
方法二:查看该机的nvidia驱动(如果不是nvidia的显卡,别费劲了)。输入命令:

ubuntu-drivers devices

安装对应的版本驱动(如:我的是nvidia-361)。输入命令:

sudo apt-get install nvidia-361

重启!


2、安装cuda

输入命令:

sudo apt-get install nvidia-cuda-toolkit
重启!虽然已经安装好了cuda,但还不能用。因为cuda不支持gcc5.0及以上版本,因此gcc需要降级。输入以下命令:

sudo apt-get install g++-4.9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9  20
sudo update-alternatives --install /usr/bin/gcc  gcc /usr/bin/gcc-5  10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.9  20
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-5  10
sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30
sudo update-alternatives --set cc /usr/bin/gcc
sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30
sudo update-alternatives --set c++ /usr/bin/g++

这些东西做完以后,我们的cuda也就安装完毕了!


(五)安装cuDNN

在nvidia官网下载cudnn(需要注册)。

1、将安装包放在‘/home’路径下。输入命令:

tar xvzf cudnn-7.5-linux-x64-v5.0-ga.tgz
sudo cp include/cudnn.h /usr/include
sudo cp lib64/libcudnn* /usr/lib/x86_64-linux-gnu/
sudo chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*


(六)为theano写配置文件


在终端输入命令:

sudo vim ~/.theanorc

输入‘i’进入insert模式。输入一下内容:

[global]
openmp=False
device=gpu
floatX=float32
allow_input-downcast=True
[blas]
ldflags=
[nvcc]
flags=-D_FORCE_INLINES

输入完毕,按‘Esc’退出,输入‘:’,再输入‘wq’,回车。配置完成!这时可以输入命令:

python  –c ‘import theano’

显示’Using gpu device 0:GeForce GTX 960’,说明显卡配置成功,否则重来一遍!




参考

http://blog.csdn.net/autocyz/article/details/51783857

http://blog.csdn.net/g0m3e/article/details/51420565

http://www.myexception.cn/cuda/2017261.html

http://www.bubuko.com/infodetail-902302.html





2019-09-15 21:47:54 dhy012345 阅读数 3
  • 基于Spark的分布式深度学习和认知计算

    你是否曾经面对多个优化算法不知所措?或者无法自由选择学习框架?又或许因为Caffe,Tensorflow, Theano, Torch的诸多参数设置而烦恼?或简单的认为只要有大数据就可以训练计算 机了?如果你不懂复杂的数学、统计学理论,还能做训练吗?...... 带着十万个为什么,让我们与深度学习技术讲师一起,了解基于Spark的分布式数据探索、机器学习/深度学习和认知计算。

    3098 人正在学习 去看看 CSDN讲师

Linux系统下python开发深度学习库Theano安装

安装Theano

Theano直接通过命令行pip install theano进行在线安装;也可以离线下载安装包,将安装包解压后将theano文件拷贝到安装包目录,再配置环境变量。

下面是离线安装的方法:
Theano下载地址:https://pypi.org/project/Theano/#files
解压到任意目录,进入安装目录,命令行运行 python setup.py install 安装文件。具体如下:

PS D:\download\install_src\WinPythonLib> cd .\Theano-1.0.4\
PS D:\download\install_src\WinPythonLib\Theano-1.0.4> python setup.py install
running install
running bdist_egg

到此,在Windows上theano安装结束。
查看安装情况:

PS D:\download\install_src\WinPythonLib\Theano-1.0.4> conda list theano
# packages in environment at d:\ProgramData\Anaconda3:
#
# Name                    Version                   Build  Channel
theano                    1.0.4                    pypi_0    pypi

在python中运行import theano,如果没有提示则表明安装成功,否则失败。 如下:

PS D:\download\install_src\WinPythonLib\Theano-1.0.4> python
Python 3.6.5 |Anaconda custom (64-bit)| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
>>> exit();
PS D:\download\install_src\WinPythonLib\Theano-1.0.4>

安装Keras

网络支持的话直接 conda install keras 。不便上网的话,先离线下载安装包再安装。可参考下面是相关下载链接和安装过程:

keras 安装依赖的包有 Keras-Preprocessing, Keras-Applications 等需要先安装。
keras下载地址:https://pypi.org/project/Keras/#files
Keras-Preprocessing 下载地址:https://pypi.org/project/Keras-Preprocessing/#files
Keras-Applications 下载地址:https://pypi.org/project/Keras-Applications/#files

解压到任意目录,进入安装目录,命令行运行 python setup.py install 安装文件。具体如下:

PS D:\download\install_src\WinPythonLib> cd .\Keras-2.2.4\
PS D:\download\install_src\WinPythonLib\Keras-2.2.4> python setup.py install
running install
running bdist_egg

.......

查看keras安装情况

PS D:\download\install_src\WinPythonLib\Keras-2.2.4> conda list Keras
# packages in environment at d:\ProgramData\Anaconda3:
#
# Name                    Version                   Build  Channel
keras-applications        1.0.7                    pypi_0    pypi
keras-preprocessing       1.0.8                    pypi_0    pypi
PS D:\download\install_src\WinPythonLib\Keras-2.2.4> python
Python 3.6.5 |Anaconda custom (64-bit)| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import keras
Using TensorFlow backend.
>>>
2016-01-23 15:26:39 mmc2015 阅读数 2953
  • 基于Spark的分布式深度学习和认知计算

    你是否曾经面对多个优化算法不知所措?或者无法自由选择学习框架?又或许因为Caffe,Tensorflow, Theano, Torch的诸多参数设置而烦恼?或简单的认为只要有大数据就可以训练计算 机了?如果你不懂复杂的数学、统计学理论,还能做训练吗?...... 带着十万个为什么,让我们与深度学习技术讲师一起,了解基于Spark的分布式数据探索、机器学习/深度学习和认知计算。

    3098 人正在学习 去看看 CSDN讲师

http://deeplearning.net/software/theano/install_windows.html


下载:

git clone https://github.com/Theano/Theano.git

配置(cd到Theano根目录):

python setup.py develop

测试:

import theano

提示:WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.

解决g++问题:

$ conda install mingw libpython

在theano的根目录下执行哈。。




测试:

import numpy as np
import time
import theano
A = np.random.rand(1000,10000).astype(theano.config.floatX)
B = np.random.rand(10000,1000).astype(theano.config.floatX)
np_start = time.time()
AB = A.dot(B)
np_end = time.time()
X,Y = theano.tensor.matrices('XY')
mf = theano.function([X,Y],X.dot(Y))
t_start = time.time()
tAB = mf(A,B)
t_end = time.time()
print "NP time: %f[s], theano time: %f[s] (times should be close when run on CPU!)" %(
                                           np_end-np_start, t_end-t_start)
print "Result difference: %f" % (np.abs(AB-tAB).max(), )   
结果:

NP time: 0.357000[s], theano time: 0.272000[s] (times should be close when run on CPU!)

Result difference: 0.000000

注意啊,如果你的第二个时间比第一个时间长,说明上面的g++问题没有解决,这里要注意(conda install命令要在theano的根目录下执行哈。。)。




完美!



http://deeplearning.net/software/theano/install_ubuntu.html#install-ubuntu

For NVIDIA Jetson TX1 embedded platform:

sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libblas-dev git
pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git --user  # Need Theano 0.8(not yet released) or more recent

For Ubuntu 11.10 through 14.04:

sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev git
sudo pip install Theano

For Ubuntu 11.04:

sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ git libatlas3gf-base libatlas-dev
sudo pip install Theano

linux下一样,我选择手动下载theano,然后

git clone git://github.com/Theano/Theano.git
cd Theano
python setup.py develop --user

2018-07-23 22:04:29 weixin_37805505 阅读数 371
  • 基于Spark的分布式深度学习和认知计算

    你是否曾经面对多个优化算法不知所措?或者无法自由选择学习框架?又或许因为Caffe,Tensorflow, Theano, Torch的诸多参数设置而烦恼?或简单的认为只要有大数据就可以训练计算 机了?如果你不懂复杂的数学、统计学理论,还能做训练吗?...... 带着十万个为什么,让我们与深度学习技术讲师一起,了解基于Spark的分布式数据探索、机器学习/深度学习和认知计算。

    3098 人正在学习 去看看 CSDN讲师

在linux在anaconda下安装theano时(参考使用conda安装theano环境

测试theano时报错use old gpu back-end,将~/.theanorc中device=gpu更改为device=cuda,再次运行,报错为缺模块pygpu。

在conda install pygpu后,仍不能运行,显示找不到cudnn.h的位置,解决办法是在~/.theanorc文件中,添加:

[dnn]
include_path=/usr/local/cuda/include/
library_path=/usr/local/cuda/lib64/

我的理解里,include_path对应cudnn.h的位置,library_path对应与之相应lib*文件,进入文件夹查看是对应的,但在测试theano程序时仍然报错,查找解决办法时根据Github上的解决办法,将~/.theanorc文件中[dnn]部分换为:

[dnn]
base_path = /usr/local/cuda

可以运行theano程序。

theano安装的问题

博文 来自: Addmana
没有更多推荐了,返回首页