Record the process of configuring nccl and horovod in these two days (original)

Installation Guide :: NVIDIA Deep Learning NCCL Documentation
NVIDIA Collective Communications Library (NCCL) Download Page | NVIDIA Developer

The above two are nccl Official website installation tutorial

One 、 Now install according to the official website （ It didn't work ）

1.

sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004 /x86_64/7fa2af80.pub

2.

sudo add-apt-repository “deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/ x86_64/ /”

( Change according to the version of your own operating system )

3.

sudo apt update

4.

(1) Install the latest version

sudo apt install libnccl2 libnccl-dev

(2) Install according to your version

sudo apt install libnccl2=2.4.8-1+cuda10.0 libnccl-dev=2.4.8-1+cuda10.0

But only installed , No compilation , What the official website said is not detailed ,/usr/local/ I can't find it nccl

Two 、 from github Next clone

nccl Of github Website

1.

git clone https://github.com/NVIDIA/nccl.git
cd nccl

2.

make -j12 src.build BUILDDIR=/home/yourname/nccl CUDA_HOME=/usr/local/cuda NVCC_GENCODE="-gencode=arch=compute_86, ode=sm_86"

(（NVCC_GENCODE Can not add , If you do not add this field , All schemas will be compiled by default ; To speed up compilation and reduce binary file size , Add this field , Specifically comute_86,sm_86 It matches the computing power of the graphics card , Specific view ：https://developer.nvidia.com/cuda-gpus）)

• -j12： Said the use of 12 Core , Use nproc Check the total number of cores , Adjust according to the specific situation ;
• BUILDDIR： Indicates that after compilation , The storage path of some files ; The default is nccl/build; Of course, if it is root Users can specify to /usr/local/ncc/;
• CUDA_HOME： Express CUDA The catalog of , The default is /usr/local/cuda, Can not add , If you make a mistake , add

3.

Finally , The compiled files are in the specified NVCC_GENCODE Field path , You need to add it to the environment variable ;

vim ~/.bashrc

This document is located in /home/yourname Next

Add the following ：

export LD_LIBRARY_PATH=$L{D}_{L}IBRAR{Y}_{P}ATH:/home/yourname/nccl/libexportPATH=$PATH:/home/yourname/nccl/bin

Press ESC,:WQ, Then execute the following command

source ~/.bashrc

4. verification NCCL Is the installation successful :

git clone https://github.com/NVIDIA/nccl-tests.git
cd nccl-tests
make -j12 CUDA_HOME=/usr/local/cuda
./build/all_reduce_perf -b 8 -e 256M -f 2 -g 4

( Machine required GPU Replace the number of , I have 4 A video card , Just designate 4;)

Linux Next NCCL Source code compilation and installation

Then install horovod（ It also needs to be installed Openmpi, gcc±5 above （horovod The official website requires ）, tensorflow>=1.15.0）

horovod

---

install horovod（ Environmental Science ） The right posture
openmpi introduction 1- Installation and testing
Horovod Installation and use
How to install Horovod？
ubuntu 18.04 install horovod

1. install gcc

• sudo apt-get install g++
• g++ --version see g++ Version of

2. see openmpi edition

• ompi_info （or mpiexec –version or mpirun –version or mpicxx --showme:version）

3. Start installation tensorflow

pip install tensorflow-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple

test tensorflow

import tensorflow as tf

Report errors

RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd

Solution one :

• pip install numpy –upgrade( upgrade Numpy, But mine is the latest version , Invalid )

Solution two ：

• uninstall numpy reinstall （ After uninstalling and installing again, I am prompted that my environment has Numpy, This shows that there are two environments just now numpy, import tensorflow as tf, Tips No module named 'numpy.core._multiarray_umath, upgrade numpy Successfully imported tensorflow）

4. install horovod

HOROVOD_GPU_OPERATIONS=NCCL pip install --no-cache-dir horovod

5. test

import tensorflow as tf
import horovod.tensorflow as hvd

( No report error , Installation successful )

6. test horovod Some examples of

horovod/examples/

---

issues

The link is provided horovod How to install