Update installation guide etc. regarding compilers and NCCL (horovod#3107)

Signed-off-by: Max H. Gerlach <[email protected]>

Author: maxhgerlach

README.rst

@@ -123,9 +123,9 @@ To install Horovod:
 
     <p/>
 
-2. If you've installed TensorFlow from `PyPI <https://pypi.org/project/tensorflow>`__, make sure that the ``g++-4.8.5`` or ``g++-4.9`` or above is installed.
+2. If you've installed TensorFlow from `PyPI <https://pypi.org/project/tensorflow>`__, make sure that ``g++-5`` or above is installed.
 
-   If you've installed PyTorch from `PyPI <https://pypi.org/project/torch>`__, make sure that the ``g++-4.9`` or above is installed.
+   If you've installed PyTorch from `PyPI <https://pypi.org/project/torch>`__, make sure that ``g++-5`` or above is installed.
 
    If you've installed either package from `Conda <https://conda.io>`_, make sure that the ``gxx_linux-64`` Conda package is installed.
 

docs/gpus.rst

@@ -32,9 +32,9 @@ by installing an `nv_peer_memory <https://github.com/Mellanox/nv_peer_memory>`__
 
    **Note**: Open MPI 3.1.3 has an issue that may cause hangs.  The recommended fix is to downgrade to Open MPI 3.1.2 or upgrade to Open MPI 4.0.0.
 
-4. If you installed TensorFlow from `PyPI <https://pypi.org/project/tensorflow>`__, make sure that the ``g++-4.8.5`` or ``g++-4.9`` or above is installed.
+4. If you installed TensorFlow from `PyPI <https://pypi.org/project/tensorflow>`__, make sure that ``g++-5`` or above is installed.
 
-   If you installed PyTorch from `PyPI <https://pypi.org/project/torch>`__, make sure that the ``g++-4.9`` or above is installed.
+   If you installed PyTorch from `PyPI <https://pypi.org/project/torch>`__, make sure that ``g++-5`` or above is installed.
 
    If you installed either package from `Conda <https://conda.io>`_, make sure that the ``gxx_linux-64`` Conda package is installed.
 

docs/index.rst

@@ -14,7 +14,7 @@ Choose your deep learning framework to learn how to get started with Horovod.
          <ol>
             <li><a href="https://www.open-mpi.org/faq/?category=building#easy-build">Install Open MPI 3.1.2 or 4.0.0</a>, or another MPI implementation. </li>
             <li>
-               If you've installed TensorFlow from <a href="https://pypi.org/project/tensorflow">PyPI</a>, make sure that the <code>g++-4.8.5</code> or <code>g++-4.9</code> or above is installed.<br/>
+               If you've installed TensorFlow from <a href="https://pypi.org/project/tensorflow">PyPI</a>, make sure that <code>g++-5</code> or above is installed.<br/>
                If you've installed TensorFlow from <a href="https://conda.io">Conda</a>, make sure that the <code>gxx_linux-64</code> Conda package is installed.
             </li>
             <li>Install the Horovod pip package: <code>pip install horovod</code></li>
@@ -30,7 +30,7 @@ Choose your deep learning framework to learn how to get started with Horovod.
          <ol>
             <li><a href="https://www.open-mpi.org/faq/?category=building#easy-build">Install Open MPI 3.1.2 or 4.0.0</a>, or another MPI implementation. </li>
             <li>
-               If you've installed TensorFlow from <a href="https://pypi.org/project/tensorflow">PyPI</a>, make sure that the <code>g++-4.8.5</code> or <code>g++-4.9</code> or above is installed.<br/>
+               If you've installed TensorFlow from <a href="https://pypi.org/project/tensorflow">PyPI</a>, make sure that <code>g++-5</code> or above is installed.<br/>
                If you've installed TensorFlow from <a href="https://conda.io">Conda</a>, make sure that the <code>gxx_linux-64</code> Conda package is installed.
             </li>
             <li>Install the Horovod pip package: <code>pip install horovod</code></li>
@@ -46,7 +46,7 @@ Choose your deep learning framework to learn how to get started with Horovod.
          <ol>
             <li><a href="https://www.open-mpi.org/faq/?category=building#easy-build">Install Open MPI 3.1.2 or 4.0.0</a>, or another MPI implementation. </li>
             <li>
-               If you've installed PyTorch from <a href="https://pypi.org/project/torch">PyPI</a>, make sure that the <code>g++-4.9</code> or above is installed.<br/>
+               If you've installed PyTorch from <a href="https://pypi.org/project/torch">PyPI</a>, make sure that <code>g++-5</code> or above is installed.<br/>
                If you've installed PyTorch from <a href="https://conda.io">Conda</a>, make sure that the <code>gxx_linux-64</code> Conda package is installed.
             </li>
             <li>Install the Horovod pip package: <code>pip install horovod</code></li>

docs/install.rst

@@ -7,6 +7,7 @@ Requirements
 ------------
 
 - Python >= 3.6
+- `g++-5` or above, or another compiler supporting C++14
 - CMake
 - TensorFlow, PyTorch, or MXNet
 - (Optional) MPI
@@ -51,9 +52,6 @@ To ensure that Horovod is built with TensorFlow support enabled:
 
 To skip TensorFlow, set ``HOROVOD_WITHOUT_TENSORFLOW=1`` in your environment.
 
-If you've installed TensorFlow from `PyPI <https://pypi.org/project/tensorflow>`__, make sure that
-the ``g++-4.8.5`` or ``g++-4.9`` or above is installed.
-
 PyTorch
 ~~~~~~~
 
@@ -65,9 +63,6 @@ To ensure that Horovod is built with PyTorch support enabled:
 
 To skip PyTorch, set ``HOROVOD_WITHOUT_PYTORCH=1`` in your environment.
 
-If you've installed PyTorch from `PyPI <https://pypi.org/project/torch>`__, make sure that the ``g++-4.9`` or
-above is installed.
-
 MXNet
 ~~~~~
 
@@ -165,12 +160,14 @@ will be used for CPU operations. You can override this by setting ``HOROVOD_CPU_
 NCCL
 ~~~~
 
-NCCL is supported for Allreduce, Allgather, and Broadcast operations.  You can enable these by setting
+NCCL is supported for Allreduce, Allgather, Broadcast, and Alltoall operations.  You can enable these by setting
 ``HOROVOD_GPU_OPERATIONS=NCCL`` during installation.
 
 NCCL operations are supported on both Nvidia (CUDA) and AMD (ROCm) GPUs. You can set ``HOROVOD_GPU`` in your
 environment to specify building with CUDA or ROCm. CUDA will be assumed if not specified.
 
+Note that Alltoall requires NCCL version >= 2.7.0.
+
 MPI
 ~~~
 

docs/summary.rst

@@ -115,9 +115,9 @@ To install Horovod:
 
     <p/>
 
-2. If you've installed TensorFlow from `PyPI <https://pypi.org/project/tensorflow>`__, make sure that the ``g++-4.8.5`` or ``g++-4.9`` or above is installed.
+2. If you've installed TensorFlow from `PyPI <https://pypi.org/project/tensorflow>`__, make sure that ``g++-5`` or above is installed.
 
-   If you've installed PyTorch from `PyPI <https://pypi.org/project/torch>`__, make sure that the ``g++-4.9`` or above is installed.
+   If you've installed PyTorch from `PyPI <https://pypi.org/project/torch>`__, make sure that ``g++-5`` or above is installed.
 
    If you've installed either package from `Conda <https://conda.io>`_, make sure that the ``gxx_linux-64`` Conda package is installed.