You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/tensor-fusion.rst
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ Tensor Fusion works by attempting to combine all the tensors that are ready to b
10
10
one reduction operation. The algorithm of Tensor Fusion is as follows:
11
11
12
12
1. Determine which tensors are ready to be reduced. Select first few tensors that fit in ``HOROVOD_FUSION_THRESHOLD`` bytes and have the same data type.
13
-
2. Allocate fusion buffer of size ``HOROVOD_FUSION_THRESHOLD`` if it was not allocated before. Default fusion buffer size is 64 MB.
13
+
2. Allocate fusion buffer of size ``HOROVOD_FUSION_THRESHOLD`` if it was not allocated before. Default fusion buffer size is 128 MB.
14
14
3. Copy data of selected tensors into the fusion buffer.
15
15
4. Execute the **allreduce** operation on the fusion buffer.
16
16
5. Copy data from the fusion buffer into the output tensors.
0 commit comments