Skip to content

Commit 74f6fd7

Browse files
authored
Change default fusion threshold from 64MB to 128MB in docs (horovod#3241)
1 parent a776186 commit 74f6fd7

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/tensor-fusion.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Tensor Fusion works by attempting to combine all the tensors that are ready to b
1010
one reduction operation. The algorithm of Tensor Fusion is as follows:
1111

1212
1. Determine which tensors are ready to be reduced. Select first few tensors that fit in ``HOROVOD_FUSION_THRESHOLD`` bytes and have the same data type.
13-
2. Allocate fusion buffer of size ``HOROVOD_FUSION_THRESHOLD`` if it was not allocated before. Default fusion buffer size is 64 MB.
13+
2. Allocate fusion buffer of size ``HOROVOD_FUSION_THRESHOLD`` if it was not allocated before. Default fusion buffer size is 128 MB.
1414
3. Copy data of selected tensors into the fusion buffer.
1515
4. Execute the **allreduce** operation on the fusion buffer.
1616
5. Copy data from the fusion buffer into the output tensors.

0 commit comments

Comments
 (0)