Tags

Force allreduce of all gradients in step(), bugfixes

FP16 hierarchical allreduce

PyTorch 1.0, TF-Keras, FP16 ops on GPU

Parallelized hierarchical allreduce

Support for the upcoming PyTorch release

Add compatibility with PyTorch 0.4.1

Support for IBM PowerAI DDL & APIs to restore optimizer state

Critical Bugfix: PyTorch must wait for GPU data before allreduce

Critical Bugfix: non-fused allreduce produces incorrect results

Hierarchical allreduce & differentiable ops

Fix PyTorch master break - use proper THTensor_storage() API

Bugfix mpi4py: Create a private MPI communicator

Collective control plane & other low latency improvements

Bugfix: `python setup.py install` requires cffi

Support TensorFlow Optimizers that override compute_gradients/apply_gradients

PyTorch support for Horovod

Avoid deadlock if worker crashes with an exception

New major release

Add hvd.local_size() API

Add Keras ImageNet training example and LearningRateScheduleCallback