Tag: mixed precision techniques

Training a language model is memory-intensive, not only because the model itself is large but also because the long sequences in the training data…