Tag: language model evaluation

A language model is a probability distribution over sequences of tokens. When you train a language model, you want to measure how accurately it…