Language-models

Cross entropy and Kullback-Leibler (KL) divergence are key concepts in training machine learning models. In this post, we derive these concepts from first principles.