ML Systems

Long-form references on training, infrastructure, and implementation practice.

Training Systems

• Frontier Model Training Methodologies
  Survey of open frontier training recipes and implementation choices.

• Scaling LLMs with JAX
  Book-length treatment of distributed training practice.

• Beyond Language Modeling: An Exploration of Multimodal Pretraining
  From-scratch multimodal pretraining study with useful details on representation choices and scaling behavior.

Embeddings And Retrieval

• How to Train the Best Embedding Model in the World
  Detailed engineering writeup on embedding model training, label noise, verification, and dataset scale.

GPU Programming

• GPU MODE Community and resource hub for GPU programming.

• CUDA Writeups by Tushar Gautam
  Implementation-forward notes on CUDA kernels and optimization.