Sequence parallelism

Pairings in the atlas

Activation sharding along tokensspecialistModel parallelism llm-training-alignment

Rivals: other methods for the same problems

Where it sits

llm-training-alignment · Machine Learning & AI