Byte pair encoding

Pairings in the atlas

Frequency-merge vocabularycanonSubword tokenization information-retrieval-nlp

Rivals: other methods for the same problems

Where it sits

information-retrieval-nlp · Databases & Information Retrieval