Tokenization

Converting continuous or structured data into discrete tokens for processing by transformer models. In VLAs, tokenization applies to: text (BPE tokenization into subword tokens), images (patch tokenization into visual tokens), and actions (discretization into action bins or clustering into action tokens). The tokenization scheme significantly impacts model capacity and inference speed.