From efc347f1b2e635753ae8a594e9714109c200fcd3 Mon Sep 17 00:00:00 2001 From: Chunyang Wen Date: Fri, 5 Jun 2026 17:12:49 +0800 Subject: [PATCH] docs: fix tokenizer optimization typo (#44066) Signed-off-by: chunyang.wen --- docs/configuration/optimization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/configuration/optimization.md b/docs/configuration/optimization.md index 5bf789a0919..80aec64ee5b 100644 --- a/docs/configuration/optimization.md +++ b/docs/configuration/optimization.md @@ -296,7 +296,7 @@ llm = LLM(model="Qwen/Qwen3-8B") The `fastokens` Python package (>= 0.2.0) must be installed; if it isn't, vLLM raises a clear `ImportError` at tokenizer load. The override applies to any `--tokenizer-mode` that ends up loading an HF fast tokenizer (`hf`, -`deepseek_v32`, `deepseek_v4`, `qwen_vl`, …). Modes that don't use the HF +`deepseek_v32`, `deepseek_v4`, `qwen_vl`, …). Models that don't use the HF fast tokenizer (`mistral`, `grok2`, `kimi_audio`) ignore the flag. Tokenizer-bound workloads — long shared prefixes, bursty short prompts,