From 33f50773cbec56cda66af786443bd13409df9bd5 Mon Sep 17 00:00:00 2001 From: MichaelCaoo <139663530+MichaelCao0@users.noreply.github.com> Date: Tue, 23 Jun 2026 10:01:22 +0800 Subject: [PATCH] [Doc] Fix typos, grammar, and broken commands across docs (#46398) Signed-off-by: MichaelCaoo Co-authored-by: Claude --- docs/benchmarking/cli.md | 8 ++++---- docs/configuration/optimization.md | 2 +- docs/design/cuda_graphs.md | 4 ++-- docs/design/metrics.md | 2 +- docs/design/prefix_caching.md | 4 ++-- docs/features/quantization/gptqmodel.md | 2 +- docs/features/quantization/llm_compressor/int8_w8a8.md | 2 -- docs/features/speculative_decoding/README.md | 2 +- docs/features/tool_calling.md | 2 +- docs/models/pooling_models/README.md | 4 ++-- docs/models/pooling_models/scoring.md | 2 +- 11 files changed, 16 insertions(+), 18 deletions(-) diff --git a/docs/benchmarking/cli.md b/docs/benchmarking/cli.md index 22406f2eaa2..7963bf58437 100644 --- a/docs/benchmarking/cli.md +++ b/docs/benchmarking/cli.md @@ -338,7 +338,7 @@ vllm bench serve \ --model meta-llama/Meta-Llama-3-8B-Instruct \ --dataset-name spec_bench \ --dataset-path "/data/spec_bench/question.jsonl" \ - --num-prompts -1 + --num-prompts -1 \ --spec-bench-category "summarization" ``` @@ -352,7 +352,7 @@ vllm bench serve \ First, download the dataset to a folder, using this one liner: ```bash -curl -LsSf https://raw.githubusercontent.com/NVIDIA-NeMo/Skills/refs/heads/main/nemo_skills/dataset/speed-bench/prepare.py \| python3 - +curl -LsSf https://raw.githubusercontent.com/NVIDIA-NeMo/Skills/refs/heads/main/nemo_skills/dataset/speed-bench/prepare.py | python3 - ``` The command supports also the following arguments: @@ -388,7 +388,7 @@ vllm bench serve \ --model meta-llama/Llama-3.3-70B-Instruct \ --dataset-name speed_bench \ --dataset-path "/data/speed_bench" \ - --num-prompts -1 + --num-prompts -1 \ --speed-bench-category "multilingual" ``` @@ -398,7 +398,7 @@ Run all categories in the Throughput split (2k ISL): vllm bench serve \ --model meta-llama/Llama-3.3-70B-Instruct \ --dataset-name speed_bench \ - --speed-bench-dataset-subset throughput_2k + --speed-bench-dataset-subset throughput_2k \ --dataset-path "/data/speed_bench/" \ --num-prompts -1 ``` diff --git a/docs/configuration/optimization.md b/docs/configuration/optimization.md index 42458d50281..32e7726cb15 100644 --- a/docs/configuration/optimization.md +++ b/docs/configuration/optimization.md @@ -109,7 +109,7 @@ from vllm import LLM # Combine pipeline and tensor parallelism llm = LLM( - model="meta-llama/Llama-3.3-70B-Instruct, + model="meta-llama/Llama-3.3-70B-Instruct", tensor_parallel_size=4, pipeline_parallel_size=2, ) diff --git a/docs/design/cuda_graphs.md b/docs/design/cuda_graphs.md index 718a4a8154d..e274b68c702 100644 --- a/docs/design/cuda_graphs.md +++ b/docs/design/cuda_graphs.md @@ -161,11 +161,11 @@ class AttentionCGSupport(enum.Enum): ALWAYS = 3 """CUDA Graphs always supported; supports mixed-prefill-decode""" UNIFORM_BATCH = 2 - """CUDA Graphs supported for batches the only contain query lengths that are + """CUDA Graphs supported for batches that only contain query lengths that are the same, this can be used for spec-decode i.e. "decodes" are 1 + num_speculative_tokens""" UNIFORM_SINGLE_TOKEN_DECODE = 1 - """CUDA Graphs supported for batches the only contain query_len==1 decodes""" + """CUDA Graphs supported for batches that only contain query_len==1 decodes""" NEVER = 0 """NO CUDA Graphs support""" ``` diff --git a/docs/design/metrics.md b/docs/design/metrics.md index 0ae42039976..7b463b8750c 100644 --- a/docs/design/metrics.md +++ b/docs/design/metrics.md @@ -685,7 +685,7 @@ documentation for this option states: > use of possibly costly and or blocking operations and hence might > have a performance impact. -The metrics were added by and who up in an OpenTelemetry trace +The metrics were added by and show up in an OpenTelemetry trace as: ```text diff --git a/docs/design/prefix_caching.md b/docs/design/prefix_caching.md index 0f3100c9b73..f783f4a1bc8 100644 --- a/docs/design/prefix_caching.md +++ b/docs/design/prefix_caching.md @@ -27,7 +27,7 @@ In the example above, the KV cache in the first block can be uniquely identified For `vllm serve`, you can control the hashing algorithm via `--prefix-caching-hash-algo`: - `sha256` (default): Uses Python's `pickle` for serialization. Hashes may not be reproducible across different Python or vLLM versions. - `sha256_cbor`: Uses `cbor2` for serialization, providing a reproducible, cross-language compatible hash. This is recommended for deterministic caching across environments. - - `xxhash`: `Uses Pickle serialization with xxHash (128-bit) for faster, non-cryptographic hashing. Requires the optional `xxhash` package. IMPORTANT: Use of a hashing algorithm that is not considered cryptographically secure theoretically increases the risk of hash collisions, which can cause undefined behavior or even leak private information in multi-tenant environments. Even if collisions are still very unlikely, it is important to consider your security risk tolerance against the performance benefits before turning this on. + - `xxhash`: Uses Pickle serialization with xxHash (128-bit) for faster, non-cryptographic hashing. Requires the optional `xxhash` package. IMPORTANT: Use of a hashing algorithm that is not considered cryptographically secure theoretically increases the risk of hash collisions, which can cause undefined behavior or even leak private information in multi-tenant environments. Even if collisions are still very unlikely, it is important to consider your security risk tolerance against the performance benefits before turning this on. - `xxhash_cbor` combines canonical CBOR serialization with xxHash for reproducible hashing. Requires the optional `xxhash` package. **A hashing example with multi-modality inputs** @@ -197,7 +197,7 @@ As can be seen, block 3 is a new full block and is cached. However, it is redund When a request is finished, we free all its blocks if no other requests are using them (reference count = 0). In this example, we free request 1 and block 2, 3, 4, 8 associated with it. We can see that the freed blocks are added to the tail of the free queue in the *reverse* order. This is because the last block of a request must hash more tokens and is less likely to be reused by other requests. As a result, it should be evicted first. -![Free queue after a request us freed](../assets/design/prefix_caching/free.png) +![Free queue after a request is freed](../assets/design/prefix_caching/free.png) ### Eviction (LRU) diff --git a/docs/features/quantization/gptqmodel.md b/docs/features/quantization/gptqmodel.md index 636a952b655..235afee5f32 100644 --- a/docs/features/quantization/gptqmodel.md +++ b/docs/features/quantization/gptqmodel.md @@ -55,7 +55,7 @@ Here is an example of how to quantize `meta-llama/Llama-3.2-1B-Instruct`: ## Running a quantized model with vLLM -To run an GPTQModel quantized model with vLLM, you can use [DeepSeek-R1-Distill-Qwen-7B-gptqmodel-4bit-vortex-v2](https://huggingface.co/ModelCloud/DeepSeek-R1-Distill-Qwen-7B-gptqmodel-4bit-vortex-v2) with the following command: +To run a GPTQModel quantized model with vLLM, you can use [DeepSeek-R1-Distill-Qwen-7B-gptqmodel-4bit-vortex-v2](https://huggingface.co/ModelCloud/DeepSeek-R1-Distill-Qwen-7B-gptqmodel-4bit-vortex-v2) with the following command: ```bash python examples/deployment/llm_engine_example.py \ diff --git a/docs/features/quantization/llm_compressor/int8_w8a8.md b/docs/features/quantization/llm_compressor/int8_w8a8.md index 21ed00d1393..64bce832c18 100644 --- a/docs/features/quantization/llm_compressor/int8_w8a8.md +++ b/docs/features/quantization/llm_compressor/int8_w8a8.md @@ -78,8 +78,6 @@ def tokenize(sample): ds = ds.map(tokenize, remove_columns=ds.column_names) ``` - - ### 3. Applying Quantization Now, apply the quantization algorithms: diff --git a/docs/features/speculative_decoding/README.md b/docs/features/speculative_decoding/README.md index 7213ef41ecd..65f396e04a3 100644 --- a/docs/features/speculative_decoding/README.md +++ b/docs/features/speculative_decoding/README.md @@ -190,7 +190,7 @@ For mitigation strategies, please refer to the FAQ entry *Can the output of a pr ## Known Feature Incompatibility -1. Pipeline parallelism is not composible with speculative decoding as of `vllm<=0.15.0` +1. Pipeline parallelism is not composable with speculative decoding as of `vllm<=0.15.0` 2. Speculative decoding with a draft models is not supported in `vllm<=0.10.0` ## Resources for vLLM contributors diff --git a/docs/features/tool_calling.md b/docs/features/tool_calling.md index 10626a254b1..ae65231919a 100644 --- a/docs/features/tool_calling.md +++ b/docs/features/tool_calling.md @@ -338,7 +338,7 @@ Supported models: Flags: `--tool-call-parser deepseek_v31 --chat-template {see_above}` -### OpenAI OSS Models ('openai`) +### OpenAI OSS Models (`openai`) Supported models: diff --git a/docs/models/pooling_models/README.md b/docs/models/pooling_models/README.md index d9ce27dd216..37fca366eba 100644 --- a/docs/models/pooling_models/README.md +++ b/docs/models/pooling_models/README.md @@ -143,7 +143,7 @@ enabling the corresponding APIs. The [classify][vllm.LLM.classify] method outputs a probability vector for each prompt. It is primarily designed for [classification models](classify.md). -For more information about `LLM.embed`, see [this page](classify.md#offline-inference). +For more information about `LLM.classify`, see [this page](classify.md#offline-inference). ### `LLM.embed` @@ -302,7 +302,7 @@ Pooling models now support token-wise task. ### Score task -`score` task have has been removed in v0.21, use `classify` instead. Only when a classification model outputs num_labels +`score` task has been removed in v0.21, use `classify` instead. Only when a classification model outputs num_labels equal to 1 can it be used as a scoring model and have its scoring API enabled. ### Pooling multitask support diff --git a/docs/models/pooling_models/scoring.md b/docs/models/pooling_models/scoring.md index a4b0fe5d2ea..e3b54b02075 100644 --- a/docs/models/pooling_models/scoring.md +++ b/docs/models/pooling_models/scoring.md @@ -440,7 +440,7 @@ More examples can be found here: [examples/pooling/score](../../../examples/pool ## Supported Features -AS cross-encoder models are a subset of classification models that accept two prompts as input and output num_labels equal to 1, cross-encoder features should be consistent with (sequence) classification. For more information, see [this page](classify.md#supported-features). +As cross-encoder models are a subset of classification models that accept two prompts as input and output num_labels equal to 1, cross-encoder features should be consistent with (sequence) classification. For more information, see [this page](classify.md#supported-features). ### Score Template