[Doc] Fix typos, grammar, and broken commands across docs (#46398)

Signed-off-by: MichaelCaoo <[email protected]>
Co-authored-by: Claude <[email protected]>
This commit is contained in:
MichaelCaoo
2026-06-23 02:01:22 +00:00
committed by GitHub
co-authored by Claude
parent fa36f86d77
commit 33f50773cb
11 changed files with 16 additions and 18 deletions
+4 -4
View File
@@ -338,7 +338,7 @@ vllm bench serve \
--model meta-llama/Meta-Llama-3-8B-Instruct \
--dataset-name spec_bench \
--dataset-path "<YOUR_DOWNLOADED_PATH>/data/spec_bench/question.jsonl" \
--num-prompts -1
--num-prompts -1 \
--spec-bench-category "summarization"
```
@@ -352,7 +352,7 @@ vllm bench serve \
First, download the dataset to a folder, using this one liner:
```bash
curl -LsSf https://raw.githubusercontent.com/NVIDIA-NeMo/Skills/refs/heads/main/nemo_skills/dataset/speed-bench/prepare.py \| python3 -
curl -LsSf https://raw.githubusercontent.com/NVIDIA-NeMo/Skills/refs/heads/main/nemo_skills/dataset/speed-bench/prepare.py | python3 -
```
The command supports also the following arguments:
@@ -388,7 +388,7 @@ vllm bench serve \
--model meta-llama/Llama-3.3-70B-Instruct \
--dataset-name speed_bench \
--dataset-path "<YOUR_DOWNLOADED_PATH>/data/speed_bench" \
--num-prompts -1
--num-prompts -1 \
--speed-bench-category "multilingual"
```
@@ -398,7 +398,7 @@ Run all categories in the Throughput split (2k ISL):
vllm bench serve \
--model meta-llama/Llama-3.3-70B-Instruct \
--dataset-name speed_bench \
--speed-bench-dataset-subset throughput_2k
--speed-bench-dataset-subset throughput_2k \
--dataset-path "<YOUR_DOWNLOADED_PATH>/data/speed_bench/" \
--num-prompts -1
```
+1 -1
View File
@@ -109,7 +109,7 @@ from vllm import LLM
# Combine pipeline and tensor parallelism
llm = LLM(
model="meta-llama/Llama-3.3-70B-Instruct,
model="meta-llama/Llama-3.3-70B-Instruct",
tensor_parallel_size=4,
pipeline_parallel_size=2,
)
+2 -2
View File
@@ -161,11 +161,11 @@ class AttentionCGSupport(enum.Enum):
ALWAYS = 3
"""CUDA Graphs always supported; supports mixed-prefill-decode"""
UNIFORM_BATCH = 2
"""CUDA Graphs supported for batches the only contain query lengths that are
"""CUDA Graphs supported for batches that only contain query lengths that are
the same, this can be used for spec-decode
i.e. "decodes" are 1 + num_speculative_tokens"""
UNIFORM_SINGLE_TOKEN_DECODE = 1
"""CUDA Graphs supported for batches the only contain query_len==1 decodes"""
"""CUDA Graphs supported for batches that only contain query_len==1 decodes"""
NEVER = 0
"""NO CUDA Graphs support"""
```
+1 -1
View File
@@ -685,7 +685,7 @@ documentation for this option states:
> use of possibly costly and or blocking operations and hence might
> have a performance impact.
The metrics were added by <https://github.com/vllm-project/vllm/pull/7089> and who up in an OpenTelemetry trace
The metrics were added by <https://github.com/vllm-project/vllm/pull/7089> and show up in an OpenTelemetry trace
as:
```text
+2 -2
View File
@@ -27,7 +27,7 @@ In the example above, the KV cache in the first block can be uniquely identified
For `vllm serve`, you can control the hashing algorithm via `--prefix-caching-hash-algo`:
- `sha256` (default): Uses Python's `pickle` for serialization. Hashes may not be reproducible across different Python or vLLM versions.
- `sha256_cbor`: Uses `cbor2` for serialization, providing a reproducible, cross-language compatible hash. This is recommended for deterministic caching across environments.
- `xxhash`: `Uses Pickle serialization with xxHash (128-bit) for faster, non-cryptographic hashing. Requires the optional `xxhash` package. IMPORTANT: Use of a hashing algorithm that is not considered cryptographically secure theoretically increases the risk of hash collisions, which can cause undefined behavior or even leak private information in multi-tenant environments. Even if collisions are still very unlikely, it is important to consider your security risk tolerance against the performance benefits before turning this on.
- `xxhash`: Uses Pickle serialization with xxHash (128-bit) for faster, non-cryptographic hashing. Requires the optional `xxhash` package. IMPORTANT: Use of a hashing algorithm that is not considered cryptographically secure theoretically increases the risk of hash collisions, which can cause undefined behavior or even leak private information in multi-tenant environments. Even if collisions are still very unlikely, it is important to consider your security risk tolerance against the performance benefits before turning this on.
- `xxhash_cbor` combines canonical CBOR serialization with xxHash for reproducible hashing. Requires the optional `xxhash` package.
**A hashing example with multi-modality inputs**
@@ -197,7 +197,7 @@ As can be seen, block 3 is a new full block and is cached. However, it is redund
When a request is finished, we free all its blocks if no other requests are using them (reference count = 0). In this example, we free request 1 and block 2, 3, 4, 8 associated with it. We can see that the freed blocks are added to the tail of the free queue in the *reverse* order. This is because the last block of a request must hash more tokens and is less likely to be reused by other requests. As a result, it should be evicted first.
![Free queue after a request us freed](../assets/design/prefix_caching/free.png)
![Free queue after a request is freed](../assets/design/prefix_caching/free.png)
### Eviction (LRU)
+1 -1
View File
@@ -55,7 +55,7 @@ Here is an example of how to quantize `meta-llama/Llama-3.2-1B-Instruct`:
## Running a quantized model with vLLM
To run an GPTQModel quantized model with vLLM, you can use [DeepSeek-R1-Distill-Qwen-7B-gptqmodel-4bit-vortex-v2](https://huggingface.co/ModelCloud/DeepSeek-R1-Distill-Qwen-7B-gptqmodel-4bit-vortex-v2) with the following command:
To run a GPTQModel quantized model with vLLM, you can use [DeepSeek-R1-Distill-Qwen-7B-gptqmodel-4bit-vortex-v2](https://huggingface.co/ModelCloud/DeepSeek-R1-Distill-Qwen-7B-gptqmodel-4bit-vortex-v2) with the following command:
```bash
python examples/deployment/llm_engine_example.py \
@@ -78,8 +78,6 @@ def tokenize(sample):
ds = ds.map(tokenize, remove_columns=ds.column_names)
```
</details>
### 3. Applying Quantization
Now, apply the quantization algorithms:
+1 -1
View File
@@ -190,7 +190,7 @@ For mitigation strategies, please refer to the FAQ entry *Can the output of a pr
## Known Feature Incompatibility
1. Pipeline parallelism is not composible with speculative decoding as of `vllm<=0.15.0`
1. Pipeline parallelism is not composable with speculative decoding as of `vllm<=0.15.0`
2. Speculative decoding with a draft models is not supported in `vllm<=0.10.0`
## Resources for vLLM contributors
+1 -1
View File
@@ -338,7 +338,7 @@ Supported models:
Flags: `--tool-call-parser deepseek_v31 --chat-template {see_above}`
### OpenAI OSS Models ('openai`)
### OpenAI OSS Models (`openai`)
Supported models:
+2 -2
View File
@@ -143,7 +143,7 @@ enabling the corresponding APIs.
The [classify][vllm.LLM.classify] method outputs a probability vector for each prompt.
It is primarily designed for [classification models](classify.md).
For more information about `LLM.embed`, see [this page](classify.md#offline-inference).
For more information about `LLM.classify`, see [this page](classify.md#offline-inference).
### `LLM.embed`
@@ -302,7 +302,7 @@ Pooling models now support token-wise task.
### Score task
`score` task have has been removed in v0.21, use `classify` instead. Only when a classification model outputs num_labels
`score` task has been removed in v0.21, use `classify` instead. Only when a classification model outputs num_labels
equal to 1 can it be used as a scoring model and have its scoring API enabled.
### Pooling multitask support
+1 -1
View File
@@ -440,7 +440,7 @@ More examples can be found here: [examples/pooling/score](../../../examples/pool
## Supported Features
AS cross-encoder models are a subset of classification models that accept two prompts as input and output num_labels equal to 1, cross-encoder features should be consistent with (sequence) classification. For more information, see [this page](classify.md#supported-features).
As cross-encoder models are a subset of classification models that accept two prompts as input and output num_labels equal to 1, cross-encoder features should be consistent with (sequence) classification. For more information, see [this page](classify.md#supported-features).
### Score Template