mirror of
https://github.com/vllm-project/vllm.git
synced 2026-08-16 02:38:12 +00:00
[Docs] Add SVG images for pooling models. (#42626)
Signed-off-by: Gracie Guo <[email protected]> Signed-off-by: wang.yuqi <[email protected]> Co-authored-by: Gracie Guo <[email protected]> Co-authored-by: wang.yuqi <[email protected]>
This commit is contained in:
co-authored by
Gracie Guo
wang.yuqi
parent
27f4ba9481
commit
9fd8487d2f
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 955 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 587 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 818 KiB |
@@ -33,6 +33,12 @@ from large language models, allowing them to benefit from the continuous improve
|
||||
similarity enables them to reuse much of vLLM’s infrastructure. If compatible, we would be happy to help them leverage
|
||||
the latest features of vLLM as well.
|
||||
|
||||
### Cheat Sheet
|
||||
|
||||
As illustrated in the figure below, we have summarized the relationships among the key elements of pooling models as a takeaway.
|
||||
|
||||

|
||||
|
||||
### Sequence-wise Task and Token-wise Task
|
||||
|
||||
The key distinction between sequence-wise task and token-wise task lies in their output granularity: sequence-wise task
|
||||
@@ -61,6 +67,8 @@ are a subset of classification models that accept two prompts as input and outpu
|
||||
|
||||
### Pooling Types
|
||||
|
||||

|
||||
|
||||
| Pooling Tasks | Granularity | Description |
|
||||
|----------------|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| `CLS` pooling | Sequence-wise | For BERT‑like (bidirectional self‑attention) models, CLS pooling is used by default. This means the last_hidden_states corresponding to the first token (the [CLS] token) is taken as the output. |
|
||||
@@ -71,6 +79,8 @@ are a subset of classification models that accept two prompts as input and outpu
|
||||
|
||||
### Score Types
|
||||
|
||||

|
||||
|
||||
The scoring models is designed to compute similarity scores between two input prompts. It supports three model types
|
||||
(aka `score_type`): `cross-encoder`, `late-interaction`, and `bi-encoder`.
|
||||
|
||||
|
||||
@@ -25,6 +25,12 @@ The score models is designed to compute similarity scores between two input prom
|
||||
!!! note
|
||||
Only when a classification model outputs num_labels equal to 1 can it be used as a scoring model and have its scoring API enabled.
|
||||
|
||||
### Score Types
|
||||
|
||||
The three supported scoring functions are as illustrated in the figure below.
|
||||
|
||||

|
||||
|
||||
## Supported Models
|
||||
|
||||
### Cross-encoder models
|
||||
|
||||
Reference in New Issue
Block a user