[Render] Add round trip parity test and docs for derender (#48617)

Signed-off-by: Martin Hickey <[email protected]>

docs/serving/online_serving/README.md

@@ -137,8 +137,12 @@ For further details on renderer APIs, please refer to [this page](renderer.md).
 
 ### Derenderer APIs
 
-- `/v1/completions/derender` - Derenderer completion requests
-- `/v1/chat/completions/derender` - Derenderer chat completion requests
+For further details on derenderer APIs, please refer to [this page](derenderer.md).
+
+- [Chat Completions Derender API](derenderer.md) (`/v1/chat/completions/derender`)
+    - Derender chat completion requests
+- [Completions Derender API](derenderer.md) (`/v1/completions/derender`)
+    - Derender completion requests
 
 ## Tokenize APIs
 

docs/serving/online_serving/derenderer.md

@@ -0,0 +1,98 @@
+# Derenderer APIs
+
+The derenderer API is the post processing counterpart to the [Renderer APIs](renderer.md). Where `/render` turns a request into token ID (preprocessing), `/derender` turns generated token IDs back into a fully formed OpenAI compatible response (detokenization, reasoning parsing, tool call parsing), all without a GPU.
+
+This closes the loop for a token-in / token-out engine in disaggregated serving:
+
+- **GPU less post processing**: Detokenization, reasoning parsing, and tool call parsing run on the same GPU less frontend that hosts `/render`
+- **Parser parity**: The derenderer reuses vLLM's tool and reasoning parsers, so a disaggregated deployment produces the same `content`/`reasoning`/ `tool_calls` split as a standard `vllm serve` server
+- **Non-streaming**: The endpoints expect a complete `GenerateResponse` with all token IDs present and perform one-shot parsing. Streaming derender would require a separate endpoint design and is not currently supported but is in the pipeline
+
+Both endpoints are hosted by the GPU less rendering server started with [`vllm launch render`](../../cli/launch/render.md), alongside the `/render`
+endpoints.
+
+## Pipeline
+
+```text
+                render                 generate                derender
+  request  ───────────────▶  token_ids  ─────────▶  token_ids  ──────────▶  response
+ (chat /            (GPU less)          (token-in /            (GPU less)   (OpenAI
+ completion)            │               token-out engine)          ▲        compatible)
+                        └─────────────── request + prompt_tokens ──┘
+```
+
+The derender step needs more than the engine's `token_ids`. It also consumes the original `chat_request`/`completion_request` and `prompt_tokens` carried over from the render step (see [Request format](#request-format)) so the tool and reasoning parsers have the context they need.
+
+## API Reference
+
+- Chat Completions Derender API (`/v1/chat/completions/derender`)
+    - Post process a single `GenerateResponse` into a `ChatCompletionResponse`
+- Completions Derender API (`/v1/completions/derender`)
+    - Post process a list of `GenerateResponse` objects (one per prompt) into a `CompletionResponse`
+
+## Request format
+
+Each request wraps the engine's `GenerateResponse`(s) together with the caller metadata needed to reconstruct the final response without a GPU.
+
+`/v1/chat/completions/derender`:
+
+??? code
+
+    ```python
+    --8<-- "vllm/entrypoints/scale_out/token_in_token_out/protocol.py:derender-chat-request"
+    ```
+
+`/v1/completions/derender`:
+
+??? code
+
+    ```python
+    --8<-- "vllm/entrypoints/scale_out/token_in_token_out/protocol.py:derender-completion-request"
+    ```
+
+Oversized payloads are rejected with a `400` before any `tokenizer.decode()` or parser runs.
+
+## Example
+
+The example below drives the full `render → generate → derender` round trip for a chat request against a GPU less render server (`/render`, `/derender`) and a token-in / token-out engine (`/inference/v1/generate`).
+
+```python
+import httpx
+
+MODEL = "meta-llama/Llama-3.2-1B-Instruct"
+RENDER = "http://localhost:8100"  # vllm launch render ...
+ENGINE = "http://localhost:8200"  # token-in / token-out engine
+
+chat_request = {
+    "model": MODEL,
+    "messages": [{"role": "user", "content": "What is 2+2?"}],
+    "max_tokens": 32,
+}
+
+with httpx.Client(timeout=60.0) as client:
+    # 1. Render: request -> token IDs (GPU less)
+    generate_request = client.post(
+        f"{RENDER}/v1/chat/completions/render", json=chat_request
+    ).json()
+    prompt_tokens = len(generate_request["token_ids"])
+
+    # 2. Generate: token IDs -> token IDs (token-in / token-out engine)
+    generate_response = client.post(
+        f"{ENGINE}/inference/v1/generate", json=generate_request
+    ).json()
+
+    # 3. Derender: token IDs -> ChatCompletionResponse (GPU less)
+    response = client.post(
+        f"{RENDER}/v1/chat/completions/derender",
+        json={
+            "model": MODEL,
+            "generate_response": generate_response,
+            "prompt_tokens": prompt_tokens,
+            "chat_request": chat_request,
+        },
+    ).json()
+
+print(response["choices"][0]["message"]["content"])
+```
+
+Passing `chat_request` lets the derenderer run the configured tool and reasoning parsers. This means `response["choices"][0]["message"]` carries the same `content` / `reasoning` / `tool_calls` split a `vllm serve` server would produce. Omit `chat_request` for plain detokenization only.

docs/serving/online_serving/renderer.md

@@ -12,3 +12,5 @@ Our renderer API is designed to disaggregate the render phase(preprocessing) and
     - Render completion requests
 - [Chat Completions Render API](renderer.md) (`/v1/chat/completions/render`)
     - Render chat completions
+
+For the post processing counterpart that turns generated token IDs back into OpenAI compatible responses, see the [Derenderer APIs](derenderer.md).

docs/usage/security.md

@@ -155,8 +155,10 @@ When `--api-key` is configured, the following `/v1` endpoints require Bearer tok
 - `/v1/chat/completions` - Chat completions
 - `/v1/chat/completions/batch` - Batch chat completions
 - `/v1/chat/completions/render` - Render chat completion requests
+- `/v1/chat/completions/derender` - Derender chat completion requests
 - `/v1/completions` - Text completions
 - `/v1/completions/render` - Render completion requests
+- `/v1/completions/derender` - Derender completion requests
 - `/v1/embeddings` - Generate embeddings
 - `/v1/audio/transcriptions` - Audio transcription
 - `/v1/audio/translations` - Audio translation

tests/entrypoints/scale_out/derender/test_derender_parity.py

@@ -0,0 +1,268 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+
+"""Round trip parity CI: render -> generate -> derender -> render == render.
+
+Pins the coupled parsing path (``/v1/chat/completions`` on a normal GPU
+server which parses generated tokens incrementally as they stream out)
+against the disaggregated path (``OnlineDerenderer.derender_chat`` via
+``/v1/chat/completions/derender`` which parses the same tokens all at once,
+out of process). A standard ``vllm serve`` GPU server mounts both, so one
+real generation lets both parsers run on identical input.
+
+Both paths consume the same generated token IDs (extracted from the coupled
+response's ``token_ids`` via ``return_token_ids=True``), so generation
+nondeterminism is irrelevant. The only variable under test is whether the
+two parsing code paths agree. Parity is asserted unconditionally as only the
+stronger per case assertions (e.g. "a tool call was produced") are gated
+behind the marker actually having been emitted since a 1.5B model is not
+guaranteed to emit ``<think>`` / ``<tool_call>``.
+"""
+
+import json
+
+import httpx
+import pytest
+import pytest_asyncio
+
+from tests.utils import RemoteOpenAIServer
+
+MODEL = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
+ARGS = [
+    "--enable-auto-tool-choice",
+    "--tool-call-parser",
+    "hermes",
+    "--reasoning-parser",
+    "deepseek_r1",
+]
+
+TOOLS = [
+    {
+        "type": "function",
+        "function": {
+            "name": "get_weather",
+            "description": "Get weather for a city",
+            "parameters": {
+                "type": "object",
+                "properties": {"city": {"type": "string"}},
+            },
+        },
+    }
+]
+FORCE_WEATHER_TOOL = {"type": "function", "function": {"name": "get_weather"}}
+
+
+@pytest.fixture(scope="module")
+def server():
+    with RemoteOpenAIServer(MODEL, ARGS) as remote_server:
+        yield remote_server
+
+
+@pytest_asyncio.fixture
+async def client(server):
+    async with httpx.AsyncClient(
+        base_url=server.url_for(""), timeout=60.0
+    ) as http_client:
+        yield http_client
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+async def _coupled(client: httpx.AsyncClient, messages: list[dict], **extra) -> dict:
+    resp = await client.post(
+        "/v1/chat/completions",
+        json={
+            "model": MODEL,
+            "messages": messages,
+            "temperature": 0,
+            "max_tokens": 128,
+            "return_token_ids": True,
+            **extra,
+        },
+    )
+    assert resp.status_code == 200, resp.text
+    return resp.json()
+
+
+async def _disagg(
+    client: httpx.AsyncClient,
+    output_ids: list[int],
+    prompt_tokens: int,
+    finish_reason: str,
+    chat_request: dict,
+    logprobs: dict | None = None,
+) -> dict:
+    resp = await client.post(
+        "/v1/chat/completions/derender",
+        json={
+            "model": MODEL,
+            "generate_response": {
+                "request_id": "parity",
+                "choices": [
+                    {
+                        "index": 0,
+                        "token_ids": output_ids,
+                        "finish_reason": finish_reason,
+                        "logprobs": logprobs,
+                    }
+                ],
+            },
+            "prompt_tokens": prompt_tokens,
+            "chat_request": chat_request,
+        },
+    )
+    assert resp.status_code == 200, resp.text
+    return resp.json()
+
+
+def _tool_sig(response_choice: dict) -> list[tuple[str, dict]]:
+    """[(name, json normalized args)] so key ordering / whitespace don't
+    cause false negatives."""
+    return [
+        (tc["function"]["name"], json.loads(tc["function"]["arguments"]))
+        for tc in (response_choice["message"].get("tool_calls") or [])
+    ]
+
+
+def _assert_parity(coupled: dict, disagg: dict) -> None:
+    """Both paths saw the same tokens, so they must agree unconditionally."""
+    c, d = coupled["choices"][0], disagg["choices"][0]
+    assert d["message"]["content"] == c["message"]["content"]
+    assert d["message"].get("reasoning") == c["message"].get("reasoning")
+    assert _tool_sig(d) == _tool_sig(c)
+    assert d["finish_reason"] == c["finish_reason"]
+    assert disagg["usage"]["prompt_tokens"] == coupled["usage"]["prompt_tokens"]
+    assert disagg["usage"]["completion_tokens"] == len(c["token_ids"])
+
+
+async def _run_parity_case(
+    client: httpx.AsyncClient, messages: list[dict], **extra
+) -> tuple[dict, dict]:
+    """Run the coupled request then feed its generated tokens into the
+    disaggregated derender endpoint. Returns (coupled, disagg)."""
+    coupled = await _coupled(client, messages, **extra)
+    ch = coupled["choices"][0]
+    chat_request = {"model": MODEL, "messages": messages, **extra}
+    disagg = await _disagg(
+        client,
+        ch["token_ids"],
+        coupled["usage"]["prompt_tokens"],
+        ch["finish_reason"],
+        chat_request,
+    )
+    return coupled, disagg
+
+
+# ---------------------------------------------------------------------------
+# Parity cases
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_parity_plain(client):
+    """Plain detokenization parity. No reasoning/tool markers involved."""
+    messages = [
+        {"role": "user", "content": "What is 2+2? Answer in one short sentence."}
+    ]
+    coupled, disagg = await _run_parity_case(client, messages)
+    _assert_parity(coupled, disagg)
+
+
+@pytest.mark.asyncio
+async def test_parity_reasoning(client):
+    """Reasoning/content split parity for <think>...</think> outputs."""
+    messages = [{"role": "user", "content": "What is 17 times 23? Think it through."}]
+    coupled, disagg = await _run_parity_case(
+        client, messages, include_reasoning=True, max_tokens=256
+    )
+    _assert_parity(coupled, disagg)
+
+    if not coupled["choices"][0]["message"].get("reasoning"):
+        pytest.skip("Model did not emit a <think> block")
+    assert disagg["choices"][0]["message"]["reasoning"]
+
+
+@pytest.mark.asyncio
+async def test_parity_tool_call(client):
+    """Tool call name+args parity."""
+    messages = [{"role": "user", "content": "What's the weather in Paris?"}]
+    coupled, disagg = await _run_parity_case(
+        client, messages, tools=TOOLS, tool_choice=FORCE_WEATHER_TOOL, max_tokens=1024
+    )
+    _assert_parity(coupled, disagg)
+
+    if not _tool_sig(coupled["choices"][0]):
+        pytest.skip("Model did not emit a tool call")
+    assert _tool_sig(disagg["choices"][0])
+
+
+@pytest.mark.asyncio
+async def test_parity_reasoning_and_tool_call(client):
+    """Combined reasoning + tool call parity means the highest drift risk
+    since it exercises both parser branches on the same output."""
+    messages = [{"role": "user", "content": "What's the weather in Paris?"}]
+    coupled, disagg = await _run_parity_case(
+        client,
+        messages,
+        tools=TOOLS,
+        tool_choice=FORCE_WEATHER_TOOL,
+        include_reasoning=True,
+        max_tokens=1024,
+    )
+    _assert_parity(coupled, disagg)
+
+    c_msg = coupled["choices"][0]["message"]
+    if not (c_msg.get("reasoning") and _tool_sig(coupled["choices"][0])):
+        pytest.skip("Model did not emit both a <think> block and a tool call")
+    d_msg = disagg["choices"][0]["message"]
+    assert d_msg["reasoning"]
+    assert _tool_sig(disagg["choices"][0])
+
+
+@pytest.mark.asyncio
+async def test_parity_logprobs(client):
+    """token_id:N resolution parity vs. the coupled server's real strings.
+
+    A real disaggregated worker only has token IDs so it emits logprobs
+    with ``token_id:N`` placeholders (``return_tokens_as_token_ids=True``
+    reproduces that shape here). ``/derender`` must resolve those
+    placeholders to the same token strings/bytes the coupled server
+    resolves them to directly.
+    """
+    messages = [{"role": "user", "content": "What is 2+2?"}]
+    extra = {"logprobs": True, "top_logprobs": 3}
+
+    # What a real GPU less worker would hand to /derender is token IDs plus
+    # logprobs still in token_id:N placeholder form
+    placeholder = await _coupled(
+        client, messages, return_tokens_as_token_ids=True, **extra
+    )
+    ch = placeholder["choices"][0]
+    chat_request = {"model": MODEL, "messages": messages, **extra}
+    disagg = await _disagg(
+        client,
+        ch["token_ids"],
+        placeholder["usage"]["prompt_tokens"],
+        ch["finish_reason"],
+        chat_request,
+        logprobs=ch["logprobs"],
+    )
+
+    # The coupled server resolving the same greedy generation
+    # to real token strings itself.
+    resolved = await _coupled(client, messages, **extra)
+    assert resolved["choices"][0]["token_ids"] == ch["token_ids"], (
+        "greedy (temperature=0) generation was expected to be deterministic "
+        "across the two coupled calls used to build this test's fixtures"
+    )
+    _assert_parity(resolved, disagg)
+
+    r_content = resolved["choices"][0]["logprobs"]["content"]
+    d_content = disagg["choices"][0]["logprobs"]["content"]
+    assert len(d_content) == len(r_content)
+    for d_entry, r_entry in zip(d_content, r_content):
+        assert d_entry["token"] == r_entry["token"]
+        assert d_entry["bytes"] == r_entry["bytes"]

vllm/entrypoints/scale_out/token_in_token_out/protocol.py

@@ -266,8 +266,13 @@ class DerenderChatRequest(BaseModel):
     and ``parser.parse_delta()`` instead of ``parser.parse()``.
     """
 
+    # --8<-- [start:derender-chat-request]
     model: str
+    """Served model name."""
+
     generate_response: GenerateResponse
+    """The complete token-in / token-out engine response to derender."""
+
     prompt_tokens: int | None = None
     """Prompt token count for usage; defaults to 0 if omitted.
 
@@ -282,6 +287,7 @@ class DerenderChatRequest(BaseModel):
     request context they expect (request.tools, request.tool_choice,
     request._grammar_from_tool_parser, etc.).
     """
+    # --8<-- [end:derender-chat-request]
 
 
 class DerenderCompletionRequest(BaseModel):
@@ -292,8 +298,14 @@ class DerenderCompletionRequest(BaseModel):
     returned by /v1/completions/render.
     """
 
+    # --8<-- [start:derender-completion-request]
     model: str
+    """Served model name."""
+
     generate_responses: list[GenerateResponse]
+    """One response per prompt, parallel to the list[GenerateRequest]
+    returned by /v1/completions/render."""
+
     prompt_tokens: list[int] | None = None
     """One prompt token count per response; each defaults to 0 if omitted.
 
@@ -306,6 +318,7 @@ class DerenderCompletionRequest(BaseModel):
     Mirrors chat_request on DerenderChatRequest. Required by the parsing
     so parsers receive the full request context.
     """
+    # --8<-- [end:derender-completion-request]
 
     @model_validator(mode="after")
     def _validate_prompt_tokens_length(self) -> "DerenderCompletionRequest":

vllm/renderers/online_derenderer.py

@@ -7,6 +7,7 @@ from vllm.entrypoints.chat_utils import ChatTemplateContentFormatOption
 from vllm.entrypoints.generate.base.serving import resolve_token_id_placeholder
 from vllm.entrypoints.openai.chat_completion.protocol import (
     ChatCompletionLogProbs,
+    ChatCompletionNamedToolChoiceParam,
     ChatCompletionRequest,
     ChatCompletionResponseChoice,
     ChatMessage,
@@ -136,6 +137,13 @@ class OnlineDerenderer:
                     else []
                 )
 
+                is_named_tool_choice = (
+                    type(chat_request.tool_choice) is ChatCompletionNamedToolChoiceParam
+                )
+                is_required_tool_choice = chat_request.tool_choice == "required"
+                if is_named_tool_choice or is_required_tool_choice:
+                    content = content or ""
+
                 message = ChatMessage(
                     role="assistant",
                     reasoning=reasoning,