[Frontend] Add endpoint plugins framework (#47454)

Signed-off-by: Martin Hickey <[email protected]>

.buildkite/test-amd.yaml

@@ -1938,6 +1938,11 @@ steps:
   - pytest -v -s plugins_tests/test_stats_logger_plugins.py
   - pip uninstall dummy_stat_logger -y
   # END: `stat_logger` plugins test
+  # BEGIN: `endpoint` plugins test
+  - pip install -e ./plugins/vllm_add_dummy_endpoint_plugin
+  - pytest -v -s plugins_tests/test_endpoint_plugins.py
+  - pip uninstall vllm_add_dummy_endpoint_plugin -y
+  # END: `endpoint` plugins test
   # BEGIN: other tests
   - pytest -v -s plugins_tests/test_scheduler_plugins.py
   - pip install -e ./plugins/vllm_add_dummy_model

.buildkite/test_areas/plugins.yaml

@@ -37,6 +37,11 @@ steps:
   - pytest -v -s plugins_tests/test_stats_logger_plugins.py
   - pip uninstall dummy_stat_logger -y
   # end stat_logger plugins test
+  # begin endpoint plugins test
+  - pip install -e ./plugins/vllm_add_dummy_endpoint_plugin
+  - pytest -v -s plugins_tests/test_endpoint_plugins.py
+  - pip uninstall vllm_add_dummy_endpoint_plugin -y
+  # end endpoint plugins test
   # other tests continue here:
   - pytest -v -s plugins_tests/test_scheduler_plugins.py
   - pip install -e ./plugins/vllm_add_dummy_model

docs/design/endpoint_plugins.md

@@ -0,0 +1,136 @@
+# Endpoint Plugins
+
+Endpoint plugins let out-of-tree packages add HTTP routes to the OpenAI compatible API server without editing `vllm/entrypoints/openai/api_server.py`. Their scope is
+the **HTTP surface only** registering routes and optionally per app state used by those routes. A plugin reaches the engine the same way an in-tree serving handler does, through the `EngineClient` it is handed at startup (e.g. `engine_client.collective_rpc(...)`). No new engine access path is introduced.
+
+!!! warning "Security"
+    Endpoint plugins are **not loaded by default** and must be explicitly allowlisted. Read [Endpoint Plugins security posture](../usage/security.md#endpoint-plugins) before enabling one, especially the route shadowing warning.
+
+## The `EndpointPlugin` protocol
+
+Endpoint plugins implement the [`EndpointPlugin`][vllm.plugins.endpoint_plugins.interface.EndpointPlugin] runtime checkable `Protocol`:
+
+```python
+class EndpointPlugin(Protocol):
+    name: str
+    required_tasks: tuple[SupportedTask, ...] | None
+
+    def attach_router(self, app: FastAPI) -> None: ...
+
+    async def init_state(
+        self, engine_client: EngineClient | None, state: State, args: Namespace
+    ) -> None: ...
+```
+
+- `name`: a unique identifier used in logs and for `VLLM_PLUGINS` allowlisting
+- `required_tasks`: the tasks the server must support for this plugin to load. `None` means the plugin has no task requirement
+- `attach_router`: registers routes on `app`
+- `init_state`: initializes per app state the routes read at request time
+
+## The two phase lifecycle
+
+Routes are registered before the engine exists. This means the interface has to expose two hooks that run at two different points in server startup:
+
+| Phase | Called from | `engine_client` available? | Work |
+| --- | --- | --- | --- |
+| A. Route registration | `build_app()` | No | `attach_router(app)` add routes. Do not touch the engine here. |
+| B. State init | `init_app_state()` | Usually but `None` on the CPU only render server | `init_state(engine_client, state, args)` build a serving handler holding `engine_client` and store it on `state`. |
+
+Because `app.state` *is* the `state` object passed to `init_app_state()`, an object stored during phase A is visible in phase B and an object stored in phase B is visible to route handlers at request time via `request.app.state`. This is the same pattern in-tree endpoints already use.
+
+### Engine less servers (the render server)
+
+The CPU only render server (`init_render_app_state()`) has no `EngineClient`. It still runs both phases for any plugin eligible for the `render` task (`required_tasks` is `None` or includes `"render"`). `attach_router` is called as usual but `init_state` is called with `engine_client=None`.
+
+A plugin that needs an engine to function has two options:
+
+- Exclude `"render"` from `required_tasks` so it is never loaded on the render server in the first place
+- Accept being loaded on `render` and check for `None` in `init_state` or in the route handler returning an error response (e.g. HTTP 503) instead of dereferencing a client that doesn't exist
+
+`tests/plugins/vllm_add_dummy_endpoint_plugin` demonstrates the second option. Its route handler returns a 503 when `state.dummy_engine_client` is `None`.
+
+### Reaching the engine from a route handler
+
+`init_state` is where a plugin captures `engine_client` into a small serving handler and stashes it on `state`. The route added in `attach_router` reads that handler off `request.app.state` at request time and calls the engine through it, typically via `engine_client.collective_rpc(...)`.
+
+This minimal example omits the `None` check from the previous section for brevity since `required_tasks` is `None` here. It is in fact eligible for `render` and should handle `engine_client=None` the way `tests/plugins/vllm_add_dummy_endpoint_plugin` does before shipping it:
+
+```python
+from fastapi import FastAPI, Request
+
+
+class MyAdminEndpointPlugin:
+    name = "my_admin_endpoint_plugin"
+    required_tasks: tuple[str, ...] | None = None
+
+    def attach_router(self, app: FastAPI) -> None:
+        @app.get("/plugins/my_admin_endpoint_plugin/scheduler_config")
+        async def scheduler_config(raw_request: Request):
+            engine_client = raw_request.app.state.my_engine_client
+            results = await engine_client.collective_rpc("get_scheduler_config")
+            return {"scheduler_config": results}
+
+    async def init_state(self, engine_client, state, args) -> None:
+        state.my_engine_client = engine_client
+```
+
+A complete and tested version of this example is in-repo as `tests/plugins/vllm_add_dummy_endpoint_plugin` and is exercised e2e (including a real HTTP request) in `tests/plugins_tests/test_endpoint_plugins.py`.
+
+## Registering the entry point
+
+Register a zero argument factory (a class or function) under the `vllm.endpoint_plugins` group. The factory must return an object satisfying `EndpointPlugin`:
+
+```toml
+# pyproject.toml
+[project.entry-points."vllm.endpoint_plugins"]
+my_admin_api = "my_pkg.endpoints:MyAdminEndpointPlugin"
+```
+
+```python
+# setup.py equivalent
+setup(
+    name="my_pkg",
+    entry_points={
+        "vllm.endpoint_plugins": [
+            "my_admin_api = my_pkg.endpoints:MyAdminEndpointPlugin"
+        ]
+    },
+)
+```
+
+The entry point name (`my_admin_api` above) is independent of the plugin's `name` attribute. `VLLM_PLUGINS` allowlisting matches on the **entry point name** following the same convention as `vllm.general_plugins` (see [Plugin System](plugin_system.md)).
+
+## Gating: `VLLM_PLUGINS` and `required_tasks`
+
+Endpoint plugins are discovered and gated by [`load_endpoint_plugins`][vllm.plugins.load_endpoint_plugins] which is stricter than the loader used for other plugin groups:
+
+- **Nothing loads unless `VLLM_PLUGINS` is set and names the plugin.** Other plugin groups load everything unless `VLLM_PLUGINS` narrows the set. Endpoint plugins invert that default because they add network exposed surface. See [Security](../usage/security.md#endpoint-plugins).
+- **`required_tasks` must intersect the server's supported tasks** unless it is `None`. Use this to keep a plugin from attaching routes on a server that can't service them (e.g. a pooling only deployment).
+- A factory that raises an issue during instantiation is logged and skipped. It does not abort server startup.
+
+Only the front end API server process loads endpoint plugins. There is no need to guard for worker or engine core processes.
+
+## Pairing with `vllm.general_plugins`
+
+Endpoint plugins cover the HTTP surface only. If a plugin also needs new engine side behavior (a new worker-side RPC method, a custom stat) that half ships separately through the existing `vllm.general_plugins` group which loads in worker processes (see [Plugin System](plugin_system.md)). The two entry points are registered and loaded **independently**. Neither implies the other. The recommended distribution shape is a single package exposing both:
+
+```toml
+[project.entry-points."vllm.general_plugins"]
+my_admin_engine = "my_pkg.engine:register"      # adds the worker side method
+
+[project.entry-points."vllm.endpoint_plugins"]
+my_admin_api = "my_pkg.endpoints:MyAdminEndpointPlugin"  # adds the HTTP route
+```
+
+Do not expect a single endpoint plugin to also mutate engine/worker state. If your route needs a worker side method that doesn't already exist then add it via a paired `general_plugins` entry point.
+
+## Path-prefix convention
+
+There is currently no route conflict enforcement (tracked as a follow-up to RFC [#46565](https://github.com/vllm-project/vllm/issues/46565)). A plugin's `attach_router` can register a path that collides with a core route and routes attached later win. To avoid surprising operators:
+
+- Namespace your routes under a distinct prefix, e.g. `/plugins/<plugin-name>/...`, rather than reusing `/v1/...` or other core prefixes
+- Only register routes under a core prefix (like the worked example's `/v1/admin/scheduler_config`) if you specifically intend to override or extend existing behavior and document that clearly for operators allowlisting your plugin
+
+## Compatibility
+
+`state`/serving handler internals (e.g. the shape of in-tree `OpenAIServing*` classes) are not a stable public contract yet. Treat them as use-at-your-own-risk and expect them to change between vLLM versions. `FastAPI`, `EngineClient` and the `EndpointPlugin` protocol itself are the supported surface.

docs/design/plugin_system.md

@@ -53,6 +53,8 @@ Every plugin has three parts:
 
 - **Stat logger plugins** (with group name `vllm.stat_logger_plugins`): The primary use case for these plugins is to register custom, out-of-the-tree loggers into vLLM. The entry point should be a class that subclasses StatLoggerBase.
 
+- **Endpoint plugins** (with group name `vllm.endpoint_plugins`): The primary use case for these plugins is to register custom, out-of-the-tree HTTP routes on the OpenAI compatible API server. Unlike the other plugin groups above, endpoint plugins are loaded only in the API server front end process and are **not loaded by default**. See [Endpoint Plugins](endpoint_plugins.md) for the interface and [Security](../usage/security.md#endpoint-plugins) for the opt-in and trust model.
+
 ## Guidelines for Writing Plugins
 
 - **Being re-entrant**: The function specified in the entry point should be re-entrant, meaning it can be called multiple times without causing issues. This is necessary because the function might be called multiple times in some processes.

docs/usage/security.md

@@ -326,6 +326,19 @@ vLLM supports dynamically loading and unloading LoRA adapters at runtime via the
 
 **Warning:** Dynamic LoRA loading is not a secure operation and should not be enabled in deployments exposed to untrusted clients. If you must enable dynamic LoRA loading, restrict access to the `/v1/load_lora_adapter` and `/v1/unload_lora_adapter` endpoints to trusted administrators only, using a reverse proxy or network-level access controls. Do not expose these endpoints to end users. For details on configuring LoRA adapters, see the [LoRA Adapters documentation](../features/lora.md).
 
+## Endpoint Plugins
+
+vLLM supports loading out-of-tree HTTP routes via the `vllm.endpoint_plugins` entry point group (see [Endpoint Plugins](../design/endpoint_plugins.md) for how to write one). An endpoint plugin can register arbitrary FastAPI routes, including routes that reach the engine via `EngineClient.collective_rpc`, so it must be treated as part of the server's trusted code base and not as sandboxed or reviewed input.
+
+**Endpoint plugins are not loaded by default.** Unlike other vLLM plugin groups (`vllm.general_plugins`, `vllm.platform_plugins`, etc.), which load every discovered plugin unless `VLLM_PLUGINS` narrows the set, endpoint plugins load **none** unless `VLLM_PLUGINS` is set and explicitly names them. This mirrors the "off by default in production" posture used for development endpoints gated behind `VLLM_SERVER_DEV_MODE`. Both surfaces are only present when an operator has explicitly opted in.
+
+### Recommended Security Practices
+
+1. **Only allowlist plugins you trust.** Set `VLLM_PLUGINS` to the exact plugin names you intend to run and never wildcard or copy an allowlist between deployments without reviewing what each named plugin does.
+2. **Audit routes before deploying.** A plugin's `attach_router` can add routes under any path, including ones that duplicate existing `/v1/*` paths. There is currently no route conflict enforcement (tracked as a follow-up to RFC [#46565](https://github.com/vllm-project/vllm/issues/46565)), so a malicious or buggy plugin can **shadow a core route** and silently replace its behavior. Prefer plugins that namespace their routes under a distinct prefix (e.g. `/plugins/<plugin-name>/...`) instead of reusing `/v1/...` and review `app.routes` after startup if you need certainty about what is actually being served.
+3. **Treat plugin routes like any other unauthenticated by default surface.** `--api-key` only protects the `/v1`, `/v2`, and `/inference` path prefixes (see [API Key Authentication Limitations](#api-key-authentication-limitations)). A plugin route outside those prefixes is unauthenticated unless the plugin implements its own authentication. Deploy behind a reverse proxy that allowlists only the plugin routes you intend to expose externally.
+4. **Remember the `vllm.general_plugins` pairing.** A plugin that also needs new engine side behavior ships that half separately via `vllm.general_plugins` which loads in every worker process under the default (load all unless restricted) posture. Allowlisting the endpoint plugin does not by itself restrict its paired engine side plugin. Need to review both.
+
 ## gRPC Interface
 
 vLLM provides an optional gRPC Generate service on a separate TCP port, enabled via the `--grpc-port` flag. When not specified, no gRPC server is started. The gRPC listener binds to the same host address as the HTTP server.

tests/plugins/vllm_add_dummy_endpoint_plugin/setup.py

@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+
+from setuptools import setup
+
+setup(
+    name="vllm_add_dummy_endpoint_plugin",
+    version="0.1",
+    packages=["vllm_add_dummy_endpoint_plugin"],
+    entry_points={
+        "vllm.endpoint_plugins": [
+            "dummy_admin_endpoint_plugin = vllm_add_dummy_endpoint_plugin:DummyAdminEndpointPlugin"  # noqa
+        ]
+    },
+)

tests/plugins/vllm_add_dummy_endpoint_plugin/vllm_add_dummy_endpoint_plugin/__init__.py

@@ -0,0 +1,37 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Worked example `vllm.endpoint_plugins` entry point.
+
+Reports scheduler config via `collective_rpc`. Demonstrates the full
+contract: `attach_router` registers the route at Phase A (`build_app`) and
+`init_state` stashes the `EngineClient` the route handler needs at Phase B
+(`init_app_state`).
+
+`required_tasks` is `None`, so this plugin is also eligible on the CPU only
+render server which has no `EngineClient`. `init_state` is called with
+`engine_client=None` in that case and the route handler returns 503 rather
+than reaching for a client that doesn't exist.
+"""
+
+from fastapi import FastAPI, HTTPException, Request
+
+
+class DummyAdminEndpointPlugin:
+    name = "dummy_admin_endpoint_plugin"
+    required_tasks: tuple[str, ...] | None = None
+
+    def attach_router(self, app: FastAPI) -> None:
+        @app.get("/v1/admin/scheduler_config")
+        async def scheduler_config(raw_request: Request):
+            engine_client = raw_request.app.state.dummy_engine_client
+            if engine_client is None:
+                raise HTTPException(
+                    status_code=503,
+                    detail="scheduler_config requires an engine, which this "
+                    "server does not have",
+                )
+            results = await engine_client.collective_rpc("get_scheduler_config")
+            return {"scheduler_config": results}
+
+    async def init_state(self, engine_client, state, args) -> None:
+        state.dummy_engine_client = engine_client

tests/plugins_tests/test_endpoint_plugins.py

@@ -0,0 +1,236 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Tests for the `vllm.endpoint_plugins` framework (RFC #46565).
+
+Uses the worked in repo example plugin (`vllm_add_dummy_endpoint_plugin`,
+installed via `tests/plugins/vllm_add_dummy_endpoint_plugin`) exercising both
+`EndpointPlugin` hooks against a fake `EngineClient`, unit tests for the
+`load_endpoint_plugins` gating matrix and an e2e test that drives a real HTTP
+request through the plugin's route.
+"""
+
+from argparse import Namespace
+from typing import Any
+
+import httpx
+import pytest
+from fastapi import FastAPI
+from vllm_add_dummy_endpoint_plugin import DummyAdminEndpointPlugin
+
+from vllm.entrypoints.openai.api_server import (
+    _attach_endpoint_plugins,
+    _init_endpoint_plugins_state,
+    build_app,
+)
+from vllm.entrypoints.openai.cli_args import make_arg_parser
+from vllm.plugins import load_endpoint_plugins
+from vllm.plugins.endpoint_plugins.interface import EndpointPlugin
+from vllm.utils.argparse_utils import FlexibleArgumentParser
+
+
+class _RaisingEndpointPlugin:
+    """Factory that raises to exercise the "instantiation fails" path."""
+
+    name = "raising_endpoint_plugin"
+    required_tasks = None
+
+    def __init__(self):
+        raise RuntimeError("boom")
+
+
+class _FakeEngineClient:
+    """Minimal stand in exercising `collective_rpc`. Not a real engine."""
+
+    def __init__(self, rpc_result: Any = None):
+        self.rpc_result = rpc_result
+        self.rpc_calls: list[tuple[str, tuple, dict]] = []
+
+    async def collective_rpc(self, method, timeout=None, args=(), kwargs=None):
+        self.rpc_calls.append((method, args, kwargs or {}))
+        return self.rpc_result
+
+
+def _build_args() -> Namespace:
+    parser = FlexibleArgumentParser()
+    subparsers = parser.add_subparsers()
+    serve_parser = subparsers.add_parser("serve")
+    make_arg_parser(serve_parser)
+    return serve_parser.parse_args([])
+
+
+def _fake_loader(factories: dict[str, Any]):
+    def _load_plugins_by_group(group: str) -> dict[str, Any]:
+        assert group == "vllm.endpoint_plugins"
+        return factories
+
+    return _load_plugins_by_group
+
+
+def test_dummy_plugin_satisfies_protocol():
+    assert isinstance(DummyAdminEndpointPlugin(), EndpointPlugin)
+
+
+def test_no_plugins_loaded_when_allowlist_unset(monkeypatch: pytest.MonkeyPatch):
+    monkeypatch.delenv("VLLM_PLUGINS", raising=False)
+
+    assert load_endpoint_plugins(("generate",)) == []
+
+
+def test_no_plugins_loaded_when_allowlist_is_empty_string(
+    monkeypatch: pytest.MonkeyPatch,
+):
+    """`VLLM_PLUGINS=""` parses to `[""]`, not `None` (see `vllm.envs`), so it
+    must be treated as a (non strict) allowlist matching no plugin name, not
+    as "unset"."""
+    monkeypatch.setenv("VLLM_PLUGINS", "")
+
+    assert load_endpoint_plugins(("generate",)) == []
+
+
+def test_plugin_loaded_when_allowlisted_and_task_matches(
+    monkeypatch: pytest.MonkeyPatch,
+):
+    monkeypatch.setenv("VLLM_PLUGINS", "dummy_admin_endpoint_plugin")
+
+    plugins = load_endpoint_plugins(("generate",))
+
+    assert len(plugins) == 1
+    assert isinstance(plugins[0], DummyAdminEndpointPlugin)
+
+
+def test_plugin_skipped_when_required_tasks_miss(monkeypatch: pytest.MonkeyPatch):
+    class _GenerateOnlyPlugin(DummyAdminEndpointPlugin):
+        required_tasks = ("generate",)
+
+    monkeypatch.setenv("VLLM_PLUGINS", "dummy_admin_endpoint_plugin")
+    monkeypatch.setattr(
+        "vllm.plugins.load_plugins_by_group",
+        _fake_loader({"dummy_admin_endpoint_plugin": _GenerateOnlyPlugin}),
+    )
+
+    assert load_endpoint_plugins(("embed",)) == []
+    assert len(load_endpoint_plugins(("generate",))) == 1
+
+
+def test_plugin_loaded_when_required_tasks_is_none(monkeypatch: pytest.MonkeyPatch):
+    monkeypatch.setenv("VLLM_PLUGINS", "dummy_admin_endpoint_plugin")
+
+    assert len(load_endpoint_plugins(supported_tasks=None)) == 1
+
+
+def test_plugin_skipped_when_required_tasks_set_but_supported_tasks_none(
+    monkeypatch: pytest.MonkeyPatch,
+):
+    class _GenerateOnlyPlugin(DummyAdminEndpointPlugin):
+        required_tasks = ("generate",)
+
+    monkeypatch.setenv("VLLM_PLUGINS", "dummy_admin_endpoint_plugin")
+    monkeypatch.setattr(
+        "vllm.plugins.load_plugins_by_group",
+        _fake_loader({"dummy_admin_endpoint_plugin": _GenerateOnlyPlugin}),
+    )
+
+    assert load_endpoint_plugins(supported_tasks=None) == []
+
+
+def test_factory_raising_is_logged_and_skipped(monkeypatch: pytest.MonkeyPatch):
+    monkeypatch.setenv(
+        "VLLM_PLUGINS", "raising_endpoint_plugin,dummy_admin_endpoint_plugin"
+    )
+    monkeypatch.setattr(
+        "vllm.plugins.load_plugins_by_group",
+        _fake_loader(
+            {
+                "raising_endpoint_plugin": _RaisingEndpointPlugin,
+                "dummy_admin_endpoint_plugin": DummyAdminEndpointPlugin,
+            }
+        ),
+    )
+
+    plugins = load_endpoint_plugins(("generate",))
+
+    assert len(plugins) == 1
+    assert isinstance(plugins[0], DummyAdminEndpointPlugin)
+
+
+def test_attach_is_noop_when_nothing_discovered(monkeypatch: pytest.MonkeyPatch):
+    monkeypatch.delenv("VLLM_PLUGINS", raising=False)
+
+    app = FastAPI()
+    _attach_endpoint_plugins(app, ("generate",))
+
+    assert app.state.endpoint_plugins == []
+
+
+@pytest.mark.asyncio
+async def test_init_state_is_noop_without_phase_a(monkeypatch: pytest.MonkeyPatch):
+    """`init_app_state` callers that never ran `build_app` (e.g.
+    `run_batch.py`, which builds a bare `State()`) must not crash just
+    because `state.endpoint_plugins` was never set."""
+    from starlette.datastructures import State
+
+    monkeypatch.setenv("VLLM_PLUGINS", "dummy_admin_endpoint_plugin")
+
+    state = State()
+    await _init_endpoint_plugins_state(_FakeEngineClient(), state, _build_args())
+
+    assert not hasattr(state, "dummy_engine_client")
+
+
+@pytest.mark.asyncio
+async def test_render_server_attaches_endpoint_plugins_with_no_engine_client(
+    monkeypatch: pytest.MonkeyPatch,
+):
+    """The CPU only render server has no `EngineClient` but a plugin eligible
+    for the `render` task (`required_tasks` is `None` or includes `"render"`)
+    still gets its routes attached at Phase A. Phase B passes `None` for
+    `engine_client` and it's up to the plugin to handle that."""
+    monkeypatch.setenv("VLLM_PLUGINS", "dummy_admin_endpoint_plugin")
+
+    args = _build_args()
+    app = build_app(args, ("render",))
+
+    assert len(app.state.endpoint_plugins) == 1
+    assert any(
+        getattr(route, "path", None) == "/v1/admin/scheduler_config"
+        for route in app.routes
+    )
+
+    await _init_endpoint_plugins_state(None, app.state, args)
+
+    assert app.state.dummy_engine_client is None
+
+    transport = httpx.ASGITransport(app=app)
+    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
+        response = await client.get("/v1/admin/scheduler_config")
+
+    assert response.status_code == 503
+
+
+@pytest.mark.asyncio
+async def test_endpoint_plugin_end_to_end(monkeypatch: pytest.MonkeyPatch):
+    """Phase A (attach) + Phase B (init) wired through `build_app` then
+    exercised with a real HTTP request against the worked example plugin."""
+    monkeypatch.setenv("VLLM_PLUGINS", "dummy_admin_endpoint_plugin")
+
+    args = _build_args()
+    app = build_app(args, supported_tasks=())
+
+    assert len(app.state.endpoint_plugins) == 1
+    assert any(
+        getattr(route, "path", None) == "/v1/admin/scheduler_config"
+        for route in app.routes
+    )
+
+    fake_engine_client = _FakeEngineClient(rpc_result=["cfg-a", "cfg-b"])
+    await _init_endpoint_plugins_state(fake_engine_client, app.state, args)
+
+    assert app.state.dummy_engine_client is fake_engine_client
+
+    transport = httpx.ASGITransport(app=app)
+    async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
+        response = await client.get("/v1/admin/scheduler_config")
+
+    assert response.status_code == 200
+    assert response.json() == {"scheduler_config": ["cfg-a", "cfg-b"]}
+    assert fake_engine_client.rpc_calls == [("get_scheduler_config", (), {})]

vllm/entrypoints/openai/api_server.py

@@ -74,6 +74,41 @@ logger = init_logger("vllm.entrypoints.openai.api_server")
 _FALLBACK_SUPPORTED_TASKS: tuple[SupportedTask, ...] = ("generate",)
 
 
+def _attach_endpoint_plugins(
+    app: FastAPI, supported_tasks: tuple["SupportedTask", ...]
+) -> None:
+    """Phase A of endpoint plugin wiring: discover, gate and attach routes.
+
+    Attached last after all core routers. This is so endpoint plugin routes can
+    shadow core routes with the same path (see `EndpointPlugin.attach_router`
+    docstring). No-ops when no plugins are discovered/allowlisted.
+    """
+    from vllm.plugins import load_endpoint_plugins
+
+    endpoint_plugins = load_endpoint_plugins(supported_tasks)
+    for plugin in endpoint_plugins:
+        plugin.attach_router(app)
+    app.state.endpoint_plugins = endpoint_plugins
+
+
+async def _init_endpoint_plugins_state(
+    engine_client: EngineClient | None, state: State, args: Namespace
+) -> None:
+    """Phase B of endpoint plugin wiring: initialize per app plugin state.
+
+    `state.endpoint_plugins` is set by `_attach_endpoint_plugins` (Phase A)
+    in `build_app`. Some `init_app_state` callers (e.g. `run_batch.py`)
+    build their own bare `State` without going through `build_app`. As a result
+    `endpoint_plugins` may be absent and are treated that the same as "none attached".
+
+    `engine_client` is `None` for the CPU only render server which has no
+    engine (see `init_render_app_state`). Plugins must handle a `None`
+    `engine_client` themselves (see `EndpointPlugin.init_state`).
+    """
+    for plugin in getattr(state, "endpoint_plugins", []):
+        await plugin.init_state(engine_client, state, args)
+
+
 @asynccontextmanager
 async def build_async_engine_client(
     args: Namespace,
@@ -230,6 +265,12 @@ def build_app(
 
         register_pooling_api_routers(app, supported_tasks, model_config)
 
+    # Endpoint plugins are attached last so their routes are registered after all core
+    # routers. This runs even for the CPU only render server. A plugin eligible for
+    # the `render` task still gets its routes registered. It receives
+    # `engine_client=None` at Phase B (see `_init_endpoint_plugins_state`).
+    _attach_endpoint_plugins(app, supported_tasks)
+
     app.root_path = args.root_path
     app.add_middleware(
         CORSMiddleware,
@@ -417,6 +458,8 @@ async def init_app_state(
 
         init_pooling_state(engine_client, state, args, request_logger, supported_tasks)
 
+    await _init_endpoint_plugins_state(engine_client, state, args)
+
     state.enable_server_load_tracking = args.enable_server_load_tracking
     state.server_load_metrics = 0
 
@@ -508,6 +551,10 @@ async def init_render_app_state(
     state.enable_server_load_tracking = False
     state.server_load_metrics = 0
 
+    # No `EngineClient` exists for the render server, so plugins get `None` and
+    # must handle it themselves (see `EndpointPlugin.init_state`).
+    await _init_endpoint_plugins_state(None, state, args)
+
 
 def create_server_socket(
     addr: tuple[str, int],

vllm/plugins/__init__.py

@@ -3,10 +3,14 @@
 
 import logging
 from collections.abc import Callable
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
 import vllm.envs as envs
 
+if TYPE_CHECKING:
+    from vllm.plugins.endpoint_plugins.interface import EndpointPlugin
+    from vllm.tasks import SupportedTask
+
 logger = logging.getLogger(__name__)
 
 # Default plugins group will be loaded in all processes(process0, engine core
@@ -20,6 +24,10 @@ PLATFORM_PLUGINS_GROUP = "vllm.platform_plugins"
 # Stat logger plugins group will be loaded in process0 only when serve vLLM with
 # async mode.
 STAT_LOGGER_PLUGINS_GROUP = "vllm.stat_logger_plugins"
+# Endpoint plugins group is loaded in the API server front end process only.
+# Each entry point resolves to a factory returning an `EndpointPlugin`
+# (see `vllm/plugins/endpoint_plugins/interface.py`).
+ENDPOINT_PLUGINS_GROUP = "vllm.endpoint_plugins"
 
 # make sure one process only loads plugins once
 plugins_loaded = False
@@ -80,3 +88,71 @@ def load_general_plugins():
     # general plugins, we only need to execute the loaded functions
     for func in plugins.values():
         func()
+
+
+def load_endpoint_plugins(
+    supported_tasks: "tuple[SupportedTask, ...] | None" = None,
+) -> "list[EndpointPlugin]":
+    """Discover, gate and instantiate `vllm.endpoint_plugins` entry points.
+
+    Endpoint plugins add HTTP routes to the API server, so they default to
+    not loading. Unlike other plugin groups, a plugin here is only
+    considered when it is explicitly named in `VLLM_PLUGINS`. This is a
+    stricter posture than `load_plugins_by_group` which "load everything unless
+    an allowlist says otherwise". This posture is taken to handle potentially
+    larger exposed network surface.
+
+    A discovered plugin is loaded only if both hold:
+      - it is named in `VLLM_PLUGINS` (enforced by not calling the loader
+        at all when `VLLM_PLUGINS` is unset). Note that `VLLM_PLUGINS=""`
+        parses to `[""]`, not `None`, so it is treated as a (non strict)
+        allowlist that matches no plugin name, not as "unset".
+      - its `required_tasks` is `None` or intersects `supported_tasks`.
+
+    Args:
+        supported_tasks: Tasks the server supports. `None` means no plugin
+            with a non `None` `required_tasks` will be loaded.
+
+    Returns:
+        Instantiated plugins that passed gating in discovery order.
+    """
+    from importlib.metadata import entry_points
+
+    if envs.VLLM_PLUGINS is None:
+        discovered = entry_points(group=ENDPOINT_PLUGINS_GROUP)
+        if discovered:
+            logger.warning(
+                "Found endpoint plugin(s) %s but VLLM_PLUGINS is not set. "
+                "Endpoint plugins add HTTP routes and must be explicitly "
+                "allowlisted via VLLM_PLUGINS to be loaded.",
+                [p.name for p in discovered],
+            )
+        return []
+
+    factories = load_plugins_by_group(ENDPOINT_PLUGINS_GROUP)
+
+    endpoint_plugins: list[EndpointPlugin] = []
+    for name, factory in factories.items():
+        try:
+            plugin = factory()
+        except Exception:
+            logger.exception("Failed to instantiate endpoint plugin %s", name)
+            continue
+
+        required_tasks = plugin.required_tasks
+        if required_tasks is not None and (
+            supported_tasks is None or not set(required_tasks) & set(supported_tasks)
+        ):
+            logger.info(
+                "Skipping endpoint plugin %s: requires one of tasks %s, "
+                "server supports %s",
+                name,
+                required_tasks,
+                supported_tasks,
+            )
+            continue
+
+        logger.info("Loaded endpoint plugin %s", name)
+        endpoint_plugins.append(plugin)
+
+    return endpoint_plugins

vllm/plugins/endpoint_plugins/interface.py

@@ -0,0 +1,87 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Contract for `vllm.endpoint_plugins` entry points.
+
+An endpoint plugin adds HTTP routes to the OpenAI compatible API server.
+Its scope is HTTP surface only. It registers routes and optionally
+per app state used by those routes. It must not open new paths into the
+engine by reaching the engine the same way an in-tree serving handler does
+via `EngineClient` (e.g. `engine_client.collective_rpc(...)`).
+
+If a plugin also needs engine side behavior (a new worker side RPC method,
+a custom stat, etc.) pair this entry point with one registered under
+`vllm.general_plugins` (see `vllm/plugins/__init__.py`). The
+`general_plugins` entry installs the engine side method and the
+`endpoint_plugins` entry exposes it over HTTP. The two are registered and
+loaded independently where neither implies the other.
+
+Plugins are opt-in. See `load_endpoint_plugins` in `vllm/plugins/__init__.py`
+for the loading/gating rules and `docs/usage/security.md` for the security
+posture of exposing plugin defined routes.
+
+The CPU only render server (see `build_and_serve_renderer` in
+`vllm/entrypoints/openai/api_server.py`) has no `EngineClient`. A plugin
+eligible for the `render` task (`required_tasks` is `None` or includes
+`"render"`) still gets `attach_router` called but `init_state` receives
+`engine_client=None`. Plugins that cannot function without an engine should
+either exclude `"render"` from `required_tasks` or check for `None` in
+`init_state`/their route handlers and degrade gracefully.
+"""
+
+from argparse import Namespace
+from typing import TYPE_CHECKING, Protocol, runtime_checkable
+
+from fastapi import FastAPI
+from starlette.datastructures import State
+
+if TYPE_CHECKING:
+    from vllm.engine.protocol import EngineClient
+    from vllm.tasks import SupportedTask
+
+
+@runtime_checkable
+class EndpointPlugin(Protocol):
+    """Protocol implemented by `vllm.endpoint_plugins` entry point factories.
+
+    An entry point registered under the `vllm.endpoint_plugins` group must
+    resolve to a zero argument callable (a class or factory function) that
+    returns an object satisfying this protocol.
+    """
+
+    name: str
+    """Unique plugin name used in logs and for `VLLM_PLUGINS` allowlisting."""
+
+    required_tasks: "tuple[SupportedTask, ...] | None"
+    """Tasks the server must support for this plugin to be loaded.
+
+    The plugin is loaded only if this set intersects the server's
+    `supported_tasks`. `None` means the plugin has no task requirement and
+    is always eligible (subject to the `VLLM_PLUGINS` allowlist).
+    """
+
+    def attach_router(self, app: FastAPI) -> None:
+        """Register this plugin's routes on `app`.
+
+        Called once during `build_app()` after all core routers have been
+        attached. Routes attached here can shadow core routes with the same
+        path. There is currently no conflict enforcement (see RFC #46565 follow ups).
+        """
+        ...
+
+    async def init_state(
+        self, engine_client: "EngineClient | None", state: State, args: Namespace
+    ) -> None:
+        """Initialize per app state consumed by this plugin's routes.
+
+        Called once during `init_app_state()` after core state has been
+        initialized. Use `engine_client` (e.g. `collective_rpc`) to reach
+        the engine. Do not open new engine access paths.
+
+        `engine_client` is `None` on the CPU only render server which has
+        no engine. This only happens for plugins eligible for the `render`
+        task (`required_tasks` is `None` or includes `"render"`). Handle
+        `None` explicitly (e.g. skip engine dependent setup, or have route
+        handlers return an error) if the plugin is loadable for `render` but
+        cannot function without an engine.
+        """
+        ...