diff --git a/docs/models/supported_models.md b/docs/models/supported_models.md index f0eb3781fba..746980b8fe1 100644 --- a/docs/models/supported_models.md +++ b/docs/models/supported_models.md @@ -682,6 +682,24 @@ Speech2Text models trained specifically for Automatic Speech Recognition. !!! note `VoxtralForConditionalGeneration` requires `mistral-common[audio]` to be installed. +#### Realtime Transcription + +Speech models that support streaming transcription via the +[`/v1/realtime`](../serving/openai_compatible_server.md#realtime-api) +WebSocket endpoint. + +| Architecture | Models | Example HF Models | [LoRA](../features/lora.md) | [PP](../serving/parallelism_scaling.md) | +| ------------ | ------ | ----------------- | -------------------- | ------------------------- | +| `VoxtralRealtimeGeneration` | Voxtral Realtime | `mistralai/Voxtral-Mini-4B-Realtime-2602` | | | +| `Qwen3ASRRealtimeGeneration` | Qwen3-ASR Realtime | `Qwen/Qwen3-ASR-0.6B` | | | + +!!! note + `VoxtralRealtimeGeneration` requires `mistral-common[audio]` to be installed, and must be served with `--tokenizer-mode mistral`. + + `Qwen3ASRRealtimeGeneration` is not auto-detected from `config.json`. + You must pass `--hf-overrides '{"architectures":["Qwen3ASRRealtimeGeneration"]}'` + when serving. + ## Pooling Models See [this page](pooling_models/README.md) for more information on how to use pooling models. diff --git a/docs/serving/openai_compatible_server.md b/docs/serving/openai_compatible_server.md index cde7d597d82..a2c90e3abd4 100644 --- a/docs/serving/openai_compatible_server.md +++ b/docs/serving/openai_compatible_server.md @@ -60,7 +60,7 @@ We currently support the following OpenAI APIs: - [Translation API](#translations-api) (`/v1/audio/translations`) - Only applicable to [Automatic Speech Recognition (ASR) models](../models/supported_models.md#transcription). - [Realtime API](#realtime-api) (`/v1/realtime`) - - Only applicable to [Automatic Speech Recognition (ASR) models](../models/supported_models.md#transcription). + - Only applicable to [Automatic Speech Recognition (ASR) models](../models/supported_models.md#realtime-transcription). In addition, we have the following custom APIs: