kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-13 22:18:36 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
49fe089470
TensorRT-LLMs/examples/serve
History
Pengyun Lin c6dc68a28e [None][doc] VDR 1.0 trtllm-serve doc enhancement (#9443) 
Signed-off-by: Pengyun Lin <81065165+LinPoly@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Mike Iovine <miovine@nvidia.com>
2025-12-05 17:50:12 -05:00
..
compatibility [TRTLLM-9242][doc] Add examples showcasing openai compatible APIs (#9520) 2025-12-03 11:47:02 +08:00
curl_chat_client_for_multimodal.sh feat: enhance trtllm serve multimodal (#3757) 2025-05-15 16:16:31 -07:00
curl_chat_client.sh feat: trtllm-serve multimodal support (#3590) 2025-04-19 05:01:28 +08:00
curl_completion_client.sh feat: trtllm-serve multimodal support (#3590) 2025-04-19 05:01:28 +08:00
deepseek_r1_reasoning_parser.sh [None][doc] VDR 1.0 trtllm-serve doc enhancement (#9443) 2025-12-05 17:50:12 -05:00
genai_perf_client_for_multimodal.sh feat: enhance trtllm serve multimodal (#3757) 2025-05-15 16:16:31 -07:00
genai_perf_client.sh [https://nvbugs/5277113][fix]genai-perf API change stress test (#4300) 2025-05-15 14:12:34 +08:00
openai_chat_client_for_multimodal.py [TRTLLM-8737][feat] Support media_io_kwargs on trtllm-serve (#8528) 2025-10-24 12:53:40 -04:00
openai_chat_client.py [TRTLLM-5277] chore: refine llmapi examples for 1.0 (part1) (#5431) 2025-07-01 19:06:41 +08:00
openai_completion_client_for_lora.py [TRTLLM-5831][feat] Add LoRA support for pytorch backend in trtllm-serve (#5376) 2025-06-29 12:46:30 +00:00
openai_completion_client_json_schema.py [None][chore] Enhance trtllm-serve example test (#6604) 2025-08-06 20:30:35 +08:00
openai_completion_client.py [TRTLLM-5277] chore: refine llmapi examples for 1.0 (part1) (#5431) 2025-07-01 19:06:41 +08:00
README.md doc: refactor trtllm-serve examples and doc (#3187) 2025-04-04 11:40:43 +08:00
requirements.txt doc: add genai-perf benchmark & slurm multi-node for trtllm-serve doc (#3407) 2025-04-16 00:11:58 +08:00

README.md

Online Serving Examples with trtllm-serve

We provide a CLI command, trtllm-serve, to launch a FastAPI server compatible with OpenAI APIs, here are some client examples to query the server, you can check the source code here or refer to the command documentation and examples for detailed information and usage guidelines.