mirror of
https://github.com/NVIDIA/TensorRT-LLM.git
synced 2026-02-11 13:33:40 +08:00
11 lines
394 B
ReStructuredText
11 lines
394 B
ReStructuredText
OpenAI Completion Client
|
|
========================
|
|
|
|
Refer to the `trtllm-serve documentation <https://nvidia.github.io/TensorRT-LLM/commands/trtllm-serve.html>`_ for starting a server.
|
|
|
|
Source https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/serve/openai_completion_client.py.
|
|
|
|
.. literalinclude:: ../../../examples/serve/openai_completion_client.py
|
|
:language: python
|
|
:linenos:
|