TensorRT-LLMs/docs
Yechan Kim c6e2111f4e
feat: enhance trtllm serve multimodal (#3757)
* feat: enhance trtllm serve multimodal

1. made the load_image and load_video asynchronous
2. add image_encoded input support to be compatible with genai-perf
3. support text-only on multimodal mdoels(currently, Qwen2-VL & Qwen2.5-VL)

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* add test

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* fix bandit

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* trimming uils

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* trimming for test

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* genai perf command fix

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* command fix

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* refactor chat_utils

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

* stress test genai-perf command

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>

---------

Signed-off-by: yechank <161688079+yechank-nvidia@users.noreply.github.com>
2025-05-15 16:16:31 -07:00
..
source feat: enhance trtllm serve multimodal (#3757) 2025-05-15 16:16:31 -07:00
Doxygen Update TensorRT-LLM (#1315) 2024-03-19 17:36:42 +08:00
make.bat Kaiyu/update main (#5) 2023-10-18 22:38:53 +08:00
Makefile Kaiyu/update main (#5) 2023-10-18 22:38:53 +08:00
README.md Kaiyu/update main (#5) 2023-10-18 22:38:53 +08:00
requirements.txt Update (#2978) 2025-03-23 16:39:35 +08:00

Docs

This directory contains the stuff for building static html documentations based on sphinx.

Build the docs

Firstly, install the sphinx:

apt-get install python3-sphinx doxygen python3-pip graphviz

Secondly, install the packages:

python3 -m pip install -r ./requirements.txt

And then, make the docs:

doxygen Doxygen # build C++ docs

make html

And the finally the generated html pages will locate in the build/html directory.

Preview the docs locally

The basic way to preview the docs is using the http.serve:

cd build/html

python3 -m http.server 8081

And you can visit the page with your web browser with url http://localhost:8081.