TensorRT-LLMs/docs
Robin Kobus cc490de92c
docs: Add KV Cache Management documentation (#3908)
* docs: Add KV Cache Management documentation

* Introduced a new document detailing the hierarchy and event system for KV cache management, including definitions for Pool, Block, and Page.
* Updated the index.rst to include a reference to the new kv-cache-management.md file.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* Update docs/source/advanced/kv-cache-management.md

Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* Update KV Cache Pool Management

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* docs: Addcross-file links

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* docs: Clarify tokens_per_block

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* docs: Clarify acronyms

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

---------

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
2025-05-21 08:39:28 +02:00
..
source docs: Add KV Cache Management documentation (#3908) 2025-05-21 08:39:28 +02:00
Doxygen Update TensorRT-LLM (#1315) 2024-03-19 17:36:42 +08:00
make.bat Kaiyu/update main (#5) 2023-10-18 22:38:53 +08:00
Makefile Kaiyu/update main (#5) 2023-10-18 22:38:53 +08:00
README.md Kaiyu/update main (#5) 2023-10-18 22:38:53 +08:00
requirements.txt Update (#2978) 2025-03-23 16:39:35 +08:00

Docs

This directory contains the stuff for building static html documentations based on sphinx.

Build the docs

Firstly, install the sphinx:

apt-get install python3-sphinx doxygen python3-pip graphviz

Secondly, install the packages:

python3 -m pip install -r ./requirements.txt

And then, make the docs:

doxygen Doxygen # build C++ docs

make html

And the finally the generated html pages will locate in the build/html directory.

Preview the docs locally

The basic way to preview the docs is using the http.serve:

cd build/html

python3 -m http.server 8081

And you can visit the page with your web browser with url http://localhost:8081.