* docs: Add KV Cache Management documentation
* Introduced a new document detailing the hierarchy and event system for KV cache management, including definitions for Pool, Block, and Page.
* Updated the index.rst to include a reference to the new kv-cache-management.md file.
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
* Update docs/source/advanced/kv-cache-management.md
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
* Update KV Cache Pool Management
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
* docs: Addcross-file links
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
* docs: Clarify tokens_per_block
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
* docs: Clarify acronyms
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
---------
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
