TensorRT-LLMs/tensorrt_llm/runtime/memory_pools/pool.py
Dan Blanaru 48686bca3a
open source 7f370deb0090d885d7518c2b146399ba3933c004 (#2273)
* Update TensorRT-LLM

---------
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
2024-09-30 13:51:19 +02:00

8 lines
109 B
Python

from dataclasses import dataclass
@dataclass
class Pool(object):
num_kv_heads: int
num_layers: int