TensorRT-LLMs/tensorrt_llm/runtime/memory_pools/pool.py
2024-11-01 19:48:44 +08:00

8 lines
109 B
Python

from dataclasses import dataclass
@dataclass
class Pool(object):
num_kv_heads: int
num_layers: int