mirror of
https://github.com/NVIDIA/TensorRT-LLM.git
synced 2026-02-04 02:02:01 +08:00
17 lines
503 B
Markdown
17 lines
503 B
Markdown
(add-model)=
|
|
|
|
# Adding a Model
|
|
|
|
This document describes how to add a model in TensorRT-LLM.
|
|
|
|
TensorRT-LLM provides:
|
|
|
|
- Low-level functions, for example, `concat`, `add`, and `sum`.
|
|
- Basic layers, such as, `Linear` and `LayerNorm`.
|
|
- High-level layers, such as, `MLP` and `Attention`.
|
|
|
|
**Steps**
|
|
|
|
1. Create a model directory in `tensorrt_llm/tensorrt_llm/models`, for example `bloom`.
|
|
2. Write a `model.py` with TensorRT-LLM low level functions and basic layers. It's optional to use high level layers.
|