mirror of
https://github.com/NVIDIA/TensorRT-LLM.git
synced 2026-02-02 01:01:35 +08:00
32ed92e449
Co-authored-by: Rong Zhou <130957722+ReginaZh@users.noreply.github.com> Co-authored-by: Onur Galoglu <33498883+ogaloglu@users.noreply.github.com> Co-authored-by: Fabian Joswig <fjosw@users.noreply.github.com>
12 lines
564 B
Markdown
12 lines
564 B
Markdown
# TensorRT-LLM Benchmarks
|
|
|
|
## Overview
|
|
|
|
There are currently three workflows to benchmark TensorRT-LLM:
|
|
* [C++ benchmarks](./cpp)
|
|
- The recommended workflow that uses TensorRT-LLM C++ API and can take advantage of the latest features of TensorRT-LLM.
|
|
* [Python benchmarks](./python)
|
|
- The Python benchmarking scripts can only benchmark the Python runtime, which do not support the latest features, such as in-flight batching.
|
|
* [The Python benchmarking suite](./Suite.md)
|
|
- This benchmarking suite is a current work in progress and is prone to large changes.
|