kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-13 22:18:36 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
75057cd036
TensorRT-LLMs/benchmarks
History
Kaiyu Xie 75057cd036
Update TensorRT-LLM (#2333) 
* Update TensorRT-LLM

---------

Co-authored-by: Puneesh Khanna <puneesh.khanna@tii.ae>
Co-authored-by: Ethan Zhang <26497102+ethnzhng@users.noreply.github.com>
2024-10-15 15:28:40 +08:00
..
cpp Update TensorRT-LLM (#2333) 2024-10-15 15:28:40 +08:00
python open source 7f370deb0090d885d7518c2b146399ba3933c004 (#2273) 2024-09-30 13:51:19 +02:00
README.md Update TensorRT-LLM 2024-08-20 18:55:15 +08:00
Suite.md open source 4dbf696ae9b74a26829d120b67ab8443d70c8e58 (#2297) 2024-10-08 12:19:19 +02:00

README.md

TensorRT-LLM Benchmarks

Overview

There are currently three workflows to benchmark TensorRT-LLM:

  • C++ benchmarks
    • The recommended workflow that uses TensorRT-LLM C++ API and can take advantage of the latest features of TensorRT-LLM.
  • Python benchmarks
    • The Python benchmarking scripts can only benchmark the Python runtime, which do not support the latest features, such as in-flight batching.
  • The Python benchmarking suite
    • This benchmarking suite is a current work in progress and is prone to large changes.