kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-02-01 16:51:11 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
a65dba7aaf
TensorRT-LLMs/benchmarks
History
Kaiyu Xie fe7dc6ad4e
Update TensorRT-LLM (#2230) 
* Update TensorRT-LLM

---------

Co-authored-by: Yi Wang <yi.wang.2005@gmail.com>
Co-authored-by: lkm2835 <lkm2835@gmail.com>
2024-09-17 14:39:09 +08:00
..
cpp Update TensorRT-LLM (#2230) 2024-09-17 14:39:09 +08:00
python Update TensorRT-LLM (#2156) 2024-08-27 18:20:59 +08:00
README.md Update TensorRT-LLM 2024-08-20 18:55:15 +08:00
Suite.md Update TensorRT-LLM (#2230) 2024-09-17 14:39:09 +08:00

README.md

TensorRT-LLM Benchmarks

Overview

There are currently three workflows to benchmark TensorRT-LLM:

  • C++ benchmarks
    • The recommended workflow that uses TensorRT-LLM C++ API and can take advantage of the latest features of TensorRT-LLM.
  • Python benchmarks
    • The Python benchmarking scripts can only benchmark the Python runtime, which do not support the latest features, such as in-flight batching.
  • The Python benchmarking suite
    • This benchmarking suite is a current work in progress and is prone to large changes.