kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-13 22:18:36 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Fix document (#1462)

Browse Source
This commit is contained in:
Kaiyu Xie 2024-04-17 13:00:20 +08:00 committed by GitHub
parent 6533c4e779
commit a9356d4b76
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 0 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
docs/source/performance/perf-overview.md
View File

@ -36,11 +36,6 @@ All data was generated using version 0.9.0
| Mistral 7B | 64 | 1 | 2048 | 128 | 2,423 |
| Mistral 7B | 56 | 1 | 2048 | 2048 | 3,867 |
| | | | | | |
| Mixtral 8x7B | 1024 | 2 | 128 | 128 | 5,444 |
| Mixtral 8x7B | 256 | 2 | 128 | 2048 | 4,973 |
| Mixtral 8x7B | 64 | 2 | 2048 | 128 | 605 |
| Mixtral 8x7B | 64 | 2 | 2048 | 2048 | 1,821 |
| | | | | | |
| LLaMA 7B | 896 | 1 | 128 | 128 | 20,618 |
| LLaMA 7B | 120 | 1 | 128 | 2048 | 8,348 |
| LLaMA 7B | 64 | 1 | 2048 | 128 | 2,391 |
@ -72,11 +67,6 @@ All data was generated using version 0.9.0
| Mistral 7B | 84 | 1 | 2048 | 128 | 2,405 |
| Mistral 7B | 56 | 1 | 2048 | 2048 | 3,731 |
| | | | | | |
| Mixtral 8x7B | 512 | 2 | 128 | 128 | 5,255 |
| Mixtral 8x7B | 128 | 2 | 128 | 2048 | 3,408 |
| Mixtral 8x7B | 64 | 2 | 2048 | 128 | 588 |
| Mixtral 8x7B | 32 | 2 | 2048 | 2048 | 1,317 |
| | | | | | |
| LLaMA 7B | 896 | 1 | 128 | 128 | 19,854 |
| LLaMA 7B | 120 | 1 | 128 | 2048 | 6,944 |
| LLaMA 7B | 84 | 1 | 2048 | 128 | 2,163 |
@ -107,11 +97,6 @@ All data was generated using version 0.9.0
| Mistral 7B | 84 | 1 | 2048 | 128 | 971 |
| Mistral 7B | 56 | 1 | 2048 | 2048 | 1,721 |
| | | | | | |
| Mixtral 8x7B | 256 | 4 | 128 | 128 | 1,266 |
| Mixtral 8x7B | 256 | 4 | 128 | 2048 | 1,490 |
| Mixtral 8x7B | 64 | 4 | 2048 | 128 | 155 |
| Mixtral 8x7B | 32 | 4 | 2048 | 2048 | 412 |
| | | | | | |
| LLaMA 7B | 256 | 1 | 128 | 128 | 5,885 |
| LLaMA 7B | 64 | 1 | 128 | 2048 | 1,654 |
| LLaMA 7B | 32 | 1 | 2048 | 128 | 574 |
@ -183,9 +168,6 @@ tasks.
| Mistral 7B | 1 | 1 | 128 | 5.9 |
| Mistral 7B | 1 | 1 | 2048 | 31.7 |
| | | | | |
| Mixtral 8x7B | 1 | 2 | 128 | 12.1 |
| Mixtral 8x7B | 1 | 2 | 2048 | 87.7 |
| | | | | |
| LLaMA 7B | 1 | 1 | 128 | 5.7 |
| LLaMA 7B | 1 | 1 | 2048 | 30.2 |
| | | | | |
@ -205,9 +187,6 @@ tasks.
| Mistral 7B | 1 | 1 | 128 | 6.5 |
| Mistral 7B | 1 | 1 | 2048 | 32.4 |
| | | | | |
| Mixtral 8x7B | 1 | 2 | 128 | 13.4 |
| Mixtral 8x7B | 1 | 2 | 2048 | 89.5 |
| | | | | |
| LLaMA 7B | 1 | 1 | 128 | 6.3 |
| LLaMA 7B | 1 | 1 | 2048 | 30.8 |
| | | | | |
@ -227,9 +206,6 @@ tasks.
| Mistral 7B | 1 | 1 | 128 | 15.4 |
| Mistral 7B | 1 | 1 | 2048 | 87.3 |
| | | | | |
| Mixtral 8x7B | 1 | 4 | 128 | 19.5 |
| Mixtral 8x7B | 1 | 4 | 2048 | 165.6 |
| | | | | |
| LLaMA 7B | 1 | 1 | 128 | 14.1 |
| LLaMA 7B | 1 | 1 | 2048 | 80.1 |
| | | | | |