kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-13 22:18:36 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
rel
TensorRT-LLMs/windows
History
Sharan Chetlur 258c7540c0 open source 09df54c0cc99354a60bbc0303e3e8ea33a96bef0 (#2725) 
Co-authored-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>

open source f8c0381a2bc50ee2739c3d8c2be481b31e5f00bd (#2736)

Co-authored-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>

Add note for blackwell (#2742)

Update the docs to workaround the extra-index-url issue (#2744)

update README.md (#2751)

Fix github io pages (#2761)

Update
2025-02-11 02:21:51 +00:00
..
docker TensorRT-LLM v0.10 update 2024-06-05 20:43:25 +08:00
examples/llama TensorRT-LLM v0.12 Update (#2164) 2024-08-29 17:25:07 +08:00
destruct_env.ps1 TensorRT-LLM v0.10 update 2024-06-05 20:43:25 +08:00
README.md TensorRT-LLM v0.10 update 2024-06-05 20:43:25 +08:00
setup_build_env.ps1 open source 09df54c0cc99354a60bbc0303e3e8ea33a96bef0 (#2725) 2025-02-11 02:21:51 +00:00
setup_env.ps1 open source 09df54c0cc99354a60bbc0303e3e8ea33a96bef0 (#2725) 2025-02-11 02:21:51 +00:00

README.md

TensorRT-LLM for Windows

  The Windows release of TensorRT-LLM is currently in beta.
  We recommend checking out the [v0.10.0 tag](https://github.com/NVIDIA/TensorRT-LLM/releases/tag/v0.10.0) for the most stable experience.

TensorRT-LLM is supported on bare-metal Windows for single-GPU inference. The release supports GeForce 40-series GPUs.

The release wheel for Windows can be installed with pip. Alternatively, you can build TensorRT-LLM for Windows from the source. Building from the source is an advanced option and is not necessary for building or running LLM engines. It is, however, required if you plan to use the C++ runtime directly or run C++ benchmarks.

Getting Started

To get started with TensorRT-LLM on Windows, visit our documentation: